first commit
This commit is contained in:
61
docs/communication/emails/domain-normalizer/README.md
Normal file
61
docs/communication/emails/domain-normalizer/README.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# @suaveplan/domain-normalizer - Normalisation, matching, and alias-canonicalisation for internet domain names
|
||||
|
||||
## Summary
|
||||
|
||||
Normalisation, matching, and alias-canonicalisation for internet domain names. Given heterogeneous URL or domain input — including common mobile/CDN subdomains and 31 built-in alias mappings (e.g. `m.youtube.com → youtube.com`, `old.reddit.com → reddit.com`) — this package produces a canonical registrable domain via `tldts`, strips `www`/`m`/`mobile` prefixes, resolves the alias map, and returns structured `NormalizedDomain` objects. It also provides Levenshtein-based similarity scoring (`computeSimilarity`), fuzzy matching (`matchDomains`), and a `groupDomains` function that clusters a mixed list by canonical root domain. Use this package for deduplicating contact email domains, canonicalising URLs before storage, and building domain-level blocklists. For full entity deduplication across arbitrary fields use `@suaveplan/dedup`; for MIME or URL content-type detection use `@suaveplan/mime`.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [domain-normalizer-error](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [types](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [domain-normalizer](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer-error](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer-error](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer-error](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [domain-normalizer](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [aliases](./src/core/aliases/aliases.md) — <!-- begin:auto-docs --> _(no submodules)_- [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ - [matcher](./src/core/matcher/matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ - [normalizer](./src/core/normalizer/normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_ - [domain-normalizer-error](./src/errors/domain-normalizer-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_ - [domain-normalizer](./src/types/domain-normalizer.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Modules
|
||||
|
||||
| Module | Description |
|
||||
|---|---|
|
||||
| [`./src/core/normalizer/normalizer.md`](./src/core/normalizer/normalizer.md) | Domain/URL normalisation with tldts, prefix stripping, alias resolution |
|
||||
| [`./src/core/aliases/aliases.md`](./src/core/aliases/aliases.md) | Built-in alias map and canonical lookup functions |
|
||||
| [`./src/core/matcher/matcher.md`](./src/core/matcher/matcher.md) | Levenshtein similarity scoring, domain grouping, and matching |
|
||||
| [`./src/errors/domain-normalizer-error.md`](./src/errors/domain-normalizer-error.md) | Local error class and stable code registry |
|
||||
| [`./src/types/domain-normalizer.md`](./src/types/domain-normalizer.md) | Shared type definitions (NormalizerConfig, NormalizedDomain, MatchResult) |
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { normalizeDomain, matchDomains, groupDomains } from "@suaveplan/domain-normalizer";
|
||||
|
||||
// Normalize a domain
|
||||
const result = normalizeDomain("m.youtube.com");
|
||||
console.log(result.normalized); // "youtube.com"
|
||||
|
||||
// Normalize a full URL
|
||||
import { normalizeUrl } from "@suaveplan/domain-normalizer";
|
||||
const urlResult = normalizeUrl("https://old.reddit.com/r/programming");
|
||||
console.log(urlResult.normalized); // "reddit.com"
|
||||
|
||||
// Check similarity
|
||||
const match = matchDomains("youtube.com", "m.youtube.com");
|
||||
console.log(match.isMatch); // true
|
||||
|
||||
// Group a list of mixed domains
|
||||
const groups = groupDomains(["m.youtube.com", "youtube.com", "reddit.com", "old.reddit.com"]);
|
||||
// Map { "youtube.com" => [...], "reddit.com" => [...] }
|
||||
```
|
||||
|
||||
## Subpath exports
|
||||
|
||||
```ts
|
||||
import { normalizeDomain } from "@suaveplan/domain-normalizer/normalizer";
|
||||
import { getCanonicalDomain } from "@suaveplan/domain-normalizer/aliases";
|
||||
import { levenshteinDistance } from "@suaveplan/domain-normalizer/matcher";
|
||||
import { DomainNormalizerError } from "@suaveplan/domain-normalizer/errors";
|
||||
```
|
||||
@@ -0,0 +1,102 @@
|
||||
# aliases
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides the built-in alias map and lookup functions for well-known domain variants. Many popular sites publish the same content under mobile subdomains (`m.`, `mobile.`), AMP subdomains (`amp.`), and legacy subdomains (`old.`, `lite.`). This module centralises the mapping from these variant domains to their canonical registrable domain, so that downstream deduplication and normalisation functions operate on a consistent canonical form.
|
||||
|
||||
The reverse-index (canonical → group of aliases) is computed lazily and cached for O(1) repeated lookups; `resetAliasCache()` clears it for test isolation.
|
||||
|
||||
## Features
|
||||
|
||||
- `BUILT_IN_ALIASES` — exported `ReadonlyMap<string, string>` covering 30+ well-known variant domains
|
||||
- `getCanonicalDomain(domain)` — returns canonical form or original input when not an alias
|
||||
- `isKnownAlias(domain)` — O(1) membership check in the alias map
|
||||
- `getAliasGroup(domain)` — returns all equivalent domains (including canonical) sorted alphabetically
|
||||
- `resetAliasCache()` — clears the lazy reverse-index for test-environment isolation
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
getCanonicalDomain,
|
||||
isKnownAlias,
|
||||
} from "@suaveplan/domain-normalizer/aliases";
|
||||
|
||||
console.log(getCanonicalDomain("m.youtube.com")); // "youtube.com"
|
||||
console.log(getCanonicalDomain("youtube.com")); // "youtube.com" (passthrough)
|
||||
console.log(isKnownAlias("old.reddit.com")); // true
|
||||
console.log(isKnownAlias("example.com")); // false
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
getAliasGroup,
|
||||
getCanonicalDomain,
|
||||
resetAliasCache,
|
||||
} from "@suaveplan/domain-normalizer/aliases";
|
||||
|
||||
// Find all equivalent domains for a given input
|
||||
const group = getAliasGroup("m.reddit.com");
|
||||
// ["i.reddit.com", "m.reddit.com", "new.reddit.com", "old.reddit.com", "reddit.com"]
|
||||
|
||||
// Canonical also returns its full group
|
||||
const sameGroup = getAliasGroup("reddit.com");
|
||||
// same array as above
|
||||
|
||||
// In tests: reset the cache after any mutation
|
||||
resetAliasCache();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `BUILT_IN_ALIASES`
|
||||
|
||||
```ts
|
||||
const BUILT_IN_ALIASES: ReadonlyMap<string, string>
|
||||
```
|
||||
|
||||
Immutable map of `variantDomain → canonicalDomain`. Contains entries for YouTube, Twitter/X, The Guardian, CNN, Facebook, Reddit, Instagram, LinkedIn, Wikipedia, GitHub, Google, Amazon, TikTok, Bing, and DuckDuckGo.
|
||||
|
||||
### `getCanonicalDomain(domain)`
|
||||
|
||||
```ts
|
||||
function getCanonicalDomain(domain: string): string
|
||||
```
|
||||
|
||||
Returns the canonical domain for `domain`. If `domain` is not in `BUILT_IN_ALIASES`, returns `domain` unchanged (making this function idempotent on canonical domains).
|
||||
|
||||
### `isKnownAlias(domain)`
|
||||
|
||||
```ts
|
||||
function isKnownAlias(domain: string): boolean
|
||||
```
|
||||
|
||||
Returns `true` if `domain` has an entry in `BUILT_IN_ALIASES`, `false` otherwise.
|
||||
|
||||
### `getAliasGroup(domain)`
|
||||
|
||||
```ts
|
||||
function getAliasGroup(domain: string): string[]
|
||||
```
|
||||
|
||||
Returns a sorted array of all domain strings that map to the same canonical as `domain`. Includes the canonical form itself. If `domain` is the canonical and has no aliases, returns `[canonical]`.
|
||||
|
||||
### `resetAliasCache()`
|
||||
|
||||
```ts
|
||||
function resetAliasCache(): void
|
||||
```
|
||||
|
||||
Clears the internal reverse-index cache. The cache is rebuilt on the next call to `getAliasGroup`. Intended for test environments only.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The reverse-index is built lazily on first call to `getAliasGroup` to avoid unnecessary work in code paths that only call `getCanonicalDomain`.
|
||||
- `BUILT_IN_ALIASES` is a `ReadonlyMap` constructed once at module load — it is not frozen at runtime but TypeScript prevents mutation at the type level.
|
||||
- `getAliasGroup` returns a new sorted array on every call to prevent callers from mutating the internal set.
|
||||
- The alias map is intentionally conservative — it only includes widely deployed subdomain patterns, not every possible variant, to minimise false-positive collapsing of genuinely distinct domains.
|
||||
@@ -0,0 +1,127 @@
|
||||
# matcher
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides domain similarity scoring and grouping functions for deduplication workflows. Uses Levenshtein edit distance on registrable domain parts (eTLD+1) to compute a normalized similarity score in `[0, 1]`, enabling fuzzy matching of typos, regional TLD variants, and subdomain variants that survive normalization.
|
||||
|
||||
The grouping function builds on the normalizer's canonical domain resolution, collecting all aliases of the same domain into a single group for deduplication pipelines.
|
||||
|
||||
## Features
|
||||
|
||||
- `levenshteinDistance(a, b)` — classic two-row DP edit distance, O(m×n) time, O(n) space
|
||||
- `computeSimilarity(a, b)` — normalized distance score: 1.0 = exact, 0.0 = fully different
|
||||
- `matchDomains(domain1, domain2)` — full `MatchResult` with similarity and boolean threshold flag
|
||||
- `findMatchingDomains(needle, haystack, threshold?)` — sorted list of haystack entries that match a needle
|
||||
- `groupDomains(domains)` — groups a list of domain strings by their canonical registrable domain
|
||||
- `DEFAULT_MATCH_THRESHOLD` — exported constant (0.8) for custom threshold decisions
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { matchDomains, computeSimilarity } from "@suaveplan/domain-normalizer/matcher";
|
||||
|
||||
const result = matchDomains("youtube.com", "m.youtube.com");
|
||||
console.log(result.similarity); // 1.0 (both normalize to youtube.com)
|
||||
console.log(result.isMatch); // true
|
||||
|
||||
const score = computeSimilarity("google.com", "googel.com");
|
||||
console.log(score); // ~0.857 (one typo in 7-char string)
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
findMatchingDomains,
|
||||
groupDomains,
|
||||
levenshteinDistance,
|
||||
} from "@suaveplan/domain-normalizer/matcher";
|
||||
|
||||
// Find duplicates of a known domain in a list
|
||||
const matches = findMatchingDomains("reddit.com", [
|
||||
"m.reddit.com",
|
||||
"old.reddit.com",
|
||||
"example.com",
|
||||
"reddlt.com",
|
||||
], 0.8);
|
||||
// ["m.reddit.com", "old.reddit.com", "reddlt.com"]
|
||||
|
||||
// Group a crawled link list by canonical domain
|
||||
const grouped = groupDomains([
|
||||
"m.youtube.com",
|
||||
"youtube.com",
|
||||
"old.reddit.com",
|
||||
"reddit.com",
|
||||
"example.com",
|
||||
]);
|
||||
// Map {
|
||||
// "youtube.com" => ["m.youtube.com", "youtube.com"],
|
||||
// "reddit.com" => ["old.reddit.com", "reddit.com"],
|
||||
// "example.com" => ["example.com"]
|
||||
// }
|
||||
|
||||
// Raw edit distance
|
||||
console.log(levenshteinDistance("kitten", "sitting")); // 3
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DEFAULT_MATCH_THRESHOLD`
|
||||
|
||||
```ts
|
||||
const DEFAULT_MATCH_THRESHOLD: number // 0.8
|
||||
```
|
||||
|
||||
### `levenshteinDistance(a, b)`
|
||||
|
||||
```ts
|
||||
function levenshteinDistance(a: string, b: string): number
|
||||
```
|
||||
|
||||
Returns the minimum number of single-character edits (insertions, deletions, substitutions) to transform `a` into `b`. Returns 0 for identical strings. Property: `distance(a, a) === 0`; triangle inequality holds.
|
||||
|
||||
### `computeSimilarity(a, b)`
|
||||
|
||||
```ts
|
||||
function computeSimilarity(a: string, b: string): number
|
||||
```
|
||||
|
||||
Returns a score in `[0, 1]`. Computes `1 - levenshtein(regA, regB) / max(len(regA), len(regB))` where `regA`/`regB` are the registrable domain parts. Returns `1.0` for two empty strings.
|
||||
|
||||
### `matchDomains(domain1, domain2)`
|
||||
|
||||
```ts
|
||||
function matchDomains(domain1: string, domain2: string): MatchResult
|
||||
```
|
||||
|
||||
Returns `{ domain1, domain2, similarity, isMatch }` where `isMatch = similarity >= DEFAULT_MATCH_THRESHOLD`.
|
||||
|
||||
### `findMatchingDomains(needle, haystack, threshold?)`
|
||||
|
||||
```ts
|
||||
function findMatchingDomains(
|
||||
needle: string,
|
||||
haystack: string[],
|
||||
threshold?: number,
|
||||
): string[]
|
||||
```
|
||||
|
||||
Returns haystack entries with `computeSimilarity(needle, entry) >= threshold`, sorted by similarity descending.
|
||||
|
||||
### `groupDomains(domains)`
|
||||
|
||||
```ts
|
||||
function groupDomains(domains: string[]): Map<string, string[]>
|
||||
```
|
||||
|
||||
Returns a `Map<canonicalDomain, string[]>` grouping all input domains by their canonical registrable domain. Preserves insertion order within each group.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `levenshteinDistance` uses the standard two-row optimization: only two rows of the DP table are kept in memory at a time, keeping space complexity at O(n).
|
||||
- `computeSimilarity` calls `getRegistrableDomain` first — two aliases that normalize to the same eTLD+1 always score 1.0.
|
||||
- `findMatchingDomains` always computes similarity for every haystack entry; it does not short-circuit for large haystacks. For very large lists (thousands of entries), consider pre-grouping first.
|
||||
- `groupDomains` relies on `getDomainGroup` which never throws, so this function is also safe to use in non-throwing map/reduce pipelines.
|
||||
@@ -0,0 +1,110 @@
|
||||
# normalizer
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides the core domain and URL normalization functions for `@suaveplan/domain-normalizer`. Uses `tldts` for accurate public suffix list (PSL) lookups to extract registrable domains (eTLD+1), and integrates with the alias module to resolve known mobile/AMP subdomain variants to their canonical form.
|
||||
|
||||
The normalizer is designed to be idempotent: `normalizeDomain(normalizeDomain(x))` produces the same result as `normalizeDomain(x)` for any valid input.
|
||||
|
||||
## Features
|
||||
|
||||
- `normalizeDomain(input, config?)` — normalizes a bare domain string with configurable stripping and alias resolution
|
||||
- `normalizeUrl(url, config?)` — extracts hostname from a full URL, then normalizes it
|
||||
- `getRegistrableDomain(input)` — returns the eTLD+1 or null for invalid inputs
|
||||
- `getSubdomain(input)` — returns the subdomain portion or null
|
||||
- `getDomainGroup(input)` — returns the canonical registrable domain as a grouping key
|
||||
- Configurable via `NormalizerConfig`: strip www, strip mobile prefixes, lowercase, resolve aliases
|
||||
- Returns rich `NormalizedDomain` objects with subdomain, publicSuffix, and isPrivateDomain metadata
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { normalizeDomain } from "@suaveplan/domain-normalizer/normalizer";
|
||||
|
||||
const result = normalizeDomain("m.youtube.com");
|
||||
console.log(result.normalized); // "youtube.com"
|
||||
console.log(result.registrableDomain); // "youtube.com"
|
||||
console.log(result.publicSuffix); // "com"
|
||||
console.log(result.original); // "m.youtube.com"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { normalizeUrl, getDomainGroup } from "@suaveplan/domain-normalizer/normalizer";
|
||||
|
||||
// Normalize a full URL
|
||||
const result = normalizeUrl("https://old.reddit.com/r/programming");
|
||||
console.log(result.normalized); // "reddit.com"
|
||||
console.log(result.original); // "https://old.reddit.com/r/programming"
|
||||
|
||||
// Group a mixed list of domains
|
||||
const domains = ["m.youtube.com", "youtube.com", "old.reddit.com", "reddit.com"];
|
||||
const groups = new Map<string, string[]>();
|
||||
for (const d of domains) {
|
||||
const key = getDomainGroup(d);
|
||||
const arr = groups.get(key) ?? [];
|
||||
arr.push(d);
|
||||
groups.set(key, arr);
|
||||
}
|
||||
// Map { "youtube.com" => ["m.youtube.com", "youtube.com"], "reddit.com" => [...] }
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `normalizeDomain(input, config?)`
|
||||
|
||||
```ts
|
||||
function normalizeDomain(input: string, config?: NormalizerConfig): NormalizedDomain
|
||||
```
|
||||
|
||||
Normalizes a bare domain string. Applies lowercase, alias resolution, www/mobile stripping, and PSL parsing in that order.
|
||||
|
||||
- **Throws** `DomainNormalizerError(INVALID_INPUT)` for empty/whitespace-only inputs
|
||||
- **Throws** `DomainNormalizerError(PARSE_ERROR)` when tldts cannot extract a registrable domain
|
||||
|
||||
### `normalizeUrl(url, config?)`
|
||||
|
||||
```ts
|
||||
function normalizeUrl(url: string, config?: NormalizerConfig): NormalizedDomain
|
||||
```
|
||||
|
||||
Extracts the hostname from `url` (using `URL` constructor with protocol injection if needed), then calls `normalizeDomain` on it. The `original` field in the result contains the full URL.
|
||||
|
||||
- **Throws** `DomainNormalizerError(INVALID_INPUT)` for empty inputs
|
||||
- **Throws** `DomainNormalizerError(PARSE_ERROR)` when hostname cannot be extracted
|
||||
|
||||
### `getRegistrableDomain(input)`
|
||||
|
||||
```ts
|
||||
function getRegistrableDomain(input: string): string | null
|
||||
```
|
||||
|
||||
Returns the eTLD+1 for `input`, or `null` if parsing fails or input is empty. Does not throw.
|
||||
|
||||
### `getSubdomain(input)`
|
||||
|
||||
```ts
|
||||
function getSubdomain(input: string): string | null
|
||||
```
|
||||
|
||||
Returns the subdomain portion of `input`, or `null` if there is none. Does not throw.
|
||||
|
||||
### `getDomainGroup(input)`
|
||||
|
||||
```ts
|
||||
function getDomainGroup(input: string): string
|
||||
```
|
||||
|
||||
Returns the canonical registrable domain for use as a deduplication/grouping key. Falls back to lowercased trimmed input if normalization fails. Does not throw.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Alias resolution is applied before prefix stripping so that an alias like `www.facebook.com` maps to `facebook.com` (rather than stripping `www.` first and getting `facebook.com` without alias resolution running).
|
||||
- `tldts` handles private domains (e.g. `github.io`) correctly when `allowPrivateDomains: true` is set.
|
||||
- The `URL` constructor is used for URL hostname extraction to handle percent-encoding, authentication components, and port numbers transparently.
|
||||
- `stripPrefixes` loops until no more changes occur to handle edge cases like hypothetical `m.m.domain.com` inputs.
|
||||
- `getDomainGroup` catches all errors and falls back to the lowercased input to ensure it never throws — making it safe for use in array mapping operations.
|
||||
@@ -0,0 +1,92 @@
|
||||
# domain-normalizer-error
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides the stable error class and error-code registry for the `@suaveplan/domain-normalizer` package. All errors thrown by normalizer, matcher, and alias functions are instances of `DomainNormalizerError` with a stable machine-readable code string, enabling programmatic error handling without string matching on messages.
|
||||
|
||||
This module deliberately avoids depending on `@suaveplan/error` so that `domain-normalizer` stays a leaf package with no workspace runtime dependencies.
|
||||
|
||||
## Features
|
||||
|
||||
- `DOMAIN_NORMALIZER_CODES` — frozen const object with every stable code string
|
||||
- `DomainNormalizerError` — extends `Error`, adds `readonly code: string` and sets `name = 'DomainNormalizerError'`
|
||||
- `DomainNormalizerCode` — union type of all defined code strings for exhaustive switch guards
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DomainNormalizerError, DOMAIN_NORMALIZER_CODES } from "@suaveplan/domain-normalizer/errors";
|
||||
|
||||
try {
|
||||
// any domain-normalizer call
|
||||
} catch (err) {
|
||||
if (err instanceof DomainNormalizerError) {
|
||||
if (err.code === DOMAIN_NORMALIZER_CODES.INVALID_INPUT) {
|
||||
console.error("Bad input:", err.message);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DomainNormalizerError, DOMAIN_NORMALIZER_CODES, type DomainNormalizerCode } from "@suaveplan/domain-normalizer/errors";
|
||||
|
||||
function handleError(err: unknown): string {
|
||||
if (!(err instanceof DomainNormalizerError)) return "unknown error";
|
||||
|
||||
const code: DomainNormalizerCode = err.code as DomainNormalizerCode;
|
||||
switch (code) {
|
||||
case DOMAIN_NORMALIZER_CODES.INVALID_INPUT:
|
||||
return `Invalid domain input: ${err.message}`;
|
||||
case DOMAIN_NORMALIZER_CODES.PARSE_ERROR:
|
||||
return `Failed to parse domain: ${err.message}`;
|
||||
default:
|
||||
return `Domain normalizer error [${code}]: ${err.message}`;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DOMAIN_NORMALIZER_CODES`
|
||||
|
||||
```ts
|
||||
const DOMAIN_NORMALIZER_CODES: {
|
||||
readonly INVALID_INPUT: "DOMAIN_NORMALIZER.INVALID_INPUT";
|
||||
readonly PARSE_ERROR: "DOMAIN_NORMALIZER.PARSE_ERROR";
|
||||
}
|
||||
```
|
||||
|
||||
Stable code strings for every error produced by this package. Use these constants for programmatic error matching rather than message strings.
|
||||
|
||||
### `DomainNormalizerError`
|
||||
|
||||
```ts
|
||||
class DomainNormalizerError extends Error {
|
||||
readonly code: string;
|
||||
constructor(message: string, code: string): DomainNormalizerError;
|
||||
}
|
||||
```
|
||||
|
||||
- `message` — human-readable description of the error condition
|
||||
- `code` — machine-readable code from `DOMAIN_NORMALIZER_CODES`
|
||||
- `name` — always `"DomainNormalizerError"` for `instanceof`-free detection
|
||||
|
||||
### `DomainNormalizerCode`
|
||||
|
||||
```ts
|
||||
type DomainNormalizerCode = "DOMAIN_NORMALIZER.INVALID_INPUT" | "DOMAIN_NORMALIZER.PARSE_ERROR";
|
||||
```
|
||||
|
||||
Union type derived from `DOMAIN_NORMALIZER_CODES` values, suitable for exhaustive switch statements.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The class intentionally calls `super(message)` before setting `this.name` to work correctly with all JS runtimes.
|
||||
- No `Error.captureStackTrace` call is made — this keeps the module universal (works in browsers without V8).
|
||||
- The `code` field is `readonly` to prevent accidental mutation after construction.
|
||||
@@ -0,0 +1,86 @@
|
||||
# domain-normalizer types
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Defines the shared TypeScript interfaces consumed by every module in `@suaveplan/domain-normalizer`. Centralising type declarations here prevents circular imports and makes the public API shape visible in one place.
|
||||
|
||||
These types are shipped as part of the package's type declarations and re-exported from the root barrel so consumers can import them without importing any runtime code.
|
||||
|
||||
## Features
|
||||
|
||||
- `NormalizerConfig` — configuration object controlling stripping and alias-resolution behaviour
|
||||
- `NormalizedDomain` — rich result object returned by `normalizeDomain` and `normalizeUrl`
|
||||
- `MatchResult` — comparison result from `matchDomains` with a numeric similarity score and boolean threshold flag
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import type { NormalizerConfig, NormalizedDomain } from "@suaveplan/domain-normalizer";
|
||||
|
||||
const config: NormalizerConfig = { stripWww: true, stripMobile: true };
|
||||
|
||||
function process(result: NormalizedDomain): void {
|
||||
console.log(result.registrableDomain, result.publicSuffix);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import type { MatchResult, NormalizerConfig } from "@suaveplan/domain-normalizer";
|
||||
import { matchDomains } from "@suaveplan/domain-normalizer/matcher";
|
||||
|
||||
function deduplicate(domains: string[]): string[] {
|
||||
const seen = new Set<string>();
|
||||
const unique: string[] = [];
|
||||
for (const d of domains) {
|
||||
let isDup = false;
|
||||
for (const u of unique) {
|
||||
const result: MatchResult = matchDomains(d, u);
|
||||
if (result.isMatch) { isDup = true; break; }
|
||||
}
|
||||
if (!isDup) { unique.push(d); seen.add(d); }
|
||||
}
|
||||
return unique;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `NormalizerConfig`
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `stripWww` | `boolean?` | `true` | Remove `www.` prefix |
|
||||
| `stripMobile` | `boolean?` | `true` | Remove `m.`, `mobile.`, `lite.`, `amp.` prefixes |
|
||||
| `toLowercase` | `boolean?` | `true` | Lowercase the result |
|
||||
| `resolveAliases` | `boolean?` | `true` | Apply built-in alias map |
|
||||
|
||||
### `NormalizedDomain`
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `original` | `string` | Unmodified input |
|
||||
| `normalized` | `string` | Canonical registrable domain |
|
||||
| `subdomain` | `string?` | Remaining subdomain after stripping |
|
||||
| `registrableDomain` | `string` | eTLD+1 (e.g. `"youtube.com"`) |
|
||||
| `publicSuffix` | `string?` | eTLD (e.g. `"com"`) |
|
||||
| `isPrivateDomain` | `boolean?` | From public suffix list private section |
|
||||
|
||||
### `MatchResult`
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `domain1` | `string` | First input |
|
||||
| `domain2` | `string` | Second input |
|
||||
| `similarity` | `number` | Score in `[0, 1]` |
|
||||
| `isMatch` | `boolean` | `true` when similarity ≥ default threshold (0.8) |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- All interfaces use optional (`?`) fields rather than union with `undefined` for better JSON round-trip compatibility.
|
||||
- `NormalizerConfig` fields are all optional so callers can pass a partial config; the normalizer merges with defaults internally.
|
||||
- `MatchResult.similarity` is always in the range `[0, 1]` inclusive — callers can apply their own threshold by comparing to `similarity` directly.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Domain Normalizer Types
|
||||
|
||||
## Purpose
|
||||
|
||||
Shared TypeScript type definitions for the `@suaveplan/domain-normalizer` package. These interfaces describe the configuration surface and output shapes used by every normalizer, alias, and matcher function. Centralising them here prevents circular imports between modules and provides a single place to extend the public API contract.
|
||||
|
||||
## Features
|
||||
|
||||
- `NormalizerConfig` — four boolean flags that control how raw input is transformed before canonicalization
|
||||
- `NormalizedDomain` — rich output record containing the original input, canonical form, subdomain, registrable domain (eTLD+1), public suffix, and private-domain flag
|
||||
- `MatchResult` — pair-wise comparison result with a `[0, 1]` similarity score and a boolean threshold indicator
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import type { NormalizerConfig, NormalizedDomain } from '@suaveplan/domain-normalizer';
|
||||
|
||||
const config: NormalizerConfig = {
|
||||
stripWww: true,
|
||||
stripMobile: true,
|
||||
toLowercase: true,
|
||||
resolveAliases: true,
|
||||
};
|
||||
|
||||
// NormalizedDomain returned by normalizeDomain()
|
||||
const result: NormalizedDomain = {
|
||||
original: 'https://m.YouTube.com/watch?v=dQw4w9WgXcQ',
|
||||
normalized: 'youtube.com',
|
||||
registrableDomain: 'youtube.com',
|
||||
publicSuffix: 'com',
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import type { MatchResult, NormalizerConfig } from '@suaveplan/domain-normalizer';
|
||||
import { matchDomains, normalizeDomain } from '@suaveplan/domain-normalizer';
|
||||
|
||||
// Use a stricter config that keeps subdomains intact
|
||||
const strictConfig: NormalizerConfig = {
|
||||
stripWww: false,
|
||||
stripMobile: false,
|
||||
toLowercase: true,
|
||||
resolveAliases: false,
|
||||
};
|
||||
|
||||
const a = normalizeDomain('blog.example.com', strictConfig);
|
||||
const b = normalizeDomain('BLOG.EXAMPLE.COM', strictConfig);
|
||||
|
||||
const result: MatchResult = matchDomains(a.normalized, b.normalized);
|
||||
// result.similarity === 1, result.isMatch === true
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `NormalizerConfig`
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `stripWww` | `boolean?` | `true` | Remove `www.` prefix |
|
||||
| `stripMobile` | `boolean?` | `true` | Remove `m.`, `mobile.`, `lite.`, `amp.` prefixes |
|
||||
| `toLowercase` | `boolean?` | `true` | Lowercase the entire domain |
|
||||
| `resolveAliases` | `boolean?` | `true` | Map known alias subdomains to their canonical registrable domain |
|
||||
|
||||
### `NormalizedDomain`
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `original` | `string` | The raw input passed to the normalizer |
|
||||
| `normalized` | `string` | The canonical domain after all transformations |
|
||||
| `subdomain` | `string?` | Remaining subdomain after stripping www/mobile |
|
||||
| `registrableDomain` | `string` | The eTLD+1 (e.g. `youtube.com`) |
|
||||
| `publicSuffix` | `string?` | The effective TLD (e.g. `com`, `co.uk`) |
|
||||
| `isPrivateDomain` | `boolean?` | `true` for private registries per public suffix list |
|
||||
|
||||
### `MatchResult`
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `domain1` | `string` | First domain in the comparison |
|
||||
| `domain2` | `string` | Second domain in the comparison |
|
||||
| `similarity` | `number` | Levenshtein-normalized score in `[0, 1]` |
|
||||
| `isMatch` | `boolean` | `true` when `similarity >= 0.8` (default threshold) |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
All interfaces are exported from `src/index.ts` and re-exported under the `./types` subpath export. The types file contains no runtime code and therefore has no test file, but it is included in `docs:check` coverage because it forms part of the package's documented public API surface.
|
||||
Reference in New Issue
Block a user