first commit

docs/foundation/data/dedup/src/core/engine/dedup-engine.md

@@ -0,0 +1,114 @@
+# dedup-engine
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+`DedupEngine` is the central orchestrator for entity deduplication. It accepts a validated `DedupConfig`, performs an O(n²) pairwise similarity scan using pluggable match keys and scoring rules, groups near-duplicate entities into connected components via union-find, and exposes a `merge()` method that delegates to caller-supplied merge strategies. The engine is generic over the entity type `T` so it can operate on any record shape.
+
+## Features
+
+- **Schema-validated configuration** — `DedupConfig` is parsed with Zod at construction time; invalid configs throw a `DedupError` with code `DEDUP.INVALID_CONFIG`.
+- **Weighted field scoring** — each `MatchKey` contributes its `weight × similarity(field_a, field_b)` to the pair score.
+- **Pluggable scoring rules** — `ScoringRule` instances can override or supplement the default key-based scoring for custom business logic.
+- **Connected-component grouping** — union-find algorithm aggregates transitive duplicate relationships so A≈B and B≈C implies A, B, C are in the same group.
+- **`maxGroupSize` guard** — prevents unbounded memory growth by capping the number of duplicates per group.
+- **OTel instrumentation** — emits `dedup.detect.total` counter, `dedup.detect.duration_ms` histogram, and per-span attributes on every `detect()` and `merge()` call.
+- **Safe merge delegation** — `merge()` wraps strategy execution in a try/catch and re-throws as `DedupError(STRATEGY_ERROR)` to preserve error code stability.
+
+## Usage: Basic Example
+
+```ts
+import { DedupEngine } from "@suaveplan/dedup/engine";
+
+const engine = new DedupEngine({
+  matchKeys: [
+    { field: "name",  weight: 0.6, strategy: "jaro-winkler" },
+    { field: "email", weight: 0.4, strategy: "exact" },
+  ],
+  scoringRules: [],
+  mergeThreshold: 0.85,
+});
+
+const records = [
+  { id: 1, name: "Alice Smith",  email: "alice@example.com" },
+  { id: 2, name: "Alice Smyth",  email: "alice@example.com" },
+  { id: 3, name: "Bob Jones",    email: "bob@example.com"   },
+];
+
+const groups = engine.detect(records);
+// groups[0].canonical → { id: 1, name: "Alice Smith",  email: "alice@example.com" }
+// groups[0].duplicates → [{ id: 2, name: "Alice Smyth", email: "alice@example.com" }]
+```
+
+## Usage: Advanced Example
+
+```ts
+import { DedupEngine } from "@suaveplan/dedup/engine";
+import { manualStrategy } from "@suaveplan/dedup/merge";
+
+const engine = new DedupEngine<{ id: number; name: string; updatedAt: string }>({
+  matchKeys: [
+    { field: "name", weight: 1, strategy: "jaro-winkler", caseSensitive: false },
+  ],
+  scoringRules: [
+    {
+      name: "domain-boost",
+      evaluate(a, b, _keys) {
+        // Boost score when both entities are from the same domain
+        const ra = a as Record<string, string>;
+        const rb = b as Record<string, string>;
+        const domainA = ra["email"]?.split("@")[1] ?? "";
+        const domainB = rb["email"]?.split("@")[1] ?? "";
+        return domainA && domainA === domainB ? 0.95 : 0.0;
+      },
+    },
+  ],
+  mergeThreshold: 0.8,
+  maxGroupSize: 50,
+});
+
+const groups = engine.detect(records);
+for (const group of groups) {
+  const result = engine.merge(group, manualStrategy);
+  console.log("merged:", result.merged);
+}
+```
+
+## API Reference
+
+### `new DedupEngine<T>(config: DedupConfig)`
+
+Constructs a new engine. `config` is validated against `DedupConfigSchema`; throws `DedupError(INVALID_CONFIG)` on failure.
+
+### `detect(entities: T[]): DuplicateGroup<T>[]`
+
+Scans all entity pairs, builds a similarity graph, finds connected components, and returns one `DuplicateGroup` per multi-entity component. Singletons (no duplicates) are omitted.
+
+**Params:**
+
+- `entities` — array of entities to deduplicate
+
+**Returns:** `DuplicateGroup<T>[]` — possibly empty if no duplicates found
+
+### `merge(group: DuplicateGroup<T>, strategy: MergeStrategy<T>): MergeResult<T>`
+
+Applies `strategy` to `group` and returns the merged result. Wraps thrown errors as `DedupError(STRATEGY_ERROR)`.
+
+**Params:**
+
+- `group` — a group returned by `detect()`
+- `strategy` — a `MergeStrategy<T>` function
+
+**Returns:** `MergeResult<T>`
+
+**Throws:** `DedupError` with code `DEDUP.STRATEGY_ERROR` if the strategy throws
+
+## Implementation Notes
+
+The pairwise scan is O(n²) in both time and the number of potential edges. For large corpora (> 10 000 entities) callers should pre-filter candidates using blocking keys (e.g. first three characters of a name) before passing them to `detect()`. A future blocking API may be added as a separate module.
+
+The union-find implementation uses path-halving rather than full path compression to keep the code concise while still achieving near-linear amortised complexity. For groups larger than `maxGroupSize`, only the first `maxGroupSize − 1` duplicates (by original array index) are retained; this is intentional to bound memory and downstream merge cost.
+
+When multiple `scoringRules` are present, each successive rule _replaces_ the accumulated score for that pair, not adds to it. This allows a single rule to act as a "veto" or "boost" override. If additive composition is needed, callers should encode that logic within a single rule.

docs/foundation/data/dedup/src/core/merge/merge-strategies.md

@@ -0,0 +1,92 @@
+# merge-strategies
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+Provides three built-in merge strategies for combining a `DuplicateGroup` into a single entity. Merge strategies are pure functions that conform to the `MergeStrategy<T>` type and can be composed or replaced with custom implementations. They are passed to `DedupEngine.merge()` as callbacks rather than configured up-front, giving callers full control over how and when merging occurs.
+
+## Features
+
+- **`latestWinsStrategy`** — selects whichever entity (canonical or duplicate) has the most recent `updatedAt` or `createdAt` timestamp; ISO 8601 strings sort correctly as plain strings so no date parsing is needed.
+- **`manualStrategy`** — no-op that returns the canonical entity unchanged, useful when a human reviewer should approve merges before they are committed to a data store.
+- **`scoredMergeStrategy`** — assembles a "best-of-breed" entity by choosing, for each field, the value from the entity with the highest similarity score; the canonical entity acts as the fallback with score 1.0.
+
+## Usage: Basic Example
+
+```ts
+import { DedupEngine } from "@suaveplan/dedup/engine";
+import { latestWinsStrategy } from "@suaveplan/dedup/merge";
+
+const engine = new DedupEngine({
+  matchKeys: [{ field: "name", weight: 1, strategy: "jaro-winkler" }],
+  scoringRules: [],
+});
+
+const groups = engine.detect([
+  { name: "Alice",   updatedAt: "2024-01-01T00:00:00Z" },
+  { name: "Alicia",  updatedAt: "2024-06-15T12:00:00Z" },
+]);
+
+for (const group of groups) {
+  const result = engine.merge(group, latestWinsStrategy);
+  // result.merged → { name: "Alicia", updatedAt: "2024-06-15T12:00:00Z" }
+  // result.strategy → "latest-wins"
+}
+```
+
+## Usage: Advanced Example
+
+```ts
+import { DedupEngine } from "@suaveplan/dedup/engine";
+import { scoredMergeStrategy, manualStrategy } from "@suaveplan/dedup/merge";
+import type { MergeStrategy } from "@suaveplan/dedup";
+
+type Person = { id: number; name: string; email: string; phone?: string };
+
+const engine = new DedupEngine<Person>({
+  matchKeys: [
+    { field: "name",  weight: 0.5, strategy: "jaro-winkler" },
+    { field: "email", weight: 0.5, strategy: "exact" },
+  ],
+  scoringRules: [],
+  mergeThreshold: 0.8,
+});
+
+const groups = engine.detect(persons);
+
+// Pick strategy based on group size
+for (const group of groups) {
+  const strategy: MergeStrategy<Person> =
+    group.duplicates.length > 5 ? manualStrategy : scoredMergeStrategy;
+  const result = engine.merge(group, strategy);
+  console.log(result.strategy, result.merged);
+}
+```
+
+## API Reference
+
+### `latestWinsStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
+
+Picks the entity with the lexicographically largest `updatedAt` or `createdAt` field value. If neither field is present, returns the canonical unchanged. All entities (canonical + duplicates) participate in the comparison.
+
+**Returns:** `MergeResult<T>` with `strategy: "latest-wins"`
+
+### `manualStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
+
+Returns the canonical as the merged entity with no field-level changes. Suitable as a placeholder when merge logic will be determined externally.
+
+**Returns:** `MergeResult<T>` with `strategy: "manual"`
+
+### `scoredMergeStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
+
+Constructs a merged object by iterating every field key that appears in any entity and assigning the value from the entity with the highest similarity score. The canonical entity always has score `1.0`. Fields with `null` or `undefined` values are skipped when selecting the best value. The result is cast to `T` — callers should ensure the resulting shape is compatible.
+
+**Returns:** `MergeResult<T>` with `strategy: "scored-merge"`
+
+## Implementation Notes
+
+All three strategies are stateless functions with no side effects. They do not mutate the `group` argument. The `scoredMergeStrategy` creates a new plain object (`{}`) and populates it field-by-field, which means prototype-based instances (classes) will lose their prototype chain in the merged result. If the entity type uses class instances, callers should wrap `scoredMergeStrategy` and reconstruct the class after merging.
+
+`latestWinsStrategy` relies on ISO 8601 string sort order. Timestamps in other formats (Unix seconds as numbers, locale-formatted strings) will not sort correctly. Callers storing dates in non-ISO formats should either normalise before dedup or supply a custom `MergeStrategy` that parses timestamps explicitly.

docs/foundation/data/dedup/src/core/scoring/similarity.md

@@ -0,0 +1,94 @@
+# similarity
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+Provides string similarity algorithms used by the dedup engine to compare entity field values. Three well-known distance / similarity metrics are implemented from scratch: Levenshtein edit distance, Jaro similarity, and Jaro-Winkler similarity. A phonetic normalisation path is also included for fuzzy name matching. All functions are pure and side-effect-free, operating only on their arguments.
+
+## Features
+
+- **Levenshtein distance** — classic Wagner–Fischer dynamic-programming algorithm with O(min(|a|, |b|)) space optimisation (two-row rolling array).
+- **Jaro similarity** — accounts for matching characters within a dynamic window and transposition pairs; returns a score in [0, 1].
+- **Jaro-Winkler similarity** — extends Jaro by boosting strings that share a common prefix (up to 4 characters) using the standard p = 0.1 scaling factor.
+- **Exact match** — binary 1 / 0 equality check for deterministic field comparison.
+- **Phonetic match** — lightweight vowel-removal normalisation followed by Jaro-Winkler, useful for names that sound alike but are spelled differently.
+- **`computeSimilarity` dispatcher** — unified entry point that routes to the right algorithm based on the `MatchKey.strategy` discriminant.
+
+## Usage: Basic Example
+
+```ts
+import { levenshteinDistance, jaroWinklerSimilarity, computeSimilarity } from "@suaveplan/dedup/scoring";
+
+levenshteinDistance("kitten", "sitting"); // 3
+jaroWinklerSimilarity("MARTHA", "MARHTA"); // ≈ 0.961
+
+// Use the dispatcher for engine integration
+const score = computeSimilarity("Alice", "Alicia", "jaro-winkler"); // ≈ 0.947
+```
+
+## Usage: Advanced Example
+
+```ts
+import { computeSimilarity } from "@suaveplan/dedup/scoring";
+import type { MatchKey } from "@suaveplan/dedup";
+
+const keys: MatchKey[] = [
+  { field: "name",  weight: 0.5, strategy: "jaro-winkler", caseSensitive: false },
+  { field: "email", weight: 0.3, strategy: "exact",         caseSensitive: false },
+  { field: "city",  weight: 0.2, strategy: "phonetic",      caseSensitive: false },
+];
+
+function pairScore(a: Record<string, string>, b: Record<string, string>): number {
+  return keys.reduce((sum, k) => {
+    const va = (a[k.field] ?? "").toLowerCase();
+    const vb = (b[k.field] ?? "").toLowerCase();
+    return sum + k.weight * computeSimilarity(va, vb, k.strategy);
+  }, 0);
+}
+
+const score = pairScore(
+  { name: "John Smith", email: "j@example.com", city: "New York" },
+  { name: "Jon Smyth",  email: "j@example.com", city: "New York" },
+);
+// score ≈ 0.93 — high confidence duplicate
+```
+
+## API Reference
+
+### `levenshteinDistance(a, b): number`
+
+Returns the minimum number of single-character edits (insertions, deletions, substitutions) required to transform `a` into `b`. Always ≥ 0. Runs in O(|a| × |b|) time with O(min(|a|, |b|)) space.
+
+### `jaroSimilarity(s1, s2): number`
+
+Jaro metric. Returns a value in [0, 1]. The matching window is `floor(max(|s1|, |s2|) / 2) - 1`. Identical strings return 1; completely different strings return 0.
+
+### `jaroWinklerSimilarity(s1, s2): number`
+
+Jaro-Winkler metric. Applies a prefix bonus: `jaro + prefix * 0.1 * (1 - jaro)` where `prefix` is the length of the shared prefix capped at 4. Returns a value in [0, 1].
+
+### `exactMatch(a, b): number`
+
+Returns `1` if `a === b`, `0` otherwise. No normalisation is applied; callers should lower-case inputs if case-insensitive comparison is required.
+
+### `computeSimilarity(a, b, strategy): number`
+
+Routes to the correct algorithm. For `"levenshtein"` the raw distance is normalised to [0, 1] by dividing by `max(|a|, |b|)` and subtracting from 1. For `"phonetic"`, both strings are first passed through a vowel-removal normaliser before Jaro-Winkler scoring.
+
+**Params:**
+
+- `a` — first string
+- `b` — second string
+- `strategy` — `"exact" | "jaro-winkler" | "levenshtein" | "phonetic"`
+
+**Returns:** `number` in [0, 1]
+
+## Implementation Notes
+
+The two-row rolling array in `levenshteinDistance` avoids allocating an (|a|+1)×(|b|+1) matrix; only the previous and current rows are kept in memory. The shorter string is always placed on the column axis to minimise allocations.
+
+The Jaro matching window formula uses `Math.max(|s1|, |s2|) / 2 - 1` and is floored, matching the original Winkler (1990) paper. Some implementations use `Math.ceil`; this library matches the floor variant which produces the widely-cited benchmark values (MARTHA/MARHTA ≈ 0.944 Jaro, ≈ 0.961 Jaro-Winkler).
+
+The phonetic normaliser is intentionally lightweight — it is not a full Soundex or Metaphone implementation. It removes vowels and collapses consecutive duplicate consonants, which is sufficient for the common use case of matching first/last names with minor spelling variations. For production-grade phonetic matching, callers should supply a custom `ScoringRule`.

docs/foundation/data/dedup/src/errors/dedup-error.md

@@ -0,0 +1,82 @@
+# DedupError
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+`DedupError` is the error substrate for the `@suaveplan/dedup` package. It extends the built-in `Error` class with a stable `code` field so callers can branch on specific failure modes without parsing message strings. Every error thrown by the dedup engine or merge strategies uses this class.
+
+## Features
+
+- Typed `code` field for reliable programmatic error handling
+- Two stable error codes: `DEDUP.INVALID_CONFIG` and `DEDUP.STRATEGY_ERROR`
+- Lightweight — no external dependencies; usable in any runtime environment
+- `name` set to `'DedupError'` for readable stack traces and `instanceof` checks
+
+## Usage: Basic Example
+
+```ts
+import { DedupError, DEDUP_CODES } from '@suaveplan/dedup/errors';
+
+try {
+  engine.detect(entities);
+} catch (err) {
+  if (err instanceof DedupError && err.code === DEDUP_CODES.INVALID_CONFIG) {
+    console.error('Configuration is invalid:', err.message);
+  }
+}
+```
+
+## Usage: Advanced Example
+
+```ts
+import { DedupError, DEDUP_CODES } from '@suaveplan/dedup/errors';
+
+function safeMerge(engine: DedupEngine, group: DuplicateGroup, strategy: MergeStrategy) {
+  try {
+    return engine.merge(group, strategy);
+  } catch (err) {
+    if (err instanceof DedupError) {
+      if (err.code === DEDUP_CODES.STRATEGY_ERROR) {
+        // Fall back to manual strategy
+        return engine.merge(group, manualStrategy);
+      }
+    }
+    throw err;
+  }
+}
+```
+
+## API Reference
+
+### `DEDUP_CODES`
+
+```ts
+const DEDUP_CODES: {
+  INVALID_CONFIG: 'DEDUP.INVALID_CONFIG';
+  STRATEGY_ERROR: 'DEDUP.STRATEGY_ERROR';
+}
+```
+
+- `INVALID_CONFIG` — thrown when `DedupConfig` fails Zod validation (missing required fields, invalid weights, etc.)
+- `STRATEGY_ERROR` — thrown when a merge strategy throws an unexpected error
+
+### `DedupError`
+
+```ts
+class DedupError extends Error {
+  readonly code: string;
+  constructor(message: string, code: string);
+}
+```
+
+- `message` — human-readable description of the failure
+- `code` — one of the `DEDUP_CODES` constants
+- `name` — always `'DedupError'`
+
+## Implementation Notes
+
+This package is a leaf (no `@suaveplan/*` runtime dependencies), so it cannot depend on `@suaveplan/error`. The local `DedupError` class provides the same `code`-based discriminator pattern used throughout the project without introducing a circular dependency. If `@suaveplan/error` is added as a dependency in the future, this class should be migrated to extend `SuaveplanError`.
+
+All errors thrown by the engine and merge strategies include a `code` value drawn from `DEDUP_CODES`. Callers should always `instanceof DedupError` before reading `.code` to avoid narrowing mistakes when third-party errors propagate through the dedup boundary. The code constants are exported as a plain `const` object rather than an enum to keep the module tree-shakeable and to avoid TypeScript enum pitfalls in strict ESM environments.

docs/foundation/data/dedup/src/schemas/dedup.schema.md

@@ -0,0 +1,97 @@
+# Dedup Schemas
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+Zod schemas for the `@suaveplan/dedup` package's configuration types. These schemas serve as the single source of truth for runtime validation of `DedupConfig`, `MatchKey`, and `ScoringRule` objects supplied by callers. All engine constructors run inputs through these schemas before use, ensuring invalid configurations fail fast with clear error messages.
+
+## Features
+
+- `MatchKeySchema` — validates field path, weight (0–1), strategy, and optional case-sensitivity flag
+- `ScoringRuleSchema` — validates the name string of a scoring rule
+- `DedupConfigSchema` — composes the above with threshold and max-group-size defaults
+- Coerced defaults: `mergeThreshold` defaults to `0.85`, `maxGroupSize` to `100`
+- All schemas export inferred TypeScript types via `z.infer`
+
+## Usage: Basic Example
+
+```ts
+import { DedupConfigSchema } from '@suaveplan/dedup';
+import type { DedupConfig } from '@suaveplan/dedup';
+
+const raw = {
+  matchKeys: [{ field: 'email', weight: 1, strategy: 'exact' }],
+  scoringRules: [],
+};
+
+const config: DedupConfig = DedupConfigSchema.parse(raw);
+```
+
+## Usage: Advanced Example
+
+```ts
+import { DedupConfigSchema } from '@suaveplan/dedup';
+
+// Validate at API boundary before constructing engine
+function createEngine(raw: unknown) {
+  const result = DedupConfigSchema.safeParse(raw);
+  if (!result.success) {
+    throw new Error(`Invalid dedup config: ${result.error.message}`);
+  }
+  return new DedupEngine(result.data);
+}
+
+// Multi-key config with custom thresholds
+const config = DedupConfigSchema.parse({
+  matchKeys: [
+    { field: 'email',     weight: 0.6, strategy: 'exact' },
+    { field: 'firstName', weight: 0.2, strategy: 'jaro-winkler' },
+    { field: 'lastName',  weight: 0.2, strategy: 'jaro-winkler' },
+  ],
+  scoringRules: [],
+  mergeThreshold: 0.75,
+  maxGroupSize: 50,
+});
+```
+
+## API Reference
+
+### `MatchKeySchema`
+
+```ts
+z.object({
+  field:         z.string().min(1),
+  weight:        z.number().min(0).max(1),
+  strategy:      z.enum(['exact', 'jaro-winkler', 'levenshtein', 'phonetic']),
+  caseSensitive: z.boolean().optional(),
+})
+```
+
+### `ScoringRuleSchema`
+
+```ts
+z.object({
+  name: z.string().min(1),
+})
+```
+
+### `DedupConfigSchema`
+
+```ts
+z.object({
+  matchKeys:      z.array(MatchKeySchema).min(1),
+  scoringRules:   z.array(ScoringRuleSchema),
+  mergeThreshold: z.number().min(0).max(1).optional().default(0.85),
+  maxGroupSize:   z.number().int().positive().optional().default(100),
+})
+```
+
+Inferred types (`MatchKey`, `ScoringRule`, `DedupConfig`) are exported from the package root alongside the schemas.
+
+## Implementation Notes
+
+Schemas are defined in a dedicated `.schema.ts` file and excluded from coverage reporting (per project convention). The `ScoringRuleSchema` currently only validates `name` because the `evaluate` function is a runtime interface — Zod cannot validate function types. Callers providing custom scoring rules must implement the `ScoringRule` interface directly.
+
+The `weight` field on `MatchKeySchema` is validated to the range `[0, 1]` but the engine does not enforce that all weights sum to exactly 1.0 — this is intentional. Weights below 1 produce a score ceiling below the default threshold, which is a valid configuration for partial-match scenarios. Callers should set `mergeThreshold` accordingly when using sub-unity weight sums.