# @suaveplan/dedup - Entity deduplication engine with pluggable match keys, scoring rules, Jaro-Winkler/Levenshtein similarity, and merge strategies

## Summary

Entity deduplication engine for detecting and merging duplicate records in any data set. Given a list of objects with varying representations of the same real-world entity (e.g. `"John Smith"` vs `"J. Smith"`, or the same email in different capitalisation), the `DedupEngine` scores pairs using a weighted combination of field-level similarity algorithms — Jaro-Winkler, Levenshtein, exact match, or phonetic — groups records into connected components via union-find, and merges each group with a pluggable strategy (`latestWins`, `manual`, or `scoredMerge`). Use this package for ETL pipelines, contact deduplication, and data cleansing. For domain-name-specific normalisation and alias resolution use `@suaveplan/domain-normalizer`; for string-similarity primitives only, the scoring module is exported as a standalone subpath.

<!-- begin:auto-docs -->
## Documentation tree

- [engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
  - [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
  - [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
  - [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
  - [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
  - [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
<!-- end:auto-docs -->  - [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_  - [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_  - [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_  - [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_  - [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_
## Modules

| Module | Description |
|---|---|
| [scoring](src/core/scoring/similarity.md) | Levenshtein, Jaro, Jaro-Winkler, exact, and phonetic similarity algorithms |
| [engine](src/core/engine/dedup-engine.md) | `DedupEngine` — detect duplicate groups and merge them |
| [merge](src/core/merge/merge-strategies.md) | Built-in merge strategies: latest-wins, manual, scored |

## Quick Start

```ts
import { DedupEngine } from "@suaveplan/dedup/engine";
import { latestWinsStrategy } from "@suaveplan/dedup/merge";

const engine = new DedupEngine({
  matchKeys: [
    { field: "name",  weight: 0.6, strategy: "jaro-winkler" },
    { field: "email", weight: 0.4, strategy: "exact" },
  ],
  scoringRules: [],
  mergeThreshold: 0.85,
});

const groups = engine.detect(records);
for (const group of groups) {
  const result = engine.merge(group, latestWinsStrategy);
  console.log(result.merged);
}
```