first commit

2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions
--- a/docs/localization/i18n/i18n-formatter-segmenter/README.md
+++ b/docs/localization/i18n/i18n-formatter-segmenter/README.md
@@ -0,0 +1,111 @@
+# @suaveplan/i18n-formatter-segmenter - Intl.Segmenter wrapper for the @suaveplan/i18n family — locale-aware text segmentation by grapheme, word, or sentence (compound-emoji-aware grapheme boundaries, ICU word-boundary rules, sentence segmentation). Per-(locale, granularity) cache, Zod-validated options, OTel histogram + cache-hit counter, structured I18nError mapping. Brand-new formatter — closes the gap from the original i18n-formatters change scope. Universal leaf package.
+
+## Summary
+
+Wrapper around `Intl.Segmenter` for the `@suaveplan/i18n` family —
+locale-aware text segmentation by grapheme, word, or sentence.
+Compound emoji count as a single grapheme; word boundaries follow
+ICU rules (handles non-space-separated scripts like Thai / Khmer /
+Japanese correctly); sentence boundaries follow ICU sentence-break
+rules.
+
+This is a **brand-new formatter** in the i18n family — `Intl.Segmenter`
+was listed in the original `i18n-formatters` change scope but was
+never implemented in the monolith. This package closes that gap.
+
+<!-- begin:auto-docs -->
+## Documentation tree
+
+- [formatter](./src/formatter/formatter.md) — `SegmenterFormatter` and the `segmentText` / `formatSegments` standalone helpers expose `Intl.Segmenter` for the `@suav…
+  - [formatter](./src/formatter/formatter.md) — `SegmenterFormatter` and the `segmentText` / `formatSegments` standalone helpers expose `Intl.Segmenter` for the `@suav…
+- [helpers](./src/helpers/helpers.md) — The three ergonomic helpers (`countGraphemes`, `splitWords`, `splitSentences`) pre-bind `segmentText` and `formatSegmen…
+  - [helpers](./src/helpers/helpers.md) — The three ergonomic helpers (`countGraphemes`, `splitWords`, `splitSentences`) pre-bind `segmentText` and `formatSegmen…
+- [observability](./src/observability/observability.md) — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…
+  - [observability](./src/observability/observability.md) — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…
+<!-- end:auto-docs -->
+
+## Why this exists
+
+The 14-package i18n modular split (cross-cutting design
+`2026-05-05-i18n-modular-split.md`) lifts each `Intl.*` wrapper into
+its own sibling package and adds the two formatters
+(`Collator` + `Segmenter`) that the original change scope listed but
+never shipped. Locale-aware segmentation is the right primitive for
+character-limited inputs (tweet-style counters), full-text search
+tokenisation, and sentence-aware UI truncation.
+
+## Install
+
+```bash
+bun add @suaveplan/i18n-formatter-segmenter
+```
+
+## Quick Start
+
+```ts
+import {
+    SegmenterFormatter,
+    segmentText,
+    formatSegments,
+    countGraphemes,
+    splitWords,
+    splitSentences,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+segmentText("ab", "en-GB"); // ["a", "b"]
+segmentText("hello world", "en-GB", { granularity: "word" });
+// ["hello", " ", "world"]
+
+countGraphemes("👨‍👩‍👧", "en-GB"); // 1 (single grapheme)
+splitWords("hello, world!", "en-GB"); // ["hello", "world"]
+splitSentences("Hi. How are you?", "en-GB");
+// ["Hi.", "How are you?"]
+
+const f = new SegmenterFormatter({ granularity: "word" });
+f.segmentWithMetadata("hi!", "en-GB");
+// [
+//   { segment: "hi", index: 0, isWordLike: true },
+//   { segment: "!", index: 2, isWordLike: false },
+// ]
+```
+
+## Modules
+
+| Subpath | Description |
+|---|---|
+| `@suaveplan/i18n-formatter-segmenter` | Root barrel — class + standalones + helpers + observability |
+| `@suaveplan/i18n-formatter-segmenter/formatter` | `SegmenterFormatter`, `segmentText`, `formatSegments`, types, schema |
+| `@suaveplan/i18n-formatter-segmenter/helpers` | `countGraphemes`, `splitWords`, `splitSentences` |
+
+## Modules Documentation
+
+- [formatter.md](src/formatter.md) — `SegmenterFormatter`, `segmentText`, `formatSegments`, `clearSegmenterCache`
+- [helpers.md](src/helpers.md) — three ergonomic helpers
+- [observability.md](src/observability.md) — OTel histogram + cache-hit counter
+
+## Public Surface (snapshot)
+
+| Export | Kind | Origin |
+|---|---|---|
+| `SegmenterFormatter` | class | `formatter.ts` |
+| `segmentText` | function | `formatter.ts` |
+| `formatSegments` | function | `formatter.ts` |
+| `clearSegmenterCache` | function | `formatter.ts` |
+| `SegmenterFormatterOptionsSchema` | Zod schema | `formatter.schemas.ts` |
+| `SegmenterFormatterOptions` | type | `formatter.types.ts` |
+| `SegmenterGranularity` | type | `formatter.types.ts` |
+| `SegmentData` | type | `formatter.types.ts` |
+| `SegmenterFormatterCacheKey` | type | `formatter.types.ts` |
+| `countGraphemes` | function | `helpers.ts` |
+| `splitWords` | function | `helpers.ts` |
+| `splitSentences` | function | `helpers.ts` |
+| `recordSegmenterFormat` | function | `observability.ts` |
+| `recordSegmenterCacheHit` | function | `observability.ts` |
+| `getSegmenterMetricsState` | function | `observability.ts` |
+| `resetSegmenterMetricsState` | function | `observability.ts` |
+
+## Versioning
+
+Part of the 14-package i18n modular-split rollout. Stays at `0.1.0`
+until the slim core (Wave 6) cuts the family `1.0.0` release.
+