first commit

docs/localization/i18n/i18n-formatter-segmenter/README.md

@@ -0,0 +1,111 @@
+# @suaveplan/i18n-formatter-segmenter - Intl.Segmenter wrapper for the @suaveplan/i18n family — locale-aware text segmentation by grapheme, word, or sentence (compound-emoji-aware grapheme boundaries, ICU word-boundary rules, sentence segmentation). Per-(locale, granularity) cache, Zod-validated options, OTel histogram + cache-hit counter, structured I18nError mapping. Brand-new formatter — closes the gap from the original i18n-formatters change scope. Universal leaf package.
+
+## Summary
+
+Wrapper around `Intl.Segmenter` for the `@suaveplan/i18n` family —
+locale-aware text segmentation by grapheme, word, or sentence.
+Compound emoji count as a single grapheme; word boundaries follow
+ICU rules (handles non-space-separated scripts like Thai / Khmer /
+Japanese correctly); sentence boundaries follow ICU sentence-break
+rules.
+
+This is a **brand-new formatter** in the i18n family — `Intl.Segmenter`
+was listed in the original `i18n-formatters` change scope but was
+never implemented in the monolith. This package closes that gap.
+
+<!-- begin:auto-docs -->
+## Documentation tree
+
+- [formatter](./src/formatter/formatter.md) — `SegmenterFormatter` and the `segmentText` / `formatSegments` standalone helpers expose `Intl.Segmenter` for the `@suav…
+  - [formatter](./src/formatter/formatter.md) — `SegmenterFormatter` and the `segmentText` / `formatSegments` standalone helpers expose `Intl.Segmenter` for the `@suav…
+- [helpers](./src/helpers/helpers.md) — The three ergonomic helpers (`countGraphemes`, `splitWords`, `splitSentences`) pre-bind `segmentText` and `formatSegmen…
+  - [helpers](./src/helpers/helpers.md) — The three ergonomic helpers (`countGraphemes`, `splitWords`, `splitSentences`) pre-bind `segmentText` and `formatSegmen…
+- [observability](./src/observability/observability.md) — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…
+  - [observability](./src/observability/observability.md) — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…
+<!-- end:auto-docs -->
+
+## Why this exists
+
+The 14-package i18n modular split (cross-cutting design
+`2026-05-05-i18n-modular-split.md`) lifts each `Intl.*` wrapper into
+its own sibling package and adds the two formatters
+(`Collator` + `Segmenter`) that the original change scope listed but
+never shipped. Locale-aware segmentation is the right primitive for
+character-limited inputs (tweet-style counters), full-text search
+tokenisation, and sentence-aware UI truncation.
+
+## Install
+
+```bash
+bun add @suaveplan/i18n-formatter-segmenter
+```
+
+## Quick Start
+
+```ts
+import {
+    SegmenterFormatter,
+    segmentText,
+    formatSegments,
+    countGraphemes,
+    splitWords,
+    splitSentences,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+segmentText("ab", "en-GB"); // ["a", "b"]
+segmentText("hello world", "en-GB", { granularity: "word" });
+// ["hello", " ", "world"]
+
+countGraphemes("👨‍👩‍👧", "en-GB"); // 1 (single grapheme)
+splitWords("hello, world!", "en-GB"); // ["hello", "world"]
+splitSentences("Hi. How are you?", "en-GB");
+// ["Hi.", "How are you?"]
+
+const f = new SegmenterFormatter({ granularity: "word" });
+f.segmentWithMetadata("hi!", "en-GB");
+// [
+//   { segment: "hi", index: 0, isWordLike: true },
+//   { segment: "!", index: 2, isWordLike: false },
+// ]
+```
+
+## Modules
+
+| Subpath | Description |
+|---|---|
+| `@suaveplan/i18n-formatter-segmenter` | Root barrel — class + standalones + helpers + observability |
+| `@suaveplan/i18n-formatter-segmenter/formatter` | `SegmenterFormatter`, `segmentText`, `formatSegments`, types, schema |
+| `@suaveplan/i18n-formatter-segmenter/helpers` | `countGraphemes`, `splitWords`, `splitSentences` |
+
+## Modules Documentation
+
+- [formatter.md](src/formatter.md) — `SegmenterFormatter`, `segmentText`, `formatSegments`, `clearSegmenterCache`
+- [helpers.md](src/helpers.md) — three ergonomic helpers
+- [observability.md](src/observability.md) — OTel histogram + cache-hit counter
+
+## Public Surface (snapshot)
+
+| Export | Kind | Origin |
+|---|---|---|
+| `SegmenterFormatter` | class | `formatter.ts` |
+| `segmentText` | function | `formatter.ts` |
+| `formatSegments` | function | `formatter.ts` |
+| `clearSegmenterCache` | function | `formatter.ts` |
+| `SegmenterFormatterOptionsSchema` | Zod schema | `formatter.schemas.ts` |
+| `SegmenterFormatterOptions` | type | `formatter.types.ts` |
+| `SegmenterGranularity` | type | `formatter.types.ts` |
+| `SegmentData` | type | `formatter.types.ts` |
+| `SegmenterFormatterCacheKey` | type | `formatter.types.ts` |
+| `countGraphemes` | function | `helpers.ts` |
+| `splitWords` | function | `helpers.ts` |
+| `splitSentences` | function | `helpers.ts` |
+| `recordSegmenterFormat` | function | `observability.ts` |
+| `recordSegmenterCacheHit` | function | `observability.ts` |
+| `getSegmenterMetricsState` | function | `observability.ts` |
+| `resetSegmenterMetricsState` | function | `observability.ts` |
+
+## Versioning
+
+Part of the 14-package i18n modular-split rollout. Stays at `0.1.0`
+until the slim core (Wave 6) cuts the family `1.0.0` release.
+

docs/localization/i18n/i18n-formatter-segmenter/src/formatter/formatter.md

@@ -0,0 +1,127 @@
+# SegmenterFormatter
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+`SegmenterFormatter` and the `segmentText` / `formatSegments`
+standalone helpers expose `Intl.Segmenter` for the `@suaveplan/i18n`
+family at the canonical Wave-3 formatter surface. Locale-aware text
+segmentation by grapheme, word, or sentence is the right primitive
+for emoji-aware character counts, ICU word-boundary tokenisation,
+and sentence splitting.
+
+This is a brand-new formatter — `Intl.Segmenter` was listed in the
+original `i18n-formatters` change scope but never implemented in the
+monolith. The split package closes that gap with the canonical Wave-3
+shape.
+
+## Features
+
+- `SegmenterFormatter` class with `segment(input, locale)` and `segmentWithMetadata(input, locale)` methods
+- `segmentText(input, locale, options?)` standalone — returns plain `string[]`
+- `formatSegments(input, locale, options?)` standalone — returns `SegmentData[]` with `segment` / `index` / `isWordLike` metadata
+- Three granularity values: `"grapheme"` (default), `"word"`, `"sentence"`
+- Construction-time Zod validation with `INVALID_FORMATTER_OPTIONS` mapping
+- `Intl.Segmenter` `RangeError` mapped to `UNSUPPORTED_LOCALE` with original error attached as `cause`
+- Runtime-availability check: when the host runtime lacks `Intl.Segmenter`, throws `I18nError(UNSUPPORTED_LOCALE)` with message key `intl.segmenter.unavailable`
+- Per-(locale, granularity) cache — repeated calls reuse the same `Intl.Segmenter`
+- `clearSegmenterCache()` test-only helper
+
+## Usage: Basic Example
+
+```ts
+import {
+    formatSegments,
+    segmentText,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+segmentText("ab", "en-GB"); // ["a", "b"]
+segmentText("hello world", "en-GB", { granularity: "word" });
+// ["hello", " ", "world"]
+
+formatSegments("hi!", "en-GB", { granularity: "word" });
+// [
+//   { segment: "hi", index: 0, isWordLike: true },
+//   { segment: "!", index: 2, isWordLike: false },
+// ]
+```
+
+## Usage: Advanced Example
+
+```ts
+import {
+    SegmenterFormatter,
+    type SegmenterFormatterOptions,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+// Emoji-aware character counter — handles compound emoji
+// (family-of-three ZWJ sequence is ONE grapheme, not 7+ code units).
+const f = new SegmenterFormatter({ granularity: "grapheme" });
+const count = f.segment("👨‍👩‍👧 hello", "en-GB").length;
+// count === 8 (1 emoji + space + 5 letters + ... locale-correct)
+
+// Build a word-level token stream for full-text search indexing.
+const search = new SegmenterFormatter({ granularity: "word" });
+const tokens = search
+    .segmentWithMetadata("Quick brown fox.", "en-GB")
+    .filter((s) => s.isWordLike === true)
+    .map((s) => s.segment.toLowerCase());
+// ["quick", "brown", "fox"]
+
+// Sentence-aware truncation for chat previews.
+const sentences = new SegmenterFormatter({ granularity: "sentence" });
+const first = sentences.segment(
+    "Hello there. How are you today?",
+    "en-GB"
+)[0];
+// "Hello there. "
+```
+
+## API Reference
+
+### `segmentText(input, locale, options?): string[]`
+
+Standalone helper. Returns a plain `string[]` of segments. Default
+`granularity` is `"grapheme"`. Throws
+`I18nError(INVALID_FORMATTER_OPTIONS)` on Zod failure;
+`I18nError(UNSUPPORTED_LOCALE)` on runtime unavailability or
+`Intl.Segmenter` `RangeError`.
+
+### `formatSegments(input, locale, options?): SegmentData[]`
+
+Standalone helper. Returns `SegmentData[]` preserving each segment's
+`index` and (for `granularity: "word"`) `isWordLike` flag. The
+`isWordLike` field is omitted on `grapheme` and `sentence`
+granularities (matches `Intl.Segmenter` semantics).
+
+### `class SegmenterFormatter`
+
+- `constructor(options?: SegmenterFormatterOptions)` — Zod-validates options
+- `segment(input, locale): string[]`
+- `segmentWithMetadata(input, locale): SegmentData[]`
+
+### `clearSegmenterCache(): void`
+
+Test-only helper that empties the per-(locale, granularity) cache.
+
+## Implementation Notes
+
+- The cache key is `${locale}|${granularity}` — only two dimensions
+  because `Intl.Segmenter` accepts only `granularity` as a configurable
+  option. Cache size stays small even with many locales.
+- Runtime-availability guard checks `typeof Intl.Segmenter === "function"`
+  before construction. This catches Node < 16 / very-old browsers /
+  test stubs that null out `Intl.Segmenter`. The error code reuses
+  `UNSUPPORTED_LOCALE` (the closest fit in the `I18N_CODES`
+  registry) with a discriminating message key
+  `intl.segmenter.unavailable` so consumers can branch on the message.
+- `Intl.Segmenter` shipped in Node 16+, Bun, and every current major
+  browser. The package does not provide a polyfill.
+- The `segmentWithMetadata` method exists on the class as the
+  programmatic-friendly companion to the standalone `formatSegments`
+  — both produce identical output.
+- Compound emoji (e.g. `"👨‍👩‍👧"`, the family-of-three ZWJ sequence)
+  count as a single grapheme per ECMA-402.

docs/localization/i18n/i18n-formatter-segmenter/src/helpers/helpers.md

@@ -0,0 +1,105 @@
+# Segmenter ergonomic helpers
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+The three ergonomic helpers (`countGraphemes`, `splitWords`,
+`splitSentences`) pre-bind `segmentText` and `formatSegments` with
+the appropriate granularity for the most common UI patterns. They
+exist because counting graphemes (for character limits) and splitting
+words (for token streams) are universal needs that should not require
+remembering the `granularity` discriminant.
+
+## Features
+
+- `countGraphemes(input, locale)` — emoji-aware grapheme count
+- `splitWords(input, locale)` — word-segments only (filters non-word-like)
+- `splitSentences(input, locale)` — sentence-segments, trimmed, empty entries dropped
+
+## Usage: Basic Example
+
+```ts
+import {
+    countGraphemes,
+    splitSentences,
+    splitWords,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+countGraphemes("hello", "en-GB"); // 5
+countGraphemes("👨‍👩‍👧", "en-GB"); // 1 (single grapheme)
+
+splitWords("hello, world!", "en-GB"); // ["hello", "world"]
+splitSentences("Hi. How are you?", "en-GB"); // ["Hi.", "How are you?"]
+```
+
+## Usage: Advanced Example
+
+```ts
+import {
+    countGraphemes,
+    splitSentences,
+    splitWords,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+// Tweet-style character limit that respects compound emoji.
+const MAX_CHARS = 280;
+function tweetFits(text: string, locale: string) {
+    return countGraphemes(text, locale) <= MAX_CHARS;
+}
+
+tweetFits("👨‍👩‍👧 hello world", "en-GB"); // true (8 graphemes)
+
+// Word-frequency map for full-text indexing.
+function wordFrequencies(input: string, locale: string) {
+    const out: Record<string, number> = {};
+    for (const word of splitWords(input, locale)) {
+        const k = word.toLowerCase();
+        out[k] = (out[k] ?? 0) + 1;
+    }
+    return out;
+}
+
+// Sentence-aware preview that always ends at a sentence boundary.
+function preview(input: string, locale: string, maxSentences: number) {
+    return splitSentences(input, locale).slice(0, maxSentences).join(" ");
+}
+```
+
+## API Reference
+
+### `countGraphemes(input, locale): number`
+
+Returns the grapheme count. Uses `granularity: "grapheme"` so
+compound emoji and combining-character sequences count as one.
+
+### `splitWords(input, locale): string[]`
+
+Returns word-segments only. Internally calls `formatSegments` with
+`granularity: "word"` and filters segments where `isWordLike === true`.
+Punctuation, whitespace, and other non-word-like segments are
+dropped.
+
+### `splitSentences(input, locale): string[]`
+
+Returns sentence-segments. Each entry is trimmed of leading/trailing
+whitespace; empty entries are dropped. Punctuation that ends a
+sentence (`.`/`?`/`!`) is included with the preceding sentence per
+ICU sentence-boundary rules.
+
+## Implementation Notes
+
+- Each helper is a thin wrapper around `segmentText` / `formatSegments`.
+  The underlying per-(locale, granularity) cache means calling
+  `splitWords` and `splitSentences` for the same locale uses two
+  cached `Intl.Segmenter` instances (one per granularity) — minimal
+  per-call cost.
+- `splitSentences` trim-and-filter chain is deliberate: ICU's
+  sentence-boundary rules emit each sentence WITH its trailing
+  whitespace; trimming gives the consumer the expected "sentence
+  text" without trailing space, and filtering drops the trailing
+  empty segment that `Intl.Segmenter` sometimes emits for terminator-
+  only input.
+- The helpers are pure functions with no shared state.

docs/localization/i18n/i18n-formatter-segmenter/src/observability/observability.md

@@ -0,0 +1,98 @@
+# Segmenter observability
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+The Segmenter formatter exports two OpenTelemetry instruments and a
+small in-memory state ring so spec-conformance tests can assert exact
+attribute payloads without standing up a full SDK exporter. The
+instrument names match the canonical Wave-3 formatter contract —
+every sibling formatter emits the same two metric names so dashboards
+and alerts compose cleanly across the family.
+
+## Features
+
+- Histogram `i18n.format.segmenter.duration_ms` (unit `ms`) — per-segment wall-clock time
+- Counter `i18n.format.cache_hit{formatter="segmenter"}` — increments on cache reuse only
+- In-memory state ring (`getSegmenterMetricsState()`) for spec-conformance
+- `resetSegmenterMetricsState()` test-only helper
+
+## Usage: Basic Example
+
+```ts
+import { metrics } from "@opentelemetry/api";
+import { MeterProvider } from "@opentelemetry/sdk-metrics";
+
+const provider = new MeterProvider({ readers: [/* reader */] });
+metrics.setGlobalMeterProvider(provider);
+
+import { segmentText } from "@suaveplan/i18n-formatter-segmenter";
+segmentText("hello", "en-GB", { granularity: "word" });
+// emits via your reader
+```
+
+## Usage: Advanced Example
+
+```ts
+import { describe, expect, it } from "@suaveplan/testing/runner";
+import {
+    clearSegmenterCache,
+    segmentText,
+} from "@suaveplan/i18n-formatter-segmenter";
+import {
+    getSegmenterMetricsState,
+    resetSegmenterMetricsState,
+} from "@suaveplan/i18n-formatter-segmenter";
+
+it("emits histogram + cache-hit counter", () => {
+    clearSegmenterCache();
+    resetSegmenterMetricsState();
+    segmentText("hello", "en-GB", { granularity: "word" });
+    segmentText("hello", "en-GB", { granularity: "word" });
+    const state = getSegmenterMetricsState();
+    expect(state.durationSamples).toHaveLength(2);
+    expect(state.cacheHitSamples).toHaveLength(1);
+    expect(state.durationSamples[0]?.attributes).toMatchObject({
+        locale: "en-GB",
+        granularity: "word",
+    });
+});
+```
+
+## API Reference
+
+### `recordSegmenterFormat(locale, granularity, durationMs): void`
+
+Records the duration histogram sample. Tags: `locale`, `granularity`.
+Called internally by `segmentText` / `formatSegments`.
+
+### `recordSegmenterCacheHit(): void`
+
+Increments the cache-hit counter with the constant tag
+`{ formatter: "segmenter" }`.
+
+### `getSegmenterMetricsState(): SegmenterMetricsState`
+
+Returns the in-memory ring of `durationSamples` and
+`cacheHitSamples`. Used by spec-conformance tests.
+
+### `resetSegmenterMetricsState(): void`
+
+Clears both ring buffers. Test-only helper.
+
+## Implementation Notes
+
+- The OTel meter is named `suaveplan.i18n.formatter.segmenter`. Every
+  formatter in the family has its own meter so OTel resource
+  attributes can be tied per-package if a consumer chooses.
+- `granularity` defaults to `"grapheme"` in the histogram tag when
+  the caller omits it; `locale` is always required so it's never
+  defaulted.
+- Sample dashboard query (PromQL/Grafana over OTLP):
+  `histogram_quantile(0.99, sum(rate(i18n_format_segmenter_duration_ms_bucket[5m])) by (le, locale, granularity))`
+- The in-memory ring is intentionally unbounded — tests are short
+  and call `resetSegmenterMetricsState()` between cases. A
+  long-running test loop should call reset to avoid memory growth.