94 lines
8.7 KiB
Markdown
94 lines
8.7 KiB
Markdown
# `@suaveplan/i18n-formatter-segmenter` — `Intl.Segmenter` wrapper (NEW formatter)
|
|
|
|
**Package**: `@suaveplan/i18n-formatter-segmenter` (NEW)
|
|
**Status**: PROPOSED
|
|
**Type**: feature (new universal-tier package — brand-new addition; no source in current monolith)
|
|
**Date**: 2026-05-05
|
|
**Wave**: 3 (depends on Wave 1 foundation; required by Wave 6 core slim)
|
|
**Design doc**: `openspec/cross-cutting-designs/2026-05-05-i18n-modular-split.md` (especially §5 formatter package shape, §7.1 — closes the "two new formatters" gap from the original `i18n-formatters` change scope)
|
|
**Path**: `packages/i18n/i18n-formatter-segmenter/`
|
|
**Tier**: universal
|
|
|
|
## Why
|
|
|
|
The original `i18n-formatters` change folder (now archived under `openspec/archive/2026-05-04-i18n-formatters/`) listed `Collator` and `Segmenter` in its scope but the implementation never landed — only the four headline formatters (number, relative-time, list, date) shipped in the monolith. This change closes the second half of that gap as part of Wave 3 of the modular split.
|
|
|
|
`Intl.Segmenter` is the canonical cross-runtime API for locale-aware text segmentation by grapheme, word, or sentence. Concrete uses across the monorepo:
|
|
|
|
- **`@suaveplan/security-codes`** today scans for sensitive substrings; word-boundary-aware redaction (e.g. masking only complete words) belongs on top of `Intl.Segmenter`.
|
|
- **`@suaveplan/react-rich-text`** (Wave 3c-bound, downstream) needs grapheme-aware cursor movement (single emoji = single grapheme even across multiple code units) — `Intl.Segmenter("en-GB", { granularity: "grapheme" })` is the right primitive.
|
|
- **`@suaveplan/i18n-icu-parser`** plural-rule selection at glyph boundaries benefits from grapheme segmentation (today the parser counts code units, which is correct for digits but wrong for emoji-prefixed numbers in some experimental ICU extensions — out of scope for this package, in scope for a follow-up).
|
|
|
|
Building this formatter from scratch follows the same Wave-3 formatter shape as every sibling package (Zod-validated options, per-(locale, options-hash) cache, OTel histogram + cache-hit counter, structured error mapping, side-effect-free).
|
|
|
|
## What changes
|
|
|
|
### New package: `@suaveplan/i18n-formatter-segmenter`
|
|
|
|
- ADDED: `packages/i18n/i18n-formatter-segmenter/` package scaffold (`package.json`, `tsconfig.json`, `vite.config.ts`, `README.md`).
|
|
- ADDED: `src/formatter.ts` — `SegmenterFormatter` class wrapping `Intl.Segmenter` with construction-time Zod option validation; `segmentText(input, locale, options?)` standalone (returns `string[]` of segments — analogue of `formatList`); `formatSegments(input, locale, options?)` standalone (returns `Intl.SegmentData[]` with full `index` / `segment` / `isWordLike` metadata for each segment); `clearSegmenterCache` test-only helper. Per-(locale, granularity) cache key `${locale}|${granularity}`.
|
|
- ADDED: `src/formatter.types.ts` — `SegmenterGranularity` (`"grapheme" | "word" | "sentence"`), `SegmenterFormatterOptions` (`{ readonly granularity?: SegmenterGranularity }`), `SegmentData` (re-export of `Intl.SegmentData` with stable shape), `SegmenterFormatterCacheKey`.
|
|
- ADDED: `src/formatter.schemas.ts` — `SegmenterFormatterOptionsSchema` Zod schema validating `granularity ∈ {grapheme, word, sentence}`.
|
|
- ADDED: `src/helpers.ts` — `countGraphemes(input, locale)` (returns the grapheme count for emoji-aware character counting); `splitWords(input, locale)` returns the word-segments only (filters `isWordLike: true`); `splitSentences(input, locale)` returns sentence-granularity segments.
|
|
- ADDED: `src/observability.ts` — local OTel histogram `i18n.format.segmenter.duration_ms` + counter `i18n.format.cache_hit{formatter="segmenter"}`.
|
|
- ADDED: `src/formatter.test.ts` — covers grapheme / word / sentence granularity; emoji handled as single grapheme; class twins helper; Zod failure → `INVALID_FORMATTER_OPTIONS`; bogus locale → `UNSUPPORTED_LOCALE`.
|
|
- ADDED: `src/formatter.spec-conformance.test.ts` — every requirement scenario in this delta runs as an executable test.
|
|
- ADDED: `src/helpers.test.ts` — covers all three ergonomic helpers (especially `countGraphemes` on emoji input).
|
|
- ADDED: `src/index.ts` — explicit named re-exports.
|
|
- ADDED: `src/formatter.md`, `src/helpers.md`, `src/observability.md` — sibling docs.
|
|
|
|
### Subpath exports
|
|
|
|
- `.` — root barrel
|
|
- `./formatter` — `SegmenterFormatter` class + `segmentText` + `formatSegments` + types + schema only
|
|
- `./helpers` — `countGraphemes`, `splitWords`, `splitSentences` only
|
|
|
|
### Dependencies
|
|
|
|
- `@suaveplan/i18n-shared workspace:^` — `I18N_CODES`, `I18nError`
|
|
- `@suaveplan/types workspace:^`
|
|
- `@suaveplan/error workspace:^`
|
|
- `@opentelemetry/api catalog:observability`
|
|
- `tslib catalog:build`
|
|
- `zod catalog:forms`
|
|
- **No** dep on any other formatter package, on `@suaveplan/i18n` core, or on any backend package.
|
|
|
|
## Out of scope
|
|
|
|
- **Custom segmentation rules** (e.g. ICU `tailoring` rules) — out of scope; matches `Intl.Segmenter` capabilities only.
|
|
- **Pre-`Intl.Segmenter` runtime polyfill** — out of scope; `Intl.Segmenter` shipped in Node 16+, Bun, and every major browser. Older runtimes are not supported.
|
|
- **`SegmentIterator` streaming over `ReadableStream`** — out of scope; `Intl.Segmenter.segment()` accepts only a string. Streaming text segmentation is a follow-up (likely a separate `@suaveplan/i18n-formatter-segmenter-stream` package).
|
|
- **Removing this from the monolith** — N/A (the monolith never shipped a segmenter).
|
|
|
|
## Source mapping
|
|
|
|
| Requirement | Source file | Source lines | Notes |
|
|
|---|---|---|---|
|
|
| All requirements | (no source in monolith) | — | Brand-new package; designed from scratch using `@suaveplan/i18n-formatter-number` (`packages/i18n/i18n/src/formatting/number/number.ts`) as the shape template |
|
|
|
|
### Requirements with no source line
|
|
|
|
| Requirement | Justification |
|
|
|---|---|
|
|
| `SegmenterFormatter` class wrapping `Intl.Segmenter` | Brand-new — closes the original `i18n-formatters` change scope; required by Wave 6 default `FormatBridge` for future grapheme-aware ICU extensions |
|
|
| `segmentText`, `formatSegments` standalones | Brand-new — `segmentText` returns `string[]` (most consumer-friendly for splitting); `formatSegments` returns `SegmentData[]` (preserves `index` and `isWordLike` metadata) |
|
|
| Per-(locale, granularity) cache | Mandatory per Wave-3 formatter shape — every other formatter in the family ships this; smaller cache key (only granularity) than `NumberFormatter` because `Intl.Segmenter` has only one option |
|
|
| Zod schema, `INVALID_FORMATTER_OPTIONS`, `UNSUPPORTED_LOCALE` mappings | Mandatory per Wave-3 formatter shape |
|
|
| OTel histogram `i18n.format.segmenter.duration_ms` + cache-hit counter `i18n.format.cache_hit{formatter="segmenter"}` | Mandatory per Wave-3 formatter shape |
|
|
| Three ergonomic helpers (`countGraphemes`, `splitWords`, `splitSentences`) | Symmetric with `formatPercent` / `formatCompact` / `formatUnit` for the number formatter; covers the highest-frequency real-world segmentation use cases |
|
|
|
|
## Verification approach
|
|
|
|
- `packages/i18n/i18n-formatter-segmenter/src/formatter.test.ts` — `segmentText("hello world", "en-GB", { granularity: "word" })` returns `["hello", " ", "world"]` (or platform-equivalent); grapheme granularity treats `"👨👩👧"` as a single segment; sentence granularity splits on `.`/`?`/`!`; class wrapper twins helper output; Zod failure → `INVALID_FORMATTER_OPTIONS`; bogus locale → `UNSUPPORTED_LOCALE`; cache reuse on identical key
|
|
- `packages/i18n/i18n-formatter-segmenter/src/formatter.spec-conformance.test.ts` — every requirement scenario runs as an executable test
|
|
- `packages/i18n/i18n-formatter-segmenter/src/helpers.test.ts` — `countGraphemes("👨👩👧", "en-GB")` returns `1`; `countGraphemes("hello", "en-GB")` returns `5`; `splitWords("hello, world!", "en-GB")` returns `["hello", "world"]` (filters non-word-like segments); `splitSentences("Hi. How are you?", "en-GB")` returns two sentence segments
|
|
- `packages/i18n/i18n-formatter-segmenter/src/observability.test.ts` — histogram named `i18n.format.segmenter.duration_ms`; counter `i18n.format.cache_hit{formatter="segmenter"}` increments only on second call with same `(locale, granularity)`
|
|
- `packages/i18n/i18n-formatter-segmenter/src/index.test.ts` — every documented public export resolves; subpaths resolve
|
|
|
|
## Impact
|
|
|
|
- New universal-tier package — joins the rest of the i18n family.
|
|
- Indirect consumer in Wave 6: `@suaveplan/i18n` slim core re-exports `SegmenterFormatter` and `formatSegments` under the back-compat root barrel (per design doc §8.1 — `formatSegments` is the listed re-export).
|
|
- Closes a gap from the original `i18n-formatters` change scope (now archived under `openspec/archive/2026-05-04-i18n-formatters/`).
|
|
- `MANIFEST.md` and `CLAUDE.md` updated post-merge.
|