Files
2026-06-09 21:06:27 +01:00
..
2026-06-09 21:06:27 +01:00

@suaveplan/i18n-formatter-segmenter - Intl.Segmenter wrapper for the @suaveplan/i18n family — locale-aware text segmentation by grapheme, word, or sentence (compound-emoji-aware grapheme boundaries, ICU word-boundary rules, sentence segmentation). Per-(locale, granularity) cache, Zod-validated options, OTel histogram + cache-hit counter, structured I18nError mapping. Brand-new formatter — closes the gap from the original i18n-formatters change scope. Universal leaf package.

Summary

Wrapper around Intl.Segmenter for the @suaveplan/i18n family — locale-aware text segmentation by grapheme, word, or sentence. Compound emoji count as a single grapheme; word boundaries follow ICU rules (handles non-space-separated scripts like Thai / Khmer / Japanese correctly); sentence boundaries follow ICU sentence-break rules.

This is a brand-new formatter in the i18n family — Intl.Segmenter was listed in the original i18n-formatters change scope but was never implemented in the monolith. This package closes that gap.

Documentation tree

  • formatter — SegmenterFormatter and the segmentText / formatSegments standalone helpers expose Intl.Segmenter for the `@suav…
    • formatter — SegmenterFormatter and the segmentText / formatSegments standalone helpers expose Intl.Segmenter for the `@suav…
  • helpers — The three ergonomic helpers (countGraphemes, splitWords, splitSentences) pre-bind segmentText and `formatSegmen…
    • helpers — The three ergonomic helpers (countGraphemes, splitWords, splitSentences) pre-bind segmentText and `formatSegmen…
  • observability — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…
    • observability — The Segmenter formatter exports two OpenTelemetry instruments and a small in-memory state ring so spec-conformance test…

Why this exists

The 14-package i18n modular split (cross-cutting design 2026-05-05-i18n-modular-split.md) lifts each Intl.* wrapper into its own sibling package and adds the two formatters (Collator + Segmenter) that the original change scope listed but never shipped. Locale-aware segmentation is the right primitive for character-limited inputs (tweet-style counters), full-text search tokenisation, and sentence-aware UI truncation.

Install

bun add @suaveplan/i18n-formatter-segmenter

Quick Start

import {
    SegmenterFormatter,
    segmentText,
    formatSegments,
    countGraphemes,
    splitWords,
    splitSentences,
} from "@suaveplan/i18n-formatter-segmenter";

segmentText("ab", "en-GB"); // ["a", "b"]
segmentText("hello world", "en-GB", { granularity: "word" });
// ["hello", " ", "world"]

countGraphemes("👨‍👩‍👧", "en-GB"); // 1 (single grapheme)
splitWords("hello, world!", "en-GB"); // ["hello", "world"]
splitSentences("Hi. How are you?", "en-GB");
// ["Hi.", "How are you?"]

const f = new SegmenterFormatter({ granularity: "word" });
f.segmentWithMetadata("hi!", "en-GB");
// [
//   { segment: "hi", index: 0, isWordLike: true },
//   { segment: "!", index: 2, isWordLike: false },
// ]

Modules

Subpath Description
@suaveplan/i18n-formatter-segmenter Root barrel — class + standalones + helpers + observability
@suaveplan/i18n-formatter-segmenter/formatter SegmenterFormatter, segmentText, formatSegments, types, schema
@suaveplan/i18n-formatter-segmenter/helpers countGraphemes, splitWords, splitSentences

Modules Documentation

Public Surface (snapshot)

Export Kind Origin
SegmenterFormatter class formatter.ts
segmentText function formatter.ts
formatSegments function formatter.ts
clearSegmenterCache function formatter.ts
SegmenterFormatterOptionsSchema Zod schema formatter.schemas.ts
SegmenterFormatterOptions type formatter.types.ts
SegmenterGranularity type formatter.types.ts
SegmentData type formatter.types.ts
SegmenterFormatterCacheKey type formatter.types.ts
countGraphemes function helpers.ts
splitWords function helpers.ts
splitSentences function helpers.ts
recordSegmenterFormat function observability.ts
recordSegmenterCacheHit function observability.ts
getSegmenterMetricsState function observability.ts
resetSegmenterMetricsState function observability.ts

Versioning

Part of the 14-package i18n modular-split rollout. Stays at 0.1.0 until the slim core (Wave 6) cuts the family 1.0.0 release.