suaveplan/openspec/changes/archive/2026-05-04-i18n-storage-backends/proposal.md

i18n storage-backed translation loaders

Package: @suaveplan/i18n Status: PROPOSED Type: feature Date: 2026-05-04

Why

The current MessageCatalog.load(locale, namespace?, loader) accepts a single async function and that is the entire resource-loading surface. Every consumer that wants translations from anything other than statically-imported objects has to hand-roll the loader, the cache key, the namespace-merge, and the error mapping. Compare this against i18next-http-backend, i18next-fs-backend, i18next-resources-to-backend, i18next-multiload-backend-adapter, chained-backend, and localstorage-backend — eight battle-tested backends behind one published interface — and @suaveplan/i18n looks anaemic.

Two specific consumer workflows are blocked today:

1. Many small JSON files in many places. Translators prefer locales/<locale>/<namespace>.json (one file per namespace per locale) over a single mega-file. Today, every consumer who wants this writes their own glob+parse+merge loop and feeds the result into mergeMessages.
2. Translations stored in a database, S3, OPFS, IndexedDB, or any other key-value store. The genesis monorepo already ships @suaveplan/storage's AbstractStorage contract with eight backends (memory, s3, opfs, indexeddb, node-fs, sqlite, garage, content-addressed). A translator-friendly admin UI that writes new strings to a database, or a runtime that pulls strings from S3 with content-addressed cache busting, should be one config line — not a fresh integration each time.

This change introduces a formal Backend plugin interface, ships four built-in backends, and wires the catalog so that MessageCatalog.load and on-demand t('auth:login.title') can both resolve through any registered backend. The StorageBackend is the headline addition: any AbstractStorage instance becomes a translation source.

What changes

@suaveplan/i18n

Backend interface

• ADDED: Backend interface in src/core/backend/backend.types.ts:
  • read(locale, namespace) → Promise<TranslationRecord> — load a single namespace
  • readMulti?(locale, namespaces[]) → Promise<Record<string, TranslationRecord>> — bulk load (optional optimisation; falls back to N read calls when absent)
  • save?(locale, namespace, record) → Promise<void> — write back (used by missing-key reporter and admin tooling; optional)
  • lastModified?(locale, namespace) → Promise<number | undefined> — for cache-busting; optional
  • init?(catalog) → Promise<void> — one-shot setup hook called when registered
  • dispose?() → Promise<void> — teardown hook
  • name: string — stable identifier for telemetry and chained-backend ordering
• ADDED: BackendError codes in the I18N.* namespace via defineCodes:
  • I18N.BACKEND_READ_FAILED — read/readMulti rejected or returned a non-record value
  • I18N.BACKEND_WRITE_FAILED — save rejected
  • I18N.BACKEND_NOT_REGISTERED — t() requested a namespace from a backend that is not registered
  • I18N.BACKEND_LOCALE_NOT_AVAILABLE — backend reported "locale unavailable" (distinct from a transport error)

Built-in backends

• ADDED: MemoryBackend (src/core/backend/memory/memory-backend.ts) — in-memory Map<locale, Map<namespace, TranslationRecord>>. Useful for tests and seeded defaults.
• ADDED: HttpBackend (src/core/backend/http/http-backend.ts) — fetches loadPath URL with {locale} / {namespace} placeholders. Configurable fetchFn (defaults to global fetch), requestOptions, parsePayload (defaults to res.json()), crossDomain flag, customHeaders callback. Forwards 5xx / network failures via @suaveplan/retry RetryPolicy (caller-supplied; defaults to no retry).
• ADDED: StorageBackend (src/core/backend/storage/storage-backend.ts) — wraps any AbstractStorage from @suaveplan/storage. Layouts:
  • single-key-per-namespace (default): key = ${prefix}/${locale}/${namespace}.json, value = JSON-encoded TranslationRecord.
  • single-key-per-locale: key = ${prefix}/${locale}.json, value = Record<namespace, TranslationRecord> (matches the i18next single-file shape).
  • flat-keys: key = ${prefix}:${locale}:${namespace}:${dotPath}, value = string (one storage key per leaf — works well with content-addressed/CDN backends and lets you ship deltas). Pluggable codec (encode/decode) so callers can swap JSON for MessagePack via @suaveplan/serialize.
• ADDED: MultiFileFsBackend (src/core/backend/multi-file-fs/multi-file-fs-backend.ts) — Node/Bun-only filesystem backend. Reads <root>/<locale>/<namespace>.json (configurable layout: flat, nested, glob). Watches for changes via node:fs.watch (optional, opt-in via watch: true); when a file changes, emits loaded event so callers can refresh React trees. Supports JSON, JSON5, YAML (when the yaml peer is installed), and PO (when gettext-parser peer is installed).
• ADDED: ChainedBackend (src/core/backend/chained/chained-backend.ts) — composes N backends. read walks them in order until one succeeds; readMulti fans out concurrently and merges (later backends win on conflict, configurable). Caching layer between backends (e.g. Chained([cacheBackend, networkBackend])) follows the i18next pattern: write-through to all earlier backends on every successful network read.
• ADDED: BackendPlugin factory exports — memoryBackend(opts), httpBackend(opts), storageBackend(opts), multiFileFsBackend(opts), chainedBackend(opts). Use Backend directly for advanced cases.

Catalog integration

• MODIFIED: MessageCatalog constructor accepts backend?: Backend (single backend) or backends?: Backend[] (auto-wrapped in ChainedBackend). When provided, the catalog registers them via the plugin manager (see i18n-completion).
• ADDED: MessageCatalog.preload(locales, namespaces?) → Promise<void> — bulk-loads via readMulti when supported, with concurrent-call dedup keyed on (locale, namespace).
• ADDED: MessageCatalog.lazyLoad: boolean option (default true when a backend is registered, false otherwise) — when on, get(key) triggers an in-flight backend read for the missing namespace and returns a Promise via the new getAsync(key) overload.
• ADDED: MessageCatalog.getAsync(key, locale?) → Promise<LookupResult> — async sibling of get that awaits in-flight backend loads. Synchronous get/t continue to behave as today (return undefined / key when missing) so existing call sites are unchanged.
• ADDED: MessageCatalog.invalidate(locale?, namespace?) — drops cached namespaces so the next read re-fetches via the backend.
• ADDED: MessageCatalog.saveMissing(locale, key, defaultValue?) → Promise<void> — when a backend implements save, this records a missing key for translator pickup. Wired automatically when saveMissing: true is passed in catalog options.
• MODIFIED: MessageCatalog.load(locale, namespace?, loader?) — loader becomes optional; when omitted, the catalog routes through the registered backend. Existing call sites that pass a loader keep working unchanged.

Zod schemas

• ADDED: BackendSchema (Zod) — validates the structural shape of a Backend (function presence, name string).
• ADDED: HttpBackendOptionsSchema, StorageBackendOptionsSchema, MultiFileFsBackendOptionsSchema, ChainedBackendOptionsSchema — option validation at construction.

OTel instrumentation

• ADDED: i18n.backend.read.duration_ms histogram — labels: backend.name, locale, namespace, outcome (hit | miss | error).
• ADDED: i18n.backend.read.total counter — labels: backend.name, outcome.
• ADDED: i18n.backend.write.total counter — labels: backend.name, outcome.
• ADDED: i18n.backend.cache.size gauge — number of (locale, namespace) pairs currently in the catalog's in-memory store.

Out of scope

• CDN-side compilation (LinguiJS / FormatJS approach where messages are compiled to JS modules at build time) — separate change once the backend interface stabilises. The flat-keys layout in StorageBackend already supports the closest equivalent.
• TMS adapters (Phrase, Lokalise, Crowdin) — these are downstream consumers of Backend + save(); ship in a separate i18n-tms-adapters change once translator workflows are scoped.
• Vite/Rollup plugin for build-time JSON inlining — separate change. The MultiFileFsBackend covers dev-mode hot reload; the build-time codepath is its own surface.
• React-side hooks for backend-driven Suspense — covered by enterprise-react-i18n.

Source mapping

This change is genesis-original: the universal @suaveplan/i18n core has no enterprise/i18n.md upstream that describes a backend layer. The closest upstream comparison is the i18next plugin ecosystem (i18next-http-backend, i18next-chained-backend, i18next-fs-backend, i18next-localstorage-backend), and the design here borrows interface shapes (read, readMulti, save, lastModified) directly from those packages so that a future i18next interop layer is a thin shim.

Requirements with no source line

Requirement	Justification
Backend interface	Industry-standard shape (i18next BackendModule); we pick it because future interop is cheaper than reinventing.
StorageBackend	The user-facing reason this change exists — translations in DB / S3 / OPFS / IndexedDB without per-consumer integration.
MultiFileFsBackend	Solves "JSON files in many places rather than one mega-file" workflow concern raised in the i18n review.
ChainedBackend write-through caching	i18next pattern; non-obvious but proven and necessary for offline-first consumers (cache from IndexedDB, fall through to HTTPS).
OTel backend.read.duration_ms	AGENTS.md §13 row 4 — every async I/O boundary emits a duration histogram.

Verification approach

Each requirement carries a **Verified by:** line in the spec delta.

• packages/i18n/i18n/src/core/backend/backend.test.ts — interface conformance suite (every backend passes the same battery)
• packages/i18n/i18n/src/core/backend/memory/memory-backend.test.ts, http/http-backend.test.ts, storage/storage-backend.test.ts, multi-file-fs/multi-file-fs-backend.test.ts, chained/chained-backend.test.ts — per-backend behaviour
• packages/i18n/i18n/src/core/backend/storage/storage-backend.spec-conformance.test.ts — uses a real MemoryStorage from @suaveplan/storage to prove the integration is not a mock
• packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts — lazyLoad, preload, invalidate, saveMissing, getAsync interactions
• packages/i18n/i18n/src/core/backend/chained/chained-backend.spec-conformance.test.ts — write-through cache, fallback ordering, readMulti merge
• packages/i18n/i18n/tests/dep-graph.test.ts — @suaveplan/storage is a dependencies entry, not devDependencies

Impact

Affected dependents

grep -rl "@suaveplan/i18n" /home/ubuntu/web/genesis/packages/ --include="package.json" — only packages/i18n/i18n/package.json itself today.

Migration impact

Pure additive. Existing consumers of MessageCatalog.load(locale, ns, loader) are unaffected — loader is now optional but still accepted. Existing consumers of MessageCatalog.get/t continue to behave synchronously and miss-on-absent. New getAsync is the explicit opt-in to backend-driven lazy loading. Adding @suaveplan/storage as a dependencies entry: this is the first runtime dep on storage; verified safe because @suaveplan/storage itself only depends on @suaveplan/error + @suaveplan/types (no cycle).