115 lines
12 KiB
Markdown
115 lines
12 KiB
Markdown
# i18n storage-backed translation loaders
|
|
|
|
**Package**: `@suaveplan/i18n`
|
|
**Status**: PROPOSED
|
|
**Type**: feature
|
|
**Date**: 2026-05-04
|
|
|
|
## Why
|
|
|
|
The current `MessageCatalog.load(locale, namespace?, loader)` accepts a single async function and that is the entire resource-loading surface. Every consumer that wants translations from anything other than statically-imported objects has to hand-roll the loader, the cache key, the namespace-merge, and the error mapping. Compare this against `i18next-http-backend`, `i18next-fs-backend`, `i18next-resources-to-backend`, `i18next-multiload-backend-adapter`, `chained-backend`, and `localstorage-backend` — eight battle-tested backends behind one published interface — and `@suaveplan/i18n` looks anaemic.
|
|
|
|
Two specific consumer workflows are blocked today:
|
|
|
|
1. **Many small JSON files in many places.** Translators prefer `locales/<locale>/<namespace>.json` (one file per namespace per locale) over a single mega-file. Today, every consumer who wants this writes their own glob+parse+merge loop and feeds the result into `mergeMessages`.
|
|
2. **Translations stored in a database, S3, OPFS, IndexedDB, or any other key-value store.** The genesis monorepo already ships `@suaveplan/storage`'s `AbstractStorage` contract with eight backends (`memory`, `s3`, `opfs`, `indexeddb`, `node-fs`, `sqlite`, `garage`, `content-addressed`). A translator-friendly admin UI that writes new strings to a database, or a runtime that pulls strings from S3 with content-addressed cache busting, should be one config line — not a fresh integration each time.
|
|
|
|
This change introduces a formal `Backend` plugin interface, ships four built-in backends, and wires the catalog so that `MessageCatalog.load` and on-demand `t('auth:login.title')` can both resolve through any registered backend. The `StorageBackend` is the headline addition: any `AbstractStorage` instance becomes a translation source.
|
|
|
|
## What changes
|
|
|
|
### `@suaveplan/i18n`
|
|
|
|
#### Backend interface
|
|
|
|
- ADDED: `Backend` interface in `src/core/backend/backend.types.ts`:
|
|
- `read(locale, namespace) → Promise<TranslationRecord>` — load a single namespace
|
|
- `readMulti?(locale, namespaces[]) → Promise<Record<string, TranslationRecord>>` — bulk load (optional optimisation; falls back to N `read` calls when absent)
|
|
- `save?(locale, namespace, record) → Promise<void>` — write back (used by missing-key reporter and admin tooling; optional)
|
|
- `lastModified?(locale, namespace) → Promise<number | undefined>` — for cache-busting; optional
|
|
- `init?(catalog) → Promise<void>` — one-shot setup hook called when registered
|
|
- `dispose?() → Promise<void>` — teardown hook
|
|
- `name: string` — stable identifier for telemetry and chained-backend ordering
|
|
- ADDED: `BackendError` codes in the `I18N.*` namespace via `defineCodes`:
|
|
- `I18N.BACKEND_READ_FAILED` — `read`/`readMulti` rejected or returned a non-record value
|
|
- `I18N.BACKEND_WRITE_FAILED` — `save` rejected
|
|
- `I18N.BACKEND_NOT_REGISTERED` — `t()` requested a namespace from a backend that is not registered
|
|
- `I18N.BACKEND_LOCALE_NOT_AVAILABLE` — backend reported "locale unavailable" (distinct from a transport error)
|
|
|
|
#### Built-in backends
|
|
|
|
- ADDED: `MemoryBackend` (`src/core/backend/memory/memory-backend.ts`) — in-memory `Map<locale, Map<namespace, TranslationRecord>>`. Useful for tests and seeded defaults.
|
|
- ADDED: `HttpBackend` (`src/core/backend/http/http-backend.ts`) — fetches `loadPath` URL with `{locale}` / `{namespace}` placeholders. Configurable `fetchFn` (defaults to global `fetch`), `requestOptions`, `parsePayload` (defaults to `res.json()`), `crossDomain` flag, `customHeaders` callback. Forwards 5xx / network failures via `@suaveplan/retry` `RetryPolicy` (caller-supplied; defaults to no retry).
|
|
- ADDED: `StorageBackend` (`src/core/backend/storage/storage-backend.ts`) — wraps any `AbstractStorage` from `@suaveplan/storage`. Layouts:
|
|
- `single-key-per-namespace` (default): key = `${prefix}/${locale}/${namespace}.json`, value = JSON-encoded `TranslationRecord`.
|
|
- `single-key-per-locale`: key = `${prefix}/${locale}.json`, value = `Record<namespace, TranslationRecord>` (matches the i18next single-file shape).
|
|
- `flat-keys`: key = `${prefix}:${locale}:${namespace}:${dotPath}`, value = string (one storage key per leaf — works well with content-addressed/CDN backends and lets you ship deltas).
|
|
Pluggable codec (`encode`/`decode`) so callers can swap JSON for MessagePack via `@suaveplan/serialize`.
|
|
- ADDED: `MultiFileFsBackend` (`src/core/backend/multi-file-fs/multi-file-fs-backend.ts`) — Node/Bun-only filesystem backend. Reads `<root>/<locale>/<namespace>.json` (configurable layout: `flat`, `nested`, `glob`). Watches for changes via `node:fs.watch` (optional, opt-in via `watch: true`); when a file changes, emits `loaded` event so callers can refresh React trees. Supports JSON, JSON5, YAML (when the `yaml` peer is installed), and PO (when `gettext-parser` peer is installed).
|
|
- ADDED: `ChainedBackend` (`src/core/backend/chained/chained-backend.ts`) — composes N backends. `read` walks them in order until one succeeds; `readMulti` fans out concurrently and merges (later backends win on conflict, configurable). Caching layer between backends (e.g. `Chained([cacheBackend, networkBackend])`) follows the i18next pattern: write-through to all earlier backends on every successful network read.
|
|
- ADDED: `BackendPlugin` factory exports — `memoryBackend(opts)`, `httpBackend(opts)`, `storageBackend(opts)`, `multiFileFsBackend(opts)`, `chainedBackend(opts)`. Use `Backend` directly for advanced cases.
|
|
|
|
#### Catalog integration
|
|
|
|
- MODIFIED: `MessageCatalog` constructor accepts `backend?: Backend` (single backend) or `backends?: Backend[]` (auto-wrapped in `ChainedBackend`). When provided, the catalog registers them via the plugin manager (see `i18n-completion`).
|
|
- ADDED: `MessageCatalog.preload(locales, namespaces?) → Promise<void>` — bulk-loads via `readMulti` when supported, with concurrent-call dedup keyed on `(locale, namespace)`.
|
|
- ADDED: `MessageCatalog.lazyLoad: boolean` option (default `true` when a backend is registered, `false` otherwise) — when on, `get(key)` triggers an in-flight backend `read` for the missing namespace and returns a Promise via the new `getAsync(key)` overload.
|
|
- ADDED: `MessageCatalog.getAsync(key, locale?) → Promise<LookupResult>` — async sibling of `get` that awaits in-flight backend loads. Synchronous `get`/`t` continue to behave as today (return `undefined` / key when missing) so existing call sites are unchanged.
|
|
- ADDED: `MessageCatalog.invalidate(locale?, namespace?)` — drops cached namespaces so the next read re-fetches via the backend.
|
|
- ADDED: `MessageCatalog.saveMissing(locale, key, defaultValue?) → Promise<void>` — when a backend implements `save`, this records a missing key for translator pickup. Wired automatically when `saveMissing: true` is passed in catalog options.
|
|
- MODIFIED: `MessageCatalog.load(locale, namespace?, loader?)` — `loader` becomes optional; when omitted, the catalog routes through the registered backend. Existing call sites that pass a loader keep working unchanged.
|
|
|
|
#### Zod schemas
|
|
|
|
- ADDED: `BackendSchema` (Zod) — validates the structural shape of a `Backend` (function presence, name string).
|
|
- ADDED: `HttpBackendOptionsSchema`, `StorageBackendOptionsSchema`, `MultiFileFsBackendOptionsSchema`, `ChainedBackendOptionsSchema` — option validation at construction.
|
|
|
|
#### OTel instrumentation
|
|
|
|
- ADDED: `i18n.backend.read.duration_ms` histogram — labels: `backend.name`, `locale`, `namespace`, `outcome` (`hit | miss | error`).
|
|
- ADDED: `i18n.backend.read.total` counter — labels: `backend.name`, `outcome`.
|
|
- ADDED: `i18n.backend.write.total` counter — labels: `backend.name`, `outcome`.
|
|
- ADDED: `i18n.backend.cache.size` gauge — number of `(locale, namespace)` pairs currently in the catalog's in-memory store.
|
|
|
|
## Out of scope
|
|
|
|
- **CDN-side compilation** (LinguiJS / FormatJS approach where messages are compiled to JS modules at build time) — separate change once the backend interface stabilises. The `flat-keys` layout in `StorageBackend` already supports the closest equivalent.
|
|
- **TMS adapters** (Phrase, Lokalise, Crowdin) — these are downstream consumers of `Backend` + `save()`; ship in a separate `i18n-tms-adapters` change once translator workflows are scoped.
|
|
- **Vite/Rollup plugin** for build-time JSON inlining — separate change. The `MultiFileFsBackend` covers dev-mode hot reload; the build-time codepath is its own surface.
|
|
- **React-side hooks** for backend-driven Suspense — covered by `enterprise-react-i18n`.
|
|
|
|
## Source mapping
|
|
|
|
This change is **genesis-original**: the universal `@suaveplan/i18n` core has no `enterprise/i18n.md` upstream that describes a backend layer. The closest upstream comparison is the i18next plugin ecosystem (`i18next-http-backend`, `i18next-chained-backend`, `i18next-fs-backend`, `i18next-localstorage-backend`), and the design here borrows interface shapes (`read`, `readMulti`, `save`, `lastModified`) directly from those packages so that a future i18next interop layer is a thin shim.
|
|
|
|
### Requirements with no source line
|
|
|
|
| Requirement | Justification |
|
|
|---|---|
|
|
| `Backend` interface | Industry-standard shape (i18next `BackendModule`); we pick it because future interop is cheaper than reinventing. |
|
|
| `StorageBackend` | The user-facing reason this change exists — translations in DB / S3 / OPFS / IndexedDB without per-consumer integration. |
|
|
| `MultiFileFsBackend` | Solves "JSON files in many places rather than one mega-file" workflow concern raised in the i18n review. |
|
|
| `ChainedBackend` write-through caching | i18next pattern; non-obvious but proven and necessary for offline-first consumers (cache from IndexedDB, fall through to HTTPS). |
|
|
| OTel `backend.read.duration_ms` | AGENTS.md §13 row 4 — every async I/O boundary emits a duration histogram. |
|
|
|
|
## Verification approach
|
|
|
|
Each requirement carries a `**Verified by:**` line in the spec delta.
|
|
|
|
- `packages/i18n/i18n/src/core/backend/backend.test.ts` — interface conformance suite (every backend passes the same battery)
|
|
- `packages/i18n/i18n/src/core/backend/memory/memory-backend.test.ts`, `http/http-backend.test.ts`, `storage/storage-backend.test.ts`, `multi-file-fs/multi-file-fs-backend.test.ts`, `chained/chained-backend.test.ts` — per-backend behaviour
|
|
- `packages/i18n/i18n/src/core/backend/storage/storage-backend.spec-conformance.test.ts` — uses a real `MemoryStorage` from `@suaveplan/storage` to prove the integration is not a mock
|
|
- `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts` — `lazyLoad`, `preload`, `invalidate`, `saveMissing`, `getAsync` interactions
|
|
- `packages/i18n/i18n/src/core/backend/chained/chained-backend.spec-conformance.test.ts` — write-through cache, fallback ordering, `readMulti` merge
|
|
- `packages/i18n/i18n/tests/dep-graph.test.ts` — `@suaveplan/storage` is a `dependencies` entry, not `devDependencies`
|
|
|
|
## Impact
|
|
|
|
### Affected dependents
|
|
|
|
`grep -rl "@suaveplan/i18n" /home/ubuntu/web/genesis/packages/ --include="package.json"` — only `packages/i18n/i18n/package.json` itself today.
|
|
|
|
### Migration impact
|
|
|
|
Pure additive. Existing consumers of `MessageCatalog.load(locale, ns, loader)` are unaffected — `loader` is now optional but still accepted. Existing consumers of `MessageCatalog.get/t` continue to behave synchronously and miss-on-absent. New `getAsync` is the explicit opt-in to backend-driven lazy loading. Adding `@suaveplan/storage` as a `dependencies` entry: this is the first runtime dep on storage; verified safe because `@suaveplan/storage` itself only depends on `@suaveplan/error` + `@suaveplan/types` (no cycle).
|