Initial Commit
This commit is contained in:
@@ -0,0 +1,114 @@
|
||||
# i18n storage-backed translation loaders
|
||||
|
||||
**Package**: `@suaveplan/i18n`
|
||||
**Status**: PROPOSED
|
||||
**Type**: feature
|
||||
**Date**: 2026-05-04
|
||||
|
||||
## Why
|
||||
|
||||
The current `MessageCatalog.load(locale, namespace?, loader)` accepts a single async function and that is the entire resource-loading surface. Every consumer that wants translations from anything other than statically-imported objects has to hand-roll the loader, the cache key, the namespace-merge, and the error mapping. Compare this against `i18next-http-backend`, `i18next-fs-backend`, `i18next-resources-to-backend`, `i18next-multiload-backend-adapter`, `chained-backend`, and `localstorage-backend` — eight battle-tested backends behind one published interface — and `@suaveplan/i18n` looks anaemic.
|
||||
|
||||
Two specific consumer workflows are blocked today:
|
||||
|
||||
1. **Many small JSON files in many places.** Translators prefer `locales/<locale>/<namespace>.json` (one file per namespace per locale) over a single mega-file. Today, every consumer who wants this writes their own glob+parse+merge loop and feeds the result into `mergeMessages`.
|
||||
2. **Translations stored in a database, S3, OPFS, IndexedDB, or any other key-value store.** The genesis monorepo already ships `@suaveplan/storage`'s `AbstractStorage` contract with eight backends (`memory`, `s3`, `opfs`, `indexeddb`, `node-fs`, `sqlite`, `garage`, `content-addressed`). A translator-friendly admin UI that writes new strings to a database, or a runtime that pulls strings from S3 with content-addressed cache busting, should be one config line — not a fresh integration each time.
|
||||
|
||||
This change introduces a formal `Backend` plugin interface, ships four built-in backends, and wires the catalog so that `MessageCatalog.load` and on-demand `t('auth:login.title')` can both resolve through any registered backend. The `StorageBackend` is the headline addition: any `AbstractStorage` instance becomes a translation source.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/i18n`
|
||||
|
||||
#### Backend interface
|
||||
|
||||
- ADDED: `Backend` interface in `src/core/backend/backend.types.ts`:
|
||||
- `read(locale, namespace) → Promise<TranslationRecord>` — load a single namespace
|
||||
- `readMulti?(locale, namespaces[]) → Promise<Record<string, TranslationRecord>>` — bulk load (optional optimisation; falls back to N `read` calls when absent)
|
||||
- `save?(locale, namespace, record) → Promise<void>` — write back (used by missing-key reporter and admin tooling; optional)
|
||||
- `lastModified?(locale, namespace) → Promise<number | undefined>` — for cache-busting; optional
|
||||
- `init?(catalog) → Promise<void>` — one-shot setup hook called when registered
|
||||
- `dispose?() → Promise<void>` — teardown hook
|
||||
- `name: string` — stable identifier for telemetry and chained-backend ordering
|
||||
- ADDED: `BackendError` codes in the `I18N.*` namespace via `defineCodes`:
|
||||
- `I18N.BACKEND_READ_FAILED` — `read`/`readMulti` rejected or returned a non-record value
|
||||
- `I18N.BACKEND_WRITE_FAILED` — `save` rejected
|
||||
- `I18N.BACKEND_NOT_REGISTERED` — `t()` requested a namespace from a backend that is not registered
|
||||
- `I18N.BACKEND_LOCALE_NOT_AVAILABLE` — backend reported "locale unavailable" (distinct from a transport error)
|
||||
|
||||
#### Built-in backends
|
||||
|
||||
- ADDED: `MemoryBackend` (`src/core/backend/memory/memory-backend.ts`) — in-memory `Map<locale, Map<namespace, TranslationRecord>>`. Useful for tests and seeded defaults.
|
||||
- ADDED: `HttpBackend` (`src/core/backend/http/http-backend.ts`) — fetches `loadPath` URL with `{locale}` / `{namespace}` placeholders. Configurable `fetchFn` (defaults to global `fetch`), `requestOptions`, `parsePayload` (defaults to `res.json()`), `crossDomain` flag, `customHeaders` callback. Forwards 5xx / network failures via `@suaveplan/retry` `RetryPolicy` (caller-supplied; defaults to no retry).
|
||||
- ADDED: `StorageBackend` (`src/core/backend/storage/storage-backend.ts`) — wraps any `AbstractStorage` from `@suaveplan/storage`. Layouts:
|
||||
- `single-key-per-namespace` (default): key = `${prefix}/${locale}/${namespace}.json`, value = JSON-encoded `TranslationRecord`.
|
||||
- `single-key-per-locale`: key = `${prefix}/${locale}.json`, value = `Record<namespace, TranslationRecord>` (matches the i18next single-file shape).
|
||||
- `flat-keys`: key = `${prefix}:${locale}:${namespace}:${dotPath}`, value = string (one storage key per leaf — works well with content-addressed/CDN backends and lets you ship deltas).
|
||||
Pluggable codec (`encode`/`decode`) so callers can swap JSON for MessagePack via `@suaveplan/serialize`.
|
||||
- ADDED: `MultiFileFsBackend` (`src/core/backend/multi-file-fs/multi-file-fs-backend.ts`) — Node/Bun-only filesystem backend. Reads `<root>/<locale>/<namespace>.json` (configurable layout: `flat`, `nested`, `glob`). Watches for changes via `node:fs.watch` (optional, opt-in via `watch: true`); when a file changes, emits `loaded` event so callers can refresh React trees. Supports JSON, JSON5, YAML (when the `yaml` peer is installed), and PO (when `gettext-parser` peer is installed).
|
||||
- ADDED: `ChainedBackend` (`src/core/backend/chained/chained-backend.ts`) — composes N backends. `read` walks them in order until one succeeds; `readMulti` fans out concurrently and merges (later backends win on conflict, configurable). Caching layer between backends (e.g. `Chained([cacheBackend, networkBackend])`) follows the i18next pattern: write-through to all earlier backends on every successful network read.
|
||||
- ADDED: `BackendPlugin` factory exports — `memoryBackend(opts)`, `httpBackend(opts)`, `storageBackend(opts)`, `multiFileFsBackend(opts)`, `chainedBackend(opts)`. Use `Backend` directly for advanced cases.
|
||||
|
||||
#### Catalog integration
|
||||
|
||||
- MODIFIED: `MessageCatalog` constructor accepts `backend?: Backend` (single backend) or `backends?: Backend[]` (auto-wrapped in `ChainedBackend`). When provided, the catalog registers them via the plugin manager (see `i18n-completion`).
|
||||
- ADDED: `MessageCatalog.preload(locales, namespaces?) → Promise<void>` — bulk-loads via `readMulti` when supported, with concurrent-call dedup keyed on `(locale, namespace)`.
|
||||
- ADDED: `MessageCatalog.lazyLoad: boolean` option (default `true` when a backend is registered, `false` otherwise) — when on, `get(key)` triggers an in-flight backend `read` for the missing namespace and returns a Promise via the new `getAsync(key)` overload.
|
||||
- ADDED: `MessageCatalog.getAsync(key, locale?) → Promise<LookupResult>` — async sibling of `get` that awaits in-flight backend loads. Synchronous `get`/`t` continue to behave as today (return `undefined` / key when missing) so existing call sites are unchanged.
|
||||
- ADDED: `MessageCatalog.invalidate(locale?, namespace?)` — drops cached namespaces so the next read re-fetches via the backend.
|
||||
- ADDED: `MessageCatalog.saveMissing(locale, key, defaultValue?) → Promise<void>` — when a backend implements `save`, this records a missing key for translator pickup. Wired automatically when `saveMissing: true` is passed in catalog options.
|
||||
- MODIFIED: `MessageCatalog.load(locale, namespace?, loader?)` — `loader` becomes optional; when omitted, the catalog routes through the registered backend. Existing call sites that pass a loader keep working unchanged.
|
||||
|
||||
#### Zod schemas
|
||||
|
||||
- ADDED: `BackendSchema` (Zod) — validates the structural shape of a `Backend` (function presence, name string).
|
||||
- ADDED: `HttpBackendOptionsSchema`, `StorageBackendOptionsSchema`, `MultiFileFsBackendOptionsSchema`, `ChainedBackendOptionsSchema` — option validation at construction.
|
||||
|
||||
#### OTel instrumentation
|
||||
|
||||
- ADDED: `i18n.backend.read.duration_ms` histogram — labels: `backend.name`, `locale`, `namespace`, `outcome` (`hit | miss | error`).
|
||||
- ADDED: `i18n.backend.read.total` counter — labels: `backend.name`, `outcome`.
|
||||
- ADDED: `i18n.backend.write.total` counter — labels: `backend.name`, `outcome`.
|
||||
- ADDED: `i18n.backend.cache.size` gauge — number of `(locale, namespace)` pairs currently in the catalog's in-memory store.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **CDN-side compilation** (LinguiJS / FormatJS approach where messages are compiled to JS modules at build time) — separate change once the backend interface stabilises. The `flat-keys` layout in `StorageBackend` already supports the closest equivalent.
|
||||
- **TMS adapters** (Phrase, Lokalise, Crowdin) — these are downstream consumers of `Backend` + `save()`; ship in a separate `i18n-tms-adapters` change once translator workflows are scoped.
|
||||
- **Vite/Rollup plugin** for build-time JSON inlining — separate change. The `MultiFileFsBackend` covers dev-mode hot reload; the build-time codepath is its own surface.
|
||||
- **React-side hooks** for backend-driven Suspense — covered by `enterprise-react-i18n`.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change is **genesis-original**: the universal `@suaveplan/i18n` core has no `enterprise/i18n.md` upstream that describes a backend layer. The closest upstream comparison is the i18next plugin ecosystem (`i18next-http-backend`, `i18next-chained-backend`, `i18next-fs-backend`, `i18next-localstorage-backend`), and the design here borrows interface shapes (`read`, `readMulti`, `save`, `lastModified`) directly from those packages so that a future i18next interop layer is a thin shim.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| `Backend` interface | Industry-standard shape (i18next `BackendModule`); we pick it because future interop is cheaper than reinventing. |
|
||||
| `StorageBackend` | The user-facing reason this change exists — translations in DB / S3 / OPFS / IndexedDB without per-consumer integration. |
|
||||
| `MultiFileFsBackend` | Solves "JSON files in many places rather than one mega-file" workflow concern raised in the i18n review. |
|
||||
| `ChainedBackend` write-through caching | i18next pattern; non-obvious but proven and necessary for offline-first consumers (cache from IndexedDB, fall through to HTTPS). |
|
||||
| OTel `backend.read.duration_ms` | AGENTS.md §13 row 4 — every async I/O boundary emits a duration histogram. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each requirement carries a `**Verified by:**` line in the spec delta.
|
||||
|
||||
- `packages/i18n/i18n/src/core/backend/backend.test.ts` — interface conformance suite (every backend passes the same battery)
|
||||
- `packages/i18n/i18n/src/core/backend/memory/memory-backend.test.ts`, `http/http-backend.test.ts`, `storage/storage-backend.test.ts`, `multi-file-fs/multi-file-fs-backend.test.ts`, `chained/chained-backend.test.ts` — per-backend behaviour
|
||||
- `packages/i18n/i18n/src/core/backend/storage/storage-backend.spec-conformance.test.ts` — uses a real `MemoryStorage` from `@suaveplan/storage` to prove the integration is not a mock
|
||||
- `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts` — `lazyLoad`, `preload`, `invalidate`, `saveMissing`, `getAsync` interactions
|
||||
- `packages/i18n/i18n/src/core/backend/chained/chained-backend.spec-conformance.test.ts` — write-through cache, fallback ordering, `readMulti` merge
|
||||
- `packages/i18n/i18n/tests/dep-graph.test.ts` — `@suaveplan/storage` is a `dependencies` entry, not `devDependencies`
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
`grep -rl "@suaveplan/i18n" /home/ubuntu/web/genesis/packages/ --include="package.json"` — only `packages/i18n/i18n/package.json` itself today.
|
||||
|
||||
### Migration impact
|
||||
|
||||
Pure additive. Existing consumers of `MessageCatalog.load(locale, ns, loader)` are unaffected — `loader` is now optional but still accepted. Existing consumers of `MessageCatalog.get/t` continue to behave synchronously and miss-on-absent. New `getAsync` is the explicit opt-in to backend-driven lazy loading. Adding `@suaveplan/storage` as a `dependencies` entry: this is the first runtime dep on storage; verified safe because `@suaveplan/storage` itself only depends on `@suaveplan/error` + `@suaveplan/types` (no cycle).
|
||||
@@ -0,0 +1,247 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: i18n (i18n-storage-backends)
|
||||
|
||||
**Package**: `@suaveplan/i18n`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. (See `openspec/AGENTS.md` Section 13.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Backend interface contract
|
||||
|
||||
The package SHALL expose a `Backend` interface that any external translation source can implement. The interface SHALL declare a stable `name: string`, a required `read(locale, namespace) → Promise<TranslationRecord>` method, and four optional methods: `readMulti?(locale, namespaces) → Promise<Record<string, TranslationRecord>>`, `save?(locale, namespace, record) → Promise<void>`, `lastModified?(locale, namespace) → Promise<number | undefined>`, and lifecycle hooks `init?(catalog)` / `dispose?()`. Implementations SHALL surface transport / parse failures as `I18nError(I18N.BACKEND_READ_FAILED)` (read path) or `I18nError(I18N.BACKEND_WRITE_FAILED)` (write path) with the underlying error attached as `cause`.
|
||||
|
||||
#### Scenario: read returns a TranslationRecord
|
||||
|
||||
- **WHEN** a registered `Backend.read('en', 'auth')` resolves with `{ login: { title: 'Sign in' } }`
|
||||
- **THEN** the catalog stores it under `(en, auth)` and a subsequent `t('auth:login.title')` returns `'Sign in'`
|
||||
|
||||
#### Scenario: read failure maps to I18N.BACKEND_READ_FAILED
|
||||
|
||||
- **WHEN** a backend's `read` rejects with a `TypeError`
|
||||
- **THEN** the catalog throws `I18nError` with code `I18N.BACKEND_READ_FAILED` and `error.cause === TypeError`
|
||||
|
||||
#### Scenario: missing readMulti falls back to N reads
|
||||
|
||||
- **WHEN** `MessageCatalog.preload(['en'], ['auth', 'common'])` is called against a backend that omits `readMulti`
|
||||
- **THEN** `read` is invoked once for each `(locale, namespace)` pair concurrently and the catalog merges the results
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/backend.test.ts::"read populates catalog"` AND `packages/i18n/i18n/src/core/backend/backend.test.ts::"read failure surfaces BACKEND_READ_FAILED"` AND `packages/i18n/i18n/src/core/backend/backend.test.ts::"falls back to per-namespace read when readMulti is absent"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MemoryBackend reference implementation
|
||||
|
||||
The package SHALL ship a `MemoryBackend` implementation backed by `Map<locale, Map<namespace, TranslationRecord>>`. It SHALL implement `read`, `readMulti`, `save`, and `lastModified`. Construction SHALL accept an optional `seed: Record<locale, Record<namespace, TranslationRecord>>` for tests and seeded defaults.
|
||||
|
||||
#### Scenario: seed is read on first request
|
||||
|
||||
- **WHEN** `new MemoryBackend({ seed: { en: { auth: { login: { title: 'Sign in' } } } } })` then `read('en', 'auth')`
|
||||
- **THEN** the resolved record deep-equals the seeded value
|
||||
|
||||
#### Scenario: save then read round-trips
|
||||
|
||||
- **WHEN** `backend.save('en', 'auth', { login: { title: 'Sign in' } })` is called, then `backend.read('en', 'auth')`
|
||||
- **THEN** the resolved record deep-equals the saved value AND `lastModified('en', 'auth')` returns a number ≥ the timestamp before `save`
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/memory/memory-backend.test.ts::"reads seeded values"` AND `packages/i18n/i18n/src/core/backend/memory/memory-backend.test.ts::"save then read round-trips with lastModified"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: HttpBackend with templated load path
|
||||
|
||||
The package SHALL ship an `HttpBackend` that fetches translations from a URL constructed from a `loadPath` template containing `{locale}` and `{namespace}` placeholders. Defaults: `fetchFn = globalThis.fetch`, `parsePayload = (res) => res.json()`. A non-2xx HTTP response SHALL surface as `I18nError(I18N.BACKEND_READ_FAILED)` with `context.status` set to the HTTP status code. A 404 SHALL surface as `I18nError(I18N.BACKEND_LOCALE_NOT_AVAILABLE)` so chained backends can fall through.
|
||||
|
||||
#### Scenario: GET /locales/en/auth.json returns translations
|
||||
|
||||
- **WHEN** `new HttpBackend({ loadPath: '/locales/{locale}/{namespace}.json', fetchFn })` then `read('en', 'auth')` with `fetchFn` resolving to `Response.json({ login: { title: 'Sign in' } })`
|
||||
- **THEN** the resolved record equals `{ login: { title: 'Sign in' } }` AND `fetchFn` was called with `'/locales/en/auth.json'`
|
||||
|
||||
#### Scenario: HTTP 404 surfaces LOCALE_NOT_AVAILABLE
|
||||
|
||||
- **WHEN** `fetchFn` resolves with status 404
|
||||
- **THEN** the call rejects with `I18nError(I18N.BACKEND_LOCALE_NOT_AVAILABLE)` and `context.status === 404`
|
||||
|
||||
#### Scenario: HTTP 500 surfaces BACKEND_READ_FAILED
|
||||
|
||||
- **WHEN** `fetchFn` resolves with status 500
|
||||
- **THEN** the call rejects with `I18nError(I18N.BACKEND_READ_FAILED)` and `context.status === 500`
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/http/http-backend.test.ts::"fetches templated URL"` AND `packages/i18n/i18n/src/core/backend/http/http-backend.test.ts::"404 surfaces LOCALE_NOT_AVAILABLE"` AND `packages/i18n/i18n/src/core/backend/http/http-backend.test.ts::"5xx surfaces BACKEND_READ_FAILED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: StorageBackend wraps any AbstractStorage
|
||||
|
||||
The package SHALL ship a `StorageBackend` that adapts any `AbstractStorage` instance from `@suaveplan/storage` into a `Backend`. It SHALL support three storage layouts via a `layout` option:
|
||||
|
||||
- `single-key-per-namespace` (default): key = `${prefix}/${locale}/${namespace}.json`, value = JSON-encoded `TranslationRecord`
|
||||
- `single-key-per-locale`: key = `${prefix}/${locale}.json`, value = `Record<namespace, TranslationRecord>`
|
||||
- `flat-keys`: key = `${prefix}:${locale}:${namespace}:${dotPath}`, value = the leaf string
|
||||
|
||||
Codec SHALL be pluggable via `encode(record): Uint8Array` / `decode(bytes): TranslationRecord` (default: `JSON.stringify` + `TextEncoder` / `TextDecoder` + `JSON.parse`).
|
||||
|
||||
#### Scenario: single-key-per-namespace round-trips through MemoryStorage
|
||||
|
||||
- **WHEN** `new StorageBackend({ storage: new MemoryStorage(), layout: 'single-key-per-namespace', prefix: 'i18n' })` and `backend.save('en', 'auth', { login: { title: 'Sign in' } })`
|
||||
- **THEN** `MemoryStorage.get('i18n/en/auth.json')` decodes to `{ login: { title: 'Sign in' } }` AND `backend.read('en', 'auth')` returns the same record
|
||||
|
||||
#### Scenario: single-key-per-locale stores all namespaces under one key
|
||||
|
||||
- **WHEN** `layout: 'single-key-per-locale'` and `backend.save('en', 'auth', X)` then `backend.save('en', 'common', Y)`
|
||||
- **THEN** `MemoryStorage.get('i18n/en.json')` decodes to `{ auth: X, common: Y }`
|
||||
|
||||
#### Scenario: flat-keys writes one storage key per leaf
|
||||
|
||||
- **WHEN** `layout: 'flat-keys'` and `backend.save('en', 'auth', { login: { title: 'Sign in' } })`
|
||||
- **THEN** `MemoryStorage.get('i18n:en:auth:login.title')` decodes to `'Sign in'`
|
||||
|
||||
#### Scenario: codec swap to MessagePack
|
||||
|
||||
- **WHEN** `new StorageBackend({ storage, encode: msgpackEncode, decode: msgpackDecode })` is constructed
|
||||
- **THEN** `save` calls `msgpackEncode` for serialisation and `read` calls `msgpackDecode`
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/storage/storage-backend.test.ts::"single-key-per-namespace round-trip"` AND `packages/i18n/i18n/src/core/backend/storage/storage-backend.test.ts::"single-key-per-locale shape"` AND `packages/i18n/i18n/src/core/backend/storage/storage-backend.test.ts::"flat-keys shape"` AND `packages/i18n/i18n/src/core/backend/storage/storage-backend.spec-conformance.test.ts::"codec swap to MessagePack"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MultiFileFsBackend reads JSON files in a per-namespace tree
|
||||
|
||||
The package SHALL ship a `MultiFileFsBackend` (Node/Bun-only; pure ESM dynamic-import of `node:fs/promises`) that loads `<root>/<locale>/<namespace>.json` from disk. It SHALL support three layouts:
|
||||
|
||||
- `nested` (default): `<root>/<locale>/<namespace>.json` (e.g. `locales/en/auth.json`)
|
||||
- `flat`: `<root>/<locale>.<namespace>.json` (e.g. `locales/en.auth.json`)
|
||||
- `glob`: caller-supplied `globPattern` with `{locale}` and `{namespace}` placeholders
|
||||
|
||||
When `watch: true`, it SHALL use `node:fs/promises.watch` to detect changes and emit a `loaded` event via the catalog event bus on each successful reload. Optional peer-dependency parsers SHALL be supported when present: `yaml` (for `.yaml`/`.yml`), `gettext-parser` (for `.po`), `json5` (for `.json5`).
|
||||
|
||||
#### Scenario: nested layout loads namespace files
|
||||
|
||||
- **WHEN** `<root>/en/auth.json` contains `{ "login": { "title": "Sign in" } }` and `backend.read('en', 'auth')` is called
|
||||
- **THEN** the resolved record equals the file's parsed content
|
||||
|
||||
#### Scenario: missing file maps to LOCALE_NOT_AVAILABLE
|
||||
|
||||
- **WHEN** `<root>/de/auth.json` does not exist and `backend.read('de', 'auth')` is called
|
||||
- **THEN** the call rejects with `I18nError(I18N.BACKEND_LOCALE_NOT_AVAILABLE)` AND `context.path` is the resolved file path
|
||||
|
||||
#### Scenario: watch emits loaded event on file change
|
||||
|
||||
- **WHEN** `watch: true` and an external process writes a new value to `<root>/en/auth.json`
|
||||
- **THEN** the catalog emits `loaded` with `{ locale: 'en', namespace: 'auth', backend: 'multi-file-fs' }` within 500 ms
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/multi-file-fs/multi-file-fs-backend.test.ts::"nested layout loads namespace files"` AND `packages/i18n/i18n/src/core/backend/multi-file-fs/multi-file-fs-backend.test.ts::"missing file maps to LOCALE_NOT_AVAILABLE"` AND `packages/i18n/i18n/src/core/backend/multi-file-fs/multi-file-fs-backend.test.ts::"watch emits loaded event on file change"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: ChainedBackend composes N backends with write-through cache
|
||||
|
||||
The package SHALL ship a `ChainedBackend` that composes a non-empty ordered list of backends. `read(locale, ns)` SHALL walk the list in order; the first backend whose `read` resolves wins. After a successful read, the resolved record SHALL be written through to every preceding backend whose `save` is implemented (the standard "cache from IndexedDB, fall through to HTTPS" pattern). `readMulti` SHALL fan out concurrently and merge per the configured `mergeStrategy: 'first-wins' | 'last-wins'` (default `'first-wins'`).
|
||||
|
||||
#### Scenario: cache hit short-circuits the chain
|
||||
|
||||
- **WHEN** `Chained([memory, http])` and `memory.read('en', 'auth')` resolves
|
||||
- **THEN** `http.read` is never called
|
||||
|
||||
#### Scenario: cache miss writes through to earlier backends
|
||||
|
||||
- **WHEN** `Chained([memory, http])`, `memory.read` rejects with `BACKEND_LOCALE_NOT_AVAILABLE`, and `http.read` resolves with `R`
|
||||
- **THEN** the chained `read` resolves with `R` AND `memory.save('en', 'auth', R)` is called exactly once
|
||||
|
||||
#### Scenario: BACKEND_READ_FAILED does not fall through
|
||||
|
||||
- **WHEN** the first backend rejects with `BACKEND_READ_FAILED` (a transport / parse error, not a 404-equivalent)
|
||||
- **THEN** the chained `read` propagates the error without calling subsequent backends
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/chained/chained-backend.test.ts::"cache hit short-circuits"` AND `packages/i18n/i18n/src/core/backend/chained/chained-backend.spec-conformance.test.ts::"cache miss writes through"` AND `packages/i18n/i18n/src/core/backend/chained/chained-backend.test.ts::"BACKEND_READ_FAILED does not fall through"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MessageCatalog accepts and routes through registered backends
|
||||
|
||||
`MessageCatalog` SHALL accept `backend?: Backend` (single) or `backends?: Backend[]` (auto-wrapped in `ChainedBackend`) at construction. When a backend is registered, `MessageCatalog.preload(locales, namespaces?) → Promise<void>` SHALL bulk-load via `readMulti` when supported and via concurrent `read` calls otherwise. Repeated `preload` calls for the same `(locale, namespace)` pair SHALL be deduplicated via the existing in-flight Promise map. `MessageCatalog.invalidate(locale?, namespace?) → void` SHALL drop the in-memory cache for the affected pairs so the next read re-fetches.
|
||||
|
||||
#### Scenario: preload populates catalog via readMulti
|
||||
|
||||
- **WHEN** a `MemoryBackend` carries seeds for `en/auth` and `en/common` and `catalog.preload(['en'], ['auth', 'common'])` is called
|
||||
- **THEN** `t('auth:login.title')` and `t('common:cancel')` resolve from the catalog without further backend calls
|
||||
|
||||
#### Scenario: invalidate forces re-read
|
||||
|
||||
- **WHEN** `catalog.preload(['en'], ['auth'])` succeeds, then `catalog.invalidate('en', 'auth')`, then `catalog.preload(['en'], ['auth'])` is called again
|
||||
- **THEN** the backend's `read` was called twice for the `(en, auth)` pair
|
||||
|
||||
#### Scenario: concurrent preload calls dedup
|
||||
|
||||
- **WHEN** `catalog.preload(['en'], ['auth'])` is called twice in immediate succession (no `await` between them)
|
||||
- **THEN** the backend's `read` is called exactly once
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"preload populates via readMulti"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"invalidate forces re-read"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"concurrent preload calls dedup"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: getAsync triggers in-flight backend load for missing namespace
|
||||
|
||||
When `lazyLoad: true` is set (default when a backend is registered) and `MessageCatalog.getAsync(key, locale?)` is called for a key in a namespace not yet present, the catalog SHALL trigger a backend `read` for that `(locale, namespace)` and resolve the returned Promise once the read completes (success or fallback). The synchronous `get(key)` and `t(key)` SHALL continue to behave as today: return `undefined` / the key string when the namespace is absent, even with `lazyLoad: true`. This separation SHALL be enforced so existing synchronous call sites are unchanged.
|
||||
|
||||
#### Scenario: getAsync resolves to backend value
|
||||
|
||||
- **WHEN** the catalog has no `auth` namespace loaded and `catalog.getAsync('auth:login.title')` is called
|
||||
- **THEN** the backend's `read('en', 'auth')` is invoked AND the returned Promise resolves to `{ value: 'Sign in', source: 'catalog', ... }`
|
||||
|
||||
#### Scenario: getAsync surfaces backend errors
|
||||
|
||||
- **WHEN** the backend's `read` rejects with `BACKEND_READ_FAILED`
|
||||
- **THEN** `getAsync` rejects with the same `I18nError`
|
||||
|
||||
#### Scenario: synchronous t remains unchanged under lazyLoad
|
||||
|
||||
- **WHEN** `lazyLoad: true` and `t('auth:login.title')` is called for a not-yet-loaded namespace
|
||||
- **THEN** the synchronous return value is `'auth:login.title'` (the key) AND the backend `read` was NOT triggered (only `getAsync` triggers backend calls)
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"getAsync resolves to backend value"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"getAsync surfaces backend errors"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"synchronous t never triggers backend reads"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: saveMissing writes back through registered backend
|
||||
|
||||
When `saveMissing: true` is passed in catalog options and a backend with a `save` method is registered, `MessageCatalog` SHALL invoke `backend.save(locale, namespace, { ...existing, [key]: defaultValue })` whenever a key miss occurs. If the backend lacks `save` or the call rejects, `MessageCatalog` SHALL emit an OTel counter sample (`i18n.backend.write.total{outcome=error}`) but SHALL NOT throw — saveMissing is best-effort by design.
|
||||
|
||||
#### Scenario: missing key triggers save
|
||||
|
||||
- **WHEN** `saveMissing: true` and `t('auth:new.unknown.key', { defaultValue: 'Default' })` is called for a key absent from the catalog
|
||||
- **THEN** the backend's `save('en', 'auth', record)` is called once with the merged record containing the new key
|
||||
|
||||
#### Scenario: save rejection does not throw
|
||||
|
||||
- **WHEN** `backend.save` rejects
|
||||
- **THEN** `t()` returns the default value AND no exception is thrown AND the OTel counter `i18n.backend.write.total{outcome=error}` increments by 1
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"saveMissing triggers backend.save"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"save rejection does not throw"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Backend OTel instrumentation
|
||||
|
||||
All backend operations SHALL emit OTel telemetry built via `getMeter('suaveplan.i18n')`:
|
||||
|
||||
- Histogram `i18n.backend.read.duration_ms` — labels: `backend.name`, `locale`, `namespace`, `outcome` (`hit | miss | error`)
|
||||
- Counter `i18n.backend.read.total` — labels: `backend.name`, `outcome`
|
||||
- Counter `i18n.backend.write.total` — labels: `backend.name`, `outcome`
|
||||
- Gauge `i18n.backend.cache.size` — observable callback returning the count of `(locale, namespace)` pairs currently held in `MessageCatalog`'s in-memory store
|
||||
|
||||
#### Scenario: read records duration sample
|
||||
|
||||
- **WHEN** `StorageBackend.read('en', 'auth')` is called once with an in-memory `MeterProvider`
|
||||
- **THEN** exactly one sample is recorded under `i18n.backend.read.duration_ms` with attributes `{ backend.name: 'storage', locale: 'en', namespace: 'auth', outcome: 'hit' }`
|
||||
|
||||
#### Scenario: cache size gauge reflects catalog state
|
||||
|
||||
- **WHEN** `catalog.preload(['en', 'fr'], ['auth'])` succeeds, then the OTel gauge is observed
|
||||
- **THEN** the observed value is `2` (one pair per locale × namespace)
|
||||
|
||||
**Verified by:** `packages/i18n/i18n/src/core/backend/storage/storage-backend.spec-conformance.test.ts::"records read.duration_ms sample"` AND `packages/i18n/i18n/src/core/catalog/catalog-backend.test.ts::"cache.size gauge reflects state"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,121 @@
|
||||
# Tasks — i18n storage-backed translation loaders
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read `openspec/specs/i18n/spec.md` end-to-end (canonical capability spec)
|
||||
- [x] 1.2 Read `openspec/AGENTS.md` Sections 13 and 14
|
||||
- [x] 1.3 Read `packages/storage/storage/src/abstract/abstract.ts` to confirm the public surface this change consumes
|
||||
- [x] 1.4 Confirm `i18n-formatters` and `i18n-spec-conformance` are landed (or land in parallel) before this change archives — the OTel meter wiring depends on them
|
||||
- [x] 1.5 `openspec change validate i18n-storage-backends` exits with no errors
|
||||
|
||||
## 2. Backend interface and error codes
|
||||
|
||||
- [x] 2.1 Create `packages/i18n/i18n/src/core/backend/backend.types.ts` with `Backend`, `BackendInitContext`, `LayoutOption`, `BackendCapabilities` types
|
||||
- [x] 2.2 Add `I18N.BACKEND_READ_FAILED`, `I18N.BACKEND_WRITE_FAILED`, `I18N.BACKEND_NOT_REGISTERED`, `I18N.BACKEND_LOCALE_NOT_AVAILABLE` to `src/error-codes.ts` via `defineCodes`
|
||||
- [x] 2.3 Sibling `backend.types.md` (≥200 words, full interface walkthrough)
|
||||
- [x] 2.4 Author the shared interface-conformance suite at `src/core/backend/backend.test.ts` (every concrete backend imports + runs it)
|
||||
|
||||
## 3. MemoryBackend
|
||||
|
||||
- [x] 3.1 Implement `MemoryBackend` at `src/core/backend/memory/memory-backend.ts` (≤200 lines hard cap)
|
||||
- [x] 3.2 `read`, `readMulti`, `save`, `lastModified` — all four implemented
|
||||
- [x] 3.3 `seed` constructor option for tests
|
||||
- [x] 3.4 `memory-backend.test.ts` — seed read, save round-trip, lastModified monotonicity, readMulti merge
|
||||
- [x] 3.5 Sibling `.md`
|
||||
|
||||
## 4. HttpBackend
|
||||
|
||||
- [x] 4.1 Implement `HttpBackend` at `src/core/backend/http/http-backend.ts`
|
||||
- [x] 4.2 `loadPath` template with `{locale}` / `{namespace}` placeholders
|
||||
- [x] 4.3 Injectable `fetchFn` (default `globalThis.fetch`), `parsePayload`, `customHeaders`, `crossDomain`
|
||||
- [x] 4.4 Map HTTP 404 → `BACKEND_LOCALE_NOT_AVAILABLE`; HTTP 5xx + `TypeError` (network) → `BACKEND_READ_FAILED`
|
||||
- [x] 4.5 Optional `retry: RetryPolicy` from `@suaveplan/retry` (peer; resolves automatically when present)
|
||||
- [x] 4.6 `HttpBackendOptionsSchema` (Zod) at `src/core/backend/http/http-backend.schema.ts`
|
||||
- [x] 4.7 `http-backend.test.ts` — templated URL, 404 mapping, 5xx mapping, parsePayload swap, custom headers callback
|
||||
- [x] 4.8 Sibling `.md`
|
||||
|
||||
## 5. StorageBackend
|
||||
|
||||
- [x] 5.1 Add `@suaveplan/storage workspace:^` to `dependencies` in `packages/i18n/i18n/package.json`
|
||||
- [x] 5.2 Implement `StorageBackend` at `src/core/backend/storage/storage-backend.ts`
|
||||
- [x] 5.3 Three layouts: `single-key-per-namespace` (default), `single-key-per-locale`, `flat-keys`
|
||||
- [x] 5.4 Pluggable codec: `encode(record): Uint8Array` / `decode(bytes): TranslationRecord`; default JSON
|
||||
- [x] 5.5 `lastModified` via `AbstractStorage.getMetadata` when available
|
||||
- [x] 5.6 `StorageBackendOptionsSchema` (Zod)
|
||||
- [x] 5.7 `storage-backend.test.ts` — all three layouts round-trip, codec swap to MessagePack, missing key surfaces `BACKEND_LOCALE_NOT_AVAILABLE`
|
||||
- [x] 5.8 `storage-backend.spec-conformance.test.ts` — uses real `MemoryStorage` from `@suaveplan/storage`, asserts OTel `read.duration_ms` sample emission
|
||||
- [x] 5.9 Sibling `.md` covering all three layouts with tradeoffs
|
||||
|
||||
## 6. MultiFileFsBackend
|
||||
|
||||
- [x] 6.1 Implement `MultiFileFsBackend` at `src/core/backend/multi-file-fs/multi-file-fs-backend.ts`
|
||||
- [x] 6.2 Three layouts: `nested` (default), `flat`, `glob`
|
||||
- [x] 6.3 Optional peer parsers: `yaml`, `gettext-parser` (PO), `json5`; auto-detect via dynamic `import()` and surface a clear `BACKEND_READ_FAILED` if peer is missing for a given file extension
|
||||
- [x] 6.4 `watch: true` opt-in via `node:fs/promises.watch`; emit `loaded` event on file change
|
||||
- [x] 6.5 `MultiFileFsBackendOptionsSchema` (Zod)
|
||||
- [x] 6.6 `multi-file-fs-backend.test.ts` — nested + flat + glob layouts, missing file → `BACKEND_LOCALE_NOT_AVAILABLE`, watch-emits-loaded with a real temp dir
|
||||
- [x] 6.7 Sibling `.md` documenting the "many JSON files in many places" use case directly
|
||||
- [x] 6.8 Subpath export `./backend/multi-file-fs` in `package.json` so browser builds tree-shake the Node-only backend
|
||||
|
||||
## 7. ChainedBackend
|
||||
|
||||
- [x] 7.1 Implement `ChainedBackend` at `src/core/backend/chained/chained-backend.ts`
|
||||
- [x] 7.2 `read` walks in order; first success wins; subsequent backends skipped
|
||||
- [x] 7.3 Cache write-through: after a downstream success, call `save` on every preceding backend that implements it
|
||||
- [x] 7.4 `readMulti` fans out concurrently, merges per `mergeStrategy: 'first-wins' | 'last-wins'`
|
||||
- [x] 7.5 `BACKEND_READ_FAILED` propagates without fallthrough; only `BACKEND_LOCALE_NOT_AVAILABLE` triggers fallthrough
|
||||
- [x] 7.6 `chained-backend.test.ts` — short-circuit on cache hit, fallthrough on locale-not-available, no fallthrough on read-failed, mergeStrategy semantics
|
||||
- [x] 7.7 `chained-backend.spec-conformance.test.ts` — write-through cache uses real `MemoryBackend` + stub `HttpBackend`; verifies exact call sequence
|
||||
- [x] 7.8 Sibling `.md`
|
||||
|
||||
## 8. Catalog integration
|
||||
|
||||
- [x] 8.1 Extend `MessageCatalog` constructor to accept `backend?: Backend` or `backends?: Backend[]`
|
||||
- [x] 8.2 Implement `MessageCatalog.preload(locales, namespaces?) → Promise<void>` with `readMulti` preference
|
||||
- [x] 8.3 Implement `MessageCatalog.invalidate(locale?, namespace?) → void`
|
||||
- [x] 8.4 Implement `MessageCatalog.getAsync(key, locale?) → Promise<LookupResult>` with in-flight load dedup
|
||||
- [x] 8.5 Synchronous `get` / `t` MUST NOT trigger backend reads (verified by spec-conformance test)
|
||||
- [x] 8.6 Implement `saveMissing: true` option — best-effort write-back via `backend.save`; OTel `backend.write.total{outcome=error}` increment on failure (no throw)
|
||||
- [x] 8.7 Modify existing `MessageCatalog.load(locale, namespace?, loader?)` to make `loader` optional and route to backend when omitted
|
||||
- [x] 8.8 `catalog-backend.test.ts` — preload, invalidate, getAsync, saveMissing, sync-get-never-triggers-backend
|
||||
- [x] 8.9 Update `catalog.md` and root `README.md` `## Quick Start` with a `StorageBackend` example
|
||||
|
||||
## 9. OTel instrumentation
|
||||
|
||||
- [x] 9.1 `i18n.backend.read.duration_ms` histogram emission wired in every concrete backend
|
||||
- [x] 9.2 `i18n.backend.read.total` and `i18n.backend.write.total` counters wired in every concrete backend
|
||||
- [x] 9.3 `i18n.backend.cache.size` observable gauge registered against `MessageCatalog`'s internal `Map.size`
|
||||
- [x] 9.4 OTel emission tested with real `InMemoryMetricExporter` in `storage-backend.spec-conformance.test.ts` and `catalog-backend.test.ts`
|
||||
|
||||
## 10. Wiring and exports
|
||||
|
||||
- [x] 10.1 Update `src/index.ts` to export `Backend` interface, all five backend factories, and `MessageCatalog` async surface
|
||||
- [x] 10.2 Add subpath exports for every backend module in `package.json`: `./backend`, `./backend/memory`, `./backend/http`, `./backend/storage`, `./backend/multi-file-fs`, `./backend/chained`
|
||||
- [x] 10.3 README `## Modules` table lists every new co-located `.md`
|
||||
- [x] 10.4 README `## Backends` section with one example per backend; the `MultiFileFsBackend` example explicitly demonstrates the `<locale>/<namespace>.json` workflow
|
||||
|
||||
## 11. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [x] 11.1 `bun run check` — Biome 0/0/0 across the monorepo
|
||||
- [x] 11.2 `bunx turbo lint typecheck test --filter=@suaveplan/i18n` — green
|
||||
- [x] 11.3 `bunx turbo test:coverage --filter=@suaveplan/i18n` — 100/100/100/100
|
||||
- [x] 11.4 `bun scripts/openspec-conformance.ts @suaveplan/i18n` — exit 0, no `error` findings
|
||||
- [x] 11.5 `bunx turbo build --filter=@suaveplan/i18n` — green
|
||||
|
||||
## 12. Pre-archive 7-row checklist (AGENTS.md Section 13)
|
||||
|
||||
- [x] 12.1 Generics threaded; `expectTypeOf` assertions on `Backend.read` return type narrowing for typed message trees
|
||||
- [x] 12.2 Zod schemas wired at every public boundary (`HttpBackendOptionsSchema`, `StorageBackendOptionsSchema`, `MultiFileFsBackendOptionsSchema`, `ChainedBackendOptionsSchema`)
|
||||
- [x] 12.3 All four new error codes thrown from at least one production path
|
||||
- [x] 12.4 Strict-mode tested: `getAsync` rejection vs sync-get fallback paths
|
||||
- [x] 12.5 OTel histograms / counters / gauge emitted from production paths and asserted with a real in-memory `MeterProvider`
|
||||
- [x] 12.6 Lifecycle dedup: concurrent `preload(en, [auth])` calls trigger one backend read; concurrent `getAsync('auth:k')` calls trigger one backend read
|
||||
- [x] 12.7 No retroactive tick — every box reflects work done in this change
|
||||
|
||||
## 13. Archive
|
||||
|
||||
- [x] 13.1 Fold spec delta into `openspec/specs/i18n/spec.md`
|
||||
- [x] 13.2 Move `openspec/changes/i18n-storage-backends/` → `openspec/archive/<yyyy-mm-dd>-i18n-storage-backends/`
|
||||
- [x] 13.3 Refresh `MANIFEST.md` if dep set changed (`@suaveplan/storage` is a new dep)
|
||||
Reference in New Issue
Block a user