SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
suaveplan/openspec/changes/schema-registry-completion/proposal.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

17 KiB
Raw Permalink Blame History

Schema-registry completion

Package: @suaveplan/schema-registry Status: PROPOSED Type: feature

Why

Genesis ships only a thin in-memory SchemaRegistry (createRegistry, register / getLatest / getVersion / getAll / listSubjects / deleteSubject / setCompatibility / checkCompatibility / clear) plus the checkCompatibility predicate and a four-code error registry. The legacy packages/universal/schema-registry/src shipped a full enterprise schema-registry surface with seven additional capabilities that genesis silently dropped:

  1. Transitive compatibility — BACKWARD_TRANSITIVE / FORWARD_TRANSITIVE / FULL_TRANSITIVE modes that check a candidate against ALL prior versions, not just the latest. Legacy src/compatibility/transitive-checker.ts (L1-L100) ships checkTransitiveAgainstAll, buildTransitiveReport, TransitiveReport, TransitiveVersionResult, isTransitiveMode, stripTransitive. Without these, schema-evolution policies that mandate "every old consumer must keep working" cannot be enforced — a real concern for event-sourced systems and message buses.
  2. Schema diff — diffSchemas(a, b) → SchemaDiffResult and formatDiffReport(result) → human string, plus the SchemaDiffChange and DiffChangeType types. Legacy src/diff/schema-differ.ts (L1-L175). Without diff, callers cannot surface "what specifically broke" to ops users.
  3. SchemaStoreAdapter interface — the contract that lets the registry persist beyond process memory. Legacy src/types/registry.types.ts (L82-L105) declares 8 methods; genesis has no analogous abstraction.
  4. MemorySchemaStore — in-memory implementation of SchemaStoreAdapter. Legacy ships this implicitly inside the registry; genesis needs it as a standalone module so consumers can swap stores.
  5. DatabaseSchemaStore — DatabaseAdapter-backed implementation. Legacy src/store/database-store.ts (L1-L147). Without it, the genesis registry is process-local.
  6. Canonicalisation utilities — canonicalizeSchema, computeFingerprint, djb2Hash, flattenFields, isOptionalOrHasDefault. Legacy src/utils/utils.ts (L1-L70). The registry needs deterministic fingerprints to deduplicate identical re-registrations and to support hash-based subject indexing.
  7. SubjectConfig — per-subject configuration object including compatibility mode plus metadata. Legacy src/types/registry.types.ts (L65-L68) ships a minimal version; the genesis registry hardcodes the compat level on SubjectState with no metadata or persisted version cap.

The genesis-original behaviour the registry already ships (auto-incrementing version, INVALID_SCHEMA validation, OTel register.total counter and check.duration_ms histogram) is preserved. This change is purely additive on the existing SchemaRegistry surface — no current export is removed or renamed.

What changes

  • ADDED: CompatibilityLevel widened to "BACKWARD" | "FORWARD" | "FULL" | "NONE" | "BACKWARD_TRANSITIVE" | "FORWARD_TRANSITIVE" | "FULL_TRANSITIVE". Existing four levels keep their meaning; the three new modes activate per-subject transitive checking.
  • ADDED: checkTransitiveCompatibility(history: readonly SchemaDefinition[], candidate: unknown, mode: CompatibilityLevel) → CompatibilityResult — convenience entry point.
  • ADDED: buildTransitiveReport(history, candidate, mode) → TransitiveReport — verbose per-version compatibility breakdown.
  • ADDED: TransitiveReport interface — { compatible: boolean; mode: CompatibilityLevel; versionResults: TransitiveVersionResult[]; totalVersions: number; failedVersions: number }.
  • ADDED: TransitiveVersionResult interface — { version: number; compatible: boolean; errors: readonly string[] }.
  • ADDED: isTransitiveMode(mode) and stripTransitive(mode) helper functions.
  • ADDED: diffSchemas(oldSchema, newSchema) → SchemaDiffResult — structural diff identifying added / removed / modified fields and flagging breaking changes.
  • ADDED: formatDiffReport(result) → human-readable string — counts plus per-change + / - / ~ rendering with [BREAKING] annotation.
  • ADDED: SchemaDiffChange interface — { changeType: DiffChangeType; path: string; breaking: boolean; description: string; oldField?: SchemaField; newField?: SchemaField }.
  • ADDED: SchemaDiffResult interface — { changes: readonly SchemaDiffChange[]; hasBreakingChanges: boolean; summary: string }.
  • ADDED: DiffChangeType union — "added" | "removed" | "modified".
  • ADDED: SchemaField interface — { name: string; type: string; required?: boolean; defaultValue?: unknown; fields?: readonly SchemaField[] }. Used by the diff and canonicalisation utilities. (Legacy registry already had this; genesis dropped it.)
  • ADDED: SchemaStoreAdapter interface — getVersions(subject), getVersion(subject, version), getLatest(subject), putVersion(version), getSubjects(), hasSubject(subject), deleteSubject(subject), getConfig(subject), setConfig(subject, config). All async.
  • ADDED: MemorySchemaStore class implementing SchemaStoreAdapter against an in-memory Map.
  • ADDED: DatabaseSchemaStore<TAdapter extends DatabaseAdapter> class implementing SchemaStoreAdapter over @suaveplan/db's DatabaseAdapter. Constructor takes (db, tableName?, configTableName?) defaulting to "schema_versions" / "schema_configs".
  • ADDED: canonicalizeSchema(schema) → stable JSON string with fields sorted at every level.
  • ADDED: computeFingerprint(schema) → 8-hex-char djb2 hash of the canonical form.
  • ADDED: djb2Hash(input) → 8-hex-char hash. Exported as a building block.
  • ADDED: flattenFields(fields, prefix?) → Map<string, SchemaField> keyed by dot-notation paths.
  • ADDED: isOptionalOrHasDefault(field) → boolean.
  • ADDED: SubjectConfig interface — { compatibility: CompatibilityLevel; maxVersions?: number; metadata?: Record<string, string> }. Replaces the inline-on-SubjectState representation.
  • MODIFIED: SchemaRegistry constructor — accepts an optional store?: SchemaStoreAdapter. When supplied, persistence routes through the store; otherwise the existing in-process Map path is used. The synchronous public API is preserved when no store is supplied; when a store IS supplied, an alternative async surface (registerAsync, getLatestAsync, getVersionAsync, getAllAsync, listSubjectsAsync, deleteSubjectAsync, checkCompatibilityAsync) is exposed alongside the existing sync methods (which throw if a store is configured but the call is sync).
  • MODIFIED: SchemaDefinition — metadata?: Record<string, string> already supported; ADDED a non-breaking fingerprint?: string field that, when present, equals computeFingerprint(schema as SchemaDefinition).
  • ADDED: error code SCHEMA_REGISTRY.STORE_FAILURE — thrown when the underlying SchemaStoreAdapter fails (network, DB).
  • ADDED: error code SCHEMA_REGISTRY.UNKNOWN_COMPATIBILITY_MODE — thrown when an invalid mode string is supplied.

Out of scope

  • Confluent Schema Registry HTTP client — the legacy package never shipped one; the integration with Confluent's REST API is a separate pending change tracked as schema-registry-confluent-adapter. Reason for splitting: the Confluent contract is large (subjects, configs, deletes, soft/hard deletes, modes, mode reset, schema-by-id lookup, version-by-schema lookup), authentication varies (basic/bearer/mTLS), and pulling it in here would dilute review of the much smaller transitive + diff + store surface.
  • Avro / Protobuf / JSON-Schema specific format adapters — SchemaFormat already includes "json-schema" | "zod" | "avro" | "custom" in genesis. This change ships only the internal Zod-described SchemaDefinition shape that drives the diff and compatibility engines; semantic adapters per format land in follow-up changes (schema-registry-avro-adapter, schema-registry-protobuf-adapter). Reason: each format has bespoke compatibility semantics — Avro reader/writer schema rules differ from JSON Schema's $ref resolution.
  • Webhook notifications on schema change — the legacy package did not ship them; any register → fan-out integration is delegated to @suaveplan/webhook-client by composition.
  • Schema validation (running data through a schema) — @suaveplan/validation is the canonical home; the registry stores schemas, the validation package validates payloads against them.
  • zod-as-SchemaDefinition autodiff — the diff engine works on SchemaDefinition ({ type, fields }) per the legacy contract. Translating a Zod schema into SchemaDefinition is a follow-up; for now consumers either store JSON-Schema-shaped definitions or supply a custom translator.

Source mapping

This change derives from the legacy packages/universal/schema-registry/src tree at ../suaveplan/. No enterprise/schema-registry.md per-feature audit doc exists; source mapping cites the upstream files directly.

Requirement Source file Source lines Notes
checkTransitiveCompatibility (= checkTransitiveAgainstAll) packages/universal/schema-registry/src/compatibility/transitive-checker.ts L18-L35 Renamed for symmetry with checkCompatibility; behaviour verbatim
buildTransitiveReport packages/universal/schema-registry/src/compatibility/transitive-checker.ts L58-L84 Verbatim
TransitiveReport interface packages/universal/schema-registry/src/compatibility/transitive-checker.ts L94-L100 Verbatim shape; widen mode to genesis CompatibilityLevel
TransitiveVersionResult interface packages/universal/schema-registry/src/compatibility/transitive-checker.ts L87-L91 Verbatim
isTransitiveMode / stripTransitive helpers packages/universal/schema-registry/src/compatibility/transitive-checker.ts L40-L52 Verbatim
diffSchemas packages/universal/schema-registry/src/diff/schema-differ.ts L16-L40 Verbatim
formatDiffReport packages/universal/schema-registry/src/diff/schema-differ.ts L45-L64 Verbatim
SchemaDiffChange interface packages/universal/schema-registry/src/types/registry.types.ts L118-L132 Verbatim
SchemaDiffResult interface packages/universal/schema-registry/src/types/registry.types.ts L134-L142 Verbatim
DiffChangeType union packages/universal/schema-registry/src/types/registry.types.ts L116 Verbatim
SchemaField interface packages/universal/schema-registry/src/types/registry.types.ts L15-L26 Verbatim
SchemaStoreAdapter interface packages/universal/schema-registry/src/types/registry.types.ts L82-L105 Verbatim shape; widen mode types to genesis CompatibilityLevel
DatabaseSchemaStore class packages/universal/schema-registry/src/store/database-store.ts L1-L147 Re-target DatabaseAdapter import to @suaveplan/db's DatabaseAdapter (legacy used an inline interface)
canonicalizeSchema / computeFingerprint / djb2Hash / flattenFields / isOptionalOrHasDefault packages/universal/schema-registry/src/utils/utils.ts L1-L70 Verbatim
SubjectConfig (compatibility, maxVersions?, metadata?) packages/universal/schema-registry/src/types/registry.types.ts L65-L68 Extended: legacy ships only compatibility; genesis adds maxVersions and metadata (already partially modelled as constructor option).

Requirements with no source line

Requirement Justification
MemorySchemaStore standalone class Legacy ships the in-memory case implicitly inside the registry; genesis exposes it as a standalone, importable module so consumers can swap stores without re-wiring. Symmetric with DatabaseSchemaStore.
SchemaRegistry.{registerAsync, getLatestAsync, ...} async surface Legacy registry is sync; legacy SchemaStoreAdapter is async. Bridging requires either making the whole API async (BREAKING) or shipping a parallel async surface. Genesis chooses the additive route to preserve the existing sync API for in-memory users.
SCHEMA_REGISTRY.STORE_FAILURE and SCHEMA_REGISTRY.UNKNOWN_COMPATIBILITY_MODE codes Genesis-wide policy — every thrown error routes through @suaveplan/error (CLAUDE.md §Workflow rule 2).
SchemaDefinition.fingerprint? field Required for de-duplication when an identical schema is re-registered; without it, every retry creates a new version. The fingerprint is computed by the registry at register-time using computeFingerprint; consumers do not supply it.

Verification approach

Each spec delta requirement carries a **Verified by:** line pointing at concrete it(...) titles. The test files that will exist after this change ships:

  • packages/core/schema-registry/src/core/transitive/transitive.test.ts — covers checkTransitiveCompatibility against multi-version histories
  • packages/core/schema-registry/src/core/transitive/transitive.spec-conformance.test.ts — pins isTransitiveMode / stripTransitive exhaustively over the seven CompatibilityLevel values
  • packages/core/schema-registry/src/core/diff/diff.test.ts — covers diffSchemas for added / removed / modified / type-changed cases; formatDiffReport rendering
  • packages/core/schema-registry/src/core/diff/diff.spec-conformance.test.ts — property tests diffSchemas(a, a) === { changes: [], hasBreakingChanges: false } and formatDiffReport(diff(a, a)) === "No changes detected."
  • packages/core/schema-registry/src/store/memory/memory.test.ts — covers MemorySchemaStore round-trips, subject delete, config get/set
  • packages/core/schema-registry/src/store/memory/memory.spec-conformance.test.ts — pins the SchemaStoreAdapter contract
  • packages/core/schema-registry/src/store/database/database.test.ts — covers DatabaseSchemaStore against an in-memory DatabaseAdapter stub
  • packages/core/schema-registry/src/store/database/database.spec-conformance.test.ts — pins the same contract; verifies STORE_FAILURE is raised on adapter exception
  • packages/core/schema-registry/src/utils/canonical/canonical.test.ts — covers canonicalizeSchema, computeFingerprint, djb2Hash, flattenFields, isOptionalOrHasDefault
  • packages/core/schema-registry/src/utils/canonical/canonical.spec-conformance.test.ts — fast-check property canonicalizeSchema(a) === canonicalizeSchema(shuffleFields(a)) (canonicalisation is order-independent)
  • packages/core/schema-registry/src/core/registry/schema-registry.test.ts — extended to cover the async surface and store wiring
  • packages/core/schema-registry/src/core/registry/schema-registry.spec-conformance.test.ts — pins the UNKNOWN_COMPATIBILITY_MODE and STORE_FAILURE error contracts; pins the SchemaDefinition.fingerprint invariant (re-register identical schema produces same fingerprint and triggers de-dup)

Migration

This is an additive change to the SchemaRegistry constructor and CompatibilityLevel union. No public API is removed; existing in-memory consumers continue to work without modification.

For consumers that want to switch to a persisted store:

  • Old behavior: new SchemaRegistry({ defaultCompatibility: "BACKWARD" }) was process-local.
  • New behavior: new SchemaRegistry({ defaultCompatibility: "BACKWARD", store: new DatabaseSchemaStore(dbAdapter) }) persists to the supplied store. The async surface (registerAsync, getLatestAsync, ...) replaces the sync one; calling sync methods on a store-backed registry throws SCHEMA_REGISTRY.STORE_FAILURE with context.reason === "sync_call_on_async_store".
  • Search command consumers should run: grep -rln "new SchemaRegistry\|createRegistry" packages/ src/.
  • Recommended replacement: for in-memory users, no change. For persisted users, switch construction to pass store and migrate call sites to the *Async variants.

For consumers using CompatibilityLevel:

  • Old behavior: the union was "BACKWARD" | "FORWARD" | "FULL" | "NONE".
  • New behavior: the union also includes "BACKWARD_TRANSITIVE" | "FORWARD_TRANSITIVE" | "FULL_TRANSITIVE". Exhaustive switch statements over CompatibilityLevel will surface a TypeScript narrowing miss until consumers add the three new arms (or fall through them to the corresponding non-transitive case).
  • Search command: grep -rln "CompatibilityLevel\|defaultCompatibility" packages/ src/.
  • Recommended replacement: in any exhaustive switch, fall through BACKWARD_TRANSITIVE to the BACKWARD arm; same for FORWARD_TRANSITIVE / FULL_TRANSITIVE. The transitive variants only differ at register-time (check against all versions vs. latest).

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/schema-registry to enumerate. From grep -rl "@suaveplan/schema-registry" /home/ubuntu/web/genesis/packages/ --include="package.json":

  • @suaveplan/events-schema — [x] tests run / [x] docs reviewed / [ ] no action needed. The events-schema package depends on the registry to track event-schema versions; it currently uses the in-memory path. After this change it can opt into transitive compatibility (event consumers commonly span multiple schema versions in flight) and persisted storage (event schemas should outlive process restarts). Tests must verify the package keeps building and passing with the new union shape.

For all other monorepo packages: no action — they do not depend on @suaveplan/schema-registry. This change is additive only.

Reference in New Issue View Git Blame Copy Permalink