Initial Commit
This commit is contained in:
138
openspec/changes/event-sourcing-completion/proposal.md
Normal file
138
openspec/changes/event-sourcing-completion/proposal.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# Event-sourcing completion
|
||||
|
||||
**Package**: `@suaveplan/event-sourcing`
|
||||
**Status**: PROPOSED
|
||||
**Type**: spec-conformance + feature
|
||||
|
||||
## Why
|
||||
|
||||
The 2026-05-02 cross-monorepo audit confirmed that `@suaveplan/event-sourcing` shipped on genesis with only the inner-loop pieces — `Aggregate`, `aggregate-repository`, `MemoryEventStore`, the `Projection` builder, and a flat `registerUpcaster` / `upcastEvents` pair. The legacy `packages/universal/event-sourcing/src` package on `../suaveplan/` carries six additional capabilities — `EventOutbox`, `DatabaseEventStore`, `DatabaseSnapshotStore`, `EventReplayer` with progress / pause / resume, a `SubscriptionFilter` + `EventBus` pair, a `SnapshotManager` with state reducer + pruning, and a versioned `UpcasterChain` with `detectSchemaVersion` + helper builders. None of those were archived as out-of-scope in the original migration; they were silently dropped.
|
||||
|
||||
Genesis policy (CLAUDE.md §Workflow rule 5) is "preserve the full public surface during rewrites — trimming is OK ONLY when code is duplicated elsewhere in-repo or can be replaced by an existing `@suaveplan/*` library." None of the six dropped capabilities are duplicated elsewhere on genesis, so each one must be ported. This change closes that gap in one coordinated pass so the next consumer (drive-network, the upcoming projection adapters, and any service that needs reliable event publishing) has a coherent surface to build on.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `EventOutbox` interface — durable transactional outbox; methods `append(events)`, `relay(publisher)`, `pendingCount()`, `failedCount()`, `cleanup(beforeIso)`.
|
||||
- ADDED: `MemoryEventOutbox` class — in-memory implementation of `EventOutbox` for tests and single-process services; preserves insertion order and per-row `attempts` counter.
|
||||
- ADDED: `OutboxPublisher` type alias — `(events: DomainEvent[]) => Promise<void>`.
|
||||
- ADDED: `OutboxRelay` worker class — drains pending records on a configurable interval through an injectable `Clock`; honours `AbortSignal`; emits `outbox.relayed` / `outbox.failed` OTel counters.
|
||||
- ADDED: `DatabaseEventStore<TEvent>` class — `EventStore` adapter backed by a `DatabaseAdapter` from `@suaveplan/db`; optimistic-concurrency check via expected version inside `transaction()`; ledger-style append-only events table with auto-create.
|
||||
- ADDED: `DatabaseSnapshotStore<TState>` class — `SnapshotStoreLike` adapter; latest-snapshot-per-aggregate persistence; configurable `serializer` / `deserializer` so non-JSON state types round-trip.
|
||||
- ADDED: `EventReplayer` class — `replay({ aggregateId | eventType, filter, batchSize, onProgress, signal })`, `pause()`, `resume()`, `cancel()`, `reset()`; `ReplayProgress` with `total` / `processed` / `percentage` / `paused` / `completed`; `ReplayFilter` (eventTypes, fromDate, toDate, aggregateId).
|
||||
- ADDED: `EventBus` class with `SubscriptionFilter` — typed `subscribe(handler, { filter, group })`, `publish(event)`, `publishAll(events)`, round-robin group delivery, `subscriberCount`, `clear()`; subscription returns an `EventSubscription { id, unsubscribe(), active }` handle.
|
||||
- ADDED: `DeadLetterSink` interface + `MemoryDeadLetterSink` — receives events whose handler threw; preserves the original `cause` on `EventSourcingError(EVENT_HANDLER_FAILED)`.
|
||||
- ADDED: `SnapshotManager<TState>` class — automatic snapshot creation at configurable intervals; `load(aggregateId, initialState, reducer)` returns `AggregateLoadResult<TState>` (`state`, `version`, `restoredFromSnapshot`, `eventsReplayed`); pruning to `maxSnapshots` per aggregate; emits `snapshot.created` and `aggregate.loaded` OTel events.
|
||||
- ADDED: `StateReducer<TState>` type alias — `(state: TState, event: DomainEvent) => TState`.
|
||||
- ADDED: `UpcasterChain` ordered registry — `register(upcaster)`, `registerAll(upcasters)`, `upcast(event)`, `upcastAll(events)`, `getUpcasters()`, `getUpcastersForType(type)`, `hasUpcast(event)`, `clear()`. Upcasters are sorted by `fromVersion` so consumers do not depend on registration order.
|
||||
- ADDED: `detectSchemaVersion(event)` — reads `metadata.schemaVersion`, defaults to `1`.
|
||||
- ADDED: `createUpcastedEvent(event, newPayload, targetVersion)` and `createRenamedEvent(event, newType, newPayload, targetVersion)` helpers.
|
||||
- ADDED: `createFieldRenameUpcaster`, `createDefaultFieldUpcaster`, `createRemoveFieldUpcaster` builder functions.
|
||||
- ADDED: `EventSubscriptionOptions`, `EventSubscription`, `SubscriptionFilter`, `OutboxConfig`, `OutboxEntryStatus`, `OutboxEntry`, `DatabaseEventStoreConfig`, `DatabaseSnapshotStoreConfig`, `ReplayConfig`, `ReplayFilter`, `ReplayProgress`, `ProgressCallback`, `ReplayHandler`, `SnapshotManagerConfig`, `AggregateLoadResult`, `VersionedEventMetadata`, `VersionedEvent` types — all colocated with their implementation modules.
|
||||
- MODIFIED: `ES_CODES` registry — adds `OUTBOX_TABLE_INIT_FAILED`, `OUTBOX_PUBLISH_FAILED`, `OUTBOX_RELAY_INTERRUPTED`, `EVENT_INVALID`, `EVENT_HANDLER_FAILED`, `REPLAY_CANCELLED`, `REPLAY_FILTER_INVALID`, `SUBSCRIPTION_INVALID`, `DATABASE_TABLE_INIT_FAILED`, `SNAPSHOT_HANDLER_FAILED`, `UPCASTER_DUPLICATE` codes; the existing `CONCURRENCY_CONFLICT`, `AGGREGATE_NOT_FOUND`, `SNAPSHOT_INVALID`, `UPCASTER_MISSING`, `STREAM_CORRUPT`, `APPEND_FAILED` codes are preserved.
|
||||
- MODIFIED: `index.ts` barrel — re-exports every new symbol; legacy `registerUpcaster` / `clearUpcasters` / `upcastEvents` continue to be exported and now delegate to a module-scoped `UpcasterChain` so both APIs operate over the same registry.
|
||||
- MODIFIED: `package.json` `exports` map — adds subpaths `./outbox/event-outbox`, `./outbox/memory-outbox`, `./outbox/relay`, `./store/database-event-store`, `./store/database-snapshot-store`, `./replay/event-replayer`, `./subscription/event-bus`, `./subscription/dead-letter`, `./snapshot/snapshot-manager`, `./upcasting/chain`.
|
||||
- MODIFIED: `package.json` `dependencies` — adds `@suaveplan/db` (`workspace:^`) for the database-backed adapters.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **SurrealDB / PostgreSQL / SQLite-specific event-store and snapshot-store adapters** — each lands as its own package (`@suaveplan/event-sourcing-surrealdb`, `@suaveplan/event-sourcing-postgres`, `@suaveplan/event-sourcing-sqlite`) so the engine-pluggable matrix mirrors `@suaveplan/crypto` / `@suaveplan/crypto-sodium` / `@suaveplan/crypto-browser`. `DatabaseEventStore` here only needs to satisfy the universal `DatabaseAdapter` contract; per-engine quirks (RANGE locks, `RETURNING` clauses, BLOB serialization) belong with the engine adapter.
|
||||
- **CQRS read-model framework** — `Projection` builder + `EventReplayer` cover the foundation; a full read-model registry with view-versioning, gap detection, and dead-letter rebuild is its own change (`@suaveplan/cqrs`).
|
||||
- **Saga / process-manager pattern** — long-running workflow coordination has dependencies (`@suaveplan/fsm`, `@suaveplan/idempotency`, distributed lock provider) that warrant a separate proposal.
|
||||
- **Distributed event bus** (Kafka / NATS / RabbitMQ adapters) — the `EventBus` class shipped here is in-process publish/subscribe; cross-process distribution is its own change once the wire-format spec is firm.
|
||||
- **`registerUpcaster` legacy free-function deprecation** — kept and delegated; removing it is a breaking change that requires its own deprecation window.
|
||||
- **Re-folding of `openspec/specs/event-sourcing/spec.md`** if a canonical spec file does not yet exist, this change creates it for the cap; full spec folding lands once all event-sourcing changes archive.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change derives from `../suaveplan/packages/universal/event-sourcing/src/`. Genesis `enterprise/event-sourcing.md` is a generic enterprise template (no per-feature requirements), so source mapping cites the upstream package source directly.
|
||||
|
||||
| Requirement | Source file | Source lines | Notes |
|
||||
|---|---|---|---|
|
||||
| `EventOutbox` interface + `MemoryEventOutbox` class | `packages/universal/event-sourcing/src/outbox/outbox-pattern.ts` | L62-L197 | Class-based upstream is database-coupled; split into interface + memory impl on genesis so the abstraction is testable without `@suaveplan/db` |
|
||||
| `OutboxPublisher` type | `packages/universal/event-sourcing/src/outbox/outbox-pattern.ts` | L15 | Verbatim |
|
||||
| `OutboxConfig`, `OutboxEntryStatus`, `OutboxRow` (renamed `OutboxEntry`) types | `packages/universal/event-sourcing/src/outbox/outbox-pattern.ts` | L18-L42 | Verbatim shape; `tableName`/`autoCreate` only apply to the database-backed variant |
|
||||
| `OutboxRelay` worker | `packages/universal/event-sourcing/src/outbox/outbox-pattern.ts` | L103-L140 | Extracted from upstream `relay()` method into a standalone worker class with injectable `Clock` and `AbortSignal` |
|
||||
| `DatabaseEventStore` class | `packages/universal/event-sourcing/src/store/database-event-store.ts` | L44-L191 | Verbatim; `EventStoreDatabaseAdapter` upstream replaced by `DatabaseAdapter` from `@suaveplan/db` |
|
||||
| `DatabaseEventStoreConfig` | `packages/universal/event-sourcing/src/store/database-event-store.ts` | L21-L28 | Verbatim |
|
||||
| Optimistic-concurrency append | `packages/universal/event-sourcing/src/store/database-event-store.ts` | L56-L104 | Verbatim; emits `EventSourcingError(CONCURRENCY_CONFLICT)` |
|
||||
| `DatabaseSnapshotStore<TState>` class | `packages/universal/event-sourcing/src/store/database-snapshot-store.ts` | L37-L137 | Verbatim; serializer / deserializer hooks added so non-JSON state types round-trip |
|
||||
| `EventReplayer` class with progress / pause / resume / cancel | `packages/universal/event-sourcing/src/replay/event-replayer.ts` | L56-L250 | Verbatim; `AbortSignal` threaded through `replay` / `replayByType` / `replayEvents` |
|
||||
| `ReplayFilter`, `ReplayProgress`, `ReplayConfig`, `ReplayHandler`, `ProgressCallback` types | `packages/universal/event-sourcing/src/replay/event-replayer.ts` | L13-L50 | Verbatim |
|
||||
| `EventBus` class with `SubscriptionFilter` | `packages/universal/event-sourcing/src/subscription/event-subscription.ts` | L93-L200 | Verbatim; subscriptionId generation switched to injectable `IdGenerator` to drop module-scope counter |
|
||||
| `SubscriptionFilter`, `SubscriptionOptions`, `EventSubscription`, `EventHandler` types | `packages/universal/event-sourcing/src/subscription/event-subscription.ts` | L13-L41 | Verbatim |
|
||||
| `matchesFilter` predicate helper | `packages/universal/event-sourcing/src/subscription/event-subscription.ts` | L60-L85 | Verbatim |
|
||||
| `SnapshotManager<TState>` class | `packages/universal/event-sourcing/src/snapshot/snapshot-manager.ts` | L49-L177 | Verbatim; `new Date()` replaced with injectable `Clock` |
|
||||
| `AggregateLoadResult<TState>`, `StateReducer<TState>`, `SnapshotManagerConfig` | `packages/universal/event-sourcing/src/snapshot/snapshot-manager.ts` | L19-L42 | Verbatim |
|
||||
| `UpcasterChain` ordered registry | `packages/universal/event-sourcing/src/versioning/event-upcaster.ts` | L91-L142 | Verbatim |
|
||||
| `detectSchemaVersion`, `createUpcastedEvent`, `createRenamedEvent` | `packages/universal/event-sourcing/src/versioning/event-upcaster.ts` | L42-L82 | Verbatim |
|
||||
| `createFieldRenameUpcaster`, `createDefaultFieldUpcaster` | `packages/universal/event-sourcing/src/versioning/event-upcaster.ts` | L147-L203 | Verbatim |
|
||||
| `VersionedEventMetadata`, `VersionedEvent`, `EventUpcaster` types | `packages/universal/event-sourcing/src/versioning/event-upcaster.ts` | L13-L36 | Verbatim |
|
||||
| `eventSourcingEvents` lifecycle emitter (`event:appended`, `snapshot:created`, `aggregate:loaded`) | `packages/universal/event-sourcing/src/event-sourcing-events.ts` | L14-L30 | Verbatim type map; emitter rewired to OTel counters/spans on genesis to avoid coupling tests to a global emitter |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Replace upstream `EventStoreDatabaseAdapter` with `DatabaseAdapter` from `@suaveplan/db` | `@suaveplan/db` already exists on genesis (`packages/database/db/src/`) and is the canonical adapter contract; introducing a second `EventStoreDatabaseAdapter` would fork the abstraction. |
|
||||
| Inject `Clock` and `IdGenerator` from `@suaveplan/types` everywhere upstream uses `Date.now()` / `new Date()` / module-scope counters | Genesis-wide CLAUDE.md determinism rule: "no `Math.random()` / `Date.now()` at module scope; injectable clock and RNG" (enterprise/event-sourcing.md L40). Applies to outbox `id` generation, snapshot `takenAt`, OTel histograms, subscription IDs. |
|
||||
| `AbortSignal` threaded through `EventReplayer.replay*`, `OutboxRelay.start`, `EventBus.publishAll` | Genesis-wide CLAUDE.md rule: "Backpressure & cancellation — `AbortSignal` everywhere async". Upstream lacks signal support in these surfaces. |
|
||||
| `DeadLetterSink` interface + `MemoryDeadLetterSink` implementation | Upstream `EventBus.publish` swallows handler errors silently when grouped subscriptions throw. Genesis policy (CLAUDE.md §Workflow rule 4: professional honesty + no silent failures) requires a structured dead-letter path with the original `cause` preserved on `EventSourcingError(EVENT_HANDLER_FAILED)`. |
|
||||
| Split `EventOutbox` into interface + `MemoryEventOutbox` + `DatabaseEventStore`-backed wiring | Upstream couples the outbox class to a database adapter. Genesis policy (CLAUDE.md §Workflow rule 3: enterprise-grade modularity) requires the contract to be testable in pure memory before any persistence engine binds. |
|
||||
| Zod schemas for `OutboxConfig`, `ReplayFilter`, `SubscriptionOptions`, `SnapshotManagerConfig`, `DatabaseEventStoreConfig`, `DatabaseSnapshotStoreConfig` | enterprise/event-sourcing.md L37: "Zod schemas for all public types". Upstream validates through TypeScript only. |
|
||||
| New `ES_*` codes for outbox / database / replay / subscription / dead-letter / snapshot / upcaster duplicate failures | Genesis-wide policy (CLAUDE.md §Workflow rule 2) that every thrown error route through `@suaveplan/error`. Upstream throws plain `Error` for several of these paths. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each spec delta requirement carries a `**Verified by:**` line pointing at concrete `it(...)` titles. The test files that will exist after this change ships:
|
||||
|
||||
- `packages/events/event-sourcing/src/outbox/event-outbox.test.ts` — interface contract, `MemoryEventOutbox` insertion order + attempts counter
|
||||
- `packages/events/event-sourcing/src/outbox/event-outbox.spec-conformance.test.ts` — Zod-rejected configs emit `OUTBOX_TABLE_INIT_FAILED`; injected `IdGenerator` deterministic; OTel counters
|
||||
- `packages/events/event-sourcing/src/outbox/relay.test.ts` — successful relay marks rows published; failure increments attempts; `AbortSignal` aborts in-flight loops with `OUTBOX_RELAY_INTERRUPTED`
|
||||
- `packages/events/event-sourcing/src/store/database-event-store.test.ts` — append + load round-trip; optimistic-concurrency conflict raises `CONCURRENCY_CONFLICT`; `getEventsByType` honours `limit`
|
||||
- `packages/events/event-sourcing/src/store/database-event-store.spec-conformance.test.ts` — table auto-create gated on `autoCreate: true`; `transaction()` rollback on adapter exception; emits `event:appended` on success
|
||||
- `packages/events/event-sourcing/src/store/database-snapshot-store.test.ts` — save / load latest-per-aggregate; configurable serializer round-trips a non-JSON state; `delete` returns `false` for unknown aggregate
|
||||
- `packages/events/event-sourcing/src/replay/event-replayer.test.ts` — full replay invokes handler in stream order; filter narrows by `eventTypes` / `aggregateId` / `fromDate` / `toDate`; pause / resume / cancel / reset state machine; `onProgress` final tick has `completed: true`
|
||||
- `packages/events/event-sourcing/src/replay/event-replayer.spec-conformance.test.ts` — abort signal mid-batch rejects with `REPLAY_CANCELLED`; bad filter rejected with `REPLAY_FILTER_INVALID`
|
||||
- `packages/events/event-sourcing/src/subscription/event-bus.test.ts` — `subscribe` / `publish` round-trip; filter narrows by `eventTypes` / `aggregateIds` / `aggregateType`; group round-robin distribution; `unsubscribe` removes record
|
||||
- `packages/events/event-sourcing/src/subscription/event-bus.spec-conformance.test.ts` — handler exceptions route to `DeadLetterSink` with `EVENT_HANDLER_FAILED` cause; injected `IdGenerator` produces deterministic subscription IDs; `subscriberCount` reflects `unsubscribe()`
|
||||
- `packages/events/event-sourcing/src/subscription/dead-letter.test.ts` — `MemoryDeadLetterSink` capture; iteration order; `clear()` resets
|
||||
- `packages/events/event-sourcing/src/snapshot/snapshot-manager.test.ts` — `load` rebuilds from snapshot + tail events; auto-create at `interval`; pruning at `maxSnapshots`; `shouldCreateSnapshot` predicate
|
||||
- `packages/events/event-sourcing/src/snapshot/snapshot-manager.spec-conformance.test.ts` — injected `Clock` stamps `takenAt` deterministically; `aggregate:loaded` OTel event carries `restoredFromSnapshot`; reducer exception raises `SNAPSHOT_HANDLER_FAILED`
|
||||
- `packages/events/event-sourcing/src/upcasting/chain.test.ts` — `register` orders by `fromVersion`; `upcast` walks chain v1→v2→v3; `getUpcastersForType` filters; `clear()` empties registry
|
||||
- `packages/events/event-sourcing/src/upcasting/chain.spec-conformance.test.ts` — `detectSchemaVersion` defaults to `1`; duplicate `(eventType, fromVersion)` registration raises `UPCASTER_DUPLICATE`; `createFieldRenameUpcaster` round-trip
|
||||
- `tests/dep-graph.test.ts` (cross-package) — `@suaveplan/event-sourcing` declares `@suaveplan/db` as a runtime dep; the package still has zero browser-only deps
|
||||
|
||||
## Migration impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/event-sourcing` to enumerate. As of `grep -rl "@suaveplan/event-sourcing" packages/ --include="package.json"` (2026-05-02), the only match is the package's own `package.json` — there are no in-repo dependents on genesis yet.
|
||||
|
||||
- (no in-repo dependents) — additive only; existing barrel exports (`Aggregate`, `MemoryEventStore`, `createAggregateRepository`, `createProjection`, `registerUpcaster`, `upcastEvents`) are preserved unchanged. New exports (`EventOutbox`, `DatabaseEventStore`, `DatabaseSnapshotStore`, `EventReplayer`, `EventBus`, `SnapshotManager`, `UpcasterChain`, `detectSchemaVersion`, `createFieldRenameUpcaster`, etc.) are additive.
|
||||
|
||||
The legacy `registerUpcaster` / `clearUpcasters` / `upcastEvents` flat API continues to work and now delegates to a module-scoped `UpcasterChain` so both APIs operate over the same registry. No breaking changes; consumers can adopt the new class API at their own pace.
|
||||
|
||||
For foundation-package safety, a `bunx turbo build typecheck test --filter=...@suaveplan/event-sourcing` sweep runs in §6 of `tasks.md`.
|
||||
|
||||
## Parity Checklist
|
||||
|
||||
| Capability | Upstream | Migrated | Notes |
|
||||
|---|---|---|---|
|
||||
| `Aggregate` base class | yes | yes | Already on genesis |
|
||||
| `aggregate-repository` factory | yes | yes | Already on genesis |
|
||||
| `MemoryEventStore` | yes | yes | Already on genesis |
|
||||
| `DatabaseEventStore` (DB-backed) | yes | yes (this change) | Ported to `@suaveplan/db` adapter |
|
||||
| `Snapshot` type | yes | yes | Already on genesis |
|
||||
| `DatabaseSnapshotStore` | yes | yes (this change) | Adds serializer hooks |
|
||||
| `SnapshotManager<TState>` | yes | yes (this change) | `new Date()` → injectable `Clock` |
|
||||
| `Projection` builder | yes | yes | Already on genesis |
|
||||
| `EventBus` + `SubscriptionFilter` | yes | yes (this change) | Adds dead-letter sink |
|
||||
| `EventReplayer` (progress / pause / resume) | yes | yes (this change) | Adds `AbortSignal` |
|
||||
| `EventOutbox` + `OutboxRelay` | yes | yes (this change) | Split interface / memory / DB |
|
||||
| `UpcasterChain` ordered registry | yes | yes (this change) | Replaces flat `registerUpcaster` |
|
||||
| `detectSchemaVersion`, `createUpcastedEvent`, `createRenamedEvent` | yes | yes (this change) | Verbatim |
|
||||
| `createFieldRenameUpcaster`, `createDefaultFieldUpcaster` | yes | yes (this change) | Adds `createRemoveFieldUpcaster` |
|
||||
| `eventSourcingEvents` lifecycle emitter | yes | yes (this change) | Rewired to OTel; back-compat type map preserved |
|
||||
| Flat `registerUpcaster` / `upcastEvents` | yes | yes | Already on genesis; now delegates to `UpcasterChain` |
|
||||
|
||||
No upstream capabilities are dropped.
|
||||
Reference in New Issue
Block a user