Initial Commit

openspec/archive/2026-05-26-api-saga/proposal.md

@@ -0,0 +1,62 @@
+# `@suaveplan/api-saga` — saga orchestrator for compensable cross-plugin workflows
+
+**Package**: `@suaveplan/api-saga`
+**Status**: PROPOSED
+**Type**: new-package
+**Stage**: 6 (Distributed transactions + Better-Auth) — see umbrella `openspec/changes/api-plugin-foundation/`
+
+## Why
+
+Cross-plugin workflows ("create customer → open subscription → send welcome email") need rollback semantics that no single plugin can give. Per design doc §9.1 the saga engine is the recommended default for these flows — eventual consistency with compensable undo. Sagas execute in declared order; on any step failure the orchestrator runs `compensate` handlers in reverse over completed steps. Without this package, the foundation has no first-class story for multi-plugin eventual consistency, and every plugin author would re-invent state machines + idempotency wrappers ad hoc.
+
+The saga engine is also a Stage-6 prerequisite for the reference plugin's integration test (umbrella §15 Stage 6 acceptance: "a multi-step saga in `api-plugin-example` runs end-to-end including a compensation path").
+
+## What changes
+
+Net-new package depending on `@suaveplan/api-types`, `@suaveplan/api-host` (consumes the metadata DB), `@suaveplan/api-manifest` (reads `sagas.owns[]` / `sagas.participatesIn[]`), `@suaveplan/idempotency` (per-step idempotency wrapper), `@suaveplan/events` (saga step event bus), `@suaveplan/error`, `@suaveplan/logger`, `@suaveplan/opentelemetry`, and `zod`.
+
+- ADDED: `createSagaOrchestrator(deps)` — factory binding the orchestrator to a host's metadata DB, event bus, idempotency store, capability registry, and OTel meter.
+- ADDED: `SagaOrchestrator` — public surface: `start(name, input, ctx)`, `resume(sagaId)`, `cancel(sagaId, reason)`, `listInstances(filter)`, `getInstance(sagaId)`.
+- ADDED: `SagaInstanceState` — `"running" | "completed" | "compensating" | "compensated" | "compensation_failed" | "cancelled"`.
+- ADDED: `SagaStepState` — `"pending" | "in_flight" | "completed" | "failed" | "compensating" | "compensated"`.
+- ADDED: `SagaParticipantRegistry` — host-side registry where plugins register their action and compensate handlers at start; orchestrator dispatches step events into them.
+- ADDED: Event payloads on `@suaveplan/events`: `saga.step.requested`, `saga.step.completed`, `saga.step.failed`, `saga.step.compensated`, `saga.instance.started`, `saga.instance.completed`, `saga.instance.compensated`, `saga.instance.compensation_failed`.
+- ADDED: `runSagaParticipantConformance(participantFactory)` — re-export of the conformance suite from `api-types/saga` (suite definition lives in this package's `./testing` subpath; abstract type contract lives in `api-types`).
+- ADDED: `SAGA_*` error codes: `SAGA_UNKNOWN_NAME`, `SAGA_UNKNOWN_PARTICIPANT`, `SAGA_STEP_TIMEOUT`, `SAGA_COMPENSATION_FAILED`, `SAGA_INVALID_STATE_TRANSITION`, `SAGA_CROSS_TENANT_DENIED`, `SAGA_DUPLICATE_OWNER`, `SAGA_INSTANCE_NOT_FOUND`, `SAGA_PERSISTENCE_FAILED`, `SAGA_COMPENSATION_MAX_ATTEMPTS_EXCEEDED`.
+- ADDED: OTel instruments: `saga.instance` span (per saga lifetime), `saga.step` span (per step attempt), `saga.compensate` span, `saga.instances_total` counter, `saga.step_duration_ms` histogram, `saga.compensations_total` counter, `saga.in_flight` gauge.
+
+## Out of scope
+
+- The host metadata DB schema for `suaveplan_saga_instances` and `suaveplan_saga_steps` — those tables ship with `api-host`'s migrations (umbrella §Stage 2). This package consumes them.
+- Cross-plugin invocation transport — step dispatch goes over the existing `@suaveplan/events` bus + `SagaParticipantRegistry`, not a new RPC channel.
+- Manifest validation — `sagas.owns[]` / `sagas.participatesIn[]` Zod schema lives in `api-manifest`. This package consumes the validated manifest tree.
+- 2PC integration — distinct engine; see `api-2pc`.
+
+## Source mapping
+
+Genesis-original. Source of design intent: `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`.
+
+| Requirement | Design doc section |
+|---|---|
+| Saga orchestrator drives steps in declared order | §9.1 |
+| Compensation in reverse on failure | §9.1 |
+| Persisted state in `suaveplan_saga_instances` / `_steps` | §9.1 (paragraph 2) |
+| Survives host restart by replaying state | §9.1 (paragraph 2) |
+| Idempotency keyed by `{sagaId, stepName}` | §9.1 (bullet "Idempotency contract") |
+| `RequestContext` carries initiator principal + target tenant | §9.1 (final paragraph) |
+| `saga.cross-tenant` capability gate | §9.1 (final paragraph) |
+| OTel instruments listed above | §9.1 (OTel surface block) |
+| Owner vs participant distinction | §9.1 (`sagas.owns` vs `sagas.participatesIn`) |
+
+## Verification approach
+
+- `src/orchestrator/orchestrator.test.ts` — step-ordering, success-path, single-step compensation, multi-step compensation, idempotency-on-retry, cross-tenant denial.
+- `src/orchestrator/orchestrator.spec-conformance.test.ts` — `runSagaParticipantConformance` pinning the contract every step handler must satisfy.
+- `src/registry/participant-registry.test.ts` — owner vs participant registration, duplicate-owner refusal, validation that every step's named plugin exists.
+- `src/persistence/persistence.test.ts` (integration, 30000ms) — uses `@suaveplan/testing-integration-postgres` to assert state survives a simulated host restart: start saga, kill orchestrator mid-flight, instantiate new orchestrator against same DB, assert it resumes correctly.
+- `src/events/events.test.ts` — every documented event payload shape pinned against its Zod schema; bus emission asserted with a real `EventBus`.
+- `tests/integration/cross-plugin-saga.test.ts` (60000ms) — full two-plugin integration: plugin A owns the saga, plugin B participates, force a step-2 failure, assert step-1 compensate fires.
+
+## Migration impact
+
+None — additive only. The umbrella's Stage 6 acceptance criterion depends on this package being green.

openspec/archive/2026-05-26-api-saga/specs/api-saga/spec.md

@@ -0,0 +1,142 @@
+<!-- markdownlint-disable MD024 -->
+
+# Capability Spec Delta: api-saga (api-saga)
+
+**Package**: `@suaveplan/api-saga`
+**Status**: PROPOSED
+
+> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. (See `openspec/AGENTS.md` Section 13.)
+
+## ADDED Requirements
+
+### Requirement: Saga ownership versus participation
+
+The package SHALL distinguish saga *owners* (plugins whose manifest declares `sagas.owns[]`) from saga *participants* (plugins whose manifest declares `sagas.participatesIn[]` AND export named `actions` / `compensations`). At plugin install, the orchestrator MUST validate that for every owned saga's declared steps the named target plugin exists, exports the named action, and — where the step declares a `compensate` — exports the named compensation. Two plugins declaring `owns` of the same saga name MUST cause install to fail with `SAGA_DUPLICATE_OWNER`.
+
+#### Scenario: owner registers a saga its participants can join
+
+- **WHEN** plugin A's manifest declares `sagas.owns: [{ name: "X", steps: [{ plugin: "B", action: "a", compensate: "c" }] }]` AND plugin B's manifest declares `sagas.participatesIn: ["A/X"]` AND plugin B exports `actions.a` and `compensations.c`
+- **THEN** the participant registry validates successfully AND `orchestrator.start("X", input, ctx)` is callable
+
+#### Scenario: duplicate owner is rejected at install
+
+- **WHEN** plugin A and plugin C both declare `owns: [{ name: "X", ... }]`
+- **THEN** install fails with `SAGA_DUPLICATE_OWNER`
+
+#### Scenario: missing participant action is rejected at install
+
+- **WHEN** the saga step names `actions.a` on plugin B AND plugin B does not export `actions.a`
+- **THEN** install fails with `SAGA_UNKNOWN_PARTICIPANT`
+
+**Verified by:** `src/registry/participant-registry.test.ts::"validates owner + participant wiring"` AND `src/registry/participant-registry.test.ts::"rejects duplicate saga owner with SAGA_DUPLICATE_OWNER"` AND `src/registry/participant-registry.test.ts::"rejects missing participant action with SAGA_UNKNOWN_PARTICIPANT"`
+
+---
+
+### Requirement: Step dispatch via event bus with idempotent execution
+
+The orchestrator SHALL drive steps in declared order by emitting `saga.step.requested` on the host event bus with `{sagaId, stepName, input}`. The target plugin's `actions[stepName]` handler runs under a `RequestContext` whose `principal` is the saga initiator and whose `tenantId` is the saga's target tenant. Every step invocation MUST be wrapped in `@suaveplan/idempotency` keyed by `{sagaId, stepName}` unless the action declares `idempotency: "manual"`.
+
+#### Scenario: step completion advances to next step
+
+- **WHEN** an action handler returns success-with-output for step N
+- **THEN** the orchestrator emits `saga.step.completed` AND immediately requests step N+1
+
+#### Scenario: retried step does not double-fire
+
+- **WHEN** the orchestrator retries a step whose handler previously committed under the same `{sagaId, stepName}` key
+- **THEN** the idempotency wrapper returns the cached outcome AND the handler is NOT re-entered
+
+#### Scenario: manual idempotency opts out of the wrapper
+
+- **WHEN** the action declares `idempotency: "manual"`
+- **THEN** the handler is invoked without the idempotency wrapper AND is solely responsible for its own dedup
+
+**Verified by:** `src/orchestrator/orchestrator.test.ts::"step completion advances to next step"` AND `src/idempotency/idempotency.test.ts::"retried step returns cached outcome without re-entry"` AND `src/idempotency/idempotency.test.ts::"manual opt-out bypasses idempotency wrapper"`
+
+---
+
+### Requirement: Reverse-order compensation on step failure
+
+On any step failure (action returns failure-with-cause OR exceeds `timeoutMs`), the orchestrator SHALL run `compensate` handlers in reverse order over completed steps. Compensation handlers MUST be idempotent + retryable up to `maxCompensationAttempts` (default 5). Successful compensation moves the instance to `compensated`; unrecoverable compensation failure moves to `compensation_failed` AND emits `saga.instance.compensation_failed`.
+
+#### Scenario: failure on step 3 compensates steps 2 and 1 in that order
+
+- **WHEN** a 3-step saga fails on step 3 AND every compensate succeeds
+- **THEN** `compensations` for step 2 and then step 1 are invoked in that order AND the instance ends in `compensated`
+
+#### Scenario: compensation max-attempts exceeded yields terminal failure
+
+- **WHEN** a compensate handler fails on every attempt up to `maxCompensationAttempts`
+- **THEN** the instance ends in `compensation_failed` AND `SAGA_COMPENSATION_MAX_ATTEMPTS_EXCEEDED` is emitted AND `saga.instance.compensation_failed` fires
+
+**Verified by:** `src/compensation/compensation.test.ts::"failure on step 3 compensates steps 2 and 1 in reverse order"` AND `src/compensation/compensation.test.ts::"compensation max-attempts exceeded yields compensation_failed"`
+
+---
+
+### Requirement: Persisted state with replay after host restart
+
+Every saga instance and every step transition SHALL be persisted to the host metadata DB (`suaveplan_saga_instances` and `suaveplan_saga_steps`, schema owned by `api-host`). On orchestrator boot the recovery runner MUST query incomplete instances and resume each — completing in-flight actions whose results have not yet been recorded, or re-emitting `saga.step.requested` where the request was never observed.
+
+#### Scenario: orchestrator boot resumes an in-flight instance
+
+- **WHEN** a saga's step 2 was `in_flight` when the previous orchestrator died AND a new orchestrator boots against the same DB
+- **THEN** the recovery runner observes the persisted state AND resumes the saga from step 2 (re-emitting `saga.step.requested` is acceptable because the action handler is idempotent)
+
+#### Scenario: completed instance is not re-replayed
+
+- **WHEN** boot recovery scans the DB
+- **THEN** instances in terminal states (`completed`, `compensated`, `compensation_failed`, `cancelled`) are NOT re-replayed
+
+**Verified by:** `src/persistence/persistence.test.ts::"orchestrator boot resumes in-flight instance from persisted state"` AND `src/persistence/persistence.test.ts::"terminal-state instances are not re-replayed on boot"`
+
+---
+
+### Requirement: Cross-tenant capability gate
+
+A saga step whose target tenant differs from the saga's initiator tenant SHALL require the saga owner's manifest to declare the `saga.cross-tenant` capability. Step dispatch without this capability MUST fail with `SAGA_CROSS_TENANT_DENIED`.
+
+#### Scenario: same-tenant step proceeds without the capability
+
+- **WHEN** a saga's target tenant equals the initiator tenant AND the owner does NOT declare `saga.cross-tenant`
+- **THEN** step dispatch proceeds normally
+
+#### Scenario: cross-tenant step without capability is refused
+
+- **WHEN** a saga's target tenant differs from the initiator AND the owner does NOT declare `saga.cross-tenant`
+- **THEN** dispatch fails with `SAGA_CROSS_TENANT_DENIED`
+
+**Verified by:** `src/tenant/cross-tenant.test.ts::"same-tenant saga proceeds without capability"` AND `src/tenant/cross-tenant.test.ts::"cross-tenant step refused without saga.cross-tenant"`
+
+---
+
+### Requirement: OTel instruments cover instance and step lifecycle
+
+The orchestrator SHALL emit, against an injected OTel meter and tracer:
+
+- `saga.instance` span per saga lifetime (attributes: `saga.name`, `saga.tenant`, `saga.outcome`).
+- `saga.step` span per step attempt (attributes: `saga.name`, `step.name`, `attempt`, `outcome`).
+- `saga.compensate` span per compensation invocation.
+- `saga.instances_total` counter labelled `{ name, outcome }`.
+- `saga.step_duration_ms` histogram labelled `{ name, step, outcome }`.
+- `saga.compensations_total` counter labelled `{ name, step }`.
+- `saga.in_flight` observable gauge of currently-running instances.
+
+#### Scenario: full instance emits matching counter + span pair
+
+- **WHEN** a saga runs to `completed`
+- **THEN** an in-memory `MeterProvider` observes one `saga.instances_total{outcome="completed"}` increment AND one `saga.instance` span for the run
+
+**Verified by:** `src/instruments/instruments.test.ts::"emits instance span and counter on completed saga"` AND `src/instruments/instruments.test.ts::"emits step_duration_ms histogram per step"`
+
+---
+
+### Requirement: Conformance suite published for participants
+
+The package SHALL publish `runSagaParticipantConformance(participantFactory)` on the `./testing` subpath. The suite MUST pin: handler signature shape, idempotency-key contract, compensation-handler signature, error-mapping into `saga.step.failed`, and the `RequestContext` field set visible inside a step handler.
+
+#### Scenario: a reference participant passes the suite
+
+- **WHEN** the suite runs against the in-package reference participant
+- **THEN** every invariant passes
+
+**Verified by:** `src/orchestrator/orchestrator.spec-conformance.test.ts::"reference participant passes runSagaParticipantConformance"`

openspec/archive/2026-05-26-api-saga/tasks.md

@@ -0,0 +1,73 @@
+# Tasks — `@suaveplan/api-saga`
+
+> Every checkbox MUST reflect work actually done in this change.
+
+## 0. Pre-flight (non-negotiable)
+
+- [x] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
+- [x] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
+- [x] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
+
+## 1. Spec authoring
+
+- [x] 1.1 Read design doc §9.1 (sagas) and §9.3 (engine choice)
+- [x] 1.2 Read umbrella `openspec/changes/api-plugin-foundation/proposal.md` (Stage 6 acceptance)
+- [x] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
+- [x] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
+- [x] 1.5 Draft `specs/api-saga/spec.md` from `openspec/templates/spec-delta.md`
+- [x] 1.6 Every requirement carries a `**Verified by:**` line naming concrete `it(...)` titles
+
+## 2. Implementation
+
+- [x] 2.1 Scaffold `packages/api/api-saga/` with `package.json` (`"version": "0.1.0"`, `"sideEffects": false`), `tsconfig.json`, `tsup.config.ts`
+- [x] 2.2 `src/orchestrator/orchestrator.ts` — `createSagaOrchestrator(deps)` factory + `SagaOrchestrator` class
+- [x] 2.3 `src/registry/participant-registry.ts` — owner + participant registration, validation of declared step → plugin → action wiring
+- [x] 2.4 `src/persistence/persistence.ts` — `SagaPersistence` interface bound against host metadata DB; CRUD over `suaveplan_saga_instances` and `suaveplan_saga_steps`
+- [x] 2.5 `src/recovery/recovery.ts` — boot-time replay: query incomplete instances, resume each through orchestrator
+- [x] 2.6 `src/idempotency/idempotency.ts` — per-step idempotency wrapper using `@suaveplan/idempotency` keyed by `{sagaId, stepName}`; honour per-action `idempotency: "manual"` opt-out
+- [x] 2.7 `src/events/events.ts` — Zod schemas for every saga event payload (`saga.step.requested`, `.completed`, `.failed`, `.compensated`, plus `.instance.*`); typed emitter helpers
+- [x] 2.8 `src/compensation/compensation.ts` — reverse-order compensation runner with retry up to `maxCompensationAttempts`; `compensation_failed` terminal state
+- [x] 2.9 `src/tenant/cross-tenant.ts` — capability check for `saga.cross-tenant`; refuse step dispatch otherwise
+- [x] 2.10 `src/state/state-machine.ts` — `SagaInstanceState` + `SagaStepState` transition table with invalid-transition guard
+- [x] 2.11 `src/instruments/instruments.ts` — OTel instrument registration (span/counter/histogram/gauge per design §9.1)
+- [x] 2.12 `src/error-codes.ts` — 10 `SAGA_*` codes; every code thrown from at least one production path
+- [x] 2.13 `./testing` subpath — `runSagaParticipantConformance` suite
+- [x] 2.14 `src/index.ts` — explicit named re-exports
+
+## 3. Tests
+
+- [x] 3.1 Unit tests per module (every `it(...)` `{ timeout: 5000 }`)
+- [x] 3.2 `src/orchestrator/orchestrator.spec-conformance.test.ts` — runs the conformance suite against a reference participant
+- [x] 3.3 `src/persistence/persistence.test.ts` — Postgres integration via `@suaveplan/testing-integration-postgres` (`{ timeout: 30000 }`); pins recovery-after-restart
+- [x] 3.4 `tests/integration/cross-plugin-saga.test.ts` — full two-plugin saga end-to-end with forced compensation (`{ timeout: 60000 }`)
+- [x] 3.5 Every test imports test-runner primitives from `@suaveplan/testing/runner`
+
+## 4. Documentation
+
+- [x] 4.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples)
+- [x] 4.2 `README.md` at package root linking every co-located `.md` under `## Modules`; calls out owner-vs-participant model
+
+## 5. Pre-archive verification (the 5-gate triple)
+
+- [x] 5.1 `bun run check` — Biome 0/0/0
+- [x] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-saga` — green
+- [x] 5.3 `bunx turbo test:coverage --filter=@suaveplan/api-saga` — 100/100/100/100
+- [x] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/api-saga` — exit 0
+- [x] 5.5 `bunx turbo build --filter=@suaveplan/api-saga` — green
+
+## 6. Pre-archive 7-row checklist (AGENTS.md Section 13)
+
+- [x] 6.1 Generics threaded through `SagaOrchestrator<TInput, TOutput>`; `expectTypeOf` assertions on every literal-type claim
+- [x] 6.2 Zod schemas wired at every event-payload boundary and at the manifest-tree consumer
+- [x] 6.3 Every `SAGA_*` error code thrown by at least one production path
+- [x] 6.4 Strict-mode behaviour tested both on and off (idempotency manual-opt-out, cross-tenant gate)
+- [x] 6.5 OTel instruments emitted from production paths and asserted with a real in-memory `MeterProvider`
+- [x] 6.6 Lifecycle dedup tested: re-dispatching a completed step is a no-op; resume after restart does not double-fire
+- [x] 6.7 No retroactive tick — every box above reflects work done in this change
+
+## 7. Archive
+
+- [x] 7.1 Fold spec delta into `openspec/specs/api-saga/spec.md`
+- [x] 7.2 Move `openspec/changes/api-saga/` → `openspec/archive/<yyyy-mm-dd>-api-saga/`
+- [x] 7.3 Tick the Stage-6 row in umbrella `openspec/changes/api-plugin-foundation/tasks.md`
+- [x] 7.4 Refresh `MANIFEST.md`