Initial Commit

openspec/archive/2026-05-26-api-2pc/proposal.md

@@ -0,0 +1,60 @@
+# `@suaveplan/api-2pc` — two-phase commit coordinator for atomic cross-plugin writes
+
+**Package**: `@suaveplan/api-2pc`
+**Status**: PROPOSED
+**Type**: new-package
+**Stage**: 6 (Distributed transactions + Better-Auth) — see umbrella `openspec/changes/api-plugin-foundation/`
+
+## Why
+
+Some workflows genuinely require atomic cross-DB writes — no partial visibility is acceptable (classic finance/inventory). Per design doc §9.2 the foundation ships a 2PC coordinator on top of engines supporting `PREPARE TRANSACTION`. Postgres is the only mainstream engine that supports it; SQLite and SurrealDB cannot participate as 2PC writers and the coordinator must refuse them at install time. Without this package, plugins needing strict atomicity have no first-class option and would have to roll bespoke coordinator code that nobody can audit.
+
+This package is also the second half of Stage-6 acceptance in the umbrella: "`api-2pc` commits + aborts an integration scenario against real Postgres."
+
+## What changes
+
+Net-new package depending on `@suaveplan/api-types`, `@suaveplan/api-host` (consumes the metadata DB), `@suaveplan/api-manifest`, `@suaveplan/db` (for the `db.preparable` capability check + `PREPARE TRANSACTION` exec), `@suaveplan/error`, `@suaveplan/logger`, `@suaveplan/opentelemetry`, and `zod`.
+
+- ADDED: `createTwoPcCoordinator(deps)` — factory binding the coordinator to a host's metadata DB, capability registry, OTel meter, and clock.
+- ADDED: `run2pc(opts, ctx)` — primary entry point. `opts: { participants: Array<{ plugin, action, input }>, timeoutMs?: number }`. Default `timeoutMs` is 5000ms, hard ceiling 30000ms. Exceeding the ceiling is refused at call time.
+- ADDED: `TwoPcParticipantRegistry` — registers participating plugins; refuses registration if the plugin's bound DB engine does not support `db.preparable` (Surreal / SQLite plugins MAY still register as *read-only voters* and contribute to the vote but MUST NOT participate in write phases).
+- ADDED: `TwoPcCoordinatorState` — `"voting" | "committing" | "aborting" | "committed" | "aborted" | "in_doubt"`.
+- ADDED: `TwoPcVote` — `"commit" | "abort"`.
+- ADDED: Boot-time orphan resolution — on coordinator start, query every participant's `pg_prepared_xacts` for prepared transactions tagged with the host's instance ID; commit those whose ledger row says "all-voted-commit", roll back the rest.
+- ADDED: `outstandingPreparedCap` — default cap of 256 outstanding prepared transactions per coordinator (design doc §14 left this as an implementation choice; this is the spec'd default and is operator-tunable via host config). Exceeding the cap rejects new `run2pc` calls with `TWOPC_OUTSTANDING_LIMIT_EXCEEDED`.
+- ADDED: `run2pcParticipantConformance(participantFactory)` — re-export of the conformance suite from `api-types/twopc`.
+- ADDED: `TWOPC_*` error codes: `TWOPC_TIMEOUT`, `TWOPC_VOTE_ABORT`, `TWOPC_PREPARE_FAILED`, `TWOPC_COMMIT_FAILED`, `TWOPC_ROLLBACK_FAILED`, `TWOPC_IN_DOUBT`, `TWOPC_PARTICIPANT_NOT_PREPARABLE`, `TWOPC_READ_ONLY_PARTICIPANT_CANNOT_WRITE`, `TWOPC_OUTSTANDING_LIMIT_EXCEEDED`, `TWOPC_TIMEOUT_CEILING_EXCEEDED`.
+- ADDED: OTel instruments: `2pc.transaction` span (per transaction lifetime), `2pc.prepare` / `2pc.commit` / `2pc.rollback` spans per participant phase, `2pc.transactions_total` counter `{ outcome }`, `2pc.prepare_duration_ms` histogram, `2pc.commit_duration_ms` histogram, `2pc.outstanding_prepared` observable gauge.
+
+## Out of scope
+
+- The `suaveplan_2pc_transactions` table schema — ships with `api-host`'s migrations. This package only reads/writes rows.
+- Engines other than Postgres — design doc §9.2 explicitly scopes the writer surface to Postgres. SQLite and SurrealDB participate only as read-only voters; this package SHALL refuse them as writers at install time and the spec pins this.
+- Cross-region / multi-host coordinator coordination — umbrella's out-of-scope list excludes multi-region.
+
+## Source mapping
+
+Genesis-original. Source of design intent: `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`.
+
+| Requirement | Design doc section |
+|---|---|
+| `run2pc` API + phase-1 prepare / phase-2 commit | §9.2 (paragraph 2 + code block) |
+| Default `timeoutMs` 5000 / hard ceiling 30000 | §9.2 (operational warning paragraph) |
+| Coordinator state in `suaveplan_2pc_transactions` survives restart | §9.2 (paragraph 3) |
+| Orphan resolution via `pg_prepared_xacts` on boot | §9.2 (paragraph 3) |
+| Postgres-only writer; Surreal/SQLite as read-only voters only | §9.2 (paragraph 1) |
+| `db.preparable` capability requirement | §9.2 (paragraph 4) |
+| `2pc.prepare_duration_ms` + `2pc.commit_duration_ms` instruments | §9.2 (operational warning paragraph) |
+
+## Verification approach
+
+- `src/coordinator/coordinator.test.ts` — happy-path commit, vote-abort path, timeout-then-rollback path, in-doubt path.
+- `src/coordinator/coordinator.spec-conformance.test.ts` — runs `run2pcParticipantConformance` against a reference Postgres participant.
+- `src/registry/participant-registry.test.ts` — refusal of non-preparable engines as writers; acceptance as read-only voters.
+- `src/recovery/recovery.test.ts` (integration, 30000ms) — kills coordinator mid-prepare, restarts, asserts orphan resolution commits or rolls back based on ledger state.
+- `src/limits/outstanding-cap.test.ts` — exceeds the 256-prepared cap, asserts `TWOPC_OUTSTANDING_LIMIT_EXCEEDED`.
+- `tests/integration/postgres-2pc.test.ts` (60000ms) — real `@suaveplan/testing-integration-postgres`: full commit + abort + restart-recovery scenarios end-to-end.
+
+## Migration impact
+
+None — additive only.

openspec/archive/2026-05-26-api-2pc/specs/api-2pc/spec.md

@@ -0,0 +1,143 @@
+<!-- markdownlint-disable MD024 -->
+
+# Capability Spec Delta: api-2pc (api-2pc)
+
+**Package**: `@suaveplan/api-2pc`
+**Status**: PROPOSED
+
+> Every requirement below carries a `**Verified by:**` line at authoring time.
+
+## ADDED Requirements
+
+### Requirement: Two-phase commit driven against Postgres `PREPARE TRANSACTION`
+
+The package SHALL expose `run2pc({ participants, timeoutMs? }, ctx)`. Phase 1 sends `prepare` to every participant; each participant runs its work inside a transaction and issues `PREPARE TRANSACTION '<participantTxId>'`, returning `vote_commit` or `vote_abort`. Phase 2: if all participants vote commit, the coordinator issues `COMMIT PREPARED` everywhere; if any participant votes abort OR the prepare-phase deadline expires, the coordinator issues `ROLLBACK PREPARED` everywhere.
+
+#### Scenario: unanimous commit completes successfully
+
+- **WHEN** every participant votes commit
+- **THEN** the coordinator issues `COMMIT PREPARED` on every participant AND the transaction ends in `committed`
+
+#### Scenario: any vote-abort rolls back everywhere
+
+- **WHEN** at least one participant votes abort
+- **THEN** the coordinator issues `ROLLBACK PREPARED` on every participant that returned `vote_commit` AND emits `TWOPC_VOTE_ABORT`
+
+**Verified by:** `src/coordinator/coordinator.test.ts::"unanimous commit issues COMMIT PREPARED on every participant"` AND `src/coordinator/coordinator.test.ts::"any vote-abort rolls back everywhere with TWOPC_VOTE_ABORT"`
+
+---
+
+### Requirement: SQLite and SurrealDB cannot participate as writers
+
+The participant registry SHALL refuse registration of a writing participant whose bound DB engine does not advertise the `db.preparable` capability. SQLite and SurrealDB plugins MAY register as read-only voters that contribute a `vote_commit`/`vote_abort` based on their own consistency check but MUST NOT be sent prepare/commit SQL. Attempting to bind one as a writer fails at install with `TWOPC_PARTICIPANT_NOT_PREPARABLE`.
+
+#### Scenario: SQLite plugin is rejected as a writer
+
+- **WHEN** a plugin bound to SQLite declares `twoPhaseCommit.participates: true` AND is registered as a write participant
+- **THEN** install fails with `TWOPC_PARTICIPANT_NOT_PREPARABLE`
+
+#### Scenario: SurrealDB plugin participates as a read-only voter
+
+- **WHEN** a SurrealDB-bound plugin is registered with `voterMode: "read-only"`
+- **THEN** registration succeeds AND the coordinator collects its vote in phase 1 but never sends commit/rollback SQL
+
+#### Scenario: read-only voter that attempts a write is refused
+
+- **WHEN** a read-only voter's prepare callback returns a write-intent payload
+- **THEN** the coordinator fails with `TWOPC_READ_ONLY_PARTICIPANT_CANNOT_WRITE`
+
+**Verified by:** `src/registry/participant-registry.test.ts::"rejects SQLite writer with TWOPC_PARTICIPANT_NOT_PREPARABLE"` AND `src/registry/participant-registry.test.ts::"accepts SurrealDB plugin as read-only voter"` AND `src/registry/participant-registry.test.ts::"refuses write-intent from read-only voter"`
+
+---
+
+### Requirement: Timeout default and hard ceiling
+
+The default `timeoutMs` SHALL be 5000ms. The hard ceiling SHALL be 30000ms — calls supplying `timeoutMs > 30000` MUST be refused at call time with `TWOPC_TIMEOUT_CEILING_EXCEEDED` (NOT silently clamped — the operational warning in §9.2 is that long prepares hold row-level locks; the coordinator surfaces the misuse, not hides it). Exceeding the runtime deadline during phase 1 triggers rollback with `TWOPC_TIMEOUT`.
+
+#### Scenario: default timeout applies when none is supplied
+
+- **WHEN** `run2pc` is called without `timeoutMs`
+- **THEN** the effective deadline is 5000ms
+
+#### Scenario: timeoutMs above ceiling is refused
+
+- **WHEN** `run2pc({ ..., timeoutMs: 30001 })` is called
+- **THEN** the call rejects with `TWOPC_TIMEOUT_CEILING_EXCEEDED` BEFORE any participant is contacted
+
+#### Scenario: deadline expiry during prepare triggers rollback
+
+- **WHEN** any participant has not returned a vote before the deadline
+- **THEN** the coordinator issues `ROLLBACK PREPARED` to every prepared participant AND emits `TWOPC_TIMEOUT`
+
+**Verified by:** `src/limits/timeout-ceiling.test.ts::"default timeoutMs is 5000"` AND `src/limits/timeout-ceiling.test.ts::"timeoutMs above 30000 rejects with TWOPC_TIMEOUT_CEILING_EXCEEDED"` AND `src/coordinator/coordinator.test.ts::"deadline expiry during prepare triggers rollback with TWOPC_TIMEOUT"`
+
+---
+
+### Requirement: Outstanding-prepared cap
+
+The coordinator SHALL enforce a default cap of 256 outstanding prepared transactions across all in-flight calls. This default is the spec'd implementation choice for design doc §14's open question on coordinator throttling — it is operator-tunable via host config but the default ships at 256. Exceeding the cap SHALL fail new `run2pc` calls with `TWOPC_OUTSTANDING_LIMIT_EXCEEDED` until in-flight transactions complete.
+
+#### Scenario: 257th concurrent transaction is refused
+
+- **WHEN** 256 transactions are in phase 1 or phase 2 AND a 257th call begins
+- **THEN** the 257th call rejects with `TWOPC_OUTSTANDING_LIMIT_EXCEEDED`
+
+#### Scenario: cap recovers as transactions complete
+
+- **WHEN** an in-flight transaction completes AND a new call arrives
+- **THEN** the new call is admitted
+
+**Verified by:** `src/limits/outstanding-cap.test.ts::"257th concurrent transaction is refused"` AND `src/limits/outstanding-cap.test.ts::"slot recovers as transactions complete"`
+
+---
+
+### Requirement: Persisted state and boot-time orphan resolution
+
+Every coordinator transition SHALL be persisted to `suaveplan_2pc_transactions` (schema owned by `api-host`). On coordinator boot the recovery runner MUST query every participant's `pg_prepared_xacts` for transactions tagged with the host's instance ID. For each orphan: if the ledger row records `all-voted-commit`, issue `COMMIT PREPARED`; otherwise issue `ROLLBACK PREPARED`. Transactions without a corresponding ledger row MUST be rolled back (safe default — no commit without explicit log).
+
+#### Scenario: orphan with "all-voted-commit" ledger row is committed
+
+- **WHEN** the coordinator boots AND finds a prepared transaction whose ledger row is `all-voted-commit`
+- **THEN** `COMMIT PREPARED` is issued AND the row is moved to `committed`
+
+#### Scenario: orphan with no ledger row is rolled back
+
+- **WHEN** the coordinator boots AND finds a prepared transaction with no matching ledger row
+- **THEN** `ROLLBACK PREPARED` is issued
+
+**Verified by:** `src/recovery/recovery.test.ts::"orphan with all-voted-commit ledger row is committed"` AND `src/recovery/recovery.test.ts::"orphan with no ledger row is rolled back"`
+
+---
+
+### Requirement: OTel instruments expose prepare and commit durations
+
+The coordinator SHALL emit, against an injected OTel meter and tracer:
+
+- `2pc.transaction` span per transaction lifetime (attributes: `tx.id`, `outcome`, `participant_count`).
+- `2pc.prepare`, `2pc.commit`, `2pc.rollback` spans per participant phase.
+- `2pc.transactions_total` counter labelled `{ outcome }`.
+- `2pc.prepare_duration_ms` histogram labelled `{ outcome }`.
+- `2pc.commit_duration_ms` histogram labelled `{ outcome }`.
+- `2pc.outstanding_prepared` observable gauge.
+
+Operators rely on `prepare_duration_ms` to spot lock-holding times before they cause throughput regressions.
+
+#### Scenario: successful commit emits both prepare and commit histograms
+
+- **WHEN** a 2PC run completes successfully
+- **THEN** an in-memory `MeterProvider` observes one observation in `2pc.prepare_duration_ms` AND one in `2pc.commit_duration_ms` AND the outstanding gauge briefly rises then falls
+
+**Verified by:** `src/instruments/instruments.test.ts::"emits prepare_duration_ms and commit_duration_ms on successful commit"`
+
+---
+
+### Requirement: Conformance suite for participants
+
+The package SHALL publish `run2pcParticipantConformance(participantFactory)` on the `./testing` subpath, pinning prepare-callback signature, vote semantics, idempotent commit/rollback contract, and the requirement that participants advertise `db.preparable`.
+
+#### Scenario: reference participant passes the suite
+
+- **WHEN** the suite runs against the in-package reference Postgres participant
+- **THEN** every invariant passes
+
+**Verified by:** `src/coordinator/coordinator.spec-conformance.test.ts::"reference Postgres participant passes run2pcParticipantConformance"`

openspec/archive/2026-05-26-api-2pc/tasks.md

@@ -0,0 +1,73 @@
+# Tasks — `@suaveplan/api-2pc`
+
+> Every checkbox MUST reflect work actually done in this change.
+
+## 0. Pre-flight (non-negotiable)
+
+- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
+- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
+- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
+
+## 1. Spec authoring
+
+- [x] 1.1 Read design doc §9.2 (2PC) and §9.3 (engine choice)
+- [x] 1.2 Read umbrella `openspec/changes/api-plugin-foundation/proposal.md` (Stage 6 acceptance)
+- [x] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
+- [x] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
+- [x] 1.5 Draft `specs/api-2pc/spec.md` from `openspec/templates/spec-delta.md`
+- [x] 1.6 Every requirement carries a `**Verified by:**` line
+
+## 2. Implementation
+
+- [x] 2.1 Scaffold `packages/api/api-2pc/` (`"version": "0.1.0"`, `"sideEffects": false`)
+- [x] 2.2 `src/coordinator/coordinator.ts` — `createTwoPcCoordinator` + `run2pc`
+- [x] 2.3 `src/registry/participant-registry.ts` — refusal of non-preparable engines as writers, read-only voter path
+- [x] 2.4 `src/phases/prepare.ts` — Phase 1: collect votes; emit `2pc.prepare` spans + histogram
+- [x] 2.5 `src/phases/commit.ts` — Phase 2 commit: `COMMIT PREPARED` against every participant; emit `2pc.commit` instruments
+- [x] 2.6 `src/phases/rollback.ts` — Phase 2 abort path: `ROLLBACK PREPARED` everywhere; idempotent
+- [x] 2.7 `src/persistence/persistence.ts` — CRUD over `suaveplan_2pc_transactions` (schema owned by `api-host`)
+- [x] 2.8 `src/recovery/recovery.ts` — boot-time orphan resolution via `pg_prepared_xacts`; commit / rollback per ledger state
+- [x] 2.9 `src/limits/outstanding-cap.ts` — enforce default 256 outstanding-prepared cap; configurable via host config
+- [x] 2.10 `src/limits/timeout-ceiling.ts` — refuse `timeoutMs > 30000` at call time
+- [x] 2.11 `src/instruments/instruments.ts` — span/counter/histogram/gauge per design §9.2
+- [x] 2.12 `src/error-codes.ts` — 10 `TWOPC_*` codes
+- [x] 2.13 `./testing` subpath — `run2pcParticipantConformance`
+- [x] 2.14 `src/index.ts` — explicit named re-exports
+
+## 3. Tests
+
+- [x] 3.1 Unit tests per module, every `it(...)` `{ timeout: 5000 }`
+- [x] 3.2 `src/coordinator/coordinator.spec-conformance.test.ts` — conformance suite
+- [x] 3.3 `src/recovery/recovery.test.ts` — Postgres integration via `@suaveplan/testing-integration-postgres` (`{ timeout: 30000 }`)
+- [x] 3.4 `tests/integration/postgres-2pc.test.ts` — full commit + abort + recovery (`{ timeout: 60000 }`)
+- [x] 3.5 All test files import runner primitives from `@suaveplan/testing/runner`
+
+## 4. Documentation
+
+- [x] 4.1 Sibling `.md` for every implementation `.ts`
+- [x] 4.2 `README.md` listing `## Modules`; call out the Postgres-only writer constraint at the top
+
+## 5. Pre-archive verification (the 5-gate triple)
+
+- [x] 5.1 `bun run check` — 0/0/0
+- [x] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-2pc` — green
+- [x] 5.3 `bunx turbo test:coverage --filter=@suaveplan/api-2pc` — 100/100/100/100
+- [x] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/api-2pc` — exit 0
+- [x] 5.5 `bunx turbo build --filter=@suaveplan/api-2pc` — green
+
+## 6. Pre-archive 7-row checklist
+
+- [x] 6.1 Generics threaded; `expectTypeOf` on every literal-type claim
+- [x] 6.2 Zod schemas wired at every boundary
+- [x] 6.3 Every `TWOPC_*` code thrown by production paths
+- [x] 6.4 Strict-mode behaviour tested both on and off (timeout ceiling, outstanding cap)
+- [x] 6.5 OTel instruments asserted against an in-memory `MeterProvider`
+- [x] 6.6 Lifecycle dedup: re-running `recovery` is a no-op once orphans are resolved
+- [x] 6.7 No retroactive tick — every box reflects work in this change
+
+## 7. Archive
+
+- [x] 7.1 Fold spec delta into `openspec/specs/api-2pc/spec.md`
+- [x] 7.2 Move to `openspec/archive/<yyyy-mm-dd>-api-2pc/`
+- [x] 7.3 Tick Stage-6 row in umbrella tasks
+- [x] 7.4 Refresh `MANIFEST.md`