Initial Commit

openspec/archive/2026-05-18-add-api-migrations/proposal.md

@@ -0,0 +1,52 @@
+# Add @suaveplan/api-migrations — DB-adapter-agnostic migration runner (4 of 5)
+
+**Package**: `@suaveplan/api-migrations` (NEW, server-tier)
+**Status**: PROPOSED
+**Type**: new-package (split from upstream `@suaveplan/api`)
+
+- **Spec delta**: ./specs/api-migrations/spec.md
+- **Companions**: `add-api-plugin`, `add-api-tenant`, `add-api-openapi`, `enterprise-api`
+- **Position in chain**: independent of plugin/tenant/openapi (only depends on `@suaveplan/types` `DatabaseLike`)
+
+## Why
+
+Upstream `@suaveplan/api`'s `database/migration-runner.ts` (331 LOC) hides three problems: (1) it's pretends to be database-adapter-agnostic via the `DatabaseAdapter` interface but bootstraps its tracking table with **SurrealQL DDL** (`DEFINE TABLE _migrations SCHEMAFULL ... DEFAULT time::now()`), locking it to SurrealDB; (2) it declares `validateChecksums: true` as default but never actually compares stored checksums against current file content — silent drift; (3) it's coupled to the plugin registry's `applyMigrations()` soft-fail path, so production deploys can have migrations silently skipped.
+
+Splitting it into a dedicated package makes the dialect-bootstrap pluggable, **actually validates checksums**, and decouples migration from plugin init so the app orchestrator runs migrations at an explicit boot phase with explicit error policy.
+
+## What changes
+
+Net-new package at `0.1.0`. ADDED:
+
+- `MigrationRunner` class — reads migration files from a directory, applies pending ones in order, records each with SHA-256 checksum. Constructor takes a `MigrationAdapter` (NOT the broader `DatabaseAdapter`) so the surface is narrow and testable.
+- `MigrationAdapter` interface — `{ dialect: MigrationDialect; ensureTable(name: string): Promise<void>; selectApplied(table: string): Promise<MigrationRecord[]>; insertApplied(table, record): Promise<void>; execute(content: string): Promise<void>; deleteApplied(table, id): Promise<void> }`. Each adapter ships its own `ensureTable` DDL.
+- `MigrationDialect` literal union: `"surrealql" | "postgres" | "sqlite" | "mysql"`. Shipped adapters: `createSurrealqlAdapter(surrealAdapter)`, `createPostgresAdapter(pgClient)`, `createSqliteAdapter(sqliteDb)`. MySQL is declared in the union but ships as a separate package (`@suaveplan/api-migrations-mysql`) — included as a typed slot to prevent the union from being closed.
+- `MigrationFile` (`{ name, path, order, content?, checksum?: string }`), `MigrationRecord` (`{ id, name, appliedAt, checksum, duration }`), `MigrationResult` (`{ name, success, duration, error? }`).
+- `MigrationRunnerOptions` — `{ tableName?: string (default "_migrations"); fileExtension?: string (default ".surql" / ".sql" per dialect); validateChecksums?: boolean (default true, ACTUALLY VALIDATES); dryRun?: boolean; logger?: LoggerLike; clock?: Clock }`.
+- **Checksum enforcement**: when `validateChecksums: true`, before applying any pending migration, the runner computes SHA-256 of every already-applied migration's file and compares against the stored checksum. Mismatch raises `MigrationError(MIGRATION_CHECKSUM_DRIFT)` with `context.{name, expected, actual}` — apply halts. Genesis behaviour change from upstream's no-op.
+- `runFromDirectory(dir)`, `run(files)`, `runMigration(file)`, `rollback()`, `getAppliedMigrations()`, `getPendingMigrations(dir)`, `scanDirectory(dir)`.
+- `MigrationError extends SuaveplanError<MigrationErrorCode>` with codes `MIGRATION_TABLE_BOOTSTRAP_FAILED`, `MIGRATION_APPLY_FAILED`, `MIGRATION_CHECKSUM_DRIFT`, `MIGRATION_DIRECTORY_NOT_FOUND`, `MIGRATION_INVALID_FILE_NAME`, `MIGRATION_ROLLBACK_NOT_SUPPORTED`.
+- `AbortSignal` propagation — `runFromDirectory(dir, { signal })` honors cancellation between migrations.
+- OTel — counter `migrations.applied_total{dialect, outcome}`, histogram `migrations.apply_duration_ms{name}`.
+- Subpath exports per feature module + per dialect adapter.
+
+## Out of scope
+
+- **Schema diffing / migration generation.** Genesis ships only the apply-runner. Tools that generate migration files from schema diffs (`drizzle-kit` etc.) are external.
+- **Down migrations.** SurrealDB has no automatic rollback support; Postgres/SQLite do but require careful authoring. Shipped `rollback()` only deletes the tracking record for the last applied migration; reverting schema is application policy.
+- **Distributed lock for multi-instance deploys.** A real distributed advisory lock (Postgres `pg_advisory_lock`, Redis `setnx`) belongs in an adapter-side wrapper. The shipped runner is single-instance-safe (transactional record insert) but not cluster-coordinated.
+- **MySQL adapter.** Declared in the dialect union but actual adapter ships separately as `@suaveplan/api-migrations-mysql`.
+
+## Source mapping
+
+| Capability | Upstream source | Notes |
+|---|---|---|
+| Migration runner + scanDirectory + checksum compute | `../suaveplan/packages/server/api/src/database/migration-runner.ts` L1-L331 | port shape + **actually validate checksums** + narrow adapter interface from `DatabaseAdapter` to `MigrationAdapter` |
+| SurrealQL DDL bootstrap | `../suaveplan/packages/server/api/src/database/migration-runner.ts` L277-L284 | extract into `createSurrealqlAdapter` so it's per-dialect, not hard-coded |
+| `MigrationFile` / `MigrationRecord` / `MigrationResult` types | `../suaveplan/packages/server/api/src/database/migration-runner.ts` L9-L38 | verbatim port |
+
+## Impact
+
+### Affected dependents
+
+None today — net-new. Will be consumed by `@suaveplan/api` orchestrator (calls `runner.runFromDirectory(...)` between plugin register and init phases) and any plugin that ships `.surql`/`.sql` files alongside its source.

openspec/archive/2026-05-18-add-api-migrations/specs/api-migrations/spec.md

@@ -0,0 +1,150 @@
+<!-- markdownlint-disable MD024 -->
+
+# Capability Spec Delta: api-migrations (add-api-migrations)
+
+**Package**: `@suaveplan/api-migrations`
+**Status**: PROPOSED
+
+## ADDED Requirements
+
+### Requirement: MigrationAdapter is the per-dialect surface
+
+The package SHALL export `MigrationAdapter` interface with `{ dialect: MigrationDialect; ensureTable(name): Promise<void>; selectApplied(table): Promise<MigrationRecord[]>; insertApplied(table, record): Promise<void>; deleteApplied(table, id): Promise<void>; execute(content, signal?): Promise<void> }`. Each dialect (`surrealql`/`postgres`/`sqlite`) ships its own `ensureTable` DDL — no hard-coded SurrealQL in the runner. `MigrationRunner` consumes the adapter, not a generic `DatabaseLike`.
+
+#### Scenario: surrealql adapter emits SurrealQL DDL on ensureTable
+
+- **WHEN** `createSurrealqlAdapter(client).ensureTable("_migrations")` is called
+- **THEN** the adapter executes `DEFINE TABLE IF NOT EXISTS _migrations SCHEMAFULL; DEFINE FIELD ...`
+
+#### Scenario: postgres adapter emits standard SQL DDL on ensureTable
+
+- **WHEN** `createPostgresAdapter(pgClient).ensureTable("_migrations")` is called
+- **THEN** the adapter executes `CREATE TABLE IF NOT EXISTS _migrations (id text primary key, name text unique not null, applied_at timestamptz not null default now(), checksum text not null, duration_ms int not null)`
+
+**Verified by:** `src/adapters/surrealql/surrealql.test.ts::"ensureTable emits SurrealQL DDL"` AND `src/adapters/postgres/postgres.test.ts::"ensureTable emits standard SQL DDL"`
+
+---
+
+### Requirement: Runner applies pending migrations in numerical order
+
+`MigrationRunner.scanDirectory(dir)` SHALL return files matching `fileExtension`, parsing leading digits from filenames (`001_initial.sql` → order `1`) and sorting ascending. `run(files)` SHALL filter out already-applied (by name match against `selectApplied`), apply each pending migration via `adapter.execute(content)`, record via `adapter.insertApplied(table, {id, name, appliedAt, checksum, duration})`, and halt on first failure with `MigrationError(MIGRATION_APPLY_FAILED)`.
+
+#### Scenario: pending applied in order
+
+- **WHEN** `_migrations` contains record `001_a` and directory contains `002_b.sql`, `003_c.sql`
+- **THEN** `runFromDirectory(dir)` applies `002_b` then `003_c`
+
+#### Scenario: failure halts the apply loop
+
+- **WHEN** `003_c` throws and there are 4+ pending migrations
+- **THEN** the runner returns `[result_for_002_b, result_for_003_c_with_success: false]` and does NOT apply `004_d`
+
+**Verified by:** `src/runner/runner.test.ts::"applies pending in numerical order"` AND `src/runner/runner.test.ts::"halts on first failure"`
+
+---
+
+### Requirement: Checksums are actually validated
+
+When `validateChecksums: true` (default), BEFORE applying any pending migration, the runner SHALL re-read every already-applied migration's file content, compute SHA-256, and compare against the stored checksum. Mismatch SHALL raise `MigrationError(MIGRATION_CHECKSUM_DRIFT)` with `context.{name, expected, actual}` — apply halts.
+
+#### Scenario: checksum drift halts apply
+
+- **WHEN** `001_initial.sql` was applied with checksum `abc`, the file is modified post-apply, and `runFromDirectory(dir)` is called
+- **THEN** the call rejects with `MigrationError(MIGRATION_CHECKSUM_DRIFT)` BEFORE attempting any pending migration
+
+#### Scenario: validateChecksums: false skips drift check
+
+- **WHEN** the same scenario runs with `validateChecksums: false`
+- **THEN** apply proceeds without drift detection
+
+**Verified by:** `src/runner/runner.test.ts::"checksum drift raises MIGRATION_CHECKSUM_DRIFT"` AND `src/runner/runner.test.ts::"validateChecksums false skips drift check"`
+
+---
+
+### Requirement: AbortSignal cancels between migrations
+
+`runFromDirectory(dir, { signal })` SHALL check `signal.aborted` before each pending migration. If aborted, the in-progress migration completes (per the adapter's `execute` semantics) and the runner returns the partial results array. No further migrations are applied. The runner does NOT inject the signal into the adapter — that's per-adapter discretion.
+
+#### Scenario: pre-aborted signal returns empty
+
+- **WHEN** `runFromDirectory(dir, { signal: AbortSignal.abort() })` is awaited
+- **THEN** the result array is empty (no migration applied)
+
+#### Scenario: abort mid-loop returns partial result
+
+- **WHEN** abort fires after 002 succeeds but before 003 starts
+- **THEN** the result array contains only the 002 entry
+
+**Verified by:** `src/runner/runner.test.ts::"pre-aborted signal applies nothing"` AND `src/runner/runner.test.ts::"abort mid-loop returns partial result"`
+
+---
+
+### Requirement: Dry-run logs but does not apply
+
+`dryRun: true` SHALL cause `runMigration(file)` to log `[DRY RUN] Would apply: ${name}` and return `{name, success: true, duration: <small>}` without invoking `adapter.execute` or `adapter.insertApplied`.
+
+#### Scenario: dry-run does not call adapter
+
+- **WHEN** `runFromDirectory(dir, { dryRun: true })` is invoked and 3 pending migrations exist
+- **THEN** `adapter.execute` is not called, `adapter.insertApplied` is not called, and the result array has 3 entries with `success: true`
+
+**Verified by:** `src/runner/runner.test.ts::"dryRun does not call adapter.execute or insertApplied"`
+
+---
+
+### Requirement: OpenTelemetry counter + histogram per apply
+
+The package SHALL register counter `migrations.applied_total` (attributes `dialect`, `outcome`) and histogram `migrations.apply_duration_ms` (attribute `name`). Both SHALL emit per migration regardless of success/failure.
+
+#### Scenario: counter increments on success
+
+- **WHEN** a successful apply runs against a real in-memory `MeterProvider`
+- **THEN** `migrations.applied_total{dialect, outcome: "success"}` has 1 observation
+
+#### Scenario: counter increments on failure
+
+- **WHEN** an apply fails
+- **THEN** `migrations.applied_total{dialect, outcome: "error"}` has 1 observation
+
+**Verified by:** `src/otel/otel.test.ts::"applied_total emits success"` AND `src/otel/otel.test.ts::"applied_total emits error"` AND `src/otel/otel.test.ts::"apply_duration_ms records per migration"`
+
+---
+
+### Requirement: Rollback removes the last tracking record only
+
+`rollback()` SHALL fetch the most recently applied migration record (sort by `appliedAt` desc, take 1), call `adapter.deleteApplied(table, lastRecord.id)`, and return `{name, success: true, duration: 0}`. It does NOT attempt to revert the schema — that's application policy. If no records exist, returns `null`.
+
+#### Scenario: rollback removes last record
+
+- **WHEN** records `001`, `002`, `003` are applied and `rollback()` is awaited
+- **THEN** `adapter.deleteApplied` is called with id of `003` only
+
+#### Scenario: no records → returns null
+
+- **WHEN** the migrations table is empty and `rollback()` is awaited
+- **THEN** the result is `null`
+
+**Verified by:** `src/runner/runner.test.ts::"rollback removes last record"` AND `src/runner/runner.test.ts::"rollback empty returns null"`
+
+---
+
+### Requirement: Every MIGRATION_* code is thrown from a production path
+
+`MIGRATION_CODES` SHALL define exactly 6 codes (`MIGRATION_TABLE_BOOTSTRAP_FAILED`, `MIGRATION_APPLY_FAILED`, `MIGRATION_CHECKSUM_DRIFT`, `MIGRATION_DIRECTORY_NOT_FOUND`, `MIGRATION_INVALID_FILE_NAME`, `MIGRATION_ROLLBACK_NOT_SUPPORTED`). Every code SHALL be raised by at least one production call site.
+
+**Verified by:** `src/api-migrations.spec-conformance.test.ts::"every MIGRATION_* code is thrown from a production path"`
+
+---
+
+### Requirement: Server-tier package shape with per-dialect subpaths
+
+The package SHALL run in Bun + Node. `package.json` SHALL declare subpath exports for the runner (`./runner/runner`), every shipped adapter (`./adapters/surrealql/surrealql`, `./adapters/postgres/postgres`, `./adapters/sqlite/sqlite`), the OTel layer, and the error codes. Every implementation `.ts` SHALL have a sibling `.md` ≥200 words. No `export *` in feature barrels. 100/100/100/100 coverage.
+
+#### Scenario: each dialect adapter is independently importable
+
+- **WHEN** a consumer imports `@suaveplan/api-migrations/adapters/postgres/postgres`
+- **THEN** the import succeeds without dragging in the SurrealQL adapter
+
+**Verified by:** `tests/dep-graph.test.ts::"each adapter has its own subpath"` AND `tests/dep-graph.test.ts::"subpath exports cover every feature module"` AND `tests/dep-graph.test.ts::"every impl .ts has a sibling .md"`
+
+<!-- markdownlint-enable MD024 -->

openspec/archive/2026-05-18-add-api-migrations/tasks.md

@@ -0,0 +1,50 @@
+# Tasks — add-api-migrations
+
+## 1. Scaffold
+
+- [ ] 1.1 Create `packages/api/api-migrations/` with `src/{runner,adapters/{surrealql,postgres,sqlite},otel,error-codes,types}/` + `tests/setup/` + `tests/integration/`
+- [ ] 1.2 `package.json` v0.1.0. Deps: `@suaveplan/error` `@suaveplan/types` `tslib`, `@opentelemetry/api`. PeerDeps (optional, dialect-specific): `surrealdb` `postgres` `@sqlite.org/sqlite-wasm` — each listed as `peerDependenciesMeta.<x>.optional: true`. Dev: standard + the actual clients for integration tests.
+- [ ] 1.3 Subpath exports: `.`, `./runner/runner`, `./adapters/surrealql/surrealql`, `./adapters/postgres/postgres`, `./adapters/sqlite/sqlite`, `./otel/instruments`, `./error-codes/error-codes`, `./types/migration`
+- [ ] 1.4 tsconfigs + vite + vitest (100/100/100/100, testTimeout 60000 for integration)
+- [ ] 1.5 `tests/setup/polyfills.ts`
+
+## 2. Implementation
+
+- [ ] 2.1 `src/error-codes.ts` + `.md` + test — 6 MIGRATION.* codes + `MigrationError`
+- [ ] 2.2 `src/types/migration.{ts,md,index.ts}` — `MigrationFile`, `MigrationRecord`, `MigrationResult`, `MigrationAdapter`, `MigrationDialect`, `MigrationRunnerOptions`
+- [ ] 2.3 `src/otel/instruments.{ts,test.ts,md,index.ts}` — counter + histogram
+- [ ] 2.4 `src/adapters/surrealql/surrealql.{ts,test.ts,md,index.ts}` — `createSurrealqlAdapter(client)`, SurrealQL DDL
+- [ ] 2.5 `src/adapters/postgres/postgres.{ts,test.ts,md,index.ts}` — `createPostgresAdapter(client)`, standard SQL DDL
+- [ ] 2.6 `src/adapters/sqlite/sqlite.{ts,test.ts,md,index.ts}` — `createSqliteAdapter(db)`, SQLite DDL
+- [ ] 2.7 `src/runner/runner.{ts,test.ts,md,index.ts}` — `MigrationRunner` class; checksum validation; dry-run; AbortSignal; rollback
+- [ ] 2.8 `src/index.ts` — named re-exports
+
+## 3. Tests
+
+- [ ] 3.1 Co-located `*.test.ts` per spec scenario (use stub `MigrationAdapter` for unit tests)
+- [ ] 3.2 `src/api-migrations.spec-conformance.test.ts` — every MIGRATION.* thrown
+- [ ] 3.3 `tests/integration/surrealql-roundtrip.test.ts` — real `@suaveplan/testing-integration-surrealdb` testcontainer; apply 3 migrations, query the tracking table, assert records + checksums
+- [ ] 3.4 `tests/integration/postgres-roundtrip.test.ts` — real `@suaveplan/testing-integration-postgres` testcontainer
+- [ ] 3.5 `tests/integration/sqlite-roundtrip.test.ts` — in-memory `@sqlite.org/sqlite-wasm`
+- [ ] 3.6 `tests/property.test.ts` — fast-check checksum stability: identical content produces identical SHA-256
+- [ ] 3.7 `tests/dep-graph.test.ts` — every adapter has its own subpath; sibling .md present
+- [ ] 3.8 Runner imports + timeouts
+
+## 4. Documentation
+
+- [ ] 4.1 Sibling `.md` per impl (≥200 words)
+- [ ] 4.2 `README.md` — quick-start for each dialect, AbortSignal pattern, checksum-drift recovery procedure
+
+## 5. Gates
+
+- [ ] 5.1 `bunx turbo lint typecheck test --filter=@suaveplan/api-migrations --force`
+- [ ] 5.2 Coverage 100/100/100/100
+- [ ] 5.3 `bun scripts/openspec-conformance.ts @suaveplan/api-migrations` exit 0
+- [ ] 5.4 Build green
+- [ ] 5.5 Banned patterns + no `export *` + no direct runner imports
+
+## 6. Archive
+
+- [ ] 6.1 Fold spec → `openspec/specs/api-migrations/spec.md`
+- [ ] 6.2 mv → archive
+- [ ] 6.3 MANIFEST + STATUS refresh