342 lines
18 KiB
Markdown
342 lines
18 KiB
Markdown
# Tasks: enterprise-db-sqlite
|
||
|
||
Status key: ✅ done | 🔄 in progress | ⬜ pending
|
||
|
||
---
|
||
|
||
## Pre-flight: dependency gate
|
||
|
||
- ⬜ P.1 Confirm `@suaveplan/db` is published on genesis (provides `DatabaseAdapter`,
|
||
`DatabaseConfig`, `MigrationConfig`, `MigrationResult`, `DatabaseError`,
|
||
`getSingleton`, `closeSingleton`, `closeAllSingletons`, `hasSingleton`).
|
||
Do not start Wave 1 until this is green.
|
||
- ⬜ P.2 Add `catalog:database` block to root `package.json`:
|
||
```jsonc
|
||
"database": {
|
||
"drizzle-orm": "<latest-exact>",
|
||
"@sqlite.org/sqlite-wasm": "<latest-exact>"
|
||
}
|
||
```
|
||
Verify no version floats; use exact versions in Bun `overrides` if needed.
|
||
|
||
---
|
||
|
||
## Wave 1 — Scaffold
|
||
|
||
- ⬜ 1.1 Create directory `universal/database/db-sqlite/`
|
||
- ⬜ 1.2 Create `package.json` with:
|
||
- `name: "@suaveplan/db-sqlite"`, `version: "0.1.0"`, `sideEffects: false`, `type: "module"`
|
||
- `dependencies`: `drizzle-orm: "catalog:database"`, `@sqlite.org/sqlite-wasm: "catalog:database"`,
|
||
`@suaveplan/db: "workspace:^"`, `@suaveplan/error: "workspace:^"`,
|
||
`@suaveplan/types: "workspace:^"`, `tslib: "catalog:build"`
|
||
- `devDependencies`: `@suaveplan/logger: "workspace:^"`, `@suaveplan/retry: "workspace:^"`,
|
||
`@types/bun: "catalog:"`, `@types/node: "catalog:"`, `vite: "catalog:build"`,
|
||
`vite-plugin-dts: "catalog:build"`, `vitest: "catalog:testing"`,
|
||
`@vitest/coverage-istanbul: "catalog:testing"`, `fast-check: "catalog:testing"`,
|
||
`mitata: "catalog:testing"`
|
||
- Conditional `exports` map:
|
||
```jsonc
|
||
{
|
||
"./package.json": "./package.json",
|
||
".": {
|
||
"import": { "types": "./dist/index.d.ts", "default": "./dist/index.js" }
|
||
},
|
||
"./adapter": {
|
||
"bun": { "types": "./dist/adapter.bun.d.ts", "default": "./dist/adapter.bun.js" },
|
||
"browser": { "types": "./dist/adapter.wasm.d.ts", "default": "./dist/adapter.wasm.js" },
|
||
"default": { "types": "./dist/adapter.bun.d.ts", "default": "./dist/adapter.bun.js" }
|
||
},
|
||
"./unsafe": {
|
||
"import": { "types": "./dist/unsafe/index.d.ts", "default": "./dist/unsafe/index.js" }
|
||
},
|
||
"./errors/errors": {
|
||
"import": { "types": "./dist/errors/errors.d.ts", "default": "./dist/errors/errors.js" }
|
||
},
|
||
"./cache/cache": {
|
||
"import": { "types": "./dist/cache/cache.d.ts", "default": "./dist/cache/cache.js" }
|
||
},
|
||
"./migration/migration": {
|
||
"import": { "types": "./dist/migration/migration.d.ts", "default": "./dist/migration/migration.js" }
|
||
},
|
||
"./drivers/bun-driver": {
|
||
"import": { "types": "./dist/drivers/bun-driver.d.ts", "default": "./dist/drivers/bun-driver.js" }
|
||
},
|
||
"./drivers/wasm-driver": {
|
||
"import": { "types": "./dist/drivers/wasm-driver.d.ts", "default": "./dist/drivers/wasm-driver.js" }
|
||
},
|
||
"./connection/connection": {
|
||
"import": { "types": "./dist/connection/connection.d.ts", "default": "./dist/connection/connection.js" }
|
||
}
|
||
}
|
||
```
|
||
- `scripts`: `build`, `lint`, `typecheck`, `test`, `test:coverage`, `docs:check`
|
||
- ⬜ 1.3 Create `tsconfig.json` (extends `@suaveplan/typescript-config/library.json`)
|
||
- ⬜ 1.4 Create `tsconfig.lib.json` (strict, composite, declaration emit, excludes tests)
|
||
- ⬜ 1.5 Create `vite.config.ts` — dual entry (`index.ts` + `unsafe/index.ts`), `lib` mode,
|
||
`external: ['bun:sqlite', '@sqlite.org/sqlite-wasm', 'drizzle-orm', ...]`
|
||
- ⬜ 1.6 Create `vitest.config.ts` — Node pool config via `createTestConfig()`:
|
||
- Primary config: `pool: 'forks'`, `setupFiles: ['@suaveplan/testing/polyfills']`
|
||
- Browser/WASM sub-config: `fileParallelism: false`, `environment: 'happy-dom'`
|
||
- ⬜ 1.7 Create `biome.json` inheriting root config
|
||
- ⬜ 1.8 Create `src/index.ts` barrel (empty, filled incrementally per wave)
|
||
|
||
---
|
||
|
||
## Wave 2 — Types and errors
|
||
|
||
- ⬜ 2.1 Create `src/errors/errors.ts`:
|
||
- `SQLITE_CODES` registry via `@suaveplan/error#defineCodes`:
|
||
`SQLITE.NOT_CONNECTED`, `SQLITE.QUERY_FAILED`, `SQLITE.EXECUTE_FAILED`,
|
||
`SQLITE.TRANSACTION_FAILED`, `SQLITE.MIGRATION_FAILED`, `SQLITE.ENVIRONMENT_ERROR`,
|
||
`SQLITE.BUSY`, `SQLITE.CONSTRAINT`, `SQLITE.CORRUPT`, `SQLITE.DRIVER_INIT_FAILED`
|
||
- `SqliteError extends SuaveplanError` with `code: keyof typeof SQLITE_CODES`
|
||
- Helper `mapSqliteCode(err: unknown): keyof typeof SQLITE_CODES` mapping raw
|
||
SQLite result-code strings/numbers to stable `SQLITE.*` codes
|
||
- ⬜ 2.2 Create `src/errors/errors.types.ts` — exported type aliases
|
||
- ⬜ 2.3 Create `src/errors/index.ts` barrel
|
||
- ⬜ 2.4 Create `src/errors/errors.md` co-located docs (≥200 words, two examples)
|
||
- ⬜ 2.5 Create `src/errors/errors.test.ts` — full coverage of error construction,
|
||
code mapping, cause-chain, Zod wire-format serialisation
|
||
- ⬜ 2.6 Create `src/schemas/schemas.ts` — Zod schemas:
|
||
- `BunDriverOptionsSchema`, `WasmDriverOptionsSchema`, `SqliteAdapterOptionsSchema`,
|
||
`MigrationRunnerOptionsSchema`, `DatabaseConfigSchema`
|
||
- ⬜ 2.7 Create `src/schemas/schemas.md`, `src/schemas/index.ts`, `src/schemas/schemas.test.ts`
|
||
|
||
---
|
||
|
||
## Wave 3 — Bun driver
|
||
|
||
- ⬜ 3.1 Create `src/drivers/bun-driver.ts`:
|
||
- `BunDriverOptions` interface: `filename?: string`, `readonly?: boolean`
|
||
- `BunDriverResult` interface: `sqlite: Database`, `close: () => Promise<void>`
|
||
- `isBunEnvironment(): boolean`
|
||
- `createBunDriver(options?: BunDriverOptions): Promise<BunDriverResult>`
|
||
— dynamic import of `bun:sqlite`; `Database(":memory:")` or `new Database(filename, { create: true, readonly })`
|
||
- ⬜ 3.2 Create `src/drivers/bun-driver.md` (purpose, OPFS note, examples)
|
||
- ⬜ 3.3 Create `src/drivers/index.ts` barrel (re-exports bun + wasm drivers)
|
||
- ⬜ 3.4 Create `src/drivers/bun-driver.test.ts` — full coverage:
|
||
- In-memory creation
|
||
- File-backed creation and close
|
||
- `isBunEnvironment()` returns `true` under Bun test runner
|
||
- `close()` is idempotent
|
||
|
||
---
|
||
|
||
## Wave 4 — WASM driver
|
||
|
||
- ⬜ 4.1 Create `src/drivers/wasm-worker.ts` (Web Worker entry):
|
||
- Handles `init` message: loads `@sqlite.org/sqlite-wasm`, installs OPFS SAH-pool VFS
|
||
if `sqlite3.oo1.OpfsDb` is available, falls back to in-memory `":memory:"` with
|
||
`console.warn` when OPFS headers are absent
|
||
- Handles `query` message: `exec` via OO1 API, returns `resultRows` in `array` mode
|
||
- Handles `close` message: `db.close()`, posts `{ type: 'closed' }`
|
||
- Handles `begin_tx` / `commit_tx` / `rollback_tx` messages for batched transactions
|
||
- ⬜ 4.2 Create `src/drivers/wasm-driver.ts`:
|
||
- `WasmDriverOptions`: `{ filename?: string }`
|
||
- `WasmDriverResult<TSchema>`: `{ db: SqliteRemoteDatabase<TSchema>; close: () => Promise<void> }`
|
||
- `createWasmDriver<TSchema>(options?, schema?): Promise<WasmDriverResult<TSchema>>`
|
||
— spawns `wasm-worker.ts` via `new Worker(new URL('./wasm-worker.js', import.meta.url), { type: 'module' })`
|
||
— awaits `{ type: 'ready' }` init handshake with reject-on-error + worker termination guard
|
||
— pending-promise map keyed by sequential `id`, resolved/rejected by worker response
|
||
— `close()` drains pending map with rejection, sends `close` message, awaits `closed`, then `worker.terminate()`
|
||
— wraps Drizzle `sqlite-proxy` driver
|
||
- ⬜ 4.3 Create `src/drivers/wasm-driver.md` — OPFS prerequisites (COOP/COEP headers),
|
||
bundler worker-URL support, fallback behaviour, close semantics
|
||
- ⬜ 4.4 Create `src/drivers/wasm-driver.test.ts` (vitest `environment: 'happy-dom'`,
|
||
`fileParallelism: false`):
|
||
- Worker init success path (mock Worker via DI or in-process worker shim)
|
||
- Worker init failure propagation + worker termination
|
||
- Query round-trip
|
||
- `close()` drains pending and terminates worker
|
||
- Note: full OPFS integration test runs only in a real browser / Playwright — unit
|
||
coverage uses an injectable worker factory to avoid WSL2 OPFS limitations
|
||
|
||
---
|
||
|
||
## Wave 5 — Prepared-statement cache
|
||
|
||
- ⬜ 5.1 Create `src/cache/cache.ts`:
|
||
- `PreparedStatementCache` — LRU eviction, configurable `maxSize` (default 128)
|
||
- `get(sql: string): Statement | undefined`
|
||
- `set(sql: string, stmt: Statement): void`
|
||
- `invalidate(): void` — called by `SqliteAdapter.migrate()`; closes all held statements
|
||
- `get size(): number`
|
||
- OTel counters: `sqlite.cache.hits`, `sqlite.cache.misses`, `sqlite.cache.evictions`
|
||
via `@opentelemetry/api` — no-op when no provider registered
|
||
- ⬜ 5.2 Create `src/cache/cache.types.ts`, `src/cache/index.ts`, `src/cache/cache.md`
|
||
- ⬜ 5.3 Create `src/cache/cache.test.ts` — LRU eviction order, invalidate clears,
|
||
OTel counter increments, property-based test (fast-check) for cache-size invariant
|
||
|
||
---
|
||
|
||
## Wave 6 — Migration runner
|
||
|
||
- ⬜ 6.1 Create `src/migration/migration.ts`:
|
||
- `MigrationRunnerOptions`: `{ migrationsFolder?: string; migrationsTable?: string; dryRun?: boolean; clock?: Clock }`
|
||
- `MigrationRunner` class:
|
||
- Constructor takes a `BunSQLiteDatabase` and `MigrationRunnerOptions`
|
||
- `run(): Promise<MigrationResult>` — reads migration files from folder, checks ledger
|
||
table for applied entries, applies in filename order (forward-only, idempotent),
|
||
writes ledger row per applied migration, returns `{ applied: number; migrations: string[] }`
|
||
- `dryRun(): Promise<MigrationResult>` — same logic without writing to the DB
|
||
- Ledger table: `__sqlite_migrations (id TEXT PRIMARY KEY, applied_at INTEGER NOT NULL)`
|
||
- Emits OTel span (`sqlite.migration.run`) with `migration.file` attribute per migration
|
||
- Throws `SqliteError(SQLITE.MIGRATION_FAILED)` on DDL error with original cause
|
||
- `MigrationRunnerOptionsSchema` (Zod) validates options at construction
|
||
- ⬜ 6.2 Create `src/migration/migration.types.ts`, `src/migration/index.ts`,
|
||
`src/migration/migration.md`
|
||
- ⬜ 6.3 Create `src/migration/migration.test.ts` — ledger idempotency (run twice = same
|
||
result), dry-run does not alter DB, missing folder error, partial-failure rollback,
|
||
injectable clock for `applied_at` assertions
|
||
|
||
---
|
||
|
||
## Wave 7 — OTel instrumentation
|
||
|
||
- ⬜ 7.1 Create `src/otel/otel.ts`:
|
||
- `withQuerySpan<T>(operationName: string, sanitisedSql: string, fn: () => Promise<T>): Promise<T>`
|
||
— creates child span under active context, sets `db.system=sqlite`, `db.statement`,
|
||
records exception + sets ERROR status on failure, no-op when no OTel provider
|
||
- `sanitizeSql(sql: string): string` — strips string literals and numeric parameters
|
||
to prevent PII leakage in span attributes
|
||
- Histograms: `sqlite.query.duration_ms` (labels: operation, result)
|
||
- Counters: `sqlite.query.count`, `sqlite.error.count`
|
||
- ⬜ 7.2 Create `src/otel/otel.md`, `src/otel/index.ts`
|
||
- ⬜ 7.3 Create `src/otel/otel.test.ts` — verify span creation, sanitizeSql strips
|
||
literals, no-op when no provider, histogram updated
|
||
|
||
---
|
||
|
||
## Wave 8 — SqliteAdapter
|
||
|
||
- ⬜ 8.1 Create `src/adapter/adapter.ts`:
|
||
- `SqliteAdapterOptions = BunDriverOptions`
|
||
- `SqliteAdapter<TSchema extends Record<string, unknown> = Record<string, never>>`
|
||
implements `DatabaseAdapter` from `@suaveplan/db`
|
||
- Constructor: validates options via `SqliteAdapterOptionsSchema`, throws
|
||
`SqliteError(SQLITE.ENVIRONMENT_ERROR)` if `!isBunEnvironment()`
|
||
- `protected _drizzle: BunSQLiteDatabase<TSchema> | null`
|
||
- `protected _sqlite: import('bun:sqlite').Database | null`
|
||
- `protected _connected: boolean`
|
||
- `protected _connectingPromise: Promise<void> | null` (idempotent concurrent connect)
|
||
- `protected _cache: PreparedStatementCache`
|
||
- `connect(config?: Partial<DatabaseConfig>): Promise<void>` — uses cache, sets
|
||
`_drizzle` via `drizzle(sqlite, { schema })`
|
||
- `disconnect(): Promise<void>` — closes sqlite, nulls state, calls `_cache.invalidate()`
|
||
- `query<T>(sql, params?, signal?: AbortSignal): Promise<T[]>` — wraps
|
||
`_cache.get/set` + `withQuerySpan`; checks `signal.aborted` before execution;
|
||
maps caught errors to `SqliteError` via `mapSqliteCode`
|
||
- `execute(sql, params?, signal?: AbortSignal): Promise<void>` — same pattern
|
||
- `transaction<T>(fn: () => Promise<T>): Promise<T>` — `BEGIN IMMEDIATE` / `COMMIT` /
|
||
`ROLLBACK`; wraps in `withQuerySpan('transaction', 'BEGIN IMMEDIATE')`
|
||
- `migrate(config?: MigrationConfig): Promise<MigrationResult>` — delegates to
|
||
`MigrationRunner`, calls `_cache.invalidate()` on success
|
||
- `get isConnected: boolean`
|
||
- `get drizzle: BunSQLiteDatabase<TSchema>` — asserts connected, returns `_drizzle!`
|
||
- `protected assertConnected(): void`
|
||
- ⬜ 8.2 Create `src/adapter/adapter.types.ts` (re-exports `BunSQLiteDatabase` type alias)
|
||
- ⬜ 8.3 Create `src/adapter/index.ts`, `src/adapter/adapter.md`
|
||
- ⬜ 8.4 Create `src/adapter/adapter.test.ts` — connect/disconnect lifecycle,
|
||
double-connect idempotency, drizzle getter throws before connect,
|
||
transaction commit and rollback, migration invalidates cache,
|
||
AbortSignal cancellation, error code mapping, property-based test for
|
||
concurrent-connect race (fast-check `schedulerFor`)
|
||
|
||
---
|
||
|
||
## Wave 9 — Connection singleton helpers
|
||
|
||
- ⬜ 9.1 Create `src/connection/connection.ts`:
|
||
- `getDefaultSqlite(options?: SqliteAdapterOptions, config?: Partial<DatabaseConfig>, name?: string): Promise<SqliteAdapter>`
|
||
— thin wrapper around `@suaveplan/db#getSingleton`
|
||
- Re-export `closeSingleton as closeSqlite`, `closeAllSingletons as closeAllSqlite`,
|
||
`hasSingleton as hasSqlite` from `@suaveplan/db`
|
||
- ⬜ 9.2 Create `src/connection/connection.md`, `src/connection/index.ts`
|
||
- ⬜ 9.3 Create `src/connection/connection.test.ts` — get creates adapter, second get
|
||
returns same instance, close removes it, `hasSqlite` reflects state
|
||
|
||
---
|
||
|
||
## Wave 10 — Unsafe subpath
|
||
|
||
- ⬜ 10.1 Create `src/unsafe/index.ts`:
|
||
- `rawQuery<T = Record<string, unknown>>(db: BunSQLiteDatabase<any> | SqliteRemoteDatabase<any>, sql: string, params?: unknown[]): Promise<T[]>`
|
||
- JSDoc warning: "Unsafe — bypasses Drizzle ORM; no SQL injection protection. Use only in migration scripts or low-level tooling."
|
||
- ⬜ 10.2 Create `src/unsafe/unsafe.md` — explicit security warning in Purpose section
|
||
- ⬜ 10.3 Create `src/unsafe/unsafe.test.ts` — basic execution, error propagation
|
||
|
||
---
|
||
|
||
## Wave 11 — Barrel and conformance suite
|
||
|
||
- ⬜ 11.1 Wire `src/index.ts` — export all public symbols from waves 2–10:
|
||
`SqliteAdapter`, `SqliteAdapterOptions`, `SqliteError`, `SQLITE_CODES`,
|
||
`createBunDriver`, `BunDriverOptions`, `BunDriverResult`, `isBunEnvironment`,
|
||
`createWasmDriver`, `WasmDriverOptions`, `WasmDriverResult`,
|
||
`getDefaultSqlite`, `closeSqlite`, `closeAllSqlite`, `hasSqlite`,
|
||
Zod schemas, `PreparedStatementCache`, `MigrationRunner`, `MigrationRunnerOptions`
|
||
- ⬜ 11.2 Create `src/conformance/conformance-suite.ts` — driver-agnostic test factory:
|
||
- Accepts a `DriverFactory` callback that returns `{ db: DrizzleClient; close(): Promise<void> }`
|
||
- Test groups: CRUD round-trip, transaction commit, transaction rollback, concurrent
|
||
writes (SQLITE_BUSY retry via `@suaveplan/retry`), schema migration idempotency,
|
||
prepared-statement cache hit ratio, `AbortSignal` cancellation mid-query
|
||
- ⬜ 11.3 Create `src/conformance/bun-conformance.test.ts` — runs suite with Bun driver
|
||
- ⬜ 11.4 Create `src/conformance/wasm-conformance.test.ts` — runs suite with WASM driver
|
||
(`fileParallelism: false`, `environment: 'happy-dom'`)
|
||
- ⬜ 11.5 Create `src/conformance/conformance-suite.md`
|
||
|
||
---
|
||
|
||
## Wave 12 — Property-based tests
|
||
|
||
- ⬜ 12.1 Add `src/adapter/adapter.property.test.ts` — fast-check arbitraries for:
|
||
- SQL parameterisation: verify no raw-string interpolation leaks by generating
|
||
adversarial inputs and confirming Drizzle always uses `?` binding
|
||
- Transaction isolation: interleaved concurrent transactions produce consistent
|
||
row counts (serialisability invariant under Bun's cooperative concurrency)
|
||
- Cache size invariant: `cache.size <= maxSize` after arbitrary insert/evict sequences
|
||
- ⬜ 12.2 Add `src/migration/migration.property.test.ts` — fast-check: arbitrary lists
|
||
of migration files always produce monotonically increasing ledger entries; re-running
|
||
produces identical result (idempotency law)
|
||
|
||
---
|
||
|
||
## Wave 13 — Benchmarks
|
||
|
||
- ⬜ 13.1 Create `src/__benchmarks__/sqlite.bench.ts` (mitata):
|
||
- Single-row insert (prepared vs. ad-hoc)
|
||
- Bulk insert — 1 000 rows in transaction
|
||
- Point lookup by primary key
|
||
- Full-table scan — 1 000 rows
|
||
- Cache-warmed vs. cold prepared statement
|
||
- ⬜ 13.2 Add `bench` script to `package.json`: `bun src/__benchmarks__/sqlite.bench.ts`
|
||
- ⬜ 13.3 Record baseline numbers in README under a `## Performance` heading
|
||
|
||
---
|
||
|
||
## Wave 14 — Documentation
|
||
|
||
- ⬜ 14.1 Write `README.md` at package root — sections: Purpose, Installation, Quick Start
|
||
(Bun server, browser WASM), API Reference (all exports), Conditional Exports map,
|
||
OPFS setup guide (COOP/COEP headers), Migration Runner usage, Unsafe subpath warning,
|
||
Performance numbers, Contributing
|
||
- ⬜ 14.2 Verify every `.ts` implementation file has a sibling `.md` with ≥200 words and
|
||
both a basic and an advanced usage example
|
||
- ⬜ 14.3 Run `bun run docs:check` and fix any missing-doc gaps
|
||
|
||
---
|
||
|
||
## Wave 15 — Pre-publish gate
|
||
|
||
- ⬜ 15.1 Run `bunx turbo lint --filter=@suaveplan/db-sqlite` — fix all Biome findings
|
||
- ⬜ 15.2 Run `bunx turbo typecheck --filter=@suaveplan/db-sqlite` — zero type errors
|
||
- ⬜ 15.3 Run `bunx turbo test --filter=@suaveplan/db-sqlite` — all tests green
|
||
- ⬜ 15.4 Run `bunx turbo test:coverage --filter=@suaveplan/db-sqlite` — Istanbul reports
|
||
100% statement, branch, function, line coverage on all touched modules
|
||
- ⬜ 15.5 Verify `sideEffects: false` tree-shake test passes
|
||
- ⬜ 15.6 Update `CLAUDE.md` package listing entry for `@suaveplan/db-sqlite`
|
||
- ⬜ 15.7 Update root `README.md` table row for `@suaveplan/db-sqlite`
|
||
- ⬜ 15.8 Run `bun scripts/publish.ts --package @suaveplan/db-sqlite`
|
||
- ⬜ 15.9 Move this change folder to
|
||
`openspec/archive/2026-04-23-enterprise-db-sqlite/`
|