SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
admin
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
openspec/archive/2026-05-19-wire-surreal-import-store/proposal.md Normal file
View File

@@ -0,0 +1,71 @@
# Wire SurrealImportStore into @suaveplan/api-plugin-import
**Package**: `@suaveplan/api-plugin-import`
**Status**: PROPOSED
**Type**: modified-requirements (follow-up to `add-api-plugin-import`)
- **Spec delta**: ./specs/api-plugin-import/spec.md
## Why
The initial migration (`openspec/archive/2026-05-19-add-api-plugin-import/`) explicitly scoped out the SurrealDB-backed `ImportStoreAdapter` implementation. The published `### Excluded from this migration` section reads:
> **`SurrealImportStore`** — requires `@suaveplan/surrealdb` migrations + the Surreal-side schema. Genesis ships `MemoryImportStore` against the same `ImportStoreAdapter` interface so the SurrealDB adapter lands as a sibling package without breaking consumers. Listed in pending follow-ups.
Both prerequisites now exist on `genesis`:
- `@suaveplan/surrealdb` ships at `packages/database/surrealdb/` with the `SurrealAdapter` class implementing `DatabaseAdapter`.
- `@suaveplan/api-plugin-database` ships at `packages/api/api-plugin-database/` exposing a pooled `SurrealAdapter` via `ServiceRegistry["db"]`.
- `@suaveplan/api-plugin-events` ships the canonical positional-SurQL adapter pattern (`$p0`, `$p1`, …) at `packages/api/api-plugin-events/src/repositories/`.
This change adds the durable `SurrealImportStore` implementation of the existing `ImportStoreAdapter` interface, ships the matching `.surql` migration co-located with the package, and registers a corresponding spec requirement. The change does NOT rewrite the existing service, routes, parsers, transform pipeline, or `MemoryImportStore` — only adds the SurrealDB store as an alternative backend selectable by the host.
## What changes
- ADDED: `SurrealImportStore implements ImportStoreAdapter` at `src/store/surreal/surreal-import-store.ts` with `create`, `get`, `update`, `delete`, `list` parity. Uses positional SurQL placeholders per the `@suaveplan/api-plugin-events` repository pattern (`$p0`, `$p1`, …).
- ADDED: `SurrealImportStoreConfig` Zod boundary schema (db adapter handle + optional table name + optional logger + optional clock).
- ADDED: `surql/tables/import_job.surql` migration defining the `import_job` table schema (id, owner_id, status, total_rows, processed_rows, success_count, error_count, errors, metadata, created_at, updated_at, completed_at) with the canonical `OVERWRITE` form used by `api-plugin-events`.
- ADDED: 1 new error code (`STORAGE_ERROR` already exists — reused for adapter failures; no registry growth).
- ADDED: OTel counters `import.store.writes_total{outcome}` and `import.store.reads_total{outcome}` registered on first use via the existing `getImportInstruments()` bundle (extension — keeps the bundle accessor stable).
- ADDED: Subpath export `./store/surreal/surreal-import-store` in `package.json#exports`.
- ADDED: Unit tests with a fake `SurrealAdapter` exercising every SurQL string, parameter ordering, and error path (mirrors `event-repository.test.ts`).
- ADDED: Integration test under `tests/integration/surreal-import-store.test.ts` using a real `SurrealDBFixture` from `@suaveplan/testing-integration-surrealdb`. Applies the `.surql` migration, round-trips a record, exercises filter/limit/offset on `list`, and asserts the `update`-on-missing error message.
- ADDED: `@suaveplan/surrealdb` + `@suaveplan/testing-integration-surrealdb` as deps if not present.
## Out of scope
Not addressed in this change — each remains a separate proposal:
- Streaming `Readable` / `AsyncIterable` input on `startImport` (§4 future enhancement).
- Foreign-key resolution across rows (§4).
- Parallel batch workers (§4).
- S3 / HTTP URL source adapters (§4).
- SSE progress push from the route layer (§4).
- Mitata benchmarks for the SurrealDB store (covered by the monorepo-wide benchmark wave).
- Postgres / SQLite store adapters (separate sibling proposals — `ImportStoreAdapter` is the contract that admits them).
- Any change to the existing `MemoryImportStore`, `ImportService`, `createImportRoutes`, parsers, transform pipeline, progress tracker, plugin descriptor, problem-json renderer, or OTel instrument bundle public surface beyond the additive counter pair noted above.
- Any modification to other `packages/api/api-plugin-*/`, `MANIFEST.md`, `STATUS.md`, `CLAUDE.md`, or the root `package.json` (except an additive `bun.lock` refresh if a workspace dep needs hoisting).
## Source mapping
This change derives from the original archive's `### Excluded from this migration` section
(`openspec/archive/2026-05-19-add-api-plugin-import/proposal.md`) and from the upstream
enterprise audit `../suaveplan/enterprise/api-plugin-import.md` §3 row 5.
| Requirement | Source | Notes |
| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `SurrealImportStore implements ImportStoreAdapter` | `openspec/archive/2026-05-19-add-api-plugin-import/proposal.md` (Excluded) + `../suaveplan/enterprise/api-plugin-import.md` §3 row 5 | Implements the SHIPPED `ImportStoreAdapter` interface — zero contract drift |
| Positional `$p0,$p1,…` SurQL placeholders | `packages/api/api-plugin-events/src/repositories/event-repository/event-repository.ts` L20-L25 | Pattern shared across all genesis SurrealDB repositories |
| Co-located `.surql` migration | `packages/api/api-plugin-events/surql/tables/event.surql` | Same `surql/tables/<table>.surql` layout |
| Zod boundary on store config | `openspec/AGENTS.md` §11 | Named-interface + parse-at-boundary policy |
| OTel counters on write / read | `openspec/AGENTS.md` §13 step 0 | Cached bundle accessor (`getImportInstruments`) extended additively |
| Integration test against real SurrealDB testcontainer | `packages/api/api-plugin-events/tests/integration/full-boot.test.ts` | Same `SurrealDBFixture` lifecycle |
| Update-on-missing rejects with `Job "<id>" not found` | `openspec/specs/api-plugin-import/spec.md` (SHIPPED `MemoryImportStore` requirement) | Contract parity with the in-memory store |
## Impact
### Affected dependents
`bunx turbo ls --filter=...@suaveplan/api-plugin-import` enumerates dependents.
- No dependents currently consume `MemoryImportStore`-specific symbols; the store is selected at host wiring time via the `ImportStoreAdapter` interface. Adding `SurrealImportStore` is purely additive. — [x] no action needed (reason: additive sibling implementation of an existing interface; existing consumers continue to receive `MemoryImportStore` unless they opt in)

100
openspec/archive/2026-05-19-wire-surreal-import-store/specs/api-plugin-import/spec.md Normal file
View File

@@ -0,0 +1,100 @@
<!-- markdownlint-disable MD024 -->
# Capability Spec Delta: api-plugin-import (wire-surreal-import-store)
**Package**: `@suaveplan/api-plugin-import`
**Status**: PROPOSED
> Every requirement below carries a `**Verified by:**` line authored at draft time.
## ADDED Requirements
### Requirement: SurrealImportStore — durable ImportStoreAdapter
The package SHALL ship `SurrealImportStore implements ImportStoreAdapter` at `src/store/surreal/surreal-import-store.ts`, providing the same `create` / `get` / `update` / `delete` / `list` surface as `MemoryImportStore` against a `SurrealAdapter` backend from `@suaveplan/surrealdb`. All SurrealQL placeholders SHALL be positional (`$p0`, `$p1`, …) per the canonical pattern in `@suaveplan/api-plugin-events`. The constructor SHALL accept a `SurrealImportStoreConfig` validated through Zod at the boundary; an invalid config SHALL throw `ImportError` with `code === IMPORT_CODES.INVALID_OPTIONS` and `cause instanceof ZodError`. The store SHALL accept a custom `table` name (default `"import_job"`). `create` SHALL upsert the record by stringified id; `update` on a missing id SHALL throw an error whose message contains `Job "<id>" not found` (parity with `MemoryImportStore`). Adapter-layer failures SHALL be wrapped as `ImportError(IMPORT_CODES.STORAGE_ERROR)` with the underlying error as `cause`. The store SHALL increment `import.store.writes_total{outcome:"success"|"error"}` on every `create` / `update` / `delete`, and `import.store.reads_total{outcome:"success"|"error"}` on every `get` / `list`.
#### Scenario: create emits a CREATE SET statement with positional params
- **WHEN** `surrealStore.create(record)` runs with `record.id = "j1"`
- **THEN** the issued SurQL begins with `CREATE import_job:j1 SET` and the parameter array contains the record's `ownerId`, `status`, numeric counters, and timestamp strings in the order they appear in the `SET` clause
#### Scenario: get returns undefined when no row exists
- **WHEN** `surrealStore.get("missing")` runs against an empty backend
- **THEN** the call resolves to `undefined` and `import.store.reads_total{outcome:"success"}` is incremented (a miss is still a successful read)
#### Scenario: update on missing id throws Job "<id>" not found
- **WHEN** `surrealStore.update("missing", { status: "completed" })` runs on a store with no record `"missing"`
- **THEN** the call rejects with an `Error` whose message contains `Job "missing" not found`
#### Scenario: list filters by status, ownerId, limit, offset
- **WHEN** `surrealStore.list({ status: "completed", ownerId: "alice", limit: 10, offset: 5 })` runs
- **THEN** the SurQL contains `WHERE status = $p0 AND owner_id = $p1`, `LIMIT $p2`, `START $p3` and the parameter array is `["completed", "alice", 10, 5]`
#### Scenario: adapter failure surfaces as STORAGE_ERROR
- **WHEN** the underlying `SurrealAdapter.query` throws
- **THEN** the store's call rejects with an `ImportError` whose `code === IMPORT_CODES.STORAGE_ERROR` and `cause` is the original throwable
#### Scenario: invalid config throws INVALID_OPTIONS
- **WHEN** `new SurrealImportStore({ db: adapter, table: "" })` runs
- **THEN** it throws `ImportError` with `code === IMPORT_CODES.INVALID_OPTIONS` and `cause instanceof ZodError`
**Verified by:** `src/store/surreal/surreal-import-store.test.ts::"create issues a CREATE SET statement with positional params"` AND `src/store/surreal/surreal-import-store.test.ts::"get returns undefined when no row exists"` AND `src/store/surreal/surreal-import-store.test.ts::"update on missing id throws Job not found"` AND `src/store/surreal/surreal-import-store.test.ts::"list filters by status, ownerId, limit, offset"` AND `src/store/surreal/surreal-import-store.test.ts::"adapter failure surfaces as STORAGE_ERROR"` AND `src/store/surreal/surreal-import-store.test.ts::"invalid config throws INVALID_OPTIONS"`
---
### Requirement: SurrealDB schema migration
The package SHALL ship `surql/tables/import_job.surql` defining the `import_job` table SCHEMAFULL with fields `owner_id (string)`, `status (string)` constrained to the `ImportJobStatus` enum, `total_rows (int)`, `processed_rows (int)`, `success_count (int)`, `error_count (int)`, `errors (array)`, `metadata (option<object>)`, `created_at (datetime)`, `updated_at (datetime)`, `completed_at (option<datetime>)`, plus an index on `owner_id`. The migration SHALL use the canonical `DEFINE TABLE OVERWRITE` / `DEFINE FIELD OVERWRITE` form mirroring `packages/api/api-plugin-events/surql/tables/event.surql`. Hosts MAY apply the migration via `@suaveplan/api-plugin-database`'s migration runner or directly via `adapter.query(readFileSync(...))`.
#### Scenario: migration declares the canonical schema
- **WHEN** the integration test reads and applies `surql/tables/import_job.surql`
- **THEN** the SurrealDB instance subsequently accepts a `CREATE import_job:j1 SET owner_id = 'alice', status = 'created', …` statement and rejects a `status = 'bogus'` write
**Verified by:** `tests/integration/surreal-import-store.test.ts::"migration enables canonical writes and rejects invalid status"`
---
### Requirement: Integration round-trip against a real SurrealDB instance
The package SHALL ship `tests/integration/surreal-import-store.test.ts` using `SurrealDBFixture` from `@suaveplan/testing-integration-surrealdb`. The test SHALL apply the `.surql` migration once in `beforeAll`, then `create` a record, `get` it back, `update` it, `list` with a `status` filter, and `delete` it — all against the real container. The integration test SHALL also assert that `update("missing", …)` rejects with the `Job "missing" not found` message and that the `errors` array round-trips intact.
#### Scenario: full round-trip against testcontainer
- **WHEN** the integration test runs against a started `SurrealDBFixture`
- **THEN** each store operation completes successfully and the final `get` after `delete` returns `undefined`
**Verified by:** `tests/integration/surreal-import-store.test.ts::"round-trips a record through the real SurrealDB instance"` AND `tests/integration/surreal-import-store.test.ts::"update on missing id surfaces the canonical message against a real instance"`
---
### Requirement: OTel store counters
The package SHALL extend `getImportInstruments()` to expose two additional counters — `import.store.writes_total` and `import.store.reads_total` — both labelled by `outcome`. The counter pair SHALL be cached on the same bundle as the existing four instruments and SHALL rebind through `resetImportInstruments()`. The existing four-instrument scenarios SHALL continue to pass (additive extension, no breaking shape change). `SurrealImportStore` SHALL emit on every adapter call; `MemoryImportStore` is out of scope for this requirement (it is an in-process test/dev adapter without observable backend health).
#### Scenario: bundle exposes the two new counters
- **WHEN** a caller invokes `getImportInstruments()`
- **THEN** the returned object exposes `storeWritesTotal` and `storeReadsTotal` Counter instruments in addition to the existing four
**Verified by:** `src/otel/instruments.test.ts::"bundle exposes the store write and read counters"`
---
### Requirement: Subpath export for the SurrealDB store
The package SHALL declare the subpath export `./store/surreal/surreal-import-store` in `package.json#exports` so consumers can tree-shake the SurrealDB store without pulling in the in-memory implementation. The `src/store/surreal/index.ts` barrel SHALL use explicit named re-exports (no `export *`).
#### Scenario: subpath export exists and the barrel uses named exports
- **WHEN** the package's `exports` map is enumerated and the new barrel is read
- **THEN** the map contains `./store/surreal/surreal-import-store` and the barrel contains no `export *` lines
**Verified by:** `src/api-plugin-import.spec-conformance.test.ts::"package exposes a subpath export for the SurrealDB store"`
<!-- markdownlint-enable MD024 -->

69
openspec/archive/2026-05-19-wire-surreal-import-store/tasks.md Normal file
View File

@@ -0,0 +1,69 @@
# Tasks — wire-surreal-import-store
> No retroactive ticks. Each item is checked only after its work is actually complete.
## 1. Author the spec delta
- [ ] 1.1 Draft `specs/api-plugin-import/spec.md` with `## ADDED Requirements` for `SurrealImportStore` + `.surql` migration + extended OTel counter pair.
- [ ] 1.2 Every `### Requirement:` carries `**Verified by:**` lines pointing at planned tests.
## 2. Implement SurrealImportStore
- [ ] 2.1 Create `packages/api/api-plugin-import/src/store/surreal/surreal-import-store.ts` implementing `ImportStoreAdapter` with positional SurQL placeholders.
- [ ] 2.2 Create `packages/api/api-plugin-import/src/store/surreal/surreal-import-store.types.ts` with `SurrealImportStoreConfig` (Zod-validated) + helper types.
- [ ] 2.3 Create `packages/api/api-plugin-import/src/store/surreal/surreal-import-store.test.ts` — fake-adapter unit tests covering every SurQL emission, parameter ordering, error normalisation, OTel counter increments, and the `Job "<id>" not found` reject.
- [ ] 2.4 Create `packages/api/api-plugin-import/src/store/surreal/surreal-import-store.md` (≥200 words, all six required sections).
- [ ] 2.5 Create `packages/api/api-plugin-import/src/store/surreal/index.ts` with explicit named re-exports (no `export *`).
## 3. Extend the OTel bundle
- [ ] 3.1 Add `import.store.writes_total{outcome}` and `import.store.reads_total{outcome}` counters to the existing `getImportInstruments()` bundle.
- [ ] 3.2 Update `src/otel/instruments.md` to document the two new counters.
- [ ] 3.3 Update `src/otel/instruments.test.ts` to assert both new counters appear in the cached bundle.
## 4. Ship the SurrealDB migration
- [ ] 4.1 Create `packages/api/api-plugin-import/surql/tables/import_job.surql` with the canonical `OVERWRITE` schema for the `import_job` table.
- [ ] 4.2 Document the path in the package README under a new "SurrealDB store" section.
## 5. Integration test against real SurrealDB
- [ ] 5.1 Create `packages/api/api-plugin-import/tests/integration/surreal-import-store.test.ts` using `SurrealDBFixture` — applies `.surql`, round-trips a record, exercises filter+limit+offset on `list`, asserts the `update`-on-missing reject.
- [ ] 5.2 Add `vitest.integration.config.ts` mirroring `api-plugin-events`'s config (hookTimeout 180_000, testTimeout 60_000).
## 6. Package wiring
- [ ] 6.1 Add `./store/surreal/surreal-import-store` to `package.json#exports`.
- [ ] 6.2 Add `@suaveplan/surrealdb` + `@suaveplan/testing-integration-surrealdb` to dependencies / devDependencies (workspace:^).
- [ ] 6.3 Add the public re-exports (`SurrealImportStore`, `SurrealImportStoreConfig`) to the package root barrel.
## 7. Pre-archive gates (all green — non-negotiable)
- [ ] 7.1 `bunx biome check packages/api/api-plugin-import/src/`
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-plugin-import`
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/api-plugin-import` — 100 / 100 / 100 / 100
- [ ] 7.4 `bun scripts/openspec-conformance.ts @suaveplan/api-plugin-import` — 0 errors
- [ ] 7.5 `bunx turbo build --filter=@suaveplan/api-plugin-import`
## 8. Pre-archive checklist (9 rows)
- [ ] 8.1 Banned-pattern scan zero hits in new files.
- [ ] 8.2 Barrel-shape scan zero hits in `src/store/surreal/index.ts`.
- [ ] 8.3 Test-runner import scan zero hits (`@suaveplan/testing/runner` only).
- [ ] 8.4 Sibling `.md` exists for every new `.ts` (excluding `.test.ts`, `.types.ts`, `index.ts`).
- [ ] 8.5 `README.md` updated with SurrealDB store section.
- [ ] 8.6 No bare version pins in `package.json` (catalog / workspace only).
- [ ] 8.7 Subpath export count matches feature module count.
- [ ] 8.8 No hardcoded secrets in new files.
- [ ] 8.9 Dependent sweep — none impacted (`ImportStoreAdapter` is the only contract; opt-in by host).
## 9. Fold + archive
- [ ] 9.1 Fold `specs/api-plugin-import/spec.md` (this change's delta) into `openspec/specs/api-plugin-import/spec.md`.
- [ ] 9.2 `mv openspec/changes/wire-surreal-import-store openspec/archive/2026-05-19-wire-surreal-import-store`.
## 10. Commit
- [ ] 10.1 Pre-push: `git fetch origin genesis && git pull --rebase --autostash origin genesis`.
- [ ] 10.2 One commit: `feat(api-plugin-import): add SurrealImportStore (was excluded in initial migration)`.
- [ ] 10.3 Never `--no-verify`. Never force-push.