72 lines
7.3 KiB
Markdown
72 lines
7.3 KiB
Markdown
# Wire SurrealImportStore into @suaveplan/api-plugin-import
|
|
|
|
**Package**: `@suaveplan/api-plugin-import`
|
|
**Status**: PROPOSED
|
|
**Type**: modified-requirements (follow-up to `add-api-plugin-import`)
|
|
|
|
- **Spec delta**: ./specs/api-plugin-import/spec.md
|
|
|
|
## Why
|
|
|
|
The initial migration (`openspec/archive/2026-05-19-add-api-plugin-import/`) explicitly scoped out the SurrealDB-backed `ImportStoreAdapter` implementation. The published `### Excluded from this migration` section reads:
|
|
|
|
> **`SurrealImportStore`** — requires `@suaveplan/surrealdb` migrations + the Surreal-side schema. Genesis ships `MemoryImportStore` against the same `ImportStoreAdapter` interface so the SurrealDB adapter lands as a sibling package without breaking consumers. Listed in pending follow-ups.
|
|
|
|
Both prerequisites now exist on `genesis`:
|
|
|
|
- `@suaveplan/surrealdb` ships at `packages/database/surrealdb/` with the `SurrealAdapter` class implementing `DatabaseAdapter`.
|
|
- `@suaveplan/api-plugin-database` ships at `packages/api/api-plugin-database/` exposing a pooled `SurrealAdapter` via `ServiceRegistry["db"]`.
|
|
- `@suaveplan/api-plugin-events` ships the canonical positional-SurQL adapter pattern (`$p0`, `$p1`, …) at `packages/api/api-plugin-events/src/repositories/`.
|
|
|
|
This change adds the durable `SurrealImportStore` implementation of the existing `ImportStoreAdapter` interface, ships the matching `.surql` migration co-located with the package, and registers a corresponding spec requirement. The change does NOT rewrite the existing service, routes, parsers, transform pipeline, or `MemoryImportStore` — only adds the SurrealDB store as an alternative backend selectable by the host.
|
|
|
|
## What changes
|
|
|
|
- ADDED: `SurrealImportStore implements ImportStoreAdapter` at `src/store/surreal/surreal-import-store.ts` with `create`, `get`, `update`, `delete`, `list` parity. Uses positional SurQL placeholders per the `@suaveplan/api-plugin-events` repository pattern (`$p0`, `$p1`, …).
|
|
- ADDED: `SurrealImportStoreConfig` Zod boundary schema (db adapter handle + optional table name + optional logger + optional clock).
|
|
- ADDED: `surql/tables/import_job.surql` migration defining the `import_job` table schema (id, owner_id, status, total_rows, processed_rows, success_count, error_count, errors, metadata, created_at, updated_at, completed_at) with the canonical `OVERWRITE` form used by `api-plugin-events`.
|
|
- ADDED: 1 new error code (`STORAGE_ERROR` already exists — reused for adapter failures; no registry growth).
|
|
- ADDED: OTel counters `import.store.writes_total{outcome}` and `import.store.reads_total{outcome}` registered on first use via the existing `getImportInstruments()` bundle (extension — keeps the bundle accessor stable).
|
|
- ADDED: Subpath export `./store/surreal/surreal-import-store` in `package.json#exports`.
|
|
- ADDED: Unit tests with a fake `SurrealAdapter` exercising every SurQL string, parameter ordering, and error path (mirrors `event-repository.test.ts`).
|
|
- ADDED: Integration test under `tests/integration/surreal-import-store.test.ts` using a real `SurrealDBFixture` from `@suaveplan/testing-integration-surrealdb`. Applies the `.surql` migration, round-trips a record, exercises filter/limit/offset on `list`, and asserts the `update`-on-missing error message.
|
|
- ADDED: `@suaveplan/surrealdb` + `@suaveplan/testing-integration-surrealdb` as deps if not present.
|
|
|
|
## Out of scope
|
|
|
|
Not addressed in this change — each remains a separate proposal:
|
|
|
|
- Streaming `Readable` / `AsyncIterable` input on `startImport` (§4 future enhancement).
|
|
- Foreign-key resolution across rows (§4).
|
|
- Parallel batch workers (§4).
|
|
- S3 / HTTP URL source adapters (§4).
|
|
- SSE progress push from the route layer (§4).
|
|
- Mitata benchmarks for the SurrealDB store (covered by the monorepo-wide benchmark wave).
|
|
- Postgres / SQLite store adapters (separate sibling proposals — `ImportStoreAdapter` is the contract that admits them).
|
|
- Any change to the existing `MemoryImportStore`, `ImportService`, `createImportRoutes`, parsers, transform pipeline, progress tracker, plugin descriptor, problem-json renderer, or OTel instrument bundle public surface beyond the additive counter pair noted above.
|
|
- Any modification to other `packages/api/api-plugin-*/`, `MANIFEST.md`, `STATUS.md`, `CLAUDE.md`, or the root `package.json` (except an additive `bun.lock` refresh if a workspace dep needs hoisting).
|
|
|
|
## Source mapping
|
|
|
|
This change derives from the original archive's `### Excluded from this migration` section
|
|
(`openspec/archive/2026-05-19-add-api-plugin-import/proposal.md`) and from the upstream
|
|
enterprise audit `../suaveplan/enterprise/api-plugin-import.md` §3 row 5.
|
|
|
|
| Requirement | Source | Notes |
|
|
| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
|
|
| `SurrealImportStore implements ImportStoreAdapter` | `openspec/archive/2026-05-19-add-api-plugin-import/proposal.md` (Excluded) + `../suaveplan/enterprise/api-plugin-import.md` §3 row 5 | Implements the SHIPPED `ImportStoreAdapter` interface — zero contract drift |
|
|
| Positional `$p0,$p1,…` SurQL placeholders | `packages/api/api-plugin-events/src/repositories/event-repository/event-repository.ts` L20-L25 | Pattern shared across all genesis SurrealDB repositories |
|
|
| Co-located `.surql` migration | `packages/api/api-plugin-events/surql/tables/event.surql` | Same `surql/tables/<table>.surql` layout |
|
|
| Zod boundary on store config | `openspec/AGENTS.md` §11 | Named-interface + parse-at-boundary policy |
|
|
| OTel counters on write / read | `openspec/AGENTS.md` §13 step 0 | Cached bundle accessor (`getImportInstruments`) extended additively |
|
|
| Integration test against real SurrealDB testcontainer | `packages/api/api-plugin-events/tests/integration/full-boot.test.ts` | Same `SurrealDBFixture` lifecycle |
|
|
| Update-on-missing rejects with `Job "<id>" not found` | `openspec/specs/api-plugin-import/spec.md` (SHIPPED `MemoryImportStore` requirement) | Contract parity with the in-memory store |
|
|
|
|
## Impact
|
|
|
|
### Affected dependents
|
|
|
|
`bunx turbo ls --filter=...@suaveplan/api-plugin-import` enumerates dependents.
|
|
|
|
- No dependents currently consume `MemoryImportStore`-specific symbols; the store is selected at host wiring time via the `ImportStoreAdapter` interface. Adding `SurrealImportStore` is purely additive. — [x] no action needed (reason: additive sibling implementation of an existing interface; existing consumers continue to receive `MemoryImportStore` unless they opt in)
|