63 lines
5.6 KiB
Markdown
63 lines
5.6 KiB
Markdown
# api-conformance-suites — `./testing` subpath conformance functions for every abstract capability
|
|
|
|
**Package**: multi-package — adds `./testing` subpaths to every Stage-1 abstract package
|
|
**Status**: PROPOSED
|
|
**Type**: feature (cross-package)
|
|
**Stage**: 5 (Testing kit + conformance suites) — see umbrella `openspec/changes/api-plugin-foundation/`
|
|
|
|
## Why
|
|
|
|
Per design doc §13.4, every abstract capability ships a conformance suite under its `./testing` subpath. The pattern mirrors `@suaveplan/i18n-shared/testing`'s `runBackendConformance`: a single function that exhaustively pins the contract; every adapter (Stage 4 production AND Stage 5 in-memory) runs the suite; new adapters that pass the suite are automatically known good; new invariants added to the suite are automatically enforced across every existing adapter.
|
|
|
|
This change adds those `./testing` subpaths. It's a multi-package change because each function lives in its respective abstract package — co-location matters (the contract owner owns the conformance gate). The companion in-memory adapters (Stage 5) provide the canonical "in-memory passes the suite" self-test, proving the suite isn't trivially empty.
|
|
|
|
## What changes
|
|
|
|
Adds a `./testing` export to each of the following abstract packages (created in Stage 1 — `api-router`, `api-auth`, `api-tenant-resolver` (currently inside `api-auth` umbrella), `api-tenant-provider`, `api-secrets`, `api-events`, `api-jobs`, `api-graphql`, `api-ws`, `api-sse`, `api-saga`, `api-2pc`, `api-db`). Each `./testing` subpath exports one conformance function:
|
|
|
|
- ADDED: `@suaveplan/api-router/testing` → `runRouterConformance(label, factory)` — pins route registration, dispatch, blue-green slot semantics, streaming bypass.
|
|
- ADDED: `@suaveplan/api-auth/testing` → `runAuthSessionConformance(label, factory)` — pins `resolveSession` contract, null-principal handling, securityScheme contribution.
|
|
- ADDED: `@suaveplan/api-tenant-resolver/testing` (or `@suaveplan/api-auth/testing` namespace) → `runTenantResolverConformance(label, factory)` — pins chain-advance sentinel, principal-derived resolution.
|
|
- ADDED: `@suaveplan/api-tenant-provider/testing` → `runTenantProviderConformance(label, factory)` — pins `list`, `listAsync`, mutation semantics.
|
|
- ADDED: `@suaveplan/api-secrets/testing` → `runSecretsConformance(label, factory)` — pins `get`, manifest validation, redaction.
|
|
- ADDED: `@suaveplan/api-events/testing` → `runEventsBusConformance(label, factory)` — pins publish/subscribe, ordering, idempotency, bounded-queue backpressure.
|
|
- ADDED: `@suaveplan/api-jobs/testing` → `runJobsConformance(label, factory)` — pins schedule, runScheduled, perTenant fan-out.
|
|
- ADDED: `@suaveplan/api-graphql/testing` → `runGraphqlEngineConformance(label, factory)` — pins schema merge, resolver execution, subscription delivery.
|
|
- ADDED: `@suaveplan/api-ws/testing` → `runWsEngineConformance(label, factory)` AND `runWsBrokerConformance(label, factory)` — pins connection lifecycle (engine) and subscription fan-out (broker).
|
|
- ADDED: `@suaveplan/api-sse/testing` → `runSseEngineConformance(label, factory)` — pins SSE stream lifecycle, reconnect-id handling.
|
|
- ADDED: `@suaveplan/api-saga/testing` → `runSagaParticipantConformance(label, factory)` — pins step idempotency, compensation, replay safety.
|
|
- ADDED: `@suaveplan/api-2pc/testing` → `run2pcParticipantConformance(label, factory)` — pins prepare/commit/rollback, in-doubt recovery.
|
|
- ADDED: `@suaveplan/api-db/testing` → `runDbAdapterConformance(label, factory)` — pins admin vs scoped surfaces, tenant injection, migration runner, unsafe-queries gating.
|
|
|
|
Self-tests:
|
|
|
|
- ADDED: For each `runXConformance`, a self-test file (`src/testing/<name>.self-test.test.ts`) that runs the function against the Stage-5 in-memory adapter (or, for `api-router-hono`'s case, the direct surface) AND asserts that the run reports at least one passing invariant per category (sanity check: a conformance function that runs zero invariants is broken).
|
|
|
|
## Out of scope
|
|
|
|
- The abstract package interfaces themselves — those are Stage-1 changes (already proposed).
|
|
- Adapter packages that consume these suites — those are separate changes (Stage 4 + Stage 5).
|
|
- Runtime-only test helpers unrelated to capability conformance — those live in `@suaveplan/testing`.
|
|
|
|
## Source mapping
|
|
|
|
Genesis-original. Source of design intent: `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`.
|
|
|
|
| Requirement | Design doc section |
|
|
|---|---|
|
|
| `./testing` subpath pattern (mirrors `i18n-shared/testing`) | §13.4 (intro) |
|
|
| One conformance fn per capability | §13.4 (final paragraph) |
|
|
| Same suite runs against every adapter | §13.4 (middle paragraph) |
|
|
| New invariants automatically enforce across adapters | §13.4 (middle paragraph) |
|
|
|
|
## Verification approach
|
|
|
|
- Each conformance function lives at `packages/api/<abstract>/src/testing/<run-name>.ts` with a sibling `.md` documenting which invariants it pins (one bullet per `it(...)` title).
|
|
- Each function's self-test (`<run-name>.self-test.test.ts`) calls it with the Stage-5 in-memory adapter as the factory and asserts non-empty pass count.
|
|
- Each Stage-4 production adapter (in its own change) and each Stage-5 in-memory adapter (in its own change) runs the function against its own factory; those tests live in the adapter package, not here.
|
|
- A dep-graph guard (`tests/dep-graph/conformance-subpath-shape.test.ts`) iterates every abstract package and asserts the `./testing` subpath export exists AND exports the expected `runXConformance` symbol.
|
|
|
|
## Migration impact
|
|
|
|
None — additive only. Existing abstract packages get a new subpath export; no existing import surface is touched.
|