suaveplan/openspec/changes/api-conformance-suites/proposal.md

api-conformance-suites — ./testing subpath conformance functions for every abstract capability

Package: multi-package — adds ./testing subpaths to every Stage-1 abstract package Status: PROPOSED Type: feature (cross-package) Stage: 5 (Testing kit + conformance suites) — see umbrella openspec/changes/api-plugin-foundation/

Why

Per design doc §13.4, every abstract capability ships a conformance suite under its ./testing subpath. The pattern mirrors @suaveplan/i18n-shared/testing's runBackendConformance: a single function that exhaustively pins the contract; every adapter (Stage 4 production AND Stage 5 in-memory) runs the suite; new adapters that pass the suite are automatically known good; new invariants added to the suite are automatically enforced across every existing adapter.

This change adds those ./testing subpaths. It's a multi-package change because each function lives in its respective abstract package — co-location matters (the contract owner owns the conformance gate). The companion in-memory adapters (Stage 5) provide the canonical "in-memory passes the suite" self-test, proving the suite isn't trivially empty.

What changes

Adds a ./testing export to each of the following abstract packages (created in Stage 1 — api-router, api-auth, api-tenant-resolver (currently inside api-auth umbrella), api-tenant-provider, api-secrets, api-events, api-jobs, api-graphql, api-ws, api-sse, api-saga, api-2pc, api-db). Each ./testing subpath exports one conformance function:

• ADDED: @suaveplan/api-router/testing → runRouterConformance(label, factory) — pins route registration, dispatch, blue-green slot semantics, streaming bypass.
• ADDED: @suaveplan/api-auth/testing → runAuthSessionConformance(label, factory) — pins resolveSession contract, null-principal handling, securityScheme contribution.
• ADDED: @suaveplan/api-tenant-resolver/testing (or @suaveplan/api-auth/testing namespace) → runTenantResolverConformance(label, factory) — pins chain-advance sentinel, principal-derived resolution.
• ADDED: @suaveplan/api-tenant-provider/testing → runTenantProviderConformance(label, factory) — pins list, listAsync, mutation semantics.
• ADDED: @suaveplan/api-secrets/testing → runSecretsConformance(label, factory) — pins get, manifest validation, redaction.
• ADDED: @suaveplan/api-events/testing → runEventsBusConformance(label, factory) — pins publish/subscribe, ordering, idempotency, bounded-queue backpressure.
• ADDED: @suaveplan/api-jobs/testing → runJobsConformance(label, factory) — pins schedule, runScheduled, perTenant fan-out.
• ADDED: @suaveplan/api-graphql/testing → runGraphqlEngineConformance(label, factory) — pins schema merge, resolver execution, subscription delivery.
• ADDED: @suaveplan/api-ws/testing → runWsEngineConformance(label, factory) AND runWsBrokerConformance(label, factory) — pins connection lifecycle (engine) and subscription fan-out (broker).
• ADDED: @suaveplan/api-sse/testing → runSseEngineConformance(label, factory) — pins SSE stream lifecycle, reconnect-id handling.
• ADDED: @suaveplan/api-saga/testing → runSagaParticipantConformance(label, factory) — pins step idempotency, compensation, replay safety.
• ADDED: @suaveplan/api-2pc/testing → run2pcParticipantConformance(label, factory) — pins prepare/commit/rollback, in-doubt recovery.
• ADDED: @suaveplan/api-db/testing → runDbAdapterConformance(label, factory) — pins admin vs scoped surfaces, tenant injection, migration runner, unsafe-queries gating.

Self-tests:

• ADDED: For each runXConformance, a self-test file (src/testing/<name>.self-test.test.ts) that runs the function against the Stage-5 in-memory adapter (or, for api-router-hono's case, the direct surface) AND asserts that the run reports at least one passing invariant per category (sanity check: a conformance function that runs zero invariants is broken).

Out of scope

• The abstract package interfaces themselves — those are Stage-1 changes (already proposed).
• Adapter packages that consume these suites — those are separate changes (Stage 4 + Stage 5).
• Runtime-only test helpers unrelated to capability conformance — those live in @suaveplan/testing.

Source mapping

Genesis-original. Source of design intent: docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md.

Requirement	Design doc section
./testing subpath pattern (mirrors i18n-shared/testing)	§13.4 (intro)
One conformance fn per capability	§13.4 (final paragraph)
Same suite runs against every adapter	§13.4 (middle paragraph)
New invariants automatically enforce across adapters	§13.4 (middle paragraph)

Verification approach

• Each conformance function lives at packages/api/<abstract>/src/testing/<run-name>.ts with a sibling .md documenting which invariants it pins (one bullet per it(...) title).
• Each function's self-test (<run-name>.self-test.test.ts) calls it with the Stage-5 in-memory adapter as the factory and asserts non-empty pass count.
• Each Stage-4 production adapter (in its own change) and each Stage-5 in-memory adapter (in its own change) runs the function against its own factory; those tests live in the adapter package, not here.
• A dep-graph guard (tests/dep-graph/conformance-subpath-shape.test.ts) iterates every abstract package and asserts the ./testing subpath export exists AND exports the expected runXConformance symbol.

Migration impact

None — additive only. Existing abstract packages get a new subpath export; no existing import surface is touched.