4.5 KiB
Why
The 2026-04-25 test-suite audit (/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md §2 row "No load/perf gate beyond mitata micro-bench") identifies an end-to-end performance gap. The canonical openspec/specs/testing/spec.md already enforces a mitata 10% benchmark regression gate, but mitata measures function-level micro-throughput in isolation. It does not exercise the request-response cycle, websocket fan-out, or concurrent-user behaviour of apps/testbed's Hono routes or the websocket-gateway. A regression in middleware ordering, async-context propagation, or Drizzle query plans can pass every micro-benchmark while doubling p95 latency under realistic load.
CLAUDE.md (/mnt/s/web/genesis/CLAUDE.md §Web App & Page Verification) requires Docker-staging verification, full Playwright sweeps, and zero-defect-to-live before pushing client-side work. Audit §7 confirms apps/testbed only partially meets this bar and that "release readiness" gating is missing. End-to-end load and perf gates close the third missing piece (after audit §2's a11y and visual regression, scheduled separately): proving the system holds up under load, not just that it renders.
What Changes
This change introduces a new @suaveplan/testing-load package and extends openspec/specs/testing/spec.md with an end-to-end performance gate that complements (does not replace) the existing mitata micro-benchmark gate.
New @suaveplan/testing-load package
- Layout:
packages/testing/testing-load. Zero@suaveplan/*runtime deps except@suaveplan/types(consistent with the rule-#9 carve-out perCLAUDE.md§Critical and the existingtesting-integration*precedent). - Public API:
defineLoadScenario({ name, target, vus, duration, thresholds, stages? }),runLoadScenario(scenario, opts)returning{ summary, raw },assertThresholds(summary)throwing on violation. - Tooling:
k6(binary, run via testcontainers) is the primary driver.autocannon(npm, catalog ref) is supported as a lighter Node-native alternative for environments without Docker. - Default thresholds:
http_req_duration{p(95)} < 500ms,http_req_failed < 1%,iteration_duration{p(99)} < 2000ms. Per-scenario overrides supported. - Run modes:
smoke(1 vu, 30s),load(10 vu, 5m),stress(50 vu, 10m),spike(0→100 vu in 30s). - Errors:
TestingLoadErrorextendingErrorwith{ code, cause?, context? }and stableTESTING_LOAD.*codes. Shape-compatible with@suaveplan/errorfor a future adapter, matching the testing-package precedent inopenspec/specs/testing/spec.md. - Quality: 100% Istanbul coverage on the wrapper, sibling
.mdper implementation file, Zod schemas in sibling*.schemas.ts, mitata bench file, fast-check property tests where applicable.
Extension to openspec/specs/testing/spec.md
A new requirement layered on the existing Benchmark gate: every app under apps/ SHALL ship a *.smoke.load.ts exercising the smoke scenario nightly, failing on threshold violation. apps/testbed SHALL define a load scenario per route declared in its routes.ts. The websocket-gateway SHALL define a connection-fan-out scenario. The release-readiness gate (cross-referenced from audit §7) consumes the smoke result as one of its inputs.
Capabilities
New Capabilities
testing-load— end-to-end load and performance scenarios with k6/autocannon drivers, threshold assertions, structured errors, and Zod-validated configs. Layer 0 utility (no@suaveplan/*runtime deps except@suaveplan/types).
Modified Capabilities
testing— adds an end-to-end performance gate requirement for every app underapps/, layered on top of the existing mitata 10% micro-benchmark gate. Does not modify or relax the mitata gate.
Impact
- Affected code: new
packages/testing/testing-load/package; per-app*.smoke.load.tsfiles added under eachapps/<name>/.apps/testbedgains one scenario per route inroutes.ts. Thewebsocket-gatewayapp gains a connection-fan-out scenario. - Affected catalogs: root
package.jsongainsautocannonundercatalogs.testing.k6is run via thegrafana/k6testcontainers image (no npm catalog entry needed). - CI cadence: smoke scenarios run nightly per app. Load/stress/spike scenarios run weekly or on-demand against staging.
- Coverage: wrapper meets 100% Istanbul; integration with k6 binary mocked via injected exec/fetch shims to keep the package self-testable without Docker.
- No replacement: the existing mitata 10% regression gate stays in place. Both gates run; both gates must pass.