suaveplan/openspec/changes/enterprise-testing-load-perf/design.md

Context

mitata benchmarks under src/__benchmarks__/ already gate per-package micro-performance at the 10% regression floor (openspec/specs/testing/spec.md "Benchmark gate in CI"). This protects pure-function throughput but says nothing about request-response latency, concurrent-user behaviour, websocket fan-out, or the full Hono middleware stack of apps/testbed. The 2026-04-25 audit (§2) flags this as a high-severity gap and recommends k6 or autocannon as the remediation.

apps/testbed already ships a Docker staging environment and a Playwright sweep. Adding a load gate against the same staging environment closes the third side of the verification triangle: render correctness (Playwright), behavioural assertions (vitest), and performance under load (k6/autocannon).

Goals / Non-Goals

Goals:

• A single, runner-agnostic public API (defineLoadScenario, runLoadScenario, assertThresholds) that callers use without knowing which driver runs underneath.
• k6 as primary driver via the grafana/k6 testcontainers image; autocannon as a lighter Node-native alternative for environments without Docker (CI matrix on macOS/Windows runners that lack Docker support).
• Default thresholds tuned for typical Hono REST workloads: http_req_duration{p(95)} < 500ms, http_req_failed < 1%, iteration_duration{p(99)} < 2000ms. All overridable per scenario.
• Four run modes: smoke (gating, 1 vu / 30s), load (10 vu / 5m), stress (50 vu / 10m), spike (0→100 vu in 30s). Smoke runs nightly in CI; the others run weekly or on demand.
• 100% Istanbul coverage on the wrapper itself; the k6 binary invocation is mocked via injected spawn/fetch shims so the package tests run without Docker.
• Stable error contract: TestingLoadError with codes prefixed TESTING_LOAD.* (e.g. TESTING_LOAD.THRESHOLD_VIOLATION, TESTING_LOAD.DRIVER_FAILED, TESTING_LOAD.CONFIG_INVALID).

Non-Goals:

• Replacing or relaxing the mitata 10% micro-benchmark gate.
• Distributed load generation. k6 cloud, k6 operator on Kubernetes, and Locust are out of scope. Single-runner load is sufficient for the smoke gate.
• Custom k6 extensions (xk6). The wrapper consumes k6's stock features only.
• Load testing against production. All scenarios target the Docker staging environment defined by apps/testbed/docker-compose.testbed.yml (or equivalent per-app stack).

Decisions

D1: Runner-agnostic API with pluggable drivers

export interface LoadDriver {
    name: "k6" | "autocannon";
    run(scenario: LoadScenario, opts: LoadRunOptions): Promise<LoadSummary>;
}

export function defineLoadScenario(spec: LoadScenarioSpec): LoadScenario;
export function runLoadScenario(scenario: LoadScenario, opts?: LoadRunOptions): Promise<{ summary: LoadSummary; raw: unknown }>;
export function assertThresholds(summary: LoadSummary): void;

LoadScenario carries { name, target, vus, duration, thresholds, stages? }. LoadRunOptions carries { driver?: LoadDriver; signal?: AbortSignal; onMetric?: (m) => void }. The driver is injected; k6 is the default when present, autocannon otherwise.

D2: k6 via testcontainers, autocannon via npm

k6 runs in the grafana/k6 container, with the script generated from the LoadScenario and mounted into the container. Stdout is parsed via k6's structured-summary JSON output (--summary-export). Autocannon runs in-process via the npm package, with options mapped from the same LoadScenario. Both drivers populate the same LoadSummary shape so assertThresholds is driver-agnostic.

D3: Threshold contract

Thresholds are RFC-style key→predicate maps:

export type Thresholds = Record<string, string[]>;
// e.g. { "http_req_duration": ["p(95)<500", "p(99)<1000"], "http_req_failed": ["rate<0.01"] }

Defaults applied if the caller omits the field; overrides merge on top. assertThresholds(summary) throws TestingLoadError("TESTING_LOAD.THRESHOLD_VIOLATION", …, { context: { failed: [...] } }) listing every failed predicate.

D4: Run modes as named presets

smoke, load, stress, spike are exported constants composing vus + stages + duration. Callers spread them into defineLoadScenario:

defineLoadScenario({ ...presets.smoke, name: "GET /healthz", target: "http://testbed:3000/healthz" });

D5: Self-testability without Docker

The k6 driver accepts an injectable spawn function and an injectable fetch. Tests stub both to feed canned k6 JSON summaries; no Docker required for the wrapper's own test suite. Same pattern as testing-integration-postgres's injectable runner function (per CLAUDE.md package listing).

D6: Error shape parallels @suaveplan/testing

TestingLoadError extends Error with { code, cause?, context? } and codes prefixed TESTING_LOAD.*. Shape-compatible with the existing TestingError so a future @suaveplan/error adapter bridges both at once. No @suaveplan/error dep on day one (keeps Layer-0 carve-out per CLAUDE.md rule 8).

D7: App integration via *.smoke.load.ts

Each app under apps/ ships at least one *.smoke.load.ts file. CI runs them nightly against the app's staging stack. apps/testbed enumerates one smoke scenario per route in routes.ts. The websocket-gateway app's smoke scenario opens N websocket connections and asserts message round-trip latency.

Risks / Trade-offs

• k6 binary availability. Mitigated by the testcontainers image plus the autocannon fallback path.
• CI wall-clock cost. Smoke scenarios are 30s × N apps; load/stress/spike are out-of-band (weekly or on demand).
• Threshold tuning. Defaults may be too tight for some routes. Mitigated by per-scenario overrides; the smoke run is intended to catch regressions, not enforce SLOs.
• Driver-summary impedance. k6 and autocannon report different metric names. Mitigated by the wrapper normalising both into LoadSummary before threshold evaluation.
• Layer-0 dep purity. The package depends on @suaveplan/types only (parallel to testing-integration*). External catalog deps cover everything else.