suaveplan/openspec/specs/testing/spec.md

testing

Purpose

The @suaveplan/testing package provides universal, runner-agnostic test utilities — FakeClock, Factory, FixtureManager, MemoryStore, RequestBuilder/ResponseBuilder, helpers, and observability adapters — that work under both Vitest and bun:test via conditional exports. The package has zero browser, Node, or React dependencies in its core runtime exports and zero @suaveplan/* workspace dependencies, making it the foundation Layer 0 testing primitive that every other package may consume safely. Determinism is enforced through dependency-injected Clock, Rng, and AbortSignal parameters, and every public surface is instrumented with @opentelemetry/api counters/histograms that no-op when no SDK is registered.

Requirements

Requirement: Universal purity

The @suaveplan/testing package SHALL depend on zero browser-only, Node-only, or React APIs at the module-graph level for its core runtime exports. The ./config subpath and ./polyfills subpath MAY reference Node built-ins because they are explicitly Node-side and are never pulled transitively by browser-runnable exports.

Scenario: Browser-runnable exports avoid Node built-ins

• WHEN a browser target imports FakeClock, RequestBuilder, ResponseBuilder, Factory, FixtureManager, MemoryStore, or any helper
• THEN the resolved module graph MUST NOT reference node:fs, node:path, node:url, process, Buffer, or any Node-only API

Scenario: Node-side subpaths are segregated

• WHEN a consumer imports @suaveplan/testing/config or @suaveplan/testing/polyfills
• THEN those imports resolve to modules that are allowed to reference Node built-ins AND the core export (@suaveplan/testing) MUST NOT transitively re-import them

Verified by: tests/tree-shake.test.ts::"FakeClock subpath does not pull MemoryStore / RequestBuilder / Factory" AND tests/tree-shake.test.ts::"RequestBuilder subpath does not pull FakeClock / MemoryStore" AND tests/tree-shake.test.ts::"MemoryStore subpath does not pull builders or factories"

Requirement: No @suaveplan/* workspace dependencies

The @suaveplan/testing package dependencies, peerDependencies, and devDependencies MUST NOT list any @suaveplan/* scoped package. External catalog deps only (vitest, happy-dom, ws, tslib, zod, @opentelemetry/api, fast-check, mitata, @types/*, build tooling).

Scenario: Dep-graph guard

• WHEN bunx turbo ls --filter=@suaveplan/testing... is run
• THEN no @suaveplan/* package appears in the forward closure of @suaveplan/testing

Verified by: tests/dep-graph.test.ts::"no @suaveplan/* dependencies, peerDependencies, or devDependencies"

Requirement: Public types validated via Zod

Every public function's input and output types SHALL be modeled by a Zod schema exported from a sibling <module>.schemas.ts file under a stable namespace (e.g. RequestBuilderSchemas.input). Schemas MUST NOT be imported by functional entry points; consumers opt in by importing *.schemas.ts directly.

Scenario: Runtime validation available on opt-in

• WHEN a consumer imports FactorySchemas.buildOptions from @suaveplan/testing/factories/factory/factory.schemas and calls .parse(value)
• THEN invalid input is rejected with a ZodError whose issues[].path points to the offending field

Scenario: Schemas do not bloat the functional bundle

• WHEN a build imports only Factory from @suaveplan/testing
• THEN the resolved bundle MUST NOT include Zod or any *.schemas.ts module

Verified by: tests/spec-conformance.spec-conformance.test.ts::"every *.schemas.ts file does NOT import its sibling implementation file" AND src/builders/request-builder/request-builder.test.ts::"output schema accepts a valid TestRequest" AND src/builders/response-builder/response-builder.test.ts::"output schema accepts a valid response"

Requirement: Structured error hierarchy

The package SHALL throw only subclasses of TestingError extends Error with a stable code of the form TESTING.<DOMAIN>.<REASON> (plain uppercase ASCII, dots as separators). Errors MUST carry { code, cause?, context? } fields. The package MUST NOT throw plain Error or TypeError from any public code path.

Scenario: Timer overflow throws a coded error

• WHEN FakeClock is asked to schedule more than MAX_TIMERS timers
• THEN it throws TestingError with code === "TESTING.CLOCK.MAX_TIMERS_EXCEEDED" AND context.limit === MAX_TIMERS

Scenario: Fixture setup failure preserves cause

• WHEN a fixture's setup function throws
• THEN FixtureManager.setupAll throws TestingError with code === "TESTING.FIXTURES.SETUP_FAILED" AND cause is the original thrown value AND context.fixtureName is the failing fixture's name

Scenario: Abort signal triggered mid-wait

• WHEN waitFor(predicate, { signal }) runs with a signal that aborts during polling
• THEN the returned promise rejects with TestingError where code === "TESTING.ABORTED" AND cause === signal.reason

Verified by: src/errors/testing-error.test.ts::"AbortedError hardcodes TESTING.ABORTED code" AND src/errors/testing-error.test.ts::"is instanceof Error and TestingError" AND src/errors/testing-error.test.ts::"each subclass preserves instanceof TestingError" AND src/clock/fake-clock/fake-clock.test.ts::"throws MAX_TIMERS_EXCEEDED when the limit is reached" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"setup failure is captured without short-circuiting" AND src/helpers/test-helpers.test.ts::"aborts on signal mid-poll"

Requirement: Deterministic clock and RNG via dependency injection

Every public function that reads time, randomness, or iterates SHALL accept an optional { clock?: Clock; rng?: Rng; signal?: AbortSignal } options parameter. Zero-config callers MUST get behaviorally stable defaults: defaultClock (backed by Date.now / performance.now), cryptoRng (backed by crypto.getRandomValues), and no abort.

Scenario: Injected clock controls observed time

• WHEN waitFor(predicate, { clock: fakeClock }) runs and fakeClock.advance(1000) is called
• THEN the helper observes 1000 ms elapsed AND does NOT read Date.now() or performance.now() directly

Scenario: Injected RNG produces deterministic factory output

• WHEN Factory.build({ rng: seededRng(42) }) is called twice with the same factory definition
• THEN both calls return byte-identical output (for factories whose definition is otherwise deterministic)

Scenario: Module-scope non-determinism absent

• WHEN the built dist/ is grep'd for Math.random( or Date.now( or performance.now(
• THEN those symbols appear only inside the defaultClock / cryptoRng module (and FakeClock's internal simulation), never at module scope in other files

Verified by: tests/spec-conformance.spec-conformance.test.ts::"Math.random / Date.now / performance.now appear only in allow-listed files" AND src/helpers/test-helpers.test.ts::"uses the injected clock for timeout detection" AND src/factories/factory/factory.test.ts::"identical seed + same definition produces identical output" AND src/context/context.test.ts::"defaultClock.hrtime falls back to Date.now when performance is missing"

Requirement: Cooperative cancellation via AbortSignal

Every iterative or async helper SHALL check its signal?.aborted property at bounded intervals (≤ 256 items or ≤ 1 KB scanned per check) and throw TestingError("TESTING.ABORTED", …) on first observed abort without returning partial results.

Scenario: Batch factory aborts within one tick

• WHEN Factory.buildBatch(10000, { signal }) is running and signal.abort() is called after 200 ms
• THEN the promise rejects with TESTING.ABORTED within one event-loop tick of the abort

Scenario: Fixture teardown aborts between fixtures, never mid-fixture

• WHEN FixtureManager.teardownAll({ signal }) is mid-teardown and the signal fires
• THEN the current fixture's teardown completes AND the next fixture is NOT started AND the promise rejects with TESTING.ABORTED

Verified by: src/factories/factory/factory.test.ts::"honors AbortSignal mid-batch at the 256-item boundary" AND src/factories/factory/factory.test.ts::"honors AbortSignal pre-loop" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"abort between fixtures stops further setup" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"teardownAll with pre-aborted signal throws AbortedError" AND src/helpers/test-helpers.test.ts::"aborts on signal pre-poll" AND src/helpers/test-helpers.test.ts::"aborts on signal mid-poll"

Requirement: Dual-runner conditional exports

The package exports map SHALL route @suaveplan/testing via a vitest condition to the vitest-backed barrel, a bun condition to the bun-test shim, and a default condition to the vitest barrel. The vitest barrel MUST re-export describe, it, test, expect, afterAll, afterEach, beforeAll, beforeEach, vi, fn, mock, spy, stub, fakeClock, restoreMocks, restoreTimers. The bun-test shim MUST expose the same names via bun-test primitives.

Scenario: Vitest consumer resolves vitest primitives

• WHEN a package with vitest in its test runner configuration imports { describe, it, expect } from @suaveplan/testing
• THEN resolution MUST pick the vitest condition and the imported symbols MUST be vitest's

Scenario: Bun-test consumer resolves bun-test primitives

• WHEN a package configured for bun test imports { describe, it, expect } from @suaveplan/testing
• THEN resolution MUST pick the bun condition and the imported symbols MUST come from bun's test runner

Verified by: tests/spec-conformance.spec-conformance.test.ts::"exports map provides distinct subpaths for every public surface"

Requirement: Observability via @opentelemetry/api

Every public entrypoint SHALL be instrumented with an OTEL counter (testing.<domain>.<op>.count), a histogram (testing.<domain>.<op>.duration_ms), and an error counter (testing.<domain>.<op>.errors with code attribute on error). Spans SHALL be created only for iterative operations (waitFor, FixtureManager.setupAll/teardownAll, Factory.buildBatch). When no OTEL SDK is registered, all instrumentation MUST be a no-op at runtime.

Scenario: Counter increments on every call

• WHEN RequestBuilder methods are called N times during a test run with a registered OTEL SDK
• THEN the metric testing.builders.request.count has recorded N increments

Scenario: Error counter records code attribute

• WHEN FakeClock.schedule throws TESTING.CLOCK.MAX_TIMERS_EXCEEDED
• THEN the metric testing.clock.schedule.errors records an increment with attribute code = "TESTING.CLOCK.MAX_TIMERS_EXCEEDED"

Scenario: Zero overhead without SDK

• WHEN the benchmark suite runs without any OTEL SDK registered
• THEN the difference between the OTEL-wrapped public entrypoint and an identical unwrapped baseline MUST be less than 5% per op

Verified by: src/observability/with-observability.test.ts::"calling the same metric name twice reuses cached instruments" AND src/observability/with-observability.test.ts::"throws synchronously inside span and records error" AND src/observability/with-observability.test.ts::"non-iterative async rejection records and rethrows" AND src/observability/with-observability.test.ts::"withObservability nowMs falls back to Date.now when performance is missing"

Requirement: Coverage and docs gates

Istanbul statement + branch + function + line coverage across packages/testing/testing/src/ (excluding *.bench.ts and *.schemas.ts) MUST be 100%. Every implementation file (*.ts other than *.test.ts, *.bench.ts, *.schemas.ts, inner-dir *.types.ts, barrel index.ts) MUST have a sibling .md file of ≥ 200 words (excluding code fences) with a simple and an advanced usage example.

Scenario: Coverage gate in CI

• WHEN bunx turbo test:coverage --filter=@suaveplan/testing runs
• THEN reported statement, branch, function, and line coverage are each 100%

Scenario: Docs coverage enforced

• WHEN a script enumerates implementation files under src/ (excluding the stated exclusions) and checks for sibling .md files
• THEN every implementation file has a sibling .md AND every .md is ≥ 200 non-code words

Verified by: vitest.config.ts (coverage thresholds set to 100/100/100/100) AND bunx turbo test:coverage --filter=@suaveplan/testing (CI run) AND bun run docs:check (sibling-.md enumeration script)

Requirement: No disallowed constructs in src/

Files under packages/testing/testing/src/ MUST NOT contain the any keyword, @ts-ignore, @ts-expect-error, TODO, FIXME, @deprecated, it.skip, describe.skip, test.skip, .only, or any mock/stub of runtime behaviour intended for production code paths.

Scenario: Biome scan clean

• WHEN bunx biome check src/ runs with the project's strict config
• THEN the scan reports zero noExplicitAny diagnostics AND zero suppressed directives

Scenario: Grep guard

• WHEN grep -rnE "TODO|FIXME|@deprecated|\\.skip\\(|\\.only\\(|ts-ignore|ts-expect-error" src/ runs
• THEN zero matches are produced

Verified by: tests/no-disallowed.test.ts::"src/ contains no disallowed constructs"

Requirement: sideEffects: false and subpath exports

The package MUST declare "sideEffects": false in package.json AND provide explicit subpath exports for every module intended as a distinct import surface. A tree-shake test MUST prove that importing one submodule does not pull unrelated submodules.

Scenario: Tree-shake test passes

• WHEN a test imports only FakeClock from @suaveplan/testing and reads the resolved bundle text
• THEN the bundle MUST NOT contain identifiers unique to RequestBuilder, MemoryStore, FixtureManager, or Factory

Verified by: tests/tree-shake.test.ts::"package.json declares sideEffects: false" AND tests/tree-shake.test.ts::"FakeClock subpath does not pull MemoryStore / RequestBuilder / Factory" AND tests/tree-shake.test.ts::"RequestBuilder subpath does not pull FakeClock / MemoryStore" AND tests/tree-shake.test.ts::"MemoryStore subpath does not pull builders or factories"

Requirement: Property-based coverage for pure helpers

Pure functions and deterministic state machines in the package (clock arithmetic, builder state transitions, factory deterministic output, memory-store TTL expiry, wait-for polling) SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants as applicable.

Scenario: Builder composition is associative

• WHEN fast-check runs over arbitrary RequestBuilder presets a, b, c applied in different groupings
• THEN (a then b) then c produces the same final state as a then (b then c)

Scenario: FakeClock advance + schedule commutes within a tick

• WHEN fast-check runs over sequences of advance and schedule calls that never cross a timer threshold
• THEN reordering within a single synchronous batch produces the same observed state

Verified by: tests/properties.test.ts::"header / query / params order is irrelevant" AND tests/properties.test.ts::"header / body / latency / status order is irrelevant" AND tests/properties.test.ts::"advance is associative within a single timer-free window" AND tests/properties.test.ts::"hrtime is monotonically non-decreasing under advance" AND tests/properties.test.ts::"build with same seed produces same items for any index" AND tests/properties.test.ts::"an entry's expiresAt equals createdAt + ttlMs (when ttl > 0)"

Requirement: Benchmark gate in CI

A mitata benchmark suite under src/__benchmarks__/ SHALL record baselines in benchmarks/baselines.json. CI SHALL rerun the benchmarks (3-strike, take fastest), compare to baselines, and fail if any op regresses by more than 10%. Baseline updates MUST land on a dedicated, reviewer-approved commit separate from functional changes.

Scenario: Regression gate trips

• WHEN a PR introduces a 15% slowdown on Factory.build
• THEN the CI benchmark step fails the build

Scenario: Baseline update is intentional

• WHEN a maintainer intentionally updates benchmarks/baselines.json
• THEN the baseline commit is separate from functional code changes and is reviewed before landing

Verified by: src/__benchmarks__/all.bench.ts (mitata baseline source) AND benchmarks/baselines.json (committed baselines re-run by CI)

Requirement: Plugin surface on FixtureManager

FixtureManager SHALL accept plugins implementing { name: string; beforeEach?; afterEach?; onError? } and invoke their hooks around each fixture's own setup/teardown lifecycle. Plugins MUST NOT be able to cancel a teardown mid-flight except via onError.

Scenario: Plugin hook fires around each fixture

• WHEN a FixtureManager with a plugin registered runs setupAll over two fixtures
• THEN the plugin's beforeEach is called exactly twice in fixture-registration order AND afterEach is called exactly twice in reverse order

Verified by: src/fixtures/fixture-manager/fixture-manager.test.ts::"beforeEach and afterEach fire around each setup" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"onError fires on setup failure" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"plugin hook error does not interrupt pipeline" AND src/fixtures/fixture-manager/fixture-manager.test.ts::"plugin hook error is logged and does not interrupt"

Requirement: createTestConfig() absorbs WSL2 pool and polyfill concerns

The @suaveplan/testing/config factory SHALL produce a Vitest config with pool: "forks", setupFiles: ["@suaveplan/testing/polyfills"], and — when the caller requests happy-dom — fileParallelism: false AND dynamically-loaded @vitejs/plugin-react if that optional peer is installed. When the caller's environment requests bun-test, the factory SHALL return an equivalent bun-test config instead.

Scenario: Happy-dom caller gets React plugin

• WHEN createTestConfig({ environment: "happy-dom" }) is awaited and @vitejs/plugin-react is installed
• THEN the returned config includes the React plugin in its plugins array AND pool === "forks" AND setupFiles includes the polyfills path

Scenario: bun-test caller gets bun config

• WHEN createTestConfig({ runner: "bun:test" }) is awaited
• THEN the returned value is a BunTestConfig with pool not set (bun-test has no concept of pool) AND any warnings about missing preload hooks are logged once

Verified by: src/config/create-test-config.test.ts::"returns canonical defaults when called with no args" AND src/config/create-test-config.test.ts::"supports happy-dom environment" AND src/config/create-test-config.test.ts::"defaults coverage thresholds to 100/100/100/100" AND src/config/create-test-config.test.ts::"layers additional setup files after the polyfills entry" AND src/config/create-test-config.test.ts::"allows overriding the pool" AND src/config/create-test-config.test.ts::"allows disabling fileParallelism"

Requirement: Builder presets thread <TBody> through .build()

Every RequestBuilder and ResponseBuilder preset factory SHALL accept an optional <TBody = unknown> type parameter. The returned builder MUST be RequestBuilder<TBody> / ResponseBuilder<TBody> and .build() MUST return TestRequest<TBody | undefined> / TestResponse<TBody | undefined>. Callers who omit the parameter MUST observe the same unknown defaults that pre-dated this change.

Scenario: post threads body type

• WHEN post<{ name: string }>("/x", { name: "a" }).build() is called
• THEN the inferred return type is TestRequest<{ name: string } | undefined>

Scenario: ok threads body type

• WHEN ok<{ id: string }>({ id: "1" }).build() is called
• THEN the inferred return type is TestResponse<{ id: string } | undefined>

Scenario: zero-arg preset preserves unknown default

• WHEN get("/x").build() or ok().build() is called without a type argument
• THEN the inferred return types are TestRequest<unknown> and TestResponse<unknown> respectively

Verified by: src/builders/request-builder/request-builder.spec-conformance.test.ts::"post<TBody>(url, body).build() returns TestRequest<TBody | undefined>" AND src/builders/request-builder/request-builder.spec-conformance.test.ts::"get<TBody>(url).build() returns TestRequest<TBody | undefined>" AND src/builders/request-builder/request-builder.spec-conformance.test.ts::"put<TBody>, patch<TBody>, del<TBody>, head<TBody>, options<TBody> all thread TBody" AND src/builders/request-builder/request-builder.spec-conformance.test.ts::"zero-arg presets default TBody to unknown" AND src/builders/request-builder/request-builder.spec-conformance.test.ts::"RequestBuilder<TBody> exposes typed body field on state and build" AND src/builders/response-builder/response-builder.spec-conformance.test.ts::"ok<TBody>(body).build() returns TestResponse<TBody | undefined>" AND src/builders/response-builder/response-builder.spec-conformance.test.ts::"created<TBody>, badRequest<TBody>, unauthorized<TBody>, notFound<TBody>, serverError<TBody> thread TBody" AND src/builders/response-builder/response-builder.spec-conformance.test.ts::"noContent<TBody>() returns ResponseBuilder<TBody>" AND src/builders/response-builder/response-builder.spec-conformance.test.ts::"zero-arg presets default TBody to unknown" AND src/builders/response-builder/response-builder.spec-conformance.test.ts::"ResponseBuilder<TBody> exposes typed body on state and build"

Requirement: FixtureManager.get accepts a typed value parameter

FixtureManager.get SHALL accept an optional <T = unknown> type parameter and return FixtureState<T> | undefined. The narrowing SHALL extend through state.value so callers see T | null without a follow-up cast. Internally the manager MAY perform a single boundary cast; consumers MUST NOT need any.

Scenario: typed get returns typed state

• WHEN mgr.get<DbFixture>("db") is called
• THEN the inferred return type is FixtureState<DbFixture> | undefined AND the inferred type of state.value (after state !== undefined) is DbFixture | null

Scenario: untyped get preserves unknown default

• WHEN mgr.get("db") is called without a type argument
• THEN the inferred return type is FixtureState<unknown> | undefined

Verified by: src/fixtures/fixture-manager/fixture-manager.spec-conformance.test.ts::"get<T>(name) returns FixtureState<T> | undefined" AND src/fixtures/fixture-manager/fixture-manager.spec-conformance.test.ts::"get<T>(name) narrows state.value to T | null" AND src/fixtures/fixture-manager/fixture-manager.spec-conformance.test.ts::"get(name) without explicit T defaults to FixtureState<unknown>"

Requirement: assertThrows returns the caller-narrowed error type

assertThrows and assertThrowsAsync SHALL accept an optional <TError extends Error = Error> type parameter and return TError / Promise<TError> respectively. Internally the helper MAY cast at the boundary; the runtime fallback (wrap a non-Error throw in new Error(String(value))) MUST remain.

Scenario: typed assertThrows returns typed error

• WHEN assertThrows<TestingError>(fn) is called and fn throws a TestingError
• THEN the inferred return type is TestingError AND the caller can read .code without a follow-up instanceof narrowing

Scenario: untyped assertThrows defaults to Error

• WHEN assertThrows(fn) is called without a type argument
• THEN the inferred return type is Error

Scenario: async variant matches sync contract

• WHEN assertThrowsAsync<TestingError>(asyncFn) is awaited
• THEN the inferred awaited type is TestingError

Verified by: src/helpers/test-helpers.spec-conformance.test.ts::"assertThrows<TError>() returns TError" AND src/helpers/test-helpers.spec-conformance.test.ts::"assertThrows() without explicit TError defaults to Error" AND src/helpers/test-helpers.spec-conformance.test.ts::"assertThrowsAsync<TError>() returns Promise<TError>" AND src/helpers/test-helpers.spec-conformance.test.ts::"assertThrowsAsync() without explicit TError defaults to Error"

Requirement: Central runner subpath

The package SHALL expose a ./runner subpath that re-exports the active test runner's primitives (describe, it, test, expect, expectTypeOf, vi, beforeAll/beforeEach/afterAll/afterEach) together with generic mock helpers (fn, spy, stub, mock, fakeClock, restoreMocks, restoreTimers) and runner-neutral types (RunnerMock<TArgs, TReturn>, SpyHandle). The subpath SHALL resolve to the vitest implementation by default and to bun:test under the bun package.json condition, so consumers swap runners without changing imports.

Scenario: vitest path re-exports primitives

• WHEN a consumer imports describe, it, expect, vi, expectTypeOf from @suaveplan/testing/runner under the default or vitest condition
• THEN the imports resolve to the corresponding vitest exports

Scenario: bun-test path re-exports primitives with expectTypeOf shim

• WHEN a consumer imports the same surface under the bun condition
• THEN describe/it/expect/vi resolve to bun:test equivalents and expectTypeOf resolves to a Proxy-backed no-op chain whose calls succeed (type assertions are erased at compile time)

Scenario: generic mock helpers preserve call signature

• WHEN a test calls fn<[string], number>() from the subpath
• THEN the returned RunnerMock<[string], number> is callable as (arg: string) => number under both the vitest and bun-test paths

Verified by: src/runner/mocks.test.ts::"creates a mock that records calls" AND src/runner/runner.spec-conformance.test.ts::"fn<[string], number>() preserves call-signature typing" AND src/runner/runner.spec-conformance.test.ts::"BunRunnerMock<TArgs, TReturn> shape mirrors the vitest path"