Initial Commit
This commit is contained in:
251
openspec/archive/2026-04-19-enterprise-testing/design.md
Normal file
251
openspec/archive/2026-04-19-enterprise-testing/design.md
Normal file
@@ -0,0 +1,251 @@
|
||||
## Context
|
||||
|
||||
`@suaveplan/testing` sits at the bottom of the genesis dependency graph. Every other `@suaveplan/*` package uses it to write unit/integration tests. The archived `2026-04-08-testing-libraries` change on main established the current shape — dual-runner conditional exports (vitest default / bun condition), sub-path exports for `./config` / `./utils` / `./polyfills`, full mock API, `createTestConfig` factory absorbing WSL2 pool + polyfill settings. That shape is load-bearing and must survive.
|
||||
|
||||
Constraints specific to genesis:
|
||||
|
||||
- **Rule #9** — no `@suaveplan/*` workspace deps. External catalog deps only.
|
||||
- **Layer 0** — nothing else on genesis has been built yet. No consumers to migrate. The audit phase is surface-preservation against main's source, not consumer churn.
|
||||
- **Version** — starts at `0.1.0`; stays there until graduation cut.
|
||||
- **Enterprise bar** — `enterprise/testing.md` §6 acceptance criteria is the gate for marking the package ✅ Enterprise.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Deliver every capability from `enterprise/testing.md` §3 and §4 within the genesis-rule constraints.
|
||||
- Preserve the full public surface inventoried below; no signature shrinkage.
|
||||
- Make the package a reference implementation for the enterprise checklist in `openspec/project.md` — so subsequent genesis packages can copy its patterns (error shape, Zod schema layout, OTEL wrapping, Clock/Rng/Signal DI, property tests, benchmarks).
|
||||
- Dual-runner condition (vitest default, bun-test as a peer) remains intact.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Introducing new user-facing APIs. The §4 "feature-rich" items (plugin interface on `FixtureManager`, `traits` on `Factory`) are additive to the existing shape, not replacements.
|
||||
- Supporting any runner beyond Vitest and bun-test.
|
||||
- Shipping a `@suaveplan/*` adapter for schemas / errors / OTEL — those integrations land when each matching package reaches enterprise on a later layer.
|
||||
- Re-routing public messages through i18n — English-only by design (rule parallel to utils D10).
|
||||
|
||||
## Surface Inventory (public exports that MUST survive)
|
||||
|
||||
From `/mnt/s/web/suaveplan/packages/universal/testing/src/` — every name below is preserved unchanged in type shape; internals are rewritten.
|
||||
|
||||
### Runner (`src/runner/`)
|
||||
|
||||
Conditional export routing via `package.json`:
|
||||
|
||||
- `vitest` condition → `src/runner/vitest.ts` (re-exports vitest primitives)
|
||||
- `bun` condition → `src/runner/bun-test.ts` (bun-test shim)
|
||||
- default → `src/runner/index.ts` (barrel for the vitest path)
|
||||
|
||||
Exports (from the default condition): `afterAll`, `afterEach`, `beforeAll`, `beforeEach`, `describe`, `expect`, `fakeClock`, `fn`, `it`, `mock`, `restoreMocks`, `restoreTimers`, `spy`, `stub`, `test`, `vi`.
|
||||
|
||||
### Builders (`src/builders/`)
|
||||
|
||||
- `RequestBuilder` class; quick helpers `get`, `post`, `put`, `patch`, `del`.
|
||||
- `ResponseBuilder` class; quick helpers `ok`, `created`, `noContent`, `badRequest`, `unauthorized`, `notFound`, `serverError`.
|
||||
- Types: `RequestBuilderPreset`, `RequestBuilderState`, `ResponseBuilderPreset`, `ResponseBuilderState`, `TestRequest`, `TestResponse`, `HttpMethod`, `HttpHeaders`, `QueryParams`.
|
||||
|
||||
### Clock (`src/clock/`)
|
||||
|
||||
- `FakeClock` class (implements the local `Clock` interface).
|
||||
- Types: `ClockSnapshot`, `FakeClockOptions`, `ScheduledTimer`.
|
||||
|
||||
### Factories (`src/factories/`)
|
||||
|
||||
- `defineFactory<T>(definition)`, `Factory<T>` class.
|
||||
- Types: `FactoryBuildOptions`, `FactoryDefaults`, `FactoryDefinition`, `BatchBuildResult`.
|
||||
|
||||
### Fixtures (`src/fixtures/`)
|
||||
|
||||
- `FixtureManager` class.
|
||||
- Types: `FixtureDefinition`, `FixtureState`, `SetupAllResult`, `TeardownAllResult`.
|
||||
|
||||
### Storage (`src/storage/`)
|
||||
|
||||
- `MemoryStore` class.
|
||||
- Types: `MemoryStoreEvent`, `MemoryStoreStats`, `MemoryStoreOptions`, `StoreEntry`.
|
||||
|
||||
### Helpers (`src/helpers/`)
|
||||
|
||||
- `assertThrows`, `assertThrowsAsync`, `createCollector`, `deferred`, `measureTime`, `resetTestIdCounter`, `sleep`, `uniqueTestId`, `waitFor`.
|
||||
- Types: `WaitForOptions`.
|
||||
|
||||
### Constants (`src/constants/`)
|
||||
|
||||
- `DEFAULT_CLOCK_OPTIONS`, `DEFAULT_HTTP_STATUS`, `DEFAULT_RESPONSE_LATENCY_MS`, `DEFAULT_STORE_OPTIONS`, `DEFAULT_WAIT_OPTIONS`, `MAX_FIXTURES`, `MAX_STORE_ENTRIES`, `MAX_TIMERS`.
|
||||
|
||||
### Config subpath (`@suaveplan/testing/config`)
|
||||
|
||||
- `createTestConfig(options?)` async factory.
|
||||
- Types: `TestConfigOptions`, `TestRunner`, `TestEnvironment`, `VitestConfigOverrides`, `BunTestConfig`, `BunTestConfigOverrides`.
|
||||
|
||||
### Utils subpath (`@suaveplan/testing/utils`)
|
||||
|
||||
Re-exports every non-runner symbol for callers that want a single import without pulling the runner surface.
|
||||
|
||||
### Polyfills subpath (`@suaveplan/testing/polyfills`)
|
||||
|
||||
Side-effecting module loaded by `setupFiles` in consumers' vitest configs. Installs whatever global shims (e.g. `crypto`) the happy-dom environment needs on WSL2.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: Local `Clock` and `Rng` interfaces in `src/context/`
|
||||
|
||||
```ts
|
||||
export interface Clock {
|
||||
now(): number; // ms since epoch
|
||||
hrtime(): bigint; // nanoseconds, monotonic
|
||||
}
|
||||
|
||||
export interface Rng {
|
||||
nextUint32(): number; // [0, 2^32)
|
||||
nextFloat(): number; // [0, 1)
|
||||
}
|
||||
|
||||
export const defaultClock: Clock;
|
||||
export const cryptoRng: Rng;
|
||||
export function seededRng(seed: number): Rng;
|
||||
```
|
||||
|
||||
Rationale: matches the shape `@suaveplan/datetime` will eventually expose, so a later genesis layer can ship an adapter without changing call sites. Keeps genesis rule #9 clean. `FakeClock` implements `Clock` so callers can swap seamlessly.
|
||||
|
||||
### D2: Local `TestingError` hierarchy
|
||||
|
||||
```ts
|
||||
export type TestingErrorCode =
|
||||
| "TESTING.CLOCK.MAX_TIMERS_EXCEEDED"
|
||||
| "TESTING.CLOCK.NEGATIVE_ADVANCE"
|
||||
| "TESTING.FIXTURES.SETUP_FAILED"
|
||||
| "TESTING.FIXTURES.TEARDOWN_FAILED"
|
||||
| "TESTING.FIXTURES.MAX_FIXTURES_EXCEEDED"
|
||||
| "TESTING.FACTORY.BUILD_FAILED"
|
||||
| "TESTING.FACTORY.TRAIT_NOT_FOUND"
|
||||
| "TESTING.STORE.CAPACITY_EXCEEDED"
|
||||
| "TESTING.STORE.KEY_NOT_FOUND"
|
||||
| "TESTING.HELPERS.WAIT_TIMEOUT"
|
||||
| "TESTING.HELPERS.ASSERTION_FAILED"
|
||||
| "TESTING.BUILDER.INVALID_STATE"
|
||||
| "TESTING.CONFIG.UNKNOWN_RUNNER"
|
||||
| "TESTING.ABORTED";
|
||||
|
||||
export class TestingError extends Error {
|
||||
readonly code: TestingErrorCode;
|
||||
readonly context?: Record<string, unknown>;
|
||||
constructor(code: TestingErrorCode, message: string, options?: {
|
||||
cause?: unknown;
|
||||
context?: Record<string, unknown>;
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Subclasses per domain (`ClockError`, `FixturesError`, `FactoryError`, `StoreError`, `HelpersError`, `BuilderError`, `ConfigError`) are thin wrappers narrowing the code union — callers can branch on `err instanceof ClockError`. All still satisfy `err instanceof TestingError`.
|
||||
|
||||
Rationale: shape-compatible with eventual `@suaveplan/error`; per-domain subclasses let TypeScript narrow the code; stable strings as machine contract.
|
||||
|
||||
### D3: Zod schemas in sibling `*.schemas.ts`
|
||||
|
||||
Every implementation file `foo/foo.ts` has a peer `foo/foo.schemas.ts` exporting input/output schemas under a stable namespace. Example:
|
||||
|
||||
```ts
|
||||
// src/builders/request-builder/request-builder.schemas.ts
|
||||
import z from "zod";
|
||||
|
||||
export const RequestBuilderSchemas = {
|
||||
preset: z.enum(["get", "post", "put", "patch", "delete"]),
|
||||
input: z.object({
|
||||
method: z.string(),
|
||||
url: z.string(),
|
||||
headers: z.record(z.string()).optional(),
|
||||
query: z.record(z.string()).optional(),
|
||||
body: z.unknown().optional(),
|
||||
}),
|
||||
output: z.object({ /* … */ }),
|
||||
} as const;
|
||||
```
|
||||
|
||||
Schemas are NOT pulled in by the functional entry points — they are consumed only when a caller opts in (`RequestBuilderSchemas.input.parse(…)`). Verified by the tree-shake test.
|
||||
|
||||
Rationale: respects the enterprise requirement (Zod on public types) without forcing runtime validation on every call. Stable names mean a future `@suaveplan/schema-registry` adapter can enumerate them without modifying this package.
|
||||
|
||||
### D4: OTEL via `@opentelemetry/api`, no-op by default
|
||||
|
||||
```ts
|
||||
// src/observability/tracer.ts
|
||||
import { trace, metrics, type Span, type Counter } from "@opentelemetry/api";
|
||||
|
||||
const tracer = trace.getTracer("@suaveplan/testing");
|
||||
const meter = metrics.getMeter("@suaveplan/testing");
|
||||
```
|
||||
|
||||
Every public entrypoint is wrapped via a local `withObservability(name, fn)` helper. When no SDK is registered (default for Vitest/Bun-test runs), `trace.getTracer()` returns a NoopTracer; overhead is a single function call and object allocation per invocation — immeasurable in practice.
|
||||
|
||||
Counters: `testing.<domain>.<op>.count`, `testing.<domain>.<op>.errors{code}`. Histograms: `testing.<domain>.<op>.duration_ms`. Spans: only for iterative operations (`waitFor`, `FixtureManager.setupAll`, `Factory.buildBatch`).
|
||||
|
||||
Rationale: `@opentelemetry/api` is the stable public spec — 2 KB, zero transitive Node deps, safe in browsers. Writing against it means a future `@suaveplan/opentelemetry` package that merely bundles an exporter + resource config will just work.
|
||||
|
||||
### D5: Dual-runner conditional exports preserved exactly
|
||||
|
||||
`package.json` retains:
|
||||
|
||||
```json
|
||||
"exports": {
|
||||
".": {
|
||||
"vitest": { "types": "./dist/index.d.ts", "default": "./dist/index.js" },
|
||||
"bun": { "types": "./dist/runner/bun-test.d.ts", "default": "./dist/runner/bun-test.js" },
|
||||
"default": { "types": "./dist/index.d.ts", "default": "./dist/index.js" }
|
||||
},
|
||||
"./config": { … },
|
||||
"./utils": { … },
|
||||
"./polyfills": "./dist/setup/polyfills.js",
|
||||
"./package.json": "./package.json"
|
||||
}
|
||||
```
|
||||
|
||||
Rationale: established shape from the archived change. Swapping a package from vitest to bun-test is a single-condition change with no test-source rewrites.
|
||||
|
||||
### D6: AbortSignal at bounded checkpoints
|
||||
|
||||
`waitFor` checks `signal?.aborted` each poll cycle. `FixtureManager.setupAll` / `teardownAll` check between fixtures. `Factory.buildBatch(n)` checks every 256 items. `MemoryStore` eviction sweeps check every 1024 entries. On abort, throws `TestingError("TESTING.ABORTED", …, { cause: signal.reason })`.
|
||||
|
||||
Rationale: deterministic cancellation with predictable cost. Enterprise doc §3 "Backpressure & cancellation".
|
||||
|
||||
### D7: Plugin interface on `FixtureManager` (feature-rich)
|
||||
|
||||
```ts
|
||||
export interface FixtureManagerPlugin {
|
||||
name: string;
|
||||
beforeEach?(ctx: { fixtureName: string; state: FixtureState }): void | Promise<void>;
|
||||
afterEach?(ctx: { fixtureName: string; state: FixtureState }): void | Promise<void>;
|
||||
onError?(ctx: { fixtureName: string; error: TestingError }): void | Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
Rationale: enterprise doc §4 "plugin / extension interface". Downstream genesis packages can contribute fixture-level concerns (e.g. database seeding for a `surrealdb` package) without forking `FixtureManager`.
|
||||
|
||||
### D8: Benchmarks in `src/__benchmarks__/<domain>.bench.ts`
|
||||
|
||||
Every domain has a bench file. `benchmarks/baselines.json` stores the baseline. CI runs benchmarks 3×, takes the fastest, compares to baseline, fails if any op regresses >10%. Baseline updates land in their own commit with reviewer approval.
|
||||
|
||||
Rationale: enterprise doc §4 "Performance". Baselines taken against the genesis implementation, not retrofit.
|
||||
|
||||
### D9: `createTestConfig()` absorbs WSL2 realities
|
||||
|
||||
The factory handles:
|
||||
|
||||
- `pool: "forks"` (vmThreads/threads fail on WSL2)
|
||||
- `setupFiles: ["@suaveplan/testing/polyfills"]`
|
||||
- `fileParallelism: false` for browser envs
|
||||
- React plugin loaded only when `environment === "happy-dom"` and `@vitejs/plugin-react` is installed (optional peer)
|
||||
- bun-test path when caller's `package.json` / env says so
|
||||
|
||||
Preserved exactly from the archived 2026-04-08 change.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **OTEL overhead on hot paths.** `trace.getTracer().startSpan()` in a noop tracer is ~5 ns — measurable in tight benchmarks. Mitigation: benchmark suite asserts overhead stays below 5% vs a no-OTEL baseline.
|
||||
- **Zod bundle cost on `@suaveplan/testing` root bundle.** Mitigation: schemas in peer `*.schemas.ts` files, not imported by functional entry points. Tree-shake test enforces.
|
||||
- **Local `TestingError` diverging from `@suaveplan/error`.** Mitigation: shape locked to `{ code, cause, context }` today. A future adapter is a one-file bridge.
|
||||
- **`@opentelemetry/api` version churn.** Mitigation: pin via catalog. The API spec itself has been stable across major versions of the SDK.
|
||||
- **fast-check timeouts on WSL2.** Mitigation: cap `numRuns: 100` in CI; larger runs nightly only. Generator sizes capped per-test.
|
||||
- **Benchmark flakiness in CI.** Mitigation: mitata stabilization (min samples, outlier trimming), 3-strike retry, 10% absolute floor.
|
||||
- **Node-only APIs sneaking in (fs/path/url in `createTestConfig`).** Mitigation: `createTestConfig` is explicitly a Node-side factory (runs in vitest's Node context), not a browser-runnable helper. Tree-shake test verifies that importing a browser-runnable helper (e.g. `FakeClock`) does NOT pull in `createTestConfig`'s Node deps.
|
||||
62
openspec/archive/2026-04-19-enterprise-testing/proposal.md
Normal file
62
openspec/archive/2026-04-19-enterprise-testing/proposal.md
Normal file
@@ -0,0 +1,62 @@
|
||||
## Why
|
||||
|
||||
`@suaveplan/testing` is the shared test infrastructure every other `@suaveplan/*` package imports from. As Layer 0 in the genesis dependency graph (zero `@suaveplan/*` workspace deps — rule #9 of `openspec/project.md`), every package downstream depends on it. Enterprise-quality testing unblocks enterprise quality everywhere else.
|
||||
|
||||
The enterprise specification at `/mnt/s/web/suaveplan/enterprise/testing.md` sets the bar; this change translates that spec into genesis form, **preserving the full public surface** established in the archived `2026-04-08-testing-libraries` change (RequestBuilder, ResponseBuilder, FakeClock, Factory, FixtureManager, MemoryStore, helpers, dual-runner conditional exports, `createTestConfig` — full inventory in `design.md`) while upgrading internals to enterprise grade.
|
||||
|
||||
This is the first implementation change on the `genesis` branch. There is no prior genesis code to audit — the "current state" is empty. Design intent comes from the enterprise doc; surface intent comes from the existing implementation on `main` at `/mnt/s/web/suaveplan/packages/universal/testing/src/`.
|
||||
|
||||
## What Changes
|
||||
|
||||
All requirements from `enterprise/testing.md` §3 are carried forward, with **four explicit substitutions** forced by genesis rule #9 (testing packages may not depend on any `@suaveplan/*` workspace package). Each substitution is called out inline below.
|
||||
|
||||
### Carried forward verbatim
|
||||
|
||||
- **Universal purity** — zero browser/Node/React deps in `src/`. Runs in Bun, Node, and modern browsers. Enforced by a tree-shake test that inspects the built bundle.
|
||||
- **Zod schemas** for all public function inputs and outputs, exported from sibling `*.schemas.ts` files alongside each implementation file. Zod is an **external catalog dep** (`catalog:forms`), not routed through `@suaveplan/schema-registry`.
|
||||
- **Property-based tests (fast-check)** for pure functions (clock arithmetic, builder composition, factory deterministic output given fixed RNG, memory-store TTL math, semver-ish helpers).
|
||||
- **Deterministic behaviour** — no `Math.random()` / `Date.now()` / `performance.now()` at module scope. Every function that reads time or randomness accepts an optional `{ clock?, rng?, signal? }` options parameter. Zero-config defaults preserved.
|
||||
- **Backpressure & cancellation** — `AbortSignal` on every iterative / async helper (`waitFor`, `FixtureManager.setupAll/teardownAll`, `Factory.build` in batch mode, `MemoryStore` eviction sweeps), checked at bounded intervals (≤ 256 items or ≤ 1 KB).
|
||||
- **Idempotency & retry primitives** wired for any helper with network-ish retry semantics (e.g. `waitFor` polling).
|
||||
- **Feature flags** — any risky new behaviour gated behind an options flag with safe defaults.
|
||||
- **Structured errors** with stable codes of the form `TESTING.<DOMAIN>.<REASON>` (e.g. `TESTING.CLOCK.MAX_TIMERS_EXCEEDED`, `TESTING.FIXTURES.SETUP_FAILED`). Every thrown error carries `{ code, cause?, context? }`.
|
||||
- **Benchmarks** via `mitata` (external catalog dep) under `src/__benchmarks__/<domain>.bench.ts`, baseline in `benchmarks/baselines.json`, CI gate at 10% regression floor.
|
||||
- **Plugin / extension interface** — `FixtureManager` accepts plugins implementing `beforeEach/afterEach/onError` hooks; `Factory` accepts `traits` that downstream packages can contribute without forking.
|
||||
- **Observability surface** — counters, histograms, spans for every public entrypoint. Gated: no-op when no OTEL SDK is registered; live when one is. Useful for diagnosing flaky test environments in CI without touching test source.
|
||||
- **Security** — input validation at the boundary via Zod, no prototype-pollution vectors in `MemoryStore`, dependency audit clean.
|
||||
- **Migration story** — N/A; genesis starts fresh at version `0.1.0`.
|
||||
- **Example / story** — `examples/` directory with runnable usage of each public entrypoint.
|
||||
|
||||
### Substitutions (enterprise-doc → genesis form)
|
||||
|
||||
| Enterprise doc requires | Genesis substitution | Why |
|
||||
|---|---|---|
|
||||
| Errors extend `@suaveplan/error` | Local `TestingError extends Error` with the same `{ code, cause, context }` shape, stable `TESTING.<DOMAIN>.<REASON>` codes | Rule #9 — can't take `@suaveplan/*` deps. Shape-compatible so a future `@suaveplan/error` adapter can bridge without API change. |
|
||||
| OTEL via `@suaveplan/opentelemetry` | `@opentelemetry/api` (external, ~2 KB, no transitive Node deps). Spans/counters/histograms created via the standard API, no-op without a registered SDK | Rule #9. `@opentelemetry/api` is the canonical public API spec; the internal package will wrap this at a later layer. |
|
||||
| Schemas via `@suaveplan/schema-registry` | `zod` direct (catalog:forms). Schemas exported from `*.schemas.ts` under stable names (e.g. `BuilderSchemas.request.input`) so a future `schema-registry` adapter can enumerate them | Rule #9. |
|
||||
| Clock via `@suaveplan/datetime` | Local `Clock` interface in `src/context/context.ts`: `{ now(): number; hrtime(): bigint }`. `defaultClock` reads `Date.now()` / `performance.now()` directly. `FakeClock` implements the same interface. | Rule #9. |
|
||||
| i18n via `@suaveplan/i18n` or hard-coded message keys | English-only `message` fields. `code` is the machine contract; `message` is developer-facing. Matches the utils D10 precedent. | Rule #9. Test helpers emit messages to developers in logs, not to end users. |
|
||||
|
||||
### Public surface preserved
|
||||
|
||||
Every export enumerated in `design.md` §Surface Inventory is retained unchanged in shape. Internals are rewritten; signatures gain the optional `{ clock?, rng?, signal? }` options parameter where appropriate (purely additive — zero-config callers are unaffected).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `testing`: shared test infrastructure for the genesis monorepo — test primitives (vitest + bun-test conditional export), mock API, request/response builders, FakeClock, Factory, FixtureManager, MemoryStore, async helpers, vitest config factory, polyfills. Layer 0 leaf.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
(none — genesis has no prior spec)
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected code:** `/mnt/s/web/genesis/packages/universal/testing/**` (fresh implementation).
|
||||
- **Affected consumers:** none on genesis yet; every subsequent genesis package depends on this one.
|
||||
- **Version:** `0.1.0` (genesis baseline; no bump until graduation cut).
|
||||
- **Catalogs added / extended in root `package.json`:** `zod` to `catalogs.forms`, `@opentelemetry/api` to a new `catalogs.observability`, `fast-check` and `mitata` to `catalogs.testing`.
|
||||
- **Root `package.json` `devDependencies` addition:** `@types/bun` already present; tooling catalogs unchanged.
|
||||
- **New capability spec:** `openspec/specs/testing/spec.md` (created on archive).
|
||||
- **Pre-publish gate:** full turbo `lint typecheck test` on `@suaveplan/testing` + istanbul coverage ≥ 95% before `bun scripts/publish.ts --package @suaveplan/testing`.
|
||||
@@ -0,0 +1,211 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Universal purity
|
||||
|
||||
The `@suaveplan/testing` package SHALL depend on zero browser-only, Node-only, or React APIs at the module-graph level for its core runtime exports. The `./config` subpath and `./polyfills` subpath MAY reference Node built-ins because they are explicitly Node-side and are never pulled transitively by browser-runnable exports.
|
||||
|
||||
#### Scenario: Browser-runnable exports avoid Node built-ins
|
||||
|
||||
- **WHEN** a browser target imports `FakeClock`, `RequestBuilder`, `ResponseBuilder`, `Factory`, `FixtureManager`, `MemoryStore`, or any helper
|
||||
- **THEN** the resolved module graph MUST NOT reference `node:fs`, `node:path`, `node:url`, `process`, `Buffer`, or any Node-only API
|
||||
|
||||
#### Scenario: Node-side subpaths are segregated
|
||||
|
||||
- **WHEN** a consumer imports `@suaveplan/testing/config` or `@suaveplan/testing/polyfills`
|
||||
- **THEN** those imports resolve to modules that are allowed to reference Node built-ins AND the core export (`@suaveplan/testing`) MUST NOT transitively re-import them
|
||||
|
||||
### Requirement: No `@suaveplan/*` workspace dependencies
|
||||
|
||||
The `@suaveplan/testing` package `dependencies`, `peerDependencies`, and `devDependencies` MUST NOT list any `@suaveplan/*` scoped package. External catalog deps only (vitest, happy-dom, ws, tslib, zod, `@opentelemetry/api`, fast-check, mitata, `@types/*`, build tooling).
|
||||
|
||||
#### Scenario: Dep-graph guard
|
||||
|
||||
- **WHEN** `bunx turbo ls --filter=@suaveplan/testing...` is run
|
||||
- **THEN** no `@suaveplan/*` package appears in the forward closure of `@suaveplan/testing`
|
||||
|
||||
### Requirement: Public types validated via Zod
|
||||
|
||||
Every public function's input and output types SHALL be modeled by a Zod schema exported from a sibling `<module>.schemas.ts` file under a stable namespace (e.g. `RequestBuilderSchemas.input`). Schemas MUST NOT be imported by functional entry points; consumers opt in by importing `*.schemas.ts` directly.
|
||||
|
||||
#### Scenario: Runtime validation available on opt-in
|
||||
|
||||
- **WHEN** a consumer imports `FactorySchemas.buildOptions` from `@suaveplan/testing/factories/factory/factory.schemas` and calls `.parse(value)`
|
||||
- **THEN** invalid input is rejected with a `ZodError` whose `issues[].path` points to the offending field
|
||||
|
||||
#### Scenario: Schemas do not bloat the functional bundle
|
||||
|
||||
- **WHEN** a build imports only `Factory` from `@suaveplan/testing`
|
||||
- **THEN** the resolved bundle MUST NOT include Zod or any `*.schemas.ts` module
|
||||
|
||||
### Requirement: Structured error hierarchy
|
||||
|
||||
The package SHALL throw only subclasses of `TestingError extends Error` with a stable `code` of the form `TESTING.<DOMAIN>.<REASON>` (plain uppercase ASCII, dots as separators). Errors MUST carry `{ code, cause?, context? }` fields. The package MUST NOT throw plain `Error` or `TypeError` from any public code path.
|
||||
|
||||
#### Scenario: Timer overflow throws a coded error
|
||||
|
||||
- **WHEN** `FakeClock` is asked to schedule more than `MAX_TIMERS` timers
|
||||
- **THEN** it throws `TestingError` with `code === "TESTING.CLOCK.MAX_TIMERS_EXCEEDED"` AND `context.limit === MAX_TIMERS`
|
||||
|
||||
#### Scenario: Fixture setup failure preserves cause
|
||||
|
||||
- **WHEN** a fixture's `setup` function throws
|
||||
- **THEN** `FixtureManager.setupAll` throws `TestingError` with `code === "TESTING.FIXTURES.SETUP_FAILED"` AND `cause` is the original thrown value AND `context.fixtureName` is the failing fixture's name
|
||||
|
||||
#### Scenario: Abort signal triggered mid-wait
|
||||
|
||||
- **WHEN** `waitFor(predicate, { signal })` runs with a signal that aborts during polling
|
||||
- **THEN** the returned promise rejects with `TestingError` where `code === "TESTING.ABORTED"` AND `cause === signal.reason`
|
||||
|
||||
### Requirement: Deterministic clock and RNG via dependency injection
|
||||
|
||||
Every public function that reads time, randomness, or iterates SHALL accept an optional `{ clock?: Clock; rng?: Rng; signal?: AbortSignal }` options parameter. Zero-config callers MUST get behaviorally stable defaults: `defaultClock` (backed by `Date.now` / `performance.now`), `cryptoRng` (backed by `crypto.getRandomValues`), and no abort.
|
||||
|
||||
#### Scenario: Injected clock controls observed time
|
||||
|
||||
- **WHEN** `waitFor(predicate, { clock: fakeClock })` runs and `fakeClock.advance(1000)` is called
|
||||
- **THEN** the helper observes 1000 ms elapsed AND does NOT read `Date.now()` or `performance.now()` directly
|
||||
|
||||
#### Scenario: Injected RNG produces deterministic factory output
|
||||
|
||||
- **WHEN** `Factory.build({ rng: seededRng(42) })` is called twice with the same factory definition
|
||||
- **THEN** both calls return byte-identical output (for factories whose definition is otherwise deterministic)
|
||||
|
||||
#### Scenario: Module-scope non-determinism absent
|
||||
|
||||
- **WHEN** the built `dist/` is grep'd for `Math.random(` or `Date.now(` or `performance.now(`
|
||||
- **THEN** those symbols appear only inside the `defaultClock` / `cryptoRng` module (and `FakeClock`'s internal simulation), never at module scope in other files
|
||||
|
||||
### Requirement: Cooperative cancellation via AbortSignal
|
||||
|
||||
Every iterative or async helper SHALL check its `signal?.aborted` property at bounded intervals (≤ 256 items or ≤ 1 KB scanned per check) and throw `TestingError("TESTING.ABORTED", …)` on first observed abort without returning partial results.
|
||||
|
||||
#### Scenario: Batch factory aborts within one tick
|
||||
|
||||
- **WHEN** `Factory.buildBatch(10000, { signal })` is running and `signal.abort()` is called after 200 ms
|
||||
- **THEN** the promise rejects with `TESTING.ABORTED` within one event-loop tick of the abort
|
||||
|
||||
#### Scenario: Fixture teardown aborts between fixtures, never mid-fixture
|
||||
|
||||
- **WHEN** `FixtureManager.teardownAll({ signal })` is mid-teardown and the signal fires
|
||||
- **THEN** the current fixture's teardown completes AND the next fixture is NOT started AND the promise rejects with `TESTING.ABORTED`
|
||||
|
||||
### Requirement: Dual-runner conditional exports
|
||||
|
||||
The package `exports` map SHALL route `@suaveplan/testing` via a `vitest` condition to the vitest-backed barrel, a `bun` condition to the bun-test shim, and a `default` condition to the vitest barrel. The vitest barrel MUST re-export `describe`, `it`, `test`, `expect`, `afterAll`, `afterEach`, `beforeAll`, `beforeEach`, `vi`, `fn`, `mock`, `spy`, `stub`, `fakeClock`, `restoreMocks`, `restoreTimers`. The bun-test shim MUST expose the same names via bun-test primitives.
|
||||
|
||||
#### Scenario: Vitest consumer resolves vitest primitives
|
||||
|
||||
- **WHEN** a package with `vitest` in its test runner configuration imports `{ describe, it, expect }` from `@suaveplan/testing`
|
||||
- **THEN** resolution MUST pick the `vitest` condition and the imported symbols MUST be vitest's
|
||||
|
||||
#### Scenario: Bun-test consumer resolves bun-test primitives
|
||||
|
||||
- **WHEN** a package configured for `bun test` imports `{ describe, it, expect }` from `@suaveplan/testing`
|
||||
- **THEN** resolution MUST pick the `bun` condition and the imported symbols MUST come from bun's test runner
|
||||
|
||||
### Requirement: Observability via `@opentelemetry/api`
|
||||
|
||||
Every public entrypoint SHALL be instrumented with an OTEL counter (`testing.<domain>.<op>.count`), a histogram (`testing.<domain>.<op>.duration_ms`), and an error counter (`testing.<domain>.<op>.errors` with `code` attribute on error). Spans SHALL be created only for iterative operations (`waitFor`, `FixtureManager.setupAll`/`teardownAll`, `Factory.buildBatch`). When no OTEL SDK is registered, all instrumentation MUST be a no-op at runtime.
|
||||
|
||||
#### Scenario: Counter increments on every call
|
||||
|
||||
- **WHEN** `RequestBuilder` methods are called N times during a test run with a registered OTEL SDK
|
||||
- **THEN** the metric `testing.builders.request.count` has recorded N increments
|
||||
|
||||
#### Scenario: Error counter records code attribute
|
||||
|
||||
- **WHEN** `FakeClock.schedule` throws `TESTING.CLOCK.MAX_TIMERS_EXCEEDED`
|
||||
- **THEN** the metric `testing.clock.schedule.errors` records an increment with attribute `code = "TESTING.CLOCK.MAX_TIMERS_EXCEEDED"`
|
||||
|
||||
#### Scenario: Zero overhead without SDK
|
||||
|
||||
- **WHEN** the benchmark suite runs without any OTEL SDK registered
|
||||
- **THEN** the difference between the OTEL-wrapped public entrypoint and an identical unwrapped baseline MUST be less than 5% per op
|
||||
|
||||
### Requirement: Coverage and docs gates
|
||||
|
||||
Istanbul statement + branch coverage across `packages/universal/testing/src/` (excluding `*.bench.ts` and `*.schemas.ts`) MUST be ≥ 95%. Every implementation file (`*.ts` other than `*.test.ts`, `*.bench.ts`, `*.schemas.ts`, inner-dir `*.types.ts`, barrel `index.ts`) MUST have a sibling `.md` file of ≥ 200 words (excluding code fences) with a simple and an advanced usage example.
|
||||
|
||||
#### Scenario: Coverage gate in CI
|
||||
|
||||
- **WHEN** `bunx turbo test:coverage --filter=@suaveplan/testing` runs
|
||||
- **THEN** reported statement coverage is ≥ 95% AND branch coverage is ≥ 95%
|
||||
|
||||
#### Scenario: Docs coverage enforced
|
||||
|
||||
- **WHEN** a script enumerates implementation files under `src/` (excluding the stated exclusions) and checks for sibling `.md` files
|
||||
- **THEN** every implementation file has a sibling `.md` AND every `.md` is ≥ 200 non-code words
|
||||
|
||||
### Requirement: No disallowed constructs in `src/`
|
||||
|
||||
Files under `packages/universal/testing/src/` MUST NOT contain the `any` keyword, `@ts-ignore`, `@ts-expect-error`, `TODO`, `FIXME`, `@deprecated`, `it.skip`, `describe.skip`, `test.skip`, `.only`, or any mock/stub of runtime behaviour intended for production code paths.
|
||||
|
||||
#### Scenario: Biome scan clean
|
||||
|
||||
- **WHEN** `bunx biome check src/` runs with the project's strict config
|
||||
- **THEN** the scan reports zero `noExplicitAny` diagnostics AND zero suppressed directives
|
||||
|
||||
#### Scenario: Grep guard
|
||||
|
||||
- **WHEN** `grep -rnE "TODO|FIXME|@deprecated|\\.skip\\(|\\.only\\(|ts-ignore|ts-expect-error" src/` runs
|
||||
- **THEN** zero matches are produced
|
||||
|
||||
### Requirement: `sideEffects: false` and subpath exports
|
||||
|
||||
The package MUST declare `"sideEffects": false` in `package.json` AND provide explicit subpath exports for every module intended as a distinct import surface. A tree-shake test MUST prove that importing one submodule does not pull unrelated submodules.
|
||||
|
||||
#### Scenario: Tree-shake test passes
|
||||
|
||||
- **WHEN** a test imports only `FakeClock` from `@suaveplan/testing` and reads the resolved bundle text
|
||||
- **THEN** the bundle MUST NOT contain identifiers unique to `RequestBuilder`, `MemoryStore`, `FixtureManager`, or `Factory`
|
||||
|
||||
### Requirement: Property-based coverage for pure helpers
|
||||
|
||||
Pure functions and deterministic state machines in the package (clock arithmetic, builder state transitions, factory deterministic output, memory-store TTL expiry, wait-for polling) SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants as applicable.
|
||||
|
||||
#### Scenario: Builder composition is associative
|
||||
|
||||
- **WHEN** fast-check runs over arbitrary RequestBuilder presets `a`, `b`, `c` applied in different groupings
|
||||
- **THEN** `(a then b) then c` produces the same final state as `a then (b then c)`
|
||||
|
||||
#### Scenario: FakeClock advance + schedule commutes within a tick
|
||||
|
||||
- **WHEN** fast-check runs over sequences of `advance` and `schedule` calls that never cross a timer threshold
|
||||
- **THEN** reordering within a single synchronous batch produces the same observed state
|
||||
|
||||
### Requirement: Benchmark gate in CI
|
||||
|
||||
A `mitata` benchmark suite under `src/__benchmarks__/` SHALL record baselines in `benchmarks/baselines.json`. CI SHALL rerun the benchmarks (3-strike, take fastest), compare to baselines, and fail if any op regresses by more than 10%. Baseline updates MUST land on a dedicated, reviewer-approved commit separate from functional changes.
|
||||
|
||||
#### Scenario: Regression gate trips
|
||||
|
||||
- **WHEN** a PR introduces a 15% slowdown on `Factory.build`
|
||||
- **THEN** the CI benchmark step fails the build
|
||||
|
||||
#### Scenario: Baseline update is intentional
|
||||
|
||||
- **WHEN** a maintainer intentionally updates `benchmarks/baselines.json`
|
||||
- **THEN** the baseline commit is separate from functional code changes and is reviewed before landing
|
||||
|
||||
### Requirement: Plugin surface on FixtureManager
|
||||
|
||||
`FixtureManager` SHALL accept plugins implementing `{ name: string; beforeEach?; afterEach?; onError? }` and invoke their hooks around each fixture's own setup/teardown lifecycle. Plugins MUST NOT be able to cancel a teardown mid-flight except via `onError`.
|
||||
|
||||
#### Scenario: Plugin hook fires around each fixture
|
||||
|
||||
- **WHEN** a `FixtureManager` with a plugin registered runs `setupAll` over two fixtures
|
||||
- **THEN** the plugin's `beforeEach` is called exactly twice in fixture-registration order AND `afterEach` is called exactly twice in reverse order
|
||||
|
||||
### Requirement: `createTestConfig()` absorbs WSL2 pool and polyfill concerns
|
||||
|
||||
The `@suaveplan/testing/config` factory SHALL produce a Vitest config with `pool: "forks"`, `setupFiles: ["@suaveplan/testing/polyfills"]`, and — when the caller requests `happy-dom` — `fileParallelism: false` AND dynamically-loaded `@vitejs/plugin-react` if that optional peer is installed. When the caller's environment requests bun-test, the factory SHALL return an equivalent bun-test config instead.
|
||||
|
||||
#### Scenario: Happy-dom caller gets React plugin
|
||||
|
||||
- **WHEN** `createTestConfig({ environment: "happy-dom" })` is awaited and `@vitejs/plugin-react` is installed
|
||||
- **THEN** the returned config includes the React plugin in its `plugins` array AND `pool === "forks"` AND `setupFiles` includes the polyfills path
|
||||
|
||||
#### Scenario: bun-test caller gets bun config
|
||||
|
||||
- **WHEN** `createTestConfig({ runner: "bun:test" })` is awaited
|
||||
- **THEN** the returned value is a `BunTestConfig` with `pool` not set (bun-test has no concept of pool) AND any warnings about missing preload hooks are logged once
|
||||
152
openspec/archive/2026-04-19-enterprise-testing/surface.md
Normal file
152
openspec/archive/2026-04-19-enterprise-testing/surface.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# `@suaveplan/testing` — Surface Inventory
|
||||
|
||||
Preservation contract. Every name below is reproduced unchanged in shape by the genesis rewrite. Signatures gain the optional `{ clock?, rng?, signal? }` options parameter where appropriate (purely additive — zero-config callers are unaffected).
|
||||
|
||||
Source: `/mnt/s/web/suaveplan/packages/universal/testing/src/` at branch `main` head `ebff1a14d`.
|
||||
|
||||
## Root barrel (`@suaveplan/testing`)
|
||||
|
||||
### Runner primitives (via conditional exports)
|
||||
|
||||
| Name | Kind | Notes |
|
||||
|---|---|---|
|
||||
| `afterAll`, `afterEach`, `beforeAll`, `beforeEach` | function | Lifecycle hooks |
|
||||
| `describe` | function | Group tests |
|
||||
| `it`, `test` | function | Declare a test |
|
||||
| `expect` | function | Assertion entry |
|
||||
| `vi` | namespace | Vitest mock namespace (vitest path only) |
|
||||
| `fn` | `<TArgs extends unknown[], TReturn>(implementation?: (...args: TArgs) => TReturn) => Mock` | Runner-agnostic mock fn |
|
||||
| `mock` | `(modulePath: string, factory?: () => unknown) => { restore(): void }` | Module mock (vitest path only) |
|
||||
| `spy` | `<T, M extends keyof T>(obj: T, method: M & string)` | Method spy |
|
||||
| `stub` | `<T, M extends keyof T>(obj: T, method: M & string, impl?: (...args: unknown[]) => unknown)` | Method stub with default implementation |
|
||||
| `fakeClock` | `() => { advance(ms: number): void; restore(): void }` | Vitest fake-timer wrapper |
|
||||
| `restoreMocks` | `() => void` | |
|
||||
| `restoreTimers` | `() => void` | |
|
||||
|
||||
Routing: `package.json` conditional exports pick vitest (`default`), bun-test (`bun` condition), or the vitest barrel (`vitest` condition) per the archived `2026-04-08-testing-libraries` change.
|
||||
|
||||
### Builders
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `RequestBuilder` | class |
|
||||
| `ResponseBuilder` | class |
|
||||
| `get(url: string): RequestBuilder` | function (preset) |
|
||||
| `post(url: string, body?: unknown): RequestBuilder` | function (preset) |
|
||||
| `put(url: string, body?: unknown): RequestBuilder` | function (preset) |
|
||||
| `patch(url: string, body?: unknown): RequestBuilder` | function (preset) |
|
||||
| `del(url: string): RequestBuilder` | function (preset, `delete` keyword avoidance) |
|
||||
| `ok(body?: unknown): ResponseBuilder` | function (preset 200) |
|
||||
| `created(body?: unknown): ResponseBuilder` | function (preset 201) |
|
||||
| `noContent(): ResponseBuilder` | function (preset 204) |
|
||||
| `badRequest(body?: unknown): ResponseBuilder` | function (preset 400) |
|
||||
| `unauthorized(body?: unknown): ResponseBuilder` | function (preset 401) |
|
||||
| `notFound(body?: unknown): ResponseBuilder` | function (preset 404) |
|
||||
| `serverError(body?: unknown): ResponseBuilder` | function (preset 500) |
|
||||
|
||||
### Clock
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `FakeClock` | class |
|
||||
|
||||
### Factories
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `defineFactory<T>(definition: FactoryDefinition<T>): Factory<T>` | function |
|
||||
| `Factory<T>` | class |
|
||||
|
||||
### Fixtures
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `FixtureManager` | class |
|
||||
|
||||
### Storage
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `MemoryStore` | class |
|
||||
|
||||
### Helpers
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `assertThrows(fn: () => void, matcher?: string \| RegExp \| ErrorConstructor): Error` | function |
|
||||
| `assertThrowsAsync(fn: () => Promise<unknown>, matcher?: string \| RegExp \| ErrorConstructor): Promise<Error>` | function |
|
||||
| `createCollector<T>(): { push(value: T): void; items(): readonly T[]; clear(): void }` | function |
|
||||
| `deferred<T>(): { promise: Promise<T>; resolve(value: T): void; reject(reason: unknown): void }` | function |
|
||||
| `measureTime<T>(fn: () => T \| Promise<T>): Promise<{ result: T; elapsedMs: number }>` | function |
|
||||
| `resetTestIdCounter(): void` | function |
|
||||
| `sleep(ms: number): Promise<void>` | function |
|
||||
| `uniqueTestId(prefix?: string): string` | function |
|
||||
| `waitFor(predicate: () => boolean \| Promise<boolean>, options?: WaitForOptions): Promise<void>` | function |
|
||||
|
||||
### Constants
|
||||
|
||||
| Name | Kind | Source |
|
||||
|---|---|---|
|
||||
| `DEFAULT_CLOCK_OPTIONS` | `Required<FakeClockOptions>` | `src/constants/` |
|
||||
| `DEFAULT_HTTP_STATUS` | `number` (200) | |
|
||||
| `DEFAULT_RESPONSE_LATENCY_MS` | `number` (0) | |
|
||||
| `DEFAULT_STORE_OPTIONS` | `Required<MemoryStoreOptions>` | |
|
||||
| `DEFAULT_WAIT_OPTIONS` | `Required<WaitForOptions>` | |
|
||||
| `MAX_FIXTURES` | `number` (100) | |
|
||||
| `MAX_STORE_ENTRIES` | `number` (1,000,000) | |
|
||||
| `MAX_TIMERS` | `number` (10,000) | |
|
||||
|
||||
### Public types
|
||||
|
||||
| Name | Kind | Definition location |
|
||||
|---|---|---|
|
||||
| `HttpMethod` | `"GET" \| "POST" \| "PUT" \| "PATCH" \| "DELETE" \| "HEAD" \| "OPTIONS"` | `src/types/testing.types.ts` |
|
||||
| `HttpHeaders` | `Record<string, string>` | |
|
||||
| `QueryParams` | `Record<string, string \| string[]>` | |
|
||||
| `TestRequest` | interface: `{ method, url, headers, query, body, params }` | |
|
||||
| `TestResponse` | interface: `{ status, headers, body, latencyMs }` | |
|
||||
| `FakeClockOptions` | interface: `{ startTime?, autoAdvance?, autoAdvanceMs? }` | |
|
||||
| `ScheduledTimer` | interface: `{ id, callback, fireAt, interval, type }` | |
|
||||
| `MemoryStoreOptions` | interface: `{ maxEntries?, defaultTtlMs? }` | |
|
||||
| `StoreEntry<T>` | interface: `{ value: T, createdAt, expiresAt }` | |
|
||||
| `FactoryDefinition<T>` | interface: `{ defaults, traits?, afterCreate? }` | |
|
||||
| `FactoryDefaults<T>` | mapped type | |
|
||||
| `FactoryBuildOptions<T>` | interface: `{ overrides?, traits?, index? }` | |
|
||||
| `FixtureDefinition<T>` | interface: `{ name, setup, teardown?, dependencies? }` | |
|
||||
| `FixtureState<T>` | interface: `{ name, value, isSetup, setupCount, teardownCount }` | |
|
||||
| `WaitForOptions` | interface: `{ timeoutMs?, intervalMs? }` | |
|
||||
| `ClockSnapshot` | feature-local | `src/clock/fake-clock/fake-clock.types.ts` |
|
||||
| `RequestBuilderPreset`, `RequestBuilderState` | feature-local | `src/builders/request-builder/` |
|
||||
| `ResponseBuilderPreset`, `ResponseBuilderState` | feature-local | `src/builders/response-builder/` |
|
||||
| `BatchBuildResult` | feature-local | `src/factories/factory/` |
|
||||
| `SetupAllResult`, `TeardownAllResult` | feature-local | `src/fixtures/fixture-manager/` |
|
||||
| `MemoryStoreEvent`, `MemoryStoreStats` | feature-local | `src/storage/memory-store/` |
|
||||
|
||||
## Subpath `./config` (`@suaveplan/testing/config`)
|
||||
|
||||
| Name | Kind |
|
||||
|---|---|
|
||||
| `createTestConfig(options?: TestConfigOptions): Promise<VitestUserConfig \| BunTestConfig>` | async factory |
|
||||
| `TestConfigOptions` | interface extending `VitestConfigOverrides` with `runner?: TestRunner` |
|
||||
| `TestRunner` | `"vitest" \| "bun:test"` |
|
||||
| `TestEnvironment` | `"node" \| "happy-dom" \| "jsdom"` |
|
||||
| `VitestConfigOverrides` | interface (passes through to vitest's config) |
|
||||
| `BunTestConfig` | interface (minimal bun-test config shape) |
|
||||
| `BunTestConfigOverrides` | interface (partial of the above) |
|
||||
|
||||
## Subpath `./utils` (`@suaveplan/testing/utils`)
|
||||
|
||||
Re-exports every non-runner symbol above for single-import convenience (`import { FakeClock, Factory, … } from "@suaveplan/testing/utils"`).
|
||||
|
||||
## Subpath `./polyfills` (`@suaveplan/testing/polyfills`)
|
||||
|
||||
Side-effecting module (`import "@suaveplan/testing/polyfills";`) loaded by `setupFiles` in consumers' vitest configs. Installs happy-dom-needed globals (e.g. `crypto.getRandomValues` polyfill, whatever else surfaces as needed on WSL2).
|
||||
|
||||
## Non-obvious constraints (from the archived 2026-04-08-testing-libraries change)
|
||||
|
||||
- **Dual-runner exports are load-bearing.** Swapping a package between vitest and bun-test is a single condition change with no test-source rewrite. The vitest path and the bun-test path expose the same names.
|
||||
- **`@vitejs/plugin-react` is an optional peer dep.** `createTestConfig` dynamically imports it only when the caller requests `environment: "happy-dom"` AND the peer is installed. Missing peer must NOT break non-React consumers.
|
||||
- **`pool: "forks"` everywhere.** `vmThreads` and `threads` both fail in Bun on WSL2. Both the vitest config factory and any direct config consumers must force `forks`.
|
||||
- **`fileParallelism: false`** required for the happy-dom path (20+ concurrent forks time out on WSL2).
|
||||
- **Polyfills import is a side-effect module**, loaded via `setupFiles: ["@suaveplan/testing/polyfills"]`. Must export nothing; merely installing globals.
|
||||
- **`mocks.ts` uses vitest's `vi` directly.** On the bun-test path, the `mocks.ts` shim is replaced by bun-test equivalents. Callers see the same function names either way.
|
||||
115
openspec/archive/2026-04-19-enterprise-testing/tasks.md
Normal file
115
openspec/archive/2026-04-19-enterprise-testing/tasks.md
Normal file
@@ -0,0 +1,115 @@
|
||||
> Archived 2026-04-19: all tasks retroactively ticked; package shipped under universal/dev/testing.
|
||||
|
||||
## 1. Surface inventory & catalog setup
|
||||
|
||||
- [x] 1.1 Read `/mnt/s/web/suaveplan/packages/universal/testing/src/` top-to-bottom; produce `openspec/changes/enterprise-testing/surface.md` enumerating every public export grouped by subdirectory, with current signature. This is the preservation contract for the rewrite.
|
||||
- [x] 1.2 Read the archived `/mnt/s/web/suaveplan/openspec/archive/2026-04-08-testing-libraries/` change notes; capture any non-obvious constraints the current shape assumes (dual-runner condition routing, `createTestConfig` WSL2 behaviours, optional React plugin peer).
|
||||
- [x] 1.3 Extend root `package.json` catalogs: add `zod@^4.1.5` to `catalogs.forms`, create `catalogs.observability` with `@opentelemetry/api@^1.9.0`, add `fast-check@^3.23.2` and `mitata@^1.0.34` to `catalogs.testing`. Run `bun install`, confirm lockfile updates cleanly.
|
||||
|
||||
## 2. Package skeleton
|
||||
|
||||
- [x] 2.1 Create `packages/universal/testing/` with `package.json` (version `0.1.0`, `type: "module"`, `sideEffects: false`, dual-runner `exports` map from design D5, external catalog deps only per spec "No `@suaveplan/*` workspace dependencies")
|
||||
- [x] 2.2 Create `tsconfig.lib.json` extending `../../../tsconfig.base.json` with `outDir: ./dist` and `include: ["src"]`
|
||||
- [x] 2.3 Create `vite.config.ts` producing `preserveModules: true` ESM output so every source file maps to a dist file (required for subpath exports to resolve)
|
||||
- [x] 2.4 Create `vitest.config.ts` using a minimal inline config (cannot self-reference `createTestConfig` yet — this is the package that defines it)
|
||||
- [x] 2.5 Create `README.md` (root; no category READMEs per project rule) with purpose, install, quick usage, links to each module's co-located `.md`
|
||||
- [x] 2.6 Add `@suaveplan/testing` row to `CLAUDE.md` package listing (when the section is re-populated) and to `STRUCTURE.yml` (when the file exists)
|
||||
|
||||
## 3. Foundational primitives (context, errors, observability)
|
||||
|
||||
- [x] 3.1 Implement `src/context/context.ts`: `Clock` and `Rng` interfaces; `defaultClock` (reads `Date.now` / `performance.now`); `cryptoRng` (reads `crypto.getRandomValues`); `seededRng(seed)` (xorshift or similar deterministic)
|
||||
- [x] 3.2 Co-locate `src/context/context.test.ts` (deterministic tests for `seededRng` reproducibility; `defaultClock` monotonicity) and `src/context/context.md` (≥ 200 words, simple + advanced examples)
|
||||
- [x] 3.3 Co-locate `src/context/context.schemas.ts` with Zod schemas for `Clock` / `Rng` shapes (for callers who want to runtime-check they've passed a compatible object)
|
||||
- [x] 3.4 Implement `src/errors/testing-error.ts`: `TestingError` base class + per-domain subclasses (`ClockError`, `FixturesError`, `FactoryError`, `StoreError`, `HelpersError`, `BuilderError`, `ConfigError`) as per design D2
|
||||
- [x] 3.5 Implement `src/errors/codes.ts`: string-literal-union constants for every `TESTING.<DOMAIN>.<REASON>` code used in the package
|
||||
- [x] 3.6 Co-locate tests (`instanceof` branch coverage, `cause` / `context` round-trip) + `.md` docs + `*.schemas.ts`
|
||||
- [x] 3.7 Implement `src/observability/tracer.ts` and `src/observability/with-observability.ts` per design D4: counters, histograms, spans via `@opentelemetry/api`; span-only-for-iterative gate
|
||||
- [x] 3.8 Co-locate tests (no-SDK overhead, counter increment, error-counter attribute) + `.md` docs
|
||||
|
||||
## 4. Public-surface modules (implementation, tests, docs, schemas)
|
||||
|
||||
Repeat the pattern for every module — implementation, test, schemas, md — preserving the public surface from §1.1:
|
||||
|
||||
- [x] 4.1 `src/clock/fake-clock/*` — `FakeClock`, `ClockSnapshot`, `FakeClockOptions`; accepts `{ clock?, rng?, signal? }`; throws `ClockError("TESTING.CLOCK.*", …)`; implements `Clock` interface
|
||||
- [x] 4.2 `src/builders/request-builder/*` — `RequestBuilder`, `get`/`post`/`put`/`patch`/`del`; state-machine transitions throw `BuilderError` on invalid state
|
||||
- [x] 4.3 `src/builders/response-builder/*` — `ResponseBuilder`, `ok`/`created`/`noContent`/`badRequest`/`unauthorized`/`notFound`/`serverError`
|
||||
- [x] 4.4 `src/factories/factory/*` — `defineFactory`, `Factory`, `BatchBuildResult`, `FactoryBuildOptions`, `FactoryDefaults`, `FactoryDefinition`; `traits` plugin surface per design D7's factory analogue; `buildBatch(n, { signal })` checks abort every 256 items
|
||||
- [x] 4.5 `src/fixtures/fixture-manager/*` — `FixtureManager`, `SetupAllResult`, `TeardownAllResult`, `FixtureDefinition`, `FixtureState`, `FixtureManagerPlugin`; `setupAll`/`teardownAll` accept `{ signal }`; plugin hooks fire around each fixture
|
||||
- [x] 4.6 `src/storage/memory-store/*` — `MemoryStore`, `MemoryStoreEvent`, `MemoryStoreStats`, `MemoryStoreOptions`, `StoreEntry`; TTL expiry via injected clock; eviction sweeps honour `signal`
|
||||
- [x] 4.7 `src/helpers/test-helpers.ts` — `assertThrows`, `assertThrowsAsync`, `createCollector`, `deferred`, `measureTime`, `resetTestIdCounter`, `sleep`, `uniqueTestId`, `waitFor`; `waitFor` checks `signal.aborted` every poll; all reads of time/randomness routed through context
|
||||
- [x] 4.8 `src/constants/constants.ts` — `DEFAULT_CLOCK_OPTIONS`, `DEFAULT_HTTP_STATUS`, `DEFAULT_RESPONSE_LATENCY_MS`, `DEFAULT_STORE_OPTIONS`, `DEFAULT_WAIT_OPTIONS`, `MAX_FIXTURES`, `MAX_STORE_ENTRIES`, `MAX_TIMERS`
|
||||
- [x] 4.9 `src/types/testing.types.ts` — shared public types; co-locate `testing.types.md` (types/ subdirectory requires `.md` per project convention)
|
||||
|
||||
## 5. Runner and config subpaths
|
||||
|
||||
- [x] 5.1 `src/runner/vitest.ts` — re-export vitest primitives under the `default` condition
|
||||
- [x] 5.2 `src/runner/bun-test.ts` — bun-test shim exposing the same surface under the `bun` condition
|
||||
- [x] 5.3 `src/runner/mocks.ts` — `spy`, `stub`, `fn`, `mock`, `fakeClock`, `restoreMocks`, `restoreTimers` backed by the host runner
|
||||
- [x] 5.4 `src/runner/index.ts` — combined barrel for the default vitest path
|
||||
- [x] 5.5 `src/config/index.ts` — `createTestConfig()` factory per design D9
|
||||
- [x] 5.6 `src/config/vitest-config.ts` — vitest config builder (pool: forks, polyfills, fileParallelism)
|
||||
- [x] 5.7 `src/config/bun-test-config.ts` — bun-test config builder
|
||||
- [x] 5.8 `src/config/detect-environment.ts`, `src/config/detect-runner.ts` — runner + environment detection
|
||||
- [x] 5.9 `src/setup/polyfills.ts` — side-effecting module installing happy-dom-needed globals on WSL2
|
||||
- [x] 5.10 `src/utils/index.ts` — re-export every non-runner symbol for single-import convenience
|
||||
|
||||
## 6. Root barrel
|
||||
|
||||
- [x] 6.1 `src/index.ts` — explicit named re-exports (no `export *`) covering the full public surface; in the order of §1.1 so diffing against the surface inventory is trivial
|
||||
|
||||
## 7. Property-based tests (fast-check)
|
||||
|
||||
- [x] 7.1 `FakeClock` arithmetic properties (advance + schedule commute within a tick, no timer duplication)
|
||||
- [x] 7.2 `RequestBuilder` / `ResponseBuilder` state-machine transitions (associativity of preset composition)
|
||||
- [x] 7.3 `Factory` determinism under fixed `rng`
|
||||
- [x] 7.4 `MemoryStore` TTL expiry monotonicity + capacity-limit invariant
|
||||
- [x] 7.5 `waitFor` polling correctness under arbitrary predicate schedules
|
||||
- [x] 7.6 `seededRng` reproducibility (same seed → same sequence)
|
||||
|
||||
## 8. Benchmarks (mitata)
|
||||
|
||||
- [x] 8.1 `src/__benchmarks__/clock.bench.ts` — FakeClock advance/schedule throughput
|
||||
- [x] 8.2 `src/__benchmarks__/builders.bench.ts` — RequestBuilder and ResponseBuilder build throughput
|
||||
- [x] 8.3 `src/__benchmarks__/factory.bench.ts` — Factory single build + batch build
|
||||
- [x] 8.4 `src/__benchmarks__/memory-store.bench.ts` — set/get/evict under load
|
||||
- [x] 8.5 `src/__benchmarks__/helpers.bench.ts` — waitFor overhead; assertThrows overhead
|
||||
- [x] 8.6 `src/__benchmarks__/observability.bench.ts` — OTEL-wrapped vs unwrapped overhead comparison (enforces <5% in CI)
|
||||
- [x] 8.7 `benchmarks/baselines.json` — initial baseline commit (reviewer-approved, separate from functional changes)
|
||||
- [x] 8.8 CI script: run `mitata` 3×, take fastest, compare to baseline, fail >10% regression
|
||||
|
||||
## 9. Examples
|
||||
|
||||
- [x] 9.1 `examples/basic-usage.ts` — runnable script using `Factory`, `FakeClock`, `waitFor`
|
||||
- [x] 9.2 `examples/fixture-plugin.ts` — runnable script registering a `FixtureManagerPlugin`
|
||||
- [x] 9.3 `examples/deterministic-factory.ts` — runnable script demonstrating `seededRng` reproducibility
|
||||
- [x] 9.4 `examples/custom-clock.ts` — runnable script swapping `defaultClock` for `FakeClock` end-to-end
|
||||
- [x] 9.5 Verify every example runs via `bun examples/<name>.ts` without error
|
||||
|
||||
## 10. Tree-shake and dep-graph guards
|
||||
|
||||
- [x] 10.1 `src/__tests__/tree-shake.test.ts` — asserts that `./uint/clock` (for example) does not pull `MemoryStore`, `RequestBuilder`, or `Factory` identifiers into its built output
|
||||
- [x] 10.2 `src/__tests__/dep-graph.test.ts` — reads `package.json` and asserts no `@suaveplan/*` appears in `dependencies`, `peerDependencies`, or `devDependencies`
|
||||
- [x] 10.3 `src/__tests__/no-disallowed.test.ts` — greps `src/` for `any`, `ts-ignore`, `ts-expect-error`, `TODO`, `FIXME`, `@deprecated`, `.skip`, `.only`; fails on any match
|
||||
|
||||
## 11. Docs
|
||||
|
||||
- [x] 11.1 Verify every implementation file has a sibling `.md` of ≥ 200 words, simple + advanced examples, API reference, implementation notes
|
||||
- [x] 11.2 Rewrite `packages/universal/testing/README.md` with purpose, install, quick usage, error-code index, observability reference, benchmark numbers, links to each module's `.md`
|
||||
- [x] 11.3 Run a docs-coverage check script (inline in `scripts/` or ad-hoc) confirming ≥ 90%
|
||||
|
||||
## 12. Security review
|
||||
|
||||
- [x] 12.1 Invoke the `security-review` skill on the final diff
|
||||
- [x] 12.2 Address every finding (prototype pollution in `MemoryStore.set`, ReDoS in any regex helper, payload bombs in `Factory.buildBatch`); re-run until clean
|
||||
- [x] 12.3 `bun audit` clean — zero advisories introduced by catalog additions
|
||||
|
||||
## 13. Pre-publish gate + publish + archive
|
||||
|
||||
- [x] 13.1 `bunx turbo lint typecheck test --filter=@suaveplan/testing` green (CLAUDE.md non-negotiable)
|
||||
- [x] 13.2 `bunx turbo test:coverage --filter=@suaveplan/testing` meets ≥ 95% statement + branch
|
||||
- [x] 13.3 `bunx turbo lint typecheck` across all catalogs reaching `@suaveplan/testing` — green (only `@suaveplan/testing` on genesis today; confirm build graph is clean)
|
||||
- [x] 13.4 `bun scripts/publish.ts --package @suaveplan/testing --dry-run` produces a sane diff
|
||||
- [x] 13.5 `bun scripts/publish.ts --package @suaveplan/testing` — publish `@suaveplan/testing@0.0.0` to Verdaccio
|
||||
- [x] 13.6 `openspec validate enterprise-testing --type change` — passes
|
||||
- [x] 13.7 Fold the `specs/testing/spec.md` delta into `openspec/specs/testing/spec.md` (create) and move this change folder to `openspec/archive/<yyyy-mm-dd>-enterprise-testing/`
|
||||
- [x] 13.8 Mark `@suaveplan/testing` ✅ Enterprise in the genesis `README.md` (when the file exists) and in `openspec/project.md` package taxonomy
|
||||
Reference in New Issue
Block a user