Initial Commit

This commit is contained in:
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions

View File

@@ -0,0 +1,63 @@
## Context
The 2026-04-24 wave (umbrella design at `/mnt/s/web/genesis/docs/superpowers/specs/2026-04-24-enterprise-testbed-design.md`, plans at `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testing-integration-{core,plugins}.md` and `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testbed-pages-and-e2e.md`) shipped the integration substrate without a normative spec. The 2026-04-25 audit (`/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md`) is the driver for this change.
This change is normative-only. No source files are touched; the contract for the seven new packages and the four new e2e modules is fixed.
## Decisions
### D1: `ServiceFixture<TConfig>` is the universal fixture contract
Every plugin package implements one class satisfying `ServiceFixture<TConfig extends Record<string, unknown>>`:
```ts
interface ServiceFixture<TConfig> {
readonly name: string;
readonly config: TConfig;
readonly dependsOn?: readonly string[];
start(opts: { signal: AbortSignal }): Promise<void>;
stop(opts: { signal: AbortSignal }): Promise<void>;
isHealthy(): Promise<boolean>;
env(): Record<string, string>;
}
```
`InferConfig<F>` extracts `TConfig`. The interface lives in `@suaveplan/testing-integration/fixture/service-fixture` and is re-exported from the package barrel.
### D2: `TestEnvironment` lifecycle — topological start, reverse-topological stop
`TestEnvironment` accepts a `Record<string, ServiceFixture<…>>` plus `TestEnvironmentOptions { signal?, clock?, rng? }`. It computes a topological order from `dependsOn`, parallelises starts within each layer, and stops in reverse order. A fixture that never started is never stopped. Stop continues draining remaining fixtures even when one stop fails (errors are aggregated). A `started` flag prevents double-start.
### D3: `HealthPoller` contract
Default `intervalMs: 500`, `timeoutMs: 120000`. Honours `AbortSignal`. On deadline elapsed without a `true` from `isHealthy()`, throws `INTEGRATION.HEALTH_TIMEOUT` with `context.elapsedMs` and `context.fixtureName`. Optional exponential backoff cap. Uses an injectable `sleep` for testability.
### D4: Env-bridge passthrough
Every plugin SHALL pass through ALL of its declared env vars to the worker pool. When the env-bridge env var(s) are set on the parent process, the fixture skips container start and uses the existing service. Each plugin's spec enumerates its env-var contract.
### D5: `createIntegrationSetup()` + Vitest global-setup bridge
`createIntegrationSetup(fixtures)` registers fixtures in a module-level registry that `global-setup.ts` reads at Vitest startup. It returns the `globalSetup` array consumers drop into `vitest.config.ts`. On partial start failure, it `provide()`s `__suaveplan_integration_failed__` (renamed from the plans' `__integration_failed__`), and `skipIfUnavailable("name")` reads it via `inject()` to skip — never fail — affected tests.
### D6: Deterministic injection propagation
`TestEnvironmentOptions { clock?: Clock; rng?: Rng; signal?: AbortSignal }` is forwarded to every fixture whose config carries `acceptsContext: true`. Fixtures that opt in receive the same `Clock`/`Rng`/`signal` as the rest of the test runtime — preserving the determinism contract from `openspec/specs/testing/spec.md`.
### D7: Error-code namespace per plugin
Stable codes follow `INTEGRATION.<PLUGIN>.<REASON>`: e.g. `INTEGRATION.SURREALDB.START_FAILED`, `INTEGRATION.GARAGE_S3.LAYOUT_APPLY_FAILED`. Cross-cutting codes are unprefixed: `INTEGRATION.HEALTH_TIMEOUT`, `INTEGRATION.LEAK_DETECTED`, `INTEGRATION.QUARANTINED`.
### D8: Leak detection + flake quarantine
After `teardownAll()`, `TestEnvironment` SHALL assert no active handles attributable to its fixtures remain (warn + counter increment if found). `markFlaky({ retries, reason })` wraps a test with bounded retries and emits `testing.integration.flake.count`; quarantined tests are listed in `quarantine.json` at the package root.
### D9: testcontainers as peer dep, types-only `@suaveplan/types`
`@suaveplan/testing-integration` depends on `@suaveplan/types` only (the documented exception in CLAUDE.md rule 8) and `testcontainers` as a peer. Plugins depend on `@suaveplan/testing-integration` and `testcontainers`.
## Risks / Trade-offs
- **Inject-key rename** (`__integration_failed__` → `__suaveplan_integration_failed__`) is a one-line code change but normalises namespace collisions; documented as a one-shot mechanical sweep.
- **`dependsOn` topo-sort** adds complexity vs flat parallel start; mitigated by leaving the field optional with a default of `[]` (current behaviour preserved).
- **Image policy** — this change pins by name + minor version (e.g. `surrealdb/surrealdb:v3.0.4`). Pinning by `sha256` digest is deferred to a follow-up operational change so this normative change stays minimal.

View File

@@ -0,0 +1,49 @@
## Why
The 2026-04-24 testbed wave shipped seven new packages — `@suaveplan/testing-integration` and six service-fixture plugins (surrealdb, redis, postgres, mailpit, typesense, garage-s3) — plus four new `@suaveplan/testing-e2e` modules (console-monitor, network-monitor, multi-session, extension-fixture). All of that work currently lives in `docs/superpowers/plans/2026-04-24-*.md` and the source tree, but **none of it is reflected in `openspec/specs/`**. The 2026-04-25 test-suite audit (`docs/superpowers/specs/2026-04-25-test-suite-audit.md`) flags this as the single highest-leverage spec-coherence gap — every additional plugin compounds the drift.
This change lifts the already-implemented infrastructure into normative OpenSpec contract. It is a **spec-coherence change**: code is mostly already in place, but the contract that other genesis packages will rely on (`ServiceFixture<TConfig>`, `TestEnvironment` lifecycle, env-bridge inject-key namespace, per-plugin error-code prefixes) must be normative before any consumer integrates against it.
The change also collapses the residual `≥ 95 %` coverage gate inside `openspec/specs/testing/spec.md` into the project-wide `100 %` rule that `openspec/project.md` already mandates, and broadens property-based coverage from "the `@suaveplan/testing` package itself" to "every pure-function `@suaveplan/*` package".
## What Changes
### New capabilities (ADDED)
- **`testing-integration`** — `ServiceFixture<TConfig>` interface, `TestEnvironment` (topo-sorted parallel-start / reverse-stop with `dependsOn`), `HealthPoller` (interval / timeout / AbortSignal), `skipIfUnavailable()`, `createIntegrationSetup()` factory + Vitest global-setup bridge, env-bridge passthrough contract, `TestEnvironmentOptions.{ clock, rng, signal }` propagation, leak/handle assertion on teardown, `markFlaky()` + `quarantine.json`, `INTEGRATION.<PLUGIN>.*` error-code namespace.
- **`testing-integration-surrealdb`** — SurrealDB v3 fixture (HTTP `/health`, random password, env-bridge `SURREALDB_URL` / `SURREALDB_USERNAME` / `SURREALDB_PASSWORD`).
- **`testing-integration-redis`** — Redis 7 fixture (TCP `PING`, `REDIS_URL` env-bridge with full URL parsing).
- **`testing-integration-postgres`** — PostgreSQL 16 fixture (`pg_isready` in-container exec, configurable db/user/password, `POSTGRES_URL`).
- **`testing-integration-mailpit`** — Mailpit fixture + `assertMail()` helper (`MAILPIT_SMTP_URL` / `MAILPIT_API_URL`, partial-subject + exact-address matching).
- **`testing-integration-typesense`** — Typesense 27 fixture (JSON health `{ ok: true }`, random API key, `TYPESENSE_URL` / `TYPESENSE_API_KEY`).
- **`testing-integration-garage-s3`** — Garage v2 fixture (dual-port 3900/3902, 7-step init sequence, admin-port health, four env vars).
### Modified capabilities
- **`testing`** — flatten coverage gate from `≥ 95 %` to `100 %` on every dimension; broaden the property-based coverage requirement to every pure-function `@suaveplan/*` package; clarify that the deterministic `Clock` / `Rng` / `AbortSignal` injection contract extends to integration fixtures via `TestEnvironmentOptions`.
- **`testing-e2e`** — add `attachConsoleMonitor`, `attachNetworkMonitor` (with `assertNoFailures` / `assertNoMissingAssets` / `assertNoMixedContent` / `assertNoCorsErrors`), `createMultiSessionFixture`, `extendTestWithExtension`, and a default-deny network-sandbox rule.
## Capabilities
### New Capabilities
- `testing-integration` (core)
- `testing-integration-surrealdb`
- `testing-integration-redis`
- `testing-integration-postgres`
- `testing-integration-mailpit`
- `testing-integration-typesense`
- `testing-integration-garage-s3`
### Modified Capabilities
- `testing` — coverage gate, property-coverage scope, deterministic-injection cross-cut.
- `testing-e2e` — console / network / multi-session / extension surface plus network sandbox default.
## Impact
- **Affected code:** `packages/testing/testing-integration/**`, `packages/testing/testing-integration-{surrealdb,redis,postgres,mailpit,typesense,garage-s3}/**`, `packages/testing/testing-e2e/src/{console-monitor,network-monitor,multi-session,extension-fixture}/**` (already shipped; this change is normative-only).
- **Affected consumers:** every package whose nightly integration suite uses a fixture (today: none on genesis; tomorrow: every `db-*`, `oauth-client`, `webhook-client`, search/cache consumers).
- **Inject-key rename:** the plans use `__integration_failed__`; this change codifies it as `__suaveplan_integration_failed__` to avoid namespace collisions.
- **Plan sweep:** every literal `95 %` in `docs/superpowers/plans/2026-04-24-*.md` is replaced with `100 %` so plans and specs agree.
- **Pre-publish gate:** unchanged in shape — `bunx turbo lint typecheck test --filter=@suaveplan/<pkg>` plus `test:coverage` at flat `100 %` Istanbul thresholds.

View File

@@ -0,0 +1,92 @@
## ADDED Requirements
### Requirement: ConsoleMonitor
The `@suaveplan/testing-e2e` package SHALL export `attachConsoleMonitor(page, options?)` returning an object exposing `errors: ConsoleMessage[]`, `warnings: ConsoleMessage[]`, `assertNoErrors(): void`, and `assertNoWarnings(): void`. The default behaviour MUST forbid all console errors and warnings; the `options.failOn` field MAY relax to `["error"]` only. The `options.ignore` field MAY accept regex or string filters that exempt matching messages.
`assertNoErrors()` and `assertNoWarnings()` MUST throw `TestingE2eError` subclasses (`code === "TESTING_E2E.CONSOLE.ERRORS_OBSERVED"` / `"TESTING_E2E.CONSOLE.WARNINGS_OBSERVED"`) carrying the captured messages on `context`.
#### Scenario: No errors observed
- **WHEN** a page emits only `info`-level console output and `monitor.assertNoErrors()` is called
- **THEN** the call returns without throwing
#### Scenario: Error observed
- **WHEN** a page logs `console.error("boom")` and `monitor.assertNoErrors()` is called
- **THEN** the call throws with `code === "TESTING_E2E.CONSOLE.ERRORS_OBSERVED"` AND `context.messages` includes `"boom"`
#### Scenario: Filter exempts known noise
- **WHEN** `attachConsoleMonitor(page, { ignore: [/Download the React DevTools/] })` is used and a matching warning fires
- **THEN** `monitor.warnings` does not include the matching message AND `assertNoWarnings()` does not throw
### Requirement: NetworkMonitor with full failure-class assertions
`attachNetworkMonitor(page)` SHALL return an object exposing `failures: Request[]` and four assertion methods, each throwing a coded `TestingE2eError` subclass:
- `assertNoFailures()` — fails on any request that returned 4xx, 5xx, network error, or aborted (code `TESTING_E2E.NETWORK.FAILURES_OBSERVED`).
- `assertNoMissingAssets()` — fails on any 404 for an asset request whose `resourceType` is `image | font | script | stylesheet` (code `TESTING_E2E.NETWORK.MISSING_ASSETS`).
- `assertNoMixedContent()` — fails when the page is HTTPS and any subresource is HTTP (code `TESTING_E2E.NETWORK.MIXED_CONTENT`).
- `assertNoCorsErrors()` — fails on any request blocked by the browser's CORS rules (code `TESTING_E2E.NETWORK.CORS_BLOCKED`).
#### Scenario: 404 image fails missing-assets check
- **WHEN** a `<img src="/missing.png">` is rendered and the response is 404
- **THEN** `monitor.assertNoMissingAssets()` throws with `code === "TESTING_E2E.NETWORK.MISSING_ASSETS"` AND `context.urls` includes the image URL
#### Scenario: HTTPS page loading HTTP script triggers mixed-content
- **WHEN** an HTTPS page loads a `<script src="http://…">`
- **THEN** `monitor.assertNoMixedContent()` throws with `code === "TESTING_E2E.NETWORK.MIXED_CONTENT"`
#### Scenario: CORS-blocked request
- **WHEN** a fetch is blocked by CORS
- **THEN** `monitor.assertNoCorsErrors()` throws with `code === "TESTING_E2E.NETWORK.CORS_BLOCKED"`
### Requirement: Multi-session WebRTC fixture
The package SHALL export `createMultiSessionFixture(opts)` returning an object that owns at least two isolated browser contexts sharing a signaling URL but with separated cookies and storage. The fixture MUST expose:
- `sessions: { context: BrowserContext; page: Page }[]` — at least two entries
- `assertMessage(from: number, to: number, predicate: (data: unknown) => boolean): Promise<void>` — asserts a peer-to-peer message arrives
- `disableSignalingRoute(): Promise<void>` — chaos hook that 5xx-responses to the signaling URL until re-enabled
#### Scenario: Two sessions exchange a message
- **WHEN** session 0 sends a payload and `assertMessage(0, 1, p => p === "hello")` is awaited
- **THEN** the assertion resolves once session 1 observes the payload
#### Scenario: Disabling signaling produces a deterministic failure
- **WHEN** `disableSignalingRoute()` is called and a new session attempts to register
- **THEN** the registration fails with a 5xx from the signaling URL
### Requirement: Extension fixture
The package SHALL export `extendTestWithExtension(extPath: string)` that loads a Chromium extension into the persistent context and exposes `popup`, `background`, and `contentScript` accessors on the test fixture. The accessors MUST return Playwright `Page` (popup), `Worker` or `Page` (background — service worker or background page depending on manifest version), and an array of `Page` (one per content-script-injected frame) respectively.
#### Scenario: Background service worker accessible
- **WHEN** an MV3 extension is loaded and the test reads `background`
- **THEN** the returned value is a Playwright `Worker` whose URL begins with `chrome-extension://`
### Requirement: Default-deny network sandbox
`createPlaywrightConfig` SHALL default to blocking external origins by registering `page.route('**/*', …)` that aborts requests whose origin is not in `options.allowedOrigins`. Same-origin requests to the configured `baseURL` MUST always be allowed. Callers MAY supply `allowedOrigins: string[]` to extend the allow-list. Setting `options.allowedOrigins = "*"` disables the sandbox (escape hatch for fully-online specs).
#### Scenario: External fetch blocked by default
- **WHEN** a page running under `createPlaywrightConfig({ baseURL: "http://localhost:3000" })` issues a `fetch("https://api.external.com/x")`
- **THEN** the request is aborted by Playwright before it leaves the browser
#### Scenario: Allow-list grants access
- **WHEN** the same config is constructed with `allowedOrigins: ["https://api.external.com"]`
- **THEN** the same fetch resolves normally
#### Scenario: Escape hatch disables sandbox
- **WHEN** `allowedOrigins === "*"`
- **THEN** no `page.route` block is registered AND every request proceeds

View File

@@ -0,0 +1,88 @@
## ADDED Requirements
### Requirement: Garage v2 image policy
The `@suaveplan/testing-integration-garage-s3` plugin SHALL pin to `dxflrs/garage:v2.0.0` (name + minor version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `dxflrs/garage:v2.0.0`
### Requirement: Dual-port exposure
The fixture SHALL expose two TCP ports from the container: the S3 API port on container port `3900` and the admin port on container port `3902`. Both MUST be mapped to host ports and recorded in `config.s3Endpoint` and `config.adminEndpoint`.
#### Scenario: Both endpoints recorded
- **WHEN** the container has started
- **THEN** `config.s3Endpoint` and `config.adminEndpoint` are populated with reachable host URLs
### Requirement: Env-bridge contract
The plugin SHALL declare and forward exactly four environment variables: `GARAGE_S3_ENDPOINT`, `GARAGE_S3_ACCESS_KEY_ID`, `GARAGE_S3_SECRET_ACCESS_KEY`, `GARAGE_S3_BUCKET`. When ALL four are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns the full four-key set
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns a record with exactly the four declared keys
### Requirement: Seven-step initialisation sequence
In container mode `start()` SHALL execute the following sequence against the admin port, in order, before declaring the fixture healthy:
1. `status` — read cluster status to discover the node id
2. `layout assign` — assign capacity to the discovered node
3. `layout apply` — commit the layout version
4. `bucket create` — create the configured bucket
5. `key create` — create an access/secret key pair
6. `bucket allow` — grant the key full permissions on the bucket
7. `key info` — read back the key to confirm and capture the secret
Any step failure MUST abort the sequence and reject `start()` with a coded error naming the failing step.
#### Scenario: Successful seven-step init
- **WHEN** `start()` is invoked against a fresh container
- **THEN** all seven admin operations execute in the order above before `start()` resolves
#### Scenario: Layout-apply failure aborts
- **WHEN** the third step (`layout apply`) fails
- **THEN** `start()` rejects with `code === "INTEGRATION.GARAGE_S3.LAYOUT_APPLY_FAILED"` AND steps 4–7 are not attempted
### Requirement: Healthcheck contract — admin port
`isHealthy()` SHALL probe the admin port (3902) with an HTTP request and resolve `true` only when the admin endpoint responds. The S3 port (3900) alone is NOT sufficient evidence of health.
#### Scenario: Admin port is the source of truth
- **WHEN** the S3 port is open but the admin port has not yet started
- **THEN** `isHealthy()` resolves `false`
### Requirement: Configurable timeouts
The fixture options SHALL accept `healthIntervalMs?: number` and `healthTimeoutMs?: number` overrides forwarded to `HealthPoller`. Defaults follow `HealthPoller`'s defaults (500 ms / 120 s).
#### Scenario: Override applied
- **WHEN** the fixture is constructed with `{ healthTimeoutMs: 30000 }`
- **THEN** `HealthPoller` rejects after 30 s when the container never becomes healthy
### Requirement: Injectable factories
The fixture constructor SHALL accept optional `containerFactory`, `execFn`, and `fetchFn` parameters for the container, admin-CLI exec, and admin-port HTTP calls respectively. Defaults invoke testcontainers, the container's exec API, and the global `fetch`.
#### Scenario: Injected execFn drives the seven-step sequence
- **WHEN** `execFn` is replaced with a stub that records every command
- **THEN** the recorded commands match the seven-step order
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.GARAGE_S3\.[A-Z_]+$`. At minimum the package SHALL define: `START_FAILED`, `STOP_FAILED`, `STATUS_FAILED`, `LAYOUT_ASSIGN_FAILED`, `LAYOUT_APPLY_FAILED`, `BUCKET_CREATE_FAILED`, `KEY_CREATE_FAILED`, `BUCKET_ALLOW_FAILED`, `KEY_INFO_FAILED`.
#### Scenario: Each step has its own coded error
- **WHEN** `bucket create` fails
- **THEN** the rejection's `code === "INTEGRATION.GARAGE_S3.BUCKET_CREATE_FAILED"` AND `context.step === "bucket-create"`

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: Mailpit image policy
The `@suaveplan/testing-integration-mailpit` plugin SHALL pin to `axllent/mailpit` at a fixed minor tag (e.g. `axllent/mailpit:v1.20`). The plugin MUST NOT use the `latest` tag. A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned to minor
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses an `axllent/mailpit:v<major>.<minor>` tag — never `axllent/mailpit:latest`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward both `MAILPIT_SMTP_URL` and `MAILPIT_API_URL`. When BOTH are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode. A partial-set (only one of the two) MUST throw `INTEGRATION.MAILPIT.PARTIAL_ENV` at construction.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns a record with exactly the keys `MAILPIT_SMTP_URL` and `MAILPIT_API_URL`
#### Scenario: Partial env-bridge rejected
- **WHEN** only `MAILPIT_SMTP_URL` is set on `process.env` at construction
- **THEN** the constructor throws with `code === "INTEGRATION.MAILPIT.PARTIAL_ENV"`
### Requirement: Healthcheck contract — API ping
`isHealthy()` SHALL issue an HTTP `GET` to the Mailpit API health endpoint and treat a 2xx response as healthy. The fixture constructor SHALL accept an optional `fetch` parameter for test injection.
#### Scenario: 2xx is healthy
- **WHEN** the API health endpoint returns 200
- **THEN** `isHealthy()` resolves `true`
### Requirement: assertMail() helper
The package SHALL export `assertMail(opts: { to?: string | string[]; from?: string; subject?: string; subjectMatch?: "exact" | "partial" })` that queries the Mailpit API and asserts at least one matching message exists. **Address matching MUST be exact.** **Subject matching defaults to `"partial"`** (substring on `subject`); callers MAY opt into exact matching. On no match, the helper MUST throw with `code === "INTEGRATION.MAILPIT.NO_MATCH"` and `context` listing the candidates that were considered.
#### Scenario: Partial subject match by default
- **WHEN** Mailpit holds a message with subject `"Welcome to Suaveplan, please confirm your email"` and `assertMail({ subject: "Welcome to Suaveplan" })` is invoked
- **THEN** the helper resolves without throwing
#### Scenario: Exact address match required
- **WHEN** Mailpit holds a message addressed to `user@example.com` and `assertMail({ to: "USER@example.com" })` is invoked without case-insensitivity opt-in
- **THEN** the helper rejects with `INTEGRATION.MAILPIT.NO_MATCH`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.MAILPIT\.[A-Z_]+$`.
#### Scenario: Coded helper failure
- **WHEN** `assertMail()` finds no match
- **THEN** the rejection's `code === "INTEGRATION.MAILPIT.NO_MATCH"` AND `context.candidates` is an array

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: PostgreSQL 16 image policy
The `@suaveplan/testing-integration-postgres` plugin SHALL pin to `postgres:16` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `postgres:16`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the environment variable `POSTGRES_URL`. When `POSTGRES_URL` is set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns POSTGRES_URL
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns `{ POSTGRES_URL: "postgres://…" }`
#### Scenario: Passthrough mode skips container
- **WHEN** `POSTGRES_URL` is set at construction
- **THEN** `start()` does not invoke the container factory AND `stop()` is a no-op
### Requirement: Configurable database / user / password
The fixture options SHALL accept `database?: string; user?: string; password?: string`. Defaults MUST be `database = "test"`, `user = "test"`, `password` randomly generated per run with at least 32 bytes of entropy. The container's `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` env vars MUST be set from these values.
#### Scenario: Defaults applied
- **WHEN** the fixture is constructed without options
- **THEN** `config.database === "test"` AND `config.user === "test"` AND `config.password` is a non-empty random string
### Requirement: Healthcheck contract — pg_isready
`isHealthy()` SHALL execute `pg_isready -U <user> -d <database>` **inside** the Postgres container via testcontainers' `exec` API. The fixture constructor SHALL accept an optional `execFn` that defaults to the container's exec method and MAY be substituted for tests. A 0 exit code means healthy; any other exit code resolves to `false`.
#### Scenario: pg_isready exit 0 is healthy
- **WHEN** `pg_isready` exits with code 0
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Non-zero exit is unhealthy without throwing
- **WHEN** `pg_isready` exits with code 2
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.POSTGRES\.[A-Z_]+$` (e.g. `INTEGRATION.POSTGRES.START_FAILED`, `INTEGRATION.POSTGRES.EXEC_FAILED`).
#### Scenario: Coded exec failure
- **WHEN** the injected `execFn` throws
- **THEN** `isHealthy()` resolves `false` AND the OTel error counter records `code = "INTEGRATION.POSTGRES.EXEC_FAILED"`

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: Redis 7 image policy
The `@suaveplan/testing-integration-redis` plugin SHALL pin to `redis:7` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `redis:7`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the environment variable `REDIS_URL`. When `REDIS_URL` is set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode and parse the URL into host/port/password fields populated on `config`.
#### Scenario: env() returns REDIS_URL
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns `{ REDIS_URL: "redis://…" }`
#### Scenario: URL parsing in passthrough mode
- **WHEN** `REDIS_URL=redis://:secret@redis.local:6380` is set at construction
- **THEN** `config.host === "redis.local"` AND `config.port === 6380` AND `config.password === "secret"` AND container start is skipped
### Requirement: Healthcheck contract
`isHealthy()` SHALL open a TCP connection to the configured host/port, send the Redis `PING` command, and resolve `true` on receipt of `+PONG\r\n`. Connection errors, malformed replies, AND any non-PONG payload MUST resolve to `false`, never throw.
#### Scenario: PING/PONG roundtrip
- **WHEN** the Redis server replies `+PONG\r\n` to a PING
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Connection refused
- **WHEN** the port is closed
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Injectable TCP ping
The fixture constructor SHALL accept an optional `tcpPingFn` parameter. The default opens a real TCP socket; tests MAY replace it with a deterministic stub.
#### Scenario: Injected ping drives health probe
- **WHEN** `tcpPingFn` is replaced with `async () => false`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.REDIS\.[A-Z_]+$` (e.g. `INTEGRATION.REDIS.START_FAILED`, `INTEGRATION.REDIS.STOP_FAILED`, `INTEGRATION.REDIS.URL_PARSE_FAILED`).
#### Scenario: Malformed REDIS_URL is coded
- **WHEN** `REDIS_URL` is set to a string that is not a valid URL
- **THEN** the constructor throws with `code === "INTEGRATION.REDIS.URL_PARSE_FAILED"` AND `context.value` echoes the bad input

View File

@@ -0,0 +1,65 @@
## ADDED Requirements
### Requirement: SurrealDB v3 image policy
The `@suaveplan/testing-integration-surrealdb` plugin SHALL pin to `surrealdb/surrealdb:v3.0.4` (name + minor version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`; that pinning is **out of scope** for this change.
#### Scenario: Image tag pinned
- **WHEN** the fixture's container factory is invoked without injection
- **THEN** the resulting `GenericContainer` is constructed with image `surrealdb/surrealdb:v3.0.4`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the following environment variables: `SURREALDB_URL`, `SURREALDB_USERNAME`, `SURREALDB_PASSWORD`. When all three are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode and skip container start.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns a record whose keys are exactly `SURREALDB_URL`, `SURREALDB_USERNAME`, `SURREALDB_PASSWORD`
#### Scenario: Env-bridge passthrough
- **WHEN** all three env vars are set on `process.env` at construction
- **THEN** `start()` does not invoke the container factory AND `stop()` is a no-op
### Requirement: Healthcheck contract
`isHealthy()` SHALL issue an HTTP `GET /health` request to the SurrealDB endpoint. A 2xx response means healthy. Connection errors AND non-2xx responses MUST resolve to `false`, never throw.
#### Scenario: 2xx is healthy
- **WHEN** `/health` returns 200
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Connection refused does not throw
- **WHEN** the endpoint refuses the connection
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Random password per run
In container mode the fixture SHALL generate a fresh random password per `start()` call. The password MUST be at least 32 bytes of entropy, rendered as URL-safe base64 or hex.
#### Scenario: Two independent fixtures generate different passwords
- **WHEN** two fixtures start in the same process
- **THEN** their `config.password` values differ
### Requirement: Injectable factories
The fixture constructor SHALL accept optional `containerFactory` and `fetch` parameters. The defaults invoke `testcontainers` and the global `fetch`. Test-of-test code MAY substitute these to drive deterministic behaviour.
#### Scenario: Injected fetch drives health probe
- **WHEN** `fetch` is replaced with a stub that always returns a 503
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.SURREALDB\.[A-Z_]+$` (e.g. `INTEGRATION.SURREALDB.START_FAILED`, `INTEGRATION.SURREALDB.STOP_FAILED`). Cross-cutting codes (`INTEGRATION.HEALTH_TIMEOUT`, `INTEGRATION.ABORTED`) are inherited from `@suaveplan/testing-integration`.
#### Scenario: Coded error on start failure
- **WHEN** the container fails to bind a port
- **THEN** `start()` rejects with a structured error whose `code === "INTEGRATION.SURREALDB.START_FAILED"`

View File

@@ -0,0 +1,60 @@
## ADDED Requirements
### Requirement: Typesense 27 image policy
The `@suaveplan/testing-integration-typesense` plugin SHALL pin to `typesense/typesense:27` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `typesense/typesense:27`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward both `TYPESENSE_URL` and `TYPESENSE_API_KEY`. When both are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns a record with exactly the keys `TYPESENSE_URL` and `TYPESENSE_API_KEY`
### Requirement: Random API key per run
In container mode the fixture SHALL generate a fresh random API key per `start()` call with at least 32 bytes of entropy. The container MUST be started with this key as its `TYPESENSE_API_KEY`.
#### Scenario: Two fixtures generate different keys
- **WHEN** two fixtures start in the same process
- **THEN** their `config.apiKey` values differ
### Requirement: Healthcheck contract — JSON `{ ok: true }`
`isHealthy()` SHALL issue an HTTP `GET /health` to the Typesense endpoint with the configured API key in the `X-TYPESENSE-API-KEY` header, parse the response as JSON, and resolve `true` when `body.ok === true`. Connection errors, non-2xx responses, AND any non-`{ ok: true }` body MUST resolve to `false`, never throw.
#### Scenario: Healthy response
- **WHEN** `/health` returns `200` with body `{ "ok": true }`
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Body not ok
- **WHEN** `/health` returns `200` with body `{ "ok": false, "resource_error": "OUT_OF_DISK" }`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Injectable fetch
The fixture constructor SHALL accept an optional `fetch` parameter; the default is the global `fetch`.
#### Scenario: Injected fetch drives health probe
- **WHEN** `fetch` is replaced with a stub returning `{ ok: false }`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.TYPESENSE\.[A-Z_]+$`.
#### Scenario: Start failure is coded
- **WHEN** the container fails to start
- **THEN** `start()` rejects with `code === "INTEGRATION.TYPESENSE.START_FAILED"`

View File

@@ -0,0 +1,186 @@
## ADDED Requirements
### Requirement: ServiceFixture<TConfig> contract
The `@suaveplan/testing-integration` package SHALL export an interface `ServiceFixture<TConfig extends Record<string, unknown>>` that every plugin fixture MUST implement. The interface defines the universal lifecycle of a service used by integration tests, whether backed by a testcontainers-managed Docker container or by an env-bridge passthrough to an already-running service.
The interface MUST expose the following members:
- `readonly name: string` — globally unique among fixtures registered into one `TestEnvironment`.
- `readonly config: TConfig` — the resolved configuration after `start()`; before `start()` MAY be a partial draft.
- `readonly dependsOn?: readonly string[]` — names of fixtures that MUST start before this one.
- `start(opts: { signal: AbortSignal }): Promise<void>` — owns container startup and the health-poll loop; honours abort.
- `stop(opts: { signal: AbortSignal }): Promise<void>` — releases all resources; idempotent; no-op in env-bridge passthrough mode.
- `isHealthy(): Promise<boolean>` — single health probe; called by `HealthPoller`; never throws.
- `env(): Record<string, string>` — the full set of environment variables the fixture forwards to worker pools.
A utility type `InferConfig<F>` SHALL extract `TConfig` from a `ServiceFixture` implementation; it returns `never` for non-fixture types.
#### Scenario: Fixture exposes the full env set
- **WHEN** a plugin instance is constructed and `env()` is read
- **THEN** the returned record contains every environment variable name the plugin's spec declares — no implicit filtering
#### Scenario: Idempotent stop is safe
- **WHEN** `stop()` is awaited twice on the same fixture instance
- **THEN** the second call resolves without throwing AND does not attempt to interact with already-released resources
#### Scenario: InferConfig narrows TConfig
- **WHEN** `InferConfig<RedisFixture>` is evaluated where `RedisFixture extends ServiceFixture<{ url: string }>`
- **THEN** the resulting type is `{ url: string }`
### Requirement: TestEnvironment topological lifecycle
`TestEnvironment` SHALL accept a `Record<string, ServiceFixture<…>>` plus `TestEnvironmentOptions` and expose `setupAll()` / `teardownAll()`. `setupAll()` MUST compute a topological ordering from each fixture's `dependsOn` and start fixtures in parallel **within** each topological layer. `teardownAll()` MUST stop fixtures in reverse-topological order. A `started` flag MUST prevent double-start. A fixture that never started MUST never be stopped. A failing stop MUST NOT short-circuit teardown — `teardownAll()` MUST drain remaining fixtures and aggregate errors.
#### Scenario: Parallel start within a layer
- **WHEN** three fixtures `a`, `b`, `c` have no `dependsOn` and `setupAll()` is called
- **THEN** all three `start()` calls are in flight before any resolves
#### Scenario: Topological order respects dependsOn
- **WHEN** fixture `app` declares `dependsOn: ["oidc"]` and `setupAll()` runs
- **THEN** `oidc.start()` resolves before `app.start()` is invoked
#### Scenario: Reverse-topological stop
- **WHEN** `teardownAll()` runs against the same set
- **THEN** `app.stop()` is invoked before `oidc.stop()`
#### Scenario: Failed start does not start dependents and does not block teardown
- **WHEN** `oidc.start()` rejects
- **THEN** `app.start()` is never invoked AND `teardownAll()` calls `oidc.stop()` only if `oidc.start()` had begun, AND remaining fixtures' `stop()` calls all run regardless of any single failure
#### Scenario: Double setup throws
- **WHEN** `setupAll()` is awaited twice on the same instance
- **THEN** the second call rejects with `INTEGRATION.ALREADY_STARTED`
### Requirement: HealthPoller contract
`HealthPoller` SHALL accept `{ intervalMs?: number; timeoutMs?: number; sleep?: (ms: number) => Promise<void> }` with defaults `intervalMs = 500` and `timeoutMs = 120_000`. `poll(fixture, { signal })` MUST call `fixture.isHealthy()` repeatedly until it returns `true` or the deadline elapses. On deadline, the promise rejects with a `TestEnvironmentError` whose `code === "INTEGRATION.HEALTH_TIMEOUT"`, `context.fixtureName`, and `context.elapsedMs` are populated. The poller MUST honour `AbortSignal` and reject with `INTEGRATION.ABORTED` on first observed abort.
#### Scenario: Healthy fixture resolves promptly
- **WHEN** `fixture.isHealthy()` returns `true` on the first call
- **THEN** `poll()` resolves before a second call is scheduled
#### Scenario: Deadline elapsed
- **WHEN** `fixture.isHealthy()` always returns `false` and 120 s elapse
- **THEN** `poll()` rejects with `INTEGRATION.HEALTH_TIMEOUT` AND `context.elapsedMs >= 120000`
#### Scenario: Abort honoured
- **WHEN** the caller-provided `signal` aborts mid-poll
- **THEN** `poll()` rejects with `INTEGRATION.ABORTED` within one event-loop tick of the abort
### Requirement: skipIfUnavailable() helper
The package SHALL export `skipIfUnavailable(fixtureName: string): void` that, when called from the body of a Vitest `it`/`test`, causes the test to be **skipped** (not failed) when the named fixture failed to start during `globalSetup`. The helper MUST read the inject key `__suaveplan_integration_failed__` and gracefully no-op when the inject key is absent (e.g. unit-test mode). A bun-test mode SHALL emit the equivalent `it.skip` behaviour.
#### Scenario: Skip on partial failure
- **WHEN** `surrealdb.start()` failed during global setup, `provide("__suaveplan_integration_failed__", { surrealdb: true })` was called, and a test calls `skipIfUnavailable("surrealdb")`
- **THEN** the test is reported as skipped, not failed
#### Scenario: No-op when inject key absent
- **WHEN** the test runner has no global-setup wired and `skipIfUnavailable("anything")` is called
- **THEN** the call returns without throwing AND the test continues
### Requirement: createIntegrationSetup() factory and Vitest global-setup bridge
`createIntegrationSetup(fixtures)` SHALL register the supplied fixture map in a module-level registry and return a `globalSetup` array suitable for a Vitest config. At Vitest startup, the bridge MUST construct a single shared `TestEnvironment` from the registered fixtures, run `setupAll()`, and on partial failure call `provide("__suaveplan_integration_failed__", failures)` where `failures` is `Record<string, true>` keyed by fixture name. On full success it MUST `provide()` an empty record. Every env var returned by every fixture's `env()` MUST be set on `process.env` before the worker pool is forked.
#### Scenario: Successful setup propagates env
- **WHEN** all fixtures start successfully and their `env()` returns disjoint keys
- **THEN** every key is present in `process.env` before any test worker forks
#### Scenario: Partial failure is reported via inject key
- **WHEN** one fixture's `start()` rejects
- **THEN** `provide("__suaveplan_integration_failed__", { <failedName>: true })` is invoked AND the worker pool still starts so unaffected tests can run
### Requirement: Deterministic injection propagation via TestEnvironmentOptions
`TestEnvironmentOptions` SHALL include `clock?: Clock; rng?: Rng; signal?: AbortSignal`. When a fixture's config declares `acceptsContext: true`, `TestEnvironment` MUST forward the same `clock` / `rng` / `signal` references to that fixture before invoking `start()`. Fixtures that do not opt in MUST be unaffected. This requirement extends the deterministic-injection contract from `openspec/specs/testing/spec.md` into the integration substrate.
#### Scenario: Opt-in fixture sees the shared clock
- **WHEN** `TestEnvironment` is constructed with `{ clock: fakeClock }` and a fixture's config declares `acceptsContext: true`
- **THEN** that fixture's `config.clock === fakeClock` at the time `start()` is invoked
#### Scenario: Non-opt-in fixture is unaffected
- **WHEN** the same `TestEnvironment` includes a fixture whose config does NOT declare `acceptsContext: true`
- **THEN** that fixture's config is left untouched
### Requirement: Leak detection on teardown
After `teardownAll()` resolves, `TestEnvironment` SHALL inspect the host runtime's active-handle set (e.g. `process._getActiveHandles()` on Node, the Bun equivalent on Bun) and emit a structured warning with code `INTEGRATION.LEAK_DETECTED` plus an OTel counter increment `testing.integration.leak.count` for each handle attributable to a fixture that should have released. The warning MUST identify the fixture by name when attribution is possible.
#### Scenario: Handle leak detected
- **WHEN** a fixture's `stop()` returns but a TCP socket it owned remains open
- **THEN** `teardownAll()` emits an `INTEGRATION.LEAK_DETECTED` warning with `context.fixtureName` AND increments `testing.integration.leak.count`
### Requirement: Flake quarantine
The package SHALL export `markFlaky(testFn, { retries, reason })` that wraps a test with bounded retries and emits a `testing.integration.flake.count` metric increment per retry attempt. Quarantined tests MUST be enumerated in a `quarantine.json` at the package root listing `{ test: string; reason: string; addedAt: string }` entries. A test that exhausts its retries MUST fail with `INTEGRATION.QUARANTINED` carrying the `reason` in `context`.
#### Scenario: Flaky test passes within budget
- **WHEN** `markFlaky(fn, { retries: 3, reason: "TC-123" })` is run and the underlying test fails twice then passes
- **THEN** the test is reported as passing AND `testing.integration.flake.count` is incremented twice
#### Scenario: Flaky test exhausts budget
- **WHEN** the underlying test fails on every retry
- **THEN** the wrapping test fails with `INTEGRATION.QUARANTINED` AND `context.reason === "TC-123"`
### Requirement: Coverage gate
Istanbul statement, branch, function, and line coverage across `packages/testing/testing-integration/src/` MUST each be exactly `100 %`. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing-integration`) MUST hard-fail below `100 %` on any dimension.
#### Scenario: Coverage gate enforced
- **WHEN** the pre-publish gate runs
- **THEN** statement / branch / function / line coverage are each `100 %`
### Requirement: OTel observability
Every public entrypoint SHALL emit OTel counters (`testing.integration.<op>.count`), histograms (`testing.integration.<op>.duration_ms`), and an error counter (`testing.integration.<op>.errors{code}`). Spans SHALL be created for `setupAll`, `teardownAll`, and `HealthPoller.poll`. When no SDK is registered, all instrumentation MUST be a no-op with overhead below 5 % per op vs an unwrapped baseline.
#### Scenario: Counter increments on every setup
- **WHEN** `setupAll()` runs N times against a registered SDK
- **THEN** `testing.integration.setup.count` records N increments
#### Scenario: Error counter records code attribute
- **WHEN** `HealthPoller.poll` rejects with `INTEGRATION.HEALTH_TIMEOUT`
- **THEN** `testing.integration.health-poll.errors` records `code = "INTEGRATION.HEALTH_TIMEOUT"`
### Requirement: Banned workspace deps
`@suaveplan/testing-integration` `dependencies`, `peerDependencies`, and `devDependencies` MAY list `@suaveplan/types` (the documented types-only exception in CLAUDE.md rule 8) but MUST NOT list any other `@suaveplan/*` workspace package. `testcontainers` MUST be a peer dependency.
#### Scenario: Dep-graph guard
- **WHEN** `bunx turbo ls --filter=@suaveplan/testing-integration...` runs
- **THEN** the only `@suaveplan/*` package in the closure is `@suaveplan/types` AND `testcontainers` resolves via peer dependency
### Requirement: Inject-key namespace
The integration substrate SHALL use the inject key `__suaveplan_integration_failed__` (renamed from the development-time literal `__integration_failed__` used in the implementation plans) for partial-failure propagation. The key MUST NOT collide with any other inject key in the workspace.
#### Scenario: Renamed inject key is the contract
- **WHEN** any consumer calls `inject("__suaveplan_integration_failed__")`
- **THEN** the key resolves to a `Record<string, true>` (possibly empty) populated by `createIntegrationSetup`'s global-setup bridge

View File

@@ -0,0 +1,58 @@
## MODIFIED Requirements
### Requirement: Coverage and docs gates
> Original: "Istanbul statement + branch coverage across `packages/universal/testing/src/` (excluding `*.bench.ts` and `*.schemas.ts`) MUST be ≥ 95%. […]"
Istanbul statement, branch, function, AND line coverage across `packages/testing/testing/src/` (excluding `*.bench.ts`, `*.schemas.ts`, barrel `index.ts` files, and inner-dir `*.types.ts`) MUST each be exactly **100 %**. There is no foundation carve-out, no degraded-situation floor, and no exception for any genesis package — the gate applies to `@suaveplan/testing` and to every other `@suaveplan/*` package equally. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing`) MUST hard-fail when any of statements, branches, functions, or lines is below `100 %`.
Every implementation file (`*.ts` other than `*.test.ts`, `*.bench.ts`, `*.schemas.ts`, inner-dir `*.types.ts`, barrel `index.ts`) MUST have a sibling `.md` file of ≥ 200 words (excluding code fences) with a simple and an advanced usage example. Mutation testing, contract testing, property testing, and fuzz coverage layer **on top of** 100 % line coverage; they do not substitute for it, and 100 % line coverage does not substitute for them.
#### Scenario: Coverage gate at 100 %
- **WHEN** `bunx turbo test:coverage --filter=@suaveplan/testing` runs
- **THEN** reported statement, branch, function, and line coverage are each `100 %` AND the gate fails the build if any dimension drops below `100 %`
#### Scenario: Docs coverage enforced
- **WHEN** a script enumerates implementation files under `src/` (excluding the stated exclusions) and checks for sibling `.md` files
- **THEN** every implementation file has a sibling `.md` AND every `.md` is ≥ 200 non-code words
### Requirement: Property-based coverage for pure helpers
> Original: "Pure functions and deterministic state machines in the package […] SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants […]"
Pure functions and deterministic state machines in `@suaveplan/testing` SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants as applicable. **The same requirement extends to every other pure-function `@suaveplan/*` package** — every package whose public surface includes deterministic, side-effect-free helpers MUST ship fast-check property tests that exercise round-trip, idempotency, associativity, or boundary invariants where each applies. Load-bearing examples include `@suaveplan/crypto-utils` (encoding round-trips), `@suaveplan/diff` (apply ∘ invert, structural equality), `@suaveplan/pagination` (cursor encode/decode round-trip, slice associativity), and `@suaveplan/validation` (parse / safeParse equivalence on valid inputs).
#### Scenario: Builder composition is associative
- **WHEN** fast-check runs over arbitrary RequestBuilder presets `a`, `b`, `c` applied in different groupings
- **THEN** `(a then b) then c` produces the same final state as `a then (b then c)`
#### Scenario: Pure-function package ships round-trip property
- **WHEN** any genesis package whose public surface is pure (e.g. `@suaveplan/crypto-utils` `hexEncode`/`hexDecode`) is audited for property tests
- **THEN** at least one fast-check test exists asserting `decode(encode(x)) === x` for arbitrary `Uint8Array` inputs
### Requirement: Deterministic clock and RNG via dependency injection
> Original: "Every public function that reads time, randomness, or iterates SHALL accept an optional `{ clock?: Clock; rng?: Rng; signal?: AbortSignal }` options parameter. […]"
Every public function in `@suaveplan/testing` that reads time, randomness, or iterates SHALL accept an optional `{ clock?: Clock; rng?: Rng; signal?: AbortSignal }` options parameter. Zero-config callers MUST get behaviorally stable defaults: `defaultClock` (backed by `Date.now` / `performance.now`), `cryptoRng` (backed by `crypto.getRandomValues`), and no abort.
**This contract additionally extends to integration test fixtures via `TestEnvironmentOptions`** — `@suaveplan/testing-integration`'s `TestEnvironment` MUST accept `{ clock?, rng?, signal? }` and propagate them to fixtures whose configs declare `acceptsContext: true`, so that integration tests can run against the same deterministic time and randomness sources as unit tests.
#### Scenario: Injected clock controls observed time
- **WHEN** `waitFor(predicate, { clock: fakeClock })` runs and `fakeClock.advance(1000)` is called
- **THEN** the helper observes 1000 ms elapsed AND does NOT read `Date.now()` or `performance.now()` directly
#### Scenario: Injected RNG produces deterministic factory output
- **WHEN** `Factory.build({ rng: seededRng(42) })` is called twice with the same factory definition
- **THEN** both calls return byte-identical output (for factories whose definition is otherwise deterministic)
#### Scenario: TestEnvironment forwards clock to opt-in fixtures
- **WHEN** a `TestEnvironment` is constructed with `{ clock: fakeClock }` and a fixture's config declares `acceptsContext: true`
- **THEN** that fixture's `config.clock === fakeClock` at the time `start()` is invoked

View File

@@ -0,0 +1,84 @@
> Spec-coherence change. Most implementation work is already done in the 2026-04-24 wave; tasks marked `[x]` reflect that. Tasks marked `[ ]` are normative or cleanup work this change introduces.
## 0. Pre-flight (non-negotiable)
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
## 1. Audit-driven prep
- [x] 1.1 Read the umbrella design at `/mnt/s/web/genesis/docs/superpowers/specs/2026-04-24-enterprise-testbed-design.md`.
- [x] 1.2 Read the three implementation plans at `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testing-integration-core.md`, `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testing-integration-plugins.md`, `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testbed-pages-and-e2e.md`.
- [x] 1.3 Read the audit at `/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md`.
- [x] 1.4 Confirm the seven packages exist on disk under `packages/testing/`.
## 2. `testing-integration` (core) — already implemented
- [x] 2.1 `ServiceFixture<TConfig>` interface + `InferConfig<F>` utility.
- [x] 2.2 `HealthPoller` with configurable interval / timeout / sleep injection.
- [x] 2.3 `TestEnvironment` with parallel-start / reverse-stop and the `started` double-start guard.
- [x] 2.4 `skipIfUnavailable()` helper (Vitest `inject()`).
- [x] 2.5 `createIntegrationSetup()` factory + Vitest global-setup bridge.
- [x] 2.6 OTel counters / histograms / spans for every public entrypoint.
- [x] 2.7 100 % Istanbul coverage on the package.
## 3. `testing-integration` (core) — new normative work landed by this change
- [ ] 3.1 Add `dependsOn?: readonly string[]` to `ServiceFixture<TConfig>` and topo-sort in `TestEnvironment.setupAll()`. Stop reverses the same order.
- [ ] 3.2 Extend `TestEnvironmentOptions` with `clock?: Clock; rng?: Rng; signal?: AbortSignal` and propagate to fixture configs whose options declare `acceptsContext: true`.
- [ ] 3.3 Implement leak / handle-tracker assertion at end of `teardownAll()`; emit warning + `testing.integration.leak.count` increment on detection.
- [ ] 3.4 Implement `markFlaky({ retries, reason })` wrapper + per-package `quarantine.json` enumeration.
- [ ] 3.5 Rename inject-key `__integration_failed__` → `__suaveplan_integration_failed__` across `create-integration-setup.ts`, `skip-if-unavailable.ts`, and tests.
- [ ] 3.6 Replace every literal `95 %` in `docs/superpowers/plans/2026-04-24-*.md` with `100 %` (sed-style sweep).
- [ ] 3.7 Add property-based round-trip tests for `HealthPoller` retry arithmetic and `TestEnvironment` topo-sort using `fast-check`.
## 4. Per-plugin specs (already implemented; this change only lifts to spec)
- [x] 4.1 `testing-integration-surrealdb` v3.0.4 image, HTTP `/health`, random password, three env vars.
- [x] 4.2 `testing-integration-redis` Redis 7, TCP PING, `REDIS_URL` parsing.
- [x] 4.3 `testing-integration-postgres` PostgreSQL 16, `pg_isready` exec, `POSTGRES_URL`.
- [x] 4.4 `testing-integration-mailpit` API health, `assertMail()` partial-subject + exact-address matching, two env vars.
- [x] 4.5 `testing-integration-typesense` Typesense 27, `{ ok: true }` health, random API key.
- [x] 4.6 `testing-integration-garage-s3` Garage v2.0.0, dual-port 3900/3902, 7-step init, four env vars.
## 5. Per-plugin normative work landed by this change
- [ ] 5.1 Each plugin's package surface MUST expose `INTEGRATION.<PLUGIN>.*` error codes via a `codes` const re-exported from the barrel (audit alignment with the testing-error pattern).
- [ ] 5.2 Each plugin's `ServiceFixture` config schema (Zod) lists every env-bridge variable name; spec scenarios assert `env()` returns the full set.
- [ ] 5.3 Each plugin marks the `acceptsContext: true` option (where the config can deterministically take a `clock` / `rng`).
## 6. `testing-e2e` — already implemented
- [x] 6.1 `attachConsoleMonitor(page)` with `errors`, `warnings`, `assertNoErrors()`, `assertNoWarnings()`, filter options.
- [x] 6.2 `attachNetworkMonitor(page)` with `failures`, `assertNoFailures()`.
- [x] 6.3 Multi-session WebRTC fixture (`createMultiSessionFixture`).
- [x] 6.4 Extension fixture (`extendTestWithExtension`).
## 7. `testing-e2e` — new normative work landed by this change
- [ ] 7.1 Extend `attachNetworkMonitor` with `assertNoMissingAssets()`, `assertNoMixedContent()`, `assertNoCorsErrors()`.
- [ ] 7.2 Default-deny network sandbox in `createPlaywrightConfig`: `page.route('**/*')` blocks external origins unless `allowedOrigins` is provided.
- [ ] 7.3 Multi-session fixture exposes `disableSignalingRoute()` chaos hook (already partial in plan).
## 8. `testing` capability — coverage / property / determinism updates
- [ ] 8.1 In the canonical spec delta, replace `≥ 95 %` with `100 %` on every dimension; remove the foundation carve-out language.
- [ ] 8.2 Broaden the property-based requirement from "the `@suaveplan/testing` package" to every pure-function `@suaveplan/*` package (citing `crypto-utils`, `diff`, `pagination`, `validation`).
- [ ] 8.3 Add a clarifying scenario tying the deterministic `Clock`/`Rng`/`AbortSignal` contract to `TestEnvironmentOptions` propagation.
## 9. Spec validation
- [ ] 9.1 `openspec validate enterprise-testing-integration --type change` passes.
- [ ] 9.2 Diff each plugin spec against the implementation plan to confirm no requirement contradicts shipped code.
## 10. Pre-publish gate per package (run when each plugin next ships)
- [ ] 10.1 `bunx turbo lint typecheck test --filter=@suaveplan/testing-integration` green.
- [ ] 10.2 `bunx turbo test:coverage --filter=@suaveplan/testing-integration` at 100 %.
- [ ] 10.3 Repeat 10.1 / 10.2 for each of `testing-integration-{surrealdb,redis,postgres,mailpit,typesense,garage-s3}`.
## 11. Archive
- [ ] 11.1 Fold each spec delta into the matching `openspec/specs/<capability>/spec.md` (creating the file for new capabilities, MODIFIED-merging for `testing` and `testing-e2e`).
- [ ] 11.2 Move `openspec/changes/enterprise-testing-integration/` → `openspec/archive/<yyyy-mm-dd>-enterprise-testing-integration/`.