Initial Commit

This commit is contained in:
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions

View File

@@ -0,0 +1,92 @@
## ADDED Requirements
### Requirement: ConsoleMonitor
The `@suaveplan/testing-e2e` package SHALL export `attachConsoleMonitor(page, options?)` returning an object exposing `errors: ConsoleMessage[]`, `warnings: ConsoleMessage[]`, `assertNoErrors(): void`, and `assertNoWarnings(): void`. The default behaviour MUST forbid all console errors and warnings; the `options.failOn` field MAY relax to `["error"]` only. The `options.ignore` field MAY accept regex or string filters that exempt matching messages.
`assertNoErrors()` and `assertNoWarnings()` MUST throw `TestingE2eError` subclasses (`code === "TESTING_E2E.CONSOLE.ERRORS_OBSERVED"` / `"TESTING_E2E.CONSOLE.WARNINGS_OBSERVED"`) carrying the captured messages on `context`.
#### Scenario: No errors observed
- **WHEN** a page emits only `info`-level console output and `monitor.assertNoErrors()` is called
- **THEN** the call returns without throwing
#### Scenario: Error observed
- **WHEN** a page logs `console.error("boom")` and `monitor.assertNoErrors()` is called
- **THEN** the call throws with `code === "TESTING_E2E.CONSOLE.ERRORS_OBSERVED"` AND `context.messages` includes `"boom"`
#### Scenario: Filter exempts known noise
- **WHEN** `attachConsoleMonitor(page, { ignore: [/Download the React DevTools/] })` is used and a matching warning fires
- **THEN** `monitor.warnings` does not include the matching message AND `assertNoWarnings()` does not throw
### Requirement: NetworkMonitor with full failure-class assertions
`attachNetworkMonitor(page)` SHALL return an object exposing `failures: Request[]` and four assertion methods, each throwing a coded `TestingE2eError` subclass:
- `assertNoFailures()` — fails on any request that returned 4xx, 5xx, network error, or aborted (code `TESTING_E2E.NETWORK.FAILURES_OBSERVED`).
- `assertNoMissingAssets()` — fails on any 404 for an asset request whose `resourceType` is `image | font | script | stylesheet` (code `TESTING_E2E.NETWORK.MISSING_ASSETS`).
- `assertNoMixedContent()` — fails when the page is HTTPS and any subresource is HTTP (code `TESTING_E2E.NETWORK.MIXED_CONTENT`).
- `assertNoCorsErrors()` — fails on any request blocked by the browser's CORS rules (code `TESTING_E2E.NETWORK.CORS_BLOCKED`).
#### Scenario: 404 image fails missing-assets check
- **WHEN** a `<img src="/missing.png">` is rendered and the response is 404
- **THEN** `monitor.assertNoMissingAssets()` throws with `code === "TESTING_E2E.NETWORK.MISSING_ASSETS"` AND `context.urls` includes the image URL
#### Scenario: HTTPS page loading HTTP script triggers mixed-content
- **WHEN** an HTTPS page loads a `<script src="http://…">`
- **THEN** `monitor.assertNoMixedContent()` throws with `code === "TESTING_E2E.NETWORK.MIXED_CONTENT"`
#### Scenario: CORS-blocked request
- **WHEN** a fetch is blocked by CORS
- **THEN** `monitor.assertNoCorsErrors()` throws with `code === "TESTING_E2E.NETWORK.CORS_BLOCKED"`
### Requirement: Multi-session WebRTC fixture
The package SHALL export `createMultiSessionFixture(opts)` returning an object that owns at least two isolated browser contexts sharing a signaling URL but with separated cookies and storage. The fixture MUST expose:
- `sessions: { context: BrowserContext; page: Page }[]` — at least two entries
- `assertMessage(from: number, to: number, predicate: (data: unknown) => boolean): Promise<void>` — asserts a peer-to-peer message arrives
- `disableSignalingRoute(): Promise<void>` — chaos hook that 5xx-responses to the signaling URL until re-enabled
#### Scenario: Two sessions exchange a message
- **WHEN** session 0 sends a payload and `assertMessage(0, 1, p => p === "hello")` is awaited
- **THEN** the assertion resolves once session 1 observes the payload
#### Scenario: Disabling signaling produces a deterministic failure
- **WHEN** `disableSignalingRoute()` is called and a new session attempts to register
- **THEN** the registration fails with a 5xx from the signaling URL
### Requirement: Extension fixture
The package SHALL export `extendTestWithExtension(extPath: string)` that loads a Chromium extension into the persistent context and exposes `popup`, `background`, and `contentScript` accessors on the test fixture. The accessors MUST return Playwright `Page` (popup), `Worker` or `Page` (background — service worker or background page depending on manifest version), and an array of `Page` (one per content-script-injected frame) respectively.
#### Scenario: Background service worker accessible
- **WHEN** an MV3 extension is loaded and the test reads `background`
- **THEN** the returned value is a Playwright `Worker` whose URL begins with `chrome-extension://`
### Requirement: Default-deny network sandbox
`createPlaywrightConfig` SHALL default to blocking external origins by registering `page.route('**/*', …)` that aborts requests whose origin is not in `options.allowedOrigins`. Same-origin requests to the configured `baseURL` MUST always be allowed. Callers MAY supply `allowedOrigins: string[]` to extend the allow-list. Setting `options.allowedOrigins = "*"` disables the sandbox (escape hatch for fully-online specs).
#### Scenario: External fetch blocked by default
- **WHEN** a page running under `createPlaywrightConfig({ baseURL: "http://localhost:3000" })` issues a `fetch("https://api.external.com/x")`
- **THEN** the request is aborted by Playwright before it leaves the browser
#### Scenario: Allow-list grants access
- **WHEN** the same config is constructed with `allowedOrigins: ["https://api.external.com"]`
- **THEN** the same fetch resolves normally
#### Scenario: Escape hatch disables sandbox
- **WHEN** `allowedOrigins === "*"`
- **THEN** no `page.route` block is registered AND every request proceeds

View File

@@ -0,0 +1,88 @@
## ADDED Requirements
### Requirement: Garage v2 image policy
The `@suaveplan/testing-integration-garage-s3` plugin SHALL pin to `dxflrs/garage:v2.0.0` (name + minor version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `dxflrs/garage:v2.0.0`
### Requirement: Dual-port exposure
The fixture SHALL expose two TCP ports from the container: the S3 API port on container port `3900` and the admin port on container port `3902`. Both MUST be mapped to host ports and recorded in `config.s3Endpoint` and `config.adminEndpoint`.
#### Scenario: Both endpoints recorded
- **WHEN** the container has started
- **THEN** `config.s3Endpoint` and `config.adminEndpoint` are populated with reachable host URLs
### Requirement: Env-bridge contract
The plugin SHALL declare and forward exactly four environment variables: `GARAGE_S3_ENDPOINT`, `GARAGE_S3_ACCESS_KEY_ID`, `GARAGE_S3_SECRET_ACCESS_KEY`, `GARAGE_S3_BUCKET`. When ALL four are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns the full four-key set
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns a record with exactly the four declared keys
### Requirement: Seven-step initialisation sequence
In container mode `start()` SHALL execute the following sequence against the admin port, in order, before declaring the fixture healthy:
1. `status` — read cluster status to discover the node id
2. `layout assign` — assign capacity to the discovered node
3. `layout apply` — commit the layout version
4. `bucket create` — create the configured bucket
5. `key create` — create an access/secret key pair
6. `bucket allow` — grant the key full permissions on the bucket
7. `key info` — read back the key to confirm and capture the secret
Any step failure MUST abort the sequence and reject `start()` with a coded error naming the failing step.
#### Scenario: Successful seven-step init
- **WHEN** `start()` is invoked against a fresh container
- **THEN** all seven admin operations execute in the order above before `start()` resolves
#### Scenario: Layout-apply failure aborts
- **WHEN** the third step (`layout apply`) fails
- **THEN** `start()` rejects with `code === "INTEGRATION.GARAGE_S3.LAYOUT_APPLY_FAILED"` AND steps 4–7 are not attempted
### Requirement: Healthcheck contract — admin port
`isHealthy()` SHALL probe the admin port (3902) with an HTTP request and resolve `true` only when the admin endpoint responds. The S3 port (3900) alone is NOT sufficient evidence of health.
#### Scenario: Admin port is the source of truth
- **WHEN** the S3 port is open but the admin port has not yet started
- **THEN** `isHealthy()` resolves `false`
### Requirement: Configurable timeouts
The fixture options SHALL accept `healthIntervalMs?: number` and `healthTimeoutMs?: number` overrides forwarded to `HealthPoller`. Defaults follow `HealthPoller`'s defaults (500 ms / 120 s).
#### Scenario: Override applied
- **WHEN** the fixture is constructed with `{ healthTimeoutMs: 30000 }`
- **THEN** `HealthPoller` rejects after 30 s when the container never becomes healthy
### Requirement: Injectable factories
The fixture constructor SHALL accept optional `containerFactory`, `execFn`, and `fetchFn` parameters for the container, admin-CLI exec, and admin-port HTTP calls respectively. Defaults invoke testcontainers, the container's exec API, and the global `fetch`.
#### Scenario: Injected execFn drives the seven-step sequence
- **WHEN** `execFn` is replaced with a stub that records every command
- **THEN** the recorded commands match the seven-step order
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.GARAGE_S3\.[A-Z_]+$`. At minimum the package SHALL define: `START_FAILED`, `STOP_FAILED`, `STATUS_FAILED`, `LAYOUT_ASSIGN_FAILED`, `LAYOUT_APPLY_FAILED`, `BUCKET_CREATE_FAILED`, `KEY_CREATE_FAILED`, `BUCKET_ALLOW_FAILED`, `KEY_INFO_FAILED`.
#### Scenario: Each step has its own coded error
- **WHEN** `bucket create` fails
- **THEN** the rejection's `code === "INTEGRATION.GARAGE_S3.BUCKET_CREATE_FAILED"` AND `context.step === "bucket-create"`

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: Mailpit image policy
The `@suaveplan/testing-integration-mailpit` plugin SHALL pin to `axllent/mailpit` at a fixed minor tag (e.g. `axllent/mailpit:v1.20`). The plugin MUST NOT use the `latest` tag. A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned to minor
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses an `axllent/mailpit:v<major>.<minor>` tag — never `axllent/mailpit:latest`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward both `MAILPIT_SMTP_URL` and `MAILPIT_API_URL`. When BOTH are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode. A partial-set (only one of the two) MUST throw `INTEGRATION.MAILPIT.PARTIAL_ENV` at construction.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns a record with exactly the keys `MAILPIT_SMTP_URL` and `MAILPIT_API_URL`
#### Scenario: Partial env-bridge rejected
- **WHEN** only `MAILPIT_SMTP_URL` is set on `process.env` at construction
- **THEN** the constructor throws with `code === "INTEGRATION.MAILPIT.PARTIAL_ENV"`
### Requirement: Healthcheck contract — API ping
`isHealthy()` SHALL issue an HTTP `GET` to the Mailpit API health endpoint and treat a 2xx response as healthy. The fixture constructor SHALL accept an optional `fetch` parameter for test injection.
#### Scenario: 2xx is healthy
- **WHEN** the API health endpoint returns 200
- **THEN** `isHealthy()` resolves `true`
### Requirement: assertMail() helper
The package SHALL export `assertMail(opts: { to?: string | string[]; from?: string; subject?: string; subjectMatch?: "exact" | "partial" })` that queries the Mailpit API and asserts at least one matching message exists. **Address matching MUST be exact.** **Subject matching defaults to `"partial"`** (substring on `subject`); callers MAY opt into exact matching. On no match, the helper MUST throw with `code === "INTEGRATION.MAILPIT.NO_MATCH"` and `context` listing the candidates that were considered.
#### Scenario: Partial subject match by default
- **WHEN** Mailpit holds a message with subject `"Welcome to Suaveplan, please confirm your email"` and `assertMail({ subject: "Welcome to Suaveplan" })` is invoked
- **THEN** the helper resolves without throwing
#### Scenario: Exact address match required
- **WHEN** Mailpit holds a message addressed to `user@example.com` and `assertMail({ to: "USER@example.com" })` is invoked without case-insensitivity opt-in
- **THEN** the helper rejects with `INTEGRATION.MAILPIT.NO_MATCH`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.MAILPIT\.[A-Z_]+$`.
#### Scenario: Coded helper failure
- **WHEN** `assertMail()` finds no match
- **THEN** the rejection's `code === "INTEGRATION.MAILPIT.NO_MATCH"` AND `context.candidates` is an array

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: PostgreSQL 16 image policy
The `@suaveplan/testing-integration-postgres` plugin SHALL pin to `postgres:16` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `postgres:16`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the environment variable `POSTGRES_URL`. When `POSTGRES_URL` is set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns POSTGRES_URL
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns `{ POSTGRES_URL: "postgres://…" }`
#### Scenario: Passthrough mode skips container
- **WHEN** `POSTGRES_URL` is set at construction
- **THEN** `start()` does not invoke the container factory AND `stop()` is a no-op
### Requirement: Configurable database / user / password
The fixture options SHALL accept `database?: string; user?: string; password?: string`. Defaults MUST be `database = "test"`, `user = "test"`, `password` randomly generated per run with at least 32 bytes of entropy. The container's `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` env vars MUST be set from these values.
#### Scenario: Defaults applied
- **WHEN** the fixture is constructed without options
- **THEN** `config.database === "test"` AND `config.user === "test"` AND `config.password` is a non-empty random string
### Requirement: Healthcheck contract — pg_isready
`isHealthy()` SHALL execute `pg_isready -U <user> -d <database>` **inside** the Postgres container via testcontainers' `exec` API. The fixture constructor SHALL accept an optional `execFn` that defaults to the container's exec method and MAY be substituted for tests. A 0 exit code means healthy; any other exit code resolves to `false`.
#### Scenario: pg_isready exit 0 is healthy
- **WHEN** `pg_isready` exits with code 0
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Non-zero exit is unhealthy without throwing
- **WHEN** `pg_isready` exits with code 2
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.POSTGRES\.[A-Z_]+$` (e.g. `INTEGRATION.POSTGRES.START_FAILED`, `INTEGRATION.POSTGRES.EXEC_FAILED`).
#### Scenario: Coded exec failure
- **WHEN** the injected `execFn` throws
- **THEN** `isHealthy()` resolves `false` AND the OTel error counter records `code = "INTEGRATION.POSTGRES.EXEC_FAILED"`

View File

@@ -0,0 +1,56 @@
## ADDED Requirements
### Requirement: Redis 7 image policy
The `@suaveplan/testing-integration-redis` plugin SHALL pin to `redis:7` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `redis:7`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the environment variable `REDIS_URL`. When `REDIS_URL` is set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode and parse the URL into host/port/password fields populated on `config`.
#### Scenario: env() returns REDIS_URL
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns `{ REDIS_URL: "redis://…" }`
#### Scenario: URL parsing in passthrough mode
- **WHEN** `REDIS_URL=redis://:secret@redis.local:6380` is set at construction
- **THEN** `config.host === "redis.local"` AND `config.port === 6380` AND `config.password === "secret"` AND container start is skipped
### Requirement: Healthcheck contract
`isHealthy()` SHALL open a TCP connection to the configured host/port, send the Redis `PING` command, and resolve `true` on receipt of `+PONG\r\n`. Connection errors, malformed replies, AND any non-PONG payload MUST resolve to `false`, never throw.
#### Scenario: PING/PONG roundtrip
- **WHEN** the Redis server replies `+PONG\r\n` to a PING
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Connection refused
- **WHEN** the port is closed
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Injectable TCP ping
The fixture constructor SHALL accept an optional `tcpPingFn` parameter. The default opens a real TCP socket; tests MAY replace it with a deterministic stub.
#### Scenario: Injected ping drives health probe
- **WHEN** `tcpPingFn` is replaced with `async () => false`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.REDIS\.[A-Z_]+$` (e.g. `INTEGRATION.REDIS.START_FAILED`, `INTEGRATION.REDIS.STOP_FAILED`, `INTEGRATION.REDIS.URL_PARSE_FAILED`).
#### Scenario: Malformed REDIS_URL is coded
- **WHEN** `REDIS_URL` is set to a string that is not a valid URL
- **THEN** the constructor throws with `code === "INTEGRATION.REDIS.URL_PARSE_FAILED"` AND `context.value` echoes the bad input

View File

@@ -0,0 +1,65 @@
## ADDED Requirements
### Requirement: SurrealDB v3 image policy
The `@suaveplan/testing-integration-surrealdb` plugin SHALL pin to `surrealdb/surrealdb:v3.0.4` (name + minor version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`; that pinning is **out of scope** for this change.
#### Scenario: Image tag pinned
- **WHEN** the fixture's container factory is invoked without injection
- **THEN** the resulting `GenericContainer` is constructed with image `surrealdb/surrealdb:v3.0.4`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward the following environment variables: `SURREALDB_URL`, `SURREALDB_USERNAME`, `SURREALDB_PASSWORD`. When all three are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode and skip container start.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started in container mode
- **THEN** `fixture.env()` returns a record whose keys are exactly `SURREALDB_URL`, `SURREALDB_USERNAME`, `SURREALDB_PASSWORD`
#### Scenario: Env-bridge passthrough
- **WHEN** all three env vars are set on `process.env` at construction
- **THEN** `start()` does not invoke the container factory AND `stop()` is a no-op
### Requirement: Healthcheck contract
`isHealthy()` SHALL issue an HTTP `GET /health` request to the SurrealDB endpoint. A 2xx response means healthy. Connection errors AND non-2xx responses MUST resolve to `false`, never throw.
#### Scenario: 2xx is healthy
- **WHEN** `/health` returns 200
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Connection refused does not throw
- **WHEN** the endpoint refuses the connection
- **THEN** `isHealthy()` resolves `false` and does not throw
### Requirement: Random password per run
In container mode the fixture SHALL generate a fresh random password per `start()` call. The password MUST be at least 32 bytes of entropy, rendered as URL-safe base64 or hex.
#### Scenario: Two independent fixtures generate different passwords
- **WHEN** two fixtures start in the same process
- **THEN** their `config.password` values differ
### Requirement: Injectable factories
The fixture constructor SHALL accept optional `containerFactory` and `fetch` parameters. The defaults invoke `testcontainers` and the global `fetch`. Test-of-test code MAY substitute these to drive deterministic behaviour.
#### Scenario: Injected fetch drives health probe
- **WHEN** `fetch` is replaced with a stub that always returns a 503
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.SURREALDB\.[A-Z_]+$` (e.g. `INTEGRATION.SURREALDB.START_FAILED`, `INTEGRATION.SURREALDB.STOP_FAILED`). Cross-cutting codes (`INTEGRATION.HEALTH_TIMEOUT`, `INTEGRATION.ABORTED`) are inherited from `@suaveplan/testing-integration`.
#### Scenario: Coded error on start failure
- **WHEN** the container fails to bind a port
- **THEN** `start()` rejects with a structured error whose `code === "INTEGRATION.SURREALDB.START_FAILED"`

View File

@@ -0,0 +1,60 @@
## ADDED Requirements
### Requirement: Typesense 27 image policy
The `@suaveplan/testing-integration-typesense` plugin SHALL pin to `typesense/typesense:27` (name + major version). A follow-up operational change tracked at `docs/superpowers/specs/2026-04-25-test-suite-audit.md §5` will pin to `sha256:<digest>`.
#### Scenario: Image tag pinned
- **WHEN** the container factory is invoked without injection
- **THEN** the resulting container uses image `typesense/typesense:27`
### Requirement: Env-bridge contract
The plugin SHALL declare and forward both `TYPESENSE_URL` and `TYPESENSE_API_KEY`. When both are set on the parent process at construction time, the fixture MUST enter env-bridge passthrough mode.
#### Scenario: env() returns the full set
- **WHEN** the fixture has started
- **THEN** `fixture.env()` returns a record with exactly the keys `TYPESENSE_URL` and `TYPESENSE_API_KEY`
### Requirement: Random API key per run
In container mode the fixture SHALL generate a fresh random API key per `start()` call with at least 32 bytes of entropy. The container MUST be started with this key as its `TYPESENSE_API_KEY`.
#### Scenario: Two fixtures generate different keys
- **WHEN** two fixtures start in the same process
- **THEN** their `config.apiKey` values differ
### Requirement: Healthcheck contract — JSON `{ ok: true }`
`isHealthy()` SHALL issue an HTTP `GET /health` to the Typesense endpoint with the configured API key in the `X-TYPESENSE-API-KEY` header, parse the response as JSON, and resolve `true` when `body.ok === true`. Connection errors, non-2xx responses, AND any non-`{ ok: true }` body MUST resolve to `false`, never throw.
#### Scenario: Healthy response
- **WHEN** `/health` returns `200` with body `{ "ok": true }`
- **THEN** `isHealthy()` resolves `true`
#### Scenario: Body not ok
- **WHEN** `/health` returns `200` with body `{ "ok": false, "resource_error": "OUT_OF_DISK" }`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Injectable fetch
The fixture constructor SHALL accept an optional `fetch` parameter; the default is the global `fetch`.
#### Scenario: Injected fetch drives health probe
- **WHEN** `fetch` is replaced with a stub returning `{ ok: false }`
- **THEN** `isHealthy()` resolves `false`
### Requirement: Error-code prefix
Errors thrown from the plugin SHALL carry codes matching `^INTEGRATION\.TYPESENSE\.[A-Z_]+$`.
#### Scenario: Start failure is coded
- **WHEN** the container fails to start
- **THEN** `start()` rejects with `code === "INTEGRATION.TYPESENSE.START_FAILED"`

View File

@@ -0,0 +1,186 @@
## ADDED Requirements
### Requirement: ServiceFixture<TConfig> contract
The `@suaveplan/testing-integration` package SHALL export an interface `ServiceFixture<TConfig extends Record<string, unknown>>` that every plugin fixture MUST implement. The interface defines the universal lifecycle of a service used by integration tests, whether backed by a testcontainers-managed Docker container or by an env-bridge passthrough to an already-running service.
The interface MUST expose the following members:
- `readonly name: string` — globally unique among fixtures registered into one `TestEnvironment`.
- `readonly config: TConfig` — the resolved configuration after `start()`; before `start()` MAY be a partial draft.
- `readonly dependsOn?: readonly string[]` — names of fixtures that MUST start before this one.
- `start(opts: { signal: AbortSignal }): Promise<void>` — owns container startup and the health-poll loop; honours abort.
- `stop(opts: { signal: AbortSignal }): Promise<void>` — releases all resources; idempotent; no-op in env-bridge passthrough mode.
- `isHealthy(): Promise<boolean>` — single health probe; called by `HealthPoller`; never throws.
- `env(): Record<string, string>` — the full set of environment variables the fixture forwards to worker pools.
A utility type `InferConfig<F>` SHALL extract `TConfig` from a `ServiceFixture` implementation; it returns `never` for non-fixture types.
#### Scenario: Fixture exposes the full env set
- **WHEN** a plugin instance is constructed and `env()` is read
- **THEN** the returned record contains every environment variable name the plugin's spec declares — no implicit filtering
#### Scenario: Idempotent stop is safe
- **WHEN** `stop()` is awaited twice on the same fixture instance
- **THEN** the second call resolves without throwing AND does not attempt to interact with already-released resources
#### Scenario: InferConfig narrows TConfig
- **WHEN** `InferConfig<RedisFixture>` is evaluated where `RedisFixture extends ServiceFixture<{ url: string }>`
- **THEN** the resulting type is `{ url: string }`
### Requirement: TestEnvironment topological lifecycle
`TestEnvironment` SHALL accept a `Record<string, ServiceFixture<…>>` plus `TestEnvironmentOptions` and expose `setupAll()` / `teardownAll()`. `setupAll()` MUST compute a topological ordering from each fixture's `dependsOn` and start fixtures in parallel **within** each topological layer. `teardownAll()` MUST stop fixtures in reverse-topological order. A `started` flag MUST prevent double-start. A fixture that never started MUST never be stopped. A failing stop MUST NOT short-circuit teardown — `teardownAll()` MUST drain remaining fixtures and aggregate errors.
#### Scenario: Parallel start within a layer
- **WHEN** three fixtures `a`, `b`, `c` have no `dependsOn` and `setupAll()` is called
- **THEN** all three `start()` calls are in flight before any resolves
#### Scenario: Topological order respects dependsOn
- **WHEN** fixture `app` declares `dependsOn: ["oidc"]` and `setupAll()` runs
- **THEN** `oidc.start()` resolves before `app.start()` is invoked
#### Scenario: Reverse-topological stop
- **WHEN** `teardownAll()` runs against the same set
- **THEN** `app.stop()` is invoked before `oidc.stop()`
#### Scenario: Failed start does not start dependents and does not block teardown
- **WHEN** `oidc.start()` rejects
- **THEN** `app.start()` is never invoked AND `teardownAll()` calls `oidc.stop()` only if `oidc.start()` had begun, AND remaining fixtures' `stop()` calls all run regardless of any single failure
#### Scenario: Double setup throws
- **WHEN** `setupAll()` is awaited twice on the same instance
- **THEN** the second call rejects with `INTEGRATION.ALREADY_STARTED`
### Requirement: HealthPoller contract
`HealthPoller` SHALL accept `{ intervalMs?: number; timeoutMs?: number; sleep?: (ms: number) => Promise<void> }` with defaults `intervalMs = 500` and `timeoutMs = 120_000`. `poll(fixture, { signal })` MUST call `fixture.isHealthy()` repeatedly until it returns `true` or the deadline elapses. On deadline, the promise rejects with a `TestEnvironmentError` whose `code === "INTEGRATION.HEALTH_TIMEOUT"`, `context.fixtureName`, and `context.elapsedMs` are populated. The poller MUST honour `AbortSignal` and reject with `INTEGRATION.ABORTED` on first observed abort.
#### Scenario: Healthy fixture resolves promptly
- **WHEN** `fixture.isHealthy()` returns `true` on the first call
- **THEN** `poll()` resolves before a second call is scheduled
#### Scenario: Deadline elapsed
- **WHEN** `fixture.isHealthy()` always returns `false` and 120 s elapse
- **THEN** `poll()` rejects with `INTEGRATION.HEALTH_TIMEOUT` AND `context.elapsedMs >= 120000`
#### Scenario: Abort honoured
- **WHEN** the caller-provided `signal` aborts mid-poll
- **THEN** `poll()` rejects with `INTEGRATION.ABORTED` within one event-loop tick of the abort
### Requirement: skipIfUnavailable() helper
The package SHALL export `skipIfUnavailable(fixtureName: string): void` that, when called from the body of a Vitest `it`/`test`, causes the test to be **skipped** (not failed) when the named fixture failed to start during `globalSetup`. The helper MUST read the inject key `__suaveplan_integration_failed__` and gracefully no-op when the inject key is absent (e.g. unit-test mode). A bun-test mode SHALL emit the equivalent `it.skip` behaviour.
#### Scenario: Skip on partial failure
- **WHEN** `surrealdb.start()` failed during global setup, `provide("__suaveplan_integration_failed__", { surrealdb: true })` was called, and a test calls `skipIfUnavailable("surrealdb")`
- **THEN** the test is reported as skipped, not failed
#### Scenario: No-op when inject key absent
- **WHEN** the test runner has no global-setup wired and `skipIfUnavailable("anything")` is called
- **THEN** the call returns without throwing AND the test continues
### Requirement: createIntegrationSetup() factory and Vitest global-setup bridge
`createIntegrationSetup(fixtures)` SHALL register the supplied fixture map in a module-level registry and return a `globalSetup` array suitable for a Vitest config. At Vitest startup, the bridge MUST construct a single shared `TestEnvironment` from the registered fixtures, run `setupAll()`, and on partial failure call `provide("__suaveplan_integration_failed__", failures)` where `failures` is `Record<string, true>` keyed by fixture name. On full success it MUST `provide()` an empty record. Every env var returned by every fixture's `env()` MUST be set on `process.env` before the worker pool is forked.
#### Scenario: Successful setup propagates env
- **WHEN** all fixtures start successfully and their `env()` returns disjoint keys
- **THEN** every key is present in `process.env` before any test worker forks
#### Scenario: Partial failure is reported via inject key
- **WHEN** one fixture's `start()` rejects
- **THEN** `provide("__suaveplan_integration_failed__", { <failedName>: true })` is invoked AND the worker pool still starts so unaffected tests can run
### Requirement: Deterministic injection propagation via TestEnvironmentOptions
`TestEnvironmentOptions` SHALL include `clock?: Clock; rng?: Rng; signal?: AbortSignal`. When a fixture's config declares `acceptsContext: true`, `TestEnvironment` MUST forward the same `clock` / `rng` / `signal` references to that fixture before invoking `start()`. Fixtures that do not opt in MUST be unaffected. This requirement extends the deterministic-injection contract from `openspec/specs/testing/spec.md` into the integration substrate.
#### Scenario: Opt-in fixture sees the shared clock
- **WHEN** `TestEnvironment` is constructed with `{ clock: fakeClock }` and a fixture's config declares `acceptsContext: true`
- **THEN** that fixture's `config.clock === fakeClock` at the time `start()` is invoked
#### Scenario: Non-opt-in fixture is unaffected
- **WHEN** the same `TestEnvironment` includes a fixture whose config does NOT declare `acceptsContext: true`
- **THEN** that fixture's config is left untouched
### Requirement: Leak detection on teardown
After `teardownAll()` resolves, `TestEnvironment` SHALL inspect the host runtime's active-handle set (e.g. `process._getActiveHandles()` on Node, the Bun equivalent on Bun) and emit a structured warning with code `INTEGRATION.LEAK_DETECTED` plus an OTel counter increment `testing.integration.leak.count` for each handle attributable to a fixture that should have released. The warning MUST identify the fixture by name when attribution is possible.
#### Scenario: Handle leak detected
- **WHEN** a fixture's `stop()` returns but a TCP socket it owned remains open
- **THEN** `teardownAll()` emits an `INTEGRATION.LEAK_DETECTED` warning with `context.fixtureName` AND increments `testing.integration.leak.count`
### Requirement: Flake quarantine
The package SHALL export `markFlaky(testFn, { retries, reason })` that wraps a test with bounded retries and emits a `testing.integration.flake.count` metric increment per retry attempt. Quarantined tests MUST be enumerated in a `quarantine.json` at the package root listing `{ test: string; reason: string; addedAt: string }` entries. A test that exhausts its retries MUST fail with `INTEGRATION.QUARANTINED` carrying the `reason` in `context`.
#### Scenario: Flaky test passes within budget
- **WHEN** `markFlaky(fn, { retries: 3, reason: "TC-123" })` is run and the underlying test fails twice then passes
- **THEN** the test is reported as passing AND `testing.integration.flake.count` is incremented twice
#### Scenario: Flaky test exhausts budget
- **WHEN** the underlying test fails on every retry
- **THEN** the wrapping test fails with `INTEGRATION.QUARANTINED` AND `context.reason === "TC-123"`
### Requirement: Coverage gate
Istanbul statement, branch, function, and line coverage across `packages/testing/testing-integration/src/` MUST each be exactly `100 %`. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing-integration`) MUST hard-fail below `100 %` on any dimension.
#### Scenario: Coverage gate enforced
- **WHEN** the pre-publish gate runs
- **THEN** statement / branch / function / line coverage are each `100 %`
### Requirement: OTel observability
Every public entrypoint SHALL emit OTel counters (`testing.integration.<op>.count`), histograms (`testing.integration.<op>.duration_ms`), and an error counter (`testing.integration.<op>.errors{code}`). Spans SHALL be created for `setupAll`, `teardownAll`, and `HealthPoller.poll`. When no SDK is registered, all instrumentation MUST be a no-op with overhead below 5 % per op vs an unwrapped baseline.
#### Scenario: Counter increments on every setup
- **WHEN** `setupAll()` runs N times against a registered SDK
- **THEN** `testing.integration.setup.count` records N increments
#### Scenario: Error counter records code attribute
- **WHEN** `HealthPoller.poll` rejects with `INTEGRATION.HEALTH_TIMEOUT`
- **THEN** `testing.integration.health-poll.errors` records `code = "INTEGRATION.HEALTH_TIMEOUT"`
### Requirement: Banned workspace deps
`@suaveplan/testing-integration` `dependencies`, `peerDependencies`, and `devDependencies` MAY list `@suaveplan/types` (the documented types-only exception in CLAUDE.md rule 8) but MUST NOT list any other `@suaveplan/*` workspace package. `testcontainers` MUST be a peer dependency.
#### Scenario: Dep-graph guard
- **WHEN** `bunx turbo ls --filter=@suaveplan/testing-integration...` runs
- **THEN** the only `@suaveplan/*` package in the closure is `@suaveplan/types` AND `testcontainers` resolves via peer dependency
### Requirement: Inject-key namespace
The integration substrate SHALL use the inject key `__suaveplan_integration_failed__` (renamed from the development-time literal `__integration_failed__` used in the implementation plans) for partial-failure propagation. The key MUST NOT collide with any other inject key in the workspace.
#### Scenario: Renamed inject key is the contract
- **WHEN** any consumer calls `inject("__suaveplan_integration_failed__")`
- **THEN** the key resolves to a `Record<string, true>` (possibly empty) populated by `createIntegrationSetup`'s global-setup bridge

View File

@@ -0,0 +1,58 @@
## MODIFIED Requirements
### Requirement: Coverage and docs gates
> Original: "Istanbul statement + branch coverage across `packages/universal/testing/src/` (excluding `*.bench.ts` and `*.schemas.ts`) MUST be ≥ 95%. […]"
Istanbul statement, branch, function, AND line coverage across `packages/testing/testing/src/` (excluding `*.bench.ts`, `*.schemas.ts`, barrel `index.ts` files, and inner-dir `*.types.ts`) MUST each be exactly **100 %**. There is no foundation carve-out, no degraded-situation floor, and no exception for any genesis package — the gate applies to `@suaveplan/testing` and to every other `@suaveplan/*` package equally. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing`) MUST hard-fail when any of statements, branches, functions, or lines is below `100 %`.
Every implementation file (`*.ts` other than `*.test.ts`, `*.bench.ts`, `*.schemas.ts`, inner-dir `*.types.ts`, barrel `index.ts`) MUST have a sibling `.md` file of ≥ 200 words (excluding code fences) with a simple and an advanced usage example. Mutation testing, contract testing, property testing, and fuzz coverage layer **on top of** 100 % line coverage; they do not substitute for it, and 100 % line coverage does not substitute for them.
#### Scenario: Coverage gate at 100 %
- **WHEN** `bunx turbo test:coverage --filter=@suaveplan/testing` runs
- **THEN** reported statement, branch, function, and line coverage are each `100 %` AND the gate fails the build if any dimension drops below `100 %`
#### Scenario: Docs coverage enforced
- **WHEN** a script enumerates implementation files under `src/` (excluding the stated exclusions) and checks for sibling `.md` files
- **THEN** every implementation file has a sibling `.md` AND every `.md` is ≥ 200 non-code words
### Requirement: Property-based coverage for pure helpers
> Original: "Pure functions and deterministic state machines in the package […] SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants […]"
Pure functions and deterministic state machines in `@suaveplan/testing` SHALL have fast-check property tests covering round-trip, idempotency, associativity, and boundary invariants as applicable. **The same requirement extends to every other pure-function `@suaveplan/*` package** — every package whose public surface includes deterministic, side-effect-free helpers MUST ship fast-check property tests that exercise round-trip, idempotency, associativity, or boundary invariants where each applies. Load-bearing examples include `@suaveplan/crypto-utils` (encoding round-trips), `@suaveplan/diff` (apply ∘ invert, structural equality), `@suaveplan/pagination` (cursor encode/decode round-trip, slice associativity), and `@suaveplan/validation` (parse / safeParse equivalence on valid inputs).
#### Scenario: Builder composition is associative
- **WHEN** fast-check runs over arbitrary RequestBuilder presets `a`, `b`, `c` applied in different groupings
- **THEN** `(a then b) then c` produces the same final state as `a then (b then c)`
#### Scenario: Pure-function package ships round-trip property
- **WHEN** any genesis package whose public surface is pure (e.g. `@suaveplan/crypto-utils` `hexEncode`/`hexDecode`) is audited for property tests
- **THEN** at least one fast-check test exists asserting `decode(encode(x)) === x` for arbitrary `Uint8Array` inputs
### Requirement: Deterministic clock and RNG via dependency injection
> Original: "Every public function that reads time, randomness, or iterates SHALL accept an optional `{ clock?: Clock; rng?: Rng; signal?: AbortSignal }` options parameter. […]"
Every public function in `@suaveplan/testing` that reads time, randomness, or iterates SHALL accept an optional `{ clock?: Clock; rng?: Rng; signal?: AbortSignal }` options parameter. Zero-config callers MUST get behaviorally stable defaults: `defaultClock` (backed by `Date.now` / `performance.now`), `cryptoRng` (backed by `crypto.getRandomValues`), and no abort.
**This contract additionally extends to integration test fixtures via `TestEnvironmentOptions`** — `@suaveplan/testing-integration`'s `TestEnvironment` MUST accept `{ clock?, rng?, signal? }` and propagate them to fixtures whose configs declare `acceptsContext: true`, so that integration tests can run against the same deterministic time and randomness sources as unit tests.
#### Scenario: Injected clock controls observed time
- **WHEN** `waitFor(predicate, { clock: fakeClock })` runs and `fakeClock.advance(1000)` is called
- **THEN** the helper observes 1000 ms elapsed AND does NOT read `Date.now()` or `performance.now()` directly
#### Scenario: Injected RNG produces deterministic factory output
- **WHEN** `Factory.build({ rng: seededRng(42) })` is called twice with the same factory definition
- **THEN** both calls return byte-identical output (for factories whose definition is otherwise deterministic)
#### Scenario: TestEnvironment forwards clock to opt-in fixtures
- **WHEN** a `TestEnvironment` is constructed with `{ clock: fakeClock }` and a fixture's config declares `acceptsContext: true`
- **THEN** that fixture's `config.clock === fakeClock` at the time `start()` is invoked