64 lines
4.8 KiB
Markdown
64 lines
4.8 KiB
Markdown
## Context
|
|
|
|
The 2026-04-24 wave (umbrella design at `/mnt/s/web/genesis/docs/superpowers/specs/2026-04-24-enterprise-testbed-design.md`, plans at `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testing-integration-{core,plugins}.md` and `/mnt/s/web/genesis/docs/superpowers/plans/2026-04-24-testbed-pages-and-e2e.md`) shipped the integration substrate without a normative spec. The 2026-04-25 audit (`/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md`) is the driver for this change.
|
|
|
|
This change is normative-only. No source files are touched; the contract for the seven new packages and the four new e2e modules is fixed.
|
|
|
|
## Decisions
|
|
|
|
### D1: `ServiceFixture<TConfig>` is the universal fixture contract
|
|
|
|
Every plugin package implements one class satisfying `ServiceFixture<TConfig extends Record<string, unknown>>`:
|
|
|
|
```ts
|
|
interface ServiceFixture<TConfig> {
|
|
readonly name: string;
|
|
readonly config: TConfig;
|
|
readonly dependsOn?: readonly string[];
|
|
start(opts: { signal: AbortSignal }): Promise<void>;
|
|
stop(opts: { signal: AbortSignal }): Promise<void>;
|
|
isHealthy(): Promise<boolean>;
|
|
env(): Record<string, string>;
|
|
}
|
|
```
|
|
|
|
`InferConfig<F>` extracts `TConfig`. The interface lives in `@suaveplan/testing-integration/fixture/service-fixture` and is re-exported from the package barrel.
|
|
|
|
### D2: `TestEnvironment` lifecycle — topological start, reverse-topological stop
|
|
|
|
`TestEnvironment` accepts a `Record<string, ServiceFixture<…>>` plus `TestEnvironmentOptions { signal?, clock?, rng? }`. It computes a topological order from `dependsOn`, parallelises starts within each layer, and stops in reverse order. A fixture that never started is never stopped. Stop continues draining remaining fixtures even when one stop fails (errors are aggregated). A `started` flag prevents double-start.
|
|
|
|
### D3: `HealthPoller` contract
|
|
|
|
Default `intervalMs: 500`, `timeoutMs: 120000`. Honours `AbortSignal`. On deadline elapsed without a `true` from `isHealthy()`, throws `INTEGRATION.HEALTH_TIMEOUT` with `context.elapsedMs` and `context.fixtureName`. Optional exponential backoff cap. Uses an injectable `sleep` for testability.
|
|
|
|
### D4: Env-bridge passthrough
|
|
|
|
Every plugin SHALL pass through ALL of its declared env vars to the worker pool. When the env-bridge env var(s) are set on the parent process, the fixture skips container start and uses the existing service. Each plugin's spec enumerates its env-var contract.
|
|
|
|
### D5: `createIntegrationSetup()` + Vitest global-setup bridge
|
|
|
|
`createIntegrationSetup(fixtures)` registers fixtures in a module-level registry that `global-setup.ts` reads at Vitest startup. It returns the `globalSetup` array consumers drop into `vitest.config.ts`. On partial start failure, it `provide()`s `__suaveplan_integration_failed__` (renamed from the plans' `__integration_failed__`), and `skipIfUnavailable("name")` reads it via `inject()` to skip — never fail — affected tests.
|
|
|
|
### D6: Deterministic injection propagation
|
|
|
|
`TestEnvironmentOptions { clock?: Clock; rng?: Rng; signal?: AbortSignal }` is forwarded to every fixture whose config carries `acceptsContext: true`. Fixtures that opt in receive the same `Clock`/`Rng`/`signal` as the rest of the test runtime — preserving the determinism contract from `openspec/specs/testing/spec.md`.
|
|
|
|
### D7: Error-code namespace per plugin
|
|
|
|
Stable codes follow `INTEGRATION.<PLUGIN>.<REASON>`: e.g. `INTEGRATION.SURREALDB.START_FAILED`, `INTEGRATION.GARAGE_S3.LAYOUT_APPLY_FAILED`. Cross-cutting codes are unprefixed: `INTEGRATION.HEALTH_TIMEOUT`, `INTEGRATION.LEAK_DETECTED`, `INTEGRATION.QUARANTINED`.
|
|
|
|
### D8: Leak detection + flake quarantine
|
|
|
|
After `teardownAll()`, `TestEnvironment` SHALL assert no active handles attributable to its fixtures remain (warn + counter increment if found). `markFlaky({ retries, reason })` wraps a test with bounded retries and emits `testing.integration.flake.count`; quarantined tests are listed in `quarantine.json` at the package root.
|
|
|
|
### D9: testcontainers as peer dep, types-only `@suaveplan/types`
|
|
|
|
`@suaveplan/testing-integration` depends on `@suaveplan/types` only (the documented exception in CLAUDE.md rule 8) and `testcontainers` as a peer. Plugins depend on `@suaveplan/testing-integration` and `testcontainers`.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- **Inject-key rename** (`__integration_failed__` → `__suaveplan_integration_failed__`) is a one-line code change but normalises namespace collisions; documented as a one-shot mechanical sweep.
|
|
- **`dependsOn` topo-sort** adds complexity vs flat parallel start; mitigated by leaving the field optional with a default of `[]` (current behaviour preserved).
|
|
- **Image policy** — this change pins by name + minor version (e.g. `surrealdb/surrealdb:v3.0.4`). Pinning by `sha256` digest is deferred to a follow-up operational change so this normative change stays minimal.
|