suaveplan/openspec/changes/enterprise-testing-integration/specs/testing-integration/spec.md

ADDED Requirements

Requirement: ServiceFixture contract

The @suaveplan/testing-integration package SHALL export an interface ServiceFixture<TConfig extends Record<string, unknown>> that every plugin fixture MUST implement. The interface defines the universal lifecycle of a service used by integration tests, whether backed by a testcontainers-managed Docker container or by an env-bridge passthrough to an already-running service.

The interface MUST expose the following members:

• readonly name: string — globally unique among fixtures registered into one TestEnvironment.
• readonly config: TConfig — the resolved configuration after start(); before start() MAY be a partial draft.
• readonly dependsOn?: readonly string[] — names of fixtures that MUST start before this one.
• start(opts: { signal: AbortSignal }): Promise<void> — owns container startup and the health-poll loop; honours abort.
• stop(opts: { signal: AbortSignal }): Promise<void> — releases all resources; idempotent; no-op in env-bridge passthrough mode.
• isHealthy(): Promise<boolean> — single health probe; called by HealthPoller; never throws.
• env(): Record<string, string> — the full set of environment variables the fixture forwards to worker pools.

A utility type InferConfig<F> SHALL extract TConfig from a ServiceFixture implementation; it returns never for non-fixture types.

Scenario: Fixture exposes the full env set

• WHEN a plugin instance is constructed and env() is read
• THEN the returned record contains every environment variable name the plugin's spec declares — no implicit filtering

Scenario: Idempotent stop is safe

• WHEN stop() is awaited twice on the same fixture instance
• THEN the second call resolves without throwing AND does not attempt to interact with already-released resources

Scenario: InferConfig narrows TConfig

• WHEN InferConfig<RedisFixture> is evaluated where RedisFixture extends ServiceFixture<{ url: string }>
• THEN the resulting type is { url: string }

Requirement: TestEnvironment topological lifecycle

TestEnvironment SHALL accept a Record<string, ServiceFixture<…>> plus TestEnvironmentOptions and expose setupAll() / teardownAll(). setupAll() MUST compute a topological ordering from each fixture's dependsOn and start fixtures in parallel within each topological layer. teardownAll() MUST stop fixtures in reverse-topological order. A started flag MUST prevent double-start. A fixture that never started MUST never be stopped. A failing stop MUST NOT short-circuit teardown — teardownAll() MUST drain remaining fixtures and aggregate errors.

Scenario: Parallel start within a layer

• WHEN three fixtures a, b, c have no dependsOn and setupAll() is called
• THEN all three start() calls are in flight before any resolves

Scenario: Topological order respects dependsOn

• WHEN fixture app declares dependsOn: ["oidc"] and setupAll() runs
• THEN oidc.start() resolves before app.start() is invoked

Scenario: Reverse-topological stop

• WHEN teardownAll() runs against the same set
• THEN app.stop() is invoked before oidc.stop()

Scenario: Failed start does not start dependents and does not block teardown

• WHEN oidc.start() rejects
• THEN app.start() is never invoked AND teardownAll() calls oidc.stop() only if oidc.start() had begun, AND remaining fixtures' stop() calls all run regardless of any single failure

Scenario: Double setup throws

• WHEN setupAll() is awaited twice on the same instance
• THEN the second call rejects with INTEGRATION.ALREADY_STARTED

Requirement: HealthPoller contract

HealthPoller SHALL accept { intervalMs?: number; timeoutMs?: number; sleep?: (ms: number) => Promise<void> } with defaults intervalMs = 500 and timeoutMs = 120_000. poll(fixture, { signal }) MUST call fixture.isHealthy() repeatedly until it returns true or the deadline elapses. On deadline, the promise rejects with a TestEnvironmentError whose code === "INTEGRATION.HEALTH_TIMEOUT", context.fixtureName, and context.elapsedMs are populated. The poller MUST honour AbortSignal and reject with INTEGRATION.ABORTED on first observed abort.

Scenario: Healthy fixture resolves promptly

• WHEN fixture.isHealthy() returns true on the first call
• THEN poll() resolves before a second call is scheduled

Scenario: Deadline elapsed

• WHEN fixture.isHealthy() always returns false and 120 s elapse
• THEN poll() rejects with INTEGRATION.HEALTH_TIMEOUT AND context.elapsedMs >= 120000

Scenario: Abort honoured

• WHEN the caller-provided signal aborts mid-poll
• THEN poll() rejects with INTEGRATION.ABORTED within one event-loop tick of the abort

Requirement: skipIfUnavailable() helper

The package SHALL export skipIfUnavailable(fixtureName: string): void that, when called from the body of a Vitest it/test, causes the test to be skipped (not failed) when the named fixture failed to start during globalSetup. The helper MUST read the inject key __suaveplan_integration_failed__ and gracefully no-op when the inject key is absent (e.g. unit-test mode). A bun-test mode SHALL emit the equivalent it.skip behaviour.

Scenario: Skip on partial failure

• WHEN surrealdb.start() failed during global setup, provide("__suaveplan_integration_failed__", { surrealdb: true }) was called, and a test calls skipIfUnavailable("surrealdb")
• THEN the test is reported as skipped, not failed

Scenario: No-op when inject key absent

• WHEN the test runner has no global-setup wired and skipIfUnavailable("anything") is called
• THEN the call returns without throwing AND the test continues

Requirement: createIntegrationSetup() factory and Vitest global-setup bridge

createIntegrationSetup(fixtures) SHALL register the supplied fixture map in a module-level registry and return a globalSetup array suitable for a Vitest config. At Vitest startup, the bridge MUST construct a single shared TestEnvironment from the registered fixtures, run setupAll(), and on partial failure call provide("__suaveplan_integration_failed__", failures) where failures is Record<string, true> keyed by fixture name. On full success it MUST provide() an empty record. Every env var returned by every fixture's env() MUST be set on process.env before the worker pool is forked.

Scenario: Successful setup propagates env

• WHEN all fixtures start successfully and their env() returns disjoint keys
• THEN every key is present in process.env before any test worker forks

Scenario: Partial failure is reported via inject key

• WHEN one fixture's start() rejects
• THEN provide("__suaveplan_integration_failed__", { <failedName>: true }) is invoked AND the worker pool still starts so unaffected tests can run

Requirement: Deterministic injection propagation via TestEnvironmentOptions

TestEnvironmentOptions SHALL include clock?: Clock; rng?: Rng; signal?: AbortSignal. When a fixture's config declares acceptsContext: true, TestEnvironment MUST forward the same clock / rng / signal references to that fixture before invoking start(). Fixtures that do not opt in MUST be unaffected. This requirement extends the deterministic-injection contract from openspec/specs/testing/spec.md into the integration substrate.

Scenario: Opt-in fixture sees the shared clock

• WHEN TestEnvironment is constructed with { clock: fakeClock } and a fixture's config declares acceptsContext: true
• THEN that fixture's config.clock === fakeClock at the time start() is invoked

Scenario: Non-opt-in fixture is unaffected

• WHEN the same TestEnvironment includes a fixture whose config does NOT declare acceptsContext: true
• THEN that fixture's config is left untouched

Requirement: Leak detection on teardown

After teardownAll() resolves, TestEnvironment SHALL inspect the host runtime's active-handle set (e.g. process._getActiveHandles() on Node, the Bun equivalent on Bun) and emit a structured warning with code INTEGRATION.LEAK_DETECTED plus an OTel counter increment testing.integration.leak.count for each handle attributable to a fixture that should have released. The warning MUST identify the fixture by name when attribution is possible.

Scenario: Handle leak detected

• WHEN a fixture's stop() returns but a TCP socket it owned remains open
• THEN teardownAll() emits an INTEGRATION.LEAK_DETECTED warning with context.fixtureName AND increments testing.integration.leak.count

Requirement: Flake quarantine

The package SHALL export markFlaky(testFn, { retries, reason }) that wraps a test with bounded retries and emits a testing.integration.flake.count metric increment per retry attempt. Quarantined tests MUST be enumerated in a quarantine.json at the package root listing { test: string; reason: string; addedAt: string } entries. A test that exhausts its retries MUST fail with INTEGRATION.QUARANTINED carrying the reason in context.

Scenario: Flaky test passes within budget

• WHEN markFlaky(fn, { retries: 3, reason: "TC-123" }) is run and the underlying test fails twice then passes
• THEN the test is reported as passing AND testing.integration.flake.count is incremented twice

Scenario: Flaky test exhausts budget

• WHEN the underlying test fails on every retry
• THEN the wrapping test fails with INTEGRATION.QUARANTINED AND context.reason === "TC-123"

Requirement: Coverage gate

Istanbul statement, branch, function, and line coverage across packages/testing/testing-integration/src/ MUST each be exactly 100 %. The pre-publish gate (bunx turbo test:coverage --filter=@suaveplan/testing-integration) MUST hard-fail below 100 % on any dimension.

Scenario: Coverage gate enforced

• WHEN the pre-publish gate runs
• THEN statement / branch / function / line coverage are each 100 %

Requirement: OTel observability

Every public entrypoint SHALL emit OTel counters (testing.integration.<op>.count), histograms (testing.integration.<op>.duration_ms), and an error counter (testing.integration.<op>.errors{code}). Spans SHALL be created for setupAll, teardownAll, and HealthPoller.poll. When no SDK is registered, all instrumentation MUST be a no-op with overhead below 5 % per op vs an unwrapped baseline.

Scenario: Counter increments on every setup

• WHEN setupAll() runs N times against a registered SDK
• THEN testing.integration.setup.count records N increments

Scenario: Error counter records code attribute

• WHEN HealthPoller.poll rejects with INTEGRATION.HEALTH_TIMEOUT
• THEN testing.integration.health-poll.errors records code = "INTEGRATION.HEALTH_TIMEOUT"

Requirement: Banned workspace deps

@suaveplan/testing-integration dependencies, peerDependencies, and devDependencies MAY list @suaveplan/types (the documented types-only exception in CLAUDE.md rule 8) but MUST NOT list any other @suaveplan/* workspace package. testcontainers MUST be a peer dependency.

Scenario: Dep-graph guard

• WHEN bunx turbo ls --filter=@suaveplan/testing-integration... runs
• THEN the only @suaveplan/* package in the closure is @suaveplan/types AND testcontainers resolves via peer dependency

Requirement: Inject-key namespace

The integration substrate SHALL use the inject key __suaveplan_integration_failed__ (renamed from the development-time literal __integration_failed__ used in the implementation plans) for partial-failure propagation. The key MUST NOT collide with any other inject key in the workspace.

Scenario: Renamed inject key is the contract

• WHEN any consumer calls inject("__suaveplan_integration_failed__")
• THEN the key resolves to a Record<string, true> (possibly empty) populated by createIntegrationSetup's global-setup bridge