suaveplan/openspec/changes/enterprise-testing-operational/specs/testing-integration/spec.md

ADDED Requirements

Requirement: Image pinning by tag and digest

Every Docker image referenced by any testing-integration-* plugin, by docker-compose.{dev,test,testbed}.yml, or by any testcontainers config helper SHALL be pinned by both tag AND sha256 digest in the canonical form image: <registry>/<name>:<tag>@sha256:<64-hex>. Tag-only pinning is a defect. The pin SHALL be verified by scripts/verify-image-pinning.ts, which fails CI on any reference that does not match ^[a-z0-9./-]+:[A-Za-z0-9._-]+@sha256:[a-f0-9]{64}$. Digest refresh follows a monthly dependabot-style cadence with reviewer approval; security CVE advisories trigger immediate refresh outside the monthly window.

Scenario: Tag-only image fails verification

• WHEN a plugin or compose file declares image: foo/bar:v1.2.3 without a @sha256: digest
• THEN scripts/verify-image-pinning.ts exits non-zero AND CI fails the lint stage

Scenario: Mailpit is pinned to v1.x with a digest

• WHEN the Mailpit plugin or testbed compose stack references the Mailpit image
• THEN the reference is axllent/mailpit:v1.<minor>@sha256:<digest> (never :latest)

Scenario: CVE refresh bypasses monthly cadence

• WHEN an upstream CVE advisory is filed against a pinned image
• THEN a digest-bump PR SHALL be opened immediately AND the monthly cadence does not block it

Requirement: Container resource limits

Every service in docker-compose.{dev,test,testbed}.yml SHALL declare deploy.resources.limits.{cpus,memory}. Defaults: 1.0 cpu / 1Gi for application services, 2.0 cpu / 2Gi for database services (Postgres, SurrealDB, Redis, Typesense), 1.5 cpu / 1.5Gi for object stores (Garage, MinIO). testcontainers-managed containers SHALL set HostConfig.Memory (bytes) and HostConfig.NanoCpus (cpu × 1e9) via the plugin config shape { memoryMiB?: number; cpuCount?: number }, with the same defaults applied when fields are omitted.

Scenario: Compose service lacking limits fails verification

• WHEN a service in any tracked docker-compose.*.yml does not declare deploy.resources.limits.cpus AND deploy.resources.limits.memory
• THEN the compose-lint step fails

Scenario: testcontainer translates plugin budget to docker HostConfig

• WHEN a plugin starts with { memoryMiB: 1024, cpuCount: 1.0 }
• THEN the resulting docker HostConfig.Memory === 1073741824 AND HostConfig.NanoCpus === 1_000_000_000

Requirement: CI matrix on Bun, OS, and architecture

CI SHALL execute on a matrix of bun-latest × bun-canary, ubuntu-22.04 × ubuntu-24.04 × macos-14 × windows-latest (Playwright-only on Windows), and amd64 × arm64 on Linux. Each testing-integration-* plugin SHALL declare archSupport: ('amd64'|'arm64')[] and the matrix-generator SHALL skip unsupported cells. The canonical Bun version is pinned via root package.json engines.bun; the bun-canary job is allowed to fail-open with a warning annotation. A separate Node 22 LTS job SHALL run the universal-purity suite for browser-runnable packages (testing, testing-dom); failures here block merge.

Scenario: Plugin without arm64 support skips arm64 cell

• WHEN a plugin declares archSupport: ['amd64']
• THEN the matrix-generator emits no arm64 job for that plugin AND CI does not flag the absence as a failure

Scenario: Bun canary fails

• WHEN the bun-canary job exits non-zero
• THEN the PR check surfaces a warning annotation AND merge is not blocked

Scenario: Node parity fails

• WHEN the Node 22 LTS parity job for browser-runnable packages exits non-zero
• THEN merge is blocked

Requirement: Airgap rebuild gate

A nightly workflow airgap-rebuild SHALL: pre-warm the Bun cache via a normal install, run bun install --offline from that cache, run bunx turbo build, then run bunx turbo test --filter=...[HEAD~7] with all required images pre-pulled and the test phase executed inside --network=none. Failure indicates a missing offline-cache entry and SHALL block the next release.

Scenario: Offline install missing dep fails the job

• WHEN any package pulls a dependency that is not present in the pre-warmed cache during bun install --offline
• THEN the airgap job fails AND release is blocked until the offline cache is fixed

Scenario: Test phase isolated from network

• WHEN the airgap job enters the test phase
• THEN the Docker network for the runner is --network=none AND any test that attempts external network egress fails fast

Requirement: Secret management policy

CI secrets SHALL be sourced from GitHub OIDC plus a Vault or SOPS bridge. The variables GARAGE_S3_SECRET_ACCESS_KEY, OIDC_CLIENT_SECRET, NPM_TOKEN, and PACT_BROKER_TOKEN MUST be referenced via the bridge — never inline in workflow YAML. .env.example SHALL document each secret's CI source and its local-dev source. scripts/scan-secrets.ts SHALL fail CI on any inline secret pattern using @suaveplan/security-codes detectors.

Scenario: Inline secret in workflow YAML fails CI

• WHEN a workflow declares env: NPM_TOKEN: <literal-token> (any pattern detected by @suaveplan/security-codes)
• THEN scripts/scan-secrets.ts exits non-zero AND CI fails

Scenario: Bridge resolution

• WHEN a workflow references ${{ secrets.NPM_TOKEN }} via the OIDC + Vault/SOPS bridge step
• THEN the secret is fetched at job start AND no inline value appears in the YAML source

Requirement: Port collision policy

Static port ranges (dev 8001-8902, test 7004-7011, testbed 3200) are reserved for compose stacks. testcontainers-managed integration tests SHALL use random ephemeral ports (the testcontainers default). docker/scripts/preflight.ts SHALL fail loudly if any reserved port is already bound, with a remediation hint pointing the user at either killing the conflicting process or remapping via PORTS_OFFSET.

Scenario: Bound reserved port aborts startup

• WHEN docker compose -f docker-compose.dev.yml up runs and port 8001 is already bound by another process
• THEN preflight exits non-zero AND prints the offending PID + a PORTS_OFFSET remediation hint

Scenario: PORTS_OFFSET shift

• WHEN PORTS_OFFSET=100 is set in the environment
• THEN every reserved port is shifted by 100 AND preflight passes if the shifted range is free

Requirement: OOM and suite wall-clock budgets

Per-spec memory cap SHALL default to 1Gi via the shared createTestConfig()'s pool.forks.maxMemory; packages MAY override per-package. Per-bunx turbo test invocation wall-clock ceiling SHALL be 30 minutes; a soft warning SHALL be emitted at 20 minutes; the run SHALL be aborted at 30 minutes. No automatic retries on timeout — timeout indicates a pathological condition that must be diagnosed.

Scenario: Single-spec OOM aborts that fork only

• WHEN a single test spec exceeds 1Gi resident
• THEN the forked vitest worker is killed with a TESTING.MEMORY.LIMIT_EXCEEDED error AND sibling specs continue

Scenario: Wall-clock 30-minute abort

• WHEN bunx turbo test exceeds 30 minutes wall-clock
• THEN the run is aborted with a non-zero exit AND no automatic retry is attempted

Requirement: Leak detection on fixture teardown

After every TestEnvironment.teardown() the runner SHALL assert no leaked timers, sockets, or file descriptors via Bun.peek / runtime handle table on Bun or process._getActiveHandles() on Node fallback. Leaks SHALL fail the suite with a categorised error: TESTING.LEAK.HANDLE, TESTING.LEAK.TIMER, or TESTING.LEAK.SOCKET, each carrying { count, sample } context.

Scenario: Unreleased timer leaks

• WHEN a fixture sets setInterval and does not clear it before teardown
• THEN teardown throws TESTING.LEAK.TIMER AND context.count >= 1

Scenario: Unclosed socket leaks

• WHEN a fixture opens a TCP socket and does not close it before teardown
• THEN teardown throws TESTING.LEAK.SOCKET AND context.sample contains the offending socket descriptor

Scenario: Clean teardown passes

• WHEN all timers are cleared and all sockets are closed before teardown returns
• THEN the leak assertion passes silently