suaveplan/openspec/changes/enterprise-testing-operational/design.md

Design notes

Image pinning enforcement

The verifier (scripts/verify-image-pinning.ts) walks every docker-compose.*.yml, every plugin's image-config TS source, and every reference inside testcontainers config helpers. It uses a strict regex ^[a-z0-9./-]+:[A-Za-z0-9._-]+@sha256:[a-f0-9]{64}$ and fails on any reference that does not match. Tag-only references are a hard error, not a warning, because partial enforcement is what allowed axllent/mailpit:latest to ship in the first place.

Digest refresh follows a dependabot-style monthly cadence: a scheduled job opens a PR with the updated digests pulled from docker manifest inspect. CVE advisories from upstream (Mailpit, Redis, Postgres, SurrealDB, Garage, Typesense, Verdaccio) trigger immediate refresh outside the monthly window. The reviewer SHALL verify the upstream changelog before merge.

Resource limits — compose vs testcontainers

Compose stacks declare limits under deploy.resources.limits. testcontainers does not honour the compose v3 schema, so plugin fixture configs accept { memoryMiB?: number; cpuCount?: number } and translate to HostConfig.Memory (bytes) and HostConfig.NanoCpus (cpu × 1e9). Defaults are identical between the two paths so behaviour is consistent regardless of which entry point CI uses.

Database services get the larger budget (2.0 cpu, 2Gi) because a parallel test sweep can drive Surreal/Postgres to its working-set limit. Garage and MinIO sit in the middle (1.5 cpu, 1.5Gi) because both have a memory-resident metadata index that grows with bucket size during long-running E2E runs.

CI matrix shape

Three axes — runtime, OS, arch — multiplied conservatively because Playwright on Windows triples job time and arm64 is unsupported by some images we depend on. The plugin-side archSupport declaration is required so the matrix-generator script can prune unsupported cells before scheduling. bun-canary is allowed to fail-open because canary breakage is an upstream signal, not a merge blocker — but the warning is loud and surfaces in PR checks.

The Node 22 LTS parity job runs only the universal-purity suite (no integration, no E2E). It catches accidental Bun-only API leakage in browser-runnable packages (testing, testing-dom, crypto-utils, etc.) which is otherwise invisible until a downstream consumer hits it.

Airgap rebuild

The nightly airgap-rebuild job exists because enterprise consumers regularly run in environments without internet egress at build/test time. The job pre-warms a Bun cache via a normal bun install in a separate step, then runs bun install --offline, bunx turbo build, and a network-isolated test phase. Failure indicates a missing offline-cache entry — typically a transitive dep that pulled at runtime — and blocks the next release. We test the smallest changed-since-last-week filter (...[HEAD~7]) to keep the job under the 30-minute wall-clock budget.

Verdaccio fixture

Verdaccio is the only "test-the-test-of-the-test" case where a @suaveplan/* package's primary use case (publishing) requires a fixture. We accept the meta-circularity: testing-integration-verdaccio does not depend on any @suaveplan/* package by Layer-0 rule, but bun scripts/publish.ts (a repo-level script, not a package) is exercised against it in CI.

Memory storage is the default for speed; tmpfs is available for tests that need filesystem persistence semantics. Users and packages config follow Verdaccio's native YAML shape, translated from the TS config object at fixture startup.

Network sandboxing

The default block is implemented via page.route('**/*') registered before each test. The allow-list is checked by URL origin (not full path) using the URL constructor; the auto-allow set is localhost, 127.0.0.1, and the configured testbed origin from env. The test.use({ allowExternalNetwork: true }) escape hatch logs a warning at suite start so the reviewer can challenge it during code review.

Leak detection

After every TestEnvironment.teardown() the runner snapshots active handles. On Bun we use Bun.peek and the runtime's internal handle table; on Node fallback we use process._getActiveHandles(). The diff between pre-setup and post-teardown is the leak set. Categories: HANDLE (file descriptors, dns lookups), TIMER (setTimeout/setInterval), SOCKET (TCP/UDP). The error code is granular so triage can route to the responsible fixture.

Open questions

• Whether to extend the airgap test to E2E (Playwright requires browser binaries that may not be in the offline cache by default).
• Whether quarantine.json should live at repo root or under openspec/. Preference is repo root so CI tooling does not need OpenSpec parsing.