suaveplan/openspec/changes/enterprise-testing-operational/tasks.md

Tasks

0. Pre-flight (non-negotiable)

• 0.1 Confirm clean working tree (git status reports nothing) and no in-flight worktrees with uncommitted work (git worktree list then git status in each)
• 0.2 If any worktrees exist under .claude/worktrees/, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (git worktree remove) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
• 0.3 Confirm the package set in MANIFEST.md matches find packages -maxdepth 3 -name package.json count. Drift here means a prior change archived without regenerating the manifest.

1. Image pinning

• 1.1 Author scripts/verify-image-pinning.ts (regex-based scan of compose files + plugin TS sources)
• 1.2 Wire verify-image-pinning into the root lint turbo task so CI hard-fails on violation
• 1.3 Replace axllent/mailpit:latest with axllent/mailpit:v1.x@sha256:… in the Mailpit plugin and docker-compose.testbed.yml
• 1.4 Pin every other image referenced in docker-compose.{dev,test,testbed}.yml and in every testing-integration-* plugin
• 1.5 Document monthly digest-refresh cadence + CVE-trigger policy in docker/README.md
• 1.6 Add a scheduled GitHub Actions workflow that opens a digest-bump PR monthly

2. Container resource limits

• 2.1 Add deploy.resources.limits.{cpus,memory} to every service in docker-compose.{dev,test,testbed}.yml using the documented defaults
• 2.2 Extend each testing-integration-* plugin's config shape with memoryMiB? + cpuCount? and translate to HostConfig.Memory / HostConfig.NanoCpus
• 2.3 Set defaults: 1.0 cpu / 1Gi (services), 2.0 cpu / 2Gi (databases — Postgres, SurrealDB, Redis, Typesense), 1.5 cpu / 1.5Gi (Garage, MinIO)
• 2.4 Add a unit test per plugin verifying the translated docker HostConfig matches the configured budget

3. CI matrix and airgap

• 3.1 Pin engines.bun in root package.json to the canonical genesis Bun version
• 3.2 Define matrix axes in .github/workflows/ci.yml: bun (latest, canary), os (ubuntu-22.04, ubuntu-24.04, macos-14, windows-latest), arch (amd64, arm64 — linux only)
• 3.3 Add archSupport: ['amd64','arm64'] declaration to every testing-integration-* plugin and skip unsupported matrix cells
• 3.4 Add a Node-22-LTS parity job running the universal-purity suite for browser-runnable packages
• 3.5 Mark bun-canary job as continue-on-error: true with a warning annotation
• 3.6 Author the nightly airgap-rebuild workflow (offline install → build → network-isolated test phase)
• 3.7 Document the matrix and airgap policy in docker/README.md

4. testing-integration-verdaccio

• 4.1 Scaffold packages/testing/testing-integration-verdaccio package per Layer-0 rules (no @suaveplan/* runtime deps; testcontainers + zod + @opentelemetry/api)
• 4.2 Implement VerdaccioFixture (image, env-bridge, healthcheck, config types)
• 4.3 Pin verdaccio/verdaccio:6@sha256:… and add to image-pinning verifier
• 4.4 Implement memory + tmpfs storage modes; users/packages/uplinks config translation
• 4.5 Add INTEGRATION.VERDACCIO.* error codes
• 4.6 Author integration test exercising bun scripts/publish.ts --package @suaveplan/<probe> against the fixture
• 4.7 Add the fixture to the testbed compose stack as an optional service for manual repro
• 4.8 100% Istanbul coverage gate

5. Secret management

• 5.1 Wire GitHub OIDC + Vault/SOPS bridge in .github/workflows/_secrets.yml (reusable workflow)
• 5.2 Refactor every workflow that references GARAGE_S3_SECRET_ACCESS_KEY, OIDC_CLIENT_SECRET, NPM_TOKEN, PACT_BROKER_TOKEN to source from the bridge
• 5.3 Author scripts/scan-secrets.ts using @suaveplan/security-codes detectors; fail CI on inline secret patterns
• 5.4 Update .env.example to document each secret's CI source vs local-dev source

6. Port collision and budgets

• 6.1 Author docker/scripts/preflight.ts that probes reserved port ranges (dev 8001-8902, test 7004-7011, testbed 3200) and fails loudly on collision
• 6.2 Implement PORTS_OFFSET env shift escape hatch
• 6.3 Add per-spec memory cap (pool.forks.maxMemory: 1Gi default) to the shared createTestConfig
• 6.4 Add suite wall-clock budget enforcement (20 min warn, 30 min abort) to the root turbo test task

7. Network sandboxing (testing-e2e)

• 7.1 Default-block external origins in createE2ETestConfig via page.route('**/*')
• 7.2 Auto-allow localhost, 127.0.0.1, and configured testbed origin
• 7.3 Add allowedOrigins: string[] option (default [])
• 7.4 Add test.use({ allowExternalNetwork: true }) escape hatch with a startup warning
• 7.5 Tests covering all four code paths

8. Flake quarantine and leak detection (testing)

• 8.1 Implement markFlaky({ retries, reason, ticket }) with bounded retries (max 3)
• 8.2 Emit testing.flake.count and testing.flake.retry_used OTel metrics
• 8.3 Validate ticket field is a non-empty issue URL (zod schema)
• 8.4 Define quarantine.json schema ({ path, name, ticket, addedAt, expiresAt }) and load at suite start
• 8.5 Fail CI on expired quarantine entries
• 8.6 Implement post-teardown leak assertion: timers / sockets / handles
• 8.7 Throw TESTING.LEAK.{HANDLE,TIMER,SOCKET} with the offending stack
• 8.8 Tests covering each leak category

9. Spec archival

• 9.1 Run openspec validate enterprise-testing-operational --strict
• 9.2 Fold deltas into openspec/specs/{testing-integration,testing-integration-verdaccio,testing,testing-e2e}/spec.md
• 9.3 Move openspec/changes/enterprise-testing-operational/ to openspec/archive/<yyyy-mm-dd>-enterprise-testing-operational/