suaveplan/openspec/changes/enterprise-testbed-verification/specs/testbed/spec.md

ADDED Requirements

Requirement: Per-page interaction spec coverage

Every route registered in apps/testbed/src/routes.ts SHALL have a corresponding <route>.interaction.spec.ts under apps/testbed/e2e/ that:

1. Navigates to the route.
2. Asserts the element with data-testid="page-title" resolves and has non-empty text content.
3. Iterates every data-testid="demo-card" rendered on the page; for each card, triggers its primary action (button click, form submit, or focus event as the card defines) AND asserts the card-scoped data-testid="output" populates with non-empty content within 5 seconds.
4. Includes at least one demo-specific assertion beyond the title and output checks (e.g. an output regex, a computed style assertion, or a returned JSON shape assertion).

A "baseline-only" spec — one that merely navigates and asserts a non-empty page title — MUST NOT be the sole coverage for a route. A baseline navigation spec MAY exist alongside an interaction spec but does not satisfy this requirement on its own.

Scenario: New route ships with interaction spec

• WHEN a new route /demos/widgets is added to apps/testbed/src/routes.ts
• AND the only e2e file referencing the route is widgets.baseline.spec.ts asserting only the page title
• THEN the testbed verification gate fails with a missing-coverage diagnostic naming widgets.interaction.spec.ts

Scenario: Demo card without output fails the spec

• WHEN an interaction spec triggers a demo card's primary action
• AND the card-scoped data-testid="output" remains empty for 5 seconds
• THEN the spec fails with a timeout error referencing the card's testid

Requirement: Per-page invariants attached on every interaction spec

Every interaction spec SHALL attach and exercise ALL of the following invariants, in addition to its functional assertions:

1. attachConsoleMonitor(page) followed by assertNoErrors() and assertNoWarnings() at end of test.
2. attachNetworkMonitor(page) followed by assertNoFailures(), assertNoMissingAssets(), assertNoMixedContent(), and assertNoCorsErrors() at end of test.
3. attachAxe(page).assertNoViolations() per the contract introduced by enterprise-testing-a11y (WCAG 2.1 AA).
4. expect(page).toHaveScreenshot() per the contract introduced by enterprise-testing-visual-regression, with a stable viewport and prefers-reduced-motion: reduce.
5. The interaction-coverage assertion defined in the previous requirement.

A spec that omits any of these five invariants does NOT satisfy this requirement.

Scenario: Spec missing axe assertion fails the gate

• WHEN an interaction spec attaches console + network + visual + interaction coverage but not attachAxe
• THEN the testbed verification gate fails with a missing-invariant diagnostic naming the omitted helper

Scenario: All invariants attached and passing

• WHEN an interaction spec attaches all five invariants and every assertion passes
• THEN the spec passes the per-page invariant requirement

Requirement: Release-readiness CI gate

A CI job named release-readiness SHALL run against apps/testbed deployed to the testbed Docker stack defined by enterprise-testing-operational. The job SHALL fail on any of:

1. Any console error or warning (per the console invariant).
2. Any network failure, missing asset, mixed content, CORS error, or unexpected request (per the network invariant).
3. Any axe accessibility violation at WCAG 2.1 AA (per enterprise-testing-a11y).
4. Any visual-regression diff exceeding 0.5% from the committed baseline (per enterprise-testing-visual-regression).
5. Any broken internal link reported by crawlInternalLinks(page, { maxDepth: 2, sameOriginOnly: true }).
6. Any HTTP 5xx response from any Hono API route under the load smoke defined by enterprise-testing-load-perf.

The release-readiness gate is mandatory for any pull request whose change set touches apps/testbed or any package in its dependency closure.

Scenario: Broken internal link blocks the gate

• WHEN the testbed footer links to /legal/terms which returns 404
• AND release-readiness runs
• THEN the gate fails with a TESTING_E2E.CRAWL.BROKEN_LINK diagnostic

Scenario: 5xx under load blocks the gate

• WHEN the load smoke for an API route observes any HTTP 5xx response
• AND release-readiness runs
• THEN the gate fails citing the responsible route and status

Scenario: Clean run passes the gate

• WHEN all six failure conditions are absent
• AND release-readiness runs
• THEN the gate passes

Requirement: Zero-defect-to-live enforcement

Failure of the release-readiness gate SHALL block merge to main, and SHALL block merge to genesis until graduation. There is no override path for "ship now, fix later". The CLAUDE.md "Web App & Page Verification" rule (bullet 6, "Zero-defect push to live") becomes machine-enforced through this requirement.

Scenario: Failing gate blocks merge

• WHEN a pull request touching apps/testbed triggers release-readiness
• AND the gate fails for any reason listed in the previous requirement
• THEN the branch protection prevents the PR from being merged to main (or to genesis pre-graduation)

Scenario: No override token

• WHEN a maintainer attempts to merge a PR whose release-readiness gate has failed
• THEN no label, comment, or commit-trailer SHALL bypass the protection

Requirement: Component-on-page coverage complements Storybook coverage

Storybook stories (per enterprise-browser-storybook-test-utils) cover components in isolation; interaction specs (defined above) cover components rendered on actual pages. The two coverage layers are complementary: a passing Storybook test SHALL NOT be accepted as a substitute for an interaction-spec assertion exercising the same component on a live page, and vice versa.

Scenario: Component covered only by Storybook

• WHEN a component is exercised only in *.stories.tsx and never rendered by any testbed route's interaction spec
• AND the component appears on a testbed route in production
• THEN the testbed verification gate fails with a missing-on-page-coverage diagnostic

Scenario: Component covered only on-page

• WHEN a component is exercised by an interaction spec but has no *.stories.tsx
• THEN the broader package quality gate (per enterprise-browser-storybook-test-utils) fails with a missing-story diagnostic; the testbed gate is not the enforcement point for that case