Initial Commit
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Per-page interaction spec coverage
|
||||
|
||||
Every route registered in `apps/testbed/src/routes.ts` SHALL have a corresponding `<route>.interaction.spec.ts` under `apps/testbed/e2e/` that:
|
||||
|
||||
1. Navigates to the route.
|
||||
2. Asserts the element with `data-testid="page-title"` resolves and has non-empty text content.
|
||||
3. Iterates every `data-testid="demo-card"` rendered on the page; for each card, triggers its primary action (button click, form submit, or focus event as the card defines) AND asserts the card-scoped `data-testid="output"` populates with non-empty content within 5 seconds.
|
||||
4. Includes at least one demo-specific assertion beyond the title and output checks (e.g. an output regex, a computed style assertion, or a returned JSON shape assertion).
|
||||
|
||||
A "baseline-only" spec — one that merely navigates and asserts a non-empty page title — MUST NOT be the sole coverage for a route. A baseline navigation spec MAY exist alongside an interaction spec but does not satisfy this requirement on its own.
|
||||
|
||||
#### Scenario: New route ships with interaction spec
|
||||
|
||||
- **WHEN** a new route `/demos/widgets` is added to `apps/testbed/src/routes.ts`
|
||||
- **AND** the only e2e file referencing the route is `widgets.baseline.spec.ts` asserting only the page title
|
||||
- **THEN** the testbed verification gate fails with a missing-coverage diagnostic naming `widgets.interaction.spec.ts`
|
||||
|
||||
#### Scenario: Demo card without output fails the spec
|
||||
|
||||
- **WHEN** an interaction spec triggers a demo card's primary action
|
||||
- **AND** the card-scoped `data-testid="output"` remains empty for 5 seconds
|
||||
- **THEN** the spec fails with a timeout error referencing the card's testid
|
||||
|
||||
### Requirement: Per-page invariants attached on every interaction spec
|
||||
|
||||
Every interaction spec SHALL attach and exercise ALL of the following invariants, in addition to its functional assertions:
|
||||
|
||||
1. `attachConsoleMonitor(page)` followed by `assertNoErrors()` and `assertNoWarnings()` at end of test.
|
||||
2. `attachNetworkMonitor(page)` followed by `assertNoFailures()`, `assertNoMissingAssets()`, `assertNoMixedContent()`, and `assertNoCorsErrors()` at end of test.
|
||||
3. `attachAxe(page).assertNoViolations()` per the contract introduced by `enterprise-testing-a11y` (WCAG 2.1 AA).
|
||||
4. `expect(page).toHaveScreenshot()` per the contract introduced by `enterprise-testing-visual-regression`, with a stable viewport and `prefers-reduced-motion: reduce`.
|
||||
5. The interaction-coverage assertion defined in the previous requirement.
|
||||
|
||||
A spec that omits any of these five invariants does NOT satisfy this requirement.
|
||||
|
||||
#### Scenario: Spec missing axe assertion fails the gate
|
||||
|
||||
- **WHEN** an interaction spec attaches console + network + visual + interaction coverage but not `attachAxe`
|
||||
- **THEN** the testbed verification gate fails with a missing-invariant diagnostic naming the omitted helper
|
||||
|
||||
#### Scenario: All invariants attached and passing
|
||||
|
||||
- **WHEN** an interaction spec attaches all five invariants and every assertion passes
|
||||
- **THEN** the spec passes the per-page invariant requirement
|
||||
|
||||
### Requirement: Release-readiness CI gate
|
||||
|
||||
A CI job named `release-readiness` SHALL run against `apps/testbed` deployed to the testbed Docker stack defined by `enterprise-testing-operational`. The job SHALL fail on any of:
|
||||
|
||||
1. Any console error or warning (per the console invariant).
|
||||
2. Any network failure, missing asset, mixed content, CORS error, or unexpected request (per the network invariant).
|
||||
3. Any axe accessibility violation at WCAG 2.1 AA (per `enterprise-testing-a11y`).
|
||||
4. Any visual-regression diff exceeding 0.5% from the committed baseline (per `enterprise-testing-visual-regression`).
|
||||
5. Any broken internal link reported by `crawlInternalLinks(page, { maxDepth: 2, sameOriginOnly: true })`.
|
||||
6. Any HTTP 5xx response from any Hono API route under the load smoke defined by `enterprise-testing-load-perf`.
|
||||
|
||||
The release-readiness gate is mandatory for any pull request whose change set touches `apps/testbed` or any package in its dependency closure.
|
||||
|
||||
#### Scenario: Broken internal link blocks the gate
|
||||
|
||||
- **WHEN** the testbed footer links to `/legal/terms` which returns 404
|
||||
- **AND** `release-readiness` runs
|
||||
- **THEN** the gate fails with a `TESTING_E2E.CRAWL.BROKEN_LINK` diagnostic
|
||||
|
||||
#### Scenario: 5xx under load blocks the gate
|
||||
|
||||
- **WHEN** the load smoke for an API route observes any HTTP 5xx response
|
||||
- **AND** `release-readiness` runs
|
||||
- **THEN** the gate fails citing the responsible route and status
|
||||
|
||||
#### Scenario: Clean run passes the gate
|
||||
|
||||
- **WHEN** all six failure conditions are absent
|
||||
- **AND** `release-readiness` runs
|
||||
- **THEN** the gate passes
|
||||
|
||||
### Requirement: Zero-defect-to-live enforcement
|
||||
|
||||
Failure of the `release-readiness` gate SHALL block merge to `main`, and SHALL block merge to `genesis` until graduation. There is no override path for "ship now, fix later". The `CLAUDE.md` "Web App & Page Verification" rule (bullet 6, "Zero-defect push to live") becomes machine-enforced through this requirement.
|
||||
|
||||
#### Scenario: Failing gate blocks merge
|
||||
|
||||
- **WHEN** a pull request touching `apps/testbed` triggers `release-readiness`
|
||||
- **AND** the gate fails for any reason listed in the previous requirement
|
||||
- **THEN** the branch protection prevents the PR from being merged to `main` (or to `genesis` pre-graduation)
|
||||
|
||||
#### Scenario: No override token
|
||||
|
||||
- **WHEN** a maintainer attempts to merge a PR whose `release-readiness` gate has failed
|
||||
- **THEN** no label, comment, or commit-trailer SHALL bypass the protection
|
||||
|
||||
### Requirement: Component-on-page coverage complements Storybook coverage
|
||||
|
||||
Storybook stories (per `enterprise-browser-storybook-test-utils`) cover components in isolation; interaction specs (defined above) cover components rendered on actual pages. The two coverage layers are complementary: a passing Storybook test SHALL NOT be accepted as a substitute for an interaction-spec assertion exercising the same component on a live page, and vice versa.
|
||||
|
||||
#### Scenario: Component covered only by Storybook
|
||||
|
||||
- **WHEN** a component is exercised only in `*.stories.tsx` and never rendered by any testbed route's interaction spec
|
||||
- **AND** the component appears on a testbed route in production
|
||||
- **THEN** the testbed verification gate fails with a missing-on-page-coverage diagnostic
|
||||
|
||||
#### Scenario: Component covered only on-page
|
||||
|
||||
- **WHEN** a component is exercised by an interaction spec but has no `*.stories.tsx`
|
||||
- **THEN** the broader package quality gate (per `enterprise-browser-storybook-test-utils`) fails with a missing-story diagnostic; the testbed gate is not the enforcement point for that case
|
||||
Reference in New Issue
Block a user