35 lines
3.2 KiB
Markdown
35 lines
3.2 KiB
Markdown
## Gate composition
|
|
|
|
The release-readiness gate is the union of five orthogonal layers. Each layer is owned by a separate change folder and contributes its own assertions; this change wires them into a single pass/fail outcome at the `apps/testbed` boundary.
|
|
|
|
| Layer | Source change | Failure condition |
|
|
|---|---|---|
|
|
| Console health | this change (per-page invariant requirement) | any error or warning logged during the spec |
|
|
| Network health | this change (`NetworkMonitor` extensions) | any failure, missing asset, mixed-content, CORS error, or unexpected request |
|
|
| Accessibility | `enterprise-testing-a11y` | any axe violation at WCAG 2.1 AA |
|
|
| Visual regression | `enterprise-testing-visual-regression` | any pixel diff > 0.5% from the committed baseline |
|
|
| Link reachability | this change (`crawlInternalLinks`) | any 4xx/5xx, circular redirect, or depth-exceeded |
|
|
| Load smoke | `enterprise-testing-load-perf` | any 5xx from any Hono API route under load smoke |
|
|
|
|
The `release-readiness` Playwright project runs these layers in series against the Docker stack stood up per `enterprise-testing-operational`. Because the layers are orthogonal, a single spec failure surfaces the precise root cause via the structured-error code (`TESTING_E2E.NETWORK.MISSING_ASSET`, `TESTING_E2E.CRAWL.BROKEN_LINK`, etc.) — no log-spelunking required.
|
|
|
|
## Why a separate `testbed` capability
|
|
|
|
`testing-e2e` is a Layer 0 library — runner-agnostic helpers with zero `@suaveplan/*` workspace deps (per `openspec/project.md` rule 9). Per-page invariants for `apps/testbed` are application-level policy: which testids are required, how many demo cards each route must expose, what counts as "baseline-only" coverage. Mixing this into `testing-e2e` would conflate library API with consumer policy. A separate `testbed` capability keeps the library re-usable by other apps while making the testbed's verification contract explicit and auditable.
|
|
|
|
## Why broken-link crawling lives in `testing-e2e`
|
|
|
|
The crawler is generically useful — any app can call `crawlInternalLinks(page)` against its own routes. It does not embed testbed-specific knowledge. Putting it in `testing-e2e` lets future apps (`apps/docs`, future consumer apps) re-use it without copying.
|
|
|
|
## Component-on-page vs Storybook
|
|
|
|
Storybook (covered by `enterprise-browser-storybook-test-utils`) renders components in isolation with controlled props. Interaction specs render the real page composition: real router, real data fetching, real layout, real error boundaries. A button can pass every Storybook test and still fail when wired into a page that misroutes its onClick. Both layers are mandatory and neither substitutes for the other; the spec encodes this so future agents do not "consolidate" them away.
|
|
|
|
## Allow-listing 404s
|
|
|
|
`monitor.allow404(predicate)` exists for legitimate cases (e.g. ad-blocker telemetry beacons, optional CDN preloads). The default is deny-by-default; opt-outs are per-test, predicate-based, and surface the allowed URL in the structured error context for audit.
|
|
|
|
## Performance budget for the gate
|
|
|
|
The crawler is bounded by `maxDepth` (default 2) and `sameOriginOnly` (default true) to keep the gate tractable on large site maps. Anything deeper belongs to a periodic crawl job, not the per-PR gate.
|