95 lines
5.2 KiB
Markdown
95 lines
5.2 KiB
Markdown
## ADDED Requirements
|
|
|
|
### Requirement: `attachAxe` accessibility helper
|
|
|
|
The `@suaveplan/testing-e2e` package SHALL export `attachAxe(page, options?)` returning an `AxeHandle` with two methods: `assertNoViolations(opts?): Promise<void>` and `getViolations(opts?): Promise<A11yViolation[]>`. The implementation MUST be built on `@axe-core/playwright` and MUST inject the axe-core engine into the page under test on each call.
|
|
|
|
#### Scenario: Helper returns handle
|
|
|
|
- **WHEN** `attachAxe(page)` is called with a Playwright `Page`
|
|
- **THEN** the returned object exposes `assertNoViolations` and `getViolations` as functions
|
|
|
|
#### Scenario: getViolations does not throw on findings
|
|
|
|
- **WHEN** the page contains accessibility issues and `getViolations()` is called
|
|
- **THEN** the promise resolves with a non-empty `A11yViolation[]` and does NOT throw
|
|
|
|
### Requirement: Default WCAG 2.1 AA rule set
|
|
|
|
`attachAxe` SHALL default `runOnly.values` to `['wcag2a','wcag2aa','wcag21a','wcag21aa']` when the caller does not supply `runOnly`. Callers MAY override `runOnly`, `disableRules`, `include`, and `exclude`. Caller-supplied values MUST take precedence over defaults.
|
|
|
|
#### Scenario: Default rule set applied
|
|
|
|
- **WHEN** `assertNoViolations()` is invoked with no `runOnly` override
|
|
- **THEN** the underlying `AxeBuilder.withTags()` call receives exactly `['wcag2a','wcag2aa','wcag21a','wcag21aa']`
|
|
|
|
#### Scenario: Caller override wins
|
|
|
|
- **WHEN** `assertNoViolations({ runOnly: ['wcag22aa'] })` is invoked
|
|
- **THEN** the `AxeBuilder.withTags()` call receives `['wcag22aa']`
|
|
|
|
### Requirement: Impact-based failure gate
|
|
|
|
`assertNoViolations` SHALL fail by default when any violation has `impact === 'serious'` or `impact === 'critical'`. When `strict: true` is passed, violations of `impact === 'moderate'` or `impact === 'minor'` MUST also fail. Violations below the active impact threshold MUST NOT cause a throw but MUST still be written to the reporter output.
|
|
|
|
#### Scenario: Serious violation fails default mode
|
|
|
|
- **WHEN** the page produces a `serious` violation and `assertNoViolations()` runs
|
|
- **THEN** the call throws `A11yError` with `code === 'TESTING_E2E.A11Y.VIOLATIONS_FOUND'`
|
|
|
|
#### Scenario: Minor violation passes default mode
|
|
|
|
- **WHEN** the page produces only `minor` violations and `assertNoViolations()` runs
|
|
- **THEN** the call resolves without throwing
|
|
|
|
#### Scenario: Strict mode escalates minor
|
|
|
|
- **WHEN** the page produces a `minor` violation and `assertNoViolations({ strict: true })` runs
|
|
- **THEN** the call throws `A11yError` with `code === 'TESTING_E2E.A11Y.VIOLATIONS_FOUND'`
|
|
|
|
### Requirement: A11y reporter output
|
|
|
|
Each call to `assertNoViolations` or `getViolations` SHALL write a JSON report to `playwright-report/a11y/<spec-name>.json` containing every detected violation with fields `id`, `impact`, `helpUrl`, `nodes[].target`, `nodes[].html`. The report MUST be written even when the assertion passes (zero-violation runs produce a report with an empty `violations` array).
|
|
|
|
#### Scenario: Report file written on pass
|
|
|
|
- **WHEN** a passing `assertNoViolations()` call finishes
|
|
- **THEN** `playwright-report/a11y/<spec>.json` exists with `violations: []`
|
|
|
|
#### Scenario: Report file written on fail
|
|
|
|
- **WHEN** a failing `assertNoViolations()` call throws
|
|
- **THEN** `playwright-report/a11y/<spec>.json` exists with at least one entry whose `impact` matches the active threshold
|
|
|
|
### Requirement: Per-spec a11y invariant
|
|
|
|
Every page-level Playwright spec under `apps/<app>/e2e/` that navigates to a rendered page SHALL invoke `axe.assertNoViolations()` after the page reaches a ready state, alongside `attachConsoleMonitor` and `attachNetworkMonitor` invariants. Specs that exclusively test API endpoints (no DOM render) are exempt.
|
|
|
|
#### Scenario: Page-level spec includes axe sweep
|
|
|
|
- **WHEN** a spec under `apps/testbed/e2e/` navigates to a route and asserts rendered DOM
|
|
- **THEN** the spec also calls `axe.assertNoViolations()` before completing
|
|
|
|
### Requirement: Structured error codes
|
|
|
|
All errors thrown from the `attachAxe` helper SHALL be subclasses of `TestingE2eError` carrying `{ code, cause?, context? }` and SHALL match the regex `^TESTING_E2E\.A11Y\.[A-Z_]+$`. The defined codes are `TESTING_E2E.A11Y.VIOLATIONS_FOUND`, `TESTING_E2E.A11Y.AXE_LOAD_FAILED`, and `TESTING_E2E.A11Y.INVALID_RULESET`.
|
|
|
|
#### Scenario: Unknown WCAG tag rejected
|
|
|
|
- **WHEN** `assertNoViolations({ runOnly: ['not-a-real-tag'] })` is invoked
|
|
- **THEN** the call throws `A11yError` with `code === 'TESTING_E2E.A11Y.INVALID_RULESET'`
|
|
|
|
#### Scenario: Axe injection failure surfaces
|
|
|
|
- **WHEN** axe-core script injection fails (CSP, navigation race, page closed)
|
|
- **THEN** the call throws `A11yError` with `code === 'TESTING_E2E.A11Y.AXE_LOAD_FAILED'` carrying the underlying cause
|
|
|
|
### Requirement: Coverage gate at 100%
|
|
|
|
Istanbul statement, branch, function, and line coverage across `packages/testing/testing-e2e/src/a11y/` (excluding `*.bench.ts`, `*.schemas.ts`, barrel `index.ts`) MUST be 100% on every dimension. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing-e2e`) hard-fails below 100%.
|
|
|
|
#### Scenario: Coverage gate enforces 100%
|
|
|
|
- **WHEN** `bunx turbo test:coverage --filter=@suaveplan/testing-e2e` runs
|
|
- **THEN** statements, branches, functions, and lines coverage all equal 100%
|