suaveplan/openspec/changes/enterprise-testing-a11y/specs/testing-e2e/spec.md

ADDED Requirements

Requirement: attachAxe accessibility helper

The @suaveplan/testing-e2e package SHALL export attachAxe(page, options?) returning an AxeHandle with two methods: assertNoViolations(opts?): Promise<void> and getViolations(opts?): Promise<A11yViolation[]>. The implementation MUST be built on @axe-core/playwright and MUST inject the axe-core engine into the page under test on each call.

Scenario: Helper returns handle

• WHEN attachAxe(page) is called with a Playwright Page
• THEN the returned object exposes assertNoViolations and getViolations as functions

Scenario: getViolations does not throw on findings

• WHEN the page contains accessibility issues and getViolations() is called
• THEN the promise resolves with a non-empty A11yViolation[] and does NOT throw

Requirement: Default WCAG 2.1 AA rule set

attachAxe SHALL default runOnly.values to ['wcag2a','wcag2aa','wcag21a','wcag21aa'] when the caller does not supply runOnly. Callers MAY override runOnly, disableRules, include, and exclude. Caller-supplied values MUST take precedence over defaults.

Scenario: Default rule set applied

• WHEN assertNoViolations() is invoked with no runOnly override
• THEN the underlying AxeBuilder.withTags() call receives exactly ['wcag2a','wcag2aa','wcag21a','wcag21aa']

Scenario: Caller override wins

• WHEN assertNoViolations({ runOnly: ['wcag22aa'] }) is invoked
• THEN the AxeBuilder.withTags() call receives ['wcag22aa']

Requirement: Impact-based failure gate

assertNoViolations SHALL fail by default when any violation has impact === 'serious' or impact === 'critical'. When strict: true is passed, violations of impact === 'moderate' or impact === 'minor' MUST also fail. Violations below the active impact threshold MUST NOT cause a throw but MUST still be written to the reporter output.

Scenario: Serious violation fails default mode

• WHEN the page produces a serious violation and assertNoViolations() runs
• THEN the call throws A11yError with code === 'TESTING_E2E.A11Y.VIOLATIONS_FOUND'

Scenario: Minor violation passes default mode

• WHEN the page produces only minor violations and assertNoViolations() runs
• THEN the call resolves without throwing

Scenario: Strict mode escalates minor

• WHEN the page produces a minor violation and assertNoViolations({ strict: true }) runs
• THEN the call throws A11yError with code === 'TESTING_E2E.A11Y.VIOLATIONS_FOUND'

Requirement: A11y reporter output

Each call to assertNoViolations or getViolations SHALL write a JSON report to playwright-report/a11y/<spec-name>.json containing every detected violation with fields id, impact, helpUrl, nodes[].target, nodes[].html. The report MUST be written even when the assertion passes (zero-violation runs produce a report with an empty violations array).

Scenario: Report file written on pass

• WHEN a passing assertNoViolations() call finishes
• THEN playwright-report/a11y/<spec>.json exists with violations: []

Scenario: Report file written on fail

• WHEN a failing assertNoViolations() call throws
• THEN playwright-report/a11y/<spec>.json exists with at least one entry whose impact matches the active threshold

Requirement: Per-spec a11y invariant

Every page-level Playwright spec under apps/<app>/e2e/ that navigates to a rendered page SHALL invoke axe.assertNoViolations() after the page reaches a ready state, alongside attachConsoleMonitor and attachNetworkMonitor invariants. Specs that exclusively test API endpoints (no DOM render) are exempt.

Scenario: Page-level spec includes axe sweep

• WHEN a spec under apps/testbed/e2e/ navigates to a route and asserts rendered DOM
• THEN the spec also calls axe.assertNoViolations() before completing

Requirement: Structured error codes

All errors thrown from the attachAxe helper SHALL be subclasses of TestingE2eError carrying { code, cause?, context? } and SHALL match the regex ^TESTING_E2E\.A11Y\.[A-Z_]+$. The defined codes are TESTING_E2E.A11Y.VIOLATIONS_FOUND, TESTING_E2E.A11Y.AXE_LOAD_FAILED, and TESTING_E2E.A11Y.INVALID_RULESET.

Scenario: Unknown WCAG tag rejected

• WHEN assertNoViolations({ runOnly: ['not-a-real-tag'] }) is invoked
• THEN the call throws A11yError with code === 'TESTING_E2E.A11Y.INVALID_RULESET'

Scenario: Axe injection failure surfaces

• WHEN axe-core script injection fails (CSP, navigation race, page closed)
• THEN the call throws A11yError with code === 'TESTING_E2E.A11Y.AXE_LOAD_FAILED' carrying the underlying cause

Requirement: Coverage gate at 100%

Istanbul statement, branch, function, and line coverage across packages/testing/testing-e2e/src/a11y/ (excluding *.bench.ts, *.schemas.ts, barrel index.ts) MUST be 100% on every dimension. The pre-publish gate (bunx turbo test:coverage --filter=@suaveplan/testing-e2e) hard-fails below 100%.

Scenario: Coverage gate enforces 100%

• WHEN bunx turbo test:coverage --filter=@suaveplan/testing-e2e runs
• THEN statements, branches, functions, and lines coverage all equal 100%