Files
suaveplan/openspec/changes/enterprise-testing-visual-regression/specs/testing-e2e/spec.md
2026-05-27 23:29:53 +01:00

5.1 KiB

ADDED Requirements

Requirement: configureVisualRegression factory

The @suaveplan/testing-e2e package SHALL export configureVisualRegression(config?) returning a PlaywrightTestConfig fragment with expect.toHaveScreenshot.maxDiffPixelRatio defaulted to 0.005, expect.toHaveScreenshot.animations defaulted to 'disabled', expect.toHaveScreenshot.caret defaulted to 'hide', and use.viewport defaulted to { width: 1280, height: 720 }. Caller-supplied values MUST take precedence over defaults.

Scenario: Defaults emitted

  • WHEN configureVisualRegression() is called with no arguments
  • THEN the returned fragment contains expect.toHaveScreenshot.maxDiffPixelRatio === 0.005 AND expect.toHaveScreenshot.animations === 'disabled' AND use.viewport.width === 1280

Scenario: Caller override wins

  • WHEN configureVisualRegression({ expect: { toHaveScreenshot: { maxDiffPixelRatio: 0.001 } } }) is called
  • THEN the returned fragment has expect.toHaveScreenshot.maxDiffPixelRatio === 0.001

Requirement: prepareForVisual determinism harness

The package SHALL export prepareForVisual(page) which MUST: disable animations via page.emulateMedia({ reducedMotion: 'reduce' }), wait for page.waitForLoadState('networkidle'), await document.fonts.ready inside the page context, and assert the active viewport matches the configured visual viewport. Failure to reach font readiness within timeoutMs MUST throw VisualRegressionError with code === 'TESTING_E2E.VISUAL.FONTS_NOT_READY'.

Scenario: Determinism levers applied

  • WHEN prepareForVisual(page) resolves successfully
  • THEN prefers-reduced-motion is 'reduce' AND document.fonts.status === 'loaded' AND the page has reached networkidle

Scenario: Font timeout

  • WHEN document.fonts.ready does not resolve within timeoutMs
  • THEN the call throws VisualRegressionError with code === 'TESTING_E2E.VISUAL.FONTS_NOT_READY'

Scenario: Viewport mismatch

  • WHEN the page viewport differs from the configured visual viewport at capture time
  • THEN the call throws VisualRegressionError with code === 'TESTING_E2E.VISUAL.VIEWPORT_MISMATCH'

Requirement: Per-OS baseline storage

Visual baselines SHALL be stored under apps/<app>/e2e/__snapshots__/<platform>/<spec>/<test>.png where <platform> is one of linux, darwin, win32. A run on a platform with no baseline directory MUST NOT pass implicitly — it MUST throw VisualRegressionError with code === 'TESTING_E2E.VISUAL.BASELINE_MISSING' unless --update-snapshots is in effect.

Scenario: Missing baseline fails

  • WHEN an e2e spec calls expect(page).toHaveScreenshot() on a platform with no baseline directory and --update-snapshots is NOT set
  • THEN the call throws VisualRegressionError with code === 'TESTING_E2E.VISUAL.BASELINE_MISSING'

Scenario: Pixel diff exceeded

  • WHEN a comparison exceeds the configured maxDiffPixelRatio
  • THEN the call throws VisualRegressionError with code === 'TESTING_E2E.VISUAL.PIXEL_DIFF_EXCEEDED' whose context contains { baselinePath, actualPath, diffPath, pixelRatio }

Requirement: Baseline-update workflow discipline

Updates to committed baselines SHALL only occur via runs invoked with --update-snapshots AND MUST land in a dedicated commit prefixed chore(visual): refresh baselines separate from any feature or fix commit.

Scenario: Baseline refresh commit isolated

  • WHEN baselines are updated
  • THEN the resulting commit contains only __snapshots__/ changes AND its message starts with chore(visual): refresh baselines

Requirement: Per-spec visual invariant

Every page-level Playwright spec under apps/<app>/e2e/ that asserts rendered DOM SHALL also call expect(page).toHaveScreenshot() after prepareForVisual(page) resolves. Specs that exclusively test API endpoints (no DOM render) are exempt.

Scenario: Page-level spec includes screenshot

  • WHEN a spec navigates to a route, runs prepareForVisual(page), and asserts rendered DOM
  • THEN the spec also calls expect(page).toHaveScreenshot() before completing

Requirement: Structured error codes

All errors thrown from the visual helpers SHALL be subclasses of TestingE2eError carrying { code, cause?, context? } and SHALL match the regex ^TESTING_E2E\.VISUAL\.[A-Z_]+$. Defined codes are TESTING_E2E.VISUAL.PIXEL_DIFF_EXCEEDED, TESTING_E2E.VISUAL.BASELINE_MISSING, TESTING_E2E.VISUAL.FONTS_NOT_READY, TESTING_E2E.VISUAL.VIEWPORT_MISMATCH.

Scenario: Coded error

  • WHEN any visual helper throws
  • THEN the thrown value instanceof TestingE2eError is true AND err.code matches the regex

Requirement: Coverage gate at 100%

Istanbul statement, branch, function, and line coverage across packages/testing/testing-e2e/src/visual/ (excluding *.bench.ts, *.schemas.ts, barrel index.ts) MUST be 100% on every dimension. The pre-publish gate hard-fails below 100%.

Scenario: Coverage gate enforces 100%

  • WHEN bunx turbo test:coverage --filter=@suaveplan/testing-e2e runs
  • THEN statements, branches, functions, and lines coverage all equal 100%