Files
2026-05-27 23:29:53 +01:00

3.7 KiB
Raw Permalink Blame History

Why

The 2026-04-25 test-suite audit (/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md §2 row 2) flagged "no visual regression" as a critical gap. Stories existing in Storybook is not visual coverage — pixel drift between commits goes unnoticed until a user reports it. CLAUDE.md's "Web App & Page Verification" section requires verifying the rendered DOM contains expected content; visual regression is the strongest automated form of that check.

This change introduces baseline-screenshot coverage on two surfaces:

  1. Playwright e2e — expect(page).toHaveScreenshot() baselines for every page in apps/<app>/e2e/.
  2. Storybook — every *.stories.tsx story produces a screenshot baseline per variant via @storybook/test-runner.

Both paths share the same threshold, viewport pinning, and animation-disabling discipline so a single --update-snapshots workflow covers them.

What Changes

New surface in @suaveplan/testing-e2e

  • configureVisualRegression(config?) — emits a PlaywrightTestConfig fragment with expect.toHaveScreenshot.maxDiffPixelRatio: 0.005, expect.toHaveScreenshot.animations: 'disabled', viewport pinned to 1280×720, prefers-reduced-motion: reduce forced.
  • prepareForVisual(page) — disables animations, waits for networkidle + document.fonts.ready, sets prefers-reduced-motion. Called inside e2e specs before toHaveScreenshot.
  • Baselines stored under apps/<app>/e2e/__snapshots__/<platform>/<spec>/<test>.png.
  • Structured error codes under TESTING_E2E.VISUAL.* (PIXEL_DIFF_EXCEEDED, BASELINE_MISSING, FONTS_NOT_READY, VIEWPORT_MISMATCH).

New surface in @suaveplan/testing-dom

  • createStorybookTestRunnerConfig(opts) — returns the @storybook/test-runner config with the visual hooks pre-wired (postVisit snapshot capture, threshold matching the e2e path).
  • pinStoryEnvironment(page) — locks viewport, font loading, and reduced-motion for stories.
  • Structured error codes under TESTING_DOM.STORYBOOK_VISUAL.* (PIXEL_DIFF_EXCEEDED, BASELINE_MISSING, RUNNER_LOAD_FAILED).

Modified surface

  • Storybook stories MUST register through the test-runner config so every variant produces a baseline.
  • Theme/locale decorators MUST honour the pinStoryEnvironment contract (deterministic theme + locale per story file).
  • --update-snapshots runs MUST land in a separate review-tagged commit per genesis policy.
  • Cross-link to the enterprise-testbed-verification change: visual diffs feed the release-readiness gate.

Capabilities

New Capabilities

(none — extensions to existing testing-e2e and testing-dom capabilities)

Modified Capabilities

  • testing-e2e: adds configureVisualRegression, prepareForVisual, TESTING_E2E.VISUAL.* error codes, baseline-storage layout requirement, per-OS matrix requirement.
  • testing-dom: adds Storybook test-runner config factory, pinStoryEnvironment, TESTING_DOM.STORYBOOK_VISUAL.* error codes, per-story baseline requirement.

Impact

  • Affected code: /mnt/s/web/genesis/packages/testing/testing-e2e/src/visual/**, /mnt/s/web/genesis/packages/testing/testing-dom/src/storybook-visual/**, every consumer with stories or e2e specs.
  • Catalog adds: @storybook/test-runner to catalogs.testing in root package.json. (Pixel diffing reuses Playwright's built-in pixelmatch-equivalent — no extra catalog dep.)
  • Coverage: 100% Istanbul on every helper module per project rule.
  • Baselines: committed to repo under __snapshots__/. Per-OS subdirectory matches CI matrix (linux / darwin / win32). CI updated to run the full matrix or skip the platform check explicitly.
  • No version bump: both packages stay at 0.1.0.