Initial Commit
This commit is contained in:
88
openspec/archive/2026-04-19-enterprise-testing-e2e/design.md
Normal file
88
openspec/archive/2026-04-19-enterprise-testing-e2e/design.md
Normal file
@@ -0,0 +1,88 @@
|
||||
## Context
|
||||
|
||||
`@suaveplan/testing-e2e` is the third Layer 0 testing-infrastructure package on genesis. It wraps Playwright with conventions, page objects, and a config factory.
|
||||
|
||||
Layer 0 dependency profile: external catalog deps only. No `@suaveplan/*` workspace deps in runtime closure. `@suaveplan/testing` permitted as `devDependency` for the package's own tests.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Thin, conventional Playwright wrapper. No forking of Playwright internals.
|
||||
- `createPlaywrightConfig()` absorbs WSL2 + CI defaults so every consumer gets stable runs without copy-paste.
|
||||
- `PageObject` base class for the typed page-object pattern.
|
||||
- Local `TestingE2eError` hierarchy with stable `TESTING_E2E.<DOMAIN>.<REASON>` codes.
|
||||
- ≥ 95% coverage on every dimension; co-located unit tests + cross-cutting `tests/` per the genesis test-placement hard rule.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- A full Playwright re-implementation. We re-export, we don't rebuild.
|
||||
- Browser launching automation. Playwright already handles that; we expose its config.
|
||||
- React-specific helpers. Those live in `@suaveplan/testing-dom`.
|
||||
- Real-browser E2E tests of the package itself — the unit tests for our helpers run under vitest with happy-dom; real Playwright E2E of the package would be circular and impractical.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: Local `TestingE2eError` hierarchy
|
||||
|
||||
Mirrors the shape from `@suaveplan/testing` and `@suaveplan/testing-dom`. Subclasses: `NavigationError`, `WaitTimeoutError`, `ConfigError`, `FixtureError`, `AbortedError`. Codes follow `TESTING_E2E.<DOMAIN>.<REASON>` pattern.
|
||||
|
||||
### D2: `createPlaywrightConfig(options)`
|
||||
|
||||
Factory accepting `{ baseURL, projects, reporter, ...overrides }` and returning a `PlaywrightTestConfig`:
|
||||
|
||||
- `fullyParallel: false` by default (WSL2 stability)
|
||||
- `forbidOnly: !!process.env.CI`
|
||||
- `retries: process.env.CI ? 2 : 0`
|
||||
- `workers: process.env.CI ? 1 : undefined`
|
||||
- `use.trace: "on-first-retry"`
|
||||
- `use.screenshot: "only-on-failure"`
|
||||
- `use.video: "retain-on-failure"`
|
||||
- Caller can override every default via `options`.
|
||||
|
||||
### D3: `PageObject` base class
|
||||
|
||||
```ts
|
||||
export abstract class PageObject {
|
||||
constructor(protected readonly page: Page) {}
|
||||
abstract readonly path: string;
|
||||
async navigate(): Promise<void> { ... }
|
||||
async waitForReady(opts?: { signal? }): Promise<void> { ... }
|
||||
findByRole(role, options?): Locator { ... }
|
||||
findByLabel(text): Locator { ... }
|
||||
findByText(text): Locator { ... }
|
||||
}
|
||||
```
|
||||
|
||||
Extension by composition: subclasses add typed locators and domain methods.
|
||||
|
||||
### D4: Test fixtures via Playwright's `test.extend`
|
||||
|
||||
`extendedTest` exposes:
|
||||
|
||||
- `apiContext`: a `request` instance auto-created per test
|
||||
- `freshPage`: a new browser context + page (isolated from any auth fixtures)
|
||||
- `defaultTimeout`: configurable per-suite default
|
||||
|
||||
### D5: Helpers
|
||||
|
||||
- `waitForNetworkIdle(page, opts?)` — wait until N consecutive ticks with no in-flight requests
|
||||
- `screenshotOnFailure(testInfo, page)` — manual fallback for non-fixture-driven captures
|
||||
- `dismissDialog(page)` — registers a one-shot dialog handler
|
||||
- `mockGeolocation(page, coords)` — install a geolocation override via context.grantPermissions + setGeolocation
|
||||
- `mockClipboard(page)` — install a programmatic clipboard via `page.evaluate`
|
||||
|
||||
### D6: OTEL gated behind env flag
|
||||
|
||||
Same pattern as testing-dom: counters/histograms behind `process.env.TESTING_E2E_OTEL_ENABLED === "1"`.
|
||||
|
||||
### D7: Tests run under vitest (not Playwright)
|
||||
|
||||
Counter-intuitive but correct: testing-e2e is a LIBRARY exposing Playwright helpers. Its OWN unit tests run under vitest because vitest is faster and the helpers themselves don't need a real browser. We test the typed shapes, error throws, config defaults, etc. Real Playwright tests live in app packages that consume testing-e2e.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Playwright version churn** — major Playwright bumps occasionally change `PlaywrightTestConfig` shapes. Mitigation: pin via catalog; bump deliberately.
|
||||
- **Page-object generality** — a one-size base class can't anticipate every consumer's pattern. Mitigation: keep `PageObject` minimal; document the intended subclassing pattern.
|
||||
- **Helpers tested under vitest, not Playwright** — covers shape but not real-browser behavior. Mitigation: mark helpers as "best effort" in docs; recommend integration tests in consumer apps.
|
||||
- **`devDependency on @suaveplan/testing`** — same as testing-dom; documented exception.
|
||||
@@ -0,0 +1,42 @@
|
||||
## Why
|
||||
|
||||
`@suaveplan/testing-e2e` is the entry point for every Playwright E2E test in the genesis monorepo. Per the testing-package separation hard rule, it owns Playwright fixtures, page objects, helpers, and a `createPlaywrightConfig()` factory. It is the third and final Layer 0 testing-infrastructure package.
|
||||
|
||||
Layer 0 — testing-infra packages take no `@suaveplan/*` workspace deps. External catalog deps only.
|
||||
|
||||
## What Changes
|
||||
|
||||
- New package `@suaveplan/testing-e2e` at `packages/universal/testing-e2e/` (genesis-fresh, version `0.1.0`).
|
||||
- **Substitutions per genesis rules:**
|
||||
- Local `TestingE2eError extends Error` with `{ code, cause, context }` shape (no `@suaveplan/error`)
|
||||
- `@opentelemetry/api` direct (no `@suaveplan/opentelemetry`)
|
||||
- `zod` direct (no `@suaveplan/schema-registry`)
|
||||
- English-only `code` machine-contract messages (no `@suaveplan/i18n`)
|
||||
- **No `@suaveplan/*` runtime deps.** `@suaveplan/testing` is allowed as a `devDependency` only (mirroring the testing-dom precedent).
|
||||
|
||||
### Public surface (this package)
|
||||
|
||||
- **Re-exports of Playwright primitives**: `test`, `expect`, `Page`, `Browser`, `BrowserContext`, `Locator`, `request`
|
||||
- **`createPlaywrightConfig(options)`** — factory that emits a `PlaywrightTestConfig` with WSL2-safe defaults (`fullyParallel: false` for stability, retries on CI, screenshot/video/trace on failure)
|
||||
- **`PageObject`** base class — typed wrapper around a Playwright `Page` exposing common patterns (`navigate`, `waitForReady`, `findByRole`-style locators)
|
||||
- **Test fixtures** — `extendedTest` extending Playwright's `test` with a stable shared fixture set: `userPage` (pre-authenticated), `apiContext` (Playwright `request` instance), `freshPage` (forced new context)
|
||||
- **Helpers**: `waitForNetworkIdle`, `screenshotOnFailure`, `dismissDialog`, `mockGeolocation`, `mockClipboard`
|
||||
- **Errors**: `TestingE2eError`, `NavigationError`, `WaitTimeoutError`, `ConfigError`, `FixtureError`, `AbortedError` with `TESTING_E2E.<DOMAIN>.<REASON>` codes
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `testing-e2e`: Playwright fixtures, page-object base class, helpers, config factory, structured errors. Layer 0; no `@suaveplan/*` runtime deps.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
(none)
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected code**: `packages/universal/testing-e2e/**` (fresh).
|
||||
- **Affected consumers**: every E2E-testing app/package (none yet on genesis).
|
||||
- **Version**: `0.1.0`.
|
||||
- **Catalogs**: `@playwright/test` is already in `catalogs.testing` on main; carry over or add the version.
|
||||
- **Pre-publish gate**: `bunx turbo lint typecheck test --filter=@suaveplan/testing-e2e` green + ≥ 95% coverage on touched modules.
|
||||
@@ -0,0 +1,104 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Layer 0 dep-graph isolation
|
||||
|
||||
The `@suaveplan/testing-e2e` `dependencies` and `peerDependencies` MUST NOT contain any `@suaveplan/*` workspace package. `devDependencies` MAY include `@suaveplan/testing` for the package's own tests.
|
||||
|
||||
#### Scenario: Runtime closure clean
|
||||
|
||||
- **WHEN** `bunx turbo ls --filter=@suaveplan/testing-e2e...` is run
|
||||
- **THEN** no `@suaveplan/*` package appears in the runtime closure
|
||||
|
||||
### Requirement: Playwright primitive re-exports
|
||||
|
||||
The package MUST re-export `test`, `expect`, `Page`, `Browser`, `BrowserContext`, `Locator`, `request` from `@playwright/test` so consumers import a single entry point.
|
||||
|
||||
#### Scenario: Single-import consumer
|
||||
|
||||
- **WHEN** a consumer imports `{ test, expect, Page }` from `@suaveplan/testing-e2e`
|
||||
- **THEN** the symbols are the Playwright originals (identity comparison passes)
|
||||
|
||||
### Requirement: `createPlaywrightConfig` produces stable defaults
|
||||
|
||||
The factory SHALL emit a `PlaywrightTestConfig` with `fullyParallel: false`, `use.trace: "on-first-retry"`, `use.screenshot: "only-on-failure"`, `use.video: "retain-on-failure"`. CI mode (detected via `process.env.CI`) MUST set `forbidOnly: true`, `retries: 2`, `workers: 1`. Caller-supplied overrides MUST take precedence.
|
||||
|
||||
#### Scenario: Default config
|
||||
|
||||
- **WHEN** `createPlaywrightConfig({ baseURL: "http://localhost:3000" })` is called
|
||||
- **THEN** the returned config has `fullyParallel === false` AND `use.trace === "on-first-retry"`
|
||||
|
||||
#### Scenario: CI overrides
|
||||
|
||||
- **WHEN** the factory runs with `process.env.CI === "1"` and no override
|
||||
- **THEN** the config has `forbidOnly === true`, `retries === 2`, `workers === 1`
|
||||
|
||||
#### Scenario: Caller override wins
|
||||
|
||||
- **WHEN** `createPlaywrightConfig({ retries: 5 })` is called with `process.env.CI === "1"`
|
||||
- **THEN** the config has `retries === 5`
|
||||
|
||||
### Requirement: `PageObject` base class
|
||||
|
||||
`PageObject` MUST be an abstract class accepting a Playwright `Page` in its constructor. It MUST expose `path: string` (abstract), `navigate(): Promise<void>`, `waitForReady(opts?): Promise<void>`, and convenience locator helpers (`findByRole`, `findByLabel`, `findByText`).
|
||||
|
||||
#### Scenario: Subclass + navigate
|
||||
|
||||
- **WHEN** a subclass extends `PageObject`, sets `path = "/users"`, and calls `await instance.navigate()`
|
||||
- **THEN** `page.goto("/users")` is called once
|
||||
|
||||
#### Scenario: waitForReady aborts on signal
|
||||
|
||||
- **WHEN** `waitForReady({ signal })` is called with a pre-aborted signal
|
||||
- **THEN** the promise rejects with `AbortedError` whose `code === "TESTING_E2E.ABORTED"`
|
||||
|
||||
### Requirement: Helpers expose stable signatures
|
||||
|
||||
The following helpers MUST be exported with the signatures below and SHALL throw `TestingE2eError` subclasses on failure (never plain `Error`):
|
||||
|
||||
- `waitForNetworkIdle(page, options?): Promise<void>` — waits until the page sees `idleMs` of zero in-flight requests
|
||||
- `screenshotOnFailure(testInfo, page): Promise<void>` — captures a screenshot named after the failing test
|
||||
- `dismissDialog(page, action?): Promise<void>` — registers a one-shot dialog handler that auto-accepts (or auto-dismisses if `action === "dismiss"`)
|
||||
- `mockGeolocation(context, coords): Promise<void>` — installs a static geolocation override
|
||||
- `mockClipboard(page): Promise<void>` — installs a programmatic clipboard
|
||||
|
||||
#### Scenario: waitForNetworkIdle resolves
|
||||
|
||||
- **WHEN** the predicate observes zero in-flight requests for the configured `idleMs` window
|
||||
- **THEN** the promise resolves
|
||||
|
||||
#### Scenario: waitForNetworkIdle times out
|
||||
|
||||
- **WHEN** the network never quiets within `timeoutMs`
|
||||
- **THEN** the promise rejects with `WaitTimeoutError` whose `code === "TESTING_E2E.WAIT.TIMEOUT"`
|
||||
|
||||
### Requirement: Structured errors with stable codes
|
||||
|
||||
All errors thrown from `@suaveplan/testing-e2e` MUST be subclasses of `TestingE2eError extends Error` carrying `{ code, cause?, context? }`. Codes match `^TESTING_E2E\.[A-Z]+\.[A-Z_]+$`.
|
||||
|
||||
#### Scenario: Coded error
|
||||
|
||||
- **WHEN** any helper throws
|
||||
- **THEN** the thrown value `instanceof TestingE2eError` is true AND `err.code` matches the regex
|
||||
|
||||
### Requirement: Coverage and docs gates
|
||||
|
||||
Istanbul statement + branch + function + line coverage across `packages/universal/testing-e2e/src/` (excluding `*.bench.ts`, `*.schemas.ts`, barrel `index.ts` files) MUST be ≥ 95% on every dimension. Every implementation file MUST have a sibling `.md` (≥ 200 words, simple + advanced examples).
|
||||
|
||||
#### Scenario: CI coverage gate
|
||||
|
||||
- **WHEN** `bunx turbo test:coverage --filter=@suaveplan/testing-e2e` runs
|
||||
- **THEN** statement / branch / function / line coverage all ≥ 95%
|
||||
|
||||
### Requirement: Test placement follows the genesis hard rule
|
||||
|
||||
Per-feature unit tests SHALL be co-located in `src/` (named `<feature>.test.ts` next to `<feature>.ts`). Cross-cutting tests (dep-graph guard, tree-shake guard, no-disallowed grep guard) MUST live in `tests/` sibling to `src/`.
|
||||
|
||||
#### Scenario: Per-feature co-location
|
||||
|
||||
- **WHEN** `src/page-objects/page-object.ts` exists
|
||||
- **THEN** its unit tests live at `src/page-objects/page-object.test.ts`
|
||||
|
||||
#### Scenario: Cross-cutting in tests/
|
||||
|
||||
- **WHEN** a guard exercises package-level concerns (dep graph, tree-shake, lint)
|
||||
- **THEN** it lives under `tests/`
|
||||
59
openspec/archive/2026-04-19-enterprise-testing-e2e/tasks.md
Normal file
59
openspec/archive/2026-04-19-enterprise-testing-e2e/tasks.md
Normal file
@@ -0,0 +1,59 @@
|
||||
> Archived 2026-04-19: all tasks retroactively ticked; package shipped under universal/dev/testing-e2e.
|
||||
|
||||
## 1. Catalogs and skeleton
|
||||
|
||||
- [x] 1.1 Confirm `@playwright/test` is in `catalogs.testing`; add if missing.
|
||||
- [x] 1.2 Create `packages/universal/testing-e2e/` skeleton: `package.json` (version `0.1.0`, runtime deps = @playwright/test, @opentelemetry/api, zod, tslib; devDep = @suaveplan/testing), `tsconfig.lib.json`, `vite.config.ts`, `vitest.config.ts`, `README.md`.
|
||||
- [x] 1.3 Verify runtime closure has zero `@suaveplan/*`.
|
||||
|
||||
## 2. Errors
|
||||
|
||||
- [x] 2.1 `src/errors/codes.ts` — `TESTING_E2E.<DOMAIN>.<REASON>` constants
|
||||
- [x] 2.2 `src/errors/testing-e2e-error.ts` — base + subclasses (`NavigationError`, `WaitTimeoutError`, `ConfigError`, `FixtureError`, `AbortedError`)
|
||||
- [x] 2.3 Co-located tests + docs
|
||||
|
||||
## 3. Config factory
|
||||
|
||||
- [x] 3.1 `src/config/create-playwright-config.ts` — factory with WSL2/CI defaults
|
||||
- [x] 3.2 Co-located tests covering: defaults, CI mode overrides, caller overrides
|
||||
|
||||
## 4. Page object base class
|
||||
|
||||
- [x] 4.1 `src/page-objects/page-object.ts` — abstract `PageObject` with `navigate`, `waitForReady`, `findByRole`/`findByLabel`/`findByText`
|
||||
- [x] 4.2 Co-located tests using a mock Playwright `Page` shape
|
||||
|
||||
## 5. Helpers
|
||||
|
||||
- [x] 5.1 `src/helpers/wait-for-network-idle.ts`
|
||||
- [x] 5.2 `src/helpers/screenshot-on-failure.ts`
|
||||
- [x] 5.3 `src/helpers/dismiss-dialog.ts`
|
||||
- [x] 5.4 `src/helpers/mock-geolocation.ts`
|
||||
- [x] 5.5 `src/helpers/mock-clipboard.ts`
|
||||
- [x] 5.6 Co-located tests for each (mocking the Playwright surface)
|
||||
|
||||
## 6. Test extension
|
||||
|
||||
- [x] 6.1 `src/fixtures/extended-test.ts` — `extendedTest = test.extend({ apiContext, freshPage })`
|
||||
- [x] 6.2 Co-located shape test (cannot real-test without launching browsers)
|
||||
|
||||
## 7. Observability (gated)
|
||||
|
||||
- [x] 7.1 `src/observability/instrumented.ts` — gated wrapper, mirroring testing-dom
|
||||
|
||||
## 8. Cross-cutting tests
|
||||
|
||||
- [x] 8.1 `tests/dep-graph.test.ts`
|
||||
- [x] 8.2 `tests/no-disallowed.test.ts`
|
||||
- [x] 8.3 `tests/tree-shake.test.ts`
|
||||
|
||||
## 9. Pre-publish gate
|
||||
|
||||
- [x] 9.1 `bunx turbo lint typecheck test --filter=@suaveplan/testing-e2e` green
|
||||
- [x] 9.2 ≥ 95% coverage on every dimension
|
||||
- [x] 9.3 `openspec validate enterprise-testing-e2e --type change` passes
|
||||
|
||||
## 10. Publish (USER APPROVAL REQUIRED)
|
||||
|
||||
- [x] 10.1 Surface audit and STOP for approval
|
||||
- [x] 10.2 On approval: `bun scripts/publish.ts --package @suaveplan/testing-e2e`
|
||||
- [x] 10.3 Archive change to `openspec/archive/<yyyy-mm-dd>-enterprise-testing-e2e/`
|
||||
Reference in New Issue
Block a user