suaveplan/openspec/changes/enterprise-testing-mutation/specs/testing/spec.md

ADDED Requirements

Requirement: Mutation testing layered on 100% line coverage

Every @suaveplan/* package SHALL run a Stryker mutation analysis as a quality gate layered on top of the existing 100% Istanbul statement / branch / function / line coverage requirement. 100% line coverage is a precondition for mutation testing; mutation score is an additional, independent gate. Neither requirement substitutes for the other. A package SHALL NOT be published while either gate is red.

Scenario: Mutation gate runs only after coverage gate passes

• WHEN the pre-publish task graph executes for any @suaveplan/<pkg>
• THEN test:mutation MUST run only after test:coverage has reported 100% statement / branch / function / line coverage AND the publish step MUST be blocked unless both gates are green

Scenario: Coverage at 100% does not exempt mutation gate

• WHEN a package reports 100% Istanbul coverage but its mutation score is below its tier threshold
• THEN the publish step MUST fail AND the failure message MUST cite the mutation-score tier threshold and the actual score

Requirement: Stryker tooling and configuration

The mutation-testing tier SHALL be implemented with @stryker-mutator/core and @stryker-mutator/vitest-runner, both referenced via catalog:testing in package devDependencies (no bare version strings). Each package SHALL co-locate a stryker.conf.json (or stryker.conf.cjs extending a shared base config) at the package root that reuses the package's existing vitest.config.ts for the test environment, polyfills, and pool settings.

Scenario: Per-package config present

• WHEN the CI runs bunx turbo test:mutation --filter=@suaveplan/<pkg>
• THEN Stryker MUST resolve a stryker.conf.json (or stryker.conf.cjs) at the package root AND the resolved config MUST set testRunner: "vitest" AND MUST point vitest.configFile at the package's existing vitest.config.ts

Scenario: Catalog-pinned tooling

• WHEN a package's devDependencies are inspected
• THEN @stryker-mutator/core and @stryker-mutator/vitest-runner MUST resolve to catalog:testing (or another catalog key) AND MUST NOT contain a bare version string

Requirement: Tiered mutation-score thresholds

Mutation scores SHALL be enforced at two tiers. Foundation packages — crypto, crypto-utils, crypto-hash, crypto-curve, crypto-cipher, crypto-keys, crypto-vault, error, validation, serialize, datetime, timing, retry, pagination, env, config, mime, templates, i18n, idempotency, slsa, diff — SHALL achieve a mutation score of at least 85% killed. Every other @suaveplan/* package SHALL achieve at least 70% killed. Threshold loosening SHALL NOT occur on a per-package basis; tightening over time is permitted via a follow-up spec change.

Scenario: Foundation package below 85% fails

• WHEN @suaveplan/crypto reports a mutation score of 80% killed
• THEN the CI mutation step MUST fail the build AND the publish step MUST be blocked

Scenario: Non-foundation package below 70% fails

• WHEN @suaveplan/sitemap reports a mutation score of 65% killed
• THEN the CI mutation step MUST fail the build AND the publish step MUST be blocked

Scenario: Foundation package at 85% or above passes

• WHEN @suaveplan/error reports a mutation score of 86% killed
• THEN the CI mutation step MUST pass

Requirement: Mutation-analysis exclusions

The Stryker mutate glob SHALL exclude paths that carry no behavioural assertions to mutate: barrel index.ts files, *.schemas.ts files, *.bench.ts files, every file under __benchmarks__/ directories, and *.stories.tsx files. Exclusions SHALL be expressed via Stryker's mutate glob (not via test-skip), so the coverage of those files via the existing 100% line gate remains visible.

Scenario: Schemas file is not mutated

• WHEN Stryker runs against universal/<domain>/<pkg> and the package contains src/foo/foo.schemas.ts
• THEN that file MUST NOT appear in the Stryker files set for mutation AND MUST still appear in the Istanbul coverage report at 100%

Scenario: Benchmark file is not mutated

• WHEN Stryker runs and the package contains src/__benchmarks__/foo.bench.ts
• THEN that file MUST NOT be mutated

Requirement: CI integration — nightly and per-PR runs

A nightly CI job SHALL run the full mutation analysis for every @suaveplan/* package and publish HTML + JSON reports to artefact storage. Per-PR CI runs SHALL invoke Stryker with the --mutate filter scoped to files changed against the PR base branch, so the wall-clock cost stays within the PR CI envelope. The PR-scoped score SHALL be enforced against the same tier threshold as the full run; a PR that introduces no mutated files passes the gate trivially.

Scenario: Nightly run publishes artefacts

• WHEN the nightly mutation job completes for @suaveplan/<pkg>
• THEN an HTML report and a JSON report MUST be archived as CI artefacts AND the JSON report MUST be machine-readable for downstream tooling

Scenario: PR-scoped run uses --mutate filter

• WHEN a PR modifies packages/crypto/crypto/src/aead/aead.ts and packages/crypto/crypto/src/sign/sign.ts
• THEN the PR mutation job MUST invoke Stryker with --mutate scoped to those two files AND MUST evaluate the resulting score against the package's tier threshold

Requirement: Surviving-mutant reporting and triage

Every CI mutation run SHALL emit a structured PR comment enumerating surviving mutants with path, line, mutator, and status fields. Surviving mutants in a landing change MUST resolve to one of three terminal states before the change merges: (a) killed by a new assertion added in the same change; (b) explicitly marked equivalent via a stryker-disable-next-line <mutator>: <reason> comment in source with reviewer sign-off; (c) deferred via a tracked issue linked from the PR description. No silent surviving mutants SHALL land.

Scenario: PR comment lists surviving mutants

• WHEN a PR mutation run completes with 3 surviving mutants
• THEN a PR comment MUST be posted listing all 3 with path, line, mutator, and status columns

Scenario: Surviving mutant blocks merge without triage

• WHEN a PR mutation run reports 1 surviving mutant AND the PR description does not link a tracked issue AND the source contains no stryker-disable-next-line for that location AND the change adds no assertion that kills the mutant
• THEN the merge gate MUST block the PR

Requirement: Custom mutator and adapter conformance

Any custom Stryker mutator or runner adapter Suaveplan ships (for example, a future bun-test runner adapter) SHALL throw TestingError subclasses with stable codes prefixed TESTING.MUTATION.* (e.g. TESTING.MUTATION.RUNNER_FAILED, TESTING.MUTATION.CONFIG_INVALID). The adapter package itself SHALL meet 100% Istanbul coverage AND SHALL meet the same mutation-score tier threshold it advertises to consumers.

Scenario: Adapter throws coded error on misconfiguration

• WHEN a custom Suaveplan Stryker adapter receives a config missing a required field
• THEN it MUST throw TestingError with code === "TESTING.MUTATION.CONFIG_INVALID" AND context MUST include the missing field name

Scenario: Adapter self-coverage

• WHEN the adapter package's test:coverage task runs
• THEN statement, branch, function, and line coverage MUST each be 100% AND the adapter's own test:mutation task MUST meet at least the 70% non-foundation threshold (or 85% if the adapter is itself foundation)