suaveplan/openspec/changes/enterprise-testing-integration-otel-collector/specs/testing/spec.md

ADDED Requirements

Requirement: Test-execution OTel export

When the test runner detects that OTEL_EXPORTER_OTLP_ENDPOINT is set in its environment, it SHALL emit one OpenTelemetry span per it / test invocation, one span event per assertion failure, and one span event per fixture lifecycle transition. When the env var is unset the runner MUST NOT create any spans, register any SDK, or perform any export work — the path MUST be a no-op.

Per-test spans SHALL carry attributes { suite: string; file: string; retry: number } and a span name equal to the test's fully-qualified description (e.g. "FakeClock > advances time"). Span status MUST be OK for passing tests, ERROR for failing tests, and UNSET for skipped tests. Failure events MUST carry attributes exception.type, exception.message, and exception.stacktrace. Fixture lifecycle events MUST be one of setup-start, setup-end, teardown-start, teardown-end.

Implementation hooks SHALL ship at @suaveplan/testing/reporters/otel for both Vitest and bun-test. The reporter SHALL accept an optional { clock?, idGenerator? } options bundle so span timestamps and ids are deterministic under test of the reporter itself.

Scenario: Span emitted per test when endpoint set

• WHEN OTEL_EXPORTER_OTLP_ENDPOINT is set AND a vitest suite runs describe("X", () => it("y", …))
• THEN exactly one span with name "X > y" is exported AND its attributes include suite === "X" AND file is the absolute spec path AND retry === 0 on the first attempt

Scenario: Assertion failure recorded as span event

• WHEN an it assertion throws an AssertionError
• THEN the test's span has exactly one event whose name is "exception" AND whose attributes include exception.type === "AssertionError" AND exception.message matches the thrown message AND exception.stacktrace is the captured stack

Scenario: Fixture lifecycle events emitted

• WHEN a FixtureManager runs setupAll over fixture db followed by teardownAll
• THEN the active span tree contains events setup-start{name:"db"}, setup-end{name:"db"}, teardown-start{name:"db"}, teardown-end{name:"db"} in that order

Scenario: Skipped test emits UNSET span

• WHEN a test is marked it.skip
• THEN a span is exported with status UNSET AND no exception events

Scenario: No-op without endpoint

• WHEN OTEL_EXPORTER_OTLP_ENDPOINT is unset AND the test suite runs
• THEN zero spans are exported AND the reporter MUST NOT instantiate any SDK AND benchmark deltas relative to the unwrapped runner MUST be < 5% per op

Requirement: Trace-on-failure persistence

For every failed test the runner SHALL serialise the test's full span tree (including child spans from instrumented code under test) to playwright-report/traces/<spec>.json on the local filesystem. When TESTING_TRACE_SINK_URL is set the runner SHALL additionally upload that file to the supplied URL via HTTP PUT (Garage S3-compatible). Upload failures MUST be logged but MUST NOT fail the test run.

Scenario: Failed test serialises trace to disk

• WHEN a test in auth.test.ts fails
• THEN the file playwright-report/traces/auth.test.json exists AND its parsed content contains the failed test's span AND every child span emitted while the test was on the stack

Scenario: Trace uploaded when sink configured

• WHEN TESTING_TRACE_SINK_URL=https://garage.example/traces/run-123/ is set AND a test fails
• THEN an HTTP PUT is issued to https://garage.example/traces/run-123/<spec>.json AND the request body is byte-identical to the on-disk trace file

Scenario: Upload failure does not fail the run

• WHEN the trace-sink PUT returns HTTP 500
• THEN a warning is logged AND the test runner exit code remains determined by test outcomes only AND the on-disk trace file is preserved