Initial Commit

openspec/changes/enterprise-testing-integration-otel-collector/specs/testing-integration-otel-collector/spec.md

@@ -0,0 +1,94 @@
+## ADDED Requirements
+
+### Requirement: Image policy
+
+The `@suaveplan/testing-integration-otel-collector` package SHALL launch the `otel/opentelemetry-collector-contrib` container pinned to a specific minor version (e.g. `0.<minor>`). A digest-pin migration MUST be tracked as a follow-up task; bare `latest` tags are forbidden.
+
+#### Scenario: Tag pin enforced
+
+- **WHEN** the fixture is constructed without a config override
+- **THEN** the resolved image string MUST match `otel/opentelemetry-collector-contrib:0.<minor>` AND MUST NOT equal `otel/opentelemetry-collector-contrib:latest`
+
+#### Scenario: Image override accepted
+
+- **WHEN** the fixture constructor is given `{ image: 'otel/opentelemetry-collector-contrib:0.99.0@sha256:...' }`
+- **THEN** the launched container uses the supplied digest-pinned reference verbatim
+
+### Requirement: Env-bridge contract
+
+The fixture SHALL set `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_PROTOCOL` (`http/protobuf` by default, `grpc` when the config requests `otlp-grpc`) on the host process during `setup()`, and MAY set `OTEL_EXPORTER_OTLP_HEADERS` when the config supplies headers. All values MUST be restored to their prior state on `teardown()`.
+
+#### Scenario: Endpoint published on setup
+
+- **WHEN** `setup()` resolves
+- **THEN** `process.env.OTEL_EXPORTER_OTLP_ENDPOINT` is a URL pointing at the running collector's OTLP port AND `process.env.OTEL_EXPORTER_OTLP_PROTOCOL === 'http/protobuf'`
+
+#### Scenario: Prior env-state restored on teardown
+
+- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` was set to `'http://prior'` before `setup()` and `teardown()` later runs
+- **THEN** `process.env.OTEL_EXPORTER_OTLP_ENDPOINT === 'http://prior'` after `teardown()` resolves
+
+### Requirement: Health-check contract
+
+The fixture SHALL consider the collector ready only when an HTTP GET on the collector's `:13133/health/status` endpoint (port configurable) returns 200 within the configured timeout (default 60 s). Health polling MUST use the `HealthPoller` from `@suaveplan/testing-integration`.
+
+#### Scenario: Health endpoint returns 200
+
+- **WHEN** the collector is fully started AND the health endpoint returns HTTP 200
+- **THEN** `setup()` resolves AND `getSpans()` becomes callable
+
+#### Scenario: Health timeout exceeded
+
+- **WHEN** the health endpoint never returns 200 within the configured timeout
+- **THEN** `setup()` rejects with `IntegrationError` whose `code === "INTEGRATION.OTEL_COLLECTOR.HEALTH_TIMEOUT"` AND whose `context.timeoutMs` matches the configured timeout
+
+#### Scenario: Injectable fetch for unit-testing the fixture
+
+- **WHEN** the fixture is constructed with `{ fetchFn }`
+- **THEN** all health-check requests MUST use the supplied `fetchFn` AND MUST NOT touch the global `fetch`
+
+### Requirement: Config shape and synthesis
+
+The fixture config object SHALL conform to `{ receivers: ('otlp-grpc' | 'otlp-http')[]; exporters: ('logging' | 'memory' | 'jaeger' | 'file')[]; configFile?: string }`. When `configFile` is omitted the fixture MUST synthesise a minimal `otelcol-config.yaml` from the `receivers` and `exporters` lists and mount it into the container. When `configFile` is supplied the fixture MUST mount that file verbatim and MUST NOT synthesise.
+
+#### Scenario: Synthesised config wires the requested receivers and exporters
+
+- **WHEN** `setup({ receivers: ['otlp-http'], exporters: ['memory'] })` runs
+- **THEN** the mounted `otelcol-config.yaml` declares an OTLP HTTP receiver AND an in-memory exporter AND a single pipeline routing the receiver to the exporter
+
+#### Scenario: External config file is mounted verbatim
+
+- **WHEN** `setup({ receivers: [], exporters: [], configFile: '/abs/path/custom.yaml' })` runs
+- **THEN** the container's `/etc/otelcol-contrib/config.yaml` is the byte-identical contents of `/abs/path/custom.yaml` AND no synthesis occurs
+
+### Requirement: Span query helper
+
+The fixture SHALL expose `getSpans({ filter })` which returns `ReadableSpan[]` from the collector's in-memory exporter. The helper MUST throw `IntegrationError("INTEGRATION.OTEL_COLLECTOR.MEMORY_EXPORTER_NOT_CONFIGURED", …)` when the active config does not include the `memory` exporter.
+
+#### Scenario: Returns spans matching filter
+
+- **WHEN** a producer has emitted spans named `op.foo` and `op.bar` AND `getSpans({ filter: span => span.name === 'op.foo' })` is called
+- **THEN** the result contains every `op.foo` span AND no `op.bar` spans
+
+#### Scenario: Helper unavailable without memory exporter
+
+- **WHEN** the fixture was started with `exporters: ['logging']` AND `getSpans()` is called
+- **THEN** the call throws `IntegrationError` with `code === "INTEGRATION.OTEL_COLLECTOR.MEMORY_EXPORTER_NOT_CONFIGURED"`
+
+### Requirement: Error code prefix
+
+Every error thrown by the package SHALL be an `IntegrationError` (or subclass) whose `code` begins with `INTEGRATION.OTEL_COLLECTOR.`. Plain `Error` / `TypeError` MUST NOT be thrown from any public code path.
+
+#### Scenario: Code namespace enforced
+
+- **WHEN** any public method on the fixture throws
+- **THEN** the thrown value `instanceof IntegrationError === true` AND `error.code.startsWith("INTEGRATION.OTEL_COLLECTOR.") === true`
+
+### Requirement: Coverage gate
+
+Istanbul statement, branch, function, and line coverage across `packages/testing/testing-integration-otel-collector/src/` (excluding `*.bench.ts` and `*.schemas.ts`) MUST be 100%. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing-integration-otel-collector`) MUST hard-fail below 100%.
+
+#### Scenario: Coverage gate trips below 100%
+
+- **WHEN** any line, branch, function, or statement is uncovered
+- **THEN** the pre-publish gate fails AND the package MUST NOT be published

openspec/changes/enterprise-testing-integration-otel-collector/specs/testing/spec.md

@@ -0,0 +1,53 @@
+## ADDED Requirements
+
+### Requirement: Test-execution OTel export
+
+When the test runner detects that `OTEL_EXPORTER_OTLP_ENDPOINT` is set in its environment, it SHALL emit one OpenTelemetry span per `it` / `test` invocation, one span event per assertion failure, and one span event per fixture lifecycle transition. When the env var is unset the runner MUST NOT create any spans, register any SDK, or perform any export work — the path MUST be a no-op.
+
+Per-test spans SHALL carry attributes `{ suite: string; file: string; retry: number }` and a span name equal to the test's fully-qualified description (e.g. `"FakeClock > advances time"`). Span status MUST be `OK` for passing tests, `ERROR` for failing tests, and `UNSET` for skipped tests. Failure events MUST carry attributes `exception.type`, `exception.message`, and `exception.stacktrace`. Fixture lifecycle events MUST be one of `setup-start`, `setup-end`, `teardown-start`, `teardown-end`.
+
+Implementation hooks SHALL ship at `@suaveplan/testing/reporters/otel` for both Vitest and bun-test. The reporter SHALL accept an optional `{ clock?, idGenerator? }` options bundle so span timestamps and ids are deterministic under test of the reporter itself.
+
+#### Scenario: Span emitted per test when endpoint set
+
+- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` is set AND a vitest suite runs `describe("X", () => it("y", …))`
+- **THEN** exactly one span with name `"X > y"` is exported AND its attributes include `suite === "X"` AND `file` is the absolute spec path AND `retry === 0` on the first attempt
+
+#### Scenario: Assertion failure recorded as span event
+
+- **WHEN** an `it` assertion throws an `AssertionError`
+- **THEN** the test's span has exactly one event whose name is `"exception"` AND whose attributes include `exception.type === "AssertionError"` AND `exception.message` matches the thrown message AND `exception.stacktrace` is the captured stack
+
+#### Scenario: Fixture lifecycle events emitted
+
+- **WHEN** a `FixtureManager` runs `setupAll` over fixture `db` followed by `teardownAll`
+- **THEN** the active span tree contains events `setup-start{name:"db"}`, `setup-end{name:"db"}`, `teardown-start{name:"db"}`, `teardown-end{name:"db"}` in that order
+
+#### Scenario: Skipped test emits UNSET span
+
+- **WHEN** a test is marked `it.skip`
+- **THEN** a span is exported with status `UNSET` AND no `exception` events
+
+#### Scenario: No-op without endpoint
+
+- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` is unset AND the test suite runs
+- **THEN** zero spans are exported AND the reporter MUST NOT instantiate any SDK AND benchmark deltas relative to the unwrapped runner MUST be < 5% per op
+
+### Requirement: Trace-on-failure persistence
+
+For every failed test the runner SHALL serialise the test's full span tree (including child spans from instrumented code under test) to `playwright-report/traces/<spec>.json` on the local filesystem. When `TESTING_TRACE_SINK_URL` is set the runner SHALL additionally upload that file to the supplied URL via HTTP PUT (Garage S3-compatible). Upload failures MUST be logged but MUST NOT fail the test run.
+
+#### Scenario: Failed test serialises trace to disk
+
+- **WHEN** a test in `auth.test.ts` fails
+- **THEN** the file `playwright-report/traces/auth.test.json` exists AND its parsed content contains the failed test's span AND every child span emitted while the test was on the stack
+
+#### Scenario: Trace uploaded when sink configured
+
+- **WHEN** `TESTING_TRACE_SINK_URL=https://garage.example/traces/run-123/` is set AND a test fails
+- **THEN** an HTTP PUT is issued to `https://garage.example/traces/run-123/<spec>.json` AND the request body is byte-identical to the on-disk trace file
+
+#### Scenario: Upload failure does not fail the run
+
+- **WHEN** the trace-sink PUT returns HTTP 500
+- **THEN** a warning is logged AND the test runner exit code remains determined by test outcomes only AND the on-disk trace file is preserved