Initial Commit
This commit is contained in:
@@ -0,0 +1,32 @@
|
||||
## Context
|
||||
|
||||
Every genesis package is required to ship `@opentelemetry/api` instrumentation (counters, histograms, spans) on its public entrypoints, with a no-op fallback when no SDK is registered (`openspec/specs/testing/spec.md` "Observability" requirement; `CLAUDE.md` Quality gates). The audit at `/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md` §1 and §4 records two related defects: there is no fixture that exercises the export path of that instrumentation, and the test runner itself emits no telemetry — so flaky-test triage has nothing to correlate against. This change addresses both at once because the runner-side requirement is meaningless without a collector to receive its export, and the collector fixture is hard to justify without a real producer driving it.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- A `ServiceFixture` plugin for `otel/opentelemetry-collector-contrib` whose lifecycle, env-bridge, and health contract match the conventions established by the existing `testing-integration-*` plugins (CLAUDE.md package list).
|
||||
- A runner reporter at `@suaveplan/testing/reporters/otel` that wraps each `it`/`test` in a span, records assertion failures as span events, and serialises the full span tree of failed tests to disk + Garage S3.
|
||||
- Deterministic span ids and timestamps under test (the reporter accepts `Clock` and `IdGenerator` from `@suaveplan/types` so OTel-of-OTel-tests is itself reproducible).
|
||||
- Zero behaviour change when `OTEL_EXPORTER_OTLP_ENDPOINT` is unset.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Replacing the per-package `@opentelemetry/api` instrumentation. The reporter wraps tests, not source code.
|
||||
- Shipping a Jaeger / Tempo / Honeycomb backend — the collector fixture supports `logging`, `memory`, `jaeger`, and `file` exporters; downstream backend choice is a consumer concern.
|
||||
- Sampling. Every test span is exported; tail-sampling is the consumer's job in their own collector config.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Memory exporter is the assertion path.** The `getSpans({ filter })` helper queries the collector's in-memory exporter rather than parsing `logging` exporter stdout — assertion-grade reliability is required.
|
||||
- **Trace sink is opt-in.** The reporter serialises failure traces unconditionally to `playwright-report/traces/<spec>.json` (zero infra), but only uploads when `TESTING_TRACE_SINK_URL` is set. This avoids surprise S3 traffic from local runs.
|
||||
- **Reporter ships in `@suaveplan/testing`** (not in the new plugin) so that runner-side OTel works without taking the testcontainers dep — testing-infrastructure rule (`CLAUDE.md` rule 8) forbids `@suaveplan/*` runtime deps in `@suaveplan/testing` but not external `@opentelemetry/*` packages, which are already permitted.
|
||||
- **Config synthesis vs `configFile`.** Most consumers want "OTLP HTTP receiver + memory exporter"; the plugin synthesises that case from the typed config object. Power users pass `configFile` and the plugin mounts it verbatim — no template language.
|
||||
- **Image pin.** Tag `0.<minor>` initially (matches existing plugin practice); `tasks.md` tracks the digest-pin migration as a follow-up so this change does not block on a registry lookup.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Collector startup latency** can dominate fast unit-test suites. Mitigated by `skipIfUnavailable()` falling through when the fixture cannot start (consistent with other `testing-integration-*` plugins) and by recommending nightly-only export in CI.
|
||||
- **Memory-exporter retention** is bounded; long suites can drop early spans. The plugin documents the bound and exposes a `flushSpans()` method that pages through the memory exporter at suite teardown.
|
||||
- **Failure-trace upload** introduces a network dependency on red runs. Mitigated by best-effort upload (failures logged, not thrown) and by the env-var gate.
|
||||
@@ -0,0 +1,39 @@
|
||||
## Why
|
||||
|
||||
Section 4 of the test-suite audit (`/mnt/s/web/genesis/docs/superpowers/specs/2026-04-25-test-suite-audit.md`) flags that observability of the test suite itself is absent: every `@suaveplan/*` package ships OTel instrumentation via `@opentelemetry/api`, but there is no fixture that exercises the export path end-to-end and no runner hook that turns each `it`/`test` into a span. Today CI can only assert "no SDK registered = no-op"; there is no way to verify that a counter, histogram, or span actually reaches a collector when one is registered. Section 1 of the same audit lists the OpenTelemetry collector as a critical missing service-fixture (alongside OIDC, Mongo, Kafka/NATS).
|
||||
|
||||
This change closes both gaps in a single capability: a `@suaveplan/testing-integration-otel-collector` plugin (a `ServiceFixture` for `otel/opentelemetry-collector-contrib`) plus a normative requirement on the `testing` capability that the runner SHALL emit a span per test, an event per assertion failure, and an event per fixture lifecycle transition when an OTLP endpoint is configured. Failed-test span trees are serialised to `playwright-report/traces/` and uploaded to a configured Garage S3 sink — directly addressing audit §4 ("Trace-on-failure exists locally but is not exported anywhere").
|
||||
|
||||
## What Changes
|
||||
|
||||
### New plugin package — `@suaveplan/testing-integration-otel-collector`
|
||||
|
||||
- Lives at `packages/testing/testing-integration-otel-collector` (mirrors the `testing-integration-*` plugin convention from `CLAUDE.md`).
|
||||
- Implements the `ServiceFixture<TConfig>` interface from `@suaveplan/testing-integration` with image `otel/opentelemetry-collector-contrib` pinned to a specific minor (digest-pin tracked in `tasks.md`).
|
||||
- Env-bridge: `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL` (`http/protobuf` default, `grpc` opt-in), optional `OTEL_EXPORTER_OTLP_HEADERS`.
|
||||
- HTTP healthcheck on `:13133/health/status` (port configurable; default timeout 60 s).
|
||||
- Config object `{ receivers: ('otlp-grpc' | 'otlp-http')[]; exporters: ('logging' | 'memory' | 'jaeger' | 'file')[]; configFile?: string }` — when no `configFile` is provided the plugin synthesises a minimal `otelcol-config.yaml` from the receivers/exporters lists.
|
||||
- Public helper `getSpans({ filter })` queries the in-memory exporter (when configured) and returns `ReadableSpan[]` for assertion in tests.
|
||||
- Errors prefixed `INTEGRATION.OTEL_COLLECTOR.*`; injectable `containerFactory` and `fetchFn` for unit-testability of the fixture itself.
|
||||
|
||||
### Test-execution OTel export — addition to the `testing` capability
|
||||
|
||||
When `OTEL_EXPORTER_OTLP_ENDPOINT` is set the test runner SHALL emit one span per `it`/`test` (`name`, `status`, `attributes={ suite, file, retry }`), one event per assertion failure (`exception.type`, `exception.message`, `exception.stacktrace`), and one event per fixture lifecycle transition (`setup-start`, `setup-end`, `teardown-start`, `teardown-end`). Implementation hooks ship at `@suaveplan/testing/reporters/otel` for both vitest and bun-test. Failed tests serialise their full span tree to `playwright-report/traces/<spec>.json` and (when `TESTING_TRACE_SINK_URL` is set) upload that file to the configured Garage S3 bucket.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `testing-integration-otel-collector` — a `ServiceFixture` for the OpenTelemetry collector (image policy, env-bridge, health-check contract, config shape, span-query helper, error code prefix).
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `testing` — adds the **Test-execution OTel export** requirement (runner emits spans/events for tests and fixture lifecycle, trace-on-failure serialised and uploaded).
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected code:** new package at `/mnt/s/web/genesis/packages/testing/testing-integration-otel-collector/**`; new reporters at `/mnt/s/web/genesis/packages/testing/testing/src/reporters/otel/**`.
|
||||
- **Affected consumers:** every `@suaveplan/*` test suite gains span export when the env var is set; zero behaviour change when unset.
|
||||
- **Catalog additions (root `package.json`):** none — `@opentelemetry/sdk-node`, `@opentelemetry/sdk-trace-node`, and `@opentelemetry/exporter-trace-otlp-http` are added under `catalogs.observability` only if not already pinned by `@suaveplan/opentelemetry`.
|
||||
- **Pre-publish gate:** plugin package must hit 100% Istanbul coverage (per `openspec/AGENTS.md` §7) and pass the standard `lint typecheck test` suite before publish.
|
||||
- **Cross-cutting:** the `testing` spec gains a new requirement; folding on archive will append it to `openspec/specs/testing/spec.md`.
|
||||
@@ -0,0 +1,94 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Image policy
|
||||
|
||||
The `@suaveplan/testing-integration-otel-collector` package SHALL launch the `otel/opentelemetry-collector-contrib` container pinned to a specific minor version (e.g. `0.<minor>`). A digest-pin migration MUST be tracked as a follow-up task; bare `latest` tags are forbidden.
|
||||
|
||||
#### Scenario: Tag pin enforced
|
||||
|
||||
- **WHEN** the fixture is constructed without a config override
|
||||
- **THEN** the resolved image string MUST match `otel/opentelemetry-collector-contrib:0.<minor>` AND MUST NOT equal `otel/opentelemetry-collector-contrib:latest`
|
||||
|
||||
#### Scenario: Image override accepted
|
||||
|
||||
- **WHEN** the fixture constructor is given `{ image: 'otel/opentelemetry-collector-contrib:0.99.0@sha256:...' }`
|
||||
- **THEN** the launched container uses the supplied digest-pinned reference verbatim
|
||||
|
||||
### Requirement: Env-bridge contract
|
||||
|
||||
The fixture SHALL set `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_PROTOCOL` (`http/protobuf` by default, `grpc` when the config requests `otlp-grpc`) on the host process during `setup()`, and MAY set `OTEL_EXPORTER_OTLP_HEADERS` when the config supplies headers. All values MUST be restored to their prior state on `teardown()`.
|
||||
|
||||
#### Scenario: Endpoint published on setup
|
||||
|
||||
- **WHEN** `setup()` resolves
|
||||
- **THEN** `process.env.OTEL_EXPORTER_OTLP_ENDPOINT` is a URL pointing at the running collector's OTLP port AND `process.env.OTEL_EXPORTER_OTLP_PROTOCOL === 'http/protobuf'`
|
||||
|
||||
#### Scenario: Prior env-state restored on teardown
|
||||
|
||||
- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` was set to `'http://prior'` before `setup()` and `teardown()` later runs
|
||||
- **THEN** `process.env.OTEL_EXPORTER_OTLP_ENDPOINT === 'http://prior'` after `teardown()` resolves
|
||||
|
||||
### Requirement: Health-check contract
|
||||
|
||||
The fixture SHALL consider the collector ready only when an HTTP GET on the collector's `:13133/health/status` endpoint (port configurable) returns 200 within the configured timeout (default 60 s). Health polling MUST use the `HealthPoller` from `@suaveplan/testing-integration`.
|
||||
|
||||
#### Scenario: Health endpoint returns 200
|
||||
|
||||
- **WHEN** the collector is fully started AND the health endpoint returns HTTP 200
|
||||
- **THEN** `setup()` resolves AND `getSpans()` becomes callable
|
||||
|
||||
#### Scenario: Health timeout exceeded
|
||||
|
||||
- **WHEN** the health endpoint never returns 200 within the configured timeout
|
||||
- **THEN** `setup()` rejects with `IntegrationError` whose `code === "INTEGRATION.OTEL_COLLECTOR.HEALTH_TIMEOUT"` AND whose `context.timeoutMs` matches the configured timeout
|
||||
|
||||
#### Scenario: Injectable fetch for unit-testing the fixture
|
||||
|
||||
- **WHEN** the fixture is constructed with `{ fetchFn }`
|
||||
- **THEN** all health-check requests MUST use the supplied `fetchFn` AND MUST NOT touch the global `fetch`
|
||||
|
||||
### Requirement: Config shape and synthesis
|
||||
|
||||
The fixture config object SHALL conform to `{ receivers: ('otlp-grpc' | 'otlp-http')[]; exporters: ('logging' | 'memory' | 'jaeger' | 'file')[]; configFile?: string }`. When `configFile` is omitted the fixture MUST synthesise a minimal `otelcol-config.yaml` from the `receivers` and `exporters` lists and mount it into the container. When `configFile` is supplied the fixture MUST mount that file verbatim and MUST NOT synthesise.
|
||||
|
||||
#### Scenario: Synthesised config wires the requested receivers and exporters
|
||||
|
||||
- **WHEN** `setup({ receivers: ['otlp-http'], exporters: ['memory'] })` runs
|
||||
- **THEN** the mounted `otelcol-config.yaml` declares an OTLP HTTP receiver AND an in-memory exporter AND a single pipeline routing the receiver to the exporter
|
||||
|
||||
#### Scenario: External config file is mounted verbatim
|
||||
|
||||
- **WHEN** `setup({ receivers: [], exporters: [], configFile: '/abs/path/custom.yaml' })` runs
|
||||
- **THEN** the container's `/etc/otelcol-contrib/config.yaml` is the byte-identical contents of `/abs/path/custom.yaml` AND no synthesis occurs
|
||||
|
||||
### Requirement: Span query helper
|
||||
|
||||
The fixture SHALL expose `getSpans({ filter })` which returns `ReadableSpan[]` from the collector's in-memory exporter. The helper MUST throw `IntegrationError("INTEGRATION.OTEL_COLLECTOR.MEMORY_EXPORTER_NOT_CONFIGURED", …)` when the active config does not include the `memory` exporter.
|
||||
|
||||
#### Scenario: Returns spans matching filter
|
||||
|
||||
- **WHEN** a producer has emitted spans named `op.foo` and `op.bar` AND `getSpans({ filter: span => span.name === 'op.foo' })` is called
|
||||
- **THEN** the result contains every `op.foo` span AND no `op.bar` spans
|
||||
|
||||
#### Scenario: Helper unavailable without memory exporter
|
||||
|
||||
- **WHEN** the fixture was started with `exporters: ['logging']` AND `getSpans()` is called
|
||||
- **THEN** the call throws `IntegrationError` with `code === "INTEGRATION.OTEL_COLLECTOR.MEMORY_EXPORTER_NOT_CONFIGURED"`
|
||||
|
||||
### Requirement: Error code prefix
|
||||
|
||||
Every error thrown by the package SHALL be an `IntegrationError` (or subclass) whose `code` begins with `INTEGRATION.OTEL_COLLECTOR.`. Plain `Error` / `TypeError` MUST NOT be thrown from any public code path.
|
||||
|
||||
#### Scenario: Code namespace enforced
|
||||
|
||||
- **WHEN** any public method on the fixture throws
|
||||
- **THEN** the thrown value `instanceof IntegrationError === true` AND `error.code.startsWith("INTEGRATION.OTEL_COLLECTOR.") === true`
|
||||
|
||||
### Requirement: Coverage gate
|
||||
|
||||
Istanbul statement, branch, function, and line coverage across `packages/testing/testing-integration-otel-collector/src/` (excluding `*.bench.ts` and `*.schemas.ts`) MUST be 100%. The pre-publish gate (`bunx turbo test:coverage --filter=@suaveplan/testing-integration-otel-collector`) MUST hard-fail below 100%.
|
||||
|
||||
#### Scenario: Coverage gate trips below 100%
|
||||
|
||||
- **WHEN** any line, branch, function, or statement is uncovered
|
||||
- **THEN** the pre-publish gate fails AND the package MUST NOT be published
|
||||
@@ -0,0 +1,53 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Test-execution OTel export
|
||||
|
||||
When the test runner detects that `OTEL_EXPORTER_OTLP_ENDPOINT` is set in its environment, it SHALL emit one OpenTelemetry span per `it` / `test` invocation, one span event per assertion failure, and one span event per fixture lifecycle transition. When the env var is unset the runner MUST NOT create any spans, register any SDK, or perform any export work — the path MUST be a no-op.
|
||||
|
||||
Per-test spans SHALL carry attributes `{ suite: string; file: string; retry: number }` and a span name equal to the test's fully-qualified description (e.g. `"FakeClock > advances time"`). Span status MUST be `OK` for passing tests, `ERROR` for failing tests, and `UNSET` for skipped tests. Failure events MUST carry attributes `exception.type`, `exception.message`, and `exception.stacktrace`. Fixture lifecycle events MUST be one of `setup-start`, `setup-end`, `teardown-start`, `teardown-end`.
|
||||
|
||||
Implementation hooks SHALL ship at `@suaveplan/testing/reporters/otel` for both Vitest and bun-test. The reporter SHALL accept an optional `{ clock?, idGenerator? }` options bundle so span timestamps and ids are deterministic under test of the reporter itself.
|
||||
|
||||
#### Scenario: Span emitted per test when endpoint set
|
||||
|
||||
- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` is set AND a vitest suite runs `describe("X", () => it("y", …))`
|
||||
- **THEN** exactly one span with name `"X > y"` is exported AND its attributes include `suite === "X"` AND `file` is the absolute spec path AND `retry === 0` on the first attempt
|
||||
|
||||
#### Scenario: Assertion failure recorded as span event
|
||||
|
||||
- **WHEN** an `it` assertion throws an `AssertionError`
|
||||
- **THEN** the test's span has exactly one event whose name is `"exception"` AND whose attributes include `exception.type === "AssertionError"` AND `exception.message` matches the thrown message AND `exception.stacktrace` is the captured stack
|
||||
|
||||
#### Scenario: Fixture lifecycle events emitted
|
||||
|
||||
- **WHEN** a `FixtureManager` runs `setupAll` over fixture `db` followed by `teardownAll`
|
||||
- **THEN** the active span tree contains events `setup-start{name:"db"}`, `setup-end{name:"db"}`, `teardown-start{name:"db"}`, `teardown-end{name:"db"}` in that order
|
||||
|
||||
#### Scenario: Skipped test emits UNSET span
|
||||
|
||||
- **WHEN** a test is marked `it.skip`
|
||||
- **THEN** a span is exported with status `UNSET` AND no `exception` events
|
||||
|
||||
#### Scenario: No-op without endpoint
|
||||
|
||||
- **WHEN** `OTEL_EXPORTER_OTLP_ENDPOINT` is unset AND the test suite runs
|
||||
- **THEN** zero spans are exported AND the reporter MUST NOT instantiate any SDK AND benchmark deltas relative to the unwrapped runner MUST be < 5% per op
|
||||
|
||||
### Requirement: Trace-on-failure persistence
|
||||
|
||||
For every failed test the runner SHALL serialise the test's full span tree (including child spans from instrumented code under test) to `playwright-report/traces/<spec>.json` on the local filesystem. When `TESTING_TRACE_SINK_URL` is set the runner SHALL additionally upload that file to the supplied URL via HTTP PUT (Garage S3-compatible). Upload failures MUST be logged but MUST NOT fail the test run.
|
||||
|
||||
#### Scenario: Failed test serialises trace to disk
|
||||
|
||||
- **WHEN** a test in `auth.test.ts` fails
|
||||
- **THEN** the file `playwright-report/traces/auth.test.json` exists AND its parsed content contains the failed test's span AND every child span emitted while the test was on the stack
|
||||
|
||||
#### Scenario: Trace uploaded when sink configured
|
||||
|
||||
- **WHEN** `TESTING_TRACE_SINK_URL=https://garage.example/traces/run-123/` is set AND a test fails
|
||||
- **THEN** an HTTP PUT is issued to `https://garage.example/traces/run-123/<spec>.json` AND the request body is byte-identical to the on-disk trace file
|
||||
|
||||
#### Scenario: Upload failure does not fail the run
|
||||
|
||||
- **WHEN** the trace-sink PUT returns HTTP 500
|
||||
- **THEN** a warning is logged AND the test runner exit code remains determined by test outcomes only AND the on-disk trace file is preserved
|
||||
@@ -0,0 +1,56 @@
|
||||
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. Catalog and package skeleton
|
||||
|
||||
- [ ] 1.1 Add `@opentelemetry/sdk-node`, `@opentelemetry/sdk-trace-node`, and `@opentelemetry/exporter-trace-otlp-http` to root `package.json` `catalogs.observability` (skip any already pinned by `@suaveplan/opentelemetry`).
|
||||
- [ ] 1.2 Create `packages/testing/testing-integration-otel-collector/` with `package.json` (version `0.1.0`, `type: "module"`, `sideEffects: false`, `workspace:^` dep on `@suaveplan/testing-integration`, catalog dep on `testcontainers`).
|
||||
- [ ] 1.3 Add `tsconfig.lib.json` extending `@suaveplan/typescript-config/library-node.json`; add `vite.config.ts` with `preserveModules: true`; add `vitest.config.ts` via `createTestConfig()`.
|
||||
- [ ] 1.4 Stub `README.md` (one-paragraph description, install, quick start, **Modules** index linking each co-located `.md`).
|
||||
|
||||
## 2. ServiceFixture implementation
|
||||
|
||||
- [ ] 2.1 Implement `src/fixture/otel-collector-fixture.ts` — `OtelCollectorFixture` implementing `ServiceFixture<OtelCollectorConfig>`; constructor accepts injectable `containerFactory` and `fetchFn`.
|
||||
- [ ] 2.2 Implement `src/config/synthesize-config.ts` — produce `otelcol-config.yaml` from `{ receivers, exporters }`; include the in-memory exporter wiring when `'memory'` is requested.
|
||||
- [ ] 2.3 Implement `src/health/health-check.ts` — HTTP GET on `:13133/health/status`, default timeout 60 s, retries via `HealthPoller` from `@suaveplan/testing-integration`.
|
||||
- [ ] 2.4 Implement `src/env-bridge/env-bridge.ts` — sets `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, optional `OTEL_EXPORTER_OTLP_HEADERS`; restores prior values on teardown.
|
||||
- [ ] 2.5 Implement `src/query/get-spans.ts` — `getSpans({ filter })` queries the memory exporter via the collector's debug HTTP endpoint; returns `ReadableSpan[]`; throws `IntegrationError("INTEGRATION.OTEL_COLLECTOR.MEMORY_EXPORTER_NOT_CONFIGURED", …)` when the memory exporter is not in the active config.
|
||||
- [ ] 2.6 Implement `src/errors/codes.ts` — string-literal-union of every `INTEGRATION.OTEL_COLLECTOR.<REASON>` code thrown by the package.
|
||||
- [ ] 2.7 Co-locate `*.test.ts`, `*.schemas.ts`, `*.md` for every implementation file (`.md` ≥ 200 words with simple + advanced examples per `openspec/AGENTS.md` §4).
|
||||
|
||||
## 3. Image pinning (tracked, follow-up)
|
||||
|
||||
- [ ] 3.1 Tag-pin `otel/opentelemetry-collector-contrib:0.<minor>` initially.
|
||||
- [ ] 3.2 Open a tracking task to migrate to digest-pin (`@sha256:…`) once the minor version is locked — referenced from audit §5 "Images pinned by tag, not digest".
|
||||
|
||||
## 4. Runner reporter — `@suaveplan/testing/reporters/otel`
|
||||
|
||||
- [ ] 4.1 Add subpath export `./reporters/otel` to `packages/testing/testing/package.json`.
|
||||
- [ ] 4.2 Implement `src/reporters/otel/vitest-reporter.ts` — Vitest reporter that opens a span per `it`/`test`, records `exception.*` events on assertion failure, and writes the full span tree to `playwright-report/traces/<spec>.json` on red.
|
||||
- [ ] 4.3 Implement `src/reporters/otel/bun-test-reporter.ts` — equivalent for bun-test (uses bun-test's reporter API).
|
||||
- [ ] 4.4 Implement `src/reporters/otel/trace-sink.ts` — uploads the failure trace to `TESTING_TRACE_SINK_URL` (Garage S3 PUT); best-effort (failures logged, not thrown).
|
||||
- [ ] 4.5 Implement `src/reporters/otel/fixture-events.ts` — hook into `FixtureManager` lifecycle to emit `setup-start`/`setup-end`/`teardown-start`/`teardown-end` span events.
|
||||
- [ ] 4.6 Co-locate tests, schemas, and docs for every new file.
|
||||
|
||||
## 5. Spec deltas
|
||||
|
||||
- [ ] 5.1 Land `specs/testing-integration-otel-collector/spec.md` with all `## ADDED Requirements` (image policy, env-bridge, health-check, config shape, span-query, errors).
|
||||
- [ ] 5.2 Land `specs/testing/spec.md` with the `## ADDED Requirements` block carrying the **Test-execution OTel export** requirement.
|
||||
- [ ] 5.3 On ship, fold both into `openspec/specs/testing-integration-otel-collector/spec.md` (new file) and `openspec/specs/testing/spec.md` (append).
|
||||
|
||||
## 6. Verification
|
||||
|
||||
- [ ] 6.1 `bunx turbo lint typecheck test --filter=@suaveplan/testing-integration-otel-collector` green.
|
||||
- [ ] 6.2 `bunx turbo test:coverage --filter=@suaveplan/testing-integration-otel-collector` at 100% statement / branch / function / line.
|
||||
- [ ] 6.3 `bunx turbo lint typecheck test --filter=@suaveplan/testing` green after reporter additions; coverage stays at 100%.
|
||||
- [ ] 6.4 End-to-end smoke: a sample test suite with `OTEL_EXPORTER_OTLP_ENDPOINT` set produces queryable spans via `getSpans()`.
|
||||
- [ ] 6.5 Banned-pattern scan (`@deprecated`, `TODO`, `FIXME`, `ts-ignore`, `ts-expect-error`, `.skip`, `.only`, bare `any`) clean.
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Move `openspec/changes/enterprise-testing-integration-otel-collector/` → `openspec/archive/<yyyy-mm-dd>-enterprise-testing-integration-otel-collector/` after all gates pass.
|
||||
Reference in New Issue
Block a user