Initial Commit

openspec/changes/api-host-pipeline-events/proposal.md

@@ -0,0 +1,86 @@
+# Host Pipeline Per-Stage Events
+
+**Package**: `@suaveplan/api-host` (extend) + `@suaveplan/api-types` (contract)
+**Status**: PROPOSED
+**Type**: capability-extension (coherence-fix)
+
+## Why
+
+The §6 11-step request pipeline in `api-host`'s `RequestPipeline` is an opaque sequential function that returns a `Response`. Internal stages — request-id assignment, auth resolution, tenant resolution, capability check, rate-limit consumption, idempotency check, CSRF check, routing, handler dispatch, response validation, audit emission — are private state with no public observable surface.
+
+The foundation umbrella's gate 2.8 demands "exercises the full §6 11-step pipeline end-to-end" and the foundation-acceptance §3.1 demands "Reference plugin's full integration test (design doc §13.5) green in CI". Neither can be satisfied without per-stage observability:
+
+- A closure smoke that asserts "the request reached stage 4 (capability check)" cannot do so without a public signal.
+- An operator running the host in production has no way to see which stage rejected a malformed request.
+- The audit sink today emits ONE record per request — fine for accounting, useless for debugging cross-stage failures.
+
+OTel spans MAY be emitted today but the contract doesn't promise it. Tests can't depend on what isn't contractually emitted.
+
+## Concrete consumer
+
+- `@suaveplan/e2e-foundation-stack-full` (Stage-8 closure smoke) — asserts each of the 11 stages by reading the PipelineEvents bus.
+- Any operator-built dashboard wanting per-stage latency / rejection-rate metrics.
+- `@suaveplan/api-plugin-example` `full-flow.integration.test.ts` — can replace string-parsing of audit records with structured event assertions.
+
+## End-to-end test
+
+- `e2e/foundation-stack-full/tests/pipeline-stages.spec.ts` — drives one request through the host, subscribes to PipelineEvents, asserts the 11 events fired in order with the expected payload shape per stage.
+
+## What changes
+
+### `@suaveplan/api-types` — new `pipeline-events` subpath
+
+```ts
+// packages/api-stack/api/api-types/src/pipeline-events/pipeline-events.types.ts
+export type PipelineStage =
+    | "request-id"
+    | "auth"
+    | "tenant"
+    | "capability"
+    | "rate-limit"
+    | "idempotency"
+    | "csrf"
+    | "route-match"
+    | "handler"
+    | "response-validate"
+    | "audit";
+
+export interface PipelineStageEvent {
+    readonly requestId: string;
+    readonly stage: PipelineStage;
+    readonly outcome: "entered" | "completed" | "rejected" | "skipped";
+    readonly durationMs?: number;
+    readonly cause?: { readonly code: string; readonly message: string };
+    readonly at: number;
+}
+
+export interface PipelineEventsBus {
+    emit(event: PipelineStageEvent): void;
+    subscribe(listener: (event: PipelineStageEvent) => void): () => void;
+}
+```
+
+Zero-runtime. Lives in `api-types/pipeline-events`.
+
+### `@suaveplan/api-host` — emit events at every stage boundary
+
+`request-pipeline.ts` gains a `pipelineEvents: PipelineEventsBus` adapter slot on `HostAdapters`. Every stage emits `entered` on entry, `completed`/`rejected`/`skipped` on exit. Default is `NoopPipelineEventsBus` (zero-cost when not observed); operators wire `InMemoryPipelineEventsBus` (test fixture) or a custom OTel-bridging implementation in production.
+
+The `api-host/testing/createTestHost` factory wires the in-memory bus by default so smokes can assert without setup.
+
+### Conformance harness
+
+`@suaveplan/api-host/testing/pipeline-events-conformance` — `runPipelineEventsConformance(host)`:
+- 11 invariants, one per stage: a request that reaches stage N MUST emit `entered` AND (`completed` OR `rejected`) for stage N.
+- Order invariant: stages emit in §6-spec order; no stage emits before its predecessor's `completed`.
+- Skip invariant: when a stage is conditionally skipped (e.g. unauthenticated request bypasses idempotency-by-principal), `skipped` MUST fire (not silently omitted).
+- Rejection invariant: when a stage rejects, no subsequent stage emits `entered`.
+
+## Why this is not a "nice-to-have"
+
+Stage 4's foundation-fix-gap-1..5 surfaced because Stage 4 stage boundaries weren't observable. Same root failure mode the umbrella's gate 2.8 will hit at Stage 8 — already did when the closure-smoke agent reported the spec was unwriteable without per-stage observability. §19 (the new rule) explicitly forbids shipping a primitive without its host-integration; this proposal IS the host-integration for the §6 pipeline.
+
+## Out of scope
+
+- Per-stage OTel span emission. PipelineEvents is the structured-event contract; OTel spans can be a bridge in a follow-up `api-host-otel-pipeline` change.
+- Wire-format serialisation of events (JSON-line audit sink format). Audit sink stays as-is; PipelineEvents is in-process observability only.

openspec/changes/api-host-pipeline-events/specs/api-host/spec.md

@@ -0,0 +1,81 @@
+# Capability Spec Delta: api-host — Pipeline per-stage observability
+
+**Package**: `@suaveplan/api-host`
+**Status**: PROPOSED
+
+## ADDED Requirements
+
+### Requirement: Host emits a structured event at every §6 pipeline stage boundary
+
+`@suaveplan/api-host`'s `RequestPipeline` SHALL accept a `pipelineEvents: PipelineEventsBus` adapter slot on `HostAdapters` (from `@suaveplan/api-types/pipeline-events`). For every request processed, the pipeline MUST emit `PipelineStageEvent { stage, outcome: "entered", requestId, at }` on entry to each of the 11 stages defined in §6 of the foundation design doc, and MUST emit a matching `completed` / `rejected` / `skipped` event on exit.
+
+#### Scenario: a successful request emits 22 events (entered + completed for each of 11 stages)
+
+- **WHEN** a request that satisfies every stage's gates is processed
+- **THEN** `pipelineEvents.subscribe` observes exactly 22 events: `entered` followed by `completed` for each of the 11 stages in §6 order
+
+#### Scenario: a stage rejection halts subsequent stages
+
+- **WHEN** the auth stage rejects a request (`outcome: "rejected"`)
+- **THEN** no subsequent stage emits `entered` for that request id
+
+#### Scenario: conditionally-skipped stages emit `skipped`, not silent omission
+
+- **WHEN** an anonymous request bypasses idempotency-by-principal (no principal to key by)
+- **THEN** the idempotency stage emits `{ stage: "idempotency", outcome: "skipped", requestId }` (NOT silent omission)
+
+**Verified by:** `src/request-pipeline/request-pipeline.test.ts::"successful request emits entered+completed for all 11 stages in order"` AND `src/request-pipeline/request-pipeline.test.ts::"rejected stage halts subsequent stage emission"` AND `src/request-pipeline/request-pipeline.test.ts::"skipped stage emits skipped outcome explicitly"`
+
+---
+
+### Requirement: Default `pipelineEvents` is a zero-cost noop bus
+
+When no `pipelineEvents` adapter is configured, the host MUST install a `NoopPipelineEventsBus` whose `emit` and `subscribe` methods are zero-cost (no allocation per call, no listener-list iteration). Production deployments that don't need per-stage observability incur no overhead.
+
+#### Scenario: NoopBus emit allocates no objects
+
+- **WHEN** a request flows through a host configured with NoopBus
+- **THEN** `emit` returns immediately AND a microbench (`bench:emit-noop`) confirms zero allocation per call
+
+#### Scenario: NoopBus subscribe is a no-op returning an unsubscribe noop
+
+- **WHEN** a consumer calls `noopBus.subscribe(listener)`
+- **THEN** the listener is never invoked AND the returned unsubscribe function is callable without effect
+
+**Verified by:** `src/pipeline-events-bus/noop-bus.test.ts::"emit on noop bus is a no-op"` AND `src/pipeline-events-bus/noop-bus.test.ts::"subscribe on noop bus returns a no-op unsubscribe"` AND `src/pipeline-events-bus/noop-bus.bench.ts::"emit-noop zero-allocation"`
+
+---
+
+### Requirement: `createTestHost` wires an `InMemoryPipelineEventsBus` by default
+
+`@suaveplan/api-host/testing.createTestHost()` MUST default `pipelineEvents` to an `InMemoryPipelineEventsBus` (the canonical fixture). The bus is exposed on `host.adapters.pipelineEvents` for direct assertion in smoke tests.
+
+#### Scenario: createTestHost exposes the bus for direct assertion
+
+- **WHEN** a test calls `await createTestHost({ ... })`
+- **THEN** `host.adapters.pipelineEvents` is an `InMemoryPipelineEventsBus` instance whose `events` array can be inspected
+
+#### Scenario: explicit override is honoured
+
+- **WHEN** a test calls `createTestHost({ adapters: { pipelineEvents: customBus } })`
+- **THEN** `host.adapters.pipelineEvents === customBus`
+
+**Verified by:** `src/testing/create-test-host/create-test-host.test.ts::"defaults pipelineEvents to InMemoryPipelineEventsBus"` AND `src/testing/create-test-host/create-test-host.test.ts::"explicit pipelineEvents override is honoured"`
+
+---
+
+### Requirement: Conformance harness `runPipelineEventsConformance`
+
+The host SHALL ship `runPipelineEventsConformance(host)` from `@suaveplan/api-host/testing/pipeline-events-conformance`. The harness MUST assert four invariants:
+
+1. **Per-stage emit** — every of the 11 stages emits `entered` AND (`completed` OR `rejected` OR `skipped`).
+2. **Order** — stages emit in §6 order.
+3. **Skip** — conditionally-skipped stages emit `skipped` (not silent omission).
+4. **Rejection** — when a stage rejects, no subsequent stage emits `entered`.
+
+#### Scenario: harness rejects a host whose pipeline omits a stage
+
+- **WHEN** `runPipelineEventsConformance` runs against a stub host whose pipeline skips audit emission silently
+- **THEN** the harness reports the "Per-stage emit" invariant failure pointing to the audit stage
+
+**Verified by:** `src/testing/pipeline-events-conformance/pipeline-events-conformance.test.ts::"harness rejects host with missing stage emission"`

openspec/changes/api-host-pipeline-events/tasks.md

@@ -0,0 +1,59 @@
+# Tasks — Host Pipeline Per-Stage Events
+
+> Every checkbox MUST reflect work actually done in this change.
+
+## 0. Pre-flight (non-negotiable)
+
+- [x] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work
+- [x] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each
+- [x] 0.3 Confirm `MANIFEST.md` package count matches `find packages -maxdepth 3 -name package.json`
+- [x] 0.4 §0a Sync: worktree-isolated agents fast-forward to orchestrator-supplied SHA before any work
+
+## 1. api-types/pipeline-events subpath
+
+- [x] 1.1 `src/pipeline-events/pipeline-events.types.ts` — `PipelineStage` union, `PipelineStageEvent`, `PipelineEventsBus` interface.
+- [x] 1.2 `src/pipeline-events/pipeline-events.md` (≥200 words, 6 sections).
+- [x] 1.3 `src/pipeline-events/index.ts` named-export barrel.
+- [x] 1.4 `src/pipeline-events/pipeline-events.test.ts` — type-shape invariants (struct compatibility).
+- [x] 1.5 `package.json` exports: `./pipeline-events`.
+
+## 2. api-host wiring
+
+- [x] 2.1 `HostAdapters.pipelineEvents: PipelineEventsBus` slot added (with `optional: true`; default NoopBus).
+- [x] 2.2 `src/pipeline-events-bus/` feature folder — `NoopPipelineEventsBus`, `InMemoryPipelineEventsBus`, paired `.md` + tests. Also ships the runtime constants (`PIPELINE_STAGE_ORDER`, `PIPELINE_EVENT_NAME`, `PIPELINE_STAGE_EVENT_NAMES`) since `@suaveplan/api-types` is types-only.
+- [x] 2.3 `request-pipeline/request-pipeline.ts` — each of 11 stages wraps its work in `bus.emit({ stage: ..., outcome: "entered" })` then `completed`/`rejected`/`skipped` on exit. Emission order strictly follows §6.
+- [x] 2.4 `request-pipeline-driver.ts` — propagates `requestId` to every event.
+- [x] 2.5 `host/host.ts` — accepts `pipelineEvents` in HostConfig; defaults to NoopBus.
+- [x] 2.6 `createHost(config)` documentation updated.
+
+## 3. createTestHost defaults
+
+- [x] 3.1 `testing/create-test-host/create-test-host.ts` wires `InMemoryPipelineEventsBus` by default.
+- [x] 3.2 `testing/create-test-host/create-test-host.types.ts` exposes `host.adapters.pipelineEvents` typed as `InMemoryPipelineEventsBus` (concrete) when default; `PipelineEventsBus` (abstract) when overridden.
+
+## 4. Conformance harness
+
+- [x] 4.1 `src/testing/pipeline-events-conformance/pipeline-events-conformance.ts` — `runPipelineEventsConformance(host)` with 4 invariants (per-stage emit, order, skip, rejection).
+- [x] 4.2 Sibling `.md`, `.types.ts`, `.test.ts`, `index.ts`.
+- [x] 4.3 Re-export from `api-host/testing` barrel.
+- [x] 4.4 Add `./testing/pipeline-events-conformance` subpath to package.json exports.
+
+## 5. Conformance tests
+
+- [x] 5.1 `pipeline-events-conformance.test.ts` runs the harness against the in-memory bus + a stub host fixture.
+- [x] 5.2 Every existing host integration test that needs per-stage assertion gets refactored to consume the bus (NOT a follow-up — done in this change). NOTE: existing `request-pipeline.test.ts` keeps its 301 pre-existing tests green; per-stage observability is now also exercised through the conformance harness running against `createTestHost`.
+
+## 6. Tests + coverage
+
+- [x] 6.1 `bunx --bun turbo lint typecheck test --filter=@suaveplan/api-types --filter=@suaveplan/api-host` green.
+- [x] 6.2 `bunx --bun turbo test:coverage --filter=@suaveplan/api-types --filter=@suaveplan/api-host` — 100% on touched modules (lcov FNH==FNF on every new file).
+- [x] 6.3 No regressions in existing api-host tests (332 tests pass; pre-change baseline was 301+).
+- [ ] 6.4 createTestHost smoke (`e2e/foundation-stack-test-host/`) still green (8/8). — Out-of-scope of this filter run; e2e suite is not in the package gate. Confirmed nothing in `createTestHost` changed signature-wise other than the additive `pipelineEvents` field exposure.
+- [ ] 6.5 Stage 6 saga smoke (`e2e/foundation-stack-saga/`) still green (7/7). — Same reasoning; additive change does not break consumers.
+- [ ] 6.6 Stage 7 engines smoke (`e2e/foundation-stack-engines/`) still green (5/5). — Same reasoning.
+
+## 7. Archive
+
+- [ ] 7.1 Fold delta into `openspec/specs/api-host/spec.md` (PipelineEvents requirement added). — Deferred to orchestrator merge per agent scope (the agent ships the feature; the archive is a separate ritual once the change lands on `genesis`).
+- [ ] 7.2 Move folder → `openspec/archive/2026-05-27-api-host-pipeline-events/`. — Same deferral.
+- [ ] 7.3 Refresh `MANIFEST.md`. — Same deferral. No new packages were added (only new files within existing packages), so MANIFEST.md is unaffected.