Initial Commit

openspec/archive/2026-05-26-api-ws-broker-redis/proposal.md

@@ -0,0 +1,56 @@
+# `@suaveplan/api-ws-broker-redis` — Redis pub/sub broker for cross-host fan-out
+
+**Package**: `@suaveplan/api-ws-broker-redis`
+**Status**: PROPOSED
+**Type**: new-package
+**Stage**: 7 (GraphQL + WebSocket / SSE engines) — see umbrella `openspec/changes/api-plugin-foundation/`
+
+## Why
+
+Per design doc §11.3 single-host deployments use an in-memory broker, but production deployments with multiple host replicas need cross-host broadcast: a `bus.broadcast` on host A must reach the locally-connected subscribers on host B. Redis pub/sub is the simplest and most operationally familiar transport for this. This package is the foundation's reference cross-host broker — required for any multi-replica deployment. The §11.6 SSE design explicitly reuses the same broker abstraction, so this package serves both `api-ws-bun` (WebSocket) and `api-sse-bun` (SSE) without duplication.
+
+Sticky sessions are NOT required because connections are not migrated across hosts — the broker just makes broadcasts visible everywhere; each host delivers to its own locally-connected subscribers.
+
+## What changes
+
+Net-new package depending on `@suaveplan/api-types`, `@suaveplan/api-ws` (broker contract), `@suaveplan/api-sse` (also consumes the broker), `@suaveplan/error`, `@suaveplan/logger`, `@suaveplan/opentelemetry`, `ioredis` (peer), and `zod`.
+
+- ADDED: `createRedisWsBroker(opts)` — factory returning a broker value implementing the `WsBroker` contract from `@suaveplan/api-ws`.
+- ADDED: `RedisWsBrokerOptions` — Zod-validated config: `connection` (URL or `ioredis` options), `keyPrefix` (default `suaveplan:ws:`), `channelPrefix` (default `ws.broadcast.`), `eventBufferTtlSec` (default 300 = 5 min — pairs with `api-ws-bun`'s reconnect TTL), `eventBufferMaxEvents` (default 256 per `(tenant, topic)`), `compressionThresholdBytes` (default 1024 — payloads above this are gzipped before publish), `reconnectStrategy`.
+- ADDED: Pub/sub transport — `broker.broadcast(topic, { tenant, payload })` publishes to `<channelPrefix><topic>` on Redis; every subscribed host receives the message and delivers to its locally-connected subscribers (looked up via local `ConnectionRegistry`).
+- ADDED: Reconnect event-buffer storage — for `api-ws-bun` when its `reconnectEventBufferStrategy` resolves to broker-backed, this package owns the event buffer: Redis sorted-set per `(tenant, topic)`, scored by `lastEventId`, capped at `eventBufferMaxEvents`, TTL `eventBufferTtlSec`. SSE consumes the same buffer for `Last-Event-Id` resume.
+- ADDED: Cross-host topology — every host subscribes to the host-wide channel pattern at start; on plugin install/uninstall the broker is informed so per-plugin topic prefixes can be added/dropped from the subscription set if optimisation is wanted. Default is whole-pattern subscription for simplicity.
+- ADDED: Health probe — `broker.healthz()` returns broker's connection state; consumed by the host's readiness endpoint.
+- ADDED: `WS_BROKER_REDIS_*` error codes: `WS_BROKER_REDIS_CONNECTION_LOST`, `WS_BROKER_REDIS_PUBLISH_FAILED`, `WS_BROKER_REDIS_SUBSCRIBE_FAILED`, `WS_BROKER_REDIS_BUFFER_WRITE_FAILED`, `WS_BROKER_REDIS_BUFFER_READ_FAILED`, `WS_BROKER_REDIS_PAYLOAD_TOO_LARGE`, `WS_BROKER_REDIS_DESERIALISE_FAILED`, `WS_BROKER_REDIS_CONFIG_INVALID`, `WS_BROKER_REDIS_RECONNECT_FAILED`, `WS_BROKER_REDIS_TIMEOUT`.
+- ADDED: OTel instruments: `ws.broker.publishes_total { topic, outcome }`, `ws.broker.publish_duration_ms`, `ws.broker.receives_total { topic }`, `ws.broker.buffer_writes_total`, `ws.broker.buffer_reads_total { hit }`, `ws.broker.connection_lost_total`.
+
+## Out of scope
+
+- A non-Redis broker — design doc §11.3 specifically calls Redis as the reference. Other brokers (NATS, Kafka) can implement the same `WsBroker` contract in future packages.
+- The WebSocket engine itself — that's `api-ws-bun`. This broker just makes broadcasts visible across hosts.
+- The SSE engine — `api-sse-bun`. Consumes this same broker for fan-out + event-buffer.
+- Connection migration across hosts — out of scope by design.
+
+## Source mapping
+
+Genesis-original. Source of design intent: `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`.
+
+| Requirement | Design doc section |
+|---|---|
+| Cross-host fan-out via Redis pub/sub | §11.3 |
+| Sticky sessions NOT required | §11.3 |
+| Reconnect event-buffer storage in Redis | §11.5 (paragraph 4) + §14 |
+| SSE reuses same broker | §11.6 |
+| Health probe | §12.6 |
+
+## Verification approach
+
+- `src/broker/redis-broker.test.ts` (integration, 30000ms) — uses `@suaveplan/testing-integration-redis`; covers publish/subscribe round-trip, multi-host fan-out simulation (two broker instances bound to same Redis), reconnect after Redis bounce.
+- `src/broker/redis-broker.spec-conformance.test.ts` — pins the `WsBroker` contract conformance suite from `api-ws/testing`.
+- `src/event-buffer/event-buffer.test.ts` (integration, 30000ms) — sorted-set ops; TTL expiry; cap enforcement; replay by `lastEventId`.
+- `src/health/health.test.ts` — `healthz` reflects connection state; recovers after reconnect.
+- `tests/integration/two-host-broadcast.test.ts` (60000ms) — boots two `api-ws-bun` engines against the same Redis; connects clients to each; broadcasts from one; asserts delivery on both.
+
+## Migration impact
+
+None — additive only.

openspec/archive/2026-05-26-api-ws-broker-redis/specs/api-ws-broker-redis/spec.md

@@ -0,0 +1,131 @@
+<!-- markdownlint-disable MD024 -->
+
+# Capability Spec Delta: api-ws-broker-redis (api-ws-broker-redis)
+
+**Package**: `@suaveplan/api-ws-broker-redis`
+**Status**: PROPOSED
+
+> Every requirement carries a `**Verified by:**` line.
+
+## ADDED Requirements
+
+### Requirement: Implements the WsBroker contract
+
+The package SHALL export `createRedisWsBroker(opts)` producing a broker value implementing the `WsBroker` contract from `@suaveplan/api-ws`. The conformance suite from `@suaveplan/api-ws/testing` MUST pass against this broker.
+
+#### Scenario: conformance suite passes
+
+- **WHEN** the conformance suite runs against `createRedisWsBroker(opts)` backed by a real Redis container
+- **THEN** every invariant passes
+
+**Verified by:** `src/broker/redis-broker.spec-conformance.test.ts::"createRedisWsBroker passes WsBroker conformance against real Redis"`
+
+---
+
+### Requirement: Cross-host broadcast via Redis pub/sub
+
+`broker.broadcast(topic, { tenant, payload })` SHALL publish to `<channelPrefix><topic>` on Redis. Every host subscribed to the channel pattern MUST receive the message and deliver it to its locally-connected subscribers via the local `ConnectionRegistry`. Connections are NOT migrated across hosts — the broker only makes broadcasts visible everywhere, and each host independently delivers to its own subscribers.
+
+#### Scenario: broadcast from host A is delivered to subscribers on host B
+
+- **WHEN** two `api-ws-bun` engines run on different processes pointed at the same Redis AND a client C subscribed on host B AND host A issues `broker.broadcast(topic, { tenant: T, payload: ... })` with C's tenant
+- **THEN** C receives the payload
+
+#### Scenario: sticky sessions are NOT required
+
+- **WHEN** a client connects to host A AND later reconnects and lands on host B
+- **THEN** subscriptions resume on host B from the event buffer with no requirement on routing back to host A
+
+**Verified by:** `tests/integration/two-host-broadcast.test.ts::"broadcast from host A delivered to subscriber on host B"` AND `tests/integration/two-host-broadcast.test.ts::"reconnect lands on different host and resumes from event buffer"`
+
+---
+
+### Requirement: Reconnect event-buffer storage in Redis sorted-sets
+
+The broker SHALL maintain a per-`(tenant, topic)` event buffer in Redis sorted-sets: `<keyPrefix>buffer:<tenant>:<topic>`, scored by `lastEventId`. Each entry SHALL be capped at `eventBufferMaxEvents` (default 256, oldest evicted on overflow) AND TTL'd at `eventBufferTtlSec` (default 300s). On reconnect, `api-ws-bun` calls `broker.readBuffer(tenant, topic, sinceEventId)` to replay events; SSE calls the same surface for `Last-Event-Id` resume.
+
+#### Scenario: buffer cap evicts oldest entries on overflow
+
+- **WHEN** `eventBufferMaxEvents: 4` AND 5 events are written for the same `(tenant, topic)`
+- **THEN** the buffer holds the 4 most recent events AND the oldest was evicted
+
+#### Scenario: TTL expires the buffer
+
+- **WHEN** no writes for `eventBufferTtlSec + 1`
+- **THEN** the buffer key has been removed from Redis
+
+#### Scenario: replay returns events with id > sinceEventId
+
+- **WHEN** the buffer holds events 1..10 AND `readBuffer(tenant, topic, 7)` is called
+- **THEN** events 8, 9, 10 are returned in that order
+
+**Verified by:** `src/event-buffer/event-buffer.test.ts::"buffer cap evicts oldest on overflow"` AND `src/event-buffer/event-buffer.test.ts::"buffer TTL expires keys"` AND `src/event-buffer/event-buffer.test.ts::"readBuffer returns events with id greater than sinceEventId"`
+
+---
+
+### Requirement: Payload compression above a configurable threshold
+
+When a payload exceeds `compressionThresholdBytes` (default 1024), the broker SHALL gzip the payload before publishing. Subscribers SHALL gunzip on receive transparently. A payload that exceeds `4_194_304` bytes (4 MiB) hard cap MUST be refused with `WS_BROKER_REDIS_PAYLOAD_TOO_LARGE` — operators see the misuse instead of silently flooding Redis.
+
+#### Scenario: payload above threshold is compressed
+
+- **WHEN** a 2 KiB payload is broadcast
+- **THEN** the bytes published to Redis are gzip-prefixed AND the subscriber receives the decompressed payload identical to input
+
+#### Scenario: payload above 4 MiB hard cap is refused
+
+- **WHEN** a 5 MiB payload is broadcast
+- **THEN** the call fails with `WS_BROKER_REDIS_PAYLOAD_TOO_LARGE` AND nothing is published
+
+**Verified by:** `src/publish/publish.test.ts::"payload above threshold is gzip-compressed end-to-end"` AND `src/publish/publish.test.ts::"payload above hard cap refused with WS_BROKER_REDIS_PAYLOAD_TOO_LARGE"`
+
+---
+
+### Requirement: Reconnect after Redis bounce
+
+If the underlying Redis connection drops, the broker SHALL attempt auto-reconnect per `reconnectStrategy` (exponential backoff default). During the outage, `healthz()` returns `degraded`; in-flight publishes either queue (for short outages, configurable) or fail with `WS_BROKER_REDIS_CONNECTION_LOST`. On reconnect, pattern subscriptions MUST be re-established AND `ws.broker.connection_lost_total` reflects the outage count.
+
+#### Scenario: short Redis bounce recovers transparently
+
+- **WHEN** Redis is restarted (under 5 seconds) AND publishes continue
+- **THEN** publishes resume after reconnect AND `ws.broker.connection_lost_total` increments by 1
+
+#### Scenario: healthz reflects connection state
+
+- **WHEN** Redis is unreachable
+- **THEN** `broker.healthz()` returns `{ status: "degraded" | "down" }`
+
+**Verified by:** `src/reconnect/reconnect.test.ts::"short Redis bounce recovers transparently and counter increments"` AND `src/health/health.test.ts::"healthz reflects Redis connection state"`
+
+---
+
+### Requirement: Same broker serves SSE and WebSocket
+
+The broker SHALL be consumable by both `api-ws-bun` (WebSocket) and `api-sse-bun` (SSE). Both engines use `broker.broadcast` for fan-out AND `broker.readBuffer` for reconnect resume. The broker MUST NOT assume one transport over the other.
+
+#### Scenario: SSE engine and WS engine share the same broker
+
+- **WHEN** an integration test boots `api-sse-bun` and `api-ws-bun` against the same broker AND a broadcast fires
+- **THEN** both engines deliver to their respective subscribers
+
+**Verified by:** `tests/integration/two-host-broadcast.test.ts::"single broker serves both WS and SSE engines"`
+
+---
+
+### Requirement: OTel surface
+
+The broker SHALL emit:
+
+- `ws.broker.publishes_total` counter `{ topic, outcome }`.
+- `ws.broker.publish_duration_ms` histogram.
+- `ws.broker.receives_total` counter `{ topic }`.
+- `ws.broker.buffer_writes_total` counter.
+- `ws.broker.buffer_reads_total` counter `{ hit }` (`"hit"` or `"miss"` based on whether events were returned).
+- `ws.broker.connection_lost_total` counter.
+
+#### Scenario: successful publish emits counter + histogram
+
+- **WHEN** a publish completes successfully
+- **THEN** `ws.broker.publishes_total{outcome="ok"}` increments AND one observation lands in `ws.broker.publish_duration_ms`
+
+**Verified by:** `src/instruments/instruments.test.ts::"successful publish emits publishes_total + publish_duration_ms"`

openspec/archive/2026-05-26-api-ws-broker-redis/tasks.md

@@ -0,0 +1,72 @@
+# Tasks — `@suaveplan/api-ws-broker-redis`
+
+> Every checkbox MUST reflect work actually done in this change.
+
+## 0. Pre-flight (non-negotiable)
+
+- [x] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
+- [x] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
+- [x] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
+
+## 1. Spec authoring
+
+- [x] 1.1 Read design doc §11.3, §11.5, §11.6, §12.6
+- [x] 1.2 Read umbrella Stage 7 acceptance
+- [x] 1.3 Read sibling `openspec/changes/api-ws-bun/` for reconnect-buffer integration
+- [x] 1.4 Read `openspec/AGENTS.md` Sections 13 and 14
+- [x] 1.5 Draft `proposal.md`
+- [x] 1.6 Draft `specs/api-ws-broker-redis/spec.md`
+- [x] 1.7 Every requirement carries a `**Verified by:**` line
+
+## 2. Implementation
+
+- [x] 2.1 Scaffold `packages/api/api-ws-broker-redis/`
+- [x] 2.2 `src/options/options.ts` — `RedisWsBrokerOptions` Zod schema
+- [x] 2.3 `src/broker/redis-broker.ts` — `createRedisWsBroker(opts)`; implements `WsBroker` contract
+- [x] 2.4 `src/publish/publish.ts` — `broker.broadcast`; gzip above `compressionThresholdBytes`
+- [x] 2.5 `src/subscribe/subscribe.ts` — channel pattern subscription; deliver to local `ConnectionRegistry`
+- [x] 2.6 `src/event-buffer/event-buffer.ts` — Redis sorted-set per `(tenant, topic)`; cap + TTL; replay-by-lastEventId
+- [x] 2.7 `src/health/health.ts` — `broker.healthz()`
+- [x] 2.8 `src/reconnect/reconnect.ts` — auto-reconnect on connection loss; replay buffered messages on resume
+- [x] 2.9 `src/instruments/instruments.ts` — every OTel instrument per proposal
+- [x] 2.10 `src/error-codes.ts` — 10 `WS_BROKER_REDIS_*` codes
+- [x] 2.11 `src/index.ts` — explicit named re-exports
+
+## 3. Tests
+
+- [x] 3.1 Unit tests, every `it(...)` `{ timeout: 5000 }`
+- [x] 3.2 Conformance test against `WsBroker` contract
+- [x] 3.3 Redis integration tests via `@suaveplan/testing-integration-redis` (`{ timeout: 30000 }`)
+- [x] 3.4 Two-host fan-out integration test (`{ timeout: 60000 }`)
+- [x] 3.5 Health-probe test pins recovery after Redis bounce
+- [x] 3.6 All tests import from `@suaveplan/testing/runner`
+
+## 4. Documentation
+
+- [x] 4.1 Sibling `.md` per implementation file
+- [x] 4.2 `README.md` documenting key-prefix conventions, sorted-set schema for the event buffer, and operational notes (Redis memory sizing tied to `eventBufferMaxEvents * eventBufferTtlSec`)
+
+## 5. Pre-archive verification
+
+- [x] 5.1 `bun run check` — 0/0/0
+- [x] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-ws-broker-redis` — green
+- [x] 5.3 `bunx turbo test:coverage --filter=@suaveplan/api-ws-broker-redis` — 100/100/100/100
+- [x] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/api-ws-broker-redis` — exit 0
+- [x] 5.5 `bunx turbo build --filter=@suaveplan/api-ws-broker-redis` — green
+
+## 6. Pre-archive 7-row checklist
+
+- [x] 6.1 Generics on broker payload types; `expectTypeOf` on literal-type claims
+- [x] 6.2 Zod schemas on options + on inbound payloads (deserialise validation)
+- [x] 6.3 Every `WS_BROKER_REDIS_*` code thrown by a production path
+- [x] 6.4 Strict-mode tested (compression on vs off; buffer TTL respected)
+- [x] 6.5 OTel instruments asserted against in-memory `MeterProvider`
+- [x] 6.6 Lifecycle dedup: reconnect does not duplicate buffer writes
+- [x] 6.7 No retroactive tick
+
+## 7. Archive
+
+- [x] 7.1 Fold spec delta into `openspec/specs/api-ws-broker-redis/spec.md`
+- [x] 7.2 Move to `openspec/archive/<yyyy-mm-dd>-api-ws-broker-redis/`
+- [x] 7.3 Tick Stage-7 row in umbrella tasks
+- [x] 7.4 Refresh `MANIFEST.md`