first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,99 @@
# `event-buffer` — Redis sorted-set reconnect buffer
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
This module backs the broker's reconnect event buffer. Each
`(tenant, topic)` pair maps to one Redis sorted-set keyed by
`<keyPrefix>buffer:<tenant>:<topic>` and scored by `eventId`. The
`api-ws-bun` and `api-sse-bun` engines call into this surface to replay
events to clients that reconnected with a `Last-Event-Id` (WS) or
`Last-Event-Id` (SSE) header.
The buffer enforces two operational guarantees: a fixed cap
(`eventBufferMaxEvents`) so a per-tenant runaway publisher cannot exhaust
Redis memory, and a sliding TTL (`eventBufferTtlSec`) so abandoned topics
expire without operator intervention.
## Features
- Sorted-set encoding `<eventId>|<base64-payload>` so the integer score
remains usable for `ZRANGEBYSCORE` queries and the binary payload survives
the round-trip without coercion.
- Cap enforcement via `ZCARD` + `ZREMRANGEBYRANK 0 (overflow - 1)` —
oldest entries evicted first.
- Per-write TTL refresh via `EXPIRE` so the key always carries the latest
TTL window.
- `readBuffered Events(...)` returns events strictly greater than
`sinceEventId` (`(<sinceEventId>` syntax) in ascending score order.
- OTel hit/miss labelling on every read via the `ws.broker.buffer_reads_total`
counter.
## Usage: Basic
```ts
import {
readBufferedEvents,
writeBufferedEvent,
} from "@suaveplan/api-ws-broker-redis/event-buffer";
const opts = { keyPrefix: "sp:", maxEvents: 256, ttlSec: 300 };
await writeBufferedEvent(client, tenant, "orders", {
eventId: 1,
payload: new Uint8Array([1, 2, 3]),
}, opts);
const events = await readBufferedEvents(client, tenant, "orders", 0, opts);
```
## Usage: Advanced
```ts
import {
bufferKey,
decodeBufferMember,
} from "@suaveplan/api-ws-broker-redis/event-buffer";
// Drop into raw Redis to debug a stuck buffer — `decodeBufferMember` parses
// each `<eventId>|<base64>` entry back into its `BufferedEvent`.
const key = bufferKey("sp:", tenant, "orders");
const range = await rawRedis.zrangebyscore(key, "-inf", "+inf");
for (const m of range) {
const event = decodeBufferMember(m);
console.log(event.eventId, event.payload.length);
}
```
## API Reference
- `writeBufferedEvent(client, tenant, topic, event, opts): Promise<void>` —
append, enforce cap, refresh TTL.
- `readBufferedEvents(client, tenant, topic, sinceEventId, opts):
Promise<readonly BufferedEvent[]>` — replay events with `id > sinceEventId`.
- `bufferKey(keyPrefix, tenant, topic): string` — key construction helper.
- `encodeBufferMember(event)` / `decodeBufferMember(s)` — sorted-set
member encoding/decoding (exposed for raw-Redis debugging).
- `BufferedEvent { eventId, payload }` and `EventBufferOptions
{ keyPrefix, maxEvents, ttlSec }`.
## Implementation Notes
The sorted-set strategy was chosen over a Redis Stream because the broker
only needs id-keyed range reads (no consumer-group coordination), and a
sorted-set's eviction and cap operations are atomic enough for the
single-tenant single-topic write path the broker uses. Streams' richer
semantics (XCLAIM, XACK) would only add operational complexity without
helping the resume use case.
The base64 step inside `encodeBufferMember` is intentional: sorted-set
members are byte strings, but the broker payload is an arbitrary
`Uint8Array` that may contain `\r\n` or `\0` bytes. Base64 round-trips
through Redis's binary-safe wire protocol without coercion and the cost is
33% bigger members, which is well below the network overhead of the
publish itself.
`writeBufferedEvent` re-throws an existing `WsBrokerRedisError` unmodified
rather than double-wrapping it; tests cover the no-double-wrap path
explicitly.