first commit
This commit is contained in:
@@ -0,0 +1,99 @@
|
||||
# `event-buffer` — Redis sorted-set reconnect buffer
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
This module backs the broker's reconnect event buffer. Each
|
||||
`(tenant, topic)` pair maps to one Redis sorted-set keyed by
|
||||
`<keyPrefix>buffer:<tenant>:<topic>` and scored by `eventId`. The
|
||||
`api-ws-bun` and `api-sse-bun` engines call into this surface to replay
|
||||
events to clients that reconnected with a `Last-Event-Id` (WS) or
|
||||
`Last-Event-Id` (SSE) header.
|
||||
|
||||
The buffer enforces two operational guarantees: a fixed cap
|
||||
(`eventBufferMaxEvents`) so a per-tenant runaway publisher cannot exhaust
|
||||
Redis memory, and a sliding TTL (`eventBufferTtlSec`) so abandoned topics
|
||||
expire without operator intervention.
|
||||
|
||||
## Features
|
||||
|
||||
- Sorted-set encoding `<eventId>|<base64-payload>` so the integer score
|
||||
remains usable for `ZRANGEBYSCORE` queries and the binary payload survives
|
||||
the round-trip without coercion.
|
||||
- Cap enforcement via `ZCARD` + `ZREMRANGEBYRANK 0 (overflow - 1)` —
|
||||
oldest entries evicted first.
|
||||
- Per-write TTL refresh via `EXPIRE` so the key always carries the latest
|
||||
TTL window.
|
||||
- `readBuffered Events(...)` returns events strictly greater than
|
||||
`sinceEventId` (`(<sinceEventId>` syntax) in ascending score order.
|
||||
- OTel hit/miss labelling on every read via the `ws.broker.buffer_reads_total`
|
||||
counter.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
readBufferedEvents,
|
||||
writeBufferedEvent,
|
||||
} from "@suaveplan/api-ws-broker-redis/event-buffer";
|
||||
|
||||
const opts = { keyPrefix: "sp:", maxEvents: 256, ttlSec: 300 };
|
||||
await writeBufferedEvent(client, tenant, "orders", {
|
||||
eventId: 1,
|
||||
payload: new Uint8Array([1, 2, 3]),
|
||||
}, opts);
|
||||
const events = await readBufferedEvents(client, tenant, "orders", 0, opts);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
bufferKey,
|
||||
decodeBufferMember,
|
||||
} from "@suaveplan/api-ws-broker-redis/event-buffer";
|
||||
|
||||
// Drop into raw Redis to debug a stuck buffer — `decodeBufferMember` parses
|
||||
// each `<eventId>|<base64>` entry back into its `BufferedEvent`.
|
||||
const key = bufferKey("sp:", tenant, "orders");
|
||||
const range = await rawRedis.zrangebyscore(key, "-inf", "+inf");
|
||||
for (const m of range) {
|
||||
const event = decodeBufferMember(m);
|
||||
console.log(event.eventId, event.payload.length);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `writeBufferedEvent(client, tenant, topic, event, opts): Promise<void>` —
|
||||
append, enforce cap, refresh TTL.
|
||||
- `readBufferedEvents(client, tenant, topic, sinceEventId, opts):
|
||||
Promise<readonly BufferedEvent[]>` — replay events with `id > sinceEventId`.
|
||||
- `bufferKey(keyPrefix, tenant, topic): string` — key construction helper.
|
||||
- `encodeBufferMember(event)` / `decodeBufferMember(s)` — sorted-set
|
||||
member encoding/decoding (exposed for raw-Redis debugging).
|
||||
- `BufferedEvent { eventId, payload }` and `EventBufferOptions
|
||||
{ keyPrefix, maxEvents, ttlSec }`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The sorted-set strategy was chosen over a Redis Stream because the broker
|
||||
only needs id-keyed range reads (no consumer-group coordination), and a
|
||||
sorted-set's eviction and cap operations are atomic enough for the
|
||||
single-tenant single-topic write path the broker uses. Streams' richer
|
||||
semantics (XCLAIM, XACK) would only add operational complexity without
|
||||
helping the resume use case.
|
||||
|
||||
The base64 step inside `encodeBufferMember` is intentional: sorted-set
|
||||
members are byte strings, but the broker payload is an arbitrary
|
||||
`Uint8Array` that may contain `\r\n` or `\0` bytes. Base64 round-trips
|
||||
through Redis's binary-safe wire protocol without coercion and the cost is
|
||||
33% bigger members, which is well below the network overhead of the
|
||||
publish itself.
|
||||
|
||||
`writeBufferedEvent` re-throws an existing `WsBrokerRedisError` unmodified
|
||||
rather than double-wrapping it; tests cover the no-double-wrap path
|
||||
explicitly.
|
||||
Reference in New Issue
Block a user