`event-buffer` — Redis sorted-set reconnect buffer

(no submodules)

Purpose

This module backs the broker's reconnect event buffer. Each (tenant, topic) pair maps to one Redis sorted-set keyed by <keyPrefix>buffer:<tenant>:<topic> and scored by eventId. The api-ws-bun and api-sse-bun engines call into this surface to replay events to clients that reconnected with a Last-Event-Id (WS) or Last-Event-Id (SSE) header.

The buffer enforces two operational guarantees: a fixed cap (eventBufferMaxEvents) so a per-tenant runaway publisher cannot exhaust Redis memory, and a sliding TTL (eventBufferTtlSec) so abandoned topics expire without operator intervention.

Features

Sorted-set encoding <eventId>|<base64-payload> so the integer score remains usable for ZRANGEBYSCORE queries and the binary payload survives the round-trip without coercion.
Cap enforcement via ZCARD + ZREMRANGEBYRANK 0 (overflow - 1) — oldest entries evicted first.
Per-write TTL refresh via EXPIRE so the key always carries the latest TTL window.
readBuffered Events(...) returns events strictly greater than sinceEventId ((<sinceEventId> syntax) in ascending score order.
OTel hit/miss labelling on every read via the ws.broker.buffer_reads_total counter.

Usage: Basic

import {
    readBufferedEvents,
    writeBufferedEvent,
} from "@suaveplan/api-ws-broker-redis/event-buffer";

const opts = { keyPrefix: "sp:", maxEvents: 256, ttlSec: 300 };
await writeBufferedEvent(client, tenant, "orders", {
    eventId: 1,
    payload: new Uint8Array([1, 2, 3]),
}, opts);
const events = await readBufferedEvents(client, tenant, "orders", 0, opts);

Usage: Advanced

import {
    bufferKey,
    decodeBufferMember,
} from "@suaveplan/api-ws-broker-redis/event-buffer";

// Drop into raw Redis to debug a stuck buffer — `decodeBufferMember` parses
// each `<eventId>|<base64>` entry back into its `BufferedEvent`.
const key = bufferKey("sp:", tenant, "orders");
const range = await rawRedis.zrangebyscore(key, "-inf", "+inf");
for (const m of range) {
    const event = decodeBufferMember(m);
    console.log(event.eventId, event.payload.length);
}

API Reference

writeBufferedEvent(client, tenant, topic, event, opts): Promise<void> — append, enforce cap, refresh TTL.
readBufferedEvents(client, tenant, topic, sinceEventId, opts): Promise<readonly BufferedEvent[]> — replay events with id > sinceEventId.
bufferKey(keyPrefix, tenant, topic): string — key construction helper.
encodeBufferMember(event) / decodeBufferMember(s) — sorted-set member encoding/decoding (exposed for raw-Redis debugging).
BufferedEvent { eventId, payload } and EventBufferOptions { keyPrefix, maxEvents, ttlSec }.

Implementation Notes

The sorted-set strategy was chosen over a Redis Stream because the broker only needs id-keyed range reads (no consumer-group coordination), and a sorted-set's eviction and cap operations are atomic enough for the single-tenant single-topic write path the broker uses. Streams' richer semantics (XCLAIM, XACK) would only add operational complexity without helping the resume use case.

The base64 step inside encodeBufferMember is intentional: sorted-set members are byte strings, but the broker payload is an arbitrary Uint8Array that may contain \r\n or \0 bytes. Base64 round-trips through Redis's binary-safe wire protocol without coercion and the cost is 33% bigger members, which is well below the network overhead of the publish itself.

writeBufferedEvent re-throws an existing WsBrokerRedisError unmodified rather than double-wrapping it; tests cover the no-double-wrap path explicitly.

3.7 KiB Raw Blame History

event-buffer — Redis sorted-set reconnect buffer