skills/docs/foundation/bitpacker/bitpacker-streaming/src/streaming-bit-packer/streaming-bit-packer.md

StreamingBitPacker

(no submodules)

- streaming-bit-packer.schema — Zod schemas for the o…

• streaming-bit-packer.schema — Zod schemas for the option bundles consumed by StreamingBitPacker and its peer streaming helpers (packStream, `unpa…

• streaming-bit-packer.schema — Zod schemas for the o…

• streaming-bit-packer.schema — Zod schemas for the option bundles consumed by StreamingBitPacker and its peer streaming helpers (packStream, `unpa…

• streaming-bit-packer.schema — Zod schemas for the o…

• streaming-bit-packer.schema — Zod schemas for the option bundles consumed by StreamingBitPacker and its peer streaming helpers (packStream, `unpa…

• streaming-bit-packer.schema — Zod schemas for the o…

• streaming-bit-packer.schema — Zod schemas for the option bundles consumed by StreamingBitPacker and its peer streaming helpers (packStream, `unpa…

• streaming-bit-packer.schema — Zod schemas for the option bundles consumed by StreamingBitPacker and its peer streaming helpers (packStream, `unpa…

Purpose

StreamingBitPacker<T> is the class-shaped façade that lets callers process pack/unpack operations as chunked pipelines instead of single-shot calls. It owns an underlying BitPacker<T> and surfaces five entry points that mirror the upstream bitpacker/src/streaming/streaming.ts API: packStream, unpackStream, createReadableStream, createPackingStream, and createUnpackingStream. Wire-format compatibility is preserved — bytes produced by packStream round-trip through unpackStream exactly as bytes produced by BitPacker.pack round-trip through BitPacker.unpack.

The streaming surface is for callers that need progress callbacks, per-chunk processing (compression, encryption), or backpressure semantics — anything where the single-shot pack/unpack is too coarse. For everything else, prefer the simpler chunk helpers (packToChunks / unpackFromChunks) that already live in this package's index.ts.

Features

• Five entry points covering the chunked-pipeline matrix: synchronous-with-known-input (packStream/unpackStream), readable-side (createReadableStream), and writable-side with in-flight pack/unpack (createPackingStream/createUnpackingStream).
• Optional chunkProcessor runs between bit-packing and emission, so the same façade composes cleanly with downstream compression/encryption stages without callers needing to wire the WHATWG streams API by hand.
• Progress reported via the canonical TransformProgress shape from @suaveplan/types — observability tools speak one vocabulary across every Suaveplan pipeline.
• Injectable Clock for throughput timing; tests run with a FakeClock to avoid real-timer flakiness.
• BitpackerError codes (STREAM_ABORTED, STREAM_CHUNK_PROCESSOR_FAILED, STREAM_INCOMPLETE_DATA) for every failure mode, replacing upstream's plain Error throws.
• Default packing strategies registered automatically (opt-out via registerDefaults: false).
• AbortSignal supported on all *Stream methods — caller can cancel mid-pipeline.

Usage: Basic Example

import { StreamingBitPacker } from "@suaveplan/bitpacker-streaming";

const packer = new StreamingBitPacker({ x: 255, y: 255 });
const packed = await packer.packStream({ x: 42, y: 99 });
const decoded = await packer.unpackStream<{ x: number; y: number }>(packed);
// → { x: 42, y: 99 }

Usage: Advanced Example

import { StreamingBitPacker } from "@suaveplan/bitpacker-streaming";
import type { TransformProgress } from "@suaveplan/types";

const packer = new StreamingBitPacker({
    position: { x: "f32", y: "f32", z: "f32" },
    velocity: { x: "f32", y: "f32", z: "f32" },
});

const progressLog: TransformProgress[] = [];
const onProgress = (p: TransformProgress): void => {
    progressLog.push(p);
};

// chunkProcessor runs between bit-pack and emission — compose with any
// downstream codec without touching the WHATWG streams API directly.
const xor = (chunk: Uint8Array): Uint8Array => {
    const out = new Uint8Array(chunk.length);
    for (let i = 0; i < chunk.length; i++) out[i] = (chunk[i] as number) ^ 0xaa;
    return out;
};

const packed = await packer.packStream(
    {
        position: { x: 1, y: 2, z: 3 },
        velocity: { x: 0.1, y: 0, z: 0 },
    },
    {
        chunkSize: 4,
        chunkProcessor: xor,
        onProgress,
    }
);

// Reverse path with the same processor undoes the XOR before unpack.
const decoded = await packer.unpackStream(packed, {
    chunkSize: 4,
    chunkProcessor: xor,
});

API Reference

new StreamingBitPacker<T>(schema, options?)

schema — a BitPackerSchema describing the data shape. options.registerDefaults? — boolean. Default true. When false, the underlying BitPacker ships with no strategies; the caller must register their own before any pack/unpack call.

packStream(data, options?) → Promise<Uint8Array>

Bit-packs data, walks the bytes in chunkSize-byte chunks (default 1024), runs each chunk through options.chunkProcessor if supplied, and returns the concatenated bytes. onProgress fires once per chunk. Throws BitpackerError(STREAM_ABORTED) if options.signal aborts before or mid-loop.

unpackStream<R>(data, options?) → Promise<R>

Reverse of packStream. Walks data in chunks (running the inverse processor if supplied), concatenates, and bit-unpacks.

createReadableStream(data, options?) → ReadableStream<Uint8Array>

Returns a WHATWG ReadableStream<Uint8Array> that emits data in chunks. Useful for piping into another transform (pipeThrough(compressionStream)). Emits all chunks during start() and immediately closes — back-pressure is delegated to the consumer.

createPackingStream(options?) → { stream, result }

stream is a WritableStream<unknown> that packs each write(item) and pushes the bytes into a buffer. result resolves to the concatenated packed bytes when the stream closes. If chunkProcessor is supplied, each packed chunk is run through it before being buffered. result rejects with BitpackerError(STREAM_ABORTED) on abort().

createUnpackingStream<R>(options?) → { stream, result }

stream is a WritableStream<Uint8Array> that pushes each write(bytes) (optionally through chunkProcessor) into a buffer. On close(), the buffer is concatenated and bit-unpacked. result rejects with BitpackerError(STREAM_INCOMPLETE_DATA) if the stream closes empty, or STREAM_ABORTED on abort().

Inherited

bitPacker — public readonly. The underlying BitPacker<T> instance. Useful for callers wanting to register a custom strategy or schema.

Implementation Notes

The port drops upstream's EventEmitter inheritance because BitPacker does not extend an emitter and the 'progress' event fires the same payload onProgress would have received — the callback is the simpler shape. This also avoids the duplicate-listener footgun upstream callers occasionally hit.

validateChunks from upstream is gone. BitPacker does not expose validate() or getValidationErrors() — those upstream helpers were thin wrappers over schema-introspection logic that suaveplan pushes onto @suaveplan/validation callers. If pre-pack validation is needed, run a Zod schema at the call site before invoking packStream.

The runChunkPipeline helper is shared between packStream and unpackStream because the per-chunk loop logic — chunk slice, optional processor, progress emit — is identical in both directions. Diverging the two paths would duplicate ~30 lines of logic for no semantic gain.

createPackingStream and createUnpackingStream deliberately do not auto-emit on start (unlike createReadableStream) — they are writable sides and emission timing is fully driven by caller writes. The result promise pattern lets callers await the closure without needing a separate completion signal.