4.6 KiB
length-prefix-decode
(no submodules)
Purpose
lengthPrefixDecode decodes a length-prefixed binary framing protocol back into
raw payload Uint8Array chunks. It is the inverse of a length-prefix encoder
and is the canonical primitive for parsing wire formats such as TCP message
frames, SLIP-style records, BSON / MessagePack blocks delimited by an explicit
size header, and any custom protocol that prefixes each payload with a
fixed-width little-endian or big-endian length field.
The transform is fully streaming: a single logical frame may span any number of upstream chunks, and an upstream chunk may carry any number of complete frames plus a residual partial frame. A small carry buffer holds bytes between calls so that no frame boundary is ever lost.
Features
- 1, 2, 4, and 8-byte prefix widths.
- Big-endian (default — network order) and little-endian byte orders.
- Configurable
maxFrameBytesguard against memory-exhaustion attacks. - Strict truncation detection: closing the upstream mid-prefix or mid-payload
raises
STREAM.TRUNCATED_FRAMEwith the expected and received byte counts in the error context. - Plugin-OO
LengthPrefixDecodeTransformwrapper with anawait transform.process(input)shorthand for one-shot decode of a singleUint8Arraycontaining one or more concatenated frames. - Zero-copy on the prefix scan; payloads are copied once into stable
Uint8Arrayinstances so consumers may retain references safely.
Usage: Basic Example
import { lengthPrefixDecode } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";
// Wire bytes: 4-byte BE length prefix per frame.
const framed = new Uint8Array([
0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, // frame: [aa bb cc]
0x00, 0x00, 0x00, 0x02, 0xde, 0xad, // frame: [de ad]
]);
const stream = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(framed);
controller.close();
},
});
const reader = stream.pipeThrough(lengthPrefixDecode(4)).getReader();
for (;;) {
const { value, done } = await reader.read();
if (done) break;
console.log(value); // Uint8Array(3) [aa bb cc], Uint8Array(2) [de ad]
}
Usage: Advanced Example
Decoding a little-endian, 2-byte prefix protocol with a 64 KiB cap and a
plugin-OO wrapper so the same transform can be wired into a StreamManager
pipeline or invoked directly:
import { LengthPrefixDecodeTransform } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";
const decoder = new LengthPrefixDecodeTransform(2, {
byteOrder: "le",
maxFrameBytes: 64 * 1024,
});
// One-shot path: a single Uint8Array carrying many concatenated frames.
const payloadsConcat = await decoder.process(framedBytes);
// Streaming path: same options, fresh stream per call.
upstream.pipeThrough(decoder.toTransformStream());
If a frame's prefix value exceeds maxFrameBytes the stream errors with
StreamFrameTooLargeError (STREAM.FRAME_TOO_LARGE) carrying
{ framePrefixValue, maxFrameBytes } in its context. If the upstream closes
mid-frame the stream errors with StreamTruncatedFrameError
(STREAM.TRUNCATED_FRAME) carrying { expectedBytes, receivedBytes }.
API Reference
lengthPrefixDecode(prefixBytes, options?)
prefixBytes: 1 | 2 | 4 | 8— width of the length prefix.options.byteOrder?: "le" | "be"— defaults to"be".options.maxFrameBytes?: number— defaults to1 << 24(16 MiB).- Returns
TransformStream<Uint8Array, Uint8Array>. - Errors the stream with
StreamFrameTooLargeErrororStreamTruncatedFrameError.
class LengthPrefixDecodeTransform extends BaseTransform
constructor(prefixBytes, options?)— same surface as the factory.process(input: Uint8Array): Promise<Uint8Array>— decodes one input buffer containing one or more frames into the concatenated payloads.toTransformStream(): TransformStream<Uint8Array, Uint8Array>— returns the factory form for pipeline use.
Implementation Notes
The carry buffer is allocated freshly on each yielded boundary so subarray
views never pin the entire upstream chunk in memory. The prefix is read with
multiplicative accumulation rather than bit-shifts so that 5–8 byte big-endian
prefixes do not silently wrap to 32-bit on V8. maxFrameBytes is checked
against the decoded prefix value before any payload bytes are buffered, so a
hostile peer cannot exhaust memory by announcing a huge frame they never
deliver. Truncation is detected in the WHATWG flush callback by inspecting
the carry buffer; this preserves the invariant that a clean stream close
always yields zero pending bytes.