Files
skills/docs/foundation/flow/streams/src/transforms/length-prefix-decode/length-prefix-decode.md
2026-05-29 08:20:56 +01:00

4.6 KiB
Raw Blame History

length-prefix-decode

(no submodules)

Purpose

lengthPrefixDecode decodes a length-prefixed binary framing protocol back into raw payload Uint8Array chunks. It is the inverse of a length-prefix encoder and is the canonical primitive for parsing wire formats such as TCP message frames, SLIP-style records, BSON / MessagePack blocks delimited by an explicit size header, and any custom protocol that prefixes each payload with a fixed-width little-endian or big-endian length field.

The transform is fully streaming: a single logical frame may span any number of upstream chunks, and an upstream chunk may carry any number of complete frames plus a residual partial frame. A small carry buffer holds bytes between calls so that no frame boundary is ever lost.

Features

  • 1, 2, 4, and 8-byte prefix widths.
  • Big-endian (default — network order) and little-endian byte orders.
  • Configurable maxFrameBytes guard against memory-exhaustion attacks.
  • Strict truncation detection: closing the upstream mid-prefix or mid-payload raises STREAM.TRUNCATED_FRAME with the expected and received byte counts in the error context.
  • Plugin-OO LengthPrefixDecodeTransform wrapper with an await transform.process(input) shorthand for one-shot decode of a single Uint8Array containing one or more concatenated frames.
  • Zero-copy on the prefix scan; payloads are copied once into stable Uint8Array instances so consumers may retain references safely.

Usage: Basic Example

import { lengthPrefixDecode } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";

// Wire bytes: 4-byte BE length prefix per frame.
const framed = new Uint8Array([
    0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, // frame: [aa bb cc]
    0x00, 0x00, 0x00, 0x02, 0xde, 0xad,       // frame: [de ad]
]);

const stream = new ReadableStream<Uint8Array>({
    start(controller) {
        controller.enqueue(framed);
        controller.close();
    },
});

const reader = stream.pipeThrough(lengthPrefixDecode(4)).getReader();
for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    console.log(value); // Uint8Array(3) [aa bb cc], Uint8Array(2) [de ad]
}

Usage: Advanced Example

Decoding a little-endian, 2-byte prefix protocol with a 64 KiB cap and a plugin-OO wrapper so the same transform can be wired into a StreamManager pipeline or invoked directly:

import { LengthPrefixDecodeTransform } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";

const decoder = new LengthPrefixDecodeTransform(2, {
    byteOrder: "le",
    maxFrameBytes: 64 * 1024,
});

// One-shot path: a single Uint8Array carrying many concatenated frames.
const payloadsConcat = await decoder.process(framedBytes);

// Streaming path: same options, fresh stream per call.
upstream.pipeThrough(decoder.toTransformStream());

If a frame's prefix value exceeds maxFrameBytes the stream errors with StreamFrameTooLargeError (STREAM.FRAME_TOO_LARGE) carrying { framePrefixValue, maxFrameBytes } in its context. If the upstream closes mid-frame the stream errors with StreamTruncatedFrameError (STREAM.TRUNCATED_FRAME) carrying { expectedBytes, receivedBytes }.

API Reference

lengthPrefixDecode(prefixBytes, options?)

  • prefixBytes: 1 | 2 | 4 | 8 — width of the length prefix.
  • options.byteOrder?: "le" | "be" — defaults to "be".
  • options.maxFrameBytes?: number — defaults to 1 << 24 (16 MiB).
  • Returns TransformStream<Uint8Array, Uint8Array>.
  • Errors the stream with StreamFrameTooLargeError or StreamTruncatedFrameError.

class LengthPrefixDecodeTransform extends BaseTransform

  • constructor(prefixBytes, options?) — same surface as the factory.
  • process(input: Uint8Array): Promise<Uint8Array> — decodes one input buffer containing one or more frames into the concatenated payloads.
  • toTransformStream(): TransformStream<Uint8Array, Uint8Array> — returns the factory form for pipeline use.

Implementation Notes

The carry buffer is allocated freshly on each yielded boundary so subarray views never pin the entire upstream chunk in memory. The prefix is read with multiplicative accumulation rather than bit-shifts so that 5–8 byte big-endian prefixes do not silently wrap to 32-bit on V8. maxFrameBytes is checked against the decoded prefix value before any payload bytes are buffered, so a hostile peer cannot exhaust memory by announcing a huge frame they never deliver. Truncation is detected in the WHATWG flush callback by inspecting the carry buffer; this preserves the invariant that a clean stream close always yields zero pending bytes.