first commit
This commit is contained in:
@@ -0,0 +1,115 @@
|
||||
# length-prefix-decode
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`lengthPrefixDecode` decodes a length-prefixed binary framing protocol back into
|
||||
raw payload `Uint8Array` chunks. It is the inverse of a length-prefix encoder
|
||||
and is the canonical primitive for parsing wire formats such as TCP message
|
||||
frames, SLIP-style records, BSON / MessagePack blocks delimited by an explicit
|
||||
size header, and any custom protocol that prefixes each payload with a
|
||||
fixed-width little-endian or big-endian length field.
|
||||
|
||||
The transform is fully streaming: a single logical frame may span any number of
|
||||
upstream chunks, and an upstream chunk may carry any number of complete frames
|
||||
plus a residual partial frame. A small carry buffer holds bytes between calls
|
||||
so that no frame boundary is ever lost.
|
||||
|
||||
## Features
|
||||
|
||||
- 1, 2, 4, and 8-byte prefix widths.
|
||||
- Big-endian (default — network order) and little-endian byte orders.
|
||||
- Configurable `maxFrameBytes` guard against memory-exhaustion attacks.
|
||||
- Strict truncation detection: closing the upstream mid-prefix or mid-payload
|
||||
raises `STREAM.TRUNCATED_FRAME` with the expected and received byte counts in
|
||||
the error context.
|
||||
- Plugin-OO `LengthPrefixDecodeTransform` wrapper with an
|
||||
`await transform.process(input)` shorthand for one-shot decode of a single
|
||||
`Uint8Array` containing one or more concatenated frames.
|
||||
- Zero-copy on the prefix scan; payloads are copied once into stable
|
||||
`Uint8Array` instances so consumers may retain references safely.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { lengthPrefixDecode } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";
|
||||
|
||||
// Wire bytes: 4-byte BE length prefix per frame.
|
||||
const framed = new Uint8Array([
|
||||
0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, // frame: [aa bb cc]
|
||||
0x00, 0x00, 0x00, 0x02, 0xde, 0xad, // frame: [de ad]
|
||||
]);
|
||||
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(framed);
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
|
||||
const reader = stream.pipeThrough(lengthPrefixDecode(4)).getReader();
|
||||
for (;;) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
console.log(value); // Uint8Array(3) [aa bb cc], Uint8Array(2) [de ad]
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Decoding a little-endian, 2-byte prefix protocol with a 64 KiB cap and a
|
||||
plugin-OO wrapper so the same transform can be wired into a `StreamManager`
|
||||
pipeline or invoked directly:
|
||||
|
||||
```ts
|
||||
import { LengthPrefixDecodeTransform } from "@suaveplan/streams/transforms/length-prefix-decode/length-prefix-decode";
|
||||
|
||||
const decoder = new LengthPrefixDecodeTransform(2, {
|
||||
byteOrder: "le",
|
||||
maxFrameBytes: 64 * 1024,
|
||||
});
|
||||
|
||||
// One-shot path: a single Uint8Array carrying many concatenated frames.
|
||||
const payloadsConcat = await decoder.process(framedBytes);
|
||||
|
||||
// Streaming path: same options, fresh stream per call.
|
||||
upstream.pipeThrough(decoder.toTransformStream());
|
||||
```
|
||||
|
||||
If a frame's prefix value exceeds `maxFrameBytes` the stream errors with
|
||||
`StreamFrameTooLargeError` (`STREAM.FRAME_TOO_LARGE`) carrying
|
||||
`{ framePrefixValue, maxFrameBytes }` in its context. If the upstream closes
|
||||
mid-frame the stream errors with `StreamTruncatedFrameError`
|
||||
(`STREAM.TRUNCATED_FRAME`) carrying `{ expectedBytes, receivedBytes }`.
|
||||
|
||||
## API Reference
|
||||
|
||||
### `lengthPrefixDecode(prefixBytes, options?)`
|
||||
|
||||
- `prefixBytes: 1 | 2 | 4 | 8` — width of the length prefix.
|
||||
- `options.byteOrder?: "le" | "be"` — defaults to `"be"`.
|
||||
- `options.maxFrameBytes?: number` — defaults to `1 << 24` (16 MiB).
|
||||
- Returns `TransformStream<Uint8Array, Uint8Array>`.
|
||||
- Errors the stream with `StreamFrameTooLargeError` or `StreamTruncatedFrameError`.
|
||||
|
||||
### `class LengthPrefixDecodeTransform extends BaseTransform`
|
||||
|
||||
- `constructor(prefixBytes, options?)` — same surface as the factory.
|
||||
- `process(input: Uint8Array): Promise<Uint8Array>` — decodes one input buffer
|
||||
containing one or more frames into the concatenated payloads.
|
||||
- `toTransformStream(): TransformStream<Uint8Array, Uint8Array>` — returns the
|
||||
factory form for pipeline use.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The carry buffer is allocated freshly on each yielded boundary so subarray
|
||||
views never pin the entire upstream chunk in memory. The prefix is read with
|
||||
multiplicative accumulation rather than bit-shifts so that 5–8 byte big-endian
|
||||
prefixes do not silently wrap to 32-bit on V8. `maxFrameBytes` is checked
|
||||
against the decoded prefix value before any payload bytes are buffered, so a
|
||||
hostile peer cannot exhaust memory by announcing a huge frame they never
|
||||
deliver. Truncation is detected in the WHATWG `flush` callback by inspecting
|
||||
the carry buffer; this preserves the invariant that a clean stream close
|
||||
always yields zero pending bytes.
|
||||
Reference in New Issue
Block a user