normalizeSource

(no submodules)

Purpose

Converts the four shapes accepted as a packer source — Uint8Array, Blob, ReadableStream<Uint8Array>, AsyncIterable<Uint8Array> — into a single uniform AsyncIterable<Uint8Array> so downstream pipelines (compressor, AEAD encrypt-stream, chunk chopper) can iterate without branching on input type. Buffered sources are sliced to chunkSize boundaries; already-streaming sources preserve their inbound chunk boundaries.

Features

Accepts buffered: Uint8Array, Blob.
Accepts streaming: ReadableStream<Uint8Array>, AsyncIterable<Uint8Array>.
Single unified output: AsyncIterable<Uint8Array>.
Releases ReadableStream reader lock in finally to prevent leaks on early termination.
Rejects unsupported inputs with PackerError(PACKER.INVALID_SOURCE).

Usage: Basic Example

import { normalizeSource } from "@suaveplan/packer/streaming";

const source = new Uint8Array([1, 2, 3, 4, 5]);
for await (const chunk of normalizeSource(source, 2)) {
    console.log(chunk); // Uint8Array(2)[1,2], (2)[3,4], (1)[5]
}

Usage: Advanced Example

import { normalizeSource } from "@suaveplan/packer/streaming";
import { open } from "node:fs/promises";

const fh = await open("/large/file.bin");
const stream = fh.readableWebStream() as ReadableStream<Uint8Array>;

let totalBytes = 0;
for await (const chunk of normalizeSource(stream, 64 * 1024)) {
    totalBytes += chunk.byteLength;
    // pass chunk through next stage of the pipeline
}
await fh.close();

API Reference

normalizeSource(source: PackSource, chunkSize: number): AsyncIterable<Uint8Array>

source — Uint8Array | Blob | ReadableStream<Uint8Array> | AsyncIterable<Uint8Array>. The input bytes.
chunkSize — positive integer. Slice boundary for buffered inputs. Ignored for streaming inputs (their boundaries pass through unchanged).
Returns — AsyncIterable<Uint8Array>.
Throws — PackerError(PACKER.INVALID_SOURCE) if source is none of the supported types.

Implementation Notes

Uint8Array.subarray is used for buffered slicing (zero-copy view, not a copy). Blob paths recurse through Blob.stream(), which itself yields the streaming branch. The Symbol.asyncIterator in source check precedes the getReader branch because modern ReadableStream instances satisfy both protocols and async iteration is the simpler path. The reader-based fallback exists for environments where ReadableStream lacks async-iteration support; the try/finally ensures the lock is released even when the consumer aborts mid-stream.

2.6 KiB Raw Blame History