loader-stream

Purpose

Progressive entry-by-entry unpack helper that powers PackedResourceLoader.loadStream. Callers pass a list of packed-archive URLs (or StreamLoadEntry records with per-entry options) and consume the returned AsyncIterableIterator<StreamedLoadedResource>. Each entry is dispatched through the standard load pipeline (fetch → maybe-decrypt → verify → unpack) in parallel; the iterator yields each result in completion order so a slow fetch can't block a fast neighbour.

Features

Completion-order yielding — Promise.race over outstanding entries so the consumer's for-await loop wakes up as soon as any entry finishes.
Per-entry options merge — entry.options overrides the iterator-level opts so a single stream can mix unencrypted and encrypted entries, different verifyKeys, or different mime allow-lists.
AbortSignal honoured at every yield. Pre-aborted signal short- circuits the iterator without issuing a fetch; a signal that fires mid-stream throws on the next next() call.
Coexists with the batch load path — both call into the same loader pipeline so behaviour (signature verification, retry policy, OTel metrics) is identical.

Usage: Basic Example

import { createPackedResourceManager } from "@suaveplan/packer-browser";

const manager = createPackedResourceManager();
const urls = ["/a.packed", "/b.packed", "/c.packed"];

for await (const entry of manager.getLoader().loadStream(urls)) {
    console.info("decoded", entry.inputUrl, "→", entry.url);
}

Usage: Advanced Example

import { MemoryKeyring, createPackedResourceManager } from "@suaveplan/packer-browser";

const keyring = new MemoryKeyring({ "tenant-a": new Uint8Array(32).fill(0x11) });
const ac = new AbortController();
const manager = createPackedResourceManager();

const stream = manager.getLoader().loadStream(
    [
        { url: "/asset-a.packed" },
        { url: "/asset-b.packed", options: { keyring } },
        { url: "/asset-c.packed", options: { allowedMimePrefixes: ["image/"] } },
    ],
    { signal: ac.signal }
);

setTimeout(() => ac.abort(), 5000);
for await (const entry of stream) {
    consume(entry);
}

API Reference

loadStream(loader, inputs, opts?) — free function used by PackedResourceLoader.loadStream. Not part of the package's public barrel.
StreamLoadEntry — { url: string; options?: PackedResourceLoadOptions }.
StreamedLoadedResource — LoadedResource & { inputUrl: string }.

Implementation Notes

The Promise.race strategy is bounded — N outstanding fetches at all times — so callers that want a sliding-window concurrency limit should chunk their input list externally. A built-in concurrency cap is intentionally out of scope; the loader's withRetry already smooths transient failures and @suaveplan/streams carries the general windowing primitives for callers that need them.
A pending entry's .promise is created eagerly when loadStream is invoked, not lazily on next(). That matches the audit's "stream sub-resources as bytes arrive" requirement: by starting every fetch immediately, the iterator yields the first completer with the lowest possible latency.
The signal check inside the while loop runs before every await Promise.race(...) so a mid-stream abort terminates cleanly without leaving the iterator suspended on an indefinite race.

3.4 KiB Raw Blame History