first commit
This commit is contained in:
@@ -0,0 +1,88 @@
|
||||
# loader-stream
|
||||
|
||||
## Purpose
|
||||
|
||||
Progressive entry-by-entry unpack helper that powers
|
||||
`PackedResourceLoader.loadStream`. Callers pass a list of packed-archive
|
||||
URLs (or `StreamLoadEntry` records with per-entry options) and consume
|
||||
the returned `AsyncIterableIterator<StreamedLoadedResource>`. Each entry
|
||||
is dispatched through the standard load pipeline (fetch → maybe-decrypt
|
||||
→ verify → unpack) in parallel; the iterator yields each result in
|
||||
**completion order** so a slow fetch can't block a fast neighbour.
|
||||
|
||||
## Features
|
||||
|
||||
- Completion-order yielding — `Promise.race` over outstanding entries
|
||||
so the consumer's `for-await` loop wakes up as soon as any entry
|
||||
finishes.
|
||||
- Per-entry options merge — `entry.options` overrides the iterator-level
|
||||
`opts` so a single stream can mix unencrypted and encrypted entries,
|
||||
different `verifyKey`s, or different mime allow-lists.
|
||||
- `AbortSignal` honoured at every yield. Pre-aborted signal short-
|
||||
circuits the iterator without issuing a fetch; a signal that fires
|
||||
mid-stream throws on the next `next()` call.
|
||||
- Coexists with the batch `load` path — both call into the same loader
|
||||
pipeline so behaviour (signature verification, retry policy, OTel
|
||||
metrics) is identical.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { createPackedResourceManager } from "@suaveplan/packer-browser";
|
||||
|
||||
const manager = createPackedResourceManager();
|
||||
const urls = ["/a.packed", "/b.packed", "/c.packed"];
|
||||
|
||||
for await (const entry of manager.getLoader().loadStream(urls)) {
|
||||
console.info("decoded", entry.inputUrl, "→", entry.url);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { MemoryKeyring, createPackedResourceManager } from "@suaveplan/packer-browser";
|
||||
|
||||
const keyring = new MemoryKeyring({ "tenant-a": new Uint8Array(32).fill(0x11) });
|
||||
const ac = new AbortController();
|
||||
const manager = createPackedResourceManager();
|
||||
|
||||
const stream = manager.getLoader().loadStream(
|
||||
[
|
||||
{ url: "/asset-a.packed" },
|
||||
{ url: "/asset-b.packed", options: { keyring } },
|
||||
{ url: "/asset-c.packed", options: { allowedMimePrefixes: ["image/"] } },
|
||||
],
|
||||
{ signal: ac.signal }
|
||||
);
|
||||
|
||||
setTimeout(() => ac.abort(), 5000);
|
||||
for await (const entry of stream) {
|
||||
consume(entry);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `loadStream(loader, inputs, opts?)` — free function used by
|
||||
`PackedResourceLoader.loadStream`. Not part of the package's public
|
||||
barrel.
|
||||
- `StreamLoadEntry` — `{ url: string; options?: PackedResourceLoadOptions }`.
|
||||
- `StreamedLoadedResource` — `LoadedResource & { inputUrl: string }`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The `Promise.race` strategy is bounded — N outstanding fetches at all
|
||||
times — so callers that want a sliding-window concurrency limit
|
||||
should chunk their input list externally. A built-in concurrency
|
||||
cap is intentionally out of scope; the loader's `withRetry` already
|
||||
smooths transient failures and `@suaveplan/streams` carries the
|
||||
general windowing primitives for callers that need them.
|
||||
- A pending entry's `.promise` is created eagerly when `loadStream` is
|
||||
invoked, not lazily on `next()`. That matches the audit's "stream
|
||||
sub-resources as bytes arrive" requirement: by starting every fetch
|
||||
immediately, the iterator yields the first completer with the lowest
|
||||
possible latency.
|
||||
- The signal check inside the `while` loop runs before every `await
|
||||
Promise.race(...)` so a mid-stream abort terminates cleanly without
|
||||
leaving the iterator suspended on an indefinite race.
|
||||
Reference in New Issue
Block a user