first commit
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
# loader-helpers
|
||||
|
||||
## Purpose
|
||||
|
||||
Small free helpers extracted from `loader.ts` so the class stays under
|
||||
the 300-line hard cap. These functions carry no state and are exercised
|
||||
indirectly through the loader's tests (every helper has at least one
|
||||
loader-level scenario that pins its behaviour) plus the
|
||||
`encrypted-archive` and `range-response` tests for their wire formats.
|
||||
|
||||
## Features
|
||||
|
||||
- `ARCHIVE_SIGNATURE_HEADER` — the canonical lowercase header name
|
||||
(`x-archive-signature`) used for Ed25519 signature transport.
|
||||
- `NO_RETRY` — the default single-attempt retry config used by the
|
||||
loader when the caller does not supply one.
|
||||
- `defaultFetch(input, init?)` — `globalThis.fetch` wrapper that's
|
||||
injectable through `PackedResourceLoaderOptions.fetch`.
|
||||
- `hexToBytes(hex)` — decode an `0x`-prefixable hex string into a
|
||||
`Uint8Array`. Used for the signature header.
|
||||
- `maybeDecryptArchive(bytes, keyring, signal, resolver)` — branch on
|
||||
the `SP-ENC1` magic prefix: pass-through unencrypted bytes; decrypt
|
||||
through the supplied keyring otherwise. Throws `KEY_UNAVAILABLE` when
|
||||
a keyring is missing for an encrypted body.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { hexToBytes, defaultFetch } from "./loader-helpers.js";
|
||||
|
||||
const sig = hexToBytes("0xab12cd34");
|
||||
const res = await defaultFetch("/asset.packed");
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maybeDecryptArchive } from "./loader-helpers.js";
|
||||
import { MemoryKeyring } from "../keyring/keyring.js";
|
||||
|
||||
const keyring = new MemoryKeyring({
|
||||
"tenant-a": new Uint8Array(32).fill(7),
|
||||
});
|
||||
|
||||
// Inside a loader pipeline, swap in the helper between fetch + unpack.
|
||||
const fetched = new Uint8Array(await response.arrayBuffer());
|
||||
const plain = await maybeDecryptArchive(
|
||||
fetched,
|
||||
keyring,
|
||||
/* signal */ undefined,
|
||||
/* resolver */ undefined
|
||||
);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `ARCHIVE_SIGNATURE_HEADER: "x-archive-signature"`.
|
||||
- `NO_RETRY: { maxAttempts: 1; delay: () => 0 }`.
|
||||
- `__testing_noDelay` — re-exported by `loader.ts` for backwards
|
||||
compatibility with tests that import the symbol from the class
|
||||
module.
|
||||
- `defaultFetch(input: string, init?: RequestInit): Promise<Response>`.
|
||||
- `hexToBytes(hex: string): Uint8Array` — accepts an optional `0x`
|
||||
prefix; otherwise expects a sequence of `<hex><hex>` byte digits.
|
||||
- `maybeDecryptArchive(bytes, keyring, signal, resolver): Promise<Uint8Array>` —
|
||||
decrypt or pass-through; throws `PackerBrowserError(KEY_UNAVAILABLE)`
|
||||
when an encrypted body is received with no keyring.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- These helpers are factored out solely to honour CLAUDE.md §3's 300-LOC
|
||||
hard cap on `loader.ts`; they are not part of the package's public
|
||||
surface (the loader's tests exercise them indirectly through the
|
||||
loader, and the `loader/index.ts` barrel does not re-export them).
|
||||
- `maybeDecryptArchive` deliberately returns the original `Uint8Array`
|
||||
reference (not a copy) on the pass-through path so the loader does
|
||||
not allocate twice for the common unencrypted case.
|
||||
- `hexToBytes` does not validate input length parity; the loader feeds
|
||||
it only the 128-character signature header (64-byte signature) so
|
||||
the contract is constrained at the call site.
|
||||
@@ -0,0 +1,88 @@
|
||||
# loader-stream
|
||||
|
||||
## Purpose
|
||||
|
||||
Progressive entry-by-entry unpack helper that powers
|
||||
`PackedResourceLoader.loadStream`. Callers pass a list of packed-archive
|
||||
URLs (or `StreamLoadEntry` records with per-entry options) and consume
|
||||
the returned `AsyncIterableIterator<StreamedLoadedResource>`. Each entry
|
||||
is dispatched through the standard load pipeline (fetch → maybe-decrypt
|
||||
→ verify → unpack) in parallel; the iterator yields each result in
|
||||
**completion order** so a slow fetch can't block a fast neighbour.
|
||||
|
||||
## Features
|
||||
|
||||
- Completion-order yielding — `Promise.race` over outstanding entries
|
||||
so the consumer's `for-await` loop wakes up as soon as any entry
|
||||
finishes.
|
||||
- Per-entry options merge — `entry.options` overrides the iterator-level
|
||||
`opts` so a single stream can mix unencrypted and encrypted entries,
|
||||
different `verifyKey`s, or different mime allow-lists.
|
||||
- `AbortSignal` honoured at every yield. Pre-aborted signal short-
|
||||
circuits the iterator without issuing a fetch; a signal that fires
|
||||
mid-stream throws on the next `next()` call.
|
||||
- Coexists with the batch `load` path — both call into the same loader
|
||||
pipeline so behaviour (signature verification, retry policy, OTel
|
||||
metrics) is identical.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { createPackedResourceManager } from "@suaveplan/packer-browser";
|
||||
|
||||
const manager = createPackedResourceManager();
|
||||
const urls = ["/a.packed", "/b.packed", "/c.packed"];
|
||||
|
||||
for await (const entry of manager.getLoader().loadStream(urls)) {
|
||||
console.info("decoded", entry.inputUrl, "→", entry.url);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { MemoryKeyring, createPackedResourceManager } from "@suaveplan/packer-browser";
|
||||
|
||||
const keyring = new MemoryKeyring({ "tenant-a": new Uint8Array(32).fill(0x11) });
|
||||
const ac = new AbortController();
|
||||
const manager = createPackedResourceManager();
|
||||
|
||||
const stream = manager.getLoader().loadStream(
|
||||
[
|
||||
{ url: "/asset-a.packed" },
|
||||
{ url: "/asset-b.packed", options: { keyring } },
|
||||
{ url: "/asset-c.packed", options: { allowedMimePrefixes: ["image/"] } },
|
||||
],
|
||||
{ signal: ac.signal }
|
||||
);
|
||||
|
||||
setTimeout(() => ac.abort(), 5000);
|
||||
for await (const entry of stream) {
|
||||
consume(entry);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `loadStream(loader, inputs, opts?)` — free function used by
|
||||
`PackedResourceLoader.loadStream`. Not part of the package's public
|
||||
barrel.
|
||||
- `StreamLoadEntry` — `{ url: string; options?: PackedResourceLoadOptions }`.
|
||||
- `StreamedLoadedResource` — `LoadedResource & { inputUrl: string }`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The `Promise.race` strategy is bounded — N outstanding fetches at all
|
||||
times — so callers that want a sliding-window concurrency limit
|
||||
should chunk their input list externally. A built-in concurrency
|
||||
cap is intentionally out of scope; the loader's `withRetry` already
|
||||
smooths transient failures and `@suaveplan/streams` carries the
|
||||
general windowing primitives for callers that need them.
|
||||
- A pending entry's `.promise` is created eagerly when `loadStream` is
|
||||
invoked, not lazily on `next()`. That matches the audit's "stream
|
||||
sub-resources as bytes arrive" requirement: by starting every fetch
|
||||
immediately, the iterator yields the first completer with the lowest
|
||||
possible latency.
|
||||
- The signal check inside the `while` loop runs before every `await
|
||||
Promise.race(...)` so a mid-stream abort terminates cleanly without
|
||||
leaving the iterator suspended on an indefinite race.
|
||||
102
docs/foundation/data/packer-browser/src/loader/loader.md
Normal file
102
docs/foundation/data/packer-browser/src/loader/loader.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# `loader`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`loader.ts` is the fetch → verify → unpack → Blob-URL pipeline for
|
||||
packed resources. It is the only place in the package that issues a
|
||||
network request, the only place that runs `Unpacker.unpack`, and the
|
||||
only place that allocates a Blob URL. Splitting it out from
|
||||
`PackedResourceManager` keeps every concern testable in isolation —
|
||||
fetch retry, signature failure, abort propagation, and OTel histogram
|
||||
emission can each be exercised without instantiating a cache.
|
||||
|
||||
## Features
|
||||
|
||||
- `fetch` routed through `@suaveplan/retry`'s `withRetry` with a
|
||||
single-attempt default policy; callers opt in to retry by passing
|
||||
`retryConfig`.
|
||||
- `AbortSignal` honoured at every yield point — pre-fetch, post-fetch,
|
||||
post-buffer, post-verify, and pre-unpack.
|
||||
- Optional Ed25519 signature verification when `verifyKey` is supplied
|
||||
and the response carries an `X-Archive-Signature` header.
|
||||
- OTel emission: increments `packed_resource_fetch_total` on every
|
||||
attempt; records `packed_resource_decode_ms` on every successful
|
||||
pipeline run.
|
||||
- Typed error surface: every failure produces `PackerBrowserError`
|
||||
with a `PACKER_BROWSER.*` code and an i18n-resolved message.
|
||||
- Injectable `fetch` for tests — no `vi.mock` required.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
createPackerBrowserObservability,
|
||||
PackedResourceLoader,
|
||||
resolveRuntime,
|
||||
} from "@suaveplan/packer-browser";
|
||||
|
||||
const loader = new PackedResourceLoader({
|
||||
runtime: resolveRuntime(),
|
||||
observability: createPackerBrowserObservability(),
|
||||
});
|
||||
|
||||
const { url, metadata } = await loader.load("/banner.packed");
|
||||
img.src = url;
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { createConstantBackoff } from "@suaveplan/retry";
|
||||
import { PackedResourceLoader } from "@suaveplan/packer-browser";
|
||||
|
||||
const controller = new AbortController();
|
||||
const loader = new PackedResourceLoader({
|
||||
runtime: resolveRuntime(),
|
||||
observability,
|
||||
fetch: async (url) => recordedFetch(url),
|
||||
});
|
||||
|
||||
const out = await loader.load("/banner.packed", {
|
||||
verifyKey: trustAnchorPublicKey,
|
||||
signal: controller.signal,
|
||||
retryConfig: {
|
||||
maxAttempts: 3,
|
||||
delay: createConstantBackoff({ delay: 250 }),
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `new PackedResourceLoader(opts: PackedResourceLoaderOptions)`.
|
||||
- `loader.load(url: string, opts?: PackedResourceLoadOptions):
|
||||
Promise<LoadedResource>`.
|
||||
- Throws `RESOURCE_NOT_FOUND` for non-2xx.
|
||||
- Throws `ARCHIVE_SIGNATURE_INVALID` for a non-verifying signature
|
||||
when `verifyKey` is supplied.
|
||||
- Throws `LOAD_FAILED` when `Unpacker.unpack` rejects.
|
||||
- Throws `ABORTED` when the supplied signal fires.
|
||||
- `PackedResourceLoadOptions`:
|
||||
- `verifyChecksum?: boolean` — default `false`.
|
||||
- `verifyKey?: Uint8Array` — optional Ed25519 public key.
|
||||
- `signal?: AbortSignal` — checked at every yield.
|
||||
- `retryConfig?: RetryConfig` — default: single attempt.
|
||||
- `messageResolver?: MessageResolver` — overrides the constructor's
|
||||
resolver for this call.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Cluster A's loader treats every fetched payload as a raw `PackedData`
|
||||
whose metadata is reconstructed from the bytes; signature verification
|
||||
covers the **archive transport** layer, while integrity verification
|
||||
is delegated to the `Unpacker` and currently skipped
|
||||
(`verifyIntegrity: false`) because Cluster A archives do not carry the
|
||||
BLAKE3 integrity bundle. Cluster B introduces the formal manifest
|
||||
decoder and re-enables integrity. The `X-Archive-Signature` header
|
||||
contract is unchanged across clusters — switching to a separate
|
||||
signature manifest is a Cluster B concern only.
|
||||
Reference in New Issue
Block a user