Initial Commit

openspec/archive/2026-05-19-enterprise-packer-resilience/proposal.md

@@ -0,0 +1,144 @@
+# Enterprise hardening — `@suaveplan/packer` resilience cluster
+
+**Package**: `@suaveplan/packer`
+**Status**: PROPOSED
+**Type**: feature
+**Design**: (inline below — small scope)
+**Spec delta**: ./specs/packer/spec.md
+
+## Why
+
+Cluster #1 (`enterprise-packer-integrity`, archived 2026-05-19) landed BLAKE3
+content-addressable digests, the bundle-level Merkle root, mandatory
+integrity verification on unpack, full OpenTelemetry instrumentation, and
+i18n-keyed error messages. The `enterprise/packer.md` audit (L43, L46–L47)
+calls out three remaining resilience gaps that block the ENTERPRISE label:
+
+1. No retry primitives on IO-backed sources (`ReadableStream.read()` calls
+   throw straight up the stack on transient errors).
+2. No idempotency layer — repeat packs over identical inputs re-run the full
+   gzip + AEAD + ed25519 pipeline every time.
+3. The whole-file `pack` / `unpack` paths do not accept `AbortSignal`; only
+   the chunked path honours cancellation today.
+4. `Date.now()` and `crypto.getRandomValues()` are called at production code
+   paths in `src/`, violating the audit's "no `Math.random()` / `Date.now()`
+   at module scope; injectable clock and RNG" line.
+
+This is cluster #2 of a 3-cluster enterprise hardening of `@suaveplan/packer`
+(integrity → resilience → extensibility). Cluster #3 is deliberately out of
+scope here — see "Out of scope" below.
+
+## What changes
+
+- ADDED: `src/retry/retryable-read.ts` — `retryRead(reader, opts)` helper that
+  wraps `ReadableStream` reads in `@suaveplan/retry`'s `withRetry`, honours
+  `AbortSignal`, and returns the same `{ value, done }` shape. Defaults:
+  `createExponentialBackoff({ initial: 25, base: 2, max: 1000, jitter: "equal" })`
+  + `maxAttempts: 4`.
+- ADDED: `RetryPolicy` option bundle on `PackerOptions`, `PackChunkedOptions`,
+  and `UnpackChunkedOptions` (`retryConfig?`, `signal?`). Threaded through to
+  `normalizeSource` and the abstract `collect` path so every `reader.read()`
+  in production code is retry-aware.
+- ADDED: `src/idempotency/idempotency.ts` — `PackIdempotencyGuard` that
+  computes a content-derived key (`blake3(payload) || compressionLevel ||
+  encryptionAlgo || signed`-bit) and short-circuits to a cached `PackedData`
+  when the same input is packed again. Backed by an injectable
+  `IdempotencyStoreAdapter` (default `MemoryIdempotencyStore`).
+- ADDED: `PackerOptions.idempotency?: { store, ttlMs?, enabled? }`.
+  `Packer.pack` consults the guard before any work; a hit returns the cached
+  bundle bytes. Misses populate the store after the pack completes.
+- ADDED: `signal?: AbortSignal` on `UnpackOptions`. `Unpacker.unpack` now
+  checks `signal.aborted` at every yield point (before integrity verify,
+  before signature verify, before AEAD decrypt, before gunzip, before
+  checksum verify, before the final emit).
+- ADDED: `PackOptions` interface (`{ filename?, signal? }`) and an additional
+  `Packer.pack(data, opts: PackOptions)` overload. The original
+  `Packer.pack(data, filename?)` overload remains for back-compat.
+- ADDED: `signal?: AbortSignal` on `PackChunkedOptions` — already honoured on
+  the unpacker; now enforced on the chunked packer between every emitted
+  chunk and the source pull loop.
+- ADDED: `Clock` and `Rng` injection on `PackerOptions`, `PackChunkedOptions`,
+  `UnpackChunkedOptions`. Defaults wire to real platform implementations
+  (`() => Date.now()` for `Clock`, a wrapper around `crypto.getRandomValues`
+  for `Rng`). Every `Date.now()` / `crypto.getRandomValues()` /
+  `performance.now()` call in production `src/` is replaced with a call
+  through the injected `Clock` / `Rng`.
+- MODIFIED: `AbstractPacker.buildMetadata` reads `timestamp` from the
+  resolved `Clock` instead of `Date.now()`.
+- MODIFIED: `packer/packer.ts` reads nonce bytes via the resolved `Rng`
+  instead of calling `crypto.getRandomValues` directly.
+- MODIFIED: every `nowMs()` helper across `packer/`, `unpacker/`,
+  `chunked/packer/`, `chunked/unpacker/` now reads through the resolved
+  `monotonicClock` (defaults to `performance.now()` for backward parity).
+
+## Out of scope
+
+- `@suaveplan/schema-registry` integration → cluster #3.
+- `@suaveplan/feature-flags` for format-version rollouts → cluster #3.
+- Plugin / extension interface → cluster #3.
+- `fast-check` property tests for round-trip invariants → cluster #3.
+- mitata benchmarks → cluster #3.
+
+## Source mapping
+
+This change derives from `../suaveplan/enterprise/packer.md`.
+
+| Requirement | Source lines | Notes |
+|---|---|---|
+| PACKER-22 (`@suaveplan/retry` on IO) | L47 | Verbatim — "**retry** primitives for IO-backed sources". |
+| PACKER-23 (idempotency cache) | L47 | Verbatim — "**Idempotency** — repeat packs over the same inputs are a no-op". |
+| PACKER-24 (whole-file AbortSignal) | L46 | Verbatim — "**Backpressure & cancellation** — `AbortSignal` everywhere async". |
+| PACKER-25 (injected `Clock` + `Rng`) | L43 | Verbatim — "no `Math.random()` / `Date.now()` at module scope; injectable clock and RNG". |
+
+### Requirements with no source line
+
+None — every requirement in this cluster maps directly to an audit bullet.
+
+## Verification approach
+
+- `src/retry/retryable-read.test.ts` + `.spec-conformance.test.ts` — pins
+  `retryRead` against an injectable failing reader, asserts the retry policy
+  is consulted, asserts `AbortSignal` cancels mid-backoff.
+- `src/idempotency/idempotency.test.ts` + `.spec-conformance.test.ts` — pins
+  identical-input dedup, cache-miss population, custom-store injection.
+- `src/packer/packer.spec-conformance.test.ts` (extended) — pins `pack`
+  abort, idempotency hit, `Clock`-driven timestamp, `Rng`-driven nonce.
+- `src/unpacker/unpacker.spec-conformance.test.ts` (extended) — pins
+  pre-flight abort, mid-pipeline abort.
+- `src/chunked/packer/packer.spec-conformance.test.ts` (extended) — pins
+  signal-driven mid-stream abort.
+
+## Migration impact
+
+Public-surface additions only. No existing signature changes shape — the new
+`Packer.pack(data, opts: PackOptions)` overload sits beside the existing
+`Packer.pack(data, filename?)`. The `Clock` / `Rng` / `idempotency` /
+`retryConfig` / `signal` options are all optional with platform defaults.
+Two consumer packages (`@suaveplan/drive-plugin-chunks`,
+`@suaveplan/drive-webrtc`) are audited and the sweep is part of the gates;
+no existing call site changes shape.
+
+## Impact
+
+### Affected dependents
+
+Run `bunx turbo ls --filter=...@suaveplan/packer` to enumerate.
+
+- `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed / [ ] no action needed
+- `@suaveplan/drive-webrtc` — [x] tests run / [x] docs reviewed / [ ] no action needed
+
+If `tests run` is unchecked for any dependent, the change MUST NOT be folded
+into the canonical spec.
+
+## Migration
+
+No behavioral change for callers that did not pass the new options. Existing
+calls keep their previous semantics:
+
+- **No `signal` passed**: pack / unpack behaviour is identical to cluster #1.
+- **No `idempotency` passed**: no cache layer is consulted.
+- **No `retryConfig` passed**: a single read is attempted (no retries) for
+  parity with cluster #1 — the default policy is `maxAttempts: 1` so the
+  retry layer is a no-op unless opted in.
+- **No `clock` / `rng` passed**: defaults call `Date.now()` /
+  `crypto.getRandomValues` exactly as before, so wire output is identical.