145 lines
7.2 KiB
Markdown
145 lines
7.2 KiB
Markdown
# Enterprise hardening — `@suaveplan/packer` resilience cluster
|
||
|
||
**Package**: `@suaveplan/packer`
|
||
**Status**: PROPOSED
|
||
**Type**: feature
|
||
**Design**: (inline below — small scope)
|
||
**Spec delta**: ./specs/packer/spec.md
|
||
|
||
## Why
|
||
|
||
Cluster #1 (`enterprise-packer-integrity`, archived 2026-05-19) landed BLAKE3
|
||
content-addressable digests, the bundle-level Merkle root, mandatory
|
||
integrity verification on unpack, full OpenTelemetry instrumentation, and
|
||
i18n-keyed error messages. The `enterprise/packer.md` audit (L43, L46–L47)
|
||
calls out three remaining resilience gaps that block the ENTERPRISE label:
|
||
|
||
1. No retry primitives on IO-backed sources (`ReadableStream.read()` calls
|
||
throw straight up the stack on transient errors).
|
||
2. No idempotency layer — repeat packs over identical inputs re-run the full
|
||
gzip + AEAD + ed25519 pipeline every time.
|
||
3. The whole-file `pack` / `unpack` paths do not accept `AbortSignal`; only
|
||
the chunked path honours cancellation today.
|
||
4. `Date.now()` and `crypto.getRandomValues()` are called at production code
|
||
paths in `src/`, violating the audit's "no `Math.random()` / `Date.now()`
|
||
at module scope; injectable clock and RNG" line.
|
||
|
||
This is cluster #2 of a 3-cluster enterprise hardening of `@suaveplan/packer`
|
||
(integrity → resilience → extensibility). Cluster #3 is deliberately out of
|
||
scope here — see "Out of scope" below.
|
||
|
||
## What changes
|
||
|
||
- ADDED: `src/retry/retryable-read.ts` — `retryRead(reader, opts)` helper that
|
||
wraps `ReadableStream` reads in `@suaveplan/retry`'s `withRetry`, honours
|
||
`AbortSignal`, and returns the same `{ value, done }` shape. Defaults:
|
||
`createExponentialBackoff({ initial: 25, base: 2, max: 1000, jitter: "equal" })`
|
||
+ `maxAttempts: 4`.
|
||
- ADDED: `RetryPolicy` option bundle on `PackerOptions`, `PackChunkedOptions`,
|
||
and `UnpackChunkedOptions` (`retryConfig?`, `signal?`). Threaded through to
|
||
`normalizeSource` and the abstract `collect` path so every `reader.read()`
|
||
in production code is retry-aware.
|
||
- ADDED: `src/idempotency/idempotency.ts` — `PackIdempotencyGuard` that
|
||
computes a content-derived key (`blake3(payload) || compressionLevel ||
|
||
encryptionAlgo || signed`-bit) and short-circuits to a cached `PackedData`
|
||
when the same input is packed again. Backed by an injectable
|
||
`IdempotencyStoreAdapter` (default `MemoryIdempotencyStore`).
|
||
- ADDED: `PackerOptions.idempotency?: { store, ttlMs?, enabled? }`.
|
||
`Packer.pack` consults the guard before any work; a hit returns the cached
|
||
bundle bytes. Misses populate the store after the pack completes.
|
||
- ADDED: `signal?: AbortSignal` on `UnpackOptions`. `Unpacker.unpack` now
|
||
checks `signal.aborted` at every yield point (before integrity verify,
|
||
before signature verify, before AEAD decrypt, before gunzip, before
|
||
checksum verify, before the final emit).
|
||
- ADDED: `PackOptions` interface (`{ filename?, signal? }`) and an additional
|
||
`Packer.pack(data, opts: PackOptions)` overload. The original
|
||
`Packer.pack(data, filename?)` overload remains for back-compat.
|
||
- ADDED: `signal?: AbortSignal` on `PackChunkedOptions` — already honoured on
|
||
the unpacker; now enforced on the chunked packer between every emitted
|
||
chunk and the source pull loop.
|
||
- ADDED: `Clock` and `Rng` injection on `PackerOptions`, `PackChunkedOptions`,
|
||
`UnpackChunkedOptions`. Defaults wire to real platform implementations
|
||
(`() => Date.now()` for `Clock`, a wrapper around `crypto.getRandomValues`
|
||
for `Rng`). Every `Date.now()` / `crypto.getRandomValues()` /
|
||
`performance.now()` call in production `src/` is replaced with a call
|
||
through the injected `Clock` / `Rng`.
|
||
- MODIFIED: `AbstractPacker.buildMetadata` reads `timestamp` from the
|
||
resolved `Clock` instead of `Date.now()`.
|
||
- MODIFIED: `packer/packer.ts` reads nonce bytes via the resolved `Rng`
|
||
instead of calling `crypto.getRandomValues` directly.
|
||
- MODIFIED: every `nowMs()` helper across `packer/`, `unpacker/`,
|
||
`chunked/packer/`, `chunked/unpacker/` now reads through the resolved
|
||
`monotonicClock` (defaults to `performance.now()` for backward parity).
|
||
|
||
## Out of scope
|
||
|
||
- `@suaveplan/schema-registry` integration → cluster #3.
|
||
- `@suaveplan/feature-flags` for format-version rollouts → cluster #3.
|
||
- Plugin / extension interface → cluster #3.
|
||
- `fast-check` property tests for round-trip invariants → cluster #3.
|
||
- mitata benchmarks → cluster #3.
|
||
|
||
## Source mapping
|
||
|
||
This change derives from `../suaveplan/enterprise/packer.md`.
|
||
|
||
| Requirement | Source lines | Notes |
|
||
|---|---|---|
|
||
| PACKER-22 (`@suaveplan/retry` on IO) | L47 | Verbatim — "**retry** primitives for IO-backed sources". |
|
||
| PACKER-23 (idempotency cache) | L47 | Verbatim — "**Idempotency** — repeat packs over the same inputs are a no-op". |
|
||
| PACKER-24 (whole-file AbortSignal) | L46 | Verbatim — "**Backpressure & cancellation** — `AbortSignal` everywhere async". |
|
||
| PACKER-25 (injected `Clock` + `Rng`) | L43 | Verbatim — "no `Math.random()` / `Date.now()` at module scope; injectable clock and RNG". |
|
||
|
||
### Requirements with no source line
|
||
|
||
None — every requirement in this cluster maps directly to an audit bullet.
|
||
|
||
## Verification approach
|
||
|
||
- `src/retry/retryable-read.test.ts` + `.spec-conformance.test.ts` — pins
|
||
`retryRead` against an injectable failing reader, asserts the retry policy
|
||
is consulted, asserts `AbortSignal` cancels mid-backoff.
|
||
- `src/idempotency/idempotency.test.ts` + `.spec-conformance.test.ts` — pins
|
||
identical-input dedup, cache-miss population, custom-store injection.
|
||
- `src/packer/packer.spec-conformance.test.ts` (extended) — pins `pack`
|
||
abort, idempotency hit, `Clock`-driven timestamp, `Rng`-driven nonce.
|
||
- `src/unpacker/unpacker.spec-conformance.test.ts` (extended) — pins
|
||
pre-flight abort, mid-pipeline abort.
|
||
- `src/chunked/packer/packer.spec-conformance.test.ts` (extended) — pins
|
||
signal-driven mid-stream abort.
|
||
|
||
## Migration impact
|
||
|
||
Public-surface additions only. No existing signature changes shape — the new
|
||
`Packer.pack(data, opts: PackOptions)` overload sits beside the existing
|
||
`Packer.pack(data, filename?)`. The `Clock` / `Rng` / `idempotency` /
|
||
`retryConfig` / `signal` options are all optional with platform defaults.
|
||
Two consumer packages (`@suaveplan/drive-plugin-chunks`,
|
||
`@suaveplan/drive-webrtc`) are audited and the sweep is part of the gates;
|
||
no existing call site changes shape.
|
||
|
||
## Impact
|
||
|
||
### Affected dependents
|
||
|
||
Run `bunx turbo ls --filter=...@suaveplan/packer` to enumerate.
|
||
|
||
- `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||
- `@suaveplan/drive-webrtc` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||
|
||
If `tests run` is unchecked for any dependent, the change MUST NOT be folded
|
||
into the canonical spec.
|
||
|
||
## Migration
|
||
|
||
No behavioral change for callers that did not pass the new options. Existing
|
||
calls keep their previous semantics:
|
||
|
||
- **No `signal` passed**: pack / unpack behaviour is identical to cluster #1.
|
||
- **No `idempotency` passed**: no cache layer is consulted.
|
||
- **No `retryConfig` passed**: a single read is attempted (no retries) for
|
||
parity with cluster #1 — the default policy is `maxAttempts: 1` so the
|
||
retry layer is a no-op unless opted in.
|
||
- **No `clock` / `rng` passed**: defaults call `Date.now()` /
|
||
`crypto.getRandomValues` exactly as before, so wire output is identical.
|