first commit
This commit is contained in:
116
docs/foundation/data/packer/src/idempotency/idempotency.md
Normal file
116
docs/foundation/data/packer/src/idempotency/idempotency.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# `@suaveplan/packer/idempotency` — Idempotent repeat-pack cache
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Repeat packs over identical inputs are a guaranteed no-op: the second call
|
||||
returns the cached `PackedData` without re-running the gzip / AEAD /
|
||||
ed25519 / BLAKE3-integrity pipeline. Cluster #1 already paid the cost of
|
||||
computing a content-addressable BLAKE3 digest per pack — this module turns
|
||||
that digest into the cache key for an opt-in idempotency layer backed by
|
||||
`@suaveplan/idempotency`. The audit line in `enterprise/packer.md` (L47)
|
||||
mandates "**Idempotency** — repeat packs over the same inputs are a no-op".
|
||||
|
||||
The layer is opt-in: callers that do not pass `PackerOptions.idempotency`
|
||||
see the legacy non-cached behaviour. Callers that opt in get a content-
|
||||
addressable cache backed by any `IdempotencyStoreAdapter` —
|
||||
`MemoryIdempotencyStore` for single-process, Redis / database adapters for
|
||||
distributed deployments.
|
||||
|
||||
## Features
|
||||
|
||||
- `PackIdempotencyGuard` — wraps an `IdempotencyStoreAdapter<PackedData>`
|
||||
into a `lookup(key)` / `record(key, packed)` pair tailored for the pack
|
||||
pipeline.
|
||||
- `computePackDedupKey(data, fingerprint)` — derives the cache key from
|
||||
`blake3(data)` plus the pipeline fingerprint (compression level,
|
||||
encryption algorithm, signing-enabled bit, filename). Two packs with
|
||||
identical bytes but different pipeline parameters do not collide.
|
||||
- Default `MemoryIdempotencyStore<PackedData>` when no store is supplied
|
||||
but the guard is enabled, so callers can opt in without choosing a
|
||||
backend up-front.
|
||||
- Configurable TTL (default 24 h) and an `enabled` toggle for runtime
|
||||
feature-flag style off-switches.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { Packer } from "@suaveplan/packer";
|
||||
import { MemoryIdempotencyStore } from "@suaveplan/idempotency";
|
||||
|
||||
const store = new MemoryIdempotencyStore();
|
||||
const packer = new Packer({ idempotency: { store } });
|
||||
|
||||
const data = new Uint8Array([1, 2, 3]);
|
||||
const a = await packer.pack(data);
|
||||
const b = await packer.pack(data); // cache hit — no pipeline work
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { Packer } from "@suaveplan/packer";
|
||||
import {
|
||||
computePackDedupKey,
|
||||
PackIdempotencyGuard,
|
||||
} from "@suaveplan/packer/idempotency";
|
||||
import { RedisIdempotencyStore } from "@suaveplan/idempotency";
|
||||
|
||||
// Distributed deployment with explicit cache key inspection.
|
||||
const store = new RedisIdempotencyStore<PackedData>(/* config */);
|
||||
const packer = new Packer({
|
||||
idempotency: { store, ttlMs: 3_600_000 }, // 1 hour
|
||||
encryption: { algorithm: "xchacha20-poly1305", key: aeadKey },
|
||||
signing: { key: signingKey },
|
||||
});
|
||||
|
||||
const data = new Uint8Array([1, 2, 3]);
|
||||
const key = computePackDedupKey(data, {
|
||||
compressionLevel: 6,
|
||||
encrypted: true,
|
||||
encryptionAlgorithm: "xchacha20-poly1305",
|
||||
signed: true,
|
||||
filename: undefined,
|
||||
});
|
||||
console.log("dedup key:", key);
|
||||
const bundle = await packer.pack(data);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `PackIdempotencyGuard.constructor(opts, clock)` — wraps an injected
|
||||
`IdempotencyStoreAdapter<PackedData>`. Uses the supplied `Clock` for TTL
|
||||
computation so determinism is preserved when callers inject a
|
||||
`FakeClock`-backed value.
|
||||
- `PackIdempotencyGuard.enabled: boolean` — reflects `opts.enabled` (true
|
||||
by default).
|
||||
- `PackIdempotencyGuard.lookup(key): Promise<PackedData | undefined>` —
|
||||
returns a cached `PackedData` on hit, `undefined` on miss / when
|
||||
disabled / when the record is not `completed`.
|
||||
- `PackIdempotencyGuard.record(key, packed): Promise<void>` — writes a
|
||||
completed record under `key` with the configured TTL. No-op when
|
||||
disabled.
|
||||
- `computePackDedupKey(data, fingerprint): string` — derives the cache key
|
||||
from `blake3(data)` + stringified pipeline fingerprint.
|
||||
- `PackPipelineFingerprint` — `{ compressionLevel, encrypted,
|
||||
encryptionAlgorithm, signed, filename }`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The cache value is the full `PackedData` (bytes + metadata). For
|
||||
encrypted bundles, the AEAD nonce is captured on the metadata so the
|
||||
cached envelope decrypts deterministically without needing the original
|
||||
RNG state.
|
||||
- `lookup` checks `record.status === "completed"`; pending and failed
|
||||
records are treated as misses so an interrupted pack from a different
|
||||
caller doesn't poison the cache.
|
||||
- The dedup key is human-readable hex (not just bytes) so operators can
|
||||
inspect Redis / DB rows without a decoder layer. The BLAKE3 output is
|
||||
256 bits so cache-key collision probability is negligible (2^-256).
|
||||
- The guard does not own the store — callers wire and tear down the
|
||||
store. A default `MemoryIdempotencyStore` is constructed only when no
|
||||
store is supplied but the guard is otherwise enabled; in that mode the
|
||||
cache lives for the lifetime of the `Packer` instance.
|
||||
Reference in New Issue
Block a user