skills/docs/foundation/data/packer/src/idempotency/idempotency.md

@suaveplan/packer/idempotency — Idempotent repeat-pack cache

(no submodules)

Purpose

Repeat packs over identical inputs are a guaranteed no-op: the second call returns the cached PackedData without re-running the gzip / AEAD / ed25519 / BLAKE3-integrity pipeline. Cluster #1 already paid the cost of computing a content-addressable BLAKE3 digest per pack — this module turns that digest into the cache key for an opt-in idempotency layer backed by @suaveplan/idempotency. The audit line in enterprise/packer.md (L47) mandates "Idempotency — repeat packs over the same inputs are a no-op".

The layer is opt-in: callers that do not pass PackerOptions.idempotency see the legacy non-cached behaviour. Callers that opt in get a content- addressable cache backed by any IdempotencyStoreAdapter — MemoryIdempotencyStore for single-process, Redis / database adapters for distributed deployments.

Features

• PackIdempotencyGuard — wraps an IdempotencyStoreAdapter<PackedData> into a lookup(key) / record(key, packed) pair tailored for the pack pipeline.
• computePackDedupKey(data, fingerprint) — derives the cache key from blake3(data) plus the pipeline fingerprint (compression level, encryption algorithm, signing-enabled bit, filename). Two packs with identical bytes but different pipeline parameters do not collide.
• Default MemoryIdempotencyStore<PackedData> when no store is supplied but the guard is enabled, so callers can opt in without choosing a backend up-front.
• Configurable TTL (default 24 h) and an enabled toggle for runtime feature-flag style off-switches.

Usage: Basic Example

import { Packer } from "@suaveplan/packer";
import { MemoryIdempotencyStore } from "@suaveplan/idempotency";

const store = new MemoryIdempotencyStore();
const packer = new Packer({ idempotency: { store } });

const data = new Uint8Array([1, 2, 3]);
const a = await packer.pack(data);
const b = await packer.pack(data); // cache hit — no pipeline work

Usage: Advanced Example

import { Packer } from "@suaveplan/packer";
import {
    computePackDedupKey,
    PackIdempotencyGuard,
} from "@suaveplan/packer/idempotency";
import { RedisIdempotencyStore } from "@suaveplan/idempotency";

// Distributed deployment with explicit cache key inspection.
const store = new RedisIdempotencyStore<PackedData>(/* config */);
const packer = new Packer({
    idempotency: { store, ttlMs: 3_600_000 }, // 1 hour
    encryption: { algorithm: "xchacha20-poly1305", key: aeadKey },
    signing: { key: signingKey },
});

const data = new Uint8Array([1, 2, 3]);
const key = computePackDedupKey(data, {
    compressionLevel: 6,
    encrypted: true,
    encryptionAlgorithm: "xchacha20-poly1305",
    signed: true,
    filename: undefined,
});
console.log("dedup key:", key);
const bundle = await packer.pack(data);

API Reference

• PackIdempotencyGuard.constructor(opts, clock) — wraps an injected IdempotencyStoreAdapter<PackedData>. Uses the supplied Clock for TTL computation so determinism is preserved when callers inject a FakeClock-backed value.
• PackIdempotencyGuard.enabled: boolean — reflects opts.enabled (true by default).
• PackIdempotencyGuard.lookup(key): Promise<PackedData | undefined> — returns a cached PackedData on hit, undefined on miss / when disabled / when the record is not completed.
• PackIdempotencyGuard.record(key, packed): Promise<void> — writes a completed record under key with the configured TTL. No-op when disabled.
• computePackDedupKey(data, fingerprint): string — derives the cache key from blake3(data) + stringified pipeline fingerprint.
• PackPipelineFingerprint — { compressionLevel, encrypted, encryptionAlgorithm, signed, filename }.

Implementation Notes

• The cache value is the full PackedData (bytes + metadata). For encrypted bundles, the AEAD nonce is captured on the metadata so the cached envelope decrypts deterministically without needing the original RNG state.
• lookup checks record.status === "completed"; pending and failed records are treated as misses so an interrupted pack from a different caller doesn't poison the cache.
• The dedup key is human-readable hex (not just bytes) so operators can inspect Redis / DB rows without a decoder layer. The BLAKE3 output is 256 bits so cache-key collision probability is negligible (2^-256).
• The guard does not own the store — callers wire and tear down the store. A default MemoryIdempotencyStore is constructed only when no store is supplied but the guard is otherwise enabled; in that mode the cache lives for the lifetime of the Packer instance.