suaveplan/openspec/changes/complete-cache-storage/design.md

Design — Complete @suaveplan/cache-storage

Non-normative. Normative requirements live in ./specs/cache-storage/spec.md.

Goal

Take @suaveplan/cache-storage from a minimum-viable StorageCache (header + 2 codecs + namespacing + lazy TTL) to a feature-complete enterprise cache: third codec, pluggable eviction, background sweep, stampede protection, stale-while-revalidate, bulk + atomic ops, configurable size limits, OTel observability, 12-code error registry, and a multi-backend conformance harness — all without touching backend implementations and without breaking the existing public surface.

Composition over rewrite

The current StorageCache.ts (240 LoC) is correct and small. Instead of replacing it, this change extends it via composition:

StorageCache (orchestrator, ~250 LoC)
  ├─ Codec<T>           — ./codec/{json,identity,structured-clone}
  ├─ EvictionPolicy     — ./eviction/{lru,lfu,fifo,ttl-only,policy.types}
  ├─ InflightMap        — ./stampede/inflight-map
  ├─ TtlSweeper         — ./sweep/ttl-sweeper
  ├─ SwrScheduler       — ./swr/swr-scheduler
  ├─ HeaderV2 codec     — ./header (extended)
  ├─ MetricsRecorder    — ./observability/metrics
  ├─ TraceRecorder      — ./observability/trace
  └─ NamespaceStats     — ./stats/namespace-stats

Every collaborator is a plain class with constructor-injected dependencies, fully unit-testable in isolation, and reachable via subpath import. StorageCache becomes the glue layer; today's 240-line file stays under the 300-line cap by extracting collaborators into their own modules.

Header v2 — version field

byte:    0 1 2 3 4 5 6 7  8 9 ...15  16 17 18 19  20 21 22 23  24 25 26 27  28 29 30 31
field:   createdAt        expiresAt   flags        hitCount     version       reserved
width:   u64 BE           u64 BE      u32 BE       u32 BE       u32 BE        zeros

• Bytes 24..27 — previously reserved-zero — now carry version: u32 BE. Legacy entries (where every reserved byte is zero) decode as version: 0. New entries written by this version of cache-storage start at version: 1 and increment on every set / successful compareAndSet.
• Bytes 28..31 remain reserved (zero). The next on-disk evolution (HMAC tag, codec discriminant, namespace hash) can claim them without breaking v2 readers.
• flags bit 2 (FLAG_HAS_VERSION = 1 << 2) is set when the writer is v2-aware. A v2 reader trusts bytes 24..27 only when this flag is set; otherwise it returns version: 0. This preserves the "reserved bytes happened to be zero on a v1 write" case.

The header codec stays in src/header/header.ts. The reserved-byte reinterpretation is an internal change — HEADER_BYTES = 32 and the existing encodeHeader/decodeHeader signatures are unchanged.

Eviction — pluggable policy

// src/eviction/policy.types.ts
export interface EvictionPolicy {
    /** Called after a successful set; returns keys to evict (may be empty). */
    onSet(key: string, sizeBytes: number, now: number): readonly string[];
    /** Called after a hit; lets the policy update its recency / frequency index. */
    onHit(key: string, now: number): void;
    /** Called after a delete / expire to drop the key from the policy's index. */
    onDelete(key: string): void;
    /** Clear the policy's internal state when the cache is cleared. */
    reset(): void;
    /** Snapshot the policy's view of size (entries, bytes). */
    snapshot(): { entries: number; bytes: number };
}

export interface EvictionConfig {
    policy: EvictionPolicy;
    maxEntries?: number;
    maxBytes?: number;
}

Four built-ins:

Policy	Eviction rule	Index
LruPolicy	Drop the least-recently-touched key when entries > maxEntries or bytes > maxBytes.	Doubly-linked list keyed by prefixedKey for O(1) touch/evict.
LfuPolicy	Drop the lowest-frequency key. Tie-break by oldest first-insertion.	Frequency buckets (Map<freq, Set<key>>) for O(1) eviction; per-key freq counter saturates at u32.
FifoPolicy	Drop the oldest-inserted key.	Insertion-ordered Map (JS spec guarantees insertion order).
TtlOnlyPolicy	Never evict on count/size; relies entirely on TTL + background sweeper.	None.

Eviction is applied by StorageCache.set after a successful write:

const victims = this.policy.onSet(prefixed, blob.length, now);
for (const victim of victims) {
    try {
        await this.storage.delete(victim);
        this.policy.onDelete(victim);
        this.size -= 1;
        this.evictions += 1;
        this.emit("evict", { key: this.stripNamespace(victim), reason: policyReason });
    } catch (cause) {
        throw new StorageCacheError(CACHE_STORAGE_CODES.EVICTION_FAILED, "error.cache-storage.eviction.delete", { cause, key: victim });
    }
}

The policy lives entirely in-memory inside StorageCache. A second StorageCache instance pointed at the same backend will keep its own policy view; this is documented as the contract — coherent eviction across instances requires a shared coordination layer (out of scope; backend-specific).

Background TTL sweep

// src/sweep/ttl-sweeper.ts
export class TtlSweeper {
    constructor(private deps: {
        storage: AbstractStorage;
        clock: Clock;
        timers: TimersLike;
        namespace: string;
        intervalMs: number;
        batchSize: number;
        onExpired: (key: string) => Promise<void>;
        onSweepError: (cause: unknown) => void;
    }) {}
    start(): void { /* setInterval via timers.setTimeout chain */ }
    stop(): void { /* clear handle */ }
}

timers: TimersLike comes from @suaveplan/types and provides setTimeout/clearTimeout/queueMicrotask/now so tests can drive the sweeper with FakeTimers without touching globals. The sweeper:

1. Lists keys via storage.list({ prefix: namespace, limit: batchSize, cursor }).
2. For each key, reads the header only (caller passes a headerProbe helper that fetches HEADER_BYTES and parses), no body decode.
3. If expiresAt !== 0 && now >= expiresAt, storage.delete(key) and onExpired(strippedKey).
4. On any error, calls onSweepError(cause) (which records CACHE_STORAGE.SWEEP_FAILED and emits a sweep-error event) but continues — one bad key never poisons the sweep.
5. After all pages, schedules the next sweep intervalMs later.

start()/stop() are idempotent. stop() clears the pending timer and resolves any in-flight page before returning.

Stampede protection — single-flight

// src/stampede/inflight-map.ts
export class InflightMap<T> {
    #map = new Map<string, Promise<T>>();
    constructor(private deps: { timers: TimersLike; defaultTimeoutMs: number }) {}

    async run(key: string, loader: () => Promise<T>, timeoutMs = this.deps.defaultTimeoutMs): Promise<T> {
        const existing = this.#map.get(key);
        if (existing !== undefined) return existing;

        const work = (async () => { ... })();
        const timed = withTimeout(work, timeoutMs, this.deps.timers);
        this.#map.set(key, timed);
        try {
            return await timed;
        } finally {
            this.#map.delete(key);
        }
    }
    get pending(): number { return this.#map.size; }
}

withTimeout races the loader against a timers.setTimeout that rejects with CACHE_STORAGE.STAMPEDE_TIMEOUT. The timeout cancels the timer on resolution to keep the event loop clean. The counter stampedePrevented increments on every run call that returns the existing promise instead of starting new work.

StorageCache.wrap consults the in-flight map inside the miss path so concurrent misses share the loader; getOrSet is just wrap with stampede enabled by default.

Stale-while-revalidate

cache.wrap(key, loader, { staleWhileRevalidate: true, staleThresholdMs }):

1. Read the entry. If clock() < expiresAt - staleThresholdMs, return the fresh value.
2. If clock() >= expiresAt - staleThresholdMs AND clock() < expiresAt, return the stale value immediately AND schedule a background refresh.
3. If clock() >= expiresAt, treat as a miss — go through the stampede-protected loader path.

The background refresh runs via timers.queueMicrotask; failure increments swrRefreshFailures and emits swr-refresh-failed but never reaches the caller. The refresh path also goes through the same InflightMap so a stampede of stale callers fires the loader once.

Bulk ops — bulk-aware backend dispatch

interface BulkCapableStorage extends AbstractStorage {
    getMultiple(keys: string[]): Promise<Map<string, Uint8Array>>;
    setMultiple(entries: Array<{ key: string; value: Uint8Array }>): Promise<void>;
}

function hasBulk(s: AbstractStorage): s is BulkCapableStorage {
    return typeof (s as Partial<BulkCapableStorage>).getMultiple === "function"
        && typeof (s as Partial<BulkCapableStorage>).setMultiple === "function";
}

StorageCache.mget checks hasBulk(storage):

• If yes, one storage.getMultiple(prefixedKeys) call; iterate the returned Map, decode each header + body, build the result map. Single TTL check per key.
• If no, fall back to Promise.all(keys.map(k => this.get(k))).

mset mirrors the dispatch. mdelete returns the count of keys that actually existed and were removed. All three open an OTel span with cache.bulk.size = keys.length.

Atomic ops — compareAndSet

async compareAndSet(key: string, expectedVersion: number, value: T, options?: CacheSetOptions): Promise<{ version: number }> {
    const prefixed = this.prefixed(key);
    const current = await this.readWithHeader(prefixed);  // throws BACKEND_ERROR / ENTRY_CORRUPT
    const observed = current?.meta.version ?? 0;
    if (observed !== expectedVersion) {
        throw new StorageCacheError(CACHE_STORAGE_CODES.CAS_CONFLICT, "error.cache-storage.cas-conflict", { key, context: { expected: expectedVersion, observed } });
    }
    const nextVersion = (observed + 1) >>> 0;
    await this.writeWithVersion(prefixed, value, options, nextVersion);
    return { version: nextVersion };
}

This is optimistic CAS — there is no backend-level read-modify-write atomicity, so two concurrent CAS calls against the same key from different processes can both observe the same version and both write. For coherent CAS across instances callers need a backend that supports atomic check-and-set (e.g. Redis WATCH/MULTI); the spec text makes this explicit and the doc points at cache-redis for the strong variant.

getWithVersion(key): Promise<{ value: T; version: number } | undefined> returns the decoded value alongside the header version so callers can drive the CAS loop without re-reading.

StructuredCloneCodec (reused from superseded change)

Lift table reused verbatim from openspec/changes/_superseded-narrow-deferrals-2026-05-17/add-cache-storage-cbor-codec/design.md:

JS type	CBOR tag	Payload
Date	0	RFC 3339 string
bigint > 0	2	byte string, big-endian magnitude
bigint < 0	3	byte string, big-endian magnitude of -1 - n
URL	32	URL.href string
RegExp	35	regex.source string (flags dropped — documented trade-off)
Uint8Array	64	byte string
Set	258	array of items
Map	259	array of [k, v] pairs

Encode walks the input, lifts carriers to Tagged(tag, walk(payload)), then calls encode from @suaveplan/cbor. Decode calls decode with the registry; any leftover Tagged post-walk triggers CODEC_UNKNOWN_TAG. Unencodable types (Function, Symbol, WeakMap, WeakSet, Promise, unregistered class instances) trigger CODEC_UNSUPPORTED_TYPE.

Conformance harnesses

Two harnesses ship inside the package:

1. runCodecConformance(codec, { name, profile }) — profile is "json" | "structured-clone" | "bytes". Each profile defines the round-trip cases the codec must pass: json is primitives + plain objects + arrays; structured-clone adds Date/Map/Set/Uint8Array/bigint/RegExp/URL + deep nesting; bytes is Uint8Array round-trip only. Self-test feeds a deliberately broken codec and asserts at least one harness check fails.
2. runStorageCacheConformance({ createStorage, name }) — accepts a factory () => AbstractStorage so the same suite can run against MemoryStorage, a Redis-backed storage, an OPFS storage, etc. Covers the full API (get/set/delete/has/clear/getStats/mget/mset/mdelete/wrap/getOrSet/compareAndSet/getWithVersion), every eviction policy, stampede dedup, SWR, and the 12 error codes. Lives in ./testing/conformance so it ships with the package but is tree-shakeable.

The conformance harness lives in this package, not in @suaveplan/testing, because it imports StorageCache directly — lifting it would invert the dep graph (rule 8).

Observability

const tracer = trace.getTracer("@suaveplan/cache-storage");
const meter = metrics.getMeter("@suaveplan/cache-storage");

const hitsCtr = meter.createCounter("cache.hits");
const missesCtr = meter.createCounter("cache.misses");
const evictionsCtr = meter.createCounter("cache.evictions");
const stampedeCtr = meter.createCounter("cache.stampede_prevented");
const swrCtr = meter.createCounter("cache.swr_refreshes");
const rejectedCtr = meter.createCounter("cache.rejected");
const payloadHist = meter.createHistogram("cache.payload_bytes");

Every public method opens a span named cache.<op> with attributes cache.namespace, cache.key (when low-cardinality safe — caller opts in via { traceKey: true }), cache.hit, cache.size_bytes. Span ends with SpanStatusCode.OK on success or SpanStatusCode.ERROR carrying the SuaveplanError.code.

No exporter registered ⇒ all of this is no-op per the @opentelemetry/api contract — zero allocations on the hot path beyond the constant tracer/meter lookups.

Error registry — 12 codes

export const CACHE_STORAGE_CODES = defineCodes("CACHE_STORAGE", [
    "ENTRY_CORRUPT",           // existing
    "CODEC_ERROR",             // existing
    "BACKEND_ERROR",           // existing
    "KEY_TOO_LONG",            // existing
    "VALUE_TOO_LARGE",         // existing
    "STAMPEDE_TIMEOUT",        // new
    "CAS_CONFLICT",            // new
    "SWEEP_FAILED",            // new
    "EVICTION_FAILED",         // new
    "CODEC_UNSUPPORTED_TYPE",  // new
    "CODEC_UNKNOWN_TAG",       // new
    "NAMESPACE_CONFLICT",      // new — raised when a child namespace tries to register a name already in use
] as const);

Every code is reachable from at least one test call site — verified by the errors.test.ts reachability assertion.

File layout

packages/flow/cache-storage/src/
  codec/
    codec.types.ts                      # unchanged
    identity.{ts,test.ts,md}            # unchanged
    json.{ts,test.ts,md}                # extended with conformance harness call
    structured-clone.{ts,test.ts,md}    # NEW
    conformance.{ts,test.ts,md}         # NEW
    index.ts
  eviction/
    policy.types.ts                     # NEW — interface only
    lru/{lru,lru.test,lru.md}.ts        # NEW
    lfu/{lfu,lfu.test,lfu.md}.ts        # NEW
    fifo/{fifo,fifo.test,fifo.md}.ts    # NEW
    ttl/{ttl,ttl.test,ttl.md}.ts        # NEW — TtlOnlyPolicy
    index.ts                            # NEW
  stampede/
    inflight-map/{inflight-map,...}.ts  # NEW
    index.ts                            # NEW
  sweep/
    ttl-sweeper/{ttl-sweeper,...}.ts    # NEW
    index.ts                            # NEW
  swr/
    swr-scheduler/{swr-scheduler,...}.ts # NEW
    index.ts                            # NEW
  observability/
    metrics/{metrics,...}.ts            # NEW
    trace/{trace,...}.ts                # NEW
    index.ts                            # NEW
  stats/
    namespace-stats/{namespace-stats,...}.ts  # NEW
    index.ts                            # NEW
  header/
    header.ts                           # MODIFIED — version field
    header.test.ts                      # extended
    header.md                           # extended
    index.ts
  errors/
    errors.ts                           # MODIFIED — 5 → 12 codes
    errors.test.ts                      # extended — reachability for all 12
    errors.md                           # extended
    index.ts
  storage-cache/
    storage-cache.ts                    # MODIFIED — composes new collaborators (~280 LoC)
    storage-cache.types.ts              # MODIFIED — new options
    storage-cache.test.ts               # extended
    storage-cache.md                    # extended
    index.ts
  registry-augment/                     # unchanged
  testing/
    conformance/{conformance,...}.ts    # NEW — runStorageCacheConformance
    index.ts                            # NEW
  index.ts                              # MODIFIED — new exports

Every new file respects the 300-line soft cap. The StorageCache itself stays under 300 lines because every cross-cutting concern (eviction, sweep, stampede, SWR, metrics) lives in a co-located collaborator.

Trade-offs

• In-memory eviction index. A StorageCache instance keeps the eviction policy's index in memory. Multiple instances pointed at the same backend do not coordinate — documented contract. Coordinated eviction belongs to the backend (e.g. Redis maxmemory-policy lru).
• Optimistic CAS. compareAndSet is not backend-atomic. The doc says so and points at backend-native primitives (WATCH/MULTI in Redis). Strong CAS requires backend support.
• Sweep cost. The background sweep reads every key's header in batchSize pages. For backends where list is cheap (memory, OPFS) this is negligible; for cloud backends (S3) callers should tune intervalMs and batchSize accordingly. Documented.
• OTel overhead when no exporter registered. Negligible (the no-op tracer is a singleton, span/counter calls are no-op functions) — verified by mitata baseline.
• Header v2 reserved-byte reinterpretation. Backwards-compatible because v1 reserved bytes were defined as zero; flag bit 2 (FLAG_HAS_VERSION) disambiguates v1-from-v2 unambiguously.

Future work (explicitly deferred — out of scope)

• Backend-specific implementations (cache-opfs, cache-s3) — each is its own package.
• Public StructuredCloneCodec tag-registration API for caller-defined carriers.
• Mutation-resistant CAS via backend primitives (Redis WATCH/MULTI) — belongs in cache-redis.
• Distributed stampede dedup across cache instances (single-flight via Redis SETNX lock) — belongs in cache-redis.
• Tiered cache (L1 memory in front of L2 storage) — belongs in a new cache-tiered package.