suaveplan/openspec/archive/2026-05-16-add-storage-plugin-cache/proposal.md

Add @suaveplan/storage-plugin-cache — read-through cache decorator for AbstractStorage

Package: @suaveplan/storage-plugin-cache Status: PROPOSED Type: new-package (genesis-original) Depends on: @suaveplan/storage (archived), @suaveplan/cache family (archived as add-cache-storage-redis-backends on 2026-05-16) Sibling: add-drive-plugin-cache (this change targets the storage tier; the drive-tier twin targets Drive.read(path))

• Design: ./design.md
• Spec delta: ./specs/storage-plugin-cache/spec.md

Why

The cache family on genesis (@suaveplan/cache, cache-storage, cache-redis, built-in MemoryCache) provides one unified abstraction — AbstractCache<T> with createCache("memory" | "storage" | "redis", config) — and @suaveplan/cache-storage lets any AbstractStorage become a cache backend (storage-as-cache). The inverse surface is missing: there is no decorator that puts a cache in front of an AbstractStorage. Concretely, a consumer holding a slow remote storage (storage-http, storage-s3, storage-cdn-rackspace, storage-cdn-bunnycdn) today cannot wrap it with a fast local cache (MemoryCache for hot keys, StorageCache over OPFS/file for cross-process durability) without hand-rolling the read-through / invalidate-on-write logic at every call site.

storage-plugin-cache is the inverse of cache-storage: it takes any AbstractStorage and returns another AbstractStorage that consults a caller-supplied AbstractCache<Uint8Array> first on every get, falls through to the inner storage on miss, populates on success, and invalidates the cache key on every set/setPartial/delete/clear. The result is structurally identical to NamespacedStorage — a decorator class extending AbstractStorage that delegates to an inner instance — so every consumer in the ecosystem that accepts a StorageLike or AbstractStorage (drive, storage-uri plugins, application code) takes the cached version with zero adjustment.

This is the structural twin of add-drive-plugin-cache: same AbstractCache<Uint8Array> injection, same error-swallow rules, same OTel surface shape, same shouldCache / maxEntryBytes defaults. Where drive-plugin-cache caches reassembled plaintext at the drive middleware tier, storage-plugin-cache caches raw stored bytes at the storage tier — the most fundamental of the three planned cache decorators (the third being a future cacheUriPlugin over the URI resolver).

What changes

• ADDED: CachedStorage extends AbstractStorage — decorator class that wraps an inner AbstractStorage and intercepts every abstract method. Constructor takes the inner storage and a CachedStorageOptions bag.
• ADDED: cachedStorage(inner, options): CachedStorage factory — convenience over new CachedStorage(inner, options) matching the createCache / createHttpStorage family conventions. Validates options synchronously and throws STORAGE_CACHE.INVALID_OPTIONS on missing/malformed cache.
• ADDED: CachedStorageOptions interface — { cache: AbstractCache<Uint8Array>; invalidateOnWrite?: boolean; invalidateOnDelete?: boolean; invalidateOnClear?: boolean; namespace?: string; shouldCache?: (key: string, bytes: Uint8Array) => boolean; maxEntryBytes?: number; logger?: LoggerLike }.
• ADDED: Read-through get(key, signal?):
  1. Compute cache key as ${namespace ?? "storage"}::${key}.
  2. await cache.get(cacheKey). If hit, return bytes; increment storage_cache.hits_total{namespace}.
  3. On miss, await inner.get(key, signal). Increment storage_cache.misses_total{namespace}.
  4. When the inner returns a Uint8Array AND shouldCache(key, bytes) returns true, await cache.set(cacheKey, bytes). Increment storage_cache.populations_total{namespace}.
  5. Cache get/set failures SHALL be swallowed (logged, counter-incremented) — a cache failure MUST NOT turn a successful inner.get into a thrown error.
• ADDED: Invalidation surface. On set(key, value, signal?), the wrapper SHALL call inner.set(key, value, signal) first; on success, when invalidateOnWrite !== false, await cache.delete(cacheKey). Symmetric on setPartial(...), delete(key, signal?) (when invalidateOnDelete !== false), and clear(signal?) (when invalidateOnClear !== false, the wrapper calls cache.clear() to drop every namespaced entry; the spec MUST document the scope as "every entry under this wrapper's namespace, not the entire cache"). Cache delete/clear failures SHALL NOT propagate.
• ADDED: getWithMetadata, getMetadata, getSize, getItemSize, exists, list — these all delegate to the inner storage unchanged. getWithMetadata does NOT populate the cache (metadata mutation invalidation is out of scope); exists is cheap on every backend we ship, so caching it would add invalidation surface without meaningful win. The wrapper SHALL document these as "passthrough".
• ADDED: getPartial and setPartial — passthrough to inner. Partial-byte caching is out of scope (range invalidation across overlapping writes is a separate design problem).
• ADDED: createReadStream and createWriteStream — passthrough to inner. Streaming bypasses the buffered cache by design; a cacheUriPlugin (planned) will cover cached streaming at the URI tier.
• ADDED: STORAGE_CACHE_CODES registry via defineCodes("STORAGE_CACHE", [...] as const) — 2 codes: INVALID_OPTIONS, INNER_MISSING. The wrapper's own failures are deliberately few — everything else surfaces from the inner storage (STORAGE_HTTP.*, STORAGE.*, etc.) or the cache (CACHE.*) unchanged.
• ADDED: CachedStorageError extends SuaveplanError<CachedStorageErrorCode> with key? and namespace? discriminants.
• ADDED: CachedStorageStats exposed via the class's getCacheStats() method (distinct from AbstractStorage.getStats() which counts the inner storage's operations) — { hits, misses, populations, invalidations, errors }.
• ADDED: OTel surface — storage_cache.{hits, misses, populations, invalidations, errors}_total{namespace} counters; storage_cache.entry_bytes{namespace} histogram observing populations; spans on @suaveplan/storage-cache tracer with storage.cache.namespace, storage.cache.key, storage.cache.outcome ∈ {hit, miss, populated, invalidated, error} attributes.
• ADDED: README composition recipes — memory cache in front of S3, OPFS in front of HTTPS, file cache in front of CDN, Redis cache in front of S3 (multi-process), namespaced isolation across multiple wrappers sharing one underlying cache, cache.wrap()-based stale-while-revalidate via the exposed cache reference.

Out of scope

• Range-byte caching. getPartial/setPartial pass through. Range invalidation (a setPartial(key, offset=10, bytes) requires evicting any cached byte slice that overlaps [10, 10+len]) is non-trivial and rare in practice. Consumers needing fast random-access reads should cache at the URI tier or use a content-addressed scheme.
• Streaming caching. createReadStream/createWriteStream pass through. Same reason as the drive-plugin-cache: cached streaming belongs at the URI layer with the planned cacheUriPlugin, where the resolver owns the stream surface.
• List-result caching. Same problem as in drive-plugin-cache: list invalidation across prefix-mutating writes is hard; consumers needing fast list should compose a higher-level index or use a content-addressed storage.
• Cache stampede protection. If two concurrent get(k) calls both miss the cache, both will call inner.get(k) and both will populate. Single-flight / lock-and-load belongs in AbstractCache.wrap(...) (which already provides it). Consumers needing stampede protection should compose at the cache tier via handle.cache.wrap() rather than at the storage decorator.
• exists caching. Cheap on every shipped backend; caching adds an invalidation surface without a meaningful latency win. Out of scope.
• Negative caching. inner.get returning undefined (key absent) is NOT cached. Caching absence requires a sentinel value and a separate TTL knob; can be added later if a real workload needs it. For now, undefined is always a fresh read.
• Metadata caching (getMetadata, getWithMetadata). Passthrough. ETag-based conditional reads already live in storage-http's EtagCache; bringing them up here would duplicate that layer.
• storage-uri-tier cacheUriPlugin. Separate proposal (sibling). Wraps any UriPlugin with caching across URI schemes.
• drive-plugin-cache parity. Separate sibling proposal (add-drive-plugin-cache). Caches Drive.read(path) plaintext at the drive middleware tier.

Source mapping

This is a genesis-original package; no upstream suaveplan/ reference. Requirements derive from:

• The AbstractStorage abstract class in @suaveplan/storage (openspec/specs/storage/spec.md) — every abstract method (get/set/delete/exists/list/clear/getWithMetadata/getMetadata/getSize/getItemSize/getPartial/setPartial/createReadStream/createWriteStream) must be implemented in CachedStorage with delegation semantics.
• The NamespacedStorage decorator precedent in @suaveplan/storage (src/abstract/abstract.ts:559) — same structural pattern: wrap inner, prefix keys, delegate operations.
• The AbstractCache<T> contract in @suaveplan/cache.
• The conventions established by add-drive-plugin-cache (sibling) — identical option shape, error-swallow rules, OTel naming, recipes.

Requirements with no source line

Requirement	Justification
Cache failures SHALL NOT propagate	Same as drive-plugin-cache. Cache is a perf optimisation, not a correctness boundary.
shouldCache default is 16 MiB	Same ceiling as drive-plugin-cache and drive-mesh-protocol's MAX_FRAME_BYTES. Avoids memory-pressure events when a large blob is read once.
clear() only clears the namespaced subset of the cache, not the whole cache	Two CachedStorage instances may share one AbstractCache with distinct namespaces (recipe in README); a wrapper.clear() call MUST NOT obliterate the other wrapper's entries. Implementation: AbstractCache doesn't currently expose "delete by prefix"; the spec mandates the behaviour and the implementation walks the cache via cache.keys() if available, falling back to a logged warning + skip when the cache backend can't enumerate.
Passthrough on exists, getMetadata, getWithMetadata, getSize, getItemSize	Listed in Out of Scope; the wrapper's value is on get/set/delete only.
Streaming passthrough	Same as drive-plugin-cache; cached streaming belongs at the URI tier.
Returns CachedStorage extends AbstractStorage (class, not pure function)	Matches NamespacedStorage precedent; lets consumers instanceof CachedStorage check; exposes getCacheStats() and the underlying cache reference on the same instance.
Wrapper's own error registry is 2 codes only	Almost every failure mode comes from the inner storage or the cache, both of which carry namespaced codes that pass through unchanged. The wrapper's surface is small by design.

Verification approach

Every requirement in the spec delta carries a **Verified by:** line referencing concrete planned it(...) titles in these files:

• src/cached-storage/cached-storage.test.ts — constructor wires the inner reference, getCacheStats() returns the snapshot, instanceof AbstractStorage and instanceof CachedStorage both hold.
• src/cached-storage/cached-storage.get.test.ts — hit short-circuits inner, miss populates, namespace used in cache key, shouldCache predicate gates population, default 16 MiB ceiling, inner.get returning undefined is NOT cached, cache.get rejection falls through to inner.
• src/cached-storage/cached-storage.write.test.ts — set invalidates post-success, set failure does not invalidate, setPartial invalidates, invalidateOnWrite:false skips, cache.delete rejection swallowed.
• src/cached-storage/cached-storage.delete.test.ts — delete invalidates post-success when inner returns true, propagates false, failure does not invalidate, invalidateOnDelete:false skips.
• src/cached-storage/cached-storage.clear.test.ts — clear invokes inner first, then iterates namespaced cache entries, invalidateOnClear:false skips, cache without keys() enumeration logs a warning and continues.
• src/cached-storage/cached-storage.passthrough.test.ts — exists / list / getSize / getMetadata / getWithMetadata / getItemSize / getPartial / setPartial / createReadStream / createWriteStream all delegate to inner without touching the cache.
• src/cached-storage/cached-storage.signal.test.ts — pre-aborted signal skips cache work and forwards to inner; live signal threaded into inner and (when accepted) into cache.
• src/factory/factory.test.ts — cachedStorage(inner, options) returns a CachedStorage, throws INVALID_OPTIONS on missing cache, throws INVALID_OPTIONS when cache lacks get/set/delete.
• src/error-codes/error-codes.test.ts — exhaustive STORAGE_CACHE_CODES registry, every code reachable, discriminants populated.
• src/otel/otel.test.ts — hit / miss / populate / invalidate / error counters fire, entry_bytes histogram observed on populations, span outcome attribute correct.
• tests/integration/memory-in-front-of-http.test.ts — real MemoryCache + real storage-http (via in-process fetch stub): two reads of the same key hit inner once, cache hit on the second; verifies the cache wrapper composes structurally with a real remote storage.
• tests/integration/file-cache-survives-restart.test.ts — real StorageCache over real storage-file + real in-memory inner storage: populate, drop the CachedStorage reference, build a new one against the same file-backed cache, assert hit on first read.
• tests/integration/namespaced-isolation.test.ts — two CachedStorage instances over distinct inner storages, sharing one AbstractCache with distinct namespaces; neither sees the other's entries; wrapper-a.clear() does not affect wrapper-b's cache entries.

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/storage-plugin-cache to enumerate. The package is new — no dependents currently. After adoption, every consumer holding an AbstractStorage reference (drive, storage-uri plugins, application code that takes a StorageLike) becomes a candidate to wrap with cachedStorage(inner, options) — no API change at the consumer.

• No dependents (verified by bunx turbo ls --filter=...@suaveplan/storage-plugin-cache returning the package itself).

Affected upstreams (none modified)

Package	Action	Breaking?
@suaveplan/storage	None — AbstractStorage extended in the canonical way (matches NamespacedStorage precedent)	no
@suaveplan/cache	None — AbstractCache<Uint8Array> consumed as-is	no
@suaveplan/cache-storage	None — used only as a devDep for the file-backed integration test	no
@suaveplan/storage-file	None — used only as a devDep for the file-backed integration test	no
@suaveplan/storage-http	None — used only as a devDep for the cross-storage integration test	no

storage-plugin-cache MUST NOT silently grow any of these packages' contracts. If a needed extension turns up during implementation (e.g. AbstractCache.keys() enumeration is needed for clear() and not all backends expose it), the spec MUST document the fallback and capture a follow-up proposal against @suaveplan/cache — never bend the contract here.

Ordering

This change is unblocked: @suaveplan/storage and the @suaveplan/cache* family both ship today on genesis with the consumed surface intact. May ship before, alongside, or after add-drive-plugin-cache — the two are independent.