Initial Commit
This commit is contained in:
348
openspec/archive/2026-05-16-add-storage-plugin-cache/design.md
Normal file
348
openspec/archive/2026-05-16-add-storage-plugin-cache/design.md
Normal file
@@ -0,0 +1,348 @@
|
||||
# Design — @suaveplan/storage-plugin-cache
|
||||
|
||||
## Tier and dependency posture
|
||||
|
||||
Universal tier. Zero `node:` / `bun:` imports in `src/`. Runtime deps:
|
||||
|
||||
- `@suaveplan/error` — `defineCodes`, `SuaveplanError`
|
||||
- `@suaveplan/storage` — `AbstractStorage` (extended), associated types (`StorageMetadata`, `StorageGetMetadataResult`, `StorageListResult`, etc.)
|
||||
- `@suaveplan/cache` — `AbstractCache<T>` (type-only consumption; the consumer passes a concrete instance)
|
||||
- `@suaveplan/types` — `LoggerLike` (optional injectable)
|
||||
- `@opentelemetry/api` — meter + tracer
|
||||
- `tslib`
|
||||
|
||||
`@suaveplan/cache-storage`, `cache-redis`, `storage-file`, `storage-http`, `storage-browser-*` are **NOT** runtime deps. The consumer wires their preferred cache backend and inner storage before passing them to `cachedStorage(inner, options)`. Same injection posture as `drive-uri` and `drive-plugin-cache` — the wrapper sees interfaces, not concrete providers.
|
||||
|
||||
DevDeps: `@suaveplan/testing`, `@suaveplan/storage-file`, `@suaveplan/storage-http`, `@suaveplan/cache-storage`, `@vitest/coverage-istanbul`, `fast-check`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/storage/storage-plugin-cache/
|
||||
package.json
|
||||
README.md
|
||||
src/
|
||||
index.ts # explicit named re-exports (rule 10)
|
||||
error-codes/
|
||||
error-codes.ts # defineCodes("STORAGE_CACHE", [...]), CachedStorageError
|
||||
error-codes.test.ts
|
||||
error-codes.md
|
||||
index.ts
|
||||
types/
|
||||
options.types.ts # CachedStorageOptions, CachedStorageStats
|
||||
options.types.md
|
||||
index.ts
|
||||
cached-storage/
|
||||
cached-storage.ts # class CachedStorage extends AbstractStorage
|
||||
cached-storage.test.ts # construct, instanceof, getCacheStats
|
||||
cached-storage.get.ts # get() read-through helper (kept ≤300 lines)
|
||||
cached-storage.get.test.ts
|
||||
cached-storage.write.ts # set() / setPartial() invalidation helper
|
||||
cached-storage.write.test.ts
|
||||
cached-storage.delete.ts # delete() invalidation helper
|
||||
cached-storage.delete.test.ts
|
||||
cached-storage.clear.ts # clear() namespaced-iteration helper
|
||||
cached-storage.clear.test.ts
|
||||
cached-storage.passthrough.ts # exists/list/getSize/getMetadata/streams delegates
|
||||
cached-storage.passthrough.test.ts
|
||||
cached-storage.signal.ts # pre-aborted signal short-circuit helper
|
||||
cached-storage.signal.test.ts
|
||||
cached-storage.md
|
||||
cached-storage.get.md
|
||||
cached-storage.write.md
|
||||
cached-storage.delete.md
|
||||
cached-storage.clear.md
|
||||
cached-storage.passthrough.md
|
||||
cached-storage.signal.md
|
||||
index.ts
|
||||
factory/
|
||||
factory.ts # cachedStorage(inner, options): CachedStorage
|
||||
factory.test.ts
|
||||
factory.md
|
||||
index.ts
|
||||
otel/
|
||||
otel.ts # meter + tracer wiring, recordOp(...)
|
||||
otel.test.ts
|
||||
otel.md
|
||||
index.ts
|
||||
tests/
|
||||
setup/polyfills.ts
|
||||
integration/
|
||||
memory-in-front-of-http.test.ts # real MemoryCache + real storage-http (in-process fetch)
|
||||
file-cache-survives-restart.test.ts # real StorageCache over real storage-file
|
||||
namespaced-isolation.test.ts # two wrappers sharing one cache, distinct namespaces
|
||||
tsconfig.json
|
||||
tsconfig.lib.json
|
||||
vite.config.ts
|
||||
vitest.config.ts
|
||||
```
|
||||
|
||||
The `cached-storage/` feature folder is split into one-concern files (`get`, `write`, `delete`, `clear`, `passthrough`, `signal`) to stay well under the 300-line hard limit while keeping each behaviour next to its tests + docs. Every implementation `.ts` has a sibling `.md`. The folder pattern mirrors `storage-uri`'s `resolver/` split.
|
||||
|
||||
Per AGENTS §10: parent `cached-storage/` with un-suffixed children. No `-plugin-` suffix because the children are not nested plugins (the package is named `storage-plugin-cache` because it functions as a storage-tier cache plugin — the noun "plugin" appears in the package name only, not in any child folder).
|
||||
|
||||
## Class shape
|
||||
|
||||
```ts
|
||||
export class CachedStorage extends AbstractStorage {
|
||||
readonly #inner: AbstractStorage;
|
||||
readonly #cache: AbstractCache<Uint8Array>;
|
||||
readonly #options: ResolvedCachedStorageOptions;
|
||||
readonly #stats: MutableStats = { hits: 0, misses: 0, populations: 0, invalidations: 0, errors: 0 };
|
||||
|
||||
public constructor(inner: AbstractStorage, options: CachedStorageOptions) {
|
||||
super(/* eventBus from inner is intentionally NOT forwarded — events are observed on the inner */);
|
||||
if (!inner) throw new CachedStorageError("INNER_MISSING", "inner storage is required");
|
||||
validateOptions(options);
|
||||
this.#inner = inner;
|
||||
this.#cache = options.cache;
|
||||
this.#options = resolveDefaults(options);
|
||||
}
|
||||
|
||||
public get cache(): AbstractCache<Uint8Array> { return this.#cache; }
|
||||
public get inner(): AbstractStorage { return this.#inner; }
|
||||
|
||||
public getCacheStats(): CachedStorageStats { return { ...this.#stats }; }
|
||||
|
||||
async get(key: string, signal?: AbortSignal): Promise<Uint8Array | undefined> { /* read-through */ }
|
||||
async set(key: string, value: Uint8Array, signal?: AbortSignal): Promise<void> { /* delegate + invalidate */ }
|
||||
async delete(key: string, signal?: AbortSignal): Promise<boolean> { /* delegate + invalidate */ }
|
||||
// … rest of AbstractStorage's 15 abstract methods …
|
||||
}
|
||||
|
||||
export function cachedStorage(
|
||||
inner: AbstractStorage,
|
||||
options: CachedStorageOptions
|
||||
): CachedStorage {
|
||||
return new CachedStorage(inner, options);
|
||||
}
|
||||
```
|
||||
|
||||
The class exposes both `cache` (the underlying `AbstractCache` for `wrap()` / `getStats()` / direct manipulation) and `inner` (the wrapped `AbstractStorage` for inspection / unwrap). The factory is a one-liner over `new CachedStorage(...)` so consumers can write either form; the factory is what `validateOptions` lives in (synchronous throw on missing cache before the class constructor runs).
|
||||
|
||||
## Read-through (`get`)
|
||||
|
||||
```ts
|
||||
async get(key: string, signal?: AbortSignal): Promise<Uint8Array | undefined> {
|
||||
if (signal?.aborted) return this.#inner.get(key, signal); // let inner raise canonical AbortError
|
||||
|
||||
const cacheKey = `${this.#options.namespace}::${key}`;
|
||||
|
||||
let cached: Uint8Array | undefined;
|
||||
try {
|
||||
cached = await this.#cache.get(cacheKey);
|
||||
} catch (err) {
|
||||
this.#stats.errors += 1;
|
||||
recordError("get", err, this.#options.namespace);
|
||||
// fall through — cache failure must not block the read
|
||||
}
|
||||
if (cached !== undefined) {
|
||||
this.#stats.hits += 1;
|
||||
recordHit(this.#options.namespace, key);
|
||||
return cached;
|
||||
}
|
||||
|
||||
this.#stats.misses += 1;
|
||||
recordMiss(this.#options.namespace, key);
|
||||
const bytes = await this.#inner.get(key, signal);
|
||||
if (bytes === undefined) return undefined; // negative results NOT cached
|
||||
|
||||
if (this.#options.shouldCache(key, bytes)) {
|
||||
try {
|
||||
await this.#cache.set(cacheKey, bytes);
|
||||
this.#stats.populations += 1;
|
||||
recordPopulate(this.#options.namespace, key, bytes.byteLength);
|
||||
} catch (err) {
|
||||
this.#stats.errors += 1;
|
||||
recordError("set", err, this.#options.namespace);
|
||||
// swallow — same rule as drive-plugin-cache
|
||||
}
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
```
|
||||
|
||||
## Invalidation (`set` / `setPartial` / `delete` / `clear`)
|
||||
|
||||
```ts
|
||||
async set(key: string, value: Uint8Array, signal?: AbortSignal): Promise<void> {
|
||||
await this.#inner.set(key, value, signal);
|
||||
if (this.#options.invalidateOnWrite) {
|
||||
await this.#invalidateKey(key);
|
||||
}
|
||||
}
|
||||
|
||||
async setPartial(key: string, offset: number, value: Uint8Array, signal?: AbortSignal): Promise<void> {
|
||||
await this.#inner.setPartial(key, offset, value, signal);
|
||||
if (this.#options.invalidateOnWrite) {
|
||||
await this.#invalidateKey(key); // partial writes invalidate the whole entry — no partial caching
|
||||
}
|
||||
}
|
||||
|
||||
async delete(key: string, signal?: AbortSignal): Promise<boolean> {
|
||||
const removed = await this.#inner.delete(key, signal);
|
||||
if (this.#options.invalidateOnDelete) {
|
||||
await this.#invalidateKey(key); // delete even on inner.delete === false to clear stale entries
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
|
||||
async clear(signal?: AbortSignal): Promise<void> {
|
||||
await this.#inner.clear(signal);
|
||||
if (!this.#options.invalidateOnClear) return;
|
||||
|
||||
// Iterate this wrapper's namespaced subset only — never wipe other wrappers' entries.
|
||||
if (typeof this.#cache.keys === "function") {
|
||||
try {
|
||||
const prefix = `${this.#options.namespace}::`;
|
||||
for await (const k of this.#cache.keys()) {
|
||||
if (k.startsWith(prefix)) await this.#cache.delete(k);
|
||||
this.#stats.invalidations += 1;
|
||||
}
|
||||
} catch (err) {
|
||||
this.#stats.errors += 1;
|
||||
recordError("clear", err, this.#options.namespace);
|
||||
}
|
||||
} else {
|
||||
// AbstractCache without enumeration: log warning, skip. Consumer can call cache.clear()
|
||||
// directly if they accept the wider blast radius.
|
||||
this.#options.logger?.warn?.({
|
||||
msg: "[storage-cache] cache backend has no keys() — clear() did not evict cache entries",
|
||||
namespace: this.#options.namespace,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async #invalidateKey(key: string): Promise<void> {
|
||||
const cacheKey = `${this.#options.namespace}::${key}`;
|
||||
try {
|
||||
await this.#cache.delete(cacheKey);
|
||||
this.#stats.invalidations += 1;
|
||||
recordInvalidate(this.#options.namespace, key);
|
||||
} catch (err) {
|
||||
this.#stats.errors += 1;
|
||||
recordError("delete", err, this.#options.namespace);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Post-success invalidation matches `drive-plugin-cache`'s rationale — pre-`next()` invalidation opens a race window where a concurrent read repopulates the cache with stale bytes during the write.
|
||||
|
||||
## Passthrough surface
|
||||
|
||||
```ts
|
||||
exists (key, signal?) → inner.exists(key, signal)
|
||||
list (opts?, signal?) → inner.list(opts, signal)
|
||||
clear // handled above with invalidation
|
||||
getWithMetadata(key, signal?) → inner.getWithMetadata(key, signal)
|
||||
getMetadata (key, signal?) → inner.getMetadata(key, signal)
|
||||
getSize (signal?) → inner.getSize(signal)
|
||||
getItemSize (key, signal?) → inner.getItemSize(key, signal)
|
||||
getPartial (key, offset, length, sig?) → inner.getPartial(...)
|
||||
createReadStream (key, signal?) → inner.createReadStream(key, signal)
|
||||
createWriteStream (key, signal?) → inner.createWriteStream(key, signal)
|
||||
```
|
||||
|
||||
These do NOT touch the cache. The cache surface is reads-of-whole-objects + writes-invalidate-whole-objects. Anything that wants cached metadata or cached streaming composes at a different tier (drive-plugin-cache for plaintext, future cacheUriPlugin for URI-resolver results, `storage-http`'s `EtagCache` for HTTP 304s).
|
||||
|
||||
## Error-code map
|
||||
|
||||
| Code | When | Notes |
|
||||
|---|---|---|
|
||||
| `STORAGE_CACHE.INVALID_OPTIONS` | Constructor or factory called without a valid `cache` (missing, wrong shape, missing `get`/`set`/`delete`) | Synchronous throw |
|
||||
| `STORAGE_CACHE.INNER_MISSING` | Constructor called with a nullish `inner` | Synchronous throw |
|
||||
|
||||
Every other failure mode (storage errors, cache errors) surfaces with its original code through the wrapper. The wrapper's own registry is intentionally narrow — there's no value in re-namespacing downstream errors.
|
||||
|
||||
## OTel surface
|
||||
|
||||
Tracer + meter: `@suaveplan/storage-cache`.
|
||||
|
||||
Histograms:
|
||||
|
||||
- `storage_cache.entry_bytes{namespace}` — distribution of cached entry sizes. Sampled on `populations_total` increments.
|
||||
|
||||
Counters:
|
||||
|
||||
- `storage_cache.hits_total{namespace}`
|
||||
- `storage_cache.misses_total{namespace}`
|
||||
- `storage_cache.populations_total{namespace}`
|
||||
- `storage_cache.invalidations_total{namespace}`
|
||||
- `storage_cache.errors_total{namespace, op}` — `op ∈ get | set | delete | clear`
|
||||
|
||||
Spans: one per public op, attributes `storage.cache.namespace`, `storage.cache.key` (or `*` for `clear`), `storage.cache.outcome ∈ {hit, miss, populated, invalidated, error}`. Keys are not URLs; no query-string redaction needed.
|
||||
|
||||
## Trade-offs accepted
|
||||
|
||||
- **No range-byte caching.** `setPartial(key, offset, bytes)` invalidates the whole entry. Avoids the overlapping-range invalidation puzzle. Consumers with random-access read workloads should compose differently (see Out of Scope).
|
||||
- **No streaming-result caching.** `createReadStream` / `createWriteStream` pass through. Cached streaming lives at the URI tier.
|
||||
- **`clear()` scope is namespace-local when the cache exposes `keys()`, otherwise no-op with a warning.** Two wrappers sharing one cache MUST NOT obliterate each other's entries. The fallback (no `keys()`) is conservative; the spec mandates the warning so operators see why their `clear()` was a no-op.
|
||||
- **Negative results not cached.** `inner.get` returning `undefined` is always a fresh check. Add later if a real workload requires it (needs a sentinel value and a separate TTL).
|
||||
- **`getStats()` (inherited from `AbstractStorage`) vs. `getCacheStats()` (new on `CachedStorage`).** The inherited stats count the *inner* storage's operations as observed through the wrapper; the new `getCacheStats()` counts the cache layer's behaviour. Two distinct surfaces; documented in the README.
|
||||
- **One `AbstractCache` per wrapper.** Two-tier caches (memory in front of OPFS in front of a remote storage) are out of scope. Compose by stacking two `cachedStorage(...)` wrappers if you really need that.
|
||||
- **`inner` and `cache` exposed as readonly properties.** Lets the consumer inspect both and bypass the wrapper deliberately (e.g. for `cache.wrap()` stale-while-revalidate). Same shape as `drive-plugin-cache`'s `handle.cache`.
|
||||
|
||||
## Composition recipes (in README)
|
||||
|
||||
```ts
|
||||
// 1. Memory cache in front of S3 (server, hot keys)
|
||||
import { S3Storage } from "@suaveplan/storage-s3";
|
||||
import { createCache } from "@suaveplan/cache";
|
||||
import { cachedStorage } from "@suaveplan/storage-plugin-cache";
|
||||
|
||||
const inner = new S3Storage({ bucket: "media", /* … */ });
|
||||
const cache = createCache("memory", { policy: "lru", maxItems: 1_000 });
|
||||
const storage = cachedStorage(inner, { cache, namespace: "media" });
|
||||
|
||||
// Drop straight into anything that takes an AbstractStorage:
|
||||
const drive = new Drive({ defaultStorage: storage });
|
||||
|
||||
// 2. OPFS cache in front of HTTPS (browser, cross-session)
|
||||
import { createOpfsStorage } from "@suaveplan/storage-browser-opfs";
|
||||
import { createHttpStorage } from "@suaveplan/storage-http";
|
||||
|
||||
const inner = createHttpStorage({ baseUrl: "https://cdn.example.com/start/here/" });
|
||||
const cache = createCache("storage", {
|
||||
storage: await createOpfsStorage("http-cache"),
|
||||
policy: "lru",
|
||||
maxBytes: 256 * 1024 * 1024,
|
||||
});
|
||||
const storage = cachedStorage(inner, { cache });
|
||||
|
||||
// 3. File cache in front of Bunny CDN (server, survives process restart)
|
||||
import { createBunnyStorage } from "@suaveplan/storage-cdn-bunnycdn";
|
||||
import { createFileStorage } from "@suaveplan/storage-file";
|
||||
|
||||
const inner = createBunnyStorage({ /* … */ });
|
||||
const cache = createCache("storage", {
|
||||
storage: createFileStorage({ rootDir: "/var/cache/bunny" }),
|
||||
policy: "ttl",
|
||||
ttlMs: 3_600_000,
|
||||
});
|
||||
const storage = cachedStorage(inner, { cache, namespace: "bunny-prod" });
|
||||
|
||||
// 4. Redis cache in front of S3 (server, cross-process shared)
|
||||
import { createRedisCache } from "@suaveplan/cache-redis";
|
||||
|
||||
const inner = new S3Storage({ /* … */ });
|
||||
const cache = createRedisCache({ client, prefix: "media-cache:" });
|
||||
const storage = cachedStorage(inner, { cache, namespace: "media" });
|
||||
|
||||
// 5. Stale-while-revalidate via cache.wrap()
|
||||
const storage = cachedStorage(inner, { cache });
|
||||
const bytes = await storage.cache.wrap(`media::clip.mov`, () => inner.get("clip.mov"), {
|
||||
staleAge: 60_000,
|
||||
refreshAhead: true,
|
||||
});
|
||||
|
||||
// 6. Composed with drive-uri / storage-uri — the cache wrapper is just another StorageLike
|
||||
import { createStorageUriResolver, fileUriPlugin } from "@suaveplan/storage-uri";
|
||||
|
||||
const cachedFile = cachedStorage(createFileStorage({ rootDir: "/var/media" }), { cache });
|
||||
const resolver = createStorageUriResolver([
|
||||
fileUriPlugin({ rootDir: "/var/media", storage: cachedFile }),
|
||||
// … other plugins …
|
||||
]);
|
||||
```
|
||||
|
||||
Recipe 6 illustrates the structural payoff: anything that took `AbstractStorage` before takes `CachedStorage` now, with zero adjustment, including `storage-uri` plugins. The cache wrapper composes uniformly across the ecosystem.
|
||||
114
openspec/archive/2026-05-16-add-storage-plugin-cache/proposal.md
Normal file
114
openspec/archive/2026-05-16-add-storage-plugin-cache/proposal.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Add @suaveplan/storage-plugin-cache — read-through cache decorator for AbstractStorage
|
||||
|
||||
**Package**: `@suaveplan/storage-plugin-cache`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (genesis-original)
|
||||
**Depends on**: `@suaveplan/storage` (archived), `@suaveplan/cache` family (archived as `add-cache-storage-redis-backends` on 2026-05-16)
|
||||
**Sibling**: `add-drive-plugin-cache` (this change targets the storage tier; the drive-tier twin targets `Drive.read(path)`)
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/storage-plugin-cache/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
The cache family on genesis (`@suaveplan/cache`, `cache-storage`, `cache-redis`, built-in `MemoryCache`) provides one unified abstraction — `AbstractCache<T>` with `createCache("memory" | "storage" | "redis", config)` — and `@suaveplan/cache-storage` lets any `AbstractStorage` become a *cache backend* (storage-as-cache). The *inverse* surface is missing: there is no decorator that puts a cache *in front of* an `AbstractStorage`. Concretely, a consumer holding a slow remote storage (`storage-http`, `storage-s3`, `storage-cdn-rackspace`, `storage-cdn-bunnycdn`) today cannot wrap it with a fast local cache (`MemoryCache` for hot keys, `StorageCache` over OPFS/file for cross-process durability) without hand-rolling the read-through / invalidate-on-write logic at every call site.
|
||||
|
||||
`storage-plugin-cache` is the inverse of `cache-storage`: it takes any `AbstractStorage` and returns another `AbstractStorage` that consults a caller-supplied `AbstractCache<Uint8Array>` first on every `get`, falls through to the inner storage on miss, populates on success, and invalidates the cache key on every `set`/`setPartial`/`delete`/`clear`. The result is structurally identical to `NamespacedStorage` — a decorator class extending `AbstractStorage` that delegates to an inner instance — so every consumer in the ecosystem that accepts a `StorageLike` or `AbstractStorage` (drive, storage-uri plugins, application code) takes the cached version with zero adjustment.
|
||||
|
||||
This is the structural twin of `add-drive-plugin-cache`: same `AbstractCache<Uint8Array>` injection, same error-swallow rules, same OTel surface shape, same `shouldCache` / `maxEntryBytes` defaults. Where `drive-plugin-cache` caches *reassembled plaintext* at the drive middleware tier, `storage-plugin-cache` caches *raw stored bytes* at the storage tier — the most fundamental of the three planned cache decorators (the third being a future `cacheUriPlugin` over the URI resolver).
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `CachedStorage extends AbstractStorage` — decorator class that wraps an inner `AbstractStorage` and intercepts every abstract method. Constructor takes the inner storage and a `CachedStorageOptions` bag.
|
||||
- ADDED: `cachedStorage(inner, options): CachedStorage` factory — convenience over `new CachedStorage(inner, options)` matching the `createCache` / `createHttpStorage` family conventions. Validates options synchronously and throws `STORAGE_CACHE.INVALID_OPTIONS` on missing/malformed `cache`.
|
||||
- ADDED: `CachedStorageOptions` interface — `{ cache: AbstractCache<Uint8Array>; invalidateOnWrite?: boolean; invalidateOnDelete?: boolean; invalidateOnClear?: boolean; namespace?: string; shouldCache?: (key: string, bytes: Uint8Array) => boolean; maxEntryBytes?: number; logger?: LoggerLike }`.
|
||||
- ADDED: Read-through `get(key, signal?)`:
|
||||
1. Compute cache key as `${namespace ?? "storage"}::${key}`.
|
||||
2. `await cache.get(cacheKey)`. If hit, return bytes; increment `storage_cache.hits_total{namespace}`.
|
||||
3. On miss, `await inner.get(key, signal)`. Increment `storage_cache.misses_total{namespace}`.
|
||||
4. When the inner returns a `Uint8Array` AND `shouldCache(key, bytes)` returns true, `await cache.set(cacheKey, bytes)`. Increment `storage_cache.populations_total{namespace}`.
|
||||
5. Cache `get`/`set` failures SHALL be swallowed (logged, counter-incremented) — a cache failure MUST NOT turn a successful `inner.get` into a thrown error.
|
||||
- ADDED: Invalidation surface. On `set(key, value, signal?)`, the wrapper SHALL call `inner.set(key, value, signal)` first; on success, when `invalidateOnWrite !== false`, `await cache.delete(cacheKey)`. Symmetric on `setPartial(...)`, `delete(key, signal?)` (when `invalidateOnDelete !== false`), and `clear(signal?)` (when `invalidateOnClear !== false`, the wrapper calls `cache.clear()` to drop every namespaced entry; the spec MUST document the scope as "every entry under this wrapper's namespace, not the entire cache"). Cache `delete`/`clear` failures SHALL NOT propagate.
|
||||
- ADDED: `getWithMetadata`, `getMetadata`, `getSize`, `getItemSize`, `exists`, `list` — these all delegate to the inner storage unchanged. `getWithMetadata` does NOT populate the cache (metadata mutation invalidation is out of scope); `exists` is cheap on every backend we ship, so caching it would add invalidation surface without meaningful win. The wrapper SHALL document these as "passthrough".
|
||||
- ADDED: `getPartial` and `setPartial` — passthrough to inner. Partial-byte caching is out of scope (range invalidation across overlapping writes is a separate design problem).
|
||||
- ADDED: `createReadStream` and `createWriteStream` — passthrough to inner. Streaming bypasses the buffered cache by design; a `cacheUriPlugin` (planned) will cover cached streaming at the URI tier.
|
||||
- ADDED: `STORAGE_CACHE_CODES` registry via `defineCodes("STORAGE_CACHE", [...] as const)` — 2 codes: `INVALID_OPTIONS`, `INNER_MISSING`. The wrapper's own failures are deliberately few — everything else surfaces from the inner storage (`STORAGE_HTTP.*`, `STORAGE.*`, etc.) or the cache (`CACHE.*`) unchanged.
|
||||
- ADDED: `CachedStorageError extends SuaveplanError<CachedStorageErrorCode>` with `key?` and `namespace?` discriminants.
|
||||
- ADDED: `CachedStorageStats` exposed via the class's `getCacheStats()` method (distinct from `AbstractStorage.getStats()` which counts the *inner* storage's operations) — `{ hits, misses, populations, invalidations, errors }`.
|
||||
- ADDED: OTel surface — `storage_cache.{hits, misses, populations, invalidations, errors}_total{namespace}` counters; `storage_cache.entry_bytes{namespace}` histogram observing populations; spans on `@suaveplan/storage-cache` tracer with `storage.cache.namespace`, `storage.cache.key`, `storage.cache.outcome ∈ {hit, miss, populated, invalidated, error}` attributes.
|
||||
- ADDED: README composition recipes — memory cache in front of S3, OPFS in front of HTTPS, file cache in front of CDN, Redis cache in front of S3 (multi-process), namespaced isolation across multiple wrappers sharing one underlying cache, `cache.wrap()`-based stale-while-revalidate via the exposed `cache` reference.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Range-byte caching.** `getPartial`/`setPartial` pass through. Range invalidation (a `setPartial(key, offset=10, bytes)` requires evicting any cached byte slice that overlaps `[10, 10+len]`) is non-trivial and rare in practice. Consumers needing fast random-access reads should cache at the URI tier or use a content-addressed scheme.
|
||||
- **Streaming caching.** `createReadStream`/`createWriteStream` pass through. Same reason as the drive-plugin-cache: cached streaming belongs at the URI layer with the planned `cacheUriPlugin`, where the resolver owns the stream surface.
|
||||
- **List-result caching.** Same problem as in drive-plugin-cache: list invalidation across prefix-mutating writes is hard; consumers needing fast `list` should compose a higher-level index or use a content-addressed storage.
|
||||
- **Cache stampede protection.** If two concurrent `get(k)` calls both miss the cache, both will call `inner.get(k)` and both will populate. Single-flight / lock-and-load belongs in `AbstractCache.wrap(...)` (which already provides it). Consumers needing stampede protection should compose at the cache tier via `handle.cache.wrap()` rather than at the storage decorator.
|
||||
- **`exists` caching.** Cheap on every shipped backend; caching adds an invalidation surface without a meaningful latency win. Out of scope.
|
||||
- **Negative caching.** `inner.get` returning `undefined` (key absent) is NOT cached. Caching absence requires a sentinel value and a separate TTL knob; can be added later if a real workload needs it. For now, `undefined` is always a fresh read.
|
||||
- **Metadata caching (`getMetadata`, `getWithMetadata`).** Passthrough. ETag-based conditional reads already live in `storage-http`'s `EtagCache`; bringing them up here would duplicate that layer.
|
||||
- **`storage-uri`-tier `cacheUriPlugin`.** Separate proposal (sibling). Wraps any `UriPlugin` with caching across URI schemes.
|
||||
- **`drive-plugin-cache` parity.** Separate sibling proposal (`add-drive-plugin-cache`). Caches `Drive.read(path)` plaintext at the drive middleware tier.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This is a **genesis-original** package; no upstream `suaveplan/` reference. Requirements derive from:
|
||||
|
||||
- The `AbstractStorage` abstract class in `@suaveplan/storage` (`openspec/specs/storage/spec.md`) — every abstract method (`get`/`set`/`delete`/`exists`/`list`/`clear`/`getWithMetadata`/`getMetadata`/`getSize`/`getItemSize`/`getPartial`/`setPartial`/`createReadStream`/`createWriteStream`) must be implemented in `CachedStorage` with delegation semantics.
|
||||
- The `NamespacedStorage` decorator precedent in `@suaveplan/storage` (`src/abstract/abstract.ts:559`) — same structural pattern: wrap inner, prefix keys, delegate operations.
|
||||
- The `AbstractCache<T>` contract in `@suaveplan/cache`.
|
||||
- The conventions established by `add-drive-plugin-cache` (sibling) — identical option shape, error-swallow rules, OTel naming, recipes.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Cache failures SHALL NOT propagate | Same as drive-plugin-cache. Cache is a perf optimisation, not a correctness boundary. |
|
||||
| `shouldCache` default is 16 MiB | Same ceiling as `drive-plugin-cache` and `drive-mesh-protocol`'s `MAX_FRAME_BYTES`. Avoids memory-pressure events when a large blob is read once. |
|
||||
| `clear()` only clears the namespaced subset of the cache, not the whole cache | Two `CachedStorage` instances may share one `AbstractCache` with distinct namespaces (recipe in README); a `wrapper.clear()` call MUST NOT obliterate the other wrapper's entries. Implementation: `AbstractCache` doesn't currently expose "delete by prefix"; the spec mandates the behaviour and the implementation walks the cache via `cache.keys()` if available, falling back to a logged warning + skip when the cache backend can't enumerate. |
|
||||
| Passthrough on `exists`, `getMetadata`, `getWithMetadata`, `getSize`, `getItemSize` | Listed in Out of Scope; the wrapper's value is on `get`/`set`/`delete` only. |
|
||||
| Streaming passthrough | Same as drive-plugin-cache; cached streaming belongs at the URI tier. |
|
||||
| Returns `CachedStorage extends AbstractStorage` (class, not pure function) | Matches `NamespacedStorage` precedent; lets consumers `instanceof CachedStorage` check; exposes `getCacheStats()` and the underlying `cache` reference on the same instance. |
|
||||
| Wrapper's own error registry is 2 codes only | Almost every failure mode comes from the inner storage or the cache, both of which carry namespaced codes that pass through unchanged. The wrapper's surface is small by design. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete planned `it(...)` titles in these files:
|
||||
|
||||
- `src/cached-storage/cached-storage.test.ts` — constructor wires the inner reference, `getCacheStats()` returns the snapshot, `instanceof AbstractStorage` and `instanceof CachedStorage` both hold.
|
||||
- `src/cached-storage/cached-storage.get.test.ts` — hit short-circuits inner, miss populates, namespace used in cache key, `shouldCache` predicate gates population, default 16 MiB ceiling, `inner.get` returning `undefined` is NOT cached, cache.get rejection falls through to inner.
|
||||
- `src/cached-storage/cached-storage.write.test.ts` — `set` invalidates post-success, `set` failure does not invalidate, `setPartial` invalidates, `invalidateOnWrite:false` skips, cache.delete rejection swallowed.
|
||||
- `src/cached-storage/cached-storage.delete.test.ts` — `delete` invalidates post-success when inner returns `true`, propagates `false`, failure does not invalidate, `invalidateOnDelete:false` skips.
|
||||
- `src/cached-storage/cached-storage.clear.test.ts` — `clear` invokes inner first, then iterates namespaced cache entries, `invalidateOnClear:false` skips, cache without `keys()` enumeration logs a warning and continues.
|
||||
- `src/cached-storage/cached-storage.passthrough.test.ts` — `exists` / `list` / `getSize` / `getMetadata` / `getWithMetadata` / `getItemSize` / `getPartial` / `setPartial` / `createReadStream` / `createWriteStream` all delegate to inner without touching the cache.
|
||||
- `src/cached-storage/cached-storage.signal.test.ts` — pre-aborted signal skips cache work and forwards to inner; live signal threaded into inner and (when accepted) into cache.
|
||||
- `src/factory/factory.test.ts` — `cachedStorage(inner, options)` returns a `CachedStorage`, throws `INVALID_OPTIONS` on missing cache, throws `INVALID_OPTIONS` when cache lacks `get`/`set`/`delete`.
|
||||
- `src/error-codes/error-codes.test.ts` — exhaustive `STORAGE_CACHE_CODES` registry, every code reachable, discriminants populated.
|
||||
- `src/otel/otel.test.ts` — hit / miss / populate / invalidate / error counters fire, `entry_bytes` histogram observed on populations, span outcome attribute correct.
|
||||
- `tests/integration/memory-in-front-of-http.test.ts` — real `MemoryCache` + real `storage-http` (via in-process fetch stub): two reads of the same key hit inner once, cache hit on the second; verifies the cache wrapper composes structurally with a real remote storage.
|
||||
- `tests/integration/file-cache-survives-restart.test.ts` — real `StorageCache` over real `storage-file` + real in-memory inner storage: populate, drop the `CachedStorage` reference, build a new one against the same file-backed cache, assert hit on first read.
|
||||
- `tests/integration/namespaced-isolation.test.ts` — two `CachedStorage` instances over distinct inner storages, sharing one `AbstractCache` with distinct namespaces; neither sees the other's entries; `wrapper-a.clear()` does not affect `wrapper-b`'s cache entries.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/storage-plugin-cache` to enumerate. The package is new — no dependents currently. After adoption, every consumer holding an `AbstractStorage` reference (drive, storage-uri plugins, application code that takes a `StorageLike`) becomes a candidate to wrap with `cachedStorage(inner, options)` — no API change at the consumer.
|
||||
|
||||
- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-plugin-cache` returning the package itself).
|
||||
|
||||
### Affected upstreams (none modified)
|
||||
|
||||
| Package | Action | Breaking? |
|
||||
|---|---|---|
|
||||
| `@suaveplan/storage` | None — `AbstractStorage` extended in the canonical way (matches `NamespacedStorage` precedent) | no |
|
||||
| `@suaveplan/cache` | None — `AbstractCache<Uint8Array>` consumed as-is | no |
|
||||
| `@suaveplan/cache-storage` | None — used only as a devDep for the file-backed integration test | no |
|
||||
| `@suaveplan/storage-file` | None — used only as a devDep for the file-backed integration test | no |
|
||||
| `@suaveplan/storage-http` | None — used only as a devDep for the cross-storage integration test | no |
|
||||
|
||||
`storage-plugin-cache` MUST NOT silently grow any of these packages' contracts. If a needed extension turns up during implementation (e.g. `AbstractCache.keys()` enumeration is needed for `clear()` and not all backends expose it), the spec MUST document the fallback and capture a follow-up proposal against `@suaveplan/cache` — never bend the contract here.
|
||||
|
||||
### Ordering
|
||||
|
||||
This change is unblocked: `@suaveplan/storage` and the `@suaveplan/cache*` family both ship today on `genesis` with the consumed surface intact. May ship before, alongside, or after `add-drive-plugin-cache` — the two are independent.
|
||||
@@ -0,0 +1,322 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: storage-plugin-cache (add-storage-plugin-cache)
|
||||
|
||||
**Package**: `@suaveplan/storage-plugin-cache`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `CachedStorage extends AbstractStorage` and `cachedStorage(inner, options)` factory
|
||||
|
||||
The package SHALL export a `CachedStorage` class that extends `AbstractStorage` from `@suaveplan/storage` and a `cachedStorage(inner, options): CachedStorage` factory. The class SHALL implement every abstract method on the base — `get`, `set`, `delete`, `exists`, `list`, `clear`, `getWithMetadata`, `getMetadata`, `getSize`, `getItemSize`, `getPartial`, `setPartial`, `createReadStream`, `createWriteStream` — and SHALL expose `readonly cache: AbstractCache<Uint8Array>`, `readonly inner: AbstractStorage`, and `getCacheStats(): CachedStorageStats`.
|
||||
|
||||
#### Scenario: instance is structurally an AbstractStorage
|
||||
|
||||
- **WHEN** `cachedStorage(innerStorage, { cache })` is called against a real `AbstractStorage` subclass
|
||||
- **THEN** the returned instance is `instanceof AbstractStorage` AND `instanceof CachedStorage` AND every abstract method on `AbstractStorage` is callable
|
||||
|
||||
#### Scenario: cache and inner are exposed as readonly properties
|
||||
|
||||
- **WHEN** `cachedStorage(innerStorage, { cache })` returns a `CachedStorage`
|
||||
- **THEN** `wrapper.cache === cache` (same reference) AND `wrapper.inner === innerStorage` (same reference)
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.test.ts::"instance is structurally an AbstractStorage and a CachedStorage"` AND `src/cached-storage/cached-storage.test.ts::"cache and inner are exposed as readonly properties"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Construction-time validation of required options
|
||||
|
||||
The `cachedStorage(inner, options)` factory and the `new CachedStorage(inner, options)` constructor SHALL throw `STORAGE_CACHE.INNER_MISSING` synchronously when `inner` is nullish, and `STORAGE_CACHE.INVALID_OPTIONS` synchronously when `options.cache` is missing or is not a function-typed `get`/`set`/`delete` carrier. The validation MUST NOT be deferred to first use.
|
||||
|
||||
#### Scenario: nullish inner throws INNER_MISSING
|
||||
|
||||
- **WHEN** `cachedStorage(null as unknown as AbstractStorage, { cache })` is called
|
||||
- **THEN** the call throws `CachedStorageError` with `code === "STORAGE_CACHE.INNER_MISSING"`
|
||||
|
||||
#### Scenario: missing cache throws INVALID_OPTIONS
|
||||
|
||||
- **WHEN** `cachedStorage(innerStorage, {} as unknown as CachedStorageOptions)` is called
|
||||
- **THEN** the call throws `CachedStorageError` with `code === "STORAGE_CACHE.INVALID_OPTIONS"`
|
||||
|
||||
#### Scenario: cache without get/set/delete methods throws INVALID_OPTIONS
|
||||
|
||||
- **WHEN** `cachedStorage(innerStorage, { cache: { name: "x" } as unknown as AbstractCache<Uint8Array> })` is called
|
||||
- **THEN** the call throws `CachedStorageError` with `code === "STORAGE_CACHE.INVALID_OPTIONS"`
|
||||
|
||||
**Verified by:** `src/factory/factory.test.ts::"throws INNER_MISSING when inner is nullish"` AND `src/factory/factory.test.ts::"throws INVALID_OPTIONS when cache is missing"` AND `src/factory/factory.test.ts::"throws INVALID_OPTIONS when cache lacks get/set/delete"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `get` returns cached bytes on hit without consulting inner
|
||||
|
||||
When `get(key, signal?)` is called and `cache.get(cacheKey)` resolves to a `Uint8Array`, the wrapper SHALL return those bytes WITHOUT invoking `inner.get`. The cache key SHALL be `${namespace ?? "storage"}::${key}`.
|
||||
|
||||
#### Scenario: cache hit short-circuits inner
|
||||
|
||||
- **WHEN** `wrapper.get("clip.mov")` is called and `cache.get("storage::clip.mov")` resolves to bytes
|
||||
- **THEN** the returned value equals those bytes AND `inner.get` is NOT invoked AND `getCacheStats().hits === 1`
|
||||
|
||||
#### Scenario: custom namespace appears in cache key
|
||||
|
||||
- **WHEN** the wrapper is constructed with `{ cache, namespace: "media" }` and `wrapper.get("clip.mov")` is called
|
||||
- **THEN** `cache.get("media::clip.mov")` is invoked exactly once
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.get.test.ts::"hit short-circuits inner and returns cached bytes"` AND `src/cached-storage/cached-storage.get.test.ts::"uses the configured namespace in the cache key"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `get` populates the cache on miss
|
||||
|
||||
When `cache.get(cacheKey)` returns `undefined`, the wrapper SHALL invoke `inner.get(key, signal)`, and when the inner returns a `Uint8Array` AND `shouldCache(key, bytes)` returns true, SHALL call `cache.set(cacheKey, bytes)`. The default `shouldCache` SHALL return true when `bytes.byteLength <= 16 * 1024 * 1024` (16 MiB). When the inner returns `undefined`, the wrapper SHALL NOT cache anything (no negative caching).
|
||||
|
||||
#### Scenario: miss invokes inner and populates the cache
|
||||
|
||||
- **WHEN** `wrapper.get("x.bin")` is called, `cache.get` returns `undefined`, and `inner.get` returns `bytes`
|
||||
- **THEN** `inner.get("x.bin", signal)` is invoked exactly once AND `cache.set("storage::x.bin", bytes)` is invoked exactly once AND `getCacheStats().misses === 1` AND `getCacheStats().populations === 1`
|
||||
|
||||
#### Scenario: shouldCache predicate gates population
|
||||
|
||||
- **WHEN** the wrapper is constructed with `{ cache, shouldCache: () => false }` and `wrapper.get("x.bin")` is called on a miss
|
||||
- **THEN** `inner.get` is invoked exactly once AND `cache.set` is NOT invoked AND `getCacheStats().populations === 0`
|
||||
|
||||
#### Scenario: default maxEntryBytes is 16 MiB
|
||||
|
||||
- **WHEN** the wrapper is constructed without `shouldCache` or `maxEntryBytes`, and `inner.get` resolves to a `Uint8Array` of `16 * 1024 * 1024 + 1` bytes
|
||||
- **THEN** `cache.set` is NOT invoked AND `getCacheStats().populations === 0`
|
||||
|
||||
#### Scenario: undefined from inner is not cached
|
||||
|
||||
- **WHEN** `wrapper.get("missing.bin")` is called and `inner.get` returns `undefined`
|
||||
- **THEN** `cache.set` is NOT invoked AND the wrapper returns `undefined`
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.get.test.ts::"miss invokes inner and populates the cache"` AND `src/cached-storage/cached-storage.get.test.ts::"shouldCache predicate gates population"` AND `src/cached-storage/cached-storage.get.test.ts::"default maxEntryBytes is 16 MiB"` AND `src/cached-storage/cached-storage.get.test.ts::"undefined from inner is not cached"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Cache failures never propagate
|
||||
|
||||
When `cache.get`, `cache.set`, or `cache.delete` throws or rejects, the wrapper SHALL increment `getCacheStats().errors`, SHALL record the failure via the OTel `storage_cache.errors_total{namespace, op}` counter, and SHALL NOT propagate the failure to the consumer. On `cache.get` failure during a `get(key)` call, the wrapper SHALL fall through to `inner.get` and return its result. On `cache.set` failure during population, the wrapper SHALL still return the bytes from `inner.get`. On `cache.delete` failure during invalidation, the wrapper SHALL still return the inner's result.
|
||||
|
||||
#### Scenario: cache.get rejection falls through to inner
|
||||
|
||||
- **WHEN** `wrapper.get("x")` is called and `cache.get` rejects with `new Error("redis down")`
|
||||
- **THEN** `inner.get` is invoked AND the returned bytes match `inner.get`'s result AND the consumer sees no error AND `getCacheStats().errors === 1`
|
||||
|
||||
#### Scenario: cache.set rejection does not propagate
|
||||
|
||||
- **WHEN** `wrapper.get("x")` is called and `cache.set` rejects after `inner.get` succeeds
|
||||
- **THEN** `wrapper.get` resolves with `inner.get`'s bytes AND `getCacheStats().errors === 1` AND `getCacheStats().populations === 0`
|
||||
|
||||
#### Scenario: cache.delete rejection on invalidation does not propagate
|
||||
|
||||
- **WHEN** `wrapper.set("x", bytes)` succeeds and `cache.delete` rejects
|
||||
- **THEN** `wrapper.set` resolves successfully AND `getCacheStats().errors === 1` AND `getCacheStats().invalidations === 0`
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.get.test.ts::"cache.get rejection falls through to inner"` AND `src/cached-storage/cached-storage.get.test.ts::"cache.set rejection does not propagate"` AND `src/cached-storage/cached-storage.write.test.ts::"cache.delete rejection on invalidation does not propagate"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `set` invalidates the cache entry post-success
|
||||
|
||||
When `set(key, value, signal?)` is called and `invalidateOnWrite !== false`, the wrapper SHALL invoke `inner.set(key, value, signal)` first; on successful resolution, SHALL call `cache.delete(cacheKey)`. When `inner.set` rejects, the wrapper SHALL propagate the rejection WITHOUT invalidating the cache.
|
||||
|
||||
#### Scenario: successful set invalidates the cache entry
|
||||
|
||||
- **WHEN** `wrapper.set("x.bin", bytes)` succeeds
|
||||
- **THEN** `cache.delete("storage::x.bin")` is invoked exactly once AND `getCacheStats().invalidations === 1`
|
||||
|
||||
#### Scenario: failed set does not invalidate
|
||||
|
||||
- **WHEN** `wrapper.set("x.bin", bytes)` rejects because `inner.set` threw
|
||||
- **THEN** `cache.delete` is NOT invoked AND `getCacheStats().invalidations === 0`
|
||||
|
||||
#### Scenario: invalidateOnWrite:false skips invalidation
|
||||
|
||||
- **WHEN** the wrapper is constructed with `{ cache, invalidateOnWrite: false }` and `wrapper.set("x.bin", bytes)` succeeds
|
||||
- **THEN** `cache.delete` is NOT invoked
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.write.test.ts::"successful set invalidates the cache entry"` AND `src/cached-storage/cached-storage.write.test.ts::"failed set does not invalidate"` AND `src/cached-storage/cached-storage.write.test.ts::"invalidateOnWrite:false skips invalidation"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `setPartial` invalidates the whole entry post-success
|
||||
|
||||
When `setPartial(key, offset, value, signal?)` is called, the wrapper SHALL invoke `inner.setPartial(...)` first; on success, when `invalidateOnWrite !== false`, SHALL call `cache.delete(cacheKey)` for the whole `key` (the cache entry for that key is now stale regardless of the partial write's range). When `inner.setPartial` rejects, the wrapper SHALL propagate the rejection WITHOUT invalidating.
|
||||
|
||||
#### Scenario: setPartial invalidates the whole cache entry
|
||||
|
||||
- **WHEN** `wrapper.setPartial("x.bin", 100, bytes)` succeeds
|
||||
- **THEN** `cache.delete("storage::x.bin")` is invoked exactly once AND `getCacheStats().invalidations === 1`
|
||||
|
||||
#### Scenario: failed setPartial does not invalidate
|
||||
|
||||
- **WHEN** `wrapper.setPartial("x.bin", 100, bytes)` rejects because `inner.setPartial` threw
|
||||
- **THEN** `cache.delete` is NOT invoked
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.write.test.ts::"setPartial invalidates the whole cache entry"` AND `src/cached-storage/cached-storage.write.test.ts::"failed setPartial does not invalidate"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `delete` invalidates the cache entry post-success
|
||||
|
||||
When `delete(key, signal?)` is called and `invalidateOnDelete !== false`, the wrapper SHALL invoke `inner.delete(key, signal)` first; on successful resolution (regardless of whether `inner.delete` returned `true` or `false`), SHALL call `cache.delete(cacheKey)` to clear any stale entry. When `inner.delete` rejects, the wrapper SHALL propagate the rejection WITHOUT invalidating.
|
||||
|
||||
#### Scenario: successful delete invalidates the cache entry
|
||||
|
||||
- **WHEN** `wrapper.delete("x.bin")` resolves
|
||||
- **THEN** `cache.delete("storage::x.bin")` is invoked exactly once AND `getCacheStats().invalidations === 1`
|
||||
|
||||
#### Scenario: failed delete does not invalidate
|
||||
|
||||
- **WHEN** `wrapper.delete("x.bin")` rejects because `inner.delete` threw
|
||||
- **THEN** `cache.delete` is NOT invoked AND the rejection propagates
|
||||
|
||||
#### Scenario: invalidateOnDelete:false skips invalidation
|
||||
|
||||
- **WHEN** the wrapper is constructed with `{ cache, invalidateOnDelete: false }` and `wrapper.delete("x.bin")` resolves to `true`
|
||||
- **THEN** `cache.delete` is NOT invoked
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.delete.test.ts::"successful delete invalidates the cache entry"` AND `src/cached-storage/cached-storage.delete.test.ts::"failed delete does not invalidate"` AND `src/cached-storage/cached-storage.delete.test.ts::"invalidateOnDelete:false skips invalidation"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `clear` invalidates only the namespaced subset of the cache
|
||||
|
||||
When `clear(signal?)` is called and `invalidateOnClear !== false`, the wrapper SHALL invoke `inner.clear(signal)` first; on success, SHALL iterate the cache's keys (when `cache.keys()` is available) and delete every key starting with `${namespace}::`. When the underlying `AbstractCache` does NOT expose `keys()`, the wrapper SHALL log a warning via `options.logger?.warn?.(...)` and SHALL NOT invoke `cache.clear()` (doing so would obliterate every other wrapper's entries sharing the cache).
|
||||
|
||||
#### Scenario: clear evicts namespaced entries only when cache supports keys()
|
||||
|
||||
- **WHEN** the underlying cache has both `"storage::a"` and `"other::b"` and `wrapper.clear()` is called with namespace `"storage"`
|
||||
- **THEN** `cache.delete("storage::a")` is invoked AND `cache.delete("other::b")` is NOT invoked
|
||||
|
||||
#### Scenario: clear logs a warning when cache lacks keys()
|
||||
|
||||
- **WHEN** the underlying `AbstractCache` does not implement `keys()` and `wrapper.clear()` is called
|
||||
- **THEN** `options.logger.warn` is invoked at least once with a message naming `[storage-cache]` AND `cache.clear()` is NOT invoked AND `getCacheStats().invalidations === 0`
|
||||
|
||||
#### Scenario: invalidateOnClear:false skips namespaced cleanup
|
||||
|
||||
- **WHEN** the wrapper is constructed with `{ cache, invalidateOnClear: false }` and `wrapper.clear()` is called
|
||||
- **THEN** `cache.delete`/`cache.clear`/`cache.keys` are NOT invoked
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.clear.test.ts::"clear evicts namespaced entries only when cache supports keys()"` AND `src/cached-storage/cached-storage.clear.test.ts::"clear logs a warning when cache lacks keys()"` AND `src/cached-storage/cached-storage.clear.test.ts::"invalidateOnClear:false skips namespaced cleanup"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Passthrough surface — `exists`, `list`, metadata, partial, streams
|
||||
|
||||
The wrapper SHALL implement `exists`, `list`, `getWithMetadata`, `getMetadata`, `getSize`, `getItemSize`, `getPartial`, `createReadStream`, and `createWriteStream` as pure delegates to the inner storage. None of these SHALL consult or mutate the cache (this iteration intentionally caches whole-object reads only).
|
||||
|
||||
#### Scenario: exists delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.exists("x")` is called and `inner.exists("x")` returns `true`
|
||||
- **THEN** the wrapper returns `true` AND `cache.get` / `cache.set` / `cache.delete` are NOT invoked
|
||||
|
||||
#### Scenario: list delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.list({ prefix: "a/" })` is called
|
||||
- **THEN** the returned value equals `inner.list({ prefix: "a/" })`'s result AND `cache.get` / `cache.set` are NOT invoked
|
||||
|
||||
#### Scenario: getMetadata delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.getMetadata("x")` is called
|
||||
- **THEN** the returned value equals `inner.getMetadata("x")`'s result AND `cache.get` is NOT invoked
|
||||
|
||||
#### Scenario: getPartial delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.getPartial("x", 100, 200)` is called
|
||||
- **THEN** the returned slice equals `inner.getPartial("x", 100, 200)`'s result AND `cache.get` is NOT invoked
|
||||
|
||||
#### Scenario: createReadStream delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.createReadStream("x")` is called
|
||||
- **THEN** the returned stream is the same reference returned by `inner.createReadStream("x")` AND `cache.get` is NOT invoked
|
||||
|
||||
#### Scenario: createWriteStream delegates to inner without touching the cache
|
||||
|
||||
- **WHEN** `wrapper.createWriteStream("x")` is called
|
||||
- **THEN** the returned writer is the same reference returned by `inner.createWriteStream("x")` AND `cache.delete` is NOT invoked at construction time (the cache is invalidated only when the writer's underlying write commits via `set`, which is out of scope here — see Implementation Notes in design.md)
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.passthrough.test.ts::"exists delegates to inner without touching the cache"` AND `src/cached-storage/cached-storage.passthrough.test.ts::"list delegates to inner without touching the cache"` AND `src/cached-storage/cached-storage.passthrough.test.ts::"getMetadata delegates to inner without touching the cache"` AND `src/cached-storage/cached-storage.passthrough.test.ts::"getPartial delegates to inner without touching the cache"` AND `src/cached-storage/cached-storage.passthrough.test.ts::"createReadStream delegates to inner without touching the cache"` AND `src/cached-storage/cached-storage.passthrough.test.ts::"createWriteStream delegates to inner without touching the cache"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AbortSignal short-circuit on pre-aborted signal
|
||||
|
||||
When `signal?.aborted === true` at the time `get`, `set`, `setPartial`, `delete`, or `clear` is invoked, the wrapper SHALL skip its own cache work (no `cache.get` / `cache.set` / `cache.delete` calls) and SHALL invoke the inner method immediately so the inner storage raises the canonical `STORAGE.*` aborted error. Live signals SHALL be forwarded into `cache.get` / `cache.set` / `cache.delete` calls when the underlying `AbstractCache` accepts a signal argument.
|
||||
|
||||
#### Scenario: pre-aborted signal skips cache work in get
|
||||
|
||||
- **WHEN** `wrapper.get("x", AbortSignal.abort())` is called
|
||||
- **THEN** `cache.get` is NOT invoked AND `inner.get` is invoked exactly once with the same aborted signal
|
||||
|
||||
#### Scenario: pre-aborted signal skips invalidation in set
|
||||
|
||||
- **WHEN** `wrapper.set("x", bytes, AbortSignal.abort())` is called and `inner.set` rejects with abort
|
||||
- **THEN** `cache.delete` is NOT invoked AND the rejection propagates
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.signal.test.ts::"pre-aborted signal skips cache work in get"` AND `src/cached-storage/cached-storage.signal.test.ts::"pre-aborted signal skips invalidation in set"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `getCacheStats()` reports hits, misses, populations, invalidations, errors
|
||||
|
||||
The class SHALL expose `getCacheStats(): CachedStorageStats` returning a snapshot of the wrapper's counters: `{ hits, misses, populations, invalidations, errors }`. Counters SHALL be monotonically non-decreasing. The method SHALL NOT mutate any state.
|
||||
|
||||
#### Scenario: counters reflect a hit, a miss, a population, and an invalidation
|
||||
|
||||
- **WHEN** one hit, one miss + population, and one set (invalidation) are exercised in sequence against a fresh wrapper
|
||||
- **THEN** `getCacheStats()` returns `{ hits: 1, misses: 1, populations: 1, invalidations: 1, errors: 0 }`
|
||||
|
||||
#### Scenario: getCacheStats does not mutate state
|
||||
|
||||
- **WHEN** `getCacheStats()` is called twice without any other operation
|
||||
- **THEN** both snapshots return the same object content AND no `cache.*` method is invoked
|
||||
|
||||
**Verified by:** `src/cached-storage/cached-storage.test.ts::"getCacheStats reflects hit, miss, population, and invalidation"` AND `src/cached-storage/cached-storage.test.ts::"getCacheStats does not mutate state"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry instrumentation
|
||||
|
||||
Every cache interaction SHALL emit a span on the `@suaveplan/storage-cache` tracer with `storage.cache.namespace` and `storage.cache.key` attributes (the key is `*` for `clear` operations) and a `storage.cache.outcome` attribute drawn from `{"hit", "miss", "populated", "invalidated", "error"}`. The wrapper SHALL emit counters `storage_cache.{hits, misses, populations, invalidations, errors}_total{namespace}` and a `storage_cache.entry_bytes{namespace}` histogram observing each `populations_total` increment with the cached entry's byte length. The `errors_total` counter SHALL include an `op` label drawn from `{"get", "set", "delete", "clear"}`.
|
||||
|
||||
#### Scenario: cache hit increments hits_total and records a hit-outcome span
|
||||
|
||||
- **WHEN** a cache hit is exercised
|
||||
- **THEN** `storage_cache.hits_total{namespace:"storage"}` increments by 1 AND the emitted span has `storage.cache.outcome === "hit"`
|
||||
|
||||
#### Scenario: population records entry_bytes histogram
|
||||
|
||||
- **WHEN** a 4096-byte payload is cached on a miss
|
||||
- **THEN** `storage_cache.entry_bytes` records one observation of `4096`
|
||||
|
||||
#### Scenario: cache error increments errors_total with op label
|
||||
|
||||
- **WHEN** `cache.set` rejects during population
|
||||
- **THEN** `storage_cache.errors_total{namespace:"storage", op:"set"}` increments by 1
|
||||
|
||||
**Verified by:** `src/otel/otel.test.ts::"hit increments hits_total and records hit-outcome span"` AND `src/otel/otel.test.ts::"population records entry_bytes histogram"` AND `src/otel/otel.test.ts::"cache error increments errors_total with op label"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Multi-namespace isolation on a shared cache
|
||||
|
||||
Two `CachedStorage` wrappers constructed over DISTINCT inner storages with the SAME `AbstractCache` reference but DISTINCT `namespace` strings SHALL NOT observe each other's entries on `get`, AND `wrapper.clear()` on one SHALL NOT evict entries belonging to the other.
|
||||
|
||||
#### Scenario: distinct namespaces isolate entries on a shared cache
|
||||
|
||||
- **WHEN** wrapper A (namespace `"a"`) populates `"clip.mov"` and wrapper B (namespace `"b"`) then reads `"clip.mov"`
|
||||
- **THEN** wrapper B observes a miss AND `cache.get("a::clip.mov")` and `cache.get("b::clip.mov")` are distinct calls
|
||||
|
||||
#### Scenario: clear on one wrapper does not evict the other's entries
|
||||
|
||||
- **WHEN** wrapper A populates entries under `"a::"` and wrapper B populates entries under `"b::"`, then `wrapperA.clear()` is called against a cache that exposes `keys()`
|
||||
- **THEN** every `"a::*"` key is deleted AND every `"b::*"` key is preserved
|
||||
|
||||
**Verified by:** `tests/integration/namespaced-isolation.test.ts::"distinct namespaces isolate entries on a shared cache"` AND `tests/integration/namespaced-isolation.test.ts::"clear on one wrapper does not evict the other's entries"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,89 @@
|
||||
# Tasks — add-storage-plugin-cache
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `openspec/specs/storage/spec.md` and confirm `AbstractStorage`'s 15 abstract methods and the `NamespacedStorage` decorator precedent
|
||||
- [ ] 1.2 Read `openspec/specs/cache/spec.md` and confirm `AbstractCache<T>` contract (especially whether `keys()` is part of the canonical surface)
|
||||
- [ ] 1.3 Read `openspec/archive/2026-05-16-add-cache-storage-redis-backends/` for the cache-family conventions
|
||||
- [ ] 1.4 Read sibling proposal `openspec/changes/add-drive-plugin-cache/proposal.md` — match option shape, error-swallow rules, OTel naming
|
||||
- [ ] 1.5 Read `openspec/AGENTS.md` §0c, §2b, §6a, §7, §11, §13
|
||||
- [ ] 1.6 Draft `proposal.md`, `design.md`, `tasks.md`, `specs/storage-plugin-cache/spec.md`
|
||||
- [ ] 1.7 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete planned `it(...)` titles
|
||||
- [ ] 1.8 Build the numbered Requirement/Scenario implementation checklist (§0c step 6) — 13 requirements, ~30 scenarios
|
||||
|
||||
## 2. Scaffolding
|
||||
|
||||
- [ ] 2.1 Create `packages/storage/storage-plugin-cache/` mirroring `packages/storage/storage-uri/` conventions
|
||||
- [ ] 2.2 Write `package.json` v0.0.0, `"sideEffects": false`, subpath exports for every feature module
|
||||
- [ ] 2.3 Wire deps: `@suaveplan/error`, `@suaveplan/storage`, `@suaveplan/cache`, `@suaveplan/types`, `@opentelemetry/api`, `tslib` (all `workspace:^` / `catalog:`)
|
||||
- [ ] 2.4 Wire devDeps: `@suaveplan/testing`, `@suaveplan/storage-file`, `@suaveplan/storage-http`, `@suaveplan/cache-storage`, `@vitest/coverage-istanbul`, `fast-check`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`
|
||||
- [ ] 2.5 Write `tsconfig.json`, `tsconfig.lib.json`, `vite.config.ts`, `vitest.config.ts` (via `createTestConfig`)
|
||||
- [ ] 2.6 Write `tests/setup/polyfills.ts` and `README.md` skeleton
|
||||
|
||||
## 3. Implementation
|
||||
|
||||
- [ ] 3.1 `src/error-codes/error-codes.ts` — `defineCodes("STORAGE_CACHE", ["INVALID_OPTIONS", "INNER_MISSING"] as const)`, `CachedStorageError` class, `key?`/`namespace?` discriminants
|
||||
- [ ] 3.2 `src/types/options.types.ts` — `CachedStorageOptions`, `CachedStorageStats`, `ResolvedCachedStorageOptions` interfaces (no inline shapes per §11)
|
||||
- [ ] 3.3 `src/cached-storage/cached-storage.signal.ts` — `AbortSignal` pre-aborted short-circuit helper
|
||||
- [ ] 3.4 `src/cached-storage/cached-storage.get.ts` — `get(key, signal?)` read-through with hit/miss/populate
|
||||
- [ ] 3.5 `src/cached-storage/cached-storage.write.ts` — `set(key, value, signal?)` and `setPartial(...)` invalidation
|
||||
- [ ] 3.6 `src/cached-storage/cached-storage.delete.ts` — `delete(key, signal?)` invalidation
|
||||
- [ ] 3.7 `src/cached-storage/cached-storage.clear.ts` — `clear(signal?)` namespaced enumeration + fallback warning
|
||||
- [ ] 3.8 `src/cached-storage/cached-storage.passthrough.ts` — `exists`/`list`/`getMetadata`/`getWithMetadata`/`getSize`/`getItemSize`/`getPartial`/`createReadStream`/`createWriteStream` delegates
|
||||
- [ ] 3.9 `src/cached-storage/cached-storage.ts` — `class CachedStorage extends AbstractStorage` assembling the helpers from `.get.ts`/`.write.ts`/`.delete.ts`/`.clear.ts`/`.passthrough.ts`/`.signal.ts`. Constructor validation; `cache`/`inner`/`getCacheStats()` accessors.
|
||||
- [ ] 3.10 `src/factory/factory.ts` — `cachedStorage(inner, options): CachedStorage` factory + synchronous `validateOptions(options)` throwing `INVALID_OPTIONS`/`INNER_MISSING`
|
||||
- [ ] 3.11 `src/otel/otel.ts` — meter + tracer wiring, counters `hits`/`misses`/`populations`/`invalidations`/`errors`, histogram `entry_bytes`, span attribute helpers
|
||||
- [ ] 3.12 `src/index.ts` — explicit named re-exports per §6a
|
||||
|
||||
## 4. Tests (every `it(...)` carries `{ timeout: 5000 }`, 30000 for async integration)
|
||||
|
||||
- [ ] 4.1 `src/error-codes/error-codes.test.ts` — both codes reachable from a real call site, discriminants populated
|
||||
- [ ] 4.2 `src/factory/factory.test.ts` — factory returns `CachedStorage`, throws `INNER_MISSING` on nullish inner, throws `INVALID_OPTIONS` on missing/malformed cache
|
||||
- [ ] 4.3 `src/cached-storage/cached-storage.test.ts` — instance is `instanceof AbstractStorage` AND `instanceof CachedStorage`, `cache`/`inner` exposed by reference, `getCacheStats` snapshot semantics
|
||||
- [ ] 4.4 `src/cached-storage/cached-storage.get.test.ts` — hit short-circuits inner, custom namespace in key, miss populates, `shouldCache` gate, 16 MiB ceiling, `undefined` not cached, cache.get rejection falls through, cache.set rejection swallowed
|
||||
- [ ] 4.5 `src/cached-storage/cached-storage.write.test.ts` — `set` invalidates post-success, failed `set` does not invalidate, `setPartial` invalidates whole entry, `invalidateOnWrite:false` skips, cache.delete rejection swallowed
|
||||
- [ ] 4.6 `src/cached-storage/cached-storage.delete.test.ts` — `delete` invalidates post-success, failed `delete` does not invalidate, `invalidateOnDelete:false` skips
|
||||
- [ ] 4.7 `src/cached-storage/cached-storage.clear.test.ts` — namespaced enumeration via `cache.keys()`, warning + skip when `keys()` absent, `invalidateOnClear:false` skips
|
||||
- [ ] 4.8 `src/cached-storage/cached-storage.passthrough.test.ts` — every passthrough method delegates without touching the cache (one `it(...)` per method)
|
||||
- [ ] 4.9 `src/cached-storage/cached-storage.signal.test.ts` — pre-aborted skips cache work in `get`, pre-aborted skips invalidation in `set`
|
||||
- [ ] 4.10 `src/otel/otel.test.ts` — counters increment, histogram observed, span outcome attribute correct, errors_total `op` label populated
|
||||
- [ ] 4.11 `tests/integration/memory-in-front-of-http.test.ts` — real `MemoryCache` + real `storage-http` (in-process fetch stub): two reads, inner invoked once
|
||||
- [ ] 4.12 `tests/integration/file-cache-survives-restart.test.ts` — real `StorageCache` over real `storage-file` + in-memory inner: cache survives wrapper re-instantiation
|
||||
- [ ] 4.13 `tests/integration/namespaced-isolation.test.ts` — two wrappers / two namespaces / one cache: isolation on `get` and on `clear`
|
||||
- [ ] 4.14 No imports from `vitest` or `bun:test` in test files — all via `@suaveplan/testing/runner` (rule 11)
|
||||
- [ ] 4.15 No `vi.mock` / `jest.mock`; all dependencies are real per CLAUDE.md rule 2
|
||||
|
||||
## 5. Documentation
|
||||
|
||||
- [ ] 5.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples, full API reference)
|
||||
- [ ] 5.2 `README.md` links every module under `## Modules`
|
||||
- [ ] 5.3 README documents the six composition recipes from `design.md` (memory→S3, OPFS→HTTPS, file→Bunny, Redis→S3, `wrap()` SWR, compose with `storage-uri`)
|
||||
- [ ] 5.4 README documents the relationship to `@suaveplan/cache-storage` (the *inverse* direction) and to `add-drive-plugin-cache` (the drive-tier twin)
|
||||
- [ ] 5.5 README documents the `shouldCache` / `maxEntryBytes` defaults and the rationale (16 MiB matches `drive-mesh-protocol`'s frame ceiling and `drive-plugin-cache`'s ceiling)
|
||||
|
||||
## 6. Gates
|
||||
|
||||
- [ ] 6.1 `bun run check`
|
||||
- [ ] 6.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-plugin-cache`
|
||||
- [ ] 6.3 `bunx turbo test:coverage --filter=@suaveplan/storage-plugin-cache` — 100/100/100/100 Istanbul
|
||||
- [ ] 6.4 `bunx turbo build --filter=@suaveplan/storage-plugin-cache`
|
||||
- [ ] 6.5 `bun scripts/lint/check-test-runner-imports.ts` (rule 11)
|
||||
- [ ] 6.6 Tier-purity grep — `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-plugin-cache/src/` returns empty
|
||||
- [ ] 6.7 Banned-pattern scan returns empty (§2)
|
||||
- [ ] 6.8 Missing-`.md` scan returns empty (§4)
|
||||
- [ ] 6.9 Non-root `export *` scan returns empty (§6a)
|
||||
- [ ] 6.10 Bare-version scan returns empty (§1)
|
||||
- [ ] 6.11 Folder-naming scan (§10) — children under `cached-storage/` and `factory/` do not include the parent's singular form as a suffix
|
||||
- [ ] 6.12 Spec-reconciliation gate — every `### Requirement` is implemented; every `**Verified by:**` test name exists verbatim (§13 step 0)
|
||||
- [ ] 6.13 File-size scan — every `src/**/*.ts` ≤ 300 lines, every `*.test.ts` ≤ 500 lines
|
||||
|
||||
## 7. Manifest & archival
|
||||
|
||||
- [ ] 7.1 Add package to `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 7.2 Update `STATUS.md` (genesis-original — add to "Completed" once shipped)
|
||||
- [ ] 7.3 Update `SUMMARY.md` with the one-paragraph description
|
||||
- [ ] 7.4 `bunx markdownlint-cli2 --fix` on touched docs
|
||||
- [ ] 7.5 Fold `specs/storage-plugin-cache/spec.md` into `openspec/specs/storage-plugin-cache/spec.md`
|
||||
- [ ] 7.6 Move `openspec/changes/add-storage-plugin-cache/` → `openspec/archive/<yyyy-mm-dd>-add-storage-plugin-cache/`
|
||||
Reference in New Issue
Block a user