Initial Commit

openspec/archive/2026-05-16-add-drive-plugin-cache/proposal.md

@@ -0,0 +1,111 @@
+# Add @suaveplan/drive-plugin-cache — read-through cache middleware for Drive
+
+**Package**: `@suaveplan/drive-plugin-cache`
+**Status**: PROPOSED
+**Type**: new-package (genesis-original)
+**Depends on**: `@suaveplan/drive` (archived), `@suaveplan/cache` family (archived as `add-cache-storage-redis-backends` on 2026-05-16)
+
+- **Design**: ./design.md
+- **Spec delta**: ./specs/drive-plugin-cache/spec.md
+
+## Why
+
+`@suaveplan/drive` exposes a `DrivePlugin` / `DriveMiddleware` pair as its extension surface: middleware can intercept `read` / `write` / `delete` / `stat` / `list` calls with a `(ctx, next)` continuation pattern. Today no shipped package fills the obvious slot — a read-through cache that sits between consumers and the storage tier, returning hot plaintext bytes from a fast in-memory or OPFS-backed cache instead of re-running the full plugin chain (manifest fetch, peer pull, AEAD verify, reassembly).
+
+The cache infrastructure is already complete and unified:
+
+- `@suaveplan/cache` — `AbstractCache<T>`, `CacheFactory`, `CacheProviderRegistry`, four eviction policies (`lru`, `lfu`, `fifo`, `ttl`), lifecycle events, `wrap()` with stale-while-revalidate, namespacing, structured `CACHE_*` codes.
+- `@suaveplan/cache-storage` — `StorageCache<T>` over any `AbstractStorage` (so OPFS / IndexedDB / file / S3 / CDN-backed caches Just Work).
+- `@suaveplan/cache-redis` — `RedisCache<T>` over `RedisLike`.
+- Built-in `MemoryCache` registered as `"memory"`.
+
+The result: `createCache("memory" | "storage" | "redis", config)` is the cache-of-your-choice surface, and every present and future `AbstractStorage` adapter is automatically a cache backend. What's missing is **the consumer that wires `Drive.read` to that surface.**
+
+`drive-plugin-cache` is that consumer: a tiny middleware (~150 LOC) that accepts a caller-supplied `AbstractCache<Uint8Array>`, intercepts every `read(path)` to check the cache first, and populates on miss. Writes / deletes invalidate the corresponding entry. The cache itself is opaque to the plugin — consumers pick the backend, the eviction policy, the TTL, the namespace, the codec, the redis/storage instance — and the plugin just talks `AbstractCache`.
+
+This closes the most common drive consumer-pattern complaint: every audio/video pipeline today calls `drive.read("clip.mov")` repeatedly across renders and pays the full reassembly cost each time. With this plugin, the second-and-onward reads of a hot path return cached bytes in microseconds.
+
+## What changes
+
+- ADDED: `driveCachePlugin(options): DrivePlugin<DriveCacheHandle>` factory. Returns a `DrivePlugin` whose `install(drive)` registers a `DriveMiddleware` that intercepts `read` (and optionally `write` / `delete` for invalidation) on the supplied `DriveLike`. The factory accepts the cache instance, an invalidation policy, and an optional namespace.
+- ADDED: `DriveCacheOptions` interface — `{ cache: AbstractCache<Uint8Array>; invalidateOnWrite?: boolean; invalidateOnDelete?: boolean; namespace?: string; shouldCache?: (path: string, bytes: Uint8Array) => boolean; maxEntryBytes?: number }`.
+- ADDED: `DriveCacheHandle` interface — the plugin handle returned from `install()`. Exposes `getStats()` (hits, misses, populations, evictions, invalidations), `invalidate(path)`, `clear()`, and `cache` (the underlying `AbstractCache` reference for advanced wiring like `wrap()` with stale-while-revalidate).
+- ADDED: Read-through middleware. On `read(ctx, next)`:
+  1. Compute the cache key as `${namespace ?? "drive"}::${ctx.path}`.
+  2. `await cache.get(key)`. If hit, return bytes; emit `cache_hit` event; increment `drive_cache.hits_total{namespace}`.
+  3. On miss, `await next()` to get plaintext from the next middleware (or the storage). Increment `drive_cache.misses_total{namespace}`.
+  4. If `shouldCache(path, bytes)` returns true (default: `bytes.byteLength <= maxEntryBytes ?? 16 MiB`), `await cache.set(key, bytes)`. Increment `drive_cache.populations_total{namespace}`. Errors from `cache.set` SHALL be swallowed (logged via `LoggerLike` when wired) — a cache write failure MUST NOT propagate to the consumer.
+  5. Return bytes.
+- ADDED: Write/delete invalidation. On `write(ctx, next)`, if `invalidateOnWrite !== false`, run `next()` first; on success, `await cache.delete(key)` (the cached plaintext is now stale). Symmetric on `delete(ctx, next)` when `invalidateOnDelete !== false`. Errors from `cache.delete` SHALL NOT propagate.
+- ADDED: `stat` / `list` passthrough — the middleware does NOT cache stat or list results in this iteration (cache invalidation across `list` prefixes is non-trivial; out of scope). `stat` and `list` SHALL pass through `next()` unchanged.
+- ADDED: AbortSignal threading — the middleware does NOT swallow `AbortSignal`. The signal in `ctx` (when present) is forwarded into `cache.get` / `cache.set` / `cache.delete` calls; pre-aborted signals SHALL cause the middleware to skip its own work and call `next()` immediately (the inner middleware/storage layer will surface `DRIVE.ABORTED`).
+- ADDED: `DRIVE_CACHE_CODES` registry via `defineCodes("DRIVE_CACHE", [...] as const)` — 3 codes: `INVALID_OPTIONS`, `CACHE_TOO_LARGE`, `INSTALL_FAILED`. The plugin's own errors are deliberately few — most failure modes come from the underlying `AbstractCache` or `Drive`, which carry their own `CACHE_*` / `DRIVE_*` codes that pass through this middleware unchanged.
+- ADDED: `DriveCacheError extends SuaveplanError<DriveCacheErrorCode>` with `path?` and `namespace?` discriminants.
+- ADDED: OTel surface — `drive_cache.{hits,misses,populations,invalidations,errors}_total{namespace}` counters; `drive_cache.entry_bytes` histogram (distribution of cached entry sizes); spans on `@suaveplan/drive-cache` tracer wrapping each `read`/`write`/`delete` middleware invocation with `drive.cache.namespace`, `drive.cache.path`, `drive.cache.outcome` attributes.
+- ADDED: README composition recipes — memory cache (default), OPFS-backed cache via `cache-storage` + `storage-browser-opfs`, file-backed cache via `cache-storage` + `storage-file`, Redis-backed cache via `cache-redis`, namespaced cache (multi-drive isolation), `wrap()`-based stale-while-revalidate.
+
+## Out of scope
+
+- **Caching `Drive.stat` and `Drive.list` results.** `stat` is cheap; `list` invalidation across prefix mutations is non-trivial (any write under `foo/` would have to invalidate every cached `list("foo/*")` result that happened to enumerate the written path). A future `add-drive-plugin-cache-list` change can take that on; reading is the dominant use case for this iteration.
+- **Partial reads / range caching.** `Drive.createReadStream` is **not** intercepted by this middleware — it bypasses the `read` middleware entirely. Range-aware caching belongs in a future iteration paired with `add-drive-mesh-range-reads`. The plugin SHALL document this constraint; consumers needing cached streaming should compose at the URI layer with the future `cacheUriPlugin`.
+- **Per-tenant key partitioning.** The `namespace` option gives one knob; multi-tenant per-request scoping (e.g. a `request.tenantId` extracted from `ctx`) would require either a `keyFn(ctx)` extension or a `NamespacedCache` factory per request. Out of scope until a concrete consumer needs it.
+- **Cache-aside `wrap()` integration baked in.** The plugin exposes the underlying cache via `handle.cache`, so consumers can call `cache.wrap(key, loader, { staleAge, refreshAhead })` themselves. Wiring `wrap()` directly into the middleware would lock the policy choices that `wrap()` deliberately leaves to the caller (stale-while-revalidate vs. lock-and-load vs. fail-open). Out of scope.
+- **A `cacheUriPlugin` for the URI resolver.** Different but related concern; will be a sibling proposal (`add-storage-uri-cache-plugin`). Drive caches plaintext bytes at the drive-middleware tier; URI cache plugins cache `StorageLike.get` results across schemes. Some consumers will want both.
+- **`AbstractCache<unknown>` support.** This plugin caches `Uint8Array` only — drive reads return bytes. Generic `T` would require a `Codec<T>` on every call site that has no actual variation; `cache-storage` already covers `Codec<T>` for the storage-backed cache itself.
+
+## Source mapping
+
+This is a **genesis-original** package; no upstream `suaveplan/` reference. Requirements derive from:
+
+- The `DrivePlugin` / `DriveMiddleware` / `MiddlewareContext` / `MiddlewareNext` contract in `@suaveplan/drive` (`openspec/specs/drive/spec.md`).
+- The `AbstractCache<T>` contract in `@suaveplan/cache` (`openspec/specs/cache/spec.md`).
+- The `CacheProviderRegistry` factory pattern from `add-cache-storage-redis-backends` (archived 2026-05-16).
+
+### Requirements with no source line
+
+| Requirement | Justification |
+|---|---|
+| Cache failures SHALL NOT propagate to the consumer | A cache is a perf optimisation, not a correctness boundary. A failed `cache.set` must not turn a successful `drive.read` into a thrown error — the bytes are still returned. The OTel surface records the failure (`drive_cache.errors_total`); the consumer never sees it. |
+| `shouldCache` default is "byte-length ≤ 16 MiB" | An unbounded cache turns a large-file read into a memory-pressure event for the entire process. 16 MiB is the same ceiling `drive-mesh-protocol`'s `MAX_FRAME_BYTES` uses for chunk framing — known-safe in every consumer that's already running drive. |
+| Invalidation is post-success, not optimistic | Optimistic invalidation (cache.delete *before* `next()`) opens a race window where a concurrent `read` repopulates the cache with stale bytes during the write. Post-success keeps the cache consistent with whatever storage actually holds. |
+| `stat`/`list` passthrough (not cached) | Listed in Out of Scope. `stat` is cheap enough that caching it costs more than it saves on most stores; `list` invalidation is a separate design problem. |
+| Plugin owns only 3 error codes | The interesting failures are upstream (`CACHE.*`, `DRIVE.*`) and pass through unchanged. Allocating a wide error registry here would be code-bloat and would mask the real diagnostic surface. |
+| `cache` parameter is `AbstractCache<Uint8Array>`, not a `CacheBackendKind` enum | Lets the consumer fully configure the cache before passing it in — eviction policy, TTL, lifecycle events, namespacing, redis/storage instance, codec — without `drive-plugin-cache` having to mirror every cache option. One injectable, one responsibility. |
+
+## Verification approach
+
+Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete planned `it(...)` titles in these files:
+
+- `src/plugin/plugin.test.ts` — factory validates options, installs as `DrivePlugin`, registers `DriveMiddleware` on the supplied drive, returns `DriveCacheHandle`.
+- `src/middleware/middleware.test.ts` — cache hit short-circuits `next()`, cache miss populates after `next()`, `shouldCache` gates population, `maxEntryBytes` default 16 MiB enforced, `cache.set` failure does NOT propagate, `cache.get` failure does NOT propagate (falls through to `next()`).
+- `src/middleware/middleware.invalidation.test.ts` — `write` invalidates post-success, `write` does NOT invalidate when `next()` throws, `delete` symmetric, `invalidateOnWrite: false` skips invalidation, `cache.delete` failure does NOT propagate.
+- `src/middleware/middleware.signal.test.ts` — pre-aborted signal short-circuits to `next()` without invoking cache; live signal threads into `cache.get`/`set`/`delete`.
+- `src/handle/handle.test.ts` — `getStats()` returns monotonically-increasing counters; `invalidate(path)` deletes one entry; `clear()` clears the underlying cache; `handle.cache` is the exact reference passed in `options.cache`.
+- `src/error-codes/error-codes.test.ts` — exhaustive `DRIVE_CACHE_CODES` registry, `DriveCacheError` discriminant fields, every code reachable from a real call site.
+- `src/otel/otel.test.ts` — `hits_total` / `misses_total` / `populations_total` / `invalidations_total` / `errors_total` increment as scenarios fire; `entry_bytes` histogram observes population events; query-string-equivalent path redaction confirmed (paths are not URLs, so no redaction needed — verified by absence of any URL-shaped attribute).
+- `tests/integration/with-memory-cache.test.ts` — real `MemoryCache` + real `Drive` over an in-memory `DriveStorage`: write bytes via `drive.write`, read twice, assert second read came from cache (hits === 1).
+- `tests/integration/with-storage-cache.test.ts` — real `StorageCache` over real `storage-file` `FileStorage` + real `Drive`: identical scenario, asserts the cache file appears on disk and surviving a fresh `driveCachePlugin` install re-hydrates from disk.
+- `tests/integration/with-namespaced-caches.test.ts` — two `driveCachePlugin` instances with distinct `namespace`s share one underlying `AbstractCache` but do not see each other's entries.
+
+## Impact
+
+### Affected dependents
+
+Run `bunx turbo ls --filter=...@suaveplan/drive-plugin-cache` to enumerate. The package is new — no dependents currently. After adoption, consumers that already call `drive.read(path)` repeatedly across renders/requests (audio/video pipelines, asset hydration, drive-template fixtures) benefit directly.
+
+- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/drive-plugin-cache` returning the package itself).
+
+### Affected upstreams (none modified)
+
+| Package | Action | Breaking? |
+|---|---|---|
+| `@suaveplan/drive` | None — `DrivePlugin` / `DriveMiddleware` / `MiddlewareContext` consumed as-is | no |
+| `@suaveplan/cache` | None — `AbstractCache<Uint8Array>` consumed as-is | no |
+| `@suaveplan/cache-storage` | None — used only as a devDep for the integration test | no |
+| `@suaveplan/storage-file` | None — used only as a devDep for the integration test | no |
+
+`drive-plugin-cache` MUST NOT silently grow any of these packages' contracts. If a needed extension turns up during implementation (e.g. `MiddlewareContext` not carrying `signal`), capture it as a separate proposal against the relevant upstream — never bend the contract here.
+
+### Ordering
+
+This change is unblocked: `@suaveplan/drive` and the `@suaveplan/cache*` family both ship today on `genesis` with the consumed surface intact.