Initial Commit

openspec/changes/complete-cache-storage/tasks.md

@@ -0,0 +1,141 @@
+# Tasks — complete-cache-storage
+
+> Every checkbox MUST reflect work actually done in this change. Mostly additive; one MODIFIED to header layout (forward-compatible) and one MODIFIED to `StorageCacheOptions` (additive, all defaults preserve current behaviour).
+
+## 0. Pre-flight (non-negotiable)
+
+- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
+- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
+- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
+
+## 1. Pre-flight
+
+- [ ] 1.1 Re-read `openspec/AGENTS.md` §0c (delta format), §13 (gate order), §6a (barrels), §7 (test-runner imports)
+- [ ] 1.2 Confirm `@suaveplan/cbor` ships on genesis at `packages/flow/cbor` with `encode`, `decode`, `Tagged`, `type TagHandler` in the root barrel
+- [ ] 1.3 Confirm `@suaveplan/types` exposes `TimersLike` (with `setTimeout`/`clearTimeout`/`queueMicrotask`/`now`); add it there first if absent
+- [ ] 1.4 Build a numbered checklist of every `### Requirement` and `#### Scenario` in `specs/cache-storage/spec.md` in working notes (AGENTS §0c step 6)
+- [ ] 1.5 Read the superseded change `openspec/changes/_superseded-narrow-deferrals-2026-05-17/add-cache-storage-cbor-codec/{design,specs/cache-storage/spec}.md` — tag table and codec design reused verbatim
+
+## 2. Package wiring
+
+- [ ] 2.1 Add `@suaveplan/cbor` to `packages/flow/cache-storage/package.json` `dependencies` at `workspace:^`
+- [ ] 2.2 Add `@opentelemetry/api` (catalog) to `dependencies`
+- [ ] 2.3 Add subpath exports for every new feature module (`./codec/structured-clone`, `./codec/conformance`, `./eviction`, `./eviction/lru`, `./eviction/lfu`, `./eviction/fifo`, `./eviction/ttl`, `./eviction/policy`, `./stampede/inflight-map`, `./sweep/ttl-sweeper`, `./swr/swr-scheduler`, `./observability/metrics`, `./observability/trace`, `./stats/namespace-stats`, `./testing/conformance`)
+- [ ] 2.4 Confirm `sideEffects` array still lists only the two real side-effect files (`./src/index.ts`, `./src/registry-augment/registry-augment.ts`) and their dist twins
+
+## 3. Error registry extension (§spec REQ-3)
+
+- [ ] 3.1 Extend `src/errors/errors.ts` `CACHE_STORAGE_CODES` from 5 codes to 12: add `STAMPEDE_TIMEOUT`, `CAS_CONFLICT`, `SWEEP_FAILED`, `EVICTION_FAILED`, `CODEC_UNSUPPORTED_TYPE`, `CODEC_UNKNOWN_TAG`, `NAMESPACE_CONFLICT`
+- [ ] 3.2 Update `src/errors/errors.md` Features + API Reference to list every new code with its raising call site (doc-drift gate §4a)
+- [ ] 3.3 Extend `src/errors/errors.test.ts` reachability assertion to walk all 12 codes
+
+## 4. Header v2 (§spec REQ-2)
+
+- [ ] 4.1 Modify `src/header/header.ts` — repurpose bytes 24..27 as `version: u32 BE`; bytes 28..31 stay reserved-zero; add `FLAG_HAS_VERSION = 1 << 2`
+- [ ] 4.2 Update `HeaderMeta` to add `version: number`
+- [ ] 4.3 Extend `encodeHeader` / `decodeHeader` to round-trip `version`; legacy headers (flag bit 2 clear) decode as `version: 0`
+- [ ] 4.4 `src/header/header.test.ts` — add forward-compat test: a v1-shaped buffer (`FLAG_HAS_VERSION` clear, bytes 24..31 zero) decodes as `version: 0`
+- [ ] 4.5 Update `src/header/header.md` to describe the v2 layout, the `FLAG_HAS_VERSION` bit, and the forward-compatibility contract
+
+## 5. StructuredCloneCodec (§spec REQ-5..REQ-9)
+
+- [ ] 5.1 `src/codec/structured-clone.ts` — `StructuredCloneCodec<T> implements Codec<T>`; lift table from `design.md`; encode via `@suaveplan/cbor` `encode`; decode via `@suaveplan/cbor` `decode` with frozen tag registry; post-walk to detect unhandled `Tagged` → throw `CODEC_UNKNOWN_TAG`
+- [ ] 5.2 `src/codec/structured-clone.md` — sibling docs (≥200 words, basic + advanced examples, API reference, implementation notes including IANA tag references and the RegExp-flags trade-off)
+- [ ] 5.3 `src/codec/structured-clone.test.ts` — every `**Verified by:**` line from spec REQ-5..REQ-9, plus direct tests for `CODEC_UNSUPPORTED_TYPE` (encode of `() => {}`, `Symbol`, `WeakMap`, `Promise`) and `CODEC_UNKNOWN_TAG` (decode of `Tagged(9999, "x")`) and `CODEC_ERROR` (malformed CBOR bytes)
+- [ ] 5.4 If the file exceeds the 300-line soft cap, extract the lift table to `src/codec/structured-clone.tags.ts`
+
+## 6. Codec conformance harness (§spec REQ-10)
+
+- [ ] 6.1 `src/codec/conformance.ts` — `runCodecConformance(codec, { name: string; profile: "json" | "structured-clone" | "bytes" })`; cases per profile match the design doc
+- [ ] 6.2 `src/codec/conformance.md` — sibling docs
+- [ ] 6.3 `src/codec/conformance.test.ts` — self-test: pass a broken codec (decode returns a fixed `null` sentinel), assert at least one harness check fails
+- [ ] 6.4 Extend `src/codec/json.test.ts` with `runCodecConformance(new JsonCodec<unknown>(), { name: "JsonCodec", profile: "json" })`
+- [ ] 6.5 Extend `src/codec/identity.test.ts` with `runCodecConformance(new IdentityCodec(), { name: "IdentityCodec", profile: "bytes" })`
+
+## 7. Eviction policies (§spec REQ-11..REQ-15)
+
+- [ ] 7.1 `src/eviction/policy.types.ts` — `EvictionPolicy` interface, `EvictionConfig` type, `EvictionReason` union (`"lru" | "lfu" | "fifo" | "ttl" | "size"`)
+- [ ] 7.2 `src/eviction/lru/lru.ts` — O(1) LRU via doubly-linked-list keyed by prefixed key; `onSet`/`onHit`/`onDelete`/`reset`/`snapshot`
+- [ ] 7.3 `src/eviction/lfu/lfu.ts` — frequency-bucket LFU; tie-break by oldest first-insertion
+- [ ] 7.4 `src/eviction/fifo/fifo.ts` — insertion-ordered `Map` FIFO
+- [ ] 7.5 `src/eviction/ttl/ttl.ts` — `TtlOnlyPolicy` — never evicts on count/size; always returns `[]` from `onSet`
+- [ ] 7.6 Each policy ships `<name>.test.ts` (property tests for ordering invariants, boundary `maxEntries` / `maxBytes`, `reset` + `snapshot` correctness) and `<name>.md` (≥200 words, basic + advanced examples, API reference, complexity notes)
+- [ ] 7.7 `src/eviction/index.ts` — explicit named re-exports of every policy and the types
+- [ ] 7.8 Files stay under the 300-line soft cap
+
+## 8. Stampede protection (§spec REQ-16)
+
+- [ ] 8.1 `src/stampede/inflight-map/inflight-map.ts` — `InflightMap<T>` with `run(key, loader, timeoutMs?)`; concurrent `run` of the same key returns the existing promise; timeout rejects with `CACHE_STORAGE.STAMPEDE_TIMEOUT`; map entry cleared in `finally`; `pending` counter
+- [ ] 8.2 `src/stampede/inflight-map/inflight-map.test.ts` — concurrent-loader dedup test using an injected `FakeTimers`; timeout test; loader-error propagation test
+- [ ] 8.3 `src/stampede/inflight-map/inflight-map.md` — sibling docs
+
+## 9. Background sweep (§spec REQ-17)
+
+- [ ] 9.1 `src/sweep/ttl-sweeper/ttl-sweeper.ts` — `TtlSweeper` class with `start()` / `stop()`; idempotent; pages via `storage.list({ prefix, limit, cursor })`; header-only probe per key; `onSweepError` callback feeds `SWEEP_FAILED`
+- [ ] 9.2 `src/sweep/ttl-sweeper/ttl-sweeper.test.ts` — fixture-driven test: seed expired + fresh entries in a `MemoryStorage`, drive the `FakeTimers`, assert expired entries are deleted and `expire` events emit
+- [ ] 9.3 `src/sweep/ttl-sweeper/ttl-sweeper.md` — sibling docs
+
+## 10. Stale-while-revalidate (§spec REQ-18)
+
+- [ ] 10.1 `src/swr/swr-scheduler/swr-scheduler.ts` — pure scheduler: `shouldServeStale(meta, now, staleThresholdMs)`, `scheduleRefresh(refresh: () => Promise<void>, timers)`; refresh failures invoke an error sink
+- [ ] 10.2 `src/swr/swr-scheduler/swr-scheduler.test.ts` — table tests over (now, expiresAt, staleThresholdMs) tuples; refresh-failure path increments error sink
+- [ ] 10.3 `src/swr/swr-scheduler/swr-scheduler.md` — sibling docs
+
+## 11. Observability (§spec REQ-19..REQ-20)
+
+- [ ] 11.1 `src/observability/metrics/metrics.ts` — `MetricsRecorder` builds counters/histogram from `@opentelemetry/api` `metrics.getMeter("@suaveplan/cache-storage")`; no-op safe
+- [ ] 11.2 `src/observability/trace/trace.ts` — `TraceRecorder.span(name, attrs, fn)` opens/closes spans via `trace.getTracer("@suaveplan/cache-storage")`; sets `SpanStatusCode.ERROR` + `error.code` on `SuaveplanError`
+- [ ] 11.3 Tests for both via an injected in-memory tracer/meter (no real exporter required); assert counter increments and span attributes
+- [ ] 11.4 Sibling `.md` for each
+
+## 12. Per-namespace stats (§spec REQ-21)
+
+- [ ] 12.1 `src/stats/namespace-stats/namespace-stats.ts` — `NamespaceStats` keyed by namespace name; `record(name, kind: "hit" | "miss" | "evict" | "rejected" | "stampedePrevented" | "swrRefresh")`; `snapshot(name)` returns counters
+- [ ] 12.2 Tests + sibling `.md`
+- [ ] 12.3 Wire into `StorageCache.namespace(sub)` factory method which returns a sub-namespaced view that records to the same stats collector
+
+## 13. StorageCache integration (§spec REQ-1, REQ-22..REQ-27)
+
+- [ ] 13.1 Modify `src/storage-cache/storage-cache.ts` to compose every collaborator (codec, policy, sweeper, in-flight map, swr scheduler, metrics, trace, namespace-stats). Defaults preserve current behaviour: no policy ⇒ `TtlOnlyPolicy`, no sweep config ⇒ sweeper never starts, no `stampede` option ⇒ in-flight map unused, no SWR ⇒ `wrap` matches parent semantics.
+- [ ] 13.2 Modify `src/storage-cache/storage-cache.types.ts` — extend `StorageCacheOptions` with `eviction?`, `sweep?`, `limits?`, `timers?`, `tracer?`, `metrics?`. Every field is optional; every default preserves today's behaviour.
+- [ ] 13.3 Implement `mget`, `mset`, `mdelete` with bulk-aware dispatch (duck-typed `getMultiple`/`setMultiple` on `AbstractStorage`)
+- [ ] 13.4 Implement `getOrSet(key, loader, options?)` — `wrap` with stampede on by default
+- [ ] 13.5 Implement `compareAndSet(key, expectedVersion, value, options?)` and `getWithVersion(key)` — read header, compare, write with `version + 1`; mismatch ⇒ `CAS_CONFLICT`
+- [ ] 13.6 Override `wrap` to add stampede + SWR + OTel span; observationally equivalent to parent when both opts are absent
+- [ ] 13.7 Implement `start()` / `stop()` lifecycle that drives the optional `TtlSweeper`
+- [ ] 13.8 Implement `namespace(sub: string): StorageCache<T>` — returns a child view sharing the same storage but with a deeper prefix; collision detection ⇒ `NAMESPACE_CONFLICT`
+- [ ] 13.9 Keep `storage-cache.ts` under the 300-line soft cap by delegating to collaborators
+- [ ] 13.10 Extend `src/storage-cache/storage-cache.test.ts` to cover every new path; add the conformance harness invocation against `MemoryStorage`
+- [ ] 13.11 Update `src/storage-cache/storage-cache.md` to describe every new option, the `namespace(sub)` factory, CAS semantics, SWR semantics, the stampede contract, and the OTel surface
+
+## 14. Multi-backend conformance harness (§spec REQ-28)
+
+- [ ] 14.1 `src/testing/conformance/conformance.ts` — `runStorageCacheConformance({ createStorage, name })`; runs every public method, every eviction policy, stampede, SWR, CAS, and asserts every one of the 12 error codes is reachable
+- [ ] 14.2 `src/testing/conformance/conformance.test.ts` — invoke against `MemoryStorage` (the only universal-tier storage available here) and assert it passes; a deliberately broken fake storage causes failure
+- [ ] 14.3 `src/testing/conformance/conformance.md` — sibling docs explaining how a backend package (`cache-redis`, future `cache-opfs`) wires its own factory
+
+## 15. Root barrel + README
+
+- [ ] 15.1 Update `src/index.ts` with explicit named re-exports for every new public surface (codecs, eviction policies, in-flight map, sweeper, SWR scheduler, observability, namespace stats, conformance harness) — no `export *`
+- [ ] 15.2 Update `packages/flow/cache-storage/README.md` Modules section with every new co-located `.md`
+- [ ] 15.3 Update README Quick Start with a sample that exercises eviction + SWR + CAS
+
+## 16. Tests + gates (§13 order)
+
+- [ ] 16.1 Spec reconciliation — walk every `### Requirement` / `#### Scenario` in the delta against implementation; every named class/function/code exists in `src/` and is exercised by a test whose name matches the `**Verified by:**` line verbatim
+- [ ] 16.2 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for async / sweep tests)
+- [ ] 16.3 All test imports come from `@suaveplan/testing/runner` — zero direct `vitest` / `bun:test` imports (§7)
+- [ ] 16.4 `bunx turbo lint typecheck test --filter=@suaveplan/cache-storage`
+- [ ] 16.5 `bunx turbo test:coverage --filter=@suaveplan/cache-storage` — 100/100/100/100
+- [ ] 16.6 `bunx turbo build --filter=@suaveplan/cache-storage`
+- [ ] 16.7 Banned-pattern scan, barrel-shape scan, runner-import scan, missing-`.md` scan, bare-version scan (zero output each)
+- [ ] 16.8 `bunx turbo test --filter=...@suaveplan/cache-storage` — dependent sweep; `@suaveplan/cache-redis` MUST stay green (header v2 is forward-compatible; bulk-op dispatch is duck-typed)
+
+## 17. Root-doc refresh + archival
+
+- [ ] 17.1 `bun scripts/folder-reorg/generate-manifest.ts` — refresh `MANIFEST.md`
+- [ ] 17.2 `bun scripts/generate-missing.ts` — refresh `MISSING.yml`
+- [ ] 17.3 Update `STATUS.md` and `SUMMARY.md` to reflect the new surface
+- [ ] 17.4 `bunx markdownlint-cli2 --fix`
+- [ ] 17.5 Fold `specs/cache-storage/spec.md` into `openspec/specs/cache-storage/spec.md` (merge `## ADDED` block into existing spec; apply `## MODIFIED` blocks)
+- [ ] 17.6 Move change folder → `openspec/archive/<yyyy-mm-dd>-complete-cache-storage/`