# Tasks: enterprise-storage Change ID: `enterprise-storage` Package: `@suaveplan/storage` Path: `universal/storage/storage` Work tasks in order. Each section is a logical gate — do not start the next section until all checkboxes in the current section are green and `bunx turbo lint typecheck test --filter=@suaveplan/storage` passes. --- ## 1. Scaffold - [x] Create `universal/storage/storage/` directory tree: ``` universal/storage/storage/ src/ abstract/ errors/ factory/ memory/ types/ __benchmarks__/ tests/ package.json tsconfig.json tsconfig.build.json vite.config.ts vitest.config.ts biome.json (extend root) ``` - [x] Write `package.json`: - `name`: `@suaveplan/storage`, `version`: `0.1.0`, `private`: false - `"sideEffects": false` - `type`: `"module"` - `main`/`module`/`types` pointing at `dist/` output - Subpath exports: - `"."` → `./dist/index.js` - `"./abstract"` → `./dist/abstract/index.js` - `"./errors"` → `./dist/errors/index.js` - `"./factory"` → `./dist/factory/index.js` - `"./memory"` → `./dist/memory/index.js` - `"./types"` → `./dist/types/index.js` - Runtime deps: `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/events workspace:^`, `zod catalog:forms`, `@opentelemetry/api catalog:observability`, `tslib catalog:build` - Dev deps: `vitest catalog:testing`, `@vitest/coverage-istanbul catalog:testing`, `fast-check catalog:testing`, `mitata catalog:testing` - Scripts: `build`, `test`, `test:coverage`, `test:bench`, `lint`, `typecheck`, `docs:check` - [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json` - [x] Write `tsconfig.build.json` (excludes test files and `__benchmarks__`) - [x] Write `vite.config.ts` for library build (entry `src/index.ts`, formats `es`/`cjs`, `dts: true`) - [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config`; set `provider: 'istanbul'`, `thresholds` at 100/100/100/100 - [x] Add package to `turbo.json` pipeline (inherits global tasks — confirm no extra config needed) - [x] Add `@suaveplan/storage` to root `bun.lockb` workspace list and run `bun install` - [x] Write guard test `tests/tree-shake.test.ts`: - Assert the built `dist/index.js` contains no `import ... from 'node:*'` (universal runtime constraint) - Assert `dist/index.js` byte size is tracked (soft limit alert at 50 KB) - [x] Confirm `bunx turbo build --filter=@suaveplan/storage` exits 0 (even with empty `src/index.ts`) --- ## 2. Types + Error Codes ### 2a. Error codes (`src/errors/`) - [x] Create `src/errors/errors.ts`: - Import `SuaveplanError`, `defineCodes` from `@suaveplan/error` - Define code registry with `defineCodes`: ``` STORAGE.GENERIC STORAGE.NOT_FOUND STORAGE.PERMISSION_DENIED STORAGE.QUOTA_EXCEEDED STORAGE.INVALID_KEY STORAGE.BATCH_ERROR STORAGE.ABORTED ``` - Implement `StorageError extends SuaveplanError` — constructor accepts `(message, code, key?, cause?)`; preserves `key` as a typed public field; emits OTel span event on construction - Implement `StorageNotFoundError extends StorageError` — constructor `(key, cause?)` - Implement `StoragePermissionError extends StorageError` — constructor `(key, operation, cause?)` - Implement `StorageQuotaError extends StorageError` — constructor `(key, cause?)` - Implement `StorageInvalidKeyError extends StorageError` — constructor `(key, reason, cause?)` - Implement `StorageBatchError extends StorageError` — constructor `(message, cause?)` - [x] Create `src/errors/errors.types.ts` with any shared error-related type aliases - [x] Create `src/errors/errors.md` (purpose, features, usage examples, API reference, implementation notes — ≥ 200 words) - [x] Create `src/errors/index.ts` barrel - [x] Create `src/errors/errors.test.ts`: - Instantiate each error class and assert `name`, `code`, `key`, `cause` fields - Assert each is `instanceof SuaveplanError` - Assert `StorageError` with each `STORAGE.*` code round-trips through `@suaveplan/error` wire format - `{ timeout: 5000 }` on all tests ### 2b. Types and Zod schemas (`src/types/`) - [x] Create `src/types/types.ts` — all interface definitions from the preserved surface (see proposal § Preserved public API): - `StorageOptions`, `StorageMetadata`, `StorageResult`, `StorageListOptions`, `StorageListResult`, `StoragePatternOptions`, `StorageBatchOperation`, `StorageBatchResult`, `StorageStats`, `StorageEventType`, `StorageEventData`, `StorageEventHandler`, `StoragePreSetHook`, `StoragePostGetHook` - All `metadata` fields typed `Record` (not `any`) - [x] Create `src/types/schemas.ts` — Zod schemas: - `StorageOptionsSchema`, `StorageMetadataSchema`, `StorageResultSchema` (generic via `z.object` + type param helper), `StorageListOptionsSchema`, `StorageListResultSchema`, `StoragePatternOptionsSchema`, `StorageBatchOperationSchema`, `StorageBatchResultSchema`, `StorageStatsSchema` - Export inferred types alongside schemas (e.g. `export type StorageOptions = z.infer`) - [x] Create `src/types/types.md` and `src/types/schemas.md` (≥ 200 words each) - [x] Create `src/types/index.ts` barrel - [x] Create `src/types/types.test.ts`: - Parse valid and invalid objects through every Zod schema - Assert discriminated union `StorageBatchOperationSchema` correctly narrows on `type` - `{ timeout: 5000 }` on all tests --- ## 3. AbstractStorage Core ### 3a. Implementation (`src/abstract/abstract.ts`) - [x] Declare `AbstractStorage implements StorageLike` (import from `@suaveplan/types`) - [x] Constructor accepts optional `eventBus?: EventBusLike` (from `@suaveplan/types`); defaults to `new EventBus()` from `@suaveplan/events` - [x] Internal `protected stats: StorageStats` initialized with all counters at 0 - [x] Internal `protected preSetHooks: StoragePreSetHook[]` and `protected postGetHooks: StoragePostGetHook[]` - [x] Internal `protected maxValueSize: number` defaulting to `10 * 1024 * 1024` - [x] Internal `protected maxTotalSize: number | undefined` - [x] Implement `protected validateKey(key: string): void` — checks: non-empty, no `..` or `\`, no absolute path, length ≤ 1024, no `<>"|*?`; throws `StorageInvalidKeyError` - [x] Implement `protected async validateValueSize(key, value, signal?)` — throws `StorageQuotaError` on single-value or total-quota breach - [x] Declare all abstract methods with `signal?: AbortSignal` on every async signature: - `get`, `set`, `delete`, `exists`, `list`, `clear` (StorageLike conformance) - `getWithMetadata`, `getMetadata`, `getSize` - `getPartial`, `getItemSize`, `createReadStream`, `createWriteStream` - [x] Provide default implementations for streaming methods that delegate to `get`/`set` (subclasses may override for efficiency): - `getPartial`: calls `get`, slices the resulting `Uint8Array` - `getItemSize`: calls `getMetadata`, returns `metadata.size` - `createReadStream`: enqueues single chunk from `get()` then closes - `createWriteStream`: accumulates all chunks, calls `set()` on close - [x] Implement concrete helpers: `batch`, `getdel`, `mdelete`, `listWithPattern` — all accept optional `signal`, check `signal?.aborted` at start and between async steps; throw `StorageError(STORAGE.ABORTED)` when aborted - [x] `batch()` rollback: on error or abort, reverse completed `set`/`delete` ops; use structured `console.warn` only if OTel is unavailable at that point, otherwise emit span event - [x] Implement `setMaxValueSize`, `setMaxTotalSize` - [x] Implement `getStats()`, `resetStats()` - [x] Implement `on(event, handler)` and `off(event, handler)` delegating to the injected `EventBusLike` - [x] Implement `addPreSetHook`, `addPostGetHook`, `removeHook` with priority sort - [x] Implement `protected async applyPreSetHooks(key, value, options?)` and `protected async applyPostGetHooks(key, value)` - [x] Implement `protected emitEvent(event, data)` — publishes to `EventBusLike` - [x] Implement `namespace(prefix): NamespacedStorage` ### 3b. OTel instrumentation (`src/abstract/abstract.ts` or extracted `src/abstract/telemetry.ts`) - [x] Import `metrics` from `@opentelemetry/api` - [x] Create meter `@suaveplan/storage` once (module-level singleton, lazy init) - [x] Counter `storage.ops` — attributes: `operation` (get/set/delete/exists/list/clear/batch/getdel/mdelete), `provider` (class name), `status` (success/error/aborted) - [x] Histogram `storage.op.duration_ms` — same attributes - [x] Histogram `storage.value.bytes` — attributes: `operation` (get/set), `provider` - [x] Instrument all concrete helper methods and wrap the OTel recording around the abstract method calls within those helpers; individual subclasses do not need to add OTel (they inherit it) ### 3c. NamespacedStorage (`src/abstract/namespace.ts`) - [x] `class NamespacedStorage extends AbstractStorage` - [x] Constructor: `(storage: AbstractStorage, prefix: string)` — validate prefix is non-empty, no `:` - [x] `prefixKey(key)` → `${prefix}:${key}`; `unprefixKey(key)` strips prefix - [x] Implement all abstract methods by delegating to underlying `storage` with prefixed key - [x] `list()` filters by `prefix:` prefix, strips prefix from returned keys, applies pagination correctly against filtered total - [x] `clear()` lists all keys in this namespace and calls `mdelete` on prefixed keys — scoped only, does not touch other namespaces - [x] `getSize()` iterates only namespace keys and sums `metadata.size` — fixes the main-branch approximate-total bug - [x] `getPartial`, `getItemSize`, `createReadStream`, `createWriteStream` delegate via `prefixKey` ### 3d. Docs and tests - [x] Create `src/abstract/abstract.md` (≥ 200 words, both basic and advanced examples) - [x] Create `src/abstract/namespace.md` (≥ 200 words) - [x] Create `src/abstract/index.ts` barrel - [x] Create `src/abstract/abstract.test.ts` — test via a minimal `ConcreteStorage` implementation (not `MemoryStorage`; keeps unit tests isolated): - `validateKey`: empty key, path-traversal variants, absolute paths, over-1024-chars, forbidden chars, valid key - `validateValueSize`: single-value limit, total-quota limit - `batch()`: happy path, rollback on mid-batch failure, abort signal mid-batch - `getdel()`: value returned and deleted, null when absent - `mdelete()`: returns count of actually-deleted keys - `listWithPattern()`: glob `*`, `?`, no-pattern passthrough, pagination - `on/off/emitEvent`: handler called on emit, removed after `off` - `addPreSetHook`/`addPostGetHook`/`removeHook`: priority ordering, named removal - `setMaxValueSize`/`setMaxTotalSize`: rejects invalid values - `getStats()`/`resetStats()`: snapshot copy, reset to zeros - `namespace()`: returns `NamespacedStorage` with correct prefix - AbortSignal: each helper aborts before work starts - `{ timeout: 5000 }` on sync tests; `{ timeout: 30000 }` on async tests - [x] Create `src/abstract/namespace.test.ts`: - Prefix isolation: keys in `ns('a')` never visible via `ns('b')` - `clear()`: only removes namespace-scoped keys - `getSize()`: returns scoped bytes only - `list()` pagination against namespace-filtered total - `{ timeout: 5000 }` / `{ timeout: 30000 }` as appropriate --- ## 4. MemoryStorage ### 4a. Implementation (`src/memory/memory.ts`) - [x] `class MemoryStorage extends AbstractStorage` - [x] Internal `Map` store - [x] Constructor accepts `MemoryStorageConfig` (Zod-validated): `maxSize?`, `cleanupIntervalMs?` (default 60000; 0 disables timer) - [x] Background TTL cleanup loop using `setInterval`; call `.unref()` in Node/Bun to avoid blocking exit - [x] `get(key, signal?)`: - Check `signal?.aborted`; throw `StorageError(STORAGE.ABORTED)` - Return `undefined` (not `null`) for missing/expired keys — matches `StorageLike` - Expire-on-read for TTL entries; update stats - Apply `postGetHooks`; emit `get` event - [x] `set(key, value, options?, signal?)`: - Check signal; `validateKey`; `validateValueSize`; apply `preSetHooks` - Write entry with metadata (`createdAt`, `modifiedAt`, `expiresAt`, `size`) - On error: roll back stats increments - [x] `delete(key, signal?)`: returns `boolean` (true if existed) — matches `StorageLike`; update stats; emit event - [x] `exists(key, signal?)`: expire-on-read; return boolean - [x] `list(options?, signal?)`: filter expired, filter by `prefix`, paginate, return `StorageListResult` - [x] `getWithMetadata(key, signal?)`: returns `StorageResult | null` - [x] `getMetadata(key, signal?)`: returns `StorageMetadata | null` without retrieving value bytes - [x] `clear(signal?)`: clears map; resets `keyCount`/`totalSize` stats; emits `clear` event - [x] `getSize(signal?)`: sums live (non-expired) entry byte lengths - [x] `get size(): number` — count of live (non-expired) entries - [x] `destroy()`: clears the cleanup interval; idempotent - [x] Streaming overrides (zero-copy from internal `Uint8Array`): - `getPartial(key, offset, length, signal?)`: `entry.value.subarray(offset, offset + length)` — handles out-of-range gracefully (clamp to actual size) - `getItemSize(key, signal?)`: `entry.metadata.size` without extra allocation - `createReadStream(key, signal?)`: `ReadableStream` that enqueues the stored `Uint8Array` (defensive copy so callers cannot mutate the store) - `createWriteStream(key, signal?)`: `WritableStream` that accumulates typed-array chunks then calls `this.set()` on close; pipes abort signal through `WritableStreamDefaultController` ### 4b. Types (`src/memory/memory.types.ts`) - [x] `MemoryEntry` — internal store entry shape - [x] `MemoryStorageConfig` — exported, used by `MemoryStorageConfigSchema` in `src/types/schemas.ts` ### 4c. Docs and tests - [x] Create `src/memory/memory.md` (≥ 200 words, includes TTL and streaming examples) - [x] Create `src/memory/index.ts` barrel - [x] Create `src/memory/memory.test.ts` (core operations): - `get`/`set`/`delete`/`exists`/`list`/`clear`/`getSize` happy paths - TTL: entry expires after `ttl` ms, returns `undefined` on late read - `getWithMetadata` / `getMetadata` field correctness - Stats accuracy across all operations including error paths - Size limit (`maxSize`) enforced on `set` - `destroy()` stops the cleanup timer - `get size` property reflects live key count - AbortSignal: each method aborts correctly - `{ timeout: 5000 }` / `{ timeout: 30000 }` as appropriate - [x] Create `src/memory/memory-streaming.test.ts`: - `getPartial`: byte-range slices, out-of-range clamping, missing key returns null - `getItemSize`: matches stored byte length - `createReadStream`: full round-trip matches original bytes - `createWriteStream`: multi-chunk write round-trips correctly via `get()` - AbortSignal on streaming methods - `{ timeout: 30000 }` on all - [x] Create `src/memory/memory-ttl-cleanup.test.ts`: - Expired entries removed by background cleanup (use `FakeClock` via injectable clock or mock `Date.now`) - Stats (`keyCount`, `totalSize`) correctly decremented after cleanup - `cleanupIntervalMs: 0` disables background timer - `{ timeout: 30000 }` on all - [x] Create `src/memory/memory-edge-cases.test.ts`: - Overwrite: `modifiedAt` updates, `createdAt` unchanged - Large value exactly at `maxValueSize` accepted; one byte over rejected - Empty `Uint8Array` stored and retrieved correctly - `clear()` then `getSize()` returns 0 - Concurrent async `set`s to the same key settle deterministically (last writer wins in single-threaded JS) - `{ timeout: 30000 }` on all - [x] Create `src/memory/memory-atomic.test.ts`: - `getdel()` is logically atomic: value returned and key absent after call - `mdelete()` returns correct count for mix of existing and missing keys - `{ timeout: 5000 }` / `{ timeout: 30000 }` as appropriate --- ## 5. NamespacedStorage (integration tests) - [x] Create `tests/namespace-integration.test.ts` (in `tests/` sibling, cross-cutting): - Full round-trip using `MemoryStorage` as the backing store: `set`→`get`→`getWithMetadata`→`delete`→`exists` - `list()` with prefix filter, pagination - Verify no key leakage: `ns('a').set('k', v)` is not visible via `ns('b').get('k')` - `getSize()` returns bytes for the namespace only - Nested namespaces: `ns1.namespace('sub')` works correctly - `{ timeout: 30000 }` on all --- ## 6. StorageFactory + Provider Registry ### 6a. Implementation (`src/factory/factory.ts`) - [x] Module-level `providers = new Map AbstractStorage>()` - [x] Auto-register `"memory"` → `MemoryStorage` at module initialisation (this is the only side effect; it is isolated to the factory module and does not affect tree-shaking of other exports) - [x] Implement all factory functions from the preserved surface (see proposal): - `registerStorageProvider(type, implementation): void` - `createStorage(options): StorageProviderImplementation` — throws `StorageError(STORAGE.GENERIC)` if provider not found (not raw `Error`) - `createStorageWithConfig(type, config?): StorageProviderImplementation` - `hasStorageProvider(type): boolean` - `getStorageProviders(): string[]` - `unregisterStorageProvider(type): void` - `clearStorageProviders(): void` — resets to memory-only - [x] `const StorageFactory = { register, create, createWithConfig, hasProvider, getProviders, unregister, clearProviders } as const` ### 6b. Factory types (`src/factory/factory.types.ts`) - [x] `StorageProviderRegistry` interface with `memory: MemoryStorage` - [x] `type InMemory = MemoryStorage` — alias export for backwards compat - [x] `StorageProviderType`, `StorageProviderImplementation`, `StorageProviderConfig`, `MemoryStorageConfig` (re-export from `memory.types.ts`), `StorageProviderConfigFor`, `StorageFactoryConfig` ### 6c. Docs and tests - [x] Create `src/factory/factory.md` (≥ 200 words, module-augmentation registration pattern example) - [x] Create `src/factory/index.ts` barrel - [x] Create `src/factory/factory.test.ts`: - `createStorage({ type: 'memory' })` returns a `MemoryStorage` instance - `createStorage` with unregistered type throws `StorageError` with code `STORAGE.GENERIC` - `registerStorageProvider` + `createStorage` with custom provider works - `hasStorageProvider` / `getStorageProviders` / `unregisterStorageProvider` / `clearStorageProviders` correctness - `StorageFactory` facade delegates to the same functions - `InMemory` type alias is exported (type-level test using `satisfies`) - `{ timeout: 5000 }` on all --- ## 7. OTel Instrumentation Verification - [x] Create `tests/otel-instrumentation.test.ts`: - Use a `TestSpanExporter` or in-memory OTel SDK setup from `@suaveplan/opentelemetry` (if available) or a minimal no-op; verify that span event / metric recording code paths are exercised without throwing - Assert `storage.ops` counter is incremented by 1 for each of: `get`, `set`, `delete`, `exists`, `list`, `clear` - Assert `storage.op.duration_ms` histogram is recorded for each operation - Assert `storage.value.bytes` histogram is recorded for `get` and `set` with the correct byte count - `{ timeout: 30000 }` on all --- ## 8. Property-Based Tests - [x] Create `tests/property-based.test.ts`: - **Key validation**: `fc.string()` — `validateKey` accepts all valid keys; rejects all strings containing `..`, `\`, `/`, `<>"|*?`, or longer than 1024 chars - **Set/get roundtrip**: `fc.tuple(fc.string({ minLength: 1, maxLength: 100 }).filter(k => !k.includes('..')), fc.uint8Array())` — `set(k, v)` then `get(k)` returns equal `Uint8Array` - **Namespace roundtrip**: `fc.tuple(fc.string(...), fc.string(...), fc.uint8Array())` — `namespace(p).set(k, v)` then `namespace(p).get(k)` equals `v`; `namespace(p2).get(k)` returns `undefined` for `p2 !== p` - **listWithPattern consistency**: for any store state and glob pattern, `listWithPattern({ pattern })` result is a subset of `list()` result and matches the pattern on every key - **Batch rollback**: inject a failure at a random operation index; assert all keys revert to pre-batch state - **TTL expiry**: `fc.nat({ max: 100 })` for ttl; advance time past ttl; `get` must return `undefined` - `{ timeout: 30000 }` on the `describe` block --- ## 9. Benchmarks - [x] Create `src/__benchmarks__/memory-ops.bench.ts`: - Benchmark `MemoryStorage` `get`, `set`, `delete` individually (1 000-key warm store) - Benchmark `list()` on a 10 000-key store with and without prefix - Use `mitata` `bench` and `group` API - [x] Create `src/__benchmarks__/batch-ops.bench.ts`: - Benchmark `batch()` of 100 mixed set/get/delete operations vs equivalent sequential calls - Measure rollback overhead by injecting a failure at index 50 - [x] Create `src/__benchmarks__/namespace-ops.bench.ts`: - Compare `MemoryStorage.get/set` direct vs through `NamespacedStorage` wrapper - `list()` with namespace prefix vs without - [x] Add `"test:bench": "vitest bench"` script to `package.json` - [x] Confirm `bun run test:bench --filter=@suaveplan/storage` runs without error and prints numbers --- ## 10. Docs Coverage Pass - [x] Ensure every `.ts` implementation file has a sibling `.md` (no co-located `.types.ts`, `.test.ts`, barrel `index.ts`, or `__benchmarks__` files require `.md`): - `src/errors/errors.md` ✓ (Task 2a) - `src/types/types.md` ✓ (Task 2b) - `src/types/schemas.md` ✓ (Task 2b) - `src/abstract/abstract.md` ✓ (Task 3d) - `src/abstract/namespace.md` ✓ (Task 3d) - `src/abstract/telemetry.md` (if telemetry extracted to its own file) - `src/memory/memory.md` ✓ (Task 4c) - `src/factory/factory.md` ✓ (Task 6c) - [x] Write root `README.md` at `universal/storage/storage/README.md`: - Package purpose (2+ sentences) - Installation snippet - Quick-start example (MemoryStorage + namespace + TTL) - Streaming example (createReadStream / createWriteStream) - Factory / custom provider registration example - Error handling example (StorageNotFoundError, StorageQuotaError) - Links to each co-located `.md` file - Dependency table - No category-level README inside `src/` subdirectories - [x] Run `bun run docs:check --filter=@suaveplan/storage` and fix any missing doc symbols --- ## 11. Verify and Publish - [x] Run the full pre-publish gate — all four commands must exit 0: ```bash bunx turbo lint --filter=@suaveplan/storage bunx turbo typecheck --filter=@suaveplan/storage bunx turbo test --filter=@suaveplan/storage bunx turbo test:coverage --filter=@suaveplan/storage ``` - [x] Confirm coverage thresholds 100/100/100/100 are met on all touched modules (istanbul reporter) - [x] Confirm `bun run docs:check --filter=@suaveplan/storage` reports 100% - [x] Confirm `bun run test:bench --filter=@suaveplan/storage` runs to completion and outputs benchmark numbers - [x] Confirm `sideEffects: false` is set in `package.json` and tree-shake guard test passes - [x] Review `src/` for any `TODO`, `FIXME`, `@deprecated`, `any`, `ts-ignore`, or `it.skip` / `describe.skip` — fix all before proceeding - [x] Add `@suaveplan/storage` to `CLAUDE.md` package listing (under Universal packages) with its full description and dep list - [x] Publish: `bun scripts/publish.ts --package @suaveplan/storage` - [x] Archive this OpenSpec change: move `openspec/changes/enterprise-storage/` to `openspec/archive/2026-04-23-enterprise-storage/` and fold spec deltas into `openspec/specs/storage/spec.md`