Initial Commit

openspec/archive/2026-04-23-enterprise-storage/proposal.md

@@ -0,0 +1,361 @@
+# OpenSpec Change: enterprise-storage
+
+- **Change ID**: enterprise-storage
+- **Package**: `@suaveplan/storage`
+- **Category**: universal
+- **Target path**: `universal/storage/storage`
+- **Status**: IN PROGRESS
+- **Date**: 2026-04-23
+- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/storage.md`
+
+## Summary
+
+Bootstrap `@suaveplan/storage` on genesis from scratch. This package provides the core key-value storage abstractions used by all downstream packages that need persistent or in-process state: `@suaveplan/drive`, `@suaveplan/idempotency`, `@suaveplan/logger` (StorageLogger), `@suaveplan/crypto-keys` (Keychain serialized-JSON adapter), and others.
+
+The genesis rewrite preserves the full public API surface from the main branch and adds enterprise-grade enhancements: `StorageLike` implementation on `AbstractStorage`, streaming primitives (`createReadStream`/`createWriteStream`/`getPartial`/`getItemSize`) required by the drive layer, structured errors extending `@suaveplan/error` with stable `STORAGE.*` codes, Zod schemas for every config and options shape, OTel counters and histograms on all public operations, `EventBusLike` integration replacing the old `EventEmitter` re-export, `AbortSignal` on every async method, property-based tests (fast-check), and a mitata benchmark suite.
+
+## Current-state audit (genesis)
+
+The package does not exist on genesis yet — this is a fresh build.
+
+## Available workspace deps on genesis
+
+| Dependency | Catalog / pin | Available |
+|---|---|---|
+| `@suaveplan/error` | `workspace:^` | ✅ |
+| `@suaveplan/types` | `workspace:^` | ✅ (includes `StorageLike`) |
+| `@suaveplan/events` | `workspace:^` | ✅ |
+| `zod` | `catalog:forms` | ✅ |
+| `@opentelemetry/api` | `catalog:observability` | ✅ |
+| `tslib` | `catalog:build` | ✅ |
+| `vitest` | `catalog:testing` (devDep) | ✅ |
+| `@vitest/coverage-istanbul` | `catalog:testing` (devDep) | ✅ |
+| `fast-check` | `catalog:testing` (devDep) | ✅ |
+| `mitata` | `catalog:testing` (devDep) | ✅ |
+
+## Preserved public API surface (from main branch)
+
+### Errors (`src/errors/`)
+
+```ts
+// Extend @suaveplan/error via defineCodes — not ad-hoc subclasses
+class StorageError extends SuaveplanError      // code: STORAGE.GENERIC
+class StorageNotFoundError extends StorageError // code: STORAGE.NOT_FOUND
+class StoragePermissionError extends StorageError // code: STORAGE.PERMISSION_DENIED
+class StorageQuotaError extends StorageError   // code: STORAGE.QUOTA_EXCEEDED
+// New on genesis:
+class StorageInvalidKeyError extends StorageError // code: STORAGE.INVALID_KEY
+class StorageBatchError extends StorageError    // code: STORAGE.BATCH_ERROR
+```
+
+### Types (`src/types/`)
+
+```ts
+// All from main branch — Zod-validated on genesis
+interface StorageOptions {
+  ttl?: number;
+  metadata?: Record<string, unknown>;
+  compress?: boolean;
+  encrypt?: boolean;
+}
+
+interface StorageMetadata {
+  createdAt: number;
+  modifiedAt: number;
+  expiresAt?: number;
+  size: number;
+  metadata?: Record<string, unknown>;
+}
+
+interface StorageResult<T> {
+  value: T;
+  metadata: StorageMetadata;
+}
+
+interface StorageListOptions {
+  limit?: number;
+  offset?: number;
+  prefix?: string;
+  continuationToken?: string;
+}
+
+interface StorageListResult {
+  keys: string[];
+  total: number;
+  hasMore: boolean;
+  continuationToken?: string;
+}
+
+interface StoragePatternOptions extends StorageListOptions {
+  pattern?: string;
+}
+
+type StorageBatchOperation =
+  | { type: 'get'; key: string }
+  | { type: 'set'; key: string; value: Uint8Array; options?: StorageOptions }
+  | { type: 'delete'; key: string };
+
+interface StorageBatchResult {
+  operation: StorageBatchOperation;
+  success: boolean;
+  value?: Uint8Array | null;
+  error?: Error;
+}
+
+interface StorageStats {
+  reads: number;
+  writes: number;
+  deletes: number;
+  hits: number;
+  misses: number;
+  bytesRead: number;
+  bytesWritten: number;
+  keyCount?: number;
+  totalGets?: number;
+  totalSets?: number;
+  totalDeletes?: number;
+  totalErrors?: number;
+  totalSize?: number;
+}
+
+// Event types — replaces EventEmitter-based surface with EventBusLike
+type StorageEventType = 'set' | 'get' | 'delete' | 'clear' | 'error';
+
+interface StorageEventData {
+  key?: string;
+  value?: Uint8Array | null;
+  error?: Error;
+}
+
+type StorageEventHandler = (data: StorageEventData) => void;
+
+// Hook types (unchanged from main)
+interface StoragePreSetHook {
+  name?: string;
+  priority?: number;
+  transform(key: string, value: Uint8Array, options?: StorageOptions):
+    Promise<{ key: string; value: Uint8Array; options?: StorageOptions }>;
+}
+
+interface StoragePostGetHook {
+  name?: string;
+  priority?: number;
+  transform(key: string, value: Uint8Array): Promise<Uint8Array>;
+}
+```
+
+### Factory types (`src/factory/`)
+
+```ts
+// Module-augmentation pattern preserved
+interface StorageProviderRegistry {
+  memory: MemoryStorage;
+  // downstream packages augment this
+}
+
+type StorageProviderType = keyof StorageProviderRegistry;
+type StorageProviderImplementation<T extends StorageProviderType> = StorageProviderRegistry[T];
+
+interface StorageProviderConfig {
+  memory?: MemoryStorageConfig;
+}
+
+interface MemoryStorageConfig {
+  maxSize?: number;
+  cleanupIntervalMs?: number; // surfaced from MemoryStorage constructor
+}
+
+type StorageProviderConfigFor<T extends StorageProviderType> = ...;
+
+interface StorageFactoryConfig<T extends StorageProviderType> {
+  type: T;
+  config?: StorageProviderConfigFor<T>;
+}
+
+// Alias export (backwards compat)
+type InMemory = MemoryStorage;
+```
+
+### AbstractStorage (`src/abstract/`)
+
+All existing public methods preserved, with `AbortSignal` added and `StorageLike` conformance:
+
+```ts
+abstract class AbstractStorage implements StorageLike {
+  // StorageLike contract (adapted: get returns Uint8Array | undefined, delete returns boolean)
+  abstract get(key: string, signal?: AbortSignal): Promise<Uint8Array | undefined>;
+  abstract set(key: string, value: Uint8Array, options?: StorageOptions, signal?: AbortSignal): Promise<void>;
+  abstract delete(key: string, signal?: AbortSignal): Promise<boolean>;
+  abstract exists(key: string, signal?: AbortSignal): Promise<boolean>;
+  abstract list(options?: StorageListOptions, signal?: AbortSignal): Promise<StorageListResult>;
+  abstract clear(signal?: AbortSignal): Promise<void>;
+
+  // Extended beyond StorageLike
+  abstract getWithMetadata(key: string, signal?: AbortSignal): Promise<StorageResult<Uint8Array> | null>;
+  abstract getMetadata(key: string, signal?: AbortSignal): Promise<StorageMetadata | null>;
+  abstract getSize(signal?: AbortSignal): Promise<number>;
+
+  // Streaming (new on genesis — required by @suaveplan/drive)
+  abstract getPartial(key: string, offset: number, length: number, signal?: AbortSignal): Promise<Uint8Array | null>;
+  abstract getItemSize(key: string, signal?: AbortSignal): Promise<number | null>;
+  abstract createReadStream(key: string, signal?: AbortSignal): ReadableStream<Uint8Array>;
+  abstract createWriteStream(key: string, signal?: AbortSignal): WritableStream<Uint8Array>;
+
+  // Concrete helpers on AbstractStorage (all accept optional AbortSignal)
+  batch(operations: StorageBatchOperation[], signal?: AbortSignal): Promise<StorageBatchResult[]>;
+  getdel(key: string, signal?: AbortSignal): Promise<Uint8Array | null>;
+  mdelete(keys: string[], signal?: AbortSignal): Promise<number>;
+  listWithPattern(options?: StoragePatternOptions, signal?: AbortSignal): Promise<StorageListResult>;
+  namespace(prefix: string): NamespacedStorage;
+
+  // Size limits
+  setMaxValueSize(bytes: number): void;
+  setMaxTotalSize(bytes?: number): void;
+
+  // Stats
+  getStats(): StorageStats;
+  resetStats(): void;
+
+  // Event hooks (EventBusLike-backed on genesis)
+  on(event: StorageEventType, handler: StorageEventHandler): void;
+  off(event: StorageEventType, handler: StorageEventHandler): void;
+
+  // Transform hooks
+  addPreSetHook(hook: StoragePreSetHook): void;
+  addPostGetHook(hook: StoragePostGetHook): void;
+  removeHook(name?: string): void;
+}
+```
+
+### NamespacedStorage (`src/abstract/`)
+
+```ts
+class NamespacedStorage extends AbstractStorage {
+  constructor(storage: AbstractStorage, prefix: string);
+  // Delegates all operations to underlying storage with prefix: key → `${prefix}:${key}`
+  // clear() only removes keys in this namespace
+  // getSize() returns scoped byte count (not total — fixed from main branch approximation)
+}
+```
+
+### MemoryStorage (`src/memory/`)
+
+```ts
+class MemoryStorage extends AbstractStorage {
+  constructor(options?: MemoryStorageConfig);
+
+  // Implements all AbstractStorage abstracts
+  get size(): number;       // number of live (non-expired) keys
+  destroy(): void;          // stops cleanup interval
+
+  // New on genesis
+  getPartial(key, offset, length, signal?): Promise<Uint8Array | null>;
+  getItemSize(key, signal?): Promise<number | null>;
+  createReadStream(key, signal?): ReadableStream<Uint8Array>;
+  createWriteStream(key, signal?): WritableStream<Uint8Array>;
+}
+```
+
+### Factory functions (`src/factory/`)
+
+```ts
+function registerStorageProvider<T extends string>(
+  type: T,
+  implementation: new (config?: unknown) => AbstractStorage
+): void;
+
+function createStorage<T extends StorageProviderType>(
+  options: StorageFactoryConfig<T>
+): StorageProviderImplementation<T>;
+
+function createStorageWithConfig<T extends StorageProviderType>(
+  type: T,
+  config?: StorageProviderConfigFor<T>
+): StorageProviderImplementation<T>;
+
+function hasStorageProvider(type: string): boolean;
+function getStorageProviders(): string[];
+function unregisterStorageProvider(type: string): void;
+function clearStorageProviders(): void;  // resets to memory-only
+
+// Object facade (legacy compat — preserved, not deprecated on genesis)
+const StorageFactory: {
+  register: typeof registerStorageProvider;
+  create: typeof createStorage;
+  createWithConfig: typeof createStorageWithConfig;
+  hasProvider: typeof hasStorageProvider;
+  getProviders: typeof getStorageProviders;
+  unregister: typeof unregisterStorageProvider;
+  clearProviders: typeof clearStorageProviders;
+};
+```
+
+## Dropped from main branch
+
+| Symbol | Reason |
+|---|---|
+| `export { EventEmitter } from './utils/event-emitter.js'` | `EventEmitter` was already just a re-export of `@suaveplan/events` on main. Genesis drops the pass-through; callers import directly from `@suaveplan/events`. `AbstractStorage` now accepts an optional injected `EventBusLike`. |
+
+## New additions for genesis
+
+### Structured errors (`src/errors/`)
+
+- `defineCodes` from `@suaveplan/error` for stable `STORAGE.*` code registry.
+- All error classes extend `SuaveplanError` (cause-chain, OTel span-event, wire format, redaction).
+- Added: `StorageInvalidKeyError` (was an inline `StorageError` with code `"INVALID_KEY"` on main), `StorageBatchError`.
+
+### Zod schemas (`src/types/`)
+
+- `StorageOptionsSchema`, `StorageMetadataSchema`, `StorageResultSchema`, `StorageListOptionsSchema`, `StorageListResultSchema`, `StoragePatternOptionsSchema`, `StorageBatchOperationSchema`, `StorageBatchResultSchema`, `StorageStatsSchema`, `MemoryStorageConfigSchema`.
+- Used for runtime boundary validation (factory config, hook inputs) and wire-format safety.
+
+### OTel instrumentation
+
+- Counters: `storage.ops` (labelled by `operation`, `provider`, `status`).
+- Histograms: `storage.op.duration_ms` (per operation latency), `storage.value.bytes` (read/write sizes).
+- Applied at the `AbstractStorage` concrete-method layer, not inside subclasses.
+
+### EventBusLike integration
+
+- `AbstractStorage` constructor accepts an optional `EventBusLike` (from `@suaveplan/types`).
+- Default: an internal minimal bus backed by `@suaveplan/events` `EventBus`.
+- `on`/`off` delegate to the bus; existing `StorageEventType`/`StorageEventHandler` surface preserved.
+
+### AbortSignal propagation
+
+- All async abstract methods and concrete helpers accept an optional trailing `signal?: AbortSignal`.
+- `MemoryStorage` checks `signal.aborted` at the top of each method and throws `StorageError` with code `STORAGE.ABORTED`.
+- `batch()` checks signal between operations and rolls back on abort.
+
+### Streaming primitives (required by `@suaveplan/drive`)
+
+- `getPartial(key, offset, length, signal?)` — byte-range read.
+- `getItemSize(key, signal?)` — size probe without data transfer.
+- `createReadStream(key, signal?)` — `ReadableStream<Uint8Array>` backed by `get()`.
+- `createWriteStream(key, signal?)` — `WritableStream<Uint8Array>` that accumulates chunks then calls `set()`.
+- Default implementations on `AbstractStorage` using `get`/`set` for subclasses that don't override.
+- `MemoryStorage` overrides all four with zero-copy slices from the internal `Uint8Array`.
+
+### Property-based tests (fast-check)
+
+- Key-validation invariants (arbitrary strings, path-traversal, length boundaries).
+- `batch()` rollback correctness under arbitrary failure injection.
+- Namespace roundtrip: `namespace(p).set(k, v)` then `namespace(p).get(k)` equals `v`.
+- TTL expiry: for any `ttl > 0`, a read after `ttl` ms must return `undefined`.
+- `listWithPattern` result consistency against brute-force `list()` + JS filter.
+
+### Benchmarks (`src/__benchmarks__/`)
+
+- `memory-ops.bench.ts` — throughput for `get`/`set`/`delete`/`list` on `MemoryStorage`.
+- `batch-ops.bench.ts` — batch vs sequential write throughput.
+- `namespace-ops.bench.ts` — namespace overhead vs direct access.
+
+### NamespacedStorage `getSize()` fix
+
+Main branch `getSize()` returned the total storage size rather than the scoped namespace size. Genesis implements a correct scoped byte count by iterating only keys under `this.prefix`.
+
+## Impact
+
+- **Unblocks**: `@suaveplan/drive` (needs `getPartial`/`createReadStream`/`createWriteStream`), `@suaveplan/idempotency` (MemoryStore adapter), `@suaveplan/crypto-keys` (Keychain serialized-JSON), `@suaveplan/logger` (StorageLogger sink).
+- **No breaking changes** to the main-branch public API surface beyond the `EventEmitter` re-export removal and the `get()` return type narrowing from `null` to `undefined` (to match `StorageLike`).
+- **`sideEffects: false`** confirmed — provider registration happens lazily inside `factory.ts`, not at module scope before the factory module is imported.