suaveplan/openspec/archive/2026-04-23-enterprise-storage/proposal.md

# OpenSpec Change: enterprise-storage

- **Change ID**: enterprise-storage
- **Package**: `@suaveplan/storage`
- **Category**: universal
- **Target path**: `universal/storage/storage`
- **Status**: IN PROGRESS
- **Date**: 2026-04-23
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/storage.md`

## Summary

Bootstrap `@suaveplan/storage` on genesis from scratch. This package provides the core key-value storage abstractions used by all downstream packages that need persistent or in-process state: `@suaveplan/drive`, `@suaveplan/idempotency`, `@suaveplan/logger` (StorageLogger), `@suaveplan/crypto-keys` (Keychain serialized-JSON adapter), and others.

The genesis rewrite preserves the full public API surface from the main branch and adds enterprise-grade enhancements: `StorageLike` implementation on `AbstractStorage`, streaming primitives (`createReadStream`/`createWriteStream`/`getPartial`/`getItemSize`) required by the drive layer, structured errors extending `@suaveplan/error` with stable `STORAGE.*` codes, Zod schemas for every config and options shape, OTel counters and histograms on all public operations, `EventBusLike` integration replacing the old `EventEmitter` re-export, `AbortSignal` on every async method, property-based tests (fast-check), and a mitata benchmark suite.

## Current-state audit (genesis)

The package does not exist on genesis yet — this is a fresh build.

## Available workspace deps on genesis

| Dependency | Catalog / pin | Available |
|---|---|---|
| `@suaveplan/error` | `workspace:^` | ✅ |
| `@suaveplan/types` | `workspace:^` | ✅ (includes `StorageLike`) |
| `@suaveplan/events` | `workspace:^` | ✅ |
| `zod` | `catalog:forms` | ✅ |
| `@opentelemetry/api` | `catalog:observability` | ✅ |
| `tslib` | `catalog:build` | ✅ |
| `vitest` | `catalog:testing` (devDep) | ✅ |
| `@vitest/coverage-istanbul` | `catalog:testing` (devDep) | ✅ |
| `fast-check` | `catalog:testing` (devDep) | ✅ |
| `mitata` | `catalog:testing` (devDep) | ✅ |

## Preserved public API surface (from main branch)

### Errors (`src/errors/`)

```ts
// Extend @suaveplan/error via defineCodes — not ad-hoc subclasses
class StorageError extends SuaveplanError      // code: STORAGE.GENERIC
class StorageNotFoundError extends StorageError // code: STORAGE.NOT_FOUND
class StoragePermissionError extends StorageError // code: STORAGE.PERMISSION_DENIED
class StorageQuotaError extends StorageError   // code: STORAGE.QUOTA_EXCEEDED
// New on genesis:
class StorageInvalidKeyError extends StorageError // code: STORAGE.INVALID_KEY
class StorageBatchError extends StorageError    // code: STORAGE.BATCH_ERROR
```

### Types (`src/types/`)

```ts
// All from main branch — Zod-validated on genesis
interface StorageOptions {
  ttl?: number;
  metadata?: Record<string, unknown>;
  compress?: boolean;
  encrypt?: boolean;
}

interface StorageMetadata {
  createdAt: number;
  modifiedAt: number;
  expiresAt?: number;
  size: number;
  metadata?: Record<string, unknown>;
}

interface StorageResult<T> {
  value: T;
  metadata: StorageMetadata;
}

interface StorageListOptions {
  limit?: number;
  offset?: number;
  prefix?: string;
  continuationToken?: string;
}

interface StorageListResult {
  keys: string[];
  total: number;
  hasMore: boolean;
  continuationToken?: string;
}

interface StoragePatternOptions extends StorageListOptions {
  pattern?: string;
}

type StorageBatchOperation =
  | { type: 'get'; key: string }
  | { type: 'set'; key: string; value: Uint8Array; options?: StorageOptions }
  | { type: 'delete'; key: string };

interface StorageBatchResult {
  operation: StorageBatchOperation;
  success: boolean;
  value?: Uint8Array | null;
  error?: Error;
}

interface StorageStats {
  reads: number;
  writes: number;
  deletes: number;
  hits: number;
  misses: number;
  bytesRead: number;
  bytesWritten: number;
  keyCount?: number;
  totalGets?: number;
  totalSets?: number;
  totalDeletes?: number;
  totalErrors?: number;
  totalSize?: number;
}

// Event types — replaces EventEmitter-based surface with EventBusLike
type StorageEventType = 'set' | 'get' | 'delete' | 'clear' | 'error';

interface StorageEventData {
  key?: string;
  value?: Uint8Array | null;
  error?: Error;
}

type StorageEventHandler = (data: StorageEventData) => void;

// Hook types (unchanged from main)
interface StoragePreSetHook {
  name?: string;
  priority?: number;
  transform(key: string, value: Uint8Array, options?: StorageOptions):
    Promise<{ key: string; value: Uint8Array; options?: StorageOptions }>;
}

interface StoragePostGetHook {
  name?: string;
  priority?: number;
  transform(key: string, value: Uint8Array): Promise<Uint8Array>;
}
```

### Factory types (`src/factory/`)

```ts
// Module-augmentation pattern preserved
interface StorageProviderRegistry {
  memory: MemoryStorage;
  // downstream packages augment this
}

type StorageProviderType = keyof StorageProviderRegistry;
type StorageProviderImplementation<T extends StorageProviderType> = StorageProviderRegistry[T];

interface StorageProviderConfig {
  memory?: MemoryStorageConfig;
}

interface MemoryStorageConfig {
  maxSize?: number;
  cleanupIntervalMs?: number; // surfaced from MemoryStorage constructor
}

type StorageProviderConfigFor<T extends StorageProviderType> = ...;

interface StorageFactoryConfig<T extends StorageProviderType> {
  type: T;
  config?: StorageProviderConfigFor<T>;
}

// Alias export (backwards compat)
type InMemory = MemoryStorage;
```

### AbstractStorage (`src/abstract/`)

All existing public methods preserved, with `AbortSignal` added and `StorageLike` conformance:

```ts
abstract class AbstractStorage implements StorageLike {
  // StorageLike contract (adapted: get returns Uint8Array | undefined, delete returns boolean)
  abstract get(key: string, signal?: AbortSignal): Promise<Uint8Array | undefined>;
  abstract set(key: string, value: Uint8Array, options?: StorageOptions, signal?: AbortSignal): Promise<void>;
  abstract delete(key: string, signal?: AbortSignal): Promise<boolean>;
  abstract exists(key: string, signal?: AbortSignal): Promise<boolean>;
  abstract list(options?: StorageListOptions, signal?: AbortSignal): Promise<StorageListResult>;
  abstract clear(signal?: AbortSignal): Promise<void>;

  // Extended beyond StorageLike
  abstract getWithMetadata(key: string, signal?: AbortSignal): Promise<StorageResult<Uint8Array> | null>;
  abstract getMetadata(key: string, signal?: AbortSignal): Promise<StorageMetadata | null>;
  abstract getSize(signal?: AbortSignal): Promise<number>;

  // Streaming (new on genesis — required by @suaveplan/drive)
  abstract getPartial(key: string, offset: number, length: number, signal?: AbortSignal): Promise<Uint8Array | null>;
  abstract getItemSize(key: string, signal?: AbortSignal): Promise<number | null>;
  abstract createReadStream(key: string, signal?: AbortSignal): ReadableStream<Uint8Array>;
  abstract createWriteStream(key: string, signal?: AbortSignal): WritableStream<Uint8Array>;

  // Concrete helpers on AbstractStorage (all accept optional AbortSignal)
  batch(operations: StorageBatchOperation[], signal?: AbortSignal): Promise<StorageBatchResult[]>;
  getdel(key: string, signal?: AbortSignal): Promise<Uint8Array | null>;
  mdelete(keys: string[], signal?: AbortSignal): Promise<number>;
  listWithPattern(options?: StoragePatternOptions, signal?: AbortSignal): Promise<StorageListResult>;
  namespace(prefix: string): NamespacedStorage;

  // Size limits
  setMaxValueSize(bytes: number): void;
  setMaxTotalSize(bytes?: number): void;

  // Stats
  getStats(): StorageStats;
  resetStats(): void;

  // Event hooks (EventBusLike-backed on genesis)
  on(event: StorageEventType, handler: StorageEventHandler): void;
  off(event: StorageEventType, handler: StorageEventHandler): void;

  // Transform hooks
  addPreSetHook(hook: StoragePreSetHook): void;
  addPostGetHook(hook: StoragePostGetHook): void;
  removeHook(name?: string): void;
}
```

### NamespacedStorage (`src/abstract/`)

```ts
class NamespacedStorage extends AbstractStorage {
  constructor(storage: AbstractStorage, prefix: string);
  // Delegates all operations to underlying storage with prefix: key → `${prefix}:${key}`
  // clear() only removes keys in this namespace
  // getSize() returns scoped byte count (not total — fixed from main branch approximation)
}
```

### MemoryStorage (`src/memory/`)

```ts
class MemoryStorage extends AbstractStorage {
  constructor(options?: MemoryStorageConfig);

  // Implements all AbstractStorage abstracts
  get size(): number;       // number of live (non-expired) keys
  destroy(): void;          // stops cleanup interval

  // New on genesis
  getPartial(key, offset, length, signal?): Promise<Uint8Array | null>;
  getItemSize(key, signal?): Promise<number | null>;
  createReadStream(key, signal?): ReadableStream<Uint8Array>;
  createWriteStream(key, signal?): WritableStream<Uint8Array>;
}
```

### Factory functions (`src/factory/`)

```ts
function registerStorageProvider<T extends string>(
  type: T,
  implementation: new (config?: unknown) => AbstractStorage
): void;

function createStorage<T extends StorageProviderType>(
  options: StorageFactoryConfig<T>
): StorageProviderImplementation<T>;

function createStorageWithConfig<T extends StorageProviderType>(
  type: T,
  config?: StorageProviderConfigFor<T>
): StorageProviderImplementation<T>;

function hasStorageProvider(type: string): boolean;
function getStorageProviders(): string[];
function unregisterStorageProvider(type: string): void;
function clearStorageProviders(): void;  // resets to memory-only

// Object facade (legacy compat — preserved, not deprecated on genesis)
const StorageFactory: {
  register: typeof registerStorageProvider;
  create: typeof createStorage;
  createWithConfig: typeof createStorageWithConfig;
  hasProvider: typeof hasStorageProvider;
  getProviders: typeof getStorageProviders;
  unregister: typeof unregisterStorageProvider;
  clearProviders: typeof clearStorageProviders;
};
```

## Dropped from main branch

| Symbol | Reason |
|---|---|
| `export { EventEmitter } from './utils/event-emitter.js'` | `EventEmitter` was already just a re-export of `@suaveplan/events` on main. Genesis drops the pass-through; callers import directly from `@suaveplan/events`. `AbstractStorage` now accepts an optional injected `EventBusLike`. |

## New additions for genesis

### Structured errors (`src/errors/`)

- `defineCodes` from `@suaveplan/error` for stable `STORAGE.*` code registry.
- All error classes extend `SuaveplanError` (cause-chain, OTel span-event, wire format, redaction).
- Added: `StorageInvalidKeyError` (was an inline `StorageError` with code `"INVALID_KEY"` on main), `StorageBatchError`.

### Zod schemas (`src/types/`)

- `StorageOptionsSchema`, `StorageMetadataSchema`, `StorageResultSchema`, `StorageListOptionsSchema`, `StorageListResultSchema`, `StoragePatternOptionsSchema`, `StorageBatchOperationSchema`, `StorageBatchResultSchema`, `StorageStatsSchema`, `MemoryStorageConfigSchema`.
- Used for runtime boundary validation (factory config, hook inputs) and wire-format safety.

### OTel instrumentation

- Counters: `storage.ops` (labelled by `operation`, `provider`, `status`).
- Histograms: `storage.op.duration_ms` (per operation latency), `storage.value.bytes` (read/write sizes).
- Applied at the `AbstractStorage` concrete-method layer, not inside subclasses.

### EventBusLike integration

- `AbstractStorage` constructor accepts an optional `EventBusLike` (from `@suaveplan/types`).
- Default: an internal minimal bus backed by `@suaveplan/events` `EventBus`.
- `on`/`off` delegate to the bus; existing `StorageEventType`/`StorageEventHandler` surface preserved.

### AbortSignal propagation

- All async abstract methods and concrete helpers accept an optional trailing `signal?: AbortSignal`.
- `MemoryStorage` checks `signal.aborted` at the top of each method and throws `StorageError` with code `STORAGE.ABORTED`.
- `batch()` checks signal between operations and rolls back on abort.

### Streaming primitives (required by `@suaveplan/drive`)

- `getPartial(key, offset, length, signal?)` — byte-range read.
- `getItemSize(key, signal?)` — size probe without data transfer.
- `createReadStream(key, signal?)` — `ReadableStream<Uint8Array>` backed by `get()`.
- `createWriteStream(key, signal?)` — `WritableStream<Uint8Array>` that accumulates chunks then calls `set()`.
- Default implementations on `AbstractStorage` using `get`/`set` for subclasses that don't override.
- `MemoryStorage` overrides all four with zero-copy slices from the internal `Uint8Array`.

### Property-based tests (fast-check)

- Key-validation invariants (arbitrary strings, path-traversal, length boundaries).
- `batch()` rollback correctness under arbitrary failure injection.
- Namespace roundtrip: `namespace(p).set(k, v)` then `namespace(p).get(k)` equals `v`.
- TTL expiry: for any `ttl > 0`, a read after `ttl` ms must return `undefined`.
- `listWithPattern` result consistency against brute-force `list()` + JS filter.

### Benchmarks (`src/__benchmarks__/`)

- `memory-ops.bench.ts` — throughput for `get`/`set`/`delete`/`list` on `MemoryStorage`.
- `batch-ops.bench.ts` — batch vs sequential write throughput.
- `namespace-ops.bench.ts` — namespace overhead vs direct access.

### NamespacedStorage `getSize()` fix

Main branch `getSize()` returned the total storage size rather than the scoped namespace size. Genesis implements a correct scoped byte count by iterating only keys under `this.prefix`.

## Impact

- **Unblocks**: `@suaveplan/drive` (needs `getPartial`/`createReadStream`/`createWriteStream`), `@suaveplan/idempotency` (MemoryStore adapter), `@suaveplan/crypto-keys` (Keychain serialized-JSON), `@suaveplan/logger` (StorageLogger sink).
- **No breaking changes** to the main-branch public API surface beyond the `EventEmitter` re-export removal and the `get()` return type narrowing from `null` to `undefined` (to match `StorageLike`).
- **`sideEffects: false`** confirmed — provider registration happens lazily inside `factory.ts`, not at module scope before the factory module is imported.