Files
2026-05-27 23:29:53 +01:00

321 lines
17 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# OpenSpec Change: enterprise-browser-storage-browser
- **Change ID**: enterprise-browser-storage-browser
- **Package**: `@suaveplan/storage-browser`
- **Category**: browser
- **Target path**: `browser/storage/storage-browser`
- **Status**: PROPOSED
- **Date**: 2026-04-24
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/storage-browser.md`
---
## Summary
Deliver `@suaveplan/storage-browser` on the genesis branch at enterprise quality. The package provides browser-side `AbstractStorage` adapters — `IndexedDBStorage` and `LocalStorageStorage` — with a unified async API over `Uint8Array` payloads, TTL expiration, prefix-based listing with pagination, `StorageQuotaError`, and auto-registration. Genesis adds BroadcastChannel cross-tab sync, encrypted + compressed wrapper adapters, proactive TTL sweep, transactional batch API, optimistic concurrency, observable store, and corrected `sideEffects` declaration.
---
## Current-state audit (genesis)
Fresh build — does not exist on genesis yet.
---
## Available workspace deps on genesis
| Package | Role |
|---|---|
| `@suaveplan/storage` (`workspace:^`) | `AbstractStorage`, `StorageError`, `StorageQuotaError`, factory |
| `@suaveplan/crypto` (`workspace:^`) | AES-GCM encrypted wrapper |
| `@suaveplan/events` (`workspace:^`) | Observable store events |
| `@suaveplan/error` (`workspace:^`) | Typed errors |
| `@suaveplan/types` (`workspace:^`) | `LoggerLike` |
| `@suaveplan/logger` (`workspace:^`) | Structured logging |
| `@suaveplan/broadcast-channel` (`workspace:^`) | Cross-tab invalidation |
| `@suaveplan/schema-registry` (`workspace:^`) | Register config schemas |
| `zod` (`catalog:forms`) | Config + entry schemas |
| `@opentelemetry/api` (`catalog:observability`) | Storage operation metrics |
| `tslib` (`catalog:build`) | Runtime helpers |
Dev: `vitest`, `@vitest/coverage-istanbul`, `fake-indexeddb (^6.0.0) — must be added to catalog:testing before implementation`, `@playwright/test`, `@suaveplan/testing`.
---
## Preserved public API surface
```ts
// IndexedDB adapter
class IndexedDBStorage extends AbstractStorage {
constructor(config: IndexedDBStorageConfig)
get(key: string): Promise<StorageEntry | null>
set(key: string, data: Uint8Array, metadata?: StorageMetadata): Promise<void>
delete(key: string): Promise<void>
clear(): Promise<void>
list(prefix?: string, options?: ListOptions): Promise<StorageEntry[]>
batch(ops: BatchOperation[]): Promise<void>
}
// LocalStorage adapter
class LocalStorageStorage extends AbstractStorage {
constructor(config: LocalStorageStorageConfig)
get(key: string): Promise<StorageEntry | null>
set(key: string, data: Uint8Array, metadata?: StorageMetadata): Promise<void>
delete(key: string): Promise<void>
clear(): Promise<void>
list(prefix?: string, options?: ListOptions): Promise<StorageEntry[]>
}
// Wrapper adapters
class EncryptedStorage extends AbstractStorage { constructor(inner: AbstractStorage, keyMaterial: CryptoKey) }
class CompressedStorage extends AbstractStorage { constructor(inner: AbstractStorage, threshold?: number) }
// Observable
function observeStorage(storage: AbstractStorage, key: string): AsyncIterable<StorageEvent>
// Register functions (explicit, not auto-import side effects)
function registerIndexedDBStorage(): void
function registerLocalStorageStorage(): void
// Config types
interface IndexedDBStorageConfig { dbName: string; storeName: string; ttl?: number; schemaVersion?: number; clock?: Clock }
interface LocalStorageStorageConfig { namespace: string; ttl?: number; clock?: Clock }
```
Subpath exports: `"."`, `"./indexeddb"`, `"./localstorage"`, `"./register"`, `"./types"`.
---
## New additions for genesis
### 1. `sideEffects` fix
Main-branch `sideEffects: false` conflicts with the two `registry.ts` side-effect imports. Genesis moves auto-registration to explicit `registerIndexedDBStorage()` / `registerLocalStorageStorage()` call-site functions under a `./register` subpath. `sideEffects` remains `false` for all other subpaths.
### 2. Proactive TTL sweep
`IndexedDBStorage` sweeps expired rows via `requestIdleCallback` + the `expiresAt` IDB index. No expired rows consume quota between reads.
### 3. Transactional batch API
`batch(ops: BatchOperation[])` executes multiple `set`/`delete` operations in a single IDB transaction for atomic multi-key updates.
### 4. Encrypted wrapper
`EncryptedStorage` wraps any `AbstractStorage` with AES-GCM (via `@suaveplan/crypto`) for at-rest encryption of payloads and metadata.
### 5. Compressed wrapper
`CompressedStorage` applies `CompressionStream` (gzip) above a configurable byte threshold before writing to the underlying adapter.
### 6. BroadcastChannel cross-tab sync
`IndexedDBStorage` and `LocalStorageStorage` accept an optional `broadcastChannel?: TypedBroadcastChannel`; emit `set`/`delete`/`clear` events so other tabs invalidate their views.
### 7. Observable store
`observeStorage(storage, key): AsyncIterable<StorageEvent>` on top of the BroadcastChannel layer.
### 8. Optimistic concurrency
Compare-and-set using `updatedAt` metadata so concurrent-tab writes don't silently clobber each other.
### 9. Error codes
`STORAGE_QUOTA_EXCEEDED`, `STORAGE_KEY_NOT_FOUND`, `STORAGE_ENCRYPTION_FAILED`, `STORAGE_COMPRESSION_FAILED`, `STORAGE_CONCURRENCY_CONFLICT`.
### 10. OTel metrics
Counter `storage.reads_total{backend}`, counter `storage.writes_total{backend}`, counter `storage.deletes_total{backend}`, counter `storage.ttl_evictions_total`, histogram `storage.op_ms`.
---
## File layout
```text
browser/storage/storage-browser/
package.json
tsconfig.json
vitest.config.ts
src/
index.ts
register.ts (explicit registerIndexedDBStorage + registerLocalStorageStorage)
error-codes.ts
indexeddb/
adapter/
indexeddb.ts (orchestrator ≤300 lines)
indexeddb.test.ts
indexeddb.md
indexeddb.types.ts
index.ts
store/ (store.ts, store.test.ts, store.md)
ttl/ (ttl.ts, ttl.test.ts, ttl.md)
migrations/ (migrations.ts, migrations.test.ts, migrations.md)
localstorage/
adapter/
localstorage.ts
localstorage.test.ts
localstorage.md
localstorage.types.ts
index.ts
store/
ttl/
wrappers/
encrypted/ (encrypted.ts, encrypted.test.ts, encrypted.md)
compressed/ (compressed.ts, compressed.test.ts, compressed.md)
sync/
broadcast/
broadcast-sync.ts
broadcast-sync.test.ts
broadcast-sync.md
index.ts
observable/
observe-storage/
observe-storage.ts
observe-storage.test.ts
observe-storage.md
index.ts
types/
types.ts
types.md
index.ts
tests/
integration.test.ts
e2e/
storage-browser.spec.ts (Chromium/WebKit/Firefox cross-tab, encrypted, compressed)
```
---
## Source mapping
> This change derives from `../suaveplan/enterprise/storage-browser.md`.
| Requirement | Source lines | Notes |
|---|---|---|
| IndexedDB adapter with CRUD, TTL, and metadata | L13, L21-L22 | Purpose and current state assessment |
| LocalStorage adapter with Base64 envelope and TTL | L13, L24-L25 | Purpose and current state assessment |
| Explicit registration — no import-time side effects | L48-L49 | Fix sideEffects vs registry modules |
| Proactive TTL sweep | L69 | Proactive TTL sweep using expiresAt index |
| Transactional batch API | L70 | Transactional batch API |
| Encrypted wrapper | L71 | Encrypted wrapper using WebCrypto AES-GCM |
| Compressed wrapper | L72 | Compressed wrapper using CompressionStream |
| BroadcastChannel cross-tab sync | L73 | BroadcastChannel cross-tab sync |
| Observable store | L78 | Observable store AsyncIterable |
| Optimistic concurrency | L77 | Optimistic concurrency via updatedAt |
| Stable error codes | L63 | Error hierarchy via SuaveplanError |
| OTel metrics per backend | L44, L72 | Observability surface — counters, histograms, spans |
### Requirements with no source line
| Requirement | Justification |
|---|---|
| Subpath exports for `./indexeddb`, `./localstorage`, `./register`, `./types` | Genesis-specific tree-shaking requirement (AGENTS.md Section 6) |
| 100% coverage threshold | Genesis-wide policy (openspec/AGENTS.md Section 7), stricter than enterprise 95% floor |
| `fake-indexeddb` for unit tests | Genesis testing strategy — no mocks in production, but test doubles permitted in tests |
| Playwright E2E cross-tab invalidation | Genesis web-app verification rules (CLAUDE.md) require multi-tab E2E |
## Out of scope
- Server-side rendering (SSR) storage emulation.
- Service Worker `CacheStorage` or `caches` API integration.
- OPFS backend (handled by `@suaveplan/storage-browser-opfs`).
- Multi-origin storage synchronization.
- Offline-first conflict resolution (CRDTs, etc.).
## Gap analysis: sub-standard → enterprise
- **Security gaps**: Prior art stored plaintext in `localStorage` without encryption wrappers. Genesis adds `EncryptedStorage` (AES-GCM via `@suaveplan/crypto`) and enforces origin-scoped namespaces.
- **Reliability gaps**: Prior art lacked atomic batch operations and could leave partial state on tab crash. Genesis adds transactional `batch()` for IndexedDB and optimistic concurrency control via `updatedAt` metadata.
- **Observability gaps**: Prior art emitted no metrics. Genesis adds per-backend OTel counters, histograms, and structured logs; BroadcastChannel cross-tab sync is observable.
- **Testability gaps**: Prior art used brittle mock-based tests. Genesis uses `fake-indexeddb` for deterministic unit tests, Playwright E2E for cross-tab validation, and 100% coverage mandate.
- **Operational gaps**: Prior art had silent TTL expiry accumulation. Genesis adds proactive TTL sweep via `requestIdleCallback` and `StorageQuotaError` before write failures.
## Security model
- **Threat model**: Same-origin attackers with JS execution can read/write storage. Threats arise from XSS injecting keys/values, and from local storage inspection on shared devices.
- **Boundary diagram**: `IndexedDBStorage` / `LocalStorageStorage` sit inside the origin boundary. `EncryptedStorage` adds a cryptographic boundary inside the origin. `BroadcastChannel` crosses tab boundaries within the same origin.
- **Input sanitization**: Keys are sanitized (no null bytes, max length). `StorageMetadata` is validated via Zod. `CryptoKey` for encryption is non-extractable and scoped to the wrapper.
- **Side-channel resistance**: `EncryptedStorage` uses AES-GCM with random IVs per write; no static nonce. Compression is applied before encryption to mitigate CRIME/BREACH (payloads are local, but defense-in-depth applied).
## Performance SLOs
| Operation | p50 | p99 | Throughput | Memory | Big-O |
|---|---|---|---|---|---|
| IndexedDB `get` | <2 ms | <10 ms | 5k ops/s | payload size + object store overhead | O(1) |
| IndexedDB `set` (small) | <3 ms | <15 ms | 3k ops/s | 2× payload (transaction journal) | O(1) |
| LocalStorage `get` | <1 ms | <5 ms | 10k ops/s | 5 MiB total quota | O(1) |
| LocalStorage `set` | <1 ms | <5 ms | 10k ops/s | 5 MiB total quota | O(1) |
| `batch` (100 ops) | <10 ms | <50 ms | 100 batches/s | sum of payloads | O(n) |
| TTL sweep | <5 ms | <20 ms | 1 sweep/min | cursor overhead | O(e) |
- **Scalability limits**: IndexedDB tested up to 100k entries / 500 MiB. LocalStorage hard-capped at ~5 MiB by browser; adapter throws `STORAGE_QUOTA_EXCEEDED` before bound.
## Observability plan
- **OTel spans**: `storage.browser.get`, `storage.browser.set`, `storage.browser.delete`, `storage.browser.list`, `storage.browser.batch`, `storage.browser.ttl_sweep`
- **Counters**: `storage.reads_total{backend}`, `storage.writes_total{backend}`, `storage.deletes_total{backend}`, `storage.ttl_evictions_total`, `storage.quota_exceeded_total`, `storage.concurrency_conflict_total`
- **Histograms**: `storage.op_ms{backend,op}`, `storage.batch_size`, `storage.encrypt_ms`, `storage.compress_ms`
- **Structured logging**: `debug` log per operation with `backend`, `key` (hashed for PII), `trace_id`, `duration_ms`, `error_code`.
- **Health checks**: `storage.browser.health` probe performs a `set`/`get`/`delete` round-trip on a sentinel key; fails if quota is exhausted or IDB is corrupt.
- **Alerting thresholds**: `storage.quota_exceeded_total` > 10/min triggers warning; `storage.op_ms` p99 > 100 ms for 5 min triggers investigation; `storage.concurrency_conflict_total` spike indicates cross-tab write collision.
## Dependency audit
| Runtime dep | Version | Justification | Supply-chain risk | Tree-shakeable | License |
|---|---|---|---|---|---|
| `zod` | `catalog:forms` | Config + entry schema validation | Low | Yes | MIT |
| `@opentelemetry/api` | `catalog:observability` | Metrics and spans | Low | Yes | Apache-2.0 |
| `tslib` | `catalog:build` | TS runtime helpers | Low | Yes | 0BSD |
| `@suaveplan/storage` | `workspace:^` | AbstractStorage contract | Low | Yes | Proprietary |
| `@suaveplan/crypto` | `workspace:^` | AES-GCM wrapper | Low | Yes | Proprietary |
| `@suaveplan/events` | `workspace:^` | Observable store events | Low | Yes | Proprietary |
| `@suaveplan/error` | `workspace:^` | Typed errors | Low | Yes | Proprietary |
| `@suaveplan/types` | `workspace:^` | LoggerLike | Low | Yes | Proprietary |
| `@suaveplan/logger` | `workspace:^` | Structured logging | Low | Yes | Proprietary |
| `@suaveplan/broadcast-channel` | `workspace:^` | Cross-tab sync | Low | Yes | Proprietary |
| `@suaveplan/schema-registry` | `workspace:^` | Config schema registration | Low | Yes | Proprietary |
## Operational readiness
- **Graceful degradation**: If `BroadcastChannel` is unavailable (e.g., Firefox private mode), sync degrades to silent mode; storage operations continue. If `CompressionStream` is missing, `CompressedStorage` bypasses compression.
- **Resource limits**: TTL sweep is gated by `requestIdleCallback` timeout (50 ms) to avoid blocking the main thread. `batch` size is unbounded by spec but documented with a recommended 500-op soft limit.
- **Retry policies**: IndexedDB transactions retry once on `TransactionInactiveError` or `QuotaExceededError` after a 100 ms delay. LocalStorage is synchronous under the hood; no retries.
- **Circuit breakers**: `QuotaExceededError` triggers a 5-second circuit breaker on writes to prevent spin-loops; reads remain open.
- **Upgrade compatibility**: Schema version bumps in IndexedDB use standard IDB `onupgradeneeded` callbacks; old data is preserved unless a migration explicitly drops it. Subpath exports ensure tree-shaking is stable across versions.
## Verification approach
- `src/indexeddb/adapter/indexeddb.test.ts` — covers IndexedDB CRUD, TTL, batch, concurrency, quota
- `src/indexeddb/adapter/indexeddb.spec-conformance.test.ts` — pins type-level conformance, OTel metrics, strict-mode behaviours
- `src/indexeddb/store/store.test.ts` — covers IDB open/upgrade/transaction internals
- `src/indexeddb/ttl/ttl.test.ts` — covers proactive sweep
- `src/indexeddb/migrations/migrations.test.ts` — covers schema-version upgrade callbacks
- `src/localstorage/adapter/localstorage.test.ts` — covers LocalStorage CRUD, namespace isolation, TTL
- `src/wrappers/encrypted/encrypted.test.ts` — covers AES-GCM encryption round-trip
- `src/wrappers/compressed/compressed.test.ts` — covers gzip threshold logic
- `src/sync/broadcast/broadcast-sync.test.ts` — covers cross-tab event emission
- `src/observable/observe-storage/observe-storage.test.ts` — covers AsyncIterable events and abort
- `src/register.test.ts` — covers explicit factory registration
- `src/error-codes.test.ts` — covers code registration
- `tests/integration.test.ts` — covers adapter stacking (encrypted + compressed over IndexedDB)
- `e2e/storage-browser.spec.ts` — Playwright cross-tab, quota, encryption visibility tests
- **100% coverage target**: Istanbul thresholds in `vitest.config.ts` enforce 100% branches, functions, lines, and statements.
- **Property-based test invariants**: `fast-check` verifies that arbitrary `batch` sequences are equivalent to individual ops; arbitrary key prefixes never collide across namespaces; TTL expiry times are monotonic.
- **Load/stress criteria**: 10k rapid `set`/`get` cycles in a single tab must complete in <30 s with no IDB transaction timeouts; cross-tab stress test with 5 tabs each writing 1k keys must converge to consistent state within 5 s.
- **Security audit checklist**:
1. AES-GCM key non-extractability — `CryptoKey` usages cannot be exported.
2. IV uniqueness — encrypted wrapper never reuses an IV across two writes.
3. No plaintext metadata leakage — `StorageMetadata` inside `EncryptedStorage` is encrypted as part of the payload envelope.
4. LocalStorage namespace isolation — two adapters with different namespaces cannot read each other’s keys.
5. XSS injection via key — keys containing `<script>` or null bytes are sanitized before storage.
6. Quota exhaustion handling — `STORAGE_QUOTA_EXCEEDED` is thrown before the underlying API throws an opaque DOMException.
## Impact
**Downstream**: `@suaveplan/cache-browser`, `@suaveplan/analytics-browser`, `@suaveplan/react-tour`, any package using browser storage.
**Breaking changes**: Auto-registration side effects removed; callers must import `./register` explicitly.
**Prerequisites**: `@suaveplan/storage`, `@suaveplan/crypto`, `@suaveplan/events`, `@suaveplan/broadcast-channel` on Verdaccio.