Initial Commit

openspec/archive/2026-05-12-storage-file/proposal.md

@@ -0,0 +1,93 @@
+# Add @suaveplan/storage-file — Node.js filesystem AbstractStorage adapter
+
+**Package**: `@suaveplan/storage-file`
+**Status**: PROPOSED
+**Type**: new-package
+
+## Why
+
+Genesis has `@suaveplan/storage` (AbstractStorage core with in-memory adapter) and `@suaveplan/storage-s3` (object-store adapter), but no filesystem-backed adapter for server-tier consumers (drive plugin, vault snapshots, i18n filesystem backends, integration test fixtures). The upstream `../suaveplan/packages/server/storage-file` exists at 90% coverage against the old `StorageLike` shim and ships with no atomic writes, no AbortSignal, no retry on transient errors, no path-traversal guard, and no streaming. The genesis rebuild extends `AbstractStorage`, adds POSIX-atomic temp-rename writes, hardens key validation, threads AbortSignal everywhere, and wires `@suaveplan/retry` for `EBUSY`/`EMFILE`/`ENFILE`. This unblocks several downstream packages (drive-template hydration, vault encrypted snapshots, server-side i18n backends, integration test storage fixtures).
+
+## What changes
+
+- ADDED: `NodeFsStorage` class extending `AbstractStorage` — full key-value storage rooted at a configurable `rootDir` with `.data`/`.meta` file pair per entry.
+- ADDED: `createNodeFsStorage(options)` factory.
+- ADDED: `NodeFsStorageOptions` interface + `NodeFsStorageOptionsSchema` Zod schema (Zod-validated boundary; throws `STORAGE_FILE.INVALID_OPTIONS` on malformed input).
+- ADDED: `NodeFsLike` structural interface + `defaultNodeFs()` adapter wrapping `node:fs/promises` (`mkdir`, `readFile`, `writeFile`, `rename`, `unlink`, `readdir`, `stat`, `access`, `createReadStream`, `createWriteStream`). Injectable for tests.
+- ADDED: `sanitizeKey(key)` strict key boundary — rejects empty, `..`, absolute paths (`/`, drive-prefixed), null bytes, control chars (0x00–0x1F, 0x7F), `KEY_TOO_LONG` over 1024 chars; normalises Windows `\` separators to POSIX `/`.
+- ADDED: `STORAGE_FILE_CODES` registry via `defineCodes("STORAGE_FILE", [...])` — 8 codes: `NOT_FOUND`, `PERMISSION_DENIED`, `IO_FAILED`, `INVALID_KEY`, `ATOMIC_WRITE_FAILED`, `INVALID_OPTIONS`, `ABORTED`, `KEY_TOO_LONG`.
+- ADDED: `StorageFileError extends SuaveplanError<StorageFileErrorCode>` with `key` discriminant.
+- ADDED: Module-augmentation registry — `declare module "@suaveplan/storage"` adds `file: NodeFsStorage` to `StorageProviderRegistry` and `file: NodeFsStorageOptions` to `StorageProviderConfig`; `registerNodeFsProvider()` registers the constructor with the factory.
+- ADDED: Atomic write helper — temp-file-then-`fs.rename` for `set` and metadata writes; rollback unlinks temp on failure.
+- ADDED: `@suaveplan/retry` integration on transient filesystem errors (`EBUSY`/`EMFILE`/`ENFILE`/`EAGAIN`) — exponential backoff with injectable `TimersLike`.
+- ADDED: Native Web Streams via `createReadStream`/`createWriteStream` consuming `node:fs.createReadStream`/`createWriteStream` (declared `capabilities.nativeStreams: true`).
+- ADDED: `getPartial(offset, length)` via `fs.read` with a positioned `FileHandle` — declared `capabilities.partialReads: true`.
+- ADDED: OTel surface — `storage_file.op_duration_ms{op,outcome}` + `storage_file.io_bytes{op,direction}` histograms; `storage_file.{reads,writes,deletes,errors}_total{outcome}` counters; per-op span on `@suaveplan/storage-file` tracer.
+
+## Out of scope
+
+- `setPartial` random-access writes — falls through to `AbstractStorage.defaultSetPartial` read-modify-write reference path. Reason: POSIX `pwrite` with sparse-file extension semantics is a follow-up; current callers don't require it and the reference fallback is correct.
+- Watch/notify mode — `fs.watch` integration belongs in a separate `storage-file-watcher` package; the i18n filesystem backend already covers its own watch needs (`@suaveplan/i18n-backend-fs-multifile`).
+- Encryption-at-rest — orthogonal concern handled by `@suaveplan/drive-plugin-chunks` (AEAD per chunk) and `@suaveplan/crypto-vault` (sealed container). Storage layer stores bytes verbatim.
+- Quota enforcement beyond `AbstractStorage.maxValueSize`/`maxTotalSize` — caller-facing concern; OS-level `ulimit` and filesystem quotas are out of scope.
+
+## Source mapping
+
+This change derives from `../suaveplan/enterprise/storage-file.md` (118 lines) and ports `../suaveplan/packages/server/storage-file/src/filesystem.ts` (259 lines) to extend `AbstractStorage` rather than the legacy `StorageLike` shim.
+
+| Requirement | Source lines | Notes |
+|---|---|---|
+| Node.js filesystem storage purpose | L10-L12 | "Node.js filesystem storage" — preserved as the package's single responsibility. |
+| Zod request/response schemas | L39 | Reworded as Zod option boundary (`NodeFsStorageOptionsSchema`); the upstream phrasing references `@suaveplan/api-schemas` which is API-tier and not in scope here. |
+| OpenTelemetry observability | L44 | Implemented as per-op spans, counters, and histograms emitted from production paths. |
+| Retries via `@suaveplan/retry` | L45 | Mapped to transient `EBUSY`/`EMFILE`/`ENFILE`/`EAGAIN` retry with exponential backoff and injectable `TimersLike`. |
+| Test coverage 100% | L52 | Adopted at the genesis-wide Istanbul 100/100/100/100 threshold. |
+| File-size budget | L54 | Adopted: ≤300 LOC hard / ≤500 LOC test. |
+| No mocks — real services | L55 | Tests use real `fs.mkdtemp` temp dirs and a real injectable `NodeFsLike` shim; no `vi.mock`. |
+| `sideEffects: false` | L57 | Set in `package.json`. |
+| Workspace deps + Bun catalog | L58 | All deps use `workspace:^` or `catalog:` refs. |
+| Bun overrides exact | L59 | Inherits root override policy. |
+| Biome clean | L60 | Enforced by `bun run check`. |
+| TypeScript strict, no `any` | L61 | Adopted; no `any`, no `ts-ignore`. |
+| Public API stability via Zod | L62 | `NodeFsStorageOptionsSchema` published; runtime validation at constructor boundary. |
+| Error hierarchy via `@suaveplan/error` | L63 | `StorageFileError extends SuaveplanError<StorageFileErrorCode>`. |
+| Performance benchmarks | L74 | Mitata benchmarks under `src/__benchmarks__/`. |
+| Resilience — retries, cancellation | L75 | Retry on transient errors; AbortSignal threaded through every public async method. |
+| Security — input validation at boundary | L76 | `sanitizeKey` strict boundary; `rootDir` isolation rejects traversal before any fs call. |
+
+### Requirements with no source line
+
+| Requirement | Justification |
+|---|---|
+| Extend `AbstractStorage` not `StorageLike` | The genesis storage core (`@suaveplan/storage`) replaces the legacy `StorageLike` shim with a richer `AbstractStorage` base class. Every adapter (memory, S3) extends it; this adapter MUST conform to the same contract for module-augmentation registry compatibility. |
+| AbortSignal on every public async method | Required by `AbstractStorage`'s abstract signatures. Upstream had no AbortSignal support. |
+| Atomic writes via temp-file-then-rename | Required for crash safety on the filesystem tier. Concurrent readers must never observe a torn write. POSIX `rename(2)` is atomic on the same filesystem; this is the standard primitive (e.g. `git`, `sqlite`). Upstream wrote directly to the destination path. |
+| Path-traversal guard (`..`, absolute paths) | Upstream had no key sanitisation. Without it, a caller-controlled key (`../../etc/passwd`) would escape `rootDir`. Required by genesis security model. |
+| Module-augmentation registry hook | The genesis `@suaveplan/storage` factory uses module augmentation (`StorageProviderRegistry`) for typed adapter dispatch. New adapters must hook in via `declare module` + `registerStorageProvider`. |
+| Native streaming declared in `capabilities` | `AbstractStorage.capabilities` discriminates backends that support native streams from those that fall through to the buffered reference implementation. `node:fs.createReadStream`/`createWriteStream` are native; consumers (drive-plugin-chunks) read this flag to decide whether to chain or to pre-buffer. |
+| Partial reads via positioned FileHandle | `capabilities.partialReads: true` is honoured by drive-plugin-chunks for range reads. Implementing it natively avoids `O(file-size)` memory for large objects. |
+
+## Verification approach
+
+Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete `it(...)` titles in these files:
+
+- `src/node-fs-storage/node-fs-storage.test.ts` — CRUD, metadata, list, clear, getSize, getPartial, native streams, AbortSignal, key sanitisation integration, atomic write under crash, retry exhaustion, OTel happy paths.
+- `src/node-fs-storage/node-fs-storage.spec-conformance.test.ts` — type-level conformance against `AbstractStorage`, capabilities matrix, OTel instruments observed via real in-memory `MeterProvider`, idempotent registry registration, error-code registry exhaustion.
+- `src/key-sanitizer/key-sanitizer.test.ts` — empty, traversal, absolute, null bytes, control chars, overlong, separator normalisation, valid paths.
+- `src/node-fs/node-fs.test.ts` — `defaultNodeFs()` adapter forwards every method to `node:fs/promises`, against a real `fs.mkdtemp` directory.
+- `src/retry-helper/retry-helper.test.ts` — transient classification (`EBUSY`/`EMFILE`/`ENFILE`/`EAGAIN` retry; `ENOENT`/`EACCES` not), exponential backoff timing under FakeClock + injectable timers, exhaustion → `IO_FAILED`.
+- `src/otel/otel.test.ts` — `recordOp` emits histogram + counter with the documented attribute set against a real `InMemoryMetricExporter`.
+- `src/streams/streams.test.ts` — large-file round-trip via Web Streams, AbortSignal mid-stream, structured error mapping.
+- `src/registry/registry.test.ts` — registers `file` provider, `createStorage({ type: "file", config })` returns `NodeFsStorage`, idempotent re-registration.
+
+## Impact
+
+### Affected dependents
+
+Run `bunx turbo ls --filter=...@suaveplan/storage-file` to enumerate. The package is new — no dependents currently. No action needed.
+
+- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-file` returning the package itself).
+
+### Affected upstreams
+
+- `@suaveplan/storage` — `StorageProviderRegistry` module-augmented to add `file: NodeFsStorage` slot. Additive only; no breaking change. [x] no action needed (purely additive type slot).

openspec/archive/2026-05-12-storage-file/tasks.md

@@ -0,0 +1,80 @@
+# Tasks — storage-file
+
+> Every checkbox MUST reflect work actually done in this change.
+
+## 1. Spec authoring
+
+- [x] 1.1 Read `../suaveplan/enterprise/storage-file.md` end-to-end
+- [x] 1.2 Read `openspec/specs/storage/spec.md` and confirm capability boundary
+- [x] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
+- [x] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
+- [x] 1.5 Fill in `## Source mapping` table — every requirement cites a line range OR appears in the "no source line" table with justification
+- [x] 1.6 Draft `specs/storage-file/spec.md` from `openspec/templates/spec-delta.md`
+- [x] 1.7 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles
+
+## 2. Scaffolding
+
+- [x] 2.1 Create `packages/storage/storage-file/` mirroring `packages/storage/storage-s3/` conventions
+- [x] 2.2 Write `package.json` v0.0.0, `"sideEffects": false`, subpath exports for every feature module
+- [x] 2.3 Wire deps: `@suaveplan/error`, `@suaveplan/storage`, `@suaveplan/retry`, `@suaveplan/types`, `@opentelemetry/api`, `zod`, `tslib` (all `workspace:^` / `catalog:`)
+- [x] 2.4 Wire devDeps: `@suaveplan/testing`, `@types/node`, `@vitest/coverage-istanbul`, `fast-check`, `mitata`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`
+- [x] 2.5 Write `tsconfig.json`, `tsconfig.lib.json`, `vite.config.ts`, `vitest.config.ts`
+- [x] 2.6 Write `tests/setup/polyfills.ts` and `README.md` skeleton
+
+## 3. Implementation
+
+- [x] 3.1 `src/error-codes.ts` — `defineCodes("STORAGE_FILE", [...])` with 8 codes, `StorageFileError` class, `key` discriminant
+- [x] 3.2 `src/types/node-fs-storage.types.ts` + `.schemas` — `NodeFsStorageOptions` interface + Zod schema
+- [x] 3.3 `src/types/fs-adapter.types.ts` — `NodeFsLike` structural interface (mkdir, readFile, writeFile, rename, unlink, readdir, stat, access, open, createReadStream, createWriteStream)
+- [x] 3.4 `src/node-fs/node-fs.ts` — `defaultNodeFs()` factory wrapping `node:fs/promises` + `node:fs` streams
+- [x] 3.5 `src/key-sanitizer/key-sanitizer.ts` — `sanitizeKey(key)` strict boundary
+- [x] 3.6 `src/retry-helper/retry-helper.ts` — `withFsRetry(fn, options)` consuming `@suaveplan/retry.retry` with transient classifier
+- [x] 3.7 `src/otel/otel.ts` — meters/histograms/counters/tracer, `recordOp(op, outcome, durationMs, bytes?)`
+- [x] 3.8 `src/streams/streams.ts` — `createNodeReadStream` and `createNodeWriteStream` mapping `node:fs` streams to Web Streams with AbortSignal
+- [x] 3.9 `src/node-fs-storage/node-fs-storage.ts` — `NodeFsStorage extends AbstractStorage` with all abstract methods, atomic writes, AbortSignal, capabilities matrix
+- [x] 3.10 `src/registry/registry.ts` — `declare module "@suaveplan/storage"` + `registerNodeFsProvider()`
+- [x] 3.11 `src/index.ts` — explicit named re-exports per §6a
+
+## 4. Tests
+
+- [x] 4.1 `src/key-sanitizer/key-sanitizer.test.ts` — valid, traversal, absolute, null bytes, control chars, overlong, separator normalisation
+- [x] 4.2 `src/node-fs/node-fs.test.ts` — adapter forwards to real `fs.mkdtemp`-rooted directory
+- [x] 4.3 `src/retry-helper/retry-helper.test.ts` — classifier, backoff timing, exhaustion
+- [x] 4.4 `src/otel/otel.test.ts` — metrics observed via real in-memory MeterProvider
+- [x] 4.5 `src/streams/streams.test.ts` — round-trip, large payload, AbortSignal mid-stream
+- [x] 4.6 `src/node-fs-storage/node-fs-storage.test.ts` — full CRUD + metadata + list + clear + getSize + getPartial + atomic write rollback + retry path
+- [x] 4.7 `src/node-fs-storage/node-fs-storage.spec-conformance.test.ts` — type conformance, capabilities, OTel instruments emitted, registry idempotency
+- [x] 4.8 `src/registry/registry.test.ts` — registers `file` slot, factory dispatch
+- [x] 4.9 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for genuinely async)
+
+## 5. Documentation
+
+- [x] 5.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples, full API reference)
+- [x] 5.2 `README.md` links every module under `## Modules`
+
+## 6. Gates
+
+- [x] 6.1 `bun run check`
+- [x] 6.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-file`
+- [x] 6.3 `bunx turbo test:coverage --filter=@suaveplan/storage-file` (100/100/100/100)
+- [x] 6.4 `bun scripts/openspec-conformance.ts @suaveplan/storage-file`
+- [x] 6.5 `bunx turbo build --filter=@suaveplan/storage-file`
+
+## 7. Pre-archive 9-row checklist
+
+- [x] 7.1 Generics threaded; structural injectable seams (`NodeFsLike`, `TimersLike`)
+- [x] 7.2 Zod schemas wired at every declared boundary
+- [x] 7.3 Every error code thrown by at least one production path
+- [x] 7.4 OTel instruments emitted from production paths; asserted via real in-memory `MeterProvider`
+- [x] 7.5 Atomic writes verified (temp-file removed on success, removed on failure)
+- [x] 7.6 No `export *` in feature-module barrels (§6a)
+- [x] 7.7 No direct `vitest` / `bun:test` imports in tests (§7)
+- [x] 7.8 No banned patterns in `src/` (`TODO`/`FIXME`/`ts-ignore`/`.skip`/`.only`)
+- [x] 7.9 README + sibling `.md` files complete
+
+## 8. Archive
+
+- [x] 8.1 Fold spec delta into `openspec/specs/storage-file/spec.md`
+- [x] 8.2 Move `openspec/changes/storage-file/` → `openspec/archive/<yyyy-mm-dd>-storage-file/`
+- [x] 8.3 Refresh `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
+- [x] 8.4 Update CLAUDE.md package listing under storage family