251 lines
15 KiB
Markdown
251 lines
15 KiB
Markdown
<!-- markdownlint-disable MD024 -->
|
||
|
||
# Capability Spec Delta: storage-file (storage-file)
|
||
|
||
**Package**: `@suaveplan/storage-file`
|
||
**Status**: PROPOSED
|
||
|
||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||
|
||
## ADDED Requirements
|
||
|
||
### Requirement: NodeFsStorage extends AbstractStorage
|
||
|
||
The package SHALL provide a `NodeFsStorage` class extending `AbstractStorage` from `@suaveplan/storage`. The class SHALL implement every abstract method on the base (`get`, `set`, `delete`, `exists`, `list`, `clear`, `getWithMetadata`, `getMetadata`, `getSize`, `getPartial`, `setPartial`, `getItemSize`, `createReadStream`, `createWriteStream`) with the signatures declared on the base, including the trailing `signal?: AbortSignal` parameter.
|
||
|
||
#### Scenario: instance is structurally an AbstractStorage
|
||
|
||
- **WHEN** a caller constructs `new NodeFsStorage({ rootDir })`
|
||
- **THEN** the instance is `instanceof AbstractStorage` and every abstract method is callable
|
||
|
||
#### Scenario: capabilities matrix declares native streams and partial reads
|
||
|
||
- **WHEN** a caller reads `storage.capabilities`
|
||
- **THEN** `partialReads === true`, `nativeStreams === true`, `atomicWrites === true`, `partialWrites === false`, `contentAddressable === false`
|
||
|
||
**Verified by:** `src/node-fs-storage/node-fs-storage.test.ts::"implements every AbstractStorage abstract method"` AND `src/node-fs-storage/node-fs-storage.spec-conformance.test.ts::"declares the documented capabilities matrix"`
|
||
|
||
---
|
||
|
||
### Requirement: CRUD operations under rootDir isolation
|
||
|
||
The adapter SHALL store every key under the configured `rootDir`. `set` SHALL create parent directories when `mkdirp` is enabled (default true). `get` SHALL return `undefined` for missing keys. `delete` SHALL return `true` on successful removal and `false` when the key did not exist. `exists` SHALL return a boolean without throwing on missing keys. `list` SHALL enumerate every key under an optional `prefix`, support `limit`/`offset`/`continuationToken` pagination, and return `{keys, total, hasMore, continuationToken}`. `clear` SHALL remove every data and metadata file under `rootDir`. Every method SHALL accept an optional `AbortSignal` and throw `STORAGE_FILE.ABORTED` when the signal is already aborted.
|
||
|
||
#### Scenario: round-trip set/get/delete preserves bytes
|
||
|
||
- **WHEN** a caller `await storage.set("k", bytes)` then `await storage.get("k")`
|
||
- **THEN** the returned `Uint8Array` is byte-equal to `bytes`; subsequent `await storage.delete("k")` returns `true`; final `await storage.get("k")` returns `undefined`
|
||
|
||
#### Scenario: AbortSignal short-circuits every method
|
||
|
||
- **WHEN** a caller passes an already-aborted signal to any method
|
||
- **THEN** the call rejects with `StorageFileError` whose `code === "STORAGE_FILE.ABORTED"` before any filesystem call is issued
|
||
|
||
**Verified by:** `src/node-fs-storage/node-fs-storage.test.ts::"round-trips Uint8Array values"` AND `src/node-fs-storage/node-fs-storage.test.ts::"rejects with STORAGE_FILE.ABORTED when signal is pre-aborted"`
|
||
|
||
---
|
||
|
||
### Requirement: Atomic writes via temp-file-then-rename
|
||
|
||
`set` and metadata writes SHALL write to a temp file in the destination directory and `fs.rename` it onto the destination path. On write failure the temp file SHALL be unlinked. Concurrent readers SHALL never observe a torn write — they either see the previous version or the new version.
|
||
|
||
#### Scenario: temp file is removed after successful write
|
||
|
||
- **WHEN** a caller `await storage.set("k", bytes)` completes
|
||
- **THEN** no temp file (with prefix `.tmp-`) remains in `rootDir`
|
||
|
||
#### Scenario: temp file is removed when the underlying write throws
|
||
|
||
- **WHEN** the injected `NodeFsLike.writeFile` throws after the temp file is created
|
||
- **THEN** the temp file is unlinked and the destination path is unchanged
|
||
|
||
**Verified by:** `src/node-fs-storage/node-fs-storage.test.ts::"writes atomically — no temp files remain after success"` AND `src/node-fs-storage/node-fs-storage.test.ts::"unlinks the temp file when the underlying writeFile throws"`
|
||
|
||
---
|
||
|
||
### Requirement: Strict key sanitisation at the boundary
|
||
|
||
Every method that accepts a key SHALL pass it through `sanitizeKey` before issuing any filesystem call. The sanitiser SHALL reject empty strings, keys containing `..` segments, keys starting with `/` or matching a drive-letter prefix (`C:\…`), keys containing null bytes (` `), keys containing control chars (``–` `, ``), and keys longer than 1024 characters. Windows `\` separators SHALL be normalised to POSIX `/`. Rejections SHALL throw `StorageFileError` with code `STORAGE_FILE.INVALID_KEY` or `STORAGE_FILE.KEY_TOO_LONG`.
|
||
|
||
#### Scenario: path-traversal attempt is rejected
|
||
|
||
- **WHEN** a caller calls `storage.get("../../etc/passwd")`
|
||
- **THEN** the call rejects with `StorageFileError` whose `code === "STORAGE_FILE.INVALID_KEY"` before any filesystem call is issued
|
||
|
||
#### Scenario: overlong key is rejected with KEY_TOO_LONG
|
||
|
||
- **WHEN** a caller calls `storage.set("a".repeat(1025), bytes)`
|
||
- **THEN** the call rejects with `StorageFileError` whose `code === "STORAGE_FILE.KEY_TOO_LONG"`
|
||
|
||
**Verified by:** `src/key-sanitizer/key-sanitizer.test.ts::"rejects path traversal segments"` AND `src/key-sanitizer/key-sanitizer.test.ts::"rejects keys longer than 1024 chars with KEY_TOO_LONG"`
|
||
|
||
---
|
||
|
||
### Requirement: Transient error retry via @suaveplan/retry
|
||
|
||
Operations SHALL retry on transient filesystem errors (`EBUSY`, `EMFILE`, `ENFILE`, `EAGAIN`) using `@suaveplan/retry.retry` with exponential backoff. Non-transient errors (`ENOENT`, `EACCES`) SHALL NOT be retried. After `maxAttempts` (default 3) the operation SHALL throw `StorageFileError` with code `STORAGE_FILE.IO_FAILED` and `cause` set to the last underlying error. The retry loop SHALL accept an injectable `TimersLike` (`setTimer`/`clearTimer`) so that tests run deterministically.
|
||
|
||
#### Scenario: EBUSY is retried then succeeds on the second attempt
|
||
|
||
- **WHEN** the injected `NodeFsLike.writeFile` fails once with `EBUSY` and succeeds on retry
|
||
- **THEN** the overall `storage.set(...)` resolves and `recordOp` reports `outcome="success"`
|
||
|
||
#### Scenario: EACCES is not retried
|
||
|
||
- **WHEN** the injected `NodeFsLike.writeFile` throws `EACCES`
|
||
- **THEN** the call rejects after a single attempt with `StorageFileError` whose `code === "STORAGE_FILE.PERMISSION_DENIED"`
|
||
|
||
**Verified by:** `src/retry-helper/retry-helper.test.ts::"retries on EBUSY and resolves on success"` AND `src/retry-helper/retry-helper.test.ts::"does not retry on EACCES"`
|
||
|
||
---
|
||
|
||
### Requirement: Native Web Streams for read and write
|
||
|
||
`createReadStream(key, signal?)` SHALL return a `ReadableStream<Uint8Array>` backed by `node:fs.createReadStream`. `createWriteStream(key, signal?)` SHALL return a `WritableStream<Uint8Array>` backed by `node:fs.createWriteStream` writing to a temp file then renaming on close. Aborting the AbortSignal mid-stream SHALL cancel the underlying Node stream and reject the consumer with `STORAGE_FILE.ABORTED`.
|
||
|
||
#### Scenario: large file round-trips through streams without buffering
|
||
|
||
- **WHEN** a caller writes 1 MiB of data through `createWriteStream` then reads it back through `createReadStream`
|
||
- **THEN** the concatenated bytes are byte-equal to the original
|
||
|
||
#### Scenario: AbortSignal mid-write cleans up the temp file
|
||
|
||
- **WHEN** a caller aborts the signal halfway through a write
|
||
- **THEN** no temp file remains in `rootDir` and the destination path is unchanged
|
||
|
||
**Verified by:** `src/streams/streams.test.ts::"round-trips a 1 MiB payload"` AND `src/streams/streams.test.ts::"aborts mid-write and removes the temp file"`
|
||
|
||
---
|
||
|
||
### Requirement: Partial reads via positioned read
|
||
|
||
`getPartial(key, offset, length, signal?)` SHALL open the data file, read `length` bytes starting at `offset`, and close the handle. The result SHALL be a `Uint8Array` of at most `length` bytes (truncated at EOF). Memory usage SHALL be `O(length)`, not `O(file-size)`.
|
||
|
||
#### Scenario: partial read returns the requested slice
|
||
|
||
- **WHEN** a caller `await storage.getPartial("k", 10, 5)` on a 100-byte file
|
||
- **THEN** the returned Uint8Array is byte-equal to bytes 10..14 of the stored value
|
||
|
||
#### Scenario: partial read past EOF returns a truncated slice
|
||
|
||
- **WHEN** a caller `await storage.getPartial("k", 95, 20)` on a 100-byte file
|
||
- **THEN** the returned Uint8Array has length 5
|
||
|
||
**Verified by:** `src/node-fs-storage/node-fs-storage.test.ts::"reads a positioned slice via getPartial"` AND `src/node-fs-storage/node-fs-storage.test.ts::"truncates getPartial at EOF"`
|
||
|
||
---
|
||
|
||
### Requirement: Zod-validated options at construction
|
||
|
||
The constructor SHALL parse its options through `NodeFsStorageOptionsSchema` and throw `StorageFileError` with code `STORAGE_FILE.INVALID_OPTIONS` on parse failure. Required fields: `rootDir: string`. Optional fields: `mkdirp: boolean` (default `true`), `fileExtension: string` (default `".data"`), `metadataExtension: string` (default `".meta"`), `retry: { maxAttempts, baseDelayMs, maxDelayMs }`, `fs: NodeFsLike`, `timers: TimersLike`, `clock: Clock`.
|
||
|
||
#### Scenario: missing rootDir is rejected with INVALID_OPTIONS
|
||
|
||
- **WHEN** a caller calls `new NodeFsStorage({} as never)`
|
||
- **THEN** the constructor throws `StorageFileError` whose `code === "STORAGE_FILE.INVALID_OPTIONS"`
|
||
|
||
**Verified by:** `src/node-fs-storage/node-fs-storage.test.ts::"rejects missing rootDir with INVALID_OPTIONS"` AND `src/node-fs-storage/node-fs-storage.spec-conformance.test.ts::"parses options via NodeFsStorageOptionsSchema at construction"`
|
||
|
||
---
|
||
|
||
### Requirement: OpenTelemetry observability
|
||
|
||
The package SHALL emit per-operation OTel signals from production paths: a span on the `@suaveplan/storage-file` tracer named `storage_file.<op>`, a counter increment on `storage_file.<reads|writes|deletes|errors>_total{outcome}`, a histogram observation on `storage_file.op_duration_ms{op,outcome}`, and a histogram observation on `storage_file.io_bytes{op,direction}` when bytes flow. The `outcome` attribute SHALL be one of `success`/`error`/`aborted`.
|
||
|
||
#### Scenario: a successful set records the expected metrics
|
||
|
||
- **WHEN** a caller `await storage.set("k", bytes)` against a real in-memory `MeterProvider`
|
||
- **THEN** `storage_file.writes_total{outcome="success"}` increments by 1 AND `storage_file.op_duration_ms{op="set"}` records a single observation AND `storage_file.io_bytes{op="set",direction="write"}` records `bytes.byteLength`
|
||
|
||
**Verified by:** `src/otel/otel.test.ts::"records duration and byte counters on a successful op"` AND `src/node-fs-storage/node-fs-storage.spec-conformance.test.ts::"emits storage_file.writes_total counter on successful set"`
|
||
|
||
---
|
||
|
||
### Requirement: Module-augmentation registry hook
|
||
|
||
The package SHALL augment `@suaveplan/storage`'s `StorageProviderRegistry` and `StorageProviderConfig` with a `file` slot keyed to `NodeFsStorage`/`NodeFsStorageOptions`. The package SHALL export `registerNodeFsProvider()` which calls `registerStorageProvider("file", NodeFsStorage)` and is idempotent — repeated calls SHALL NOT throw.
|
||
|
||
#### Scenario: factory resolves `file` to NodeFsStorage after registration
|
||
|
||
- **WHEN** a caller `registerNodeFsProvider(); const s = createStorage({ type: "file", config: { rootDir } });`
|
||
- **THEN** `s instanceof NodeFsStorage` is `true`
|
||
|
||
**Verified by:** `src/registry/registry.test.ts::"registers the file provider and resolves it via createStorage"` AND `src/registry/registry.test.ts::"is idempotent across repeated registerNodeFsProvider calls"`
|
||
|
||
---
|
||
|
||
### Requirement: defaultBunFs satisfies NodeFsLike via Bun-native read/write
|
||
|
||
The package SHALL export a `defaultBunFs(): NodeFsLike` factory from
|
||
`src/bun-fs/bun-fs.ts`. The returned object's `readFile(path)` SHALL be
|
||
implemented in terms of `Bun.file(path).bytes()` and its `writeFile(path, data)`
|
||
SHALL be implemented in terms of `Bun.write(path, data)`. Every other method
|
||
on the returned `NodeFsLike` (`mkdir`, `readdir`, `rename`, `unlink`, `stat`,
|
||
`access`, `open`, `createReadStream`, `createWriteStream`) SHALL delegate to
|
||
the same `node:fs/promises` / `node:fs` calls used by `defaultNodeFs()`. The
|
||
factory SHALL return a fresh object per call so callers can selectively
|
||
override individual methods without mutating shared state.
|
||
|
||
#### Scenario: readFile round-trips Bun.write output
|
||
|
||
- **WHEN** a caller calls `defaultBunFs().writeFile(path, bytes)` followed by
|
||
`defaultBunFs().readFile(path)` against a real temp dir
|
||
- **THEN** the returned `Uint8Array` is byte-equal to `bytes`
|
||
|
||
#### Scenario: every other method forwards to node:fs/promises behaviour
|
||
|
||
- **WHEN** a caller exercises `mkdir`, `readdir`, `stat`, `access`, `rename`,
|
||
`unlink`, `open`, `createReadStream`, and `createWriteStream` on a
|
||
`defaultBunFs()` instance against a real temp dir
|
||
- **THEN** every call returns the same shape and side effects as the
|
||
matching method on `defaultNodeFs()`
|
||
|
||
**Verified by:** `src/bun-fs/bun-fs.test.ts::"round-trips bytes through Bun.write and Bun.file"` AND `src/bun-fs/bun-fs.test.ts::"forwards mkdir, readdir, stat, access, rename, unlink, open, and the stream constructors"`
|
||
|
||
---
|
||
|
||
### Requirement: pickDefaultFs selects the Bun adapter at runtime
|
||
|
||
The package SHALL export a `pickDefaultFs(): NodeFsLike` factory from
|
||
`src/bun-fs/bun-fs.ts` that returns `defaultBunFs()` when `globalThis.Bun`
|
||
is defined and `defaultNodeFs()` otherwise. The probe SHALL be performed on
|
||
each call (never cached at module scope) so test harnesses can deterministically
|
||
switch adapters by injecting an explicit `fs` option to `NodeFsStorage`.
|
||
|
||
#### Scenario: probe selects the Bun adapter when globalThis.Bun is present
|
||
|
||
- **WHEN** a caller installs a Bun-shaped global and invokes `pickDefaultFs()`
|
||
- **THEN** the returned `NodeFsLike` `readFile` and `writeFile` are sourced
|
||
from `defaultBunFs()`, observable by performing a round-trip and confirming
|
||
the byte contents are preserved against a real temp file
|
||
|
||
#### Scenario: probe falls back to defaultNodeFs when Bun is absent
|
||
|
||
- **WHEN** `globalThis.Bun` is undefined and a caller invokes `pickDefaultFs()`
|
||
- **THEN** the returned `NodeFsLike` resolves a round-trip via
|
||
`defaultNodeFs()` without throwing
|
||
|
||
**Verified by:** `src/bun-fs/bun-fs.test.ts::"pickDefaultFs returns a working NodeFsLike under Bun"` AND `src/bun-fs/bun-fs.test.ts::"pickDefaultFs falls back to defaultNodeFs when Bun is absent"`
|
||
|
||
---
|
||
|
||
### Requirement: NodeFsStorage default fs honours runtime probe
|
||
|
||
`resolveConfig` in `src/node-fs-storage/node-fs-storage.helpers.ts` SHALL
|
||
resolve a missing `options.fs` by calling `pickDefaultFs()` rather than
|
||
`defaultNodeFs()`. The behaviour SHALL be backwards compatible: when the
|
||
caller passes `options.fs` explicitly, the injected adapter is used
|
||
unchanged.
|
||
|
||
#### Scenario: missing fs option under Bun uses defaultBunFs
|
||
|
||
- **WHEN** a caller constructs `new NodeFsStorage({ rootDir })` while
|
||
`globalThis.Bun` is present and performs
|
||
`await storage.set("k", bytes)` followed by `await storage.get("k")`
|
||
- **THEN** the returned `Uint8Array` is byte-equal to `bytes` and the
|
||
recorded Bun-native call counter is non-zero
|
||
|
||
**Verified by:** `src/bun-fs/bun-fs.test.ts::"NodeFsStorage round-trips through the auto-picked Bun fs"`
|
||
|
||
<!-- markdownlint-enable MD024 -->
|