89 lines
8.6 KiB
Markdown
89 lines
8.6 KiB
Markdown
# @suaveplan/storage-file - Node.js filesystem AbstractStorage adapter for @suaveplan/storage: atomic writes via temp-rename, AbortSignal everywhere, strict key sanitisation, native Web Streams, partial reads, transient-error retry via @suaveplan/retry, full OTel surface.
|
|
|
|
## Summary
|
|
|
|
Node.js filesystem `AbstractStorage` adapter for `@suaveplan/storage`. Implements atomic writes via temp-file-then-rename, AbortSignal threaded through every async method, strict key sanitisation, transient-error retry via `@suaveplan/retry`, native Web Streams via `node:fs.createReadStream`/`createWriteStream`, and positioned partial reads. Full OpenTelemetry surface: per-op spans, latency histograms, byte-flow histograms, kind-split counters.
|
|
|
|
<!-- begin:auto-docs -->
|
|
## Documentation tree
|
|
|
|
- [bun-fs](./src/bun-fs/bun-fs.md) — `bun-fs` is a sibling of `node-fs` that satisfies the same structural `NodeFsLike` contract consumed by `NodeFsStorage`…
|
|
- [bun-fs](./src/bun-fs/bun-fs.md) — `bun-fs` is a sibling of `node-fs` that satisfies the same structural `NodeFsLike` contract consumed by `NodeFsStorage`…
|
|
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [node-fs](./src/node-fs/node-fs.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [node-fs](./src/node-fs/node-fs.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [node-fs-storage](./src/node-fs-storage/node-fs-storage.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs --> - node-fs-storage.ops — Houses the file-system operat…
|
|
- [node-fs-storage.helpers](./src/node-fs-storage/node-fs-storage.helpers.md) — Houses the pure helpers consumed by `NodeFsStorage` so the orchestrator class stays under the file-size budget and the…
|
|
- [node-fs-storage](./src/node-fs-storage/node-fs-storage.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs --> - node-fs-storage.ops — Houses the file-system operat…
|
|
- [node-fs-storage.ops](./src/node-fs-storage/node-fs-storage.ops.md) — Houses the file-system operation helpers consumed by `NodeFsStorage`. Each function here wraps a single boundary call w…
|
|
- [otel](./src/otel/otel.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [otel](./src/otel/otel.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [registry](./src/registry/registry.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [registry](./src/registry/registry.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [retry-helper](./src/retry-helper/retry-helper.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [retry-helper](./src/retry-helper/retry-helper.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [streams](./src/streams/streams.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [streams](./src/streams/streams.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
|
- [error-codes](./src/error-codes.md) — Defines the frozen `STORAGE_FILE_CODES` registry and the `StorageFileError` class used by every public path in `@suavep…
|
|
<!-- end:auto-docs -->
|
|
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — <!-- begin:auto-docs --> _(no submodules)_- [node-fs](./src/node-fs/node-fs.md) — <!-- begin:auto-docs --> _(no submodules)_ - [node-fs](./src/node-fs/node-fs.md) — <!-- begin:auto-docs --> _(no submodules)_- [node-fs-storage](./src/node-fs-storage/node-fs-storage.md) — <!-- begin:auto-docs --> _(no submodules)_ - [node-fs-storage.helpers](./src/node-fs-storage/node-fs-storage.helpers.md) — Houses the pure helpers consumed by `NodeFsStorage` so the orchestrator class stays under the file-size budget and the…
|
|
- [node-fs-storage](./src/node-fs-storage/node-fs-storage.md) — <!-- begin:auto-docs --> _(no submodules)_ - [node-fs-storage.ops](./src/node-fs-storage/node-fs-storage.ops.md) — Houses the file-system operation helpers consumed by `NodeFsStorage`. Each function here wraps a single boundary call w…
|
|
- [otel](./src/otel/otel.md) — <!-- begin:auto-docs --> _(no submodules)_ - [otel](./src/otel/otel.md) — <!-- begin:auto-docs --> _(no submodules)_- [registry](./src/registry/registry.md) — <!-- begin:auto-docs --> _(no submodules)_ - [registry](./src/registry/registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [retry-helper](./src/retry-helper/retry-helper.md) — <!-- begin:auto-docs --> _(no submodules)_ - [retry-helper](./src/retry-helper/retry-helper.md) — <!-- begin:auto-docs --> _(no submodules)_- [streams](./src/streams/streams.md) — <!-- begin:auto-docs --> _(no submodules)_ - [streams](./src/streams/streams.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the frozen `STORAGE_FILE_CODES` registry and the `StorageFileError` class used by every public path in `@suavep…
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
bun add @suaveplan/storage-file
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
```ts
|
|
import { NodeFsStorage } from "@suaveplan/storage-file";
|
|
|
|
const storage = new NodeFsStorage({ rootDir: "/var/lib/app/storage" });
|
|
|
|
await storage.set("orders/42.json", new TextEncoder().encode("{\"id\":42}"));
|
|
const bytes = await storage.get("orders/42.json");
|
|
console.log(new TextDecoder().decode(bytes!));
|
|
|
|
// Streams + partial reads
|
|
const writer = storage.createWriteStream("large.bin").getWriter();
|
|
await writer.write(new Uint8Array(1024 * 1024));
|
|
await writer.close();
|
|
const slice = await storage.getPartial("large.bin", 0, 16);
|
|
|
|
// Factory registration (optional)
|
|
import { registerNodeFsProvider } from "@suaveplan/storage-file";
|
|
import { createStorage } from "@suaveplan/storage/factory";
|
|
registerNodeFsProvider();
|
|
const viaFactory = createStorage({ type: "file", config: { rootDir: "/tmp/store" } });
|
|
```
|
|
|
|
## Modules
|
|
|
|
- [`src/error-codes`](src/error-codes.md) — `STORAGE_FILE_CODES`, `StorageFileError`, `StorageFileErrorCode`
|
|
- [`src/key-sanitizer/key-sanitizer`](src/key-sanitizer/key-sanitizer.md) — `sanitizeKey` strict boundary
|
|
- [`src/node-fs/node-fs`](src/node-fs/node-fs.md) — `defaultNodeFs` adapter wrapping `node:fs/promises`
|
|
- [`src/bun-fs/bun-fs`](src/bun-fs/bun-fs.md) — `defaultBunFs` / `pickDefaultFs` — Bun-native `Bun.file` / `Bun.write` fast path
|
|
- [`src/retry-helper/retry-helper`](src/retry-helper/retry-helper.md) — `withFsRetry`, transient classifier
|
|
- [`src/otel/otel`](src/otel/otel.md) — OTel instruments and `recordOp`
|
|
- [`src/streams/streams`](src/streams/streams.md) — Web Streams bridge with atomic write
|
|
- [`src/node-fs-storage/node-fs-storage`](src/node-fs-storage/node-fs-storage.md) — main `NodeFsStorage` class
|
|
- [`src/node-fs-storage/node-fs-storage.helpers`](src/node-fs-storage/node-fs-storage.helpers.md) — pure helpers
|
|
- [`src/registry/registry`](src/registry/registry.md) — factory registration hook
|
|
- [`src/types/fs-adapter.types`](src/types/fs-adapter.types.md) — `NodeFsLike` interface
|
|
- [`src/types/node-fs-storage.types`](src/types/node-fs-storage.types.md) — options + Zod schema
|
|
|
|
## Architecture
|
|
|
|
- **Tier**: server (consumes `node:fs/promises` and `node:fs` stream constructors). Browser consumers should pick `@suaveplan/storage/memory`, an HTTP/IndexedDB adapter, or `@suaveplan/storage-s3`.
|
|
- **Base class**: extends `AbstractStorage` from `@suaveplan/storage/abstract`. Every consumer that talks to `AbstractStorage` (or the `StorageLike` shim) works against this adapter unchanged.
|
|
- **Atomicity**: every public write (`set`, metadata sidecar, `createWriteStream.close`) writes to a temp file in the destination directory and `fs.rename`s onto the destination. POSIX rename is atomic within a filesystem.
|
|
- **Cancellation**: every public async method accepts an optional `AbortSignal`. Pre-aborted signals throw `STORAGE_FILE.ABORTED` before any IO call.
|
|
- **Resilience**: transient errors (`EBUSY`, `EMFILE`, `ENFILE`, `EAGAIN`) are retried via `@suaveplan/retry` with exponential backoff and injectable `TimersLike`. Non-transient errors surface as `STORAGE_FILE.IO_FAILED` or `STORAGE_FILE.PERMISSION_DENIED`.
|
|
- **Security**: every public method passes its key through `sanitizeKey`. Path-traversal, absolute paths, drive-letter prefixes, null bytes, control chars, and over-length keys all surface as `STORAGE_FILE.INVALID_KEY`/`KEY_TOO_LONG` before any filesystem call.
|
|
- **Observability**: OTel counters, histograms, and a tracer emit from production paths. Tests assert against a real in-memory MeterProvider — no `vi.mock`.
|
|
|