first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,92 @@
# storage-adapter / persister
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Bridges `@suaveplan/storage` `AbstractStorage` onto the queue's
`ChunkPersister` contract. Persistence enables resuming an interrupted
transfer: every received chunk is written through to a key-value
backend (IndexedDB in browsers, file-backed stores on the server)
keyed by `file:<fileId>:chunk:<chunkId>` so a fresh queue with the
same `fileId` can hydrate via `loadAllChunks`.
## Features
- Deterministic key layout (`chunkStorageKey`, `chunkStoragePrefix`).
- Persists chunk bytes alongside the metadata via the storage backend's
optional `metadata` slot.
- Loads a single chunk or lists every persisted chunk id for the
configured `fileId`, in ascending order.
- Structural `ChunkStorageLike` view so tests can substitute a Map.
## Usage: Basic
```ts
import { MemoryStorage } from "@suaveplan/storage";
import { StorageChunkPersister } from "@suaveplan/streams-chunk-queue";
const storage = new MemoryStorage();
const persister = new StorageChunkPersister({ storage, fileId: 12345n });
```
## Usage: Advanced
```ts
import { IndexedDBStorage } from "@suaveplan/storage";
const storage = await IndexedDBStorage.open("downloads", "chunks");
const persister = new StorageChunkPersister({
storage,
fileId: 12345n,
});
const queue = new StreamingChunkQueue({
fileId: 12345n,
totalChunks: 100n,
persister,
});
await queue.loadAllChunks();
```
## API Reference
### `chunkStorageKey(fileId, chunkId)` → `string`
`file:<fileId>:chunk:<chunkId>`.
### `chunkStoragePrefix(fileId)` → `string`
`file:<fileId>:chunk:`.
### `ChunkPersister`
```ts
interface ChunkPersister {
persist(chunk: StoredChunk): Promise<void>;
load(chunkId: bigint): Promise<Uint8Array | undefined>;
listChunkIds(): Promise<bigint[]>;
}
```
### `StorageChunkPersister`
```ts
new StorageChunkPersister({ storage, fileId });
```
Implements `ChunkPersister` over any `ChunkStorageLike`. `listChunkIds`
filters by prefix, parses the trailing segment as `bigint`, and sorts
ascending — the ordering means
`StreamingChunkQueue.loadAllChunks` ingests in the canonical chunk
order.
## Implementation Notes
`BigInt` is converted to `string` before being stored in storage
metadata so the JSON wire form is portable. The persister does NOT
attempt to delete entries — chunks are durable until the consumer
explicitly clears the underlying storage. Keys with an empty trailing
segment (`file:7:chunk:`) are silently dropped from `listChunkIds`
because they cannot be parsed as a bigint.