skills/docs/foundation/flow/streams-chunk-queue/src/storage-adapter/persister.md

storage-adapter / persister

(no submodules)

Purpose

Bridges @suaveplan/storage AbstractStorage onto the queue's ChunkPersister contract. Persistence enables resuming an interrupted transfer: every received chunk is written through to a key-value backend (IndexedDB in browsers, file-backed stores on the server) keyed by file:<fileId>:chunk:<chunkId> so a fresh queue with the same fileId can hydrate via loadAllChunks.

Features

• Deterministic key layout (chunkStorageKey, chunkStoragePrefix).
• Persists chunk bytes alongside the metadata via the storage backend's optional metadata slot.
• Loads a single chunk or lists every persisted chunk id for the configured fileId, in ascending order.
• Structural ChunkStorageLike view so tests can substitute a Map.

Usage: Basic

import { MemoryStorage } from "@suaveplan/storage";
import { StorageChunkPersister } from "@suaveplan/streams-chunk-queue";

const storage = new MemoryStorage();
const persister = new StorageChunkPersister({ storage, fileId: 12345n });

Usage: Advanced

import { IndexedDBStorage } from "@suaveplan/storage";

const storage = await IndexedDBStorage.open("downloads", "chunks");
const persister = new StorageChunkPersister({
    storage,
    fileId: 12345n,
});
const queue = new StreamingChunkQueue({
    fileId: 12345n,
    totalChunks: 100n,
    persister,
});
await queue.loadAllChunks();

API Reference

chunkStorageKey(fileId, chunkId) → string

file:<fileId>:chunk:<chunkId>.

chunkStoragePrefix(fileId) → string

file:<fileId>:chunk:.

ChunkPersister

interface ChunkPersister {
    persist(chunk: StoredChunk): Promise<void>;
    load(chunkId: bigint): Promise<Uint8Array | undefined>;
    listChunkIds(): Promise<bigint[]>;
}

StorageChunkPersister

new StorageChunkPersister({ storage, fileId });

Implements ChunkPersister over any ChunkStorageLike. listChunkIds filters by prefix, parses the trailing segment as bigint, and sorts ascending — the ordering means StreamingChunkQueue.loadAllChunks ingests in the canonical chunk order.

Implementation Notes

BigInt is converted to string before being stored in storage metadata so the JSON wire form is portable. The persister does NOT attempt to delete entries — chunks are durable until the consumer explicitly clears the underlying storage. Keys with an empty trailing segment (file:7:chunk:) are silently dropped from listChunkIds because they cannot be parsed as a bigint.