@suaveplan/drive-plugin-chunks - Chunk-addressing plugin for @suaveplan/drive. Defines the version 0x0001 binary chunk format, server-signed manifests, storage layouts, and streaming encrypt-then-chunk pipeline.

Summary

Chunk-addressing plugin for @suaveplan/drive. Splits files into signed, encrypted chunks using the locked version 0x0001 binary format, with server-signed manifests and two storage layouts (sparse-file and chunk-keys).

Documentation tree

chunk — Wraps the chunk wire format above the bare header: takes raw chunk bytes (header + signature + ciphertext) and parses t…
header — Encodes and parses the fixed-width 100-byte metadata header that prefixes every chunk in the `@suaveplan/drive-plugin-c…
inline — (no submodules)
- inline-hint — (no submodules)
auto — Resolves the best chunk layout for a given storage at runtime: prefers sparse-file when the storage advertises `capab…
chunk-keys — Implements the chunk-keys layout strategy that stores each chunk under a discrete storage key (`chunks//<c…
sparse-file — Implements the sparse-file chunk layout that stores all chunks of a file as offset-addressable bytes inside a single…
build — Constructs a server-signed Manifest for an ingested file: bundles the file metadata (fileId, fileSize, `chunkSize…
load — Loads server-signed manifests from the two supported sources: a sentinel chunk inside the file's own chunk space (`mani…
source — Validates the manifestSource configuration before the plugin starts wiring its read middleware. The function is small…
decrypt-chunk — The single-chunk decryption helper that runs on the read side of the pipeline. Given chunk metadata (already parsed via…
encrypt-stream — The encryption pipeline that transforms a plaintext file into a stream of signed encrypted chunks ready for persistence…
sign-chunk — Signs a chunk under the server's Ed25519 private key. Signatures cover the full encoded chunk (metadata header bytes pl…
verify-chunk — Verifies a chunk's Ed25519 signature against the server's public key as the gate that runs on receipt of every chunk —…
schemas — (no submodules)
- manifest.schema — (no submodules)
error-codes — Defines the structured error registry for @suaveplan/drive-plugin-chunks. Every chunk-format failure mode the plugin…
plugin — Implements the @suaveplan/drive-plugin-chunks plugin object that consumes the Drive plugin slot, splits files into…

- [inline-hint](./src/inline/inline-hint.md) — _(no submodules)_- [auto](./src/layout/auto.md) — Resolves the best chunk layout for a given storage at runtime: prefers `sparse-file` when the storage advertises `capab…

chunk-keys — Implements the chunk-keys layout strategy that stores each chunk under a discrete storage key (`chunks//<c…
sparse-file — Implements the sparse-file chunk layout that stores all chunks of a file as offset-addressable bytes inside a single…
build — Constructs a server-signed Manifest for an ingested file: bundles the file metadata (fileId, fileSize, `chunkSize…
load — Loads server-signed manifests from the two supported sources: a sentinel chunk inside the file's own chunk space (`mani…
source — Validates the manifestSource configuration before the plugin starts wiring its read middleware. The function is small…
decrypt-chunk — The single-chunk decryption helper that runs on the read side of the pipeline. Given chunk metadata (already parsed via…
encrypt-stream — The encryption pipeline that transforms a plaintext file into a stream of signed encrypted chunks ready for persistence…
sign-chunk — Signs a chunk under the server's Ed25519 private key. Signatures cover the full encoded chunk (metadata header bytes pl…
verify-chunk — Verifies a chunk's Ed25519 signature against the server's public key as the gate that runs on receipt of every chunk —…
schemas — (no submodules)- manifest.schema — (no submodules)- error-codes — Defines the structured error registry for @suaveplan/drive-plugin-chunks. Every chunk-format failure mode the plugin…
plugin — Implements the @suaveplan/drive-plugin-chunks plugin object that consumes the Drive plugin slot, splits files into…
inline-hint — (no submodules)- auto — Resolves the best chunk layout for a given storage at runtime: prefers sparse-file when the storage advertises `capab…
chunk-keys — Implements the chunk-keys layout strategy that stores each chunk under a discrete storage key (`chunks//<c…
sparse-file — Implements the sparse-file chunk layout that stores all chunks of a file as offset-addressable bytes inside a single…
build — Constructs a server-signed Manifest for an ingested file: bundles the file metadata (fileId, fileSize, `chunkSize…
load — Loads server-signed manifests from the two supported sources: a sentinel chunk inside the file's own chunk space (`mani…
source — Validates the manifestSource configuration before the plugin starts wiring its read middleware. The function is small…
decrypt-chunk — The single-chunk decryption helper that runs on the read side of the pipeline. Given chunk metadata (already parsed via…
encrypt-stream — The encryption pipeline that transforms a plaintext file into a stream of signed encrypted chunks ready for persistence…
sign-chunk — Signs a chunk under the server's Ed25519 private key. Signatures cover the full encoded chunk (metadata header bytes pl…
verify-chunk — Verifies a chunk's Ed25519 signature against the server's public key as the gate that runs on receipt of every chunk —…
schemas — (no submodules)- manifest.schema — (no submodules)- error-codes — Defines the structured error registry for @suaveplan/drive-plugin-chunks. Every chunk-format failure mode the plugin…
plugin — Implements the @suaveplan/drive-plugin-chunks plugin object that consumes the Drive plugin slot, splits files into…
inline-hint — (no submodules)- auto — Resolves the best chunk layout for a given storage at runtime: prefers sparse-file when the storage advertises `capab…
chunk-keys — Implements the chunk-keys layout strategy that stores each chunk under a discrete storage key (`chunks//<c…
sparse-file — Implements the sparse-file chunk layout that stores all chunks of a file as offset-addressable bytes inside a single…
build — Constructs a server-signed Manifest for an ingested file: bundles the file metadata (fileId, fileSize, `chunkSize…
load — Loads server-signed manifests from the two supported sources: a sentinel chunk inside the file's own chunk space (`mani…
source — Validates the manifestSource configuration before the plugin starts wiring its read middleware. The function is small…
decrypt-chunk — The single-chunk decryption helper that runs on the read side of the pipeline. Given chunk metadata (already parsed via…
encrypt-stream — The encryption pipeline that transforms a plaintext file into a stream of signed encrypted chunks ready for persistence…
sign-chunk — Signs a chunk under the server's Ed25519 private key. Signatures cover the full encoded chunk (metadata header bytes pl…
verify-chunk — Verifies a chunk's Ed25519 signature against the server's public key as the gate that runs on receipt of every chunk —…
schemas — (no submodules)- manifest.schema — (no submodules)- error-codes — Defines the structured error registry for @suaveplan/drive-plugin-chunks. Every chunk-format failure mode the plugin…
plugin — Implements the @suaveplan/drive-plugin-chunks plugin object that consumes the Drive plugin slot, splits files into…
inline-hint — (no submodules)- auto — Resolves the best chunk layout for a given storage at runtime: prefers sparse-file when the storage advertises `capab…
chunk-keys — Implements the chunk-keys layout strategy that stores each chunk under a discrete storage key (`chunks//<c…
sparse-file — Implements the sparse-file chunk layout that stores all chunks of a file as offset-addressable bytes inside a single…
build — Constructs a server-signed Manifest for an ingested file: bundles the file metadata (fileId, fileSize, `chunkSize…
load — Loads server-signed manifests from the two supported sources: a sentinel chunk inside the file's own chunk space (`mani…
source — Validates the manifestSource configuration before the plugin starts wiring its read middleware. The function is small…
decrypt-chunk — The single-chunk decryption helper that runs on the read side of the pipeline. Given chunk metadata (already parsed via…
encrypt-stream — The encryption pipeline that transforms a plaintext file into a stream of signed encrypted chunks ready for persistence…
sign-chunk — Signs a chunk under the server's Ed25519 private key. Signatures cover the full encoded chunk (metadata header bytes pl…
verify-chunk — Verifies a chunk's Ed25519 signature against the server's public key as the gate that runs on receipt of every chunk —…
schemas — (no submodules) - manifest.schema — (no submodules)- error-codes — Defines the structured error registry for @suaveplan/drive-plugin-chunks. Every chunk-format failure mode the plugin…
plugin — Implements the @suaveplan/drive-plugin-chunks plugin object that consumes the Drive plugin slot, splits files into…

Install

bun add @suaveplan/drive-plugin-chunks

Quick start

import { Drive } from "@suaveplan/drive";
import { chunksPlugin } from "@suaveplan/drive-plugin-chunks";
import { MemoryStorage } from "@suaveplan/storage/memory";

const storage = Object.assign(new MemoryStorage(), { id: "local" });
const drive = new Drive(storage);
await drive.initialize();

const chunks = await drive.usePlugin(
    chunksPlugin({
        serverPrivKey: ed25519PrivKey,
        serverPubKey: ed25519PubKey,
        chunkSize: 65536,
    })
);

await drive.write("hello.txt", new TextEncoder().encode("world"));
await chunks.ingest("hello.txt");

const manifest = await chunks.loadManifest(fileId);
console.log(manifest.totalChunks);

await drive.dispose();

Modules

src/codec/header.md — binary header encoding / parsing
src/codec/chunk.md — full chunk encode / parse
src/pipeline/encrypt-stream.md — XChaCha20-Poly1305 streaming encryption
src/pipeline/chop-stream.md — ciphertext slicing into chunk payloads
src/pipeline/sign-chunk.md — ed25519 server signing
src/pipeline/verify-chunk.md — signature verification
src/layout/sparse-file.md — single-object sparse layout
src/layout/chunk-keys.md — one-key-per-chunk layout
src/layout/auto.md — automatic layout selection
src/manifest/build.md — manifest assembly and signing
src/manifest/load.md — manifest fetching and validation
src/manifest/source.md — distribution strategy switch
src/plugin.md — plugin handle and Drive middleware
src/types/chunks.types.md — public TypeScript interfaces
src/schemas/manifest.schema.md — Zod boundary schemas
src/error-codes.md — CHUNKS_CODES and ChunksError

14 KiB Raw Blame History