skills/plugins/document-navigator/docs/foundation/data/packer/README.md

@suaveplan/packer - Binary packing/unpacking: whole-file (gzip + AEAD + ed25519) and chunked (locked v1 byte format, streaming pipeline, server-signed manifests, peer-verifiable).

Summary

Binary packing/unpacking utilities. Two modes:

• Whole-file (@suaveplan/packer) — gzip + AEAD (XChaCha20-Poly1305) + ed25519 sign in one shot, schema-driven via @suaveplan/bitpacker. Self-contained pack-and-go for small assets.
• Chunked (@suaveplan/packer/chunked) — chunk-addressed streaming pack/unpack/verify emitting the locked v1 byte format. Used by @suaveplan/drive-plugin-chunks for P2P-distributable file chunks.

Status: scaffolding. See the design document for the in-flight port from ../suaveplan/packages/universal/packer.

Install

bun add @suaveplan/packer

Quick start

import { Packer, Unpacker } from "@suaveplan/packer";

const p = new Packer();
const u = new Unpacker();
const packed = await p.pack(new Uint8Array([1, 2, 3]));
const result = await u.unpack(packed);
// result.metadata.integrity carries BLAKE3 entryDigest + Merkle root

Modules

• integrity/integrity — BLAKE3 per-entry digest, bundle-level Merkle root, computeWholeFileIntegrity, bytesEqual.
• observability/observability — OpenTelemetry counters, histograms, and spans; createPackerObservability, withPackerSpan, withPackerSpanSync.
• i18n/messages — PACKER_MESSAGES English-defaults catalog and defaultPackerResolver / resolvePackerMessage for MessageResolver injection.
• packer/packer — whole-file Packer.pack (gzip + AEAD + ed25519 + BLAKE3 integrity).
• unpacker/unpacker — whole-file Unpacker.unpack with mandatory BLAKE3 integrity verification.
• chunked/packer/packer — streaming v1 chunked pack pipeline.
• chunked/unpacker/unpacker — chunked unpack (unpackChunks, unpackChunksWithManifest).
• chunked/manifest/manifest — server-signed manifest builder/verifier carrying entryDigest + merkleRoot.
• chunked/codec/chunk and chunked/codec/header — v1 byte-layout encoder/parser.
• chunked/crypto/aead — XChaCha20-Poly1305 wrapper + per-chunk nonce derivation.
• chunked/verifier/chunk-verifier — server-pubkey-only verifier for peer-mesh consumers.
• streaming/normalize-source — source-shape normaliser.
• error-codes — PACKER_CODES registry, including PACKER.INTEGRITY_FAILED.

Documentation tree

• abstract — (no submodules)
  • abstract — (no submodules)
• chunk — Encodes and parses the wire-format v1 chunk in its entirety: metadata(100) || sig(64) || ciphertext(payloadLen). Comp…
• header — Encodes and parses the 100-byte metadata portion of a v1 chunk on its own (without sig or ciphertext). Every chunk on t…
• crypto — (no submodules)
  • aead — (no submodules)
• manifest — Builds, signs, and verifies the v1 manifest — the server-signed object that maps a fileId to its full chunk list, enc…
  • manifest — Builds, signs, and verifies the v1 manifest — the server-signed object that maps a fileId to its full chunk list, enc…
• packer — (no submodules)
  • packer — (no submodules)
• unpacker — (no submodules)
  • unpacker — (no submodules)
• verifier — (no submodules)
  • chunk-verifier — (no submodules)
• flags — flags.ts wires @suaveplan/packer into the @suaveplan/feature-flags ecosystem so future wire-format versions can b…
  • flags — flags.ts wires @suaveplan/packer into the @suaveplan/feature-flags ecosystem so future wire-format versions can b…
• i18n — @suaveplan/packer/i18n carries the per-package message catalog every error thrown by @suaveplan/packer flows throug…
  • messages — @suaveplan/packer/i18n carries the per-package message catalog every error thrown by @suaveplan/packer flows throug…
• idempotency — Repeat packs over identical inputs are a guaranteed no-op: the second call returns the cached PackedData without re-r…
  • idempotency — Repeat packs over identical inputs are a guaranteed no-op: the second call returns the cached PackedData without re-r…
• integrity — @suaveplan/packer/integrity provides the content-addressable integrity layer the rest of the package uses to detect a…
  • integrity — @suaveplan/packer/integrity provides the content-addressable integrity layer the rest of the package uses to detect a…
• observability — @suaveplan/packer/observability centralises every OpenTelemetry counter, histogram, and span the package emits, so a…
  • observability — @suaveplan/packer/observability centralises every OpenTelemetry counter, histogram, and span the package emits, so a…
• packer — (no submodules)
  • packer — (no submodules)
• plugins — plugins.ts exposes a small, opinionated extension surface that lets downstream packages contribute behaviour to `Pack…
  • plugins — plugins.ts exposes a small, opinionated extension surface that lets downstream packages contribute behaviour to `Pack…
• retry — Wraps ReadableStream reads with the retry primitives from @suaveplan/retry so transient IO failures (network blip,…
  • retryable-read — Wraps ReadableStream reads with the retry primitives from @suaveplan/retry so transient IO failures (network blip,…
• runtime — Every production code path in @suaveplan/packer reads wall-clock time and randomness through an injected `PackerRunti…
  • runtime — Every production code path in @suaveplan/packer reads wall-clock time and randomness through an injected `PackerRunti…
• schemas — registration.ts wires every Zod schema exported from packer.schemas.ts and a canonical packer-chunk-format wire-f…
  • registration — registration.ts wires every Zod schema exported from packer.schemas.ts and a canonical packer-chunk-format wire-f…
• streaming — (no submodules)
  • normalize-source — (no submodules)
• unpacker — (no submodules)
  • unpacker — (no submodules)
• error-codes — Defines the stable error code registry for @suaveplan/packer. Every failure surfaced by the package — invalid inputs,…

- [abstract](./src/abstract/abstract.md) — _(no submodules)_- [chunk](./src/chunked/codec/chunk.md) — Encodes and parses the wire-format v1 chunk in its entirety: `metadata(100) || sig(64) || ciphertext(payloadLen)`. Comp…

• header — Encodes and parses the 100-byte metadata portion of a v1 chunk on its own (without sig or ciphertext). Every chunk on t…
• crypto — (no submodules)- aead — (no submodules)- manifest — (no submodules)- manifest — (no submodules)- packer — (no submodules)- packer — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- verifier — (no submodules)- chunk-verifier — (no submodules)- packer — (no submodules)- packer — (no submodules)- streaming — (no submodules)- normalize-source — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- error-codes — Defines the stable error code registry for @suaveplan/packer. Every failure surfaced by the package — invalid inputs,…
• abstract — (no submodules)- chunk — Encodes and parses the wire-format v1 chunk in its entirety: metadata(100) || sig(64) || ciphertext(payloadLen). Comp…
• header — Encodes and parses the 100-byte metadata portion of a v1 chunk on its own (without sig or ciphertext). Every chunk on t…
• crypto — (no submodules)- aead — (no submodules)- manifest — (no submodules)- manifest — (no submodules)- packer — (no submodules)- packer — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- verifier — (no submodules)- chunk-verifier — (no submodules)- packer — (no submodules)- packer — (no submodules)- streaming — (no submodules)- normalize-source — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- error-codes — Defines the stable error code registry for @suaveplan/packer. Every failure surfaced by the package — invalid inputs,…
• abstract — (no submodules)- chunk — Encodes and parses the wire-format v1 chunk in its entirety: metadata(100) || sig(64) || ciphertext(payloadLen). Comp…
• header — Encodes and parses the 100-byte metadata portion of a v1 chunk on its own (without sig or ciphertext). Every chunk on t…
• crypto — (no submodules)- aead — (no submodules)- manifest — (no submodules)- manifest — (no submodules)- packer — (no submodules)- packer — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- verifier — (no submodules)- chunk-verifier — (no submodules)- packer — (no submodules)- packer — (no submodules)- streaming — (no submodules)- normalize-source — (no submodules)- unpacker — (no submodules)- unpacker — (no submodules)- error-codes — Defines the stable error code registry for @suaveplan/packer. Every failure surfaced by the package — invalid inputs,…
• abstract — (no submodules)- chunk — Encodes and parses the wire-format v1 chunk in its entirety: metadata(100) || sig(64) || ciphertext(payloadLen). Comp…
• header — Encodes and parses the 100-byte metadata portion of a v1 chunk on its own (without sig or ciphertext). Every chunk on t…
• crypto — (no submodules) - aead — (no submodules)- manifest — (no submodules) - manifest — (no submodules)- packer — (no submodules) - packer — (no submodules)- unpacker — (no submodules) - unpacker — (no submodules)- verifier — (no submodules) - chunk-verifier — (no submodules)- packer — (no submodules) - packer — (no submodules)- streaming — (no submodules) - normalize-source — (no submodules)- unpacker — (no submodules) - unpacker — (no submodules)- error-codes — Defines the stable error code registry for @suaveplan/packer. Every failure surfaced by the package — invalid inputs,…

Subpaths

Subpath	Surface
@suaveplan/packer	Packer, Unpacker, whole-file types
@suaveplan/packer/chunked	ChunkedPacker, ChunkedUnpacker, ChunkVerifier, manifest helpers
@suaveplan/packer/streaming	normalizeSource(...) source-normalization helper
@suaveplan/packer/errors	PACKER_CODES, PackerError
@suaveplan/packer/flags	defaultPackerFlags, resolvePackerFlags, PACKER_FLAG_KEYS
@suaveplan/packer/plugins	PackerPlugin, PluginPipeline
@suaveplan/packer/schemas	registerPackerSchemas, getPackerSchemaRegistry, PACKER_CHUNK_FORMAT_V1

Benchmarks

Baseline numbers collected on bun 1.3.14 (x64-linux) — Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz, WSL2. Run locally with bun run benchmark.

Case	Throughput (avg)	Notes
whole-file pack — 1 KiB	~254 µs/iter	gzip+sha256+blake3 hot path
whole-file unpack — 1 KiB	~47 µs/iter	integrity verify + gunzip
chunked pack — single chunk	~654 µs/iter	XChaCha20-Poly1305 + ed25519
chunked pack — 1000 chunks	~562 ms/iter	1 MiB streamed in 1 KiB chunks
integrity verify — 1 KiB	~21 µs/iter	BLAKE3 entry + Merkle root
plugin overhead — pack + noop plugin	~147 µs/iter	observed over the 1 KiB pack baseline

The benchmark suite is advisory: bun run benchmark runs mitata and emits the numbers above. The suite is intentionally excluded from the test gate (vitest excludes src/__benchmarks__/**) — regression tracking is opt-in for local profiling.

Chunk binary format — v1

metadata(100 B) ‖ sig(64 B) ‖ ciphertext(payloadLen B)

See the specification for the byte-exact field offsets pinned in PACKER-CHUNK-01.