suaveplan/openspec/archive/2026-04-28-packer-port/proposal.md

OpenSpec Change: packer-port

• Change ID: packer-port
• Package: @suaveplan/packer (new)
• Status: PROPOSED
• Date: 2026-04-28

Summary

Port @suaveplan/packer from ../suaveplan/packages/universal/packer into genesis. Two modes ship as subpaths:

• @suaveplan/packer — whole-file pack/unpack: gzip + AEAD + ed25519 sign in one shot, schema-driven via bitpacker. Self-contained pack-and-go for small assets. Carried over with minor cleanup.
• @suaveplan/packer/chunked — chunk-addressed streaming pack/unpack/verify emitting the locked v1 byte format specced in drive-plugin-chunks-port. Partial rewrite: keeps source's pipeline (streaming source normalisation, AEAD streaming, per-chunk signing, verifier) but drops bitpacker on the chunked path in favour of a byte-aligned fixed-layout codec.

Three structural changes versus source:

1. Chunked wire format = locked v1 (164-byte fixed header + ciphertext). Source's u8/u64/u40 bitpacker schema is replaced. Whole-file mode keeps bitpacker (no fixed-size constraint there).
2. No signature recovery. Source uses signWithRecovery/verifyWithRecovery (1-bit recovery flag); v1 is plain ed25519_sign because every consumer knows the server pubkey out-of-band. Saves a bit, simplifies verify path.
3. Streaming-first chunked pipeline. Encryption runs through a streaming AEAD; the ciphertext stream is chopped at payload_per_chunk = chunkSize - 164 byte offsets. Last chunk is naturally short (no zero-padding). Header payloadLen is authoritative.

Source-side scope

../suaveplan/packages/universal/packer/ v2.0.0 — 37 .ts files, ~3 460 src + ~4 997 test LOC.

src/
├── abstract/        — AbstractPacker base (compression / AEAD / signing helpers)
├── streaming/       — normalizeSource(Uint8Array | ReadableStream | AsyncIterable)
├── packer/          — Packer (whole-file)
├── unpacker/        — Unpacker (whole-file)
├── chunked/
│   ├── packer/      — ChunkedPacker
│   ├── unpacker/    — ChunkedUnpacker
│   └── verifier/    — ChunkVerifier
├── types/
└── integration/

Source whole-file public surface: Packer, Unpacker, signedPacketSchema. Source chunked surface: ChunkedPacker, ChunkedUnpacker, ChunkVerifier, chunkedPacketSchema (replaced by v1 codec).

Dependencies

All present in genesis:

• @suaveplan/bitpacker (packages/primitives/bitpacker) — used by whole-file mode
• @suaveplan/crypto (packages/crypto/crypto) — encrypt/decrypt, AEAD streaming, ed25519 sign/verify, hashing
• @suaveplan/events (packages/events) — EventEmitter base for progress events
• @suaveplan/mime (packages/primitives/mime) — MIME detection on pack input
• @suaveplan/error, @suaveplan/types, @suaveplan/streams, @opentelemetry/api, zod

No new packages are required. No source-side blockers.

What changes (port-time)

1. Layout

packages/data/packer/
├── package.json                       # @suaveplan/packer @ 0.0.0
├── tsconfig.json                      # extends @suaveplan/typescript-config/library
├── src/
│   ├── abstract/                      # ported with minor cleanup
│   ├── streaming/                     # ported as-is
│   ├── packer/                        # whole-file Packer (bitpacker schema)
│   ├── unpacker/                      # whole-file Unpacker
│   ├── chunked/
│   │   ├── codec/                     # NEW: v1 byte-format header encode/decode
│   │   ├── packer/                    # ChunkedPacker — emits v1 chunks via streaming pipeline
│   │   ├── unpacker/                  # ChunkedUnpacker — consumes v1 chunks
│   │   └── verifier/                  # ChunkVerifier — sig + AEAD verify, no decrypt
│   ├── error-codes.ts                 # PACKER_CODES via defineCodes
│   ├── schemas/                       # Zod boundary schemas for options
│   └── index.ts                       # barrel — whole-file surface
└── src/chunked/index.ts               # barrel — chunked surface (subpath ./chunked)

2. Subpath exports

"exports": {
    ".":          { "import": "./dist/index.js", "types": "./dist/index.d.ts" },
    "./chunked":  { "import": "./dist/chunked/index.js", "types": "./dist/chunked/index.d.ts" },
    "./streaming":{ "import": "./dist/streaming/index.js", "types": "./dist/streaming/index.d.ts" },
    "./errors":   { "import": "./dist/error-codes.js", "types": "./dist/error-codes.d.ts" }
}

3. Chunked v1 codec (fixed-layout, byte-aligned, big-endian)

Header layout per drive-plugin-chunks-port CHUNKS-FORMAT-V1-01. Repeated here for completeness:

metadata(100 B):
  0..2     version    uint16 = 0x0001
  2..34    fileId     bytes(32)         // content hash of plaintext
  34..38   chunkId    uint32
  38..42   totalChunks uint32
  42..50   fileSize   uint64
  50..54   chunkSize  uint32            // budget for non-final chunks
  54..58   payloadLen uint32
  58..60   flags      uint16            // v1 MUST be 0x0000
  60..84   nonce      bytes(24)         // XChaCha20 nonce
  84..100  aeadTag    bytes(16)         // Poly1305 tag

sig(64 B):
  100..164 ed25519 signature over metadata[0..100] ‖ ciphertext[0..payloadLen]

ciphertext(payloadLen B):
  164..(164+payloadLen)

Last chunk: payloadLen = fileSize − chunkId * (chunkSize − 164). Non-final chunks: payloadLen = chunkSize − 164. No zero padding.

4. Streaming chunked pipeline (write path)

PackSource ─► normalizeSource ─► AsyncIterable<Uint8Array>
            ─► hash(plaintext) → fileId (sha256, 32 B)
            ─► aead-encrypt-stream (XChaCha20-Poly1305, file key + per-chunk nonce)
            ─► chop at (chunkSize − 164) byte boundaries
            ─► for each slice:
                 derive nonce = baseNonce ⊕ (chunkId as 8 B little-endian, zero-extended to 24 B)
                 build metadata (100 B)
                 ed25519-sign(metadata ‖ ciphertext) → 64 B sig
                 emit metadata ‖ sig ‖ ciphertext
            ─► finally: build manifest (server-signed, fileId-addressed)

End-to-end streaming. No whole-file buffering.

5. Streaming chunked pipeline (read/verify path)

chunk-bytes ─► parseHeader (structural validation: version, flags, payloadLen invariants)
            ─► verifySig (metadata ‖ ciphertext) — reject on fail
            ─► aead-decrypt(ciphertext, nonce, aeadTag, fileKey)
            ─► emit plaintext slice

ChunkVerifier performs structural-parse + sig-verify only (no decrypt) — used by mesh peers that forward chunks without holding the file key.

6. Drops

• signWithRecovery/verifyWithRecovery calls on the chunked path — replaced with plain ed25519_sign/ed25519_verify. Whole-file mode keeps recovery semantics if the source had them (audit during port).
• Bitpacker schema for chunks (chunkedPacketSchema) — replaced by the v1 byte codec.
• Source's EventEmitter -based progress events — replaced by genesis @suaveplan/events EventBus<TMap> typed events.

7. Error codes

PACKER_CODES = defineCodes("PACKER", [INVALID_SOURCE, COMPRESSION_FAILED, ENCRYPTION_FAILED, SIGNATURE_FAILED, VERIFICATION_FAILED, INVALID_FORMAT, VERSION_UNSUPPORTED, FLAGS_RESERVED_NONZERO, PAYLOAD_LEN_INVALID, ABORTED] as const). PackerError extends SuaveplanError<PackerErrorCode>. Frozen.

Impact

• New package packages/data/packer/ at 0.1.0.
• New OpenSpec capability packer at openspec/specs/packer/spec.md.
• Unblocks drive-plugin-chunks-port (which depends on the chunked subpath).
• Source-side compatibility: wire-format break on the chunked side. Whole-file packed data from source v2.0.0 is unaffected (whole-file format unchanged).

Out of scope

• Source's integration/integration.test.ts end-to-end suite — re-authored as conformance tests against the new format, not ported byte-for-byte.
• Compression on chunked path — v1 reserved flags bit 0 for compressed; not implemented in v1.
• Algorithm agility — v1 hardwires XChaCha20-Poly1305 + ed25519. Algo selector via flags bit 1 reserved for v2.
• Browser-specific packer-browser — separate change after this lands.