suaveplan/openspec/archive/2026-05-02-bitpacker-canonical-spec/proposal.md

Bitpacker canonical spec — port adaptive blob from upstream

Package: @suaveplan/bitpacker Status: PROPOSED — decision required (genesis blob strategy diverged from upstream during the modular restructure; see §"Decision required" below) Type: spec port + behavior decision

Decision required

During implementation of this change (2026-05-02), it surfaced that genesis's BlobPackingStrategy does NOT match the upstream contract. The two implementations are wire-incompatible.

Aspect	Upstream	Genesis
canHandle schema test	schema === "~"	schema === "blob" or { _blob: { maxLength: number } }
Wire format	[2-bit tag][2–5 byte length][raw bytes]	[bitsRequired(maxLength)-bit length][raw bytes] (if maxLength set) or variable-length encoded differently for "blob"
Length range	up to 1 TiB via adaptive tag	up to caller-specified maxLength
Schema reservedness	"~" is a single-character reserved keyword	"blob" is a regular keyword + an object form

Three options. The user must pick before this change can fold:

1. Re-align genesis to upstream: re-port BlobPackingStrategy to use the "~" schema and 2-bit width tag exactly as upstream. This is a behavior change with potential wire-incompatibility for any current genesis consumer of "blob" schemas. The upstream canonical spec then folds verbatim into openspec/specs/bitpacker/spec.md.
2. Re-author canonical spec to describe genesis: keep the genesis "blob"/_blob implementation, and write a new canonical spec that documents what genesis actually does (max-length-bounded length-prefix). The upstream "~" spec is then NOT folded into genesis.
3. Coexist: re-port the "~" adaptive strategy alongside the existing "blob" strategy, so consumers can pick. Both contracts get folded; the strategy chain ordering needs to put "~" before "blob" (and both before string), which is documented as a separate requirement.

Recommendation: Option 2 for now (genesis already diverged; the "blob" implementation is in active use; re-aligning would risk wire-compat breaks elsewhere in the monorepo). Confirm before continuing.

This proposal as currently authored assumes Option 1. The spec delta MUST be re-written if Option 2 or 3 is chosen.

Why

Why

Genesis has no openspec/specs/bitpacker/spec.md. Upstream has one at /mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md (32 L) defining three canonical requirements: the adaptive blob schema type "~", the strategy chain ordering that resolves "~" via BlobPackingStrategy before StringPackingStrategy, and safety-limit enforcement on every variable-length type. Those requirements describe behavior that is already implemented in genesis — bitpacker-strategies/strategies/blob/blob.ts exists and bitpacker-validation/safety/ exists — but no spec pins the contract, which means a future refactor can quietly break the wire format without any check failing.

This change folds the upstream canonical requirements into openspec/specs/bitpacker/spec.md. It is a port, not a redesign: every requirement is verbatim from upstream, with **Verified by:** lines added per genesis policy.

What changes

• ADDED: openspec/specs/bitpacker/spec.md (created from this change's spec delta at archive time) with the three canonical requirements.
• ADDED: per-requirement **Verified by:** lines naming the existing test files that pin each scenario today (no new tests written).
• ADDED: a ## Purpose section ported verbatim from upstream — gives the spec the same reading entry point as upstream so a port to a third codebase later remains 1:1 traceable.

Out of scope

• New behavior. This change only documents what is already true.
• Restructuring bitpacker-modular-restructure's spec deltas. That change still has its own specs/bitpacker/spec.md covering the package-restructure-specific requirements (BitWriter / BitReader / strategy manager / BitPacker class). Both this change and the modular restructure fold into the same canonical openspec/specs/bitpacker/spec.md; OpenSpec folding is additive, so the requirements coexist.
• Spec deltas for bitpacker-strategies (where BlobPackingStrategy lives) or bitpacker-validation (where SafetyChecker lives). The upstream canonical spec puts all three requirements under the bitpacker capability and we keep that grouping for traceability; the per-strategy / per-safety-checker behaviors that aren't in upstream's canonical spec remain owned by their own caps.

Source mapping

The entire spec delta derives from upstream's canonical spec.

Requirement	Source file	Source lines	Notes
Adaptive blob schema type "~"	/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md	L9-L26	Verbatim
Blob strategy runs before string strategy	/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md	L28-L29	Verbatim
Safety-limit enforcement	/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md	L31-L32	Verbatim
## Purpose preamble	/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md	L1-L7	Verbatim

Requirements with no source line

Requirement	Justification
**Verified by:** lines on every requirement	Genesis-wide AGENTS.md §13 policy — every requirement carries one at authoring time. Upstream did not enforce this. The lines name the existing tests that pin each scenario today.

Verification approach

No new tests. Each canonical requirement is already pinned by an existing test in genesis — this change names those tests in the **Verified by:** lines.

Requirement	Existing genesis test
Adaptive blob "~" (4 scenarios)	packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts
Blob-before-string strategy ordering	packages/bitpacker/bitpacker-strategies/src/register-default-strategies.test.ts
Safety-limit enforcement	packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts

If any of those test files is missing the named it(...) titles at archive time, the change cannot fold; the missing it(...) must be added first. Adding it is in-scope for this change (it's part of "documenting what is already true").

Migration impact

None — additive only. No package surface changes; no consumer is affected.

This change does NOT block on bitpacker-modular-restructure or bitpacker-stub-completion or bitpacker-crypto-port. All four can archive independently into the same openspec/specs/bitpacker/spec.md (folding is additive). The recommended archive order is canonical-spec → modular-restructure → stub-completion → crypto-port, but it is not enforced.