82 lines
6.6 KiB
Markdown
82 lines
6.6 KiB
Markdown
# Bitpacker canonical spec — port adaptive blob from upstream
|
||
|
||
**Package**: `@suaveplan/bitpacker`
|
||
**Status**: PROPOSED — **decision required** (genesis blob strategy diverged from upstream during the modular restructure; see §"Decision required" below)
|
||
**Type**: spec port + behavior decision
|
||
|
||
## Decision required
|
||
|
||
During implementation of this change (2026-05-02), it surfaced that genesis's `BlobPackingStrategy` does NOT match the upstream contract. The two implementations are wire-incompatible.
|
||
|
||
| Aspect | Upstream | Genesis |
|
||
|---|---|---|
|
||
| `canHandle` schema test | `schema === "~"` | `schema === "blob"` or `{ _blob: { maxLength: number } }` |
|
||
| Wire format | `[2-bit tag][2–5 byte length][raw bytes]` | `[bitsRequired(maxLength)-bit length][raw bytes]` (if `maxLength` set) or variable-length encoded differently for `"blob"` |
|
||
| Length range | up to 1 TiB via adaptive tag | up to caller-specified `maxLength` |
|
||
| Schema reservedness | `"~"` is a single-character reserved keyword | `"blob"` is a regular keyword + an object form |
|
||
|
||
Three options. The user must pick before this change can fold:
|
||
|
||
1. **Re-align genesis to upstream**: re-port `BlobPackingStrategy` to use the `"~"` schema and 2-bit width tag exactly as upstream. This is a behavior change with potential wire-incompatibility for any current genesis consumer of `"blob"` schemas. The upstream canonical spec then folds verbatim into `openspec/specs/bitpacker/spec.md`.
|
||
2. **Re-author canonical spec to describe genesis**: keep the genesis `"blob"`/`_blob` implementation, and write a new canonical spec that documents what genesis actually does (max-length-bounded length-prefix). The upstream `"~"` spec is then NOT folded into genesis.
|
||
3. **Coexist**: re-port the `"~"` adaptive strategy alongside the existing `"blob"` strategy, so consumers can pick. Both contracts get folded; the strategy chain ordering needs to put `"~"` before `"blob"` (and both before string), which is documented as a separate requirement.
|
||
|
||
Recommendation: **Option 2** for now (genesis already diverged; the "blob" implementation is in active use; re-aligning would risk wire-compat breaks elsewhere in the monorepo). Confirm before continuing.
|
||
|
||
This proposal as currently authored assumes Option 1. The spec delta MUST be re-written if Option 2 or 3 is chosen.
|
||
|
||
## Why
|
||
|
||
## Why
|
||
|
||
Genesis has no `openspec/specs/bitpacker/spec.md`. Upstream has one at `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` (32 L) defining three canonical requirements: the adaptive blob schema type `"~"`, the strategy chain ordering that resolves `"~"` via `BlobPackingStrategy` before `StringPackingStrategy`, and safety-limit enforcement on every variable-length type. Those requirements describe behavior that is already implemented in genesis — `bitpacker-strategies/strategies/blob/blob.ts` exists and `bitpacker-validation/safety/` exists — but no spec pins the contract, which means a future refactor can quietly break the wire format without any check failing.
|
||
|
||
This change folds the upstream canonical requirements into `openspec/specs/bitpacker/spec.md`. It is a port, not a redesign: every requirement is verbatim from upstream, with `**Verified by:**` lines added per genesis policy.
|
||
|
||
## What changes
|
||
|
||
- ADDED: `openspec/specs/bitpacker/spec.md` (created from this change's spec delta at archive time) with the three canonical requirements.
|
||
- ADDED: per-requirement `**Verified by:**` lines naming the existing test files that pin each scenario today (no new tests written).
|
||
- ADDED: a `## Purpose` section ported verbatim from upstream — gives the spec the same reading entry point as upstream so a port to a third codebase later remains 1:1 traceable.
|
||
|
||
## Out of scope
|
||
|
||
- New behavior. This change only documents what is already true.
|
||
- Restructuring `bitpacker-modular-restructure`'s spec deltas. That change still has its own `specs/bitpacker/spec.md` covering the package-restructure-specific requirements (BitWriter / BitReader / strategy manager / BitPacker class). Both this change and the modular restructure fold into the same canonical `openspec/specs/bitpacker/spec.md`; OpenSpec folding is additive, so the requirements coexist.
|
||
- Spec deltas for `bitpacker-strategies` (where `BlobPackingStrategy` lives) or `bitpacker-validation` (where `SafetyChecker` lives). The upstream canonical spec puts all three requirements under the `bitpacker` capability and we keep that grouping for traceability; the per-strategy / per-safety-checker behaviors that aren't in upstream's canonical spec remain owned by their own caps.
|
||
|
||
## Source mapping
|
||
|
||
The entire spec delta derives from upstream's canonical spec.
|
||
|
||
| Requirement | Source file | Source lines | Notes |
|
||
|---|---|---|---|
|
||
| Adaptive blob schema type `"~"` | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L9-L26 | Verbatim |
|
||
| Blob strategy runs before string strategy | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L28-L29 | Verbatim |
|
||
| Safety-limit enforcement | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L31-L32 | Verbatim |
|
||
| `## Purpose` preamble | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L1-L7 | Verbatim |
|
||
|
||
### Requirements with no source line
|
||
|
||
| Requirement | Justification |
|
||
|---|---|
|
||
| `**Verified by:**` lines on every requirement | Genesis-wide AGENTS.md §13 policy — every requirement carries one at authoring time. Upstream did not enforce this. The lines name the existing tests that pin each scenario today. |
|
||
|
||
## Verification approach
|
||
|
||
No new tests. Each canonical requirement is already pinned by an existing test in genesis — this change names those tests in the `**Verified by:**` lines.
|
||
|
||
| Requirement | Existing genesis test |
|
||
|---|---|
|
||
| Adaptive blob `"~"` (4 scenarios) | `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts` |
|
||
| Blob-before-string strategy ordering | `packages/bitpacker/bitpacker-strategies/src/register-default-strategies.test.ts` |
|
||
| Safety-limit enforcement | `packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts` |
|
||
|
||
If any of those test files is missing the named `it(...)` titles at archive time, the change cannot fold; the missing `it(...)` must be added first. Adding it is in-scope for this change (it's part of "documenting what is already true").
|
||
|
||
## Migration impact
|
||
|
||
None — additive only. No package surface changes; no consumer is affected.
|
||
|
||
This change does NOT block on `bitpacker-modular-restructure` or `bitpacker-stub-completion` or `bitpacker-crypto-port`. All four can archive independently into the same `openspec/specs/bitpacker/spec.md` (folding is additive). The recommended archive order is canonical-spec → modular-restructure → stub-completion → crypto-port, but it is not enforced.
|