Initial Commit

openspec/archive/2026-05-02-bitpacker-canonical-spec/proposal.md

@@ -0,0 +1,81 @@
+# Bitpacker canonical spec — port adaptive blob from upstream
+
+**Package**: `@suaveplan/bitpacker`
+**Status**: PROPOSED — **decision required** (genesis blob strategy diverged from upstream during the modular restructure; see §"Decision required" below)
+**Type**: spec port + behavior decision
+
+## Decision required
+
+During implementation of this change (2026-05-02), it surfaced that genesis's `BlobPackingStrategy` does NOT match the upstream contract. The two implementations are wire-incompatible.
+
+| Aspect | Upstream | Genesis |
+|---|---|---|
+| `canHandle` schema test | `schema === "~"` | `schema === "blob"` or `{ _blob: { maxLength: number } }` |
+| Wire format | `[2-bit tag][2–5 byte length][raw bytes]` | `[bitsRequired(maxLength)-bit length][raw bytes]` (if `maxLength` set) or variable-length encoded differently for `"blob"` |
+| Length range | up to 1 TiB via adaptive tag | up to caller-specified `maxLength` |
+| Schema reservedness | `"~"` is a single-character reserved keyword | `"blob"` is a regular keyword + an object form |
+
+Three options. The user must pick before this change can fold:
+
+1. **Re-align genesis to upstream**: re-port `BlobPackingStrategy` to use the `"~"` schema and 2-bit width tag exactly as upstream. This is a behavior change with potential wire-incompatibility for any current genesis consumer of `"blob"` schemas. The upstream canonical spec then folds verbatim into `openspec/specs/bitpacker/spec.md`.
+2. **Re-author canonical spec to describe genesis**: keep the genesis `"blob"`/`_blob` implementation, and write a new canonical spec that documents what genesis actually does (max-length-bounded length-prefix). The upstream `"~"` spec is then NOT folded into genesis.
+3. **Coexist**: re-port the `"~"` adaptive strategy alongside the existing `"blob"` strategy, so consumers can pick. Both contracts get folded; the strategy chain ordering needs to put `"~"` before `"blob"` (and both before string), which is documented as a separate requirement.
+
+Recommendation: **Option 2** for now (genesis already diverged; the "blob" implementation is in active use; re-aligning would risk wire-compat breaks elsewhere in the monorepo). Confirm before continuing.
+
+This proposal as currently authored assumes Option 1. The spec delta MUST be re-written if Option 2 or 3 is chosen.
+
+## Why
+
+## Why
+
+Genesis has no `openspec/specs/bitpacker/spec.md`. Upstream has one at `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` (32 L) defining three canonical requirements: the adaptive blob schema type `"~"`, the strategy chain ordering that resolves `"~"` via `BlobPackingStrategy` before `StringPackingStrategy`, and safety-limit enforcement on every variable-length type. Those requirements describe behavior that is already implemented in genesis — `bitpacker-strategies/strategies/blob/blob.ts` exists and `bitpacker-validation/safety/` exists — but no spec pins the contract, which means a future refactor can quietly break the wire format without any check failing.
+
+This change folds the upstream canonical requirements into `openspec/specs/bitpacker/spec.md`. It is a port, not a redesign: every requirement is verbatim from upstream, with `**Verified by:**` lines added per genesis policy.
+
+## What changes
+
+- ADDED: `openspec/specs/bitpacker/spec.md` (created from this change's spec delta at archive time) with the three canonical requirements.
+- ADDED: per-requirement `**Verified by:**` lines naming the existing test files that pin each scenario today (no new tests written).
+- ADDED: a `## Purpose` section ported verbatim from upstream — gives the spec the same reading entry point as upstream so a port to a third codebase later remains 1:1 traceable.
+
+## Out of scope
+
+- New behavior. This change only documents what is already true.
+- Restructuring `bitpacker-modular-restructure`'s spec deltas. That change still has its own `specs/bitpacker/spec.md` covering the package-restructure-specific requirements (BitWriter / BitReader / strategy manager / BitPacker class). Both this change and the modular restructure fold into the same canonical `openspec/specs/bitpacker/spec.md`; OpenSpec folding is additive, so the requirements coexist.
+- Spec deltas for `bitpacker-strategies` (where `BlobPackingStrategy` lives) or `bitpacker-validation` (where `SafetyChecker` lives). The upstream canonical spec puts all three requirements under the `bitpacker` capability and we keep that grouping for traceability; the per-strategy / per-safety-checker behaviors that aren't in upstream's canonical spec remain owned by their own caps.
+
+## Source mapping
+
+The entire spec delta derives from upstream's canonical spec.
+
+| Requirement | Source file | Source lines | Notes |
+|---|---|---|---|
+| Adaptive blob schema type `"~"` | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L9-L26 | Verbatim |
+| Blob strategy runs before string strategy | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L28-L29 | Verbatim |
+| Safety-limit enforcement | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L31-L32 | Verbatim |
+| `## Purpose` preamble | `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` | L1-L7 | Verbatim |
+
+### Requirements with no source line
+
+| Requirement | Justification |
+|---|---|
+| `**Verified by:**` lines on every requirement | Genesis-wide AGENTS.md §13 policy — every requirement carries one at authoring time. Upstream did not enforce this. The lines name the existing tests that pin each scenario today. |
+
+## Verification approach
+
+No new tests. Each canonical requirement is already pinned by an existing test in genesis — this change names those tests in the `**Verified by:**` lines.
+
+| Requirement | Existing genesis test |
+|---|---|
+| Adaptive blob `"~"` (4 scenarios) | `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts` |
+| Blob-before-string strategy ordering | `packages/bitpacker/bitpacker-strategies/src/register-default-strategies.test.ts` |
+| Safety-limit enforcement | `packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts` |
+
+If any of those test files is missing the named `it(...)` titles at archive time, the change cannot fold; the missing `it(...)` must be added first. Adding it is in-scope for this change (it's part of "documenting what is already true").
+
+## Migration impact
+
+None — additive only. No package surface changes; no consumer is affected.
+
+This change does NOT block on `bitpacker-modular-restructure` or `bitpacker-stub-completion` or `bitpacker-crypto-port`. All four can archive independently into the same `openspec/specs/bitpacker/spec.md` (folding is additive). The recommended archive order is canonical-spec → modular-restructure → stub-completion → crypto-port, but it is not enforced.

openspec/archive/2026-05-02-bitpacker-canonical-spec/specs/bitpacker/spec.md

@@ -0,0 +1,78 @@
+<!-- markdownlint-disable MD024 -->
+
+# Capability Spec Delta: bitpacker (bitpacker-canonical-spec)
+
+**Package**: `@suaveplan/bitpacker`
+**Status**: PROPOSED
+
+> Every requirement below carries a `**Verified by:**` line at authoring time. The requirements are ported from upstream `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` (cited in the proposal source mapping); SHALL phrasing has been added to satisfy the genesis OpenSpec validator. Behavior described here is already implemented in genesis; this change documents the contract.
+
+## Purpose (preamble)
+
+`@suaveplan/bitpacker` provides bit-level binary serialization driven by declarative schemas. It supports fixed-width primitives (`u8`, `i16`, etc.), variable-length strings/arrays with max-sized length prefixes, and — as of 2026-04-10 — an adaptive blob type (`"~"`) whose length-prefix width is self-describing via a 2-bit tag so payloads ranging from bytes to 1 TiB share a schema without wasting bits on small entries.
+
+## ADDED Requirements
+
+### Requirement: Adaptive blob schema type `"~"`
+
+The package SHALL treat schema value `"~"` as a binary blob whose length prefix is self-describing via a 2-bit width tag selecting 2, 3, 4, or 5 bytes of length. Pack input and unpack output SHALL be `Uint8Array` with no encoding.
+
+#### Scenario: tag selected for a small payload
+
+- **WHEN** packing 200 bytes with schema field `data: "~"`
+- **THEN** the written bits are `[tag=0][2-byte length=200][200 raw bytes]`
+
+#### Scenario: tag selected for a large payload
+
+- **WHEN** packing 500 MiB
+- **THEN** the tag is `2` and the length prefix is 4 bytes
+
+#### Scenario: payload over 1 TiB rejected
+
+- **WHEN** packing a buffer whose length exceeds `0xFFFFFFFFFF`
+- **THEN** the packer throws
+
+#### Scenario: round-trip preserves bytes
+
+- **WHEN** a `Uint8Array` is packed then unpacked
+- **THEN** the output is byte-identical to the input
+
+**Verified by:** `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts::"selects 2-byte length tag for payloads under 64 KiB"` AND `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts::"selects 4-byte length tag for payloads over 16 MiB"` AND `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts::"throws on payloads exceeding 1 TiB"` AND `packages/bitpacker/bitpacker-strategies/src/strategies/blob/blob.test.ts::"round-trip preserves bytes byte-identical"`
+
+---
+
+### Requirement: Blob strategy runs before string strategy
+
+The strategy chain SHALL resolve `"~"` via `BlobPackingStrategy` before `StringPackingStrategy` sees it, so the reserved keyword MUST NOT be misinterpreted.
+
+#### Scenario: registerDefaultStrategies orders blob before string
+
+- **WHEN** `registerDefaultStrategies(manager)` runs and `findStrategy("~")` is called
+- **THEN** the returned strategy is `BlobPackingStrategy`, not `StringPackingStrategy`
+
+**Verified by:** `packages/bitpacker/bitpacker-strategies/src/register-default-strategies.test.ts::"BlobPackingStrategy resolves '~' before StringPackingStrategy"`
+
+---
+
+### Requirement: Safety-limit enforcement
+
+The package SHALL validate every variable-length type (`"-N"`, `"-N[M]"`, `"~"`) against `safetyLimits.maxBufferSize` before writing.
+
+#### Scenario: variable-length string above maxBufferSize rejected
+
+- **WHEN** a payload of `("-32")` exceeds `safetyLimits.maxBufferSize`
+- **THEN** the packer throws before allocating a write buffer
+
+#### Scenario: dynamic array above maxBufferSize rejected
+
+- **WHEN** an array `("-32[16]")` whose total bytes exceed `safetyLimits.maxBufferSize` is packed
+- **THEN** the packer throws before writing
+
+#### Scenario: adaptive blob above maxBufferSize rejected
+
+- **WHEN** a blob `"~"` whose length exceeds `safetyLimits.maxBufferSize` is packed
+- **THEN** the packer throws before writing
+
+**Verified by:** `packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts::"rejects variable-length string above maxBufferSize"` AND `packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts::"rejects dynamic array above maxBufferSize"` AND `packages/bitpacker/bitpacker-validation/src/safety/safety-checks.test.ts::"rejects adaptive blob above maxBufferSize"`
+
+<!-- markdownlint-enable MD024 -->

openspec/archive/2026-05-02-bitpacker-canonical-spec/tasks.md

@@ -0,0 +1,32 @@
+# Tasks — Bitpacker canonical spec
+
+## 1. Spec authoring
+
+- [ ] 1.1 Re-read upstream `/mnt/s/web/suaveplan/openspec/specs/bitpacker/spec.md` end-to-end
+- [ ] 1.2 Confirm none of the three canonical requirements have already landed in `bitpacker-modular-restructure/specs/bitpacker/spec.md`
+- [ ] 1.3 Draft `specs/bitpacker/spec.md` with the three upstream requirements verbatim plus the `## Purpose` preamble
+- [ ] 1.4 Add `**Verified by:**` lines naming concrete existing tests per the proposal table
+
+## 2. Existing-test confirmation
+
+- [ ] 2.1 `bitpacker-strategies/src/strategies/blob/blob.test.ts` contains `it(...)` titles matching the four adaptive-blob scenarios. If any title is missing, ADD the test in this change so the spec delta's `**Verified by:**` line is honest.
+- [ ] 2.2 `bitpacker-strategies/src/register-default-strategies.test.ts` contains an `it(...)` pinning blob-before-string ordering. If missing, ADD it.
+- [ ] 2.3 `bitpacker-validation/src/safety/safety-checks.test.ts` contains an `it(...)` pinning maxBufferSize gating for `"-N"`, `"-N[M]"`, and `"~"`. If missing, ADD it.
+- [ ] 2.4 Run `bunx turbo test --filter=@suaveplan/bitpacker-strategies --filter=@suaveplan/bitpacker-validation` — all named `it(...)` titles pass
+
+## 3. Pre-archive verification
+
+- [ ] 3.1 `openspec change validate bitpacker-canonical-spec --strict` exits clean
+- [ ] 3.2 `bun run check` — Biome 0/0/0
+- [ ] 3.3 The three named `it(...)` titles each exist in their stated test file (grep guard)
+
+## 4. Archive
+
+- [ ] 4.1 Fold spec delta into `openspec/specs/bitpacker/spec.md` (creating the file)
+- [ ] 4.2 Move `openspec/changes/bitpacker-canonical-spec/` → `openspec/archive/<yyyy-mm-dd>-bitpacker-canonical-spec/`
+- [ ] 4.3 STATUS.md update: bitpacker canonical spec landed
+
+## Audit Note (2026-05-02)
+
+DEFERRED in the genesis spec-cleanup pass on 2026-05-02. Rationale:
+- The proposal explicitly requires a USER DECISION (3 options for genesis blob strategy alignment). Cannot auto-resolve without user input. Genesis currently uses Option 2 semantics ("blob"/_blob shape); upstream uses Option 1 ("~" tag). 14 tasks open.