Initial Commit
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
# `@suaveplan/bitpacker` — Full Port from Suaveplan (v2.0.0)
|
||||
|
||||
**Status:** **SUPERSEDED (2026-05-03)** — fold complete via the modular family; remaining hardening carved to `bitpacker-hardening`.
|
||||
**Target:** `@suaveplan/bitpacker` 0.0.0 → 2.0.0 (full port, replacing the cut-down genesis version)
|
||||
**Source:** `/home/ubuntu/web/suaveplan/packages/universal/bitpacker/` (400 source files, v1.2.0)
|
||||
|
||||
## Triage outcome (2026-05-03)
|
||||
|
||||
This proposal was authored when `@suaveplan/bitpacker` was a 20-file stub. The reality on 2026-05-03 is a 10-package family of ~232 `.ts` files at 100/100/100/100 coverage, delivered through six smaller, archived changes:
|
||||
|
||||
| Archived change | What it delivered |
|
||||
|---|---|
|
||||
| `2026-04-24-enterprise-bitpacker` | Initial enterprise-grade reshape |
|
||||
| `2026-05-02-bitpacker-modular-restructure` | Split the monolith into the 10-package family |
|
||||
| `2026-05-02-bitpacker-stub-completion` | Filled the DeltaManager / NetworkSimulator / CompressedBitPacker / StreamingBitPacker stubs |
|
||||
| `2026-05-02-bitpacker-canonical-spec` | Canonical `openspec/specs/bitpacker/spec.md` (adaptive `"~"` blob, strategy ordering, safety enforcement) |
|
||||
| `2026-05-02-bitpacker-crypto-port` | First-pass `bitpacker-crypto` |
|
||||
| `2026-05-02-bitpacker-crypto-rewrite` | Slimmed to non-overlapping primitives (`SchemaFingerprint`, `HashAlgorithmRegistry`) |
|
||||
|
||||
**Of the original 51 tasks:** 28 were already checked when this triage began, and a further 17 were verified done by the modular work. **6 are genuinely outstanding** and carved into `bitpacker-hardening`:
|
||||
|
||||
1. Zod schemas for every public type (partial today — only `bitpacker-crypto`, `-delta`, `-schema`)
|
||||
2. Register Zod with `@suaveplan/schema-registry` (zero usage today)
|
||||
3. OTel instrumentation across the core (only `bitpacker-crypto` has spans today)
|
||||
4. `fast-check` property tests (zero, dependency not installed)
|
||||
5. `mitata` benchmark suite + CI regression gate (zero, dependency not installed)
|
||||
6. `@suaveplan/feature-flags` integration — decision pending (may be unnecessary)
|
||||
|
||||
**Plus one tracked-separately:** 47 source files in the bitpacker family lack a sibling `.md` (mostly `bitpacker` core 30, `bitpacker-validation` 9, `bitpacker-game` 7, `bitpacker-crypto` 1). Tracked as a documentation pass, not blocking.
|
||||
|
||||
This change folder is retained for archival traceability. **Do not start work here.** Do new work in `bitpacker-hardening`.
|
||||
|
||||
---
|
||||
|
||||
> Original proposal text below — preserved for traceability. The "What changes" sections are accurate as historical intent; for current state, see per-package CLAUDE.md entries and the canonical spec at `openspec/specs/bitpacker/spec.md`.
|
||||
|
||||
## Why
|
||||
|
||||
The genesis `bitpacker` at `packages/primitives/bitpacker` is a cut-down greenfield build (~20 source files) providing only low-level `BitWriter`/`BitReader` and a minimal `defineBitSchema`. It is missing the entire schema-driven packing system that makes suaveplan's bitpacker useful:
|
||||
|
||||
- **Six schema-reachable packers** (`enum`, `datetime`, `varint`, `map`, `set`, `bitarray`) — missing entirely.
|
||||
- **Custom type registry** — `_customType` marker has no wiring.
|
||||
- **Advanced schema primitives** — no optional fields (`?`), unions (`_union`), tuples (`_tuple`), shared sub-schemas (`$ref`), versioning (`_version`), or conditional presence (`_if`/`_when`).
|
||||
- **Game state schemas** — no delta encoding, no compression pipeline.
|
||||
- **Network simulation** — no jitter/latency helpers.
|
||||
- **Streaming packers** — no chunked/iterative encode/decode.
|
||||
- **Enterprise quality gaps** — no Zod schemas, no structured errors, no OTel, no benchmarks, no property tests.
|
||||
|
||||
This violates the directive: "ensure we don't write basic / cut down versions of the original libraries ever again."
|
||||
|
||||
---
|
||||
|
||||
## What changes
|
||||
|
||||
### Phase 1 — Core schema system port
|
||||
Replace the genesis bitpacker source with the full suaveplan bitpacker source, preserving genesis additions (zigzag encoding, `peek`/`seek`, alignment helpers) where they don't conflict.
|
||||
|
||||
### Phase 2 — Schema flexibility (Tier 1–3)
|
||||
Port the `bitpacker-schema-flexibility` change on top of the core:
|
||||
- Tier 1: Register `EnumPacker`, `DateTimePacker`, `VarIntPacker`, `MapPacker`, `SetPacker`, `BitArrayPacker` in `createPackingStrategyManager`.
|
||||
- Tier 2: Wire `CustomTypeRegistry` + `_customType` into the pack/unpack path.
|
||||
- Tier 3: Add `?`, `_union`, `_tuple`, `_def`/`$ref`, `_version`, `_if`/`_when`.
|
||||
|
||||
### Phase 3 — Enterprise hardening
|
||||
Port the `enterprise-bitpacker` requirements:
|
||||
- Zod schemas registered with `@suaveplan/schema-registry`.
|
||||
- Structured error hierarchy (`BITPACKER.*` codes) via `@suaveplan/error`.
|
||||
- OTel instrumentation via `@suaveplan/opentelemetry`.
|
||||
- Injectable `Clock` and `Rng` — no module-scope `Math.random()` / `Date.now()`.
|
||||
- `AbortSignal` on iterative APIs.
|
||||
- `fast-check` property tests + `mitata` benchmark suite.
|
||||
- Explicit named re-exports, expanded `package.json` `exports` map.
|
||||
- Coverage ≥ 95%, docs ≥ 90%.
|
||||
|
||||
### Non-goals
|
||||
|
||||
- No runtime code generation, `eval`, or schema → JS compilation.
|
||||
- No replacement of `BitOperations` / standalone-packer APIs (escape hatch stays).
|
||||
- No general expression language for conditional predicates.
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/bitpacker` to enumerate.
|
||||
|
||||
- `@suaveplan/packer` — imports bitpacker for envelope encoding; must be verified after port
|
||||
- `@suaveplan/network-*` — imports bitpacker for protocol encoding; must be verified after port
|
||||
- `@suaveplan/storage-*` — imports bitpacker for payload encoding; must be verified after port
|
||||
|
||||
### Breaking changes
|
||||
|
||||
The genesis `BitWriter`/`BitReader`/`defineBitSchema` API surface is preserved where possible, but:
|
||||
- `defineBitSchema` return type will change to match suaveplan's richer `BitSchema` interface.
|
||||
- Error types change from plain `Error` to `BitpackerError extends SuaveplanError`.
|
||||
- New runtime deps: `@suaveplan/schema-registry`, `@suaveplan/opentelemetry`, `@suaveplan/feature-flags`.
|
||||
|
||||
---
|
||||
|
||||
## Migration
|
||||
|
||||
For consumers of the genesis cut-down API:
|
||||
- `BitWriter` / `BitReader` — unchanged signatures, but now live under subpath imports.
|
||||
- `defineBitSchema` — return type gains `pack`/`unpack` methods and metadata; existing call sites compile without change.
|
||||
- Error handling — catch `BitpackerError` instead of `Error` for programmatic handling.
|
||||
@@ -0,0 +1,85 @@
|
||||
# Tasks — bitpacker full port
|
||||
|
||||
> **STATUS (2026-05-03 triage):** This change has been **superseded** by the
|
||||
> modular bitpacker family delivered across multiple smaller changes:
|
||||
> `enterprise-bitpacker`, `bitpacker-modular-restructure`,
|
||||
> `bitpacker-stub-completion`, `bitpacker-canonical-spec`,
|
||||
> `bitpacker-crypto-port`, `bitpacker-crypto-rewrite` (all archived
|
||||
> 2026-04-24 → 2026-05-02).
|
||||
>
|
||||
> The remaining genuine hardening items (Zod-everywhere,
|
||||
> schema-registry integration, OTel across core, fast-check property
|
||||
> tests, mitata benchmarks, feature-flags integration) are carved into
|
||||
> the focused **`bitpacker-hardening`** change folder. This folder is
|
||||
> retained for archival traceability and is not active work.
|
||||
>
|
||||
> See `proposal.md` "Triage outcome" section for the full mapping.
|
||||
|
||||
## Phase 0 — Audit & Planning
|
||||
|
||||
- [x] Inventory all 400 suaveplan bitpacker source files by directory — done as part of Phases 3a-3e modular ports; genesis family is 10 packages, ~232 .ts files
|
||||
- [x] Identify which genesis additions (zigzag, peek/seek, align) must be preserved — preserved through every port
|
||||
- [x] Map suaveplan `package.json` exports to genesis target structure — every package has explicit `exports` map
|
||||
- [x] Verify all required workspace deps exist in genesis (`@suaveplan/schema-registry`, `@suaveplan/opentelemetry`, `@suaveplan/feature-flags`, `fast-check`, `mitata`) — `schema-registry` / `opentelemetry` / `feature-flags` exist but are unused by bitpacker; `fast-check` and `mitata` not installed (deferred to `bitpacker-hardening`)
|
||||
|
||||
## Phase 1 — Core Schema System Port
|
||||
|
||||
- [x] Port `src/core/` — BitPacker, BitOperations, PackingStrategyManager, packing context
|
||||
- [x] Port `src/packers/` — all built-in packing strategies (blob, string, array, object, enum, datetime, varint, map, set, bitarray, etc.)
|
||||
- [x] Port `src/schema/` — schema parsing, validation, token registry
|
||||
- [x] Port `src/types/` — type definitions, schema shapes
|
||||
- [x] Port `src/handlers/` — strategy dispatch and handler registry
|
||||
- [x] Port `src/utils/` — utility functions (keep genesis zigzag, pack-bits where they don't conflict)
|
||||
- [x] Port `src/validation/` — safety limits, schema validation
|
||||
- [x] Port `src/iterators/` — streaming/iterative pack/unpack
|
||||
- [x] Port `src/streaming/` — chunked streaming APIs (delivered as `@suaveplan/bitpacker-streaming`, Phase 3d)
|
||||
- [x] Port `src/compressed/` — compression pipeline (delivered as `@suaveplan/bitpacker-compressed`, Phase 3c)
|
||||
- [x] Port `src/crypto/` — checksum helpers (delivered as `@suaveplan/bitpacker-crypto`)
|
||||
- [x] Port `src/network/` — network simulation helpers (delivered as `@suaveplan/bitpacker-network`, Phase 3b)
|
||||
- [x] Port `src/game/` — game state schemas, delta encoding (delivered as `@suaveplan/bitpacker-game` + `@suaveplan/bitpacker-delta`, Phase 3a)
|
||||
- [x] Port `src/serialization/` — serialize/deserialize helpers
|
||||
- [x] Rewrite `src/index.ts` with explicit named re-exports grouped by subpath
|
||||
- [x] Expand `package.json` `exports` map to match every public subpath
|
||||
|
||||
## Phase 2 — Schema Flexibility (Tier 1–3)
|
||||
|
||||
- [x] Register 6 missing packer tokens in `registerDefaultStrategies` (genesis equivalent of `createPackingStrategyManager`)
|
||||
- [x] Wire `CustomTypeRegistry` + `_customType` into `new BitPacker(schema)`
|
||||
- [x] Add `_optional` / `?` suffix support
|
||||
- [x] Add `_union` for discriminated unions
|
||||
- [x] Add `_tuple` for heterogeneous fixed arrays (via `ArraySchemaPackingStrategy` on plain arrays)
|
||||
- [x] Add `_def` / `$ref` for shared sub-schemas (`_ref` + `_defs` strategies exist)
|
||||
- [x] Add `_version` + forward-compat rules — `VersionPackingStrategy` exists and enforces version match; "forward-compat" semantics (tolerating newer/older versions) is a feature decision deferred to `bitpacker-hardening`
|
||||
- [x] Add `_if` / `_when` for conditional presence (JSON-only predicates)
|
||||
- [x] Add regression test asserting schemas stay pure JSON (round-trip through `JSON.stringify`/`JSON.parse`)
|
||||
|
||||
## Phase 3 — Enterprise Hardening
|
||||
|
||||
- [x] Replace all `catch (error: any)` with `unknown` + typed narrowing — banned-pattern scan over `src/` returns 0
|
||||
- [ ] Introduce Zod schemas for every public type — partial (bitpacker-crypto, bitpacker-delta, bitpacker-schema have them; core does not). Carved to `bitpacker-hardening`.
|
||||
- [ ] Register Zod schemas with `@suaveplan/schema-registry` — zero usage today. Carved to `bitpacker-hardening`.
|
||||
- [x] Define structured error hierarchy (`BITPACKER.*` codes) via `@suaveplan/error` — `BitpackerError extends SuaveplanError<BitpackerErrorCode>` with `defineCodes("BITPACKER", [...])` registry covering 30+ codes
|
||||
- [ ] Wire OTel instrumentation (counters, histograms, spans) — only `bitpacker-crypto` has spans today. Carved to `bitpacker-hardening`.
|
||||
- [x] Add injectable `Clock` and `Rng` — done in `bitpacker-network`, `bitpacker-streaming`, `bitpacker-schema`; each has `defaults/default-clock.ts` allow-listed module-scope `Date.now()`
|
||||
- [x] Add `AbortSignal` support to iterative APIs — done in `bitpacker-streaming` (every `*Stream` method) and `bitpacker-crypto` async paths
|
||||
- [ ] Add `fast-check` property tests (round-trip, delta idempotency, checksum distribution) — zero. `fast-check` not installed. Carved to `bitpacker-hardening`.
|
||||
- [ ] Add `mitata` benchmark suite with baselines — zero. `mitata` not installed. Carved to `bitpacker-hardening`.
|
||||
- [ ] Add CI benchmark regression gate — N/A until benchmarks exist. Carved to `bitpacker-hardening`.
|
||||
- [ ] Integrate `@suaveplan/feature-flags` for risky primitives — zero usage. Carved to `bitpacker-hardening` (decision pending — may be unnecessary).
|
||||
- [x] Raise coverage to ≥ 95% — exceeded: 100/100/100/100 across `bitpacker`, `bitpacker-strategies`, `bitpacker-validation`, `bitpacker-game`, and every Phase-3 module
|
||||
- [ ] Ensure every exported symbol has a co-located `.md` — 47 sources without sibling `.md` (bulk: bitpacker 30, bitpacker-validation 9, bitpacker-game 7, bitpacker-crypto 1). Tracked separately as a documentation pass; not blocking.
|
||||
|
||||
## Phase 4 — Verification
|
||||
|
||||
- [x] `bunx turbo lint typecheck test --filter=@suaveplan/bitpacker`
|
||||
- [x] `bunx turbo test:coverage --filter=@suaveplan/bitpacker` — 100/100/100/100
|
||||
- [x] `bunx turbo test --filter=...@suaveplan/bitpacker` (dependent sweep) — 69/69 tasks succeed
|
||||
- [x] Banned-pattern scan (zero output) — clean over `src/`
|
||||
- [ ] Missing `.md` scan (zero output) — 47 missing; documentation pass tracked separately
|
||||
- [x] Tree-shake test on rewritten `index.ts` — `"sideEffects": false` set on all 10 packages; explicit assertion test deferred (low value given the Vite build already verifies tree-shaking via dts-only outputs)
|
||||
|
||||
## Acceptance
|
||||
|
||||
- [x] All suaveplan bitpacker tests pass on genesis (or equivalent coverage) — equivalent: 100/100/100/100 across the family
|
||||
- [x] Downstream consumers (`packer`, `network-*`, `storage-*`) compile and test green — `...@suaveplan/bitpacker` sweep: 69/69 successful
|
||||
- [x] No `any`, `TODO`, `FIXME`, `ts-ignore`, `ts-expect-error` in `src/` — banned-pattern scan returns 0 over the bitpacker family
|
||||
Reference in New Issue
Block a user