Initial Commit
This commit is contained in:
@@ -0,0 +1,129 @@
|
||||
# Bitpacker stub completion
|
||||
|
||||
**Packages**: `@suaveplan/bitpacker-delta`, `@suaveplan/bitpacker-network`, `@suaveplan/bitpacker-compressed`, `@suaveplan/bitpacker-streaming`, `@suaveplan/bitpacker-schema`
|
||||
**Status**: PROPOSED
|
||||
**Type**: spec-conformance + feature
|
||||
|
||||
## Why
|
||||
|
||||
The 2026-05-02 audit of the `bitpacker-modular-restructure` change (now reflected in its updated parity table) found five places where the modular restructure ticked a box but the underlying work was either a stub, misplaced, or contained a determinism rule violation. Genesis policy is "no half-baked or mediocre libraries" (CLAUDE.md §Workflow rule 3) and "no module-scope `Math.random()` / `Date.now()`" (enterprise/bitpacker.md L40). This change closes those five gaps in one coordinated pass so downstream consumers (drive, network, packer) can rely on a single coherent bitpacker surface.
|
||||
|
||||
The five gaps:
|
||||
|
||||
1. **`bitpacker-delta`**: 35 L inline `DeltaManager` in `index.ts` violates the determinism rule (`Date.now()`, `JSON.stringify`); upstream has a 269 L `DeltaManager` + 149 L `DeltaStateManager` with field-level deltas, configurable XOR/string-diff/threshold modes, and a state-key-sharded LRU.
|
||||
2. **`bitpacker-network`**: 48 L stub in `index.ts` uses module-scope `Math.random()` and exposes only three free functions; upstream has a 598 L `NetworkSimulator` class supporting bandwidth limits, jitter, packet reordering, duplication, corruption, burstiness, random disconnects, and statistics — but the full implementation was misplaced into `bitpacker-game/src/network/network-simulation.ts` (622 L) instead of `bitpacker-network/`.
|
||||
3. **`bitpacker-compressed`**: 61 L of standalone RLE labelled "placeholder for a more sophisticated compression algorithm"; upstream has a 327 L `CompressedBitPacker<T> extends BitPacker<T>` class plus a 336 L `CompressionRegistry` with `NoCompressionAdapter`, `GzipCompressionAdapter`, and `DeflateCompressionAdapter`.
|
||||
4. **`bitpacker-streaming`**: 87 L of chunked helpers; the 406 L `StreamingBitPacker<T> extends EventEmitter` class with `packStream` / `unpackStream`, validation, progress events, and per-chunk processors is absent.
|
||||
5. **`bitpacker-schema`**: 50 L `SchemaRegistry` is a `Map` wrapper; upstream's 576 L file ships `SchemaRegistryStorage` interface, `InMemorySchemaRegistryStorage`, `LocalStorageSchemaRegistryStorage`, plus the real `SchemaRegistry` (versioned register / load / list / delete with Zod-validated metadata) and the `createInMemorySchemaRegistry` / `createLocalStorageSchemaRegistry` factories.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/bitpacker-delta`
|
||||
|
||||
- ADDED: `DeltaStateManager` class — keys-sharded LRU of previous states, deep-cloned on store, evicted in insertion order; default capacity 100.
|
||||
- MODIFIED: `DeltaManager` class — replaces the 35 L inline class; takes a `DeltaConfig` (mode `none` / `xor` / `string-diff` / `field-level`, `xorNumbers`, `diffStrings`, `threshold`, `maxFieldCount`, `stateKey`); `detectChanges(current, schema, stateKey?)` returns `FieldChange[]` indexed by schema field order; takes an injectable `Clock` (no module-scope `Date.now()`); reports `JSON.stringify` only behind a feature flag, defaulting to a structural equality walker that respects `BitPackerSchema` field order.
|
||||
- ADDED: `FieldChange` interface — `{ index, key, previousValue, currentValue }`.
|
||||
- REMOVED: the 35 L inline `DeltaManager` from `bitpacker-delta/src/index.ts` (violates determinism rule).
|
||||
|
||||
### `@suaveplan/bitpacker-network`
|
||||
|
||||
- ADDED: `NetworkSimulator extends EventEmitter` class — bandwidth limit, latency baseline + jitter, packet loss probability, ordered/unordered delivery, duplication probability, corruption probability, burstiness scheduler, random-disconnect scheduler, `NetworkSimulationStats` collector.
|
||||
- ADDED: `NETWORK_CONDITIONS` preset map — `IDEAL`, `HOME`, `MOBILE_4G`, `MOBILE_3G`, `SATELLITE`, `LOSSY_WIFI`, `CUSTOM` (matches upstream constants).
|
||||
- ADDED: `createNetworkSimulator(options)` factory.
|
||||
- ADDED: `PacketStatus` enum and `PacketInfo` interface.
|
||||
- MODIFIED: every `Math.random()` call replaced with an injectable `Rng` (`@suaveplan/types::Rng`) supplied through options; default uses `crypto.getRandomValues` per CLAUDE.md determinism rule.
|
||||
- MODIFIED: every `Date.now()` call replaced with an injectable `Clock`.
|
||||
- REMOVED: the 48 L stub at `bitpacker-network/src/index.ts`.
|
||||
- REMOVED: the misplaced 622 L copy at `bitpacker-game/src/network/network-simulation.ts` (deleted; bitpacker-game re-exports from `@suaveplan/bitpacker-network` for back-compat in its public surface).
|
||||
|
||||
### `@suaveplan/bitpacker-compressed`
|
||||
|
||||
- ADDED: `CompressedBitPacker<T extends BitPackerSchema> extends BitPacker<T>` class — pack/unpack run through a configurable compression adapter; constructor honours `BitPackerConfig.compression` (`algorithm`, `options`, `enabled`); accessors `compressionAlgorithm`, `compressionOptions`, `compressionEnabled`.
|
||||
- ADDED: `CompressionAdapter` interface and `CompressionRegistry` class with built-in `NoCompressionAdapter`, `GzipCompressionAdapter`, `DeflateCompressionAdapter`.
|
||||
- ADDED: `compressData(data, algorithm, options)` and `decompressData(data, algorithm, options)` async helpers.
|
||||
- KEPT: the existing `compressRle` / `decompressRle` helpers for the bytes-only fast path (re-exported under `./rle`).
|
||||
- REMOVED: the "placeholder" disclaimer in the package README.
|
||||
|
||||
### `@suaveplan/bitpacker-streaming`
|
||||
|
||||
- ADDED: `StreamingBitPacker<T extends BitPackerSchema> extends EventEmitter` class — `packStream(data, options)` and `unpackStream(packed, options)`, both async, both with `chunkSize`, `validateChunks`, `emitProgress`, and an optional `chunkProcessor`; emits `progress` and `chunk` events.
|
||||
- KEPT: `packToChunks` / `unpackFromChunks` / `unpackFromChunksAsync` helpers (synchronous, allocation-friendly path); they delegate to `StreamingBitPacker` under the hood.
|
||||
- KEPT: `transform/*` framed Web-Streams transformers.
|
||||
- ADDED: `AbortSignal` propagation through `packStream` / `unpackStream` (existing helpers already honour it).
|
||||
|
||||
### `@suaveplan/bitpacker-schema`
|
||||
|
||||
- ADDED: `SchemaRegistryStorage` interface — `saveSchema`, `loadSchema(namespace, name, version?)`, `listSchemas(namespace?, tag?)`, `deleteSchema`.
|
||||
- ADDED: `InMemorySchemaRegistryStorage` class — in-memory implementation of the storage interface.
|
||||
- ADDED: `LocalStorageSchemaRegistryStorage` class — `localStorage`-backed implementation; constructor accepts a `storageKey` (default `bitpacker-schema-registry`).
|
||||
- MODIFIED: `SchemaRegistry` class — now takes a `SchemaRegistryStorage` (default `InMemorySchemaRegistryStorage`); adds `registerSchema`, `registerVersionedSchema`, `loadSchema`, `listSchemas`, `deleteSchema` async methods that apply auto-versioning (`existing.version + 1`) and stamp `createdAt` / `updatedAt` from an injectable `Clock`.
|
||||
- ADDED: `createInMemorySchemaRegistry()` factory.
|
||||
- ADDED: `createLocalStorageSchemaRegistry(storageKey?)` factory.
|
||||
- KEPT: legacy 50 L `Map`-based surface re-exported from `./legacy` for one release; deprecated.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Crypto subsystem** (provider, registry, noop / libsodium / elliptic adapters; ~1906 L) — tracked separately as `bitpacker-crypto-port`. That port is large enough on its own and depends on crypto-engine primitives that already exist in genesis under `packages/crypto/*`; folding it into this change would dilute review.
|
||||
- **`appendAdler32` / `verifyAdler32` helper wrappers** — polish only, not stub completion. Will land with §14.7 in `bitpacker-modular-restructure/tasks.md`.
|
||||
- **⚠️-marked partials audit** (xor-encoder 34%, string-diff 50%, interpolation 60%, transform 67%) — covered by §14.8; each requires a per-file diff that this change does not undertake.
|
||||
- **`openspec/specs/bitpacker/spec.md`** (the canonical core spec) — drafting it from scratch is its own change; this proposal only adds capability specs for the five caps it touches.
|
||||
- **Replacing `BitPacker` simplification** (1203 L upstream → 178 L genesis) — the simplification is documented in `bitpacker-modular-restructure/design.md` Decision 2 and is intentional. Restoring upstream's `EventEmitter` / built-in delta / built-in checksum on `BitPacker` would re-couple plugins back into core.
|
||||
|
||||
## Source mapping
|
||||
|
||||
Genesis `enterprise/bitpacker.md` is a generic enterprise-quality template (no per-feature requirements), so source mapping cites the upstream package source directly.
|
||||
|
||||
| Requirement | Source file | Source lines | Notes |
|
||||
|---|---|---|---|
|
||||
| Field-level `DeltaManager` with `DeltaConfig` | `packages/universal/bitpacker/src/core/delta/delta-manager.ts` | L1-L269 | Verbatim port; replace `Date.now()` / `JSON.stringify` per CLAUDE.md determinism rule |
|
||||
| `DeltaStateManager` with LRU eviction | `packages/universal/bitpacker/src/core/delta/delta-state.ts` | L1-L149 | Verbatim port; default cap 100 |
|
||||
| `FieldChange` interface | `packages/universal/bitpacker/src/core/delta/delta-manager.ts` | L5-L10 | Verbatim |
|
||||
| `NetworkSimulator extends EventEmitter` | `packages/universal/bitpacker/src/network/network-simulation.ts` | L15-L593 | Move into `bitpacker-network`; replace module-scope `Math.random()` with injectable `Rng` |
|
||||
| `createNetworkSimulator` factory | `packages/universal/bitpacker/src/network/network-simulation.ts` | L594-L598 | Verbatim |
|
||||
| `NETWORK_CONDITIONS` presets, `PacketStatus`, `PacketInfo`, `NetworkSimulationOptions`, `NetworkSimulationStats` | `packages/universal/bitpacker/src/types.ts` | (re-export) | Move type companions alongside the simulator |
|
||||
| `CompressedBitPacker<T> extends BitPacker<T>` | `packages/universal/bitpacker/src/compressed/bitpacker.ts` | L1-L327 | Verbatim |
|
||||
| `CompressionAdapter` + `CompressionRegistry` + built-in `NoCompressionAdapter` / `GzipCompressionAdapter` / `DeflateCompressionAdapter` | `packages/universal/bitpacker/src/compressed/compression.ts` | L1-L336 | Verbatim |
|
||||
| `compressData` / `decompressData` async helpers | `packages/universal/bitpacker/src/compressed/compression.ts` | L75-L120 | Verbatim |
|
||||
| `StreamingBitPacker<T> extends EventEmitter` | `packages/universal/bitpacker/src/streaming/streaming.ts` | L1-L406 | Verbatim; thread `AbortSignal` through |
|
||||
| `SchemaRegistryStorage` interface | `packages/universal/bitpacker/src/types.ts` | (interface re-host) | Co-locate with implementations |
|
||||
| `InMemorySchemaRegistryStorage` | `packages/universal/bitpacker/src/schema/registry/registry.ts` | L13-L141 | Verbatim |
|
||||
| `LocalStorageSchemaRegistryStorage` | `packages/universal/bitpacker/src/schema/registry/registry.ts` | L146-L368 | Verbatim |
|
||||
| `SchemaRegistry` (versioned register / load / list / delete) | `packages/universal/bitpacker/src/schema/registry/registry.ts` | L372-L554 | Replace `new Date()` with injectable `Clock` |
|
||||
| `createInMemorySchemaRegistry` / `createLocalStorageSchemaRegistry` | `packages/universal/bitpacker/src/schema/registry/registry.ts` | L560-L576 | Verbatim |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Drop `Math.random()` / `Date.now()` in favour of injectable `Rng` / `Clock` | Genesis-wide CLAUDE.md determinism rule (rule 9: "NEVER write `// biome-ignore` …" and enterprise/bitpacker.md L40 "Deterministic behaviour — no `Math.random()` / `Date.now()` at module scope"). Not in upstream. |
|
||||
| `BitpackerError` codes registered for compression-failed, decompression-failed, unknown-compression-algorithm, schema-conflict, schema-not-found, network-stat-overflow | Genesis-wide policy that every thrown error route through `@suaveplan/error` (CLAUDE.md §Workflow rule 2). Upstream throws plain `Error`. |
|
||||
| `bitpacker-game` re-export shim for `@suaveplan/bitpacker-network` | Back-compat fence so consumers currently importing `network-simulation` from `bitpacker-game` keep compiling for one release. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each spec delta requirement carries a `**Verified by:**` line pointing at concrete `it(...)` titles. The test files that will exist after this change ships:
|
||||
|
||||
- `packages/bitpacker/bitpacker-delta/src/delta-manager/delta-manager.test.ts` — field-level delta scenarios, state-key sharding, eviction
|
||||
- `packages/bitpacker/bitpacker-delta/src/delta-manager/delta-manager.spec-conformance.test.ts` — `DeltaConfig` strict-mode, deterministic clock, error-code emission
|
||||
- `packages/bitpacker/bitpacker-network/src/simulator/simulator.test.ts` — bandwidth, jitter, loss, duplication, corruption, burstiness, disconnect
|
||||
- `packages/bitpacker/bitpacker-network/src/simulator/simulator.spec-conformance.test.ts` — injected `Rng` / `Clock` used; statistics; preset bindings
|
||||
- `packages/bitpacker/bitpacker-compressed/src/compressed-bit-packer/compressed-bit-packer.test.ts` — round-trip per algorithm
|
||||
- `packages/bitpacker/bitpacker-compressed/src/compression-registry/compression-registry.spec-conformance.test.ts` — adapter registration / replacement
|
||||
- `packages/bitpacker/bitpacker-streaming/src/streaming-bit-packer/streaming-bit-packer.test.ts` — `packStream` / `unpackStream` round-trip, progress events, abort
|
||||
- `packages/bitpacker/bitpacker-streaming/src/streaming-bit-packer/streaming-bit-packer.spec-conformance.test.ts` — chunk processor invocation, validation gating
|
||||
- `packages/bitpacker/bitpacker-schema/src/registry/registry.test.ts` — versioned register, load latest, list by tag, delete by version
|
||||
- `packages/bitpacker/bitpacker-schema/src/registry/registry.spec-conformance.test.ts` — `SchemaRegistryStorage` contract pinned for both adapters; injected `Clock` stamps `createdAt` / `updatedAt`
|
||||
- `tests/bitpacker-game-network-shim.test.ts` (cross-package) — pins the back-compat re-export so removal of the shim is a deliberate spec change
|
||||
|
||||
## Migration impact
|
||||
|
||||
Consumers grepping for current bitpacker stubs:
|
||||
|
||||
- **`@suaveplan/bitpacker-delta`** — only the inline `DeltaManager` in `index.ts` is exported. The rewrite preserves that name (`DeltaManager`), so existing imports compile; the constructor signature changes from `()` to `(config?: DeltaConfig, options?: { clock?: Clock })` — call sites that did `new DeltaManager()` continue to work.
|
||||
- **`@suaveplan/bitpacker-network`** — only `simulateLatency`, `shouldDropPacket`, `applyNetworkConditions` are currently exported. They are kept (now backed by the full simulator) and a new class export is added. Existing call sites compile.
|
||||
- **`@suaveplan/bitpacker-compressed`** — only `compressRle` / `decompressRle` are currently exported. Kept; moved to `./rle` subpath but also re-exported from root for one release.
|
||||
- **`@suaveplan/bitpacker-streaming`** — `packToChunks` / `unpackFromChunks` / `unpackFromChunksAsync` and the `transform/*` exports are kept; `StreamingBitPacker` is a new class.
|
||||
- **`@suaveplan/bitpacker-schema`** — current `SchemaRegistry` `Map` API (`register`, `resolve`, `unregister`, `has`, `names`, `clear`) is preserved on a `LegacySchemaRegistry` alias re-exported from `./legacy`; the canonical name is replaced with the storage-backed version. One-release deprecation window.
|
||||
- **`@suaveplan/bitpacker-game`** — drops the misplaced `network/network-simulation.ts`; ships a thin re-export of `NetworkSimulator` from `@suaveplan/bitpacker-network` to keep the public surface stable for one release. Any downstream that imported `network-simulation.ts` directly via a deep path needs to switch to the package surface — flagged as a BREAKING **deep-path** change in `proposal.md`.
|
||||
|
||||
A `bunx turbo build typecheck test --filter=...@suaveplan/bitpacker-*` sweep runs in §6.5 of `tasks.md` to verify all in-repo dependents compile and test green before archive.
|
||||
Reference in New Issue
Block a user