SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
suaveplan/openspec/archive/2026-05-02-bitpacker-stub-completion/proposal.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

15 KiB
Raw Permalink Blame History

Bitpacker stub completion

Packages: @suaveplan/bitpacker-delta, @suaveplan/bitpacker-network, @suaveplan/bitpacker-compressed, @suaveplan/bitpacker-streaming, @suaveplan/bitpacker-schema Status: PROPOSED Type: spec-conformance + feature

Why

The 2026-05-02 audit of the bitpacker-modular-restructure change (now reflected in its updated parity table) found five places where the modular restructure ticked a box but the underlying work was either a stub, misplaced, or contained a determinism rule violation. Genesis policy is "no half-baked or mediocre libraries" (CLAUDE.md §Workflow rule 3) and "no module-scope Math.random() / Date.now()" (enterprise/bitpacker.md L40). This change closes those five gaps in one coordinated pass so downstream consumers (drive, network, packer) can rely on a single coherent bitpacker surface.

The five gaps:

  1. bitpacker-delta: 35 L inline DeltaManager in index.ts violates the determinism rule (Date.now(), JSON.stringify); upstream has a 269 L DeltaManager + 149 L DeltaStateManager with field-level deltas, configurable XOR/string-diff/threshold modes, and a state-key-sharded LRU.
  2. bitpacker-network: 48 L stub in index.ts uses module-scope Math.random() and exposes only three free functions; upstream has a 598 L NetworkSimulator class supporting bandwidth limits, jitter, packet reordering, duplication, corruption, burstiness, random disconnects, and statistics — but the full implementation was misplaced into bitpacker-game/src/network/network-simulation.ts (622 L) instead of bitpacker-network/.
  3. bitpacker-compressed: 61 L of standalone RLE labelled "placeholder for a more sophisticated compression algorithm"; upstream has a 327 L CompressedBitPacker<T> extends BitPacker<T> class plus a 336 L CompressionRegistry with NoCompressionAdapter, GzipCompressionAdapter, and DeflateCompressionAdapter.
  4. bitpacker-streaming: 87 L of chunked helpers; the 406 L StreamingBitPacker<T> extends EventEmitter class with packStream / unpackStream, validation, progress events, and per-chunk processors is absent.
  5. bitpacker-schema: 50 L SchemaRegistry is a Map wrapper; upstream's 576 L file ships SchemaRegistryStorage interface, InMemorySchemaRegistryStorage, LocalStorageSchemaRegistryStorage, plus the real SchemaRegistry (versioned register / load / list / delete with Zod-validated metadata) and the createInMemorySchemaRegistry / createLocalStorageSchemaRegistry factories.

What changes

@suaveplan/bitpacker-delta

  • ADDED: DeltaStateManager class — keys-sharded LRU of previous states, deep-cloned on store, evicted in insertion order; default capacity 100.
  • MODIFIED: DeltaManager class — replaces the 35 L inline class; takes a DeltaConfig (mode none / xor / string-diff / field-level, xorNumbers, diffStrings, threshold, maxFieldCount, stateKey); detectChanges(current, schema, stateKey?) returns FieldChange[] indexed by schema field order; takes an injectable Clock (no module-scope Date.now()); reports JSON.stringify only behind a feature flag, defaulting to a structural equality walker that respects BitPackerSchema field order.
  • ADDED: FieldChange interface — { index, key, previousValue, currentValue }.
  • REMOVED: the 35 L inline DeltaManager from bitpacker-delta/src/index.ts (violates determinism rule).

@suaveplan/bitpacker-network

  • ADDED: NetworkSimulator extends EventEmitter class — bandwidth limit, latency baseline + jitter, packet loss probability, ordered/unordered delivery, duplication probability, corruption probability, burstiness scheduler, random-disconnect scheduler, NetworkSimulationStats collector.
  • ADDED: NETWORK_CONDITIONS preset map — IDEAL, HOME, MOBILE_4G, MOBILE_3G, SATELLITE, LOSSY_WIFI, CUSTOM (matches upstream constants).
  • ADDED: createNetworkSimulator(options) factory.
  • ADDED: PacketStatus enum and PacketInfo interface.
  • MODIFIED: every Math.random() call replaced with an injectable Rng (@suaveplan/types::Rng) supplied through options; default uses crypto.getRandomValues per CLAUDE.md determinism rule.
  • MODIFIED: every Date.now() call replaced with an injectable Clock.
  • REMOVED: the 48 L stub at bitpacker-network/src/index.ts.
  • REMOVED: the misplaced 622 L copy at bitpacker-game/src/network/network-simulation.ts (deleted; bitpacker-game re-exports from @suaveplan/bitpacker-network for back-compat in its public surface).

@suaveplan/bitpacker-compressed

  • ADDED: CompressedBitPacker<T extends BitPackerSchema> extends BitPacker<T> class — pack/unpack run through a configurable compression adapter; constructor honours BitPackerConfig.compression (algorithm, options, enabled); accessors compressionAlgorithm, compressionOptions, compressionEnabled.
  • ADDED: CompressionAdapter interface and CompressionRegistry class with built-in NoCompressionAdapter, GzipCompressionAdapter, DeflateCompressionAdapter.
  • ADDED: compressData(data, algorithm, options) and decompressData(data, algorithm, options) async helpers.
  • KEPT: the existing compressRle / decompressRle helpers for the bytes-only fast path (re-exported under ./rle).
  • REMOVED: the "placeholder" disclaimer in the package README.

@suaveplan/bitpacker-streaming

  • ADDED: StreamingBitPacker<T extends BitPackerSchema> extends EventEmitter class — packStream(data, options) and unpackStream(packed, options), both async, both with chunkSize, validateChunks, emitProgress, and an optional chunkProcessor; emits progress and chunk events.
  • KEPT: packToChunks / unpackFromChunks / unpackFromChunksAsync helpers (synchronous, allocation-friendly path); they delegate to StreamingBitPacker under the hood.
  • KEPT: transform/* framed Web-Streams transformers.
  • ADDED: AbortSignal propagation through packStream / unpackStream (existing helpers already honour it).

@suaveplan/bitpacker-schema

  • ADDED: SchemaRegistryStorage interface — saveSchema, loadSchema(namespace, name, version?), listSchemas(namespace?, tag?), deleteSchema.
  • ADDED: InMemorySchemaRegistryStorage class — in-memory implementation of the storage interface.
  • ADDED: LocalStorageSchemaRegistryStorage class — localStorage-backed implementation; constructor accepts a storageKey (default bitpacker-schema-registry).
  • MODIFIED: SchemaRegistry class — now takes a SchemaRegistryStorage (default InMemorySchemaRegistryStorage); adds registerSchema, registerVersionedSchema, loadSchema, listSchemas, deleteSchema async methods that apply auto-versioning (existing.version + 1) and stamp createdAt / updatedAt from an injectable Clock.
  • ADDED: createInMemorySchemaRegistry() factory.
  • ADDED: createLocalStorageSchemaRegistry(storageKey?) factory.
  • KEPT: legacy 50 L Map-based surface re-exported from ./legacy for one release; deprecated.

Out of scope

  • Crypto subsystem (provider, registry, noop / libsodium / elliptic adapters; ~1906 L) — tracked separately as bitpacker-crypto-port. That port is large enough on its own and depends on crypto-engine primitives that already exist in genesis under packages/crypto/*; folding it into this change would dilute review.
  • appendAdler32 / verifyAdler32 helper wrappers — polish only, not stub completion. Will land with §14.7 in bitpacker-modular-restructure/tasks.md.
  • ⚠️-marked partials audit (xor-encoder 34%, string-diff 50%, interpolation 60%, transform 67%) — covered by §14.8; each requires a per-file diff that this change does not undertake.
  • openspec/specs/bitpacker/spec.md (the canonical core spec) — drafting it from scratch is its own change; this proposal only adds capability specs for the five caps it touches.
  • Replacing BitPacker simplification (1203 L upstream → 178 L genesis) — the simplification is documented in bitpacker-modular-restructure/design.md Decision 2 and is intentional. Restoring upstream's EventEmitter / built-in delta / built-in checksum on BitPacker would re-couple plugins back into core.

Source mapping

Genesis enterprise/bitpacker.md is a generic enterprise-quality template (no per-feature requirements), so source mapping cites the upstream package source directly.

Requirement Source file Source lines Notes
Field-level DeltaManager with DeltaConfig packages/universal/bitpacker/src/core/delta/delta-manager.ts L1-L269 Verbatim port; replace Date.now() / JSON.stringify per CLAUDE.md determinism rule
DeltaStateManager with LRU eviction packages/universal/bitpacker/src/core/delta/delta-state.ts L1-L149 Verbatim port; default cap 100
FieldChange interface packages/universal/bitpacker/src/core/delta/delta-manager.ts L5-L10 Verbatim
NetworkSimulator extends EventEmitter packages/universal/bitpacker/src/network/network-simulation.ts L15-L593 Move into bitpacker-network; replace module-scope Math.random() with injectable Rng
createNetworkSimulator factory packages/universal/bitpacker/src/network/network-simulation.ts L594-L598 Verbatim
NETWORK_CONDITIONS presets, PacketStatus, PacketInfo, NetworkSimulationOptions, NetworkSimulationStats packages/universal/bitpacker/src/types.ts (re-export) Move type companions alongside the simulator
CompressedBitPacker<T> extends BitPacker<T> packages/universal/bitpacker/src/compressed/bitpacker.ts L1-L327 Verbatim
CompressionAdapter + CompressionRegistry + built-in NoCompressionAdapter / GzipCompressionAdapter / DeflateCompressionAdapter packages/universal/bitpacker/src/compressed/compression.ts L1-L336 Verbatim
compressData / decompressData async helpers packages/universal/bitpacker/src/compressed/compression.ts L75-L120 Verbatim
StreamingBitPacker<T> extends EventEmitter packages/universal/bitpacker/src/streaming/streaming.ts L1-L406 Verbatim; thread AbortSignal through
SchemaRegistryStorage interface packages/universal/bitpacker/src/types.ts (interface re-host) Co-locate with implementations
InMemorySchemaRegistryStorage packages/universal/bitpacker/src/schema/registry/registry.ts L13-L141 Verbatim
LocalStorageSchemaRegistryStorage packages/universal/bitpacker/src/schema/registry/registry.ts L146-L368 Verbatim
SchemaRegistry (versioned register / load / list / delete) packages/universal/bitpacker/src/schema/registry/registry.ts L372-L554 Replace new Date() with injectable Clock
createInMemorySchemaRegistry / createLocalStorageSchemaRegistry packages/universal/bitpacker/src/schema/registry/registry.ts L560-L576 Verbatim

Requirements with no source line

Requirement Justification
Drop Math.random() / Date.now() in favour of injectable Rng / Clock Genesis-wide CLAUDE.md determinism rule (rule 9: "NEVER write // biome-ignore …" and enterprise/bitpacker.md L40 "Deterministic behaviour — no Math.random() / Date.now() at module scope"). Not in upstream.
BitpackerError codes registered for compression-failed, decompression-failed, unknown-compression-algorithm, schema-conflict, schema-not-found, network-stat-overflow Genesis-wide policy that every thrown error route through @suaveplan/error (CLAUDE.md §Workflow rule 2). Upstream throws plain Error.
bitpacker-game re-export shim for @suaveplan/bitpacker-network Back-compat fence so consumers currently importing network-simulation from bitpacker-game keep compiling for one release.

Verification approach

Each spec delta requirement carries a **Verified by:** line pointing at concrete it(...) titles. The test files that will exist after this change ships:

  • packages/bitpacker/bitpacker-delta/src/delta-manager/delta-manager.test.ts — field-level delta scenarios, state-key sharding, eviction
  • packages/bitpacker/bitpacker-delta/src/delta-manager/delta-manager.spec-conformance.test.ts — DeltaConfig strict-mode, deterministic clock, error-code emission
  • packages/bitpacker/bitpacker-network/src/simulator/simulator.test.ts — bandwidth, jitter, loss, duplication, corruption, burstiness, disconnect
  • packages/bitpacker/bitpacker-network/src/simulator/simulator.spec-conformance.test.ts — injected Rng / Clock used; statistics; preset bindings
  • packages/bitpacker/bitpacker-compressed/src/compressed-bit-packer/compressed-bit-packer.test.ts — round-trip per algorithm
  • packages/bitpacker/bitpacker-compressed/src/compression-registry/compression-registry.spec-conformance.test.ts — adapter registration / replacement
  • packages/bitpacker/bitpacker-streaming/src/streaming-bit-packer/streaming-bit-packer.test.ts — packStream / unpackStream round-trip, progress events, abort
  • packages/bitpacker/bitpacker-streaming/src/streaming-bit-packer/streaming-bit-packer.spec-conformance.test.ts — chunk processor invocation, validation gating
  • packages/bitpacker/bitpacker-schema/src/registry/registry.test.ts — versioned register, load latest, list by tag, delete by version
  • packages/bitpacker/bitpacker-schema/src/registry/registry.spec-conformance.test.ts — SchemaRegistryStorage contract pinned for both adapters; injected Clock stamps createdAt / updatedAt
  • tests/bitpacker-game-network-shim.test.ts (cross-package) — pins the back-compat re-export so removal of the shim is a deliberate spec change

Migration impact

Consumers grepping for current bitpacker stubs:

  • @suaveplan/bitpacker-delta — only the inline DeltaManager in index.ts is exported. The rewrite preserves that name (DeltaManager), so existing imports compile; the constructor signature changes from () to (config?: DeltaConfig, options?: { clock?: Clock }) — call sites that did new DeltaManager() continue to work.
  • @suaveplan/bitpacker-network — only simulateLatency, shouldDropPacket, applyNetworkConditions are currently exported. They are kept (now backed by the full simulator) and a new class export is added. Existing call sites compile.
  • @suaveplan/bitpacker-compressed — only compressRle / decompressRle are currently exported. Kept; moved to ./rle subpath but also re-exported from root for one release.
  • @suaveplan/bitpacker-streaming — packToChunks / unpackFromChunks / unpackFromChunksAsync and the transform/* exports are kept; StreamingBitPacker is a new class.
  • @suaveplan/bitpacker-schema — current SchemaRegistry Map API (register, resolve, unregister, has, names, clear) is preserved on a LegacySchemaRegistry alias re-exported from ./legacy; the canonical name is replaced with the storage-backed version. One-release deprecation window.
  • @suaveplan/bitpacker-game — drops the misplaced network/network-simulation.ts; ships a thin re-export of NetworkSimulator from @suaveplan/bitpacker-network to keep the public surface stable for one release. Any downstream that imported network-simulation.ts directly via a deep path needs to switch to the package surface — flagged as a BREAKING deep-path change in proposal.md.

A bunx turbo build typecheck test --filter=...@suaveplan/bitpacker-* sweep runs in §6.5 of tasks.md to verify all in-repo dependents compile and test green before archive.

Reference in New Issue View Git Blame Copy Permalink