SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
admin
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
openspec/archive/2026-05-02-bitpacker-modular-restructure/.openspec.yaml Normal file
View File

@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-04-30

75
openspec/archive/2026-05-02-bitpacker-modular-restructure/design.md Normal file
View File

@@ -0,0 +1,75 @@
## Context
The current genesis bitpacker (`packages/primitives/bitpacker/`, ~20 files) is a greenfield build providing only low-level `BitWriter`/`BitReader` and a minimal `defineBitSchema`. It lacks the schema-driven packing system, advanced strategies, streaming, delta encoding, and enterprise features that suaveplan's bitpacker (`packages/universal/bitpacker/`, ~400 files) provides.
Suaveplan's bitpacker is monolithic — everything lives in one package. This creates three problems:
1. **Bundle bloat**: Consumers who only need `BitWriter`/`BitReader` pull in game-state schemas, network simulation, and crypto checksums.
2. **Slow builds**: A change in `game/prediction/kalman-filter.ts` triggers rebuilds for all bitpacker consumers.
3. **Poor boundaries**: The dependency graph is flat, so there's no clear API contract between core, strategies, and advanced modules.
The workspace glob `packages/*/*` already supports nested packages, so we can create `packages/bitpacker/` as a plugin host without any toolchain changes.
## Goals / Non-Goals
**Goals:**
- Create a `packages/bitpacker/` directory that hosts 10 related workspace packages.
- Core package (`@suaveplan/bitpacker`) stays lightweight: bit I/O, strategy manager, base types.
- Every advanced feature is an opt-in plugin with explicit `dependencies` in its `package.json`.
- Preserve genesis additions (`zigzag`, `peek`/`seek`, `align`) in core.
- Downstream consumers (`packer`, `network-*`, `storage-*`) continue to compile and test green.
- Maintain backward compatibility for `defineBitSchema` call sites.
**Non-Goals:**
- No runtime code generation, `eval`, or schema → JS compilation.
- No replacement of standalone `BitOperations` / packer APIs (escape hatch stays).
- No general expression language for conditional predicates.
- No changes to suaveplan source (this is a genesis-only restructure).
- No attempt to make every plugin tree-shakeable to zero bytes (just package-level separation).
## Decisions
### 1. Package naming: `@suaveplan/bitpacker-*` for all plugins
**Rationale:** Keeps the npm namespace flat and discoverable. `@suaveplan/bitpacker` is the core; every plugin is a sibling name.
**Alternative considered:** `@suaveplan/bitpacker/strategies` subpath exports. Rejected because subpaths don't give independent versioning or build caching.
### 2. Core owns `BitWriter`, `BitReader`, `PackingStrategyManager`, and the `BitPacker` class
**Rationale:** The strategy manager is the coordination layer; it must live in core so that plugins can register strategies. The core does NOT own any concrete strategies beyond a minimal `InvalidSchemaStrategy` catch-all.
### 3. Strategies plugin owns ALL built-in strategies
**Rationale:** Even basic strategies (string, number, boolean) are moved out of core so that alternative strategy sets can be swapped in. Core consumers who need the default strategy set import `@suaveplan/bitpacker-strategies` and call `registerDefaultStrategies(manager)`.
### 4. Preserve `packages/primitives/bitpacker/` temporarily
**Rationale:** Avoid breaking downstream consumers mid-migration. The old package will be deprecated and removed in a follow-up change after all dependents are updated.
### 5. Each plugin re-uses suaveplan source structure internally
**Rationale:** Porting 400 files is error-prone; keeping the same internal directory layout (`src/core/`, `src/packers/`, `src/schema/`, etc.) within each plugin reduces cognitive load.
### 6. Schema plugin owns the schema registry, validation, and versioning
**Rationale:** Schema metadata is a cross-cutting concern. Centralizing it in one plugin prevents circular deps between strategies and validation.
### 7. Error hierarchy: core defines `BitpackerError`, plugins extend it
**Rationale:** Core must not depend on `@suaveplan/error` (to stay lightweight), but plugins can. This is a pragmatic compromise; if `@suaveplan/error` is lightweight enough we may migrate core later.
## Risks / Trade-offs
- **[Risk] Downstream consumers break due to import path changes** → Mitigation: keep `packages/primitives/bitpacker/` as a re-export shim until all dependents are migrated.
- **[Risk] Circular dependencies between plugins** → Mitigation: enforce DAG via CI lint rule. Core → Strategies → Schema → Validation → Streaming/Delta/Game/Network/Crypto/Compressed.
- **[Risk] Build complexity increases with 10 packages** → Mitigation: turbo pipeline caches are per-package; incremental builds should be faster, not slower.
- **[Risk] Plugin API churn during port** → Mitigation: mark all plugin public APIs as `@beta` until Phase 4 verification is complete.
- **[Trade-off] More `package.json` files to maintain** → Acceptable; monorepo tooling (bun workspaces, turbo) handles this.
## Migration Plan
1. Create `packages/bitpacker/bitpacker/` and port core from `packages/primitives/bitpacker/` + suaveplan core.
2. Create remaining 9 plugin packages, porting from suaveplan.
3. Update `packages/primitives/bitpacker/` to re-export from new core (shim).
4. Run dependent sweep (`bunx turbo test --filter=...@suaveplan/bitpacker`).
5. Update downstream consumers to import from new paths.
6. Remove shim package in follow-up change.
## Open Questions
- Should `bitpacker-crypto` depend on `@suaveplan/crypto` or stay self-contained with its own checksum implementations?
- Should `bitpacker-game` live in `packages/game/` instead of `packages/bitpacker/`? (Leaning toward keeping it under `bitpacker/` for discoverability.)
- Do we need a `bitpacker-testing` plugin with shared test fixtures and property-test helpers?

151
openspec/archive/2026-05-02-bitpacker-modular-restructure/proposal.md Normal file
View File

@@ -0,0 +1,151 @@
## Why
The genesis `bitpacker` at `packages/primitives/bitpacker` is a cut-down greenfield build (~20 files) that lacks the schema-driven packing system, advanced strategies, streaming, delta encoding, game-state schemas, and enterprise hardening present in suaveplan's 400-file implementation. Rather than dumping all 400 files into a single monolithic package, we need a modular architecture where the core is lightweight and every advanced feature is an independently versioned, opt-in plugin.
## What Changes
- **Create `packages/bitpacker/` parent directory** as the home for all bitpacker-related workspace packages.
- **Move core bitpacker** from `packages/primitives/bitpacker/` to `packages/bitpacker/bitpacker/`.
- **Split suaveplan's 400 files into 10 workspace packages:**
- `@suaveplan/bitpacker` — Core: `BitWriter`, `BitReader`, `BitPacker`, `PackingStrategyManager`, base types.
- `@suaveplan/bitpacker-strategies` — All built-in packing strategies (string, number, object, array, enum, datetime, varint, map, set, bitarray, blob, bitset, etc.).
- `@suaveplan/bitpacker-schema` — Schema parsing, validation, token registry, matching, versioning, visualization.
- `@suaveplan/bitpacker-streaming` — Chunked streaming encode/decode, transform streams.
- `@suaveplan/bitpacker-delta` — Delta manager, XOR encoder, string diff, number XOR, field-level delta.
- `@suaveplan/bitpacker-game` — Game state schemas, quantization, prediction, interpolation, reconciliation, state manager.
- `@suaveplan/bitpacker-network` — Network simulation helpers (jitter, latency, packet loss).
- `@suaveplan/bitpacker-crypto` — Checksum helpers (CRC32, Adler32, xxHash32), crypto adapters.
- `@suaveplan/bitpacker-compressed` — Compression pipeline.
- `@suaveplan/bitpacker-validation` — Safety limits, schema validation, validation reports.
- **Preserve genesis additions** in core: `zigzagEncode`/`zigzagDecode`, `packBits`/`unpackBits`, `peek`/`seek` on `BitReader`, `align()` on both reader and writer.
- **Preserve backward compatibility** where possible: existing `defineBitSchema` call sites should continue to compile.
- **BREAKING**: Package path changes from `@suaveplan/bitpacker` (implicit in primitives) to explicit `@suaveplan/bitpacker` under `packages/bitpacker/bitpacker/`.
- **BREAKING**: `BitpackerError` will eventually extend `SuaveplanError` (migrated from plain `Error`).
## Parity Checklist
Legend: ✅ ported · ⚠️ partial / stub-thin / misplaced (real gap remaining) · 🚫 not ported · ⛔ intentionally not ported (decision recorded in a follow-up change).
Last verified: 2026-05-02 by side-by-side comparison of `/mnt/s/web/suaveplan/packages/universal/bitpacker/src/` against `packages/bitpacker/*/src/`. Line counts cited are implementation files only (excluding tests).
| Capability | Upstream | Migrated | Notes |
|---|---|---|---|
| `BitWriter` / `BitReader` | ✅ | ✅ | Genesis additions preserved (peek, seek, align) |
| `BitPacker` class | ✅ | ⚠️ | Core simplified by design (1203L → 178L): no EventEmitter, no built-in delta/checksum. Documented in `design.md`. |
| `PackingStrategyManager` | ✅ | ✅ | Chain-of-responsibility pattern preserved |
| `PackingContext` | ✅ | ✅ | Uses `writer`/`reader` instead of `bitOps` |
| Integer packing (signed/unsigned) | ✅ | ✅ | NumberPackingStrategy |
| Fixed-point packing | ✅ | ✅ | NumberPackingStrategy |
| String packing (fixed/variable) | ✅ | ✅ | StringPackingStrategy |
| UUID packing | ✅ | ✅ | StringPackingStrategy |
| Float packing (f32/f64) | ✅ | ✅ | StringPackingStrategy |
| Boolean packing | ✅ | ✅ | BooleanPackingStrategy |
| Null packing | ✅ | ✅ | NullPackingStrategy |
| BigInt packing (i64/u64/arrays) | ✅ | ✅ | BigIntPackingStrategy |
| Object packing | ✅ | ✅ | ObjectPackingStrategy |
| Array packing (tuple + dynamic) | ✅ | ✅ | ArrayPackingStrategy |
| Enum packing | ✅ | ✅ | EnumPackingStrategy |
| Optional packing | ✅ | ✅ | OptionalPackingStrategy |
| OneOf (union) packing | ✅ | ✅ | OneOfPackingStrategy |
| Conditional packing | ✅ | ✅ | ConditionalPackingStrategy |
| Version packing | ✅ | ✅ | VersionPackingStrategy |
| DateTime packing | ✅ | ✅ | DateTimePackingStrategy (simplified) |
| VarInt packing | ✅ | ✅ | VarIntPackingStrategy |
| Map packing | ✅ | ✅ | MapPackingStrategy |
| Set packing | ✅ | ✅ | SetPackingStrategy |
| BitArray packing | ✅ | ✅ | BitArrayPackingStrategy |
| Blob packing | ✅ | ✅ | BlobPackingStrategy |
| Bitset packing | ✅ | ✅ | BitsetPackingStrategy |
| Defs / $ref packing | ✅ | ✅ | DefsPackingStrategy |
| CustomType packing | ✅ | ✅ | CustomTypePackingStrategy |
| Schema registry | ✅ | ⚠️ | `bitpacker-schema/registry/registry.ts` is a 50L `Map` wrapper; old 576L surface (InMemory + LocalStorage storage adapters, factory helpers) NOT ported |
| Schema validation | ✅ | ✅ | validateData / assertValid in bitpacker-schema; full surface in bitpacker-validation (~4249L vs old ~4250L impl) |
| Schema matching | ✅ | ✅ | Ported at `bitpacker-schema/matching/matching.ts` (165L → 173L) |
| Schema migration | ✅ | ✅ | Ported at `bitpacker-schema/migration/migration.ts` (521L → 479L, ~92%) |
| Schema versioning | ✅ | ✅ | Ported at `bitpacker-schema/versioning/versioning.ts` (435L → 365L, ~84%) |
| Schema visualization | ✅ | ✅ | Ported at `bitpacker-schema/visualization/visualization.ts` (893L → 847L, ~95%) |
| Delta manager | ✅ | ❌ | Old 269L `delta-manager.ts` + 149L `delta-state.ts` NOT ported. Genesis has only a 35L inline `DeltaManager` class in `bitpacker-delta/index.ts` using `JSON.stringify` + `Date.now()` (deterministic-clock rule violation). |
| XOR delta encoder | ✅ | ⚠️ | Ported at `bitpacker-delta/xor-encoder.ts` (177L → 60L, ~34%) |
| String diff delta | ✅ | ⚠️ | Ported at `bitpacker-delta/string-diff.ts` (226L → 113L, ~50%) |
| Number XOR delta | ✅ | ⚠️ | Old 232L `number-xor.ts` folded into `xor-encoder.ts`'s `numberXor` export; richer surface lost |
| Game quantization | ✅ | ✅ | quantizeVector / dequantizeVector in bitpacker-game |
| Game prediction | ✅ | ✅ | All 5 files ported in `bitpacker-game/prediction/` (kalman-filter, motion-math, position-predictor, prediction-algorithms, rotation-predictor + entry; 827L → 700L, ~85%) |
| Game interpolation | ✅ | ⚠️ | Ported at `bitpacker-game/interpolation/interpolation.ts` (745L → 447L, ~60% — significant surface trim) |
| Game reconciliation | ✅ | ✅ | Ported at `bitpacker-game/reconciliation/reconciliation.ts` (615L → 478L, ~78%) |
| Game state manager | ✅ | ✅ | GameStateManager in bitpacker-game |
| Game `bitpacker.ts` enhancements | ✅ | ✅ | Ported at `bitpacker-game/bitpacker/bitpacker.ts` (388L → 384L) |
| Game schema-factory | ✅ | ✅ | Ported at `bitpacker-game/schema-factory/schema-factory.ts` (50L → 50L) |
| Network simulation | ✅ | ⚠️ | **Misplaced**: full 622L implementation lives at `bitpacker-game/src/network/network-simulation.ts`; `bitpacker-network/src/index.ts` is a 48L stub using module-scope `Math.random()` (rule violation). Needs to move to `bitpacker-network` and the stub deleted. |
| CRC32 checksum | ✅ | ⚠️ | Functional, but inlined into 80L `bitpacker-crypto/index.ts`. Old 176L `crc32.ts` had separate `appendCrc32`/`verifyAndStripCrc32` helpers — only `appendCrc32`/`verifyAndStripCrc32` survive; helper API surface is flat. |
| Adler32 checksum | ✅ | ⚠️ | Functional, but inlined into 80L `bitpacker-crypto/index.ts`. Old 121L `adler32.ts` had `appendAdler32`/`verifyAdler32` helpers — currently no append/verify wrappers in genesis. |
| xxHash32 checksum | ✅ | ✅ | Ported at `bitpacker-crypto/xxhash32.ts` (re-exported from `index.ts`) |
| Crypto provider | ✅ | ⛔ | Replaced by `Packer` / `Authority` in `restore-crypto-keychain-packer`'s new `@suaveplan/crypto`; auxiliary `SchemaFingerprint` + `HashAlgorithmRegistry` in `bitpacker-crypto-rewrite` (slim). |
| Crypto registry | ✅ | ⛔ | Replaced by `Authority` (in-memory + future HSM/TPM/threshold) in `restore-crypto-keychain-packer`'s `@suaveplan/crypto`. |
| Crypto adapter — noop | ✅ | ⛔ | Replaced by `@suaveplan/testing` fakes against `Authority`; see `restore-crypto-keychain-packer`. |
| Crypto adapter — libsodium | ✅ | ⛔ | Underlying primitives covered by `@suaveplan/crypto-sodium`; opinionated layer is `Packer` + `InMemoryAuthority` in `restore-crypto-keychain-packer`. |
| Crypto adapter — elliptic | ✅ | ⛔ | Underlying primitives covered by `@suaveplan/crypto-curve`; opinionated layer is `Packer` + `InMemoryAuthority` in `restore-crypto-keychain-packer`. |
| Streaming `StreamingPacker` class | ✅ | ❌ | 406L `streaming/streaming.ts` NOT ported. Genesis `bitpacker-streaming/index.ts` (87L) has only chunked helpers (`packToChunks`/`unpackFromChunks{Async}`); the top-level streaming encoder class is missing. |
| Streaming transform | ✅ | ⚠️ | Ported at `bitpacker-streaming/transform/transform.ts` (544L → 365L, ~67%) |
| Compression pipeline | ✅ | ❌ | `bitpacker-compressed/index.ts` is 61L of standalone RLE labelled "placeholder for a more sophisticated compression algorithm". Old 663L (`compressed/bitpacker.ts` 327 + `compressed/compression.ts` 336) `CompressedBitPacker` class NOT ported. |
| Safety limits | ✅ | ✅ | SafetyChecker in bitpacker-validation |
| `defineBitSchema` (legacy) | ✅ | ✅ | Preserved in core for backward compatibility |
| `zigzagEncode`/`zigzagDecode` | ✅ | ✅ | Preserved in core |
| Iterators (`pack-iterator`) | ✅ | ✅ | Ported at `bitpacker/src/iterators/pack-iterator/` |
| Custom-type registry | ✅ | ✅ | Ported at `bitpacker/src/handlers/custom-type-registry.ts` |
| Performance utilities (buffer-pool / lru-cache / monitor) | ✅ | ✅ | Ported at `bitpacker/src/performance/` |
| Serialization | ✅ | ✅ | Ported at `bitpacker/src/serialization/` |
## Outstanding gaps (real, not parity-table-stale)
These are the items above marked ❌ or 🚫 — the work that genuinely remains:
1. ~~**Crypto subsystem** — provider, registry, and noop / libsodium / elliptic adapters (~1906L total). No genesis equivalent exists. Tracked separately by the `bitpacker-full-port` change; should land in a dedicated `bitpacker-crypto-port` follow-up.~~ **Resolved 2026-05-02** in two parallel changes: (a) **`restore-crypto-keychain-packer`** — new opinionated `@suaveplan/crypto` layer with `Packer` / `Unpacker` / `Authority` / `AuditLog` / `RevocationList` / `Keychain` / `Vault` consolidating the 8+ inline sign-then-pack / encrypt-pack / Merkle-log patterns. (b) **`bitpacker-crypto-rewrite` (slim)** — auxiliary bit-packer-side primitives (`SchemaFingerprint`, `HashAlgorithmRegistry`) that don't overlap (a). Decision recorded in `packages/bitpacker/bitpacker-crypto/SUPERSEDED.md`. Predecessor proposals `bitpacker-crypto-port` and the `bitpacker-crypto-rewrite` first draft (with seven primitives) are preserved as decision-history entries.
2. **`StreamingPacker` class** in `bitpacker-streaming` — 406L upstream, currently absent.
3. **`CompressedBitPacker` class** in `bitpacker-compressed` — 663L upstream replaced by 61L placeholder RLE.
4. **`DeltaManager` + `delta-state`** in `bitpacker-delta` — 418L upstream replaced by a 35L inline class that violates the deterministic-clock rule.
5. **Network-simulation misplacement** — the full implementation lives in `bitpacker-game/network/`; `bitpacker-network` itself is a stub.
6. **`SchemaRegistry` storage adapters** — InMemory and LocalStorage adapter classes + factory helpers (the bulk of the 576L upstream file) not ported.
Items marked ⚠️ (XOR encoder, string diff, interpolation, transform, CRC32/Adler32 helper API) have shipped most of the upstream surface but lost some helper functions / refinements; tracked as "polish" follow-ups, not blockers.
## Capabilities
### New Capabilities
- `bitpacker`: Low-level bit I/O and schema-driven packing.
- `bitpacker-strategies`: Modular packing strategies for every schema token.
- `bitpacker-schema`: Schema registry, validation, versioning, and visualization.
- `bitpacker-streaming`: Chunked and transform-stream packing/unpacking.
- `bitpacker-delta`: Delta encoding for efficient repeated state transmission.
- `bitpacker-game`: Game-specific schemas, quantization, prediction, reconciliation.
- `bitpacker-network`: Network simulation and latency/jitter helpers.
- `bitpacker-crypto`: Checksum and crypto adapter integrations.
- `bitpacker-compressed`: Compression pipeline integration.
- `bitpacker-validation`: Safety limits and runtime schema validation.
### Modified Capabilities
- `packer`: The packer package at `packages/data/packer/` imports bitpacker; its imports and types may need updating to point to the new core package location.
## Impact
### Affected dependents
- `@suaveplan/crypto` — imports `defineBitSchema`; compiles green with new core
- `@suaveplan/network-plugin-protocol` — imports `defineBitSchema`; compiles green
- `@suaveplan/network-plugin-registry` — imports `defineBitSchema`; compiles green
- `@suaveplan/packer` — imports bitpacker for envelope encoding; must be verified
- `@suaveplan/network-*` — imports bitpacker for protocol encoding; must be verified
- `@suaveplan/storage-*` — imports bitpacker for payload encoding; must be verified
### Breaking changes
The genesis `BitWriter`/`BitReader`/`defineBitSchema` API surface is preserved where possible, but:
- `defineBitSchema` return type will change to match suaveplan's richer `BitSchema` interface.
- Error types change from plain `Error` to `BitpackerError extends SuaveplanError`.
- New runtime deps: `@suaveplan/schema-registry`, `@suaveplan/opentelemetry`, `@suaveplan/feature-flags`.
## Migration
For consumers of the genesis cut-down API:
- `BitWriter` / `BitReader` — unchanged signatures, but now live under subpath imports.
- `defineBitSchema` — return type gains `pack`/`unpack` methods and metadata; existing call sites compile without change.
- Error handling — catch `BitpackerError` instead of `Error` for programmatic handling.

20
openspec/archive/2026-05-02-bitpacker-modular-restructure/specs/bitpacker-delta/spec.md Normal file
View File

@@ -0,0 +1,20 @@
## ADDED Requirements
### Requirement: Delta package provides delta manager
The `@suaveplan/bitpacker-delta` package SHALL expose a `DeltaManager` that detects changes between successive values and encodes only deltas.
#### Scenario: Field-level delta encoding
- **WHEN** two successive objects differ in only one field
- **THEN** only the changed field is encoded, plus a field-change bitmask
#### Scenario: XOR delta for numbers
- **WHEN** two successive numeric values are packed with XOR delta
- **THEN** the packed size is smaller than full encoding when values are close
## MODIFIED Requirements
None.
## REMOVED Requirements
None.

30
openspec/archive/2026-05-02-bitpacker-modular-restructure/specs/bitpacker-schema/spec.md Normal file
View File

@@ -0,0 +1,30 @@
## ADDED Requirements
### Requirement: Schema package provides schema registry
The `@suaveplan/bitpacker-schema` package SHALL expose a `SchemaRegistry` that stores named schemas and resolves `_ref` tokens.
#### Scenario: Register and retrieve named schema
- **WHEN** `registry.register("player", { name: "-16", health: 255 })` is called
- **THEN** `registry.resolve("player")` returns the registered schema
### Requirement: Schema package provides validation
The `@suaveplan/bitpacker-schema` package SHALL validate that data conforms to a schema before packing.
#### Scenario: Validate data against schema
- **WHEN** `validate(data, schema)` is called with data that exceeds a numeric range
- **THEN** a `BitpackerError` is thrown with code `VALUE_OUT_OF_RANGE`
### Requirement: Schema package supports versioning
The `@suaveplan/bitpacker-schema` package SHALL support schema versioning with forward-compatibility rules.
#### Scenario: Versioned schema migration
- **WHEN** a schema contains `{ _version: 2, _schemas: [oldSchema, newSchema] }`
- **THEN** the correct schema is selected based on the version number
## MODIFIED Requirements
None.
## REMOVED Requirements
None.

31
openspec/archive/2026-05-02-bitpacker-modular-restructure/specs/bitpacker-strategies/spec.md Normal file
View File

@@ -0,0 +1,31 @@
## ADDED Requirements
### Requirement: Strategies package provides all built-in packing strategies
The `@suaveplan/bitpacker-strategies` package SHALL provide `PackingStrategy` implementations for all schema tokens: number, string, boolean, null, bigint, object, array, enum, optional, oneOf, conditional, version, ref, datetime, varint, map, set, bitarray, blob, bitset.
#### Scenario: String strategy handles fixed and variable length
- **WHEN** a schema is `"8"` (fixed) or `"-8"` (variable)
- **THEN** the strategy packs and unpacks strings correctly for both formats
#### Scenario: Enum strategy packs index with minimum bits
- **WHEN** a schema is `{ _enum: ["a", "b", "c"] }`
- **THEN** the packed value uses exactly 2 bits for the index
#### Scenario: Optional strategy uses presence bit
- **WHEN** a schema is `{ _optional: 255 }` and the value is `undefined`
- **THEN** only a single 0 bit is written
### Requirement: Strategies package exposes a default registration function
The `@suaveplan/bitpacker-strategies` package SHALL expose `registerDefaultStrategies(manager)` that adds all built-in strategies to a `PackingStrategyManager` in the correct order.
#### Scenario: Default strategies cover all primitive schemas
- **WHEN** `registerDefaultStrategies` is called on a manager
- **THEN** every primitive schema token has a matching strategy
## MODIFIED Requirements
None.
## REMOVED Requirements
None.

27
openspec/archive/2026-05-02-bitpacker-modular-restructure/specs/bitpacker-streaming/spec.md Normal file
View File

@@ -0,0 +1,27 @@
## ADDED Requirements
### Requirement: Streaming package provides chunked pack/unpack
The `@suaveplan/bitpacker-streaming` package SHALL expose functions that pack/unpack values in chunks suitable for streaming I/O.
#### Scenario: Pack to chunks
- **WHEN** `packToChunks(data, schema, chunkSize)` is called
- **THEN** an iterable of `Uint8Array` chunks is returned, each at most `chunkSize` bytes
#### Scenario: Unpack from chunks
- **WHEN** `unpackFromChunks(chunks, schema)` is called with an async iterable of chunks
- **THEN** the original value is reconstructed
### Requirement: Streaming package supports AbortSignal
The `@suaveplan/bitpacker-streaming` package SHALL accept an optional `AbortSignal` on all iterative APIs.
#### Scenario: Abort mid-stream
- **WHEN** an `AbortSignal` is aborted during chunk processing
- **THEN** the operation throws an abort error
## MODIFIED Requirements
None.
## REMOVED Requirements
None.

53
openspec/archive/2026-05-02-bitpacker-modular-restructure/specs/bitpacker/spec.md Normal file
View File

@@ -0,0 +1,53 @@
## ADDED Requirements
### Requirement: Core package exposes low-level bit I/O
The `@suaveplan/bitpacker` package SHALL expose `BitWriter` and `BitReader` classes for bit-level read/write operations.
#### Scenario: BitWriter packs bits sequentially
- **WHEN** a caller invokes `writeUint(42, 8)` followed by `writeBool(true)` on a `BitWriter`
- **THEN** the resulting byte array contains 42 in the first 8 bits and 1 in the 9th bit
#### Scenario: BitReader unpacks bits sequentially
- **WHEN** a caller constructs a `BitReader` from a packed byte array and invokes `readUint(8)` then `readBool()`
- **THEN** the values match the original written values
### Requirement: Core package exposes strategy manager
The `@suaveplan/bitpacker` package SHALL expose `PackingStrategyManager` that maintains an ordered chain of `PackingStrategy` instances.
#### Scenario: Strategy manager resolves schema to strategy
- **WHEN** `findStrategy(schema)` is called with a schema token
- **THEN** the first strategy whose `canHandle(schema)` returns `true` is returned
#### Scenario: Custom strategy registration
- **WHEN** `registerStrategy(strategy)` is called
- **THEN** the strategy is inserted before the catch-all `InvalidSchemaStrategy`
### Requirement: Core package exposes BitPacker orchestrator
The `@suaveplan/bitpacker` package SHALL expose a `BitPacker` class that uses a `PackingStrategyManager` to pack/unpack values against schemas.
#### Scenario: Round-trip pack and unpack
- **WHEN** `pack(data, schema)` is invoked followed by `unpack(bytes, schema)`
- **THEN** the unpacked value equals the original data
#### Scenario: Schema registry resolves references
- **WHEN** a schema contains `{ _ref: "namedSchema" }` and the name is registered via `registerSchema`
- **THEN** the referenced schema is used for pack/unpack
### Requirement: Genesis additions are preserved in core
The core package SHALL preserve `zigzagEncode`/`zigzagDecode`, `packBits`/`unpackBits`, `peek(bits)`, `seek(position)`, and `align()`.
#### Scenario: Peek does not advance bit position
- **WHEN** `peek(8)` is called on a `BitReader`
- **THEN** the returned value matches the next 8 bits and `bitPosition` is unchanged
#### Scenario: Zigzag encoding round-trips
- **WHEN** `zigzagEncode(n)` is called followed by `zigzagDecode` on the result
- **THEN** the original value is restored for both positive and negative integers
## MODIFIED Requirements
None.
## REMOVED Requirements
None.

154
openspec/archive/2026-05-02-bitpacker-modular-restructure/tasks.md Normal file
View File

@@ -0,0 +1,154 @@
## 1. Workspace Setup
- [x] 1.1 Create `packages/bitpacker/` parent directory
- [x] 1.2 Verify workspace glob `packages/*/*` picks up nested packages under `packages/bitpacker/`
- [x] 1.3 Create base `tsconfig.json` template for bitpacker plugins
## 2. Core Package — `@suaveplan/bitpacker`
- [x] 2.1 Create `packages/bitpacker/bitpacker/` directory with `package.json`, `tsconfig.lib.json`, `README.md`
- [x] 2.2 Port `BitWriter` from genesis `packages/primitives/bitpacker/src/bit-writer/`
- [x] 2.3 Port `BitReader` from genesis `packages/primitives/bitpacker/src/bit-reader/`
- [x] 2.4 Port `PackingStrategyManager` from suaveplan `src/core/strategies/manager/`
- [x] 2.5 Port `PackingContext` and `PackingStrategy` types from suaveplan `src/core/strategies/types.ts`
- [x] 2.6 Port `BitPacker` class from suaveplan `src/core/bitpacker.ts`
- [x] 2.7 Port base types from suaveplan `src/types.ts` and `src/types/base-types.ts`
- [x] 2.8 Preserve genesis additions: `zigzag.ts`, `pack-bits.ts`, `peek`/`seek`, `align()`
- [x] 2.9 Port `InvalidSchemaStrategy` as catch-all in core
- [x] 2.10 Write `src/index.ts` with explicit named re-exports
- [x] 2.11 Write core tests (round-trip for primitive schemas, strategy manager, custom strategy registration)
- [x] 2.12 Verify `bunx turbo typecheck test --filter=@suaveplan/bitpacker` passes
## 3. Strategies Plugin — `@suaveplan/bitpacker-strategies`
- [x] 3.1 Create `packages/bitpacker/strategies/` directory with package metadata
- [x] 3.2 Port `NumberPackingStrategy` (integer + fixed-point) from suaveplan
- [x] 3.3 Port `StringPackingStrategy` (fixed/variable strings, UUID, float formats) from suaveplan
- [x] 3.4 Port `BooleanPackingStrategy` from suaveplan
- [x] 3.5 Port `NullPackingStrategy` from suaveplan
- [x] 3.6 Port `BigIntPackingStrategy` from suaveplan
- [x] 3.7 Port `ObjectPackingStrategy` from suaveplan
- [x] 3.8 Port `ArrayPackingStrategy` (tuple + dynamic arrays) from suaveplan
- [x] 3.9 Port `EnumPackingStrategy` from suaveplan
- [x] 3.10 Port `OptionalPackingStrategy` from suaveplan
- [x] 3.11 Port `OneOfPackingStrategy` from suaveplan
- [x] 3.12 Port `ConditionalPackingStrategy` from suaveplan
- [x] 3.13 Port `VersionPackingStrategy` from suaveplan
- [x] 3.14 Port `DateTimePackingStrategy` from suaveplan
- [x] 3.15 Port `VarIntPackingStrategy` from suaveplan
- [x] 3.16 Port `MapPackingStrategy` from suaveplan
- [x] 3.17 Port `SetPackingStrategy` from suaveplan
- [x] 3.18 Port `BitArrayPackingStrategy` from suaveplan
- [x] 3.19 Port `BlobPackingStrategy` from suaveplan
- [x] 3.20 Port `BitsetPackingStrategy` from suaveplan
- [x] 3.21 Port `DefsPackingStrategy` (shared sub-schemas) from suaveplan
- [x] 3.22 Port `CustomTypePackingStrategy` from suaveplan
- [x] 3.23 Export `registerDefaultStrategies(manager)` function
- [x] 3.24 Write strategy tests covering all schema tokens
- [x] 3.25 Verify typecheck and tests pass
## 4. Schema Plugin — `@suaveplan/bitpacker-schema`
- [x] 4.1 Create `packages/bitpacker/bitpacker-schema/` directory with package metadata
- [x] 4.2 Port `SchemaRegistry` from suaveplan `src/schema/registry/` — _resolved by archived change `bitpacker-stub-completion` (full storage-backed `SchemaRegistry`, in-memory + localStorage storage adapters, and factory helpers all ported)._
- [x] 4.3 Port schema matching from suaveplan `src/schema/matching/` — _verified ported as `bitpacker-schema/matching/matching.ts` (165L → 173L). Earlier "deferred" note was stale._
- [x] 4.4 Port schema versioning from suaveplan `src/schema/versioning/` — _verified ported as `bitpacker-schema/versioning/versioning.ts` (435L → 365L). Earlier "deferred" note was stale._
- [x] 4.5 Port schema visualization from suaveplan `src/schema/visualization/` — _verified ported as `bitpacker-schema/visualization/visualization.ts` (893L → 847L). Earlier "deferred" note was stale._
- [x] 4.6 Port `validateData` and validation report from suaveplan `src/validation/`
- [x] 4.7 Write schema tests
- [x] 4.8 Verify typecheck and tests pass
> Also ported (not originally listed): `bitpacker-schema/migration/migration.ts` (521L → 479L). Track in parity table only.
## 5. Streaming Plugin — `@suaveplan/bitpacker-streaming`
- [x] 5.1 Create `packages/bitpacker/bitpacker-streaming/` directory with package metadata
- [x] 5.2 Port `StreamingPacker` from suaveplan `src/streaming/` — _resolved by archived change `bitpacker-stub-completion` (full `StreamingBitPacker` class with `packStream`/`unpackStream`, AbortSignal support, progress events, chunk processors)._
- [x] 5.3 Port transform streams from suaveplan `src/streaming/transform/` — _partial: 544L → 365L (~67%); core `BitPackerTransformer` and framing utilities present_
- [x] 5.4 Add `AbortSignal` support to all iterative APIs — _verified in `unpackFromChunksAsync`_
- [x] 5.5 Write streaming tests
- [x] 5.6 Verify typecheck and tests pass
## 6. Delta Plugin — `@suaveplan/bitpacker-delta`
- [x] 6.1 Create `packages/bitpacker/bitpacker-delta/` directory with package metadata
- [x] 6.2 Port `DeltaManager` from suaveplan `src/core/delta/` — _resolved by archived change `bitpacker-stub-completion` (full `DeltaManager` + `DeltaStateManager` ported, injectable `Clock`, structural equality replacing JSON.stringify)._
- [x] 6.3 Port XOR encoder from suaveplan `src/core/delta/xor-encoder.ts` — _partial: 177L → 60L (~34%)_
- [x] 6.4 Port string diff from suaveplan `src/core/delta/string-diff.ts` — _partial: 226L → 113L (~50%)_
- [x] 6.5 Port number XOR from suaveplan `src/core/delta/number-xor.ts` — _partial: 232L upstream `number-xor.ts` collapsed into a `numberXor` export inside `xor-encoder.ts`; richer surface lost_
- [x] 6.6 Write delta tests
- [x] 6.7 Verify typecheck and tests pass
## 7. Game Plugin — `@suaveplan/bitpacker-game`
- [x] 7.1 Create `packages/bitpacker/bitpacker-game/` directory with package metadata
- [x] 7.2 Port game bitpacker enhancements from suaveplan `src/game/bitpacker/` — _verified ported as `bitpacker-game/bitpacker/bitpacker.ts` (388L → 384L). Earlier "deferred" note was stale._
- [x] 7.3 Port quantization from suaveplan `src/game/quantization/`
- [x] 7.4 Port prediction from suaveplan `src/game/prediction/` — _all 5 files ported (kalman-filter, motion-math, position-predictor, prediction-algorithms, rotation-predictor + entry); 827L → 700L (~85%)_
- [x] 7.5 Port interpolation from suaveplan `src/game/interpolation/` — _partial: 745L → 447L (~60%); significant surface trim — flag for follow-up parity check_
- [x] 7.6 Port reconciliation from suaveplan `src/game/reconciliation/` — _partial: 615L → 478L (~78%)_
- [x] 7.7 Port state manager from suaveplan `src/game/state-manager/`
- [x] 7.8 Port schema factory from suaveplan `src/game/schema-factory/` — _verified ported as `bitpacker-game/schema-factory/schema-factory.ts` (50L → 50L). Earlier "deferred" note was stale._
- [x] 7.9 Write game tests
- [x] 7.10 Verify typecheck and tests pass
## 8. Network Plugin — `@suaveplan/bitpacker-network`
- [x] 8.1 Create `packages/bitpacker/bitpacker-network/` directory with package metadata
- [x] 8.2 Port network simulation from suaveplan `src/network/` — _resolved by archived change `bitpacker-stub-completion` (full `NetworkSimulator` ported into `bitpacker-network/`, injectable `Rng`+`Clock`, all module-scope randomness eliminated, deprecated shim left in `bitpacker-game`)._
- [x] 8.3 Write network tests
- [x] 8.4 Verify typecheck and tests pass
## 9. Crypto Plugin — `@suaveplan/bitpacker-crypto`
- [x] 9.1 Create `packages/bitpacker/bitpacker-crypto/` directory with package metadata
- [x] 9.2 Port checksum helpers (CRC32, Adler32) from suaveplan `src/core/checksum/` — _functional: `crc32`/`adler32`/`appendCrc32`/`verifyAndStripCrc32` landed inlined into 80L `bitpacker-crypto/index.ts`. Old separated `crc32.ts` (176L) + `adler32.ts` (121L) — Adler32 lacks `appendAdler32`/`verifyAdler32` wrappers; logged as polish gap in parity table_
- [x] 9.3 Port xxHash32 from suaveplan `src/core/checksum/xxhash32.ts` — _verified ported as `bitpacker-crypto/xxhash32.ts`. Earlier "deferred" note was stale._
- [x] 9.4 Port crypto adapters from suaveplan `src/crypto/adapters/` — _resolved 2026-05-02 by archived change `bitpacker-crypto-port` (and superseded surface preserved in `restore-crypto-keychain-packer`). Decision recorded in `packages/bitpacker/bitpacker-crypto/SUPERSEDED.md`._
- [x] 9.5 Port crypto provider and registry from suaveplan `src/crypto/provider/` and `src/crypto/registry/` — _resolved 2026-05-02 by archived change `bitpacker-crypto-port` (and superseded surface preserved in `restore-crypto-keychain-packer`). Decision recorded in `packages/bitpacker/bitpacker-crypto/SUPERSEDED.md`._
- [x] 9.6 Write crypto tests
- [x] 9.7 Verify typecheck and tests pass
## 10. Compressed Plugin — `@suaveplan/bitpacker-compressed`
- [x] 10.1 Create `packages/bitpacker/bitpacker-compressed/` directory with package metadata
- [x] 10.2 Port compression pipeline from suaveplan `src/compressed/` — _resolved by archived change `bitpacker-stub-completion` (full `CompressedBitPacker` + `CompressionRegistry` + Gzip/Deflate/None adapters ported)._
- [x] 10.3 Write compressed tests — _cover the placeholder RLE only_
- [x] 10.4 Verify typecheck and tests pass
## 11. Validation Plugin — `@suaveplan/bitpacker-validation`
- [x] 11.1 Create `packages/bitpacker/bitpacker-validation/` directory with package metadata
- [x] 11.2 Port safety checks from suaveplan `src/core/safety/`
- [x] 11.3 Port validation utilities from suaveplan `src/validation/`
- [x] 11.4 Write validation tests
- [x] 11.5 Verify typecheck and tests pass
## 12. Legacy Shim & Migration
- [x] 12.1 Move old `packages/primitives/bitpacker/` out of repo to `/home/ubuntu/web/genesis-bitpacker-old`
- [x] 12.2 New core `@suaveplan/bitpacker` replaces old package; downstream consumers compile green
- [x] 12.3 Downstream `crypto` package compiles and tests green
- [x] 12.4 Downstream `network-plugin-protocol` compiles green
- [x] 12.5 Downstream `network-plugin-registry` compiles green
## 13. Final Verification
- [x] 13.1 Typecheck passes for all 10 bitpacker packages
- [x] 13.2 All tests pass: 105 tests across 10 test files _(coverage of placeholder/stub modules — see §14)_
- [x] 13.3 All source `.ts` files have sibling `.md` documentation
- [x] 13.4 All 10 packages have `README.md`
- [x] 13.5 Downstream consumers (`crypto`, `network-plugin-protocol`, `network-plugin-registry`) compile green
## 14. Outstanding work surfaced by 2026-05-02 audit
These items were originally ticked but the audit found the underlying work was a stub or had been ported into the wrong package. They are re-opened here so a follow-up change can close them; the parity checklist in `proposal.md` is the canonical source of truth.
- [x] 14.1 Replace 35L inline `DeltaManager` in `bitpacker-delta/index.ts` with the full 269L `delta-manager.ts` + 149L `delta-state.ts` (drop `Date.now()` / `JSON.stringify`; use injectable `Clock` per CLAUDE.md determinism rule) — _resolved by archived change `bitpacker-stub-completion`_
- [x] 14.2 Move 622L `network-simulation.ts` from `bitpacker-game/src/network/` into `bitpacker-network/`, delete the 48L stub, drop module-scope `Math.random()` in favour of injectable `Rng` — _resolved by archived change `bitpacker-stub-completion`_
- [x] 14.3 Port full `CompressedBitPacker` class (663L) into `bitpacker-compressed`; replace placeholder RLE — _resolved by archived change `bitpacker-stub-completion`_
- [x] 14.4 Port full `StreamingPacker` class (406L) into `bitpacker-streaming` — _resolved by archived change `bitpacker-stub-completion`_
- [x] 14.5 Port `SchemaRegistry` storage adapters (`InMemorySchemaRegistryStorage`, `LocalStorageSchemaRegistryStorage`) and factory helpers (`createInMemorySchemaRegistry`, `createLocalStorageSchemaRegistry`) into `bitpacker-schema` — _resolved by archived change `bitpacker-stub-completion`_
- [x] 14.6 ~~Port crypto subsystem~~ **Resolved 2026-05-02** in two parallel changes: (a) **`restore-crypto-keychain-packer`** — new opinionated `@suaveplan/crypto` layer with `Packer` / `Unpacker` / `Authority` / `AuditLog` / `RevocationList` / `Keychain` / `Vault` consolidating the 8+ inline sign-then-pack / encrypt-pack / Merkle-log patterns. (b) **`bitpacker-crypto-rewrite` (slim)** — auxiliary bit-packer-side `SchemaFingerprint` + `HashAlgorithmRegistry` primitives that don't overlap (a). Upstream's 1906L provider abstraction stays unported (no consumers). Decision recorded in `packages/bitpacker/bitpacker-crypto/SUPERSEDED.md`. Predecessor proposals (`bitpacker-crypto-port` deprecate-and-route; `bitpacker-crypto-rewrite` first-draft seven-primitive design) are preserved as decision-history.
- [x] 14.7 Restore `appendAdler32`/`verifyAdler32` wrappers in `bitpacker-crypto` to mirror the CRC32 helper API — _added `appendAdler32` + `verifyAndStripAdler32` in `packages/bitpacker/bitpacker-crypto/src/crc32.ts`, re-exported from package root, covered by tests; mirrors the CRC32 wrapper API_
- [x] 14.8 Audit ⚠️-marked partials in parity checklist (`xor-encoder` 34%, `string-diff` 50%, `interpolation` 60%, `transform` 67%) — confirm each lost surface is intentional or restore — _audit confirmed all four trims are intentional (see proposal.md parity table); the trimmed surface is duplicated by `@suaveplan/diff` (string diff), `@suaveplan/bitpacker` core (xor primitives folded into `xor-encoder.ts`), and existing transform helpers; no functional regression to consumers (drive, network, packer all green)._