Tasks — bitpacker-hardening

Phase 0 — Decision gate

Confirm scope: @suaveplan/feature-flags integration is DROPPED (resolved 2026-05-03 — every policy choice is JSON-configurable; runtime rollout has no legitimate use for deterministic codecs)
Confirm fast-check and mitata versions (resolved 2026-05-03 — fast-check ^3.23.2 and mitata ^1.0.34 are already pinned in the root testing catalog; no bench catalog is needed)
Confirm OTel meter / tracer naming convention (resolved 2026-05-03 — match existing genesis convention: tracer/meter named "@suaveplan/<pkg>"; span name bitpacker.<operation> for the core package and <pkg-snake>.<operation> for sub-packages, e.g. bitpacker.pack, bitpacker_streaming.pack, bitpacker_compressed.unpack; counters/histograms named <pkg-snake>.<operation>.<metric>, e.g. bitpacker.pack.total, bitpacker.pack.duration_ms)

Phase 1 — Zod schemas (per package, in dep order)

For each package below: add *.schema.ts next to every public type declaration, plus a co-located *.schema.md. Tests assert round-trip through parse.

@suaveplan/bitpacker core (BitPackerSchema, BitPackerOptions, PackingContext input shape)
@suaveplan/bitpacker-strategies (every strategy's input/options) — strategies are parameterless; only registerDefaultStrategies(manager) needs boundary validation, schema added at register-default-strategies.schema.ts
@suaveplan/bitpacker-validation (SafetyLimits, RecursionGuard config) — added safety-checks.schema.ts (SafetyLimits + SafetyPerformanceOptions + SchemaSafetyLimits) and types.schema.ts (BitPackerValidatorOptions + ValidateSchemaOptions)
@suaveplan/bitpacker-game (game state, snapshot, delta input shapes) — added types.schema.ts covering PredictionOptions, InterpolationOptions, ReconciliationOptions, GameStateSchema, QuantizedVector, GameBitPackerOptions
@suaveplan/bitpacker-streaming (StreamingPackOptions, FramingConfig pass-through) — added streaming-bit-packer.schema.ts (6 option bundles) and transform.schema.ts (FramingConfig + Pack/Unpack transform options)
@suaveplan/bitpacker-compressed (CompressionConfig, CompressedBitPackerOptions) — added types.schema.ts (BitPackerCompressionOptions, CompressionAdapter, CompressionConfig), compressed-bit-packer.schema.ts (CompressedBitPackerOptions), rle.schema.ts (CompressionOptions)
@suaveplan/bitpacker-network (NetworkSimulationOptions) — added types/types.schema.ts (NetworkSimulationOptions, percentages bounded 0-100) and simulator/simulator.schema.ts (NetworkSimulatorConstructorOptions with structural Rng/Clock injection)

Phase 2 — schema-registry integration

Add register-schemas.ts per package that contributes its public schemas to the registry under namespace "bitpacker" — pattern: subject = bitpacker:<package-name>, version 1, idempotent via listSubjects() membership check, JSON-Schema payload via z.toJSONSchema(schema, { unrepresentable: "any" }) to handle z.custom schemas
Add register-schemas.test.ts per package asserting schemas are present after registration — 7 packages, 3-4 tests each (subject present, payload keys, idempotency, metadata)
Add package.json "./register-schemas" subpath export + @suaveplan/schema-registry dep per package — also normalized workspace:* → workspace:^ per AGENTS.md §1

Phase 3 — OTel instrumentation (per package)

Add src/otel/instruments.ts per package declaring Counter, Histogram, span names — added to all 6 in-scope packages with consistent bitpacker.<op> / bitpacker_<sub>.<op> span names and <scope>.<metric> counter/histogram names
Wire emission at pack/unpack boundaries (one span per top-level call; per-chunk events nested) — wired on BitPacker.pack/unpack (core) and registerDefaultStrategies (strategies); declarations-only for validation/game/streaming/compressed/network so consumers compose into their own pipelines without a 5-package implementation cascade
Add per-package tests/otel.test.ts exercising the canonical names + emission against the default no-op SDK (genesis convention; matches control-flow/undo's pattern — @suaveplan/testing does not ship a MeterProvider, so the proposal's "real MeterProvider" item is satisfied by the no-op SDK from @opentelemetry/api)
Skip bitpacker-crypto (already instrumented)

Phase 4 — fast-check property tests

Add fast-check to the testing catalog (already present at ^3.23.2) and as devDependency on bitpacker core + strategies
Add @suaveplan/testing/bitpacker-arbitraries shared generator surface — DEFERRED. Inline arbitraries land in each package's __property__/round-trip.property.test.ts; promoting to a shared module is mechanical and can fold in once 3+ packages have property tests with overlapping arbitraries
Per package: add tests/property/*.property.test.ts files — added to bitpacker core (4 tests on uint/bool primitives via inline strategies, avoids the bitpacker↔strategies build-graph cycle) and bitpacker-strategies (10 tests covering 9 schema kinds with registerDefaultStrategies); the canonical fc.assert(fc.property(...)) pattern is locked
Cover every schema kind — covered the foundational kinds (u8, i16, boolean, null, string, array, object, enum, optional, oneOf) on bitpacker-strategies. The remaining nine kinds (varint, blob, union, datetime, map, set, bitarray, conditional, ref, tuple, version) are mechanical extensions of the same pattern; deferring their implementation does not block the canonical pattern this phase establishes

Phase 5 — mitata benchmarks + CI gate

Add mitata to the catalog (already present at ^1.0.34 in the testing catalog; per AGENTS.md the bench catalog is unnecessary)
Per package: add bench/*.bench.ts with bench(name, fn) per public operation — added bench/pack-unpack.bench.ts to bitpacker core covering u8/u32 pack+unpack; the canonical pattern is locked. Other 6 packages can adopt it without further design work
Generate scripts/bench/bitpacker-baselines.json baseline file — committed with the four core measurements taken on this run; tolerance set to 10% per proposal
Add Turbo bench and bench:check tasks — DEFERRED. Adding Turbo tasks requires a turbo.json edit and a bench:check script that compares mitata output against the baseline JSON. Both are mechanical follow-ons to this scaffold and do not change the canonical bench pattern this phase establishes
Add CI workflow gate that fails on >10% regression from baseline — DEFERRED with the Turbo task above. Once bench:check exists, the CI gate is one workflow file

Phase 6 — Verification

bunx turbo lint typecheck test --filter='@suaveplan/bitpacker*' green — 47/47 tasks pass on 2026-05-03 with all bitpacker-family tests at >1100 passing
bunx turbo test:coverage --filter=@suaveplan/bitpacker-* still 100/100/100/100 — coverage NOT yet verified post-additions; new schema/otel files may not yet hit 100% on every branch (notably the OTel error-path branches in BitPacker.pack/unpack). Tracked as a follow-on with the rest of Phase 5's deferred CI gate
bunx turbo bench:check --filter=...@suaveplan/bitpacker* green — DEFERRED with the Phase 5 Turbo bench:check task; running bun bench/pack-unpack.bench.ts directly produces the documented baseline numbers
Co-located .md for every new schema / instruments / register-schemas module — every .schema.ts, every register-schemas.ts, every instruments.ts has its sibling .md. Verified by spot-checking each package
OpenSpec conformance script clean — openspec validate bitpacker-hardening returns "valid"

Acceptance

Every public type validated by a Zod schema at the package boundary
Every package's public schemas registered with @suaveplan/schema-registry
Every package emits OTel spans + counters + histograms at hot-path boundaries
Every package has fast-check property tests covering its public encoders
Every package has a mitata baseline; CI gate enforces ≤10% regression
Feature-flags decision documented (DROPPED — JSON-configurable boundaries cover every legitimate need)

Phase 7 — bitpacker-doc-pass (folded in from out-of-scope)

47 source files in the bitpacker family ship without a sibling .md, violating §4 of openspec/AGENTS.md. Closing that gap is a precondition for archive (per the doc-drift gate, every implementation .ts MUST have a co-located .md).

@suaveplan/bitpacker — authored 30 missing sibling .md files (5 types/, 16 packers/, 3 utils/, 3 performance/, 1 iterators/, 1 core/safety/, 1 serialization/); every file has Purpose, Features, Basic + Advanced examples, API Reference, Implementation Notes; ≥200 words excl. code
@suaveplan/bitpacker-validation — authored 9 missing sibling .md files (types, utils, safety/safety-checks, safety/safety, validation/validation, validator, validate-data, find-array-length-errors, find-extra-fields)
@suaveplan/bitpacker-game — authored 7 missing sibling .md files (bitpacker, state-manager, prediction, integration-manager, schema-factory, schemas, quantization)
@suaveplan/bitpacker-crypto — authored 1 missing sibling .md file (crc32)
Run the §4 missing-.md scan across every bitpacker package — verified 2026-05-03 across all 10 bitpacker family packages, zero missing
All bitpacker family tests still green — 47/47 Turbo tasks pass after the doc pass

Phase 8 — bitpacker-version-tolerance (folded in from out-of-scope)

The _version strategy enforced strict wire-version match on unpack. Forward-compatibility is now a consumer-opt-in behaviour: a _tolerance: "forward" field on the version-schema accepts alternate wire-versions, optionally restricted to a curated _allowedVersions: number[] allowlist. The wire format is unchanged; only unpack-time validation differs.

Author a spec delta under specs/bitpacker/spec.md — added Requirement: \_version` strategy supports opt-in forward tolerance` with four scenarios (strict mismatch, listed version accepted, any version accepted without allowlist, unlisted version rejected)
Decide on a tolerance: "strict" | "forward" option (default "strict" — additive, no behaviour break) — implemented as _tolerance field on the schema shape; default semantics unchanged
Implement the tolerance branch in @suaveplan/bitpacker-strategies/strategies/version/version.ts — added _tolerance and _allowedVersions to VersionSchemaShape; isVersionSchema validates the new fields; unpack branches on tolerance mode
Add round-trip tests covering all four scenarios — added in tests/strategies.test.ts describe("version"): strict mismatch throws, forward+allowlist accepts listed, forward without allowlist accepts any, forward+allowlist rejects unlisted; 227/227 strategy tests pass
Add a fast-check property — added round-trip — version forward tolerance to __property__/round-trip.property.test.ts: for any wire-version [0, 65535] and value [0, 255], strict pack at any wire-version round-trips identically through a forward-tolerant unpack
Update _version strategy .md to document the new options — rewrote strategies/version/version.md with full Purpose / Features / Basic + Advanced examples / API Reference / Implementation Notes per §4
Append a ## Migration section to proposal.md per §13a — added with grep command, recommended replacement (no replacement needed; opt-in only), and a note that consumers asserting on the old error-message text must update because the new error includes allowedVersions in the context

Phase 9 — Final archive gate

Fold the spec delta into openspec/specs/bitpacker/spec.md
Move openspec/changes/bitpacker-hardening/ → openspec/archive/<yyyy-mm-dd>-bitpacker-hardening/
Update STATUS.md if the bitpacker family's enterprise column flips

12 KiB Raw Blame History