Initial Commit

openspec/changes/archive/2026-05-04-network-plugin-protocol/proposal.md

@@ -0,0 +1,97 @@
+# OpenSpec Change: network-plugin-protocol
+
+- **Change ID**: network-plugin-protocol
+- **Package**: `@suaveplan/network-plugin-protocol` (new)
+- **Status**: PROPOSED
+- **Type**: new-package
+- **Date**: 2026-04-29
+
+## Why
+
+The tiered network needs a compact, versioned binary protocol for all messages between peers. The genesis `bitpacker` package provides the binary serialization, and `serialize` provides schemas. This plugin implements a 13-byte header + binary payload format that replaces the suaveplan `network-protocol` stub. It integrates with the `streams` transforms for real encryption, compression, and framing.
+
+## What changes
+
+- **ADDED**: `ProtocolPlugin` class implementing `NetworkPlugin`.
+- **ADDED**: `BinaryProtocol` — 13-byte header: `version(1) | flags(1) | messageType(1) | timestamp(8) | payloadLength(2)`.
+- **ADDED**: `MessageEncoder` / `MessageDecoder` — schema-aware encoding/decoding using `@suaveplan/bitpacker`.
+- **ADDED**: `ProtocolPipeline` — wires `lengthPrefixEncode` → `CryptoTransform` → `compressStream` for outgoing, and reverse for incoming.
+- **ADDED**: `FastPathDispatcher` — zero-copy fast path for common message types (heartbeat, ping, registry delta). Bypasses full pipeline when message type is in the fast-path allowlist and no encryption/compression is required.
+- **ADDED**: `NETWORK_PROTO_CODES` — frozen error registry.
+
+## Out of scope
+
+- Protocol negotiation / version handshake → future `network-plugin-protocol-negotiation`.
+- Custom message types beyond registry/governance/topology/failover → future plugins register their own.
+
+## Source mapping
+
+This change derives from `../suaveplan/enterprise/network-protocol.md` and `../suaveplan/packages/universal/network-protocol/`.
+
+| Requirement | Source lines | Notes |
+|---|---|---|
+| 13-byte binary header | `network-protocol/README.md` §Binary Protocol | verbatim |
+| Version, flags, messageType, timestamp, payloadLength | `network-protocol/README.md` §Binary Protocol | verbatim |
+| Schema-aware encoding | `network-protocol/README.md` §Message Encoder/Decoder | reworded; uses `@suaveplan/bitpacker` |
+
+### Requirements with no source line
+
+| Requirement | Justification |
+|---|---|
+| `ProtocolPipeline` using `streams` transforms | Genesis has real crypto/compression/framing; protocol should use them |
+| `ProtocolPlugin` implements `NetworkPlugin` | Required to plug into `NetworkNode` |
+
+## Verification approach
+
+- `src/protocol/protocol.spec-conformance.test.ts` — pins header format, version, flags.
+- `src/encoder/encoder.spec-conformance.test.ts` — pins schema encoding, roundtrip.
+- `src/pipeline/pipeline.spec-conformance.test.ts` — pins pipeline ordering, error handling.
+- `tests/dep-graph.test.ts` — pins direct import of `@suaveplan/bitpacker`, `@suaveplan/streams`.
+
+## Enhancement: Serialization Strategy Framework
+
+**Source**: `docs/old-network-planning/PHASE_9.md` §Serialization Strategies
+
+The protocol plugin currently assumes a single binary encoding. The old docs defined multiple serialization strategies with a factory pattern:
+
+- **JSON Serializer** — human-readable, widely compatible
+- **Binary Serializer** — compact, schema-based, optional compression
+- **MessagePack Serializer** — compact binary with extensions
+- **Protocol Buffers Serializer** — schema-based with versioning
+- **SerializerFactory** — selects serializer by content type with fallback
+
+**Added to this change**:
+
+- `SerializerFactory` — registers serializers by content type, returns default on unknown.
+- `JsonSerializer`, `BinarySerializer`, `MessagePackSerializer` — implement `ISerializer`.
+- Content-type negotiation in 13-byte header flags.
+
+## Enhancement: Message Priority Queue & Adaptive Batching
+
+**Source**: `docs/old-network-planning/PHASE_3.md` §PriorityMessageQueue, `DESCRIPTION.md` §Adaptive Batching
+
+The protocol needs explicit message queueing and batching:
+
+- `PriorityMessageQueue` — multiple internal queues (one per priority tier). FIFO within same tier. CRITICAL messages bypass queue.
+- `MessageBatcher` — groups related messages by type/recipient. Flushes on size, time, or priority trigger. Adaptive interval based on network conditions.
+- `BackpressureController` — detects overwhelmed downstream and slows production.
+
+## Enhancement: Message TTL & Loop Detection
+
+**Source**: `docs/old-network-planning/PHASE_3.md` §Message Expiration
+
+- `MessageTTL` — time-to-live in hops. Drops message when `ttl <= 0`.
+- `LoopDetector` — tracks `visitedNodes` in envelope header. Drops on cycle detection.
+- `DuplicateDetector` — deduplicates broadcast messages via `messageId` + bloom filter.
+
+## Enhancement: Route Caching
+
+**Source**: `docs/old-network-planning/PHASE_3.md` §Route Caching
+
+- `RouteCache` — caches frequently used `(source, target) → nextHop` mappings.
+- Cache invalidation on `topology:changed` bus event.
+- Adaptive cache size based on memory availability.
+
+## Migration impact
+
+None — additive only.

openspec/changes/archive/2026-05-04-network-plugin-protocol/tasks.md

@@ -0,0 +1,98 @@
+# Tasks — network-plugin-protocol
+
+> Blocked on: `network-core`.
+
+## 1. Scaffold
+
+- [x] 1.1 `packages/network/network-plugin-protocol/` with `package.json` (`@suaveplan/network-plugin-protocol@0.0.0`), `tsconfig.json`, `vitest.config.ts`, `README.md`.
+- [x] 1.2 Workspace deps: `@suaveplan/network`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/bitpacker`, `@suaveplan/streams`, `@suaveplan/timing`, `@opentelemetry/api`, `zod`.
+- [x] 1.3 `sideEffects: false`.
+
+## 2. Types
+
+- [x] 2.1 `src/types/protocol.types.ts` — `ProtocolHeader`, `ProtocolMessage`, `MessageType`.
+- [x] 2.2 `src/types/pipeline.types.ts` — `PipelineConfig`, `PipelineStage`.
+- [x] 2.3 `src/error-codes.ts` — `defineCodes("NET_PROTO", [...])`.
+
+## 3. Binary protocol
+
+- [x] 3.1 `src/protocol/binary-protocol.ts` — `BinaryProtocol`.
+- [x] 3.2 Header: `version(1) | flags(1) | messageType(1) | timestamp(8) | payloadLength(2)` = 13 bytes.
+- [x] 3.3 Version negotiation placeholder (v1 only for now).
+
+## 4. Message encoder/decoder
+
+- [x] 4.1 `src/encoder/message-encoder.ts` — schema-aware encoding with `@suaveplan/bitpacker`.
+- [x] 4.2 `src/encoder/message-decoder.ts` — reverse decoding.
+- [x] 4.3 Roundtrip test for all `MessageType` variants.
+
+## 5. Protocol pipeline
+
+- [x] 5.1 `src/pipeline/protocol-pipeline.ts` — `ProtocolPipeline`.
+- [x] 5.2 Outgoing: `MessageEncoder` → `lengthPrefixEncode` → `CryptoTransform` → `compressStream`.
+- [x] 5.3 Incoming: reverse order with `decompressStream`, `CryptoTransform`, `lengthPrefixDecode`, `MessageDecoder`.
+- [x] 5.4 Error handling: `StreamPipeFailedError` on any stage failure.
+
+## 6. Plugin wrapper
+
+- [x] 6.1 `src/plugin/protocol-plugin.ts` — `ProtocolPlugin` implements `NetworkPlugin`.
+- [x] 6.2 Installs protocol, encoder, decoder, pipeline on `install()`.
+
+## 7. Spec
+
+- [x] 7.1 Update `openspec/specs/network/spec.md` with `NET-16`, `NET-17`.
+
+## 8. Conformance tests
+
+- [x] 8.1 `src/protocol/protocol.spec-conformance.test.ts` — header format, version, flags.
+- [x] 8.2 `src/encoder/encoder.spec-conformance.test.ts` — schema encoding, roundtrip.
+- [x] 8.3 `src/pipeline/pipeline.spec-conformance.test.ts` — pipeline ordering, error handling.
+- [x] 8.4 `src/error-codes.spec-conformance.test.ts` — frozen registry.
+
+## 9. Serialization strategy framework
+
+- [x] 9.1 `src/serialization/serializer-factory.ts` — `SerializerFactory`, registers by content type.
+- [x] 9.2 `src/serialization/json-serializer.ts` — `JsonSerializer`.
+- [x] 9.3 `src/serialization/binary-serializer.ts` — `BinarySerializer` with optional compression.
+- [x] 9.4 `src/serialization/msgpack-serializer.ts` — `MessagePackSerializer`.
+- [x] 9.5 Content-type negotiation via header flags.
+
+## 10. Priority queue & adaptive batching
+
+- [x] 10.1 `src/queue/priority-message-queue.ts` — `PriorityMessageQueue`, 5 tiers, FIFO within tier.
+- [x] 10.2 `src/batcher/message-batcher.ts` — `MessageBatcher`, flush on size/time/priority.
+- [x] 10.3 `src/backpressure/backpressure-controller.ts` — `BackpressureController`, slows on overwhelm.
+- [x] 10.4 CRITICAL messages bypass queue entirely.
+
+## 11. Message TTL & loop detection
+
+- [x] 11.1 `src/ttl/message-ttl.ts` — `MessageTTL`, decrements per hop, drops at zero.
+- [x] 11.2 `src/loop/loop-detector.ts` — `LoopDetector`, checks `visitedNodes` for cycles.
+- [x] 11.3 `src/dedup/duplicate-detector.ts` — `DuplicateDetector`, bloom filter for broadcast dedup.
+
+## 12. Route caching
+
+- [x] 12.1 `src/cache/route-cache.ts` — `RouteCache`, `(source, target) → nextHop`.
+- [x] 12.2 Invalidation on `topology:changed` bus event.
+- [x] 12.3 Adaptive size based on memory availability.
+
+## 13. Spec
+
+- [x] 13.1 Update `openspec/specs/network/spec.md` with `NET-45`, `NET-57`, `NET-58`, `NET-59`.
+
+## 14. Conformance tests
+
+- [x] 14.1 `src/serialization/serialization.spec-conformance.test.ts` — factory, JSON, Binary, MessagePack.
+- [x] 14.2 `src/queue/queue.spec-conformance.test.ts` — 5 tiers, FIFO, CRITICAL bypass.
+- [x] 14.3 `src/batcher/batcher.spec-conformance.test.ts` — grouping, flush triggers, adaptive.
+- [x] 14.4 `src/backpressure/backpressure.spec-conformance.test.ts` — detection, slowdown, recovery.
+- [x] 14.5 `src/ttl/ttl.spec-conformance.test.ts` — hop decrement, drop at zero.
+- [x] 14.6 `src/loop/loop.spec-conformance.test.ts` — cycle detection, visited tracking.
+- [x] 14.7 `src/dedup/dedup.spec-conformance.test.ts` — bloom filter, false positive tolerance.
+- [x] 14.8 `src/cache/cache.spec-conformance.test.ts` — caching, invalidation, memory bounds.
+
+## 15. Verify
+
+- [x] 15.1 `bunx turbo lint typecheck test --filter=@suaveplan/network-plugin-protocol` green.
+- [x] 15.2 `bunx turbo test:coverage --filter=@suaveplan/network-plugin-protocol` 100/100/100/100.
+- [x] 15.3 `bun scripts/openspec-conformance.ts @suaveplan/network-plugin-protocol` PASS.