SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
9b28db724f7c99ad304963ee7d8a876190dd4167
suaveplan/openspec/archive/2026-05-04-drive-mesh-protocol-v1/proposal.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

11 KiB
Raw Blame History

OpenSpec Change: drive-mesh-protocol-v1

  • Change ID: drive-mesh-protocol-v1
  • Package: — (cross-cutting wire-protocol spec consumed by @suaveplan/drive-mesh and @suaveplan/network-plugin-content-delivery)
  • Capability: drive-mesh-protocol (new)
  • Status: PROPOSED
  • Date: 2026-05-02

Summary

Define the canonical wire protocol for chunk discovery, request, and delivery across the mesh. Three envelope types — cd:advertise, cd:request, cd:response — with three request forms (whole / specific / bitset), bitpacker-compressed presence bitsets, response-side presence piggyback, and a probabilistic trust model. The protocol is a contract, not an implementation: it lives in its own capability spec so that @suaveplan/drive-mesh (the requester side) and @suaveplan/network-plugin-content-delivery (the forwarder/redundancy side) can be developed and tested against the same source of truth.

Why

The current docs (drive-mesh-port, network-plugin-content-delivery) describe cd:request per chunk. This produces:

  1. Wire amplification. A 1,000-chunk file with 800 missing chunks generates 800 request envelopes — ~16 KiB of pure framing overhead per file before any bytes flow. With a 50-peer mesh gossiping 200 active files, the control plane saturates before the data plane gets a turn.
  2. No standardized advertisement encoding. MESH-GOSSIP-01 says "gzipped bitset" without pinning a schema, codec, or framing. The 10 bitpacker packages already shipped (@suaveplan/bitpacker, bitpacker-compressed, bitpacker-network, bitpacker-schema, bitpacker-streaming, bitpacker-strategies, bitpacker-delta, bitpacker-game, bitpacker-validation, bitpacker-crypto) cover this exactly — but without a binding contract every implementer would pick a different one.
  3. No trust model beyond bad-sig. A peer can advertise chunks it doesn't have indefinitely without consequence (until we ask, get nothing, and waste the round-trip). The existing MESH-SCORE-01 only triggers on bad signatures.
  4. No response piggyback. Every request that returns chunks is a missed opportunity to learn the holder's full inventory for that file. With piggyback, an active partner avoids a separate advertisement cycle.
  5. Empty-array semantics ambiguous. "Empty chunkIds array" could mean "request nothing" or "request everything." Current docs leave the reader guessing.

The architecture this proposal codifies is documented in docs/drive/mesh-protocol.md. This change folder is the formal OpenSpec mandate for that design.

What changes

ADDED — drive-mesh-protocol capability spec

22 ADDED requirements under specs/drive-mesh-protocol/spec.md:

  • DMP-01..05 — ChunkRequest discriminated union, three forms, encoding selection threshold (>64 chunks → bitset).
  • DMP-06..09 — ChunkRequestEnvelope priority tier, maxChunks cap, presence-bitset length invariant, generation counter monotonicity.
  • DMP-10..12 — Advertisement throttling, response-side presence piggyback, chunkId ordering.
  • DMP-13..17 — Verification (Ed25519 + AEAD + BLAKE3), trust-decay model, per-file scope, sample-verification rate.
  • DMP-18..19 — Error envelope shape and codes.
  • DMP-20..22 — Codec mandates (bitpacker-network framing, bitpacker-schema validation), empty-request rejection.

MODIFIED — references in downstream proposals

  • drive-mesh-port/proposal.md — replace inline cd:request/cd:advertise descriptions with references to this capability.
  • network-plugin-content-delivery/proposal.md — same for the forwarder/redundancy side.

REMOVED — none.

This is purely additive at the protocol layer; no existing requirements are revoked.

Source mapping

Genesis-original — no enterprise/<pkg>.md reference exists for this protocol because it is being designed for the first time on the genesis branch. Antecedents: BitTorrent's bitfield message (BEP-3), IPFS Bitswap (want-list reconciliation), and Hypercore's wire protocol (have/want).

Requirement Source / inspiration Notes
Three request forms Genesis-original; partial parallel to BEP-3 bitfield + request We add kind: 'whole' to disambiguate the BitTorrent "I have nothing yet" case.
Bitset encoding via @suaveplan/bitpacker-compressed Genesis — uses an already-shipped foundation library RLE + DEFLATE; ~12 bytes for an 800-bit map with long runs.
Generation counter Hypercore protocol; vector-clock-lite Wraps mod 2^32; one-cycle anomaly tolerated.
Response presence piggyback Genesis-original Saves a round-trip for active partners.
Probabilistic trust decay Genesis-original Bad-sig still terminal; missed-advertisement is recoverable.
Error codes Standard pattern from @suaveplan/error defineCodes("DMP", [...]).

Requirements with no source line

Requirement Justification
64-chunk threshold for bitset cutover Empirically: 4 bytes/chunkId × 64 = 256 bytes; bitpacker-compressed of 64-bit map is typically <40 bytes; cutover in the 60-70 range is correct.
1024-entry generation LRU Bounded memory while supporting 1k peer × 1k file fan-out.
5% miss rate threshold for trust demotion Allows for clock skew between advertise and request without false positives; aggressive enough to neutralize a buggy peer within ~20 requests.

Parity Checklist

No upstream suaveplan/main analogue exists. Genesis-original protocol.

Capability Upstream Migrated Notes
Bitset advertisements n/a ✅ Genesis-original; MESH-GOSSIP-01 ambiguous, this proposal pins it.
Three request forms n/a ✅ Whole / specific / bitset.
Per-file trust scoping n/a ✅ New on genesis.
Response presence piggyback n/a ✅ New on genesis.
Generation counter n/a ✅ New on genesis.
Per-(peerId, fileId) trust score partial (suaveplan had per-peer) ⬆️ Enhanced: per-file scope prevents reputation laundering.

Out of scope

  • Implementation of @suaveplan/drive-mesh itself. Handled by drive-mesh-port which depends on this protocol.
  • Implementation of @suaveplan/network-plugin-content-delivery. Handled by its own proposal which consumes this protocol.
  • DHT-style chunk discovery. Out of scope — this protocol assumes peers are already connected via the mesh; how they got there is a network-layer concern (network-plugin-registry, network-discovery-*).
  • Encryption of envelopes. Already covered by network-plugin-protocol (length-prefix → CryptoTransform → compressStream pipeline).
  • Cross-mesh forwarding. network-plugin-relay carries these envelopes across mesh boundaries; loop-prevention and TTL are its concern, not this protocol's.

Verification approach

  • tests/three-form-roundtrip.test.ts — every ChunkRequest form roundtrips through encode/decode.
  • tests/bitset-rle.test.ts — fast-check property: decode(encode(bitset)) === bitset for random bit-densities.
  • tests/generation-monotonic.test.ts — out-of-order advertisements drop correctly.
  • tests/piggyback-presence.test.ts — receiving a cd:response updates peer-index without a separate cd:advertise.
  • tests/trust-decay.test.ts — repeatedly-advertising-but-not-delivering peer loses trust score predictably; bad sig terminates.
  • tests/cap-respected.test.ts — maxChunks is honored by responders; over-cap requests rejected with DMP.MAX_CHUNKS_EXCEEDED.
  • tests/empty-specific-rejected.test.ts — kind: 'specific' with chunkIds: [] rejected as DMP.EMPTY_REQUEST.
  • tests/bitset-length-invariant.test.ts — advertisement with presence.length ≠ totalChunks rejected.
  • tests/error-envelope-shape.test.ts — every ChunkErrorCode round-trips.
  • tests/codec-mandate.test.ts — payloads not encoded via bitpacker-network framing fail validation.

These belong in a new package @suaveplan/drive-mesh-protocol (types + codecs + validators only — zero runtime side effects), which both drive-mesh and network-plugin-content-delivery will consume.

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/drive-mesh-protocol once the package exists.

  • @suaveplan/drive-mesh — [ ] tests run / [ ] docs reviewed / [ ] no action needed (reason: depends on this protocol; will be implemented after this lands)
  • @suaveplan/network-plugin-content-delivery — [ ] tests run / [ ] docs reviewed / [ ] no action needed (reason: forwards these envelopes; will reference this spec)
  • @suaveplan/network-plugin-relay — [ ] tests run / [ ] docs reviewed / [ ] no action needed (reason: carries these envelopes across meshes via CrossMeshEnvelope)
  • @suaveplan/drive-transport-webrtc — [ ] tests run / [ ] docs reviewed / [ ] no action needed (reason: low-level transport; doesn't parse envelope semantics, but framing must round-trip)
  • @suaveplan/drive-transport-https — [ ] tests run / [ ] docs reviewed / [ ] no action needed (reason: server-side fallback; only relevant insofar as the same envelope shapes are accepted via HTTP)

This is a foundation-tier change for the drive/network boundary: every package that exchanges chunks across the mesh references it. Per AGENTS.md §13 step 10, the dependent sweep is mandatory before archive.

Migration

From the previous design

Authors of drive-mesh-port and network-plugin-content-delivery MUST update their proposals to:

  1. Replace any inline description of "cd:request per chunk" with a reference to DMP-01..05.
  2. Replace any inline description of "gzipped bitset advertisement" with a reference to DMP-08..09 and the @suaveplan/bitpacker-compressed codec mandate (DMP-20).
  3. Replace any per-chunk reputation logic with the (peerId, fileId)-scoped model in DMP-14..17.

Search command:

grep -rln "cd:request\|cd:advertise\|cd:response" openspec/changes/ --include="*.md"

From MESH-GOSSIP-01 (in drive-mesh-port spec)

MESH-GOSSIP-01 becomes a thin wrapper over DMP-08..09. Recommended replacement text:

The plugin SHALL emit ChunkAdvertiseEnvelope per drive-mesh-protocol/spec.md DMP-08. Receivers SHALL handle generation ordering per DMP-09. Encoding via @suaveplan/bitpacker-compressed per DMP-20.

This keeps MESH-GOSSIP-01 as the entry point while delegating wire-format details to the protocol spec.

References

Reference in New Issue View Git Blame Copy Permalink