# Mesh Protocol — Chunk Requests, Advertisements, Responses This is the wire-protocol spec for how drives request and exchange chunks across the mesh. It encodes the three-form chunk request, the bitpacker-compressed presence bitset, the response piggyback, and the trust model. This document is the resolution of [`system-review.md`](../system-review.md) §1, §2, §10, §11. The spec belongs in `openspec/specs/drive-mesh/spec.md` once the proposal lands; for now this is the architectural reference. --- ## 1. Why bitsets A 50 MiB file at 65,536-byte chunks = 800 chunks. Two clients meeting on the mesh must: 1. Tell each other what they have (advertise). 2. Request what they need. A naïve protocol uses one envelope per chunk. With 800-chunk files and a 50-peer mesh, every advertisement round = 40,000 envelopes. A bitset is two orders of magnitude smaller. With `@suaveplan/bitpacker-compressed` (RLE + DEFLATE), a typical 800-bit presence map encodes to <100 bytes when most chunks are present or absent. The mesh protocol is therefore **bitset-first**: every advertisement and every dense request uses a bitset. Sparse selections (≤64 chunks) use an array; the cutover threshold matches the bytes-on-wire break-even. --- ## 2. The three request forms ```ts type ChunkRequest = | { kind: 'whole' } | { kind: 'specific'; chunkIds: number[] } | { kind: 'bitset'; bytes: Uint8Array }; ``` | Form | When to use | Wire size (800-chunk file) | |---|---|---| | `whole` | Initial fetch; receiver returns everything it has | 1 byte (the discriminator) | | `specific` | Sparse selection (≤64 chunks needed) | 4 bytes per chunkId × N | | `bitset` | Dense selection (>64 chunks) | typically <100 bytes after `bitpacker-compressed` | The drive-mesh layer chooses the form per-request based on the chunk-presence delta. Switching is automatic; the receiver supports all three. ### `ChunkRequestEnvelope` ```ts interface ChunkRequestEnvelope extends MessageEnvelope { type: 'cd:request'; payload: Uint8Array; // bitpacker-encoded ChunkRequestPayload } interface ChunkRequestPayload { fileId: string; // 32-byte hex request: ChunkRequest; priority: 'critical' | 'high' | 'normal' | 'low' | 'bulk'; maxChunks?: number; // cap on response payload (default 64) signal?: AbortSignal; // not on wire — local cancellation } ``` Encoded via `@suaveplan/bitpacker-network` for the envelope frame; the `bytes` of the bitset use `@suaveplan/bitpacker-compressed`. ### Empty-array semantics The user's brief mentioned "supplying the file id and an empty chunk ids array" to mean "everything." The current spec uses `{ kind: 'whole' }` instead of `{ kind: 'specific'; chunkIds: [] }` to disambiguate — empty array could mean "nothing" or "everything" depending on reader bias. `kind: 'whole'` is unambiguous; `kind: 'specific'` with `chunkIds: []` is rejected at validation as a no-op request. --- ## 3. The advertisement envelope ```ts interface ChunkAdvertiseEnvelope extends MessageEnvelope { type: 'cd:advertise'; payload: Uint8Array; // bitpacker-encoded ChunkAdvertisePayload } interface ChunkAdvertisePayload { fileId: string; presence: Uint8Array; // bitset, bit i = 1 iff peer has chunk i totalChunks: number; // Generation counter — increments on every change to this peer's presence // map for this file. Lets receivers ignore out-of-order advertisements. generation: number; timestamp: number; // injectable Clock } ``` ### Encoding - `presence` uses `@suaveplan/bitpacker-compressed`'s RLE encoder. Run-length is ideal for chunk presence because chunks tend to arrive in bursts (sequential downloads) — most bitsets have long 1-runs and long 0-runs. - `presence.length` (in bits) MUST equal `totalChunks`. Padding bits in the final byte MUST be zero. - A bit set to 1 means: "I have a verified chunk for this position." A bit set to 0 means either "I don't have it" or "I have it but haven't verified the signature yet." ### When to advertise Drives gossip advertisements: - On startup, for every file in the FileLocationIndex. - On `cd:advertise` from another peer (transitive — but only the delta since our last advert; throttled to one outgoing per file per second). - On every `cd:response` we receive (we now have new chunks; tell our neighbors). ### Generation counter `generation` lets receivers handle out-of-order advertisements: ```text if (advert.generation < lastSeen[peerId][fileId]) drop; else lastSeen[peerId][fileId] = advert.generation; ``` Wraps modulo 2^32 — wrap is rare enough that we tolerate a one-cycle anomaly. --- ## 4. The response envelope ```ts interface ChunkResponseEnvelope extends MessageEnvelope { type: 'cd:response'; payload: Uint8Array; // bitpacker-encoded ChunkResponsePayload } interface ChunkResponsePayload { fileId: string; chunks: ChunkBytes[]; // ordered by chunkId asc presence: Uint8Array; // piggyback: my full bitset for this file (free advert) generation: number; } interface ChunkBytes { chunkId: number; bytes: Uint8Array; // 164-byte chunk header + ciphertext (signed by writer) } ``` ### Why piggyback presence A response is the perfect time to update peer-index for `(peerId, fileId)`. The receiver got the chunks they wanted *and* learned the holder's full inventory in one round-trip. This eliminates a separate advertisement cycle for active partners. Cost: ~100 bytes per response. Benefit: skip ~200 bytes + RTT for a follow-up advertisement. ### Per-chunk verification Each `ChunkBytes.bytes` is the locked v0x0001 chunk format from [`encryption.md`](./encryption.md): ```text metadata(100) ‖ sig(64) ‖ ciphertext(variable) ``` The receiver verifies the Ed25519 sig against the writer's pubkey (from the registry), checks the AEAD tag, and only then commits the bytes. A bad sig drops the response and demotes the responder's score (see §6). --- ## 5. Response sizing and chunking-of-the-response A 64-chunk response at 65,536 bytes per chunk = 4 MiB. That is too large for a single WebRTC datachannel message (WebRTC SCTP fragmentation breaks down past 16 KiB cleanly). The drive-transport layer fragments the envelope; the protocol layer caps: - `maxChunks` defaults to 64; receivers MUST honor it. - Responses larger than 1 MiB are sent as multiple `cd:response` envelopes with the same correlation id. - WebRTC backpressure (`bufferedAmount > 16 MiB`) blocks further sends — see `TRANSPORT-WRTC-04`. For very large transfers, the responder may choose to return a partial response and emit `cd:advertise` for the rest, letting the requester re-pull on its own pace. --- ## 6. Trust and verification ### Probabilistic trust model A peer that consistently advertises chunks it doesn't actually have wastes the requester's time even if it never sends a bad signature. The trust model handles this: ```text For each (peerId, fileId): advertised: bitset of "has it according to advertisements" delivered: bitset of "actually delivered when asked" missing: bitset of advertised-but-failed-on-request trustScore = 1 - (popcount(missing) / popcount(advertised)) P(skip request) = max(0, 0.05 - trustScore) * 20 // demotes harshly past 5% miss ``` Bad signature: immediate `trustScore = -Infinity` (existing `MESH-SCORE-01`); we never request from this peer for this file again. Per-file scope: a peer good for one file doesn't earn blanket trust for others. Avoids reputation laundering. ### Sample verification on advertisement Optionally, the receiver can probe: pick one chunk from `presence`, request it, verify. If the bit is set but the chunk isn't deliverable, demote. Sampling rate is adaptive: 1-in-100 by default, raised to 1-in-10 if a peer enters "suspect" status. --- ## 7. Worked example — 800-chunk file, 50% present locally Alice's drive needs file `f-1729...` (800 chunks). She has chunks 0–399 locally. ```text 1. Alice constructs a "needed" bitset: 0b0...01...1 (400 zeros, 400 ones). bitpacker-compressed RLE encode: ~12 bytes. 2. Alice broadcasts cd:request: { fileId: 'f-1729...', request: { kind: 'bitset', bytes: <12 bytes> }, priority: 'normal', maxChunks: 64 } Total envelope: ~50 bytes. 3. Three peers (Bob, Carol, Dave) reply: - Bob: has 400-799. Sends 64 chunks (bytes ~4.2 MiB, fragmented), plus piggyback presence (RLE: ~10 bytes). - Carol: has 400-700. Sends 64 chunks of 400-463. - Dave: has 500-700 only. Sends 64 chunks of 500-563. 4. Alice's mesh layer races them; first verifying response wins per chunk. - Chunks 400-463: Carol wins (lower latency). - Chunks 464-499: Bob wins (Carol doesn't have). - Chunks 500-563: race between Bob/Carol/Dave. - Chunks 564-700: race between Bob/Carol. - Chunks 700-799: Bob wins (sole holder). 5. Each verified chunk is stored, presence bit flipped, gossiped via cd:advertise. 6. Alice's progress UI: t=0: ███░░░░░░░ 50% t=2s: █████░░░░░ 60% (Carol's batch verified) t=4s: ███████░░░ 80% (Bob's batch verified) t=6s: ██████████ 100% ``` Total round trips: 3 (one per responder). Total bandwidth (excluding chunk bytes): ~200 bytes of metadata. Without bitset compression, the same exchange would have generated 800 single-chunk requests. --- ## 8. Error envelopes ```ts interface ChunkErrorEnvelope extends MessageEnvelope { type: 'cd:error'; payload: { fileId: string; correlationId: string; // matches the cd:request id code: ChunkErrorCode; chunkId?: number; // present for chunk-specific errors retryAfterMs?: number; // backpressure hint }; } type ChunkErrorCode = | 'NOT_FOUND' // requested chunk genuinely absent | 'ACCESS_DENIED' // policy prevents serving (e.g. unsigned trust token) | 'BANDWIDTH_EXCEEDED' // QoS quota hit; retryAfterMs set | 'GENERATION_STALE' // requester's presence info is older than ours | 'INTERNAL'; ``` `GENERATION_STALE` is informational — the responder included its current generation in the error so the requester can update without a separate advertisement. --- ## 9. Integration with `network-plugin-content-delivery` `drive-mesh` produces and consumes these envelopes via `NetworkBus`. `network-plugin-content-delivery` is the *transport-level* plugin that routes them: | Direction | Bus topic | Producer | Consumer | |---|---|---|---| | Outbound | `cd:request` | `drive-mesh` | `network-plugin-content-delivery` (forwards to peers) | | Outbound | `cd:advertise` | `drive-mesh` | `network-plugin-content-delivery` (gossips to peers) | | Outbound | `cd:response` | `drive-mesh` (when serving) | `network-plugin-content-delivery` (forwards to requester) | | Inbound | `cd:request` | `network-plugin-content-delivery` | `drive-mesh` (decides what to serve) | | Inbound | `cd:response` | `network-plugin-content-delivery` | `drive-mesh` (verifies + stores) | | Inbound | `cd:advertise` | `network-plugin-content-delivery` | `drive-mesh` (updates peer-index) | `network-plugin-content-delivery` does not parse the bitsets or verify signatures. It is a forwarder + redundancy enforcer. All chunk-level intelligence lives in `drive-mesh` + `drive-plugin-chunks`. --- ## 10. Bitpacker package usage map | Concern | Package | |---|---| | Bitset encode/decode (presence, request) | `@suaveplan/bitpacker-compressed` | | Envelope schema validation | `@suaveplan/bitpacker-schema` | | On-wire framing | `@suaveplan/bitpacker-network` | | Streaming for large responses | `@suaveplan/bitpacker-streaming` | | Future: delta-encoded advertisements | `@suaveplan/bitpacker-delta` (when generation diffs become large) | The codecs are interoperable — a bitset encoded by `bitpacker-compressed` is decoded the same way regardless of which sub-package framed the envelope. --- ## 11. Test contract The `drive-mesh-protocol-v1` change folder will ship these conformance tests: - `tests/three-form-roundtrip.test.ts` — every `ChunkRequest` form roundtrips through encode/decode. - `tests/bitset-rle.test.ts` — fast-check property: encode(decode(x)) = x for random bitsets. - `tests/generation-monotonic.test.ts` — out-of-order advertisements drop correctly. - `tests/piggyback-presence.test.ts` — receiving a `cd:response` updates peer-index without a separate `cd:advertise`. - `tests/trust-decay.test.ts` — repeatedly-advertising-but-not-delivering peer loses trust score predictably. - `tests/cap-respected.test.ts` — `maxChunks` is honored by responders. --- ## Reading further - [`mesh-and-sharing.md`](./mesh-and-sharing.md) — the `drive-mesh` plugin proper. - [`encryption.md`](./encryption.md) — chunk format that the responses carry. - [`dehydrated-drives.md`](./dehydrated-drives.md) — how mesh protocol is bypassed when CDN fallback is in play. - [`../network/plugins-catalog.md`](../network/plugins-catalog.md) — `network-plugin-content-delivery` and `network-plugin-relay`. - [`../system-review.md`](../system-review.md) — the review this doc resolves.