suaveplan/docs/drive/mesh-protocol.md

# Mesh Protocol — Chunk Requests, Advertisements, Responses

This is the wire-protocol spec for how drives request and exchange chunks across the mesh. It encodes the three-form chunk request, the bitpacker-compressed presence bitset, the response piggyback, and the trust model.

This document is the resolution of [`system-review.md`](../system-review.md) §1, §2, §10, §11. The spec belongs in `openspec/specs/drive-mesh/spec.md` once the proposal lands; for now this is the architectural reference.

---

## 1. Why bitsets

A 50 MiB file at 65,536-byte chunks = 800 chunks. Two clients meeting on the mesh must:

1. Tell each other what they have (advertise).
2. Request what they need.

A naïve protocol uses one envelope per chunk. With 800-chunk files and a 50-peer mesh, every advertisement round = 40,000 envelopes. A bitset is two orders of magnitude smaller. With `@suaveplan/bitpacker-compressed` (RLE + DEFLATE), a typical 800-bit presence map encodes to <100 bytes when most chunks are present or absent.

The mesh protocol is therefore **bitset-first**: every advertisement and every dense request uses a bitset. Sparse selections (≤64 chunks) use an array; the cutover threshold matches the bytes-on-wire break-even.

---

## 2. The three request forms

```ts
type ChunkRequest =
  | { kind: 'whole' }
  | { kind: 'specific'; chunkIds: number[] }
  | { kind: 'bitset'; bytes: Uint8Array };
```

| Form | When to use | Wire size (800-chunk file) |
|---|---|---|
| `whole` | Initial fetch; receiver returns everything it has | 1 byte (the discriminator) |
| `specific` | Sparse selection (≤64 chunks needed) | 4 bytes per chunkId × N |
| `bitset` | Dense selection (>64 chunks) | typically <100 bytes after `bitpacker-compressed` |

The drive-mesh layer chooses the form per-request based on the chunk-presence delta. Switching is automatic; the receiver supports all three.

### `ChunkRequestEnvelope`

```ts
interface ChunkRequestEnvelope extends MessageEnvelope {
  type: 'cd:request';
  payload: Uint8Array;  // bitpacker-encoded ChunkRequestPayload
}

interface ChunkRequestPayload {
  fileId: string;            // 32-byte hex
  request: ChunkRequest;
  priority: 'critical' | 'high' | 'normal' | 'low' | 'bulk';
  maxChunks?: number;        // cap on response payload (default 64)
  signal?: AbortSignal;      // not on wire — local cancellation
}
```

Encoded via `@suaveplan/bitpacker-network` for the envelope frame; the `bytes` of the bitset use `@suaveplan/bitpacker-compressed`.

### Empty-array semantics

The user's brief mentioned "supplying the file id and an empty chunk ids array" to mean "everything." The current spec uses `{ kind: 'whole' }` instead of `{ kind: 'specific'; chunkIds: [] }` to disambiguate — empty array could mean "nothing" or "everything" depending on reader bias. `kind: 'whole'` is unambiguous; `kind: 'specific'` with `chunkIds: []` is rejected at validation as a no-op request.

---

## 3. The advertisement envelope

```ts
interface ChunkAdvertiseEnvelope extends MessageEnvelope {
  type: 'cd:advertise';
  payload: Uint8Array;  // bitpacker-encoded ChunkAdvertisePayload
}

interface ChunkAdvertisePayload {
  fileId: string;
  presence: Uint8Array;  // bitset, bit i = 1 iff peer has chunk i
  totalChunks: number;
  // Generation counter — increments on every change to this peer's presence
  // map for this file. Lets receivers ignore out-of-order advertisements.
  generation: number;
  timestamp: number;     // injectable Clock
}
```

### Encoding

- `presence` uses `@suaveplan/bitpacker-compressed`'s RLE encoder. Run-length is ideal for chunk presence because chunks tend to arrive in bursts (sequential downloads) — most bitsets have long 1-runs and long 0-runs.
- `presence.length` (in bits) MUST equal `totalChunks`. Padding bits in the final byte MUST be zero.
- A bit set to 1 means: "I have a verified chunk for this position." A bit set to 0 means either "I don't have it" or "I have it but haven't verified the signature yet."

### When to advertise

Drives gossip advertisements:

- On startup, for every file in the FileLocationIndex.
- On `cd:advertise` from another peer (transitive — but only the delta since our last advert; throttled to one outgoing per file per second).
- On every `cd:response` we receive (we now have new chunks; tell our neighbors).

### Generation counter

`generation` lets receivers handle out-of-order advertisements:

```text
if (advert.generation < lastSeen[peerId][fileId]) drop;
else lastSeen[peerId][fileId] = advert.generation;
```

Wraps modulo 2^32 — wrap is rare enough that we tolerate a one-cycle anomaly.

---

## 4. The response envelope

```ts
interface ChunkResponseEnvelope extends MessageEnvelope {
  type: 'cd:response';
  payload: Uint8Array;  // bitpacker-encoded ChunkResponsePayload
}

interface ChunkResponsePayload {
  fileId: string;
  chunks: ChunkBytes[];        // ordered by chunkId asc
  presence: Uint8Array;        // piggyback: my full bitset for this file (free advert)
  generation: number;
}

interface ChunkBytes {
  chunkId: number;
  bytes: Uint8Array;            // 164-byte chunk header + ciphertext (signed by writer)
}
```

### Why piggyback presence

A response is the perfect time to update peer-index for `(peerId, fileId)`. The receiver got the chunks they wanted *and* learned the holder's full inventory in one round-trip. This eliminates a separate advertisement cycle for active partners. Cost: ~100 bytes per response. Benefit: skip ~200 bytes + RTT for a follow-up advertisement.

### Per-chunk verification

Each `ChunkBytes.bytes` is the locked v0x0001 chunk format from [`encryption.md`](./encryption.md):

```text
metadata(100) ‖ sig(64) ‖ ciphertext(variable)
```

The receiver verifies the Ed25519 sig against the writer's pubkey (from the registry), checks the AEAD tag, and only then commits the bytes. A bad sig drops the response and demotes the responder's score (see §6).

---

## 5. Response sizing and chunking-of-the-response

A 64-chunk response at 65,536 bytes per chunk = 4 MiB. That is too large for a single WebRTC datachannel message (WebRTC SCTP fragmentation breaks down past 16 KiB cleanly). The drive-transport layer fragments the envelope; the protocol layer caps:

- `maxChunks` defaults to 64; receivers MUST honor it.
- Responses larger than 1 MiB are sent as multiple `cd:response` envelopes with the same correlation id.
- WebRTC backpressure (`bufferedAmount > 16 MiB`) blocks further sends — see `TRANSPORT-WRTC-04`.

For very large transfers, the responder may choose to return a partial response and emit `cd:advertise` for the rest, letting the requester re-pull on its own pace.

---

## 6. Trust and verification

### Probabilistic trust model

A peer that consistently advertises chunks it doesn't actually have wastes the requester's time even if it never sends a bad signature. The trust model handles this:

```text
For each (peerId, fileId):
  advertised: bitset of "has it according to advertisements"
  delivered:  bitset of "actually delivered when asked"
  missing:    bitset of advertised-but-failed-on-request

  trustScore = 1 - (popcount(missing) / popcount(advertised))

  P(skip request) = max(0, 0.05 - trustScore) * 20  // demotes harshly past 5% miss
```

Bad signature: immediate `trustScore = -Infinity` (existing `MESH-SCORE-01`); we never request from this peer for this file again.

Per-file scope: a peer good for one file doesn't earn blanket trust for others. Avoids reputation laundering.

### Sample verification on advertisement

Optionally, the receiver can probe: pick one chunk from `presence`, request it, verify. If the bit is set but the chunk isn't deliverable, demote. Sampling rate is adaptive: 1-in-100 by default, raised to 1-in-10 if a peer enters "suspect" status.

---

## 7. Worked example — 800-chunk file, 50% present locally

Alice's drive needs file `f-1729...` (800 chunks). She has chunks 0–399 locally.

```text
1. Alice constructs a "needed" bitset: 0b0...01...1 (400 zeros, 400 ones).
   bitpacker-compressed RLE encode: ~12 bytes.

2. Alice broadcasts cd:request:
   { fileId: 'f-1729...', request: { kind: 'bitset', bytes: <12 bytes> },
     priority: 'normal', maxChunks: 64 }
   Total envelope: ~50 bytes.

3. Three peers (Bob, Carol, Dave) reply:
   - Bob: has 400-799. Sends 64 chunks (bytes ~4.2 MiB, fragmented), plus
     piggyback presence (RLE: ~10 bytes).
   - Carol: has 400-700. Sends 64 chunks of 400-463.
   - Dave: has 500-700 only. Sends 64 chunks of 500-563.

4. Alice's mesh layer races them; first verifying response wins per chunk.
   - Chunks 400-463: Carol wins (lower latency).
   - Chunks 464-499: Bob wins (Carol doesn't have).
   - Chunks 500-563: race between Bob/Carol/Dave.
   - Chunks 564-700: race between Bob/Carol.
   - Chunks 700-799: Bob wins (sole holder).

5. Each verified chunk is stored, presence bit flipped, gossiped via cd:advertise.

6. Alice's progress UI:
   t=0: ███░░░░░░░ 50%
   t=2s: █████░░░░░ 60%  (Carol's batch verified)
   t=4s: ███████░░░ 80%  (Bob's batch verified)
   t=6s: ██████████ 100%
```

Total round trips: 3 (one per responder). Total bandwidth (excluding chunk bytes): ~200 bytes of metadata. Without bitset compression, the same exchange would have generated 800 single-chunk requests.

---

## 8. Error envelopes

```ts
interface ChunkErrorEnvelope extends MessageEnvelope {
  type: 'cd:error';
  payload: {
    fileId: string;
    correlationId: string;     // matches the cd:request id
    code: ChunkErrorCode;
    chunkId?: number;          // present for chunk-specific errors
    retryAfterMs?: number;     // backpressure hint
  };
}

type ChunkErrorCode =
  | 'NOT_FOUND'                // requested chunk genuinely absent
  | 'ACCESS_DENIED'            // policy prevents serving (e.g. unsigned trust token)
  | 'BANDWIDTH_EXCEEDED'       // QoS quota hit; retryAfterMs set
  | 'GENERATION_STALE'         // requester's presence info is older than ours
  | 'INTERNAL';
```

`GENERATION_STALE` is informational — the responder included its current generation in the error so the requester can update without a separate advertisement.

---

## 9. Integration with `network-plugin-content-delivery`

`drive-mesh` produces and consumes these envelopes via `NetworkBus`. `network-plugin-content-delivery` is the *transport-level* plugin that routes them:

| Direction | Bus topic | Producer | Consumer |
|---|---|---|---|
| Outbound | `cd:request` | `drive-mesh` | `network-plugin-content-delivery` (forwards to peers) |
| Outbound | `cd:advertise` | `drive-mesh` | `network-plugin-content-delivery` (gossips to peers) |
| Outbound | `cd:response` | `drive-mesh` (when serving) | `network-plugin-content-delivery` (forwards to requester) |
| Inbound | `cd:request` | `network-plugin-content-delivery` | `drive-mesh` (decides what to serve) |
| Inbound | `cd:response` | `network-plugin-content-delivery` | `drive-mesh` (verifies + stores) |
| Inbound | `cd:advertise` | `network-plugin-content-delivery` | `drive-mesh` (updates peer-index) |

`network-plugin-content-delivery` does not parse the bitsets or verify signatures. It is a forwarder + redundancy enforcer. All chunk-level intelligence lives in `drive-mesh` + `drive-plugin-chunks`.

---

## 10. Bitpacker package usage map

| Concern | Package |
|---|---|
| Bitset encode/decode (presence, request) | `@suaveplan/bitpacker-compressed` |
| Envelope schema validation | `@suaveplan/bitpacker-schema` |
| On-wire framing | `@suaveplan/bitpacker-network` |
| Streaming for large responses | `@suaveplan/bitpacker-streaming` |
| Future: delta-encoded advertisements | `@suaveplan/bitpacker-delta` (when generation diffs become large) |

The codecs are interoperable — a bitset encoded by `bitpacker-compressed` is decoded the same way regardless of which sub-package framed the envelope.

---

## 11. Test contract

The `drive-mesh-protocol-v1` change folder will ship these conformance tests:

- `tests/three-form-roundtrip.test.ts` — every `ChunkRequest` form roundtrips through encode/decode.
- `tests/bitset-rle.test.ts` — fast-check property: encode(decode(x)) = x for random bitsets.
- `tests/generation-monotonic.test.ts` — out-of-order advertisements drop correctly.
- `tests/piggyback-presence.test.ts` — receiving a `cd:response` updates peer-index without a separate `cd:advertise`.
- `tests/trust-decay.test.ts` — repeatedly-advertising-but-not-delivering peer loses trust score predictably.
- `tests/cap-respected.test.ts` — `maxChunks` is honored by responders.

---

## Reading further

- [`mesh-and-sharing.md`](./mesh-and-sharing.md) — the `drive-mesh` plugin proper.
- [`encryption.md`](./encryption.md) — chunk format that the responses carry.
- [`dehydrated-drives.md`](./dehydrated-drives.md) — how mesh protocol is bypassed when CDN fallback is in play.
- [`../network/plugins-catalog.md`](../network/plugins-catalog.md) — `network-plugin-content-delivery` and `network-plugin-relay`.
- [`../system-review.md`](../system-review.md) — the review this doc resolves.