151 lines
7.7 KiB
Markdown
151 lines
7.7 KiB
Markdown
# OpenSpec Change: drive-network-integration
|
|
|
|
- **Change ID**: drive-network-integration
|
|
- **Package**: — (spec-only cross-cutting change)
|
|
- **Status**: PROPOSED
|
|
- **Date**: 2026-05-01
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Clarify the architectural relationship between the drive ecosystem (`drive-mesh`, `drive-plugin-chunks`, `drive-transport-*`) and the network mesh ecosystem (`network-plugin-content-delivery`, `network-plugin-registry`, `network-plugin-relay`). This change does not create a new package; it updates existing proposals and the canonical spec to eliminate ambiguity about layering, duplicate peer registries, and responsibility boundaries.
|
|
|
|
---
|
|
|
|
## Why
|
|
|
|
**Current ambiguity**:
|
|
|
|
1. `drive-mesh-port` proposes a `PeerRegistry` that tracks live peers, availability bitsets, and latency EMAs.
|
|
2. `network-plugin-registry` already maintains a fixed-capacity peer registry with binary encoding.
|
|
3. `network-plugin-content-delivery` proposes P2P chunk distribution with redundancy and prefetch.
|
|
4. `drive-mesh-port` proposes parallel chunk pull, peer scoring, and upload backpressure.
|
|
|
|
These two registries and two chunk-distribution layers could diverge or duplicate effort.
|
|
|
|
**Required clarity**:
|
|
|
|
- Which layer owns peer discovery and registry state?
|
|
- Which layer owns chunk redundancy and prefetch?
|
|
- How does `drive-mesh` consume network mesh events?
|
|
|
|
---
|
|
|
|
## Layered architecture
|
|
|
|
```text
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ APPLICATION │
|
|
│ fetchFile(fileId) → DriveMesh.fetchFile() │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ DRIVE MESH LAYER @suaveplan/drive-mesh │
|
|
│ - Chunk scheduling (parallel pull, racing window) │
|
|
│ - Peer scoring (latency, success rate, sig-verify) │
|
|
│ - Upload backpressure │
|
|
│ - Availability gossip (compressed bitsets per file) │
|
|
│ Consumes: ChunkSource (drive-plugin-chunks) │
|
|
│ Consumes: DriveTransport (drive-transport-webrtc/https) │
|
|
│ Consumes: NetworkBus events from network-plugin-registry │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ NETWORK CONTENT DELIVERY @suaveplan/network-plugin- │
|
|
│ LAYER content-delivery │
|
|
│ - Mesh-wide chunk redundancy (N copies) │
|
|
│ - Gossip index of chunk holders │
|
|
│ - Prefetch engine │
|
|
│ - BLAKE3 integrity verification │
|
|
│ - Re-replication on peer departure │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ NETWORK MESH ORCHESTRATION @suaveplan/network + plugins │
|
|
│ - Peer registry (network-plugin-registry) │
|
|
│ - Topology / tiers (network-plugin-topology) │
|
|
│ - Transport manager (network-plugin-transport-manager) │
|
|
│ - Relay / federation (network-plugin-relay) │
|
|
│ - Governance / elections (network-plugin-governance) │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ STORAGE BACKENDS @suaveplan/storage-* │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Responsibility contract
|
|
|
|
| Concern | Owner | Reason |
|
|
|---|---|---|
|
|
| Peer identity & lifecycle | `network-plugin-registry` | Single source of truth for all mesh peers |
|
|
| Peer availability for **files** | `drive-mesh` | File-specific bitsets don't belong in network registry |
|
|
| Chunk **scheduling** (which peer, racing window) | `drive-mesh` | Drive knows chunk boundaries and urgency |
|
|
| Chunk **redundancy** (how many copies, re-replication) | `network-plugin-content-delivery` | Mesh-wide policy, not file-system concern |
|
|
| Chunk **integrity** | `network-plugin-content-delivery` | BLAKE3 verification is a delivery concern |
|
|
| Chunk **prefetch** | `network-plugin-content-delivery` | Predictive loading based on mesh-wide access patterns |
|
|
| Wire transport | `drive-transport-webrtc` / `drive-transport-https` | Lowest layer, shared by both drive-mesh and content-delivery |
|
|
|
|
## Integration points
|
|
|
|
### 1. Drive-mesh consumes registry events
|
|
|
|
`drive-mesh` MUST NOT maintain a separate `PeerRegistry` for peer identity. Instead, it listens to `network-plugin-registry` events on the `NetworkBus`:
|
|
|
|
```ts
|
|
// In drive-mesh startup
|
|
bus.on("peer:joined", ({ peerId, tier, latency }) => {
|
|
driveMesh.peerIndex.add(peerId, { tier, latency });
|
|
});
|
|
bus.on("peer:left", ({ peerId }) => {
|
|
driveMesh.peerIndex.remove(peerId);
|
|
});
|
|
```
|
|
|
|
`drive-mesh` maintains an **index** (not a registry) that maps `peerId` → file availability bitsets. The peer identity comes from the network layer.
|
|
|
|
### 2. Content-delivery drives redundancy; drive-mesh drives fetch scheduling
|
|
|
|
`network-plugin-content-delivery` decides **where** chunks should be stored (redundancy policy). `drive-mesh` decides **from where** to fetch a chunk given the peers that have it (scheduling policy).
|
|
|
|
```ts
|
|
// Content-delivery pushes chunks to peers
|
|
cd.redundancyManager.ensureCopies(fileId, chunkId, n = 3);
|
|
|
|
// Drive-mesh pulls chunks from peers
|
|
driveMesh.scheduler.fetchChunk(fileId, chunkId, candidates);
|
|
```
|
|
|
|
### 3. Shared transport abstraction
|
|
|
|
Both `drive-mesh` and `network-plugin-content-delivery` use `DriveTransport` (or `Transport`) implementations. They share the same WebRTC data channels but use different message types:
|
|
|
|
- `drive-mesh` uses `chunk:request` / `chunk:response`
|
|
- `network-plugin-content-delivery` uses `cd:advertise` / `cd:push`
|
|
|
|
## Proposed changes to existing proposals
|
|
|
|
### Update `drive-mesh-port/proposal.md`
|
|
|
|
- Replace `PeerRegistry` with `PeerIndex` (clarify it's an index overlay, not a registry).
|
|
- Add "Registry consumption" section documenting `NetworkBus` event subscription.
|
|
- Add dependency on `network-plugin-registry`.
|
|
|
|
### Update `network-plugin-content-delivery/proposal.md`
|
|
|
|
- Add explicit boundary: "This plugin does NOT schedule individual chunk fetches; that is `drive-mesh`."
|
|
- Add dependency on `drive-mesh` for fetch completion events.
|
|
|
|
### Update `network-suite/proposal.md`
|
|
|
|
- Add `drive-network-integration` as a child change or reference.
|
|
- Update the "Drive integration" bullet to reference the layered architecture above.
|
|
|
|
---
|
|
|
|
## Verification approach
|
|
|
|
- `tests/cross-layer/dep-graph.test.ts` — pins no import of `drive-mesh` from `network-plugin-registry`, and no import of `network-plugin-registry` from `drive-mesh`.
|
|
- `tests/cross-layer/event-contract.test.ts` — pins that `peer:joined` / `peer:left` events from `network-plugin-registry` have the shape `drive-mesh` expects.
|
|
- Update existing conformance tests in `drive-mesh-port` and `network-plugin-content-delivery` to reference this contract.
|
|
|
|
---
|
|
|
|
## Out of scope
|
|
|
|
- **New code packages** — this is a spec/clarity change only.
|
|
- **Implementation of the integration** — handled by the respective package changes (`drive-mesh-port`, `network-plugin-content-delivery`).
|