first commit
This commit is contained in:
158
docs/networking/drive/drive-mesh/README.md
Normal file
158
docs/networking/drive/drive-mesh/README.md
Normal file
@@ -0,0 +1,158 @@
|
||||
# @suaveplan/drive-mesh - P2P mesh orchestrator for the drive chunks plugin. Peer registry, availability gossip, parallel-pull scheduler with racing window, server fallback, and bad-sig denylist. Consumes ChunkSource (drive-plugin-chunks) and DriveTransport (drive-transport-*).
|
||||
|
||||
## Summary
|
||||
|
||||
P2P mesh orchestrator over the drive chunks plugin. Consumes
|
||||
`ChunkSource` (from `@suaveplan/drive-plugin-chunks`) and one or more
|
||||
`DriveTransport` implementations (HTTPS, WebRTC, libp2p, in-memory).
|
||||
Provides parallel chunk pull with a first-byte racing window, peer
|
||||
scoring, bad-signature denylist, availability bitset gossip, and
|
||||
server fallback.
|
||||
|
||||
The package is universal: zero browser- or React-specific code.
|
||||
It composes on the server (Node) for vault / CDN sources, in the
|
||||
browser for distributed clients, and inside test harnesses with
|
||||
in-memory transports.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
- [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
- [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
- [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
- [gossip-engine](./src/gossip/gossip-engine/gossip-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_ - [typed-event-emitter](./src/internal/typed-event-emitter/typed-event-emitter.md) — <!-- begin:auto-docs --> _(no submodules)_- [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_ - [drive-mesh](./src/mesh/drive-mesh/drive-mesh.md) — <!-- begin:auto-docs --> _(no submodules)_- [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_ - [peer-registry](./src/peer-registry/peer-registry/peer-registry.md) — <!-- begin:auto-docs --> _(no submodules)_- [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_ - [parallel-puller](./src/scheduler/parallel-puller/parallel-puller.md) — <!-- begin:auto-docs --> _(no submodules)_- [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_ - [server-fallback](./src/server-fallback/server-fallback/server-fallback.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the frozen error-code registry for `@suaveplan/drive-mesh` via `defineCodes("DRIVE_MESH", …)` from `@suaveplan/…
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { DriveMesh } from "@suaveplan/drive-mesh";
|
||||
|
||||
const mesh = new DriveMesh({
|
||||
chunks, // ChunkSource from chunksPlugin
|
||||
transports: [httpsTransport], // any DriveTransport-compatible impl
|
||||
serverPubKey, // server's ed25519 public key
|
||||
serverFallback: { url: "https://cdn.example.com/chunks", transport: "https" },
|
||||
});
|
||||
await mesh.start();
|
||||
const chunkBytes = await mesh.requestChunk(fileIdHex, 0);
|
||||
await mesh.stop();
|
||||
```
|
||||
|
||||
## Layering
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ application (vault / drive) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ @suaveplan/drive-mesh (this package) │
|
||||
│ peer registry · gossip · racing puller · server fallback │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ @suaveplan/drive-mesh-protocol (envelopes + bitsets) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ @suaveplan/drive-plugin-chunks (sig-verify, manifest) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ DriveTransport implementations (HTTPS / WebRTC / libp2p) │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
`DriveMesh` is the only public class most callers construct. The four
|
||||
subsystems (`PeerRegistry`, `GossipEngine`, `ParallelPuller`,
|
||||
`ServerFallback`) are exported under subpaths so advanced consumers
|
||||
can compose them directly.
|
||||
|
||||
## Subpath table
|
||||
|
||||
| Subpath | What |
|
||||
| ---------------------------------------- | ----------------------------------- |
|
||||
| `@suaveplan/drive-mesh` | Root barrel — `DriveMesh` + types. |
|
||||
| `@suaveplan/drive-mesh/mesh/drive-mesh` | `DriveMesh` class. |
|
||||
| `@suaveplan/drive-mesh/peer-registry/peer-registry` | `PeerRegistry`. |
|
||||
| `@suaveplan/drive-mesh/gossip/gossip-engine` | `GossipEngine`. |
|
||||
| `@suaveplan/drive-mesh/scheduler/parallel-puller` | `ParallelPuller`. |
|
||||
| `@suaveplan/drive-mesh/server-fallback/server-fallback` | `ServerFallback`. |
|
||||
| `@suaveplan/drive-mesh/types` | All public types. |
|
||||
| `@suaveplan/drive-mesh/schemas` | Zod schemas. |
|
||||
| `@suaveplan/drive-mesh/errors` | `DRIVE_MESH_CODES`, `DriveMeshError`. |
|
||||
|
||||
## Trust model
|
||||
|
||||
The server's ed25519 public key (`serverPubKey`) is the only root of
|
||||
trust. Every chunk returned by a peer or the fallback URL must pass
|
||||
`verifyChunkSig` against this key before drive-mesh resolves its
|
||||
`requestChunk` promise. A peer that ever returns a chunk with an
|
||||
invalid signature is moved to a denylist — `denylistMode: "session"`
|
||||
(default) clears the list on `mesh.stop()`; `"persistent"` survives
|
||||
stops.
|
||||
|
||||
Mesh never holds file decryption keys. Plaintext only ever materialises
|
||||
inside the chunks plugin's decrypt pipeline, which is keyed by the
|
||||
recipient's per-vault key.
|
||||
|
||||
## Scoring formula
|
||||
|
||||
`PeerRegistry` ranks peers per `(fileId, chunkId)` using a weighted
|
||||
sum:
|
||||
|
||||
```text
|
||||
score = 2 · (100 / (rttMs + 1))
|
||||
+ 3 · successRate
|
||||
− 5 · errorRate
|
||||
```
|
||||
|
||||
- `rttMs` is an EMA (α = 0.3) of observed chunk-fetch wall time.
|
||||
- `successRate` and `errorRate` are over the lifetime of the registry
|
||||
entry. Peers with no observations start at `successRate = 0.5`
|
||||
(neutral) and `errorRate = 0`.
|
||||
- Bad-sig peers are pinned at `-Infinity` so they never appear in
|
||||
`rank()` output once denied.
|
||||
|
||||
## OpenTelemetry metrics
|
||||
|
||||
The mesh emits the following events; consumers wire them to whichever
|
||||
OTel meter they prefer. Drive-mesh stays metric-emitter-agnostic so it
|
||||
does not pin a meter provider on universal consumers.
|
||||
|
||||
| Event | Suggested metric | Type |
|
||||
| ----------------- | -------------------------------------------------- | ----------- |
|
||||
| `chunk:served` | `drive.mesh.chunks_served_total{outcome="peer"}` | counter |
|
||||
| `chunk:fallback` | `drive.mesh.chunks_served_total{outcome="fallback"}` | counter |
|
||||
| `chunk:failed` | `drive.mesh.chunks_served_total{outcome="fail"}` | counter |
|
||||
| `chunk:served` | `drive.mesh.chunk_fetch_ms{outcome="peer"}` | histogram |
|
||||
| `chunk:fallback` | `drive.mesh.chunk_fetch_ms{outcome="fallback"}` | histogram |
|
||||
| `peer:joined` | `drive.mesh.peers_admitted_total` | counter |
|
||||
| `peer:denied` | `drive.mesh.peers_denied_total{reason}` | counter |
|
||||
| `gossip:sent` | `drive.mesh.gossip_size_bytes` | histogram |
|
||||
|
||||
## Error codes
|
||||
|
||||
| Code | When |
|
||||
| ----------------------------- | ----------------------------------------------- |
|
||||
| `DRIVE_MESH.INVALID_OPTIONS` | Construction or `fileId` shape rejected. |
|
||||
| `DRIVE_MESH.NOT_STARTED` | `requestChunk` before `start()`. |
|
||||
| `DRIVE_MESH.CHUNK_UNAVAILABLE`| No candidate peers and no fallback configured. |
|
||||
| `DRIVE_MESH.SIG_VERIFY_FAILED`| Winning peer returned a chunk with bad sig. |
|
||||
| `DRIVE_MESH.ALL_PEERS_FAILED` | Every candidate failed; no fallback configured. |
|
||||
| `DRIVE_MESH.FALLBACK_FAILED` | Peers + fallback both failed. |
|
||||
| `DRIVE_MESH.INVALID_BITSET` | `advertise` got bad `totalChunks`. |
|
||||
| `DRIVE_MESH.TRANSPORT_FAILED` | Single-attempt transport failure (internal). |
|
||||
| `DRIVE_MESH.PEER_DENIED` | Peer on denylist (manual operations). |
|
||||
| `DRIVE_MESH.ABORTED` | Caller-supplied AbortSignal aborted. |
|
||||
|
||||
## License
|
||||
|
||||
MIT © Suaveplan
|
||||
|
||||
Reference in New Issue
Block a user