first commit
This commit is contained in:
@@ -0,0 +1,113 @@
|
||||
# `ChunkDiscoveryManager`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`ChunkDiscoveryManager` orchestrates pull-side chunk discovery for the WebRTC drive transport. It accepts a `chunkId` from the caller, ranks every connected peer that advertises the chunk by their `PeerBandwidthTracker` score, dispatches a `request-chunk` wire message to the best candidate, awaits a `chunk-data` (or reassembled `chunk-data-part` stream) reply within a per-attempt timeout, verifies the bytes through the `ChunkIntegrityManager`, and falls back to the next-best peer on any failure.
|
||||
|
||||
The manager deduplicates concurrent `request(chunkId)` calls so a noisy drive layer (multiple workers, retried hot paths) never produces more than one wire request per chunk, and enforces a global concurrency cap so a fleet-wide download burst cannot saturate every peer at once. Every state transition is published through a typed `TypedEventEmitter` channel so observability and metrics packages can consume the same surface.
|
||||
|
||||
## Features
|
||||
|
||||
- Best-score peer ranking via `PeerBandwidthTracker`.
|
||||
- Multi-peer fallback on timeout / integrity failure / channel error.
|
||||
- In-flight de-duplication keyed by `chunkId`.
|
||||
- Bounded global concurrency with FIFO slot queue, both abortable and `closeAll`-cancellable.
|
||||
- Sub-chunk reassembly through `SubChunkReassembler` for chunks exceeding the 16 KiB SCTP wire limit.
|
||||
- Real-crypto integrity check (SHA-256 + optional Ed25519 signature) via `ChunkIntegrityManager`.
|
||||
- AbortSignal threaded through queue-wait, wire-send, and response-await paths.
|
||||
- Five typed events: `request:start`, `request:attempt`, `request:success`, `request:fail`, `request:exhausted`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
ChunkDiscoveryManager,
|
||||
ChunkIntegrityManager,
|
||||
DataChannelManager,
|
||||
PeerBandwidthTracker,
|
||||
PeerManager,
|
||||
WebRTCClient,
|
||||
} from "@suaveplan/drive-webrtc";
|
||||
|
||||
const client = new WebRTCClient({ peerId: "self", config });
|
||||
const bandwidth = new PeerBandwidthTracker();
|
||||
const peers = new PeerManager({ selfPeerId: "self", bandwidth });
|
||||
const channels = new DataChannelManager({ client });
|
||||
const integrity = new ChunkIntegrityManager();
|
||||
|
||||
const discovery = new ChunkDiscoveryManager({
|
||||
peers,
|
||||
bandwidth,
|
||||
dataChannels: channels,
|
||||
integrity,
|
||||
});
|
||||
|
||||
const bytes = await discovery.request("chunk-abc");
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Wire telemetry, supply an Ed25519 publisher key, and cap concurrency at four:
|
||||
|
||||
```ts
|
||||
const discovery = new ChunkDiscoveryManager({
|
||||
peers,
|
||||
bandwidth,
|
||||
dataChannels: channels,
|
||||
integrity,
|
||||
requestTimeoutMs: 8_000,
|
||||
maxConcurrentRequests: 4,
|
||||
});
|
||||
|
||||
discovery.on("request:fail", ({ chunkId, peerId, reason }) => {
|
||||
metrics.failures.inc({ peerId, reason });
|
||||
});
|
||||
|
||||
discovery.on("request:success", ({ chunkId, bytes, durationMs }) => {
|
||||
metrics.latency.observe(durationMs);
|
||||
});
|
||||
|
||||
const controller = new AbortController();
|
||||
const bytes = await discovery.request("chunk-abc", {
|
||||
signal: controller.signal,
|
||||
publicKey: publisherKey,
|
||||
});
|
||||
```
|
||||
|
||||
`closeAll(reason?)` aborts every in-flight attempt and rejects every queued slot-waiter so a clean shutdown surfaces deterministically.
|
||||
|
||||
## API Reference
|
||||
|
||||
### `new ChunkDiscoveryManager(options)`
|
||||
|
||||
- `peers: PeerManager` — peer registry.
|
||||
- `bandwidth: PeerBandwidthTracker` — score source.
|
||||
- `dataChannels: DataChannelManager` — wire transport.
|
||||
- `integrity: ChunkIntegrityManager` — verifier.
|
||||
- `defaultLabel?: string` (default `"drive-chunks"`).
|
||||
- `requestTimeoutMs?: number` (default `10_000`).
|
||||
- `maxConcurrentRequests?: number` (default `16`).
|
||||
- `clock?: { now(): number }`.
|
||||
|
||||
### `request(chunkId, options?)` → `Promise<Uint8Array>`
|
||||
|
||||
Returns verified chunk bytes from the highest-scoring peer that advertises `chunkId`. De-dup keyed by `chunkId`. Throws `DriveWebRtcError(CHUNK_NOT_AVAILABLE)` when no peer advertises or every candidate fails.
|
||||
|
||||
### `inFlight` (getter) → `number`
|
||||
|
||||
Active + queued requests.
|
||||
|
||||
### `peerManager` (getter) → `PeerManager`
|
||||
|
||||
Read-only handle for `WebRTCChunkSource.health` derivation.
|
||||
|
||||
### `closeAll(reason?)`
|
||||
|
||||
Aborts every active and queued request. Idempotent.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The slot queue is a plain FIFO array; releasing a slot wakes exactly one waiter so order is preserved. Slot acquisition itself is abortable, which means a queued request whose `AbortSignal` fires never enters the dispatch path. The integrity check trusts the locally-verifiable hash bound to the chunk identity (publisher-signed when a `publicKey` is provided); a tampered response always fails before the bytes are delivered to the caller, and the manager records `request:fail` then continues to the next candidate without surfacing the half-trusted data.
|
||||
Reference in New Issue
Block a user