Initial Commit
This commit is contained in:
35
docs/network/README.md
Normal file
35
docs/network/README.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Network
|
||||
|
||||
The **network** layer is the resilient, tiered, plugin-driven mesh that lets every drive in a deployment talk to every other drive — and keeps them talking when the central server goes away.
|
||||
|
||||
The user's framing: *"a resilient tiered network system where if the server goes down, it's all in place with meshed pockets of users with their nominated hosts / those hosts on a tier nominate their tier host."*
|
||||
|
||||
That is what this layer is. The canonical capability spec is the largest spec in the monorepo: [`openspec/specs/network/spec.md`](../../openspec/specs/network/spec.md) — 125 numbered requirements (`NET-01..NET-125`) covering plugins, transports, sessions, governance, partitions, relay, content delivery, and four hosting models.
|
||||
|
||||
## Documents in this folder
|
||||
|
||||
| File | Purpose |
|
||||
| --- | --- |
|
||||
| [`architecture.md`](./architecture.md) | `NetworkNode`, `NetworkPlugin`, `NetworkBus`, `MessageEnvelope`, `Transport`, `MessageRouter`. The plugin substrate that everything else lives on. |
|
||||
| [`topology-and-failover.md`](./topology-and-failover.md) | The tier model (Tier 0 / 1 / 2), meshed pockets, host election, graceful degradation when the server disconnects, and how server reintegration reconciles. The four hosting models (`PURE_CLIENT`, `SERVER_BASED`, `SERVER_TO_CLIENT_FAILOVER`, `CLIENT_WITH_SERVER_BACKUP`). |
|
||||
| [`plugins-catalog.md`](./plugins-catalog.md) | Every network plugin and what it does — protocol, registry, governance, failover, topology, transport-manager, webrtc, content-delivery, audit-mesh, relay, assignment, partition, circuit-breaker, qos, migration, auth, server-bridge, session, simulation, monitor, middleware. |
|
||||
| [`security-and-tokens.md`](./security-and-tokens.md) | Ed25519 message signatures, server-signed mesh/session/capability tokens, peer identity challenge-response, plugin security verifier, tier-specific policies, cross-mesh trust validators. |
|
||||
| [`offline-and-reintegration.md`](./offline-and-reintegration.md) | What the mesh does without the server — token grace periods, signed log buffering, reconciliation on reconnect, log compaction. |
|
||||
| [`implementation-status.md`](./implementation-status.md) | What's shipped, pending, and the network-suite-wide cleanup needed before all 21 plugins compose cleanly. |
|
||||
|
||||
## One-paragraph summary
|
||||
|
||||
`NetworkNode` is the single entry-point: it owns a `NetworkBus` for typed pub/sub, a `MessageRouter` for envelope delivery, a transport pool, and a plugin chain. Every feature the network provides is a plugin; the core does plugin lifecycle (`install → start → stop → dispose`) and message routing, nothing else. Plugins talk to each other only via bus topics — no direct imports — so any plugin can be replaced or removed without touching the others. The result is a network that scales from a single in-process node (using `MemoryTransport`) to a worldwide tiered mesh (using `WebRTC`/`WebSocket`/`HTTPS` transports with relay, partition handling, circuit breakers, and a server bridge). Every message is signed with the sender's Ed25519 key. Every host election produces a server-attested capability token. Every cross-mesh hop is loop-detected and TTL-bounded. When the server vanishes, the mesh keeps running on signed tokens with grace-period extensions and buffers signed log entries until it returns.
|
||||
|
||||
## The bet
|
||||
|
||||
The bet is that **plugin composition** beats **monolithic protocol**. Instead of one giant network library that does everything, the genesis design has 21 small plugins each owning exactly one concern, talking only through `NetworkBus`. Add `network-plugin-relay` and you can hop across meshes; remove it and you can't. Add `network-plugin-content-delivery` and you have P2P chunks; remove it and you're a control-plane-only mesh. This is what makes the same `NetworkNode` work as a phone-to-phone WebRTC pair, a server-anchored corporate mesh, or a worldwide tiered overlay.
|
||||
|
||||
## Reading order
|
||||
|
||||
1. [`architecture.md`](./architecture.md) — the plugin substrate.
|
||||
2. [`topology-and-failover.md`](./topology-and-failover.md) — tiers, meshed pockets, server failover. **Start here for resilience questions.**
|
||||
3. [`plugins-catalog.md`](./plugins-catalog.md) — what each plugin does.
|
||||
4. [`security-and-tokens.md`](./security-and-tokens.md) — the trust model.
|
||||
5. [`offline-and-reintegration.md`](./offline-and-reintegration.md) — autonomous-mesh mode.
|
||||
6. [`../drive-storage-network.md`](../drive-storage-network.md) — the whole composition.
|
||||
199
docs/network/architecture.md
Normal file
199
docs/network/architecture.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# Network Architecture
|
||||
|
||||
Canonical capability spec: [`openspec/specs/network/spec.md`](../../openspec/specs/network/spec.md) (`NET-01..NET-125`).
|
||||
|
||||
## The five core abstractions
|
||||
|
||||
| Abstraction | Role | Spec |
|
||||
| --- | --- | --- |
|
||||
| `NetworkPlugin` | The universal extension contract every plugin implements | NET-01 |
|
||||
| `NetworkNode` | The single entry-point — owns lifecycle, bus, router, transports | NET-02 |
|
||||
| `MessageEnvelope` | Typed wire format for every message | NET-03 |
|
||||
| `Transport` | The contract every transport plugin satisfies | NET-04 |
|
||||
| `NetworkBus` | The cross-plugin event broker (no plugin imports another plugin) | NET-05 |
|
||||
|
||||
These are the only five things `network-core` ships. Everything else — even WebRTC — is a plugin.
|
||||
|
||||
## `NetworkPlugin` (NET-01)
|
||||
|
||||
```ts
|
||||
interface NetworkPlugin {
|
||||
readonly name: string;
|
||||
install(node: NetworkNode): Promise<void> | void;
|
||||
start(): Promise<void> | void;
|
||||
stop(): Promise<void> | void;
|
||||
dispose?(): Promise<void> | void;
|
||||
}
|
||||
```
|
||||
|
||||
Lifecycle is strict: `install → start → ... → stop → dispose`. The core calls them in that order; plugins must be idempotent across `start`/`stop` (you can restart without disposing). **Plugins do not import each other.** Cross-plugin communication goes through `NetworkBus`.
|
||||
|
||||
## `NetworkNode` (NET-02)
|
||||
|
||||
```ts
|
||||
class NetworkNode {
|
||||
use(plugin: NetworkPlugin): this;
|
||||
start(): Promise<void>;
|
||||
stop(): Promise<void>;
|
||||
|
||||
send(targetId: PeerId, envelope: MessageEnvelope): Promise<void>;
|
||||
broadcast(envelope: MessageEnvelope): Promise<void>;
|
||||
on<E extends NetworkEvents>(event: E, handler: Handler<E>): () => void;
|
||||
|
||||
readonly bus: NetworkBus;
|
||||
readonly router: MessageRouter;
|
||||
|
||||
bootstrap(options: BootstrapOptions): Promise<void>; // NET-113
|
||||
}
|
||||
```
|
||||
|
||||
The node owns plugin lifecycle, transport pool, and message routing. Sending is transport-agnostic — the node hands the envelope to whichever active transport can reach the target peer.
|
||||
|
||||
## `MessageEnvelope` (NET-03)
|
||||
|
||||
```ts
|
||||
interface MessageEnvelope {
|
||||
id: string; // unique
|
||||
type: string; // e.g. 'cd:request', 'election:vote'
|
||||
senderId: PeerId;
|
||||
targetId: PeerId | "*"; // "*" = broadcast
|
||||
payload: Uint8Array; // already-encoded; protocol plugin handles serialisation
|
||||
timestamp: number; // injectable Clock
|
||||
headers: Record<string, string>;
|
||||
}
|
||||
```
|
||||
|
||||
Validated via Zod before routing. The 13-byte binary header from `network-plugin-protocol` (`NET-15`) prefixes this when serialising for the wire: `version(1) | flags(1) | messageType(1) | timestamp(8) | payloadLength(2)`.
|
||||
|
||||
## `Transport` (NET-04)
|
||||
|
||||
```ts
|
||||
interface Transport {
|
||||
connect(peerId: PeerId, address: string): Promise<void>;
|
||||
disconnect(peerId: PeerId): Promise<void>;
|
||||
send(peerId: PeerId, envelope: MessageEnvelope): Promise<void>;
|
||||
onMessage(handler: (env: MessageEnvelope) => void): () => void;
|
||||
getConnectedPeers(): PeerId[];
|
||||
}
|
||||
```
|
||||
|
||||
Concrete transports: WebRTC (`network-plugin-webrtc`), WebSocket (`network-websockets`), HTTPS (planned), libp2p (`network-libp2p` — pending), Hyperswarm (`network-hyperswarm` — pending), Trystero (`network-discovery-trystero` — pending), GunDB (`network-discovery-gundb` — pending), MemoryTransport (test-only, in `network-testing`).
|
||||
|
||||
`network-plugin-transport-manager` (NET-23..NET-26) maintains multiple active transports per peer simultaneously, exchanges capability bitmaps during handshake (NET-24), and routes messages by priority across them (NET-25). CRITICAL messages may go redundantly across two transports; BULK uses backpressure-aware selection.
|
||||
|
||||
## `NetworkBus` (NET-05)
|
||||
|
||||
Typed pub/sub with FIFO delivery, bounded-queue backpressure, and interceptor support. Topics are constants from the frozen `NetworkEvents` enum (NET-55):
|
||||
|
||||
```text
|
||||
node:created, node:connected, node:disconnected, node:updated, node:removed,
|
||||
message:sent, message:received, message:routed, message:dropped,
|
||||
hierarchy:changed, hierarchy:parent-changed, hierarchy:child-added, ...,
|
||||
plugin:registered, plugin:unregistered,
|
||||
peer:joined, peer:left, peer:updated, peer:latency,
|
||||
transport:connected, transport:disconnected,
|
||||
election:vote, election:completed, election:cancelled,
|
||||
failover:triggered, failover:completed,
|
||||
topology:changed, latency:rebalance,
|
||||
cd:advertise, cd:request, cd:response,
|
||||
partition:detected, partition:healed,
|
||||
qos:congestion, monitor:alert, system:error
|
||||
```
|
||||
|
||||
Plugins subscribe; plugins publish; the bus enforces ordering and backpressure. **No plugin ever imports another plugin's code.** That is the wire that lets `network-suite` compose 21 plugins without circular deps.
|
||||
|
||||
## `MessageRouter`
|
||||
|
||||
Owns:
|
||||
|
||||
- Per-message type → handler dispatch.
|
||||
- TTL decrement (`NET-58`); drop when `ttl <= 0`.
|
||||
- Loop detection via `visitedNodes` header (`NET-58`).
|
||||
- Broadcast deduplication via `messageId` bloom filter.
|
||||
- Route caching for frequently-used `(source, target) → nextHop` mappings (`NET-59`); invalidated on `topology:changed`.
|
||||
|
||||
## How a plugin actually talks to another plugin
|
||||
|
||||
Concrete example — `drive-mesh` wants a chunk from a peer:
|
||||
|
||||
```text
|
||||
drive-mesh.fetchChunk(fileId, chunkId)
|
||||
↓
|
||||
networkNode.bus.publish('cd:request', { fileId, chunkId, requesterId })
|
||||
↓
|
||||
network-plugin-content-delivery.onCdRequest(...)
|
||||
↓
|
||||
resolves a peer holding the chunk via gossip index
|
||||
↓
|
||||
networkNode.send(holderPeerId, { type: 'cd:fetch', payload: { fileId, chunkId } })
|
||||
↓
|
||||
(transport-manager picks WebRTC, WebSocket, or HTTPS based on capability negotiation)
|
||||
↓
|
||||
holder runs network-plugin-content-delivery.onCdFetch and sends back chunk bytes
|
||||
↓
|
||||
networkNode.bus.publish('cd:response', { fileId, chunkId, bytes })
|
||||
↓
|
||||
drive-mesh.onCdResponse(...) → verify sig + AEAD via chunks plugin → return to caller
|
||||
```
|
||||
|
||||
Three plugins coordinated; none imported any other; all communication via bus topics. Replace `network-plugin-content-delivery` with a different chunk-distribution implementation and the rest of the chain doesn't notice.
|
||||
|
||||
## `BootstrapOptions` (NET-113, NET-114)
|
||||
|
||||
```ts
|
||||
interface BootstrapOptions {
|
||||
mode: 'cold' | 'warm';
|
||||
initialPeers?: PeerAddress[];
|
||||
bootstrapServers?: string[];
|
||||
connectionConcurrency?: number;
|
||||
discoveryTimeout?: number;
|
||||
prioritizeLocalNetwork?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
`NetworkNode.bootstrap(options)` brings up the node from a clean state — does discovery, opens initial connections, and (with `ConnectionPreWarmer`) pre-opens transports to known peers before they are needed.
|
||||
|
||||
## `PluginRegistry` and `PluginDiscovery` (NET-116)
|
||||
|
||||
```ts
|
||||
class PluginRegistry {
|
||||
register(plugin: NetworkPlugin): void;
|
||||
unregister(name: string): void;
|
||||
has(name: string): boolean;
|
||||
get(name: string): NetworkPlugin | undefined;
|
||||
list(): NetworkPlugin[];
|
||||
}
|
||||
|
||||
class PluginDiscovery {
|
||||
findPluginsByCapability(capability: string): NetworkPlugin[];
|
||||
}
|
||||
```
|
||||
|
||||
Runtime plugin discovery and registration — the same pattern the rest of the monorepo uses (storage factory, db factory, etc.).
|
||||
|
||||
## `TransportTester` PDK (NET-115)
|
||||
|
||||
Third-party transport plugins are validated by a conformance harness:
|
||||
|
||||
```ts
|
||||
class TransportTester {
|
||||
testBasicFunctionality(transport: Transport): Promise<TestReport>;
|
||||
testReliability(transport: Transport): Promise<TestReport>;
|
||||
testBackpressure(transport: Transport): Promise<TestReport>;
|
||||
testReconnection(transport: Transport): Promise<TestReport>;
|
||||
}
|
||||
```
|
||||
|
||||
Anyone shipping a custom transport (e.g., a corporate VPN tunnel transport) runs `TransportTester` against it before registering with `network-plugin-transport-manager`. This is what makes "swap WebRTC for $YOUR_TRANSPORT" a one-line change.
|
||||
|
||||
## Why this shape
|
||||
|
||||
The core's job is to do **as little as possible**. It owns plugin lifecycle, the bus, the router, the envelope shape, and the transport contract — and stops. Every protocol decision (binary header layout, encryption, signing, election algorithm, partition recovery) lives in a plugin where it can be replaced.
|
||||
|
||||
This is the lesson the old planning corpus was groping toward but never quite landed: a "core + plugins" model where the core is genuinely small and the plugins genuinely don't depend on each other. The `network-suite` change folder finally delivers it; the canonical spec encodes it; this is the design genesis ships.
|
||||
|
||||
## Reading further
|
||||
|
||||
- [`topology-and-failover.md`](./topology-and-failover.md) — the resilience model.
|
||||
- [`plugins-catalog.md`](./plugins-catalog.md) — what each plugin does.
|
||||
- [`security-and-tokens.md`](./security-and-tokens.md) — the trust model.
|
||||
124
docs/network/implementation-status.md
Normal file
124
docs/network/implementation-status.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# Network — Implementation Status
|
||||
|
||||
Source of truth: [`STATUS.md`](../../STATUS.md), [`openspec/specs/network/spec.md`](../../openspec/specs/network/spec.md), and the in-flight changes under `openspec/changes/network-*` and `openspec/changes/enterprise-network-*`.
|
||||
|
||||
## Shipped on `genesis` (genesis-original — newer plugin model)
|
||||
|
||||
| Package | Notes |
|
||||
| --- | --- |
|
||||
| `@suaveplan/network` | The substrate. `NetworkNode`, `NetworkPlugin`, `NetworkBus`, `MessageEnvelope`, `Transport`, `MessageRouter`, `PluginManager`, `NetworkTimestamp`, `VersionInfo`. NET-01..05, NET-12, NET-53..55. |
|
||||
| `@suaveplan/network-plugin-protocol` | 13-byte binary header, encoder/decoder, pipeline, priority queue, batcher, TTL, loop detector, three serializers (JSON/Binary/MessagePack). NET-15..17, NET-25, NET-45, NET-57..59. |
|
||||
| `@suaveplan/network-plugin-registry` | Fixed-capacity peer registry with bitpacker delta sync. NET-09, NET-13. |
|
||||
| `@suaveplan/network-transport-webrtc` | WebRTC transport + PingManager. NET-06..07. |
|
||||
| `@suaveplan/network-failover` | Heartbeat FSM, failover trigger. NET-10. |
|
||||
| `@suaveplan/network-governance` | Election, voting, host selection. NET-08. |
|
||||
| `@suaveplan/network-topology` | Tier 0/1/2 graph. NET-11. |
|
||||
| `@suaveplan/network-transport-websockets` | WebSocket transport, queue, heartbeat, adapters. |
|
||||
| `@suaveplan/network-testing` | MemoryTransport, FakeWebSocketAdapter, ConditionSimulator, TestPeer factory, EventCapture. |
|
||||
|
||||
## Pending — to be ported from `main` (older naming)
|
||||
|
||||
| Package | Description (from main `README.md`) |
|
||||
| --- | --- |
|
||||
| `@suaveplan/network-protocol` | Network protocol abstractions. **Older naming — most content already in `network-plugin-protocol`.** |
|
||||
| `@suaveplan/network-registry` | Network service/peer registry. **Older naming — most content already in `network-plugin-registry`.** |
|
||||
| `@suaveplan/network-webrtc` | WebRTC peer-to-peer networking. **Older naming — most content in `network-transport-webrtc`.** |
|
||||
| `@suaveplan/network-libp2p` | libp2p-based decentralized networking. |
|
||||
| `@suaveplan/network-libp2p-server` | libp2p server with auto-relay. |
|
||||
| `@suaveplan/network-hyperswarm` | Hyperswarm DHT networking. |
|
||||
| `@suaveplan/network-trystero-server` | Trystero serverless P2P "server" relay. |
|
||||
| `@suaveplan/network-gundb-server` | Gun.js relay peer. |
|
||||
| `@suaveplan/network-discovery-{crdt,gundb,peerjs,trystero,webtorrent,hybrid}` | Six discovery plugins. |
|
||||
| `@suaveplan/network-bootstrap` | Network bootstrap and initialisation. |
|
||||
| `@suaveplan/network-quality` | Network quality monitoring. |
|
||||
| `@suaveplan/network-observability` | Application monitoring. |
|
||||
| `@suaveplan/network-assignment` | Role/resource assignment. |
|
||||
| `@suaveplan/network-standby-transport` | Pre-emptive WebRTC standby transport. |
|
||||
|
||||
The "older naming" rows reflect a 2025-era split where each subsystem lived in its own package; the newer `network-plugin-*` naming is the genesis plugin-architecture form. Some of these will be **archived without a port** (`network-protocol`, `network-registry`, `network-webrtc`) because their content is already in the genesis equivalents. Verify before porting.
|
||||
|
||||
## Planned — new on `genesis` (the `network-suite` 21 plugins)
|
||||
|
||||
The full set of plugins from `network-suite` ([proposal](../../openspec/changes/network-suite/proposal.md)). Already-shipped subset is omitted; the remaining are:
|
||||
|
||||
| Package | Spec | Notes |
|
||||
| --- | --- | --- |
|
||||
| `@suaveplan/network-plugin-transport-manager` | NET-23..26, 46, 60, 123..124 | Multi-transport orchestration. |
|
||||
| `@suaveplan/network-plugin-content-delivery` | NET-35..36 | Mesh-wide chunk redundancy. The drive-mesh consumer side. |
|
||||
| `@suaveplan/network-plugin-audit-mesh` | NET-37..39 | Signed append-only logs. |
|
||||
| `@suaveplan/network-plugin-relay` | NET-40, 61..70 | Cross-mesh forwarding. |
|
||||
| `@suaveplan/network-plugin-assignment` | NET-71..77 | Peer→mesh assignment. |
|
||||
| `@suaveplan/network-plugin-partition` | NET-78..83 | Partition detection + merge. |
|
||||
| `@suaveplan/network-plugin-circuit-breaker` | NET-84..90 | Per-path breaker. |
|
||||
| `@suaveplan/network-plugin-qos` | NET-91..98 | QoS classes + quotas. |
|
||||
| `@suaveplan/network-plugin-migration` | NET-99..105 | 8-step peer migration. |
|
||||
| `@suaveplan/network-plugin-auth` | NET-106..112 | Mesh + capability tokens, cross-mesh trust. |
|
||||
| `@suaveplan/network-plugin-server-bridge` | NET-31..34, 56 | Server WebSocket bridge + 4 hosting models + offline mode. |
|
||||
| `@suaveplan/network-plugin-session` | NET-18..22 | Session trees, nested hierarchies, per-session election, capacity. |
|
||||
| `@suaveplan/network-plugin-crypto-messages` | NET-27..30, 47..48 | Ed25519 message signing. Plugin security verifier. Tier policies. |
|
||||
| `@suaveplan/network-plugin-middleware` | NET-41 | Priority middleware pipeline. |
|
||||
| `@suaveplan/network-plugin-state` | NET-42 | Versioned path-based state. |
|
||||
| `@suaveplan/network-plugin-monitor` | NET-44 | OTel-compatible metrics + tracing. |
|
||||
| `@suaveplan/network-plugin-simulation` | NET-43 | Deterministic virtual mesh + scenarios. |
|
||||
| `@suaveplan/network-plugin-governance` enhancements | NET-118..119 | HostSelector with HostHistory (some shipped). |
|
||||
| `@suaveplan/network-plugin-topology` enhancements | NET-49..52, 120..122 | Adaptive tier depth, partition handling, geo, battery, message pool. |
|
||||
|
||||
`network-suite/REVIEW-GAPS.md` lists 10 cross-cutting gaps that the per-plugin proposals address. The package-name collision (`enterprise-network` vs `network-core`) was resolved by renaming `enterprise-network` to `network-client`.
|
||||
|
||||
## Pending changes to consolidate (cleanup work)
|
||||
|
||||
There are a number of `enterprise-network-*` and `enterprise-browser-network-*` change folders that pre-date the `network-suite` reorganisation. Most are now subsumed:
|
||||
|
||||
| Folder | Status |
|
||||
| --- | --- |
|
||||
| `enterprise-network` | Rename to `network-client`; subset of features now in `network-suite`. |
|
||||
| `enterprise-network-bootstrap` | Subsumed by `network-core` `BootstrapOptions`. |
|
||||
| `enterprise-network-failover` | Subsumed by `network-plugin-failover` (already shipped). |
|
||||
| `enterprise-network-governance` | Subsumed by `network-plugin-governance` (already shipped). |
|
||||
| `enterprise-network-protocol` | Subsumed by `network-plugin-protocol` (already shipped). |
|
||||
| `enterprise-network-registry` | Subsumed by `network-plugin-registry` (already shipped). |
|
||||
| `enterprise-network-quality` | To become `network-plugin-monitor`. |
|
||||
| `enterprise-network-observability` | To become `network-plugin-monitor`. |
|
||||
| `enterprise-network-rename` | A no-code change to rename `enterprise-network` → `network-client`. |
|
||||
| `enterprise-network-assignment` | To become `network-plugin-assignment`. |
|
||||
| `enterprise-network-standby-transport` | To become `network-plugin-transport-manager`'s standby/preemptive feature. |
|
||||
| `enterprise-network-testing` | Already shipped as `@suaveplan/network-testing`. |
|
||||
| `enterprise-network-transport-websockets` | Already shipped as `@suaveplan/network-transport-websockets`. |
|
||||
| `enterprise-browser-network-*` | All to be consolidated under the discovery plugins or `network-transport-webrtc`. |
|
||||
|
||||
These folders should be reviewed, archived, or rewritten to point at the canonical `network-suite` proposals. They are clutter, not architecture.
|
||||
|
||||
## Critical-path order (network)
|
||||
|
||||
1. **Land the `enterprise-types-cross-cutting` archive** (shared with drive prerequisite).
|
||||
2. **Land `network-suite` REVIEW-GAPS resolutions** (no code; just spec consolidation). Includes the rename `enterprise-network` → `network-client` and archive of the duplicate `enterprise-network-*` folders.
|
||||
3. **Implement `network-plugin-crypto-messages`** — message signing is foundational for everything else.
|
||||
4. **Implement `network-plugin-transport-manager`** — multi-transport orchestration. Existing webrtc + websockets transports plug in.
|
||||
5. **Implement `network-plugin-session`** — sessions are the unit of group membership.
|
||||
6. **Implement `network-plugin-server-bridge`** — enables `SERVER_TO_CLIENT_FAILOVER` model. Brings hosting-model selection into the mesh.
|
||||
7. **Implement `network-plugin-auth`** — token issuance + cross-mesh trust.
|
||||
8. **Implement `network-plugin-relay`** — cross-mesh hops.
|
||||
9. **Implement `network-plugin-content-delivery`** — drive's bus consumer.
|
||||
10. **Implement `network-plugin-audit-mesh`** — signed logs.
|
||||
11. **Implement `network-plugin-partition` + `network-plugin-circuit-breaker` + `network-plugin-qos` + `network-plugin-migration`**. Resilience tier.
|
||||
12. **Implement `network-plugin-state`, `network-plugin-monitor`, `network-plugin-simulation`, `network-plugin-middleware`**. Operational tier.
|
||||
13. **Port the discovery plugins** in any order — they feed `network-plugin-registry`.
|
||||
|
||||
Steps 3–4 are blocking; the rest can parallelise within their tier.
|
||||
|
||||
## Test debt to inherit / avoid
|
||||
|
||||
The `docs/old-network-planning/` folder has several files documenting real test-isolation bugs in the main-branch network code: shared `MemoryTransport._instances`, singleton `TimerController`, real `setTimeout` in time-sensitive tests, voting history not cleared between tests. The genesis fix is structural:
|
||||
|
||||
- `MemoryTransport` from `@suaveplan/network-testing` is per-test (no static registry).
|
||||
- `FakeClock` from `@suaveplan/testing` is the only timer source.
|
||||
- Each `NetworkNode` is fresh per test.
|
||||
- All integration tests use `@suaveplan/testing-integration` fixtures (parallel-start, reverse-stop, idempotent teardown).
|
||||
|
||||
Don't re-import the old test patterns when porting. Use the genesis testing infrastructure.
|
||||
|
||||
## Related cross-cutting work
|
||||
|
||||
- `enterprise-streams-redesign` — `BaseTransform`, `CryptoTransform`, `BaseDuplexStream`, `NetworkStreamImpl`. Required for `network-plugin-protocol`'s pipeline (CryptoTransform/compressStream) and for `drive-plugin-chunks`'s real streaming AEAD.
|
||||
- `enterprise-streams-chunk-queue` — bounded backpressure-aware `ChunkQueue` bridging sync producers and async WHATWG consumers.
|
||||
- `enterprise-streams-metadata` — pass-through `TransformStream` extracting MIME/EXIF/audio metadata via side-channel `metadata: Promise<ExtractedMetadata>`. Drive `metadata` plugin consumes this.
|
||||
126
docs/network/offline-and-reintegration.md
Normal file
126
docs/network/offline-and-reintegration.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Offline Mode and Reintegration
|
||||
|
||||
The mesh keeps running when the central server is unreachable. This document is the reference for how that works in detail.
|
||||
|
||||
The central plugin is `network-plugin-server-bridge`, governed by NET-31..NET-34 and NET-56.
|
||||
|
||||
## Detection (NET-31)
|
||||
|
||||
Tier 0 hosts maintain a persistent WebSocket to the central server with bidirectional heartbeats. The server-bridge declares the server `dead` within 2× the heartbeat interval after the last successful heartbeat. Standard verification rules apply (re-ping through alternate paths; consensus among multiple Tier 0 hosts) to avoid declaring a dead server on a transient blip.
|
||||
|
||||
## Autonomous operation (NET-32)
|
||||
|
||||
Once the server is declared dead, the mesh transitions to autonomous mode. The transitions are:
|
||||
|
||||
| Subsystem | Online → Offline |
|
||||
| --- | --- |
|
||||
| Token grace periods | extended by configurable factor (e.g., expiresAt += 24h) |
|
||||
| New mesh / session creation | paused; new requests rejected |
|
||||
| New peer admission | requires a still-valid pre-issued `CapabilityToken`; raw onboarding paused |
|
||||
| Audit-log finalisation | local append continues; finalisation deferred |
|
||||
| Audit-log replication to server | buffered |
|
||||
| Cross-mesh trust | existing `MeshTrustToken`s remain valid; new ones cannot be issued |
|
||||
| Topology, failover, elections | unchanged — fully client-side |
|
||||
| Drive read/write | unchanged — every operation is signed by drive identity, verifiable later |
|
||||
|
||||
The bus emits `server:disconnected` so plugins can react. Most plugins do nothing — they don't depend on the server. Plugins that *do* depend on the server (server-bridge, auth, audit-mesh) flip into offline mode.
|
||||
|
||||
## What gets buffered (NET-32)
|
||||
|
||||
Three buffer streams, all in local storage (typically `OpfsStorage` in the browser, `FileStorage` on a server peer):
|
||||
|
||||
1. **Pending audit-log entries** — every signed log entry that would have been finalised by the server.
|
||||
2. **Pending token-issuance requests** — operations the user attempted that need a server-signed token.
|
||||
3. **Pending mesh state deltas** — bitpacker-encoded registry/topology deltas that the server would have authoritatively reconciled.
|
||||
|
||||
All three streams are append-only and signed by the issuing peer. Storage is bounded; on overflow, the oldest *non-CRITICAL* entries are dropped first. CRITICAL entries are never dropped — if the buffer is full of CRITICAL, the system surfaces an error and refuses to accept more until the user clears space (or the server returns).
|
||||
|
||||
## Reconnection (NET-33)
|
||||
|
||||
When the server-bridge re-establishes the WebSocket, the reconciliation pipeline runs:
|
||||
|
||||
```text
|
||||
1. Handshake
|
||||
Server-bridge sends its mesh-bridge identity and the highest seq the server has.
|
||||
Server replies with the seq it expects.
|
||||
|
||||
2. Replay buffered entries in seq order
|
||||
For each:
|
||||
a. Server verifies Ed25519 signature against the issuing peer's pubkey.
|
||||
b. Server checks for conflicts with other partitions' replays.
|
||||
c. On accept: append to authoritative log, advance seq.
|
||||
d. On conflict: invoke partition merge protocol (NET-80, NET-81).
|
||||
|
||||
3. Process pending token requests
|
||||
For each queued request:
|
||||
a. Re-validate authorization (the issuer may no longer be eligible).
|
||||
b. Issue server-signed token.
|
||||
c. Notify the requesting peer via bus.
|
||||
|
||||
4. Reconcile state
|
||||
Server broadcasts `state:reconciled { meshState, sessionStates, ... }`.
|
||||
Each plugin updates its local view to match.
|
||||
|
||||
5. Hosting transition (model-dependent)
|
||||
In SERVER_TO_CLIENT_FAILOVER, the acting Tier 0 demotes; server resumes Tier 0.
|
||||
In CLIENT_WITH_SERVER_BACKUP, the user remains Tier 0; the server returns to its
|
||||
participant role.
|
||||
|
||||
6. Bus emits `server:reconnected` and `mesh:reconciled`.
|
||||
Suspended operations (new mesh creation, etc.) resume.
|
||||
```
|
||||
|
||||
The replay is idempotent: replaying an already-finalised entry is a no-op (the server tracks `(peerId, seq)` to dedupe). This means a flaky reconnect that drops mid-replay can simply retry from the last acknowledged seq.
|
||||
|
||||
## Conflict resolution
|
||||
|
||||
Two partitions running autonomously can produce conflicting state. Examples:
|
||||
|
||||
- Both elected a different acting Tier 0.
|
||||
- Both wrote different state to the same path.
|
||||
- Both admitted a peer the other doesn't know about.
|
||||
|
||||
The merge protocol (NET-81) reconciles using the partition `QuorumResolver` strategy (NET-80) — `size`, `performance`, `age`, or `hybrid`. Whichever partition wins becomes the authoritative source; the other partition's peers rejoin (NET-82).
|
||||
|
||||
For state-level conflicts that don't involve elections, `network-plugin-state` reconciliation runs (NET-42) with one of five strategies: `latest-wins`, `remote-wins`, `local-wins`, `deep-merge`, or `custom`. The strategy is configurable per state path via `PathStrategyRegistry`.
|
||||
|
||||
## Audit-log compaction (NET-39)
|
||||
|
||||
After reintegration, the audit log can grow unbounded. Compaction folds old entries into signed checkpoints:
|
||||
|
||||
```text
|
||||
checkpoint {
|
||||
seqRange: [from, to],
|
||||
stateSnapshot: bitpacker-encoded snapshot at seq=to,
|
||||
hostSignature: Ed25519 over (seqRange ‖ stateHash),
|
||||
}
|
||||
```
|
||||
|
||||
Replicas use the latest checkpoint to skip the entire prefix when bringing up a fresh peer. The full history remains available for forensic queries; the on-the-wire replay path skips it.
|
||||
|
||||
## Hosting-model interactions
|
||||
|
||||
| Model | What happens at server-down | What happens at server-up |
|
||||
| --- | --- | --- |
|
||||
| `PURE_CLIENT` | No-op — there's no server to lose. | No-op. |
|
||||
| `SERVER_BASED` | Mesh stops. Operations queue locally; users see "offline" UI. | Replay queue drains; mesh resumes. |
|
||||
| `SERVER_TO_CLIENT_FAILOVER` | Election promotes Tier 1 → acting Tier 0. Mesh continues with full operations except new-token issuance. | Acting Tier 0 demotes; server resumes. Buffered log replays. |
|
||||
| `CLIENT_WITH_SERVER_BACKUP` | User remains Tier 0; mesh continues normally. The server is "missing" but it wasn't authoritative anyway. | Server rejoins as participant. Buffered log replays. |
|
||||
|
||||
The mesh token (NET-29) carries the model, so the server-bridge knows which transitions to run.
|
||||
|
||||
## Common pitfalls and how the design avoids them
|
||||
|
||||
| Pitfall | Avoidance |
|
||||
| --- | --- |
|
||||
| Replay stuck on a single bad entry | Server processes entries individually; rejects-but-continues. |
|
||||
| Buffer overflow loses history | CRITICAL entries are never dropped. Surface error before silent loss. |
|
||||
| Two acting Tier 0s in different partitions both finalising tokens | Tokens issued in autonomous mode have a special bit set; on merge, one partition's tokens are voided per the quorum strategy and re-issued. |
|
||||
| Clock drift causes apparent expiry mid-outage | `extendedExpiresAt` is computed from the *peer's* clock, not the server's, so lack of server time doesn't cause spurious expiry. |
|
||||
| Long outage ⇒ everyone's CapabilityToken expires | Grace-period extension is automatic; if the outage exceeds the configured maximum, the mesh degrades to read-only. |
|
||||
|
||||
## Reading further
|
||||
|
||||
- [`topology-and-failover.md`](./topology-and-failover.md) — election + Tier-0 promotion.
|
||||
- [`security-and-tokens.md`](./security-and-tokens.md) — the cryptographic chain that makes replay safe.
|
||||
- [`plugins-catalog.md`](./plugins-catalog.md) — `network-plugin-server-bridge`, `network-plugin-audit-mesh`, `network-plugin-partition`.
|
||||
96
docs/network/plugins-catalog.md
Normal file
96
docs/network/plugins-catalog.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Network Plugins — Catalogue
|
||||
|
||||
Every concrete piece of network behaviour lives in a plugin. The core is small; the plugins are many. This document is the reference for what each plugin does, which capability spec block it satisfies, and its migration status.
|
||||
|
||||
The full plugin list comes from the `network-suite` meta-change ([proposal](../../openspec/changes/network-suite/proposal.md)) which orchestrates 21 child changes. Each plugin has its own change folder under `openspec/changes/network-plugin-<name>/`.
|
||||
|
||||
## Quick map
|
||||
|
||||
| Plugin | Spec block | Status | One-line role |
|
||||
| --- | --- | --- | --- |
|
||||
| `network-core` | NET-01..05, NET-12, NET-53..55, NET-113..116 | shipped (genesis) | The substrate. `NetworkNode`, `NetworkPlugin`, `NetworkBus`, `MessageEnvelope`, `Transport`. |
|
||||
| `network-plugin-protocol` | NET-15..17, NET-25, NET-45, NET-57..59, NET-125 | shipped (genesis) | 13-byte header. Encoder/decoder. Pipeline. Priority queue. TTL. Loop detector. Fast path. |
|
||||
| `network-plugin-registry` | NET-09, NET-13, NET-71 | shipped (genesis) | Peer identity registry. Bitpacker delta sync. |
|
||||
| `network-plugin-webrtc` | NET-06..07, NET-117 | shipped (genesis) | WebRTC `Transport`. PingManager (EMA RTT). |
|
||||
| `network-plugin-failover` | NET-10 | shipped (genesis) | Heartbeat FSM. Failover trigger. |
|
||||
| `network-plugin-governance` | NET-08, NET-118..119 | shipped (genesis) | Elections. HostSelector + HostHistory. |
|
||||
| `network-plugin-topology` | NET-11, NET-14, NET-49..52, NET-120..122 | shipped (genesis) | Tier 0/1/2 management. Latency groups. Geographic clustering. Battery-aware. Partition detection. |
|
||||
| `network-websockets` | (own subpath) | shipped (genesis) | WebSocket `Transport`. Reconnect, queue, heartbeat, adapters. |
|
||||
| `network-testing` | (own subpath) | shipped (genesis) | MemoryTransport, FakeWebSocket, ConditionSimulator, TestPeer, EventCapture. |
|
||||
| `network-plugin-transport-manager` | NET-23..26, NET-46, NET-60, NET-123..124 | planned | Multi-transport orchestration. Pre-emptive switching. Battery-aware scheduler. Bandwidth conservator. |
|
||||
| `network-plugin-content-delivery` | NET-35..36 | planned | Mesh-wide chunk redundancy + integrity. The drive layer's bus consumer for `cd:request`/`cd:response`. |
|
||||
| `network-plugin-audit-mesh` | NET-37..39 | planned | Signed append-only logs. Replication across siblings. Compaction + checkpoints. |
|
||||
| `network-plugin-relay` | NET-40, NET-61..70 | planned | Cross-mesh forwarding. Backbone transport. Mesh directory. Loop prevention. |
|
||||
| `network-plugin-assignment` | NET-71..77 | planned | Peer→mesh assignment. Capacity tracking. Imbalance rebalancing. |
|
||||
| `network-plugin-partition` | NET-78..83 | planned | Partition detection (epoch-based). Quorum resolution. Merge protocol. |
|
||||
| `network-plugin-circuit-breaker` | NET-84..90 | planned | Per-path closed/open/half-open breaker. Probe + registry. |
|
||||
| `network-plugin-qos` | NET-91..98 | planned | QoS classes. Per-mesh/peer/relay bandwidth quotas. Congestion alerts. |
|
||||
| `network-plugin-migration` | NET-99..105 | planned | 8-step peer migration: request, accept, snapshot, transfer, tunnel, cutover, drain, complete. |
|
||||
| `network-plugin-auth` | NET-106..112 | planned | MeshMembershipVerifier, CapabilityTokenIssuer, CrossMeshTrustValidator. |
|
||||
| `network-plugin-server-bridge` | NET-31..34, NET-56 | planned | Tier 0 ↔ central server WebSocket. Offline mode. Log replay. Hosting-model selector. |
|
||||
| `network-plugin-session` | NET-18..22 | planned | Session trees, nested hierarchies, per-session election, capacity, transitions. |
|
||||
| `network-plugin-crypto-messages` | NET-27..30, NET-47..48 | planned | Ed25519 message signing. Mesh/session tokens. Plugin security verifier. Tier policies. |
|
||||
| `network-plugin-middleware` | NET-41 | planned | Priority middleware pipeline. Drop-on-null. Runtime add/remove/enable/disable. |
|
||||
| `network-plugin-state` | NET-42 | planned | Versioned path-based state with five reconciliation strategies. Bloom-filter divergence. |
|
||||
| `network-plugin-monitor` | NET-44 | planned | Counter/Gauge/Histogram/Summary metrics. OTel-compatible distributed tracing. |
|
||||
| `network-plugin-simulation` | NET-43 | planned | Deterministic virtual mesh. Time acceleration. Pre-defined scenarios (ServerFailure, NetworkPartition, ClientSurge, CascadingFailure). |
|
||||
|
||||
## Discovery plugins (pending — port from `main`)
|
||||
|
||||
These provide *peer discovery* — finding peers without an existing topology — and feed their results to `network-plugin-registry`:
|
||||
|
||||
| Plugin | Status | Mechanism |
|
||||
| --- | --- | --- |
|
||||
| `network-discovery-crdt` | pending — `main` | Yjs-based CRDT discovery. |
|
||||
| `network-discovery-gundb` | pending — `main` | GunDB gossip. |
|
||||
| `network-discovery-peerjs` | pending — `main` | PeerJS broker. |
|
||||
| `network-discovery-trystero` | pending — `main` | Multi-strategy (BitTorrent, Nostr, MQTT). |
|
||||
| `network-discovery-webtorrent` | pending — `main` | WebTorrent tracker. |
|
||||
| `network-discovery-hybrid` | pending — `main` | Composes several discovery plugins with deduplication. |
|
||||
|
||||
## Server-side network plugins (pending — port from `main`)
|
||||
|
||||
These run on a central server and participate in the mesh as Tier 0:
|
||||
|
||||
| Plugin | Status | Role |
|
||||
| --- | --- | --- |
|
||||
| `network-libp2p` | pending — `main` | libp2p transport. Heavy but battle-tested. |
|
||||
| `network-libp2p-server` | pending — `main` | libp2p server with auto-relay. |
|
||||
| `network-hyperswarm` | pending — `main` | Hyperswarm DHT-based discovery. |
|
||||
| `network-trystero-server` | pending — `main` | Trystero serverless P2P "server" relay. |
|
||||
| `network-gundb-server` | pending — `main` | Gun.js relay peer. |
|
||||
| `network-standby-transport` | pending — `main` | Pre-emptive WebRTC standby transport (warm spare). |
|
||||
|
||||
## Plugins by concern — what to install for what you need
|
||||
|
||||
| If you need... | Install |
|
||||
| --- | --- |
|
||||
| Just talk between two peers, no mesh | `network-core` + `network-plugin-webrtc` (or `network-websockets`) |
|
||||
| Local-area mesh with simple roles | + `network-plugin-registry`, `network-plugin-topology`, `network-plugin-failover` |
|
||||
| Resilient tiered mesh with elections | + `network-plugin-governance`, `network-plugin-session` |
|
||||
| Cross-mesh forwarding (peer-to-peer-across-different-meshes) | + `network-plugin-relay`, `network-plugin-auth` |
|
||||
| Partition tolerance | + `network-plugin-partition`, `network-plugin-circuit-breaker` |
|
||||
| Backpressure-aware transport switching | + `network-plugin-transport-manager`, `network-plugin-qos` |
|
||||
| Verified message authenticity | + `network-plugin-crypto-messages` |
|
||||
| Server failover (`SERVER_TO_CLIENT_FAILOVER`) | + `network-plugin-server-bridge` |
|
||||
| Audit trail | + `network-plugin-audit-mesh` |
|
||||
| Drive chunk distribution over the mesh | + `network-plugin-content-delivery` (drive-mesh consumes its bus events) |
|
||||
| Production observability | + `network-plugin-monitor` |
|
||||
| Deterministic mesh testing | + `network-plugin-simulation` |
|
||||
| Smooth peer migration during rebalances | + `network-plugin-migration`, `network-plugin-assignment` |
|
||||
| Pluggable middleware | + `network-plugin-middleware` |
|
||||
| State sync with conflict resolution | + `network-plugin-state` |
|
||||
|
||||
You don't have to install all 21. Most production deployments install ~10. Test deployments often install three (core + memory transport + registry).
|
||||
|
||||
## A note on the `enterprise-network` vs `network-core` collision
|
||||
|
||||
`network-suite` REVIEW-GAPS resolved a package-name collision: there were two proposals targeting `@suaveplan/network`. The resolution was to rename `enterprise-network` (which was a different angle entirely — a high-level network *client* abstraction) to `@suaveplan/network-client`, leaving `@suaveplan/network` to mean the genesis plugin core. If you encounter old references to `enterprise-network`, treat them as historical and read `network-core` instead.
|
||||
|
||||
## Reading further
|
||||
|
||||
- [`architecture.md`](./architecture.md) — the substrate every plugin extends.
|
||||
- [`topology-and-failover.md`](./topology-and-failover.md) — how these plugins compose into the resilience model.
|
||||
- [`security-and-tokens.md`](./security-and-tokens.md) — the trust model `network-plugin-crypto-messages` and `network-plugin-auth` enforce.
|
||||
- [`offline-and-reintegration.md`](./offline-and-reintegration.md) — `network-plugin-server-bridge` in autonomous mode.
|
||||
- [`implementation-status.md`](./implementation-status.md) — what's shipped on genesis.
|
||||
178
docs/network/security-and-tokens.md
Normal file
178
docs/network/security-and-tokens.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Network Security and Tokens
|
||||
|
||||
The genesis network has three orthogonal layers of cryptographic enforcement:
|
||||
|
||||
1. **Identity** — every peer has a stable Ed25519 identity. Every message is signed.
|
||||
2. **Authority** — server-signed tokens grant a peer the right to do specific things in a specific mesh/session.
|
||||
3. **Trust** — cross-mesh interactions require a `MeshTrustToken` that establishes one mesh's willingness to accept envelopes from another.
|
||||
|
||||
This document is the reference for all three.
|
||||
|
||||
## Layer 1 — Message-level integrity (NET-27, NET-30)
|
||||
|
||||
`network-plugin-crypto-messages` is the layer.
|
||||
|
||||
### Per-message Ed25519 signatures (NET-27)
|
||||
|
||||
Every outbound `MessageEnvelope` from a host (Tier 0 or Tier 1) carries an Ed25519 signature in its headers. Inbound signatures are verified using peer public keys from the registry. Invalid signatures cause the message to be **dropped silently** (and the sender's reputation in `HostHistory` to take a hit).
|
||||
|
||||
The key material is the same drive-identity key used by the chunks plugin — one Ed25519 keypair per peer, used at multiple layers. Generated and stored by `@suaveplan/crypto-vault` (BIP39 mnemonic + Argon2id KDF + zeroize on lock).
|
||||
|
||||
### Peer identity challenge-response (NET-30)
|
||||
|
||||
When two peers handshake, they perform a challenge-response to verify identity:
|
||||
|
||||
```text
|
||||
A → B: peerA_pubkey, nonceA
|
||||
B → A: peerB_pubkey, nonceB, sigB(peerA_pubkey ‖ nonceA)
|
||||
A → B: sigA(peerB_pubkey ‖ nonceB)
|
||||
```
|
||||
|
||||
Any peer claiming `peerB_pubkey` who can't produce `sigB(...)` is rejected. This catches spoofed peer-IDs at handshake before they make it into the registry.
|
||||
|
||||
## Layer 2 — Authority tokens (NET-28, NET-29, NET-106..NET-109)
|
||||
|
||||
Three token shapes, all server-signed (in models with a server):
|
||||
|
||||
### `MeshToken` (NET-29)
|
||||
|
||||
```ts
|
||||
interface MeshToken {
|
||||
meshId: string;
|
||||
maxCapacity: number;
|
||||
allowedTiers: ('tier-0' | 'tier-1' | 'tier-2')[];
|
||||
hostingModel: 'PURE_CLIENT' | 'SERVER_BASED' | 'SERVER_TO_CLIENT_FAILOVER' | 'CLIENT_WITH_SERVER_BACKUP';
|
||||
expiresAt: number;
|
||||
serverSignature: Uint8Array; // Ed25519 over canonicalSerialize(this without the sig)
|
||||
}
|
||||
```
|
||||
|
||||
Required for mesh creation. The mesh's hosting model lives here. Long expiry (days/weeks).
|
||||
|
||||
### `SessionToken` (NET-28)
|
||||
|
||||
```ts
|
||||
interface SessionToken {
|
||||
sessionId: string;
|
||||
parentSessionId: string | null;
|
||||
capacity: number;
|
||||
rulesHash: string; // hash of the session's policy doc
|
||||
expiresAt: number;
|
||||
serverSignature: Uint8Array;
|
||||
}
|
||||
```
|
||||
|
||||
Required for session creation. Sessions can nest (NET-19) up to depth 4. The `rulesHash` pins the session's policy at issuance time so it can't be silently mutated.
|
||||
|
||||
### `CapabilityToken` (NET-108, NET-109)
|
||||
|
||||
```ts
|
||||
interface CapabilityToken {
|
||||
peerId: string;
|
||||
meshId: string;
|
||||
tier: 'tier-0' | 'tier-1' | 'tier-2';
|
||||
capabilities: string[]; // e.g. ['relay', 'vote', 'admit-peer']
|
||||
issuedAt: number;
|
||||
expiresAt: number;
|
||||
serverSignature: Uint8Array;
|
||||
}
|
||||
```
|
||||
|
||||
Issued when a peer joins. The `capabilities` array constrains what the peer is *authorised* to do regardless of what the protocol allows it to *attempt*. A Tier-0 capability token includes `admit-peer`; a Tier-2 doesn't. Cross-tier escalation is blocked at the verifier (NET-48).
|
||||
|
||||
`network-plugin-auth` (NET-106..NET-109) is the enforcement plugin. `MeshMembershipVerifier` checks the `authToken` on join against `@suaveplan/auth`. `CapabilityTokenIssuer` mints them. `CrossMeshTrustValidator` (NET-110..NET-112) handles inter-mesh acceptance.
|
||||
|
||||
### Tier-specific security policies (NET-48)
|
||||
|
||||
```text
|
||||
Tier 0 — full authority. Can issue tokens (when server isn't available, with grace-period restrictions).
|
||||
Tier 1 — relay + voting authority. Cannot issue tokens.
|
||||
Tier 2 — read-only for sensitive operations.
|
||||
```
|
||||
|
||||
`SecurityManager.processTierSecurity(tierId, operation, data)` enforces this on every operation. Cross-tier delegation requires explicit credential transfer from the higher tier.
|
||||
|
||||
## Layer 3 — Cross-mesh trust (NET-110..NET-112)
|
||||
|
||||
`MeshTrustToken` is what one mesh hands another mesh to say "I'll accept your relay traffic":
|
||||
|
||||
```ts
|
||||
interface MeshTrustToken {
|
||||
fromMeshId: string;
|
||||
toMeshId: string;
|
||||
allowedOperations: string[]; // e.g. ['relay-message', 'cd:request']
|
||||
bandwidthQuotaPerHour?: number;
|
||||
expiresAt: number;
|
||||
bothServerSignatures: [Uint8Array, Uint8Array]; // signed by both meshes' Tier 0
|
||||
}
|
||||
```
|
||||
|
||||
`CrossMeshTrustValidator` rejects relay envelopes from untrusted mesh ids (NET-110). Tier 0 hosts verify trust tokens before accepting backbone connections (NET-112). Trust decisions are cached with configurable TTL (NET-111) so we don't re-verify on every envelope.
|
||||
|
||||
This means a phone-mesh and a corporate-mesh don't automatically interoperate. Two Tier 0 hosts must explicitly establish trust (out of band: typically via a co-signed token issued by both servers, or via a bilateral capability exchange in `PURE_CLIENT` mode).
|
||||
|
||||
## Plugin security verifier (NET-47)
|
||||
|
||||
`network-plugin-crypto-messages` also validates plugins themselves:
|
||||
|
||||
```ts
|
||||
class PluginSecurityVerifier {
|
||||
verifyPlugin(pluginId: string, pluginData: Uint8Array): Promise<boolean>;
|
||||
authorizePluginOperation(pluginId, operation, context?): Promise<boolean>;
|
||||
monitorPluginActivity(pluginId): void;
|
||||
setPluginSecurityPolicy(policy): void;
|
||||
}
|
||||
```
|
||||
|
||||
The intent is plugin-code signing: a deployment may want to allow only plugins signed by a known publisher key. Plugins requesting capabilities they didn't declare are rejected. Anomalous plugin activity (excessive broadcast, unauthorised operations) is detected and reported.
|
||||
|
||||
The old planning corpus admits (`PHASE_6_README.md`) that the resource-monitoring side of this is "currently simulated and would need real metrics in a production environment." Genesis treats it as a hardening backlog item — install the plugin, get signature verification today; resource accounting follows.
|
||||
|
||||
## What about the model where the server is gone?
|
||||
|
||||
Tokens issued before the server vanished remain valid until their `expiresAt`. `network-plugin-server-bridge` (NET-32) extends grace periods so existing tokens don't expire mid-outage. New tokens cannot be issued — operations that require a fresh token (admit a brand-new peer, create a new session) are queued until the server returns. Existing peers, with valid `CapabilityToken`s, keep operating with full authority.
|
||||
|
||||
When the server returns:
|
||||
|
||||
1. Buffered signed-log entries replay (NET-33).
|
||||
2. The server validates each entry's Ed25519 signature against the issuing peer's pubkey.
|
||||
3. Conflicts are detected (e.g., two peers both claimed Tier-0 in different partitions). Resolution depends on the partition merge strategy (NET-80).
|
||||
4. The server broadcasts the reconciled state.
|
||||
5. Queued token-issuance operations execute.
|
||||
|
||||
The cryptographic chain — every message signed, every token signed, every audit-log entry signed — is what makes this replay possible. There's no ambiguity about who did what; every action is provenance-stamped.
|
||||
|
||||
## Audit log (NET-37..NET-39)
|
||||
|
||||
`network-plugin-audit-mesh` keeps an append-only log where each entry includes:
|
||||
|
||||
```text
|
||||
{ seq, prevHash, payload, hostSignature }
|
||||
```
|
||||
|
||||
- `seq` is monotonic.
|
||||
- `prevHash` makes the log a hash chain — break detection on any tampering.
|
||||
- `hostSignature` is the issuing host's Ed25519 over `(seq, prevHash, payload)`.
|
||||
|
||||
Logs are replicated to at least R sibling hosts (NET-38). CRITICAL events replicate synchronously. Old logs are compacted into signed checkpoints (NET-39) — a snapshot at a given seq, signed by the host, that downstream replicas use to skip the prefix.
|
||||
|
||||
This is what gives you a verifiable audit trail across the entire mesh, surviving partitions and server outages.
|
||||
|
||||
## Where keys live, end to end
|
||||
|
||||
| Key | Owner | Storage | Used for |
|
||||
| --- | --- | --- | --- |
|
||||
| Per-peer Ed25519 (drive identity) | The peer | `@suaveplan/crypto-vault` | Signing every message, every chunk, every audit entry |
|
||||
| Per-mesh server Ed25519 | Central server | Server HSM / `@suaveplan/keys` (when available) | Signing `MeshToken`, `SessionToken`, `CapabilityToken` |
|
||||
| Cross-mesh trust signing | Both meshes' servers | As above | Co-signing `MeshTrustToken` |
|
||||
| Per-file `fileKey` (AEAD) | Drive | Wrapped by partition KEK; lives next to manifest | Encrypting chunks |
|
||||
| Per-partition KEK | `drive-sharing` | Wrapped per-grantee with X25519 sealed-box | Wrapping `fileKey`s |
|
||||
|
||||
Keys never cross trust boundaries unwrapped. Wrapping is the only mechanism for delegation.
|
||||
|
||||
## Reading further
|
||||
|
||||
- [`architecture.md`](./architecture.md) — `MessageEnvelope` and `NetworkBus`.
|
||||
- [`topology-and-failover.md`](./topology-and-failover.md) — when token issuance is paused.
|
||||
- [`offline-and-reintegration.md`](./offline-and-reintegration.md) — buffered log replay.
|
||||
- [`../drive/encryption.md`](../drive/encryption.md) — the drive layer's use of the per-peer Ed25519 key.
|
||||
205
docs/network/topology-and-failover.md
Normal file
205
docs/network/topology-and-failover.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# Topology, Tiers, Meshed Pockets, and Failover
|
||||
|
||||
The user's framing — verbatim — was: *"a resilient tiered network system where if the server goes down, it's all in place with meshed pockets of users with their nominated hosts / those hosts on a tier nominate their tier host."*
|
||||
|
||||
This is the document for that. It is the load-bearing piece of the network design.
|
||||
|
||||
## The tier model (NET-11, NET-49, NET-122)
|
||||
|
||||
Every peer occupies exactly one of three tiers at any moment:
|
||||
|
||||
| Tier | Role | Authority | Sibling mesh |
|
||||
| --- | --- | --- | --- |
|
||||
| **Tier 0** — Host | Top of the local mesh. Sends/receives from the central server (when present). | Token issuance, peer admission, log finalisation. | Other Tier 0 hosts (cross-mesh peers). |
|
||||
| **Tier 1** — Sub-host | Aggregator for a group of leaves. Relay between Tier 0 and Tier 2. | Vote in elections, relay messages, no token issuance. | Other Tier 1 sub-hosts in the same mesh. |
|
||||
| **Tier 2** — Leaf | Regular client. Speaks only to its assigned Tier 1 (and laterally to other Tier 2 in its session). | Read-only for sensitive ops. | Other Tier 2 in the same session pocket. |
|
||||
|
||||
`network-plugin-topology` owns this. Promotion/demotion is automatic, based on capability scores (latency, uptime, reliability, device profile from NET-49) and group-size limits. Peers MUST meet a stability threshold and a cooldown (`network-plugin-topology` NET-52) to prevent thrashing.
|
||||
|
||||
**Tier depth is dynamic.** A 5-node mesh runs flat (everyone Tier 0/Tier 2; no sub-host). A 5,000-node mesh runs three-tier with hundreds of Tier 1 sub-hosts. The topology plugin adapts depth based on size; collapsing or expanding tiers as needed (`AdaptiveTierDepth`).
|
||||
|
||||
## The four hosting models (NET-56)
|
||||
|
||||
`network-plugin-server-bridge` lets a deployment pick how clients and the central server interact:
|
||||
|
||||
| Model | Behaviour |
|
||||
| --- | --- |
|
||||
| **`PURE_CLIENT`** | No central server. Mesh is fully autonomous. A user creates a session and acts as Tier 0 host. Subsequent users join, get assigned Tier 1/Tier 2 by topology, and the host slot transfers via election when the host leaves. |
|
||||
| **`SERVER_BASED`** | Server is always Tier 0. All clients are Tier 1 or Tier 2. Server enforces permissions, holds authoritative state, issues all tokens. If the server disconnects, the mesh **stops** (this model trades resilience for auditability). |
|
||||
| **`SERVER_TO_CLIENT_FAILOVER`** | Server is Tier 0 while online. On disconnect, an election promotes the strongest Tier 1 to acting Tier 0, the mesh continues, and on server reconnect the server smoothly reintegrates as Tier 0. **This is the model that matches your user request.** |
|
||||
| **`CLIENT_WITH_SERVER_BACKUP`** | A user is Tier 0; the server joins as a participant (typically Tier 1). If the user disconnects or becomes unsuitable, hosting can transfer to the server. Hybrid authority — server provides persistence, client provides agency. |
|
||||
|
||||
The model is selected at mesh creation and affects election behaviour, token authority, and reconciliation direction. The mesh token (NET-29) carries it.
|
||||
|
||||
## Meshed pockets
|
||||
|
||||
A "pocket" is a group of peers that can talk to each other without going through Tier 0. Pockets emerge from two structural primitives:
|
||||
|
||||
1. **Sibling links at every tier.** A Tier 1 sub-host has up to 10 sibling Tier 1 sub-hosts in the same mesh. A Tier 2 leaf has up to N session-mates (capped per-session, NET-22). When a tier above goes silent, a pocket continues to function via siblings.
|
||||
2. **Session trees** (NET-18..NET-22). The mesh organises peers into sessions — typed groupings (a video call, a shared drive, a multiplayer game). A pocket is, concretely, the peers in one leaf session plus that session's host. Sessions can nest up to depth 4; child sessions inherit read-only state from parents.
|
||||
|
||||
If the central server vanishes:
|
||||
|
||||
- Each Tier 0 host keeps running its mesh autonomously (modulo authority limits — see below).
|
||||
- Each Tier 1 keeps relaying for its Tier 2 leaves.
|
||||
- Each session pocket keeps gossiping among its members.
|
||||
- Cross-pocket traffic is brokered by `network-plugin-relay` over the Tier 0 backbone (or, if Tier 0 is also gone in a partition, via the elected acting Tier 0).
|
||||
|
||||
Nothing about pocket operation requires the server. The server is needed only for the *authoritative* operations the mesh declines to do without it.
|
||||
|
||||
## Host election — how peers nominate hosts (NET-08, NET-118, NET-119)
|
||||
|
||||
`network-plugin-governance` runs elections. The algorithm is:
|
||||
|
||||
1. **Collect candidates** — peers eligible by capability/stability threshold.
|
||||
2. **Score each candidate** — `HostSelector` (NET-118) does weighted multi-factor evaluation:
|
||||
- Latency (EMA of measured RTT to other peers).
|
||||
- Stability (jitter + packet loss).
|
||||
- Uptime (since last reconnect).
|
||||
- Capabilities (CPU, bandwidth, NAT type).
|
||||
- **Trust score** — historical performance from `HostHistory` (NET-119), which keeps an append-only record of successful/unsuccessful hosting sessions per peer.
|
||||
3. **Vote** — `VotingManager` collects votes via the bus; configurable participation threshold and consensus requirement.
|
||||
4. **Tie-break** — lowest lexicographic peerId wins.
|
||||
5. **Publish** — `election:completed { winnerId, term }` on the bus.
|
||||
|
||||
Three modes (NET-08): `metric` (pure score), `direct` (assigned by Tier 0), `hybrid` (score-suggested + Tier 0-confirmed).
|
||||
|
||||
Each session also runs its own host election independently of the mesh Tier 0 host (NET-20). One peer can host multiple sessions.
|
||||
|
||||
## Server failover — what happens when the server goes down
|
||||
|
||||
This is the resilience pipeline (`SERVER_TO_CLIENT_FAILOVER` model):
|
||||
|
||||
```text
|
||||
1. Heartbeat timeout
|
||||
`network-plugin-failover` heartbeat (NET-10): alive → suspect → dead.
|
||||
`network-plugin-server-bridge` (NET-31) detects server disconnect within 2× heartbeat interval.
|
||||
|
||||
2. Verification
|
||||
Multiple Tier 1 peers re-ping the server through alternate paths.
|
||||
Avoid declaring the server dead on a single network blip.
|
||||
|
||||
3. Consensus
|
||||
Bus topic 'server:dead' published when N Tier 1 peers agree.
|
||||
|
||||
4. Election
|
||||
`network-plugin-governance` runs `HOST_SELECTION` among eligible Tier 1 peers.
|
||||
Winning peer is promoted to Tier 0 (acting host).
|
||||
|
||||
5. Authority handover
|
||||
Tokens issued before server-down remain valid.
|
||||
New token issuance is paused (NET-32) — see "graceful degradation" below.
|
||||
|
||||
6. Operation continues
|
||||
Each Tier 0 host keeps running its mesh autonomously.
|
||||
Cross-mesh traffic uses `network-plugin-relay` Tier 0 backbone (NET-62).
|
||||
|
||||
7. Server returns
|
||||
`network-plugin-server-bridge` reconciliation (NET-33):
|
||||
- Buffered signed log entries replayed in sequence.
|
||||
- Server validates signatures, detects conflicts, broadcasts reconciled state.
|
||||
- Server resumes Tier 0 (election may run to confirm — depends on hosting model).
|
||||
Acting Tier 0 demotes back to Tier 1.
|
||||
```
|
||||
|
||||
## Graceful degradation — server-vs-autonomous authority split
|
||||
|
||||
When the server is offline (NET-32), the mesh continues with reduced authority:
|
||||
|
||||
| Operation | Server present | Server offline |
|
||||
| --- | --- | --- |
|
||||
| Send/receive messages | ✓ | ✓ |
|
||||
| Read existing files (drive layer) | ✓ | ✓ |
|
||||
| Write existing files | ✓ | ✓ (signed by drive identity, verifiable later) |
|
||||
| Vote in elections | ✓ | ✓ |
|
||||
| Reorganise topology | ✓ | ✓ |
|
||||
| Issue new mesh / session tokens | ✓ | ✗ (queued; replayed on reconnect) |
|
||||
| Admit a brand-new peer | ✓ | ✗ (unless server-pre-issued capability token still valid) |
|
||||
| Cross-mesh trust establishment | ✓ | ✗ (existing trust links remain valid) |
|
||||
| Audit-log finalisation | ✓ | ✗ (entries are buffered, signed, replayed on reconnect) |
|
||||
|
||||
`network-plugin-server-bridge` extends token grace periods (NET-32) so existing tokens don't expire mid-outage. New session creation is paused. Signed log entries are buffered locally and replayed on reconnect (NET-33).
|
||||
|
||||
## Cross-mesh relay — connecting pockets across meshes (NET-40, NET-61..NET-70)
|
||||
|
||||
When a peer in Mesh A needs to send to a peer in Mesh B, `network-plugin-relay` handles it.
|
||||
|
||||
- `RelayRouter` maintains a routing table `(destMeshId, destPeerId) → nextHopHostId`.
|
||||
- `HostBackboneTransport` connects Tier 0 hosts of different meshes (capability-negotiated).
|
||||
- `CrossMeshEnvelope` extends `MessageEnvelope` with `senderMeshId`, `targetMeshId`, `hopCount`, `maxHops` (default 8), `visitedNodes`, `ttl`.
|
||||
- Loop prevention is strict (NET-64..NET-66): drop on `hopCount >= maxHops`, `ttl <= 0`, or `visitedNodes` already contains current peer.
|
||||
- Cross-mesh broadcast is deduplicated at mesh boundaries (NET-68).
|
||||
- Relay chooses paths via `LowestLatency` / `HighestReliability` / `FewestHops` / `TierOptimized` / `LoadBalanced` strategies (NET-40).
|
||||
|
||||
This is what makes "meshed pockets" actually composable: a phone-mesh, a corporate-mesh, and a public-server-mesh can interoperate by establishing trust between Tier 0 hosts (NET-110, NET-112) without merging into one big mesh.
|
||||
|
||||
## Partition handling (NET-50, NET-78..NET-83)
|
||||
|
||||
A network partition splits the mesh into two or more disjoint groups, each unable to reach the others. `network-plugin-partition` handles it:
|
||||
|
||||
1. **Detection** — `PartitionDetector` tracks epochs from all known peers; partition declared when a majority is unreachable. Deterministic `partitionId = hash(sorted reachable peer set)`.
|
||||
2. **Autonomous operation** — each partition operates with its local hierarchy. New elections within the partition fill any gaps.
|
||||
3. **Quorum resolution** — when partitions heal, `QuorumResolver` (NET-80) decides the authoritative partition using one of: `size`, `performance`, `age`, `hybrid`.
|
||||
4. **Merge** — `MergeProtocol` (NET-81) reconciles registry, topology, and audit state. Non-authoritative peers rejoin (NET-82). Bus events `partition:detected` / `partition:healed`.
|
||||
|
||||
## Resource awareness (NET-49, NET-120..NET-124)
|
||||
|
||||
The topology plugin adapts to device profile (`CONSTRAINED`, `STANDARD`, `POWERFUL`, `SERVER`):
|
||||
|
||||
- Constrained devices use `MessagePool` (NET-120) to reuse message objects.
|
||||
- Zero-copy passing (NET-121) where the platform allows.
|
||||
- `BatteryAwareMode` (NET-122) reduces heartbeat frequency, defers non-urgent reorganisations, lowers transport retry aggression.
|
||||
- `BandwidthConservator` (NET-124) coalesces small messages, increases batching, pauses redundant CRITICAL sends.
|
||||
|
||||
So a phone running on battery in a constrained network drops to a lighter footprint without being kicked out of the mesh.
|
||||
|
||||
## Putting it all together — the resilience picture
|
||||
|
||||
```text
|
||||
Central Server (Tier 0 of meshes M1, M2, M3)
|
||||
│ (signed tokens, audit log, authoritative state)
|
||||
│
|
||||
┌───────────┼───────────┐
|
||||
│ │ │
|
||||
mesh M1 mesh M2 mesh M3
|
||||
│ │ │
|
||||
┌────┴────┐ ┌────┴────┐ ┌────┴────┐
|
||||
│ Tier 0 │ │ Tier 0 │ │ Tier 0 │
|
||||
│ host │ │ host │ │ host │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
┌────┴────┐ ┌────┴────┐ ┌────┴────┐
|
||||
│ Tier 1 │ │ Tier 1 │ │ Tier 1 │
|
||||
│ sub-hosts ......
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
[pockets of Tier 2 leaves, organised into sessions]
|
||||
|
||||
|
||||
When the server vanishes:
|
||||
|
||||
╳ Central Server ╳
|
||||
(offline)
|
||||
┌─────────────────────────────────┐
|
||||
│ │
|
||||
mesh M1 ─── relay ─── mesh M2 ─── relay ─── mesh M3
|
||||
│ │
|
||||
each mesh elects an acting Tier 0
|
||||
each Tier 0 keeps running its mesh
|
||||
each session pocket keeps gossiping
|
||||
tokens issued before outage stay valid (grace period)
|
||||
new token issuance queued; replayed on reconnect
|
||||
signed audit-log entries buffered locally
|
||||
│
|
||||
server returns
|
||||
│
|
||||
buffered logs replayed; conflicts reconciled
|
||||
acting Tier 0s demote to Tier 1
|
||||
server resumes as Tier 0 of all meshes
|
||||
```
|
||||
|
||||
## Reading further
|
||||
|
||||
- [`security-and-tokens.md`](./security-and-tokens.md) — the cryptographic basis for token grace periods and signed log replay.
|
||||
- [`offline-and-reintegration.md`](./offline-and-reintegration.md) — the offline-mode buffering and reconciliation in detail.
|
||||
- [`plugins-catalog.md`](./plugins-catalog.md) — every plugin called out above, with one-paragraph descriptions.
|
||||
Reference in New Issue
Block a user