first commit
This commit is contained in:
20
docs/networking/network/network-plugin-migration/README.md
Normal file
20
docs/networking/network/network-plugin-migration/README.md
Normal file
@@ -0,0 +1,20 @@
|
||||
# @suaveplan/network-plugin-migration - Peer state migration between meshes: coordinator, state-transfer protocol, forwarding tunnel for graceful handoff
|
||||
|
||||
## Summary
|
||||
|
||||
Peer state migration between meshes: 8-step `MigrationCoordinator`, snapshot serialize/deserialize via `StateTransferProtocol`, graceful-handoff `ForwardingTunnel` that buffers traffic and drains to the new host.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [migration-coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [state-transfer](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [state-transfer-protocol](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [forwarding-tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [error-codes](./src/error-codes.md) — Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECT…
|
||||
<!-- end:auto-docs --> - [migration-coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer-protocol](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [forwarding-tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECT…
|
||||
- [migration-coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer-protocol](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [forwarding-tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECT…
|
||||
- [migration-coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer-protocol](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [forwarding-tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECT…
|
||||
- [migration-coordinator](./src/coordinator/migration-coordinator.md) — <!-- begin:auto-docs --> _(no submodules)_- [state-transfer](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_ - [state-transfer-protocol](./src/state-transfer/state-transfer-protocol.md) — <!-- begin:auto-docs --> _(no submodules)_- [tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_ - [forwarding-tunnel](./src/tunnel/forwarding-tunnel.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECT…
|
||||
@@ -0,0 +1,97 @@
|
||||
# MigrationCoordinator
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Orchestrates the end-to-end migration flow for moving a peer's state from one mesh to another. Walks through eight ordered steps (validate, open tunnel, accept, snapshot, pack, receive, drain, finish) and surfaces every failure as a typed `MigrateError` so callers can pick the right recovery path. Centralising the coordination prevents the assignment / topology plugins from each implementing their own (drifty) migration logic.
|
||||
|
||||
## Features
|
||||
|
||||
- 8-step pipeline with a single typed exception per failure mode.
|
||||
- Pluggable `MigrationDestination` (accept + receive), `captureSnapshot` factory, and `forwarder` for in-flight traffic.
|
||||
- Configurable `timeoutMs` (default 30 s) checked at two natural break points (after pack, after drain) so partial-progress migrations can roll back.
|
||||
- Injectable `clock` for deterministic tests.
|
||||
- Every step's failure mode maps to `MIGRATION_REJECTED`, `STATE_TRANSFER_FAILED`, `MIGRATION_TIMEOUT`, or `TUNNEL_DRAINED`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
MigrationCoordinator,
|
||||
StateTransferProtocol,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
const coordinator = new MigrationCoordinator({
|
||||
localPeerId: "alice",
|
||||
localMeshId: "mesh-A",
|
||||
stateTransfer: new StateTransferProtocol({ serializer }),
|
||||
destination: {
|
||||
accept: async () => true,
|
||||
receive: async () => {},
|
||||
},
|
||||
forwarder: async () => {},
|
||||
captureSnapshot: () => ({
|
||||
peerId: "alice",
|
||||
schemaVersion: 1,
|
||||
state: { sessionId: "x" },
|
||||
capturedAt: Date.now(),
|
||||
}),
|
||||
});
|
||||
|
||||
const result = await coordinator.initiateMigration({
|
||||
peerId: "alice",
|
||||
fromMeshId: "mesh-A",
|
||||
toMeshId: "mesh-B",
|
||||
reason: "rebalance",
|
||||
priority: "graceful",
|
||||
});
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
MIGRATE_CODES,
|
||||
MigrateError,
|
||||
MigrationCoordinator,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
let now = 0;
|
||||
const coordinator = new MigrationCoordinator({
|
||||
localPeerId: "alice",
|
||||
localMeshId: "mesh-A",
|
||||
stateTransfer,
|
||||
destination,
|
||||
forwarder,
|
||||
captureSnapshot,
|
||||
timeoutMs: 5_000,
|
||||
clock: () => now,
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await coordinator.initiateMigration(request);
|
||||
} catch (err) {
|
||||
if (
|
||||
err instanceof MigrateError &&
|
||||
err.code === MIGRATE_CODES.MIGRATION_TIMEOUT
|
||||
) {
|
||||
await rollbackToOriginalMesh(request);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface MigrationRequest { peerId, fromMeshId, toMeshId, reason, priority }`.
|
||||
- `interface MigrationResult { success, peerId, newMeshId, transferredBytes, forwardedEnvelopes, durationMs }`.
|
||||
- `interface MigrationDestination { accept(req): Promise<boolean>; receive(bytes): Promise<void> }`.
|
||||
- `interface MigrationCoordinatorConfig { localPeerId, localMeshId, stateTransfer, destination, forwarder, captureSnapshot, timeoutMs?, clock? }`.
|
||||
- `initiateMigration(request): Promise<MigrationResult>`.
|
||||
- `forwardDuringMigration(tunnel, envelope)` — enqueue envelope into the active tunnel.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The tunnel is a fresh `ForwardingTunnel` per migration so two concurrent migrations cannot leak each other's buffered traffic. Timeout is checked at two break points — after `pack` (which may stall on serializer I/O) and after `drain` (which may stall on the new host's recv pipe) — rather than via a single `setTimeout` that would race across awaits. `accept` returning `false` is treated as a clean refusal (`MIGRATION_REJECTED`), while a thrown error from `accept` propagates unchanged so consumers can distinguish "destination said no" from "destination crashed".
|
||||
@@ -0,0 +1,63 @@
|
||||
# MIGRATE_CODES
|
||||
|
||||
## Purpose
|
||||
|
||||
Frozen registry of error codes for the network-plugin-migration plugin. Four codes cover the surface: `MIGRATION_REJECTED` when the destination mesh refuses the request, `STATE_TRANSFER_FAILED` when the snapshot serialization or transmission fails, `TUNNEL_DRAINED` when the forwarding tunnel attempts a write after drain, and `MIGRATION_TIMEOUT` when the end-to-end flow exceeds its deadline. Companion `MigrateError` extends `SuaveplanError` so consumers pattern-match on `code` rather than parse messages.
|
||||
|
||||
## Features
|
||||
|
||||
- Four frozen codes registered under the `NET_MIGRATE` namespace.
|
||||
- `MigrateError extends SuaveplanError<MigrateErrorCode>`.
|
||||
- `name: "MigrateError"` on every instance — useful for log filters and dashboard alerts.
|
||||
- `cause` and `context` propagation via `SuaveplanErrorOptions`.
|
||||
- Codes are stable across versions; the registry only ever grows, never shrinks.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
MIGRATE_CODES,
|
||||
MigrateError,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
throw new MigrateError(
|
||||
MIGRATE_CODES.MIGRATION_REJECTED,
|
||||
"migration.rejected",
|
||||
{ context: { peerId: "alice", toMeshId: "mesh-B" } }
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
MIGRATE_CODES,
|
||||
MigrateError,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
try {
|
||||
await coordinator.initiateMigration(request);
|
||||
} catch (err) {
|
||||
if (err instanceof MigrateError) {
|
||||
if (err.code === MIGRATE_CODES.MIGRATION_TIMEOUT) {
|
||||
metrics.counter("migration.timeout").increment();
|
||||
return rollbackToOriginalMesh(request);
|
||||
}
|
||||
if (err.code === MIGRATE_CODES.STATE_TRANSFER_FAILED) {
|
||||
await retryWithSmallerSnapshot(request);
|
||||
return;
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `MIGRATE_CODES` — frozen `{ MIGRATION_REJECTED, STATE_TRANSFER_FAILED, TUNNEL_DRAINED, MIGRATION_TIMEOUT }`.
|
||||
- `type MigrateErrorCode` — union of registered code values.
|
||||
- `class MigrateError extends SuaveplanError<MigrateErrorCode>` — typed throwable.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The codes split rejection (declined upfront by destination), transfer-failed (network/serialization fault mid-flow), drain (write-after-drain in the forwarding tunnel), and timeout (deadline exceeded across the full handoff) so callers can choose distinct recovery paths. The package barrel side-effect-imports `./error-codes.js` so subpath consumers register the codes too. `MIGRATION_TIMEOUT` is checked at two natural break points (after `pack` and after `drain`) rather than via a wall-clock `setTimeout`, because mid-await timer races would leave the coordinator in indeterminate states.
|
||||
@@ -0,0 +1,79 @@
|
||||
# StateTransferProtocol
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Runs the snapshot serialize → wire bytes → deserialize loop for peer migrations and wraps every failure mode as `STATE_TRANSFER_FAILED`. The serializer is injected so consumers can use any encoding (JSON, MessagePack, BitPacker frames) without touching this layer. Centralising the size cap and error wrapping here means the coordinator never has to think about how state actually crosses the wire.
|
||||
|
||||
## Features
|
||||
|
||||
- Pluggable `SnapshotSerializer` — any object with `serialize` / `deserialize`.
|
||||
- Configurable `maxBytes` (default 16 MiB) enforced on both pack and unpack.
|
||||
- Single error code (`STATE_TRANSFER_FAILED`) with `cause` chained from the underlying serializer error.
|
||||
- Fully async; serializers can do I/O if they need to.
|
||||
- `getMaxBytes()` exposed for observability dashboards.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
StateTransferProtocol,
|
||||
type SnapshotSerializer,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
const serializer: SnapshotSerializer = {
|
||||
serialize: (s) => new TextEncoder().encode(JSON.stringify(s)),
|
||||
deserialize: (b) => JSON.parse(new TextDecoder().decode(b)),
|
||||
};
|
||||
const proto = new StateTransferProtocol({ serializer });
|
||||
|
||||
const bytes = await proto.pack({
|
||||
peerId: "alice",
|
||||
schemaVersion: 1,
|
||||
state: { sessionId: "x" },
|
||||
capturedAt: Date.now(),
|
||||
});
|
||||
const snapshot = await proto.unpack(bytes);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
MIGRATE_CODES,
|
||||
MigrateError,
|
||||
StateTransferProtocol,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
const proto = new StateTransferProtocol({
|
||||
serializer: msgpackSerializer,
|
||||
maxBytes: 1_048_576,
|
||||
});
|
||||
|
||||
try {
|
||||
const bytes = await proto.pack(largeSnapshot);
|
||||
await transport.send("alice", bytes);
|
||||
} catch (err) {
|
||||
if (
|
||||
err instanceof MigrateError &&
|
||||
err.code === MIGRATE_CODES.STATE_TRANSFER_FAILED
|
||||
) {
|
||||
await splitSnapshotIntoChunks(largeSnapshot);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface MigrationSnapshot { peerId, schemaVersion, state, capturedAt }`.
|
||||
- `interface SnapshotSerializer { serialize, deserialize }`.
|
||||
- `interface StateTransferProtocolConfig { serializer, maxBytes? }`.
|
||||
- `pack(snapshot): Promise<Uint8Array>` — throws `STATE_TRANSFER_FAILED` on serializer failure or oversize output.
|
||||
- `unpack(bytes): Promise<MigrationSnapshot>` — throws `STATE_TRANSFER_FAILED` on oversize input or deserializer failure.
|
||||
- `getMaxBytes(): number`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Size enforcement runs both at pack and at unpack so an attacker injecting an oversized payload at the destination cannot bypass the limit by skipping the serialize step. We deliberately do not implement chunking here — that's a layer up. Serializer exceptions are chained through `cause` so root-cause analysis sees both the high-level migration error and the underlying serializer fault.
|
||||
@@ -0,0 +1,73 @@
|
||||
# ForwardingTunnel
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Buffers traffic addressed to a migrating peer at the old host and forwards it to the new host once the migration coordinator says the new home is ready. The tunnel is one-shot: after `drain()` resolves, no further `enqueue` calls are accepted — that signal lets the surrounding plugin pipeline switch routing back to the normal mesh transport.
|
||||
|
||||
## Features
|
||||
|
||||
- `enqueue(envelope)` is the only ingestion point.
|
||||
- Configurable `bufferLimit` (default 1000) with drop-oldest backpressure.
|
||||
- `drain()` flushes the buffer through the injected `forwarder` in FIFO order, returns the count of forwarded envelopes.
|
||||
- Post-drain `enqueue` raises `TUNNEL_DRAINED` so callers see the closure deterministically rather than silently losing traffic.
|
||||
- `size()` and `isDrained()` exposed for observability.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { ForwardingTunnel } from "@suaveplan/network-plugin-migration";
|
||||
|
||||
const tunnel = new ForwardingTunnel({
|
||||
forwarder: async (envelope) => {
|
||||
await newTransport.send("bob", envelope);
|
||||
},
|
||||
});
|
||||
|
||||
tunnel.enqueue(inboundEnvelope);
|
||||
await tunnel.drain();
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
ForwardingTunnel,
|
||||
MIGRATE_CODES,
|
||||
MigrateError,
|
||||
} from "@suaveplan/network-plugin-migration";
|
||||
|
||||
const tunnel = new ForwardingTunnel({
|
||||
forwarder: realForwarder,
|
||||
bufferLimit: 100,
|
||||
});
|
||||
|
||||
for (const env of incoming) tunnel.enqueue(env);
|
||||
const forwarded = await tunnel.drain();
|
||||
metrics.gauge("migration.forwarded", forwarded);
|
||||
|
||||
try {
|
||||
tunnel.enqueue(lateArrival);
|
||||
} catch (err) {
|
||||
if (
|
||||
err instanceof MigrateError &&
|
||||
err.code === MIGRATE_CODES.TUNNEL_DRAINED
|
||||
) {
|
||||
await newTransport.send("bob", lateArrival);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `type TunnelForwarder = (envelope) => Promise<void>`.
|
||||
- `interface ForwardingTunnelConfig { forwarder, bufferLimit? }`.
|
||||
- `enqueue(envelope)` — throws `TUNNEL_DRAINED` after `drain()` has run.
|
||||
- `size(): number` / `isDrained(): boolean`.
|
||||
- `drain(): Promise<number>` — returns the number of forwarded envelopes.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Drop-oldest is the right backpressure because the new host can request a missing message via the registry-sync layer, but a stalled tunnel that filled up its buffer would block the migration entirely. The tunnel does not try to dedupe envelopes by id — that's the responsibility of the recipient's idempotency layer. `drain()` is single-shot and the FIFO order matches the order of `enqueue` calls, which matters for stateful recipients that rely on insertion order.
|
||||
Reference in New Issue
Block a user