Files
2026-05-27 23:29:53 +01:00

6.5 KiB

OpenSpec Change: network-plugin-partition

  • Change ID: network-plugin-partition
  • Package: @suaveplan/network-plugin-partition (new)
  • Status: PROPOSED
  • Date: 2026-05-01

Summary

Implement split-brain detection and resolution for the mesh. The network-plugin-failover proposal explicitly excludes network partition detection: "Network partition detection (split-brain) → future hardening change." This plugin fills that exclusion. Without it, a network partition can cause:

  • Duplicate hosts in the same mesh
  • Messages routed to the wrong partition
  • Infinite relay loops across partitioned meshes

Why

Current state: network-plugin-failover handles host death (alive → suspect → dead) but does not detect when the mesh splits into two or more isolated groups, each believing it is the authoritative partition.

Why this matters: In a multi-mesh system with relay, split-brain can cause:

  • Two Tier 0 hosts both claiming to be the primary host for the same mesh
  • network-plugin-relay forwarding messages into a partition that has diverged
  • network-plugin-registry peers replicating to hosts that are no longer reachable

Core components

1. PartitionDetector

Detects network partitions using vector clocks or epoch numbers.

class PartitionDetector {
  constructor(options: {
    heartbeatSource: HeartbeatSource; // consumes from failover plugin
    meshId: string;
    quorumRatio: number; // default 0.5 (majority)
    epochIntervalMs: number; // default 5000
  });
  start(): void;
  stop(): void;
  getCurrentEpoch(): number;
  getPartitionView(): PartitionView;
}

interface PartitionView {
  epoch: number;
  knownPeers: string[];
  reachablePeers: string[];
  isPartitioned: boolean;
  partitionId: string; // deterministic hash of reachable set
  primary: boolean; // true if this partition believes it is the primary
}

Detection strategy:

  • Each peer emits an epoch heartbeat every epochIntervalMs.
  • Peers track the highest epoch they've seen from every other peer.
  • If a peer misses epochs from a majority of known peers for suspectThreshold * 2, a partition is suspected.
  • PartitionView.partitionId is a deterministic hash (e.g. BLAKE3) of the sorted reachable peer set.
  • Two peers with different partitionId values are in different partitions.

2. QuorumResolver

Determines which partition is authoritative when split-brain is detected.

class QuorumResolver {
  constructor(options: {
    strategy: QuorumStrategy; // "size" | "performance" | "age" | "hybrid"
  });
  resolve(partitions: PartitionView[]): PartitionView | null; // null = cannot resolve
}

type QuorumStrategy = "size" | "performance" | "age" | "hybrid";

Strategies:

  • size: the partition with the most peers wins.
  • performance: the partition with the lowest aggregate latency / highest bandwidth wins.
  • age: the partition containing the oldest-established host wins.
  • hybrid: weighted composite of size (0.4), performance (0.4), age (0.2).

3. MergeProtocol

Reconciles diverged state when partitions heal.

class MergeProtocol {
  constructor(options: {
    registryReconciler: RegistryReconciler;
    topologyReconciler: TopologyReconciler;
    auditReconciler: AuditReconciler;
  });
  initiateMerge(localView: PartitionView, remoteView: PartitionView): Promise<MergeResult>;
}

interface MergeResult {
  success: boolean;
  mergedRegistry: PeerRegistrySnapshot;
  mergedTopology: TopologySnapshot;
  conflicts: MergeConflict[];
}

Merge steps:

  1. Exchange PartitionView and validate epoch sequences.
  2. Compare registry snapshots: add missing peers, remove stale entries.
  3. Compare topology: re-run host election if both partitions had a Tier 0 host.
  4. Replay audit logs from the non-authoritative partition into the authoritative one.
  5. Emit partition:healed on the NetworkBus.

Integration with failover

network-plugin-partition consumes heartbeat data from network-plugin-failover via the NetworkBus:

  • Listens to failover:triggered events — if multiple hosts trigger failover simultaneously, partition is suspected.
  • Does NOT replace failover; it complements it. Failover handles single-host death; partition handles group isolation.

Error codes

const PARTITION_CODES = defineCodes("PARTITION", [
  "SPLIT_BRAIN_DETECTED",
  "QUORUM_NOT_REACHED",
  "MERGE_CONFLICT",
  "MERGE_FAILED",
  "EPOCH_SYNC_ERROR",
  "INVALID_PARTITION_VIEW",
  "PARTITION_HEAL_TIMEOUT",
] as const);

PartitionError extends SuaveplanError<PartitionErrorCode>.


Dependencies

  • Consumes: @suaveplan/network (NetworkNode, NetworkBus), @suaveplan/network-plugin-failover (heartbeat events, FSM states), @suaveplan/network-plugin-registry (registry snapshots), @suaveplan/network-plugin-topology (topology snapshots), @suaveplan/network-plugin-audit-mesh (audit log replay), @suaveplan/error, @suaveplan/types, @opentelemetry/api.
  • Blocked on: network-core (shipped), network-plugin-failover, network-plugin-registry, network-plugin-topology.

Verification approach

  • src/detector/detector.spec-conformance.test.ts — pins epoch tracking, partition detection, reachable-set hashing.
  • src/quorum/quorum.spec-conformance.test.ts — pins all four quorum strategies, tie-breaking.
  • src/merge/merge.spec-conformance.test.ts — pins registry merge, topology merge, conflict reporting.
  • tests/integration/partition.spec-conformance.test.ts — end-to-end: simulate partition, detect, resolve, heal.
  • 100/100/100/100 Istanbul coverage.

Conformance requirements (proposed)

  • NET-78 — PartitionDetector tracks epochs from all known peers and detects partition when a majority is unreachable.
  • NET-79 — PartitionView.partitionId is a deterministic hash of the sorted reachable peer set.
  • NET-80 — QuorumResolver supports size, performance, age, and hybrid strategies.
  • NET-81 — MergeProtocol reconciles registry, topology, and audit state when partitions heal.
  • NET-82 — Non-authoritative partition peers rejoin the authoritative partition after merge.
  • NET-83 — partition:detected and partition:healed events are emitted on the NetworkBus.

Out of scope

  • Byzantine fault tolerance → v2; v1 assumes honest peers.
  • Automatic partition creation → v2; v1 pauses new mesh operations during partition.
  • Cross-mesh partition detection → handled by network-plugin-relay loop prevention; this plugin focuses on intra-mesh splits.