suaveplan/openspec/changes/archive/2026-05-04-network-plugin-circuit-breaker/proposal.md

# OpenSpec Change: network-plugin-circuit-breaker

- **Change ID**: network-plugin-circuit-breaker
- **Package**: `@suaveplan/network-plugin-circuit-breaker` (new)
- **Status**: PROPOSED
- **Date**: 2026-05-01

---

## Summary

Implement circuit breakers at mesh boundaries. `@suaveplan/retry` provides per-request retry with exponential backoff, but there is no circuit breaker on cross-mesh relay paths. If Mesh A's Tier 0 host cannot reach Mesh B's Tier 0 host, repeated retries waste bandwidth, delay failure detection, and can cascade load onto alternative paths. A circuit breaker allows fast-fail and graceful degradation.

---

## Why

**Current state**: `@suaveplan/retry` exists but no circuit breaker is integrated at the mesh level. The `enterprise-network-standby-transport` proposal has transport-level failover but no mesh-level circuit breaking.

**Why this matters**: In a multi-mesh federation:

- A dead host on a relay path causes every cross-mesh message to retry N times before failing.
- Without a circuit breaker, the mesh cannot distinguish between "slow" and "dead" quickly.
- Cascading retries can saturate the remaining healthy paths.

---

## Core components

### 1. `MeshCircuitBreaker`

Per-path circuit breaker state machine: `CLOSED` → `OPEN` → `HALF_OPEN`.

```ts
class MeshCircuitBreaker {
  constructor(options: {
    pathId: string; // e.g. "meshA:host1→meshB:host2"
    failureThreshold: number; // default 5
    successThreshold: number; // default 3
    openDurationMs: number; // default 30000
    halfOpenMaxCalls: number; // default 3
  });
  getState(): "CLOSED" | "OPEN" | "HALF_OPEN";
  recordSuccess(): void;
  recordFailure(): void;
  canExecute(): boolean;
  on(event: "state:changed", handler: (state: string) => void): void;
}
```

**States**:

- `CLOSED`: Normal operation. Failures are counted. When `failureThreshold` is reached → `OPEN`.
- `OPEN`: All executions are rejected immediately (fast-fail). After `openDurationMs` → `HALF_OPEN`.
- `HALF_OPEN`: A limited number (`halfOpenMaxCalls`) of trial executions are allowed. If `successThreshold` successes occur → `CLOSED`. If any failure → `OPEN`.

### 2. `CircuitBreakerRegistry`

Manages circuit breakers for all relay paths in the mesh.

```ts
class CircuitBreakerRegistry {
  getOrCreate(pathId: string, options?: CircuitBreakerOptions): MeshCircuitBreaker;
  getState(pathId: string): "CLOSED" | "OPEN" | "HALF_OPEN" | undefined;
  remove(pathId: string): void;
  listOpenPaths(): string[];
}
```

### 3. `HalfOpenProbe`

Sends lightweight probe messages during `HALF_OPEN` to test path health without risking real traffic.

```ts
class HalfOpenProbe {
  constructor(options: {
    registry: CircuitBreakerRegistry;
    probeIntervalMs: number; // default 5000
    probeMessageFactory: () => MessageEnvelope;
  });
  start(): void;
  stop(): void;
}
```

Probes are ping-like messages with `type: "relay:probe"` and expect an ack. They do not carry application payload.

### 4. `CircuitBreakerMiddleware`

Integrates with `network-plugin-relay` by wrapping `HostBackboneTransport.send()`:

```ts
function createCircuitBreakerMiddleware(
  registry: CircuitBreakerRegistry
): RelayMiddleware {
  return {
    beforeSend(envelope, nextHopHostId) {
      const pathId = buildPathId(localHostId, nextHopHostId);
      const cb = registry.getOrCreate(pathId);
      if (!cb.canExecute()) {
        throw new CircuitBreakerError(CB.PATH_OPEN, { pathId });
      }
      try {
        const result = next();
        cb.recordSuccess();
        return result;
      } catch (err) {
        cb.recordFailure();
        throw err;
      }
    },
  };
}
```

---

## Error codes

```ts
const CB_CODES = defineCodes("CB", [
  "PATH_OPEN",
  "PATH_HALF_OPEN_REJECTED",
  "PROBE_FAILED",
  "REGISTRY_PATH_NOT_FOUND",
] as const);
```

`CircuitBreakerError extends SuaveplanError<CircuitBreakerErrorCode>`.

---

## Dependencies

- **Consumes**: `@suaveplan/network` (`NetworkBus`, `MessageEnvelope`), `@suaveplan/network-plugin-relay` (`HostBackboneTransport`, `RelayManager`), `@suaveplan/retry` (for underlying retry policy configuration), `@suaveplan/error`, `@suaveplan/types`, `@opentelemetry/api`.
- **Blocked on**: `network-core` (shipped), `network-plugin-relay`.

---

## Verification approach

- `src/breaker/breaker.spec-conformance.test.ts` — pins CLOSED→OPEN→HALF_OPEN→CLOSED transition, threshold behaviour.
- `src/registry/registry.spec-conformance.test.ts` — pins getOrCreate, listOpenPaths, removal.
- `src/probe/probe.spec-conformance.test.ts` — pins probe emission, ack handling, failure counting.
- `src/middleware/middleware.spec-conformance.test.ts` — pins relay middleware integration, fast-fail on OPEN.
- 100/100/100/100 Istanbul coverage.

---

## Conformance requirements (proposed)

- `NET-84` — `MeshCircuitBreaker` transitions from `CLOSED` to `OPEN` after `failureThreshold` consecutive failures.
- `NET-85` — `MeshCircuitBreaker` transitions from `OPEN` to `HALF_OPEN` after `openDurationMs`.
- `NET-86` — `MeshCircuitBreaker` transitions from `HALF_OPEN` to `CLOSED` after `successThreshold` consecutive successes.
- `NET-87` — `MeshCircuitBreaker` in `OPEN` state rejects all executions immediately.
- `NET-88` — `HalfOpenProbe` sends lightweight probe messages during `HALF_OPEN`.
- `NET-89` — `CircuitBreakerRegistry` manages per-path circuit breakers.
- `NET-90` — Circuit breaker middleware wraps `HostBackboneTransport.send()` and records success/failure.

---

## Out of scope

- **Adaptive thresholds** → v2; v1 uses fixed configuration.
- **ML-based anomaly detection** → v2.
- **Per-peer circuit breakers** → handled by `@suaveplan/retry` at the transport layer; this plugin focuses on mesh-boundary paths.