196 lines
5.8 KiB
Markdown
196 lines
5.8 KiB
Markdown
# OpenSpec Change: enterprise-network-failover
|
|
|
|
- **Change ID**: enterprise-network-failover
|
|
- **Package**: `@suaveplan/network-failover`
|
|
- **Category**: universal/network
|
|
- **Target path**: `universal/network/network-failover`
|
|
- **Status**: SUPERSEDED-BY-GENESIS-WAVE
|
|
- **Date**: 2026-04-24
|
|
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/network-failover.md`
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Deliver `@suaveplan/network-failover` on the genesis branch from scratch at enterprise quality. The package provides network failover and recovery: health-checked connection groups, automatic failover on primary failure, configurable failover strategies (round-robin, fastest, weighted), recovery detection and fail-back, and in-flight request replay after failover. Genesis adds structured errors via `@suaveplan/error`, Zod schemas for all failover configuration, OTel instrumentation on failover events, `AbortSignal` on recovery operations, injectable `Clock` for health-check intervals, event-bus integration for failover state changes, and property-based tests for strategy correctness.
|
|
|
|
---
|
|
|
|
## Current-state audit (genesis)
|
|
|
|
The package does not exist on genesis yet — this is a fresh build.
|
|
|
|
---
|
|
|
|
## Available workspace deps on genesis
|
|
|
|
| Package | Role |
|
|
|---|---|
|
|
| `@suaveplan/error` (`workspace:^`) | Base class for `NetworkFailoverError`; stable error codes |
|
|
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `EventBusLike` |
|
|
| `@suaveplan/events` (`workspace:^`) | Event bus for failover events |
|
|
| `@suaveplan/logger` (`workspace:^`) | Structured logging |
|
|
| `@suaveplan/retry` (`workspace:^`) | Retry wired into health checks and recovery probes |
|
|
| `@suaveplan/timing` (`workspace:^`) | Interval timer for health checks |
|
|
| `zod` (`catalog:forms`) | Zod schemas |
|
|
| `@opentelemetry/api` (`catalog:observability`) | Spans + counters on failover |
|
|
| `tslib` (`catalog:build`) | Runtime helpers |
|
|
|
|
Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check` (all `catalog:testing`).
|
|
|
|
---
|
|
|
|
## Public API surface
|
|
|
|
### Core
|
|
|
|
```ts
|
|
function createFailoverGroup(options: FailoverGroupOptions): FailoverGroup
|
|
|
|
interface FailoverGroup {
|
|
readonly primary: FailoverTarget | null
|
|
readonly targets: ReadonlyArray<FailoverTarget>
|
|
readonly state: FailoverState
|
|
addTarget(target: FailoverTarget): void
|
|
removeTarget(targetId: string): void
|
|
execute<T>(fn: (target: FailoverTarget, signal?: AbortSignal) => Promise<T>, signal?: AbortSignal): Promise<T>
|
|
probe(targetId: string, signal?: AbortSignal): Promise<HealthStatus>
|
|
on(event: FailoverEvent, handler: (data: FailoverEventData) => void): () => void
|
|
}
|
|
```
|
|
|
|
### Target types
|
|
|
|
```ts
|
|
interface FailoverTarget {
|
|
id: string
|
|
address: string
|
|
port?: number
|
|
weight?: number
|
|
priority?: number
|
|
tags?: string[]
|
|
healthCheck?: HealthCheckConfig
|
|
}
|
|
|
|
interface HealthCheckConfig {
|
|
intervalMs?: number
|
|
timeoutMs?: number
|
|
successThreshold?: number
|
|
failureThreshold?: number
|
|
probe?: (target: FailoverTarget, signal?: AbortSignal) => Promise<boolean>
|
|
}
|
|
|
|
type HealthStatus = {
|
|
healthy: boolean
|
|
latencyMs?: number
|
|
checkedAt: number
|
|
consecutiveFailures: number
|
|
consecutiveSuccesses: number
|
|
}
|
|
```
|
|
|
|
### Failover types
|
|
|
|
```ts
|
|
type FailoverState = 'healthy' | 'failing-over' | 'degraded' | 'recovering' | 'no-targets'
|
|
type FailoverStrategy = 'round-robin' | 'fastest' | 'weighted' | 'priority'
|
|
type FailoverEvent = 'failover' | 'recovery' | 'target-failed' | 'target-recovered' | 'state-change'
|
|
|
|
interface FailoverEventData {
|
|
previousPrimary?: FailoverTarget
|
|
newPrimary?: FailoverTarget
|
|
target?: FailoverTarget
|
|
reason?: string
|
|
}
|
|
|
|
interface FailoverGroupOptions {
|
|
strategy?: FailoverStrategy
|
|
failoverDelayMs?: number
|
|
failBackDelayMs?: number
|
|
maxRetries?: number
|
|
logger?: LoggerLike
|
|
clock?: Clock
|
|
eventBus?: EventBusLike
|
|
}
|
|
```
|
|
|
|
### Error codes
|
|
|
|
```ts
|
|
type NetworkFailoverErrorCode =
|
|
| 'FAILOVER_NO_TARGETS'
|
|
| 'FAILOVER_ALL_UNHEALTHY'
|
|
| 'FAILOVER_EXECUTE_FAILED'
|
|
| 'FAILOVER_PROBE_FAILED'
|
|
| 'FAILOVER_ABORT'
|
|
|
|
function registerNetworkFailoverErrorCodes(): void
|
|
```
|
|
|
|
---
|
|
|
|
## Genesis additions
|
|
|
|
1. **Structured `NetworkFailoverError`** with stable `FAILOVER.*` codes
|
|
2. **Zod schemas** for `FailoverGroupOptions`, `FailoverTarget`, `HealthCheckConfig`
|
|
3. **Four failover strategies**: round-robin, fastest (lowest latency), weighted, priority
|
|
4. **Automatic health checking** via injectable interval timer and `@suaveplan/timing`
|
|
5. **Fail-back with delay** — prevents oscillation; configurable `failBackDelayMs`
|
|
6. **In-flight request retry** on failover — `execute()` replays on new primary with `AbortSignal`
|
|
7. **Injectable `Clock`** for health-check interval testing
|
|
8. **OTel instrumentation** — span `failover.execute`, counter `failover.events_total`, histogram `failover.latency_ms`
|
|
9. **Property-based tests** — failover always selects a healthy target when available; strategy returns consistent ordering
|
|
|
|
---
|
|
|
|
## File layout
|
|
|
|
```text
|
|
universal/network/network-failover/
|
|
package.json
|
|
tsconfig.json
|
|
vitest.config.ts
|
|
src/
|
|
index.ts
|
|
error-codes.ts
|
|
constants/
|
|
index.ts
|
|
types/
|
|
common.ts
|
|
common.md
|
|
core/
|
|
failover-group/
|
|
failover-group.ts
|
|
failover-group.md
|
|
failover-group.types.ts
|
|
failover-group.test.ts
|
|
index.ts
|
|
health-checker/
|
|
health-checker.ts
|
|
health-checker.md
|
|
health-checker.test.ts
|
|
index.ts
|
|
strategies/
|
|
round-robin.ts
|
|
round-robin.md
|
|
round-robin.test.ts
|
|
fastest.ts
|
|
fastest.md
|
|
fastest.test.ts
|
|
weighted.ts
|
|
weighted.md
|
|
weighted.test.ts
|
|
priority.ts
|
|
priority.md
|
|
priority.test.ts
|
|
tests/
|
|
integration.test.ts
|
|
property.test.ts
|
|
```
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
**Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/logger`, `@suaveplan/retry`, `@suaveplan/timing` must be published.
|