SuavePlan/suaveplan

Fork 0

Files

admin fbc70f96e6 Initial Commit

2026-05-27 23:29:53 +01:00

31 KiB

Raw Permalink Blame History

Capability Spec Delta: storage-mux (storage-mux)

Package: @suaveplan/storage-mux Status: PROPOSED

Every requirement below carries a **Verified by:** line at authoring time. No exceptions.

ADDED Requirements

Requirement: StorageMux extends AbstractStorage

The package SHALL provide a StorageMux class extending AbstractStorage from @suaveplan/storage. The constructor SHALL accept StorageMuxOptions containing an ordered non-empty backends: BackendEntry[] and a mode: MuxMode. The class SHALL implement every method of the StorageLike surface (get, set, setPartial, has, delete, list, stat, getStream, setStream) by delegating to a BackendSelector chosen from the configured mode. The class SHALL expose capabilities reflecting the intersection of backend capabilities.

Scenario: subclass is an instance of AbstractStorage

WHEN a StorageMux is constructed with at least one BackendEntry
THEN the instance is instanceof AbstractStorage AND exposes every StorageLike method

Scenario: empty backend list is rejected at construction

WHEN a caller invokes new StorageMux({ backends: [], mode: "failover" })
THEN the constructor throws StorageMuxError with code === "STORAGE_MUX.NO_BACKENDS_CONFIGURED"

Scenario: duplicate backend names rejected

WHEN two BackendEntry values share the same name
THEN the constructor throws StorageMuxError with code === "STORAGE_MUX.BACKEND_NAME_CONFLICT"

Verified by: src/mux/mux.test.ts::"is instance of AbstractStorage and exposes StorageLike surface" AND src/mux/mux.test.ts::"rejects empty backend list with NO_BACKENDS_CONFIGURED" AND src/mux/mux.test.ts::"rejects duplicate backend names with BACKEND_NAME_CONFLICT"

Requirement: MuxMode discriminated union with six modes

The package SHALL declare MuxMode as the string union "failover" | "round-robin" | "weighted" | "sticky-by-key" | "shield" | "mirror". Each mode SHALL be implemented as a BackendSelector strategy class in its own subdirectory under src/modes/. The StorageMuxOptions.mode field SHALL be validated by a Zod schema that rejects any value outside this whitelist with STORAGE_MUX.INVALID_MODE.

Scenario: unknown mode string is rejected

WHEN a caller invokes new StorageMux({ backends, mode: "magic" as MuxMode })
THEN the constructor throws StorageMuxError with code === "STORAGE_MUX.INVALID_MODE"

Verified by: src/mux/mux.test.ts::"rejects unknown mode with INVALID_MODE"

Requirement: failover mode falls through on transient errors and surfaces semantic errors

Under failover the mux SHALL invoke backends[0] first. On a transient classified error (per design.md §4) the mux SHALL advance to backends[1], then backends[2], and so on. On a semantic error (STORAGE.KEY_NOT_FOUND, STORAGE.INVALID_KEY, 4xx other than 408/429, CREDENTIAL_EXPIRED) the mux SHALL surface the error directly without advancing. When every backend has been tried and the last attempt was transient, the mux SHALL throw StorageMuxError with code === "STORAGE_MUX.ALL_BACKENDS_DOWN", the prior error as cause, and attemptCount equal to the number of backends tried.

Scenario: transient error advances to next backend

WHEN backends[0].get(key) rejects with a 503 and backends[1].get(key) resolves with bytes
THEN the mux returns backends[1]'s bytes AND emits a storage-mux.failover event naming both backends

Scenario: semantic KEY_NOT_FOUND is not retried across backends

WHEN backends[0].get(key) throws STORAGE.KEY_NOT_FOUND
THEN the mux rethrows KEY_NOT_FOUND without invoking backends[1]

Scenario: exhausting all backends throws ALL_BACKENDS_DOWN with cause chain

WHEN every backend rejects with a transient error
THEN the mux throws StorageMuxError with code === "STORAGE_MUX.ALL_BACKENDS_DOWN" AND cause is the last backend's error AND attemptCount === backends.length

Verified by: src/modes/failover/failover.test.ts::"falls through to next backend on transient 503" AND src/modes/failover/failover.test.ts::"surfaces semantic KEY_NOT_FOUND without fall-through" AND src/modes/failover/failover.test.ts::"throws ALL_BACKENDS_DOWN after exhausting list"

Requirement: round-robin mode distributes across healthy backends

Under round-robin the mux SHALL maintain a monotonic counter and select healthyBackends[counter mod healthyBackends.length] for each call. Backends marked down or draining by BackendHealth SHALL be skipped. When zero backends are healthy the mux SHALL throw STORAGE_MUX.ALL_BACKENDS_DOWN.

Scenario: nine calls across three healthy backends distribute evenly

WHEN the caller issues 9 get calls against a mux with 3 healthy backends
THEN each backend receives exactly 3 calls

Scenario: down backend is skipped

WHEN backends[1] is marked down and the caller issues 4 calls
THEN backends[0] and backends[2] each receive 2 calls AND backends[1] receives 0

Verified by: src/modes/round-robin/round-robin.test.ts::"distributes nine calls evenly across three backends" AND src/modes/round-robin/round-robin.test.ts::"skips backends marked down"

Requirement: weighted mode selects via Walker alias-method

Under weighted the mux SHALL build a Walker alias-method table over BackendEntry.weight values and select a backend in O(1) per call via two random draws (slot + alias coin). The mux SHALL rebuild the alias table on addBackend, removeBackend, setWeight, and any health transition. Construction SHALL throw STORAGE_MUX.WEIGHTED_REQUIRES_WEIGHTS when mode === "weighted" AND any BackendEntry.weight is undefined. The mux SHALL throw STORAGE_MUX.INVALID_WEIGHT when any weight < 0 or non-finite. On a transient error the mux SHALL re-roll up to healthy.length times before throwing ALL_BACKENDS_DOWN.

Scenario: weights 1/3/6 distribute roughly 10/30/60 percent over 1000 calls

WHEN 1000 calls are issued under a Rng seeded for determinism with weights [1, 3, 6]
THEN each backend's call count is within ±5% of its expected share (100/300/600)

Scenario: weighted without weights rejects at construction

WHEN a caller invokes new StorageMux({ backends: [{ name: "a", storage }, { name: "b", storage }], mode: "weighted" })
THEN the constructor throws STORAGE_MUX.WEIGHTED_REQUIRES_WEIGHTS

Scenario: negative weight rejected

WHEN any backend's weight is -1
THEN the constructor throws STORAGE_MUX.INVALID_WEIGHT

Verified by: src/modes/weighted/weighted.test.ts::"distributes calls roughly per weight under seeded rng" AND src/modes/weighted/weighted.test.ts::"rejects mode=weighted with any undefined weight as WEIGHTED_REQUIRES_WEIGHTS" AND src/modes/weighted/weighted.test.ts::"rejects negative weight with INVALID_WEIGHT"

Requirement: sticky-by-key mode is deterministic and skips down backends

Under sticky-by-key the mux SHALL compute index = fnv1a(utf8(key)) mod healthyBackends.length and route every operation for that key to the same backend. The hash SHALL be pure-JS FNV-1a (no native or crypto dep). When a backend goes down its keys SHALL be redistributed across the remaining healthy backends via the same formula recomputed against the shrunken set. Operations without a key (e.g. list) SHALL fall back to round-robin selection.

Scenario: same key resolves to same backend across 100 calls

WHEN the caller issues 100 get("alpha") calls against a mux with 3 healthy backends
THEN exactly one backend receives all 100 calls

Scenario: removed backend triggers redistribution

WHEN the backend currently serving "alpha" is marked down
THEN subsequent get("alpha") calls land on one of the remaining healthy backends AND the choice is stable until the health set changes again

Verified by: src/modes/sticky-by-key/sticky-by-key.test.ts::"routes the same key to the same backend across 100 calls" AND src/modes/sticky-by-key/sticky-by-key.test.ts::"redistributes keys when a backend goes down"

Requirement: shield mode cascades reads and fans out writes with origin priority

Under shield the mux SHALL treat backends[0] as the origin and backends[1..n] as mirrors. Construction SHALL throw STORAGE_MUX.MIRROR_ORIGIN_REQUIRED when fewer than two backends are configured. For read operations (get, has, stat, getStream) the mux SHALL invoke backends[0] first and on transient error or timeout fall through to mirrors in order. For write operations (set, setPartial, delete, setStream) the mux SHALL invoke backends[0] AND every mirror via Promise.allSettled. If the origin write rejects the mux SHALL throw with the origin error as cause. If the origin write succeeds but one or more mirror writes reject the mux SHALL emit storage-mux.fanout-partial carrying every failed mirror name AND return successfully — unless StorageMuxOptions.throwOnFanoutPartial === true, in which case the mux SHALL additionally throw StorageMuxError with code === "STORAGE_MUX.WRITE_FANOUT_PARTIAL".

Scenario: shield with single backend rejected at construction

WHEN new StorageMux({ backends: [origin], mode: "shield" })
THEN the constructor throws STORAGE_MUX.MIRROR_ORIGIN_REQUIRED

Scenario: read falls through to mirror after origin timeout

WHEN backends[0].get(key) hangs past backendTimeoutMs and backends[1].get(key) resolves with bytes
THEN the mux returns backends[1]'s bytes

Scenario: write succeeds when origin accepts and one mirror rejects (default)

WHEN backends[0].set resolves and backends[1].set rejects with a transient error
THEN the mux returns successfully AND emits storage-mux.fanout-partial naming backends[1]

Scenario: write throws WRITE_FANOUT_PARTIAL when caller opts in

WHEN throwOnFanoutPartial: true AND origin succeeds while a mirror rejects
THEN the mux throws StorageMuxError with code === "STORAGE_MUX.WRITE_FANOUT_PARTIAL"

Scenario: write fails when origin rejects regardless of mirror outcomes

WHEN backends[0].set rejects with any error
THEN the mux throws AND cause is the origin error AND mirror outcomes are not consulted

Verified by: src/modes/shield/shield.test.ts::"rejects single-backend shield with MIRROR_ORIGIN_REQUIRED" AND src/modes/shield/shield.test.ts::"reads fall through to mirror after origin timeout" AND src/modes/shield/shield.test.ts::"write succeeds when origin accepts and one mirror rejects" AND src/modes/shield/shield.test.ts::"write throws WRITE_FANOUT_PARTIAL when opt-in flag is set" AND src/modes/shield/shield.test.ts::"write fails when origin rejects regardless of mirror outcomes"

Requirement: mirror mode races reads and fans out writes across symmetric peers

Under mirror every backend is a symmetric peer — no designated origin. Read operations SHALL race healthy backends via Promise.any; the first successful result wins; all rejections SHALL produce STORAGE_MUX.ALL_BACKENDS_DOWN (cause = AggregateError of underlying errors). Write operations SHALL invoke every healthy backend via Promise.allSettled; any single rejection SHALL emit storage-mux.fanout-partial; all rejections SHALL throw STORAGE_MUX.ALL_BACKENDS_DOWN. The MIRROR_ORIGIN_REQUIRED code SHALL NOT be thrown in this mode.

Scenario: read returns first successful peer

WHEN backends[0].get resolves at 30ms and backends[1].get resolves at 5ms
THEN the mux returns backends[1]'s value (no wait for backends[0])

Scenario: write fans out to all peers and emits partial when one rejects

WHEN writes to backends 0 and 1 succeed but backend 2 rejects
THEN the mux returns successfully AND emits storage-mux.fanout-partial naming backends[2]

Scenario: all peers reject a read throws ALL_BACKENDS_DOWN

WHEN every peer rejects a get
THEN the mux throws STORAGE_MUX.ALL_BACKENDS_DOWN AND cause instanceof AggregateError

Verified by: src/modes/mirror/mirror.test.ts::"read returns first successful peer" AND src/modes/mirror/mirror.test.ts::"write fans out and emits partial on single rejection" AND src/modes/mirror/mirror.test.ts::"all peers rejecting read throws ALL_BACKENDS_DOWN with AggregateError cause"

Requirement: bulkhead — per-backend timeout via Promise.race

Every backend invocation SHALL be wrapped in Promise.race against a Clock-driven timeout of backendTimeoutMs (default 30000, configurable per-mux). When the race rejects due to timeout the mux SHALL classify the timeout as transient AND proceed per mode policy. The underlying backend promise MAY remain pending; the mux SHALL NOT await it. A backend whose timeout fires consecutively without recovery SHALL surface STORAGE_MUX.BACKEND_TIMEOUT on exhaustion.

Scenario: hung primary does not delay fallback beyond backendTimeoutMs

WHEN backends[0].get returns a never-resolving promise and backends[1].get resolves in 10ms
THEN the caller receives backends[1]'s value within backendTimeoutMs + 50ms measured against the injected FakeClock

Scenario: exhausted timeouts surface BACKEND_TIMEOUT

WHEN every backend hangs past backendTimeoutMs
THEN the mux throws STORAGE_MUX.ALL_BACKENDS_DOWN AND cause.code === "STORAGE_MUX.BACKEND_TIMEOUT"

Verified by: src/bulkhead/bulkhead.test.ts::"hung primary does not block fallback past backendTimeoutMs" AND src/bulkhead/bulkhead.test.ts::"exhausted timeouts surface BACKEND_TIMEOUT in cause chain"

Requirement: AbortSignal.any composition surfaces BACKEND_ABORTED

The per-attempt signal forwarded to each backend SHALL be composed via AbortSignal.any([userSignal, timeoutSignal, drainSignal]). When the caller's AbortSignal aborts during a mux operation, the mux SHALL surface a StorageMuxError with code === "STORAGE_MUX.BACKEND_ABORTED" rather than falling through to subsequent backends. The mux SHALL NOT classify a caller-driven abort as a transient backend failure.

Scenario: caller abort surfaces BACKEND_ABORTED and does not advance

WHEN the caller passes a pre-aborted signal to mux.get(key, signal)
THEN the mux throws StorageMuxError with code === "STORAGE_MUX.BACKEND_ABORTED" AND backends[1].get is NEVER invoked

Scenario: composed signal aborts backend mid-flight on user cancellation

WHEN the user aborts signal 5ms after backends[0].get begins
THEN backends[0].get's received AbortSignal reports aborted === true within 1 tick

Verified by: src/mux/mux.test.ts::"caller abort surfaces BACKEND_ABORTED without falling through" AND src/mux/mux.test.ts::"composed AbortSignal propagates user cancellation to in-flight backend"

Requirement: BackendHealth — passive degrade with capped exponential cooldown

The package SHALL maintain a BackendHealth tracker per backend. A backend SHALL transition from up to down on the first classified-transient error. The cooldown SHALL be min(cooldownMs * 2 ** min(consecutiveFailures, 6), maxCooldownMs) (default cooldownMs 1000, default maxCooldownMs 64000). A down backend SHALL be skipped by every selector. Once Clock.now() >= cooldownUntil the next operation SHALL probe the backend; success transitions it back to up and resets consecutiveFailures.

Scenario: failing backend cools down for 2000ms after second failure

WHEN a backend rejects twice in succession (default cooldownMs 1000)
THEN the backend is down AND remains skipped until FakeClock advances by 2000ms

Scenario: cooldown is capped at maxCooldownMs

WHEN a backend rejects 20 consecutive times with cooldownMs: 1000, maxCooldownMs: 5000
THEN the cooldown never exceeds 5000ms

Scenario: successful probe after cooldown restores backend and emits recovered event

WHEN FakeClock advances past cooldownUntil AND the next probe succeeds
THEN the backend is up AND a storage-mux.backend.recovered event is emitted AND consecutiveFailures is reset to 0

Verified by: src/health/health.test.ts::"exponential cooldown after consecutive failures" AND src/health/health.test.ts::"cooldown capped at maxCooldownMs" AND src/health/health.test.ts::"successful probe after cooldown restores backend and emits recovered"

Requirement: Active healthchecks via injected Clock

When BackendEntry.healthcheck is provided AND StorageMuxOptions.healthcheckIntervalMs is set, the mux SHALL schedule the healthcheck via the injected Clock.setInterval and invoke it with a derived AbortSignal. A rejecting healthcheck SHALL mark the backend down AND emit storage-mux.backend.down carrying STORAGE_MUX.HEALTHCHECK_FAILED as the cause. The mux SHALL clear all schedules when disposed.

Scenario: rejecting active healthcheck marks backend down

WHEN an active healthcheck rejects with a network error
THEN the backend transitions to down AND a storage-mux.backend.down event is emitted AND the failure cause is STORAGE_MUX.HEALTHCHECK_FAILED

Scenario: dispose cancels scheduled healthchecks

WHEN mux[Symbol.asyncDispose]() is invoked
THEN FakeClock advancing past the next interval does NOT invoke the healthcheck

Verified by: src/health/health.test.ts::"rejecting active healthcheck marks backend down with HEALTHCHECK_FAILED" AND src/health/health.test.ts::"dispose cancels scheduled healthchecks"

Requirement: Retry composition via @suaveplan/retry — no inline retry

Every per-backend invocation SHALL be wrapped by runWithRetry(retryStrategy, ...) from @suaveplan/retry using the strategy supplied in StorageMuxOptions.retry (defaulting to exponentialBackoff() from @suaveplan/retry). The mux SHALL NOT reimplement backoff, jitter, or attempt counting; all in-backend retry behavior MUST be expressed by composing the retry strategy. Cross-backend fall-through is policy logic; in-backend retry is runWithRetry.

Scenario: backend that fails twice then succeeds is retried in place before fall-through

WHEN backends[0].get rejects twice then resolves on the third attempt under a 3-attempt retry strategy
THEN the mux returns backends[0]'s value AND backends[1].get is NEVER invoked

Scenario: no inline retry implementation in package

WHEN the package source is grepped for backoff, setTimeout, or retryCount outside src/health/
THEN the only references are imports from @suaveplan/retry

Verified by: src/modes/failover/failover.test.ts::"retries in place before falling through to next backend" AND src/modes/failover/failover.test.ts::"package contains no inline retry implementation outside health"

Requirement: Backend isolation — one backend's failure cannot corrupt mux state

A throw, hang, or rejection inside one backend SHALL NOT leave shared mux state (BackendHealth table, round-robin counter, weighted alias table, stats counters) in a partially-mutated state observable to any other backend. State updates SHALL be made through dedicated keyed helpers; no helper SHALL mutate state for backend B while handling an error from backend A.

Scenario: throw from backend A leaves backend B health unchanged

WHEN backends[0].get throws a transient error and backends[1] has never been called
THEN mux.stats().backends[1] shows status === "up", consecutiveFailures === 0, errors === 0

Scenario: concurrent in-flight ops on different backends do not interleave state writes

WHEN 100 concurrent get calls fan out over 4 round-robin backends with one of them throwing
THEN mux.stats().backends totals exactly equal the issued call count (no lost or double-counted attempts)

Verified by: src/mux/mux.test.ts::"throw from one backend leaves siblings health untouched" AND src/mux/mux.test.ts::"concurrent calls across backends produce consistent stats totals"

Requirement: Recursive composition — StorageMux acceptable as a backend

A StorageMux SHALL itself satisfy StorageLike AND be acceptable as a BackendEntry.storage. Configurations like Mux(failover, [Mux(mirror, [...]), Mux(mirror, [...])]) SHALL function correctly. Recursion depth SHALL be capped at 8; constructing a mux whose backend tree exceeds the cap SHALL throw STORAGE_MUX.RECURSION_DEPTH_EXCEEDED.

Scenario: two-level recursive mux passes basic round-trip

WHEN a Mux(failover, [Mux(mirror, [a, b]), Mux(mirror, [c, d])]) is constructed and a value is set then read
THEN the round-trip succeeds AND OTel spans show a parent-child hierarchy with both mux levels named

Scenario: deeply nested mux beyond depth 8 throws RECURSION_DEPTH_EXCEEDED

WHEN a caller constructs a mux tree 9 levels deep
THEN the outermost constructor throws STORAGE_MUX.RECURSION_DEPTH_EXCEEDED

Verified by: src/mux/mux.spec-conformance.test.ts::"two-level recursive mux passes round-trip and emits nested spans" AND src/mux/mux.test.ts::"nine-level recursive construction throws RECURSION_DEPTH_EXCEEDED"

Requirement: Runtime mutation API — add / remove / setWeight / drain

The mux SHALL expose addBackend(entry), removeBackend(name), setWeight(name, weight), and drain(name, options?) methods. Mutations SHALL be atomic relative to in-flight operations: in-flight requests targeting a draining backend complete normally; new requests skip it. addBackend SHALL throw BACKEND_NAME_CONFLICT on duplicate names. setWeight and removeBackend SHALL throw BACKEND_NOT_FOUND for unknown names. setWeight SHALL reject weight < 0 with INVALID_WEIGHT. Each mutation SHALL emit a corresponding event (backend.added, backend.removed, weight.changed).

Scenario: addBackend with duplicate name rejected

WHEN mux.addBackend({ name: "existing", storage }) is called for a name already present
THEN the call throws STORAGE_MUX.BACKEND_NAME_CONFLICT AND no event is emitted

Scenario: setWeight rebuilds alias table under weighted mode

WHEN the mode is weighted AND mux.setWeight("b", 10) doubles backend b's weight
THEN subsequent 1000 calls under the same seeded Rng show b's share roughly doubled

Verified by: src/mutate/mutate.test.ts::"addBackend rejects duplicate name with BACKEND_NAME_CONFLICT" AND src/mutate/mutate.test.ts::"setWeight rebuilds weighted alias and shifts distribution"

Requirement: Drain — graceful backend removal

mux.drain(name, { signal? }): Promise<void> SHALL mark the named backend as draining, remove it from new-selection eligibility, await all in-flight operations to complete, then transition the backend to down and remove from the active set. The returned promise SHALL resolve when in-flight count reaches zero (or rejects when the caller-supplied signal aborts). storage-mux.drain.started SHALL be emitted on transition; .drain.completed on resolution. A backend in draining status SHALL surface BACKEND_DRAINING to selectors as a re-selection signal (classified transient).

Scenario: drain awaits in-flight calls before completing

WHEN mux.drain("primary") is called while one get is in flight on primary
THEN the drain promise does NOT resolve until that get settles AND the storage-mux.drain.completed event carries a non-zero drainedDurationMs

Scenario: draining backend is skipped by new selections

WHEN mux.drain("primary") is in progress under round-robin
THEN new calls bypass primary AND distribute across remaining healthy backends

Verified by: src/mutate/mutate.test.ts::"drain awaits in-flight operations before completing" AND src/mutate/mutate.test.ts::"draining backend is skipped by new round-robin selections"

Requirement: Stats API — per-backend counters and sampled latencies

The mux SHALL expose mux.stats(): MuxStats returning a snapshot containing the configured mode, aggregate totals (totalCalls, totalFailovers, totalFanoutPartials), and per-backend records with name, status, hits, misses, errors, timeouts, lastLatencyMs, p50LatencyMs, p99LatencyMs, consecutiveFailures, lastTransitionAt, inFlightCount. Latency percentiles SHALL be computed from a 256-slot Vitter Algorithm R reservoir sample to bound memory.

Scenario: stats reflect a successful call and a failed call

WHEN one successful get and one failing get are issued against a single-backend mux
THEN mux.stats().backends[0].hits === 1 AND errors === 1 AND lastLatencyMs > 0

Scenario: reservoir bounds memory under high call volume

WHEN 100000 calls are issued
THEN internal latency-sample arrays never exceed 256 entries per backend

Verified by: src/stats/stats.test.ts::"reflects one success and one failure in counters" AND src/stats/stats.test.ts::"latency reservoir stays bounded at 256 samples under 100k calls"

Requirement: STORAGE_MUX error registry — all 14 codes reachable

The package SHALL declare STORAGE_MUX_CODES via defineCodes("STORAGE_MUX", [...]) with exactly these 14 codes: ALL_BACKENDS_DOWN, NO_BACKENDS_CONFIGURED, HEALTHCHECK_FAILED, BACKEND_TIMEOUT, INVALID_MODE, INVALID_WEIGHT, WRITE_FANOUT_PARTIAL, BACKEND_ABORTED, BACKEND_NOT_FOUND, BACKEND_NAME_CONFLICT, BACKEND_DRAINING, WEIGHTED_REQUIRES_WEIGHTS, MIRROR_ORIGIN_REQUIRED, RECURSION_DEPTH_EXCEEDED. The package SHALL declare StorageMuxError extends SuaveplanError<StorageMuxErrorCode> with optional backendName, attemptIndex, attemptCount, and mode discriminants. Every code SHALL be reachable from a real call site exercised by tests in this package.

Scenario: every declared code is reachable from a real call site

WHEN the test suite is run
THEN every code in STORAGE_MUX_CODES has at least one test that triggers the call site that throws or surfaces it

Verified by: src/error-codes/error-codes.test.ts::"every STORAGE_MUX code is reachable from a real call site"

Requirement: Observability — EventBusLike events on lifecycle transitions

The mux SHALL emit events through an injected EventBusLike (from @suaveplan/types) on the following lifecycle transitions: storage-mux.backend.down, storage-mux.backend.recovered, storage-mux.failover, storage-mux.fanout-partial, storage-mux.drain.started, storage-mux.drain.completed, storage-mux.backend.added, storage-mux.backend.removed, storage-mux.weight.changed. When no EventBusLike is provided the mux SHALL operate normally without emitting (silent fallback).

Scenario: failover emits storage-mux.failover with both backend names

WHEN the failover mode advances from backends[0] to backends[1] due to a 503
THEN exactly one storage-mux.failover event is emitted carrying both backend names AND the underlying error

Scenario: no bus configured does not throw

WHEN the mux is constructed without events and a failover occurs
THEN the operation completes successfully AND no error is thrown for the missing bus

Verified by: src/observability/events.test.ts::"failover emits storage-mux.failover with from and to backend names" AND src/observability/events.test.ts::"absent EventBus does not throw on lifecycle transitions"

Requirement: Observability — OTel parent + per-attempt child spans

Every public operation SHALL open a parent OTel span named storage-mux.<op> with attributes mux.mode, mux.key (when present). Every backend attempt SHALL open a child span named storage-mux.attempt carrying mux.backend.name, mux.attempt.index (0-based), mux.mode, mux.outcome ∈ {ok, transient, semantic, abort, timeout}, and mux.latency.ms. When no tracer (TracerLike from @suaveplan/types) is provided the mux SHALL operate normally without recording spans.

Scenario: failover attempt records two child spans under one parent

WHEN the failover mode invokes backends[0] (rejects) then backends[1] (succeeds)
THEN the test tracer records one parent storage-mux.get span AND two child storage-mux.attempt spans with mux.attempt.index 0 and 1 AND mux.outcome transient and ok

Verified by: src/observability/otel.test.ts::"records parent op span and one child span per backend attempt with outcome"

Requirement: StorageLike conformance under every mode and recursive composition

The mux SHALL pass runStorageConformance (from @suaveplan/storage) when configured with three MemoryStorage backends under each of the six modes (failover, round-robin, weighted, sticky-by-key, shield, mirror). The mux SHALL additionally pass conformance when configured recursively as Mux(failover, [Mux(mirror, mems), Mux(mirror, mems)]). The conformance suite SHALL include setPartial sparse-extension and overwrite semantics, capability-honest reporting, and AbortSignal propagation.

Scenario: conformance passes under all six modes

WHEN runStorageConformance(mux) runs six times — once per mode — against three MemoryStorage backends
THEN every conformance assertion passes for every mode

Scenario: conformance passes under recursive composition

WHEN runStorageConformance(recursiveMux) runs against the two-level configuration
THEN every conformance assertion passes

Verified by: src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under failover" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under round-robin" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under weighted" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under sticky-by-key" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under shield" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under mirror" AND src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under recursive failover-of-mirrors"

Requirement: Tier purity — universal, no Node or DOM imports

The package SHALL NOT import from node:*, bun:*, or any DOM-only API. Hashing SHALL be pure-JS (FNV-1a). Randomness SHALL come from an injectable Rng (@suaveplan/types). Timeouts SHALL use the injected Clock. The package SHALL run unmodified under Node 18+, Bun, Deno, modern browsers, and Cloudflare Workers.

Scenario: tier-purity grep finds no Node or DOM imports

WHEN grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-mux/src/ is run
THEN the output is empty

Verified by: src/mux/mux.spec-conformance.test.ts::"src tree contains no node: or bun: imports"

31 KiB Raw Permalink Blame History