suaveplan/openspec/archive/2026-05-17-add-storage-uri-plugin-fallback/design.md

Design — @suaveplan/storage-uri-plugin-fallback

Non-normative companion to proposal.md and specs/storage-uri-plugin-fallback/spec.md.

1. URI grammar — decision matrix

The fallback:// scheme must encode an ordered list of embedded URIs, each of which is itself a complete URI (so contains :// and arbitrary reserved characters). Four candidate grammars were evaluated:

#	Form	Example	Pros	Cons
1	Percent-encoded comma-separated authority	fallback://https%3A%2F%2Fcdn.example.com%2Ffoo,bundle%3A%2F%2Fpkg%2Ffoo	Pure URI shape.	Authority section is not designed for , separators per RFC 3986 §3.2; hard to read; hard to author; awkward when embedded URIs carry their own query strings.
2	Opaque body after fallback: (no //), pipe-delimited	fallback:https://cdn.example.com/foo|bundle://pkg/foo	Readable.	Not a hierarchical URI shape; | is not URI-safe; needs a custom parser; pipes inside any candidate path break the parse.
3	Query-param form with empty authority/path	fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo	Natural URL handling via new URL / URLSearchParams; encoding done by stdlib; any number of candidates; each candidate is a single query value with stdlib-validated escaping; round-trips through any URL parser unchanged.	URLs can be long (rare in practice; cap at 8 candidates × 4096 chars).
4	List object passed via factory + symbolic short URI	fallback://my-list-id, list registered out-of-band	Shortest URI.	Requires out-of-band registration; loses the property that the URI is self-describing; not portable between processes.

Decision: Option 3 (query-param form).

Rationale:

• The Web URL standard (URL constructor, URLSearchParams) parses and serialises it correctly with zero custom code; every JavaScript runtime (browser, Bun, Node, Workers) handles encoding identically.
• Each candidate is one ?u= value; URLSearchParams.getAll("u") returns the ordered list in O(n) without ambiguity.
• Order is preserved because URLSearchParams is ordered.
• Encoding rules are RFC 3986-compliant out of the box; no edge case for ?, &, #, /, : inside candidates.
• The URI is self-describing — no out-of-band registry; the URI string IS the manifest.
• Length is bounded but ample: policy.maxCandidates (default 16, hard ceiling 64) × INVALID_URI threshold from @suaveplan/storage-uri (4096 chars per candidate) = ~64 KiB worst case, comfortably within any real-world URI handler.

The grammar formally:

fallback-uri  = "fallback:" "//" authority "/" "?" candidates [ "#" fragment ]
authority     = ""                    ; always empty
candidates    = "u=" pct-encoded-uri *( "&" "u=" pct-encoded-uri )
                ; ordered; at least 1, at most maxCandidates

Canonical examples:

fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo
fallback://?u=file%3A%2F%2F%2Ftmp%2Ffixtures%2Fa.json&u=https%3A%2F%2Ffixtures.test%2Fa.json
fallback://?u=cache%3A%2F%2Flocal%2Favatars%2Falice.jpg&u=cdn%3A%2F%2Fzone-eu%2Favatars%2Falice.jpg&u=drive%3A%2F%2Fprimary%2Favatars%2Falice.jpg

A helper buildFallbackUri(uris: readonly string[]): string is exported so authors never construct the URI by hand; it uses URLSearchParams and rejects empty / oversized inputs at the boundary.

2. File layout

src/
  index.ts                                  # explicit named re-exports (rule 10)
  error-codes/
    error-codes.ts                          # defineCodes("FALLBACK_URI", [...]) + FallbackUriError
    error-codes.md
    error-codes.test.ts                     # reachability sweep — every code triggered
  types/
    policy.types.ts                         # FallbackPolicy, FallbackPluginOptions
    policy.types.md
    policy.schemas.ts                       # Zod boundary schemas
    candidate.types.ts                      # CandidateError, CandidateAttempt
    candidate.types.md
  parse-uri/
    parse-fallback-uri.ts                   # parseFallbackUri + buildFallbackUri
    parse-fallback-uri.md
    parse-fallback-uri.test.ts
  classify/
    classify.ts                             # error → "transient" | "semantic" | "abort"
    classify.md
    classify.test.ts
  walk/
    sequential.ts                           # ordered fall-through walk
    sequential.md
    sequential.test.ts
    parallel.ts                             # Promise.any race; first-to-resolve wins
    parallel.md
    parallel.test.ts
  recursion/
    depth-context.ts                        # AsyncLocalStorage-equivalent depth + visited-set
    depth-context.md
    depth-context.test.ts
  ops/
    read.ts                                 # get / createReadStream / range
    read.md
    read.test.ts
    write.ts                                # set / createWriteStream / mirror fan-out
    write.md
    write.test.ts
    metadata.ts                             # exists / getMetadata / stat
    metadata.md
    metadata.test.ts
    delete.ts                               # mirror delete
    delete.md
    delete.test.ts
    list.ts                                 # union-with-dedup list
    list.md
    list.test.ts
  observability/
    events.ts                               # EventBusLike emit helpers
    events.md
    events.test.ts
    otel.ts                                 # counter / histogram / span hierarchy
    otel.md
    otel.test.ts
  plugin/
    plugin.ts                               # fallbackUriPlugin factory
    plugin.md
    plugin.test.ts
    plugin.spec-conformance.test.ts         # runUriPluginConformance pass
  conformance/
    fallback-conformance.ts                 # runFallbackConformance harness
    fallback-conformance.md
    fallback-conformance.test.ts

Every implementation file is ≤ 300 lines (250 soft); every test file ≤ 500 lines. The walk/, ops/, and recursion/ directories are the natural responsibility boundaries — splitting walk from ops keeps the candidate-iteration algorithm separate from the per-operation surface so each can be tested in isolation.

3. Recursion-prevention algorithm

fallback:// URIs can be nested (a candidate of a fallback:// may itself be a fallback://). Two invariants protect the process:

1. Depth cap — each plugin call increments a depth counter carried on a context token (DepthContext). Construction injects an AsyncLocalStorage-equivalent abstraction (DepthContext.run(state, fn)); under the hood the universal-tier default is a closure-passed state object (the plugin does not depend on node:async_hooks). Exceeding policy.maxDepth (default 4, hard ceiling 8) throws FALLBACK_URI.MAX_DEPTH_EXCEEDED.
2. Visited set — every recursion frame appends the original input URI (not the parsed candidate) to a visited: Set<string>. When a candidate URI normalises to a value already in visited, the walk skips that candidate and emits storage-uri.fallback.cycle-detected; if every candidate cycles, the walk throws FALLBACK_URI.CIRCULAR_REFERENCE with cycleChain: readonly string[] populated.

Normalisation: the visited-set key is the URI string after parseFallbackUri reserialisation (canonical query-param order, lowercased scheme, no fragment). This catches ?u=A&u=B vs ?u=B&u=A re-orderings of the same set.

Pseudocode:

function resolveCandidate(uri: string, ctx: DepthContext) {
  if (ctx.depth >= ctx.maxDepth) {
    throw new FallbackUriError("MAX_DEPTH_EXCEEDED", { uri, depth: ctx.depth });
  }
  const canonical = canonicalise(uri);
  if (ctx.visited.has(canonical)) {
    ctx.events?.emit("storage-uri.fallback.cycle-detected", { uri: canonical });
    return { kind: "cycle" } as const;
  }
  return ctx.resolver.dispatch(
    uri,
    { ...ctx, depth: ctx.depth + 1, visited: new Set([...ctx.visited, canonical]) }
  );
}

4. Write-mirror semantics

By default, writes mirror to all candidates so subsequent reads see consistent content regardless of which candidate wins the race. Trade-offs documented:

writeMode	Behaviour	Use case
"all" (default)	Promise.allSettled over every candidate; any rejection throws FALLBACK_URI.WRITE_MIRROR_FAILED with mirrorOutcomes: readonly { uri, status, error? }[].	CDN + origin pairing where every mirror must stay current.
"first-success"	Walk in order; first success returns; pending writes are allowed to settle in the background (logged via events but not awaited).	Caching tiers where the upstream origin is authoritative and lower tiers are best-effort populated.

Per-candidate write failures preserve the underlying SuaveplanError via mirrorOutcomes[i].error; non-SuaveplanError throwables are wrapped as FALLBACK_URI.ALL_CANDIDATES_FAILED-style sub-errors via the resolver's normal STORAGE_URI.ADAPTER_ERROR path (the plugin trusts the resolver's wrap rules; it never re-wraps a namespaced error).

When writeMode === "all" and all candidates fail, the plugin throws FALLBACK_URI.WRITE_MIRROR_FAILED (not ALL_CANDIDATES_FAILED) so callers can pattern-match on intent — WRITE_MIRROR_FAILED is "the operation was a mirror; some leg failed"; ALL_CANDIDATES_FAILED is "the operation was a walk; every leg failed."

delete follows the same matrix with one variation: success is defined as "at least one candidate deleted" by default (matches StorageLike.delete's boolean return semantics — any consistent mirror has been removed). Opt in to strict deletion with policy.deleteMode === "all" for parity with writes.

5. List-union dedup

list() collects results from every candidate, then deduplicates by canonical URI string:

const seen = new Set<string>();
const merged: string[] = [];
for (const result of perCandidateResults) {
  for (const uri of result.uris) {
    if (seen.has(uri)) continue;
    seen.add(uri);
    merged.push(uri);
  }
}

Order is stable across candidates: first-occurrence wins. hasMore is true if any candidate reports hasMore: true. continuationToken is intentionally NOT propagated by default — the plugin returns continuationToken: undefined because tokens are opaque per-adapter and cannot be merged. Callers needing paginated listing across mirrors must drive each candidate's list themselves; the plugin documents this in ops/list.md.

Under listMode: "concat" (opt-in), the plugin returns the raw concatenation without dedup — useful when callers want to count per-candidate hits.

When per-candidate list calls partially fail, the plugin throws FALLBACK_URI.LIST_UNION_FAILED carrying the per-candidate outcomes; the policy may relax this to "emit warning event, return partial union" via policy.listFailureMode: "throw" | "partial".

6. Complementarity with @suaveplan/storage-mux

storage-mux and storage-uri-plugin-fallback solve adjacent problems at different layers:

Concern	storage-mux	storage-uri-plugin-fallback
Layer	AbstractStorage (adapter)	URI scheme
Input	Pre-constructed StorageLike instances + policy	URI string with embedded candidate URIs
Backend identity	BackendEntry.name	URI string
Cross-scheme support	All backends must satisfy StorageLike; the mux does not know about schemes	Every candidate may use a different scheme (https, drive, bundle, cache, …)
Composability	Recursive (a mux is a StorageLike)	Recursive (a fallback:// candidate may itself be fallback://)
Policy surface	Six modes (failover, round-robin, weighted, sticky, shield, mirror)	One mode (sequential walk with optional race), simpler
State	Per-backend health, weights, drain	Stateless per call
Observability	Per-backend span hierarchy + stats	Per-candidate span hierarchy + per-candidate counter

When to choose which:

• If the calling code holds StorageLike instances directly (the storage layer is a private composition concern), use storage-mux.
• If the calling code receives URIs from manifests / config / wire formats and needs failover without changing its interface, use fallback://.
• They can be combined: a fallback:// candidate may be a URI that ultimately resolves through a storage-mux-backed plugin, and vice versa.

The two packages share a classification table (design.md §7) so transient/semantic/abort semantics are uniform across the stack — copy-paste sync between storage-mux/src/classify and storage-uri-plugin-fallback/src/classify is enforced by a per-PR check in CI.

7. Error classification table

Mirrors storage-mux design §4 for cross-package consistency:

Error shape	Class	Triggers fall-through?	Code surfaced
AbortError / signal.aborted from caller	abort	NO (stop walk)	FALLBACK_URI.ABORTED
STORAGE_URI.NOT_FOUND	semantic-but-fall-through (policy)	YES (configurable; default YES)	wrapped, last in cause chain on exhaustion
STORAGE_URI.INVALID_URI / STORAGE_URI.UNKNOWN_SCHEME	semantic	NO (rethrow — caller bug)	rethrow
STORAGE_URI.WRITE_UNSUPPORTED	semantic	NO (rethrow)	rethrow
Per-candidate STORAGE_URI.ADAPTER_ERROR	transient	YES	last in cause chain
Per-candidate STORAGE_URI.ABORTED from inner timeout	transient	YES (if stopOnAbort: false)	wrapped
Unknown thrown value (non-Error)	transient	YES	wrapped (resolver normalises first)
FALLBACK_URI.MAX_DEPTH_EXCEEDED	semantic	NO	rethrow (programmer bug)
FALLBACK_URI.CIRCULAR_REFERENCE	semantic	NO	rethrow

policy.errorClass: "any" re-classifies all semantic errors as fall-through-eligible; the default ("transient" + NOT_FOUND) is the safer choice because it preserves the resolver's caller-bug surfaces (INVALID_URI, UNKNOWN_SCHEME) without forcing every consumer to opt out.

8. Error-code matrix

Code	Thrown from	Discriminants populated
FALLBACK_URI.NO_CANDIDATES	parseFallbackUri when ?u= count is 0	uri
FALLBACK_URI.INVALID_URI	parseFallbackUri on malformed URI (missing scheme, non-empty authority, > 4096 chars total)	uri
FALLBACK_URI.ALL_CANDIDATES_FAILED	walk/sequential.ts, walk/parallel.ts on exhaustion (reads)	uri, candidateErrors, cause (last error)
FALLBACK_URI.WRITE_MIRROR_FAILED	ops/write.ts under writeMode: "all" when any candidate rejects	uri, mirrorOutcomes
FALLBACK_URI.LIST_UNION_FAILED	ops/list.ts under listFailureMode: "throw" when any candidate rejects	uri, candidateErrors
FALLBACK_URI.ABORTED	walk entry when signal.aborted === true, or mid-walk when stopOnAbort: true	uri
FALLBACK_URI.MAX_DEPTH_EXCEEDED	recursion/depth-context.ts when ctx.depth >= ctx.maxDepth	uri, depth
FALLBACK_URI.CIRCULAR_REFERENCE	walk/*.ts when every candidate has already been visited	uri, cycleChain

Every code is reachable from a real call site exercised by src/error-codes/error-codes.test.ts (the reachability sweep).

9. Tier purity

Universal — no node:*, no bun:*, no DOM. Specifically:

• URL and URLSearchParams are Web standards (available in every runtime).
• AbortSignal.any is ES2024; a tiny polyfill is provided under src/parse-uri/abort-any.ts for runtimes that lack it (same approach as storage-mux).
• Depth context uses closure passing (DepthContext value threaded through call options) rather than node:async_hooks. This is sufficient because the plugin's recursion is all within a single call chain — there is no cross-await-boundary state to track that the caller doesn't already own.
• Clock is injected via @suaveplan/types#Clock for any time-keeping (latency histograms only); the plugin has no scheduled work of its own.

10. Trade-offs

• Query-param URI grammar vs. shorter forms. Option 3 trades a few characters of length for zero custom parsing and standards alignment. Worth it; the URIs are not user-facing.
• Mirror writes by default. Cheaper to mirror up-front than to discover a stale mirror on the next read. Callers that explicitly want first-success-wins opt in via writeMode: "first-success".
• List dedup by canonical URI. Cannot detect content drift between mirrors; callers needing byte-equality must use a content-addressed scheme. Acceptable because content-verification is out of scope (proposal §"Out of scope").
• Depth cap 4 default, 8 hard ceiling. Most use cases are 2–3 levels (cache → CDN → origin). Eight covers pathological compositions without enabling DoS via runaway recursion.
• Stateless plugin. Unlike storage-mux, there is no per-candidate health tracking. The fallback walk is one attempt per candidate per call; if you need backoff/health, wrap each candidate's underlying plugin in @suaveplan/retry or stack a storage-mux underneath. Keeps the plugin's surface small and the semantics auditable.
• Cycle detection by URI string. Cheap and correct for the common case; cannot catch a cycle that goes through a non-fallback:// indirection (e.g. an HTTP redirect chain). That's not the plugin's problem to solve.