Initial Commit
This commit is contained in:
@@ -0,0 +1,253 @@
|
||||
# Design — @suaveplan/storage-uri-plugin-fallback
|
||||
|
||||
Non-normative companion to `proposal.md` and `specs/storage-uri-plugin-fallback/spec.md`.
|
||||
|
||||
## 1. URI grammar — decision matrix
|
||||
|
||||
The `fallback://` scheme must encode an **ordered list of embedded URIs**, each of which is itself a complete URI (so contains `://` and arbitrary reserved characters). Four candidate grammars were evaluated:
|
||||
|
||||
| # | Form | Example | Pros | Cons |
|
||||
|---|---|---|---|---|
|
||||
| 1 | Percent-encoded comma-separated authority | `fallback://https%3A%2F%2Fcdn.example.com%2Ffoo,bundle%3A%2F%2Fpkg%2Ffoo` | Pure URI shape. | Authority section is not designed for `,` separators per RFC 3986 §3.2; hard to read; hard to author; awkward when embedded URIs carry their own query strings. |
|
||||
| 2 | Opaque body after `fallback:` (no `//`), pipe-delimited | `fallback:https://cdn.example.com/foo\|bundle://pkg/foo` | Readable. | Not a hierarchical URI shape; `\|` is not URI-safe; needs a custom parser; pipes inside any candidate path break the parse. |
|
||||
| 3 | **Query-param form with empty authority/path** | `fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo` | Natural URL handling via `new URL` / `URLSearchParams`; encoding done by stdlib; any number of candidates; each candidate is a single query value with stdlib-validated escaping; round-trips through any URL parser unchanged. | URLs can be long (rare in practice; cap at 8 candidates × 4096 chars). |
|
||||
| 4 | List object passed via factory + symbolic short URI | `fallback://my-list-id`, list registered out-of-band | Shortest URI. | Requires out-of-band registration; loses the property that the URI is self-describing; not portable between processes. |
|
||||
|
||||
**Decision: Option 3 (query-param form).**
|
||||
|
||||
Rationale:
|
||||
|
||||
- The Web URL standard (`URL` constructor, `URLSearchParams`) parses and serialises it correctly with **zero** custom code; every JavaScript runtime (browser, Bun, Node, Workers) handles encoding identically.
|
||||
- Each candidate is one `?u=` value; `URLSearchParams.getAll("u")` returns the ordered list in O(n) without ambiguity.
|
||||
- Order is preserved because `URLSearchParams` is ordered.
|
||||
- Encoding rules are RFC 3986-compliant out of the box; no edge case for `?`, `&`, `#`, `/`, `:` inside candidates.
|
||||
- The URI is self-describing — no out-of-band registry; the URI string IS the manifest.
|
||||
- Length is bounded but ample: `policy.maxCandidates` (default 16, hard ceiling 64) × `INVALID_URI` threshold from `@suaveplan/storage-uri` (4096 chars per candidate) = ~64 KiB worst case, comfortably within any real-world URI handler.
|
||||
|
||||
The grammar formally:
|
||||
|
||||
```text
|
||||
fallback-uri = "fallback:" "//" authority "/" "?" candidates [ "#" fragment ]
|
||||
authority = "" ; always empty
|
||||
candidates = "u=" pct-encoded-uri *( "&" "u=" pct-encoded-uri )
|
||||
; ordered; at least 1, at most maxCandidates
|
||||
```
|
||||
|
||||
Canonical examples:
|
||||
|
||||
```text
|
||||
fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo
|
||||
fallback://?u=file%3A%2F%2F%2Ftmp%2Ffixtures%2Fa.json&u=https%3A%2F%2Ffixtures.test%2Fa.json
|
||||
fallback://?u=cache%3A%2F%2Flocal%2Favatars%2Falice.jpg&u=cdn%3A%2F%2Fzone-eu%2Favatars%2Falice.jpg&u=drive%3A%2F%2Fprimary%2Favatars%2Falice.jpg
|
||||
```
|
||||
|
||||
A helper `buildFallbackUri(uris: readonly string[]): string` is exported so authors never construct the URI by hand; it uses `URLSearchParams` and rejects empty / oversized inputs at the boundary.
|
||||
|
||||
## 2. File layout
|
||||
|
||||
```text
|
||||
src/
|
||||
index.ts # explicit named re-exports (rule 10)
|
||||
error-codes/
|
||||
error-codes.ts # defineCodes("FALLBACK_URI", [...]) + FallbackUriError
|
||||
error-codes.md
|
||||
error-codes.test.ts # reachability sweep — every code triggered
|
||||
types/
|
||||
policy.types.ts # FallbackPolicy, FallbackPluginOptions
|
||||
policy.types.md
|
||||
policy.schemas.ts # Zod boundary schemas
|
||||
candidate.types.ts # CandidateError, CandidateAttempt
|
||||
candidate.types.md
|
||||
parse-uri/
|
||||
parse-fallback-uri.ts # parseFallbackUri + buildFallbackUri
|
||||
parse-fallback-uri.md
|
||||
parse-fallback-uri.test.ts
|
||||
classify/
|
||||
classify.ts # error → "transient" | "semantic" | "abort"
|
||||
classify.md
|
||||
classify.test.ts
|
||||
walk/
|
||||
sequential.ts # ordered fall-through walk
|
||||
sequential.md
|
||||
sequential.test.ts
|
||||
parallel.ts # Promise.any race; first-to-resolve wins
|
||||
parallel.md
|
||||
parallel.test.ts
|
||||
recursion/
|
||||
depth-context.ts # AsyncLocalStorage-equivalent depth + visited-set
|
||||
depth-context.md
|
||||
depth-context.test.ts
|
||||
ops/
|
||||
read.ts # get / createReadStream / range
|
||||
read.md
|
||||
read.test.ts
|
||||
write.ts # set / createWriteStream / mirror fan-out
|
||||
write.md
|
||||
write.test.ts
|
||||
metadata.ts # exists / getMetadata / stat
|
||||
metadata.md
|
||||
metadata.test.ts
|
||||
delete.ts # mirror delete
|
||||
delete.md
|
||||
delete.test.ts
|
||||
list.ts # union-with-dedup list
|
||||
list.md
|
||||
list.test.ts
|
||||
observability/
|
||||
events.ts # EventBusLike emit helpers
|
||||
events.md
|
||||
events.test.ts
|
||||
otel.ts # counter / histogram / span hierarchy
|
||||
otel.md
|
||||
otel.test.ts
|
||||
plugin/
|
||||
plugin.ts # fallbackUriPlugin factory
|
||||
plugin.md
|
||||
plugin.test.ts
|
||||
plugin.spec-conformance.test.ts # runUriPluginConformance pass
|
||||
conformance/
|
||||
fallback-conformance.ts # runFallbackConformance harness
|
||||
fallback-conformance.md
|
||||
fallback-conformance.test.ts
|
||||
```
|
||||
|
||||
Every implementation file is ≤ 300 lines (250 soft); every test file ≤ 500 lines. The `walk/`, `ops/`, and `recursion/` directories are the natural responsibility boundaries — splitting `walk` from `ops` keeps the candidate-iteration algorithm separate from the per-operation surface so each can be tested in isolation.
|
||||
|
||||
## 3. Recursion-prevention algorithm
|
||||
|
||||
`fallback://` URIs can be nested (a candidate of a `fallback://` may itself be a `fallback://`). Two invariants protect the process:
|
||||
|
||||
1. **Depth cap** — each plugin call increments a depth counter carried on a context token (`DepthContext`). Construction injects an `AsyncLocalStorage`-equivalent abstraction (`DepthContext.run(state, fn)`); under the hood the universal-tier default is a closure-passed state object (the plugin does not depend on `node:async_hooks`). Exceeding `policy.maxDepth` (default 4, hard ceiling 8) throws `FALLBACK_URI.MAX_DEPTH_EXCEEDED`.
|
||||
2. **Visited set** — every recursion frame appends the **original input URI** (not the parsed candidate) to a `visited: Set<string>`. When a candidate URI normalises to a value already in `visited`, the walk skips that candidate and emits `storage-uri.fallback.cycle-detected`; if **every** candidate cycles, the walk throws `FALLBACK_URI.CIRCULAR_REFERENCE` with `cycleChain: readonly string[]` populated.
|
||||
|
||||
Normalisation: the visited-set key is the URI string after `parseFallbackUri` reserialisation (canonical query-param order, lowercased scheme, no fragment). This catches `?u=A&u=B` vs `?u=B&u=A` re-orderings of the same set.
|
||||
|
||||
Pseudocode:
|
||||
|
||||
```ts
|
||||
function resolveCandidate(uri: string, ctx: DepthContext) {
|
||||
if (ctx.depth >= ctx.maxDepth) {
|
||||
throw new FallbackUriError("MAX_DEPTH_EXCEEDED", { uri, depth: ctx.depth });
|
||||
}
|
||||
const canonical = canonicalise(uri);
|
||||
if (ctx.visited.has(canonical)) {
|
||||
ctx.events?.emit("storage-uri.fallback.cycle-detected", { uri: canonical });
|
||||
return { kind: "cycle" } as const;
|
||||
}
|
||||
return ctx.resolver.dispatch(
|
||||
uri,
|
||||
{ ...ctx, depth: ctx.depth + 1, visited: new Set([...ctx.visited, canonical]) }
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## 4. Write-mirror semantics
|
||||
|
||||
By default, writes mirror to **all** candidates so subsequent reads see consistent content regardless of which candidate wins the race. Trade-offs documented:
|
||||
|
||||
| `writeMode` | Behaviour | Use case |
|
||||
|---|---|---|
|
||||
| `"all"` (default) | `Promise.allSettled` over every candidate; any rejection throws `FALLBACK_URI.WRITE_MIRROR_FAILED` with `mirrorOutcomes: readonly { uri, status, error? }[]`. | CDN + origin pairing where every mirror must stay current. |
|
||||
| `"first-success"` | Walk in order; first success returns; pending writes are allowed to settle in the background (logged via events but not awaited). | Caching tiers where the upstream origin is authoritative and lower tiers are best-effort populated. |
|
||||
|
||||
Per-candidate write failures preserve the underlying `SuaveplanError` via `mirrorOutcomes[i].error`; non-`SuaveplanError` throwables are wrapped as `FALLBACK_URI.ALL_CANDIDATES_FAILED`-style sub-errors via the resolver's normal `STORAGE_URI.ADAPTER_ERROR` path (the plugin trusts the resolver's wrap rules; it never re-wraps a namespaced error).
|
||||
|
||||
When `writeMode === "all"` and **all** candidates fail, the plugin throws `FALLBACK_URI.WRITE_MIRROR_FAILED` (not `ALL_CANDIDATES_FAILED`) so callers can pattern-match on intent — `WRITE_MIRROR_FAILED` is "the operation was a mirror; some leg failed"; `ALL_CANDIDATES_FAILED` is "the operation was a walk; every leg failed."
|
||||
|
||||
`delete` follows the same matrix with one variation: success is defined as "at least one candidate deleted" by default (matches `StorageLike.delete`'s boolean return semantics — any consistent mirror has been removed). Opt in to strict deletion with `policy.deleteMode === "all"` for parity with writes.
|
||||
|
||||
## 5. List-union dedup
|
||||
|
||||
`list()` collects results from every candidate, then deduplicates by canonical URI string:
|
||||
|
||||
```ts
|
||||
const seen = new Set<string>();
|
||||
const merged: string[] = [];
|
||||
for (const result of perCandidateResults) {
|
||||
for (const uri of result.uris) {
|
||||
if (seen.has(uri)) continue;
|
||||
seen.add(uri);
|
||||
merged.push(uri);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Order is **stable across candidates**: first-occurrence wins. `hasMore` is true if **any** candidate reports `hasMore: true`. `continuationToken` is intentionally NOT propagated by default — the plugin returns `continuationToken: undefined` because tokens are opaque per-adapter and cannot be merged. Callers needing paginated listing across mirrors must drive each candidate's `list` themselves; the plugin documents this in `ops/list.md`.
|
||||
|
||||
Under `listMode: "concat"` (opt-in), the plugin returns the raw concatenation without dedup — useful when callers want to count per-candidate hits.
|
||||
|
||||
When per-candidate `list` calls partially fail, the plugin throws `FALLBACK_URI.LIST_UNION_FAILED` carrying the per-candidate outcomes; the policy may relax this to "emit warning event, return partial union" via `policy.listFailureMode: "throw" | "partial"`.
|
||||
|
||||
## 6. Complementarity with `@suaveplan/storage-mux`
|
||||
|
||||
`storage-mux` and `storage-uri-plugin-fallback` solve adjacent problems at different layers:
|
||||
|
||||
| Concern | `storage-mux` | `storage-uri-plugin-fallback` |
|
||||
|---|---|---|
|
||||
| Layer | `AbstractStorage` (adapter) | URI scheme |
|
||||
| Input | Pre-constructed `StorageLike` instances + policy | URI string with embedded candidate URIs |
|
||||
| Backend identity | `BackendEntry.name` | URI string |
|
||||
| Cross-scheme support | All backends must satisfy `StorageLike`; the mux does not know about schemes | Every candidate may use a different scheme (`https`, `drive`, `bundle`, `cache`, …) |
|
||||
| Composability | Recursive (a mux is a `StorageLike`) | Recursive (a `fallback://` candidate may itself be `fallback://`) |
|
||||
| Policy surface | Six modes (failover, round-robin, weighted, sticky, shield, mirror) | One mode (sequential walk with optional race), simpler |
|
||||
| State | Per-backend health, weights, drain | Stateless per call |
|
||||
| Observability | Per-backend span hierarchy + stats | Per-candidate span hierarchy + per-candidate counter |
|
||||
|
||||
**When to choose which:**
|
||||
|
||||
- If the calling code holds `StorageLike` instances directly (the storage layer is a private composition concern), use `storage-mux`.
|
||||
- If the calling code receives **URIs from manifests / config / wire formats** and needs failover without changing its interface, use `fallback://`.
|
||||
- They can be combined: a `fallback://` candidate may be a URI that ultimately resolves through a `storage-mux`-backed plugin, and vice versa.
|
||||
|
||||
The two packages share a **classification table** (`design.md` §7) so transient/semantic/abort semantics are uniform across the stack — copy-paste sync between `storage-mux/src/classify` and `storage-uri-plugin-fallback/src/classify` is enforced by a per-PR check in CI.
|
||||
|
||||
## 7. Error classification table
|
||||
|
||||
Mirrors `storage-mux` design §4 for cross-package consistency:
|
||||
|
||||
| Error shape | Class | Triggers fall-through? | Code surfaced |
|
||||
|---|---|---|---|
|
||||
| `AbortError` / `signal.aborted` from caller | abort | NO (stop walk) | `FALLBACK_URI.ABORTED` |
|
||||
| `STORAGE_URI.NOT_FOUND` | semantic-but-fall-through (policy) | YES (configurable; default YES) | wrapped, last in cause chain on exhaustion |
|
||||
| `STORAGE_URI.INVALID_URI` / `STORAGE_URI.UNKNOWN_SCHEME` | semantic | NO (rethrow — caller bug) | rethrow |
|
||||
| `STORAGE_URI.WRITE_UNSUPPORTED` | semantic | NO (rethrow) | rethrow |
|
||||
| Per-candidate `STORAGE_URI.ADAPTER_ERROR` | transient | YES | last in cause chain |
|
||||
| Per-candidate `STORAGE_URI.ABORTED` from inner timeout | transient | YES (if `stopOnAbort: false`) | wrapped |
|
||||
| Unknown thrown value (non-`Error`) | transient | YES | wrapped (resolver normalises first) |
|
||||
| `FALLBACK_URI.MAX_DEPTH_EXCEEDED` | semantic | NO | rethrow (programmer bug) |
|
||||
| `FALLBACK_URI.CIRCULAR_REFERENCE` | semantic | NO | rethrow |
|
||||
|
||||
`policy.errorClass: "any"` re-classifies all semantic errors as fall-through-eligible; the default (`"transient"` + NOT_FOUND) is the safer choice because it preserves the resolver's caller-bug surfaces (`INVALID_URI`, `UNKNOWN_SCHEME`) without forcing every consumer to opt out.
|
||||
|
||||
## 8. Error-code matrix
|
||||
|
||||
| Code | Thrown from | Discriminants populated |
|
||||
|---|---|---|
|
||||
| `FALLBACK_URI.NO_CANDIDATES` | `parseFallbackUri` when `?u=` count is 0 | `uri` |
|
||||
| `FALLBACK_URI.INVALID_URI` | `parseFallbackUri` on malformed URI (missing scheme, non-empty authority, > 4096 chars total) | `uri` |
|
||||
| `FALLBACK_URI.ALL_CANDIDATES_FAILED` | `walk/sequential.ts`, `walk/parallel.ts` on exhaustion (reads) | `uri`, `candidateErrors`, `cause` (last error) |
|
||||
| `FALLBACK_URI.WRITE_MIRROR_FAILED` | `ops/write.ts` under `writeMode: "all"` when any candidate rejects | `uri`, `mirrorOutcomes` |
|
||||
| `FALLBACK_URI.LIST_UNION_FAILED` | `ops/list.ts` under `listFailureMode: "throw"` when any candidate rejects | `uri`, `candidateErrors` |
|
||||
| `FALLBACK_URI.ABORTED` | walk entry when `signal.aborted === true`, or mid-walk when `stopOnAbort: true` | `uri` |
|
||||
| `FALLBACK_URI.MAX_DEPTH_EXCEEDED` | `recursion/depth-context.ts` when `ctx.depth >= ctx.maxDepth` | `uri`, `depth` |
|
||||
| `FALLBACK_URI.CIRCULAR_REFERENCE` | `walk/*.ts` when every candidate has already been visited | `uri`, `cycleChain` |
|
||||
|
||||
Every code is reachable from a real call site exercised by `src/error-codes/error-codes.test.ts` (the reachability sweep).
|
||||
|
||||
## 9. Tier purity
|
||||
|
||||
Universal — no `node:*`, no `bun:*`, no DOM. Specifically:
|
||||
|
||||
- `URL` and `URLSearchParams` are Web standards (available in every runtime).
|
||||
- `AbortSignal.any` is ES2024; a tiny polyfill is provided under `src/parse-uri/abort-any.ts` for runtimes that lack it (same approach as `storage-mux`).
|
||||
- Depth context uses closure passing (`DepthContext` value threaded through call options) rather than `node:async_hooks`. This is sufficient because the plugin's recursion is all within a single call chain — there is no cross-`await`-boundary state to track that the caller doesn't already own.
|
||||
- `Clock` is injected via `@suaveplan/types#Clock` for any time-keeping (latency histograms only); the plugin has no scheduled work of its own.
|
||||
|
||||
## 10. Trade-offs
|
||||
|
||||
- **Query-param URI grammar vs. shorter forms.** Option 3 trades a few characters of length for zero custom parsing and standards alignment. Worth it; the URIs are not user-facing.
|
||||
- **Mirror writes by default.** Cheaper to mirror up-front than to discover a stale mirror on the next read. Callers that explicitly want first-success-wins opt in via `writeMode: "first-success"`.
|
||||
- **List dedup by canonical URI.** Cannot detect content drift between mirrors; callers needing byte-equality must use a content-addressed scheme. Acceptable because content-verification is out of scope (proposal §"Out of scope").
|
||||
- **Depth cap 4 default, 8 hard ceiling.** Most use cases are 2–3 levels (cache → CDN → origin). Eight covers pathological compositions without enabling DoS via runaway recursion.
|
||||
- **Stateless plugin.** Unlike `storage-mux`, there is no per-candidate health tracking. The fallback walk is one attempt per candidate per call; if you need backoff/health, wrap each candidate's underlying plugin in `@suaveplan/retry` or stack a `storage-mux` underneath. Keeps the plugin's surface small and the semantics auditable.
|
||||
- **Cycle detection by URI string.** Cheap and correct for the common case; cannot catch a cycle that goes through a non-`fallback://` indirection (e.g. an HTTP redirect chain). That's not the plugin's problem to solve.
|
||||
@@ -0,0 +1,106 @@
|
||||
# Add @suaveplan/storage-uri-plugin-fallback — multi-source URI resolution via `fallback://` meta-scheme
|
||||
|
||||
**Package**: `@suaveplan/storage-uri-plugin-fallback`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (genesis-original)
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/storage-uri-plugin-fallback/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
Consumers of `@suaveplan/storage-uri` increasingly need **multi-source resolution** — try several URIs in order (or in parallel), return the first that resolves. The parallel audio-IR loader session almost shipped this as ad-hoc manifest data (a plain `uris: string[]` field on each impulse-response manifest); that solution only benefits the audio loader and forces every future multi-source consumer to reinvent the same control flow on top of the resolver.
|
||||
|
||||
A **URI-scheme** approach generalises the capability: any callsite that already accepts a single URI gains failover by changing one string. The resolver dispatch table does not change. Every existing `UriPlugin` (`drive`, `file`, `https`, `bundle`, `s3`, `cdn`, …) becomes a valid candidate. This is the URI-layer analogue of `@suaveplan/storage-mux` (which composes `StorageLike` backends at the adapter layer) — they are complementary, not competing (see `design.md` §6).
|
||||
|
||||
The plugin reuses the resolver it is registered into for per-candidate dispatch — a `fallback://` URI is resolved by recursively calling the **same** `StorageUriResolver`, with cycle detection and a hard depth cap to prevent self-reference DoS.
|
||||
|
||||
### ≥2-consumer justification (per `openspec/AGENTS.md` §0 step 6)
|
||||
|
||||
Forward-declared consumers that share the `fallback://` primitive:
|
||||
|
||||
1. **Audio-IR loader (parallel session)** — primary `bundle://ir-pack-v3/halls/concert.wav` with fallback to `https://cdn.example.com/irs/halls/concert.wav`. The loader becomes scheme-agnostic — it no longer needs manifest-level multi-source logic.
|
||||
2. **Drive + CDN pairing** — read from `cdn://zone-eu/avatars/alice.jpg`; if the CDN returns NOT_FOUND or a transient 5xx, fall back to `drive://primary/avatars/alice.jpg` (the authoritative origin). Same primitive used by `storage-plugin-cache` for the URI tier.
|
||||
3. **Generic CDN multi-origin failover at the URI tier** — `cdn-bunnycdn://…` first, `cdn-rackspace://…` second. `storage-mux` already covers this at the adapter layer when you control the storage instance; `fallback://` covers it when you only have URI strings (HTTP redirects, manifests, config).
|
||||
4. **Test fixtures** — primary points at a local `file:///tmp/fixtures/...`, fallback at a network fixture URL, second fallback at an inline `data:` URI. Lets test setups degrade gracefully when fixture servers are unavailable.
|
||||
5. **Drive-template hydration** — a template descriptor names multiple chunk-origin URIs; the consumer wraps them in a `fallback://` so the chunk-puller can keep its existing single-URI interface.
|
||||
6. **Cache warm-from-cold** — `storage-plugin-cache` reads from `cache://local/<key>`; on miss, fall back to the cold-storage URI. Mirrors the same composition story as `storage-mux`'s `shield` mode but expressed at the URI scheme tier.
|
||||
|
||||
All six scenarios share the same primitive (ordered candidate list + error classification + abort propagation + mirror-write semantics); without the shared plugin each consumer reinvents the same walker and the `FALLBACK_URI.*` error registry fragments.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `@suaveplan/storage-uri-plugin-fallback` package (universal tier, zero browser/Node API dependencies).
|
||||
- ADDED: `fallbackUriPlugin(options): UriPlugin` factory taking `{ resolver: StorageUriResolver, policy?: FallbackPolicy, logger?: LoggerLike, events?: EventBusLike, clock?: Clock }` and returning a `UriPlugin` bound to the `fallback` scheme.
|
||||
- ADDED: `fallback://` URI grammar — **query-param form** with embedded URIs in `?u=` repeated parameters. Example: `fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo`. Decision matrix and rationale in `design.md` §1.
|
||||
- ADDED: `parseFallbackUri(uri)` helper exporting the canonical candidate list with `INVALID_URI` / `NO_CANDIDATES` rejection.
|
||||
- ADDED: `FallbackPolicy` discriminated bundle: `errorClass: "transient" | "any"` (default `"transient"` + NOT_FOUND), `maxParallel: number` (default 1; race mode when > 1), `stopOnAbort: boolean` (default true), `writeMode: "all" | "first-success"` (default `"all"` — mirror writes keep candidates in sync), `listMode: "union" | "concat"` (default `"union"` with dedup), `maxDepth: number` (default 4, hard ceiling 8).
|
||||
- ADDED: Recursive resolver dispatch — the plugin holds a `StorageUriResolver` reference; each candidate URI is resolved through that resolver so nested `fallback://` is supported (depth-tracked via an `AsyncLocalStorage`-equivalent injection token passed via options).
|
||||
- ADDED: **Cycle detection** — every recursion frame appends the original URI to a visited set; re-encountering the same URI throws `FALLBACK_URI.CIRCULAR_REFERENCE`.
|
||||
- ADDED: **Depth cap** — recursion through nested `fallback://` URIs is capped at `policy.maxDepth` (default 4, hard ceiling 8); exceeding throws `FALLBACK_URI.MAX_DEPTH_EXCEEDED`.
|
||||
- ADDED: Full `StorageLike` adapter surface attached to the plugin:
|
||||
- `get(key)` / `createReadStream(key, signal)` — sequential or parallel walk; first success returns.
|
||||
- `exists(key)` / `getMetadata(key, signal)` — sequential walk; first success returns.
|
||||
- `set(key, bytes)` / `createWriteStream(key, body, signal)` — mirror to all candidates by default; configurable to first-success-only.
|
||||
- `delete(key)` — mirror delete to all candidates; success = any candidate deletes (configurable).
|
||||
- `list({ prefix, limit })` — union with dedup by URI string.
|
||||
- ADDED: **Range read** support — `{ offset, length }` propagated to each candidate via the resolver's existing surface.
|
||||
- ADDED: **AbortSignal composition** — caller signal + per-candidate timeout signal composed via `AbortSignal.any`. `stopOnAbort: true` cuts the walk immediately on caller abort; `false` lets the current candidate complete.
|
||||
- ADDED: **All-failed reporting** — when every candidate errors, throws `FALLBACK_URI.ALL_CANDIDATES_FAILED` with `candidateErrors: readonly { uri: string; error: SuaveplanError }[]` preserved on the structured error options and cross-linked via `cause` to the last failure.
|
||||
- ADDED: **Write-mirror semantics** — write fan-out uses `Promise.allSettled`; under `writeMode: "all"` any rejection throws `FALLBACK_URI.WRITE_MIRROR_FAILED` with per-candidate outcomes; under `"first-success"` the first success returns and pending writes are allowed to settle in the background.
|
||||
- ADDED: **List-union dedup** — concatenated key sets are deduped by canonical URI string; collisions where two candidates yield the same URI count once (caller's responsibility to verify content parity).
|
||||
- ADDED: `FALLBACK_URI_CODES` via `defineCodes("FALLBACK_URI", […])` — `NO_CANDIDATES`, `MAX_DEPTH_EXCEEDED`, `ALL_CANDIDATES_FAILED`, `INVALID_URI`, `ABORTED`, `WRITE_MIRROR_FAILED`, `LIST_UNION_FAILED`, `CIRCULAR_REFERENCE` (8 codes total; each reachable from a real call site exercised by tests).
|
||||
- ADDED: `FallbackUriError extends SuaveplanError<FallbackUriErrorCode>` with `uri`, `candidateIndex`, `candidateUri`, `candidateErrors` discriminants.
|
||||
- ADDED: **Observability**:
|
||||
- OTel counter `storage_uri.fallback_total{candidate_index, scheme, outcome}` per candidate attempt.
|
||||
- OTel histogram `storage_uri.fallback_walk_ms{outcome}` per top-level operation.
|
||||
- Parent span `storage-uri.fallback.<op>` with child span `storage-uri.fallback.attempt` per candidate (attrs `fallback.candidate.index`, `fallback.candidate.scheme`, `fallback.candidate.outcome ∈ {ok, transient, semantic, abort}`).
|
||||
- `EventBusLike` emissions: `storage-uri.fallback.attempt-failed`, `.attempt-succeeded`, `.write-mirror-partial`, `.cycle-detected`, `.depth-exceeded`.
|
||||
- ADDED: `runUriPluginConformance` pass (inherited contract from `@suaveplan/storage-uri/conformance`).
|
||||
- ADDED: **Fallback-specific conformance harness** — `runFallbackConformance(resolver, fixtures)` that asserts:
|
||||
- With N candidates where the first K fail and the (K+1)th succeeds, the result is the (K+1)th value and K events are emitted on the bus.
|
||||
- All-candidates-fail throws `ALL_CANDIDATES_FAILED` with `candidateErrors.length === N`.
|
||||
- Pre-aborted signal short-circuits with `ABORTED`.
|
||||
- Cycle yields `CIRCULAR_REFERENCE`.
|
||||
- Depth cap yields `MAX_DEPTH_EXCEEDED`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Authentication / signing across candidates.** Each candidate URI's scheme owns its own credentials; the plugin does not propagate auth state between candidates.
|
||||
- **Content verification across candidates.** If `cdn://...` and `drive://...` disagree on byte content, the plugin does NOT detect or reconcile that — first-success-wins. Content parity is a manifest-layer concern (use a content-addressed scheme like `chunks://<hash>`).
|
||||
- **Per-candidate retry tuning.** The walk is one attempt per candidate. Inner retry (backoff per candidate) is composed by wrapping each candidate's underlying plugin in `@suaveplan/retry`; the fallback walk is orthogonal.
|
||||
- **Adapter-layer multi-backend.** That is `@suaveplan/storage-mux`'s remit — see `design.md` §6.
|
||||
|
||||
## Source mapping
|
||||
|
||||
Genesis-original; no upstream in `../suaveplan/`. References:
|
||||
|
||||
- `@suaveplan/storage-uri` — `UriPlugin`, `StorageUriResolver`, `runUriPluginConformance`, error-wrap semantics.
|
||||
- `@suaveplan/storage-mux` proposal (`openspec/changes/complete-storage-mux/`) — error classification table, fan-out write semantics, span hierarchy.
|
||||
- `@suaveplan/retry` — `RetryStrategy` (not consumed directly; consumers wrap candidates).
|
||||
- RFC 3986 §3.4 (query component) for the URI grammar choice.
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement in `specs/storage-uri-plugin-fallback/spec.md` carries a `**Verified by:**` line naming planned test files. Coverage gate: 100/100/100/100. The conformance harness runs `runUriPluginConformance` against the fallback plugin (registered alongside an in-memory plugin in test fixtures), plus the fallback-specific `runFallbackConformance` exercising sequential walk, parallel race, cycle detection, depth cap, write-mirror fan-out, and abort short-circuit.
|
||||
|
||||
## Migration
|
||||
|
||||
Not applicable — net-new package.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/storage-uri-plugin-fallback` to enumerate. The package is new — no current dependents.
|
||||
|
||||
- No current dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-uri-plugin-fallback` returning the package itself).
|
||||
- Forward-declared consumers in §"≥2-consumer justification" each land their own OpenSpec change to wire the plugin in.
|
||||
|
||||
### Affected upstreams
|
||||
|
||||
- `@suaveplan/storage-uri` — consumed via `StorageUriResolver`, `UriPlugin`, `runUriPluginConformance`. No augmentation required. [x] no action needed.
|
||||
- `@suaveplan/error` — `defineCodes`, `SuaveplanError`. [x] no action needed.
|
||||
- `@suaveplan/events` — `EventBusLike` consumer only. [x] no action needed.
|
||||
- `@suaveplan/types` — `Clock`, `LoggerLike`, `StorageLike` (only the read/write/list shape). [x] no action needed.
|
||||
- `@suaveplan/storage` — devDep for in-memory test backends only. [x] no action needed.
|
||||
@@ -0,0 +1,294 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: storage-uri-plugin-fallback (storage-uri-plugin-fallback)
|
||||
|
||||
**Package**: `@suaveplan/storage-uri-plugin-fallback`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: URI grammar — query-param form with ordered `?u=` candidates
|
||||
|
||||
The package SHALL define the `fallback://` URI grammar as an empty authority and empty path followed by a query string containing one or more ordered `u=<pct-encoded-uri>` parameters. The exported `parseFallbackUri(uri)` helper SHALL return `{ candidates: readonly string[]; canonicalUri: string }` where candidate order matches the input parameter order, and the exported `buildFallbackUri(uris)` helper SHALL produce a canonical `fallback://?u=…&u=…` string using `URLSearchParams`. Inputs missing the `fallback:` scheme, carrying a non-empty authority, longer than 4096 characters total, or containing fewer than one `u=` parameter SHALL be rejected with `FALLBACK_URI.NO_CANDIDATES` (zero candidates) or `FALLBACK_URI.INVALID_URI` (any other malformed input). Candidate count above `policy.maxCandidates` (default 16, hard ceiling 64) SHALL be rejected with `FALLBACK_URI.INVALID_URI`.
|
||||
|
||||
#### Scenario: well-formed two-candidate URI parses preserving order
|
||||
|
||||
- **WHEN** `parseFallbackUri("fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo")` is called
|
||||
- **THEN** the result is `{ candidates: ["https://cdn.example.com/foo", "drive://primary/foo"], canonicalUri: "fallback://?u=https%3A%2F%2Fcdn.example.com%2Ffoo&u=drive%3A%2F%2Fprimary%2Ffoo" }`
|
||||
|
||||
#### Scenario: zero candidates throws NO_CANDIDATES
|
||||
|
||||
- **WHEN** `parseFallbackUri("fallback://")` or `parseFallbackUri("fallback://?")` is called
|
||||
- **THEN** the call throws `FallbackUriError` with `code === "FALLBACK_URI.NO_CANDIDATES"`
|
||||
|
||||
#### Scenario: non-empty authority throws INVALID_URI
|
||||
|
||||
- **WHEN** `parseFallbackUri("fallback://host/?u=https%3A%2F%2Fx")` is called
|
||||
- **THEN** the call throws `FallbackUriError` with `code === "FALLBACK_URI.INVALID_URI"`
|
||||
|
||||
#### Scenario: build round-trips through parse
|
||||
|
||||
- **WHEN** `parseFallbackUri(buildFallbackUri(["https://a/x", "drive://b/y"]))` is called for any non-empty list
|
||||
- **THEN** the returned `candidates` array deep-equals the input
|
||||
|
||||
**Verified by:** `src/parse-uri/parse-fallback-uri.test.ts::"parses two-candidate URI preserving order"` AND `src/parse-uri/parse-fallback-uri.test.ts::"throws NO_CANDIDATES for empty candidate list"` AND `src/parse-uri/parse-fallback-uri.test.ts::"throws INVALID_URI for non-empty authority"` AND `src/parse-uri/parse-fallback-uri.test.ts::"build round-trips through parse"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Plugin factory binds to `fallback` scheme and exposes full UriPlugin surface
|
||||
|
||||
The package SHALL export `fallbackUriPlugin(options: FallbackPluginOptions): UriPlugin` where `options.resolver: StorageUriResolver` is required and is the same resolver the returned plugin will be registered into. The returned plugin's `scheme` SHALL be `"fallback"`, its `parse` SHALL delegate to `parseFallbackUri`, its `build` SHALL delegate to `buildFallbackUri`, and its `adapter` SHALL satisfy the full streaming-extended `StorageLike` surface (`get`, `set`, `setPartial`, `has`, `exists`, `delete`, `list`, `getMetadata`, `createReadStream`, `createWriteStream`). The plugin SHALL satisfy `runUriPluginConformance` against the storage-uri conformance fixtures.
|
||||
|
||||
#### Scenario: constructed plugin advertises `fallback` scheme
|
||||
|
||||
- **WHEN** `fallbackUriPlugin({ resolver })` is called against any non-undefined resolver
|
||||
- **THEN** the returned plugin has `scheme === "fallback"` and `typeof plugin.parse === "function"` and `typeof plugin.build === "function"`
|
||||
|
||||
#### Scenario: missing resolver throws on construction
|
||||
|
||||
- **WHEN** `fallbackUriPlugin({ resolver: undefined as never })` is called
|
||||
- **THEN** the call throws an error (Zod validation failure) before returning a plugin
|
||||
|
||||
#### Scenario: storage-uri conformance harness passes
|
||||
|
||||
- **WHEN** `runUriPluginConformance(plugin, { validUris: [buildFallbackUri(["https://a/x"])], invalidUris: ["fallback://"], buildRoundTrip: [{ key: "<encoded>", uri: buildFallbackUri(["https://a/x"]) }] })` is called
|
||||
- **THEN** the harness returns without throwing
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"advertises the fallback scheme"` AND `src/plugin/plugin.test.ts::"throws when resolver option is missing"` AND `src/plugin/plugin.spec-conformance.test.ts::"passes runUriPluginConformance"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Sequential read walk returns first successful candidate
|
||||
|
||||
When `policy.maxParallel === 1` (default), the plugin SHALL attempt each candidate URI in declared order by dispatching it through the bound `StorageUriResolver`. The first candidate that resolves SHALL produce the returned value. Each transient failure SHALL emit `storage-uri.fallback.attempt-failed` on the configured `EventBusLike` (if provided) with `{ uri, candidateIndex, candidateUri, error, code }`, increment `storage_uri.fallback_total{candidate_index, scheme, outcome:"transient"}`, and continue to the next candidate. Semantic failures classified per the package's classification table SHALL surface immediately without walking the remaining candidates.
|
||||
|
||||
#### Scenario: first candidate succeeds, second is not attempted
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://a", "mem://b"]))` is called and the `mem://a` candidate returns bytes
|
||||
- **THEN** the returned bytes equal the `mem://a` adapter's stored value AND the `mem://b` candidate is never dispatched (verified via spy on the second plugin's `parse`)
|
||||
|
||||
#### Scenario: first candidate transient-fails, second succeeds
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://missing", "mem://b"]))` is called against a resolver where `mem://missing` raises `STORAGE_URI.NOT_FOUND` and `mem://b` returns bytes
|
||||
- **THEN** the returned bytes equal the `mem://b` adapter's stored value AND `EventBusLike.emit` is invoked exactly once with topic `"storage-uri.fallback.attempt-failed"` carrying `candidateIndex === 0`
|
||||
|
||||
#### Scenario: caller-bug semantic error surfaces without fall-through
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["xyz://nope", "mem://b"]))` is called and `xyz` is unregistered, so the resolver throws `STORAGE_URI.UNKNOWN_SCHEME`
|
||||
- **THEN** the call rejects with `STORAGE_URI.UNKNOWN_SCHEME` (NOT `FALLBACK_URI.ALL_CANDIDATES_FAILED`) AND `mem://b` is never dispatched
|
||||
|
||||
**Verified by:** `src/walk/sequential.test.ts::"returns first candidate success without dispatching subsequent candidates"` AND `src/walk/sequential.test.ts::"falls through on transient NOT_FOUND and emits attempt-failed event"` AND `src/walk/sequential.test.ts::"surfaces UNKNOWN_SCHEME from caller-bug candidate without fall-through"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Parallel read race returns first-to-resolve
|
||||
|
||||
When `policy.maxParallel > 1`, the plugin SHALL dispatch up to `maxParallel` candidates concurrently via `Promise.any`-style race, return the first to resolve, and abort the pending candidates via an `AbortSignal.any`-composed signal so they may release resources. Candidate failures that arrive before the first success SHALL be collected; if every candidate fails, the plugin SHALL throw `FALLBACK_URI.ALL_CANDIDATES_FAILED` with `candidateErrors.length === N`.
|
||||
|
||||
#### Scenario: parallel race returns the fastest candidate
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["slow://a", "fast://b"]))` is called against a resolver where `slow://a` delays 50ms and `fast://b` resolves immediately, with `policy.maxParallel: 2`
|
||||
- **THEN** the call resolves with the `fast://b` bytes AND the pending `slow://a` candidate sees its abort signal
|
||||
|
||||
**Verified by:** `src/walk/parallel.test.ts::"resolves with the fastest candidate and aborts pending"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Exhausted walk throws ALL_CANDIDATES_FAILED with per-candidate cause chain
|
||||
|
||||
When every candidate fails with a fall-through-eligible error, the plugin SHALL throw `FALLBACK_URI.ALL_CANDIDATES_FAILED` with `candidateErrors: readonly { uri, error: SuaveplanError }[]` containing one entry per candidate (in attempt order) and `cause` set to the final candidate's error. The error SHALL include the original `fallback://` URI on its `uri` field. SuaveplanError subclasses thrown by candidates SHALL be preserved verbatim inside `candidateErrors[i].error` (no re-wrapping).
|
||||
|
||||
#### Scenario: every candidate fails — error carries all causes
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://x", "mem://y", "mem://z"]))` is called against a resolver where every candidate raises `STORAGE_URI.NOT_FOUND`
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.ALL_CANDIDATES_FAILED"`, `candidateErrors.length === 3`, `candidateErrors[0].uri === "mem://x"`, every `candidateErrors[i].error.code === "STORAGE_URI.NOT_FOUND"`, and `cause === candidateErrors[2].error`
|
||||
|
||||
**Verified by:** `src/walk/sequential.test.ts::"throws ALL_CANDIDATES_FAILED carrying every per-candidate error"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Write fan-out mirrors to all candidates by default
|
||||
|
||||
When `policy.writeMode === "all"` (default), `set` and `createWriteStream` SHALL dispatch the write to **every** candidate URI via `Promise.allSettled`. If **every** write resolves, the operation SHALL resolve. If **any** write rejects, the plugin SHALL throw `FALLBACK_URI.WRITE_MIRROR_FAILED` with `mirrorOutcomes: readonly { uri, status: "fulfilled" | "rejected", error?: SuaveplanError }[]` describing each candidate's outcome. When `policy.writeMode === "first-success"`, the plugin SHALL walk candidates sequentially and return on the first success, allowing remaining writes to settle in the background.
|
||||
|
||||
#### Scenario: write to two candidates fans out to both
|
||||
|
||||
- **WHEN** `write(buildFallbackUri(["mem://a", "mem://b"]), bytes)` is called
|
||||
- **THEN** both `mem://a` and `mem://b` adapters receive the exact bytes (verified via direct reads after) AND the operation resolves successfully
|
||||
|
||||
#### Scenario: partial mirror failure throws WRITE_MIRROR_FAILED
|
||||
|
||||
- **WHEN** `write(buildFallbackUri(["mem://a", "ro-mem://b"]), bytes)` is called against a resolver where `ro-mem://b` is read-only and raises `STORAGE_URI.WRITE_UNSUPPORTED`
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.WRITE_MIRROR_FAILED"`, `mirrorOutcomes[0].status === "fulfilled"`, `mirrorOutcomes[1].status === "rejected"`, `mirrorOutcomes[1].error.code === "STORAGE_URI.WRITE_UNSUPPORTED"`
|
||||
|
||||
#### Scenario: first-success mode returns on first ok and backgrounds remainder
|
||||
|
||||
- **WHEN** `write(buildFallbackUri(["mem://a", "mem://b"]), bytes)` is called with `policy.writeMode: "first-success"`
|
||||
- **THEN** the call resolves as soon as `mem://a` completes; `mem://b` still receives the write asynchronously (verified by a follow-up read after a microtask flush)
|
||||
|
||||
**Verified by:** `src/ops/write.test.ts::"mirrors write to every candidate by default"` AND `src/ops/write.test.ts::"throws WRITE_MIRROR_FAILED with per-candidate outcomes"` AND `src/ops/write.test.ts::"first-success mode returns early and backgrounds remainder"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: List operation returns deduplicated union of candidate results
|
||||
|
||||
`list({ prefix, limit })` SHALL invoke `list` on every candidate URI and return the union of their key sets, deduplicated by canonical URI string in stable first-occurrence order. `hasMore` SHALL be `true` if **any** candidate reports `hasMore: true`. `continuationToken` SHALL be `undefined` because per-adapter tokens cannot be merged. Under `listMode: "concat"` the plugin SHALL return the raw concatenation without dedup. Under `listFailureMode: "throw"` (default), any candidate `list` failure SHALL cause the call to throw `FALLBACK_URI.LIST_UNION_FAILED` with `candidateErrors`; under `"partial"`, failures SHALL emit a warning event and be excluded from the merged result.
|
||||
|
||||
#### Scenario: union list dedupes overlapping URIs
|
||||
|
||||
- **WHEN** `list(buildFallbackUri(["mem://a/", "mem://b/"]))` is called where `mem://a/` returns `["mem://a/x", "mem://a/y"]` and `mem://b/` returns `["mem://b/x"]`, and an alias makes the second candidate also yield `"mem://a/x"`
|
||||
- **THEN** the returned `uris` contain `["mem://a/x", "mem://a/y", "mem://b/x"]` with `"mem://a/x"` appearing exactly once
|
||||
|
||||
#### Scenario: candidate list failure under throw mode raises LIST_UNION_FAILED
|
||||
|
||||
- **WHEN** `list(buildFallbackUri(["mem://a/", "fail-mem://b/"]))` is called and the second candidate raises an adapter error
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.LIST_UNION_FAILED"` and `candidateErrors.length === 1`
|
||||
|
||||
**Verified by:** `src/ops/list.test.ts::"returns deduplicated union of candidate keys in first-occurrence order"` AND `src/ops/list.test.ts::"throws LIST_UNION_FAILED on per-candidate failure under throw mode"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Recursive `fallback://` is supported with depth cap and cycle detection
|
||||
|
||||
The plugin SHALL accept candidates that are themselves `fallback://` URIs by recursively dispatching through the bound resolver. Recursion depth SHALL be tracked via a `DepthContext` value threaded through call options; exceeding `policy.maxDepth` (default 4, hard ceiling 8) SHALL throw `FALLBACK_URI.MAX_DEPTH_EXCEEDED`. Every recursion frame SHALL append the canonicalised `fallback://` URI to a `visited: Set<string>`; when every candidate has been visited the plugin SHALL throw `FALLBACK_URI.CIRCULAR_REFERENCE` with `cycleChain: readonly string[]` populated.
|
||||
|
||||
#### Scenario: two-level nested fallback resolves correctly
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["fallback://?u=mem%3A%2F%2Fa", "mem://b"]))` is called against a resolver where `mem://a` succeeds
|
||||
- **THEN** the call resolves with the `mem://a` bytes AND depth never exceeds 2
|
||||
|
||||
#### Scenario: depth cap exceeded throws MAX_DEPTH_EXCEEDED
|
||||
|
||||
- **WHEN** the resolver dispatches a `fallback://` whose only candidate is itself a `fallback://` chain longer than `policy.maxDepth`
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.MAX_DEPTH_EXCEEDED"` and `depth === policy.maxDepth + 1`
|
||||
|
||||
#### Scenario: cycle yields CIRCULAR_REFERENCE
|
||||
|
||||
- **WHEN** `read(uri)` is called where `uri = buildFallbackUri([uri])` (self-reference, hand-crafted via the visited-set canonicalisation key)
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.CIRCULAR_REFERENCE"` and `cycleChain[0]` equal to the canonicalised input URI
|
||||
|
||||
**Verified by:** `src/recursion/depth-context.test.ts::"two-level nested fallback resolves correctly"` AND `src/recursion/depth-context.test.ts::"throws MAX_DEPTH_EXCEEDED past maxDepth"` AND `src/walk/sequential.test.ts::"throws CIRCULAR_REFERENCE on self-reference"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AbortSignal propagation and stopOnAbort policy
|
||||
|
||||
Every public operation on the plugin's adapter SHALL accept an `AbortSignal` and SHALL forward an `AbortSignal.any`-composed signal (caller + per-candidate timeout, if any) to each candidate dispatch. When `policy.stopOnAbort === true` (default), an aborted caller signal SHALL cut the walk immediately; the plugin SHALL throw `FALLBACK_URI.ABORTED` without dispatching further candidates. When `policy.stopOnAbort === false`, the plugin SHALL let the in-flight candidate complete and SHALL then throw `FALLBACK_URI.ABORTED`. A pre-aborted signal SHALL short-circuit before dispatching the first candidate.
|
||||
|
||||
#### Scenario: pre-aborted signal short-circuits before any dispatch
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://a"]), AbortSignal.abort())` is called
|
||||
- **THEN** the call rejects with `FallbackUriError` having `code === "FALLBACK_URI.ABORTED"` AND the `mem://a` candidate is never dispatched
|
||||
|
||||
#### Scenario: mid-walk abort under stopOnAbort=true cuts immediately
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["slow-mem://a", "mem://b"]), signal)` is called and the caller fires `signal.abort()` after the first candidate is dispatched but before it completes
|
||||
- **THEN** the call rejects with `FALLBACK_URI.ABORTED` AND `mem://b` is never dispatched
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"short-circuits on pre-aborted signal"` AND `src/plugin/plugin.test.ts::"cuts walk immediately on mid-walk abort under stopOnAbort=true"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Error classification governs fall-through eligibility
|
||||
|
||||
The plugin SHALL classify every error raised by a candidate dispatch as `"transient"`, `"semantic"`, or `"abort"` per the table in `design.md` §7. `"transient"` errors SHALL trigger fall-through to the next candidate. `"semantic"` errors SHALL surface immediately without walking remaining candidates, **except** that `STORAGE_URI.NOT_FOUND` SHALL be treated as fall-through-eligible by default. `"abort"` errors SHALL terminate the walk per the AbortSignal requirement. Setting `policy.errorClass: "any"` SHALL re-classify every semantic error (except `MAX_DEPTH_EXCEEDED` and `CIRCULAR_REFERENCE`) as fall-through-eligible.
|
||||
|
||||
#### Scenario: NOT_FOUND from first candidate falls through
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://missing", "mem://present"]))` is called where the first raises `NOT_FOUND` and the second succeeds
|
||||
- **THEN** the call resolves with the second candidate's bytes
|
||||
|
||||
#### Scenario: UNKNOWN_SCHEME surfaces immediately under default policy
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["xyz://unregistered", "mem://present"]))` is called with default policy
|
||||
- **THEN** the call rejects with `STORAGE_URI.UNKNOWN_SCHEME` (the namespaced error from the resolver passes through unchanged)
|
||||
|
||||
#### Scenario: errorClass=any treats UNKNOWN_SCHEME as fall-through
|
||||
|
||||
- **WHEN** the same call is made with `policy.errorClass: "any"`
|
||||
- **THEN** the call resolves with the second candidate's bytes AND an `attempt-failed` event records `UNKNOWN_SCHEME` for `candidateIndex === 0`
|
||||
|
||||
**Verified by:** `src/classify/classify.test.ts::"classifies NOT_FOUND as fall-through-eligible"` AND `src/walk/sequential.test.ts::"surfaces UNKNOWN_SCHEME under default policy"` AND `src/walk/sequential.test.ts::"falls through UNKNOWN_SCHEME under errorClass=any"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry instrumentation per candidate
|
||||
|
||||
The plugin SHALL emit a parent span `storage-uri.fallback.<op>` for every public operation and a child span `storage-uri.fallback.attempt` per candidate attempt with attributes `fallback.candidate.index`, `fallback.candidate.scheme`, `fallback.candidate.outcome` ∈ `{ok, transient, semantic, abort}`. The plugin SHALL increment the counter `storage_uri.fallback_total{candidate_index, scheme, outcome}` per candidate attempt and SHALL record the histogram `storage_uri.fallback_walk_ms{outcome}` per top-level operation.
|
||||
|
||||
#### Scenario: successful walk records counter for both candidates plus histogram
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://miss", "mem://hit"]))` is called against fixture adapters where the first misses and the second hits
|
||||
- **THEN** `storage_uri.fallback_total{candidate_index:0, scheme:"mem", outcome:"transient"}` is incremented by 1 AND `storage_uri.fallback_total{candidate_index:1, scheme:"mem", outcome:"ok"}` is incremented by 1 AND `storage_uri.fallback_walk_ms{outcome:"ok"}` is recorded once
|
||||
|
||||
#### Scenario: parent span carries policy attributes
|
||||
|
||||
- **WHEN** any public operation is invoked with `policy.maxParallel: 2`
|
||||
- **THEN** the parent span carries attribute `fallback.policy.max_parallel === 2`
|
||||
|
||||
**Verified by:** `src/observability/otel.test.ts::"increments fallback_total per candidate and records walk_ms"` AND `src/observability/otel.test.ts::"records policy attributes on parent span"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: EventBusLike emissions for observability hooks
|
||||
|
||||
When `options.events: EventBusLike` is provided, the plugin SHALL emit events on the topics `storage-uri.fallback.attempt-failed`, `storage-uri.fallback.attempt-succeeded`, `storage-uri.fallback.write-mirror-partial`, `storage-uri.fallback.cycle-detected`, and `storage-uri.fallback.depth-exceeded`. Each event payload SHALL include `uri` (the original `fallback://` URI), `candidateIndex` (where applicable), `candidateUri`, and a code or cause discriminant per topic. When no `events` are provided the plugin SHALL be a no-op on the bus path.
|
||||
|
||||
#### Scenario: attempt-failed event payload contains candidate metadata
|
||||
|
||||
- **WHEN** `read(buildFallbackUri(["mem://miss", "mem://hit"]))` is called with an `EventBusLike` provided and the first candidate raises `NOT_FOUND`
|
||||
- **THEN** `events.emit` is invoked with topic `"storage-uri.fallback.attempt-failed"` and payload containing `candidateIndex: 0`, `candidateUri: "mem://miss"`, `code: "STORAGE_URI.NOT_FOUND"`
|
||||
|
||||
#### Scenario: omitted events option is a no-op
|
||||
|
||||
- **WHEN** the plugin is constructed without `options.events` and a candidate fails
|
||||
- **THEN** no error is raised by the missing-bus path AND the operation otherwise behaves identically
|
||||
|
||||
**Verified by:** `src/observability/events.test.ts::"emits attempt-failed with candidate metadata"` AND `src/observability/events.test.ts::"is a no-op when no EventBusLike is provided"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Fallback-specific conformance harness
|
||||
|
||||
The package SHALL export `runFallbackConformance(resolver, fixtures)` that consumers call from their own tests to verify their wiring exercises the fallback contract end-to-end. The harness SHALL assert that:
|
||||
|
||||
- With N candidates where the first K fail with fall-through-eligible errors and the (K+1)th succeeds, the result equals the (K+1)th candidate's value and K events are emitted on the provided bus.
|
||||
- All-candidates-fail throws `FALLBACK_URI.ALL_CANDIDATES_FAILED` with `candidateErrors.length === N`.
|
||||
- A pre-aborted signal short-circuits with `FALLBACK_URI.ABORTED` and zero dispatches occur.
|
||||
- A self-referencing `fallback://` URI yields `FALLBACK_URI.CIRCULAR_REFERENCE`.
|
||||
- A chain longer than `policy.maxDepth` yields `FALLBACK_URI.MAX_DEPTH_EXCEEDED`.
|
||||
- Mirror writes under `writeMode: "all"` hit every candidate (verified via direct reads).
|
||||
|
||||
#### Scenario: harness passes against the in-package plugin
|
||||
|
||||
- **WHEN** `runFallbackConformance(resolver, fixtures)` is invoked with a resolver carrying the fallback plugin plus two `MemoryStorage`-backed plugins
|
||||
- **THEN** every assertion passes without throwing
|
||||
|
||||
#### Scenario: harness fails when the resolver omits the fallback plugin
|
||||
|
||||
- **WHEN** `runFallbackConformance(resolver, fixtures)` is invoked against a resolver missing the fallback plugin
|
||||
- **THEN** the harness throws with a message naming the missing `fallback` scheme
|
||||
|
||||
**Verified by:** `src/conformance/fallback-conformance.test.ts::"passes against the in-package plugin"` AND `src/conformance/fallback-conformance.test.ts::"fails when the resolver omits the fallback plugin"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Tier purity — universal, no Node/Bun/DOM dependencies
|
||||
|
||||
The package SHALL be universal-tier. The implementation SHALL NOT import from `node:*`, `bun:*`, or any DOM-only API. The grammar parser SHALL use the Web-standard `URL` and `URLSearchParams` only. `AbortSignal.any` SHALL be feature-detected with a runtime polyfill fallback for environments lacking ES2024 support. Time-keeping SHALL go through the injected `Clock` from `@suaveplan/types`. The package's `package.json` SHALL declare no `node:*`-only peer dependencies and SHALL maintain `"sideEffects": false`.
|
||||
|
||||
#### Scenario: source tree contains no Node or Bun specifier imports
|
||||
|
||||
- **WHEN** the verification command `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-uri-plugin-fallback/src/` is executed
|
||||
- **THEN** the command produces no matches
|
||||
|
||||
#### Scenario: AbortSignal.any shim activates on runtimes without native support
|
||||
|
||||
- **WHEN** the package is loaded under a runtime where `AbortSignal.any` is undefined
|
||||
- **THEN** the exported shim provides equivalent composition behaviour validated by the unit test suite
|
||||
|
||||
**Verified by:** `src/parse-uri/parse-fallback-uri.test.ts::"abort-any shim composes signals identically to native"` AND `src/plugin/plugin.test.ts::"tier-purity grep produces no matches"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,113 @@
|
||||
# Tasks — add-storage-uri-plugin-fallback
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `openspec/AGENTS.md` §0, §0a, §0b, §0c, §13, §14
|
||||
- [ ] 1.2 Read `openspec/project.md`
|
||||
- [ ] 1.3 Read `openspec/specs/storage-uri/spec.md` to confirm `UriPlugin` + `StorageUriResolver` contract
|
||||
- [ ] 1.4 Read `packages/storage/storage-uri/src/` (resolver, types, conformance) for recursive-dispatch surface
|
||||
- [ ] 1.5 Read `packages/flow/retry/src/` to confirm `RetryStrategy` surface for documentation cross-links
|
||||
- [ ] 1.6 Read `openspec/changes/complete-storage-mux/` for the multi-backend failover precedent (error classification, fan-out semantics, span hierarchy)
|
||||
- [ ] 1.7 Draft `proposal.md`, `design.md`, `specs/storage-uri-plugin-fallback/spec.md`, `tasks.md`
|
||||
- [ ] 1.8 Numbered Requirement/Scenario checklist per §0c step 6 — every requirement and every scenario in the delta tracked as an unchecked item until implementation lands
|
||||
|
||||
## 2. Scaffolding
|
||||
|
||||
- [ ] 2.1 Create `packages/storage/storage-uri-plugin-fallback/`
|
||||
- [ ] 2.2 `package.json` v0.0.0, `"sideEffects": false`, subpath exports per feature directory
|
||||
- [ ] 2.3 Deps: `@suaveplan/error`, `@suaveplan/events`, `@suaveplan/storage-uri`, `@suaveplan/types`, `@opentelemetry/api`, `zod`, `tslib`
|
||||
- [ ] 2.4 DevDeps: `@suaveplan/storage`, `@suaveplan/storage-file`, `@suaveplan/testing`, `@vitest/coverage-istanbul`, `fast-check`, `mitata`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`
|
||||
- [ ] 2.5 `tsconfig.json`, `vite.config.ts`, `vitest.config.ts` via `createTestConfig` with `thresholds: 100/100/100/100`
|
||||
- [ ] 2.6 Root `README.md` with Modules section linking every co-located `.md`
|
||||
|
||||
## 3. Error registry
|
||||
|
||||
- [ ] 3.1 `src/error-codes/error-codes.ts` — `defineCodes("FALLBACK_URI", [...])` with all 8 codes: `NO_CANDIDATES`, `INVALID_URI`, `ALL_CANDIDATES_FAILED`, `WRITE_MIRROR_FAILED`, `LIST_UNION_FAILED`, `ABORTED`, `MAX_DEPTH_EXCEEDED`, `CIRCULAR_REFERENCE`
|
||||
- [ ] 3.2 `FallbackUriError extends SuaveplanError<FallbackUriErrorCode>` with `uri`, `candidateIndex`, `candidateUri`, `candidateErrors`, `mirrorOutcomes`, `cycleChain`, `depth` discriminants (only the ones relevant to each code are populated)
|
||||
- [ ] 3.3 `error-codes.md` + `error-codes.test.ts` — reachability sweep proving every code is thrown from a real call site
|
||||
|
||||
## 4. Types & schemas
|
||||
|
||||
- [ ] 4.1 `src/types/policy.types.ts` — `FallbackPolicy` (`errorClass`, `maxParallel`, `stopOnAbort`, `writeMode`, `deleteMode`, `listMode`, `listFailureMode`, `maxDepth`, `maxCandidates`), `FallbackPluginOptions`
|
||||
- [ ] 4.2 `src/types/policy.schemas.ts` — Zod schemas: `maxParallel >= 1`, `maxDepth >= 1 && <= 8`, `maxCandidates >= 1 && <= 64`, mode enums
|
||||
- [ ] 4.3 `src/types/candidate.types.ts` — `CandidateError`, `CandidateAttempt`, `MirrorOutcome`
|
||||
|
||||
## 5. URI grammar
|
||||
|
||||
- [ ] 5.1 `src/parse-uri/parse-fallback-uri.ts` — `parseFallbackUri(uri)` returning `{ candidates: readonly string[]; canonicalUri: string }`; `buildFallbackUri(uris)` with `URLSearchParams`; reject empty / oversized / non-`fallback` scheme / non-empty authority
|
||||
- [ ] 5.2 `src/parse-uri/parse-fallback-uri.md` + `.test.ts` — exhaustive validation matrix; fast-check round-trip property `buildFallbackUri ∘ parseFallbackUri = identity` on a small generator
|
||||
- [ ] 5.3 `src/parse-uri/abort-any.ts` — `AbortSignal.any` shim guarded by `typeof AbortSignal.any !== "function"`
|
||||
|
||||
## 6. Classification
|
||||
|
||||
- [ ] 6.1 `src/classify/classify.ts` + `.md` + `.test.ts` — pure `classify(err, policy) => "transient" | "semantic" | "abort"` per design §7; consumes `@suaveplan/error#SuaveplanError` for code inspection
|
||||
|
||||
## 7. Recursion context
|
||||
|
||||
- [ ] 7.1 `src/recursion/depth-context.ts` + `.md` + `.test.ts` — `DepthContext` value type, `withChildContext` helper that bumps depth + extends visited set, depth-cap enforcement throwing `MAX_DEPTH_EXCEEDED`, visited-set canonicalisation via `parseFallbackUri` reserialisation
|
||||
|
||||
## 8. Walk strategies
|
||||
|
||||
- [ ] 8.1 `src/walk/sequential.ts` + `.md` + `.test.ts` — ordered fall-through; emits attempt-failed events; throws `ALL_CANDIDATES_FAILED` on exhaustion with `candidateErrors` populated
|
||||
- [ ] 8.2 `src/walk/parallel.ts` + `.md` + `.test.ts` — `Promise.any`-style race with concurrency cap (`maxParallel`); cancels pending candidates via `AbortSignal.any` once one succeeds
|
||||
|
||||
## 9. Operations (full UriPlugin surface)
|
||||
|
||||
- [ ] 9.1 `src/ops/read.ts` + `.md` + `.test.ts` — `get(key)` + `createReadStream(key, signal)` + range `{ offset, length }` propagation
|
||||
- [ ] 9.2 `src/ops/write.ts` + `.md` + `.test.ts` — `set(key, bytes)` + `createWriteStream(key, body, signal)`; `writeMode: "all"` → `Promise.allSettled` + `WRITE_MIRROR_FAILED`; `writeMode: "first-success"` → walk + background settle
|
||||
- [ ] 9.3 `src/ops/metadata.ts` + `.md` + `.test.ts` — `exists` + `getMetadata` sequential walk
|
||||
- [ ] 9.4 `src/ops/delete.ts` + `.md` + `.test.ts` — mirror delete with `deleteMode: "any-success" | "all"` (default `"any-success"`)
|
||||
- [ ] 9.5 `src/ops/list.ts` + `.md` + `.test.ts` — union with dedup; `listMode: "union" | "concat"`; `listFailureMode: "throw" | "partial"`
|
||||
|
||||
## 10. Observability
|
||||
|
||||
- [ ] 10.1 `src/observability/events.ts` + `.md` + `.test.ts` — topic constants (`storage-uri.fallback.attempt-failed`, `.attempt-succeeded`, `.write-mirror-partial`, `.cycle-detected`, `.depth-exceeded`); emit helpers consuming `EventBusLike`
|
||||
- [ ] 10.2 `src/observability/otel.ts` + `.md` + `.test.ts` — counter `storage_uri.fallback_total{candidate_index, scheme, outcome}`, histogram `storage_uri.fallback_walk_ms{outcome}`, parent + per-attempt child spans with attribute set
|
||||
|
||||
## 11. Plugin factory
|
||||
|
||||
- [ ] 11.1 `src/plugin/plugin.ts` + `.md` — `fallbackUriPlugin(options)`; wires walk + ops + observability + recursion; binds to `fallback` scheme; produces `UriPlugin` whose `adapter` exposes the full streaming-extended `StorageLike` surface
|
||||
- [ ] 11.2 `src/plugin/plugin.test.ts` — construction validation (missing resolver, bad policy), abort short-circuit, sequential success, sequential exhaustion, parallel race, mirror write, cycle, depth cap
|
||||
- [ ] 11.3 `src/plugin/plugin.spec-conformance.test.ts` — `runUriPluginConformance(plugin, fixtures)` pass with `validUris` / `invalidUris` / `buildRoundTrip` populated from `parseFallbackUri` / `buildFallbackUri`
|
||||
|
||||
## 12. Fallback-specific conformance
|
||||
|
||||
- [ ] 12.1 `src/conformance/fallback-conformance.ts` + `.md` — `runFallbackConformance(resolver, fixtures)` exporting the harness consumers (audio-IR loader, drive+CDN pairing) call from their own tests
|
||||
- [ ] 12.2 `src/conformance/fallback-conformance.test.ts` — self-test against the in-package plugin: K-fail-then-success, all-fail, abort, cycle, depth, mirror-write
|
||||
|
||||
## 13. Barrel + exports
|
||||
|
||||
- [ ] 13.1 `src/index.ts` — explicit named re-exports per §6a (NO `export *`)
|
||||
- [ ] 13.2 `package.json` subpath exports — one entry per feature directory; verify count matches
|
||||
|
||||
## 14. Documentation
|
||||
|
||||
- [ ] 14.1 Sibling `.md` for every implementation `.ts` (≥ 200 words, all required sections, real-import examples)
|
||||
- [ ] 14.2 `README.md` — install, URI grammar, simple example (`fallbackUriPlugin` registered alongside other plugins), advanced example (mirror writes + dedup list across CDN + drive), policy reference, complementarity note linking to `@suaveplan/storage-mux`
|
||||
|
||||
## 15. Gates
|
||||
|
||||
- [ ] 15.1 `bun run check`
|
||||
- [ ] 15.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-uri-plugin-fallback`
|
||||
- [ ] 15.3 `bunx turbo test:coverage --filter=@suaveplan/storage-uri-plugin-fallback` (100/100/100/100)
|
||||
- [ ] 15.4 `bun scripts/lint/check-test-runner-imports.ts` → zero output for this package
|
||||
- [ ] 15.5 Tier-purity grep `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-uri-plugin-fallback/src/` → empty
|
||||
- [ ] 15.6 Banned-pattern scan `grep -rn "@deprecated\|TODO\|FIXME\|ts-ignore\|ts-expect-error\|\.skip\|\.only" packages/storage/storage-uri-plugin-fallback/src/ --include="*.ts"` → empty
|
||||
- [ ] 15.7 Barrel-shape scan per §6a → empty
|
||||
- [ ] 15.8 Bare-version scan → empty
|
||||
- [ ] 15.9 Missing `.md` scan → empty
|
||||
- [ ] 15.10 README present
|
||||
- [ ] 15.11 Subpath export count matches feature module count
|
||||
- [ ] 15.12 `bun scripts/openspec-conformance.ts @suaveplan/storage-uri-plugin-fallback`
|
||||
- [ ] 15.13 Dependent sweep `bunx turbo test --filter=...@suaveplan/storage-uri-plugin-fallback`
|
||||
|
||||
## 16. Manifest & archival
|
||||
|
||||
- [ ] 16.1 `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 16.2 `bun scripts/generate-missing.ts`
|
||||
- [ ] 16.3 Update `STATUS.md` and `SUMMARY.md`
|
||||
- [ ] 16.4 `bunx markdownlint-cli2 --fix`
|
||||
- [ ] 16.5 Fold `specs/storage-uri-plugin-fallback/spec.md` → `openspec/specs/storage-uri-plugin-fallback/spec.md`
|
||||
- [ ] 16.6 Move change folder → `openspec/archive/<yyyy-mm-dd>-add-storage-uri-plugin-fallback/`
|
||||
Reference in New Issue
Block a user