Initial Commit

This commit is contained in:
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions

View File

@@ -0,0 +1,221 @@
# Design — @suaveplan/storage-uri
## Tier and dependency posture
Universal tier. Zero `node:`/`bun:` imports. Runtime deps:
- `@suaveplan/error` — `defineCodes`, `SuaveplanError`
- `@suaveplan/types` — `StorageLike` structural contract
- `@opentelemetry/api` — meter + tracer
- `tslib` — runtime helpers
`@suaveplan/storage` is **not** a dep. The resolver types against `StorageLike` from `@suaveplan/types` so an in-memory test stub satisfies the contract without pulling in the `AbstractStorage` runtime.
## File layout
```
packages/storage/storage-uri/
package.json
README.md
src/
index.ts # explicit named re-exports (rule 10)
error-codes/
error-codes.ts
error-codes.test.ts
error-codes.md
index.ts
parse-uri/
parse-uri.ts # RFC 3986 split — scheme + authority + path + query
parse-uri.test.ts
parse-uri.md
index.ts
types/
plugin.types.ts # UriPlugin, ParsedUri
plugin.types.md
resolver.types.ts # ResolverOptions, StatResult
resolver.types.md
index.ts
resolver/
resolver.ts # StorageUriResolver, dispatch + delegation
resolver.test.ts
resolver.streams.ts # open() / createReadStream fallback
resolver.streams.test.ts
resolver.writes.ts # write() / createWriteStream drain
resolver.writes.test.ts
resolver.list.ts # list() prefix → URIs re-serialisation
resolver.list.test.ts
resolver.spec-conformance.test.ts
resolver.md
index.ts
conformance/
conformance.ts # runUriPluginConformance(plugin, fixtures)
conformance.test.ts
conformance.md
index.ts
otel/
otel.ts
otel.test.ts
otel.md
index.ts
tests/
setup/polyfills.ts
integration/cross-scheme.test.ts # storage-file + storage-http composition
tsconfig.json
tsconfig.lib.json
vite.config.ts
vitest.config.ts
```
Every `.ts` outside the listed exclusions (`.test.ts`, `.types.ts` inside a feature dir, `index.ts` barrels) has a sibling `.md`.
## Resolver dispatch
```ts
class StorageUriResolver {
readonly #plugins: ReadonlyMap<string, UriPlugin>;
constructor(plugins: readonly UriPlugin[]) {
const map = new Map<string, UriPlugin>();
for (const plugin of plugins) {
const existing = map.get(plugin.scheme);
if (existing && existing !== plugin) {
throw new StorageUriError("SCHEME_CONFLICT", { scheme: plugin.scheme });
}
map.set(plugin.scheme, plugin);
}
this.#plugins = map;
}
#dispatch(uri: string): { plugin: UriPlugin; key: string; meta?: Record<string, string> } {
const parsed = parseUri(uri); // throws INVALID_URI
const plugin = this.#plugins.get(parsed.scheme); // throws UNKNOWN_SCHEME
if (!plugin) throw new StorageUriError("UNKNOWN_SCHEME", { uri, scheme: parsed.scheme });
return { plugin, ...plugin.parse(uri) };
}
}
```
Dispatch is O(1) in plugin count. The resolver is frozen after construction; mutation is impossible.
## Stream fallback strategy
```ts
async open(uri: string, signal?: AbortSignal): Promise<ReadableStream<Uint8Array>> {
const { plugin, key } = this.#dispatch(uri);
const adapter = plugin.adapter as StorageLike & { createReadStream?(k: string, s?: AbortSignal): ReadableStream<Uint8Array> };
if (typeof adapter.createReadStream === "function") {
return adapter.createReadStream(key, signal);
}
// Fallback: one-chunk stream from get()
this.#otel.streamFallback(plugin.scheme);
const bytes = await adapter.get(key);
if (!bytes) throw new StorageUriError("NOT_FOUND", { uri });
return new ReadableStream({
start(ctrl) { ctrl.enqueue(bytes); ctrl.close(); },
});
}
```
The fallback path materialises the whole object in memory. The `storage_uri.stream_fallback_total{scheme}` counter is the operator's alert signal that a misuse is occurring (large object behind a non-streaming adapter).
## Write buffering strategy
Symmetric: `write(uri, body)` accepts `Uint8Array` or `ReadableStream<Uint8Array>`. Streams are drained via `Response(body).arrayBuffer()` into a single `Uint8Array`, then handed to `adapter.set(key, bytes)`. Counter: `storage_uri.buffered_write_total{scheme}`.
Adapters with native `createWriteStream` SHALL be used directly when present — the fallback runs only when the adapter has no streaming write surface.
## Per-scheme parse contract
Each adapter package owns its scheme parser. Examples:
| Scheme | Plugin owner | `parse(uri)` returns |
|---|---|---|
| `file:` | `storage-file` | `{ key: <abs path resolved against rootDir>, meta: {} }` |
| `http:` / `https:` | `storage-http` | `{ key: <pathname>, meta: { query: search } }` |
| `rackspace:` | `storage-cdn-rackspace` | `{ key: <pathname>, meta: { container: <authority> } }` |
| `bunny:` | `storage-cdn-bunnycdn` | `{ key: <pathname>, meta: { zone: <authority> } }` |
| `s3:` | `storage-s3` | `{ key: <pathname>, meta: { bucket: <authority> } }` |
Adapters that need scheme-specific addressing (bucket, container, zone) populate `meta` and are responsible for forwarding those values to their pre-bound adapter instance. The resolver never inspects `meta`.
## Error-code map
| Code | When | HTTP-ish analogue |
|---|---|---|
| `STORAGE_URI.UNKNOWN_SCHEME` | URI scheme has no plugin registered | 404 (route miss) |
| `STORAGE_URI.INVALID_URI` | `parseUri` rejects the input | 400 |
| `STORAGE_URI.SCHEME_CONFLICT` | Two distinct plugins claim the same scheme | configuration error |
| `STORAGE_URI.NOT_FOUND` | Adapter returned `undefined` for `get`/`open` | 404 |
| `STORAGE_URI.LIST_UNSUPPORTED` | Plugin has no `build()` | 501 |
| `STORAGE_URI.WRITE_UNSUPPORTED` | Adapter throws on `set` (e.g. read-only HTTP) | 405 |
| `STORAGE_URI.ABORTED` | `AbortSignal` fired | 499 |
| `STORAGE_URI.ADAPTER_ERROR` | Adapter threw something else | 502 (upstream) |
`ADAPTER_ERROR` preserves the original via `cause` — operators trace back to the underlying `STORAGE_HTTP.*`/`STORAGE_FILE.*`/`STORAGE_S3.*` code.
## Conformance harness
```ts
export function runUriPluginConformance(plugin: UriPlugin, fx: {
validUris: string[];
invalidUris: string[];
buildRoundTrip?: Array<{ key: string; meta?: Record<string, string>; uri: string }>;
}): void;
```
Called from each adapter package's test suite (e.g. `storage-file/src/uri-plugin/uri-plugin.conformance.test.ts`). Asserts:
1. Every `validUris[i]` parses without throwing, and `plugin.parse(uri).key` is a non-empty string.
2. Every `invalidUris[i]` throws `STORAGE_URI.INVALID_URI` (or a wrapped equivalent).
3. If `build` is implemented, `build(parse(uri)) === uri` for the round-trip fixtures.
4. The plugin's scheme matches the URI's scheme prefix.
This is the bridge between the resolver's spec and each adapter's contribution — the spec stays in `storage-uri`, the test fixtures live with the adapter.
## OTel surface
Tracer: `@suaveplan/storage-uri`. Meter: same.
Spans: one per public op (`storage.uri.read`, `storage.uri.write`, `storage.uri.stat`, `storage.uri.exists`, `storage.uri.delete`, `storage.uri.list`, `storage.uri.open`). Attributes:
- `storage.uri` — the full input URI (redacted to authority + path; query string dropped to avoid signed-URL leaks).
- `storage.uri.scheme` — extracted scheme (no colon).
- `storage.uri.key` — post-parse key (subject to adapter-side redaction).
Histograms:
- `storage_uri.op_duration_ms{op, scheme, outcome}` — `outcome` ∈ `success` / `not_found` / `error` / `aborted`.
Counters:
- `storage_uri.reads_total{scheme, outcome}`
- `storage_uri.writes_total{scheme, outcome}`
- `storage_uri.deletes_total{scheme, outcome}`
- `storage_uri.errors_total{scheme, code}` — `code` is the `STORAGE_URI.*` enum (`ADAPTER_ERROR` covers all downstream)
- `storage_uri.stream_fallback_total{scheme}`
- `storage_uri.buffered_write_total{scheme}`
## Composition recipes (in README)
```ts
// Single resolver for a media pipeline.
const resolver = createStorageUriResolver([
fileUriPlugin({ rootDir: "/var/media" }),
httpsUriPlugin(httpStorage),
rackspaceUriPlugin(rackspace, "media-prod"),
bunnyUriPlugin(bunny, "cdn-prod"),
]);
const stream = await resolver.open("rackspace://media-prod/raw/clip.mov");
await resolver.write("file:///cache/thumbs/clip.jpg", thumbBytes);
const meta = await resolver.stat("https://cdn.example.com/clip/master.m3u8");
```
## Trade-offs accepted
- **No global registry.** Every consumer constructs its own resolver. Cost: a little boot-time wiring. Benefit: tree-shaking works, tests can build isolated resolvers, multi-tenant configurations are explicit.
- **No copy/move primitives.** Reason given in proposal Out of Scope. A future `storage-pipeline` package can offer streaming copy with progress, retries, and integrity checks.
- **`meta` is a `Record<string, string>`.** Adapter-specific addressing (bucket, container, zone) is stringly-typed at the plugin interface; type safety is the plugin's job (each adapter exports its own typed `parseXxxUri`).
- **`stat()` returns `undefined` for adapters that can't answer cheaply.** Avoids forcing a `HEAD` request inside `storage-file` or a full-object read inside CDN adapters that don't support `HEAD`. Consumers that need metadata SHOULD compose with `storage-http`'s ETag cache directly.

View File

@@ -0,0 +1,93 @@
# Add @suaveplan/storage-uri — URI-routed dispatch over StorageLike adapters
**Package**: `@suaveplan/storage-uri`
**Status**: PROPOSED
**Type**: new-package (genesis-original)
- **Design**: ./design.md
- **Spec delta**: ./specs/storage-uri/spec.md
## Why
Genesis now ships seven `StorageLike`-conformant adapters — `storage-file`, `storage-http`, `storage-cdn`, `storage-cdn-rackspace`, `storage-cdn-bunnycdn`, `storage-s3`, and three `storage-browser-*` flavours — but every consumer has to pick **one** adapter at construction time and address its keys directly. Real workloads cross adapters: a media pipeline reads source from `s3://uploads/raw/clip.mov`, a poster from `file:///var/cache/posters/clip.jpg`, and a delivery copy from `https://cdn.example.com/clip/master.m3u8` — all in the same call site. Today that means three injected adapters, three key conventions, and bespoke fan-out logic.
`storage-uri` collapses that to **one string in, one stream out**. A consumer hands `read("s3://bucket/key")` to a resolver that owns a `scheme → { adapter, parseUri }` registry, gets a `ReadableStream<Uint8Array>` back, and never has to know which adapter was selected. The registry is composed once at boot from any subset of the existing storage packages; each adapter package ships its own `uriPlugin` so the resolver stays scheme-agnostic and dependency-light.
The package is a **router**, not a new storage tier. It performs no I/O of its own — every operation delegates to a registered `StorageLike` after URI parsing.
## What changes
- ADDED: `StorageUriResolver` class — owns a `Map<scheme, UriPlugin>` registry, exposes `open`/`read`/`write`/`stat`/`exists`/`remove`/`list` keyed by URI string.
- ADDED: `createStorageUriResolver(plugins)` factory — accepts an array of `UriPlugin` and produces a frozen resolver. Idempotent re-registration is a no-op when the plugin reference is identical; conflicting registrations throw `STORAGE_URI.SCHEME_CONFLICT`.
- ADDED: `UriPlugin` interface — `{ scheme: string; adapter: StorageLike; parse(uri): { key: string; meta?: Record<string, string> }; build?(key, meta?): string }`. The `adapter` field is `StorageLike` (not `AbstractStorage`) so the resolver works against the structural contract from `@suaveplan/types`.
- ADDED: `parseUri(uri)` standalone helper — RFC 3986 scheme + authority + path split that adapter plugins compose with their own scheme-specific addressing (`s3://bucket/key`, `file:///abs/path`, `bunny://zone/path`, `https://host/path`).
- ADDED: `open(uri, signal?): Promise<ReadableStream<Uint8Array>>` — single-shot streaming read. When the underlying adapter exposes `createReadStream`, it is forwarded; otherwise the resolver falls back to wrapping `adapter.get(key)` in a one-chunk `ReadableStream`. The fallback path is observable via `storage_uri.stream_fallback_total`.
- ADDED: `read(uri, signal?): Promise<Uint8Array>` — buffered read. Delegates straight to `adapter.get(key)`; throws `STORAGE_URI.NOT_FOUND` when the adapter returns `undefined`.
- ADDED: `write(uri, body, signal?): Promise<void>` — accepts `Uint8Array` or `ReadableStream<Uint8Array>`. Streams are drained into a single `set` call when the adapter lacks `createWriteStream`; the buffered fallback is observable via `storage_uri.buffered_write_total`.
- ADDED: `stat(uri, signal?): Promise<{ size?: number; etag?: string; lastModified?: Date; contentType?: string } | undefined>` — surfaces adapter metadata when available; returns `undefined` for adapters that cannot answer without a full read.
- ADDED: `exists(uri, signal?): Promise<boolean>` and `remove(uri, signal?): Promise<boolean>` — direct delegates to `StorageLike.exists` and `StorageLike.delete` after parse.
- ADDED: `list(uri, options?, signal?): Promise<{ uris: string[]; hasMore: boolean; continuationToken?: string }>` — the URI is treated as a prefix; results are re-serialised back to URIs via `plugin.build()`. Plugins without a `build()` function throw `STORAGE_URI.LIST_UNSUPPORTED`.
- ADDED: Sibling `uriPlugin` exports in every storage adapter package — `storage-file` ships `fileUriPlugin(rootDir)`, `storage-http` ships `httpsUriPlugin(httpStorage)` and `httpUriPlugin(httpStorage)`, `storage-cdn-rackspace` ships `rackspaceUriPlugin(adapter, container)`, `storage-cdn-bunnycdn` ships `bunnyUriPlugin(adapter, zone)`, `storage-s3` ships `s3UriPlugin(adapter, bucket)`. Each owns its scheme parser; the resolver knows nothing about S3 buckets, CDN zones, or filesystem roots.
- ADDED: `STORAGE_URI_CODES` registry via `defineCodes("STORAGE_URI", [...])` — 8 codes: `UNKNOWN_SCHEME`, `INVALID_URI`, `SCHEME_CONFLICT`, `NOT_FOUND`, `LIST_UNSUPPORTED`, `WRITE_UNSUPPORTED`, `ABORTED`, `ADAPTER_ERROR`.
- ADDED: `StorageUriError extends SuaveplanError<StorageUriErrorCode>` with `uri` and `scheme` discriminants; adapter-originated errors are re-wrapped with `ADAPTER_ERROR` preserving the original as `cause`.
- ADDED: OTel surface — `storage_uri.op_duration_ms{op,scheme,outcome}` histogram; `storage_uri.{reads,writes,deletes,errors,stream_fallback,buffered_write}_total{scheme,outcome}` counters; per-op span on `@suaveplan/storage-uri` tracer with `storage.uri.scheme`, `storage.uri.key`, `storage.uri` attributes.
- ADDED: `runUriPluginConformance(plugin, fixtures)` test harness — every adapter package's `uriPlugin` SHALL pass this conformance suite (round-trip parse↔build, key boundary forwarding, list re-serialisation, error mapping).
## Out of scope
- **A new storage adapter.** `storage-uri` performs zero I/O; every byte goes through an existing `StorageLike`. Reason: this is a routing concern, not a storage tier.
- **Cross-scheme copy/move.** `copy("s3://a/x", "file:///b/x")` is tempting but commits the resolver to streaming semantics, transactional rollback, and bandwidth governance that belong in a dedicated `storage-pipeline` package. Reason: scope creep; the consumer can compose `read` + `write` today.
- **URI signing.** Signed URLs (S3 presigned, CDN token-auth) are produced by the underlying adapters; the resolver MUST NOT introduce its own signing layer. Reason: every provider's signing scheme is different and already lives in the adapter package.
- **Caching.** Repeat reads of the same URI hit the adapter every time. A caching resolver belongs in a separate `storage-uri-cache` (compose with `cache-storage`). Reason: cache invalidation policy is application-specific.
- **Schema federation across resolvers.** A single resolver owns one plugin per scheme. Multiple Rackspace containers behind one `rackspace://` scheme need to be addressed by including the container in the authority (`rackspace://container-a/key`) — handled inside `rackspaceUriPlugin`. Reason: keeps the resolver O(1) on dispatch.
- **Browser-tier auto-registration.** The resolver is universal-tier; consumers register plugins explicitly. No global mutable registry. Reason: mutable singletons break testability and tree-shaking.
## Source mapping
This is a **genesis-original** package; there is no upstream in `../suaveplan/`. All requirements derive from the `StorageLike` contract in `@suaveplan/types` (`openspec/specs/types/spec.md`) and the existing adapter surface.
### Requirements with no source line
| Requirement | Justification |
|---|---|
| Type against `StorageLike`, not `AbstractStorage` | Keeps the resolver in the universal tier with a zero-runtime dep on `@suaveplan/types`. Adapters that extend `AbstractStorage` satisfy `StorageLike` structurally. |
| Plugins live in each adapter package, not in `storage-uri` | Avoids forcing the resolver to depend on every storage provider. Consumers pull in only the adapter packages they actually use; the resolver itself stays tiny. |
| `open()` returns `ReadableStream<Uint8Array>` | Web Streams is the universal cross-runtime stream type (Node ≥18, Bun, browsers, Deno). Matches `Response.body` from `storage-http` and existing `createReadStream` signatures. |
| `stream_fallback_total` + `buffered_write_total` counters | Quietly upgrading a buffered adapter to a "streaming" surface is a memory-pressure hazard. Operators alert on these counters to identify mis-composed adapters. |
| `parse(uri) → { key, meta }` rather than `{ key }` alone | CDN zones, S3 buckets, Rackspace containers all need an extra addressing dimension; `meta` is the documented escape hatch so the resolver doesn't grow a separate "authority" field per scheme. |
| `LIST_UNSUPPORTED` distinct from `WRITE_UNSUPPORTED` | Listing requires `plugin.build()`; write requires `adapter` write capability. They're independently optional and need distinct error codes for capability probing. |
| Idempotent registration is a no-op only when the plugin reference is identical | Reference equality is the cheapest correct test; deep-equality would mask configuration drift between modules that import the same scheme. |
## Verification approach
Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete `it(...)` titles in these files:
- `src/resolver/resolver.test.ts` — registry construction, scheme dispatch, conflict detection, idempotent re-registration, `UNKNOWN_SCHEME` for unregistered scheme, AbortSignal propagation to every operation.
- `src/resolver/resolver.streams.test.ts` — `open` forwards native `createReadStream` when present, falls back to one-chunk wrap when absent, `stream_fallback_total` increments on fallback path.
- `src/resolver/resolver.writes.test.ts` — `write` accepts `Uint8Array` straight, drains `ReadableStream` into single `set` when adapter lacks `createWriteStream`, `buffered_write_total` increments on drain path.
- `src/resolver/resolver.spec-conformance.test.ts` — type-level conformance against `StorageLike`, every `STORAGE_URI` code reachable, OTel instruments observed via real in-memory `MeterProvider`.
- `src/parse-uri/parse-uri.test.ts` — RFC 3986 happy paths (`s3://bucket/key`, `file:///abs/path`, `https://host/path?q=1`), invalid forms (`bare-string`, `scheme:`, embedded null), `INVALID_URI` mapping.
- `src/error-codes/error-codes.test.ts` — exhaustive code registry, `StorageUriError` discriminant fields, adapter-error re-wrap preserves `cause`.
- `src/otel/otel.test.ts` — metrics via real in-memory `MeterProvider`, error span via in-memory `SpanExporter`, scheme attribute present on every emission.
- `src/conformance/conformance.test.ts` — `runUriPluginConformance` against a fixture in-memory `StorageLike` and a fixture `parse`/`build` pair.
- `tests/integration/cross-scheme.test.ts` — composes a resolver with `storage-file` + `storage-http` real adapters; reads `file:///tmp/...` and `https://...` round-trip without code path divergence.
## Impact
### Affected dependents
The package is new — no dependents currently. After adopting, the obvious upstream beneficiaries are `drive-template` (URI-addressed hydration sources), `enterprise-media`, the audio/video pipelines, and any consumer that currently injects multiple `StorageLike`s side-by-side.
- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-uri` returning the package itself).
### Affected upstreams (sibling adapter packages — additive `uriPlugin` export)
| Package | Action | Breaking? |
|---|---|---|
| `@suaveplan/storage-file` | ADD `fileUriPlugin(rootDir)` named export | no |
| `@suaveplan/storage-http` | ADD `httpsUriPlugin(adapter)` and `httpUriPlugin(adapter)` named exports | no |
| `@suaveplan/storage-cdn-rackspace` | ADD `rackspaceUriPlugin(adapter, container)` named export | no |
| `@suaveplan/storage-cdn-bunnycdn` | ADD `bunnyUriPlugin(adapter, zone)` named export | no |
| `@suaveplan/storage-s3` | ADD `s3UriPlugin(adapter, bucket)` named export | no |
All additions are purely additive named exports — no existing surface changes. Each adapter package gets a separate spec delta in this change folder ONLY if its `uriPlugin` introduces non-trivial requirements; otherwise the export is documented in the adapter's existing spec and verified by the conformance harness.

View File

@@ -0,0 +1,238 @@
<!-- markdownlint-disable MD024 -->
# Capability Spec Delta: storage-uri (add-storage-uri)
**Package**: `@suaveplan/storage-uri`
**Status**: PROPOSED
> Every requirement carries a `**Verified by:**` line at authoring time.
## ADDED Requirements
### Requirement: URI scheme dispatch
The resolver SHALL select a registered `UriPlugin` by the scheme component of the input URI and SHALL throw `STORAGE_URI.UNKNOWN_SCHEME` when no plugin is registered for that scheme. The lookup SHALL be case-insensitive on the scheme per RFC 3986 §3.1.
#### Scenario: registered scheme dispatches to its plugin
- **WHEN** `read("s3://bucket/key")` is called against a resolver with an `s3` plugin
- **THEN** the `s3` plugin's `parse` is invoked exactly once and the resulting `adapter.get(key)` call returns the stored bytes
#### Scenario: unregistered scheme throws UNKNOWN_SCHEME
- **WHEN** `read("ftp://host/path")` is called against a resolver with no `ftp` plugin
- **THEN** the call rejects with `StorageUriError` having `code === "STORAGE_URI.UNKNOWN_SCHEME"` and `scheme === "ftp"`
#### Scenario: scheme matching is case-insensitive
- **WHEN** `read("S3://bucket/key")` is called against a resolver registered with the lowercase `s3` scheme
- **THEN** dispatch succeeds without throwing `UNKNOWN_SCHEME`
**Verified by:** `src/resolver/resolver.test.ts::"dispatches by scheme to the registered plugin"` AND `src/resolver/resolver.test.ts::"throws UNKNOWN_SCHEME for unregistered scheme"` AND `src/resolver/resolver.test.ts::"matches scheme case-insensitively"`
---
### Requirement: Plugin registration is immutable and conflict-detected
The `createStorageUriResolver(plugins)` factory SHALL freeze the registry after construction and SHALL throw `STORAGE_URI.SCHEME_CONFLICT` when two distinct plugin references claim the same scheme. Registering the same plugin reference twice for the same scheme SHALL be a no-op.
#### Scenario: duplicate scheme with distinct plugins is rejected
- **WHEN** `createStorageUriResolver([s3PluginA, s3PluginB])` is called where both declare `scheme: "s3"` and `s3PluginA !== s3PluginB`
- **THEN** the call throws `StorageUriError` with `code === "STORAGE_URI.SCHEME_CONFLICT"` and `scheme === "s3"`
#### Scenario: identical plugin reference registered twice is accepted
- **WHEN** `createStorageUriResolver([fileUriPlugin, fileUriPlugin])` is called with the same reference twice
- **THEN** the resolver is constructed without throwing and dispatch behaves identically to single registration
**Verified by:** `src/resolver/resolver.test.ts::"rejects conflicting scheme registrations"` AND `src/resolver/resolver.test.ts::"accepts idempotent reference re-registration"`
---
### Requirement: Streaming reads with buffered fallback
The `open(uri, signal?)` method SHALL return a `ReadableStream<Uint8Array>` of the resource's bytes. When the underlying adapter exposes `createReadStream`, the resolver SHALL forward it. When the adapter exposes only `get`, the resolver SHALL wrap the buffered bytes in a one-chunk `ReadableStream` and SHALL increment the `storage_uri.stream_fallback_total{scheme}` counter.
#### Scenario: adapter with createReadStream is forwarded
- **WHEN** `open("https://host/path")` is called against a plugin whose adapter implements `createReadStream`
- **THEN** the returned stream is the same reference returned by `adapter.createReadStream`, and `stream_fallback_total` is NOT incremented
#### Scenario: adapter without createReadStream falls back to single-chunk stream
- **WHEN** `open("memory://x/y")` is called against a plugin whose adapter implements only `get`
- **THEN** the returned stream emits exactly one chunk equal to `adapter.get` result, then closes; `stream_fallback_total{scheme:"memory"}` is incremented by 1
#### Scenario: open of a missing resource throws NOT_FOUND
- **WHEN** `open("file:///does/not/exist")` is called and `adapter.get` returns `undefined`
- **THEN** the call rejects with `StorageUriError` having `code === "STORAGE_URI.NOT_FOUND"`
**Verified by:** `src/resolver/resolver.streams.test.ts::"forwards adapter.createReadStream when available"` AND `src/resolver/resolver.streams.test.ts::"falls back to one-chunk stream and increments counter"` AND `src/resolver/resolver.streams.test.ts::"throws NOT_FOUND when adapter returns undefined"`
---
### Requirement: Streaming writes with buffered fallback
The `write(uri, body, signal?)` method SHALL accept `Uint8Array` or `ReadableStream<Uint8Array>` for `body`. When the underlying adapter exposes `createWriteStream`, the resolver SHALL forward it. When the adapter exposes only `set`, the resolver SHALL drain the stream into a single `Uint8Array`, call `set` once, and SHALL increment `storage_uri.buffered_write_total{scheme}`. Adapters that throw on `set` SHALL surface `STORAGE_URI.WRITE_UNSUPPORTED` re-wrapping the original error as `cause`.
#### Scenario: Uint8Array body is forwarded directly
- **WHEN** `write("file:///a.bin", new Uint8Array([1,2,3]))` is called
- **THEN** `adapter.set` is invoked exactly once with the exact byte array
#### Scenario: stream body is drained into a single set call
- **WHEN** `write("file:///a.bin", stream)` is called where `stream` yields three chunks
- **THEN** `adapter.set` is invoked exactly once with the concatenated bytes; `buffered_write_total{scheme:"file"}` is incremented by 1
#### Scenario: read-only adapter surfaces WRITE_UNSUPPORTED
- **WHEN** `write("https://cdn/x", bytes)` is called against a plugin whose adapter throws `STORAGE_HTTP.READ_ONLY`
- **THEN** the call rejects with `StorageUriError` having `code === "STORAGE_URI.WRITE_UNSUPPORTED"` and `cause` set to the original `StorageHttpError`
**Verified by:** `src/resolver/resolver.writes.test.ts::"forwards Uint8Array body directly to adapter.set"` AND `src/resolver/resolver.writes.test.ts::"drains ReadableStream into single set call"` AND `src/resolver/resolver.writes.test.ts::"wraps read-only adapter error as WRITE_UNSUPPORTED"`
---
### Requirement: Delegated metadata, existence, and deletion
The `stat(uri)`, `exists(uri)`, and `remove(uri)` methods SHALL delegate to the underlying adapter after URI parse. `stat` SHALL return `undefined` when the adapter cannot answer without a full read. `exists` SHALL forward `StorageLike.exists`. `remove` SHALL forward `StorageLike.delete` and SHALL return the boolean from that call.
#### Scenario: exists delegates to StorageLike.exists
- **WHEN** `exists("file:///a.bin")` is called and `adapter.exists("a.bin")` returns `true`
- **THEN** the resolver returns `true` without invoking any other adapter method
#### Scenario: remove returns the adapter's boolean
- **WHEN** `remove("s3://bucket/missing")` is called and `adapter.delete` returns `false`
- **THEN** the resolver returns `false`; no error is raised
#### Scenario: stat returns undefined when adapter offers no metadata path
- **WHEN** `stat("memory://x/y")` is called against an adapter with no `getMetadata`/`HEAD` surface
- **THEN** the call resolves with `undefined`
**Verified by:** `src/resolver/resolver.test.ts::"exists delegates to adapter.exists"` AND `src/resolver/resolver.test.ts::"remove returns adapter.delete boolean"` AND `src/resolver/resolver.test.ts::"stat returns undefined for adapters without metadata path"`
---
### Requirement: Prefix listing with URI re-serialisation
The `list(uri, options?)` method SHALL treat the input URI's key portion as a prefix, delegate to `StorageLike.list({ prefix, limit, continuationToken })`, and re-serialise each returned key back to a URI via `plugin.build(key, meta)`. Plugins without a `build` function SHALL cause `list` to throw `STORAGE_URI.LIST_UNSUPPORTED`.
#### Scenario: list re-serialises keys to URIs via build()
- **WHEN** `list("s3://bucket/raw/", { limit: 10 })` is called and `adapter.list({ prefix: "raw/", limit: 10 })` returns `{ keys: ["raw/a.mov", "raw/b.mov"], hasMore: false }`
- **THEN** the resolver returns `{ uris: ["s3://bucket/raw/a.mov", "s3://bucket/raw/b.mov"], hasMore: false }`
#### Scenario: plugin without build() throws LIST_UNSUPPORTED
- **WHEN** `list("memory://x/y/")` is called against a plugin that omits `build`
- **THEN** the call rejects with `StorageUriError` having `code === "STORAGE_URI.LIST_UNSUPPORTED"`
#### Scenario: continuation token is round-tripped opaquely
- **WHEN** `list("s3://bucket/x/", { continuationToken: "abc" })` is called and the adapter returns `{ keys: [...], hasMore: true, continuationToken: "def" }`
- **THEN** the resolver returns the same `continuationToken: "def"` unchanged
**Verified by:** `src/resolver/resolver.list.test.ts::"re-serialises keys to URIs"` AND `src/resolver/resolver.list.test.ts::"throws LIST_UNSUPPORTED when plugin omits build"` AND `src/resolver/resolver.list.test.ts::"round-trips continuation token opaquely"`
---
### Requirement: URI parsing (RFC 3986 subset)
The `parseUri(uri)` helper SHALL accept any string of the form `<scheme>:<rest>` where `<scheme>` matches `/^[A-Za-z][A-Za-z0-9+\-.]*$/`. It SHALL throw `STORAGE_URI.INVALID_URI` for inputs missing a scheme, containing embedded null bytes (``), or longer than 4096 characters.
#### Scenario: well-formed URIs parse without throwing
- **WHEN** `parseUri("s3://bucket/key")`, `parseUri("file:///abs/path")`, and `parseUri("https://host/p?q=1")` are called
- **THEN** all three return a parsed object with the expected scheme
#### Scenario: malformed inputs throw INVALID_URI
- **WHEN** `parseUri("bare-string")`, `parseUri("scheme:")`, `parseUri("s3://bucket/key")`, or `parseUri("x".repeat(4097))` is called
- **THEN** each call throws `StorageUriError` with `code === "STORAGE_URI.INVALID_URI"`
**Verified by:** `src/parse-uri/parse-uri.test.ts::"parses well-formed URIs"` AND `src/parse-uri/parse-uri.test.ts::"throws INVALID_URI for malformed inputs"`
---
### Requirement: AbortSignal propagation
Every public method of the resolver (`open`, `read`, `write`, `stat`, `exists`, `remove`, `list`) SHALL accept an `AbortSignal` and SHALL forward it unchanged to the underlying adapter call. When the signal is already aborted at the time of the call, the resolver SHALL reject with `STORAGE_URI.ABORTED` without invoking the adapter.
#### Scenario: pre-aborted signal short-circuits
- **WHEN** `read("file:///a", AbortSignal.abort())` is called
- **THEN** the call rejects with `code === "STORAGE_URI.ABORTED"` and `adapter.get` is NEVER invoked
#### Scenario: signal is forwarded to adapter
- **WHEN** `open("file:///a", signal)` is called with a live signal
- **THEN** the adapter receives the same signal reference
**Verified by:** `src/resolver/resolver.test.ts::"short-circuits on pre-aborted signal"` AND `src/resolver/resolver.test.ts::"forwards AbortSignal to adapter unchanged"`
---
### Requirement: Adapter error re-wrapping
Errors thrown by the underlying adapter that are NOT `StorageUriError` SHALL be wrapped in `StorageUriError` with `code === "STORAGE_URI.ADAPTER_ERROR"`, the original preserved as `cause`, and `uri` + `scheme` discriminants populated.
#### Scenario: arbitrary adapter error is re-wrapped
- **WHEN** `read("file:///a")` is called and `adapter.get` throws `new Error("disk fault")`
- **THEN** the rejection is a `StorageUriError` with `code === "STORAGE_URI.ADAPTER_ERROR"`, `cause.message === "disk fault"`, `uri === "file:///a"`, and `scheme === "file"`
#### Scenario: existing StorageUriError is re-thrown unchanged
- **WHEN** the adapter (via a nested resolver) throws `new StorageUriError("NOT_FOUND", ...)`
- **THEN** the same instance is propagated; no re-wrap occurs
**Verified by:** `src/resolver/resolver.test.ts::"wraps adapter errors as ADAPTER_ERROR"` AND `src/resolver/resolver.test.ts::"re-throws StorageUriError unchanged"`
---
### Requirement: OpenTelemetry instrumentation
Every public operation SHALL emit a span on the `@suaveplan/storage-uri` tracer and SHALL record `storage_uri.op_duration_ms{op, scheme, outcome}` on the meter. Errors SHALL increment `storage_uri.errors_total{scheme, code}` with the `STORAGE_URI.*` code (or `ADAPTER_ERROR` for re-wrapped). Fallback paths SHALL increment `storage_uri.stream_fallback_total` and `storage_uri.buffered_write_total` respectively.
#### Scenario: successful read records duration histogram and counter
- **WHEN** `read("file:///a")` succeeds
- **THEN** the meter records one observation on `storage_uri.op_duration_ms{op:"read", scheme:"file", outcome:"success"}` and one increment of `storage_uri.reads_total{scheme:"file", outcome:"success"}`
#### Scenario: error emits errors_total with the resolved code
- **WHEN** `read("ftp://x/y")` rejects with `UNKNOWN_SCHEME`
- **THEN** `storage_uri.errors_total{scheme:"ftp", code:"STORAGE_URI.UNKNOWN_SCHEME"}` is incremented by 1
#### Scenario: query strings are stripped from span attributes
- **WHEN** `read("https://host/path?token=secret")` is called
- **THEN** the emitted span's `storage.uri` attribute is `"https://host/path"` (no `?token=...`)
**Verified by:** `src/otel/otel.test.ts::"records duration histogram on success"` AND `src/otel/otel.test.ts::"increments errors_total with resolved code"` AND `src/otel/otel.test.ts::"strips query string from span attributes"`
---
### Requirement: Plugin conformance harness
The package SHALL export `runUriPluginConformance(plugin, fixtures)` that adapter packages call from their own test suites to verify their `uriPlugin` satisfies the resolver's contract. The harness SHALL assert that `parse` accepts every URI in `fixtures.validUris`, rejects every URI in `fixtures.invalidUris` with `INVALID_URI`, and (when `build` is implemented) round-trips every entry in `fixtures.buildRoundTrip`.
#### Scenario: harness passes for a well-formed plugin
- **WHEN** `runUriPluginConformance(fileUriPlugin, { validUris: ["file:///a"], invalidUris: ["bad"], buildRoundTrip: [{ key: "a", uri: "file:///a" }] })` is invoked
- **THEN** all assertions pass without throwing
#### Scenario: harness fails for a plugin with broken round-trip
- **WHEN** the harness is invoked against a plugin whose `build` does not invert `parse`
- **THEN** the assertion fails with a message naming the failing fixture
**Verified by:** `src/conformance/conformance.test.ts::"passes for a well-formed plugin"` AND `src/conformance/conformance.test.ts::"fails on broken round-trip"`
<!-- markdownlint-enable MD024 -->

View File

@@ -0,0 +1,83 @@
# Tasks — add-storage-uri
> Every checkbox MUST reflect work actually done in this change.
## 1. Spec authoring
- [ ] 1.1 Read `openspec/specs/types/spec.md` and confirm `StorageLike` is the right structural target
- [ ] 1.2 Read `openspec/AGENTS.md` §0c, §13, §14
- [ ] 1.3 Read `openspec/archive/2026-05-16-add-storage-http/` end-to-end as the closest precedent
- [ ] 1.4 Read `packages/core/types/src/storage-like.ts` to confirm the structural surface available
- [ ] 1.5 Draft `proposal.md` from `openspec/templates/proposal.md`
- [ ] 1.6 Draft `design.md` covering tier, file layout, dispatch, stream/write fallback, per-scheme parse contract, OTel surface
- [ ] 1.7 Draft `specs/storage-uri/spec.md` — every requirement carries a `**Verified by:**` line at authoring time
- [ ] 1.8 Build the numbered Requirement/Scenario implementation checklist (per §0c step 6)
## 2. Scaffolding
- [ ] 2.1 Create `packages/storage/storage-uri/` mirroring `packages/storage/storage-http/` conventions
- [ ] 2.2 Write `package.json` v0.0.0, `"sideEffects": false`, subpath exports for every feature module
- [ ] 2.3 Wire deps: `@suaveplan/error`, `@suaveplan/types`, `@opentelemetry/api`, `tslib` (all `workspace:^` / `catalog:`)
- [ ] 2.4 Wire devDeps: `@suaveplan/testing`, `@suaveplan/storage-file`, `@suaveplan/storage-http`, `@vitest/coverage-istanbul`, `fast-check`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`
- [ ] 2.5 Write `tsconfig.json`, `tsconfig.lib.json`, `vite.config.ts`, `vitest.config.ts` (via `createTestConfig`)
- [ ] 2.6 Write `tests/setup/polyfills.ts` and `README.md` skeleton
## 3. Core implementation
- [ ] 3.1 `src/error-codes/error-codes.ts` — `defineCodes("STORAGE_URI", [...])` with 8 codes, `StorageUriError` class, `uri`/`scheme` discriminants
- [ ] 3.2 `src/types/plugin.types.ts` — `UriPlugin`, `ParsedUri` interfaces
- [ ] 3.3 `src/types/resolver.types.ts` — `StorageUriResolver` shape, `StatResult`, `ListResult`
- [ ] 3.4 `src/parse-uri/parse-uri.ts` — RFC 3986 scheme+authority+path split, `INVALID_URI` mapping, 4096-char cap, null-byte rejection
- [ ] 3.5 `src/otel/otel.ts` — meter+tracer wiring, `recordOp(op, scheme, outcome, ms)`, URI redaction (drop query string)
- [ ] 3.6 `src/resolver/resolver.ts` — registry construction, frozen map, scheme dispatch (case-insensitive), AbortSignal short-circuit, ADAPTER_ERROR wrap, exists/remove/stat delegates
- [ ] 3.7 `src/resolver/resolver.streams.ts` — `open()` with `createReadStream` forwarding + one-chunk fallback + counter
- [ ] 3.8 `src/resolver/resolver.writes.ts` — `write()` accepting `Uint8Array | ReadableStream`, `createWriteStream` forwarding + drain fallback + counter
- [ ] 3.9 `src/resolver/resolver.list.ts` — prefix list, `plugin.build` re-serialise, `LIST_UNSUPPORTED` for plugins without `build`
- [ ] 3.10 `src/conformance/conformance.ts` — `runUriPluginConformance(plugin, fixtures)` harness
- [ ] 3.11 `src/index.ts` — explicit named re-exports per §6a (no `export *` in non-root barrels)
## 4. Tests (every `it(...)` carries `{ timeout: 5000 }`)
- [ ] 4.1 `src/parse-uri/parse-uri.test.ts` — valid scheme set, malformed inputs → `INVALID_URI`, null-byte rejection, 4096-char cap, case-insensitive scheme
- [ ] 4.2 `src/error-codes/error-codes.test.ts` — every code reachable, discriminants populated, `cause` preserved
- [ ] 4.3 `src/resolver/resolver.test.ts` — scheme dispatch, `UNKNOWN_SCHEME`, case-insensitive scheme match, `SCHEME_CONFLICT`, idempotent re-registration, `AbortSignal` pre/forward, `ADAPTER_ERROR` wrap, exists/remove/stat delegation
- [ ] 4.4 `src/resolver/resolver.streams.test.ts` — native forwarding, one-chunk fallback + counter, `NOT_FOUND` on undefined
- [ ] 4.5 `src/resolver/resolver.writes.test.ts` — `Uint8Array` forwarding, stream drain + counter, `WRITE_UNSUPPORTED` re-wrap
- [ ] 4.6 `src/resolver/resolver.list.test.ts` — re-serialisation via `build`, `LIST_UNSUPPORTED` when `build` absent, continuation token round-trip
- [ ] 4.7 `src/resolver/resolver.spec-conformance.test.ts` — type conformance against `StorageLike`, every `STORAGE_URI` code reachable, capability probing
- [ ] 4.8 `src/otel/otel.test.ts` — duration histogram on success, `errors_total` on failure, query-string redaction in span attributes (via in-memory `MeterProvider` + `SpanExporter`)
- [ ] 4.9 `src/conformance/conformance.test.ts` — harness passes for a good plugin, fails for broken round-trip
- [ ] 4.10 `tests/integration/cross-scheme.test.ts` — real `storage-file` + real `storage-http` composition, read both via the same resolver
- [ ] 4.11 No imports from `vitest` or `bun:test` in test files — all via `@suaveplan/testing/runner` (§7)
## 5. Sibling adapter plugins (additive — each in its own package)
- [ ] 5.1 `storage-file`: add `src/uri-plugin/uri-plugin.ts` exporting `fileUriPlugin(options)`, sibling `.md`, conformance test using `runUriPluginConformance`
- [ ] 5.2 `storage-http`: add `httpUriPlugin(adapter)` and `httpsUriPlugin(adapter)` exports + conformance test
- [ ] 5.3 `storage-cdn-rackspace`: add `rackspaceUriPlugin(adapter, container)` export + conformance test
- [ ] 5.4 `storage-cdn-bunnycdn`: add `bunnyUriPlugin(adapter, zone)` export + conformance test
- [ ] 5.5 (deferred) `storage-s3`: `s3UriPlugin(adapter, bucket)` — gated on s3 package landing
## 6. Documentation
- [ ] 6.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples, full API reference)
- [ ] 6.2 `README.md` links every module under `## Modules`
- [ ] 6.3 README documents the cross-scheme media-pipeline composition recipe
- [ ] 6.4 README documents how to author a third-party `UriPlugin`
## 7. Gates
- [ ] 7.1 `bun run check`
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-uri`
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/storage-uri` (100/100/100/100 Istanbul)
- [ ] 7.4 `bun scripts/lint/check-test-runner-imports.ts` (rule 11)
- [ ] 7.5 Tier-purity grep — `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-uri/src/` returns empty
- [ ] 7.6 `bun scripts/openspec-conformance.ts @suaveplan/storage-uri`
- [ ] 7.7 Every sibling adapter package still green under its own pre-publish gate after adding its `uriPlugin`
## 8. Manifest & archival
- [ ] 8.1 Add package to `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
- [ ] 8.2 Update `STATUS.md` (genesis-original — add to "Completed" once shipped)
- [ ] 8.3 Fold `specs/storage-uri/spec.md` into `openspec/specs/storage-uri/spec.md`
- [ ] 8.4 Move `openspec/changes/add-storage-uri/` → `openspec/archive/<yyyy-mm-dd>-add-storage-uri/`