Initial Commit
This commit is contained in:
93
openspec/archive/2026-05-16-add-storage-uri/proposal.md
Normal file
93
openspec/archive/2026-05-16-add-storage-uri/proposal.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Add @suaveplan/storage-uri — URI-routed dispatch over StorageLike adapters
|
||||
|
||||
**Package**: `@suaveplan/storage-uri`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (genesis-original)
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/storage-uri/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
Genesis now ships seven `StorageLike`-conformant adapters — `storage-file`, `storage-http`, `storage-cdn`, `storage-cdn-rackspace`, `storage-cdn-bunnycdn`, `storage-s3`, and three `storage-browser-*` flavours — but every consumer has to pick **one** adapter at construction time and address its keys directly. Real workloads cross adapters: a media pipeline reads source from `s3://uploads/raw/clip.mov`, a poster from `file:///var/cache/posters/clip.jpg`, and a delivery copy from `https://cdn.example.com/clip/master.m3u8` — all in the same call site. Today that means three injected adapters, three key conventions, and bespoke fan-out logic.
|
||||
|
||||
`storage-uri` collapses that to **one string in, one stream out**. A consumer hands `read("s3://bucket/key")` to a resolver that owns a `scheme → { adapter, parseUri }` registry, gets a `ReadableStream<Uint8Array>` back, and never has to know which adapter was selected. The registry is composed once at boot from any subset of the existing storage packages; each adapter package ships its own `uriPlugin` so the resolver stays scheme-agnostic and dependency-light.
|
||||
|
||||
The package is a **router**, not a new storage tier. It performs no I/O of its own — every operation delegates to a registered `StorageLike` after URI parsing.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `StorageUriResolver` class — owns a `Map<scheme, UriPlugin>` registry, exposes `open`/`read`/`write`/`stat`/`exists`/`remove`/`list` keyed by URI string.
|
||||
- ADDED: `createStorageUriResolver(plugins)` factory — accepts an array of `UriPlugin` and produces a frozen resolver. Idempotent re-registration is a no-op when the plugin reference is identical; conflicting registrations throw `STORAGE_URI.SCHEME_CONFLICT`.
|
||||
- ADDED: `UriPlugin` interface — `{ scheme: string; adapter: StorageLike; parse(uri): { key: string; meta?: Record<string, string> }; build?(key, meta?): string }`. The `adapter` field is `StorageLike` (not `AbstractStorage`) so the resolver works against the structural contract from `@suaveplan/types`.
|
||||
- ADDED: `parseUri(uri)` standalone helper — RFC 3986 scheme + authority + path split that adapter plugins compose with their own scheme-specific addressing (`s3://bucket/key`, `file:///abs/path`, `bunny://zone/path`, `https://host/path`).
|
||||
- ADDED: `open(uri, signal?): Promise<ReadableStream<Uint8Array>>` — single-shot streaming read. When the underlying adapter exposes `createReadStream`, it is forwarded; otherwise the resolver falls back to wrapping `adapter.get(key)` in a one-chunk `ReadableStream`. The fallback path is observable via `storage_uri.stream_fallback_total`.
|
||||
- ADDED: `read(uri, signal?): Promise<Uint8Array>` — buffered read. Delegates straight to `adapter.get(key)`; throws `STORAGE_URI.NOT_FOUND` when the adapter returns `undefined`.
|
||||
- ADDED: `write(uri, body, signal?): Promise<void>` — accepts `Uint8Array` or `ReadableStream<Uint8Array>`. Streams are drained into a single `set` call when the adapter lacks `createWriteStream`; the buffered fallback is observable via `storage_uri.buffered_write_total`.
|
||||
- ADDED: `stat(uri, signal?): Promise<{ size?: number; etag?: string; lastModified?: Date; contentType?: string } | undefined>` — surfaces adapter metadata when available; returns `undefined` for adapters that cannot answer without a full read.
|
||||
- ADDED: `exists(uri, signal?): Promise<boolean>` and `remove(uri, signal?): Promise<boolean>` — direct delegates to `StorageLike.exists` and `StorageLike.delete` after parse.
|
||||
- ADDED: `list(uri, options?, signal?): Promise<{ uris: string[]; hasMore: boolean; continuationToken?: string }>` — the URI is treated as a prefix; results are re-serialised back to URIs via `plugin.build()`. Plugins without a `build()` function throw `STORAGE_URI.LIST_UNSUPPORTED`.
|
||||
- ADDED: Sibling `uriPlugin` exports in every storage adapter package — `storage-file` ships `fileUriPlugin(rootDir)`, `storage-http` ships `httpsUriPlugin(httpStorage)` and `httpUriPlugin(httpStorage)`, `storage-cdn-rackspace` ships `rackspaceUriPlugin(adapter, container)`, `storage-cdn-bunnycdn` ships `bunnyUriPlugin(adapter, zone)`, `storage-s3` ships `s3UriPlugin(adapter, bucket)`. Each owns its scheme parser; the resolver knows nothing about S3 buckets, CDN zones, or filesystem roots.
|
||||
- ADDED: `STORAGE_URI_CODES` registry via `defineCodes("STORAGE_URI", [...])` — 8 codes: `UNKNOWN_SCHEME`, `INVALID_URI`, `SCHEME_CONFLICT`, `NOT_FOUND`, `LIST_UNSUPPORTED`, `WRITE_UNSUPPORTED`, `ABORTED`, `ADAPTER_ERROR`.
|
||||
- ADDED: `StorageUriError extends SuaveplanError<StorageUriErrorCode>` with `uri` and `scheme` discriminants; adapter-originated errors are re-wrapped with `ADAPTER_ERROR` preserving the original as `cause`.
|
||||
- ADDED: OTel surface — `storage_uri.op_duration_ms{op,scheme,outcome}` histogram; `storage_uri.{reads,writes,deletes,errors,stream_fallback,buffered_write}_total{scheme,outcome}` counters; per-op span on `@suaveplan/storage-uri` tracer with `storage.uri.scheme`, `storage.uri.key`, `storage.uri` attributes.
|
||||
- ADDED: `runUriPluginConformance(plugin, fixtures)` test harness — every adapter package's `uriPlugin` SHALL pass this conformance suite (round-trip parse↔build, key boundary forwarding, list re-serialisation, error mapping).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **A new storage adapter.** `storage-uri` performs zero I/O; every byte goes through an existing `StorageLike`. Reason: this is a routing concern, not a storage tier.
|
||||
- **Cross-scheme copy/move.** `copy("s3://a/x", "file:///b/x")` is tempting but commits the resolver to streaming semantics, transactional rollback, and bandwidth governance that belong in a dedicated `storage-pipeline` package. Reason: scope creep; the consumer can compose `read` + `write` today.
|
||||
- **URI signing.** Signed URLs (S3 presigned, CDN token-auth) are produced by the underlying adapters; the resolver MUST NOT introduce its own signing layer. Reason: every provider's signing scheme is different and already lives in the adapter package.
|
||||
- **Caching.** Repeat reads of the same URI hit the adapter every time. A caching resolver belongs in a separate `storage-uri-cache` (compose with `cache-storage`). Reason: cache invalidation policy is application-specific.
|
||||
- **Schema federation across resolvers.** A single resolver owns one plugin per scheme. Multiple Rackspace containers behind one `rackspace://` scheme need to be addressed by including the container in the authority (`rackspace://container-a/key`) — handled inside `rackspaceUriPlugin`. Reason: keeps the resolver O(1) on dispatch.
|
||||
- **Browser-tier auto-registration.** The resolver is universal-tier; consumers register plugins explicitly. No global mutable registry. Reason: mutable singletons break testability and tree-shaking.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This is a **genesis-original** package; there is no upstream in `../suaveplan/`. All requirements derive from the `StorageLike` contract in `@suaveplan/types` (`openspec/specs/types/spec.md`) and the existing adapter surface.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Type against `StorageLike`, not `AbstractStorage` | Keeps the resolver in the universal tier with a zero-runtime dep on `@suaveplan/types`. Adapters that extend `AbstractStorage` satisfy `StorageLike` structurally. |
|
||||
| Plugins live in each adapter package, not in `storage-uri` | Avoids forcing the resolver to depend on every storage provider. Consumers pull in only the adapter packages they actually use; the resolver itself stays tiny. |
|
||||
| `open()` returns `ReadableStream<Uint8Array>` | Web Streams is the universal cross-runtime stream type (Node ≥18, Bun, browsers, Deno). Matches `Response.body` from `storage-http` and existing `createReadStream` signatures. |
|
||||
| `stream_fallback_total` + `buffered_write_total` counters | Quietly upgrading a buffered adapter to a "streaming" surface is a memory-pressure hazard. Operators alert on these counters to identify mis-composed adapters. |
|
||||
| `parse(uri) → { key, meta }` rather than `{ key }` alone | CDN zones, S3 buckets, Rackspace containers all need an extra addressing dimension; `meta` is the documented escape hatch so the resolver doesn't grow a separate "authority" field per scheme. |
|
||||
| `LIST_UNSUPPORTED` distinct from `WRITE_UNSUPPORTED` | Listing requires `plugin.build()`; write requires `adapter` write capability. They're independently optional and need distinct error codes for capability probing. |
|
||||
| Idempotent registration is a no-op only when the plugin reference is identical | Reference equality is the cheapest correct test; deep-equality would mask configuration drift between modules that import the same scheme. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete `it(...)` titles in these files:
|
||||
|
||||
- `src/resolver/resolver.test.ts` — registry construction, scheme dispatch, conflict detection, idempotent re-registration, `UNKNOWN_SCHEME` for unregistered scheme, AbortSignal propagation to every operation.
|
||||
- `src/resolver/resolver.streams.test.ts` — `open` forwards native `createReadStream` when present, falls back to one-chunk wrap when absent, `stream_fallback_total` increments on fallback path.
|
||||
- `src/resolver/resolver.writes.test.ts` — `write` accepts `Uint8Array` straight, drains `ReadableStream` into single `set` when adapter lacks `createWriteStream`, `buffered_write_total` increments on drain path.
|
||||
- `src/resolver/resolver.spec-conformance.test.ts` — type-level conformance against `StorageLike`, every `STORAGE_URI` code reachable, OTel instruments observed via real in-memory `MeterProvider`.
|
||||
- `src/parse-uri/parse-uri.test.ts` — RFC 3986 happy paths (`s3://bucket/key`, `file:///abs/path`, `https://host/path?q=1`), invalid forms (`bare-string`, `scheme:`, embedded null), `INVALID_URI` mapping.
|
||||
- `src/error-codes/error-codes.test.ts` — exhaustive code registry, `StorageUriError` discriminant fields, adapter-error re-wrap preserves `cause`.
|
||||
- `src/otel/otel.test.ts` — metrics via real in-memory `MeterProvider`, error span via in-memory `SpanExporter`, scheme attribute present on every emission.
|
||||
- `src/conformance/conformance.test.ts` — `runUriPluginConformance` against a fixture in-memory `StorageLike` and a fixture `parse`/`build` pair.
|
||||
- `tests/integration/cross-scheme.test.ts` — composes a resolver with `storage-file` + `storage-http` real adapters; reads `file:///tmp/...` and `https://...` round-trip without code path divergence.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
The package is new — no dependents currently. After adopting, the obvious upstream beneficiaries are `drive-template` (URI-addressed hydration sources), `enterprise-media`, the audio/video pipelines, and any consumer that currently injects multiple `StorageLike`s side-by-side.
|
||||
|
||||
- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-uri` returning the package itself).
|
||||
|
||||
### Affected upstreams (sibling adapter packages — additive `uriPlugin` export)
|
||||
|
||||
| Package | Action | Breaking? |
|
||||
|---|---|---|
|
||||
| `@suaveplan/storage-file` | ADD `fileUriPlugin(rootDir)` named export | no |
|
||||
| `@suaveplan/storage-http` | ADD `httpsUriPlugin(adapter)` and `httpUriPlugin(adapter)` named exports | no |
|
||||
| `@suaveplan/storage-cdn-rackspace` | ADD `rackspaceUriPlugin(adapter, container)` named export | no |
|
||||
| `@suaveplan/storage-cdn-bunnycdn` | ADD `bunnyUriPlugin(adapter, zone)` named export | no |
|
||||
| `@suaveplan/storage-s3` | ADD `s3UriPlugin(adapter, bucket)` named export | no |
|
||||
|
||||
All additions are purely additive named exports — no existing surface changes. Each adapter package gets a separate spec delta in this change folder ONLY if its `uriPlugin` introduces non-trivial requirements; otherwise the export is documented in the adapter's existing spec and verified by the conformance harness.
|
||||
Reference in New Issue
Block a user