Move docs/ package mirror into document-navigator plugin
Relocate the ~6k-file docs tree from the repo root into plugins/document-navigator/docs/ so the plugin is self-contained and portable. docs/superpowers/ (specs and plans) stays at the repo root. Moved via cp+rm rather than git mv because directory rename() fails with EPERM on this WSL /mnt DrvFs mount; git rename-detection pairs the identical-content files at commit time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,146 @@
|
||||
# @suaveplan/storage-uri - Universal URI-routed dispatch over @suaveplan StorageLike adapters. One resolver, one URI string in, one stream/byte/metadata out; scheme→plugin registry, AbortSignal everywhere, full OTel surface, zero I/O of its own.
|
||||
|
||||
Universal URI-routed dispatch over `@suaveplan` `StorageLike` adapters.
|
||||
|
||||
One resolver, one URI string in, one stream/byte/metadata out.
|
||||
Scheme→plugin registry; the resolver performs no I/O of its own — every
|
||||
operation delegates to a registered `StorageLike` after URI parsing.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [conformance](./src/conformance/conformance.md) — `runUriPluginConformance(plugin, fixtures)` is the test harness adapter packages call from their own suites to prove th…
|
||||
- [conformance](./src/conformance/conformance.md) — `runUriPluginConformance(plugin, fixtures)` is the test harness adapter packages call from their own suites to prove th…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the canonical `STORAGE_URI_CODES` registry and the `StorageUriError` class for the `@suaveplan/storage-uri` pac…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the canonical `STORAGE_URI_CODES` registry and the `StorageUriError` class for the `@suaveplan/storage-uri` pac…
|
||||
- [otel](./src/otel/otel.md) — Owns the OpenTelemetry instrument bag for `@suaveplan/storage-uri`: meter, tracer, and the six counter + one histogram…
|
||||
- [otel](./src/otel/otel.md) — Owns the OpenTelemetry instrument bag for `@suaveplan/storage-uri`: meter, tracer, and the six counter + one histogram…
|
||||
- [parse-uri](./src/parse-uri/parse-uri.md) — `parseUri(uri)` is the RFC 3986 subset parser the resolver uses to extract a scheme, optional authority, path, and opti…
|
||||
- [parse-uri](./src/parse-uri/parse-uri.md) — `parseUri(uri)` is the RFC 3986 subset parser the resolver uses to extract a scheme, optional authority, path, and opti…
|
||||
- [resolver](./src/resolver/resolver.md) — The `StorageUriResolver` class is the entry point for URI-routed dispatch in `@suaveplan/storage-uri`. Constructed once…
|
||||
- [resolver.list](./src/resolver/resolver.list.md) — Implements the `list(uri, options?, signal?)` semantics. Treats the input URI's key portion as a listing prefix, delega…
|
||||
- [resolver](./src/resolver/resolver.md) — The `StorageUriResolver` class is the entry point for URI-routed dispatch in `@suaveplan/storage-uri`. Constructed once…
|
||||
- [resolver.streams](./src/resolver/resolver.streams.md) — Implements the `open(uri, signal?)` semantics for the resolver: forward a native `createReadStream` when the adapter ex…
|
||||
- [resolver.writes](./src/resolver/resolver.writes.md) — Implements the `write(uri, body, signal?)` semantics. Accepts either `Uint8Array` or `ReadableStream<Uint8Array>` for t…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/storage-uri
|
||||
```
|
||||
|
||||
Pair with one or more adapter packages — each ships its own `uriPlugin`:
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/storage-file @suaveplan/storage-http
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
| Subpath | What |
|
||||
|---|---|
|
||||
| `@suaveplan/storage-uri` | Full surface — re-exports everything below |
|
||||
| `@suaveplan/storage-uri/error-codes` | `STORAGE_URI_CODES`, `StorageUriError` |
|
||||
| `@suaveplan/storage-uri/parse-uri` | `parseUri(uri)` RFC 3986 subset |
|
||||
| `@suaveplan/storage-uri/resolver` | `StorageUriResolver`, `createStorageUriResolver` |
|
||||
| `@suaveplan/storage-uri/otel` | OTel meter+tracer instruments |
|
||||
| `@suaveplan/storage-uri/conformance` | `runUriPluginConformance` for adapter authors |
|
||||
| `@suaveplan/storage-uri/types` | `UriPlugin`, `ParsedUri`, `StatResult`, list types |
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { createStorageUriResolver } from "@suaveplan/storage-uri";
|
||||
import { fileUriPlugin } from "@suaveplan/storage-file";
|
||||
import { httpsUriPlugin, createHttpStorage } from "@suaveplan/storage-http";
|
||||
|
||||
const resolver = createStorageUriResolver([
|
||||
fileUriPlugin({ rootDir: "/var/cache/media" }),
|
||||
httpsUriPlugin(createHttpStorage({ baseUrl: "https://cdn.example.com" })),
|
||||
]);
|
||||
|
||||
const stream = await resolver.open("https://cdn.example.com/clip/master.m3u8");
|
||||
await resolver.write("file:///cache/poster.jpg", thumbBytes);
|
||||
```
|
||||
|
||||
## Cross-scheme media-pipeline recipe
|
||||
|
||||
```ts
|
||||
const resolver = createStorageUriResolver([
|
||||
fileUriPlugin({ rootDir: "/var/media" }),
|
||||
httpsUriPlugin(httpStorage),
|
||||
]);
|
||||
|
||||
async function thumbnail(srcUri: string, dstUri: string): Promise<void> {
|
||||
const src = await resolver.open(srcUri);
|
||||
const out = await encodeJpegFromStream(src);
|
||||
await resolver.write(dstUri, out);
|
||||
}
|
||||
```
|
||||
|
||||
A single call site composes a filesystem cache, a CDN origin, and any other
|
||||
adapter you register — the resolver hides which scheme served the request.
|
||||
|
||||
## Authoring a third-party `UriPlugin`
|
||||
|
||||
```ts
|
||||
import type { UriPlugin } from "@suaveplan/storage-uri";
|
||||
import type { StorageLike } from "@suaveplan/types";
|
||||
|
||||
export function memoryUriPlugin(adapter: StorageLike): UriPlugin {
|
||||
return {
|
||||
scheme: "memory",
|
||||
adapter,
|
||||
parse(uri) {
|
||||
const url = new URL(uri);
|
||||
return { key: url.pathname.replace(/^\//, ""), meta: { authority: url.host } };
|
||||
},
|
||||
build(key, meta) {
|
||||
const auth = meta?.authority ?? "default";
|
||||
return `memory://${auth}/${key}`;
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Verify your plugin against the conformance harness from your own test
|
||||
suite:
|
||||
|
||||
```ts
|
||||
import { runUriPluginConformance } from "@suaveplan/storage-uri/conformance";
|
||||
|
||||
runUriPluginConformance(memoryUriPlugin(adapter), {
|
||||
validUris: ["memory://x/a"],
|
||||
invalidUris: ["bad", "memory:"],
|
||||
buildRoundTrip: [{ key: "a", uri: "memory://x/a", meta: { authority: "x" } }],
|
||||
});
|
||||
```
|
||||
|
||||
## Error codes
|
||||
|
||||
| Code | When |
|
||||
|---|---|
|
||||
| `STORAGE_URI.UNKNOWN_SCHEME` | URI scheme has no plugin registered |
|
||||
| `STORAGE_URI.INVALID_URI` | `parseUri` rejected the input |
|
||||
| `STORAGE_URI.SCHEME_CONFLICT` | Two distinct plugins claim the same scheme |
|
||||
| `STORAGE_URI.NOT_FOUND` | Adapter returned `undefined` for `get`/`open` |
|
||||
| `STORAGE_URI.LIST_UNSUPPORTED` | Plugin has no `build()` |
|
||||
| `STORAGE_URI.WRITE_UNSUPPORTED` | Adapter threw on `set` (e.g. read-only HTTP) |
|
||||
| `STORAGE_URI.ABORTED` | `AbortSignal` fired |
|
||||
| `STORAGE_URI.ADAPTER_ERROR` | Adapter threw something else — cause preserved |
|
||||
|
||||
## OpenTelemetry
|
||||
|
||||
Tracer + meter name: `@suaveplan/storage-uri`. Counters:
|
||||
`reads_total`, `writes_total`, `deletes_total`, `errors_total`,
|
||||
`stream_fallback_total`, `buffered_write_total`. Histogram:
|
||||
`op_duration_ms{op,scheme,outcome}`. Spans on every public op with
|
||||
`storage.uri`, `storage.uri.scheme`, `storage.uri.key` attributes;
|
||||
`storage.uri` is redacted (query string + fragment stripped) so presigned
|
||||
URLs cannot leak into telemetry sinks.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# conformance
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`runUriPluginConformance(plugin, fixtures)` is the test harness adapter
|
||||
packages call from their own suites to prove their `uriPlugin` satisfies
|
||||
the resolver's contract. It checks scheme alignment, parse success for the
|
||||
valid fixture set, parse rejection (`STORAGE_URI.INVALID_URI`) for the
|
||||
invalid set, and — when `build` is implemented — round-trip equality for
|
||||
every entry in `buildRoundTrip`.
|
||||
|
||||
The harness throws `UriPluginConformanceFailure` (a `Error` subclass) on
|
||||
the first failure rather than collecting them; tests typically wrap one
|
||||
plugin per `describe` block so a single failure already names the offending
|
||||
fixture.
|
||||
|
||||
## Features
|
||||
|
||||
- Scheme alignment check on every valid URI.
|
||||
- Required-throw assertion on the invalid set.
|
||||
- Optional round-trip assertion gated on `plugin.build`.
|
||||
- Single-failure semantics with descriptive messages.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { runUriPluginConformance } from "@suaveplan/storage-uri/conformance";
|
||||
|
||||
runUriPluginConformance(memoryUriPlugin, {
|
||||
validUris: ["memory://x/a", "memory://x/b"],
|
||||
invalidUris: ["bare", "memory:"],
|
||||
buildRoundTrip: [{ key: "a", uri: "memory://x/a", meta: { authority: "x" } }],
|
||||
});
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { describe, it } from "@suaveplan/testing/runner";
|
||||
import { runUriPluginConformance } from "@suaveplan/storage-uri/conformance";
|
||||
import { fileUriPlugin } from "./uri-plugin.js";
|
||||
|
||||
describe("fileUriPlugin conformance", () => {
|
||||
it("passes the storage-uri harness", { timeout: 5000 }, () => {
|
||||
runUriPluginConformance(fileUriPlugin({ rootDir: "/tmp" }), {
|
||||
validUris: ["file:///a", "file:///b/c"],
|
||||
invalidUris: ["bad", "file:"],
|
||||
buildRoundTrip: [{ key: "/a", uri: "file:///a" }],
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface UriPluginFixtures { validUris; invalidUris; buildRoundTrip? }`
|
||||
- `class UriPluginConformanceFailure extends Error`
|
||||
- `function runUriPluginConformance(plugin, fixtures): void`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The harness re-uses `parseUri` from this package as a baseline scheme
|
||||
extractor so plugins that bypass it still get checked against the canonical
|
||||
RFC 3986 split. `plugin.build` is bound via `.bind(plugin)` for the same
|
||||
reason the resolver does it — `this`-bound implementations need to survive
|
||||
indirect invocation. On round-trip failure the message includes both the
|
||||
input and the produced output, so the failing fixture is uniquely
|
||||
identified without log-spelunking.
|
||||
@@ -0,0 +1,82 @@
|
||||
# error-codes
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines the canonical `STORAGE_URI_CODES` registry and the `StorageUriError`
|
||||
class for the `@suaveplan/storage-uri` package. Every error surfaced by the
|
||||
resolver — whether raised directly or re-wrapped from a downstream adapter —
|
||||
carries one of these eight codes and the `uri`/`scheme` discriminants so
|
||||
operators can correlate failures to the offending request without parsing
|
||||
free-form message text.
|
||||
|
||||
The registry is built via `defineCodes("STORAGE_URI", [...] as const)` from
|
||||
`@suaveplan/error`, which produces the runtime strings
|
||||
`STORAGE_URI.UNKNOWN_SCHEME`, `STORAGE_URI.INVALID_URI`,
|
||||
`STORAGE_URI.SCHEME_CONFLICT`, `STORAGE_URI.NOT_FOUND`,
|
||||
`STORAGE_URI.LIST_UNSUPPORTED`, `STORAGE_URI.WRITE_UNSUPPORTED`,
|
||||
`STORAGE_URI.ABORTED`, and `STORAGE_URI.ADAPTER_ERROR`.
|
||||
|
||||
## Features
|
||||
|
||||
- Frozen 8-code registry, type-narrowing `instanceof` via the
|
||||
literal-typed `TCode` parameter on `SuaveplanError`.
|
||||
- `uri` and `scheme` discriminants on every `StorageUriError` instance.
|
||||
- `cause` chain preserved when the resolver re-wraps an adapter throwable as
|
||||
`ADAPTER_ERROR` or `WRITE_UNSUPPORTED`.
|
||||
- Zero runtime dependencies beyond `@suaveplan/error`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { STORAGE_URI_CODES, StorageUriError } from "@suaveplan/storage-uri/error-codes";
|
||||
|
||||
throw new StorageUriError(
|
||||
STORAGE_URI_CODES.UNKNOWN_SCHEME,
|
||||
"no plugin for ftp",
|
||||
{ uri: "ftp://x/y", scheme: "ftp" },
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
try {
|
||||
await resolver.read("file:///missing");
|
||||
} catch (err) {
|
||||
if (err instanceof StorageUriError) {
|
||||
if (err.code === STORAGE_URI_CODES.NOT_FOUND) return undefined;
|
||||
if (err.code === STORAGE_URI_CODES.ADAPTER_ERROR) {
|
||||
// Trace back to the original adapter code via the cause chain.
|
||||
console.error("upstream failure", err.cause);
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `STORAGE_URI_CODES` — frozen record `{ UNKNOWN_SCHEME, INVALID_URI,
|
||||
SCHEME_CONFLICT, NOT_FOUND, LIST_UNSUPPORTED, WRITE_UNSUPPORTED, ABORTED,
|
||||
ADAPTER_ERROR }`. Each value is the fully-qualified `STORAGE_URI.<REASON>`
|
||||
string.
|
||||
- `type StorageUriErrorCode` — union of every value in the registry.
|
||||
- `interface StorageUriErrorOptions extends SuaveplanErrorOptions` — adds
|
||||
`uri?: string` and `scheme?: string`.
|
||||
- `class StorageUriError extends SuaveplanError<StorageUriErrorCode>` —
|
||||
exposes `readonly uri`, `readonly scheme`, inherits `code`, `cause`,
|
||||
`context`, `traceId`, `timestamp` from the base class.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`defineCodes` returns a frozen object via `Object.create(null)` so the
|
||||
registry has no prototype-pollution surface. `StorageUriError`'s constructor
|
||||
mirrors `StorageHttpError` and `StorageFileError` — `uri`/`scheme` are
|
||||
captured by direct field assignment rather than through a context map so
|
||||
they survive structured-clone serialisation. The `name` is set to
|
||||
`"StorageUriError"` explicitly so `err.name` is stable across V8/JSC/SM
|
||||
where `new.target.name` could otherwise be munged by minification.
|
||||
@@ -0,0 +1,70 @@
|
||||
# otel
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Owns the OpenTelemetry instrument bag for `@suaveplan/storage-uri`: meter,
|
||||
tracer, and the six counter + one histogram metrics emitted on every
|
||||
resolver operation. The module is intentionally side-effect-free at import
|
||||
time; instruments are created lazily on first access so tests can swap in
|
||||
an in-memory `MeterProvider` before any measurement is recorded.
|
||||
|
||||
The same module also owns the URI redaction helper `redactUri(uri)` used by
|
||||
`StorageUriResolver.#run` to strip query strings and fragments from the
|
||||
`storage.uri` span attribute before emission — presigned-URL secrets must
|
||||
never leak into telemetry sinks.
|
||||
|
||||
## Features
|
||||
|
||||
- Lazy instrument cache rebuilt automatically when the meter provider
|
||||
changes via `setStorageUriMeterProvider(provider)`.
|
||||
- Six counters: `reads_total`, `writes_total`, `deletes_total`,
|
||||
`errors_total`, `stream_fallback_total`, `buffered_write_total`.
|
||||
- One histogram: `op_duration_ms` with `{op, scheme, outcome}` attributes.
|
||||
- `redactUri(uri)` — drops `?query` and `#fragment` segments.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { recordOp } from "@suaveplan/storage-uri/otel";
|
||||
|
||||
recordOp({ op: "read", scheme: "file", outcome: "success", durationMs: 12 });
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
import { setStorageUriMeterProvider, resetInstruments } from "@suaveplan/storage-uri/otel";
|
||||
|
||||
const provider = new MyInMemoryMeterProvider();
|
||||
setStorageUriMeterProvider(provider);
|
||||
resetInstruments();
|
||||
// ... run resolver ops ...
|
||||
const snapshot = provider.collect();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `setStorageUriMeterProvider(provider | undefined): void`
|
||||
- `getInstruments(force?: boolean): StorageUriInstruments`
|
||||
- `resetInstruments(): void`
|
||||
- `recordOp({ op, scheme, outcome, durationMs, code? }): void`
|
||||
- `recordStreamFallback(scheme: string): void`
|
||||
- `recordBufferedWrite(scheme: string): void`
|
||||
- `redactUri(uri: string): string`
|
||||
- Types: `StorageUriOp`, `StorageUriOutcome`, `RecordOpInput`,
|
||||
`StorageUriInstruments`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`recordOp` routes the success counter by op category (`read`/`open`/`stat`/
|
||||
`exists`/`list` → reads, `write` → writes, `remove` → deletes) so consumers
|
||||
need not subscribe to seven separate counters. Errors always increment
|
||||
`errors_total` with the resolved `STORAGE_URI.*` code on the `code`
|
||||
attribute. The instrument cache is intentionally module-local rather than
|
||||
class-bound — the resolver is constructed many times in tests, but the
|
||||
instrument bag is provider-scoped, not resolver-scoped.
|
||||
@@ -0,0 +1,71 @@
|
||||
# parse-uri
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`parseUri(uri)` is the RFC 3986 subset parser the resolver uses to extract a
|
||||
scheme, optional authority, path, and optional query from a URI string. It
|
||||
exists to keep `StorageUriResolver` scheme-agnostic — every adapter plugin
|
||||
owns its own scheme-specific addressing and composes against this base
|
||||
parser.
|
||||
|
||||
The parser intentionally rejects more inputs than RFC 3986 strictly
|
||||
requires: whitespace, NUL bytes, and strings longer than 4096 characters are
|
||||
all hard-failures. The 4096-cap matches the de-facto path-length ceiling
|
||||
observed across S3, Rackspace, Bunny, and most filesystems and prevents
|
||||
quadratic blow-ups inside plugin parsers.
|
||||
|
||||
## Features
|
||||
|
||||
- Scheme regex `^[A-Za-z][A-Za-z0-9+\-.]*$` per RFC 3986 §3.1.
|
||||
- Case-insensitive scheme — the returned `scheme` is lower-cased.
|
||||
- Authority extraction when the rest starts with `//`.
|
||||
- Query/fragment split with both kept off the path.
|
||||
- Single error code on rejection: `STORAGE_URI.INVALID_URI` with the
|
||||
offending input on the `uri` discriminant.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { parseUri } from "@suaveplan/storage-uri/parse-uri";
|
||||
|
||||
const parts = parseUri("s3://bucket/raw/clip.mov");
|
||||
// { scheme: "s3", authority: "bucket", path: "/raw/clip.mov" }
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { parseUri } from "@suaveplan/storage-uri/parse-uri";
|
||||
import { STORAGE_URI_CODES, StorageUriError } from "@suaveplan/storage-uri/error-codes";
|
||||
|
||||
function parseOrNull(uri: string) {
|
||||
try {
|
||||
return parseUri(uri);
|
||||
} catch (err) {
|
||||
if (err instanceof StorageUriError && err.code === STORAGE_URI_CODES.INVALID_URI) {
|
||||
return null;
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface ParsedUriParts { scheme: string; authority?: string; path: string; query?: string }`
|
||||
- `function parseUri(uri: string): ParsedUriParts` — throws
|
||||
`StorageUriError(STORAGE_URI.INVALID_URI)` on rejection.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Authority termination is hand-rolled instead of regex-based so the parser
|
||||
avoids re-walking the input. The `findAuthorityEnd` helper scans for the
|
||||
first `/`, `?`, or `#` character — these are the three RFC 3986 component
|
||||
delimiters that may follow an authority. Empty path is permitted (e.g.
|
||||
`s3://bucket`); empty rest after the colon (`scheme:`) is not. The
|
||||
maximum-length check happens BEFORE the regex test so pathological inputs
|
||||
cannot pin a regex engine.
|
||||
@@ -0,0 +1,66 @@
|
||||
# resolver.list
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements the `list(uri, options?, signal?)` semantics. Treats the input
|
||||
URI's key portion as a listing prefix, delegates to
|
||||
`StorageLike.list({ prefix, limit, continuationToken })`, then re-serialises
|
||||
each returned adapter key back to a fully-qualified URI via the plugin's
|
||||
`build(key, meta)` function. Plugins without `build` cause `list()` to throw
|
||||
`STORAGE_URI.LIST_UNSUPPORTED`.
|
||||
|
||||
The function lives in its own file so the prefix-and-rebuild logic stays
|
||||
isolated from the rest of the resolver and so its specific error mapping
|
||||
can evolve without touching adjacent ops.
|
||||
|
||||
## Features
|
||||
|
||||
- Prefix derived from the plugin's `parse(uri).key`.
|
||||
- Limit and continuation-token forwarded verbatim when supplied.
|
||||
- `LIST_UNSUPPORTED` raised at the top of the function when `build` is
|
||||
missing — before any adapter call is issued.
|
||||
- `hasMore`/`continuationToken` round-tripped opaquely from the adapter.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { listViaPlugin } from "@suaveplan/storage-uri/resolver";
|
||||
|
||||
const out = await listViaPlugin(plugin, "s3://bucket/raw/", "raw/", { bucket: "bucket" }, { limit: 10 });
|
||||
// { uris: ["s3://bucket/raw/a.mov", ...], hasMore: false }
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
async function listAll(plugin, uri, prefix, meta) {
|
||||
const acc: string[] = [];
|
||||
let token: string | undefined;
|
||||
do {
|
||||
const opts = token === undefined ? { limit: 100 } : { limit: 100, continuationToken: token };
|
||||
const page = await listViaPlugin(plugin, uri, prefix, meta, opts);
|
||||
acc.push(...page.uris);
|
||||
token = page.continuationToken;
|
||||
} while (token !== undefined);
|
||||
return acc;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `function listViaPlugin(plugin, uri, prefix, meta?, options?): Promise<ListUriResult>`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`build` is captured via `.bind(plugin)` before the map so `this`-bound
|
||||
plugin implementations work without surprise. The output object is
|
||||
constructed via spread so `continuationToken` is omitted entirely when the
|
||||
adapter returned `undefined` — matters under
|
||||
`exactOptionalPropertyTypes: true` and keeps the resolver's contract
|
||||
consistent with `StorageLike.list`. The capability-check fires before any
|
||||
adapter call so an unsupported list does not waste an upstream request:
|
||||
operators can probe for list support cheaply by catching
|
||||
`STORAGE_URI.LIST_UNSUPPORTED` once and caching the result per-plugin.
|
||||
`meta` is forwarded verbatim to `build` so plugins that carry a bucket,
|
||||
container, or zone in `meta` get the same value back; downstream consumers
|
||||
that paginate across pages will see the same authority on every URI.
|
||||
@@ -0,0 +1,94 @@
|
||||
# resolver
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
The `StorageUriResolver` class is the entry point for URI-routed dispatch in
|
||||
`@suaveplan/storage-uri`. Constructed once at boot from an array of
|
||||
`UriPlugin`s, it exposes `open`/`read`/`write`/`stat`/`exists`/`remove`/
|
||||
`list` keyed by URI string and delegates each call to the plugin matching
|
||||
the URI's scheme. The resolver performs no I/O of its own — every byte goes
|
||||
through an existing `StorageLike` adapter — and is fully immutable after
|
||||
construction.
|
||||
|
||||
`createStorageUriResolver(plugins)` is the preferred factory. It throws
|
||||
`STORAGE_URI.SCHEME_CONFLICT` when two distinct plugin references claim the
|
||||
same scheme; registering the same reference twice is a silent no-op. The
|
||||
result is frozen via `Object.freeze`.
|
||||
|
||||
## Features
|
||||
|
||||
- O(1) scheme dispatch via `Map<scheme, UriPlugin>` (case-insensitive).
|
||||
- `AbortSignal` propagated to every adapter call and short-circuited
|
||||
pre-aborted with `STORAGE_URI.ABORTED`.
|
||||
- `SuaveplanError` subclasses thrown by an adapter (any package's namespaced
|
||||
error — `DriveUriError`, `StorageHttpError`, etc.) pass through the resolver
|
||||
unchanged so consumers can `switch (err.code)` on the original code. Only
|
||||
non-namespaced throwables (plain `Error`, foreign objects) are wrapped as
|
||||
`STORAGE_URI.ADAPTER_ERROR` with the original preserved as `cause`.
|
||||
- One span per public op (`storage.uri.{open,read,write,stat,exists,remove,list}`)
|
||||
on the `@suaveplan/storage-uri` tracer.
|
||||
- `op_duration_ms`, `reads_total`, `writes_total`, `deletes_total`,
|
||||
`errors_total` counters emitted automatically.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { createStorageUriResolver } from "@suaveplan/storage-uri";
|
||||
|
||||
const resolver = createStorageUriResolver([fileUriPlugin, httpsUriPlugin]);
|
||||
const bytes = await resolver.read("file:///cache/poster.jpg");
|
||||
await resolver.write("file:///cache/poster.jpg", newBytes);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
const stream = await resolver.open("https://cdn.example.com/clip/master.m3u8");
|
||||
const reader = stream.getReader();
|
||||
try {
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
sink.write(value);
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
|
||||
// Prefix list with continuation paging.
|
||||
let token: string | undefined;
|
||||
do {
|
||||
const page = await resolver.list("s3://bucket/raw/", { limit: 100, ...(token !== undefined ? { continuationToken: token } : {}) });
|
||||
for (const uri of page.uris) consume(uri);
|
||||
token = page.continuationToken;
|
||||
} while (token !== undefined);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class StorageUriResolver`
|
||||
- `constructor(plugins: readonly UriPlugin[])`
|
||||
- `get schemeCount(): number`
|
||||
- `hasScheme(scheme: string): boolean`
|
||||
- `open(uri, signal?): Promise<ReadableStream<Uint8Array>>`
|
||||
- `read(uri, signal?): Promise<Uint8Array>`
|
||||
- `write(uri, body, signal?): Promise<void>`
|
||||
- `stat(uri, signal?): Promise<StatResult | undefined>`
|
||||
- `exists(uri, signal?): Promise<boolean>`
|
||||
- `remove(uri, signal?): Promise<boolean>`
|
||||
- `list(uri, options?, signal?): Promise<ListUriResult>`
|
||||
- `function createStorageUriResolver(plugins): StorageUriResolver`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Dispatch is split into `#dispatch(uri)` (parse + lookup + plugin-`parse`)
|
||||
and per-op handlers. Every public method funnels through the `#run`
|
||||
generic, which owns the pre-aborted check, span start/end, error wrap, and
|
||||
duration emission. Splitting `open`/`write`/`list` into sibling modules
|
||||
keeps each file under the 300-line code budget without forcing the call
|
||||
site to know which file owns which method. The class field is `#plugins`,
|
||||
not `_plugins`, to make accidental external access a hard syntax error.
|
||||
@@ -0,0 +1,61 @@
|
||||
# resolver.streams
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements the `open(uri, signal?)` semantics for the resolver: forward a
|
||||
native `createReadStream` when the adapter exposes one, otherwise fall back
|
||||
to wrapping `adapter.get(key)` in a one-chunk `ReadableStream` and increment
|
||||
`storage_uri.stream_fallback_total{scheme}`. Splitting this branch into its
|
||||
own file keeps `resolver.ts` under the code-line budget while letting the
|
||||
streaming path evolve independently.
|
||||
|
||||
The function is exported so adapter authors and integration tests can call
|
||||
it directly without instantiating the full `StorageUriResolver` — useful for
|
||||
isolating the fallback behaviour in unit tests.
|
||||
|
||||
## Features
|
||||
|
||||
- Duck-typed `createReadStream` detection on the adapter.
|
||||
- One-chunk `ReadableStream` synthesis from a buffered `get` result.
|
||||
- `STORAGE_URI.NOT_FOUND` mapping when `get` returns `undefined`.
|
||||
- `storage_uri.stream_fallback_total` counter increment on the fallback
|
||||
path (not on the forwarding path).
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { openViaPlugin } from "@suaveplan/storage-uri/resolver";
|
||||
|
||||
const stream = await openViaPlugin(plugin, "memory://x/a", "a", undefined);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
const ctrl = new AbortController();
|
||||
try {
|
||||
const stream = await openViaPlugin(plugin, uri, key, ctrl.signal);
|
||||
pipe(stream, sink, { signal: ctrl.signal });
|
||||
} finally {
|
||||
ctrl.abort();
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `function openViaPlugin(plugin, uri, key, signal): Promise<ReadableStream<Uint8Array>>`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The function does NOT do the pre-aborted check; that lives in `#run` so
|
||||
every public method shares one short-circuit point. The check for
|
||||
`typeof adapter.createReadStream === "function"` is done on the casted
|
||||
`adapter` value because `StorageLike` does not declare the method — the
|
||||
duck-typing is intentional so any adapter exposing the right shape is
|
||||
forwarded automatically. The fallback path materialises the entire object
|
||||
in memory, which is correct for small payloads but a hazard for large ones
|
||||
behind a non-streaming adapter; operators should alert on the
|
||||
`storage_uri.stream_fallback_total` counter to detect mis-composed
|
||||
deployments before they OOM. The one-chunk `ReadableStream` close is
|
||||
deterministic — the queue is enqueued and the controller closed in the
|
||||
same microtask, so consumers see exactly one chunk before `done: true`.
|
||||
@@ -0,0 +1,63 @@
|
||||
# resolver.writes
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements the `write(uri, body, signal?)` semantics. Accepts either
|
||||
`Uint8Array` or `ReadableStream<Uint8Array>` for the body, forwards a native
|
||||
`createWriteStream` when the adapter exposes one and the body is a stream,
|
||||
and otherwise drains the stream (via `new Response(stream).arrayBuffer()`)
|
||||
into a single `adapter.set` call. The drain path increments
|
||||
`storage_uri.buffered_write_total{scheme}` so operators can alert on
|
||||
mis-composed adapters that quietly buffer large payloads.
|
||||
|
||||
`SuaveplanError` subclasses thrown by `adapter.set` (any package's
|
||||
namespaced error — `DriveUriError`, `StorageHttpError`, etc.) pass through
|
||||
unchanged so consumers can route on the original `code`. Non-namespaced
|
||||
throwables (plain `Error`, foreign objects) are wrapped as
|
||||
`STORAGE_URI.WRITE_UNSUPPORTED` with the original preserved as `cause`,
|
||||
giving a normalised failure mode for read-only adapters that lack their
|
||||
own error class.
|
||||
|
||||
## Features
|
||||
|
||||
- Branch on `body` shape via `getReader` duck-test.
|
||||
- Forward `createWriteStream` only when present AND body is a stream.
|
||||
- Drain-and-set fallback for both buffered bodies and missing
|
||||
`createWriteStream`.
|
||||
- `SuaveplanError` pass-through on `adapter.set` throw; `WRITE_UNSUPPORTED`
|
||||
wrap with `cause` preserved for non-namespaced throwables only.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { writeViaPlugin } from "@suaveplan/storage-uri/resolver";
|
||||
|
||||
await writeViaPlugin(plugin, "file:///a.bin", "a.bin", new Uint8Array([1, 2, 3]), undefined);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
async function writeStream(resolver: StorageUriResolver, uri: string, src: AsyncIterable<Uint8Array>): Promise<void> {
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
async start(ctrl) {
|
||||
for await (const chunk of src) ctrl.enqueue(chunk);
|
||||
ctrl.close();
|
||||
},
|
||||
});
|
||||
await resolver.write(uri, stream);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `function writeViaPlugin(plugin, uri, key, body, signal): Promise<void>`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`new Response(stream).arrayBuffer()` is the cheapest cross-runtime way to
|
||||
drain a Web Stream into a contiguous buffer; it works in Node 18+, Bun, and
|
||||
every browser without any polyfill. The `WRITE_UNSUPPORTED` re-wrap is
|
||||
distinct from the `ADAPTER_ERROR` wrap in `resolver.ts` because callers
|
||||
typically branch on these two conditions differently — `WRITE_UNSUPPORTED`
|
||||
is a capability miss while `ADAPTER_ERROR` is a transient upstream fault.
|
||||
@@ -0,0 +1,72 @@
|
||||
# plugin.types
|
||||
|
||||
## Purpose
|
||||
|
||||
Declares the public type surface every adapter package targets when shipping
|
||||
its own `uriPlugin`. The resolver knows nothing about S3 buckets, CDN zones,
|
||||
or filesystem roots; instead it accepts any object conforming to
|
||||
`UriPlugin`, which is the structural contract between this package and every
|
||||
sibling `storage-*` adapter.
|
||||
|
||||
The same module declares the optional `StreamingStorageExtensions` interface
|
||||
the resolver duck-types for `createReadStream`, `createWriteStream`, and
|
||||
`getMetadata`, plus the `StatResult` shape returned by
|
||||
`StorageUriResolver.stat`. None of these types carry runtime code.
|
||||
|
||||
## Features
|
||||
|
||||
- `UriPlugin` — `{ scheme, adapter, parse, build? }` carrying the entire
|
||||
scheme-handler contract.
|
||||
- `ParsedUri` — return type of `plugin.parse(uri)`; `{ key, meta? }`.
|
||||
- `StreamingStorageExtensions` — duck-typed optional surface.
|
||||
- `StatResult` — `{ size?, etag?, lastModified?, contentType? }`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import type { UriPlugin } from "@suaveplan/storage-uri/types";
|
||||
|
||||
const plugin: UriPlugin = {
|
||||
scheme: "memory",
|
||||
adapter,
|
||||
parse(uri) { return { key: uri.slice("memory://".length) }; },
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import type { UriPlugin, StreamingStorageExtensions } from "@suaveplan/storage-uri/types";
|
||||
|
||||
function makePlugin(adapter: StorageLike & StreamingStorageExtensions): UriPlugin {
|
||||
return {
|
||||
scheme: "rackspace",
|
||||
adapter,
|
||||
parse(uri) {
|
||||
const url = new URL(uri);
|
||||
return { key: url.pathname.slice(1), meta: { container: url.host } };
|
||||
},
|
||||
build(key, meta) {
|
||||
const container = meta?.container ?? "";
|
||||
return `rackspace://${container}/${key}`;
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface ParsedUri { key: string; meta?: Record<string,string> }`
|
||||
- `interface UriPlugin { scheme; adapter; parse; build? }`
|
||||
- `interface StreamingStorageExtensions { createReadStream?; createWriteStream?; getMetadata? }`
|
||||
- `interface StatResult { size?; etag?; lastModified?; contentType? }`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`adapter` is typed as `StorageLike` (from `@suaveplan/types`), NOT
|
||||
`AbstractStorage` from `@suaveplan/storage`. That lets in-memory
|
||||
test stubs satisfy the contract without
|
||||
pulling in the storage runtime. `meta` is a `Record<string, string>` rather
|
||||
than a generic shape so all adapter-specific addressing (bucket, zone,
|
||||
container, query) is stringly-typed at the plugin boundary — each adapter is
|
||||
free to expose typed helpers on top.
|
||||
@@ -0,0 +1,65 @@
|
||||
# resolver.types
|
||||
|
||||
## Purpose
|
||||
|
||||
Declares the option and result types carried by `StorageUriResolver.list`.
|
||||
These live in their own module so consumers can import the shapes without
|
||||
pulling in the resolver class itself — handy when type-checking plugin
|
||||
authoring code that never instantiates a resolver but does need to type its
|
||||
own list results.
|
||||
|
||||
## Features
|
||||
|
||||
- `ListUriOptions` — `{ limit?, continuationToken? }`. The `prefix` is
|
||||
derived from the input URI by the resolver; consumers never set it
|
||||
directly.
|
||||
- `ListUriResult` — `{ uris, hasMore, continuationToken? }` returned by
|
||||
`StorageUriResolver.list`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import type { ListUriOptions, ListUriResult } from "@suaveplan/storage-uri/types";
|
||||
|
||||
const opts: ListUriOptions = { limit: 50 };
|
||||
const out: ListUriResult = await resolver.list("s3://bucket/raw/", opts);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import type { ListUriOptions, ListUriResult } from "@suaveplan/storage-uri/types";
|
||||
|
||||
async function* drain(
|
||||
resolver: StorageUriResolver,
|
||||
base: string,
|
||||
): AsyncIterable<string> {
|
||||
let token: string | undefined;
|
||||
do {
|
||||
const opts: ListUriOptions = token === undefined ? {} : { continuationToken: token };
|
||||
const page: ListUriResult = await resolver.list(base, opts);
|
||||
yield* page.uris;
|
||||
token = page.continuationToken;
|
||||
} while (token !== undefined);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface ListUriOptions { limit?: number; continuationToken?: string }`
|
||||
- `interface ListUriResult { uris: readonly string[]; hasMore: boolean; continuationToken?: string }`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`uris` is a `readonly string[]` rather than a mutable array so consumers
|
||||
cannot accidentally mutate the resolver's return value. `continuationToken`
|
||||
is intentionally opaque — the resolver forwards whatever the adapter
|
||||
returned, and consumers MUST pass it back unchanged. Adapters that cannot
|
||||
paginate return `hasMore: false` and never populate `continuationToken`;
|
||||
consumers SHOULD treat `hasMore === false` as the loop's termination
|
||||
condition rather than `continuationToken === undefined`. The split between
|
||||
`ListUriOptions` and `ListUriResult` mirrors `StorageLike.list`'s own
|
||||
input/output shapes one-for-one, so adapter authors who already implement
|
||||
`StorageLike.list` need no extra adaptation layer to satisfy
|
||||
`StorageUriResolver.list` — only `build()` for URI re-serialisation. Both
|
||||
shapes are deeply readonly so consumers cannot mutate them after receipt.
|
||||
Reference in New Issue
Block a user