290 lines
20 KiB
Markdown
290 lines
20 KiB
Markdown
<!-- markdownlint-disable MD024 -->
|
||
|
||
# Capability Spec Delta: storage-http (storage-http)
|
||
|
||
**Package**: `@suaveplan/storage-http`
|
||
**Status**: PROPOSED
|
||
|
||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||
|
||
## Requirements
|
||
|
||
### Requirement: HttpStorage extends AbstractStorage
|
||
|
||
The package SHALL provide an `HttpStorage` class extending `AbstractStorage` from `@suaveplan/storage`. The class SHALL implement every abstract method on the base (`get`, `set`, `delete`, `exists`, `list`, `clear`, `getWithMetadata`, `getMetadata`, `getSize`, `getPartial`, `setPartial`, `getItemSize`, `createReadStream`, `createWriteStream`) with the signatures declared on the base, including the trailing `signal?: AbortSignal` parameter. Methods unsupported in read-only mode SHALL throw `StorageHttpError` with code `STORAGE_HTTP.READ_ONLY`.
|
||
|
||
#### Scenario: instance is structurally an AbstractStorage
|
||
|
||
- **WHEN** a caller constructs `new HttpStorage({ baseUrl: "https://cdn.example.com/start/here/" })`
|
||
- **THEN** the instance is `instanceof AbstractStorage` and every abstract method is callable
|
||
|
||
#### Scenario: capabilities matrix declares partial reads and native streams
|
||
|
||
- **WHEN** a caller reads `storage.capabilities`
|
||
- **THEN** `partialReads === true`, `nativeStreams === true`, `atomicWrites === false`, `partialWrites === false`, `contentAddressable === false`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"implements every AbstractStorage abstract method"` AND `src/http-storage/http-storage.spec-conformance.test.ts::"declares the documented capabilities matrix"`
|
||
|
||
---
|
||
|
||
### Requirement: Options validated at the boundary via Zod
|
||
|
||
The package SHALL expose `HttpStorageOptions` (interface) and `HttpStorageOptionsSchema` (Zod). The constructor SHALL parse options through the schema and throw `StorageHttpError` with code `STORAGE_HTTP.INVALID_OPTIONS` on validation failure. `baseUrl` SHALL be required, MUST be an absolute `http://` or `https://` URL, AND MUST end with `/`. `headers` MAY be a static record or a function `(key, op) => Record<string, string> | Promise<…>` called per request. `fetch`, `uploader`, `keyEncoder`, `timeout`, `retry`, and `etagCache` SHALL be optional.
|
||
|
||
#### Scenario: baseUrl without trailing slash is rejected
|
||
|
||
- **WHEN** a caller constructs `new HttpStorage({ baseUrl: "https://cdn.example.com/start/here" })`
|
||
- **THEN** the constructor throws `StorageHttpError` with `code === "STORAGE_HTTP.INVALID_OPTIONS"`
|
||
|
||
#### Scenario: non-http(s) baseUrl is rejected
|
||
|
||
- **WHEN** a caller constructs `new HttpStorage({ baseUrl: "ftp://example.com/" })`
|
||
- **THEN** the constructor throws `StorageHttpError` with `code === "STORAGE_HTTP.INVALID_OPTIONS"`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"rejects baseUrl missing trailing slash with INVALID_OPTIONS"` AND `src/http-storage/http-storage.test.ts::"rejects non-http(s) baseUrl with INVALID_OPTIONS"`
|
||
|
||
---
|
||
|
||
### Requirement: GET retrieves bytes from baseUrl + key
|
||
|
||
`get(key, signal?)` SHALL issue an HTTP `GET` to `new URL(encodedKey, baseUrl)` and return the response body as `Uint8Array`. The injected `headers` callback (if provided) SHALL be invoked with `(key, "get")` and its result merged into the request init. On `200 OK`, the body SHALL be returned. On `404 Not Found`, the method SHALL return `undefined` (not throw). On other non-success status, the method SHALL throw `StorageHttpError` per the status → code map in `design.md §3b`.
|
||
|
||
#### Scenario: round-trip GET returns the body bytes
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `new Response(new Uint8Array([1,2,3]), { status: 200 })` for `https://cdn.example.com/start/here/a/b/test.doc`
|
||
- **AND** the caller invokes `await storage.get("a/b/test.doc")`
|
||
- **THEN** the result is a `Uint8Array` byte-equal to `[1,2,3]`
|
||
|
||
#### Scenario: 404 Not Found yields undefined, not a throw
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `new Response(null, { status: 404 })`
|
||
- **AND** the caller invokes `await storage.get("missing")`
|
||
- **THEN** the result is `undefined` and no error is thrown
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"GET round-trips body bytes from baseUrl + key"` AND `src/http-storage/http-storage.test.ts::"GET returns undefined on 404 without throwing"`
|
||
|
||
---
|
||
|
||
### Requirement: Strict key sanitisation at the boundary
|
||
|
||
Every method that accepts a key SHALL pass it through `sanitizeKey` before issuing any fetch call. The sanitiser SHALL reject empty strings, keys containing `..` segments, keys starting with `/` or matching a drive-letter prefix (`C:\…`), keys containing null bytes (` `), keys containing control chars (`0x00`–`0x1F`, `0x7F`), and keys longer than 2048 characters. Windows `\` separators SHALL be normalised to POSIX `/`. The default `keyEncoder` SHALL apply `encodeURIComponent` per `/`-separated segment, preserving `/`. Rejections SHALL throw `StorageHttpError` with code `STORAGE_HTTP.INVALID_KEY` or `STORAGE_HTTP.KEY_TOO_LONG`.
|
||
|
||
#### Scenario: path-traversal attempt is rejected before fetch
|
||
|
||
- **WHEN** a caller calls `storage.get("../../etc/passwd")`
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.INVALID_KEY"` AND the injected `FetchLike` is not invoked
|
||
|
||
#### Scenario: overlong key is rejected with KEY_TOO_LONG
|
||
|
||
- **WHEN** a caller calls `storage.get("a".repeat(2049))`
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.KEY_TOO_LONG"`
|
||
|
||
#### Scenario: spaces in key segments are URL-encoded, slashes preserved
|
||
|
||
- **WHEN** a caller calls `storage.get("hello world/sub dir/file.txt")`
|
||
- **THEN** the fetched URL is `https://cdn.example.com/start/here/hello%20world/sub%20dir/file.txt`
|
||
|
||
**Verified by:** `src/key-sanitizer/key-sanitizer.test.ts::"rejects path traversal segments"` AND `src/key-sanitizer/key-sanitizer.test.ts::"rejects keys longer than 2048 chars with KEY_TOO_LONG"` AND `src/key-sanitizer/key-sanitizer.test.ts::"encodes spaces per segment, preserves slashes"`
|
||
|
||
---
|
||
|
||
### Requirement: HEAD-backed exists and getMetadata
|
||
|
||
`exists(key, signal?)` SHALL issue `HEAD <URL>` and return `true` on `2xx`, `false` on `404`/`410`, and throw on other non-success status. `getMetadata(key, signal?)` SHALL issue `HEAD <URL>` (or read from the ETag cache if populated) and return `{ size: number, contentType?: string, lastModified?: Date, etag?: string }`. `size` SHALL come from `Content-Length`; absence yields `size: -1` (sentinel for "unknown").
|
||
|
||
#### Scenario: HEAD 200 yields exists=true and parsed metadata
|
||
|
||
- **WHEN** the injected `FetchLike` resolves a `HEAD` to `new Response(null, { status: 200, headers: { "content-length": "42", "content-type": "text/plain", "etag": "\"abc\"" } })`
|
||
- **THEN** `await storage.exists("k")` is `true` AND `await storage.getMetadata("k")` is `{ size: 42, contentType: "text/plain", etag: '"abc"' }`
|
||
|
||
#### Scenario: HEAD 404 yields exists=false without throwing
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `new Response(null, { status: 404 })`
|
||
- **THEN** `await storage.exists("k")` is `false`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"HEAD 200 yields exists=true and parsed metadata"` AND `src/http-storage/http-storage.test.ts::"HEAD 404 yields exists=false without throwing"`
|
||
|
||
---
|
||
|
||
### Requirement: Range reads via getPartial
|
||
|
||
`getPartial(key, offset, length, signal?)` SHALL issue `GET <URL>` with header `Range: bytes=<offset>-<offset+length-1>`. On `206 Partial Content`, the body SHALL be returned verbatim. On `200 OK` (origin ignored the Range), the full body SHALL be consumed, sliced to `[offset, offset+length)`, returned, AND the counter `storage_http.range_fallback_total{outcome="ok"}` SHALL be incremented. On `416 Range Not Satisfiable`, the method SHALL throw `StorageHttpError` with code `STORAGE_HTTP.NOT_FOUND` when the requested range exceeds known size. `capabilities.partialReads` SHALL be `true`.
|
||
|
||
#### Scenario: 206 returns the partial bytes verbatim
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `new Response(new Uint8Array([2,3]), { status: 206, headers: { "content-range": "bytes 1-2/10" } })`
|
||
- **AND** the caller invokes `await storage.getPartial("k", 1, 2)`
|
||
- **THEN** the result is byte-equal to `[2,3]`
|
||
|
||
#### Scenario: 200 fallback slices client-side and increments counter
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `new Response(new Uint8Array([0,1,2,3,4]), { status: 200 })` for a Range request
|
||
- **AND** the caller invokes `await storage.getPartial("k", 1, 2)`
|
||
- **THEN** the result is byte-equal to `[1,2]` AND `storage_http.range_fallback_total` has been incremented exactly once
|
||
|
||
**Verified by:** `src/range/range.test.ts::"206 Partial Content returns body verbatim"` AND `src/range/range.test.ts::"200 fallback slices client-side and increments range_fallback_total"`
|
||
|
||
---
|
||
|
||
### Requirement: Native streams via createReadStream
|
||
|
||
`createReadStream(key, signal?)` SHALL return a `ReadableStream<Uint8Array>`. The stream SHALL be the `Response.body` of a `GET <URL>` when present (no buffering). If the response status is non-success, the returned stream SHALL error with `StorageHttpError`. Cancelling the stream SHALL abort the underlying fetch via the injected `AbortSignal`. `capabilities.nativeStreams` SHALL be `true`.
|
||
|
||
#### Scenario: response body is returned without buffering
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to a `Response` whose body is a `ReadableStream` of three 1KiB chunks
|
||
- **AND** the caller invokes `storage.createReadStream("k")` and reads to completion
|
||
- **THEN** the reader observes the same three chunks in order without coalescing
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"createReadStream returns Response.body without buffering"`
|
||
|
||
---
|
||
|
||
### Requirement: Conditional reads via ETag cache
|
||
|
||
When `options.etagCache` is enabled, `get` SHALL attach `If-None-Match: <etag>` whenever a cached ETag exists for the key. On `304 Not Modified`, the cached body SHALL be returned without retransmission. On `200 OK`, the cache entry SHALL be replaced (subject to LRU eviction at the configured capacity). The cache SHALL be in-memory and per-instance.
|
||
|
||
#### Scenario: 304 returns cached body without retransmitting
|
||
|
||
- **WHEN** a first `get("k")` populates the cache with `etag: '"v1"'` and body `B1`
|
||
- **AND** a second `get("k")` is issued with the injected `FetchLike` responding `304 Not Modified` to a request carrying `If-None-Match: "v1"`
|
||
- **THEN** the second call returns a body byte-equal to `B1` AND no body bytes are consumed from the fetch
|
||
|
||
**Verified by:** `src/etag-cache/etag-cache.test.ts::"304 Not Modified returns cached body without retransmission"`
|
||
|
||
---
|
||
|
||
### Requirement: Retries on transient status and Retry-After honour
|
||
|
||
The adapter SHALL classify HTTP status `408`, `425`, `429`, and `5xx` as transient. Network rejection from `FetchLike` SHALL be classified as transient. On transient outcome, the adapter SHALL retry via `@suaveplan/retry.retry` using injectable `TimersLike` for backoff. When the response carries `Retry-After` (delta-seconds or HTTP-date), the next attempt SHALL wait at least that duration. Non-transient `4xx` SHALL NOT retry. Exhausted retries SHALL surface as `STORAGE_HTTP.UPSTREAM_ERROR` (HTTP exhaustion) or `STORAGE_HTTP.NETWORK_ERROR` (fetch-reject exhaustion).
|
||
|
||
#### Scenario: 503 with Retry-After is retried after the specified delay
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `503` with `Retry-After: 2` on the first call, then `200` on the second
|
||
- **AND** the injected `TimersLike` is a `FakeClock`
|
||
- **THEN** the caller observes one `setTimeout` advance of at least 2000 ms AND the final result is the `200` body
|
||
|
||
#### Scenario: 401 Unauthorized is NOT retried
|
||
|
||
- **WHEN** the injected `FetchLike` resolves to `401 Unauthorized`
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.UNAUTHORIZED"` AND `FetchLike` was invoked exactly once
|
||
|
||
**Verified by:** `src/retry-helper/retry-helper.test.ts::"retries 503 honouring Retry-After"` AND `src/retry-helper/retry-helper.test.ts::"does not retry 401 Unauthorized"`
|
||
|
||
---
|
||
|
||
### Requirement: AbortSignal threaded into every fetch
|
||
|
||
Every public async method SHALL accept an optional `signal: AbortSignal` and pass it into the underlying `FetchLike` `init.signal`. When the caller passes an already-aborted signal, the method SHALL throw `StorageHttpError` with code `STORAGE_HTTP.ABORTED` before any fetch is issued. When the signal aborts mid-request, the in-flight fetch SHALL be aborted and the method SHALL reject with `STORAGE_HTTP.ABORTED`.
|
||
|
||
#### Scenario: pre-aborted signal short-circuits before fetch
|
||
|
||
- **WHEN** a caller passes an already-aborted signal to any read method
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.ABORTED"` AND the injected `FetchLike` is not invoked
|
||
|
||
#### Scenario: mid-flight abort propagates into fetch
|
||
|
||
- **WHEN** a `get` is in flight and the caller calls `controller.abort()`
|
||
- **THEN** the injected `FetchLike` observes the aborted signal AND the call rejects with `STORAGE_HTTP.ABORTED`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"rejects with STORAGE_HTTP.ABORTED when signal is pre-aborted"` AND `src/http-storage/http-storage.test.ts::"propagates mid-flight abort into the underlying fetch"`
|
||
|
||
---
|
||
|
||
### Requirement: Writes via injected HttpUploader
|
||
|
||
When `options.uploader` is provided, `set(key, bytes, signal?)` SHALL invoke `uploader.put(key, bytes, headersForKey, signal)` exactly once with the sanitised key and the per-request headers (from the `headers` callback). When `uploader.delete` is defined, `delete(key, signal?)` SHALL invoke it. When no uploader is provided, `set` and `delete` SHALL throw `StorageHttpError` with code `STORAGE_HTTP.READ_ONLY`. `createWriteStream` SHALL collect to memory and dispatch `uploader.put` on `close`; without an uploader it SHALL throw `STORAGE_HTTP.READ_ONLY` immediately.
|
||
|
||
#### Scenario: set with injected uploader invokes put exactly once
|
||
|
||
- **WHEN** an `HttpUploader` is injected and the caller invokes `await storage.set("a/b.bin", new Uint8Array([1,2,3]))`
|
||
- **THEN** `uploader.put` was invoked exactly once with `("a/b.bin", <bytes>, <headers>, <signal>)`
|
||
|
||
#### Scenario: set without uploader throws READ_ONLY
|
||
|
||
- **WHEN** the storage was constructed without `uploader` AND a caller invokes `storage.set("k", bytes)`
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.READ_ONLY"`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"set invokes injected uploader.put exactly once"` AND `src/http-storage/http-storage.test.ts::"set without uploader throws READ_ONLY"`
|
||
|
||
---
|
||
|
||
### Requirement: list and setPartial are read-only by default
|
||
|
||
`list(prefix?, options?)` SHALL always throw `StorageHttpError` with code `STORAGE_HTTP.READ_ONLY` — HTTP origins do not have a uniform directory protocol (see proposal "Out of scope"). `setPartial` SHALL throw `STORAGE_HTTP.READ_ONLY` when no `uploader` is wired; when an uploader is wired, `setPartial` SHALL fall through to `AbstractStorage.defaultSetPartial` (read-modify-write) and re-upload the full object. `clear()` SHALL throw `STORAGE_HTTP.READ_ONLY` in all configurations.
|
||
|
||
#### Scenario: list always throws READ_ONLY
|
||
|
||
- **WHEN** any caller invokes `storage.list()` or `storage.list("a/")`
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.READ_ONLY"`
|
||
|
||
#### Scenario: clear always throws READ_ONLY
|
||
|
||
- **WHEN** any caller invokes `storage.clear()` even with an uploader wired
|
||
- **THEN** the call rejects with `StorageHttpError` whose `code === "STORAGE_HTTP.READ_ONLY"`
|
||
|
||
**Verified by:** `src/http-storage/http-storage.test.ts::"list always throws READ_ONLY"` AND `src/http-storage/http-storage.test.ts::"clear always throws READ_ONLY"`
|
||
|
||
---
|
||
|
||
### Requirement: STORAGE_HTTP error registry
|
||
|
||
The package SHALL declare `STORAGE_HTTP_CODES` via `defineCodes("STORAGE_HTTP", [...])` with exactly these 10 codes: `NOT_FOUND`, `UNAUTHORIZED`, `FORBIDDEN`, `NETWORK_ERROR`, `INVALID_KEY`, `KEY_TOO_LONG`, `INVALID_OPTIONS`, `READ_ONLY`, `ABORTED`, `UPSTREAM_ERROR`. The package SHALL declare `StorageHttpError extends SuaveplanError<StorageHttpErrorCode>` with optional `key`, `httpStatus`, and `url` discriminants. Every code SHALL be reachable from a real call site exercised by tests.
|
||
|
||
#### Scenario: every declared code is reachable from a test
|
||
|
||
- **WHEN** the test suite is run
|
||
- **THEN** every code in `STORAGE_HTTP_CODES` has at least one test that exercises a code path producing that code
|
||
|
||
**Verified by:** `src/http-storage/http-storage.spec-conformance.test.ts::"every STORAGE_HTTP code is reachable from a real call site"`
|
||
|
||
---
|
||
|
||
### Requirement: Module-augmentation registry slot
|
||
|
||
The package SHALL declare `declare module "@suaveplan/storage"` augmenting `StorageProviderRegistry` with `http: HttpStorage` and `StorageProviderConfig` with `http: HttpStorageOptions`. The package SHALL export `registerHttpProvider(): void` that registers the constructor with `StorageFactory`. Registration SHALL be idempotent — calling it twice SHALL not throw and SHALL not double-register.
|
||
|
||
#### Scenario: createStorage with type "http" yields HttpStorage
|
||
|
||
- **WHEN** `registerHttpProvider()` has been called
|
||
- **AND** the caller invokes `createStorage({ type: "http", config: { baseUrl: "https://cdn.example.com/start/here/" } })`
|
||
- **THEN** the returned instance is `instanceof HttpStorage`
|
||
|
||
#### Scenario: registration is idempotent
|
||
|
||
- **WHEN** `registerHttpProvider()` is invoked twice
|
||
- **THEN** neither call throws AND `createStorage({ type: "http", config })` continues to yield a single registered constructor
|
||
|
||
**Verified by:** `src/registry/registry.test.ts::"createStorage with type http returns HttpStorage"` AND `src/registry/registry.test.ts::"registerHttpProvider is idempotent"`
|
||
|
||
---
|
||
|
||
### Requirement: OpenTelemetry instrumentation
|
||
|
||
The package SHALL emit histograms `storage_http.op_duration_ms{op,outcome,status_class}` and `storage_http.io_bytes{op,direction}`, and counters `storage_http.reads_total`, `storage_http.writes_total`, `storage_http.deletes_total`, `storage_http.errors_total`, `storage_http.range_fallback_total` — each with `outcome` attribute. Every public operation SHALL open a span on the `@suaveplan/storage-http` tracer with attributes `http.url`, `http.method`, `http.status_code` (when known), `http.response_content_length` (when known), and `storage.key` (sanitised, never raw caller input). Spans SHALL record exceptions on failure and end in the `error` status.
|
||
|
||
#### Scenario: successful GET emits one read counter and one duration histogram observation
|
||
|
||
- **WHEN** a `get` succeeds against an in-memory `MeterProvider`
|
||
- **THEN** `storage_http.reads_total{outcome="ok"}` increments by 1 AND `storage_http.op_duration_ms{op="get",outcome="ok",status_class="2xx"}` records exactly one observation
|
||
|
||
#### Scenario: failed GET records an error span and increments errors_total
|
||
|
||
- **WHEN** a `get` resolves to `500` against an in-memory `SpanExporter` and `MeterProvider`
|
||
- **THEN** the exported span has status `error` AND `storage_http.errors_total{outcome="upstream_error"}` increments by 1
|
||
|
||
**Verified by:** `src/otel/otel.test.ts::"successful GET emits read counter and duration histogram"` AND `src/otel/otel.test.ts::"failed GET records error span and increments errors_total"`
|
||
|
||
---
|
||
|
||
### Requirement: Tier purity — universal, fetch-only
|
||
|
||
The package SHALL NOT import from `node:*`, `bun:*`, or any DOM-only API (e.g. `window`, `document`). Every runtime dependency SHALL be available in the universal subset: `fetch`, `URL`, `URLSearchParams`, `AbortController`, `Uint8Array`, `ReadableStream`. The package SHALL run unmodified under Node 18+, Bun, Deno, modern browsers, and Cloudflare Workers.
|
||
|
||
#### Scenario: tier-purity grep finds no Node or DOM imports
|
||
|
||
- **WHEN** `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-http/src/` is run
|
||
- **THEN** the output is empty
|
||
|
||
**Verified by:** `src/http-storage/http-storage.spec-conformance.test.ts::"src tree contains no node: or bun: imports"`
|