86 lines
11 KiB
Markdown
86 lines
11 KiB
Markdown
# Add @suaveplan/storage-http — URL/CDN-backed AbstractStorage adapter
|
||
|
||
**Package**: `@suaveplan/storage-http`
|
||
**Status**: PROPOSED
|
||
**Type**: new-package (genesis-original)
|
||
|
||
- **Design**: ./design.md
|
||
- **Spec delta**: ./specs/storage-http/spec.md
|
||
|
||
## Why
|
||
|
||
Genesis ships filesystem (`storage-file`), S3 (`storage-s3`), browser (`storage-browser-{opfs,indexeddb,localstorage}`), and in-memory (`@suaveplan/storage`) adapters, but no adapter for the most common read-side surface on the public internet: **a base URL plus a key**. Consumers that need to read static assets served from a CDN (`https://cdn.example.com/start/here/` + `a/b/test.doc`) currently have to bypass the `AbstractStorage` contract and hand-roll `fetch` — losing OTel instrumentation, AbortSignal threading, structured error mapping, capability discovery, and the module-augmentation registry. The drive plugin (`drive-template` hydration), i18n HTTP backend, and any browser-tier consumer that pulls signed-URL assets all benefit from a real `AbstractStorage` over HTTPS.
|
||
|
||
The adapter is **read-first** with a pluggable write strategy. Plain CDNs (Cloudflare, Fastly, CloudFront read-paths, static origins) are read-only and the adapter SHALL declare that via `capabilities` and throw `STORAGE_HTTP.READ_ONLY` on writes. CDNs backed by an upload origin (S3 presigned PUT, R2 signed URLs, custom upload endpoints) plug in via an injectable `HttpUploader` interface; the adapter ships **no** baked-in uploader to avoid coupling to any specific provider.
|
||
|
||
## What changes
|
||
|
||
- ADDED: `HttpStorage` class extending `AbstractStorage` — full key-value read surface over `fetch`, with optional write via injected `HttpUploader`.
|
||
- ADDED: `createHttpStorage(options)` factory.
|
||
- ADDED: `HttpStorageOptions` interface + `HttpStorageOptionsSchema` Zod schema. Required `baseUrl` (must end with `/`); optional `headers` (static or per-request callback), `fetch` (injectable `FetchLike` for testing), `timeout`, `retry` (delegates to `@suaveplan/retry`), `uploader` (`HttpUploader` for writes), `keyEncoder` (default: `encodeURIComponent` per path segment, preserving `/`), `etagCache` (in-memory weak cache for conditional requests).
|
||
- ADDED: `FetchLike` structural interface — minimal `(input, init?) => Promise<Response>` shape; `defaultFetch()` adapter wrapping `globalThis.fetch` with `AbortSignal` plumbing.
|
||
- ADDED: `HttpUploader` injectable interface — `{ put(key, body, headers?, signal?), delete?(key, signal?) }`. Adapter MAY ship a thin `presignedPutUploader(presigner)` helper but no provider-specific uploaders.
|
||
- ADDED: `sanitizeKey(key)` strict key boundary — rejects empty, `..` segments, absolute paths (`/`-prefixed), null bytes, control chars (`0x00`–`0x1F`, `0x7F`), keys over 2048 chars (`KEY_TOO_LONG`); normalises Windows `\` to POSIX `/`; URL-encodes each path segment via the `keyEncoder` while preserving `/`.
|
||
- ADDED: `STORAGE_HTTP_CODES` registry via `defineCodes("STORAGE_HTTP", [...])` — 10 codes: `NOT_FOUND`, `UNAUTHORIZED`, `FORBIDDEN`, `NETWORK_ERROR`, `INVALID_KEY`, `KEY_TOO_LONG`, `INVALID_OPTIONS`, `READ_ONLY`, `ABORTED`, `UPSTREAM_ERROR`.
|
||
- ADDED: `StorageHttpError extends SuaveplanError<StorageHttpErrorCode>` with `key`, `httpStatus`, and `url` discriminants for diagnostics.
|
||
- ADDED: Module-augmentation registry — `declare module "@suaveplan/storage"` adds `http: HttpStorage` to `StorageProviderRegistry` and `http: HttpStorageOptions` to `StorageProviderConfig`; `registerHttpProvider()` registers the constructor with the factory.
|
||
- ADDED: Range-read support via HTTP `Range` header — `getPartial(offset, length)` issues `Range: bytes=<offset>-<offset+length-1>`; declared `capabilities.partialReads: true`. Servers that reply `200` instead of `206` are slow-pathed (full body sliced on the client) and counted under `storage_http.range_fallback_total`.
|
||
- ADDED: `exists(key)` via `HEAD` request; `getMetadata(key)` reads `Content-Length`, `Content-Type`, `Last-Modified`, and `ETag` from a `HEAD` response (or piggy-backed on the last `GET` via `etagCache`).
|
||
- ADDED: Conditional reads — when `etagCache` is enabled, `get` sends `If-None-Match` and treats `304 Not Modified` as a cache hit, returning the cached body without retransmission.
|
||
- ADDED: Native Web Streams via `createReadStream(key, signal)` returning the `Response.body` directly when present (declared `capabilities.nativeStreams: true`). `createWriteStream` is unavailable on read-only configurations and throws `STORAGE_HTTP.READ_ONLY`; when an `HttpUploader` is injected, the write stream collects to memory and calls `uploader.put` on `close` (the underlying surface is request-bodied, not chunked).
|
||
- ADDED: `@suaveplan/retry` integration on transient status codes (`408`, `425`, `429`, `5xx`) and network failures, with `Retry-After` honoured when present and exponential backoff otherwise. `4xx` other than the transient set SHALL NOT retry.
|
||
- ADDED: OTel surface — `storage_http.op_duration_ms{op,outcome,status_class}` and `storage_http.io_bytes{op,direction}` histograms; `storage_http.{reads,writes,deletes,errors,range_fallback}_total{outcome}` counters; per-op span on `@suaveplan/storage-http` tracer with `http.url`, `http.method`, `http.status_code`, `http.response_content_length`, `storage.key` attributes.
|
||
|
||
## Out of scope
|
||
|
||
- **Provider-specific uploaders.** S3 presigned PUTs, R2 signed URLs, Azure Blob SAS, GCS signed URLs — each ships in its own package (e.g. `storage-http-uploader-s3`) or is composed by the consumer via the `HttpUploader` slot. Reason: keeps the core adapter zero-coupling and dependency-light.
|
||
- **Listing (`list(prefix)`).** HTTP origins do not have a uniform directory protocol. `list` SHALL throw `STORAGE_HTTP.READ_ONLY` (capability `listing: false` declared on the adapter). Consumers that need listing should use `storage-s3` directly. Reason: any sensible implementation requires provider-specific knowledge (S3 `ListObjectsV2`, WebDAV `PROPFIND`, etc.); shipping an opinionated default would mislead callers.
|
||
- **Persistent disk cache.** The optional `etagCache` is an in-memory LRU; cross-process caches belong to the consumer (a `NamespacedStorage` over `storage-file` is the recommended composition for that). Reason: cache invalidation policy is application-specific.
|
||
- **Authentication state machine.** The `headers` callback runs on every request, so callers can plug in token rotation, but the adapter does not own the auth lifecycle. OAuth refresh, mTLS, AWS SigV4 — out of scope. Reason: orthogonal concern; standard composition pattern.
|
||
- **`setPartial` random-access writes.** HTTP does not expose `pwrite` semantics uniformly (S3 `multipart-upload`, R2 `multipart`, GCS `resumable` are all incompatible). `setPartial` SHALL fall through to `AbstractStorage.defaultSetPartial` only when an `HttpUploader` is wired AND the storage is read-write; otherwise it throws `STORAGE_HTTP.READ_ONLY`. `capabilities.partialWrites: false`.
|
||
- **WebDAV.** A genuine `PROPFIND`/`COPY`/`MOVE` adapter is a separate package.
|
||
|
||
## Source mapping
|
||
|
||
This is a **genesis-original** package; there is no upstream in `../suaveplan/`. All requirements derive from the genesis `AbstractStorage` contract (`openspec/specs/storage/spec.md`) and the `storage-file`/`storage-s3` adapter precedents. The package contract follows the same shape as `storage-file` (atomic writes are inapplicable; the analogue is "request body is committed in one call").
|
||
|
||
### Requirements with no source line
|
||
|
||
| Requirement | Justification |
|
||
|---|---|
|
||
| Extend `AbstractStorage` | Every storage adapter in genesis extends `AbstractStorage` for `StorageProviderRegistry` compatibility. |
|
||
| `AbortSignal` on every public async method | Required by `AbstractStorage` base signatures; threaded straight into `fetch`'s `init.signal`. |
|
||
| `baseUrl` must end with `/` | Avoids the classic `URL` resolution footgun where `new URL("a", "https://x.com/b")` yields `https://x.com/a` instead of `https://x.com/b/a`. Documented; validated by Zod schema. |
|
||
| `keyEncoder` injectable | Some CDNs require unencoded `+` or specific casing for signed-URL signatures; per-segment encoding is the sane default but must be overridable. |
|
||
| `HttpUploader` slot rather than baked-in writes | HTTP write APIs are not standardised; coupling the adapter to one provider would force every consumer to depend on that provider. The slot keeps the core read-only-by-default while permitting any write strategy. |
|
||
| `READ_ONLY` capability + error code | Lets consumers test write capability without trying-and-catching, and surfaces the failure with a stable error code when they don't. |
|
||
| Range fallback metric | Quietly accepting `200` for a `Range` request is a correctness hazard for large objects; emitting a counter lets operators alert on misconfigured origins. |
|
||
| `etagCache` is in-memory only | Cross-process caching is the consumer's job (compose with `storage-file`). |
|
||
| `STORAGE_HTTP.UPSTREAM_ERROR` distinct from `NETWORK_ERROR` | `5xx` is a real server response; `NETWORK_ERROR` is `fetch` reject (DNS, TLS, abort-without-signal). Distinguishing them helps retry classifiers and operators. |
|
||
|
||
## Verification approach
|
||
|
||
Every requirement in the spec delta carries a `**Verified by:**` line referencing concrete `it(...)` titles in these files:
|
||
|
||
- `src/http-storage/http-storage.test.ts` — CRUD via injected `FetchLike`, range reads, conditional reads with ETag cache, AbortSignal threading, retry on 429/503 + `Retry-After`, `READ_ONLY` enforcement when no uploader, write path with injected uploader, OTel happy paths.
|
||
- `src/http-storage/http-storage.spec-conformance.test.ts` — type-level conformance against `AbstractStorage`, capabilities matrix, OTel instruments observed via real in-memory `MeterProvider`, idempotent registry registration, error-code registry exhaustion.
|
||
- `src/key-sanitizer/key-sanitizer.test.ts` — empty, traversal, absolute, null bytes, control chars, overlong (>2048), separator normalisation, default per-segment encoding (preserves `/`, encodes `?`, `#`, space).
|
||
- `src/fetch-adapter/fetch-adapter.test.ts` — `defaultFetch()` forwards to `globalThis.fetch`, propagates `AbortSignal`, maps network rejection to `STORAGE_HTTP.NETWORK_ERROR`.
|
||
- `src/uploader/uploader.test.ts` — `presignedPutUploader` helper round-trips a presigned URL signer; injected uploader is invoked exactly once on `set`.
|
||
- `src/range/range.test.ts` — `Range` header format, `206` happy path, `200` fallback path increments `range_fallback_total`, `416 Range Not Satisfiable` maps to `NOT_FOUND` when length unknown.
|
||
- `src/etag-cache/etag-cache.test.ts` — LRU eviction, `304 Not Modified` returns cached body without retransmission, cache miss falls through.
|
||
- `src/otel/otel.test.ts` — `recordOp` emits histogram + counter with `status_class` attribute against a real `InMemoryMetricExporter`.
|
||
- `src/registry/registry.test.ts` — registers `http` provider, `createStorage({ type: "http", config })` returns `HttpStorage`, idempotent re-registration.
|
||
|
||
## Impact
|
||
|
||
### Affected dependents
|
||
|
||
Run `bunx turbo ls --filter=...@suaveplan/storage-http` to enumerate. The package is new — no dependents currently. No action needed.
|
||
|
||
- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-http` returning the package itself).
|
||
|
||
### Affected upstreams
|
||
|
||
- `@suaveplan/storage` — `StorageProviderRegistry` module-augmented to add `http: HttpStorage` slot. Additive only; no breaking change. [x] no action needed (purely additive type slot).
|