Add @suaveplan/storage-http — URL/CDN-backed AbstractStorage adapter

Package: @suaveplan/storage-http Status: PROPOSED Type: new-package (genesis-original)

Design: ./design.md
Spec delta: ./specs/storage-http/spec.md

Why

Genesis ships filesystem (storage-file), S3 (storage-s3), browser (storage-browser-{opfs,indexeddb,localstorage}), and in-memory (@suaveplan/storage) adapters, but no adapter for the most common read-side surface on the public internet: a base URL plus a key. Consumers that need to read static assets served from a CDN (https://cdn.example.com/start/here/ + a/b/test.doc) currently have to bypass the AbstractStorage contract and hand-roll fetch — losing OTel instrumentation, AbortSignal threading, structured error mapping, capability discovery, and the module-augmentation registry. The drive plugin (drive-template hydration), i18n HTTP backend, and any browser-tier consumer that pulls signed-URL assets all benefit from a real AbstractStorage over HTTPS.

The adapter is read-first with a pluggable write strategy. Plain CDNs (Cloudflare, Fastly, CloudFront read-paths, static origins) are read-only and the adapter SHALL declare that via capabilities and throw STORAGE_HTTP.READ_ONLY on writes. CDNs backed by an upload origin (S3 presigned PUT, R2 signed URLs, custom upload endpoints) plug in via an injectable HttpUploader interface; the adapter ships no baked-in uploader to avoid coupling to any specific provider.

What changes

ADDED: HttpStorage class extending AbstractStorage — full key-value read surface over fetch, with optional write via injected HttpUploader.
ADDED: createHttpStorage(options) factory.
ADDED: HttpStorageOptions interface + HttpStorageOptionsSchema Zod schema. Required baseUrl (must end with /); optional headers (static or per-request callback), fetch (injectable FetchLike for testing), timeout, retry (delegates to @suaveplan/retry), uploader (HttpUploader for writes), keyEncoder (default: encodeURIComponent per path segment, preserving /), etagCache (in-memory weak cache for conditional requests).
ADDED: FetchLike structural interface — minimal (input, init?) => Promise<Response> shape; defaultFetch() adapter wrapping globalThis.fetch with AbortSignal plumbing.
ADDED: HttpUploader injectable interface — { put(key, body, headers?, signal?), delete?(key, signal?) }. Adapter MAY ship a thin presignedPutUploader(presigner) helper but no provider-specific uploaders.
ADDED: sanitizeKey(key) strict key boundary — rejects empty, .. segments, absolute paths (/-prefixed), null bytes, control chars (0x00–0x1F, 0x7F), keys over 2048 chars (KEY_TOO_LONG); normalises Windows \ to POSIX /; URL-encodes each path segment via the keyEncoder while preserving /.
ADDED: STORAGE_HTTP_CODES registry via defineCodes("STORAGE_HTTP", [...]) — 10 codes: NOT_FOUND, UNAUTHORIZED, FORBIDDEN, NETWORK_ERROR, INVALID_KEY, KEY_TOO_LONG, INVALID_OPTIONS, READ_ONLY, ABORTED, UPSTREAM_ERROR.
ADDED: StorageHttpError extends SuaveplanError<StorageHttpErrorCode> with key, httpStatus, and url discriminants for diagnostics.
ADDED: Module-augmentation registry — declare module "@suaveplan/storage" adds http: HttpStorage to StorageProviderRegistry and http: HttpStorageOptions to StorageProviderConfig; registerHttpProvider() registers the constructor with the factory.
ADDED: Range-read support via HTTP Range header — getPartial(offset, length) issues Range: bytes=<offset>-<offset+length-1>; declared capabilities.partialReads: true. Servers that reply 200 instead of 206 are slow-pathed (full body sliced on the client) and counted under storage_http.range_fallback_total.
ADDED: exists(key) via HEAD request; getMetadata(key) reads Content-Length, Content-Type, Last-Modified, and ETag from a HEAD response (or piggy-backed on the last GET via etagCache).
ADDED: Conditional reads — when etagCache is enabled, get sends If-None-Match and treats 304 Not Modified as a cache hit, returning the cached body without retransmission.
ADDED: Native Web Streams via createReadStream(key, signal) returning the Response.body directly when present (declared capabilities.nativeStreams: true). createWriteStream is unavailable on read-only configurations and throws STORAGE_HTTP.READ_ONLY; when an HttpUploader is injected, the write stream collects to memory and calls uploader.put on close (the underlying surface is request-bodied, not chunked).
ADDED: @suaveplan/retry integration on transient status codes (408, 425, 429, 5xx) and network failures, with Retry-After honoured when present and exponential backoff otherwise. 4xx other than the transient set SHALL NOT retry.
ADDED: OTel surface — storage_http.op_duration_ms{op,outcome,status_class} and storage_http.io_bytes{op,direction} histograms; storage_http.{reads,writes,deletes,errors,range_fallback}_total{outcome} counters; per-op span on @suaveplan/storage-http tracer with http.url, http.method, http.status_code, http.response_content_length, storage.key attributes.

Out of scope

Provider-specific uploaders. S3 presigned PUTs, R2 signed URLs, Azure Blob SAS, GCS signed URLs — each ships in its own package (e.g. storage-http-uploader-s3) or is composed by the consumer via the HttpUploader slot. Reason: keeps the core adapter zero-coupling and dependency-light.
Listing (list(prefix)). HTTP origins do not have a uniform directory protocol. list SHALL throw STORAGE_HTTP.READ_ONLY (capability listing: false declared on the adapter). Consumers that need listing should use storage-s3 directly. Reason: any sensible implementation requires provider-specific knowledge (S3 ListObjectsV2, WebDAV PROPFIND, etc.); shipping an opinionated default would mislead callers.
Persistent disk cache. The optional etagCache is an in-memory LRU; cross-process caches belong to the consumer (a NamespacedStorage over storage-file is the recommended composition for that). Reason: cache invalidation policy is application-specific.
Authentication state machine. The headers callback runs on every request, so callers can plug in token rotation, but the adapter does not own the auth lifecycle. OAuth refresh, mTLS, AWS SigV4 — out of scope. Reason: orthogonal concern; standard composition pattern.
setPartial random-access writes. HTTP does not expose pwrite semantics uniformly (S3 multipart-upload, R2 multipart, GCS resumable are all incompatible). setPartial SHALL fall through to AbstractStorage.defaultSetPartial only when an HttpUploader is wired AND the storage is read-write; otherwise it throws STORAGE_HTTP.READ_ONLY. capabilities.partialWrites: false.
WebDAV. A genuine PROPFIND/COPY/MOVE adapter is a separate package.

Source mapping

This is a genesis-original package; there is no upstream in ../suaveplan/. All requirements derive from the genesis AbstractStorage contract (openspec/specs/storage/spec.md) and the storage-file/storage-s3 adapter precedents. The package contract follows the same shape as storage-file (atomic writes are inapplicable; the analogue is "request body is committed in one call").

Requirements with no source line

Requirement	Justification
Extend `AbstractStorage`	Every storage adapter in genesis extends `AbstractStorage` for `StorageProviderRegistry` compatibility.
`AbortSignal` on every public async method	Required by `AbstractStorage` base signatures; threaded straight into `fetch`'s `init.signal`.
`baseUrl` must end with `/`	Avoids the classic `URL` resolution footgun where `new URL("a", "https://x.com/b")` yields `https://x.com/a` instead of `https://x.com/b/a`. Documented; validated by Zod schema.
`keyEncoder` injectable	Some CDNs require unencoded `+` or specific casing for signed-URL signatures; per-segment encoding is the sane default but must be overridable.
`HttpUploader` slot rather than baked-in writes	HTTP write APIs are not standardised; coupling the adapter to one provider would force every consumer to depend on that provider. The slot keeps the core read-only-by-default while permitting any write strategy.
`READ_ONLY` capability + error code	Lets consumers test write capability without trying-and-catching, and surfaces the failure with a stable error code when they don't.
Range fallback metric	Quietly accepting `200` for a `Range` request is a correctness hazard for large objects; emitting a counter lets operators alert on misconfigured origins.
`etagCache` is in-memory only	Cross-process caching is the consumer's job (compose with `storage-file`).
`STORAGE_HTTP.UPSTREAM_ERROR` distinct from `NETWORK_ERROR`	`5xx` is a real server response; `NETWORK_ERROR` is `fetch` reject (DNS, TLS, abort-without-signal). Distinguishing them helps retry classifiers and operators.

Verification approach

Every requirement in the spec delta carries a **Verified by:** line referencing concrete it(...) titles in these files:

src/http-storage/http-storage.test.ts — CRUD via injected FetchLike, range reads, conditional reads with ETag cache, AbortSignal threading, retry on 429/503 + Retry-After, READ_ONLY enforcement when no uploader, write path with injected uploader, OTel happy paths.
src/http-storage/http-storage.spec-conformance.test.ts — type-level conformance against AbstractStorage, capabilities matrix, OTel instruments observed via real in-memory MeterProvider, idempotent registry registration, error-code registry exhaustion.
src/key-sanitizer/key-sanitizer.test.ts — empty, traversal, absolute, null bytes, control chars, overlong (>2048), separator normalisation, default per-segment encoding (preserves /, encodes ?, #, space).
src/fetch-adapter/fetch-adapter.test.ts — defaultFetch() forwards to globalThis.fetch, propagates AbortSignal, maps network rejection to STORAGE_HTTP.NETWORK_ERROR.
src/uploader/uploader.test.ts — presignedPutUploader helper round-trips a presigned URL signer; injected uploader is invoked exactly once on set.
src/range/range.test.ts — Range header format, 206 happy path, 200 fallback path increments range_fallback_total, 416 Range Not Satisfiable maps to NOT_FOUND when length unknown.
src/etag-cache/etag-cache.test.ts — LRU eviction, 304 Not Modified returns cached body without retransmission, cache miss falls through.
src/otel/otel.test.ts — recordOp emits histogram + counter with status_class attribute against a real InMemoryMetricExporter.
src/registry/registry.test.ts — registers http provider, createStorage({ type: "http", config }) returns HttpStorage, idempotent re-registration.

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/storage-http to enumerate. The package is new — no dependents currently. No action needed.

No dependents (verified by bunx turbo ls --filter=...@suaveplan/storage-http returning the package itself).

Affected upstreams

@suaveplan/storage — StorageProviderRegistry module-augmented to add http: HttpStorage slot. Additive only; no breaking change. [x] no action needed (purely additive type slot).

11 KiB Raw Blame History Unescape Escape