suaveplan/packages/persistence/storage/storage-http/README.md

@suaveplan/storage-http - Universal HTTP/CDN AbstractStorage adapter for @suaveplan/storage: GET/HEAD/Range reads against any baseUrl + key origin, ETag cache, AbortSignal everywhere, strict key sanitisation, native Web Streams, transient-status retry with Retry-After, optional writes via pluggable HttpUploader, full OTel surface.

Universal HTTP/CDN-backed AbstractStorage adapter for @suaveplan/storage. Reads via GET/HEAD/Range against any <baseUrl><key> origin, with optional writes through a pluggable HttpUploader. Threads AbortSignal everywhere, retries transient outcomes honouring Retry-After, supports conditional reads via an ETag cache, and ships native ReadableStream streaming — all in a single tier-pure package that runs unmodified under Node 18+, Bun, Deno, browsers, and Cloudflare Workers.

Documentation tree

• constants — Centralises the tunable defaults used across @suaveplan/storage-http: key-length cap, retry attempts/backoff, ETag-ca…
  • constants — Centralises the tunable defaults used across @suaveplan/storage-http: key-length cap, retry attempts/backoff, ETag-ca…
• error-codes — Declares the canonical STORAGE_HTTP error namespace used by every module in this package. The codes are produced via…
  • error-codes — Declares the canonical STORAGE_HTTP error namespace used by every module in this package. The codes are produced via…
• etag-cache — In-memory LRU keyed by sanitised storage key, valued by { etag, body, lastModified }. Backs HttpStorage's condition…
  • etag-cache — In-memory LRU keyed by sanitised storage key, valued by { etag, body, lastModified }. Backs HttpStorage's condition…
• fetch-adapter — Wraps globalThis.fetch with the contract HttpStorage actually consumes. Translates raw network rejections (DNS fail…
  • fetch-adapter — Wraps globalThis.fetch with the contract HttpStorage actually consumes. Translates raw network rejections (DNS fail…
• http-storage — HttpStorage is the universal AbstractStorage adapter that maps every storage operation onto an HTTP request against…
  • http-storage.helpers — Internal helpers used by HttpStorage. Splits boundary concerns (option parsing, URL composition, header resolution, a…
  • http-storage — HttpStorage is the universal AbstractStorage adapter that maps every storage operation onto an HTTP request against…
  • http-storage.ops — Reusable read-side operations used by HttpStorage: full-body GET with ETag-cache support, HEAD for metadata, `Ran…
  • http-storage.streams — Builds the ReadableStream and WritableStream instances HttpStorage returns from createReadStream / `createWrite…
• key-sanitizer — Validates and normalises caller-supplied storage keys before they reach any fetch call, and provides the default per-…
  • key-sanitizer — Validates and normalises caller-supplied storage keys before they reach any fetch call, and provides the default per-…
• otel — Owns the OpenTelemetry instrument set for @suaveplan/storage-http: counters (reads_total, writes_total, `deletes_…
  • otel — Owns the OpenTelemetry instrument set for @suaveplan/storage-http: counters (reads_total, writes_total, `deletes_…
• range — Encodes HTTP Range semantics for HttpStorage.getPartial. Builds the request header from (offset, length) and inte…
  • range — Encodes HTTP Range semantics for HttpStorage.getPartial. Builds the request header from (offset, length) and inte…
• registry — Module-augments @suaveplan/storage/factory's StorageProviderRegistry and StorageProviderConfig to add the http…
  • registry — Module-augments @suaveplan/storage/factory's StorageProviderRegistry and StorageProviderConfig to add the http…
• retry-helper — Coordinates retry logic for HTTP requests issued by HttpStorage. Distinguishes transient outcomes (HTTP 408/425/`…
  • retry-helper — Coordinates retry logic for HTTP requests issued by HttpStorage. Distinguishes transient outcomes (HTTP 408/425/`…
• uploader — Defines the pluggable write strategy for HttpStorage and ships the only generic uploader implementation: `presignedPu…
  • uploader — Defines the pluggable write strategy for HttpStorage and ships the only generic uploader implementation: `presignedPu…

Install

bun add @suaveplan/storage-http

Quick start

import { HttpStorage } from "@suaveplan/storage-http";

const storage = new HttpStorage({
    baseUrl: "https://cdn.example.com/start/here/",
});

const bytes = await storage.get("a/b/test.doc");

Writes require an injected HttpUploader:

import { HttpStorage, presignedPutUploader } from "@suaveplan/storage-http";

const storage = new HttpStorage({
    baseUrl: "https://cdn.example.com/start/here/",
    uploader: presignedPutUploader({
        signer: async (key) => ({ url: await signMyProvider("PUT", key, 60) }),
    }),
});

await storage.set("a/b/test.doc", new Uint8Array([1, 2, 3]));

list, clear, and getSize always throw STORAGE_HTTP.READ_ONLY because HTTP origins do not expose a uniform directory protocol.

Composition recipe — disk-backed ETag cache

For cross-process cache survival, layer @suaveplan/storage-file's NamespacedStorage underneath:

import { HttpStorage } from "@suaveplan/storage-http";
import { NodeFsStorage } from "@suaveplan/storage-file";

const local = new NodeFsStorage({ rootDir: "/var/cache/cdn-mirror" });
const remote = new HttpStorage({
    baseUrl: "https://cdn.example.com/start/here/",
    etagCache: { capacity: 1024 },
});

async function readCached(key: string): Promise<Uint8Array | undefined> {
    const cached = await local.get(key);
    if (cached !== undefined) return cached;
    const remoteBody = await remote.get(key);
    if (remoteBody !== undefined) await local.set(key, remoteBody);
    return remoteBody;
}

Modules

• error-codes — STORAGE_HTTP_CODES registry + StorageHttpError.
• constants — tunable defaults.
• key-sanitizer — strict key validation + per-segment encoder.
• fetch-adapter — defaultFetch() wrapping globalThis.fetch.
• range — Range header + 206/200/416 disambiguation.
• etag-cache — in-memory LRU for conditional GETs.
• uploader — HttpUploader contract + presignedPutUploader helper.
• retry-helper — transient classifier + Retry-After honour.
• otel — meter, tracer, counters, histograms.
• http-storage — the HttpStorage class itself.
• registry — module-augmentation slot for createStorage({ type: "http", ... }).

Features

• Universal purity — no node:* or DOM-only imports. Runs unmodified everywhere Node 18+ runs.
• Full AbstractStorage surface — get, set, delete, exists, getMetadata, getWithMetadata, getItemSize, getPartial, createReadStream, createWriteStream, setPartial. list/clear/getSize throw READ_ONLY.
• Native streaming — createReadStream returns Response.body directly (no client-side buffering).
• Range reads with fallback metric — 200-when-Range-was-asked is sliced client-side and counted under storage_http.range_fallback_total.
• ETag-cached conditional reads — If-None-Match on hit, 304 returns cached body.
• AbortSignal everywhere — pre-aborted signal short-circuits before fetch; mid-flight abort propagates to fetch.init.signal.
• Retry on transient outcomes — 408/425/429/5xx and network rejection retried with exponential backoff or Retry-After.
• Pluggable writes — HttpUploader slot keeps the core read-only-by-default while permitting any write strategy.
• OTel built-in — duration / IO bytes histograms with status_class attribute; counters per outcome; spans per call.
• 100 / 100 / 100 / 100 Istanbul coverage — see bunx turbo test:coverage --filter=@suaveplan/storage-http.

Capabilities matrix

{
    partialReads: true,
    partialWrites: false,
    nativeStreams: true,
    atomicWrites: false,
    contentAddressable: false,
}

License

MIT — see LICENSE at the repo root.