Initial Commit
This commit is contained in:
135
packages/persistence/storage/storage-http/README.md
Normal file
135
packages/persistence/storage/storage-http/README.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# @suaveplan/storage-http - Universal HTTP/CDN AbstractStorage adapter for @suaveplan/storage: GET/HEAD/Range reads against any baseUrl + key origin, ETag cache, AbortSignal everywhere, strict key sanitisation, native Web Streams, transient-status retry with Retry-After, optional writes via pluggable HttpUploader, full OTel surface.
|
||||
|
||||
Universal HTTP/CDN-backed `AbstractStorage` adapter for `@suaveplan/storage`. Reads via `GET`/`HEAD`/`Range` against any `<baseUrl><key>` origin, with optional writes through a pluggable `HttpUploader`. Threads `AbortSignal` everywhere, retries transient outcomes honouring `Retry-After`, supports conditional reads via an ETag cache, and ships native `ReadableStream` streaming — all in a single tier-pure package that runs unmodified under Node 18+, Bun, Deno, browsers, and Cloudflare Workers.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [constants](./src/constants/constants.md) — Centralises the tunable defaults used across `@suaveplan/storage-http`: key-length cap, retry attempts/backoff, ETag-ca…
|
||||
- [constants](./src/constants/constants.md) — Centralises the tunable defaults used across `@suaveplan/storage-http`: key-length cap, retry attempts/backoff, ETag-ca…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Declares the canonical `STORAGE_HTTP` error namespace used by every module in this package. The codes are produced via…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Declares the canonical `STORAGE_HTTP` error namespace used by every module in this package. The codes are produced via…
|
||||
- [etag-cache](./src/etag-cache/etag-cache.md) — In-memory LRU keyed by sanitised storage key, valued by `{ etag, body, lastModified }`. Backs `HttpStorage`'s condition…
|
||||
- [etag-cache](./src/etag-cache/etag-cache.md) — In-memory LRU keyed by sanitised storage key, valued by `{ etag, body, lastModified }`. Backs `HttpStorage`'s condition…
|
||||
- [fetch-adapter](./src/fetch-adapter/fetch-adapter.md) — Wraps `globalThis.fetch` with the contract `HttpStorage` actually consumes. Translates raw network rejections (DNS fail…
|
||||
- [fetch-adapter](./src/fetch-adapter/fetch-adapter.md) — Wraps `globalThis.fetch` with the contract `HttpStorage` actually consumes. Translates raw network rejections (DNS fail…
|
||||
- [http-storage](./src/http-storage/http-storage.md) — `HttpStorage` is the universal `AbstractStorage` adapter that maps every storage operation onto an HTTP request against…
|
||||
- [http-storage.helpers](./src/http-storage/http-storage.helpers.md) — Internal helpers used by `HttpStorage`. Splits boundary concerns (option parsing, URL composition, header resolution, a…
|
||||
- [http-storage](./src/http-storage/http-storage.md) — `HttpStorage` is the universal `AbstractStorage` adapter that maps every storage operation onto an HTTP request against…
|
||||
- [http-storage.ops](./src/http-storage/http-storage.ops.md) — Reusable read-side operations used by `HttpStorage`: full-body `GET` with ETag-cache support, `HEAD` for metadata, `Ran…
|
||||
- [http-storage.streams](./src/http-storage/http-storage.streams.md) — Builds the `ReadableStream` and `WritableStream` instances `HttpStorage` returns from `createReadStream` / `createWrite…
|
||||
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — Validates and normalises caller-supplied storage keys before they reach any `fetch` call, and provides the default per-…
|
||||
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — Validates and normalises caller-supplied storage keys before they reach any `fetch` call, and provides the default per-…
|
||||
- [otel](./src/otel/otel.md) — Owns the OpenTelemetry instrument set for `@suaveplan/storage-http`: counters (`reads_total`, `writes_total`, `deletes_…
|
||||
- [otel](./src/otel/otel.md) — Owns the OpenTelemetry instrument set for `@suaveplan/storage-http`: counters (`reads_total`, `writes_total`, `deletes_…
|
||||
- [range](./src/range/range.md) — Encodes HTTP `Range` semantics for `HttpStorage.getPartial`. Builds the request header from `(offset, length)` and inte…
|
||||
- [range](./src/range/range.md) — Encodes HTTP `Range` semantics for `HttpStorage.getPartial`. Builds the request header from `(offset, length)` and inte…
|
||||
- [registry](./src/registry/registry.md) — Module-augments `@suaveplan/storage/factory`'s `StorageProviderRegistry` and `StorageProviderConfig` to add the `http`…
|
||||
- [registry](./src/registry/registry.md) — Module-augments `@suaveplan/storage/factory`'s `StorageProviderRegistry` and `StorageProviderConfig` to add the `http`…
|
||||
- [retry-helper](./src/retry-helper/retry-helper.md) — Coordinates retry logic for HTTP requests issued by `HttpStorage`. Distinguishes transient outcomes (HTTP `408`/`425`/`…
|
||||
- [retry-helper](./src/retry-helper/retry-helper.md) — Coordinates retry logic for HTTP requests issued by `HttpStorage`. Distinguishes transient outcomes (HTTP `408`/`425`/`…
|
||||
- [uploader](./src/uploader/uploader.md) — Defines the pluggable write strategy for `HttpStorage` and ships the only generic uploader implementation: `presignedPu…
|
||||
- [uploader](./src/uploader/uploader.md) — Defines the pluggable write strategy for `HttpStorage` and ships the only generic uploader implementation: `presignedPu…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/storage-http
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { HttpStorage } from "@suaveplan/storage-http";
|
||||
|
||||
const storage = new HttpStorage({
|
||||
baseUrl: "https://cdn.example.com/start/here/",
|
||||
});
|
||||
|
||||
const bytes = await storage.get("a/b/test.doc");
|
||||
```
|
||||
|
||||
Writes require an injected `HttpUploader`:
|
||||
|
||||
```ts
|
||||
import { HttpStorage, presignedPutUploader } from "@suaveplan/storage-http";
|
||||
|
||||
const storage = new HttpStorage({
|
||||
baseUrl: "https://cdn.example.com/start/here/",
|
||||
uploader: presignedPutUploader({
|
||||
signer: async (key) => ({ url: await signMyProvider("PUT", key, 60) }),
|
||||
}),
|
||||
});
|
||||
|
||||
await storage.set("a/b/test.doc", new Uint8Array([1, 2, 3]));
|
||||
```
|
||||
|
||||
`list`, `clear`, and `getSize` always throw `STORAGE_HTTP.READ_ONLY` because HTTP origins do not expose a uniform directory protocol.
|
||||
|
||||
## Composition recipe — disk-backed ETag cache
|
||||
|
||||
For cross-process cache survival, layer `@suaveplan/storage-file`'s `NamespacedStorage` underneath:
|
||||
|
||||
```ts
|
||||
import { HttpStorage } from "@suaveplan/storage-http";
|
||||
import { NodeFsStorage } from "@suaveplan/storage-file";
|
||||
|
||||
const local = new NodeFsStorage({ rootDir: "/var/cache/cdn-mirror" });
|
||||
const remote = new HttpStorage({
|
||||
baseUrl: "https://cdn.example.com/start/here/",
|
||||
etagCache: { capacity: 1024 },
|
||||
});
|
||||
|
||||
async function readCached(key: string): Promise<Uint8Array | undefined> {
|
||||
const cached = await local.get(key);
|
||||
if (cached !== undefined) return cached;
|
||||
const remoteBody = await remote.get(key);
|
||||
if (remoteBody !== undefined) await local.set(key, remoteBody);
|
||||
return remoteBody;
|
||||
}
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
- [error-codes](./src/error-codes.md) — `STORAGE_HTTP_CODES` registry + `StorageHttpError`.
|
||||
- [constants](./src/constants/constants.md) — tunable defaults.
|
||||
- [key-sanitizer](./src/key-sanitizer/key-sanitizer.md) — strict key validation + per-segment encoder.
|
||||
- [fetch-adapter](./src/fetch-adapter/fetch-adapter.md) — `defaultFetch()` wrapping `globalThis.fetch`.
|
||||
- [range](./src/range/range.md) — `Range` header + 206/200/416 disambiguation.
|
||||
- [etag-cache](./src/etag-cache/etag-cache.md) — in-memory LRU for conditional GETs.
|
||||
- [uploader](./src/uploader/uploader.md) — `HttpUploader` contract + `presignedPutUploader` helper.
|
||||
- [retry-helper](./src/retry-helper/retry-helper.md) — transient classifier + `Retry-After` honour.
|
||||
- [otel](./src/otel/otel.md) — meter, tracer, counters, histograms.
|
||||
- [http-storage](./src/http-storage/http-storage.md) — the `HttpStorage` class itself.
|
||||
- [registry](./src/registry/registry.md) — module-augmentation slot for `createStorage({ type: "http", ... })`.
|
||||
|
||||
## Features
|
||||
|
||||
- **Universal purity** — no `node:*` or DOM-only imports. Runs unmodified everywhere Node 18+ runs.
|
||||
- **Full `AbstractStorage` surface** — `get`, `set`, `delete`, `exists`, `getMetadata`, `getWithMetadata`, `getItemSize`, `getPartial`, `createReadStream`, `createWriteStream`, `setPartial`. `list`/`clear`/`getSize` throw `READ_ONLY`.
|
||||
- **Native streaming** — `createReadStream` returns `Response.body` directly (no client-side buffering).
|
||||
- **Range reads with fallback metric** — 200-when-Range-was-asked is sliced client-side and counted under `storage_http.range_fallback_total`.
|
||||
- **ETag-cached conditional reads** — `If-None-Match` on hit, `304` returns cached body.
|
||||
- **AbortSignal everywhere** — pre-aborted signal short-circuits before fetch; mid-flight abort propagates to `fetch.init.signal`.
|
||||
- **Retry on transient outcomes** — `408`/`425`/`429`/`5xx` and network rejection retried with exponential backoff or `Retry-After`.
|
||||
- **Pluggable writes** — `HttpUploader` slot keeps the core read-only-by-default while permitting any write strategy.
|
||||
- **OTel built-in** — duration / IO bytes histograms with `status_class` attribute; counters per outcome; spans per call.
|
||||
- **100 / 100 / 100 / 100 Istanbul coverage** — see `bunx turbo test:coverage --filter=@suaveplan/storage-http`.
|
||||
|
||||
## Capabilities matrix
|
||||
|
||||
```ts
|
||||
{
|
||||
partialReads: true,
|
||||
partialWrites: false,
|
||||
nativeStreams: true,
|
||||
atomicWrites: false,
|
||||
contentAddressable: false,
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT — see `LICENSE` at the repo root.
|
||||
|
||||
Reference in New Issue
Block a user