first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,140 @@
# S3
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements the `AbstractStorage` interface from `@suaveplan/storage` backed by the AWS SDK v3 `@aws-sdk/client-s3`. The `S3` class provides full lifecycle management for objects in any S3-compatible bucket, including AWS S3, Cloudflare R2, MinIO, and Garage. It adds enterprise capabilities not present in the main-branch reference: structured error codes, presigned URL generation, multipart upload with automatic part splitting and abort-on-failure, injectable retry strategy, AbortSignal propagation, and OpenTelemetry span/counter/histogram instrumentation.
The class is intentionally thin: it translates between `AbstractStorage`'s key-value contract and S3's object model, adds a key prefix namespace, and delegates all batching, pattern-listing, and hook machinery to the base class. The `S3ClientFactory` interface enables full testing via in-memory fakes without touching real AWS infrastructure.
## Features
- Implements all `AbstractStorage` abstract methods: `get`, `set`, `delete`, `exists`, `list`, `clear`, `getWithMetadata`, `getMetadata`, `getSize`, `getPartial`, `getItemSize`, `createReadStream`, `createWriteStream`
- S3-specific extensions: `presign` (GET/PUT), `copy`, `move`, `setMultipart`
- Key prefix namespacing: `keyPrefix` option prepends a path segment to all keys
- Retry with exponential backoff for retryable S3 errors (5xx, SlowDown, RequestTimeout)
- AbortSignal propagation to all AWS SDK calls
- OpenTelemetry spans, counters, and histograms for every operation
- Structured `S3StorageError` with stable `S3_CODES` for all failure modes
- Injectable `S3ClientFactory` for testability
- AWS credential format validation via `validateCredentials`
## Usage: Basic Example
```ts
import { S3, createS3 } from "@suaveplan/storage-s3";
const storage = createS3({
bucket: "my-bucket",
region: "us-east-1",
accessKeyId: "AKIAIOSFODNN7EXAMPLE",
secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
});
await storage.set("hello.txt", new TextEncoder().encode("hello"));
const data = await storage.get("hello.txt");
console.log(new TextDecoder().decode(data)); // "hello"
```
## Usage: Advanced Example
```ts
import { S3 } from "@suaveplan/storage-s3";
import type { S3ClientFactory, S3Options } from "@suaveplan/storage-s3";
// Inject a fake client for testing
const fakeClient = new FakeS3Client(); // your in-memory implementation
const fakeFactory: S3ClientFactory = {
create: () => fakeClient as unknown as S3Client,
};
const storage = new S3(
{
bucket: "test-bucket",
region: "us-east-1",
endpoint: "http://localhost:9000",
keyPrefix: "uploads/v2",
retry: { maxAttempts: 3, baseDelayMs: 50 },
},
fakeFactory
);
// Multipart upload
const stream = new ReadableStream<Uint8Array>({
start(ctrl) {
ctrl.enqueue(new Uint8Array(10 * 1024 * 1024)); // 10 MiB
ctrl.close();
},
});
await storage.setMultipart("large-file.bin", stream, { partSize: 5 * 1024 * 1024 });
// Presigned URL
const url = await storage.presign("large-file.bin", { expiresIn: 300, method: "GET" });
console.log(url); // https://... signed URL
```
## API Reference
### `new S3(opts: S3Options, clientFactory?: S3ClientFactory)`
Creates an S3 adapter.
| Param | Type | Description |
|---|---|---|
| `opts` | `S3Options` | Bucket, region, credentials, endpoint, prefix, retry, logger |
| `clientFactory` | `S3ClientFactory?` | Injectable factory; defaults to real AWS SDK client |
**Throws**: `S3StorageError(S3.CREDENTIALS_INVALID)` if AWS credential format is invalid.
### `createS3(opts, clientFactory?): S3`
Factory function equivalent to `new S3(opts, clientFactory)`.
### `get(key, signal?): Promise<Uint8Array | undefined>`
Downloads an object. Returns `undefined` if the key does not exist.
### `set(key, value, options?, signal?): Promise<void>`
Uploads a `Uint8Array` as an object. Throws `S3StorageError(S3.UPLOAD_FAILED)` on failure.
### `delete(key, signal?): Promise<boolean>`
Deletes an object. Returns `false` if it did not exist, `true` if deleted.
### `exists(key, signal?): Promise<boolean>`
Checks existence via `HeadObject`.
### `list(options?, signal?): Promise<StorageListResult>`
Lists objects under an optional prefix. Follows continuation tokens internally.
### `clear(signal?): Promise<void>`
Deletes all objects under the configured key prefix.
### `presign(key, opts?): Promise<string>`
Returns a presigned URL (GET by default). Throws `S3StorageError(S3.PRESIGN_FAILED)`.
### `copy(source, destination, signal?): Promise<void>`
Copies an object within the same bucket. Throws `S3StorageError(S3.UPLOAD_FAILED)`.
### `move(source, destination, signal?): Promise<void>`
Moves an object: copy then delete source.
### `setMultipart(key, stream, opts?): Promise<void>`
Uploads a `ReadableStream` in parts. Automatically aborts the multipart session on error or signal. Throws `S3StorageError(S3.MULTIPART_FAILED)`.
## Implementation Notes
The `withRetry` helper is a lightweight internal implementation rather than using `@suaveplan/retry` directly, to avoid the need for the retry package to be a hard runtime dep on the critical storage path. The `RetryOptions` type is a minimal subset, and consumers can opt in by passing `retry: { maxAttempts: 3 }`. Retryable errors are detected by checking `S3ServiceException.$retryable.throttling`, `name` for `SlowDown`/`RequestTimeout`, and the HTTP status code ≥ 500.
For `setMultipart`, the stream is read in a single pass, buffering chunks until a full `partSize` accumulates. The final part may be smaller than `partSize`. AWS requires each part except the last to be at least 5 MiB, so the default `partSize` matches that minimum. If the multipart session fails after `CreateMultipartUpload`, the session is aborted via `AbortMultipartUploadCommand` to avoid incurring storage costs for incomplete uploads.