suaveplan/openspec/changes/complete-storage-s3/proposal.md

# OpenSpec Change: complete-storage-s3

- **Change ID**: complete-storage-s3
- **Package**: `@suaveplan/storage-s3` (modify — complete to full S3 parity)
- **Status**: PROPOSED
- **Date**: 2026-05-17
- **Design**: ./design.md
- **Spec delta**: ./specs/storage-s3/spec.md

## Why

`@suaveplan/storage-s3` ships a working but thin `AbstractStorage`
implementation: it does `get` / `set` / `delete` / `list` / `head`, a serial
multipart `setMultipart`, presigned URLs limited to `GET` / `PUT`, and one copy
helper. Almost every other surface S3 exposes — and that consumers actually
need to ship production object-store workloads — is absent:

- No auto-promote of `set` to multipart for large payloads; the existing
  `setMultipart` is serial and not wired into the `AbstractStorage` surface.
- No streaming `set` from `ReadableStream<Uint8Array>` without buffering.
- No `getSignedDeleteUrl`; presigning is fixed to `GET` / `PUT` with no TTL
  semantics, no response/request header pinning, no content-type or
  content-disposition control.
- No server-side encryption (SSE-S3, SSE-KMS, SSE-C) on `set` or `get`.
- No object metadata controls — content-type, content-encoding,
  content-disposition, cache-control, user-defined `x-amz-meta-*`.
- No ACLs (canned or grant policies), no object tagging, no versioning
  surface, no bucket lifecycle, no bucket CORS, no bucket-level ops (create
  / delete / exists / region), no conditional ops (`If-Match`,
  `If-None-Match`, `If-Modified-Since`, `If-Unmodified-Since`).
- No range reads via `get`; `getPartial` round-trips the whole object via
  the default fallback.
- No checksum integrity (SHA-256 / CRC32C) per AWS-recommended guard.
- Pluggable HTTP transport (`FetchLike`), pluggable SigV4 signer, and retry
  composition via `@suaveplan/retry` are not yet integrated; retries are
  hand-rolled in `withRetry`.
- Error coverage is 11 codes for a surface that needs ~50.
- No real-service test harness wiring against the existing
  `@suaveplan/testing-integration-garage-s3` fixture for the multi-feature
  paths above.

The supersession marker
`openspec/changes/_superseded-narrow-deferrals-2026-05-17/add-storage-s3-multipart/`
confirms that the prior narrow deferral (multipart only) was rolled back in
favour of a single change that lands the full S3 surface at parity. This
change is that rollup. No cherry-picking, no Phase 2 split.

## What Changes

This change brings `@suaveplan/storage-s3` to **full S3 feature parity** in a
single shipped change. Every capability below is in scope; none are deferred.

### AbstractStorage surface (production-grade)

- `get`, `set`, `setPartial`, `has`/`exists`, `delete`, `list`, `head`/`stat`
  — each becomes production-grade with full options, conditional headers,
  SSE, and checksum integrity passthrough.
- `getPartial` uses native S3 `Range:` reads — no more fetch-the-whole-object
  fallback.
- `setPartial` for `offset === 0` AND `bytes.byteLength > threshold` routes
  through the native multipart engine; non-zero offsets retain
  `defaultSetPartial` r-m-w because S3 has no byte-range PUT.
  `capabilities.partialWrites` stays `false` to honestly advertise that.

### Multipart (auto-promote, bounded concurrency, abort+cleanup)

- `S3.set` auto-promotes to multipart when `value.byteLength` exceeds a
  configurable threshold (default 16 MiB).
- Internal `multipart-engine` owns `CreateMultipartUpload` → bounded-parallel
  `UploadPart` (default concurrency 4) → `CompleteMultipartUpload`, with
  full abort propagation to in-flight workers and guaranteed
  `AbortMultipartUpload` cleanup.
- Per-stage error codes: `MULTIPART_INIT_FAILED`, `MULTIPART_PART_FAILED`,
  `MULTIPART_COMPLETE_FAILED`, `MULTIPART_ABORT_FAILED`. Legacy
  `MULTIPART_FAILED` retained as alias.

### Streaming put

- `S3.setStream(key, stream, opts?)` consumes `ReadableStream<Uint8Array>`
  without buffering the whole stream — chunked into parts on the fly with a
  ceiling of 2 part-sized buffers in memory.

### Presigned URLs

- `getSignedGetUrl`, `getSignedPutUrl`, `getSignedDeleteUrl` — each with
  TTL, custom request headers, response-content-type and
  response-content-disposition overrides, content-type pinning, and SSE
  parameter forwarding.

### Server-side encryption

- SSE-S3 (`AES256`), SSE-KMS (`aws:kms` with optional `SSEKMSKeyId` +
  `BucketKeyEnabled`), SSE-C (caller-supplied key material) on every
  `set`, `setStream`, `setMultipart`, presigned PUT, and `copy`.
- `get` accepts the matching SSE-C key for decryption.

### Object metadata

- `set` accepts `contentType`, `contentEncoding`, `contentDisposition`,
  `contentLanguage`, `cacheControl`, `expires`, and a free-form
  `userMetadata: Record<string, string>` mapped to `x-amz-meta-*`.
- `head` / `getMetadata` returns the full metadata bag including
  `eTag`, `versionId`, `lastModified`, `storageClass`, and the
  caller-set headers above.

### ACLs

- Canned ACLs (`private`, `public-read`, `public-read-write`,
  `aws-exec-read`, `authenticated-read`, `bucket-owner-read`,
  `bucket-owner-full-control`, `log-delivery-write`) per put.
- Explicit grant policies (`grantRead`, `grantReadACP`, `grantWriteACP`,
  `grantFullControl`) per put.
- `getObjectAcl` / `putObjectAcl` for post-write changes.

### Copy

- `copyObject(source, destination, opts?)` — server-side copy within bucket
  AND across buckets via `{ sourceBucket }`. Supports metadata directive
  (`COPY` / `REPLACE`), tagging directive, conditional headers, SSE
  parameter overrides.
- `UploadPartCopy` engine for objects > 5 GiB (multipart server-side copy).

### Tagging

- `getObjectTags`, `putObjectTags`, `deleteObjectTags`.
- `set` accepts inline `tags: Record<string, string>` (URL-encoded into the
  `x-amz-tagging` header — no extra round trip).

### Versioning

- `listObjectVersions` paginator.
- `get(key, { versionId })`, `head(key, { versionId })`,
  `delete(key, { versionId })`.

### Bucket-level operations

- `createBucket`, `deleteBucket`, `bucketExists`, `getBucketRegion`,
  `listObjectsV2` (paginator with `prefix`, `delimiter`, `maxKeys`,
  `continuationToken`, `startAfter`).
- Lifecycle config: `getBucketLifecycle`, `putBucketLifecycle`,
  `deleteBucketLifecycle`.
- CORS config: `getBucketCors`, `putBucketCors`, `deleteBucketCors`.

### Conditional ops

- `IfMatch`, `IfNoneMatch`, `IfModifiedSince`, `IfUnmodifiedSince` on
  `get` / `head` / `delete` / `copy` — translated to the precondition error
  code `PRECONDITION_FAILED` (412) and `NOT_MODIFIED` (304).

### Range reads

- `get(key, { range: { offset, length } })` issues a native `Range:` GET
  rather than a full-object fetch + slice.

### Checksums

- Per-request `SHA256` (default) and `CRC32C` integrity guards per the
  AWS-recommended additional checksum modes. Returned `ChecksumSHA256` /
  `ChecksumCRC32C` are surfaced on the metadata bag and verified on
  `get`-with-checksum.

### Pluggable HTTP transport, signer, retry

- `S3Options.transport?: FetchLike` to swap the underlying HTTP transport
  (browser `fetch`, Node `undici`, custom edge fetch). Default uses the
  AWS SDK's built-in.
- `S3Options.signer?: SigV4Signer` to swap the signer (default uses AWS SDK
  SigV4 implementation).
- Retry is composed through `@suaveplan/retry` (`createRetryStrategy`) —
  the hand-rolled `withRetry` is replaced. Throttling / 5xx classification
  preserved.

### Error codes

`S3_CODES` expands from 11 to ~50 codes covering every failure path:
SSE (`SSE_KEY_MISSING`, `SSE_KEY_MISMATCH`, `SSE_KMS_DENIED`),
ACLs (`ACL_INVALID`, `ACL_PUT_FAILED`),
tagging (`TAG_PUT_FAILED`, `TAG_DELETE_FAILED`, `TAG_LIMIT_EXCEEDED`),
versioning (`VERSION_NOT_FOUND`, `VERSIONING_DISABLED`),
lifecycle (`LIFECYCLE_INVALID`, `LIFECYCLE_PUT_FAILED`),
CORS (`CORS_INVALID`, `CORS_PUT_FAILED`),
bucket ops (`BUCKET_ALREADY_EXISTS`, `BUCKET_NOT_EMPTY`, `BUCKET_DELETE_FAILED`,
`BUCKET_CREATE_FAILED`, `BUCKET_REGION_MISMATCH`),
conditional (`PRECONDITION_FAILED`, `NOT_MODIFIED`),
copy (`COPY_FAILED`, `COPY_PART_FAILED`),
multipart (4 new codes above; `MULTIPART_FAILED` retained),
presign (`PRESIGN_TTL_INVALID`, `PRESIGN_HEADER_INVALID`),
checksums (`CHECKSUM_MISMATCH`, `CHECKSUM_ALGORITHM_UNSUPPORTED`),
range (`RANGE_INVALID`),
auth (`SIGNATURE_INVALID`, `TOKEN_EXPIRED`),
transport (`TRANSPORT_FAILED`).

### Tests via real Garage container

- `@suaveplan/testing-integration-garage-s3` (already shipped) hosts
  every integration test. No mocks. The fixture is extended in this
  change if any capability surface (e.g. `PutBucketLifecycleCommand`)
  needs additional helper exposure.

### 100% coverage gate

- Per project rule: `100/100/100/100` istanbul, gated by
  `bunx turbo test:coverage --filter=@suaveplan/storage-s3`.

## Parity Checklist

This change has no upstream `../suaveplan/packages/storage-s3/` to migrate
from (the upstream package was never authored there — confirmed by `ls`
returning `No such file or directory`). The parity target is **the full
public S3 API surface** as documented by AWS, not an upstream library.
Therefore the table below is structured by S3 capability rather than upstream
file:

| Capability | S3 native | Migrated | Notes |
|---|---|---|---|
| `get` / `set` / `delete` / `list` / `head` | ✅ | ✅ | Production-grade with full options. |
| `setPartial` (whole-object via multipart) | ✅ | ✅ | Non-zero offset retains r-m-w; `partialWrites: false`. |
| `getPartial` via native `Range:` | ✅ | ✅ | Replaces fetch-whole-then-slice. |
| Multipart auto-promote + parallel + abort | ✅ | ✅ | Default threshold 16 MiB, concurrency 4. |
| Streaming `setStream` | ✅ | ✅ | `ReadableStream<Uint8Array>` without buffering. |
| Presigned GET / PUT / DELETE | ✅ | ✅ | TTL + headers + content-type pin. |
| SSE-S3 / SSE-KMS / SSE-C | ✅ | ✅ | Per put / get / copy. |
| Object metadata (content-* + user-metadata) | ✅ | ✅ | Full bag. |
| Canned ACLs + grant policies + getObjectAcl/putObjectAcl | ✅ | ✅ | |
| `copyObject` (single-PUT + UploadPartCopy) | ✅ | ✅ | Cross-bucket supported. |
| Object tagging (get/put/delete + inline at set) | ✅ | ✅ | |
| Versioning (list versions, versionId on get/head/delete) | ✅ | ✅ | |
| Bucket lifecycle (get/put/delete) | ✅ | ✅ | |
| Bucket CORS (get/put/delete) | ✅ | ✅ | |
| Bucket ops (create / delete / exists / region) | ✅ | ✅ | |
| `listObjectsV2` paginator | ✅ | ✅ | Prefix/delimiter/max/continuation/startAfter. |
| Conditional ops (IfMatch / IfNoneMatch / If-*Modified-Since) | ✅ | ✅ | |
| Checksums (SHA256, CRC32C) | ✅ | ✅ | |
| Pluggable transport + signer + `@suaveplan/retry` | n/a | ✅ | Genesis-only requirement. |
| Comprehensive `STORAGE_S3.*` codes | n/a | ✅ | ~50 codes. |
| Real Garage integration tests | n/a | ✅ | Via `testing-integration-garage-s3`. |

No row is 🚫. Every capability lands in this change.

## Out of scope

Strictly things outside the S3 protocol identity:

- **CloudFront integration** (signed-cookie / signed-URL generation against
  a CloudFront key-pair, OAI / OAC origin-access-control). Belongs in a
  separate `@suaveplan/storage-cdn-cloudfront` package.
- **AWS-account-level operations** — IAM policy authoring, STS role
  assumption beyond the standard credential provider chain, KMS key
  creation / rotation. Out of `storage-s3`'s remit.
- **Resumable uploads with cross-process persistence** of `UploadId` +
  completed-part list. Multipart resume mid-process is supported via the
  engine; cross-restart resume requires a separate coordinator.
- **S3 Object Lambda / Object Lock retention modes** — distinct surface,
  not part of the core object-store contract.
- **Bucket policies / public-access blocks** beyond the ACL surface above.

## Impact

### Affected dependents

Run `bunx turbo ls --filter=...@suaveplan/storage-s3` to enumerate. Expected
dependents (verified at implementation time):

- `@suaveplan/storage` — [ ] tests run / [ ] docs reviewed / [x] no action
  needed (reason: `storage-s3` consumes `AbstractStorage`; no
  surface-of-`storage` change.)
- `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed /
  [ ] no action needed (reason: declared consumer of `setPartial` and
  range reads via `DriveStorage`; benefits directly from the native range
  GET and large-object write path.)
- `@suaveplan/drive-plugin-cache` — [ ] tests run / [ ] docs reviewed /
  [x] no action needed (reason: cache layer round-trips byte ranges; new
  range path is observable but contract unchanged.)
- `@suaveplan/storage-cdn` / `storage-cdn-*` — [ ] tests run / [ ] docs
  reviewed / [x] no action needed (reason: CDN adapters do not consume
  `storage-s3` directly.)

`storage-s3` is not a foundation package per §14a, but the dependent sweep
(§13 step 10, `bunx turbo test --filter=...@suaveplan/storage-s3`) MUST run
before archive because the multipart / range / SSE wiring is observable to
chunked-drive consumers.

## Migration

The change is mostly additive on the public surface; the few behaviour
changes that consumers can observe are listed below with grep commands:

1. **`set` for large payloads now issues multipart** — observable via
   `S3ServiceException` shape; no behaviour change for size ≤ 16 MiB.
2. **`getPartial` now uses native Range** — observable via wire traffic;
   contract identical (same bytes returned).
3. **Retry composition now flows through `@suaveplan/retry`** — observable
   only if callers introspect retry timing. Error codes preserved.
4. **`S3_CODES` gains ~40 new entries** — consumers asserting on the legacy
   11 codes are unaffected; new failure paths surface the new codes.

Search command for consumers depending on the legacy error-code surface:

```bash
grep -rln "S3_CODES\." packages/ | grep -v "/dist/\|/coverage/\|/storage-s3/"
```