suaveplan/openspec/changes/complete-storage-s3/proposal.md

OpenSpec Change: complete-storage-s3

• Change ID: complete-storage-s3
• Package: @suaveplan/storage-s3 (modify — complete to full S3 parity)
• Status: PROPOSED
• Date: 2026-05-17
• Design: ./design.md
• Spec delta: ./specs/storage-s3/spec.md

Why

@suaveplan/storage-s3 ships a working but thin AbstractStorage implementation: it does get / set / delete / list / head, a serial multipart setMultipart, presigned URLs limited to GET / PUT, and one copy helper. Almost every other surface S3 exposes — and that consumers actually need to ship production object-store workloads — is absent:

• No auto-promote of set to multipart for large payloads; the existing setMultipart is serial and not wired into the AbstractStorage surface.
• No streaming set from ReadableStream<Uint8Array> without buffering.
• No getSignedDeleteUrl; presigning is fixed to GET / PUT with no TTL semantics, no response/request header pinning, no content-type or content-disposition control.
• No server-side encryption (SSE-S3, SSE-KMS, SSE-C) on set or get.
• No object metadata controls — content-type, content-encoding, content-disposition, cache-control, user-defined x-amz-meta-*.
• No ACLs (canned or grant policies), no object tagging, no versioning surface, no bucket lifecycle, no bucket CORS, no bucket-level ops (create / delete / exists / region), no conditional ops (If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since).
• No range reads via get; getPartial round-trips the whole object via the default fallback.
• No checksum integrity (SHA-256 / CRC32C) per AWS-recommended guard.
• Pluggable HTTP transport (FetchLike), pluggable SigV4 signer, and retry composition via @suaveplan/retry are not yet integrated; retries are hand-rolled in withRetry.
• Error coverage is 11 codes for a surface that needs ~50.
• No real-service test harness wiring against the existing @suaveplan/testing-integration-garage-s3 fixture for the multi-feature paths above.

The supersession marker openspec/changes/_superseded-narrow-deferrals-2026-05-17/add-storage-s3-multipart/ confirms that the prior narrow deferral (multipart only) was rolled back in favour of a single change that lands the full S3 surface at parity. This change is that rollup. No cherry-picking, no Phase 2 split.

What Changes

This change brings @suaveplan/storage-s3 to full S3 feature parity in a single shipped change. Every capability below is in scope; none are deferred.

AbstractStorage surface (production-grade)

• get, set, setPartial, has/exists, delete, list, head/stat — each becomes production-grade with full options, conditional headers, SSE, and checksum integrity passthrough.
• getPartial uses native S3 Range: reads — no more fetch-the-whole-object fallback.
• setPartial for offset === 0 AND bytes.byteLength > threshold routes through the native multipart engine; non-zero offsets retain defaultSetPartial r-m-w because S3 has no byte-range PUT. capabilities.partialWrites stays false to honestly advertise that.

Multipart (auto-promote, bounded concurrency, abort+cleanup)

• S3.set auto-promotes to multipart when value.byteLength exceeds a configurable threshold (default 16 MiB).
• Internal multipart-engine owns CreateMultipartUpload → bounded-parallel UploadPart (default concurrency 4) → CompleteMultipartUpload, with full abort propagation to in-flight workers and guaranteed AbortMultipartUpload cleanup.
• Per-stage error codes: MULTIPART_INIT_FAILED, MULTIPART_PART_FAILED, MULTIPART_COMPLETE_FAILED, MULTIPART_ABORT_FAILED. Legacy MULTIPART_FAILED retained as alias.

Streaming put

• S3.setStream(key, stream, opts?) consumes ReadableStream<Uint8Array> without buffering the whole stream — chunked into parts on the fly with a ceiling of 2 part-sized buffers in memory.

Presigned URLs

• getSignedGetUrl, getSignedPutUrl, getSignedDeleteUrl — each with TTL, custom request headers, response-content-type and response-content-disposition overrides, content-type pinning, and SSE parameter forwarding.

Server-side encryption

• SSE-S3 (AES256), SSE-KMS (aws:kms with optional SSEKMSKeyId + BucketKeyEnabled), SSE-C (caller-supplied key material) on every set, setStream, setMultipart, presigned PUT, and copy.
• get accepts the matching SSE-C key for decryption.

Object metadata

• set accepts contentType, contentEncoding, contentDisposition, contentLanguage, cacheControl, expires, and a free-form userMetadata: Record<string, string> mapped to x-amz-meta-*.
• head / getMetadata returns the full metadata bag including eTag, versionId, lastModified, storageClass, and the caller-set headers above.

ACLs

• Canned ACLs (private, public-read, public-read-write, aws-exec-read, authenticated-read, bucket-owner-read, bucket-owner-full-control, log-delivery-write) per put.
• Explicit grant policies (grantRead, grantReadACP, grantWriteACP, grantFullControl) per put.
• getObjectAcl / putObjectAcl for post-write changes.

Copy

• copyObject(source, destination, opts?) — server-side copy within bucket AND across buckets via { sourceBucket }. Supports metadata directive (COPY / REPLACE), tagging directive, conditional headers, SSE parameter overrides.
• UploadPartCopy engine for objects > 5 GiB (multipart server-side copy).

Tagging

• getObjectTags, putObjectTags, deleteObjectTags.
• set accepts inline tags: Record<string, string> (URL-encoded into the x-amz-tagging header — no extra round trip).

Versioning

• listObjectVersions paginator.
• get(key, { versionId }), head(key, { versionId }), delete(key, { versionId }).

Bucket-level operations

• createBucket, deleteBucket, bucketExists, getBucketRegion, listObjectsV2 (paginator with prefix, delimiter, maxKeys, continuationToken, startAfter).
• Lifecycle config: getBucketLifecycle, putBucketLifecycle, deleteBucketLifecycle.
• CORS config: getBucketCors, putBucketCors, deleteBucketCors.

Conditional ops

• IfMatch, IfNoneMatch, IfModifiedSince, IfUnmodifiedSince on get / head / delete / copy — translated to the precondition error code PRECONDITION_FAILED (412) and NOT_MODIFIED (304).

Range reads

• get(key, { range: { offset, length } }) issues a native Range: GET rather than a full-object fetch + slice.

Checksums

• Per-request SHA256 (default) and CRC32C integrity guards per the AWS-recommended additional checksum modes. Returned ChecksumSHA256 / ChecksumCRC32C are surfaced on the metadata bag and verified on get-with-checksum.

Pluggable HTTP transport, signer, retry

• S3Options.transport?: FetchLike to swap the underlying HTTP transport (browser fetch, Node undici, custom edge fetch). Default uses the AWS SDK's built-in.
• S3Options.signer?: SigV4Signer to swap the signer (default uses AWS SDK SigV4 implementation).
• Retry is composed through @suaveplan/retry (createRetryStrategy) — the hand-rolled withRetry is replaced. Throttling / 5xx classification preserved.

Error codes

S3_CODES expands from 11 to ~50 codes covering every failure path: SSE (SSE_KEY_MISSING, SSE_KEY_MISMATCH, SSE_KMS_DENIED), ACLs (ACL_INVALID, ACL_PUT_FAILED), tagging (TAG_PUT_FAILED, TAG_DELETE_FAILED, TAG_LIMIT_EXCEEDED), versioning (VERSION_NOT_FOUND, VERSIONING_DISABLED), lifecycle (LIFECYCLE_INVALID, LIFECYCLE_PUT_FAILED), CORS (CORS_INVALID, CORS_PUT_FAILED), bucket ops (BUCKET_ALREADY_EXISTS, BUCKET_NOT_EMPTY, BUCKET_DELETE_FAILED, BUCKET_CREATE_FAILED, BUCKET_REGION_MISMATCH), conditional (PRECONDITION_FAILED, NOT_MODIFIED), copy (COPY_FAILED, COPY_PART_FAILED), multipart (4 new codes above; MULTIPART_FAILED retained), presign (PRESIGN_TTL_INVALID, PRESIGN_HEADER_INVALID), checksums (CHECKSUM_MISMATCH, CHECKSUM_ALGORITHM_UNSUPPORTED), range (RANGE_INVALID), auth (SIGNATURE_INVALID, TOKEN_EXPIRED), transport (TRANSPORT_FAILED).

Tests via real Garage container

• @suaveplan/testing-integration-garage-s3 (already shipped) hosts every integration test. No mocks. The fixture is extended in this change if any capability surface (e.g. PutBucketLifecycleCommand) needs additional helper exposure.

100% coverage gate

• Per project rule: 100/100/100/100 istanbul, gated by bunx turbo test:coverage --filter=@suaveplan/storage-s3.

Parity Checklist

This change has no upstream ../suaveplan/packages/storage-s3/ to migrate from (the upstream package was never authored there — confirmed by ls returning No such file or directory). The parity target is the full public S3 API surface as documented by AWS, not an upstream library. Therefore the table below is structured by S3 capability rather than upstream file:

Capability	S3 native	Migrated	Notes
get / set / delete / list / head	✅	✅	Production-grade with full options.
setPartial (whole-object via multipart)	✅	✅	Non-zero offset retains r-m-w; partialWrites: false.
getPartial via native Range:	✅	✅	Replaces fetch-whole-then-slice.
Multipart auto-promote + parallel + abort	✅	✅	Default threshold 16 MiB, concurrency 4.
Streaming setStream	✅	✅	ReadableStream<Uint8Array> without buffering.
Presigned GET / PUT / DELETE	✅	✅	TTL + headers + content-type pin.
SSE-S3 / SSE-KMS / SSE-C	✅	✅	Per put / get / copy.
Object metadata (content-* + user-metadata)	✅	✅	Full bag.
Canned ACLs + grant policies + getObjectAcl/putObjectAcl	✅	✅
copyObject (single-PUT + UploadPartCopy)	✅	✅	Cross-bucket supported.
Object tagging (get/put/delete + inline at set)	✅	✅
Versioning (list versions, versionId on get/head/delete)	✅	✅
Bucket lifecycle (get/put/delete)	✅	✅
Bucket CORS (get/put/delete)	✅	✅
Bucket ops (create / delete / exists / region)	✅	✅
listObjectsV2 paginator	✅	✅	Prefix/delimiter/max/continuation/startAfter.
Conditional ops (IfMatch / IfNoneMatch / If-*Modified-Since)	✅	✅
Checksums (SHA256, CRC32C)	✅	✅
Pluggable transport + signer + @suaveplan/retry	n/a	✅	Genesis-only requirement.
Comprehensive STORAGE_S3.* codes	n/a	✅	~50 codes.
Real Garage integration tests	n/a	✅	Via testing-integration-garage-s3.

No row is 🚫. Every capability lands in this change.

Out of scope

Strictly things outside the S3 protocol identity:

• CloudFront integration (signed-cookie / signed-URL generation against a CloudFront key-pair, OAI / OAC origin-access-control). Belongs in a separate @suaveplan/storage-cdn-cloudfront package.
• AWS-account-level operations — IAM policy authoring, STS role assumption beyond the standard credential provider chain, KMS key creation / rotation. Out of storage-s3's remit.
• Resumable uploads with cross-process persistence of UploadId + completed-part list. Multipart resume mid-process is supported via the engine; cross-restart resume requires a separate coordinator.
• S3 Object Lambda / Object Lock retention modes — distinct surface, not part of the core object-store contract.
• Bucket policies / public-access blocks beyond the ACL surface above.

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/storage-s3 to enumerate. Expected dependents (verified at implementation time):

• @suaveplan/storage — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: storage-s3 consumes AbstractStorage; no surface-of-storage change.)
• @suaveplan/drive-plugin-chunks — [x] tests run / [x] docs reviewed / [ ] no action needed (reason: declared consumer of setPartial and range reads via DriveStorage; benefits directly from the native range GET and large-object write path.)
• @suaveplan/drive-plugin-cache — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: cache layer round-trips byte ranges; new range path is observable but contract unchanged.)
• @suaveplan/storage-cdn / storage-cdn-* — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: CDN adapters do not consume storage-s3 directly.)

storage-s3 is not a foundation package per §14a, but the dependent sweep (§13 step 10, bunx turbo test --filter=...@suaveplan/storage-s3) MUST run before archive because the multipart / range / SSE wiring is observable to chunked-drive consumers.

Migration

The change is mostly additive on the public surface; the few behaviour changes that consumers can observe are listed below with grep commands:

1. set for large payloads now issues multipart — observable via S3ServiceException shape; no behaviour change for size ≤ 16 MiB.
2. getPartial now uses native Range — observable via wire traffic; contract identical (same bytes returned).
3. Retry composition now flows through @suaveplan/retry — observable only if callers introspect retry timing. Error codes preserved.
4. S3_CODES gains ~40 new entries — consumers asserting on the legacy 11 codes are unaffected; new failure paths surface the new codes.

Search command for consumers depending on the legacy error-code surface:

grep -rln "S3_CODES\." packages/ | grep -v "/dist/\|/coverage/\|/storage-s3/"