5.4 KiB
5.4 KiB
Tasks: add-storage-s3-multipart
1. Error codes + constants
- 1.1 Extend
S3_CODESviadefineCodeswithMULTIPART_INIT_FAILED,MULTIPART_PART_FAILED,MULTIPART_COMPLETE_FAILED,MULTIPART_ABORT_FAILED. RetainMULTIPART_FAILEDas a documented legacy alias (no new call site raises it). - 1.2 Add
DEFAULT_MULTIPART_THRESHOLD = 16 * 1024 * 1024andDEFAULT_MULTIPART_CONCURRENCY = 4tosrc/constants/index.ts. VerifyDEFAULT_PART_SIZEfloor (≥ 5 MiB) and add a constantMIN_PART_SIZE = 5 * 1024 * 1024. - 1.3 Add
MultipartTuninginterface tosrc/types/s3.types.ts; extendS3Optionswithmultipart?: MultipartTuning.
2. Internal engine
- 2.1 Create
src/multipart/part-chunker.ts+ sibling.md+.test.ts—chunkStream(stream, partSize): AsyncIterable<PartInput>; covers exact multiples, ragged final part, empty stream, abort mid-read. - 2.2 Create
src/multipart/worker-pool.ts+ sibling.md+.test.ts— bounded-concurrency scheduler usingPromise.race+ a linkedAbortController; covers cap enforcement, first-failure aborts peers, caller-abort cancels in-flight workers. - 2.3 Create
src/multipart/multipart-engine.ts+ sibling.md+.test.ts—runMultipart(client, bucket, key, parts, tuning, signal)withinitMultipart/completePartsList/abortMultiparthelpers; maps each stage failure to its dedicated code. - 2.4 Create
src/multipart/index.ts— explicit named re-exports only (rule 10 / §6a). Noexport *.
3. Wire into S3.set + S3.setPartial
- 3.1 In
s3/s3.ts, branchS3.setonvalue.byteLength > (this.options.multipart?.threshold ?? DEFAULT_MULTIPART_THRESHOLD)to call the engine via a synchronous part iterator built from the buffer; below the threshold keep the existingPutObjectCommand. - 3.2 In
S3.setPartial, whenoffset === 0ANDbytes.byteLength > threshold, delegate to the engine (whole-object write); otherwise keepdefaultSetPartial. Document ins3/s3.mdthatcapabilities.partialWritesstaysfalseand explain why. - 3.3 Rewrite
S3.setMultipartas a thin wrapper over the new engine usingchunkStreamfor the inputReadableStream. Preserve public signature; preservesetMultipartsubpath export. - 3.4 Validate
partSize >= MIN_PART_SIZEat construction time; throwS3StorageError(MULTIPART_INIT_FAILED, ...)with a clear message.
4. Subpath exports + barrels
- 4.1 Add
./multipart/multipart-engine,./multipart/part-chunker,./multipart/worker-poolentries topackage.jsonexports. - 4.2 Update
src/index.tsto re-export the new multipart helpers via a namespace export (export * as multipart from "./multipart/index.js") so consumers can reach them without breaking the root barrel rule.
5. Docs
- 5.1 Update
README.mdModules + Features sections to mention auto multipart, tuning, and the four new error codes. - 5.2 Update
s3/s3.md(Implementation Notes) — threshold rationale, capability honesty, abort/cleanup contract. - 5.3 Add
multipart/multipart-engine.md,part-chunker.md,worker-pool.mdper §4 (≥200 words, basic + advanced examples, API reference, implementation notes).
6. Integration tests (real Garage container)
- 6.1 Confirm
@suaveplan/testing-integration-garage-s3fixture surface covers MPU operations end-to-end; if a missing capability is found (e.g. mid-flightListPartsCommandexposure), extend the fixture in the same change. - 6.2 Add
tests/integration/multipart.test.tscovering the 10 cases indesign.md#test-plan. No mocks — use a wrappingS3ClientFactoryto inject controlled delays / failures around the real Garage client. - 6.3 Wire integration suite into
vitest.integration.config.ts(already present in the package).
7. Verify (per §13 pre-ship sequence)
- 7.1 Spec reconciliation (§13 step 0): every
### Requirementinspecs/storage-s3/spec.mdhas matching impl + tests; every**Verified by:**path resolves verbatim. - 7.2
bunx turbo lint typecheck test --filter=@suaveplan/storage-s3green. - 7.3
bunx turbo test:coverage --filter=@suaveplan/storage-s3100/100/100/100. - 7.4 Integration sweep:
bunx turbo test:integration --filter=@suaveplan/storage-s3green against the Garage testcontainer. - 7.5 Dependent sweep:
bunx turbo test --filter=...@suaveplan/storage-s3green. - 7.6 Banned-pattern / barrel-shape / runner-import scans (§13 steps 4 / 4a / 4b) — zero output.
- 7.7
bun scripts/openspec-conformance.ts @suaveplan/storage-s3PASS. - 7.8 Run
openspec validate add-storage-s3-multipart --strict— must report PASS.
8. Archive
- 8.1 Refresh root docs per §13b —
MANIFEST.md,MISSING.yml,STATUS.md,SUMMARY.md; runbunx markdownlint-cli2 --fix. - 8.2 Fold
specs/storage-s3/spec.mdintoopenspec/specs/storage-s3/spec.md(this is the first change to establish that canonical capability file — create the folder underopenspec/specs/only at this archival step, never before). - 8.3 Move
openspec/changes/add-storage-s3-multipart/→openspec/archive/<yyyy-mm-dd>-add-storage-s3-multipart/.