Files
suaveplan/openspec/archive/2026-04-24-enterprise-storage-s3/tasks.md
2026-05-27 23:29:53 +01:00

169 lines
9.1 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Tasks: enterprise-storage-s3
**Change ID**: enterprise-storage-s3
**Package**: `@suaveplan/storage-s3`
**Target path**: `universal/storage/storage-s3`
**Prerequisite packages on Verdaccio**: `@suaveplan/storage`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/retry`
---
## 1. Scaffold
- [x] Create directory `universal/storage/storage-s3/` with `src/`, `tests/`, `src/__benchmarks__/`, and all feature subdirectories per the layout in `proposal.md`.
- [x] Write `package.json` at version `0.1.0` with:
- `"type": "module"`, `"sideEffects": false`
- `exports` map: top-level `"."` barrel plus subpaths `./s3/s3`, `./types/s3.types`, `./types/schemas`
- `dependencies`: `@suaveplan/storage workspace:^`, `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/retry workspace:^`, `@aws-sdk/client-s3 catalog:storage`, `@aws-sdk/s3-request-presigner catalog:storage`, `zod catalog:forms`, `@opentelemetry/api catalog:observability`, `tslib catalog:build`
- `devDependencies`: `vitest catalog:testing`, `@vitest/coverage-istanbul catalog:testing`, `fast-check catalog:testing`, `mitata catalog:testing`, `@suaveplan/testing workspace:^`
- [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`.
- [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config` (pool: `forks`).
- [x] Add `lint`, `typecheck`, `test`, `test:coverage`, `build`, `docs:check`, `bench` scripts to `package.json`.
---
## 2. Types + errors
### 2a. S3 types (`src/types/s3.types.ts` + `src/types/s3.types.md`)
- [x] Define `S3Options`: `bucket`, `region`, `accessKeyId?`, `secretAccessKey?`, `endpoint?`, `keyPrefix?` — preserved from main.
- [x] Add genesis-only optional fields: `retry?: RetryStrategyOptions`, `logger?: LoggerLike`, `signal?: AbortSignal` (default for all ops).
- [x] Define `PresignOptions`: `expiresIn?: number`, `method?: 'GET' | 'PUT'`.
- [x] Define `MultipartUploadOptions`: `partSize?: number`, `concurrency?: number`, `signal?: AbortSignal`.
- [x] Write `src/types/s3.types.md` (≥200 words, API reference, examples).
### 2b. Error codes (`src/error-codes.ts`)
- [x] Import `CodeMetadata`, `ErrorCodes`, `SuaveplanError` from `@suaveplan/error`.
- [x] Define `S3ErrorCode` union — ten codes: `S3_CREDENTIALS_MISSING`, `S3_CREDENTIALS_INVALID`, `S3_OBJECT_NOT_FOUND`, `S3_BUCKET_NOT_FOUND`, `S3_UPLOAD_FAILED`, `S3_DELETE_FAILED`, `S3_LIST_FAILED`, `S3_PRESIGN_FAILED`, `S3_MULTIPART_FAILED`, `S3_REGION_MISMATCH`.
- [x] Define `S3_ERROR_METADATA` with `httpStatus`, `severity`, `retryable`, `category`, `description` per code.
- [x] Export `registerS3ErrorCodes()`. Idempotent; called eagerly at module scope.
- [x] Export `S3StorageError` extending `SuaveplanError` carrying `S3ErrorCode` discriminant.
### 2c. Zod schemas (`src/types/schemas.ts` + `src/types/schemas.md`)
- [x] Write `S3OptionsSchema`, `PresignOptionsSchema`, `MultipartUploadOptionsSchema`.
- [x] Write `src/types/schemas.md` (≥200 words).
---
## 3. Constants (`src/constants/index.ts`)
- [x] Export `DEFAULT_PRESIGN_EXPIRES_SECONDS = 3600`, `DEFAULT_MULTIPART_PART_SIZE = 5 * 1024 * 1024`, `DEFAULT_MULTIPART_CONCURRENCY = 4`.
---
## 4. Credential validator (`src/utils/credential-validator.ts`)
- [x] Implement `isAwsS3(endpoint?: string): boolean` — returns `true` if endpoint is undefined or hostname contains `amazonaws.com`.
- [x] Implement `validateCredentials(options: S3Options): void`:
- Throws `S3StorageError(S3_CREDENTIALS_MISSING)` if `accessKeyId` or `secretAccessKey` absent (neither in options nor env).
- Throws `S3StorageError(S3_CREDENTIALS_INVALID)` for invalid AWS AKIA key format when `isAwsS3(options.endpoint)` is true.
- No validation for S3-compatible endpoints.
- [x] Write `credential-validator.test.ts`: valid AWS creds, invalid format, missing creds, S3-compatible endpoint skips validation.
- [x] Write `credential-validator.md` (≥200 words).
---
## 5. S3 class (`src/s3/s3/`)
- [x] Implement `S3 extends AbstractStorage` (import from `@suaveplan/storage`):
- Constructor calls `validateCredentials(options)`, builds `S3Client` with `{ region, credentials: { accessKeyId, secretAccessKey }, endpoint }`.
- Internal `_resolveKey(key): string` — applies `options.keyPrefix` stripping double-slashes.
- **`get(key, options?)`**:
- Sends `GetObjectCommand`; reads response body as `Uint8Array`.
- Throws `S3StorageError(S3_OBJECT_NOT_FOUND)` on `NoSuchKey`.
- OTel span + counter. Retry wrapper if configured.
- **`set(key, value, options?)`**:
- Sends `PutObjectCommand`.
- Throws `S3StorageError(S3_UPLOAD_FAILED)` on error.
- OTel span + histogram (object size attribute).
- **`delete(key)`**:
- Sends `DeleteObjectCommand`.
- Throws `S3StorageError(S3_DELETE_FAILED)` on error.
- **`list(prefix?, options?)`**:
- Sends `ListObjectsV2Command` with pagination (handles `NextContinuationToken`).
- Throws `S3StorageError(S3_LIST_FAILED)` on error.
- **`exists(key)`**: sends `HeadObjectCommand`; returns `true` on 200, `false` on `NoSuchKey`.
- **`metadata(key)`**: parses `HeadObjectCommand` response into `StorageMetadata`.
- **`copy(source, destination)`**: sends `CopyObjectCommand`.
- **`move(source, destination)`**: `copy` then `delete`.
- **`presign(key, options?)`**: uses `getSignedUrl` from `@aws-sdk/s3-request-presigner`. Throws `S3StorageError(S3_PRESIGN_FAILED)`.
- **`setMultipart(key, stream, options?)`**:
- `CreateMultipartUploadCommand` → upload parts with concurrency cap → `CompleteMultipartUploadCommand`.
- On any error: `AbortMultipartUploadCommand` then rethrow as `S3StorageError(S3_MULTIPART_FAILED)`.
- Respects `options.signal` (abort mid-upload).
- All methods accept per-call `signal?: AbortSignal` that overrides the instance-level default.
- Log via injected `logger` at `debug` level.
- [x] Write `s3.test.ts` (unit: inject a mock `S3Client` via constructor option; no vi.mock — DI only). Cover: get hit, get miss (404), set, delete, list with pagination, exists true/false, copy, move, presign, credential validation failure.
- [x] Write `s3.md` (≥200 words, both basic + advanced example with retry config).
---
## 6. Registry (`src/registry/registry/`)
- [x] `registry.ts` — imports `S3` and calls `StorageFactory.register('s3', S3)` as a side-effect on import.
- [x] Write `registry.md` explaining the side-effect and tree-shaking implications.
---
## 7. Public barrel (`src/index.ts`)
- [x] Re-export: `S3`, `S3StorageError`, `S3OptionsSchema`, `PresignOptionsSchema`, `MultipartUploadOptionsSchema`, all types, error-codes, constants.
- [x] Import `./registry/registry` to trigger provider registration.
---
## 8. Property-based tests (`tests/property.test.ts`)
Use `fast-check`. All timeouts: `{ timeout: 30000 }`.
- [x] **Key prefix normalisation**: for arbitrary key and prefix strings (alphanum + `/`), assert `resolveKey(prefix, key)` never produces double slashes and always starts with prefix.
- [x] **`isAwsS3` detection**: for arbitrary endpoint strings, assert `isAwsS3` returns true iff hostname contains `amazonaws.com`.
- [x] **List pagination assembly**: given M pages of N/M keys each, assert the assembled list equals the concatenation of all pages in order.
---
## 9. Integration tests (`tests/integration.test.ts`)
- [x] Gate all tests behind `process.env.GARAGE_ENDPOINT` skip guard.
- [x] Test: put → get → exists → metadata → list → copy → move → delete against a real Garage S3-compatible server.
- [x] Test: multipart upload of a 15 MiB buffer (3 × 5 MiB parts).
- [x] Test: presigned GET URL is accessible without credentials.
---
## 10. Benchmarks (`src/__benchmarks__/s3.bench.ts`)
Use `mitata`. Gate IO benchmarks behind `SKIP_S3_BENCHMARKS` env flag.
- [x] `get_hot_100` (conditional): 100 `get()` calls on a pre-existing 1 KiB object.
- [x] `set_1mb` (conditional): upload a 1 MiB buffer.
- [x] `list_1000` (conditional): list 1000 objects.
- [x] `credential_validate_1k` (pure): 1000 credential validation calls.
---
## 11. Docs
- [x] `src/types/s3.types.md`
- [x] `src/types/schemas.md`
- [x] `src/s3/s3/s3.md`
- [x] `src/registry/registry/registry.md`
- [x] `src/utils/credential-validator.md`
- [x] Root `universal/storage/storage-s3/README.md`.
---
## 12. Verify and publish
- [x] Run `bunx turbo lint --filter=@suaveplan/storage-s3` — zero errors.
- [x] Run `bunx turbo typecheck --filter=@suaveplan/storage-s3` — zero errors.
- [x] Run `bunx turbo test --filter=@suaveplan/storage-s3` — all tests pass.
- [x] Run `bunx turbo test:coverage --filter=@suaveplan/storage-s3` — 100% on all touched modules.
- [x] Run `bun run docs:check --filter=@suaveplan/storage-s3` — zero errors.
- [x] Confirm no `TODO`, `FIXME`, `@deprecated`, untyped `any`, `ts-ignore`, or `.skip` in `src/` or `tests/`.
- [x] Confirm `"sideEffects": false` in `package.json`.
- [x] Run `bun scripts/publish.ts --package @suaveplan/storage-s3`.
- [x] Update `CLAUDE.md` package listing to include `@suaveplan/storage-s3`.
- [x] Archive: move `openspec/changes/enterprise-storage-s3/` to `openspec/archive/2026-04-24-enterprise-storage-s3/` and fold spec delta into `openspec/specs/storage/spec.md`.