Initial Commit
This commit is contained in:
168
openspec/archive/2026-04-24-enterprise-storage-s3/tasks.md
Normal file
168
openspec/archive/2026-04-24-enterprise-storage-s3/tasks.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# Tasks: enterprise-storage-s3
|
||||
|
||||
**Change ID**: enterprise-storage-s3
|
||||
**Package**: `@suaveplan/storage-s3`
|
||||
**Target path**: `universal/storage/storage-s3`
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/storage`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/retry`
|
||||
|
||||
---
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [x] Create directory `universal/storage/storage-s3/` with `src/`, `tests/`, `src/__benchmarks__/`, and all feature subdirectories per the layout in `proposal.md`.
|
||||
- [x] Write `package.json` at version `0.1.0` with:
|
||||
- `"type": "module"`, `"sideEffects": false`
|
||||
- `exports` map: top-level `"."` barrel plus subpaths `./s3/s3`, `./types/s3.types`, `./types/schemas`
|
||||
- `dependencies`: `@suaveplan/storage workspace:^`, `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/retry workspace:^`, `@aws-sdk/client-s3 catalog:storage`, `@aws-sdk/s3-request-presigner catalog:storage`, `zod catalog:forms`, `@opentelemetry/api catalog:observability`, `tslib catalog:build`
|
||||
- `devDependencies`: `vitest catalog:testing`, `@vitest/coverage-istanbul catalog:testing`, `fast-check catalog:testing`, `mitata catalog:testing`, `@suaveplan/testing workspace:^`
|
||||
- [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`.
|
||||
- [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config` (pool: `forks`).
|
||||
- [x] Add `lint`, `typecheck`, `test`, `test:coverage`, `build`, `docs:check`, `bench` scripts to `package.json`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Types + errors
|
||||
|
||||
### 2a. S3 types (`src/types/s3.types.ts` + `src/types/s3.types.md`)
|
||||
|
||||
- [x] Define `S3Options`: `bucket`, `region`, `accessKeyId?`, `secretAccessKey?`, `endpoint?`, `keyPrefix?` — preserved from main.
|
||||
- [x] Add genesis-only optional fields: `retry?: RetryStrategyOptions`, `logger?: LoggerLike`, `signal?: AbortSignal` (default for all ops).
|
||||
- [x] Define `PresignOptions`: `expiresIn?: number`, `method?: 'GET' | 'PUT'`.
|
||||
- [x] Define `MultipartUploadOptions`: `partSize?: number`, `concurrency?: number`, `signal?: AbortSignal`.
|
||||
- [x] Write `src/types/s3.types.md` (≥200 words, API reference, examples).
|
||||
|
||||
### 2b. Error codes (`src/error-codes.ts`)
|
||||
|
||||
- [x] Import `CodeMetadata`, `ErrorCodes`, `SuaveplanError` from `@suaveplan/error`.
|
||||
- [x] Define `S3ErrorCode` union — ten codes: `S3_CREDENTIALS_MISSING`, `S3_CREDENTIALS_INVALID`, `S3_OBJECT_NOT_FOUND`, `S3_BUCKET_NOT_FOUND`, `S3_UPLOAD_FAILED`, `S3_DELETE_FAILED`, `S3_LIST_FAILED`, `S3_PRESIGN_FAILED`, `S3_MULTIPART_FAILED`, `S3_REGION_MISMATCH`.
|
||||
- [x] Define `S3_ERROR_METADATA` with `httpStatus`, `severity`, `retryable`, `category`, `description` per code.
|
||||
- [x] Export `registerS3ErrorCodes()`. Idempotent; called eagerly at module scope.
|
||||
- [x] Export `S3StorageError` extending `SuaveplanError` carrying `S3ErrorCode` discriminant.
|
||||
|
||||
### 2c. Zod schemas (`src/types/schemas.ts` + `src/types/schemas.md`)
|
||||
|
||||
- [x] Write `S3OptionsSchema`, `PresignOptionsSchema`, `MultipartUploadOptionsSchema`.
|
||||
- [x] Write `src/types/schemas.md` (≥200 words).
|
||||
|
||||
---
|
||||
|
||||
## 3. Constants (`src/constants/index.ts`)
|
||||
|
||||
- [x] Export `DEFAULT_PRESIGN_EXPIRES_SECONDS = 3600`, `DEFAULT_MULTIPART_PART_SIZE = 5 * 1024 * 1024`, `DEFAULT_MULTIPART_CONCURRENCY = 4`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Credential validator (`src/utils/credential-validator.ts`)
|
||||
|
||||
- [x] Implement `isAwsS3(endpoint?: string): boolean` — returns `true` if endpoint is undefined or hostname contains `amazonaws.com`.
|
||||
- [x] Implement `validateCredentials(options: S3Options): void`:
|
||||
- Throws `S3StorageError(S3_CREDENTIALS_MISSING)` if `accessKeyId` or `secretAccessKey` absent (neither in options nor env).
|
||||
- Throws `S3StorageError(S3_CREDENTIALS_INVALID)` for invalid AWS AKIA key format when `isAwsS3(options.endpoint)` is true.
|
||||
- No validation for S3-compatible endpoints.
|
||||
- [x] Write `credential-validator.test.ts`: valid AWS creds, invalid format, missing creds, S3-compatible endpoint skips validation.
|
||||
- [x] Write `credential-validator.md` (≥200 words).
|
||||
|
||||
---
|
||||
|
||||
## 5. S3 class (`src/s3/s3/`)
|
||||
|
||||
- [x] Implement `S3 extends AbstractStorage` (import from `@suaveplan/storage`):
|
||||
- Constructor calls `validateCredentials(options)`, builds `S3Client` with `{ region, credentials: { accessKeyId, secretAccessKey }, endpoint }`.
|
||||
- Internal `_resolveKey(key): string` — applies `options.keyPrefix` stripping double-slashes.
|
||||
- **`get(key, options?)`**:
|
||||
- Sends `GetObjectCommand`; reads response body as `Uint8Array`.
|
||||
- Throws `S3StorageError(S3_OBJECT_NOT_FOUND)` on `NoSuchKey`.
|
||||
- OTel span + counter. Retry wrapper if configured.
|
||||
- **`set(key, value, options?)`**:
|
||||
- Sends `PutObjectCommand`.
|
||||
- Throws `S3StorageError(S3_UPLOAD_FAILED)` on error.
|
||||
- OTel span + histogram (object size attribute).
|
||||
- **`delete(key)`**:
|
||||
- Sends `DeleteObjectCommand`.
|
||||
- Throws `S3StorageError(S3_DELETE_FAILED)` on error.
|
||||
- **`list(prefix?, options?)`**:
|
||||
- Sends `ListObjectsV2Command` with pagination (handles `NextContinuationToken`).
|
||||
- Throws `S3StorageError(S3_LIST_FAILED)` on error.
|
||||
- **`exists(key)`**: sends `HeadObjectCommand`; returns `true` on 200, `false` on `NoSuchKey`.
|
||||
- **`metadata(key)`**: parses `HeadObjectCommand` response into `StorageMetadata`.
|
||||
- **`copy(source, destination)`**: sends `CopyObjectCommand`.
|
||||
- **`move(source, destination)`**: `copy` then `delete`.
|
||||
- **`presign(key, options?)`**: uses `getSignedUrl` from `@aws-sdk/s3-request-presigner`. Throws `S3StorageError(S3_PRESIGN_FAILED)`.
|
||||
- **`setMultipart(key, stream, options?)`**:
|
||||
- `CreateMultipartUploadCommand` → upload parts with concurrency cap → `CompleteMultipartUploadCommand`.
|
||||
- On any error: `AbortMultipartUploadCommand` then rethrow as `S3StorageError(S3_MULTIPART_FAILED)`.
|
||||
- Respects `options.signal` (abort mid-upload).
|
||||
- All methods accept per-call `signal?: AbortSignal` that overrides the instance-level default.
|
||||
- Log via injected `logger` at `debug` level.
|
||||
- [x] Write `s3.test.ts` (unit: inject a mock `S3Client` via constructor option; no vi.mock — DI only). Cover: get hit, get miss (404), set, delete, list with pagination, exists true/false, copy, move, presign, credential validation failure.
|
||||
- [x] Write `s3.md` (≥200 words, both basic + advanced example with retry config).
|
||||
|
||||
---
|
||||
|
||||
## 6. Registry (`src/registry/registry/`)
|
||||
|
||||
- [x] `registry.ts` — imports `S3` and calls `StorageFactory.register('s3', S3)` as a side-effect on import.
|
||||
- [x] Write `registry.md` explaining the side-effect and tree-shaking implications.
|
||||
|
||||
---
|
||||
|
||||
## 7. Public barrel (`src/index.ts`)
|
||||
|
||||
- [x] Re-export: `S3`, `S3StorageError`, `S3OptionsSchema`, `PresignOptionsSchema`, `MultipartUploadOptionsSchema`, all types, error-codes, constants.
|
||||
- [x] Import `./registry/registry` to trigger provider registration.
|
||||
|
||||
---
|
||||
|
||||
## 8. Property-based tests (`tests/property.test.ts`)
|
||||
|
||||
Use `fast-check`. All timeouts: `{ timeout: 30000 }`.
|
||||
|
||||
- [x] **Key prefix normalisation**: for arbitrary key and prefix strings (alphanum + `/`), assert `resolveKey(prefix, key)` never produces double slashes and always starts with prefix.
|
||||
- [x] **`isAwsS3` detection**: for arbitrary endpoint strings, assert `isAwsS3` returns true iff hostname contains `amazonaws.com`.
|
||||
- [x] **List pagination assembly**: given M pages of N/M keys each, assert the assembled list equals the concatenation of all pages in order.
|
||||
|
||||
---
|
||||
|
||||
## 9. Integration tests (`tests/integration.test.ts`)
|
||||
|
||||
- [x] Gate all tests behind `process.env.GARAGE_ENDPOINT` skip guard.
|
||||
- [x] Test: put → get → exists → metadata → list → copy → move → delete against a real Garage S3-compatible server.
|
||||
- [x] Test: multipart upload of a 15 MiB buffer (3 × 5 MiB parts).
|
||||
- [x] Test: presigned GET URL is accessible without credentials.
|
||||
|
||||
---
|
||||
|
||||
## 10. Benchmarks (`src/__benchmarks__/s3.bench.ts`)
|
||||
|
||||
Use `mitata`. Gate IO benchmarks behind `SKIP_S3_BENCHMARKS` env flag.
|
||||
|
||||
- [x] `get_hot_100` (conditional): 100 `get()` calls on a pre-existing 1 KiB object.
|
||||
- [x] `set_1mb` (conditional): upload a 1 MiB buffer.
|
||||
- [x] `list_1000` (conditional): list 1000 objects.
|
||||
- [x] `credential_validate_1k` (pure): 1000 credential validation calls.
|
||||
|
||||
---
|
||||
|
||||
## 11. Docs
|
||||
|
||||
- [x] `src/types/s3.types.md`
|
||||
- [x] `src/types/schemas.md`
|
||||
- [x] `src/s3/s3/s3.md`
|
||||
- [x] `src/registry/registry/registry.md`
|
||||
- [x] `src/utils/credential-validator.md`
|
||||
- [x] Root `universal/storage/storage-s3/README.md`.
|
||||
|
||||
---
|
||||
|
||||
## 12. Verify and publish
|
||||
|
||||
- [x] Run `bunx turbo lint --filter=@suaveplan/storage-s3` — zero errors.
|
||||
- [x] Run `bunx turbo typecheck --filter=@suaveplan/storage-s3` — zero errors.
|
||||
- [x] Run `bunx turbo test --filter=@suaveplan/storage-s3` — all tests pass.
|
||||
- [x] Run `bunx turbo test:coverage --filter=@suaveplan/storage-s3` — 100% on all touched modules.
|
||||
- [x] Run `bun run docs:check --filter=@suaveplan/storage-s3` — zero errors.
|
||||
- [x] Confirm no `TODO`, `FIXME`, `@deprecated`, untyped `any`, `ts-ignore`, or `.skip` in `src/` or `tests/`.
|
||||
- [x] Confirm `"sideEffects": false` in `package.json`.
|
||||
- [x] Run `bun scripts/publish.ts --package @suaveplan/storage-s3`.
|
||||
- [x] Update `CLAUDE.md` package listing to include `@suaveplan/storage-s3`.
|
||||
- [x] Archive: move `openspec/changes/enterprise-storage-s3/` to `openspec/archive/2026-04-24-enterprise-storage-s3/` and fold spec delta into `openspec/specs/storage/spec.md`.
|
||||
Reference in New Issue
Block a user