Initial Commit

openspec/archive/2026-04-24-enterprise-etl/tasks.md

@@ -0,0 +1,89 @@
+# Tasks: enterprise-etl
+
+- **Change ID**: enterprise-etl
+- **Package**: `@suaveplan/etl`
+- **Date**: 2026-04-24
+
+## Prerequisites (must be published to Verdaccio first)
+
+- [x] `@suaveplan/error`
+- [x] `@suaveplan/types`
+- [x] `@suaveplan/streams`
+- [x] `@suaveplan/serialize`
+- [x] `@suaveplan/retry`
+- [x] `@suaveplan/events`
+
+## Tasks
+
+### 1. Scaffold package
+
+- [x] Create `universal/data/etl/` directory tree per file layout in `proposal.md`
+- [x] Write `package.json` (name `@suaveplan/etl`, version `0.1.0`, `sideEffects: false`, correct workspace deps and catalog externals)
+- [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`
+- [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config`
+- [x] Add package to root `turbo.json` pipeline if not already covered by glob
+
+### 2. Error codes (`src/error-codes.ts`)
+
+- [x] Define `EtlErrorCode` union type
+- [x] Call `defineCodes` to register all `ETL.*` codes with severity and retryable flags
+- [x] Export `registerEtlErrorCodes()` factory
+- [x] Export `EtlError extends SuaveplanError<EtlErrorCode>`
+
+### 3. Types (`src/types/common.ts`)
+
+- [x] Define and export all public interfaces: `Pipeline`, `PipelineOptions`, `PipelineResult`, `PipelineStats`, `Source`, `Transform`, `TransformContext`, `Sink`, `ErrorHandler`, `ErrorAction`
+- [x] Export `SKIP` unique symbol
+- [x] Write `common.md`
+
+### 4. Stage implementations
+
+- [x] Implement `Source` interface and built-in sources (`arraySource`, `generatorSource`, `streamSource`) in `src/stages/source/`
+- [x] Implement `Transform` interface and built-in transforms (`mapTransform`, `filterTransform`, `validateTransform`, `batchTransform`, `flatMapTransform`) in `src/stages/transform/`
+- [x] Implement `Sink` interface and built-in sinks (`arraySink`, `nullSink`, `transformSink`) in `src/stages/sink/`
+- [x] Each feature file has co-located `.md`, `.types.ts`, `.test.ts`, `index.ts` barrel
+
+### 5. Pipeline orchestrator (`src/core/pipeline/`)
+
+- [x] Implement `createPipeline` with stage chaining, batch processing, `AbortSignal` propagation, and configurable `ErrorHandler`
+- [x] Integrate `@suaveplan/streams` `ReadableStream`/`TransformStream`/`WritableStream` internally
+- [x] Emit progress events via `@suaveplan/events` bus when configured
+- [x] Write `pipeline.md`, `pipeline.types.ts`, `pipeline.test.ts`, `index.ts`
+
+### 6. Retry adapter (`src/utils/retry-adapter.ts`)
+
+- [x] Wrap source/sink IO methods with `@suaveplan/retry` when `retryOptions` is provided
+- [x] Write `retry-adapter.md` and `retry-adapter.test.ts`
+
+### 7. OTel instrumentation
+
+- [x] Add spans, counters, histograms per `proposal.md` §New additions §3
+
+### 8. Tests
+
+- [x] Unit tests co-located with each feature module (100% coverage target)
+- [x] `tests/integration.test.ts` — full pipeline: array source → validate → map → array sink
+- [x] `tests/property.test.ts` — fast-check properties per `proposal.md` §5
+
+### 9. Benchmarks (`src/__benchmarks__/pipeline.bench.ts`)
+
+- [x] Implement benchmark cases per `proposal.md` §6
+
+### 10. Docs
+
+- [x] Every `.ts` implementation file has a sibling `.md` with all required sections (≥200 words, both examples)
+- [x] Root `README.md` with feature table, usage examples, API reference links
+
+### 11. Pre-publish gate
+
+- [x] `bunx turbo lint typecheck test --filter=@suaveplan/etl` green
+- [x] `bunx turbo test:coverage --filter=@suaveplan/etl` ≥ 100% on touched modules
+
+### 12. Publish
+
+- [x] `bun scripts/publish.ts --package @suaveplan/etl`
+
+### 13. Archive
+
+- [x] Fold spec delta into `openspec/specs/data/spec.md`
+- [x] `mv openspec/changes/enterprise-etl openspec/archive/2026-04-24-enterprise-etl`