# Tasks: enterprise-etl

- **Change ID**: enterprise-etl
- **Package**: `@suaveplan/etl`
- **Date**: 2026-04-24

## Prerequisites (must be published to Verdaccio first)

- [x] `@suaveplan/error`
- [x] `@suaveplan/types`
- [x] `@suaveplan/streams`
- [x] `@suaveplan/serialize`
- [x] `@suaveplan/retry`
- [x] `@suaveplan/events`

## Tasks

### 1. Scaffold package

- [x] Create `universal/data/etl/` directory tree per file layout in `proposal.md`
- [x] Write `package.json` (name `@suaveplan/etl`, version `0.1.0`, `sideEffects: false`, correct workspace deps and catalog externals)
- [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`
- [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config`
- [x] Add package to root `turbo.json` pipeline if not already covered by glob

### 2. Error codes (`src/error-codes.ts`)

- [x] Define `EtlErrorCode` union type
- [x] Call `defineCodes` to register all `ETL.*` codes with severity and retryable flags
- [x] Export `registerEtlErrorCodes()` factory
- [x] Export `EtlError extends SuaveplanError<EtlErrorCode>`

### 3. Types (`src/types/common.ts`)

- [x] Define and export all public interfaces: `Pipeline`, `PipelineOptions`, `PipelineResult`, `PipelineStats`, `Source`, `Transform`, `TransformContext`, `Sink`, `ErrorHandler`, `ErrorAction`
- [x] Export `SKIP` unique symbol
- [x] Write `common.md`

### 4. Stage implementations

- [x] Implement `Source` interface and built-in sources (`arraySource`, `generatorSource`, `streamSource`) in `src/stages/source/`
- [x] Implement `Transform` interface and built-in transforms (`mapTransform`, `filterTransform`, `validateTransform`, `batchTransform`, `flatMapTransform`) in `src/stages/transform/`
- [x] Implement `Sink` interface and built-in sinks (`arraySink`, `nullSink`, `transformSink`) in `src/stages/sink/`
- [x] Each feature file has co-located `.md`, `.types.ts`, `.test.ts`, `index.ts` barrel

### 5. Pipeline orchestrator (`src/core/pipeline/`)

- [x] Implement `createPipeline` with stage chaining, batch processing, `AbortSignal` propagation, and configurable `ErrorHandler`
- [x] Integrate `@suaveplan/streams` `ReadableStream`/`TransformStream`/`WritableStream` internally
- [x] Emit progress events via `@suaveplan/events` bus when configured
- [x] Write `pipeline.md`, `pipeline.types.ts`, `pipeline.test.ts`, `index.ts`

### 6. Retry adapter (`src/utils/retry-adapter.ts`)

- [x] Wrap source/sink IO methods with `@suaveplan/retry` when `retryOptions` is provided
- [x] Write `retry-adapter.md` and `retry-adapter.test.ts`

### 7. OTel instrumentation

- [x] Add spans, counters, histograms per `proposal.md` §New additions §3

### 8. Tests

- [x] Unit tests co-located with each feature module (100% coverage target)
- [x] `tests/integration.test.ts` — full pipeline: array source → validate → map → array sink
- [x] `tests/property.test.ts` — fast-check properties per `proposal.md` §5

### 9. Benchmarks (`src/__benchmarks__/pipeline.bench.ts`)

- [x] Implement benchmark cases per `proposal.md` §6

### 10. Docs

- [x] Every `.ts` implementation file has a sibling `.md` with all required sections (≥200 words, both examples)
- [x] Root `README.md` with feature table, usage examples, API reference links

### 11. Pre-publish gate

- [x] `bunx turbo lint typecheck test --filter=@suaveplan/etl` green
- [x] `bunx turbo test:coverage --filter=@suaveplan/etl` ≥ 100% on touched modules

### 12. Publish

- [x] `bun scripts/publish.ts --package @suaveplan/etl`

### 13. Archive

- [x] Fold spec delta into `openspec/specs/data/spec.md`
- [x] `mv openspec/changes/enterprise-etl openspec/archive/2026-04-24-enterprise-etl`