suaveplan/openspec/archive/2026-04-24-enterprise-etl/tasks.md

Tasks: enterprise-etl

• Change ID: enterprise-etl
• Package: @suaveplan/etl
• Date: 2026-04-24

Prerequisites (must be published to Verdaccio first)

• @suaveplan/error
• @suaveplan/types
• @suaveplan/streams
• @suaveplan/serialize
• @suaveplan/retry
• @suaveplan/events

Tasks

1. Scaffold package

• Create universal/data/etl/ directory tree per file layout in proposal.md
• Write package.json (name @suaveplan/etl, version 0.1.0, sideEffects: false, correct workspace deps and catalog externals)
• Write tsconfig.json extending @suaveplan/typescript-config/library.json
• Write vitest.config.ts using createTestConfig() from @suaveplan/testing/config
• Add package to root turbo.json pipeline if not already covered by glob

2. Error codes (src/error-codes.ts)

• Define EtlErrorCode union type
• Call defineCodes to register all ETL.* codes with severity and retryable flags
• Export registerEtlErrorCodes() factory
• Export EtlError extends SuaveplanError<EtlErrorCode>

3. Types (src/types/common.ts)

• Define and export all public interfaces: Pipeline, PipelineOptions, PipelineResult, PipelineStats, Source, Transform, TransformContext, Sink, ErrorHandler, ErrorAction
• Export SKIP unique symbol
• Write common.md

4. Stage implementations

• Implement Source interface and built-in sources (arraySource, generatorSource, streamSource) in src/stages/source/
• Implement Transform interface and built-in transforms (mapTransform, filterTransform, validateTransform, batchTransform, flatMapTransform) in src/stages/transform/
• Implement Sink interface and built-in sinks (arraySink, nullSink, transformSink) in src/stages/sink/
• Each feature file has co-located .md, .types.ts, .test.ts, index.ts barrel

5. Pipeline orchestrator (src/core/pipeline/)

• Implement createPipeline with stage chaining, batch processing, AbortSignal propagation, and configurable ErrorHandler
• Integrate @suaveplan/streams ReadableStream/TransformStream/WritableStream internally
• Emit progress events via @suaveplan/events bus when configured
• Write pipeline.md, pipeline.types.ts, pipeline.test.ts, index.ts

6. Retry adapter (src/utils/retry-adapter.ts)

• Wrap source/sink IO methods with @suaveplan/retry when retryOptions is provided
• Write retry-adapter.md and retry-adapter.test.ts

7. OTel instrumentation

• Add spans, counters, histograms per proposal.md §New additions §3

8. Tests

• Unit tests co-located with each feature module (100% coverage target)
• tests/integration.test.ts — full pipeline: array source → validate → map → array sink
• tests/property.test.ts — fast-check properties per proposal.md §5

9. Benchmarks (src/__benchmarks__/pipeline.bench.ts)

• Implement benchmark cases per proposal.md §6

10. Docs

• Every .ts implementation file has a sibling .md with all required sections (≥200 words, both examples)
• Root README.md with feature table, usage examples, API reference links

11. Pre-publish gate

• bunx turbo lint typecheck test --filter=@suaveplan/etl green
• bunx turbo test:coverage --filter=@suaveplan/etl ≥ 100% on touched modules

12. Publish

• bun scripts/publish.ts --package @suaveplan/etl

13. Archive

• Fold spec delta into openspec/specs/data/spec.md
• mv openspec/changes/enterprise-etl openspec/archive/2026-04-24-enterprise-etl