Files
2026-05-27 23:29:53 +01:00

10 KiB

Tasks — enterprise-streams-redesign

Every checkbox MUST reflect work actually done in this change. Retroactive ticking is forbidden.

0. Pre-flight (non-negotiable)

  • 0.1 Confirm clean working tree (git status reports nothing) and no in-flight worktrees with uncommitted work (git worktree list then git status in each)
  • 0.2 If any worktrees exist under .claude/worktrees/, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (git worktree remove) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
  • 0.3 Confirm the package set in MANIFEST.md matches find packages -maxdepth 3 -name package.json count. Drift here means a prior change archived without regenerating the manifest.

1. Spec authoring

  • 1.1 Read ../suaveplan/enterprise/streams.md end-to-end
  • 1.2 Read openspec/specs/streams/spec.md (does not exist on genesis — clean slate)
  • 1.3 Read openspec/AGENTS.md Sections 13 and 14
  • 1.4 Draft proposal.md from openspec/templates/proposal.md
  • 1.5 Fill in ## Source mapping table — every requirement cites a line range OR appears in the "no source line" table
  • 1.6 Draft specs/streams/spec.md from openspec/templates/spec-delta.md
  • 1.7 Every requirement in the spec delta carries a **Verified by:** line naming concrete it(...) titles

2. Implementation — substrate

  • 2.1 Reset packages/flow/streams/src/ layout — create new directory tree per the rename matrix in proposal.md; delete old -stream-suffixed directories
  • 2.2 src/errors/error/error.ts + error.md — StreamError base class + subclasses for every code
  • 2.3 src/error-codes.ts — defineCodes registry; named exports for each code constant
  • 2.4 src/types/common.ts + common.md — TransformOptions, StreamSource, StreamSink, DuplexStreamOptions, ReadableOptions, InstrumentOptions shared interfaces
  • 2.5 src/types/common.schemas.ts — Zod schemas for every public option type
  • 2.6 src/base/transform/transform.ts + .md + .types.ts + .test.ts + .spec-conformance.test.ts — BaseTransform abstract class
  • 2.7 src/manager/manager/manager.ts + .md + .types.ts + .test.ts — StreamManager
  • 2.8 src/factory/factory/factory.ts + .md + .types.ts + .test.ts + .spec-conformance.test.ts — TransformFactory registry
  • 2.9 src/primitives/duplex/duplex.ts + .md + .types.ts + .test.ts — BaseDuplexStream + createDuplexStream
  • 2.10 src/primitives/network/network.ts + .md + .types.ts + .test.ts — NetworkStreamImpl + createNetworkStream

3. Implementation — bundled BaseTransform subclasses (port + new)

  • 3.1 src/transforms/map/ — MapTransform + mapStream factory + .md + .test.ts (rename from map-stream)
  • 3.2 src/transforms/filter/ — FilterTransform + filterStream (rename)
  • 3.3 src/transforms/flat-map/ — FlatMapTransform + flatMapStream (rename)
  • 3.4 src/transforms/batch/ — BatchTransform + batchStream (rename)
  • 3.5 src/transforms/window/ — WindowTransform + windowStream (rename)
  • 3.6 src/transforms/throttle/ — ThrottleTransform + throttleStream (rename)
  • 3.7 src/transforms/limit/ — LimitTransform + limitStream (rename)
  • 3.8 src/transforms/dedup/ — DedupTransform + dedupStream (rename)
  • 3.9 src/transforms/tap/ — TapTransform + tapStream (rename)
  • 3.10 src/transforms/text-codec/ — TextCodecTransform + encodeTextStream/decodeTextStream (no rename)
  • 3.11 src/transforms/compress/ — CompressTransform + compressStream/decompressStream (no rename)
  • 3.12 src/transforms/crypto/ — CryptoTransform (port from main, depends on @suaveplan/crypto streaming AEAD)
  • 3.13 src/transforms/split-join/ — SplitJoinTransform (port from main)

4. Implementation — byte-framing (genesis-original)

  • 4.1 src/transforms/chunk-bytes/ — ChunkBytesTransform + chunkBytes factory
  • 4.2 src/transforms/coalesce-bytes/ — CoalesceBytesTransform + coalesceBytes factory
  • 4.3 src/transforms/sliding-bytes/ — SlidingBytesTransform + slidingBytes factory
  • 4.4 src/transforms/length-prefix-encode/ — LengthPrefixEncodeTransform + lengthPrefixEncode factory
  • 4.5 src/transforms/length-prefix-decode/ — LengthPrefixDecodeTransform + lengthPrefixDecode factory

5. Implementation — pipeline ops

  • 5.1 src/pipeline/builder/builder.ts — pipeline() + PipelineBuilder (rename from pipeline-builder)
  • 5.2 src/pipeline/fan-out/ — preserved as-is (no rename)
  • 5.3 src/pipeline/merge/ — preserved as-is (no rename)
  • 5.4 src/pipeline/partition/ — partition() single-pass split
  • 5.5 src/pipeline/route/ — route() + routeBy() N-way split
  • 5.6 src/pipeline/zip/ — zip() sample-aligned merge
  • 5.7 src/pipeline/concat/ — concat() sequential drain
  • 5.8 src/pipeline/merge-ordered/ — mergeOrdered() k-way merge
  • 5.9 src/pipeline/safe-branch/ — safeBranch() error isolation

6. Implementation — sources, sinks, utils, instrumentation

  • 6.1 src/readable/from-array/, src/readable/from-bytes/, src/readable/from-generator/ — preserved
  • 6.2 src/sinks/to-array/, src/sinks/to-bytes/, src/sinks/for-each/ — preserved
  • 6.3 src/sinks/to-async-iterable/ — new
  • 6.4 src/instrument/transform/ — preserved (rename from instrument-transform)
  • 6.5 src/node/adapters/ — preserved (rename from node-adapters.ts to feature dir)
  • 6.6 src/utils/buffer/get-data-from-file/, src/utils/buffer/buffer-to-uint8/, src/utils/buffer/uint8-slice/ — port from main
  • 6.7 src/utils/stream/round-trip/ (streamToUint8Array + uint8ArrayToStream), src/utils/stream/create-data-stream/, src/utils/stream/process-data-stream/, src/utils/stream/validate-integrity/ — port from main

7. Tests

  • 7.1 Unit tests at every src/<feature>/<feature>.test.ts covering every scenario in the spec delta
  • 7.2 Spec-conformance tests at *.spec-conformance.test.ts for type-level / observability / lifecycle scenarios
  • 7.3 tests/integration.test.ts — end-to-end pipelines (encrypt + length-prefix + decrypt round-trip; partition + safe-branch + merge composition)
  • 7.4 tests/property.test.ts — fast-check invariants (chunkBytes byte-conservation, partition order preservation, zip/unzip round-trip, mergeOrdered sortedness)
  • 7.5 tests/dep-graph.test.ts — workspace-dep audit, side-effect audit, no node:fs in browser entry
  • 7.6 tests/tree-shake.test.ts — sideEffects: false round-trip
  • 7.7 tests/error-coverage.test.ts — every StreamErrorCode thrown by at least one production path
  • 7.8 Every it(...) carries { timeout: 5000 } (or 30000 for genuinely async)

8. Documentation

  • 8.1 Sibling .md for every implementation .ts (≥200 words, basic + advanced examples, full API reference)
  • 8.2 README.md at package root rewritten with all new modules linked under ## Modules
  • 8.3 Update CLAUDE.md streams package listing to reflect new surface (BaseTransform/StreamManager/TransformFactory/CryptoTransform/etc.)

9. Package configuration

  • 9.1 Update package.json — add @suaveplan/crypto (workspace:^), @suaveplan/events (workspace:^); keep @suaveplan/error, @suaveplan/types; ensure tslib, zod, @opentelemetry/api via catalog
  • 9.2 Update package.json exports — one subpath per feature module per openspec/AGENTS.md Section 6
  • 9.3 Confirm sideEffects: false and tree-shake guard test passes
  • 9.4 vitest.config.ts thresholds remain 100/100/100/100 with provider: 'istanbul'

10. Consumer migration

  • 10.1 grep -rln "@suaveplan/streams/transforms/.*-stream" packages/ — update any internal subpath consumer to new paths
  • 10.2 grep -rln "@suaveplan/streams/pipeline/pipeline-builder" packages/ — update to pipeline/builder
  • 10.3 grep -rln "@suaveplan/streams/instrument/instrument-transform" packages/ — update to instrument/transform
  • 10.4 grep -rln "@suaveplan/streams/node/node-adapters" packages/ — update to node/adapters/adapters
  • 10.5 grep -rln "@suaveplan/streams/errors/stream-error" packages/ — update to errors/error/error

11. Pre-archive verification (the 5-gate triple)

  • 11.1 bun run check — Biome 0 errors / 0 warnings / 0 info across the whole monorepo
  • 11.2 bunx turbo lint typecheck test --filter=@suaveplan/streams — green
  • 11.3 bunx turbo test:coverage --filter=@suaveplan/streams — 100/100/100/100
  • 11.4 bun scripts/openspec-conformance.ts @suaveplan/streams — exit 0, no error findings
  • 11.5 bunx turbo build --filter=@suaveplan/streams — green

12. Pre-archive 7-row checklist (AGENTS.md Section 13)

  • 12.1 Generics threaded; expectTypeOf assertions on BaseTransform.process<T>, TransformFactory.create<T> literal-type narrowing, route() Record<K, …> keying
  • 12.2 Zod schemas wired at every declared boundary (TransformOptions, ChunkQueueOptions-style option bundles, public factories)
  • 12.3 Every error code in error-codes.ts thrown by at least one production path (verified by tests/error-coverage.test.ts)
  • 12.4 Strict-mode behaviors tested: onUnknownKey: 'drop' | 'error', tail: 'emit' | 'pad' | 'drop', onShortInput: 'close' | 'error' | 'pad'
  • 12.5 OTel instruments emitted from production paths (stream.chunks_total, stream.transform_duration_ms) and asserted with a real in-memory MeterProvider
  • 12.6 Lifecycle dedup tested: dispose() idempotent on BaseTransform and CryptoTransform; clear() idempotent on StreamManager
  • 12.7 No retroactive tick — every box above reflects work done in this change

13. Archive

  • 13.1 Fold spec delta into openspec/specs/streams/spec.md (creates the folded spec for the first time)
  • 13.2 Move openspec/changes/enterprise-streams-redesign/ → openspec/archive/<yyyy-mm-dd>-enterprise-streams-redesign/
  • 13.3 Refresh MANIFEST.md via bun scripts/folder-reorg/generate-manifest.ts (deps changed: @suaveplan/crypto, @suaveplan/events added)