suaveplan/openspec/changes/pagination-streaming/tasks.md

# Tasks — Pagination streaming

> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick boxes for work that was already in main when this change opened.

## 0. Pre-flight (non-negotiable)

- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.

## Audit Note (2026-05-02)

**Minimal progress** — DEFERRED, do not archive.

- No `packages/data/pagination/src/streaming/` directory exists. Existing dirs: `cursor`, `keyset`, `offset`, `helpers`, `adapters`, `tokens`, `errors`, `otel`, `schemas`, `types`.
- `grep -rln "streamPaginated\|stream-pagination\|collectStream\|countStream\|takeFromStream\|filterStream\|mapStream" packages/data/pagination/src/` returns zero hits.
- `bunx turbo lint typecheck test --filter=@suaveplan/pagination` is GREEN — but the streaming module is not implemented.
- All 57 tasks remain: `streamPaginated<T>` AsyncIterable, five operator helpers (`collectStream`, `countStream`, `takeFromStream`, `filterStream`, `mapStream`), `DEFAULT_STREAM_PAGE_SIZE`, AbortSignal contract, subpath export, sibling `.md`, full test matrix.

## 1. Spec authoring

- [ ] 1.1 Read `../suaveplan/enterprise/pagination.md` end-to-end
- [ ] 1.2 Read `openspec/specs/pagination/spec.md` (already exists — confirm PG-12 AbortSignal contract still aligned)
- [ ] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
- [ ] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
- [ ] 1.5 Fill in `## Source mapping` table — every requirement cites a line range OR appears in the "no source line" table with justification
- [ ] 1.6 Draft `specs/pagination/spec.md` from `openspec/templates/spec-delta.md` — note: this is a delta against the existing canonical spec, so add as ADDED Requirements (PG-14, PG-15) numbered after the current PG-13
- [ ] 1.7 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles

## 2. Implementation — `streaming/stream-pagination`

- [ ] 2.1 Create `packages/data/pagination/src/streaming/stream-pagination.ts` exporting `streamPaginated<T>(adapter, options?): AsyncIterable<T>`
- [ ] 2.2 Default `pageSize` to 50 (constant `DEFAULT_STREAM_PAGE_SIZE`); export the constant from `constants/index.ts`
- [ ] 2.3 Honour the one-page-in-memory invariant — lazy-init on first `next()`; fetch next page only when current page exhausted; mark exhausted when `currentPage.length < pageSize`
- [ ] 2.4 Forward `options.filter` and `options.sort` to `adapter.fetchPage(offset, pageSize, sort, filter)` byte-for-byte with upstream
- [ ] 2.5 Honour `options.signal` — check `signal.aborted` before each `fetchPage` AND before yielding each item; on abort throw `PaginationError(PAGINATION.ABORTED)`
- [ ] 2.6 Add `StreamPaginationOptions` type (`pageSize`, `filter`, `sort`, `signal`) to `types/pagination.types.ts`
- [ ] 2.7 Add `streaming/stream-pagination.md` sibling doc (≥200 words, basic + advanced examples)

## 3. Implementation — operator helpers

- [ ] 3.1 Add `collectStream<T>(iterable): Promise<T[]>` — drains; verify it does NOT short-circuit
- [ ] 3.2 Add `countStream<T>(iterable): Promise<number>` — counts without retaining
- [ ] 3.3 Add `takeFromStream<T>(iterable, count): Promise<T[]>` — short-circuits once `items.length >= count`
- [ ] 3.4 Add `filterStream<T>(iterable, predicate): AsyncIterable<T>` — lazy filter passthrough
- [ ] 3.5 Add `mapStream<T, U>(iterable, mapper): AsyncIterable<U>` — lazy map passthrough
- [ ] 3.6 All five helpers honour `AbortSignal` indirectly via the upstream iterable (no new signal parameter; cancellation propagates through the `for await` chain)

## 4. Subpath exports + barrel

- [ ] 4.1 Add `"./streaming/stream-pagination"` to `packages/data/pagination/package.json` exports map (matching existing per-feature subpath pattern)
- [ ] 4.2 Re-export `streamPaginated`, `collectStream`, `countStream`, `takeFromStream`, `filterStream`, `mapStream`, `StreamPaginationOptions`, `DEFAULT_STREAM_PAGE_SIZE` from `src/index.ts`

## 5. Tests

- [ ] 5.1 `streaming/stream-pagination.test.ts::"streams items from MemoryPaginationAdapter"` — basic round-trip over 100 items with `pageSize=10`
- [ ] 5.2 `streaming/stream-pagination.test.ts::"defaults pageSize to 50 when omitted"` — instrument `fetchPage` and assert called with `pageSize=50`
- [ ] 5.3 `streaming/stream-pagination.test.ts::"holds at most one page in memory"` — instrument iterator state; verify `currentPage` size ≤ `pageSize` at every step
- [ ] 5.4 `streaming/stream-pagination.test.ts::"forwards filter and sort to adapter.fetchPage"` — assert call args
- [ ] 5.5 `streaming/stream-pagination.test.ts::"is lazy — no fetch before first next()"` — instrument; verify zero calls before first `next()`
- [ ] 5.6 `streaming/stream-pagination.test.ts::"stops when partial page returned"` — verify exhausted-on-partial behavior
- [ ] 5.7 `streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED before first fetchPage when signal already aborted"` — verify `fetchPage` is NOT called
- [ ] 5.8 `streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED on next() after signal abort between yields"` — abort mid-iteration
- [ ] 5.9 `streaming/stream-pagination.spec-conformance.test.ts::"propagates rejection when signal aborts in-flight fetchPage"` — adapter aware of signal rejects naturally
- [ ] 5.10 `streaming/stream-pagination.spec-conformance.test.ts::"pins StreamPaginationOptions shape via expectTypeOf"` — type-level assertion
- [ ] 5.11 `streaming/stream-pagination.test.ts::"collectStream drains all items"` — happy path
- [ ] 5.12 `streaming/stream-pagination.test.ts::"countStream returns total without retention"` — verify result is correct count
- [ ] 5.13 `streaming/stream-pagination.test.ts::"takeFromStream short-circuits at count"` — instrument fetches; verify only enough pages fetched to satisfy take
- [ ] 5.14 `streaming/stream-pagination.test.ts::"filterStream is lazy and passthrough"` — verify items skipped don't pass through
- [ ] 5.15 `streaming/stream-pagination.test.ts::"mapStream is lazy and passthrough"` — verify mapper called once per yielded item
- [ ] 5.16 Every `it(...)` in 5.1–5.15 carries `{ timeout: 5000 }`

## 6. Documentation

- [ ] 6.1 Sibling `.md` for `stream-pagination.ts` (≥200 words, basic + advanced examples)
- [ ] 6.2 Update `packages/data/pagination/README.md` Modules section — add link to streaming module; add Quick-start snippet showing `streamPaginated` + `for await`
- [ ] 6.3 Update `CLAUDE.md` package listing line for `@suaveplan/pagination` to mention streaming

## 7. Pre-archive verification (the 5-gate triple)

- [ ] 7.1 `bun run check` — Biome 0 errors / 0 warnings / 0 info across the whole monorepo
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/pagination` — green
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/pagination` — 100/100/100/100
- [ ] 7.4 `bun scripts/openspec-conformance.ts @suaveplan/pagination` — exit 0, no `error` findings
- [ ] 7.5 `bunx turbo build --filter=@suaveplan/pagination` — green
- [ ] 7.6 Dependent sweep: `bunx turbo test --filter=...@suaveplan/pagination` (covers `@suaveplan/search`)

## 8. Pre-archive 7-row checklist (AGENTS.md Section 13)

- [ ] 8.1 Generics threaded through `streamPaginated<T>`, `mapStream<T, U>`, `filterStream<T>`, etc.
- [ ] 8.2 Zod schemas wired at the `StreamPaginationOptions` boundary if exposed across a network edge (likely no — purely in-memory contract)
- [ ] 8.3 `PAGINATION.ABORTED` thrown by at least one production path (the streaming helper)
- [ ] 8.4 Strict-mode behaviors tested both on and off (signal supplied vs absent)
- [ ] 8.5 OTel instruments emitted (page-fetch counter, page-fetch duration histogram) and asserted with a real in-memory `MeterProvider`
- [ ] 8.6 Lifecycle dedup not applicable here — streaming is single-iterator instances, no shared registry
- [ ] 8.7 No retroactive tick — every box above reflects work done in this change

## 9. Archive

- [ ] 9.1 Fold spec delta into `openspec/specs/pagination/spec.md` — add as PG-14 (streaming pagination contract) and PG-15 (streaming operator helpers)
- [ ] 9.2 Move `openspec/changes/pagination-streaming/` → `openspec/archive/<yyyy-mm-dd>-pagination-streaming/`
- [ ] 9.3 Refresh `MANIFEST.md` if package set or deps changed (no expected change here)