suaveplan/openspec/changes/pagination-streaming/tasks.md

Tasks — Pagination streaming

Every checkbox MUST reflect work actually done in this change. Do not retroactively tick boxes for work that was already in main when this change opened.

0. Pre-flight (non-negotiable)

• 0.1 Confirm clean working tree (git status reports nothing) and no in-flight worktrees with uncommitted work (git worktree list then git status in each)
• 0.2 If any worktrees exist under .claude/worktrees/, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (git worktree remove) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
• 0.3 Confirm the package set in MANIFEST.md matches find packages -maxdepth 3 -name package.json count. Drift here means a prior change archived without regenerating the manifest.

Audit Note (2026-05-02)

Minimal progress — DEFERRED, do not archive.

• No packages/data/pagination/src/streaming/ directory exists. Existing dirs: cursor, keyset, offset, helpers, adapters, tokens, errors, otel, schemas, types.
• grep -rln "streamPaginated\|stream-pagination\|collectStream\|countStream\|takeFromStream\|filterStream\|mapStream" packages/data/pagination/src/ returns zero hits.
• bunx turbo lint typecheck test --filter=@suaveplan/pagination is GREEN — but the streaming module is not implemented.
• All 57 tasks remain: streamPaginated<T> AsyncIterable, five operator helpers (collectStream, countStream, takeFromStream, filterStream, mapStream), DEFAULT_STREAM_PAGE_SIZE, AbortSignal contract, subpath export, sibling .md, full test matrix.

1. Spec authoring

• 1.1 Read ../suaveplan/enterprise/pagination.md end-to-end
• 1.2 Read openspec/specs/pagination/spec.md (already exists — confirm PG-12 AbortSignal contract still aligned)
• 1.3 Read openspec/AGENTS.md Sections 13 and 14
• 1.4 Draft proposal.md from openspec/templates/proposal.md
• 1.5 Fill in ## Source mapping table — every requirement cites a line range OR appears in the "no source line" table with justification
• 1.6 Draft specs/pagination/spec.md from openspec/templates/spec-delta.md — note: this is a delta against the existing canonical spec, so add as ADDED Requirements (PG-14, PG-15) numbered after the current PG-13
• 1.7 Every requirement in the spec delta carries a **Verified by:** line naming concrete it(...) titles

2. Implementation — streaming/stream-pagination

• 2.1 Create packages/data/pagination/src/streaming/stream-pagination.ts exporting streamPaginated<T>(adapter, options?): AsyncIterable<T>
• 2.2 Default pageSize to 50 (constant DEFAULT_STREAM_PAGE_SIZE); export the constant from constants/index.ts
• 2.3 Honour the one-page-in-memory invariant — lazy-init on first next(); fetch next page only when current page exhausted; mark exhausted when currentPage.length < pageSize
• 2.4 Forward options.filter and options.sort to adapter.fetchPage(offset, pageSize, sort, filter) byte-for-byte with upstream
• 2.5 Honour options.signal — check signal.aborted before each fetchPage AND before yielding each item; on abort throw PaginationError(PAGINATION.ABORTED)
• 2.6 Add StreamPaginationOptions type (pageSize, filter, sort, signal) to types/pagination.types.ts
• 2.7 Add streaming/stream-pagination.md sibling doc (≥200 words, basic + advanced examples)

3. Implementation — operator helpers

• 3.1 Add collectStream<T>(iterable): Promise<T[]> — drains; verify it does NOT short-circuit
• 3.2 Add countStream<T>(iterable): Promise<number> — counts without retaining
• 3.3 Add takeFromStream<T>(iterable, count): Promise<T[]> — short-circuits once items.length >= count
• 3.4 Add filterStream<T>(iterable, predicate): AsyncIterable<T> — lazy filter passthrough
• 3.5 Add mapStream<T, U>(iterable, mapper): AsyncIterable<U> — lazy map passthrough
• 3.6 All five helpers honour AbortSignal indirectly via the upstream iterable (no new signal parameter; cancellation propagates through the for await chain)

4. Subpath exports + barrel

• 4.1 Add "./streaming/stream-pagination" to packages/data/pagination/package.json exports map (matching existing per-feature subpath pattern)
• 4.2 Re-export streamPaginated, collectStream, countStream, takeFromStream, filterStream, mapStream, StreamPaginationOptions, DEFAULT_STREAM_PAGE_SIZE from src/index.ts

5. Tests

• 5.1 streaming/stream-pagination.test.ts::"streams items from MemoryPaginationAdapter" — basic round-trip over 100 items with pageSize=10
• 5.2 streaming/stream-pagination.test.ts::"defaults pageSize to 50 when omitted" — instrument fetchPage and assert called with pageSize=50
• 5.3 streaming/stream-pagination.test.ts::"holds at most one page in memory" — instrument iterator state; verify currentPage size ≤ pageSize at every step
• 5.4 streaming/stream-pagination.test.ts::"forwards filter and sort to adapter.fetchPage" — assert call args
• 5.5 streaming/stream-pagination.test.ts::"is lazy — no fetch before first next()" — instrument; verify zero calls before first next()
• 5.6 streaming/stream-pagination.test.ts::"stops when partial page returned" — verify exhausted-on-partial behavior
• 5.7 streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED before first fetchPage when signal already aborted" — verify fetchPage is NOT called
• 5.8 streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED on next() after signal abort between yields" — abort mid-iteration
• 5.9 streaming/stream-pagination.spec-conformance.test.ts::"propagates rejection when signal aborts in-flight fetchPage" — adapter aware of signal rejects naturally
• 5.10 streaming/stream-pagination.spec-conformance.test.ts::"pins StreamPaginationOptions shape via expectTypeOf" — type-level assertion
• 5.11 streaming/stream-pagination.test.ts::"collectStream drains all items" — happy path
• 5.12 streaming/stream-pagination.test.ts::"countStream returns total without retention" — verify result is correct count
• 5.13 streaming/stream-pagination.test.ts::"takeFromStream short-circuits at count" — instrument fetches; verify only enough pages fetched to satisfy take
• 5.14 streaming/stream-pagination.test.ts::"filterStream is lazy and passthrough" — verify items skipped don't pass through
• 5.15 streaming/stream-pagination.test.ts::"mapStream is lazy and passthrough" — verify mapper called once per yielded item
• 5.16 Every it(...) in 5.1–5.15 carries { timeout: 5000 }

6. Documentation

• 6.1 Sibling .md for stream-pagination.ts (≥200 words, basic + advanced examples)
• 6.2 Update packages/data/pagination/README.md Modules section — add link to streaming module; add Quick-start snippet showing streamPaginated + for await
• 6.3 Update CLAUDE.md package listing line for @suaveplan/pagination to mention streaming

7. Pre-archive verification (the 5-gate triple)

• 7.1 bun run check — Biome 0 errors / 0 warnings / 0 info across the whole monorepo
• 7.2 bunx turbo lint typecheck test --filter=@suaveplan/pagination — green
• 7.3 bunx turbo test:coverage --filter=@suaveplan/pagination — 100/100/100/100
• 7.4 bun scripts/openspec-conformance.ts @suaveplan/pagination — exit 0, no error findings
• 7.5 bunx turbo build --filter=@suaveplan/pagination — green
• 7.6 Dependent sweep: bunx turbo test --filter=...@suaveplan/pagination (covers @suaveplan/search)

8. Pre-archive 7-row checklist (AGENTS.md Section 13)

• 8.1 Generics threaded through streamPaginated<T>, mapStream<T, U>, filterStream<T>, etc.
• 8.2 Zod schemas wired at the StreamPaginationOptions boundary if exposed across a network edge (likely no — purely in-memory contract)
• 8.3 PAGINATION.ABORTED thrown by at least one production path (the streaming helper)
• 8.4 Strict-mode behaviors tested both on and off (signal supplied vs absent)
• 8.5 OTel instruments emitted (page-fetch counter, page-fetch duration histogram) and asserted with a real in-memory MeterProvider
• 8.6 Lifecycle dedup not applicable here — streaming is single-iterator instances, no shared registry
• 8.7 No retroactive tick — every box above reflects work done in this change

9. Archive

• 9.1 Fold spec delta into openspec/specs/pagination/spec.md — add as PG-14 (streaming pagination contract) and PG-15 (streaming operator helpers)
• 9.2 Move openspec/changes/pagination-streaming/ → openspec/archive/<yyyy-mm-dd>-pagination-streaming/
• 9.3 Refresh MANIFEST.md if package set or deps changed (no expected change here)