107 lines
7.4 KiB
Markdown
107 lines
7.4 KiB
Markdown
<!-- markdownlint-disable MD024 -->
|
|
|
|
# Capability Spec Delta: pagination (pagination-streaming)
|
|
|
|
**Package**: `@suaveplan/pagination`
|
|
**Status**: PROPOSED
|
|
|
|
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. (See `openspec/AGENTS.md` Section 13.)
|
|
>
|
|
> Adds PG-14 and PG-15 to the canonical capability spec at `openspec/specs/pagination/spec.md`.
|
|
|
|
## ADDED Requirements
|
|
|
|
### Requirement: Streaming pagination iterable over PaginationAdapter
|
|
|
|
The package SHALL export `streamPaginated<T>(adapter: PaginationAdapter<T>, options?: StreamPaginationOptions): AsyncIterable<T>`. The returned iterable SHALL drain `adapter.fetchPage(offset, pageSize, sort?, filter?)` lazily — fetching the next page only when the current page is exhausted — so memory use is bounded by `pageSize`. `StreamPaginationOptions` SHALL be `{ pageSize?: number; filter?: FilterConfig; sort?: SortConfig; signal?: AbortSignal }`. When `pageSize` is omitted, the iterable SHALL use `DEFAULT_STREAM_PAGE_SIZE` (50). When `signal` is supplied and aborted, `next()` SHALL throw `PaginationError` with code `PAGINATION.ABORTED` without calling `adapter.fetchPage`.
|
|
|
|
#### Scenario: drains items from MemoryPaginationAdapter
|
|
|
|
- **WHEN** `streamPaginated(adapter, { pageSize: 10 })` is iterated and `adapter` holds 100 items
|
|
- **THEN** the consumer receives all 100 items in order AND `adapter.fetchPage` is called exactly 11 times (10 full pages + one empty/partial page to detect end)
|
|
|
|
#### Scenario: defaults pageSize to 50 when omitted
|
|
|
|
- **WHEN** `streamPaginated(adapter)` is iterated with no options
|
|
- **THEN** every call to `adapter.fetchPage` is made with `pageSize: 50`
|
|
|
|
#### Scenario: holds at most one page in memory
|
|
|
|
- **WHEN** the iterable is mid-drain and instrumentation observes the internal `currentPage` array length
|
|
- **THEN** the length is always less than or equal to `pageSize`
|
|
|
|
#### Scenario: lazy — no fetch before first next()
|
|
|
|
- **WHEN** `streamPaginated(adapter)` is constructed but `Symbol.asyncIterator` is not invoked
|
|
- **THEN** `adapter.fetchPage` has been called zero times
|
|
|
|
#### Scenario: stops when partial page returned
|
|
|
|
- **WHEN** the adapter returns a page whose length is less than `pageSize`
|
|
- **THEN** the iterable yields the remaining items and then completes (no further `fetchPage` calls)
|
|
|
|
#### Scenario: forwards filter and sort to adapter.fetchPage
|
|
|
|
- **WHEN** `streamPaginated(adapter, { pageSize: 10, filter: { status: "active" }, sort: { field: "id", direction: "asc" } })` is iterated
|
|
- **THEN** every `adapter.fetchPage` call is made with the same `filter` and `sort` arguments byte-for-byte
|
|
|
|
#### Scenario: signal already aborted before first next()
|
|
|
|
- **WHEN** `signal.aborted === true` at iterator construction and `next()` is called
|
|
- **THEN** the call throws `PaginationError` with code `PAGINATION.ABORTED` AND `adapter.fetchPage` is NEVER called
|
|
|
|
#### Scenario: signal aborted between yields
|
|
|
|
- **WHEN** the consumer aborts the signal between two `next()` calls mid-page
|
|
- **THEN** the next `next()` call throws `PaginationError` with code `PAGINATION.ABORTED`
|
|
|
|
**Verified by:** `packages/data/pagination/src/streaming/stream-pagination.test.ts::"streams items from MemoryPaginationAdapter"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"defaults pageSize to 50 when omitted"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"holds at most one page in memory"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"is lazy — no fetch before first next()"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"stops when partial page returned"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"forwards filter and sort to adapter.fetchPage"` AND `packages/data/pagination/src/streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED before first fetchPage when signal already aborted"` AND `packages/data/pagination/src/streaming/stream-pagination.spec-conformance.test.ts::"throws PAGINATION.ABORTED on next() after signal abort between yields"`
|
|
|
|
---
|
|
|
|
### Requirement: Streaming operator helpers — collect, count, take, filter, map
|
|
|
|
The package SHALL export five helper functions over `AsyncIterable<T>` produced by `streamPaginated`:
|
|
|
|
- `collectStream<T>(iterable: AsyncIterable<T>): Promise<T[]>` — drains the entire iterable into an array.
|
|
- `countStream<T>(iterable: AsyncIterable<T>): Promise<number>` — counts items without retaining them.
|
|
- `takeFromStream<T>(iterable: AsyncIterable<T>, count: number): Promise<T[]>` — collects the first `count` items, short-circuiting once enough are gathered.
|
|
- `filterStream<T>(iterable: AsyncIterable<T>, predicate: (item: T) => boolean): AsyncIterable<T>` — produces a new lazy iterable yielding only items matching `predicate`.
|
|
- `mapStream<T, U>(iterable: AsyncIterable<T>, mapper: (item: T) => U): AsyncIterable<U>` — produces a new lazy iterable yielding `mapper(item)` for each input item.
|
|
|
|
`takeFromStream` SHALL stop pulling items as soon as `count` items have been collected — the underlying iterable's pagination SHALL NOT continue beyond what was needed. `filterStream` and `mapStream` SHALL be lazy — they SHALL NOT pre-fetch or pre-compute beyond what consumers `next()` request. AbortSignal cancellation propagates naturally through the `for await` chain — these helpers do not introduce a separate signal parameter.
|
|
|
|
#### Scenario: collectStream drains all items
|
|
|
|
- **WHEN** `collectStream(streamPaginated(adapter))` is awaited against a 100-item adapter
|
|
- **THEN** the result is an array of 100 items in order
|
|
|
|
#### Scenario: countStream returns total without retention
|
|
|
|
- **WHEN** `countStream(streamPaginated(adapter))` is awaited against a 100-item adapter
|
|
- **THEN** the result is `100` AND no array of size 100 is held in memory at any point
|
|
|
|
#### Scenario: takeFromStream short-circuits at count
|
|
|
|
- **WHEN** `takeFromStream(streamPaginated(adapter, { pageSize: 10 }), 5)` is awaited against a 100-item adapter
|
|
- **THEN** the result is the first 5 items AND `adapter.fetchPage` is called exactly once
|
|
|
|
#### Scenario: takeFromStream returns empty for count <= 0
|
|
|
|
- **WHEN** `takeFromStream(iter, 0)` is awaited
|
|
- **THEN** the result is `[]` AND `iter` is not advanced at all
|
|
|
|
#### Scenario: filterStream is lazy
|
|
|
|
- **WHEN** `filterStream(streamPaginated(adapter), (x) => x.id % 2 === 0)` is iterated and only the first matching item is consumed
|
|
- **THEN** the underlying adapter is queried at most enough times to find that one item — not the entire dataset
|
|
|
|
#### Scenario: mapStream is lazy
|
|
|
|
- **WHEN** `mapStream(streamPaginated(adapter), (x) => x.id)` is iterated for two items
|
|
- **THEN** the mapper is called exactly twice — not once per item in the underlying adapter
|
|
|
|
**Verified by:** `packages/data/pagination/src/streaming/stream-pagination.test.ts::"collectStream drains all items"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"countStream returns total without retention"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"takeFromStream short-circuits at count"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"takeFromStream returns empty for count <= 0"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"filterStream is lazy and passthrough"` AND `packages/data/pagination/src/streaming/stream-pagination.test.ts::"mapStream is lazy and passthrough"`
|
|
|
|
<!-- markdownlint-enable MD024 -->
|