first commit
This commit is contained in:
113
docs/foundation/flow/streams/src/pipeline/concat/concat.md
Normal file
113
docs/foundation/flow/streams/src/pipeline/concat/concat.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# concat
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`concat` joins multiple `ReadableStream<T>` inputs end-to-end into a single
|
||||
`ReadableStream<T>`, draining each source to completion before pulling from the
|
||||
next. This is the sequential counterpart to `merge`: where `merge` races readers
|
||||
and emits chunks in arrival order, `concat` preserves the input ordering
|
||||
deterministically. Use it when downstream consumers depend on the relative
|
||||
order of upstream chunks — for example, replaying a header stream followed by a
|
||||
body stream, or stitching paginated results into one contiguous flow.
|
||||
|
||||
## Features
|
||||
|
||||
- Deterministic, sequential ordering across stream boundaries.
|
||||
- Lazy: the next source is only acquired (its reader pulled) once the previous
|
||||
one has reported `done`.
|
||||
- Backpressure-aware: pulls from the current source only when the output is
|
||||
being read.
|
||||
- Error propagation: any input error halts the concatenation immediately and is
|
||||
surfaced as a `StreamPipeFailedError` (`STREAM.PIPE_FAILED`) with the original
|
||||
error attached as `cause`. Unread later sources are never touched.
|
||||
- Cancellation: cancelling the output cancels the currently-reading source.
|
||||
Sources that have not yet been acquired are left untouched (the caller still
|
||||
owns them).
|
||||
- Fast paths for empty input (`[]` returns an immediately-closed stream) and
|
||||
single input (returned verbatim, identity).
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { concat } from "@suaveplan/streams/pipeline/concat/concat";
|
||||
import { fromArray } from "@suaveplan/streams/readable/from-array/from-array";
|
||||
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
|
||||
|
||||
const head = fromArray([1, 2, 3]);
|
||||
const tail = fromArray([4, 5, 6]);
|
||||
|
||||
const all = await toArray(concat([head, tail]));
|
||||
// → [1, 2, 3, 4, 5, 6]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Concatenating paginated fetch results, surfacing transport errors:
|
||||
|
||||
```ts
|
||||
import { concat } from "@suaveplan/streams/pipeline/concat/concat";
|
||||
import { StreamPipeFailedError } from "@suaveplan/streams/errors/stream-error";
|
||||
|
||||
function pageStream(pageNumber: number): ReadableStream<Uint8Array> {
|
||||
return new ReadableStream({
|
||||
async pull(controller) {
|
||||
const res = await fetch(`/api/items?page=${pageNumber}`);
|
||||
if (!res.ok) throw new Error(`page ${pageNumber} failed`);
|
||||
const buf = new Uint8Array(await res.arrayBuffer());
|
||||
controller.enqueue(buf);
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const stitched = concat([pageStream(1), pageStream(2), pageStream(3)]);
|
||||
|
||||
try {
|
||||
const reader = stitched.getReader();
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
process(value);
|
||||
}
|
||||
} catch (err) {
|
||||
if (err instanceof StreamPipeFailedError) {
|
||||
// err.cause holds the original transport / page error
|
||||
report(err);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `concat<T>(streams: ReadableStream<T>[]): ReadableStream<T>`
|
||||
|
||||
- **Parameters**
|
||||
- `streams` — an ordered array of `ReadableStream<T>` to drain in sequence.
|
||||
- **Returns** — a new `ReadableStream<T>` that emits every chunk from
|
||||
`streams[0]`, then every chunk from `streams[1]`, and so on, closing once the
|
||||
final source closes. For `streams.length === 0`, returns an
|
||||
immediately-closed stream. For `streams.length === 1`, returns
|
||||
`streams[0]` unchanged (identity).
|
||||
- **Throws (downstream)** — `StreamPipeFailedError` with stable code
|
||||
`STREAM.PIPE_FAILED` whenever any input source errors. The original error is
|
||||
attached as `cause`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The implementation owns at most one reader at a time. After a source reports
|
||||
`done`, its reader is `releaseLock`-ed before the next source's reader is
|
||||
acquired. This avoids holding multiple locks open and keeps memory bounded.
|
||||
- Sources are acquired lazily inside `pull`. Streams that the consumer never
|
||||
reaches (e.g., because of an early error or cancellation) are never read,
|
||||
which is intentional — concat must not eagerly drain inputs.
|
||||
- Cancellation only cancels the currently-acquired source. Un-acquired sources
|
||||
are still owned by the caller and may be reused or cancelled independently.
|
||||
- Errors are wrapped, never re-thrown raw. This guarantees that consumers can
|
||||
reliably pattern-match on `StreamPipeFailedError` / `STREAM.PIPE_FAILED` for
|
||||
any concat-mediated failure, regardless of the underlying cause.
|
||||
- The single-source identity fast path is observable via reference equality —
|
||||
consumers can detect it with `concat([s]) === s`.
|
||||
Reference in New Issue
Block a user