Initial Commit

packages/foundation/flow/streams/src/pipeline/concat/concat.md

@@ -0,0 +1,113 @@
+# concat
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+`concat` joins multiple `ReadableStream<T>` inputs end-to-end into a single
+`ReadableStream<T>`, draining each source to completion before pulling from the
+next. This is the sequential counterpart to `merge`: where `merge` races readers
+and emits chunks in arrival order, `concat` preserves the input ordering
+deterministically. Use it when downstream consumers depend on the relative
+order of upstream chunks — for example, replaying a header stream followed by a
+body stream, or stitching paginated results into one contiguous flow.
+
+## Features
+
+- Deterministic, sequential ordering across stream boundaries.
+- Lazy: the next source is only acquired (its reader pulled) once the previous
+  one has reported `done`.
+- Backpressure-aware: pulls from the current source only when the output is
+  being read.
+- Error propagation: any input error halts the concatenation immediately and is
+  surfaced as a `StreamPipeFailedError` (`STREAM.PIPE_FAILED`) with the original
+  error attached as `cause`. Unread later sources are never touched.
+- Cancellation: cancelling the output cancels the currently-reading source.
+  Sources that have not yet been acquired are left untouched (the caller still
+  owns them).
+- Fast paths for empty input (`[]` returns an immediately-closed stream) and
+  single input (returned verbatim, identity).
+
+## Usage: Basic Example
+
+```ts
+import { concat } from "@suaveplan/streams/pipeline/concat/concat";
+import { fromArray } from "@suaveplan/streams/readable/from-array/from-array";
+import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
+
+const head = fromArray([1, 2, 3]);
+const tail = fromArray([4, 5, 6]);
+
+const all = await toArray(concat([head, tail]));
+// → [1, 2, 3, 4, 5, 6]
+```
+
+## Usage: Advanced Example
+
+Concatenating paginated fetch results, surfacing transport errors:
+
+```ts
+import { concat } from "@suaveplan/streams/pipeline/concat/concat";
+import { StreamPipeFailedError } from "@suaveplan/streams/errors/stream-error";
+
+function pageStream(pageNumber: number): ReadableStream<Uint8Array> {
+    return new ReadableStream({
+        async pull(controller) {
+            const res = await fetch(`/api/items?page=${pageNumber}`);
+            if (!res.ok) throw new Error(`page ${pageNumber} failed`);
+            const buf = new Uint8Array(await res.arrayBuffer());
+            controller.enqueue(buf);
+            controller.close();
+        },
+    });
+}
+
+const stitched = concat([pageStream(1), pageStream(2), pageStream(3)]);
+
+try {
+    const reader = stitched.getReader();
+    while (true) {
+        const { done, value } = await reader.read();
+        if (done) break;
+        process(value);
+    }
+} catch (err) {
+    if (err instanceof StreamPipeFailedError) {
+        // err.cause holds the original transport / page error
+        report(err);
+    }
+    throw err;
+}
+```
+
+## API Reference
+
+### `concat<T>(streams: ReadableStream<T>[]): ReadableStream<T>`
+
+- **Parameters**
+  - `streams` — an ordered array of `ReadableStream<T>` to drain in sequence.
+- **Returns** — a new `ReadableStream<T>` that emits every chunk from
+  `streams[0]`, then every chunk from `streams[1]`, and so on, closing once the
+  final source closes. For `streams.length === 0`, returns an
+  immediately-closed stream. For `streams.length === 1`, returns
+  `streams[0]` unchanged (identity).
+- **Throws (downstream)** — `StreamPipeFailedError` with stable code
+  `STREAM.PIPE_FAILED` whenever any input source errors. The original error is
+  attached as `cause`.
+
+## Implementation Notes
+
+- The implementation owns at most one reader at a time. After a source reports
+  `done`, its reader is `releaseLock`-ed before the next source's reader is
+  acquired. This avoids holding multiple locks open and keeps memory bounded.
+- Sources are acquired lazily inside `pull`. Streams that the consumer never
+  reaches (e.g., because of an early error or cancellation) are never read,
+  which is intentional — concat must not eagerly drain inputs.
+- Cancellation only cancels the currently-acquired source. Un-acquired sources
+  are still owned by the caller and may be reused or cancelled independently.
+- Errors are wrapped, never re-thrown raw. This guarantees that consumers can
+  reliably pattern-match on `StreamPipeFailedError` / `STREAM.PIPE_FAILED` for
+  any concat-mediated failure, regardless of the underlying cause.
+- The single-source identity fast path is observable via reference equality —
+  consumers can detect it with `concat([s]) === s`.

packages/foundation/flow/streams/src/pipeline/concat/concat.test.ts

@@ -0,0 +1,255 @@
+import { describe, expect, it } from "@suaveplan/testing/runner";
+import { StreamPipeFailedError } from "../../errors/stream-error.js";
+import { fromArray } from "../../readable/from-array/from-array.js";
+import { toArray } from "../../sinks/to-array/to-array.js";
+import { concat } from "./concat.js";
+
+describe("concat", () => {
+    it("preserves order across stream boundaries", {
+        timeout: 5000,
+    }, async () => {
+        const a = fromArray([1, 2, 3]);
+        const b = fromArray([4, 5, 6]);
+        const result = await toArray(concat([a, b]));
+        expect(result).toEqual([1, 2, 3, 4, 5, 6]);
+    });
+
+    it("produces empty output for empty input array", {
+        timeout: 5000,
+    }, async () => {
+        const result = await toArray(concat<number>([]));
+        expect(result).toEqual([]);
+    });
+
+    it("passes through a single stream verbatim", { timeout: 5000 }, () => {
+        const source = fromArray([1, 2, 3]);
+        const out = concat([source]);
+        expect(out).toBe(source);
+    });
+
+    it("propagates errors as StreamPipeFailedError and stops reading subsequent streams", {
+        timeout: 5000,
+    }, async () => {
+        let bGetReaderCount = 0;
+        let cGetReaderCount = 0;
+
+        const a = fromArray([1, 2]);
+        const rawB = new ReadableStream<number>({
+            pull(controller) {
+                controller.error(new Error("middle stream boom"));
+            },
+        });
+        const rawC = fromArray([99]);
+
+        // Wrap b and c so we can detect when concat acquires their readers.
+        const b = new Proxy(rawB, {
+            get(target, prop, recv) {
+                if (prop === "getReader") {
+                    return (...args: unknown[]) => {
+                        bGetReaderCount++;
+                        return (
+                            target.getReader as unknown as (
+                                ...a: unknown[]
+                            ) => ReadableStreamDefaultReader<number>
+                        ).apply(target, args);
+                    };
+                }
+                return Reflect.get(target, prop, recv);
+            },
+        });
+        const c = new Proxy(rawC, {
+            get(target, prop, recv) {
+                if (prop === "getReader") {
+                    return (...args: unknown[]) => {
+                        cGetReaderCount++;
+                        return (
+                            target.getReader as unknown as (
+                                ...a: unknown[]
+                            ) => ReadableStreamDefaultReader<number>
+                        ).apply(target, args);
+                    };
+                }
+                return Reflect.get(target, prop, recv);
+            },
+        });
+
+        const out = concat([a, b, c]);
+        const reader = out.getReader();
+        const collected: number[] = [];
+        let caught: unknown;
+        try {
+            while (true) {
+                const r = await reader.read();
+                if (r.done) break;
+                collected.push(r.value);
+            }
+        } catch (err) {
+            caught = err;
+        }
+
+        expect(collected).toEqual([1, 2]);
+        expect(bGetReaderCount).toBe(1);
+        expect(cGetReaderCount).toBe(0);
+        expect(caught).toBeInstanceOf(StreamPipeFailedError);
+        expect((caught as StreamPipeFailedError).code).toBe(
+            "STREAM.PIPE_FAILED"
+        );
+        expect(((caught as StreamPipeFailedError).cause as Error).message).toBe(
+            "middle stream boom"
+        );
+    });
+
+    it("cancels the currently-reading stream on output cancel", {
+        timeout: 5000,
+    }, async () => {
+        let aCancelled = false;
+        let bGetReaderCount = 0;
+        const a = new ReadableStream<number>({
+            start(controller) {
+                controller.enqueue(1);
+            },
+            pull() {
+                return new Promise(() => {
+                    /* hang */
+                });
+            },
+            cancel() {
+                aCancelled = true;
+            },
+        });
+        const rawB = new ReadableStream<number>({
+            pull(controller) {
+                controller.enqueue(2);
+                controller.close();
+            },
+        });
+        const b = new Proxy(rawB, {
+            get(target, prop, recv) {
+                if (prop === "getReader") {
+                    return (...args: unknown[]) => {
+                        bGetReaderCount++;
+                        return (
+                            target.getReader as unknown as (
+                                ...a: unknown[]
+                            ) => ReadableStreamDefaultReader<number>
+                        ).apply(target, args);
+                    };
+                }
+                return Reflect.get(target, prop, recv);
+            },
+        });
+
+        const out = concat([a, b]);
+        const reader = out.getReader();
+        const first = await reader.read();
+        expect(first.value).toBe(1);
+        await reader.cancel("user-cancel");
+        await new Promise((r) => setTimeout(r, 20));
+        expect(aCancelled).toBe(true);
+        expect(bGetReaderCount).toBe(0);
+    });
+
+    it("only pulls from current stream while output is being pulled (backpressure)", {
+        timeout: 5000,
+    }, async () => {
+        let bGetReaderCount = 0;
+        const a = new ReadableStream<number>({
+            start(controller) {
+                controller.enqueue(1);
+                controller.enqueue(2);
+                controller.close();
+            },
+        });
+        const rawB = new ReadableStream<number>({
+            start(controller) {
+                controller.enqueue(100);
+                controller.close();
+            },
+        });
+        const b = new Proxy(rawB, {
+            get(target, prop, recv) {
+                if (prop === "getReader") {
+                    return (...args: unknown[]) => {
+                        bGetReaderCount++;
+                        return (
+                            target.getReader as unknown as (
+                                ...a: unknown[]
+                            ) => ReadableStreamDefaultReader<number>
+                        ).apply(target, args);
+                    };
+                }
+                return Reflect.get(target, prop, recv);
+            },
+        });
+
+        const out = concat([a, b]);
+        const reader = out.getReader();
+
+        const r1 = await reader.read();
+        expect(r1.value).toBe(1);
+        // After only one read, concat must not have acquired b yet — a still
+        // has a chunk to give.
+        expect(bGetReaderCount).toBe(0);
+
+        const r2 = await reader.read();
+        expect(r2.value).toBe(2);
+
+        const r3 = await reader.read();
+        expect(r3.value).toBe(100);
+        // Concat acquired b only after a was exhausted.
+        expect(bGetReaderCount).toBe(1);
+
+        const r4 = await reader.read();
+        expect(r4.done).toBe(true);
+    });
+
+    it("handles multiple empty streams interleaved with data", {
+        timeout: 5000,
+    }, async () => {
+        const out = concat<number>([
+            fromArray<number>([]),
+            fromArray([1, 2]),
+            fromArray<number>([]),
+            fromArray([3]),
+            fromArray<number>([]),
+        ]);
+        const result = await toArray(out);
+        expect(result).toEqual([1, 2, 3]);
+    });
+
+    it("cancelled flag prevents enqueue after in-flight read resolves concurrently", {
+        timeout: 5000,
+    }, async () => {
+        // Exercises line 70: `if (cancelled)` check — cancel() fires while
+        // reader.read() is in-flight; when it resolves the value is abandoned.
+        let resolvePull!: () => void;
+        const slow = new ReadableStream<number>({
+            start(controller) {
+                controller.enqueue(1);
+            },
+            pull() {
+                return new Promise<void>((r) => {
+                    resolvePull = r;
+                });
+            },
+            cancel() {
+                // Release the pending pull so reader.read() can resolve
+                resolvePull?.();
+            },
+        });
+
+        const out = concat([slow, fromArray([99])]);
+        const reader = out.getReader();
+
+        // Consume the first chunk so the stream advances to the hanging pull
+        const first = await reader.read();
+        expect(first.value).toBe(1);
+
+        // Cancel while the second pull is in-flight; cancel() releases resolvePull
+        await reader.cancel("test-cancel");
+
+        // After cancel, any subsequent read must return done
+        const next = await reader.read();
+        expect(next.done).toBe(true);
+    });
+});

packages/foundation/flow/streams/src/pipeline/concat/concat.ts

@@ -0,0 +1,97 @@
+import { StreamPipeFailedError } from "../../errors/stream-error.js";
+
+/**
+ * Concatenates multiple ReadableStreams into one, sequentially.
+ *
+ * Each input stream is drained to completion before the next stream is pulled
+ * from. Output ordering is therefore deterministic and identical to the input
+ * order — unlike {@link merge}, which is race-based.
+ *
+ * Errors from any input stream halt the concatenation immediately and are
+ * propagated wrapped in a {@link StreamPipeFailedError}; subsequent streams
+ * are never read. Output cancellation cancels the currently-reading source
+ * and releases any unread sources without pulling from them.
+ */
+export function concat<T>(streams: ReadableStream<T>[]): ReadableStream<T> {
+    if (streams.length === 0) {
+        return new ReadableStream({
+            start(c) {
+                c.close();
+            },
+        });
+    }
+    if (streams.length === 1) {
+        return streams[0]!;
+    }
+
+    const sources = [...streams];
+    let index = 0;
+    let currentReader: ReadableStreamDefaultReader<T> | undefined;
+    let cancelled = false;
+
+    function acquireReader(): ReadableStreamDefaultReader<T> | undefined {
+        if (currentReader) return currentReader;
+        if (index >= sources.length) return undefined;
+        currentReader = sources[index]!.getReader();
+        return currentReader;
+    }
+
+    return new ReadableStream<T>({
+        async pull(controller) {
+            while (!cancelled) {
+                const reader = acquireReader();
+                if (!reader) {
+                    controller.close();
+                    return;
+                }
+                let result: ReadableStreamReadResult<T>;
+                try {
+                    result = await reader.read();
+                } catch (cause) {
+                    controller.error(
+                        new StreamPipeFailedError({
+                            cause,
+                            message: "concat: input stream errored",
+                        })
+                    );
+                    // Release the failed reader; do not touch unread sources.
+                    try {
+                        reader.releaseLock();
+                    } catch {
+                        // ignore — reader may already be released
+                    }
+                    currentReader = undefined;
+                    return;
+                }
+
+                if (result.done) {
+                    try {
+                        reader.releaseLock();
+                    } catch {
+                        // ignore
+                    }
+                    currentReader = undefined;
+                    index++;
+                    // Loop to acquire next reader (or close if exhausted).
+                    continue;
+                }
+
+                controller.enqueue(result.value);
+                return;
+            }
+        },
+        async cancel(reason) {
+            cancelled = true;
+            const reader = currentReader;
+            currentReader = undefined;
+            try {
+                await reader?.cancel(reason);
+            } catch {
+                // best-effort cancellation
+            }
+            // Subsequent un-acquired sources are intentionally NOT pulled or
+            // cancelled — concat consumes them lazily, so unread sources were
+            // never attached and remain owned by the caller.
+        },
+    });
+}

packages/foundation/flow/streams/src/pipeline/concat/concat.types.ts

@@ -0,0 +1,11 @@
+/**
+ * Type definitions for the `concat` pipeline op.
+ *
+ * `concat` takes an ordered list of `ReadableStream<T>` inputs and produces a
+ * single `ReadableStream<T>` that drains each input to completion before
+ * pulling from the next. The output type matches the element type of the
+ * input streams.
+ */
+
+/** Input bundle for {@link concat}: an ordered array of streams of the same element type. */
+export type ConcatInputs<T> = ReadableStream<T>[];

packages/foundation/flow/streams/src/pipeline/concat/index.ts

@@ -0,0 +1,2 @@
+export { concat } from "./concat.js";
+export type { ConcatInputs } from "./concat.types.js";