first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,69 @@
# from-array
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Converts a JavaScript array into a WHATWG `ReadableStream`. Each array element is enqueued as a separate chunk in index order. The stream is closed automatically after all elements are pushed. This factory is the simplest entry point for creating a stream from in-memory data and is the building block for test fixtures and benchmark pipelines throughout the package.
The `highWaterMark` option controls the WHATWG backpressure queue depth. Keeping it at the default (`1`) applies maximum backpressure, ensuring the consumer controls the pull rate. Higher values pre-buffer more chunks, which is useful when the consumer is consistently slower than the source.
## Features
- Converts any `T[]` to `ReadableStream<T>`
- `AbortSignal` support: errors the stream immediately when the signal fires
- Configurable `highWaterMark` via `ReadableOptions`
- Zero external runtime dependencies
## Usage: Basic Example
```ts
import { fromArray } from "@suaveplan/streams/readable/from-array/from-array";
const stream = fromArray([1, 2, 3]);
const reader = stream.getReader();
let result;
while (!(result = await reader.read()).done) {
console.log(result.value); // 1, 2, 3
}
```
## Usage: Advanced Example
```ts
import { fromArray } from "@suaveplan/streams/readable/from-array/from-array";
import { mapStream } from "@suaveplan/streams/transforms/map-stream/map-stream";
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
const ac = new AbortController();
const stream = fromArray(["a", "b", "c"], { signal: ac.signal, highWaterMark: 4 });
const result = await toArray(stream.pipeThrough(mapStream((x) => x.toUpperCase())));
console.log(result); // ["A", "B", "C"]
// Cancellation: abort mid-flight
const longStream = fromArray(new Array(1_000_000).fill(0), { signal: ac.signal });
ac.abort();
const reader = longStream.getReader();
const { done } = await reader.read().catch(() => ({ done: true }));
console.log(done); // true (stream was aborted)
```
## API Reference
### `fromArray<T>(items: T[], options?: ReadableOptions): ReadableStream<T>`
**Parameters**:
- `items` — the source array; elements are enqueued in insertion order.
- `options.signal` — optional `AbortSignal`; errors the stream on abort.
- `options.highWaterMark` — WHATWG queue depth (default: `1`).
**Returns**: `ReadableStream<T>` that emits each element and then closes.
**Throws**: never directly; the stream's reader will throw the abort reason when the signal fires.
## Implementation Notes
The `pull` callback checks `signal.aborted` on every pull cycle rather than using an `abort` event listener. This avoids the need to clean up listeners and keeps the implementation leak-free. When the signal fires between two pull cycles the worst case is one additional chunk being enqueued before the abort is detected — this is acceptable because WHATWG streams do not guarantee atomic cancellation at the chunk boundary.

View File

@@ -0,0 +1,73 @@
# from-bytes
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Provides three factories and utilities for working with binary `Uint8Array` streams: `fromBytes` slices a buffer into a chunked `ReadableStream<Uint8Array>`, `concatByteStreams` sequentially reads multiple streams into one, and `sliceByteStream` lazily emits a byte range without buffering the full stream. All operations use `Uint8Array.subarray` for zero-copy views wherever the platform permits.
These utilities are the foundation for file-processing pipelines, compression/decompression, and any feature that needs to move binary data through a backpressure-aware pipeline without unnecessary copying.
## Features
- `fromBytes`: splits a `Uint8Array` into configurable chunk sizes (default 64 KiB)
- `concatByteStreams`: sequential fan-in of multiple byte streams without buffering
- `sliceByteStream`: range-based lazy slice of a byte stream (`start`/`end` offsets)
- Zero-copy `subarray` views — no `slice` allocations unless the platform requires it
- AbortSignal support in `fromBytes`
- Configurable `highWaterMark` for WHATWG backpressure
## Usage: Basic Example
```ts
import { fromBytes } from "@suaveplan/streams/readable/from-bytes/from-bytes";
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
const data = new Uint8Array([0, 1, 2, 3, 4, 5, 6, 7]);
const stream = fromBytes(data, 3); // chunks: [0,1,2], [3,4,5], [6,7]
const chunks = await toArray(stream);
console.log(chunks.map(c => Array.from(c)));
// [[0,1,2], [3,4,5], [6,7]]
```
## Usage: Advanced Example
```ts
import {
fromBytes,
concatByteStreams,
sliceByteStream,
} from "@suaveplan/streams/readable/from-bytes/from-bytes";
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
const a = fromBytes(new Uint8Array([1, 2, 3]));
const b = fromBytes(new Uint8Array([4, 5, 6]));
const combined = concatByteStreams([a, b]);
// Slice bytes 1 to 4 (exclusive) from the combined stream
const sliced = sliceByteStream(combined, 1, 4);
const result = await toArray(sliced);
// result: Uint8Array([2, 3, 4])
```
## API Reference
### `fromBytes(data: Uint8Array, chunkSize?: number, options?: ReadableOptions): ReadableStream<Uint8Array>`
- `data` — source buffer; no copy is made.
- `chunkSize` — bytes per chunk (default: 65536).
- `options.signal` — AbortSignal, errors the stream on abort.
- `options.highWaterMark` — WHATWG queue depth (default: 1).
### `concatByteStreams(streams: ReadableStream<Uint8Array>[]): ReadableStream<Uint8Array>`
Reads each stream to completion before starting the next. The returned stream closes after all input streams close.
### `sliceByteStream(stream: ReadableStream<Uint8Array>, start: number, end?: number): ReadableStream<Uint8Array>`
Emits only bytes whose position falls within `[start, end)`. Cancels the underlying reader once `end` is reached.
## Implementation Notes
`fromBytes` uses `data.subarray(offset, end)` which returns a view sharing the same underlying `ArrayBuffer` — no allocation. `concatByteStreams` maintains a single active reader reference, releasing it before advancing to the next stream. `sliceByteStream` reads chunks from the underlying stream, computing per-chunk sub-ranges in a single loop without materialising the full stream.

View File

@@ -0,0 +1,75 @@
# from-generator
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Converts an async generator factory function into a WHATWG `ReadableStream`. The factory is invoked lazily — only when the first consumer starts pulling — which avoids eager resource acquisition. Generator teardown on cancellation is handled via `iterator.return()`, ensuring that `finally` blocks in the generator are executed and file handles, network sockets, or other resources are released.
This factory is the preferred entry point for wrapping any asynchronous data source that is naturally expressed as an `async function*` — database cursors, paginated API clients, file-line readers, or infinite event sequences.
## Features
- Lazy generator construction (factory called on first pull)
- Clean teardown via `iterator.return()` on stream cancellation
- AbortSignal support: errors the stream when the signal fires
- Configurable `highWaterMark` for WHATWG backpressure control
- Propagates generator `throw` errors through the stream's error channel
## Usage: Basic Example
```ts
import { fromAsyncGenerator } from "@suaveplan/streams/readable/from-generator/from-generator";
async function* counter(limit: number) {
for (let i = 0; i < limit; i++) yield i;
}
const stream = fromAsyncGenerator(() => counter(3));
const reader = stream.getReader();
console.log((await reader.read()).value); // 0
console.log((await reader.read()).value); // 1
console.log((await reader.read()).value); // 2
console.log((await reader.read()).done); // true
```
## Usage: Advanced Example
```ts
import { fromAsyncGenerator } from "@suaveplan/streams/readable/from-generator/from-generator";
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
async function* fetchPages(baseUrl: string): AsyncGenerator<string> {
let cursor: string | undefined;
do {
const url = cursor ? `${baseUrl}?cursor=${cursor}` : baseUrl;
const res = await fetch(url);
const data = await res.json();
yield* data.items;
cursor = data.nextCursor;
} while (cursor);
}
const ac = new AbortController();
const stream = fromAsyncGenerator(() => fetchPages("/api/items"), { signal: ac.signal });
const items = await toArray(stream);
```
## API Reference
### `fromAsyncGenerator<T>(gen: () => AsyncGenerator<T>, options?: ReadableOptions): ReadableStream<T>`
**Parameters**:
- `gen` — factory function that returns an `AsyncGenerator<T>`; called once on first pull.
- `options.signal` — optional `AbortSignal`; errors the stream on abort.
- `options.highWaterMark` — WHATWG queue depth (default: `1`).
**Returns**: `ReadableStream<T>` that yields each value from the generator in order.
**Throws**: never directly. Generator errors propagate through the stream's error channel and surface as rejected `reader.read()` promises.
## Implementation Notes
The generator instance is created lazily inside `pull` rather than in `start`. This means that if a `ReadableStream` is constructed but never read, the generator factory is never invoked and no resources are acquired. Cancellation calls `iterator.return(undefined)` which triggers any `try/finally` cleanup in the generator. The `cancel` method is `async` to await the return value in case the generator performs async cleanup.