first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,119 @@
# mergeOrdered
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`mergeOrdered` performs a k-way merge of pre-sorted `ReadableStream`s
into a single ordered output stream. It is the streaming analogue of
the merge step in merge-sort: when every input is sorted under the
same comparator, the output is sorted under that comparator. This is
the right tool whenever a pipeline needs to consume multiple already-
ordered sources — sharded query results, time-keyed event logs,
externally-sorted partitions — without re-sorting the union in memory.
Unlike the unordered `merge`, which races inputs and emits in arrival
order, `mergeOrdered` emits the global minimum at every step using a
binary min-heap of `(value, sourceIdx)` entries. Each pull dequeues
the heap root, enqueues it on the output, and replaces the slot with
the next chunk from the same source. Throughput is dominated by the
`O(log k)` heap operations where `k` is the number of inputs.
## Features
- K-way merge over `ReadableStream<T>` arrays of any cardinality
- Caller-supplied comparator (ascending, descending, or domain-specific)
- Single-stream and empty-array fast paths
- Streams of differing lengths handled without truncation
- `cancel` propagates to every input stream
- Errors on any input cancel the remaining inputs and surface on the
merged stream
- Backpressure-respecting — only pulls one chunk per source per pull cycle
## Usage: Basic Example
```ts
import { mergeOrdered } from "@suaveplan/streams/pipeline/merge-ordered/merge-ordered";
import { fromArray } from "@suaveplan/streams/readable/from-array/from-array";
import { toArray } from "@suaveplan/streams/sinks/to-array/to-array";
const s1 = fromArray([1, 4, 7]);
const s2 = fromArray([2, 5, 8]);
const s3 = fromArray([3, 6, 9]);
const merged = mergeOrdered([s1, s2, s3], (a, b) => a - b);
const out = await toArray(merged);
// out === [1, 2, 3, 4, 5, 6, 7, 8, 9]
```
## Usage: Advanced Example
Merge sharded events keyed by timestamp, in descending order, and
forward the result into a downstream consumer that may cancel mid-flight:
```ts
import { mergeOrdered } from "@suaveplan/streams/pipeline/merge-ordered/merge-ordered";
interface Event {
ts: number;
payload: unknown;
}
declare const shardA: ReadableStream<Event>;
declare const shardB: ReadableStream<Event>;
declare const shardC: ReadableStream<Event>;
const merged = mergeOrdered<Event>(
[shardA, shardB, shardC],
(a, b) => b.ts - a.ts, // newest-first
);
const reader = merged.getReader();
try {
const first = await reader.read();
if (!first.done) {
// process newest event
}
} finally {
// Cancellation propagates to every shard.
await reader.cancel("consumer-done");
}
```
## API Reference
### `mergeOrdered<T>(streams, compare): ReadableStream<T>`
- `streams: ReadableStream<T>[]` — input streams, each pre-sorted
under `compare`. May be empty (returns an immediately-closed stream)
or singleton (returns a passthrough merge with the same heap path).
- `compare: (a: T, b: T) => number` — comparator. Negative when `a`
precedes `b`, zero when equal, positive otherwise. Identical contract
to `Array.prototype.sort`.
- **Returns** a `ReadableStream<T>` that emits values from the union of
all inputs in `compare` order, then closes.
- **Throws / errors** — if any input reader rejects, the remaining
inputs are cancelled and the rejection is forwarded as a stream error.
### `MergeOrderedCompare<T>`
Type alias for the comparator argument; exported for downstream typing
of comparator-producing factories.
## Implementation Notes
- The heap stores at most `k` entries (one per active source) so peak
memory is `O(k)`, independent of the total chunk count.
- Heap operations (`siftUp` / `siftDown`) are iterative and avoid
recursion to keep call stacks shallow on large `k`.
- The first `pull` primes every reader in parallel via `Promise.all`;
subsequent pulls only read from the source whose value was just
emitted, which preserves backpressure on the other inputs.
- `cancel` on the output uses `Promise.allSettled` to ensure every
input is cancelled even if one rejects.
- Output sortedness is conditional on input sortedness — passing
unsorted inputs is a caller bug, not a runtime check, because
validating sortedness would defeat the streaming guarantees.
- The single-stream path still goes through the heap for behavioural
parity with k≥2 (cancellation, error propagation).