first commit
This commit is contained in:
98
docs/devtools/tooling/readline/README.md
Normal file
98
docs/devtools/tooling/readline/README.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# @suaveplan/readline - Terminal I/O primitive: AsyncGenerator line reader and interactive prompt interface
|
||||
|
||||
## Summary
|
||||
|
||||
AsyncGenerator line reader and interactive prompt interface for Node.js streams, file paths, and Web Streams.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/errors.md) — <!-- begin:auto-docs --> _(no submodules)_- [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_ - [interface](./src/interface/interface.md) — <!-- begin:auto-docs --> _(no submodules)_- [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_ - [reader](./src/reader/reader.md) — <!-- begin:auto-docs --> _(no submodules)_- [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_ - [scanner](./src/scanner/scanner.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/readline
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Description | Docs |
|
||||
|--------|-------------|------|
|
||||
| `reader` | `readLines()` — multi-source AsyncGenerator line reader | [reader.md](src/reader/reader.md) |
|
||||
| `interface` | `createInterface()` — interactive prompt / REPL interface | [interface.md](src/interface/interface.md) |
|
||||
| `errors` | `ReadlineError` and stable error-code registry | [errors.md](src/errors/errors.md) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### `readLines()` — iterate lines from any source
|
||||
|
||||
```ts
|
||||
import { readLines } from "@suaveplan/readline/reader/reader";
|
||||
|
||||
// From a file path
|
||||
for await (const line of readLines("/var/log/app.log")) {
|
||||
console.log(line);
|
||||
}
|
||||
|
||||
// From a Node.js Readable
|
||||
import { createReadStream } from "node:fs";
|
||||
for await (const line of readLines(createReadStream("data.csv"), { skipEmpty: true })) {
|
||||
process(line);
|
||||
}
|
||||
|
||||
// With abort support and line-length guard
|
||||
const ac = new AbortController();
|
||||
setTimeout(() => ac.abort(), 5_000);
|
||||
for await (const line of readLines("huge.txt", { maxLineLength: 4096, signal: ac.signal })) {
|
||||
console.log(line);
|
||||
}
|
||||
```
|
||||
|
||||
### `createInterface()` — interactive prompt
|
||||
|
||||
```ts
|
||||
import { createInterface } from "@suaveplan/readline/interface/interface";
|
||||
|
||||
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
||||
|
||||
const name = await rl.question("Your name: ");
|
||||
console.log(`Hello, ${name}!`);
|
||||
|
||||
// Iterate all lines (useful for piped input)
|
||||
for await (const line of rl) {
|
||||
if (line === "quit") break;
|
||||
console.log(`echo: ${line}`);
|
||||
}
|
||||
|
||||
rl.close();
|
||||
```
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Measured on Linux 6.6 / WSL2, Bun v1.x, AMD Ryzen 9 5900X.
|
||||
|
||||
### Scanner throughput (`createScanner`)
|
||||
|
||||
| Benchmark | ops/sec | avg latency |
|
||||
|-----------|---------|-------------|
|
||||
| scanner 1 MB (50 k lines) | PLACEHOLDER | PLACEHOLDER |
|
||||
| scanner 10 MB (500 k lines) | PLACEHOLDER | PLACEHOLDER |
|
||||
|
||||
### Reader throughput (`readLines`)
|
||||
|
||||
| Benchmark | ops/sec | avg latency |
|
||||
|-----------|---------|-------------|
|
||||
| readLines() file path (10 k lines) | PLACEHOLDER | PLACEHOLDER |
|
||||
| readLines() Readable (10 k lines) | PLACEHOLDER | PLACEHOLDER |
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
162
docs/devtools/tooling/readline/src/errors/errors.md
Normal file
162
docs/devtools/tooling/readline/src/errors/errors.md
Normal file
@@ -0,0 +1,162 @@
|
||||
# errors
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Defines the complete error taxonomy for `@suaveplan/readline`. Every failure surface
|
||||
in the package — scanner overflows, interface lifecycle violations, and reader I/O
|
||||
faults — maps to a stable, machine-readable code exposed through `READLINE_CODES`.
|
||||
Using `defineCodes` from `@suaveplan/error` guarantees that codes are interned as
|
||||
readonly string literals at build time, so TypeScript narrows `err.code` to the
|
||||
exact union member without requiring string comparisons at runtime.
|
||||
|
||||
## Features
|
||||
|
||||
- **Three sub-namespaces** (`SCANNER`, `INTERFACE`, `READER`) keep codes from
|
||||
colliding and communicate which layer of the stack raised the error.
|
||||
- **Five error codes**: `LINE_TOO_LONG`, `SCANNER.STREAM_ERROR`, `INTERFACE.CLOSED`,
|
||||
`INTERFACE.ABORTED`, and `READER.STREAM_ERROR`.
|
||||
- **`ReadlineError`** extends `SuaveplanError<ReadlineCode>` — gives callers a single
|
||||
`instanceof ReadlineError` guard that covers all readline failures.
|
||||
- **`messageKey` property** stored on the error instance for i18n lookup or
|
||||
structured-log emission without re-parsing the message string.
|
||||
- **Cause chaining** via the standard `options.cause` field, forwarded to the
|
||||
`SuaveplanError` constructor and surfaced on `err.cause`.
|
||||
- **Structured context** via `options.context` — arbitrary key/value pairs attached
|
||||
as `err.context` for diagnostic payloads (e.g., `{ maxLineLength: 4096 }`).
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { ReadlineError, READLINE_CODES } from "@suaveplan/readline/errors/errors";
|
||||
|
||||
throw new ReadlineError(
|
||||
READLINE_CODES.SCANNER.LINE_TOO_LONG,
|
||||
"readline.scanner.line_too_long"
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Compose cause-chaining and structured context for a scanner overflow that wraps an
|
||||
underlying stream error:
|
||||
|
||||
```ts
|
||||
import { ReadlineError, READLINE_CODES } from "@suaveplan/readline/errors/errors";
|
||||
|
||||
function assertLineLength(line: string, max: number): void {
|
||||
if (line.length > max) {
|
||||
throw new ReadlineError(
|
||||
READLINE_CODES.SCANNER.LINE_TOO_LONG,
|
||||
"readline.scanner.line_too_long",
|
||||
{ context: { maxLineLength: max, actual: line.length } }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleStreamError(raw: unknown): Promise<never> {
|
||||
const cause = raw instanceof Error ? raw : new Error(String(raw));
|
||||
throw new ReadlineError(
|
||||
READLINE_CODES.READER.STREAM_ERROR,
|
||||
"readline.reader.stream_error",
|
||||
{ cause }
|
||||
);
|
||||
}
|
||||
|
||||
// Discriminate at the call site:
|
||||
try {
|
||||
await readLines(stream);
|
||||
} catch (err) {
|
||||
if (err instanceof ReadlineError) {
|
||||
console.error(err.code, err.context); // "READLINE.SCANNER.LINE_TOO_LONG" { maxLineLength: 4096, actual: 5000 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `READLINE_CODES`
|
||||
|
||||
```ts
|
||||
export const READLINE_CODES: {
|
||||
SCANNER: {
|
||||
LINE_TOO_LONG: "READLINE.SCANNER.LINE_TOO_LONG";
|
||||
STREAM_ERROR: "READLINE.SCANNER.STREAM_ERROR";
|
||||
};
|
||||
INTERFACE: {
|
||||
CLOSED: "READLINE.INTERFACE.CLOSED";
|
||||
ABORTED: "READLINE.INTERFACE.ABORTED";
|
||||
};
|
||||
READER: {
|
||||
STREAM_ERROR: "READLINE.READER.STREAM_ERROR";
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| `READLINE.SCANNER.LINE_TOO_LONG` | Input line exceeds the configured maximum byte/character limit. |
|
||||
| `READLINE.SCANNER.STREAM_ERROR` | The underlying readable stream emitted an `error` event during scanning. |
|
||||
| `READLINE.INTERFACE.CLOSED` | An operation was attempted after the readline interface was closed. |
|
||||
| `READLINE.INTERFACE.ABORTED` | An operation was cancelled via `AbortSignal` or an explicit `close()` call. |
|
||||
| `READLINE.READER.STREAM_ERROR` | A reader-level stream I/O failure occurred (e.g., broken pipe, permission denied). |
|
||||
|
||||
### `ReadlineCode`
|
||||
|
||||
```ts
|
||||
export type ReadlineCode =
|
||||
| "READLINE.SCANNER.LINE_TOO_LONG"
|
||||
| "READLINE.SCANNER.STREAM_ERROR"
|
||||
| "READLINE.INTERFACE.CLOSED"
|
||||
| "READLINE.INTERFACE.ABORTED"
|
||||
| "READLINE.READER.STREAM_ERROR";
|
||||
```
|
||||
|
||||
Union of all five readline error code literals. Use as a type parameter or for
|
||||
exhaustive `switch` discrimination.
|
||||
|
||||
### `ReadlineError`
|
||||
|
||||
```ts
|
||||
export class ReadlineError extends SuaveplanError<ReadlineCode> {
|
||||
public readonly messageKey: string;
|
||||
public constructor(
|
||||
code: ReadlineCode,
|
||||
messageKey: string,
|
||||
options?: SuaveplanErrorOptions
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Param | Type | Description |
|
||||
|---|---|---|
|
||||
| `code` | `ReadlineCode` | One of the five `READLINE_CODES` values. |
|
||||
| `messageKey` | `string` | i18n message key (e.g. `"readline.scanner.line_too_long"`). |
|
||||
| `options.cause` | `unknown` | Wraps a lower-level error as the causal chain. |
|
||||
| `options.context` | `ErrorContext` | Arbitrary key/value diagnostic payload attached to `err.context`. |
|
||||
| `options.traceId` | `string` | Optional OTel trace ID for cross-service correlation. |
|
||||
|
||||
**Returns** A `ReadlineError` instance. Inherits `err.code`, `err.context`,
|
||||
`err.cause`, `err.timestamp`, and `err.traceId` from `SuaveplanError`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**Why three `defineCodes` calls?** A single flat namespace (`READLINE.*`) risks code
|
||||
collisions as the package grows. Partitioning into `SCANNER`, `INTERFACE`, and
|
||||
`READER` mirrors the three distinct layers of the readline stack: byte scanning,
|
||||
the high-level interface, and the raw stream reader. This also allows call sites to
|
||||
catch and re-throw at layer boundaries without string-prefix matching.
|
||||
|
||||
**Why store `messageKey` on the instance?** `SuaveplanError` resolves the message
|
||||
string via `resolveMessage()` at construction time and discards the key. Retaining it
|
||||
as a public readonly property lets structured loggers and telemetry sinks emit the
|
||||
key alongside the resolved string, enabling i18n round-trips and log aggregation by
|
||||
key without parsing the human-readable message.
|
||||
|
||||
**`defineCodes` internment** produces `"READLINE.SCANNER.LINE_TOO_LONG"` (not a
|
||||
Symbol or numeric constant), which survives JSON serialisation, log shipping, and
|
||||
wire-format inclusion in `SuaveplanError`'s Zod-validated payload unchanged.
|
||||
116
docs/devtools/tooling/readline/src/interface/interface.md
Normal file
116
docs/devtools/tooling/readline/src/interface/interface.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# interface
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`createInterface()` produces an interactive readline interface that wraps the lower-level `createScanner()` generator with a higher-level API suited for CLI tools, prompts, and line-by-line pipeline processing. It serves as the primary user-facing entry point for `@suaveplan/readline`, replacing `node:readline`'s callback-heavy `Interface` class with a fully promise-based, abort-signal-aware, and OpenTelemetry-instrumented alternative. Every interface instance is covered by a single OTel span that begins at construction and ends when `close()` is called or the async iterator drains naturally.
|
||||
|
||||
## Features
|
||||
|
||||
- **`question(prompt, signal?)`** — writes a prompt string to the optional output stream, then awaits the next line from input; rejects with `INTERFACE.CLOSED` when the interface is closed and `INTERFACE.ABORTED` when an `AbortSignal` fires
|
||||
- **`close()`** — idempotent close method; sets `closed = true` and ends the OTel span; safe to call multiple times
|
||||
- **`closed` getter** — synchronous boolean reflecting the current open/closed state
|
||||
- **Async iterator** (`for await...of`) — iterate all lines from the underlying scanner; automatically marks the interface closed when the stream drains
|
||||
- **AbortSignal support** — both the interface-level signal (passed to `createInterface`) and a per-question signal are honoured; abort mid-question rejects with `INTERFACE.ABORTED`
|
||||
- **OTel span lifetime** — a `readline.interface` span is started on construction and ended on close or stream drain; abort sets `SpanStatusCode.ERROR` before ending
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { Readable } from "node:stream";
|
||||
import { createInterface } from "@suaveplan/readline";
|
||||
|
||||
const input = Readable.from("Alice\nBob\n");
|
||||
const rl = createInterface({ input });
|
||||
|
||||
const name = await rl.question("Enter name: ");
|
||||
console.log(`Hello, ${name}!`); // "Hello, Alice!"
|
||||
rl.close();
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { Readable, Writable } from "node:stream";
|
||||
import { createInterface } from "@suaveplan/readline";
|
||||
|
||||
// Iterate all lines with abort support
|
||||
const controller = new AbortController();
|
||||
const input = Readable.from("line1\nline2\nline3\n");
|
||||
const rl = createInterface({ input, signal: controller.signal });
|
||||
|
||||
const collected: string[] = [];
|
||||
for await (const line of rl) {
|
||||
collected.push(line);
|
||||
if (line === "line2") controller.abort(); // stop early
|
||||
}
|
||||
// collected === ["line1", "line2"]
|
||||
|
||||
// Question with per-call signal override
|
||||
const perQuestion = new AbortController();
|
||||
const rl2 = createInterface({ input: Readable.from("response\n") });
|
||||
setTimeout(() => perQuestion.abort(), 100);
|
||||
try {
|
||||
const ans = await rl2.question("Say something: ", perQuestion.signal);
|
||||
console.log(ans);
|
||||
} catch (err) {
|
||||
// ReadlineError with code READLINE_CODES.INTERFACE.ABORTED
|
||||
}
|
||||
rl2.close();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `createInterface(options: ReadlineOptions): ReadlineInterface`
|
||||
|
||||
Creates and returns a new readline interface wrapping the provided `Readable` stream.
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `options.input` | `Readable` | Yes | The source readable stream to scan line-by-line |
|
||||
| `options.output` | `Writable` | No | Optional writable stream; prompts are written here |
|
||||
| `options.signal` | `AbortSignal` | No | Optional interface-level abort signal |
|
||||
|
||||
**Returns** `ReadlineInterface`
|
||||
|
||||
---
|
||||
|
||||
### `ReadlineInterface`
|
||||
|
||||
| Member | Signature | Description |
|
||||
|--------|-----------|-------------|
|
||||
| `question` | `(prompt: string, signal?: AbortSignal) => Promise<string>` | Write prompt and await the next line. Throws `INTERFACE.CLOSED` if the interface is closed; throws `INTERFACE.ABORTED` if a signal fires |
|
||||
| `close` | `() => void` | Idempotent; marks the interface closed and ends the OTel span |
|
||||
| `closed` | `readonly boolean` | `true` after `close()` has been called |
|
||||
| `[Symbol.asyncIterator]` | `() => AsyncIterator<string>` | Iterate all remaining lines; stops when closed or stream drains |
|
||||
|
||||
---
|
||||
|
||||
### `ReadlineOptions`
|
||||
|
||||
```ts
|
||||
interface ReadlineOptions {
|
||||
input: Readable;
|
||||
output?: Writable;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
```
|
||||
|
||||
**Throws** `ReadlineError` with codes from `READLINE_CODES.INTERFACE`:
|
||||
|
||||
- `INTERFACE.CLOSED` — method called after `close()` or stream drained
|
||||
- `INTERFACE.ABORTED` — abort signal fired before or during a `question()` call
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**OTel span design** — The span starts immediately in `createInterface()` (not lazily) so it faithfully represents the full interface lifetime. `span.end()` is called exactly once: either from `close()`, from the async iterator when the stream drains, or when an abort surfaces in the iterator. An abort path also calls `span.setStatus(ERROR)` before ending. Double-close is guarded by a `if (closed) return` early exit.
|
||||
|
||||
**Why `iter.next()` instead of `for await`** — `question()` needs to await exactly one line at a time, interleaved with prompt writes, which rules out `for await`. Calling `iter.next()` directly gives precise per-line control over the shared iterator and lets the abort handler race cleanly against the next-line promise without leaving dangling iterations.
|
||||
|
||||
**Double-abort guard** — `question()` checks `effectiveSignal?.aborted` synchronously before attaching the listener, preventing a TOCTOU where the signal is already aborted when `question()` is entered. The abort listener is removed via `removeEventListener` after `iter.next()` resolves to avoid memory leaks on long-lived interfaces.
|
||||
|
||||
**Shared iterator state** — `createScanner` is a single generator; both `question()` and the async iterator share the same `iter`. Callers must not mix `question()` calls and `for await` on the same interface concurrently — the semantics are sequential.
|
||||
114
docs/devtools/tooling/readline/src/reader/reader.md
Normal file
114
docs/devtools/tooling/readline/src/reader/reader.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# reader
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
The `reader` module exposes `readLines()`, a unified async-generator entry point for line-by-line reading from any of three input forms: a Node.js `Readable` stream, a WHATWG `ReadableStream`, or a plain file-path string. It bridges the gap between the low-level `createScanner` byte-parser and the diverse I/O surfaces consumers work with in real applications.
|
||||
|
||||
By accepting all three input kinds under a single function signature, `readLines()` lets callers swap between reading from stdin, a web `fetch()` response body, or an on-disk log file without changing any surrounding code. The function delegates all byte-parsing and line-assembly work to `createScanner`, keeping its own concerns limited to source normalisation and the `skipEmpty` filter.
|
||||
|
||||
## Features
|
||||
|
||||
- **File-path string input** — pass a filesystem path and `readLines` opens a read stream automatically via `createReadStream`
|
||||
- **Node.js `Readable` input** — pass any `Readable` (stdin, `net.Socket`, custom, etc.) directly
|
||||
- **WHATWG `ReadableStream` input** — pass a Web Streams `ReadableStream` (e.g. from `fetch().body`); bridged to Node `Readable` via `Readable.fromWeb`
|
||||
- **`skipEmpty` option** — filter out blank lines before they reach the caller
|
||||
- **`AbortSignal` support** — stop iteration at any point by aborting the controller; the generator exits cleanly on the next yield boundary
|
||||
- **`maxLineLength` and `encoding` passthrough** — forwarded verbatim to `createScanner`
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
Read every line from a local file:
|
||||
|
||||
```ts
|
||||
import { readLines } from "@suaveplan/readline/reader/reader";
|
||||
|
||||
for await (const line of readLines("/var/log/app.log")) {
|
||||
console.log(line);
|
||||
}
|
||||
```
|
||||
|
||||
Read from stdin:
|
||||
|
||||
```ts
|
||||
import { readLines } from "@suaveplan/readline/reader/reader";
|
||||
import { stdin } from "node:process";
|
||||
|
||||
for await (const line of readLines(stdin)) {
|
||||
process.stdout.write(line + "\n");
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Fetch a remote newline-delimited JSON stream, skip blank lines, and abort after the first error event:
|
||||
|
||||
```ts
|
||||
import { readLines } from "@suaveplan/readline/reader/reader";
|
||||
|
||||
const controller = new AbortController();
|
||||
const response = await fetch("https://example.com/events", {
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.body) throw new Error("No body");
|
||||
|
||||
const results: unknown[] = [];
|
||||
|
||||
try {
|
||||
for await (const line of readLines(response.body, {
|
||||
skipEmpty: true,
|
||||
signal: controller.signal,
|
||||
maxLineLength: 65536,
|
||||
})) {
|
||||
const event = JSON.parse(line) as { type: string; payload: unknown };
|
||||
results.push(event.payload);
|
||||
if (event.type === "error") {
|
||||
controller.abort();
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// generator exits cleanly; partially collected results are still usable
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `readLines(input, options?)`
|
||||
|
||||
```ts
|
||||
async function* readLines(
|
||||
input: Readable | ReadableStream | string,
|
||||
options?: ReadLinesOptions
|
||||
): AsyncGenerator<string, void, unknown>
|
||||
```
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|---|---|---|
|
||||
| `input` | `Readable \| ReadableStream \| string` | The data source. A string is treated as a file path. |
|
||||
| `options.maxLineLength` | `number` (optional) | Maximum bytes per line before a `LINE_TOO_LONG` error is thrown. Defaults to `4096`. |
|
||||
| `options.encoding` | `BufferEncoding` (optional) | Text encoding for decoded lines. Defaults to `"utf8"`. |
|
||||
| `options.skipEmpty` | `boolean` (optional) | When `true`, lines that are empty strings after decoding are silently dropped. |
|
||||
| `options.signal` | `AbortSignal` (optional) | Aborts iteration early. The generator exits on the next yield boundary without throwing. |
|
||||
|
||||
**Returns** `AsyncGenerator<string>` — each yielded value is one decoded line, CR/LF stripped.
|
||||
|
||||
**Throws** `ReadlineError` with code `SCANNER.STREAM_ERROR` on underlying stream errors, or `SCANNER.LINE_TOO_LONG` when a line exceeds `maxLineLength`.
|
||||
|
||||
### `ReadLinesOptions`
|
||||
|
||||
Interface re-exported from `reader.types.ts`. All fields are optional.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**WHATWG stream bridging.** `Readable.fromWeb` (available in Node ≥ 17 and Bun ≥ 1.0) converts a `ReadableStream<Uint8Array>` into a Node `Readable`. The import of `node:stream` is deferred (`await import`) so the static module graph does not load it for callers that only use WHATWG paths — though in practice all three code paths require `node:stream`, so this is mainly a style alignment with future environments where the module may be conditionally present.
|
||||
|
||||
**Detection heuristic.** WHATWG streams are identified by the presence of a `.getReader` method, which is part of the `ReadableStream` spec and absent from Node `Readable`. This avoids an `instanceof` check that would fail across realm boundaries (e.g. cross-context `ReadableStream` objects from a `fetch` polyfill).
|
||||
|
||||
**Delegation to `createScanner`.** All byte-parsing, CR/LF normalisation, buffer management, and `AbortSignal` checking live in `createScanner`. `readLines` is intentionally thin — it adds only source normalisation and the `skipEmpty` filter. This keeps each layer testable in isolation and avoids duplicating byte-level logic.
|
||||
|
||||
**`skipEmpty` placement.** The empty-line filter is applied after `createScanner` yields, not inside the scanner, so that callers who need raw empty lines (e.g. protocol parsers that use blank lines as delimiters) can obtain them by omitting the option.
|
||||
93
docs/devtools/tooling/readline/src/scanner/scanner.md
Normal file
93
docs/devtools/tooling/readline/src/scanner/scanner.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# scanner
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`createScanner` is the internal byte-level line-reading engine for `@suaveplan/readline`. It consumes a Node.js `Readable` stream and yields decoded string lines one at a time via an `AsyncGenerator`. This module is intentionally kept off the public `package.json` subpath exports — it is an implementation detail consumed exclusively by the `reader` and `interface` modules within this package.
|
||||
|
||||
The scanner solves a subtle problem: Node.js streams are push-based (they emit `"data"` events synchronously), while `AsyncGenerator` consumers are pull-based (they request values via `next()`). Bridging the two correctly requires a bounded async push queue that buffers emitted items until the consumer is ready to receive them.
|
||||
|
||||
## Features
|
||||
|
||||
- **LF and CRLF line termination** — bare `\n` and `\r\n` sequences are both handled; bare `\r` (old Mac line endings) is treated as whitespace.
|
||||
- **Unterminated final line** — a line that ends at stream EOF without a trailing newline is emitted correctly.
|
||||
- **Empty input** — an empty stream yields zero lines without error.
|
||||
- **Max line length guard** — if `maxLineLength` is exceeded before a line terminator is found, a `ReadlineError` with code `READLINE_CODES.SCANNER.LINE_TOO_LONG` is thrown immediately.
|
||||
- **Stream error propagation** — any `"error"` event on the input stream is wrapped in a `ReadlineError` with code `READLINE_CODES.SCANNER.STREAM_ERROR` and rethrown to the caller.
|
||||
- **AbortSignal support** — passing `signal` causes the generator to stop yielding and return cleanly when the signal fires. The abort check is performed both before and after each `dequeue()` await to catch signals that arrive while the generator is suspended.
|
||||
- **Async push queue** — a lightweight internal queue (not `EventEmitter`) decouples the push-based stream from the pull-based generator without leaking abstractions.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { Readable } from "node:stream";
|
||||
import { createScanner } from "./scanner.js";
|
||||
|
||||
const readable = Readable.from("line one\nline two\nline three\n");
|
||||
|
||||
for await (const line of createScanner(readable)) {
|
||||
console.log(line);
|
||||
}
|
||||
// Output:
|
||||
// line one
|
||||
// line two
|
||||
// line three
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { createReadStream } from "node:fs";
|
||||
import { createScanner } from "./scanner.js";
|
||||
|
||||
const controller = new AbortController();
|
||||
const stream = createReadStream("/var/log/app.log");
|
||||
|
||||
// Abort after reading 100 lines
|
||||
let count = 0;
|
||||
for await (const line of createScanner(stream, {
|
||||
maxLineLength: 8192,
|
||||
encoding: "utf8",
|
||||
signal: controller.signal,
|
||||
})) {
|
||||
console.log(line);
|
||||
if (++count >= 100) controller.abort();
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `createScanner(input, opts?)`
|
||||
|
||||
Creates an `AsyncGenerator<string>` that yields lines read from `input`.
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Name | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `input` | `Readable` | — | Node.js `Readable` stream to scan |
|
||||
| `opts.maxLineLength` | `number` | `4096` | Maximum byte length of any single line before a `LINE_TOO_LONG` error is thrown |
|
||||
| `opts.encoding` | `BufferEncoding` | `"utf8"` | Character encoding used when decoding bytes to strings |
|
||||
| `opts.signal` | `AbortSignal` | `undefined` | Optional cancellation signal; generator stops cleanly when aborted |
|
||||
|
||||
**Returns**
|
||||
|
||||
`AsyncGenerator<string, void, unknown>` — yields one decoded line per iteration (newline terminators stripped).
|
||||
|
||||
**Throws**
|
||||
|
||||
- `ReadlineError` with code `READLINE_CODES.SCANNER.LINE_TOO_LONG` when a line exceeds `maxLineLength` bytes without a line terminator.
|
||||
- `ReadlineError` with code `READLINE_CODES.SCANNER.STREAM_ERROR` wrapping the original `Error` when the underlying stream emits an `"error"` event.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**Why a bounded async push queue instead of `EventEmitter`?**
|
||||
Node.js streams emit `"data"` synchronously on whichever tick the chunk arrives. If we naively awaited an `EventEmitter`-backed `Promise` we would lose chunks that fired between `await` points. The bounded queue (`queue: QueueItem[]` + a single pending `waitResolve` resolver) solves this: if the generator is currently suspended in `dequeue()`, the new item is delivered directly to the waiting resolver; otherwise it is buffered in the queue until the next `dequeue()` call. This gives correct back-pressure semantics without a heavyweight library.
|
||||
|
||||
**Why two AbortSignal checks per iteration?**
|
||||
The `while (true)` loop checks `signal?.aborted` both _before_ calling `dequeue()` and _after_ the `await` resolves. The first check handles the case where the signal was already set when the loop ticked. The second handles the race where the signal fires _while_ the generator is suspended inside `dequeue()` waiting for the next queue item. Without the post-await check, a late abort could cause one spurious extra line to be yielded before the generator noticed the signal.
|
||||
|
||||
**`lineBuffer` pre-allocation**
|
||||
A single `Buffer.alloc(maxLen)` is allocated once per `createScanner` call, avoiding repeated heap allocations on every `"data"` event. Bytes are accumulated in-place; `subarray` is used on emit to avoid a copy.
|
||||
Reference in New Issue
Block a user