Initial Commit

This commit is contained in:
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions

View File

@@ -0,0 +1,86 @@
# `parsers/csv` — CSV parsers
## Purpose
Provides the CSV-family parsers consumed by the canonical
`ImportService.preview` / `startImport` pipeline and by any caller that
needs to drive imports outside the plugin (CLI tools, batch workers).
Ships both a high-level `parseCsv` returning row objects and a
streaming variant (`streamParseCsv`) that surfaces per-row validation
errors without aborting the run — important when the caller needs to
collect every failed row in one pass rather than restart after each
exception.
The kernel (`parseCsvLine`, `parseStreamLine`) handles double-quote
escaping (`""`), custom delimiters, embedded newlines inside quoted
fields, and trims field whitespace. Empty input or header-only input
returns an empty rows array — not an exception — so route handlers can
distinguish "no data" from "parse failed."
## Features
- `parseCsv` / `parseCsvLine` — happy-path one-shot parsing with
default `,` delimiter.
- `parseStreamLine` / `detectHeaders` — primitives used by
`streamParseCsv`; exported for hosts needing custom-delimited
variants (semicolons, pipes).
- `streamParseCsv` — line-by-line streaming with optional `onRow`,
`onError`, and `validate` callbacks. Validation failures are
collected, not thrown.
- `parseCsvWithHeaders` — header-aware result shape
(`{ headers, rows, totalRows }`) matching `parseJson` and `parseXlsx`
so routes can branch on `format` once.
- `applyColumnMappings` — maps `sourceColumn` → `targetField` with
optional per-mapping transform.
- `detectColumns` — union of keys across heterogeneous rows.
## Usage: Basic Example
```ts
import { parseCsv } from "@suaveplan/api-plugin-import";
const rows = parseCsv("name,city\nAda,London\nBob,Paris");
// [{ name: "Ada", city: "London" }, { name: "Bob", city: "Paris" }]
```
## Usage: Advanced Example
```ts
import { streamParseCsv } from "@suaveplan/api-plugin-import";
const result = streamParseCsv("a,b\n1,2\n,3", {
delimiter: ",",
validate: (row, idx) => (row["a"] === "" ? `row ${idx}: a is required` : undefined),
onError: (err) => logger.warn({ row: err.row, msg: err.message }, "drop"),
});
// result.errors → [{ row: 1, line: ",3", message: "row 1: a is required" }]
// result.rows → [{ a: "1", b: "2" }]
// result.validRows === 1
```
## API Reference
- `parseCsv(data: string): Record<string, string>[]`
- `parseCsvLine(line: string): string[]`
- `detectColumns(rows: Record<string, unknown>[]): string[]`
- `parseStreamLine(line, delimiter, quoteChar): string[]`
- `detectHeaders(firstLine, config: StreamParserConfig): string[]`
- `streamParseCsv(data, config?): StreamParseResult`
- `parseCsvWithHeaders(content, options?: { delimiter?; hasHeaders? }): CsvParseResult`
- `applyColumnMappings(rows, mappings: ColumnMapping[]): Record<string, unknown>[]`
- `interface StreamParserConfig { delimiter?; quoteChar?; hasHeaders?; onRow?; onError?; validate? }`
- `interface StreamParseResult { headers; totalRows; validRows; errors; rows }`
- `interface StreamRowError { row; line; message }`
- `interface CsvParseResult { headers; rows; totalRows }`
## Implementation Notes
- Quote handling is single-state-machine (no regex backtracking) — O(n)
in input length, no exponential pathological inputs.
- `splitLines` keeps an in-quotes counter across newlines so RFC 4180
multi-line records survive the split.
- `applyColumnMappings` only invokes `transform` when the source value
is a string — JSON-sourced numbers / booleans pass through unchanged.
- The high-level `parseCsv` strips the header line by index (line 0);
if the input contains only a header row, the function returns `[]`
rather than `[{ header: undefined }]`.