Initial Commit
This commit is contained in:
86
packages/api-stack/api/api-plugin-import/src/parsers/csv.md
Normal file
86
packages/api-stack/api/api-plugin-import/src/parsers/csv.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# `parsers/csv` — CSV parsers
|
||||
|
||||
## Purpose
|
||||
|
||||
Provides the CSV-family parsers consumed by the canonical
|
||||
`ImportService.preview` / `startImport` pipeline and by any caller that
|
||||
needs to drive imports outside the plugin (CLI tools, batch workers).
|
||||
Ships both a high-level `parseCsv` returning row objects and a
|
||||
streaming variant (`streamParseCsv`) that surfaces per-row validation
|
||||
errors without aborting the run — important when the caller needs to
|
||||
collect every failed row in one pass rather than restart after each
|
||||
exception.
|
||||
|
||||
The kernel (`parseCsvLine`, `parseStreamLine`) handles double-quote
|
||||
escaping (`""`), custom delimiters, embedded newlines inside quoted
|
||||
fields, and trims field whitespace. Empty input or header-only input
|
||||
returns an empty rows array — not an exception — so route handlers can
|
||||
distinguish "no data" from "parse failed."
|
||||
|
||||
## Features
|
||||
|
||||
- `parseCsv` / `parseCsvLine` — happy-path one-shot parsing with
|
||||
default `,` delimiter.
|
||||
- `parseStreamLine` / `detectHeaders` — primitives used by
|
||||
`streamParseCsv`; exported for hosts needing custom-delimited
|
||||
variants (semicolons, pipes).
|
||||
- `streamParseCsv` — line-by-line streaming with optional `onRow`,
|
||||
`onError`, and `validate` callbacks. Validation failures are
|
||||
collected, not thrown.
|
||||
- `parseCsvWithHeaders` — header-aware result shape
|
||||
(`{ headers, rows, totalRows }`) matching `parseJson` and `parseXlsx`
|
||||
so routes can branch on `format` once.
|
||||
- `applyColumnMappings` — maps `sourceColumn` → `targetField` with
|
||||
optional per-mapping transform.
|
||||
- `detectColumns` — union of keys across heterogeneous rows.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { parseCsv } from "@suaveplan/api-plugin-import";
|
||||
|
||||
const rows = parseCsv("name,city\nAda,London\nBob,Paris");
|
||||
// [{ name: "Ada", city: "London" }, { name: "Bob", city: "Paris" }]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { streamParseCsv } from "@suaveplan/api-plugin-import";
|
||||
|
||||
const result = streamParseCsv("a,b\n1,2\n,3", {
|
||||
delimiter: ",",
|
||||
validate: (row, idx) => (row["a"] === "" ? `row ${idx}: a is required` : undefined),
|
||||
onError: (err) => logger.warn({ row: err.row, msg: err.message }, "drop"),
|
||||
});
|
||||
// result.errors → [{ row: 1, line: ",3", message: "row 1: a is required" }]
|
||||
// result.rows → [{ a: "1", b: "2" }]
|
||||
// result.validRows === 1
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `parseCsv(data: string): Record<string, string>[]`
|
||||
- `parseCsvLine(line: string): string[]`
|
||||
- `detectColumns(rows: Record<string, unknown>[]): string[]`
|
||||
- `parseStreamLine(line, delimiter, quoteChar): string[]`
|
||||
- `detectHeaders(firstLine, config: StreamParserConfig): string[]`
|
||||
- `streamParseCsv(data, config?): StreamParseResult`
|
||||
- `parseCsvWithHeaders(content, options?: { delimiter?; hasHeaders? }): CsvParseResult`
|
||||
- `applyColumnMappings(rows, mappings: ColumnMapping[]): Record<string, unknown>[]`
|
||||
- `interface StreamParserConfig { delimiter?; quoteChar?; hasHeaders?; onRow?; onError?; validate? }`
|
||||
- `interface StreamParseResult { headers; totalRows; validRows; errors; rows }`
|
||||
- `interface StreamRowError { row; line; message }`
|
||||
- `interface CsvParseResult { headers; rows; totalRows }`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Quote handling is single-state-machine (no regex backtracking) — O(n)
|
||||
in input length, no exponential pathological inputs.
|
||||
- `splitLines` keeps an in-quotes counter across newlines so RFC 4180
|
||||
multi-line records survive the split.
|
||||
- `applyColumnMappings` only invokes `transform` when the source value
|
||||
is a string — JSON-sourced numbers / booleans pass through unchanged.
|
||||
- The high-level `parseCsv` strips the header line by index (line 0);
|
||||
if the input contains only a header row, the function returns `[]`
|
||||
rather than `[{ header: undefined }]`.
|
||||
Reference in New Issue
Block a user