first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,112 @@
# `transform/pipeline` — declarative transform pipeline
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements a three-stage row-transform pipeline that any consumer can
drive without depending on the canonical `ImportService`: per-mapping
coerce + transform, computed fields, then field validation. The
pipeline is pure (no side effects, no I/O) and never throws — every
failure is captured as a `TransformError` and returned in the result
object, so callers can collect every failed row in one pass.
This module powers both the standalone `transformRow` / `transformRows`
helpers exported to consumers and the per-row inner loop of
`ImportService.startImport`. Keeping the pipeline separate from the
service means CLI tools, batch workers, and tests can drive the
transform logic against arbitrary input without spinning up an event
emitter or store adapter.
## Features
- `coerceType` — explicit-result coercion for `number` / `boolean` /
`date` with documented "valid" sets (`1`/`0`/`yes`/`no`/`true`/`false`
for booleans, anything `Date.parse`-able for dates).
- `validateField` — five built-in rules (`required`, `min`, `max`,
`pattern`, `custom`); `custom` is an opt-in extension point that
always returns `undefined` so callers wire their own.
- `transformRow` — applies mappings, computed fields, and validations
in order; pushes `TransformError` for each failure and returns
`{ success, row, errors }`.
- `transformRows` — array fan-out; errored rows are dropped from `rows`
but their errors are still collected in `errors`.
- `toRowError` — bridge to the service-layer `RowError` shape so the
service can re-emit transform failures on the `import.row_error`
event topic without duplicating shapes.
## Usage: Basic Example
```ts
import { transformRow } from "@suaveplan/api-plugin-import";
const result = transformRow(
{ name: "Ada", age: "37" },
0,
{
mappings: [
{ sourceColumn: "name", targetField: "displayName" },
{ sourceColumn: "age", targetField: "age", type: "number" },
],
}
);
// result.row → { displayName: "Ada", age: 37 }
```
## Usage: Advanced Example
```ts
import { transformRow } from "@suaveplan/api-plugin-import";
const result = transformRow(
{ first: "Ada", last: "Lovelace", born: "1815" },
0,
{
mappings: [
{ sourceColumn: "born", targetField: "birthYear", type: "number" },
],
computedFields: [
{
targetField: "fullName",
sourceColumns: ["first", "last"],
compute: (v) => `${String(v.first)} ${String(v.last)}`,
},
],
validations: [
{ field: "birthYear", rule: "min", param: 1800 },
],
}
);
// result.row → { birthYear: 1815, fullName: "Ada Lovelace" }
```
## API Reference
- `coerceType(value: unknown, type: string): CoerceResult`
- `validateField(value, rule: FieldValidation, rowIndex): TransformError | undefined`
- `transformRow(source, rowIndex, config: TransformPipelineConfig): TransformRowResult`
- `transformRows(sources, config): { rows; errors }`
- `toRowError(err: TransformError): RowError`
- `interface TransformMapping { sourceColumn; targetField; type?; defaultValue?; required?; transform? }`
- `interface ComputedField { targetField; sourceColumns; compute }`
- `interface FieldValidation { field; rule; param?; message? }`
- `interface TransformError { row; field; message; value? }`
- `interface TransformRowResult { success; row; errors }`
- `interface TransformPipelineConfig { mappings; computedFields?; validations? }`
- `interface CoerceResult { ok; value }`
## Implementation Notes
- Coercion short-circuits on empty / null / undefined to `{ ok: true,
value: undefined }` so a downstream `defaultValue` can still apply.
- The transform function (`mapping.transform`) is wrapped in `try` so a
thrown library never escapes the pipeline; the row's `success` flag
reports the failure and the rest of the mappings still process.
- `custom` validation rules are intentionally no-op — extension is via
the `transform` callback on the mapping, which has access to the raw
value. Custom rules without a default are documented in the
enterprise audit as "extended by consumers."
- `validateField` runs after both mapping and computed-field stages, so
validations target the post-coercion result, not the raw source row.