first commit
This commit is contained in:
112
docs/api-stack/api/api-plugin-import/src/transform/pipeline.md
Normal file
112
docs/api-stack/api/api-plugin-import/src/transform/pipeline.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# `transform/pipeline` — declarative transform pipeline
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements a three-stage row-transform pipeline that any consumer can
|
||||
drive without depending on the canonical `ImportService`: per-mapping
|
||||
coerce + transform, computed fields, then field validation. The
|
||||
pipeline is pure (no side effects, no I/O) and never throws — every
|
||||
failure is captured as a `TransformError` and returned in the result
|
||||
object, so callers can collect every failed row in one pass.
|
||||
|
||||
This module powers both the standalone `transformRow` / `transformRows`
|
||||
helpers exported to consumers and the per-row inner loop of
|
||||
`ImportService.startImport`. Keeping the pipeline separate from the
|
||||
service means CLI tools, batch workers, and tests can drive the
|
||||
transform logic against arbitrary input without spinning up an event
|
||||
emitter or store adapter.
|
||||
|
||||
## Features
|
||||
|
||||
- `coerceType` — explicit-result coercion for `number` / `boolean` /
|
||||
`date` with documented "valid" sets (`1`/`0`/`yes`/`no`/`true`/`false`
|
||||
for booleans, anything `Date.parse`-able for dates).
|
||||
- `validateField` — five built-in rules (`required`, `min`, `max`,
|
||||
`pattern`, `custom`); `custom` is an opt-in extension point that
|
||||
always returns `undefined` so callers wire their own.
|
||||
- `transformRow` — applies mappings, computed fields, and validations
|
||||
in order; pushes `TransformError` for each failure and returns
|
||||
`{ success, row, errors }`.
|
||||
- `transformRows` — array fan-out; errored rows are dropped from `rows`
|
||||
but their errors are still collected in `errors`.
|
||||
- `toRowError` — bridge to the service-layer `RowError` shape so the
|
||||
service can re-emit transform failures on the `import.row_error`
|
||||
event topic without duplicating shapes.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { transformRow } from "@suaveplan/api-plugin-import";
|
||||
|
||||
const result = transformRow(
|
||||
{ name: "Ada", age: "37" },
|
||||
0,
|
||||
{
|
||||
mappings: [
|
||||
{ sourceColumn: "name", targetField: "displayName" },
|
||||
{ sourceColumn: "age", targetField: "age", type: "number" },
|
||||
],
|
||||
}
|
||||
);
|
||||
// result.row → { displayName: "Ada", age: 37 }
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { transformRow } from "@suaveplan/api-plugin-import";
|
||||
|
||||
const result = transformRow(
|
||||
{ first: "Ada", last: "Lovelace", born: "1815" },
|
||||
0,
|
||||
{
|
||||
mappings: [
|
||||
{ sourceColumn: "born", targetField: "birthYear", type: "number" },
|
||||
],
|
||||
computedFields: [
|
||||
{
|
||||
targetField: "fullName",
|
||||
sourceColumns: ["first", "last"],
|
||||
compute: (v) => `${String(v.first)} ${String(v.last)}`,
|
||||
},
|
||||
],
|
||||
validations: [
|
||||
{ field: "birthYear", rule: "min", param: 1800 },
|
||||
],
|
||||
}
|
||||
);
|
||||
// result.row → { birthYear: 1815, fullName: "Ada Lovelace" }
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `coerceType(value: unknown, type: string): CoerceResult`
|
||||
- `validateField(value, rule: FieldValidation, rowIndex): TransformError | undefined`
|
||||
- `transformRow(source, rowIndex, config: TransformPipelineConfig): TransformRowResult`
|
||||
- `transformRows(sources, config): { rows; errors }`
|
||||
- `toRowError(err: TransformError): RowError`
|
||||
- `interface TransformMapping { sourceColumn; targetField; type?; defaultValue?; required?; transform? }`
|
||||
- `interface ComputedField { targetField; sourceColumns; compute }`
|
||||
- `interface FieldValidation { field; rule; param?; message? }`
|
||||
- `interface TransformError { row; field; message; value? }`
|
||||
- `interface TransformRowResult { success; row; errors }`
|
||||
- `interface TransformPipelineConfig { mappings; computedFields?; validations? }`
|
||||
- `interface CoerceResult { ok; value }`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Coercion short-circuits on empty / null / undefined to `{ ok: true,
|
||||
value: undefined }` so a downstream `defaultValue` can still apply.
|
||||
- The transform function (`mapping.transform`) is wrapped in `try` so a
|
||||
thrown library never escapes the pipeline; the row's `success` flag
|
||||
reports the failure and the rest of the mappings still process.
|
||||
- `custom` validation rules are intentionally no-op — extension is via
|
||||
the `transform` callback on the mapping, which has access to the raw
|
||||
value. Custom rules without a default are documented in the
|
||||
enterprise audit as "extended by consumers."
|
||||
- `validateField` runs after both mapping and computed-field stages, so
|
||||
validations target the post-coercion result, not the raw source row.
|
||||
Reference in New Issue
Block a user