82 lines
2.9 KiB
Markdown
82 lines
2.9 KiB
Markdown
# OpenSpec Change: enterprise-api-plugin-import
|
|
|
|
- **Change ID**: enterprise-api-plugin-import
|
|
- **Package**: `@suaveplan/api-plugin-import`
|
|
- **Category**: api/data
|
|
- **Target path**: `api/data/api-plugin-import`
|
|
- **Status**: PROPOSED
|
|
- **Date**: 2026-04-24
|
|
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/api-plugin-import.md`
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Deliver `@suaveplan/api-plugin-import` on the genesis branch from scratch at enterprise quality. Data import plugin: async import job queue supporting CSV, JSONL, and XML (via `fast-xml-parser`); auto-format detection; row-level validation with per-row error collection; configurable column mapping and transformation; dry-run mode; duplicate detection via `@suaveplan/dedup`; PII masking via `@suaveplan/data-masking`; streaming ingestion via Web Streams for large files. Routes under `/imports`. Per-table SurrealQL. Events: `import:job:started`, `import:job:completed`, `import:job:failed`, `import:row:error`.
|
|
|
|
---
|
|
|
|
## Available workspace deps on genesis
|
|
|
|
| Package | Role |
|
|
|---|---|
|
|
| `@suaveplan/api` (`workspace:^`) | `definePlugin` |
|
|
| `@suaveplan/api-shared` (`workspace:^`) | Helpers |
|
|
| `@suaveplan/api-plugin-auth` (`workspace:^`) | `requireAuth` |
|
|
| `@suaveplan/api-plugin-database` (`workspace:^`) | SurrealDB |
|
|
| `@suaveplan/api-plugin-file-upload` (`workspace:^`) | Source file staging |
|
|
| `@suaveplan/dedup` (`workspace:^`) | Duplicate detection |
|
|
| `@suaveplan/data-masking` (`workspace:^`) | PII masking |
|
|
| `@suaveplan/serialize` (`workspace:^`) | Streaming deserialization |
|
|
| `@suaveplan/events` (`workspace:^`) | Event bus |
|
|
| `@suaveplan/error` (`workspace:^`) | `ImportError` hierarchy |
|
|
| `@suaveplan/rate-limiter` (`workspace:^`) | Rate limits |
|
|
| `fast-xml-parser` (`catalog:data`) | XML parsing |
|
|
| `hono` + `@hono/zod-openapi` + `zod` | Routes |
|
|
| `@opentelemetry/api` | OTel |
|
|
|
|
---
|
|
|
|
## Public API surface
|
|
|
|
```ts
|
|
function createImportPlugin(config?: ImportPluginConfig): PluginDescriptor
|
|
// Services: ImportJobService, FormatDetectionService, ValidationService, IngestionService
|
|
// Routes: POST /imports, GET /imports/:id, DELETE /imports/:id
|
|
interface ImportTargetAdapter { upsert(rows: unknown[]): Promise<ImportBatchResult> }
|
|
```
|
|
|
|
### Error codes
|
|
|
|
```ts
|
|
type ImportErrorCode =
|
|
| "IMPORT_JOB_NOT_FOUND"
|
|
| "IMPORT_FORMAT_UNKNOWN"
|
|
| "IMPORT_VALIDATION_FAILED"
|
|
| "IMPORT_TARGET_ERROR"
|
|
| "IMPORT_FILE_TOO_LARGE"
|
|
```
|
|
|
|
---
|
|
|
|
## File layout
|
|
|
|
```text
|
|
api/data/api-plugin-import/
|
|
package.json tsconfig.json vitest.config.ts
|
|
surql/ (import_job, import_row_error tables)
|
|
src/
|
|
index.ts events.ts types/ errors/
|
|
adapters/import-target.port.ts
|
|
services/import-job/ services/format-detection/
|
|
services/validation/ services/ingestion/
|
|
routes/imports/
|
|
tests/integration.test.ts
|
|
```
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
Prerequisites: `@suaveplan/api`, `@suaveplan/api-plugin-auth`, `@suaveplan/api-plugin-database`, `@suaveplan/api-plugin-file-upload`, `@suaveplan/dedup`, `@suaveplan/data-masking`, `@suaveplan/serialize`.
|