Initial Commit

openspec/changes/enterprise-api-plugin-import/proposal.md

@@ -0,0 +1,81 @@
+# OpenSpec Change: enterprise-api-plugin-import
+
+- **Change ID**: enterprise-api-plugin-import
+- **Package**: `@suaveplan/api-plugin-import`
+- **Category**: api/data
+- **Target path**: `api/data/api-plugin-import`
+- **Status**: PROPOSED
+- **Date**: 2026-04-24
+- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/api-plugin-import.md`
+
+---
+
+## Summary
+
+Deliver `@suaveplan/api-plugin-import` on the genesis branch from scratch at enterprise quality. Data import plugin: async import job queue supporting CSV, JSONL, and XML (via `fast-xml-parser`); auto-format detection; row-level validation with per-row error collection; configurable column mapping and transformation; dry-run mode; duplicate detection via `@suaveplan/dedup`; PII masking via `@suaveplan/data-masking`; streaming ingestion via Web Streams for large files. Routes under `/imports`. Per-table SurrealQL. Events: `import:job:started`, `import:job:completed`, `import:job:failed`, `import:row:error`.
+
+---
+
+## Available workspace deps on genesis
+
+| Package | Role |
+|---|---|
+| `@suaveplan/api` (`workspace:^`) | `definePlugin` |
+| `@suaveplan/api-shared` (`workspace:^`) | Helpers |
+| `@suaveplan/api-plugin-auth` (`workspace:^`) | `requireAuth` |
+| `@suaveplan/api-plugin-database` (`workspace:^`) | SurrealDB |
+| `@suaveplan/api-plugin-file-upload` (`workspace:^`) | Source file staging |
+| `@suaveplan/dedup` (`workspace:^`) | Duplicate detection |
+| `@suaveplan/data-masking` (`workspace:^`) | PII masking |
+| `@suaveplan/serialize` (`workspace:^`) | Streaming deserialization |
+| `@suaveplan/events` (`workspace:^`) | Event bus |
+| `@suaveplan/error` (`workspace:^`) | `ImportError` hierarchy |
+| `@suaveplan/rate-limiter` (`workspace:^`) | Rate limits |
+| `fast-xml-parser` (`catalog:data`) | XML parsing |
+| `hono` + `@hono/zod-openapi` + `zod` | Routes |
+| `@opentelemetry/api` | OTel |
+
+---
+
+## Public API surface
+
+```ts
+function createImportPlugin(config?: ImportPluginConfig): PluginDescriptor
+// Services: ImportJobService, FormatDetectionService, ValidationService, IngestionService
+// Routes: POST /imports, GET /imports/:id, DELETE /imports/:id
+interface ImportTargetAdapter { upsert(rows: unknown[]): Promise<ImportBatchResult> }
+```
+
+### Error codes
+
+```ts
+type ImportErrorCode =
+  | "IMPORT_JOB_NOT_FOUND"
+  | "IMPORT_FORMAT_UNKNOWN"
+  | "IMPORT_VALIDATION_FAILED"
+  | "IMPORT_TARGET_ERROR"
+  | "IMPORT_FILE_TOO_LARGE"
+```
+
+---
+
+## File layout
+
+```text
+api/data/api-plugin-import/
+  package.json  tsconfig.json  vitest.config.ts
+  surql/ (import_job, import_row_error tables)
+  src/
+    index.ts  events.ts  types/  errors/
+    adapters/import-target.port.ts
+    services/import-job/  services/format-detection/
+    services/validation/  services/ingestion/
+    routes/imports/
+  tests/integration.test.ts
+```
+
+---
+
+## Impact
+
+Prerequisites: `@suaveplan/api`, `@suaveplan/api-plugin-auth`, `@suaveplan/api-plugin-database`, `@suaveplan/api-plugin-file-upload`, `@suaveplan/dedup`, `@suaveplan/data-masking`, `@suaveplan/serialize`.

openspec/changes/enterprise-api-plugin-import/tasks.md

@@ -0,0 +1,56 @@
+# Tasks: enterprise-api-plugin-import
+
+**Change ID**: enterprise-api-plugin-import
+**Package**: `@suaveplan/api-plugin-import`
+**Target path**: `api/data/api-plugin-import`
+**Prerequisite packages on Verdaccio**: `@suaveplan/api`, `@suaveplan/api-shared`, `@suaveplan/api-plugin-auth`, `@suaveplan/api-plugin-database`, `@suaveplan/api-plugin-file-upload`, `@suaveplan/dedup`, `@suaveplan/data-masking`, `@suaveplan/serialize`, `@suaveplan/events`, `@suaveplan/rate-limiter`, `@suaveplan/error`
+
+---
+
+## 0. Pre-flight (non-negotiable)
+
+- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
+- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
+- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
+
+## 1. Scaffold + types + errors + events
+
+- [ ] Create directory; `package.json` v0.0.0; standard tooling.
+- [ ] Types: `ImportJob`, `ImportRowError`, `ImportTargetAdapter`, `ImportBatchResult`, `ImportPluginConfig` (Zod).
+- [ ] Errors: 5-code union + registration. Test + doc.
+- [ ] Events: `ImportEventMap` (import:job:started, import:job:completed, import:job:failed, import:row:error).
+
+---
+
+## 2. SurrealQL
+
+- [ ] `import_job.surql`, `import_row_error.surql`. tenantId indexes.
+
+---
+
+## 3. Services
+
+- [ ] **FormatDetectionService**: detect CSV/JSONL/XML from MIME type + first bytes. Test + doc.
+- [ ] **ValidationService**: run per-row Zod schema; collect `ImportRowError` records; return summary. Test + doc.
+- [ ] **IngestionService**: stream-parse file; apply column mapping; deduplicate via `@suaveplan/dedup`; batch upsert via `ImportTargetAdapter`. Dry-run mode: validate only, no upsert. Test + doc.
+- [ ] **ImportJobService**: `createJob` (file ref from `api-plugin-file-upload`), `getJob`, `listJobs`, `deleteJob`. Persist row-error records. Test + doc.
+
+---
+
+## 4. Routes
+
+- [ ] `POST /imports` — `requireAuth` + rate-limit; create async import job with file ref + config.
+- [ ] `GET /imports` — `requireAuth`; list jobs.
+- [ ] `GET /imports/:id` — `requireAuth`; status + progress + row errors.
+- [ ] `DELETE /imports/:id` — `requireAuth`.
+- [ ] Route `.test.ts` + `.md`.
+
+---
+
+## 5. Plugin factory + integration tests + publish
+
+- [ ] `createImportPlugin`: `definePlugin`, `dependencies: ["database", "auth"]`, `migrationsPath: "./surql"`.
+- [ ] Integration: upload CSV → import → row errors collected → retry corrected rows (real SurrealDB).
+- [ ] `bunx turbo lint typecheck test --filter=@suaveplan/api-plugin-import` — green; 100% coverage.
+- [ ] `bun scripts/publish.ts --package @suaveplan/api-plugin-import`.
+- [ ] Archive → `openspec/archive/2026-04-24-enterprise-api-plugin-import/`.