683 lines
49 KiB
Markdown
683 lines
49 KiB
Markdown
<!-- markdownlint-disable MD024 -->
|
||
|
||
# Capability Spec Delta: api-host (api-host)
|
||
|
||
**Package**: `@suaveplan/api-host`
|
||
**Status**: PROPOSED
|
||
|
||
> Every requirement below carries a `**Verified by:**` line at authoring time. (See `openspec/AGENTS.md` Section 13.) Requirements are grouped by surface cluster: lifecycle, hot reload, capability registry, plugin ledger, admin API, error boundary, OTel, `definePlugin` DX.
|
||
|
||
## ADDED Requirements
|
||
|
||
### Requirement: Lifecycle FSM drives every plugin through documented states
|
||
|
||
The package SHALL implement a per-plugin lifecycle driver covering every state and transition from design doc §4: `uninstalled`, `installing`, `installed`, `starting`, `started`, `rolled_back`, `crashed`, `stopping`, `stopped`, `uninstalling`, `install_failed`, `start_failed`, `uninstall_failed`. Each plugin SHALL have a per-plugin mutex; concurrent transition triggers MUST queue (only one transition in flight at a time). The FSM MUST emit `plugin.lifecycle_transitions_total{plugin, from, to, outcome}` on every transition. A handler throw inside a route or job tick MUST NOT transition the FSM — that is the error boundary's job (see "Error boundary" requirement).
|
||
|
||
#### Scenario: every documented transition is reachable
|
||
|
||
- **WHEN** a freshly-installed plugin is driven `install → installed → starting → started → stopping → stopped → uninstalling → uninstalled`
|
||
- **THEN** every transition fires AND `plugin.lifecycle_transitions_total` records each one
|
||
|
||
#### Scenario: per-plugin mutex collapses concurrent triggers
|
||
|
||
- **WHEN** two `start()` calls arrive in the same microtask
|
||
- **THEN** the second awaits the first AND only one `starting → started` transition is recorded
|
||
|
||
#### Scenario: handler throw does not transition the FSM
|
||
|
||
- **WHEN** a route handler throws `SuaveplanError`
|
||
- **THEN** the error boundary catches it AND the plugin's state remains `started`
|
||
|
||
**Verified by:** `src/lifecycle/lifecycle.test.ts::"every documented transition fires"` AND `src/lifecycle/lifecycle.test.ts::"per-plugin mutex collapses concurrent start triggers"` AND `src/error-boundary/error-boundary.test.ts::"handler throw does not transition FSM"`
|
||
|
||
---
|
||
|
||
### Requirement: `crashed` state is reachable only from worker tiers
|
||
|
||
The package SHALL transition a plugin to `crashed` ONLY when its `isolation` tier is `isolated` or `sandboxed` AND its worker dies, OOMs, or exceeds its wall-clock budget. Trusted-tier plugins cannot reach `crashed` — a handler throw stays in `started` via the error boundary. Auto-restart with exponential backoff MUST trigger up to `crashRestartMax` (default 3) within `crashRestartWindowMs`; after the budget is exhausted the plugin parks in `crashed` until an operator calls `POST /admin/plugins/{name}/restart`.
|
||
|
||
#### Scenario: trusted plugin handler throw stays started
|
||
|
||
- **WHEN** a `trusted`-tier plugin's handler throws an uncaught exception
|
||
- **THEN** the plugin state remains `started` AND no `crashed` transition is recorded
|
||
|
||
#### Scenario: isolated plugin worker death triggers `crashed`
|
||
|
||
- **WHEN** an `isolated`-tier plugin's worker emits an `exit` event with non-zero code
|
||
- **THEN** the plugin transitions to `crashed` AND auto-restart fires once after backoff
|
||
|
||
**Verified by:** `src/lifecycle/lifecycle.spec-conformance.test.ts::"trusted tier handler throw does not crash"` AND `src/lifecycle/lifecycle.spec-conformance.test.ts::"isolated tier worker exit transitions to crashed"`
|
||
|
||
---
|
||
|
||
### Requirement: Hot reload uses blue-green primitive by default
|
||
|
||
The package SHALL implement the §8.1 blue-green reload primitive. Steps 1–8 of §8.1 MUST execute in order: snapshot to `RollbackJournal`, re-resolve manifest, import new module under `?v=N+1`, run migrations if `manifest.database.schemaVersion` increased, mount shadow router slot, run new `start()` + health probe, atomically flip the live-version pointer, drain the old slot. `host.config.blueGreen: false` MUST opt into the 503-window mode (§8.5) for environments that cannot tolerate two concurrent module instances. Native bindings present in the dep graph MUST refuse reload with `RELOAD_NATIVE_BINDING_HELD`.
|
||
|
||
#### Scenario: blue-green flip is atomic and serves both old and new
|
||
|
||
- **WHEN** a reload is in progress AND the version pointer has been flipped from N to N+1
|
||
- **THEN** new requests hit module N+1 AND in-flight requests continue against N until they settle or `drainDeadlineMs` (default 30s) elapses
|
||
|
||
#### Scenario: native binding refuses reload
|
||
|
||
- **WHEN** the dep graph of the new module imports a `.node` file
|
||
- **THEN** the loader throws `RELOAD_NATIVE_BINDING_HELD` AND the shadow slot is discarded AND the live module continues serving
|
||
|
||
#### Scenario: 503-window mode is opt-in via host config
|
||
|
||
- **WHEN** `host.config.blueGreen === false`
|
||
- **THEN** reload runs in single-slot drain-then-replace mode AND new requests during the drain window receive 503 + `Retry-After: 1`
|
||
|
||
**Verified by:** `src/hot-reload/blue-green.test.ts::"blue-green flip is atomic and drains old slot"` AND `src/hot-reload/blue-green.test.ts::"native binding refuses reload"` AND `src/hot-reload/blue-green.test.ts::"503-window mode opt-in"`
|
||
|
||
---
|
||
|
||
### Requirement: RollbackJournal pins last 3 known-good specifiers and auto-rolls-back
|
||
|
||
The package SHALL maintain a per-plugin ring buffer of depth 3 (configurable). Each entry SHALL match the §8.1a shape: `specifier`, `manifestHash`, `moduleRef: WeakRef<Module> | null`, `migrationCursor`, `capturedAt`, `version`. Entries SHALL be persisted to the plugin ledger so that after a host restart the rollback path still works (with one extra module import; the in-memory `moduleRef` is lost on restart). A `start_failed` event during a reload MUST auto-transition the plugin to `rolled_back` by re-activating the most-recent journal entry's specifier — operator action is NOT required for the happy rollback path.
|
||
|
||
#### Scenario: ring buffer truncates to last 3 entries
|
||
|
||
- **WHEN** five successful reloads happen
|
||
- **THEN** `journal.length === 3` AND only the last three entries remain
|
||
|
||
#### Scenario: auto-rollback on `start_failed` during reload
|
||
|
||
- **WHEN** the new module's `start()` throws during a hot reload
|
||
- **THEN** the plugin transitions `start_failed → rolled_back` AND the previous specifier from the journal is re-activated AND `plugin.rolled_back` is emitted
|
||
|
||
#### Scenario: persisted cursor survives host restart
|
||
|
||
- **WHEN** the host restarts AND the in-memory `moduleRef` is gone
|
||
- **THEN** the persisted `specifier` + `migrationCursor` + `manifestHash` from the ledger let the host re-import the previous version with one additional `import(...)` call
|
||
|
||
**Verified by:** `src/hot-reload/rollback-journal.test.ts::"ring buffer truncates to last 3"` AND `src/hot-reload/auto-rollback.test.ts::"start_failed during reload auto-rolls-back"` AND `src/hot-reload/rollback-journal.test.ts::"persisted cursor survives restart"`
|
||
|
||
---
|
||
|
||
### Requirement: CapabilityRegistry validates version compat at install
|
||
|
||
The package SHALL expose a `CapabilityRegistry` accepting `defineCapability({ id, version, scope, provider, disposer })` registrations. On `install()` of any plugin, the registry MUST check every entry in the plugin's `requiredCapabilities` against (a) presence in the registry AND (b) version compatibility against the plugin's `apiHost` semver range. Missing → `CAPABILITY_NOT_REGISTERED`. Mismatch → `CAPABILITY_VERSION_MISMATCH`. Duplicate registration of the same `id` → `CAPABILITY_DUPLICATE_REGISTRATION`. Disposers MUST run in reverse-registration order on `stop()`.
|
||
|
||
#### Scenario: missing required capability blocks install
|
||
|
||
- **WHEN** a plugin declares `requiredCapabilities: ["db"]` AND no `db` adapter is registered
|
||
- **THEN** install throws `CAPABILITY_NOT_REGISTERED` AND the plugin remains `installing` → `install_failed`
|
||
|
||
#### Scenario: disposer order is reverse-registration
|
||
|
||
- **WHEN** capabilities are registered in order `[logger, events, db]` AND `stop()` runs
|
||
- **THEN** `db.disposer` runs first AND `logger.disposer` runs last
|
||
|
||
**Verified by:** `src/capability-registry/capability-registry.test.ts::"missing required capability throws"` AND `src/capability-registry/capability-registry.test.ts::"disposer order is reverse-registration"`
|
||
|
||
---
|
||
|
||
### Requirement: Plugin ledger metadata DB ships its own forward-only migrations
|
||
|
||
The package SHALL own a metadata DB whose schema is created and migrated by this package's own migration runner. Tables `suaveplan_plugin_ledger`, `suaveplan_rollback_journal`, `suaveplan_saga_instances`, `suaveplan_saga_steps`, `suaveplan_2pc_transactions` MUST exist after the runner completes. Migrations MUST be forward-only AND idempotent — running the runner twice MUST be a no-op. The default DB SHALL be SQLite at `host.config.metadataDbPath` (default `./suaveplan-host.db`); operators MAY override to Postgres or Surreal via `@suaveplan/db`. The ledger DB connection MUST NOT be shared with any plugin's own DB connection.
|
||
|
||
#### Scenario: every documented table exists after migration
|
||
|
||
- **WHEN** the migration runner completes against a fresh SQLite file
|
||
- **THEN** every documented table is present with the documented columns
|
||
|
||
#### Scenario: migration runner is idempotent
|
||
|
||
- **WHEN** the runner is invoked twice in sequence
|
||
- **THEN** the second run is a no-op AND no error is thrown
|
||
|
||
**Verified by:** `src/plugin-ledger/migrations.test.ts::"every table exists after migration"` AND `src/plugin-ledger/migrations.test.ts::"migration runner is idempotent"`
|
||
|
||
---
|
||
|
||
### Requirement: Admin API surface is gated by `host:admin` RBAC and rate-limited
|
||
|
||
The package SHALL mount the §8.4 admin endpoints under `host.config.adminPrefix` (default `/admin`, `null` disables HTTP admin entirely). Every endpoint SHALL require auth (no `publicRoutes` bypass) AND the `host:admin` scope. Per-principal rate limit MUST be 60 ops/min on read endpoints AND 1 op/min on destructive endpoints (`uninstall`, `force-uninstall`, `disable`). Every state-changing call MUST emit a `host.admin_action` audit envelope per §12.5 AND a `host.admin_action_total{action, outcome}` counter.
|
||
|
||
#### Scenario: every documented endpoint is mounted
|
||
|
||
- **WHEN** the host boots with default config
|
||
- **THEN** every endpoint from §8.4 (`POST /admin/plugins/{name}/reload`, `disable`, `enable`, `install`, `uninstall`, `retry-install`, `retry-start`, `force-uninstall`, `GET /admin/plugins`, `GET /admin/plugins/{name}`, `GET /admin/capabilities`) is mounted
|
||
|
||
#### Scenario: `adminPrefix: null` disables HTTP admin
|
||
|
||
- **WHEN** `host.config.adminPrefix === null`
|
||
- **THEN** no admin routes are mounted AND `GET /admin/plugins` returns 404
|
||
|
||
#### Scenario: destructive endpoint rate-limit kicks in at 2 requests / min
|
||
|
||
- **WHEN** a single principal calls `POST /admin/plugins/x/uninstall` twice within 60 seconds
|
||
- **THEN** the second call returns 429
|
||
|
||
**Verified by:** `src/admin-api/admin.test.ts::"every documented endpoint is mounted"` AND `src/admin-api/disabled.test.ts::"adminPrefix null disables admin routes"` AND `src/admin-api/admin.test.ts::"destructive rate-limit is 1 per minute"`
|
||
|
||
---
|
||
|
||
### Requirement: Error boundary produces the standard envelope and never leaks internals
|
||
|
||
The package SHALL wrap every pipeline step (§6 steps 1–11) in an error boundary. The boundary MUST map: (a) `SuaveplanError` → status from the plugin manifest's `errors[code].status` map; codes not in the map → 500 with the code preserved in the envelope's `code` field; (b) `AbortError` → 499 (client gone); (c) anything else → 500 + log of the cause chain. The response body SHALL be the `ErrorEnvelope` from `api-types/errors` — `{ error: { code, message, status, requestId, traceId, details } }`. The `X-Trace-Id` response header MUST mirror `traceId`. `details` SHALL be redacted per manifest's `errors.redact[code]` policy AND auto-redacted by `@suaveplan/data-masking` for emails / CC shapes / JWT shapes.
|
||
|
||
#### Scenario: unmapped code becomes 500 with code preserved
|
||
|
||
- **WHEN** a handler throws `SuaveplanError` with code `PLUGIN.X.NOT_IN_MAP`
|
||
- **THEN** the response status is 500 AND `body.error.code === "PLUGIN.X.NOT_IN_MAP"`
|
||
|
||
#### Scenario: AbortError becomes 499
|
||
|
||
- **WHEN** the client drops the connection mid-request
|
||
- **THEN** the boundary returns 499 AND no `api.errors_total` counter is incremented
|
||
|
||
#### Scenario: trace id mirrors response header
|
||
|
||
- **WHEN** any error response is produced
|
||
- **THEN** `res.headers["x-trace-id"] === body.error.traceId`
|
||
|
||
**Verified by:** `src/error-boundary/error-boundary.test.ts::"unmapped code preserves code in envelope"` AND `src/error-boundary/error-boundary.test.ts::"AbortError maps to 499"` AND `src/error-boundary/error-boundary.test.ts::"x-trace-id mirrors envelope traceId"`
|
||
|
||
---
|
||
|
||
### Requirement: Per-request pipeline orders the documented 11 steps and exposes OTel spans
|
||
|
||
The package SHALL execute the §6 pipeline in the documented order: HTTP intake → route match → auth resolver → tenant resolver chain → (optional) rate limit → (optional) RBAC → body/params/query parse+validate → (optional) idempotency → capability scoping → plugin handler → response validation → response composition → telemetry close. Plugins MAY insert middleware between steps 7 (capability scoping) and 8 (handler) and NOWHERE ELSE. Streaming routes (`streaming: true`) MUST bypass steps 6 (body buffering) and 10 (response composition's buffering). Cookie-based auth MUST trigger CSRF verification via `Bun.CSRF` on non-GET/HEAD methods unless the route opts out with `csrf: false`. Response-validation strictness SHALL default to `enforce` (production: 500 on violation) and `warn` (development: log + counter, body passes through).
|
||
|
||
#### Scenario: steps execute in documented order
|
||
|
||
- **WHEN** a request flows through an instrumented pipeline
|
||
- **THEN** the observed span tree matches §12.3 exactly (`api.request > api.route_match > api.auth.resolve > api.tenant.resolve > api.rbac.check > api.body.parse > api.body.validate > plugin.handler > api.response.validate > api.response.compose`)
|
||
|
||
#### Scenario: plugin middleware insertion is restricted
|
||
|
||
- **WHEN** a plugin attempts to register middleware that runs BEFORE auth
|
||
- **THEN** the host throws `PIPELINE_MIDDLEWARE_SLOT_INVALID` at register time
|
||
|
||
#### Scenario: cookie auth triggers CSRF on POST
|
||
|
||
- **WHEN** an `auth.session` adapter sets a cookie-source flag AND a request arrives as `POST /x` without a CSRF token
|
||
- **THEN** the pipeline returns 403 before reaching the handler
|
||
|
||
**Verified by:** `src/request-pipeline/pipeline.test.ts::"steps execute in documented order"` AND `src/request-pipeline/pipeline.test.ts::"middleware insertion only between steps 7 and 8"` AND `src/request-pipeline/pipeline.test.ts::"cookie auth triggers CSRF on non-GET"`
|
||
|
||
---
|
||
|
||
### Requirement: OTel surface emits every documented metric and span
|
||
|
||
The package SHALL emit every metric listed in §12.3 from at least one production path: `api.requests_total`, `api.request_duration_ms`, `api.body.bytes`, `api.response.bytes`, `api.errors_total`, `api.auth.failures_total`, `api.rbac.denials_total`, `api.tenant.unresolved_total`, `api.validation.failures_total`, `api.idempotency.replays_total`, `api.rate_limit.exceeded_total`, `plugin.lifecycle_transitions_total`, `plugin.reload_total`, `plugin.reload_duration_ms`, `plugin.unsafe_query_total`, `host.admin_action_total`. Each metric SHALL carry the documented label set. Tenant label MUST be opt-in via `host.config.otel.tenantLabelOptIn` to bound cardinality.
|
||
|
||
#### Scenario: every documented metric appears in the in-memory exporter
|
||
|
||
- **WHEN** the test exercises one production path per metric AND collects from `InMemoryMetricExporter`
|
||
- **THEN** every metric name from §12.3 is present in the collected output with non-empty data points
|
||
|
||
**Verified by:** `src/otel/otel.test.ts::"every documented metric is emitted"`
|
||
|
||
---
|
||
|
||
### Requirement: `definePlugin` types the capability map from `requires` literals
|
||
|
||
The package SHALL export `definePlugin<const Requires extends readonly CapabilityId[]>(spec)`. The `ctx` parameter passed to user lifecycle / route handlers SHALL have exactly the fields named in `requires` plus the always-present base fields (`plugin`, `logger`, `events`, `config`, `jobs`, `abortSignal`, `instruments`). Accessing an undeclared capability MUST be a compile-time TypeScript error — `ctx.secrets` on a plugin that didn't declare `secrets` does NOT type-check.
|
||
|
||
#### Scenario: declared capability is typed and reachable
|
||
|
||
- **WHEN** a plugin declares `requires: ["db", "auth.session"] as const`
|
||
- **THEN** `ctx.db` is typed as `AdminDatabaseLike` in lifecycle hooks AND `ctx.principal` is typed as `Principal | null` in route handlers
|
||
|
||
#### Scenario: undeclared capability is a compile error
|
||
|
||
- **WHEN** a plugin declares `requires: ["db"] as const` AND references `ctx.secrets.get("X")`
|
||
- **THEN** TypeScript reports an error AND `expectTypeOf<typeof ctx>().not.toHaveProperty("secrets")` passes
|
||
|
||
**Verified by:** `src/define-plugin/define-plugin.spec-conformance.test.ts::"declared capability is typed in ctx"` AND `src/define-plugin/define-plugin.spec-conformance.test.ts::"undeclared capability is a TS error"`
|
||
|
||
---
|
||
|
||
### Requirement: Health and readiness endpoints reflect plugin states
|
||
|
||
The package SHALL mount `GET /internal/livez` (200 if process is responding; no plugin checks) AND `GET /internal/readyz` (200 only if every plugin's state is `started` OR appears in an explicit allowlist; 409 + structured body otherwise). A periodic background health check (default every 10s) SHALL poll each plugin's optional `health` capability hook; a plugin reporting `unhealthy` for longer than `healthFailWindowMs` MUST emit a `plugin.unhealthy` event AND optionally auto-disable per host config (off by default).
|
||
|
||
#### Scenario: livez is independent of plugin state
|
||
|
||
- **WHEN** every plugin is `install_failed` AND `/internal/livez` is called
|
||
- **THEN** the response is 200
|
||
|
||
#### Scenario: readyz returns 409 when any plugin is not started
|
||
|
||
- **WHEN** one plugin is `crashed` AND `/internal/readyz` is called
|
||
- **THEN** the response is 409 AND the body lists the unready plugins with their states
|
||
|
||
**Verified by:** `src/host/health.test.ts::"livez is independent of plugin state"` AND `src/host/health.test.ts::"readyz returns 409 when a plugin is not started"`
|
||
|
||
---
|
||
|
||
### Requirement: `./testing` subpath is reserved in Stage 2
|
||
|
||
The package SHALL declare `./testing` as a subpath export AND ship a typed placeholder module so that consumers can wire build-tooling resolution today. The placeholder SHALL export NOTHING runtime. Stage 5 populates this subpath with `createTestHost` + in-memory adapter wiring; until then, importing it is a deliberate no-op that signals "the API exists; the implementation lands in Stage 5."
|
||
|
||
#### Scenario: subpath resolves but exports nothing runtime
|
||
|
||
- **WHEN** the test imports `@suaveplan/api-host/testing`
|
||
- **THEN** the import succeeds at type-check time AND `Object.keys(mod)` is empty at runtime
|
||
|
||
**Verified by:** `tests/testing-subpath.test.ts::"./testing subpath is reserved and empty"`
|
||
|
||
---
|
||
|
||
## ADDED Requirements — Chunk B (Lifecycle + Hot-Reload drivers)
|
||
|
||
### Requirement: `LifecycleDriver` executes the FSM side-effect descriptors
|
||
|
||
The package SHALL ship `class LifecycleDriver` that orchestrates per-plugin lifecycle through `createPluginFsm` (from `@suaveplan/api-lifecycle`) and executes each `SideEffectOp` in the discriminated union against pluggable adapter surfaces. The driver SHALL expose verbs `install`, `start`, `stop`, `uninstall`, `crash`, `retryInstall`, `retryStart`, `clearRollback`, `forceUninstall`, `repair`, `startForReload`. The driver SHALL emit `plugin.lifecycle_transitions_total{plugin, from, to, trigger, outcome}` on every transition. Per-plugin mutex semantics are inherited from `createPluginFsm`; concurrent triggers queue FIFO.
|
||
|
||
#### Scenario: every documented transition is reachable from operator verbs
|
||
|
||
- **WHEN** a freshly-registered plugin is driven `install → installed → start → started → stop → stopped → uninstall → uninstalled`
|
||
- **THEN** every transition fires AND every side-effect op (`mount-routes`, `register-jobs`, `subscribe-events`, `inject-capabilities`, `call-user-hook`, `mark-health`, `write-ledger`, `register-scopes`, `unmount-routes`, `pause-jobs`, `unsubscribe-events`, `dispose-capabilities`, `force-cancel-stragglers`, `apply-data-policy`, `drop-ledger`, `drop-scopes`) executes against the configured adapter.
|
||
|
||
#### Scenario: typed events fire on the host bus
|
||
|
||
- **WHEN** the driver completes a lifecycle transition that carries an `emit-event` side-effect
|
||
- **THEN** the matching topic (`plugin.installed` / `plugin.started` / `plugin.stopped` / `plugin.uninstalled`) is published on the host `EventBus<LifecycleEventMap>`.
|
||
|
||
#### Scenario: install hook failure parks in `install_failed`
|
||
|
||
- **WHEN** a plugin's `install` hook throws
|
||
- **THEN** the FSM dispatches `installFailed` AND the plugin lands in `install_failed` AND `lastError` records the cause code + message.
|
||
|
||
#### Scenario: start hook auto-retries before parking
|
||
|
||
- **WHEN** a plugin's `start` hook throws on the first attempt but succeeds on retry
|
||
- **THEN** the `start_failed` retry budget is consumed AND the FSM reaches `started` via the auto-retry loop.
|
||
|
||
**Verified by:** `src/lifecycle/lifecycle.test.ts::"drives install → installed → starting → started → stopping → stopped → uninstalling → uninstalled"` AND `src/lifecycle/lifecycle.test.ts::"emits typed events on the host bus"` AND `src/lifecycle/lifecycle.test.ts::"install hook throw parks the plugin in install_failed"` AND `src/lifecycle/lifecycle-coverage.test.ts::"retryStart in runStartRetries can succeed before budget exhaustion"`
|
||
|
||
---
|
||
|
||
### Requirement: `LifecycleDriver.crash` honours the per-plugin budget
|
||
|
||
The driver SHALL transition `started → crashed` on `crash(plugin, cause)` and auto-restart via `crashRestart` up to `crashRestartMax` (default 3) within the `CrashBudgetTracker` window. After the budget is exhausted, the plugin SHALL remain parked in `crashed` until an operator calls a recovery verb. `plugin.crashed` SHALL be emitted on the host bus.
|
||
|
||
#### Scenario: single crash auto-restarts to started
|
||
|
||
- **WHEN** `crash(plugin, new Error("worker died"))` is invoked on a `started` plugin
|
||
- **THEN** the FSM transitions `started → crashed → starting → started` AND `plugin.crashed` is emitted.
|
||
|
||
**Verified by:** `src/lifecycle/lifecycle.test.ts::"crash transitions to crashed then auto-restarts to started"` AND `src/lifecycle/lifecycle.test.ts::"crash budget exhaustion parks the plugin"`
|
||
|
||
---
|
||
|
||
### Requirement: `ReloadDriver` implements the §8.1 blue-green primitive
|
||
|
||
The package SHALL ship `class ReloadDriver` that orchestrates the design §8.1 blue-green hot-reload. Each `reload(plugin, opts)` call SHALL:
|
||
|
||
1. Refuse a concurrent reload with `RELOAD.IN_PROGRESS`.
|
||
2. Snapshot the active activation into the rollback journal.
|
||
3. Apply `requiresReinstall(prev, next)` (from `api-lifecycle`).
|
||
4. Import the new module under `bustSpecifier(url, N+1)` via the injected `ModuleResolver`.
|
||
5. Walk native bindings and refuse the reload with `RELOAD.NATIVE_BINDING_HELD` when any are present.
|
||
6. Stage the new handle in the blue-green shadow slot.
|
||
7. Drive `lifecycle.startForReload(plugin, probe)` which fires the FSM `start` and the host-level health probe; failure dispatches `rollbackTriggered` so the FSM lands in `rolled_back`.
|
||
8. On success, atomically flip the blue-green slot and drain the prior slot.
|
||
9. Emit `plugin.reload_total{plugin, trigger, outcome}` + `plugin.reload_duration_ms` histogram.
|
||
|
||
`blueGreen: false` SHALL opt into the §8.5 503-window mode for environments that cannot host two module instances concurrently.
|
||
|
||
#### Scenario: happy reload flips blue-green and persists the journal
|
||
|
||
- **WHEN** the new module imports cleanly AND `start()` AND `probe()` succeed
|
||
- **THEN** the blue-green adapter receives `stageShadow → flip → releaseDraining` AND the journal gains an entry AND `lastReloadTrigger` matches the trigger argument.
|
||
|
||
#### Scenario: concurrent reload is rejected
|
||
|
||
- **WHEN** a second `reload()` is dispatched while the first is in-flight
|
||
- **THEN** the second rejects with `HostError(RELOAD.IN_PROGRESS)`.
|
||
|
||
#### Scenario: native binding refuses reload
|
||
|
||
- **WHEN** the resolver's `walkNativeBindings()` returns a non-empty array
|
||
- **THEN** the reload throws `HostError(RELOAD.NATIVE_BINDING_HELD)` AND the shadow slot is discarded.
|
||
|
||
#### Scenario: 503-window opt-in skips blue-green slot manipulation
|
||
|
||
- **WHEN** the driver is constructed with `blueGreen: false`
|
||
- **THEN** the blue-green adapter records no `stage / flip / release` calls AND the reload still completes.
|
||
|
||
**Verified by:** `src/hot-reload/hot-reload.test.ts::"imports, stages, flips, drains, and persists the journal"` AND `src/hot-reload/hot-reload.test.ts::"rejects a concurrent reload with RELOAD.IN_PROGRESS"` AND `src/hot-reload/hot-reload.test.ts::"throws NATIVE_BINDING_HELD when walk reports bindings"` AND `src/hot-reload/hot-reload.test.ts::"skips blue-green slot manipulation when blueGreen=false"`
|
||
|
||
---
|
||
|
||
### Requirement: `RollbackJournal` ring buffer + auto-rollback
|
||
|
||
The package SHALL maintain a per-plugin `RollbackJournal` (default depth 3, configurable via `journalDepth`) via the in-memory `createRollbackJournal` from `@suaveplan/api-loader`. Each successful activation SHALL push a `RollbackJournalEntry` carrying `specifier`, `manifestHash`, `moduleRef: WeakRef<object>`, `migrationCursor`, `capturedAt`, `version`. The persisted subset (everything except `moduleRef`) SHALL be written to the configured `RollbackPersistence` adapter so host-restart recovery is possible with one additional `import()` call. On a `start` or health-probe failure during reload, the lifecycle FSM SHALL be auto-driven into `rolled_back` via `startForReload`'s `rollbackTriggered` dispatch AND `plugin.rolled_back` SHALL be emitted on the host bus.
|
||
|
||
#### Scenario: ring buffer truncates to depth on overflow
|
||
|
||
- **WHEN** five successful activations are recorded against a depth-3 journal
|
||
- **THEN** `journal.length === 3` AND only the last three entries remain.
|
||
|
||
#### Scenario: start failure auto-rolls-back
|
||
|
||
- **WHEN** the new module's `start` hook throws during reload
|
||
- **THEN** the lifecycle FSM lands in `rolled_back` AND `plugin.rolled_back` is emitted AND the persisted journal preserves the prior known-good specifier.
|
||
|
||
#### Scenario: persisted entries hydrate after restart
|
||
|
||
- **WHEN** `hydrateFromPersistence(plugin)` is called against a populated `RollbackPersistence`
|
||
- **THEN** the journal is re-populated with entries whose `moduleRef` is `null` (re-imported on demand).
|
||
|
||
#### Scenario: manual rollback flips to the prior journal entry
|
||
|
||
- **WHEN** `rollback(plugin, 0)` is invoked on a populated journal
|
||
- **THEN** the blue-green adapter restages + flips to the prior specifier AND `plugin.rolled_back` is emitted with `cause.code === "ADMIN"`.
|
||
|
||
**Verified by:** `src/hot-reload/hot-reload.test.ts::"emits plugin.rolled_back and drives lifecycle into rolled_back"` AND `src/hot-reload/hot-reload.test.ts::"rolls back to the prior journal entry"` AND `src/hot-reload/hot-reload.test.ts::"hydrates the journal from the ledger"` AND `src/hot-reload/hot-reload.test.ts::"throws JOURNAL_EMPTY when index exceeds depth"`
|
||
|
||
---
|
||
|
||
### Requirement: `FileWatcher` debounces dev-mode reload triggers
|
||
|
||
The package SHALL ship `class FileWatcher` for the §8.3 dev-mode trigger path. `register(plugin, path, trigger)` SHALL wire a watch via an injected `FsWatchAdapter` and collapse a burst of fs events into a single `trigger(plugin)` invocation per debounce window (default 500ms). The watcher SHALL refuse duplicate registration for the same plugin name. `disposeAll()` SHALL cancel every pending debounce timer + tear down every watch.
|
||
|
||
#### Scenario: a burst of fs events fires once
|
||
|
||
- **WHEN** three fs events are emitted in rapid succession
|
||
- **THEN** `trigger` is invoked exactly once after the debounce window elapses.
|
||
|
||
#### Scenario: duplicate registration throws
|
||
|
||
- **WHEN** `register(plugin, ...)` is called twice with the same `plugin` name
|
||
- **THEN** the second call throws synchronously.
|
||
|
||
#### Scenario: disposer cancels pending fires
|
||
|
||
- **WHEN** an fs event is debounced AND the disposer is invoked before the timer fires
|
||
- **THEN** `trigger` is never invoked AND no timers remain.
|
||
|
||
**Verified by:** `src/hot-reload/file-watcher.test.ts::"debounces a burst of events into a single trigger"` AND `src/hot-reload/file-watcher.test.ts::"throws on duplicate registration for the same plugin"` AND `src/hot-reload/file-watcher.test.ts::"disposer cancels pending fires + removes the watch"`
|
||
|
||
---
|
||
|
||
### Requirement: `PluginLedger` persists the host-owned plugin record
|
||
|
||
The package SHALL ship `class PluginLedger` backed by a SQLite-shaped `LedgerDb` handle. The ledger SHALL expose `upsertPlugin(record)`, `getPlugin(name)`, `listPlugins()`, `removePlugin(name)`, `pushJournalEntry(plugin, entry)`, `peekJournal(plugin, n)`, `latestJournal(plugin)`, `journalLength(plugin)`. Adapter views `asLifecycleLedger()` and `asRollbackPersistence()` SHALL conform to the lifecycle driver's `LedgerAdapter` and the reload driver's `RollbackPersistence` contracts so production wiring is mechanical. Construction SHALL run host-owned migrations eagerly unless `autoMigrate: false` is passed.
|
||
|
||
#### Scenario: upsert + read round-trip
|
||
|
||
- **WHEN** `upsertPlugin({ name: "billing", ... })` is followed by `getPlugin("billing")`
|
||
- **THEN** the returned row carries every persisted field (state, version, manifest hash, isolation, counters, capability set).
|
||
|
||
#### Scenario: malformed `capabilities_json` raises `LEDGER.INVALID_ROW`
|
||
|
||
- **WHEN** the underlying row's `capabilities_json` is hand-corrupted to a non-JSON string
|
||
- **THEN** `getPlugin` throws a `HostError(LEDGER.INVALID_ROW)`.
|
||
|
||
#### Scenario: `asLifecycleLedger.write` preserves existing counters
|
||
|
||
- **WHEN** the lifecycle driver writes a partial row against a plugin already carrying `crashCount: 4`
|
||
- **THEN** the resulting row carries `crashCount: 4` AND the new `state`, `version`, and `manifestHash`.
|
||
|
||
**Verified by:** `src/plugin-ledger/plugin-ledger.test.ts::"upserts + reads a plugin row"` AND `src/plugin-ledger/plugin-ledger.test.ts::"raises INVALID_ROW for malformed capabilities JSON"` AND `src/plugin-ledger/plugin-ledger.test.ts::"asLifecycleLedger.write preserves existing counters + isolation"`
|
||
|
||
---
|
||
|
||
### Requirement: `PluginLedger` ring-buffers the rollback journal
|
||
|
||
The journal subset SHALL be stored in `suaveplan_rollback_journal` keyed on `(plugin_name, position)` with `position ∈ [0, journalDepth)` (default depth 3). `pushJournalEntry` SHALL overwrite the oldest slot (by `captured_at` then `position` for ties) when the depth is exceeded. `peekJournal(plugin, n)` SHALL return at most `n` entries ordered newest-first AND validate every row through `rollbackJournalEntryPersistedSchema` from `@suaveplan/api-loader`.
|
||
|
||
#### Scenario: ring buffer trims oldest on overflow
|
||
|
||
- **WHEN** four entries are pushed against a depth-3 journal
|
||
- **THEN** the journal contains three entries AND the trimmed slot now holds the latest entry.
|
||
|
||
#### Scenario: malformed entry on push raises a Zod error
|
||
|
||
- **WHEN** `pushJournalEntry` is called with `manifestHash: "bad"` (not matching the `sha256:` shape)
|
||
- **THEN** the call throws synchronously before any row is written.
|
||
|
||
**Verified by:** `src/plugin-ledger/plugin-ledger.test.ts::"ring-buffer trims oldest on overflow"` AND `src/plugin-ledger/plugin-ledger.test.ts::"rejects malformed entries on push"` AND `src/plugin-ledger/plugin-ledger.test.ts::"ring-buffer breaks ties by lower position"`
|
||
|
||
---
|
||
|
||
### Requirement: `HostMigrationRunner` is forward-only and idempotent
|
||
|
||
The package SHALL ship `class HostMigrationRunner` consuming a `LedgerDb` and an ordered `LedgerMigration[]` (default `HOST_LEDGER_MIGRATIONS`). `migrate()` SHALL apply every declared migration whose `id` has no row in `__host_migrations`, in ascending id order. Re-applying SHALL be a no-op. The runner SHALL refuse to apply when applied ids reference undeclared migrations (`LEDGER.MIGRATION_OUT_OF_ORDER`) and when pending ids are ≤ the maximum applied id. A failed `up` callback SHALL raise `LEDGER.MIGRATION_FAILED` AND NOT record the migration id (so a subsequent retry re-runs it).
|
||
|
||
#### Scenario: applying twice is a no-op
|
||
|
||
- **WHEN** `migrate()` is invoked twice against the same DB
|
||
- **THEN** the second call returns an empty `readonly number[]`.
|
||
|
||
#### Scenario: every host-owned table is created
|
||
|
||
- **WHEN** `migrate()` runs against a fresh DB
|
||
- **THEN** `suaveplan_plugin_ledger`, `suaveplan_rollback_journal`, `suaveplan_saga_instances`, `suaveplan_saga_steps`, `suaveplan_2pc_transactions`, `__host_migrations` all exist.
|
||
|
||
#### Scenario: migration failure parks the runner
|
||
|
||
- **WHEN** a migration callback throws
|
||
- **THEN** the runner raises `LEDGER.MIGRATION_FAILED` AND the migration id is NOT recorded as applied.
|
||
|
||
**Verified by:** `src/plugin-ledger/plugin-ledger.test.ts::"re-running migrate is a no-op (idempotent)"` AND `src/plugin-ledger/plugin-ledger.test.ts::"creates every host-owned table"` AND `src/plugin-ledger/plugin-ledger.test.ts::"raises MIGRATION_FAILED when an up callback throws"`
|
||
|
||
---
|
||
|
||
### Requirement: Admin API mounts every §8.4 endpoint
|
||
|
||
The package SHALL ship `createAdminApi(options)` returning a `Hono` instance exposing every endpoint listed in design §8.4: `GET /plugins`, `GET /plugins/:name`, `GET /plugins/:name/journal`, `GET /capabilities`, `GET /capabilities/:id`, `GET /health`; `POST /plugins/:name/install`, `POST /plugins/:name/uninstall`, `POST /plugins/:name/start`, `POST /plugins/:name/stop`, `POST /plugins/:name/reload`, `POST /plugins/:name/rollback`, `POST /plugins/:name/restart`. The router SHALL short-circuit the plugin pipeline entirely — every endpoint dispatches against the host-owned `LifecycleDriver`, `ReloadDriver`, `CapabilityRegistry`, and `PluginLedger` references.
|
||
|
||
#### Scenario: `GET /plugins` returns the ledger snapshot
|
||
|
||
- **WHEN** a `host:admin` principal hits `GET /plugins`
|
||
- **THEN** the response carries `{ plugins: PluginLedgerRow[] }` and 200 status.
|
||
|
||
#### Scenario: `GET /plugins/:name` returns 404 for unknown
|
||
|
||
- **WHEN** the requested plugin has no ledger row
|
||
- **THEN** the response is `404` with body `{ error: { code: "ADMIN.PLUGIN_NOT_FOUND" } }`.
|
||
|
||
#### Scenario: `GET /capabilities` enumerates registered ids
|
||
|
||
- **WHEN** the registry contains at least one capability
|
||
- **THEN** the response body's `capabilities` array carries one entry per registered id with `{ id, version, scope }`.
|
||
|
||
**Verified by:** `src/admin-api/admin-api.test.ts::"returns the ledger snapshot"` AND `src/admin-api/admin-api.test.ts::"returns 404 for unknown plugin"` AND `src/admin-api/admin-api.test.ts::"lists registered capabilities"`
|
||
|
||
---
|
||
|
||
### Requirement: Admin API enforces `host:admin` scope + per-principal rate limit
|
||
|
||
Every admin endpoint SHALL require an authenticated `AdminPrincipal` carrying the configured `requiredScope` (default `host:admin`). The package SHALL ship `AdminRateLimiter` enforcing a per-principal token bucket — `readRatePerMinute` (default 60) for read endpoints and `destructiveRatePerMinute` (default 1) for mutating endpoints. Missing principal or missing scope SHALL raise `ADMIN.RBAC_DENIED` (HTTP 403); bucket exhaustion SHALL raise `ADMIN.RATE_LIMIT_EXCEEDED` (HTTP 429).
|
||
|
||
#### Scenario: principal without `host:admin` is denied
|
||
|
||
- **WHEN** the resolver returns a principal whose `scopes` lacks `host:admin`
|
||
- **THEN** the response is `403` with `error.code === "ADMIN.RBAC_DENIED"`.
|
||
|
||
#### Scenario: rate-limit bucket exhaustion rejects with 429
|
||
|
||
- **WHEN** the configured read RPM is `1` AND the principal makes two `GET /plugins` calls in the same minute
|
||
- **THEN** the second response is `429` with `error.code === "ADMIN.RATE_LIMIT_EXCEEDED"`.
|
||
|
||
#### Scenario: missing principal short-circuits with audit
|
||
|
||
- **WHEN** the resolver returns `null`
|
||
- **THEN** the response is `403` AND the audit sink records a `failure` envelope with `cause.code === "ADMIN.RBAC_DENIED"`.
|
||
|
||
**Verified by:** `src/admin-api/admin-api.test.ts::"denies principal without scope"` AND `src/admin-api/admin-api.test.ts::"rejects when read bucket is exhausted"` AND `src/admin-api/admin-api.test.ts::"denies missing principal"`
|
||
|
||
---
|
||
|
||
### Requirement: Admin API requires `Idempotency-Key` on mutating endpoints
|
||
|
||
Every mutating endpoint (`install`, `uninstall`, `start`, `stop`, `reload`, `rollback`, `restart`) SHALL require the caller to supply an `Idempotency-Key` header. The header value SHALL be checked against the configured `IdempotencyStore` — a missing header raises `ADMIN.IDEMPOTENCY_KEY_MISSING` (HTTP 400); a replayed key within the TTL window (default 10 minutes) raises `ADMIN.IDEMPOTENCY_KEY_REPLAY` (HTTP 409). The default in-process store SHALL be `InMemoryIdempotencyStore` with lazy TTL pruning.
|
||
|
||
#### Scenario: install without `Idempotency-Key` is rejected
|
||
|
||
- **WHEN** a `POST /plugins/:name/install` is dispatched with no `Idempotency-Key` header
|
||
- **THEN** the response is `400` with `error.code === "ADMIN.IDEMPOTENCY_KEY_MISSING"`.
|
||
|
||
#### Scenario: replayed key rejects with 409
|
||
|
||
- **WHEN** the same `Idempotency-Key` is replayed within the TTL window
|
||
- **THEN** the response is `409` with `error.code === "ADMIN.IDEMPOTENCY_KEY_REPLAY"`.
|
||
|
||
**Verified by:** `src/admin-api/admin-api.test.ts::"install rejects missing Idempotency-Key"` AND `src/admin-api/admin-api.test.ts::"install rejects replayed Idempotency-Key"`
|
||
|
||
---
|
||
|
||
### Requirement: Admin API emits audit envelope + OTel instruments
|
||
|
||
Every admin endpoint outcome SHALL emit a structured `AdminAuditRecord` through the operator-supplied `AdminAuditSink`. The envelope SHALL carry `{ at, principal, action, plugin, params, outcome, cause? }`. Successful + failed outcomes SHALL both produce envelopes so the audit trail captures denied requests. Each call SHALL also increment `host.admin_action_total{action, outcome[, plugin]}` and record the elapsed time on `host.admin_call_duration_ms`. A throwing audit sink SHALL NOT fail the underlying admin operation — the failure SHALL be reported via the metric counter under `action: "host.audit.emit", outcome: "fail"`.
|
||
|
||
#### Scenario: success envelope is emitted on the happy path
|
||
|
||
- **WHEN** a `GET /plugins` succeeds
|
||
- **THEN** the audit sink receives one record with `outcome: "success"`, `action: "host.plugins.list"`, the resolved principal.
|
||
|
||
#### Scenario: failure envelope is emitted on RBAC denial
|
||
|
||
- **WHEN** a principal lacks the required scope
|
||
- **THEN** the audit sink receives one record with `outcome: "failure"`, `cause.code: "ADMIN.RBAC_DENIED"`.
|
||
|
||
#### Scenario: audit sink throw is recorded as a metric, not a 500
|
||
|
||
- **WHEN** the audit sink throws on `emit`
|
||
- **THEN** the admin call still returns 200 AND `host.admin_action_total{action: "host.audit.emit", outcome: "fail"}` increments.
|
||
|
||
**Verified by:** `src/admin-api/admin-api.test.ts::"records a success envelope on a happy path"` AND `src/admin-api/admin-api.test.ts::"records a failure envelope on rate-limit + idempotency + RBAC denials"` AND `src/admin-api/admin-api.test.ts::"emits audit failure when sink throws"`
|
||
|
||
---
|
||
|
||
### Requirement: `LEDGER.*` + new `ADMIN.*` codes are published
|
||
|
||
The package SHALL extend the `ADMIN_*` namespace with `IDEMPOTENCY_KEY_MISSING` and `IDEMPOTENCY_KEY_REPLAY`. It SHALL publish a new `LEDGER.*` namespace with `MIGRATION_FAILED`, `MIGRATION_OUT_OF_ORDER`, `ROW_NOT_FOUND`, `INVALID_ROW`, `DB_UNAVAILABLE`. Every new code SHALL have a non-zero `HOST_DEFAULT_STATUS_MAP` entry.
|
||
|
||
#### Scenario: every new code is mapped
|
||
|
||
- **WHEN** every `LEDGER.*` value is looked up via `statusFor`
|
||
- **THEN** the returned status is a 4xx or 5xx integer.
|
||
|
||
**Verified by:** `src/error-codes/error-codes.test.ts::"LEDGER_CODES carries every documented LEDGER.* reason"` AND `src/error-codes/error-codes.test.ts::"ADMIN_CODES carries every documented ADMIN.* reason"` AND `src/error-codes/error-codes.test.ts::"HOST_DEFAULT_STATUS_MAP maps every published code to a sane HTTP status"`
|
||
|
||
---
|
||
|
||
### Requirement: ErrorBoundary projects every captured throw to a stable envelope shape
|
||
|
||
The package SHALL provide an `ErrorBoundary` consumed by every per-request pipeline step. Each invocation SHALL accept a step name, plugin name (`null` for pre-match steps), per-plugin status-map overlay, optional span, and the work function; it SHALL ALWAYS resolve with a `BoundaryOutcome` (`ok: true | false`), never throw. Projections SHALL follow design doc §12.7:
|
||
|
||
- `SuaveplanError` / `HostError` → manifest `statusMap[code]` ?? `HOST_DEFAULT_STATUS_MAP[code]` ?? `500` (code preserved);
|
||
- `AbortError` (or `code === "ABORT_ERR"`) → `HOST.CLIENT_ABORTED` (499);
|
||
- anything else → `HOST.INTERNAL_INVARIANT` (500) — `runHandler` substitutes `HOST.HANDLER_THREW` so handler bugs are slice-able in OTel.
|
||
|
||
Every failure SHALL increment `api.errors_total{step, code, status[, plugin]}`. When a span is supplied, the boundary SHALL call `recordException` + `setStatus({code: 2, message: <code>})` but SHALL NOT `.end()` the span — the pipeline owns termination. A structured log line SHALL accompany every failure with a cause chain truncated at depth 5.
|
||
|
||
#### Scenario: manifest statusMap overlays default
|
||
|
||
- **WHEN** the manifest declares `errors.statusMap["HOST.SESSION_REQUIRED"] = 418`
|
||
- **THEN** a `HostError("HOST.SESSION_REQUIRED")` thrown inside the wrapped step produces a failure with `status === 418`.
|
||
|
||
#### Scenario: unmapped code falls back to 500 with code preserved
|
||
|
||
- **WHEN** a plugin throws `PLUGIN.X.Y` with no entry in either map
|
||
- **THEN** the boundary returns `status: 500` AND `code: "PLUGIN.X.Y"`.
|
||
|
||
#### Scenario: AbortError → 499
|
||
|
||
- **WHEN** a step throws `new Error("aborted")` with `name === "AbortError"`
|
||
- **THEN** the failure carries `code: "HOST.CLIENT_ABORTED"` and `status: 499`.
|
||
|
||
#### Scenario: runHandler distinguishes handler throws
|
||
|
||
- **WHEN** a `runHandler(...)` call captures `throw new Error("plain")`
|
||
- **THEN** the failure carries `code: "HOST.HANDLER_THREW"`.
|
||
|
||
**Verified by:** `src/error-boundary/error-boundary.test.ts::"manifest statusMap overrides default"` AND `src/error-boundary/error-boundary.test.ts::"unmapped code falls back to 500"` AND `src/error-boundary/error-boundary.test.ts::"AbortError → HOST.CLIENT_ABORTED (499)"` AND `src/error-boundary/error-boundary.test.ts::"runHandler substitutes HOST.HANDLER_THREW on unknown throw"`
|
||
|
||
---
|
||
|
||
### Requirement: Request pipeline runs §6 steps in order and is the only entry point for plugin traffic
|
||
|
||
The package SHALL provide a `RequestPipeline.handle(request: Request)` driver that executes design doc §6 steps in the exact documented order:
|
||
|
||
1. Route match (host) — returns `HOST.ROUTE_NOT_FOUND` (404) when no route matches.
|
||
2. Tenant resolve — non-public routes that fail to resolve a tenant return `HOST.TENANT_NOT_RESOLVED` (400); public routes get a stub `{id: "public", slug: "public"}` context.
|
||
3. Session resolve via the configured `SessionResolverChain` — resolver throws emit `api.auth.failures_total`.
|
||
4. CSRF — required when `route.options.csrf === true`, the request method is not `GET`/`HEAD`, the route is not `streaming`, AND a principal was resolved. Invalid token → `HOST.CSRF_TOKEN_INVALID` (403).
|
||
5. RBAC — `null` principal on a non-public route → `HOST.SESSION_REQUIRED` (401); missing scope → `HOST.RBAC_DENIED` (403) + `api.rbac.denials_total`.
|
||
6. Rate limit — keyed by principal id (when present) else tenant id; bucket-empty → `HOST.RATE_LIMIT_EXCEEDED` (429) + `api.rate_limit.exceeded_total`.
|
||
7. Idempotency — `route.options.idempotent === true` + `Idempotency-Key` header present: replay → `HOST.IDEMPOTENCY_REPLAY` (409) + `api.idempotency.replays_total`.
|
||
8. Request validation — Zod body / params / query when `route.options.runtimeValidation !== false` and route is not streaming; failures emit `HOST.REQUEST_VALIDATION_FAILED` (400) + `api.validation.failures_total{where=request}`.
|
||
9. Plugin handler — invoked through `ErrorBoundary.runHandler` with a `RequestContext` carrying tenant, principal, capabilities, parsed body/params/query.
|
||
10. Response validation — `enforce` (production) → `HOST.RESPONSE_VALIDATION_FAILED` (500) on mismatch; `warn` (development) → success + `api.validation.failures_total{where=response.warn}`.
|
||
11. Audit envelope (when `auditable: true`) + OTel telemetry close — every request emits `api.requests_total{route, plugin, status, outcome}` and `api.request_duration_ms` histogram.
|
||
|
||
The pipeline SHALL always set the `X-Request-Id` response header. Streaming routes (`streaming: true`) SHALL bypass CSRF, request validation buffering, and response validation.
|
||
|
||
#### Scenario: 11 steps execute in order on happy path
|
||
|
||
- **WHEN** a `POST /billing/invoices` with valid principal + scope is dispatched
|
||
- **THEN** the response is 200 AND `X-Request-Id` is set AND an audit envelope is emitted for `auditable: true` routes.
|
||
|
||
#### Scenario: CSRF gates cookie-bound non-GET requests
|
||
|
||
- **WHEN** the route is `csrf: true`, the method is `POST`, the principal is non-null, and the CSRF validator returns `false`
|
||
- **THEN** the response is 403 with `code: "HOST.CSRF_TOKEN_INVALID"`.
|
||
|
||
#### Scenario: streaming bypasses CSRF
|
||
|
||
- **WHEN** the route is `streaming: true, csrf: true` AND CSRF validator would return `false`
|
||
- **THEN** the response still succeeds.
|
||
|
||
#### Scenario: response validation enforce → 500
|
||
|
||
- **WHEN** the handler returns `{wrong: true}` against a `responses["200"].body = z.object({ok: z.boolean()})`
|
||
- **THEN** the response is 500 with `code: "HOST.RESPONSE_VALIDATION_FAILED"`.
|
||
|
||
#### Scenario: response validation warn → success + counter
|
||
|
||
- **WHEN** the pipeline is constructed with `responseValidation: "warn"` and the handler's body mismatches the response schema
|
||
- **THEN** the response is 200 AND `api.validation.failures_total{where: "response.warn"}` increments.
|
||
|
||
**Verified by:** `src/request-pipeline/request-pipeline.test.ts::"happy path 200 with X-Request-Id + audit envelope"` AND `src/request-pipeline/request-pipeline.test.ts::"CSRF required on cookie-auth non-GET, invalid → 403"` AND `src/request-pipeline/request-pipeline.test.ts::"streaming routes bypass CSRF"` AND `src/request-pipeline/request-pipeline.test.ts::"response Zod mismatch in enforce mode → 500"` AND `src/request-pipeline/request-pipeline.test.ts::"response Zod mismatch in warn mode → success"`
|
||
|
||
---
|
||
|
||
### Requirement: `createHost(config)` composes every internal driver
|
||
|
||
The package SHALL expose a top-level `createHost(config: HostConfig)` entry point that composes the lifecycle driver, reload driver, plugin ledger, capability registry, request pipeline, and admin API into a `Host` whose `fetch(request)` method drives the full §6 pipeline. Config SHALL be Zod-validated at the entry; failures raise `HOST.CONFIG_INVALID`. `host.start()` SHALL topo-sort `config.plugins` via `@suaveplan/api-manifest`'s `resolveInstallOrder` and install each plugin in dependency order. `host.stop(graceMs)` SHALL stop plugins in reverse-topo order with a `HOST.SHUTDOWN_TIMEOUT` deadline. The host SHALL mount the admin API at `config.adminPrefix` (default `/admin`, `null` to disable). It SHALL expose `/internal/livez` (200 when responsive) and `/internal/readyz` (200 + plugin list). `installPlugin`/`uninstallPlugin` SHALL operate at runtime; pre-start fetches SHALL return a 503 envelope.
|
||
|
||
#### Scenario: empty host boots; livez/readyz respond
|
||
|
||
- **WHEN** `createHost({adapters, plugins: []})` + `host.start()` runs
|
||
- **THEN** `host.fetch(GET /internal/livez)` returns 200 AND `host.fetch(GET /internal/readyz)` returns 200 with `plugins: []`.
|
||
|
||
#### Scenario: admin disabled via `adminPrefix: null`
|
||
|
||
- **WHEN** `adminPrefix: null` is configured
|
||
- **THEN** a fetch to `/admin/plugins` is treated as an unknown plugin route and returns 404.
|
||
|
||
#### Scenario: start twice raises HOST.ALREADY_STARTED
|
||
|
||
- **WHEN** `host.start()` is called twice
|
||
- **THEN** the second call rejects with `code: "HOST.ALREADY_STARTED"`.
|
||
|
||
#### Scenario: stop before start raises HOST.NOT_STARTED
|
||
|
||
- **WHEN** `host.stop()` is called before `host.start()`
|
||
- **THEN** the call rejects with `code: "HOST.NOT_STARTED"`.
|
||
|
||
**Verified by:** `src/host/host.test.ts::"boots with zero plugins + serves /internal/livez 200"` AND `src/host/host.test.ts::"/internal/readyz lists empty plugin set"` AND `src/host/host.test.ts::"adminPrefix: null disables admin surface"` AND `src/host/host.test.ts::"start twice → HOST.ALREADY_STARTED"` AND `src/host/host.test.ts::"stop before start → HOST.NOT_STARTED"` AND `src/host/host.test.ts::"validates config via Zod"`
|
||
|
||
---
|
||
|
||
### Requirement: per-request `HOST.*` error codes are published
|
||
|
||
The package SHALL publish the following `HOST.*` codes for the per-request pipeline: `TENANT_NOT_RESOLVED` (400), `SESSION_REQUIRED` (401), `CSRF_TOKEN_INVALID` (403), `RBAC_DENIED` (403), `RATE_LIMIT_EXCEEDED` (429), `REQUEST_VALIDATION_FAILED` (400), `RESPONSE_VALIDATION_FAILED` (500), `IDEMPOTENCY_REPLAY` (409), `ROUTE_NOT_FOUND` (404), `HANDLER_THREW` (500), `CLIENT_ABORTED` (499). Each new code SHALL have a non-zero `HOST_DEFAULT_STATUS_MAP` entry AND SHALL be thrown by at least one production path covered by tests.
|
||
|
||
#### Scenario: every new code is mapped
|
||
|
||
- **WHEN** every Chunk-D `HOST.*` code is looked up via `statusFor`
|
||
- **THEN** the returned status is the documented HTTP code above.
|
||
|
||
**Verified by:** `src/error-codes/error-codes.test.ts::"HOST_CODES carries every documented HOST.* reason"` AND `src/error-codes/error-codes.test.ts::"HOST_DEFAULT_STATUS_MAP maps every published code to a sane HTTP status"` AND `src/request-pipeline/request-pipeline.test.ts` (each code thrown from a real handler path)
|