279 lines
16 KiB
Markdown
279 lines
16 KiB
Markdown
<!-- markdownlint-disable MD024 -->
|
|
|
|
# Capability Spec Delta: sandbox (sandbox)
|
|
|
|
**Package**: `@suaveplan/sandbox`
|
|
**Status**: PROPOSED
|
|
|
|
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
|
|
|
## ADDED Requirements
|
|
|
|
### Requirement: Public facade composes every subsystem
|
|
|
|
The package SHALL export a `createSandbox(config?: Partial<SandboxConfig>)` factory returning a `Sandbox` whose `execute` / `executeFunction` / `registerModule` / `unregisterModule` / `getAuditLog` / `getStatistics` / `reset` methods compose the configured engine, `ModuleResolver`, `ResourceLimiter`, `CodeTransformer`, and `AuditLogger`. Default `engine` is `"vm"`.
|
|
|
|
#### Scenario: execute returns structured result
|
|
|
|
- **WHEN** `createSandbox()` is called and `sandbox.execute("2 + 3")` resolves
|
|
- **THEN** the resolved value has `success === true`, `result === 5`, `duration >= 0`, `logs` is an array
|
|
|
|
#### Scenario: executeFunction wraps function source
|
|
|
|
- **WHEN** `sandbox.executeFunction((a, b) => a * b, [6, 7])` resolves
|
|
- **THEN** the resolved value has `success === true` and `result === 42`
|
|
|
|
**Verified by:** `src/sandbox/sandbox.test.ts::"executes a simple expression"` AND `src/sandbox/sandbox.test.ts::"executeFunction multiplies arguments"`
|
|
|
|
---
|
|
|
|
### Requirement: SANDBOX error registry
|
|
|
|
The package SHALL declare `SANDBOX_CODES` via `defineCodes("SANDBOX", [...])` covering `INVALID_CONFIG`, `INVALID_OPTIONS`, `INVALID_MODULE`, `BLOCKED_MODULE`, `RATE_LIMITED`, `TIMEOUT`, `MEMORY_EXCEEDED`, `CODE_BLOCKED`, `CODE_TOO_LONG`, `ENGINE_DISPOSED`, `ENGINE_FAILED`, `WORKER_FAILED`, `SPAWN_FAILED`, `POOL_EXHAUSTED`, `STREAM_CLOSED`, and a `SandboxError` class extending `SuaveplanError`. Every code MUST be reachable from a real call site exercised by a test.
|
|
|
|
#### Scenario: invalid module name is denylisted
|
|
|
|
- **WHEN** `sandbox.registerModule({ name: "fs", exports: {} })` is called
|
|
- **THEN** the call throws `SandboxError` with `code === SANDBOX_CODES.BLOCKED_MODULE`
|
|
|
|
#### Scenario: code transformer block
|
|
|
|
- **WHEN** `sandbox.execute("process.exit(1)")` resolves
|
|
- **THEN** the resolved value has `success === false`, `error` contains `"block-process"`
|
|
|
|
**Verified by:** `src/error-codes/error-codes.test.ts::"every SANDBOX code is registered"` AND `src/module-resolver/module-resolver.test.ts::"register denylisted module throws BLOCKED_MODULE"` AND `src/sandbox/sandbox.test.ts::"transformer blocks process.exit"`
|
|
|
|
---
|
|
|
|
### Requirement: Zod-validated public surface
|
|
|
|
The package SHALL guard `createSandbox`, `execute`, and `registerModule` with Zod schemas (`SandboxConfigPartialSchema`, `SandboxOptionsSchema`, `SandboxModuleSchema`). Parse failures raise `SandboxError` with `INVALID_CONFIG`, `INVALID_OPTIONS`, or `INVALID_MODULE` respectively.
|
|
|
|
#### Scenario: negative timeout rejected
|
|
|
|
- **WHEN** `createSandbox({ timeout: -1 })` is called
|
|
- **THEN** the call throws `SandboxError` with `code === SANDBOX_CODES.INVALID_CONFIG`
|
|
|
|
**Verified by:** `src/schemas/schemas.test.ts::"rejects negative timeout"` AND `src/sandbox/sandbox.test.ts::"rejects invalid config"`
|
|
|
|
---
|
|
|
|
### Requirement: OTel self-telemetry
|
|
|
|
The package SHALL register four OpenTelemetry instruments on the `@suaveplan/sandbox` meter: counters `sandbox_executions_total` (tagged `engine`/`outcome`), `sandbox_rule_blocks_total` (tagged `rule`), `sandbox_pool_acquisitions_total` (tagged `result`), and a histogram `sandbox_execution_duration_ms` (tagged `engine`/`outcome`). Recording helpers (`recordExecution`, `recordRuleBlock`, `recordPoolAcquisition`, `recordDuration`) MUST be called from real call sites.
|
|
|
|
#### Scenario: telemetry helpers are exposed
|
|
|
|
- **WHEN** the telemetry barrel is imported
|
|
- **THEN** `sandboxExecutionsCounter`, `sandboxRuleBlocksCounter`, `sandboxPoolAcquisitionsCounter`, `sandboxExecutionDurationHistogram`, `recordExecution`, `recordRuleBlock`, `recordPoolAcquisition`, `recordDuration` are all defined
|
|
|
|
**Verified by:** `src/telemetry/telemetry.test.ts::"exports four instruments and four recorders"`
|
|
|
|
---
|
|
|
|
### Requirement: AuditLogger
|
|
|
|
The package SHALL provide an `AuditLogger` class that records each execution with `timestamp` (ISO 8601), `codeHash` (first 16 chars of SHA-256 hex), `duration`, `success`, `memoryUsed`, and optional `error`. The logger MUST support `getEntries`, `search({ success?, after?, before?, limit? })`, `getStatistics`, `clear`, and a constructor-provided `onLog` callback invoked synchronously per entry.
|
|
|
|
#### Scenario: search filters by success
|
|
|
|
- **WHEN** an audit logger has logged one success and one failure, and `search({ success: false })` is called
|
|
- **THEN** exactly the failure entry is returned
|
|
|
|
#### Scenario: statistics aggregate
|
|
|
|
- **WHEN** the logger has two success and one failure entries with durations [10, 20, 30]
|
|
- **THEN** `getStatistics()` returns `{ totalRuns: 3, successCount: 2, failureCount: 1, successRate: 2/3, avgDuration: 20, totalMemoryUsed: <sum> }`
|
|
|
|
**Verified by:** `src/audit-logger/audit-logger.test.ts::"search filters by success"` AND `src/audit-logger/audit-logger.test.ts::"statistics aggregate across entries"`
|
|
|
|
---
|
|
|
|
### Requirement: ResourceLimiter
|
|
|
|
The package SHALL provide a `ResourceLimiter` class supporting `maxExecutionsPerWindow` over a sliding `rateLimitWindow` (default 60_000 ms). `checkRateLimit()` MUST throw `SandboxError` with `RATE_LIMITED` when the window count is at or above the limit. The limiter MUST accept an injectable `Clock` (defaulting to `Date.now`) and provide `truncateOutput`, `recordExecution`, `getTotalExecutions`, `getWindowExecutions`, `reset`.
|
|
|
|
#### Scenario: sliding window allows execution after window expires
|
|
|
|
- **WHEN** the limiter is configured with `maxExecutionsPerWindow: 2` and the clock advances past `rateLimitWindow`
|
|
- **THEN** a third call to `checkRateLimit()` succeeds
|
|
|
|
#### Scenario: output truncation appends sentinel
|
|
|
|
- **WHEN** `truncateOutput(string-longer-than-max)` is called
|
|
- **THEN** the result is truncated to `maxOutputSize` bytes with the `[output truncated at N bytes]` sentinel appended
|
|
|
|
**Verified by:** `src/resource-limiter/resource-limiter.test.ts::"sliding window resets after expiry"` AND `src/resource-limiter/resource-limiter.test.ts::"truncates oversize output"`
|
|
|
|
---
|
|
|
|
### Requirement: ModuleResolver
|
|
|
|
The package SHALL provide a `ModuleResolver` class with a denylist (`fs`, `net`, `child_process`, `cluster`, `dgram`, `dns`, `http`, `https`, `http2`, `tls`, `worker_threads`, `process`, `os`, `path`, `readline`, `repl`, `vm`) and a built-in safe-module registry (`Math`, `JSON`, `Date`). `register` MUST throw `SandboxError` with `BLOCKED_MODULE` for denylisted names. `unregister`, `getModules`, `isBlocked`, `isAllowed`, `getBlockedModules` SHALL all be public.
|
|
|
|
#### Scenario: register denylisted module is rejected
|
|
|
|
- **WHEN** `resolver.register({ name: "fs", exports: {} })`
|
|
- **THEN** throws `SandboxError` with `code === SANDBOX_CODES.BLOCKED_MODULE`
|
|
|
|
#### Scenario: builtin Math is auto-loaded when allowed
|
|
|
|
- **WHEN** `new ModuleResolver(["Math"])` is constructed
|
|
- **THEN** `getModules()` returns a module named `Math` whose exports include `Math.PI`
|
|
|
|
**Verified by:** `src/module-resolver/module-resolver.test.ts::"register denylisted module throws BLOCKED_MODULE"` AND `src/module-resolver/module-resolver.test.ts::"builtin Math is auto-loaded"`
|
|
|
|
---
|
|
|
|
### Requirement: CodeTransformer
|
|
|
|
The package SHALL provide a `CodeTransformer` class with eleven built-in rules: `block-process`, `block-require`, `block-import-dynamic`, `block-eval`, `block-function-constructor`, `block-global-this`, `block-proto-access`, `block-constructor-access`, `warn-while-true`, `warn-infinite-for`, plus a `max-code-length` guard (default 100000). `transform` MUST return a `TransformResult` (`code`, `blocked`, `blockedBy`, `applied`, `warnings`); `wouldBlock` MUST be a pure check. Custom rules SHALL be additive.
|
|
|
|
#### Scenario: block-process trips on process.exit
|
|
|
|
- **WHEN** `transformer.transform("process.exit(1)")`
|
|
- **THEN** result has `blocked === true`, `blockedBy === "block-process"`
|
|
|
|
#### Scenario: code over maxCodeLength is blocked
|
|
|
|
- **WHEN** `transformer.transform("x".repeat(maxCodeLength + 1))`
|
|
- **THEN** result has `blocked === true`, `blockedBy === "max-code-length"`
|
|
|
|
**Verified by:** `src/code-transformer/code-transformer.test.ts::"blocks process.exit via block-process rule"` AND `src/code-transformer/code-transformer.test.ts::"blocks oversize code"`
|
|
|
|
---
|
|
|
|
### Requirement: VmEngine — node:vm execution
|
|
|
|
The package SHALL provide a `VmEngine` class implementing `SandboxEngine` backed by real `node:vm`. It MUST compile the code with `new vm.Script(...)`, run via `script.runInContext(ctx, { timeout })`, capture console output up to `maxOutputSize`, surface thrown errors as `success: false` results, and snapshot V8 heap usage via `v8.getHeapStatistics()`.
|
|
|
|
#### Scenario: vm engine executes 1+1
|
|
|
|
- **WHEN** `new VmEngine({ timeout: 5000, maxOutputSize: 1024, memoryLimit: 50_000_000 })` is created and `engine.execute("1 + 1")` is awaited
|
|
- **THEN** the result has `success === true` and `result === 2`
|
|
|
|
#### Scenario: vm engine throws TIMEOUT-class for infinite loop
|
|
|
|
- **WHEN** `engine.execute("while(true){}", {}, { timeout: 100 })` is awaited
|
|
- **THEN** the result has `success === false`, `error` is non-empty (contains `timed out` or `Script execution timed out`)
|
|
|
|
#### Scenario: vm engine console capture
|
|
|
|
- **WHEN** `engine.execute("console.log('hello'); 1")` is awaited
|
|
- **THEN** the result has `logs` containing `"hello"`
|
|
|
|
**Verified by:** `src/vm-engine/vm-engine.test.ts::"executes a simple expression"` AND `src/vm-engine/vm-engine.test.ts::"enforces sync timeout"` AND `src/vm-engine/vm-engine.test.ts::"captures console output"`
|
|
|
|
---
|
|
|
|
### Requirement: WorkerEngine — node:worker_threads execution
|
|
|
|
The package SHALL provide a `WorkerEngine` class implementing `SandboxEngine` backed by real `node:worker_threads`. Each `execute` call spawns a fresh `Worker` (`eval: true`), posts the code + context + modules + maxOutputSize via the message channel, receives the structured result on `message`, and enforces the timeout by calling `worker.terminate()`.
|
|
|
|
#### Scenario: worker engine executes 2+2
|
|
|
|
- **WHEN** `engine.execute("2 + 2")` resolves
|
|
- **THEN** the result has `success === true`, `result === 4`
|
|
|
|
#### Scenario: worker engine times out via terminate
|
|
|
|
- **WHEN** `engine.execute("while(true){}", {}, { timeout: 200 })` resolves
|
|
- **THEN** the result has `success === false`, `error` references the timeout
|
|
|
|
**Verified by:** `src/worker-engine/worker-engine.test.ts::"executes 2+2"` AND `src/worker-engine/worker-engine.test.ts::"terminates on timeout"`
|
|
|
|
---
|
|
|
|
### Requirement: SpawnEngine — node:child_process execution
|
|
|
|
The package SHALL provide a `SpawnEngine` class implementing the async `SandboxEngine` contract backed by real `node:child_process.spawn`. Each `execute` call spawns a child process (defaulting to `process.execPath`), writes a JSON envelope (code + context + modules + maxOutputSize) to stdin, reads JSON from stdout, sanitises the environment by stripping keys that match `SECRET`/`KEY`/`TOKEN`/`PASSWORD`/`CREDENTIAL`/`AUTH` (case-insensitive contains) and dropping `HOME`/`USER`/`SHELL`/`LOGNAME`, and enforces the timeout via `Promise.race` + `child.kill("SIGKILL")`.
|
|
|
|
#### Scenario: spawn engine sanitises env
|
|
|
|
- **WHEN** the engine sanitises its env with `process.env.SECRET_X = "..."` set
|
|
- **THEN** the sanitised env does NOT contain `SECRET_X` (or `HOME`/`USER`/`SHELL`/`LOGNAME`)
|
|
|
|
#### Scenario: spawn engine times out by killing the child
|
|
|
|
- **WHEN** `engine.execute("while(true){}", {}, { timeout: 200 })` resolves
|
|
- **THEN** the result has `success === false`, `error` mentions the timeout
|
|
|
|
**Verified by:** `src/spawn-engine/spawn-engine.test.ts::"strips sensitive env keys"` AND `src/spawn-engine/spawn-engine.test.ts::"kills child on timeout"`
|
|
|
|
---
|
|
|
|
### Requirement: Runner script (spawn-engine child entry)
|
|
|
|
The package SHALL ship a child-process entry point (`runner.ts`) on a `./runner` subpath export. The runner reads JSON from stdin, evaluates the code in a strict-mode function scope (with `console` and modules injected), and writes a JSON `RunnerOutput` to stdout. It MUST capture `memoryUsed` via `process.memoryUsage().heapUsed` deltas and produce a usable `error` field for both `Error` and non-`Error` throws.
|
|
|
|
#### Scenario: runner handles a simple expression
|
|
|
|
- **WHEN** the runner is invoked over a pipe with `{ code: "1 + 1", context: {}, modules: [], maxOutputSize: 1024 }`
|
|
- **THEN** stdout contains a JSON envelope with `success: true`, `result: 2`
|
|
|
|
**Verified by:** `src/spawn-engine/spawn-engine.test.ts::"end-to-end runner pipeline returns 1+1"`
|
|
|
|
---
|
|
|
|
### Requirement: IsolateEngine (isolated-vm contract)
|
|
|
|
The package SHALL provide an `IsolateEngine` class implementing `SandboxEngine` over a structural `IsolateFactory` contract that mirrors the `isolated-vm` API (`createIsolate`, `createContextSync`, `compileScriptSync`, `runSync`, `getHeapStatisticsSync`, `dispose`). The structural type keeps the real `isolated-vm` peer dep optional. The engine MUST surface heap statistics via `getHeapStatistics()` and `null` after `destroy()`.
|
|
|
|
#### Scenario: returns null heap after destroy
|
|
|
|
- **WHEN** an `IsolateEngine` is destroyed and `getHeapStatistics()` is called
|
|
- **THEN** the return value is `null`
|
|
|
|
**Verified by:** `src/isolate-engine/isolate-engine.test.ts::"returns null heap after destroy"`
|
|
|
|
---
|
|
|
|
### Requirement: SandboxPool
|
|
|
|
The package SHALL provide a `SandboxPool` class that pre-warms up to `minSize` engines, lazily grows up to `maxSize`, recycles engines that exceed `maxExecutionsPerEngine`, and reclaims idle engines older than `idleTimeoutMs` down to `minSize` via a periodic cleanup. `execute` MUST surface `POOL_EXHAUSTED` via `success: false` when the pool cannot acquire. The pool MUST accept an injectable `Clock`.
|
|
|
|
#### Scenario: pool grows to maxSize then reports exhaustion
|
|
|
|
- **WHEN** a pool of `minSize: 1, maxSize: 2` has both engines acquired and a third caller invokes execute on a non-releasing engine
|
|
- **THEN** the third result has `success === false` and `error` references `POOL_EXHAUSTED`
|
|
|
|
#### Scenario: idle cleanup reclaims excess engines
|
|
|
|
- **WHEN** a pool grows above `minSize` and the clock advances past `idleTimeoutMs`
|
|
- **THEN** running the cleanup tick reduces `pool.size()` to `minSize`
|
|
|
|
**Verified by:** `src/pool/pool.test.ts::"reports POOL_EXHAUSTED when saturated"` AND `src/pool/pool.test.ts::"cleanup reclaims idle engines to minSize"`
|
|
|
|
---
|
|
|
|
### Requirement: ResultStreamer
|
|
|
|
The package SHALL provide a `ResultStreamer` class supporting `subscribe(listener) -> unsubscribe`, `pushLog`, `pushError`, `pushResult`, `complete(result)`, `flush`, `stop`, `destroy`. Chunks MUST be buffered up to `maxBufferSize` then flushed; an interval flush MUST run every `flushIntervalMs`. Calls to `pushChunk` after `complete()` MUST raise `SandboxError` with `STREAM_CLOSED`.
|
|
|
|
#### Scenario: push after complete throws STREAM_CLOSED
|
|
|
|
- **WHEN** a streamer is `complete()`d and `pushLog("late")` is called
|
|
- **THEN** the call throws `SandboxError` with `code === SANDBOX_CODES.STREAM_CLOSED`
|
|
|
|
#### Scenario: subscribe receives buffered chunks on flush
|
|
|
|
- **WHEN** a listener subscribes, `pushLog("a")` then `flush()` is called
|
|
- **THEN** the listener has received exactly one `{ type: "log", data: "a" }` chunk
|
|
|
|
**Verified by:** `src/result-streamer/result-streamer.test.ts::"raises STREAM_CLOSED after complete"` AND `src/result-streamer/result-streamer.test.ts::"flush emits buffered chunks to subscribers"`
|
|
|
|
---
|
|
|
|
### Requirement: Subpath exports per feature
|
|
|
|
The `package.json` `exports` map SHALL expose `.`, `./package.json`, `./types`, `./error-codes`, `./schemas`, `./telemetry`, `./audit-logger`, `./resource-limiter`, `./module-resolver`, `./code-transformer`, `./vm-engine`, `./worker-engine`, `./spawn-engine`, `./runner`, `./isolate-engine`, `./pool`, `./result-streamer`, `./sandbox`. Every subpath SHALL point at the feature's barrel.
|
|
|
|
#### Scenario: subpath count matches feature module count
|
|
|
|
- **WHEN** the `exports` map is enumerated (excluding `./package.json`)
|
|
- **THEN** the resulting key set contains every feature subpath listed above
|
|
|
|
**Verified by:** `src/sandbox/sandbox.test.ts::"exports every documented subpath"`
|
|
|
|
---
|