SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
suaveplan/openspec/specs/sandbox/spec.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

16 KiB
Raw Permalink Blame History

Capability Spec Delta: sandbox (sandbox)

Package: @suaveplan/sandbox Status: PROPOSED

Every requirement below carries a **Verified by:** line at authoring time. No exceptions.

ADDED Requirements

Requirement: Public facade composes every subsystem

The package SHALL export a createSandbox(config?: Partial<SandboxConfig>) factory returning a Sandbox whose execute / executeFunction / registerModule / unregisterModule / getAuditLog / getStatistics / reset methods compose the configured engine, ModuleResolver, ResourceLimiter, CodeTransformer, and AuditLogger. Default engine is "vm".

Scenario: execute returns structured result

  • WHEN createSandbox() is called and sandbox.execute("2 + 3") resolves
  • THEN the resolved value has success === true, result === 5, duration >= 0, logs is an array

Scenario: executeFunction wraps function source

  • WHEN sandbox.executeFunction((a, b) => a * b, [6, 7]) resolves
  • THEN the resolved value has success === true and result === 42

Verified by: src/sandbox/sandbox.test.ts::"executes a simple expression" AND src/sandbox/sandbox.test.ts::"executeFunction multiplies arguments"

Requirement: SANDBOX error registry

The package SHALL declare SANDBOX_CODES via defineCodes("SANDBOX", [...]) covering INVALID_CONFIG, INVALID_OPTIONS, INVALID_MODULE, BLOCKED_MODULE, RATE_LIMITED, TIMEOUT, MEMORY_EXCEEDED, CODE_BLOCKED, CODE_TOO_LONG, ENGINE_DISPOSED, ENGINE_FAILED, WORKER_FAILED, SPAWN_FAILED, POOL_EXHAUSTED, STREAM_CLOSED, and a SandboxError class extending SuaveplanError. Every code MUST be reachable from a real call site exercised by a test.

Scenario: invalid module name is denylisted

  • WHEN sandbox.registerModule({ name: "fs", exports: {} }) is called
  • THEN the call throws SandboxError with code === SANDBOX_CODES.BLOCKED_MODULE

Scenario: code transformer block

  • WHEN sandbox.execute("process.exit(1)") resolves
  • THEN the resolved value has success === false, error contains "block-process"

Verified by: src/error-codes/error-codes.test.ts::"every SANDBOX code is registered" AND src/module-resolver/module-resolver.test.ts::"register denylisted module throws BLOCKED_MODULE" AND src/sandbox/sandbox.test.ts::"transformer blocks process.exit"

Requirement: Zod-validated public surface

The package SHALL guard createSandbox, execute, and registerModule with Zod schemas (SandboxConfigPartialSchema, SandboxOptionsSchema, SandboxModuleSchema). Parse failures raise SandboxError with INVALID_CONFIG, INVALID_OPTIONS, or INVALID_MODULE respectively.

Scenario: negative timeout rejected

  • WHEN createSandbox({ timeout: -1 }) is called
  • THEN the call throws SandboxError with code === SANDBOX_CODES.INVALID_CONFIG

Verified by: src/schemas/schemas.test.ts::"rejects negative timeout" AND src/sandbox/sandbox.test.ts::"rejects invalid config"

Requirement: OTel self-telemetry

The package SHALL register four OpenTelemetry instruments on the @suaveplan/sandbox meter: counters sandbox_executions_total (tagged engine/outcome), sandbox_rule_blocks_total (tagged rule), sandbox_pool_acquisitions_total (tagged result), and a histogram sandbox_execution_duration_ms (tagged engine/outcome). Recording helpers (recordExecution, recordRuleBlock, recordPoolAcquisition, recordDuration) MUST be called from real call sites.

Scenario: telemetry helpers are exposed

  • WHEN the telemetry barrel is imported
  • THEN sandboxExecutionsCounter, sandboxRuleBlocksCounter, sandboxPoolAcquisitionsCounter, sandboxExecutionDurationHistogram, recordExecution, recordRuleBlock, recordPoolAcquisition, recordDuration are all defined

Verified by: src/telemetry/telemetry.test.ts::"exports four instruments and four recorders"

Requirement: AuditLogger

The package SHALL provide an AuditLogger class that records each execution with timestamp (ISO 8601), codeHash (first 16 chars of SHA-256 hex), duration, success, memoryUsed, and optional error. The logger MUST support getEntries, search({ success?, after?, before?, limit? }), getStatistics, clear, and a constructor-provided onLog callback invoked synchronously per entry.

Scenario: search filters by success

  • WHEN an audit logger has logged one success and one failure, and search({ success: false }) is called
  • THEN exactly the failure entry is returned

Scenario: statistics aggregate

  • WHEN the logger has two success and one failure entries with durations [10, 20, 30]
  • THEN getStatistics() returns { totalRuns: 3, successCount: 2, failureCount: 1, successRate: 2/3, avgDuration: 20, totalMemoryUsed: <sum> }

Verified by: src/audit-logger/audit-logger.test.ts::"search filters by success" AND src/audit-logger/audit-logger.test.ts::"statistics aggregate across entries"

Requirement: ResourceLimiter

The package SHALL provide a ResourceLimiter class supporting maxExecutionsPerWindow over a sliding rateLimitWindow (default 60_000 ms). checkRateLimit() MUST throw SandboxError with RATE_LIMITED when the window count is at or above the limit. The limiter MUST accept an injectable Clock (defaulting to Date.now) and provide truncateOutput, recordExecution, getTotalExecutions, getWindowExecutions, reset.

Scenario: sliding window allows execution after window expires

  • WHEN the limiter is configured with maxExecutionsPerWindow: 2 and the clock advances past rateLimitWindow
  • THEN a third call to checkRateLimit() succeeds

Scenario: output truncation appends sentinel

  • WHEN truncateOutput(string-longer-than-max) is called
  • THEN the result is truncated to maxOutputSize bytes with the [output truncated at N bytes] sentinel appended

Verified by: src/resource-limiter/resource-limiter.test.ts::"sliding window resets after expiry" AND src/resource-limiter/resource-limiter.test.ts::"truncates oversize output"

Requirement: ModuleResolver

The package SHALL provide a ModuleResolver class with a denylist (fs, net, child_process, cluster, dgram, dns, http, https, http2, tls, worker_threads, process, os, path, readline, repl, vm) and a built-in safe-module registry (Math, JSON, Date). register MUST throw SandboxError with BLOCKED_MODULE for denylisted names. unregister, getModules, isBlocked, isAllowed, getBlockedModules SHALL all be public.

Scenario: register denylisted module is rejected

  • WHEN resolver.register({ name: "fs", exports: {} })
  • THEN throws SandboxError with code === SANDBOX_CODES.BLOCKED_MODULE

Scenario: builtin Math is auto-loaded when allowed

  • WHEN new ModuleResolver(["Math"]) is constructed
  • THEN getModules() returns a module named Math whose exports include Math.PI

Verified by: src/module-resolver/module-resolver.test.ts::"register denylisted module throws BLOCKED_MODULE" AND src/module-resolver/module-resolver.test.ts::"builtin Math is auto-loaded"

Requirement: CodeTransformer

The package SHALL provide a CodeTransformer class with eleven built-in rules: block-process, block-require, block-import-dynamic, block-eval, block-function-constructor, block-global-this, block-proto-access, block-constructor-access, warn-while-true, warn-infinite-for, plus a max-code-length guard (default 100000). transform MUST return a TransformResult (code, blocked, blockedBy, applied, warnings); wouldBlock MUST be a pure check. Custom rules SHALL be additive.

Scenario: block-process trips on process.exit

  • WHEN transformer.transform("process.exit(1)")
  • THEN result has blocked === true, blockedBy === "block-process"

Scenario: code over maxCodeLength is blocked

  • WHEN transformer.transform("x".repeat(maxCodeLength + 1))
  • THEN result has blocked === true, blockedBy === "max-code-length"

Verified by: src/code-transformer/code-transformer.test.ts::"blocks process.exit via block-process rule" AND src/code-transformer/code-transformer.test.ts::"blocks oversize code"

Requirement: VmEngine — node:vm execution

The package SHALL provide a VmEngine class implementing SandboxEngine backed by real node:vm. It MUST compile the code with new vm.Script(...), run via script.runInContext(ctx, { timeout }), capture console output up to maxOutputSize, surface thrown errors as success: false results, and snapshot V8 heap usage via v8.getHeapStatistics().

Scenario: vm engine executes 1+1

  • WHEN new VmEngine({ timeout: 5000, maxOutputSize: 1024, memoryLimit: 50_000_000 }) is created and engine.execute("1 + 1") is awaited
  • THEN the result has success === true and result === 2

Scenario: vm engine throws TIMEOUT-class for infinite loop

  • WHEN engine.execute("while(true){}", {}, { timeout: 100 }) is awaited
  • THEN the result has success === false, error is non-empty (contains timed out or Script execution timed out)

Scenario: vm engine console capture

  • WHEN engine.execute("console.log('hello'); 1") is awaited
  • THEN the result has logs containing "hello"

Verified by: src/vm-engine/vm-engine.test.ts::"executes a simple expression" AND src/vm-engine/vm-engine.test.ts::"enforces sync timeout" AND src/vm-engine/vm-engine.test.ts::"captures console output"

Requirement: WorkerEngine — node:worker_threads execution

The package SHALL provide a WorkerEngine class implementing SandboxEngine backed by real node:worker_threads. Each execute call spawns a fresh Worker (eval: true), posts the code + context + modules + maxOutputSize via the message channel, receives the structured result on message, and enforces the timeout by calling worker.terminate().

Scenario: worker engine executes 2+2

  • WHEN engine.execute("2 + 2") resolves
  • THEN the result has success === true, result === 4

Scenario: worker engine times out via terminate

  • WHEN engine.execute("while(true){}", {}, { timeout: 200 }) resolves
  • THEN the result has success === false, error references the timeout

Verified by: src/worker-engine/worker-engine.test.ts::"executes 2+2" AND src/worker-engine/worker-engine.test.ts::"terminates on timeout"

Requirement: SpawnEngine — node:child_process execution

The package SHALL provide a SpawnEngine class implementing the async SandboxEngine contract backed by real node:child_process.spawn. Each execute call spawns a child process (defaulting to process.execPath), writes a JSON envelope (code + context + modules + maxOutputSize) to stdin, reads JSON from stdout, sanitises the environment by stripping keys that match SECRET/KEY/TOKEN/PASSWORD/CREDENTIAL/AUTH (case-insensitive contains) and dropping HOME/USER/SHELL/LOGNAME, and enforces the timeout via Promise.race + child.kill("SIGKILL").

Scenario: spawn engine sanitises env

  • WHEN the engine sanitises its env with process.env.SECRET_X = "..." set
  • THEN the sanitised env does NOT contain SECRET_X (or HOME/USER/SHELL/LOGNAME)

Scenario: spawn engine times out by killing the child

  • WHEN engine.execute("while(true){}", {}, { timeout: 200 }) resolves
  • THEN the result has success === false, error mentions the timeout

Verified by: src/spawn-engine/spawn-engine.test.ts::"strips sensitive env keys" AND src/spawn-engine/spawn-engine.test.ts::"kills child on timeout"

Requirement: Runner script (spawn-engine child entry)

The package SHALL ship a child-process entry point (runner.ts) on a ./runner subpath export. The runner reads JSON from stdin, evaluates the code in a strict-mode function scope (with console and modules injected), and writes a JSON RunnerOutput to stdout. It MUST capture memoryUsed via process.memoryUsage().heapUsed deltas and produce a usable error field for both Error and non-Error throws.

Scenario: runner handles a simple expression

  • WHEN the runner is invoked over a pipe with { code: "1 + 1", context: {}, modules: [], maxOutputSize: 1024 }
  • THEN stdout contains a JSON envelope with success: true, result: 2

Verified by: src/spawn-engine/spawn-engine.test.ts::"end-to-end runner pipeline returns 1+1"

Requirement: IsolateEngine (isolated-vm contract)

The package SHALL provide an IsolateEngine class implementing SandboxEngine over a structural IsolateFactory contract that mirrors the isolated-vm API (createIsolate, createContextSync, compileScriptSync, runSync, getHeapStatisticsSync, dispose). The structural type keeps the real isolated-vm peer dep optional. The engine MUST surface heap statistics via getHeapStatistics() and null after destroy().

Scenario: returns null heap after destroy

  • WHEN an IsolateEngine is destroyed and getHeapStatistics() is called
  • THEN the return value is null

Verified by: src/isolate-engine/isolate-engine.test.ts::"returns null heap after destroy"

Requirement: SandboxPool

The package SHALL provide a SandboxPool class that pre-warms up to minSize engines, lazily grows up to maxSize, recycles engines that exceed maxExecutionsPerEngine, and reclaims idle engines older than idleTimeoutMs down to minSize via a periodic cleanup. execute MUST surface POOL_EXHAUSTED via success: false when the pool cannot acquire. The pool MUST accept an injectable Clock.

Scenario: pool grows to maxSize then reports exhaustion

  • WHEN a pool of minSize: 1, maxSize: 2 has both engines acquired and a third caller invokes execute on a non-releasing engine
  • THEN the third result has success === false and error references POOL_EXHAUSTED

Scenario: idle cleanup reclaims excess engines

  • WHEN a pool grows above minSize and the clock advances past idleTimeoutMs
  • THEN running the cleanup tick reduces pool.size() to minSize

Verified by: src/pool/pool.test.ts::"reports POOL_EXHAUSTED when saturated" AND src/pool/pool.test.ts::"cleanup reclaims idle engines to minSize"

Requirement: ResultStreamer

The package SHALL provide a ResultStreamer class supporting subscribe(listener) -> unsubscribe, pushLog, pushError, pushResult, complete(result), flush, stop, destroy. Chunks MUST be buffered up to maxBufferSize then flushed; an interval flush MUST run every flushIntervalMs. Calls to pushChunk after complete() MUST raise SandboxError with STREAM_CLOSED.

Scenario: push after complete throws STREAM_CLOSED

  • WHEN a streamer is complete()d and pushLog("late") is called
  • THEN the call throws SandboxError with code === SANDBOX_CODES.STREAM_CLOSED

Scenario: subscribe receives buffered chunks on flush

  • WHEN a listener subscribes, pushLog("a") then flush() is called
  • THEN the listener has received exactly one { type: "log", data: "a" } chunk

Verified by: src/result-streamer/result-streamer.test.ts::"raises STREAM_CLOSED after complete" AND src/result-streamer/result-streamer.test.ts::"flush emits buffered chunks to subscribers"

Requirement: Subpath exports per feature

The package.json exports map SHALL expose ., ./package.json, ./types, ./error-codes, ./schemas, ./telemetry, ./audit-logger, ./resource-limiter, ./module-resolver, ./code-transformer, ./vm-engine, ./worker-engine, ./spawn-engine, ./runner, ./isolate-engine, ./pool, ./result-streamer, ./sandbox. Every subpath SHALL point at the feature's barrel.

Scenario: subpath count matches feature module count

  • WHEN the exports map is enumerated (excluding ./package.json)
  • THEN the resulting key set contains every feature subpath listed above

Verified by: src/sandbox/sandbox.test.ts::"exports every documented subpath"

Reference in New Issue View Git Blame Copy Permalink