336 lines
12 KiB
Markdown
336 lines
12 KiB
Markdown
# OpenSpec Change: enterprise-browser-audio-engine
|
|
|
|
- **Change ID**: enterprise-browser-audio-engine
|
|
- **Package**: `@suaveplan/audio-engine`
|
|
- **Category**: browser
|
|
- **Target path**: `browser/audio/audio-engine`
|
|
- **Status**: IN PROGRESS
|
|
- **Date**: 2026-04-24
|
|
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/audio-engine.md`
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Deliver `@suaveplan/audio-engine` on the genesis branch from scratch at enterprise quality. The package is the core browser-side Web Audio engine consumed by every React and R3F audio surface in the monorepo. It owns the global `AudioContext` lifecycle, a typed plugin system (`AudioPlugin` base class, `PluginRegistry`, `AudioEngine`), an `AudioGraph` node-routing layer, an `AudioWorkletManager` for loading and messaging worklet modules, an `AudioQueueEngine` for cue/sequence playback, and an `EventBus` facade. Zero React dependencies — consumed by `@suaveplan/audio-controllers`, `@suaveplan/audio-plugins-fx`, and React audio packages.
|
|
|
|
---
|
|
|
|
## Current-state audit (genesis)
|
|
|
|
The package does not exist on genesis yet — this is a fresh build. No prior source, no test results, no coverage data.
|
|
|
|
---
|
|
|
|
## Available workspace deps on genesis
|
|
|
|
| Package | Role |
|
|
|---|---|
|
|
| `@suaveplan/error` (`workspace:^`) | Base class for `AudioEngineError`; `defineCodes` for `AUDIO.*` error codes |
|
|
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `EventBusLike`, `Clock` for injectable clock support |
|
|
| `@suaveplan/events` (`workspace:^`) | `EventEmitter` for `AudioEngineEventMap`; typed event bus |
|
|
| `@suaveplan/schema-registry` (`workspace:^`) | Register `AudioPluginDescriptor` Zod schema |
|
|
| `@suaveplan/api-schemas` (`workspace:^`) | Shared schema registration surface |
|
|
| `zod` (`catalog:forms`) | Zod schemas for `PluginMetadata`, `AudioPluginDescriptor`, configs |
|
|
| `@opentelemetry/api` (`catalog:observability`) | Counters + histograms for plugin lifecycle and latency |
|
|
| `standardized-audio-context` (`catalog:browser`) | Cross-browser `AudioContext` + `OfflineAudioContext` shim |
|
|
| `tslib` (`catalog:build`) | Runtime TS helpers |
|
|
|
|
Dev deps: `vitest`, `@vitest/coverage-istanbul`, `mitata` (all `catalog:testing`).
|
|
|
|
---
|
|
|
|
## Preserved public API surface (from main branch)
|
|
|
|
### Core
|
|
|
|
```ts
|
|
class AudioEngine {
|
|
constructor(config?: AudioEngineConfig, context?: BaseAudioContext)
|
|
start(): Promise<void>
|
|
stop(): Promise<void>
|
|
getState(): AudioEngineState
|
|
getStateInfo(): AudioEngineStateInfo
|
|
getMetrics(): AudioEngineMetrics
|
|
loadPlugin(id: string, plugin: AudioPlugin): Promise<void>
|
|
unloadPlugin(id: string): Promise<void>
|
|
getPlugin<T extends AudioPlugin>(id: string): T | undefined
|
|
on<K extends keyof AudioEngineEventMap>(event: K, listener: AudioEngineEventMap[K]): void
|
|
off<K extends keyof AudioEngineEventMap>(event: K, listener: AudioEngineEventMap[K]): void
|
|
}
|
|
|
|
interface AudioEngineConfig { sampleRate?: number; latencyHint?: AudioContextLatencyCategory; bufferSize?: number; maxPlugins?: number }
|
|
type AudioEngineState = 'uninitialized' | 'starting' | 'running' | 'suspended' | 'stopping' | 'stopped' | 'error'
|
|
interface AudioEngineStateInfo { state: AudioEngineState; previousState: AudioEngineState; timestamp: number }
|
|
interface AudioEngineMetrics { pluginCount: number; activeNodes: number; cpuUsage: number; sampleRate: number; currentTime: number }
|
|
type AudioEngineEventMap = { 'state:change': (info: AudioEngineStateInfo) => void; 'plugin:loaded': (id: string) => void; 'plugin:unloaded': (id: string) => void; 'plugin:error': (id: string, error: Error) => void; 'error': (error: Error) => void }
|
|
```
|
|
|
|
### AudioGraph
|
|
|
|
```ts
|
|
class AudioGraph {
|
|
addNode(id: string, node: AudioNode): void
|
|
removeNode(id: string): void
|
|
connect(sourceId: string, destinationId: string, outputIndex?: number, inputIndex?: number): void
|
|
disconnect(sourceId: string, destinationId?: string): void
|
|
getNode(id: string): AudioNode | undefined
|
|
traverse(startId: string): string[]
|
|
snapshot(): AudioGraphSnapshot
|
|
restore(snapshot: AudioGraphSnapshot): void
|
|
}
|
|
```
|
|
|
|
### AudioQueueEngine
|
|
|
|
```ts
|
|
class AudioQueueEngine {
|
|
enqueue(cue: AudioCue): string
|
|
dequeue(cueId: string): void
|
|
play(cueId: string): Promise<void>
|
|
stop(cueId: string): void
|
|
getQueue(): AudioCue[]
|
|
getSequences(): AudioSequence[]
|
|
addSequence(seq: AudioSequence): void
|
|
removeSequence(seqId: string): void
|
|
}
|
|
|
|
interface AudioCue { id?: string; buffer: AudioBuffer; startTime?: number; offset?: number; duration?: number; gain?: number; loop?: boolean }
|
|
interface AudioSequence { id: string; cues: AudioCue[]; loop?: boolean; tempo?: number }
|
|
```
|
|
|
|
### AudioWorkletManager
|
|
|
|
```ts
|
|
class AudioWorkletManager {
|
|
addModule(url: string, options?: WorkletModuleOptions): Promise<void>
|
|
createNode(processorName: string, options?: AudioWorkletNodeOptions): AudioWorkletNode
|
|
postMessage(processorName: string, message: unknown, transfer?: Transferable[]): void
|
|
onMessage(processorName: string, handler: (event: MessageEvent) => void): void
|
|
isLoaded(url: string): boolean
|
|
}
|
|
|
|
interface WorkletModuleOptions { allowedUrls?: string[]; credentials?: RequestCredentials }
|
|
```
|
|
|
|
### Plugin system
|
|
|
|
```ts
|
|
abstract class AudioPlugin {
|
|
abstract readonly metadata: PluginMetadata
|
|
state: PluginState
|
|
abstract initialize(context: BaseAudioContext): Promise<void>
|
|
abstract connect(graph: AudioGraph): void
|
|
abstract disconnect(): void
|
|
abstract destroy(): Promise<void>
|
|
setParameter(id: string, value: number): void
|
|
getParameter(id: string): number | undefined
|
|
getParameters(): PluginParameter[]
|
|
}
|
|
|
|
type PluginState = 'uninitialized' | 'initializing' | 'initialized' | 'active' | 'inactive' | 'error' | 'destroyed'
|
|
interface PluginMetadata { id: string; name: string; version: string; category: string; description?: string }
|
|
interface PluginParameter { id: string; name: string; value: number; min: number; max: number; default: number; automatable?: boolean }
|
|
|
|
class PluginRegistry {
|
|
register(descriptor: AudioPluginDescriptor): void
|
|
unregister(id: string): void
|
|
get(id: string): AudioPluginDescriptor | undefined
|
|
list(): AudioPluginDescriptor[]
|
|
instantiate(id: string): AudioPlugin
|
|
}
|
|
|
|
interface AudioPluginDescriptor { id: string; name: string; version: string; factory: () => AudioPlugin; metadata: PluginMetadata }
|
|
```
|
|
|
|
### EventBus facade
|
|
|
|
```ts
|
|
class EventBus {
|
|
constructor(bus?: EventBusLike)
|
|
emit<K extends keyof AudioEngineEventMap>(event: K, ...args: Parameters<AudioEngineEventMap[K]>): void
|
|
on<K extends keyof AudioEngineEventMap>(event: K, listener: AudioEngineEventMap[K]): void
|
|
off<K extends keyof AudioEngineEventMap>(event: K, listener: AudioEngineEventMap[K]): void
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## New additions for genesis
|
|
|
|
### 1. Structured errors via `@suaveplan/error`
|
|
|
|
Error codes with stable `AUDIO.*` namespace:
|
|
|
|
| Code | Severity | Retryable |
|
|
|---|---|---|
|
|
| `AUDIO_ENGINE_STATE_INVALID` | `error` | `false` |
|
|
| `AUDIO_PLUGIN_NOT_FOUND` | `error` | `false` |
|
|
| `AUDIO_PLUGIN_LOAD_FAILED` | `error` | `true` |
|
|
| `AUDIO_WORKLET_LOAD_FAILED` | `error` | `true` |
|
|
| `AUDIO_WORKLET_URL_REJECTED` | `error` | `false` |
|
|
| `AUDIO_GRAPH_CONNECT_FAILED` | `error` | `false` |
|
|
| `AUDIO_CONTEXT_UNAVAILABLE` | `fatal` | `false` |
|
|
|
|
### 2. AudioContext injection seam
|
|
|
|
`AudioEngine` constructor accepts an optional pre-built `BaseAudioContext` so tests and hosts can inject a `standardized-audio-context` context or a shared singleton without touching `globalThis`.
|
|
|
|
### 3. Plugin quarantine / error isolation
|
|
|
|
A plugin that throws during `process` or `initialize` is quarantined: marked `ERROR` state, disconnected from the graph, and a `plugin:error` event is emitted. The engine and other plugins continue operating.
|
|
|
|
### 4. Typed `AudioEngineEventMap`
|
|
|
|
Replace string-enum `AudioEngineEvents` with `AudioEngineEventMap` so `EventEmitter<AudioEngineEventMap>` provides per-event argument typing.
|
|
|
|
### 5. Graph snapshot / restore
|
|
|
|
`AudioGraph.snapshot()` serializes the routing topology (node IDs and connections) to a plain JSON object; `restore()` re-wires from that snapshot.
|
|
|
|
### 6. Worklet URL allow-list
|
|
|
|
`AudioWorkletManager` rejects `addModule` calls for URLs not present in the engine's configured `allowedWorkletUrls` list; throws `AUDIO_WORKLET_URL_REJECTED`.
|
|
|
|
### 7. OTel metrics
|
|
|
|
- Counter `audio.plugin_loaded_total`
|
|
- Counter `audio.plugin_failed_total`
|
|
- Histogram `audio.engine_latency_ms`
|
|
- Histogram `audio.worklet_roundtrip_ms`
|
|
- Histogram `audio.graph_connect_ms`
|
|
|
|
### 8. Render-on-demand offline mode
|
|
|
|
`AudioEngine.renderOffline(durationSeconds, sampleRate)` drives an `OfflineAudioContext` and returns an `AudioBuffer` for export / bounce workflows.
|
|
|
|
### 9. Parameter automation
|
|
|
|
`AudioPlugin.setParameterCurve(id, curve: ParameterCurve)` schedules sample-accurate linear/exponential/target-at-time automation via the Web Audio API param scheduling methods.
|
|
|
|
### 10. Benchmarks
|
|
|
|
`src/__benchmarks__/graph.bench.ts`, `queue.bench.ts`, `worklet.bench.ts` with mitata.
|
|
|
|
---
|
|
|
|
## File layout
|
|
|
|
```text
|
|
browser/audio/audio-engine/
|
|
package.json
|
|
tsconfig.json
|
|
vitest.config.ts
|
|
src/
|
|
index.ts
|
|
error-codes.ts
|
|
constants/
|
|
index.ts
|
|
types/
|
|
common.ts
|
|
common.md
|
|
engine/
|
|
engine/
|
|
engine.ts
|
|
engine.md
|
|
engine.types.ts
|
|
engine.test.ts
|
|
index.ts
|
|
engine-state/
|
|
engine-state.ts
|
|
engine-state.md
|
|
engine-state.test.ts
|
|
index.ts
|
|
engine-metrics/
|
|
engine-metrics.ts
|
|
engine-metrics.md
|
|
engine-metrics.test.ts
|
|
index.ts
|
|
graph/
|
|
graph/
|
|
graph.ts
|
|
graph.md
|
|
graph.types.ts
|
|
graph.test.ts
|
|
index.ts
|
|
graph-connect/
|
|
graph-connect.ts
|
|
graph-connect.md
|
|
graph-connect.test.ts
|
|
index.ts
|
|
graph-traverse/
|
|
graph-traverse.ts
|
|
graph-traverse.md
|
|
graph-traverse.test.ts
|
|
index.ts
|
|
queue-engine/
|
|
queue-engine/
|
|
queue-engine.ts
|
|
queue-engine.md
|
|
queue-engine.types.ts
|
|
queue-engine.test.ts
|
|
index.ts
|
|
cue/
|
|
cue.ts
|
|
cue.md
|
|
cue.test.ts
|
|
index.ts
|
|
scheduler/
|
|
scheduler.ts
|
|
scheduler.md
|
|
scheduler.test.ts
|
|
index.ts
|
|
worklet/
|
|
worklet-loader/
|
|
worklet-loader.ts
|
|
worklet-loader.md
|
|
worklet-loader.test.ts
|
|
index.ts
|
|
worklet-messaging/
|
|
worklet-messaging.ts
|
|
worklet-messaging.md
|
|
worklet-messaging.test.ts
|
|
index.ts
|
|
worklet-registry/
|
|
worklet-registry.ts
|
|
worklet-registry.md
|
|
worklet-registry.test.ts
|
|
index.ts
|
|
plugin/
|
|
plugin/
|
|
plugin.ts
|
|
plugin.md
|
|
plugin.types.ts
|
|
plugin.test.ts
|
|
index.ts
|
|
plugin-registry/
|
|
plugin-registry.ts
|
|
plugin-registry.md
|
|
plugin-registry.types.ts
|
|
plugin-registry.test.ts
|
|
index.ts
|
|
event-bus/
|
|
event-bus/
|
|
event-bus.ts
|
|
event-bus.md
|
|
event-bus.types.ts
|
|
event-bus.test.ts
|
|
index.ts
|
|
__benchmarks__/
|
|
graph.bench.ts
|
|
queue.bench.ts
|
|
worklet.bench.ts
|
|
tests/
|
|
integration.test.ts
|
|
```
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
**Downstream packages that will consume `@suaveplan/audio-engine`**: `@suaveplan/audio-plugins-fx`, `@suaveplan/audio-controllers`, `@suaveplan/react-audio`, `@suaveplan/react-audio-player`, `@suaveplan/r3f-audio`, `@suaveplan/react-audio-stream`.
|
|
|
|
**Breaking changes from main-branch API**: none. All main-branch call sites compile unchanged. New optional fields and methods are additive.
|
|
|
|
**Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events` must be published to Verdaccio before this package can be built or tested.
|