Design: enterprise-audio-engine-browser

Change ID: enterprise-audio-engine-browser Package: @suaveplan/audio-engine-browser

@suaveplan/audio-engine-browser is the browser rendering engine of the audio family's three-layer model — the layer that turns universal timeline state into actual sound through the Web Audio API. It is the only browser-only package in the audio core: everything below it (audio-core, audio-timeline, audio-transport, audio-plugins-fx) is universal, and everything above it (react-audio*, r3f-audio) is application-tier. This document carries the package-specific design; the cross-cutting architecture for the whole @suaveplan/audio-* ecosystem lives in enterprise-audio-core/design.md §1–§4 (the three-layer separation, the six plugin slots, the genesis-package reuse table, the universal Segment shape, and the error hierarchy) and is referenced here, not duplicated.

1. Position in the three-layer model

Per enterprise-audio-core/design.md §1.2, the audio ecosystem separates the data model (audio-core) from the timeline/transport layer (audio-timeline, audio-transport) from the rendering engines (audio-engine-*). This package is one of those rendering engines — the browser one. Its sibling audio-engine-server renders the same timeline state to Node/Bun file I/O and speakers; both implement the same TimelineAdapter interface (plugin slot 5) so a TimelineEngine is engine-agnostic.

The design consequence: this package contains no timeline logic. Track ordering, clip placement, musical-time conversion, undo/redo, and tempo maps all live in @suaveplan/audio-timeline. BrowserTimelineAdapter is a pure translation layer — it receives already-resolved Track[] and playback commands and maps them onto Web Audio AudioNode graphs. This is what makes the legacy "browser-only" audio openspecs obsolete: they tangled timeline state into the rendering code; the adapter-based architecture keeps them apart.

2. Why an injectable context seam

Every browser audio API the package touches — AudioContext, OfflineAudioContext, AudioWorklet, AudioNode — is reached through one of two seams:

The context option on BrowserAdapterOptions / the WebAudioGraph and AudioWorkletManager constructors. When supplied, the package uses it verbatim and never reads globalThis.
standardized-audio-context when no context is injected — a cross-browser shim that smooths over Safari/Firefox/Chrome AudioContext differences and, critically, provides a real OfflineAudioContext implementation usable under happy-dom.

This is the same testability discipline audio-core applies to its adapters (enterprise-audio-core "Universal-tier purity" requirement), adapted for the browser tier. The package may name browser APIs — that is its job — but it must be drivable in CI without a real browser. Every test injects an OfflineAudioContext; globalThis.AudioContext is never read in the test path.

Tier purity: the package must not import Node-only APIs (node:fs, node:path, process, Buffer). There is no filesystem in this layer — file I/O belongs to audio-engine-server. The purity test greps the built dist/ for those identifiers and asserts zero hits.

3. WebAudioGraph build strategy

WebAudioGraph is a thin, id-keyed registry over a BaseAudioContext. It exists so the adapter can address nodes by stable string id (e.g. track:42:gain) rather than juggling live AudioNode references, and so a graph can be snapshotted to a plain serializable object and restored.

class WebAudioGraph {
  private readonly nodes = new Map<string, AudioNode>()
  private readonly edges: Connection[] = []

  connect(sourceId: string, destinationId: string, outputIndex = 0, inputIndex = 0): void {
    const source = this.nodes.get(sourceId)
    const destination = this.nodes.get(destinationId)
    if (!source) throw new BrowserAudioError('AUDIO_ENGINE_BROWSER.NODE_NOT_FOUND', { context: { id: sourceId } })
    if (!destination) throw new BrowserAudioError('AUDIO_ENGINE_BROWSER.NODE_NOT_FOUND', { context: { id: destinationId } })
    source.connect(destination as AudioNode, outputIndex, inputIndex)
    this.edges.push({ source: sourceId, destination: destinationId, outputIndex, inputIndex })
  }

  snapshot(): AudioGraphSnapshot {
    const nodes: Record<string, AudioNodeType> = {}
    for (const [id, node] of this.nodes) nodes[id] = classifyNode(node)
    return { nodes, connections: this.edges.map((e) => ({ ...e })) }
  }
}

snapshot() records node types, not node instances — an AudioGraphSnapshot is JSON-safe so it can be persisted in a session file or diffed for debugging. restore() rebuilds typed nodes against the graph's context and replays the edge list. The snapshot is Zod-validated on restore so a corrupt session file fails fast at the boundary rather than mid-render.

4. Per-track subgraph

Each Track becomes a fixed-shape subgraph. The chain is deliberately the same for every track so that mute/solo, volume, and pan map to predictable nodes:

AudioBufferSourceNode (per clip)
  → GainNode            (clip volume)
  → GainNode            (track volume)   ← setTrackVolume / mute / solo write here
  → StereoPannerNode    (track pan)      ← setTrackPan writes here
  → DynamicsCompressorNode (limiter)
  → GainNode            (master mix)     ← setMasterVolume writes here
  → AnalyserNode        (master analysis)
  → context.destination

Solo semantics. DAW solo is a whole-mix operation, not a per-track flag. When any track is soloed, every non-soloed track's track-volume GainNode is set to 0; when the last solo is cleared, each track's gain reverts to its mute-governed value. The adapter keeps a Set<soloed> and recomputes effective gains on every setTrackMute / setTrackSolo call — this is the one piece of real state the adapter owns, and it is small and easy to test against an OfflineAudioContext.

removeTrack / updateTrack / the per-track setters throw AUDIO_ENGINE_BROWSER.TRACK_NOT_FOUND for an unknown id rather than silently no-op'ing — a missing track id is a caller bug, not a recoverable condition.

5. FX plugin hosting — three paths

An AudioFxPlugin from @suaveplan/audio-fx-plugin-base is dual-mode by design (plugin slot 6): it always exposes a universal processBlock, and optionally a browser-native createAudioNode. The host picks the best available path:

function createFxNode(plugin: AudioFxPlugin, context: BaseAudioContext, worklets: AudioWorkletManager): AudioNode {
  // 1. AudioNode path — native, lowest latency, runs on the audio thread
  if (plugin.createAudioNode) return plugin.createAudioNode(context)

  // 2. Worklet path — universal processBlock compiled into an AudioWorkletProcessor
  if (context.audioWorklet && plugin.getWorkletProcessor) {
    worklets.addModule(plugin.getWorkletProcessor())
    return worklets.createNode(plugin.name)
  }

  // 3. Fallback path — ScriptProcessorNode on the main thread (deprecated, but universal)
  const sp = context.createScriptProcessor(BLOCK_SIZE, 2, 2)
  sp.onaudioprocess = (e) => {
    const input = [e.inputBuffer.getChannelData(0), e.inputBuffer.getChannelData(1)]
    const output = [e.outputBuffer.getChannelData(0), e.outputBuffer.getChannelData(1)]
    plugin.processBlock(input, output, { sampleRate: context.sampleRate, blockSize: BLOCK_SIZE })
  }
  return sp
}

Path 1 is preferred because a native chain runs entirely on the audio rendering thread. Path 2 keeps DSP off the main thread when the plugin only ships universal block code. Path 3 exists so every FX plugin works somewhere — ScriptProcessorNode is deprecated but universally available, and the fallback is honest about its main-thread cost. Whichever node is chosen, it is inserted into the track subgraph through WebAudioGraph so it sits in the signal path between the clip gain and the track gain.

6. Offline render

renderOffline is the bridge to non-realtime export. It wires a BrowserTimelineAdapter to an OfflineAudioContext and lets the Web Audio engine render faster than realtime:

async function renderOffline(timeline: TimelineEngine, duration: number, sampleRate: number): Promise<AudioBuffer> {
  const offline = new OfflineAudioContext(2, Math.ceil(duration * sampleRate), sampleRate)
  const adapter = new BrowserTimelineAdapter({ context: offline })
  await adapter.init()
  timeline.setAdapter(adapter)
  timeline.play(0)
  return offline.startRendering()
}

Because the adapter takes an injected context, offline render reuses the exact same rendering code path as realtime playback — there is no separate offline renderer to keep in sync. This is also why the test suite can exercise the whole adapter against an OfflineAudioContext: the "tests" and the "export feature" are the same seam.

7. Html5AudioFallback

For environments with no Web Audio API at all (locked-down embedded webviews, very old browsers), Html5AudioFallback implements the same TimelineAdapter interface using a bare HTMLAudioElement. It supports only what an <audio> element can honour — play / pause / stop / seek / getCurrentTime / setLoop / setMasterVolume. Operations it cannot perform (per-track volume/mute/solo/pan, setTempo, FX hosting, sample-accurate scheduling) are defined no-ops or throw BrowserAudioError; they never silently corrupt state. The fallback never constructs an AudioContext. It is a graceful-degradation path, not a feature-equivalent engine, and the spec is explicit about that boundary.

8. File layout

packages/audio/audio-engine-browser/
  package.json, tsconfig.json, vitest.config.ts
  src/
    index.ts
    error-codes.ts (+ .test.ts)
    adapter/   browser-adapter.ts (+ .md .types.ts .test.ts) · fx-host.ts (+ .md .test.ts) · index.ts
    graph/     web-audio-graph.ts (+ .md .types.ts .test.ts) · index.ts
    worklet/   worklet-manager.ts (+ .md .types.ts .test.ts) · index.ts
    fallback/  html5-fallback.ts (+ .md .test.ts) · index.ts
    offline/   offline-render.ts (+ .md .test.ts) · index.ts
    __benchmarks__/  graph.bench.ts  worklet.bench.ts
  tests/
    integration.test.ts  purity.test.ts  exports.test.ts

Each public options object (BrowserAdapterOptions, WorkletManagerOptions, WorkletModuleOptions, AudioGraphSnapshot) gets a sibling *.schemas.ts with a Zod schema, validated at the boundary.

9. Integration notes

@suaveplan/audio-timeline — supplies the TimelineAdapter interface (plugin slot 5), TimelineEngine, Track, and Clip. BrowserTimelineAdapter and Html5AudioFallback both implement TimelineAdapter; the adapter receives resolved Track[] and never re-derives timeline state.
@suaveplan/audio-transport — supplies the Transport musical clock; lookAhead and ppqn on the adapter feed the transport's lookahead scheduler.
@suaveplan/audio-fx-plugin-base — supplies the AudioFxPlugin interface (plugin slot 6) hosted by createFxNode.
@suaveplan/audio-core — supplies shared audio primitives (BLOCK_SIZE, the BlockProcessor contract that processBlock satisfies) used by the ScriptProcessorNode fallback path.
@suaveplan/events — backs the adapter's typed event surface.
standardized-audio-context — the cross-browser AudioContext / OfflineAudioContext shim; the sole non-@suaveplan runtime dependency beyond @opentelemetry/api.

10. Benchmarks

graph.bench.ts — add 100 nodes, connect 200 edges, then snapshot() + restore().
worklet.bench.ts — load 10 worklet modules and create 50 AudioWorkletNodes.

11. Decisions and trade-offs

Adapter, not engine. The package is a TimelineAdapter implementation with zero timeline logic. This makes it swappable with audio-engine-server and obsoletes the legacy browser-only audio openspecs that tangled timeline state into rendering code.
One injectable context seam. Every browser API goes through the context option or standardized-audio-context. The cost is one extra constructor parameter; the payoff is that the whole package — including offline render — is testable under happy-dom and that offline export and realtime playback share one code path.
Fixed-shape per-track subgraph. Every track gets the identical node chain. A uniform shape makes mute/solo/volume/pan map to predictable nodes and keeps snapshot()/restore() trivial, at the cost of a few unused nodes on a track with no pan or compression — acceptable, since GainNode/StereoPannerNode at unity are effectively free.
Three FX hosting paths, preference-ordered. Native createAudioNode first, worklet second, ScriptProcessorNode last. ScriptProcessorNode is deprecated but kept as the universal fallback so no FX plugin is ever unhostable; the spec is explicit that it runs on the main thread.
Html5AudioFallback is degradation, not parity. It implements TimelineAdapter so callers need no branching, but unsupported operations throw or no-op rather than pretending to work.
No design.md for the React/r3f tier yet. react-audio* and r3f-audio consume this package but are out of scope for this 13-change batch; they land in a later wave.

13 KiB Raw Blame History Unescape Escape