SuavePlan/suaveplan

Fork 0

Files

admin fbc70f96e6 Initial Commit

2026-05-27 23:29:53 +01:00

20 KiB

Raw Blame History

OpenSpec Change: enterprise-audio-plugins-fx

Change ID: enterprise-audio-plugins-fx
Package: @suaveplan/audio-plugins-fx
Category: universal (with optional browser-native fast paths)
Target path: packages/audio/audio-plugins-fx
Status: PROPOSED
Date: 2026-05-15
Design: ./design.md
Spec delta: ./specs/audio-plugins-fx/spec.md

Summary

Deliver @suaveplan/audio-plugins-fx on the genesis branch from scratch at enterprise quality. The package provides:

A future-proof AudioFxPlugin contract built on @suaveplan/audio-core's BlockProcessor — universal DSP that runs identically in Node/Bun/server, browser worklets, and offline render, with an optional createAudioNode() fast path for browser-native Web Audio rendering.
A PluginChain universal primitive — the missing piece. A first-class, headless, ordered plugin list with insertion-time channel validation, plugin delay compensation, hot-swap, snapshot/restore, an event bus, and chain-level bypass wrapping. The chain is what enterprise DAWs use as the "insert FX slot list"; it lives in audio-plugins-fx so every engine adapter (browser, server, offline) reuses one implementation.
A ChainContext / AmbientState propagation system — every plugin in a chain receives a read-only view of the chain's structure (predecessors, successors, total length), of ambient state (track gain/pan/mute/solo/layout, transport BPM/time/state, listener position for 3D), and of project-level sidechain buses. This is the "config tree plugins can traverse" requirement: a late-chain compressor can read ctx.ambient.track.pan, a tempo-synced delay can read ctx.ambient.transport.tempo, a sidechain compressor can pull from ctx.buses.get('kick'). No global state, no prop drilling, fully typed.
22 first-party plugins — EQ, filter, compressor, limiter, gate, reverb, delay, chorus, phaser, flanger, tremolo, auto-filter, distortion, pitch shifter, synthesizer, sampler, mixer, stereo widener, plus four newly-added plugins from upstream (microphone, glados, audio-player) and the building blocks for game audio (spatial-positional, proximity-voice, music-layer-mixer, voice-transform).
Subpath exports per plugin — sideEffects: false, one package.json exports entry per plugin so importing ./eq does not bundle ./synthesizer.

The package is universal-tier: zero browser globals, zero Node globals on the BlockProcessor path. Browser-native rendering and worklet hosting are optional surfaces that engine adapters consume via capability flags.

Current-state audit (genesis)

The package does not exist on genesis yet — this is a fresh build. The upstream packages/browser/audio-plugins-fx/ is browser-only and built on the obsolete monolithic @suaveplan/audio-engine; it is being replaced with this universal architecture.

Parity Checklist (vs upstream `../suaveplan/packages/browser/audio-plugins-fx/`)

Capability	Upstream	Migrated	Notes
`AudioPlugin` base class	✅	⬆️	Replaced by `AudioFxPlugin` interface extending `BlockProcessor` from `@suaveplan/audio-core`. Now universal — runs in browser, server, offline, worklet contexts identically. Optional `createAudioNode` / `getWorkletProcessor` for browser-native fast paths.
`EQPlugin` (8-band parametric)	✅	✅	Same surface; now extends `AbstractAudioFxPlugin`; presets validated via Zod.
`CompressorPlugin`	✅	⬆️	Adds sidechain via `setSidechain(bus)` consuming the project-level `BusRegistry`; adds mid/side mode reading `ChainContext.ambient.track.layout`.
`ReverbPlugin` (IR + algorithmic)	✅	⬆️	`loadImpulseResponse` now accepts `AbortSignal`.
`DelayPlugin` (ping-pong + BPM sync)	✅	⬆️	BPM sync now reads `ChainContext.ambient.transport.tempo` instead of a separate transport reference.
`PitchShifterPlugin` (phase vocoder + worklet)	✅	⬆️	Reports FFT-frame `latencySamples` via the new `onLatencyChanged` event so chain PDC stays accurate.
`SynthesizerPlugin` (oscillator + ADSR + voice pool + portamento)	✅	✅	Source plugin (`io.inputs[0].count === 0`); validates note range with `AUDIO_FX.INVALID_NOTE`.
`MixerPlugin` (channel strips + sends + returns + master + VU)	✅	✅	Same surface; integrated with the `PluginChain`'s ambient state for solo across the project.
`GLaDOSPlugin` (character vocoder)	✅	✅	Preset envelope validated against the plugin's Zod schema.
`MicrophonePlugin` (full mic processing chain)	✅	✅	Worklet path is the browser fast path; universal `BlockProcessor` is the server path.
`AudioPlayerPlugin` (track-style playback)	✅	✅	Source plugin; distinct from `@suaveplan/audio-sprite-player` (cue triggering).
`AudioAnalyzerPlugin` (FFT / tempo / pitch / spectral)	✅	🚫	Scoped out: replaced by dedicated `@suaveplan/audio-analysis-{beat,pitch,loudness,spectrum}` packages so analysis plugins stay headless and don't depend on the FX-plugin surface. The plugin's old features are 100 % covered by the four split packages.
`GameAudioPlugin` (spatial + voice-chat + music layers + voice transform)	✅	⬆️	Decomposed into four building blocks (`spatial-positional`, `proximity-voice`, `music-layer-mixer`, `voice-transform`) so non-R3F games can consume them directly; `r3f-audio` composes them for the R3F surface.
New plugins (`filter`, `limiter`, `gate`, `chorus`, `phaser`, `flanger`, `tremolo`, `auto-filter`, `distortion`, `sampler`, `widener`)	–	⬆️	Genesis additions covering the standard FX surface every DAW expects.
Chain primitive (`PluginChain`)	–	⬆️	Genesis addition — universal insert list with insertion-time IO validation, chain-level bypass, ambient-state propagation, plugin delay compensation, snapshot/restore, typed event surface.
Project-level `BusRegistry`	–	⬆️	Genesis addition — sidechain routing crossing tracks/chains.
Parameter automation curves + transport-time scheduling	partial (per-plugin)	⬆️	Universal `ParameterSchedule` evaluator owns wall-clock curves + transport-time events with shared queue; transport events override in-flight curves.
Preset system	partial	⬆️	Universal `PresetSnapshot` envelope with `schemaVersion`, Zod validation at the boundary, atomic apply.
Subpath exports per plugin	partial	⬆️	One subpath per plugin + `./base` + `./chain` + `./registry`; bundle-size tests prove tree-shaking.
Unit tests / coverage	partial	⬆️	100/100/100/100 across every module; property-based tests; per-plugin and chain benchmarks.

One 🚫 row, justified: audio-analyzer is replaced by the dedicated audio-analysis-* family so analysis stays headless and composable with non-FX consumers (CLIs, server pipelines).

Available workspace deps on genesis

Package	Role
`@suaveplan/audio-core` (`workspace:^`)	`BlockProcessor`, `BlockContext`, `ProcessorConfig`, `AudioError` base
`@suaveplan/audio-transport` (`workspace:^`)	`Transport` reference for `setParameterAt(transportTime)` resolution
`@suaveplan/schema-registry` (`workspace:^`)	Plugin preset Zod schema registration
`@suaveplan/events` (`workspace:^`)	Typed event bus for chain events
`@suaveplan/error` (`workspace:^`)	`defineCodes`, `SuaveplanError`
`@suaveplan/types` (`workspace:^`)	`LoggerLike`, `Clock`
`@suaveplan/opentelemetry` (`workspace:^`)	OTel spans, counters, histograms
`zod` (`catalog:forms`)	Source of truth for parameter shapes + preset schemas

Dev deps: vitest, @vitest/coverage-istanbul, fast-check, mitata (all catalog:testing).

Public API

Core contract

interface AudioFxPlugin extends BlockProcessor {
  readonly metadata: PluginMetadata
  readonly schema: ZodType<unknown>                            // source of truth for options + preset
  readonly parameters: ReadonlyArray<PluginParameter>          // declaration order, for UI iteration
  readonly parameterIndex: ReadonlyMap<string, PluginParameter> // O(1) lookup
  readonly io: PluginIo                                        // declared channel topology
  readonly capabilities: PluginCapabilities                    // explicit feature flags

  // Lifecycle
  setup(config: ProcessorConfig, chain: ChainContext, opts?: { signal?: AbortSignal }): Promise<void>
  process(input: Float32Array[], output: Float32Array[], ctx: BlockContext): void
  reset(): void
  dispose(): void
  getLatencySamples(): number                                  // for plugin delay compensation

  // Parameters
  getParameter(id: string): number
  setParameter(id: string, value: number): void
  setParameterCurve(id: string, curve: ParameterCurve): void           // ramp over seconds
  setParameterAt(id: string, value: number, transportTime: number): void // ramp to musical-time
  onLatencyChanged(listener: (samples: number) => void): UnsubscribeFn

  // Atomic snapshots
  getPreset(): PresetSnapshot
  applyPreset(p: PresetSnapshot): void

  // Optional browser-native fast paths
  createAudioNode?(ctx: BaseAudioContext, chain: ChainContext): AudioNode
  getWorkletProcessor?(): string

  // Optional sidechain consumer
  setSidechain?(bus: SidechainBus | undefined): void
}

interface PluginIo {
  readonly inputs:  ReadonlyArray<ChannelSpec>
  readonly outputs: ReadonlyArray<ChannelSpec>
}

interface ChannelSpec {
  readonly count: number | 'preserve' | 'any'
  readonly layout?: ChannelLayout                              // 'mono' | 'stereo' | 'mid-side' | '5.1' | '7.1' | 'ambisonic-1'
}

interface PluginCapabilities {
  readonly hasAudioNode: boolean
  readonly hasWorklet: boolean
  readonly hasSidechain: boolean
  readonly sampleAccurate: boolean
  readonly stateful: boolean
  readonly asyncSetup: boolean
  readonly preservesChannelCount: boolean
  readonly latencySamples: number                              // initial; dynamic via getLatencySamples()
}

interface PluginParameter {
  readonly id: string
  readonly name: string
  readonly value: number
  readonly min: number
  readonly max: number
  readonly default: number
  readonly unit?: ParameterUnit                                // 'db' | 'hz' | 'ms' | 'percent' | 'semitones' | …
  readonly scale?: 'linear' | 'log' | 'exponential'
  readonly automatable?: boolean
}

interface ParameterCurve {
  readonly type: 'linear' | 'exponential' | 'target'
  readonly value: number
  readonly duration: number
  readonly startTime?: number
}

The PluginChain primitive

class PluginChain {
  constructor(opts?: PluginChainOptions)                       // { id, name, channels, sampleRate, blockSize, buses?, ambient? }

  // Composition (insertion-time IO validation; events emitted)
  add(plugin: AudioFxPlugin, at?: number): void
  remove(idOrIndex: string | number): void
  move(from: number, to: number): void
  replace(idOrIndex, plugin, opts?: { crossfadeMs?: number }): Promise<void>
  bypass(idOrIndex, bypassed: boolean): void                   // chain-level wrapper; standard DAW behavior
  // Note: dry/wet is per-plugin (matching Pro Tools / Logic / Cubase / Reaper / Studio One);
  // the chain does NOT auto-wrap dry/wet. Plugins that need it expose a `wet` parameter.

  // Lifecycle
  setup(config: ProcessorConfig, ambient: AmbientState, opts?: { signal?: AbortSignal }): Promise<void>
  process(input: Float32Array[], output: Float32Array[], ctx: BlockContext): void
  reset(): void
  dispose(): void

  // Inspection — the "config tree plugins can traverse"
  readonly plugins: ReadonlyArray<AudioFxPlugin>
  findById(id: string): { plugin: AudioFxPlugin; index: number } | undefined
  getEffectiveLatencySamples(): number                         // sum across non-bypassed plugins (PDC)
  snapshot(): ChainSnapshot                                    // Zod-validated session save
  restore(s: ChainSnapshot): Promise<void>

  // Ambient updates pushed from track / transport / scene
  updateAmbient(patch: Partial<AmbientState>): void

  // Events
  on(event: ChainEvent, listener): UnsubscribeFn
  // events: 'plugin-added' | 'plugin-removed' | 'plugin-moved' | 'plugin-replaced'
  //       | 'parameter-changed' | 'preset-applied' | 'bypass-changed'
  //       | 'ambient-changed' | 'latency-changed'
}

Ambient state and ChainContext

interface ChainContext {
  readonly chain: ChainHandle                                  // read-only view of the chain
  readonly position: { readonly index: number; readonly total: number }
  readonly predecessors: ReadonlyArray<PluginHandle>           // upstream peers
  readonly successors:   ReadonlyArray<PluginHandle>           // downstream peers
  readonly ambient: AmbientState
  readonly buses: BusRegistry                                  // project-level (engine-owned)
}

interface PluginHandle {
  readonly id: string
  readonly metadata: PluginMetadata
  readonly io: PluginIo
  readonly capabilities: PluginCapabilities
  readonly bypassed: boolean
  getParameter(id: string): number                             // READ-ONLY view of peer parameters
  getLatencySamples(): number
}

interface AmbientState {
  readonly track: {
    readonly id: string
    readonly gainDb: number
    readonly pan: number                                       // -1 (full L) .. +1 (full R) — the LR-balance case
    readonly mute: boolean
    readonly solo: boolean
    readonly layout: ChannelLayout
  }
  readonly transport: {
    readonly tempo: number                                     // BPM
    readonly timeSignature: TimeSignature
    readonly transportTimeSec: number
    readonly state: 'stopped' | 'playing' | 'paused' | 'recording'
    readonly ppqn: number
  }
  readonly listener?: { readonly position: Vec3; readonly orientation: Quat; readonly velocity: Vec3 }
  readonly extension: Readonly<Record<string, unknown>>        // open extension point
}

Project-level bus registry (engine-owned, chain references it)

interface BusRegistry {
  get(name: string): SidechainBus | undefined
  list(): ReadonlyArray<string>
  publish(name: string, channels: number): SidechainPublisher  // create or get
  subscribe(name: string, listener: (bus: SidechainBus | undefined) => void): UnsubscribeFn
}

interface SidechainBus {
  readonly name: string
  readonly channels: number
  read(into: Float32Array[]): void                             // block-aligned read
}

interface SidechainPublisher {
  write(samples: Float32Array[]): void                         // chain writes its post-fx output
  close(): void
}

The chain reads from ctx.buses via its ChainContext. The actual BusRegistry instance lives on the engine adapter (audio-engine-browser / audio-engine-server) so cross-track / cross-chain routing works across the whole project.

Subpath exports (one per plugin + the chain + base)

@suaveplan/audio-plugins-fx
├── ./base       AudioFxPlugin, PluginIo, PluginCapabilities, ParameterCurve, PresetSnapshot, AbstractAudioFxPlugin
├── ./chain      PluginChain, ChainContext, AmbientState, BusRegistry, ChainSnapshot, ChainEvent
├── ./registry   PluginRegistry (id → factory; for session restore)
├── ./eq, ./filter, ./compressor, ./limiter, ./gate
├── ./reverb, ./delay, ./chorus, ./phaser, ./flanger
├── ./tremolo, ./auto-filter
├── ./distortion, ./pitch-shifter
├── ./synthesizer, ./sampler
├── ./mixer, ./widener
├── ./spatial-positional         ← building block
├── ./proximity-voice            ← building block
├── ./music-layer-mixer          ← building block
├── ./voice-transform            ← building block
├── ./microphone                 ← new from upstream
├── ./glados                     ← new from upstream
├── ./audio-player               ← new from upstream
└── ./presets/*                  lazy preset packs

New additions for genesis (beyond upstream)

Universal BlockProcessor base — every plugin runs headless. Browser-native rendering is optional.
PluginChain primitive with chain-level bypass, snapshot/restore, hot-swap, events, PDC computation.
ChainContext + AmbientState — read-only ambient propagation for track / transport / listener / sidechain.
PluginIo channel topology — explicit input/output channel specs validated at chain insertion; supports upmix, downmix, channel-count change.
PluginCapabilities flags — explicit feature detection; replaces opacity of optional methods.
Project-level BusRegistry — first-class sidechain routing.
Plugin delay compensation — every plugin reports getLatencySamples(); chains sum it; engines apply per-track compensation.
Dynamic latency — onLatencyChanged event for lookahead limiters etc.
Transport-synced parameter automation — setParameterAt(transportTime) for DAW automation lanes.
Async setup with AbortSignal — reverb IRs, sampler files, worklet modules load via setup(): Promise<void> cancellable mid-load.
Zod-typed parameter schemas — single source of truth; parameter list + preset shape derived from one schema.
Game-audio decomposed into building blocks — spatial-positional, proximity-voice, music-layer-mixer, voice-transform — usable in r3f-audio and non-R3F games alike.
Three plugins ported from upstream — microphone, glados, audio-player.
OTel instrumentation per-plugin and per-chain — latency histograms, block counters, parameter-change counters, preset-apply counters.
Property-based tests — gain plugin: 0 dB → identity, -∞ dB → silence; bypass → bit-identity; chain order invariance for commutative pairs.
Per-plugin and chain benchmark suite — block throughput, chain composition overhead.

Impact

Breaking changes: none — fresh build, package does not yet exist on genesis.

Prerequisites: @suaveplan/audio-core, @suaveplan/audio-transport, @suaveplan/schema-registry, @suaveplan/error, @suaveplan/types, @suaveplan/opentelemetry, @suaveplan/events published to Verdaccio.

Affected dependents

@suaveplan/audio-plugins-fx sits in the FX tier. At the time of this change none of its dependents exist on genesis yet — they land in later changes in this same batch — so there is no existing dependent surface to test or migrate. Each dependent is created against this package's spec delta, not retrofitted.

@suaveplan/audio-engine-browser — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in enterprise-audio-engine-browser. It hosts one PluginChain per track via the chain's process() method, owns the project BusRegistry, and maps each plugin's createAudioNode() into its node graph where present.)
@suaveplan/audio-engine-server — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in enterprise-audio-engine-server. It hosts the chain's BlockProcessor path per track and owns the project BusRegistry.)
@suaveplan/audio-controllers — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in enterprise-audio-controllers. It binds MIDI CCs to plugin.parameterIndex.get(id) lookups for live parameter control.)
@suaveplan/audio-cli — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in enterprise-audio-cli.)
@suaveplan/r3f-audio — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: requires re-authoring per openspec/changes/REACT_AUDIO_WAVE_REWRITE_COMPLETE.md. The rewritten version composes the four spatial-positional / proximity-voice / music-layer-mixer / voice-transform building blocks from this package.)

Once landed, any subsequent change to this package's public surface MUST run bunx turbo test --filter=...@suaveplan/audio-plugins-fx per openspec/AGENTS.md §13 step 10.

20 KiB Raw Blame History Unescape Escape