suaveplan/openspec/archive/2026-05-23-enterprise-audio-fx-plugin-spessasynth/proposal.md

@suaveplan/audio-fx-plugin-spessasynth — SF2/SF3 SoundFont synthesizer plugin

• Target path: packages/media/audio-fx/audio-fx-plugin-spessasynth/
• Tier: browser (lazy import() of spessasynth_lib; universal-tier core fallback for offline rendering)
• Date: 2026-05-23
• Status: PROPOSED

Why

The audio-fx-plugin family covers EQ, mixer, distortion, delay, gate, pitch-shifter, phaser, tremolo, sampler, microphone, audio-player, and 5 others — but offers no general-purpose virtual instrument. Piano, organ, drum kits, orchestral instruments, classic synths — all of these are commodity-grade content delivered as SoundFont files (.sf2 / .sf3). SpessaSynth is an MIT-licensed pure-JS SoundFont 2.04 + 2.04+ synth engine with full GM/GS/XG support, 24-bit ogg-compressed SF3 support, polyphony management, modulators, and per-voice DSP — exactly the right primitive to wrap.

Wrapping SpessaSynth as an AudioFxPlugin (the same contract every existing fx plugin implements) means it composes inside any PluginChain, accepts MIDI from @suaveplan/react-midi, renders into @suaveplan/audio-engine-browser or @suaveplan/audio-engine-server, and exposes its parameter set (per-channel volume/pan/program-change/reverb-send/chorus-send + global master volume) through the same automation/snapshot/preset machinery every other plugin uses.

What changes

1. NEW package @suaveplan/audio-fx-plugin-spessasynth at packages/media/audio-fx/audio-fx-plugin-spessasynth/.
2. Workspace catalog — add spessasynth_lib (browser-tier) and the SF2 spec test files (vendored fixtures) under catalog:audio.
3. SpessaSynthPlugin implementing AudioFxPlugin from @suaveplan/audio-fx-plugin-base:
   • process(input, output, ctx) — input is ignored (instrument is a source); output is the synth's rendered PCM at the chain's sample rate.
   • init({ soundFontUrl?, soundFontBytes?, sampleRate }) — lazy import("spessasynth_lib"), async-loads SF2/SF3 from URL or bytes; rejects with SPESSASYNTH.SOUNDFONT_LOAD_FAILED on parse failure.
   • dispose() — releases the underlying synth + frees voice buffers.
4. MIDI input — accepts MidiMessage events via a structural MidiPortLike consumer; routes channel-voice messages to the synth, program-change to the appropriate SF2 preset, NRPN/RPN to the standard handlers (per-note tuning, master fine-tune).
5. Parameter surface — 16 channels × { volume, pan, program, reverbSend, chorusSend } + master gain + per-channel mute/solo. Each is an automatable parameter the chain can drive (so a knob on a control surface can pan channel 1 in real time).
6. Voice management — exposes getActiveVoiceCount(), setMaxPolyphony(n), allNotesOff(channel?). Voice stealing per the SF2 spec.
7. Preset / snapshot integration — saves selected SoundFont URL, per-channel program selection, per-channel parameter values via PluginChain's standard snapshot API.
8. Browser + server tiers — same plugin works under audio-engine-browser (WebAudio) and audio-engine-server (offline render); SpessaSynth runs in either context since it's pure JS.
9. Error codes — SPESSASYNTH.{SOUNDFONT_LOAD_FAILED, SOUNDFONT_PARSE_FAILED, UNSUPPORTED_FORMAT, VOICE_LIMIT_EXCEEDED, NOT_INITIALIZED} via defineCodes.
10. OTel instrumentation — counter spessasynth_voices_triggered_total, gauge spessasynth_voices_active, histogram spessasynth_process_duration_us.
11. Demo SF2 fixture — small (≤2 MiB) public-domain SoundFont vendored under tests/fixtures/ for round-trip integration tests.

Impact

• New package in the audio-fx-plugin-* family — automatically picked up by the existing audio-fx-plugin-base plugin-discovery / chain machinery.
• Unblocks: virtual piano / drum-kit / synth UIs in @suaveplan/react-control-surface, the live-DJ scenario (DJ plays piano remotely, sound renders locally via SpessaSynth), GM/GS/XG playback in @suaveplan/audio-cli, multimedia exports in @suaveplan/audio-engine-server.
• spessasynth_lib (~200 KiB minified) added as a lazy import() — gated by optionalDependencies so consumers who don't use this plugin don't pay the cost.