Initial Commit
This commit is contained in:
@@ -0,0 +1,35 @@
|
||||
# @suaveplan/audio-fx-plugin-spessasynth — SF2/SF3 SoundFont synthesizer plugin
|
||||
|
||||
- **Target path**: `packages/media/audio-fx/audio-fx-plugin-spessasynth/`
|
||||
- **Tier**: browser (lazy `import()` of `spessasynth_lib`; universal-tier core fallback for offline rendering)
|
||||
- **Date**: 2026-05-23
|
||||
- **Status**: PROPOSED
|
||||
|
||||
## Why
|
||||
|
||||
The audio-fx-plugin family covers EQ, mixer, distortion, delay, gate, pitch-shifter, phaser, tremolo, sampler, microphone, audio-player, and 5 others — but offers no general-purpose virtual instrument. Piano, organ, drum kits, orchestral instruments, classic synths — all of these are commodity-grade content delivered as SoundFont files (`.sf2` / `.sf3`). [SpessaSynth](https://github.com/spessasus/SpessaSynth) is an MIT-licensed pure-JS SoundFont 2.04 + 2.04+ synth engine with full GM/GS/XG support, 24-bit ogg-compressed SF3 support, polyphony management, modulators, and per-voice DSP — exactly the right primitive to wrap.
|
||||
|
||||
Wrapping SpessaSynth as an `AudioFxPlugin` (the same contract every existing fx plugin implements) means it composes inside any `PluginChain`, accepts MIDI from `@suaveplan/react-midi`, renders into `@suaveplan/audio-engine-browser` or `@suaveplan/audio-engine-server`, and exposes its parameter set (per-channel volume/pan/program-change/reverb-send/chorus-send + global master volume) through the same automation/snapshot/preset machinery every other plugin uses.
|
||||
|
||||
## What changes
|
||||
|
||||
1. **NEW package** `@suaveplan/audio-fx-plugin-spessasynth` at `packages/media/audio-fx/audio-fx-plugin-spessasynth/`.
|
||||
2. **Workspace catalog** — add `spessasynth_lib` (browser-tier) and the SF2 spec test files (vendored fixtures) under `catalog:audio`.
|
||||
3. **`SpessaSynthPlugin`** implementing `AudioFxPlugin` from `@suaveplan/audio-fx-plugin-base`:
|
||||
- `process(input, output, ctx)` — input is ignored (instrument is a source); output is the synth's rendered PCM at the chain's sample rate.
|
||||
- `init({ soundFontUrl?, soundFontBytes?, sampleRate })` — lazy `import("spessasynth_lib")`, async-loads SF2/SF3 from URL or bytes; rejects with `SPESSASYNTH.SOUNDFONT_LOAD_FAILED` on parse failure.
|
||||
- `dispose()` — releases the underlying synth + frees voice buffers.
|
||||
4. **MIDI input** — accepts `MidiMessage` events via a structural `MidiPortLike` consumer; routes channel-voice messages to the synth, program-change to the appropriate SF2 preset, NRPN/RPN to the standard handlers (per-note tuning, master fine-tune).
|
||||
5. **Parameter surface** — 16 channels × `{ volume, pan, program, reverbSend, chorusSend }` + master gain + per-channel mute/solo. Each is an automatable parameter the chain can drive (so a knob on a control surface can pan channel 1 in real time).
|
||||
6. **Voice management** — exposes `getActiveVoiceCount()`, `setMaxPolyphony(n)`, `allNotesOff(channel?)`. Voice stealing per the SF2 spec.
|
||||
7. **Preset / snapshot integration** — saves selected SoundFont URL, per-channel program selection, per-channel parameter values via `PluginChain`'s standard snapshot API.
|
||||
8. **Browser + server tiers** — same plugin works under `audio-engine-browser` (WebAudio) and `audio-engine-server` (offline render); SpessaSynth runs in either context since it's pure JS.
|
||||
9. **Error codes** — `SPESSASYNTH.{SOUNDFONT_LOAD_FAILED, SOUNDFONT_PARSE_FAILED, UNSUPPORTED_FORMAT, VOICE_LIMIT_EXCEEDED, NOT_INITIALIZED}` via `defineCodes`.
|
||||
10. **OTel instrumentation** — counter `spessasynth_voices_triggered_total`, gauge `spessasynth_voices_active`, histogram `spessasynth_process_duration_us`.
|
||||
11. **Demo SF2 fixture** — small (≤2 MiB) public-domain SoundFont vendored under `tests/fixtures/` for round-trip integration tests.
|
||||
|
||||
## Impact
|
||||
|
||||
- New package in the `audio-fx-plugin-*` family — automatically picked up by the existing `audio-fx-plugin-base` plugin-discovery / chain machinery.
|
||||
- Unblocks: virtual piano / drum-kit / synth UIs in `@suaveplan/react-control-surface`, the live-DJ scenario (DJ plays piano remotely, sound renders locally via SpessaSynth), GM/GS/XG playback in `@suaveplan/audio-cli`, multimedia exports in `@suaveplan/audio-engine-server`.
|
||||
- `spessasynth_lib` (~200 KiB minified) added as a lazy `import()` — gated by `optionalDependencies` so consumers who don't use this plugin don't pay the cost.
|
||||
@@ -0,0 +1,96 @@
|
||||
# audio-fx-plugin-spessasynth spec delta
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: SpessaSynthPlugin implements AudioFxPlugin
|
||||
|
||||
`SpessaSynthPlugin` SHALL implement `AudioFxPlugin` from `@suaveplan/audio-fx-plugin-base`, exposing `init`, `process`, `dispose`, `getParameters`, `snapshot`, `restore`. It is a **source** plugin (no audio input; output is the synth's rendered PCM).
|
||||
|
||||
#### Scenario: Render a single noteOn into a buffer
|
||||
|
||||
- **WHEN** `init({ soundFontUrl: "fixtures/gm.sf2", sampleRate: 48_000 })` resolves and the consumer feeds a `{ type: "noteOn", channel: 0, note: 60, velocity: 100 }` MIDI message followed by 48,000 samples of `process()` calls
|
||||
- **THEN** the produced output has non-zero RMS (audible voice rendered)
|
||||
|
||||
**Verified by:** `tests/integration.test.ts::"renders a noteOn"`
|
||||
|
||||
### Requirement: Lazy SF2 loading via dynamic import
|
||||
|
||||
`init()` SHALL `import("spessasynth_lib")` lazily so consumers who never load this plugin pay no cost.
|
||||
|
||||
#### Scenario: Bundle without import does not eagerly load SpessaSynth
|
||||
|
||||
- **WHEN** the package's `dist/index.js` is statically analysed for imports of `spessasynth_lib`
|
||||
- **THEN** zero static imports are found; only one dynamic `import("spessasynth_lib")` exists, inside `soundfont-loader.ts`
|
||||
|
||||
**Verified by:** `tests/bundle.test.ts::"spessasynth_lib is lazy-only"`
|
||||
|
||||
### Requirement: SoundFont parse failures surface typed errors
|
||||
|
||||
A corrupt SF2/SF3 file SHALL reject `init()` with `SpessaSynthError("SPESSASYNTH.SOUNDFONT_PARSE_FAILED")` carrying the underlying parse error on `cause`.
|
||||
|
||||
#### Scenario: Invalid SF2 bytes
|
||||
|
||||
- **WHEN** `init({ soundFontBytes: new Uint8Array([0,0,0,0]), sampleRate: 48_000 })` is called
|
||||
- **THEN** it rejects with code `SPESSASYNTH.SOUNDFONT_PARSE_FAILED`
|
||||
|
||||
**Verified by:** `src/loader/soundfont-loader.test.ts`
|
||||
|
||||
### Requirement: MIDI routing — 16 channels + program change + NRPN/RPN
|
||||
|
||||
The plugin SHALL route every channel-voice message to the synth, program-change messages to the corresponding SF2 preset (bank + program), and NRPN/RPN events to the registered handlers (per-note tuning, master fine-tune).
|
||||
|
||||
#### Scenario: Program-change selects piano patch on channel 0
|
||||
|
||||
- **WHEN** the synth is initialised with GM and a `programChange` to program 0 arrives on channel 0
|
||||
- **THEN** subsequent `noteOn` events on channel 0 render with the "Acoustic Grand Piano" preset
|
||||
|
||||
#### Scenario: NRPN per-note tuning
|
||||
|
||||
- **WHEN** an NRPN with paramHigh=0x65, paramLow=0x4c (per-note tuning) is received followed by a `noteOn` of note 60
|
||||
- **THEN** the rendered fundamental matches the tuned frequency (verified via FFT peak detection on the output)
|
||||
|
||||
**Verified by:** `tests/midi-routing.test.ts`
|
||||
|
||||
### Requirement: Parameter surface with chain-driven automation
|
||||
|
||||
The plugin SHALL expose 16-channel × 5-parameter (`volume`, `pan`, `program`, `reverbSend`, `chorusSend`) + master gain via `getParameters()`. Every parameter SHALL be drivable by the host chain's automation lanes.
|
||||
|
||||
#### Scenario: Automate channel 0 volume from 0.0 to 1.0 over 1 second
|
||||
|
||||
- **WHEN** the host chain schedules a volume automation on `channel/0/volume` from 0 to 1 over 48,000 samples
|
||||
- **THEN** the rendered output envelope tracks the schedule (RMS at sample 24,000 is approximately half RMS at sample 48,000, within 5%)
|
||||
|
||||
**Verified by:** `tests/automation.test.ts`
|
||||
|
||||
### Requirement: Voice management
|
||||
|
||||
The plugin SHALL expose `getActiveVoiceCount(): number`, `setMaxPolyphony(n: number): void`, and `allNotesOff(channel?: number): void`. When polyphony is exceeded, the oldest-released or quietest voice is stolen per the SF2 spec.
|
||||
|
||||
#### Scenario: Polyphony cap steals voices
|
||||
|
||||
- **WHEN** `setMaxPolyphony(4)` is called and 5 simultaneous noteOn events are sent
|
||||
- **THEN** `getActiveVoiceCount()` equals 4, not 5
|
||||
|
||||
**Verified by:** `src/voice-manager/voice-manager.test.ts`
|
||||
|
||||
### Requirement: Snapshot / restore round-trips state
|
||||
|
||||
Calling `snapshot()` SHALL produce a JSON-serialisable object capturing: soundfont URL, per-channel program selection, per-channel parameter values, master gain, polyphony cap. `restore(snapshot)` SHALL bring a fresh plugin instance to identical state.
|
||||
|
||||
#### Scenario: snapshot/restore round-trip
|
||||
|
||||
- **WHEN** plugin A is configured (load SF2, set channel-0 program 24, set channel-3 pan 0.5), `snap = A.snapshot()`, and plugin B is `init`ed then `restore(snap)`
|
||||
- **THEN** B's `getParameters()` matches A's exactly
|
||||
|
||||
**Verified by:** `src/plugin/spessasynth-plugin.test.ts`
|
||||
|
||||
### Requirement: OTel instrumentation
|
||||
|
||||
Every `noteOn` SHALL increment `spessasynth_voices_triggered_total`. `getActiveVoiceCount()` SHALL be reflected in gauge `spessasynth_voices_active`. Every `process()` call SHALL record `spessasynth_process_duration_us`.
|
||||
|
||||
#### Scenario: Voice gauge tracks live count
|
||||
|
||||
- **WHEN** 3 simultaneous noteOn events are active and the in-memory `MeterProvider` is sampled
|
||||
- **THEN** `spessasynth_voices_active` reads 3
|
||||
|
||||
**Verified by:** `src/telemetry/telemetry.test.ts`
|
||||
@@ -0,0 +1,53 @@
|
||||
# Tasks — enterprise-audio-fx-plugin-spessasynth
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 Account for every `.claude/worktrees/*` entry; prune stale ones via `bun scripts/worktree-prune.ts --remove`.
|
||||
- [ ] 0.3 Confirm `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json | wc -l`.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Draft `proposal.md` (this file's sibling), `tasks.md`, `specs/audio-fx-plugin-spessasynth/spec.md`
|
||||
|
||||
## 2. Catalog additions
|
||||
|
||||
- [ ] 2.1 Add `spessasynth_lib` (latest stable, pinned) to `catalog:audio` in root `package.json`
|
||||
- [ ] 2.2 `bun install` to update lockfile
|
||||
|
||||
## 3. Implementation
|
||||
|
||||
- [ ] 3.1 `src/error-codes.ts` — `SPESSASYNTH.{SOUNDFONT_LOAD_FAILED, SOUNDFONT_PARSE_FAILED, UNSUPPORTED_FORMAT, VOICE_LIMIT_EXCEEDED, NOT_INITIALIZED}` via `defineCodes`
|
||||
- [ ] 3.2 `src/loader/soundfont-loader.ts` — lazy `import("spessasynth_lib")`, async URL/bytes fetch, SF2/SF3 parse, validation
|
||||
- [ ] 3.3 `src/midi-router/midi-router.ts` — `MidiPortLike` consumer; channel-voice → synth, program-change → preset, NRPN/RPN → registered handlers
|
||||
- [ ] 3.4 `src/parameters/parameters.ts` — 16-channel × 5-param + master surface; bound through `audio-fx-plugin-base` automation
|
||||
- [ ] 3.5 `src/voice-manager/voice-manager.ts` — `getActiveVoiceCount`, `setMaxPolyphony`, `allNotesOff`, voice-stealing
|
||||
- [ ] 3.6 `src/plugin/spessasynth-plugin.ts` — implements `AudioFxPlugin` (`init`, `process`, `dispose`, `getParameters`, `snapshot`, `restore`)
|
||||
- [ ] 3.7 `src/telemetry/telemetry.ts` — counter `spessasynth_voices_triggered_total`, gauge `spessasynth_voices_active`, histogram `spessasynth_process_duration_us`
|
||||
- [ ] 3.8 `src/register/register.ts` — register the plugin with `PluginChain`'s discovery
|
||||
|
||||
## 4. Tests
|
||||
|
||||
- [ ] 4.1 Per-feature `.test.ts` covering every scenario
|
||||
- [ ] 4.2 `tests/integration.test.ts` — load a vendored SF2 fixture, render a C-major chord through `process()`, assert non-zero RMS
|
||||
- [ ] 4.3 `tests/midi-routing.test.ts` — feed a 4-bar MIDI sequence, assert correct voice activations
|
||||
- [ ] 4.4 `tests/automation.test.ts` — automate volume on channel 1 over a render, assert output envelope matches the schedule
|
||||
- [ ] 4.5 Every `it(...)` has `{ timeout: 5000 }` (30000 for the integration that loads SF2)
|
||||
|
||||
## 5. Documentation
|
||||
|
||||
- [ ] 5.1 Sibling `.md` per `.ts` (≥200 words, basic + advanced examples)
|
||||
- [ ] 5.2 README with quickstart + SF2 loading example + control-surface integration example
|
||||
|
||||
## 6. Pre-archive verification
|
||||
|
||||
- [ ] 6.1 `bunx turbo lint typecheck test --filter=@suaveplan/audio-fx-plugin-spessasynth` — green
|
||||
- [ ] 6.2 `bunx turbo test:coverage --filter=@suaveplan/audio-fx-plugin-spessasynth` — 100/100/100/100
|
||||
- [ ] 6.3 `bunx turbo build --filter=@suaveplan/audio-fx-plugin-spessasynth` — green
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold spec delta → `openspec/specs/audio-fx-plugin-spessasynth/spec.md`
|
||||
- [ ] 7.2 Move folder → `openspec/archive/<yyyy-mm-dd>-enterprise-audio-fx-plugin-spessasynth/`
|
||||
- [ ] 7.3 Refresh `MANIFEST.md`
|
||||
- [ ] 7.4 Prune worktree
|
||||
Reference in New Issue
Block a user