Initial Commit
This commit is contained in:
@@ -0,0 +1,135 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: audio-plugins-fx (extend-audio-plugins-fx-enterprise-extras)
|
||||
|
||||
**Package**: `@suaveplan/audio-plugins-fx`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. (See `openspec/AGENTS.md` Section 13.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Audio analyzer plugin
|
||||
|
||||
The `./audio-analyzer` subpath SHALL export `AudioAnalyzerPlugin`, an analysis-only tap point that passes input audio through unchanged while extracting frequency-domain and time-domain features. The plugin SHALL expose synchronous `getFrequencyData()` and `getWaveformData()` accessors returning the latest FFT magnitudes and the most recent input window respectively, plus async `getTempo()`, `getPitch()`, `getSpectralFeatures()`, `getEnergy()`, and `getKey()` getters that return immutable result records. It SHALL expose `onAnalysisUpdate(callback)` returning an unsubscribe function and call `callback` on a configurable cadence with a snapshot of the enabled analyses. It SHALL expose `applyPreset(name)` accepting one of `"music" | "speech" | "percussion" | "ambient"`, each preset toggling a documented subset of analyses on or off. `enable<X>Detection(boolean)` setters and `is<X>DetectionEnabled()` accessors SHALL be available for tempo / pitch / spectral / beat analyses so callers can tune the CPU budget per use case.
|
||||
|
||||
#### Scenario: passes audio through unchanged
|
||||
|
||||
- **WHEN** `AudioAnalyzerPlugin` processes a non-silent input block
|
||||
- **THEN** the output block equals the input block sample-for-sample within IEEE-clean tolerance
|
||||
|
||||
#### Scenario: applyPreset toggles the documented subset of analyses
|
||||
|
||||
- **WHEN** `applyPreset("speech")` is called on a default instance
|
||||
- **THEN** `isPitchDetectionEnabled()` becomes `true`, `isBeatDetectionEnabled()` becomes `false`, and the same toggles persist across the next block
|
||||
|
||||
#### Scenario: onAnalysisUpdate fires on the configured cadence
|
||||
|
||||
- **WHEN** `onAnalysisUpdate(callback)` is registered and the plugin processes blocks for at least the configured `updateIntervalMs`
|
||||
- **THEN** `callback` is invoked at least once with an `AnalysisUpdate` carrying the enabled analyses, and the returned unsubscribe stops further invocations
|
||||
|
||||
**Verified by:** `src/plugins/audio-analyzer/audio-analyzer.test.ts::"output equals input sample-for-sample on the audio path"` AND `src/plugins/audio-analyzer/audio-analyzer.test.ts::"applyPreset('speech') toggles the documented analyses"` AND `src/plugins/audio-analyzer/audio-analyzer.test.ts::"onAnalysisUpdate fires once per updateInterval and unsubscribe stops it"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Microphone simulation plugin
|
||||
|
||||
The `./microphone` subpath SHALL export `MicrophoneSimulationPlugin` and a `createMicrophonePlugin(options)` factory. The plugin SHALL be a source (`io.inputs[0].count === 0`, `io.outputs[0].count === 1`) that synthesises microphone-coloured audio from an oscillator + filtered-noise + dynamics + wow-and-flutter chain. Construction options SHALL include the structured control groups documented in the main-branch `microphone.types.ts` (`FrequencyControls`, `ResonanceControls`, `DistortionControls`, `NoiseControls`, `DynamicsControls`, `CompressorControls`, `GainControls`, `CharacterControls`, `WowAndFlutterControls`). The plugin SHALL expose a `getUISections()` accessor returning a `ReadonlyArray<MicrophoneUISection>` describing every control as a typed `MicrophoneUIControl` (id, label, min, max, step, unit, group) so editors can render the UI without hard-coding the layout.
|
||||
|
||||
#### Scenario: source plugin produces non-silent output
|
||||
|
||||
- **WHEN** `MicrophoneSimulationPlugin` is constructed with defaults and processes a 4096-sample block
|
||||
- **THEN** the output's RMS exceeds `1e-4` and every sample is finite
|
||||
|
||||
#### Scenario: UI section descriptor mirrors the configured controls
|
||||
|
||||
- **WHEN** `getUISections()` is called on a default instance
|
||||
- **THEN** the returned sections enumerate every `*Controls` field in `MicrophoneConfig` with stable `id` strings and bounded `min`/`max` values
|
||||
|
||||
#### Scenario: preset round-trip preserves the full control state
|
||||
|
||||
- **WHEN** `getPreset()` is called, the resulting payload is fed into `applyPreset()` on a fresh instance, and the second instance's controls are read
|
||||
- **THEN** every nested field matches the source instance's pre-snapshot state
|
||||
|
||||
**Verified by:** `src/plugins/microphone/microphone.test.ts::"default microphone produces non-silent output"` AND `src/plugins/microphone/microphone.test.ts::"getUISections enumerates every configured control"` AND `src/plugins/microphone/microphone.test.ts::"preset round-trip preserves the full microphone control state"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: GLaDOS voice-effect plugin
|
||||
|
||||
The `./glados` subpath SHALL export `GLaDOSPlugin`, an insert plugin (`io.inputs[0].count === io.outputs[0].count === "preserve"`) that runs the input audio through a fixed signal chain of pitch-shift → formant-shift → robotic-intensity tremolo → distortion → low-pass → reverb. The plugin SHALL expose typed setters for every stage: `setPitchShift`, `setFormantShift`, `setRoboticIntensity`, `setDistortionAmount`, `setReverbAmount`, `setLowPassCutoff`. It SHALL expose `applyPreset(name)` accepting one of `"glados-mainline" | "glados-test-chamber" | "glados-cake-deception" | "wheatley" | "cave-johnson" | "robotic-generic"`, each preset setting all six stage values together. Calling a setter with an out-of-range value SHALL throw `AudioFxError` with code `AUDIO_FX.INVALID_OPTIONS`.
|
||||
|
||||
#### Scenario: applyPreset sets every stage at once
|
||||
|
||||
- **WHEN** `applyPreset("wheatley")` is called on a default instance
|
||||
- **THEN** the six stage parameter values match the documented Wheatley preset record
|
||||
|
||||
#### Scenario: voice chain colours the input audibly
|
||||
|
||||
- **WHEN** the plugin processes a non-silent input block at the default `"glados-mainline"` preset
|
||||
- **THEN** the output differs from the input (per-sample RMS difference > `1e-3`) and every sample remains finite
|
||||
|
||||
#### Scenario: out-of-range setter is rejected
|
||||
|
||||
- **WHEN** `setRoboticIntensity(99)` is called with the parameter range documented as `[0, 1]`
|
||||
- **THEN** the call throws `AudioFxError` whose `code === "AUDIO_FX.INVALID_OPTIONS"` with the offending value in `context`
|
||||
|
||||
**Verified by:** `src/plugins/glados/glados.test.ts::"applyPreset('wheatley') sets every stage to the wheatley preset values"` AND `src/plugins/glados/glados.test.ts::"glados-mainline preset colours the input audibly"` AND `src/plugins/glados/glados.test.ts::"setRoboticIntensity rejects out-of-range values with AUDIO_FX.INVALID_OPTIONS"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Game audio plugin
|
||||
|
||||
The `./game` subpath SHALL export `GameAudioPlugin`, a composite source plugin that drives game-audio scenarios. It SHALL maintain a registry of spatial sources, music layers, and proximity-voice-chat participants:
|
||||
|
||||
- Spatial sources: `createSpatialSource(id, position)`, `updateSpatialSourcePosition(id, position)`, `applyDopplerEffect(id, velocity)`, `applyOcclusion(id, level)`, `cullDistantSources(maxDistance)`, `getActiveSpatialSourceCount()`.
|
||||
- Music layers: `addMusicLayer(name, buffer)`, `removeMusicLayer(name)`, `getMusicLayers()`, `crossfadeMusicLayers(from, to, durationMs)`, `isCrossfading()`.
|
||||
- Proximity voice chat: `enableProximityVoiceChat(config)`, `addVoiceChatParticipant(id, position)`, `removeVoiceChatParticipant(id)`, `updateVoiceChatParticipantPosition(id, position)`, `calculateVoiceAttenuation(sourcePosition)`, `optimizeVoiceChatBandwidth()`, `getVoiceChatOptimizationLevel()`.
|
||||
|
||||
The plugin SHALL expose `setListenerPosition` / `setListenerOrientation` and apply distance attenuation per the Web Audio inverse-distance model (matching `SpatialAudioPlugin`'s contract). Voice attenuation SHALL fall off monotonically as the participant moves away from the listener through the configured `[minDistance, maxDistance]` band.
|
||||
|
||||
#### Scenario: spatial source lifecycle is symmetric
|
||||
|
||||
- **WHEN** a caller `createSpatialSource("a", { x: 0, y: 0, z: 0 })` then queries `getActiveSpatialSourceCount()`
|
||||
- **THEN** the count is `1`; after `cullDistantSources(0)` the count is `0`
|
||||
|
||||
#### Scenario: voice attenuation falls off monotonically with distance
|
||||
|
||||
- **WHEN** `calculateVoiceAttenuation` is called for participants placed at increasing distances within `[minDistance, maxDistance]`
|
||||
- **THEN** the returned attenuation values strictly decrease across the sequence
|
||||
|
||||
#### Scenario: music-layer crossfade reports its in-flight state
|
||||
|
||||
- **WHEN** `crossfadeMusicLayers("intro", "main", 200)` is called and immediately after `isCrossfading()` is queried
|
||||
- **THEN** the call returns `true`; after at least `200` ms of process time has elapsed (verified via the universal `nowSec` opt), it returns `false`
|
||||
|
||||
**Verified by:** `src/plugins/game/game-audio.test.ts::"createSpatialSource + cullDistantSources balance the source registry"` AND `src/plugins/game/game-audio.test.ts::"calculateVoiceAttenuation decreases monotonically with distance"` AND `src/plugins/game/game-audio.test.ts::"crossfadeMusicLayers flips isCrossfading until the configured duration elapses"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Audio player plugin
|
||||
|
||||
The `./audio-player` subpath SHALL export `AudioPlayerPlugin`, a source plugin (`io.inputs[0].count === 0`, `io.outputs[0].count === 1`) that decodes a raw Float32 PCM stream from a URL into an internal buffer and plays it back through the universal `BlockProcessor` path. The plugin SHALL expose `load(url, { signal? }): Promise<void>` accepting an `AbortSignal`; pre-aborted signal SHALL throw `AudioFxError` with code `AUDIO_FX.ABORTED` before any fetch. After load it SHALL expose `play()`, `pause()`, `stop()`, `seek(timeSec)`, `setVolume(0..1)`, `setLoop(boolean)`, `setPlaybackRate(0.25..4)`, and `getState()` returning `{ playing, paused, currentTimeSec, durationSec, volume, loop, playbackRate }`. Triggering `play()` before any `load()` SHALL throw `AudioFxError` with code `AUDIO_FX.SAMPLE_NOT_LOADED`.
|
||||
|
||||
#### Scenario: load + play renders the loaded audio
|
||||
|
||||
- **WHEN** `load(dataUrl)` resolves and `play()` is called, then the plugin processes a block
|
||||
- **THEN** the output buffer contains the first samples of the loaded PCM scaled by `volume`
|
||||
|
||||
#### Scenario: play without load is rejected
|
||||
|
||||
- **WHEN** `play()` is called before any `load()`
|
||||
- **THEN** the call throws `AudioFxError` whose `code === "AUDIO_FX.SAMPLE_NOT_LOADED"`
|
||||
|
||||
#### Scenario: pre-aborted signal short-circuits load
|
||||
|
||||
- **WHEN** `load(url, { signal })` is called with an already-aborted signal
|
||||
- **THEN** the call throws `AudioFxError` whose `code === "AUDIO_FX.ABORTED"` before any fetch is issued
|
||||
|
||||
#### Scenario: setLoop with `true` re-plays after the end of the buffer
|
||||
|
||||
- **WHEN** a short buffer is loaded, `setLoop(true)` is called, and the plugin processes more samples than the buffer contains
|
||||
- **THEN** the output stream wraps back to the start of the buffer without producing silence in between
|
||||
|
||||
**Verified by:** `src/plugins/audio-player/audio-player.test.ts::"load + play renders the loaded buffer on the next process block"` AND `src/plugins/audio-player/audio-player.test.ts::"play before load throws AUDIO_FX.SAMPLE_NOT_LOADED"` AND `src/plugins/audio-player/audio-player.test.ts::"pre-aborted signal short-circuits before fetch"` AND `src/plugins/audio-player/audio-player.test.ts::"setLoop wraps playback back to the buffer start"`
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
Reference in New Issue
Block a user