119 lines
6.2 KiB
Markdown
119 lines
6.2 KiB
Markdown
# OpenSpec Change: enterprise-gaming-r3f-audio
|
||
|
||
- **Change ID**: enterprise-gaming-r3f-audio
|
||
- **Package**: `@suaveplan/r3f-audio`
|
||
- **Category**: gaming
|
||
- **Target path**: `packages/gaming/r3f-audio`
|
||
- **Status**: PROPOSED
|
||
- **Date**: 2026-05-15
|
||
- **Design**: `./design.md`
|
||
- **Spec delta**: `./specs/r3f-audio/spec.md`
|
||
|
||
> Replaces the prior 2026-04-24 draft that was flagged `NEEDS-REWRITE`. See `../REACT_AUDIO_WAVE_REWRITE_COMPLETE.md`.
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
Deliver `@suaveplan/r3f-audio` on genesis from scratch at enterprise quality. The package is the React-Three-Fiber surface for the unified audio architecture: it composes the four game-audio building blocks shipped by `@suaveplan/audio-plugins-fx` (`SpatialPositionalPlugin`, `ProximityVoicePlugin`, `MusicLayerMixerPlugin`, `VoiceTransformPlugin`) into R3F components that bridge the Three.js scene graph to the universal `AmbientState` system:
|
||
|
||
1. **`<AudioListener>`** — attaches to a Three.js camera (or any `Object3D`) and pushes its world-space position / orientation / velocity into the active `PluginChain`'s `AmbientState.listener` every frame, so every plugin in every chain reads consistent listener state.
|
||
2. **`<PositionalAudio>`** — wraps a `SpatialPositionalPlugin` instance and syncs its `position` parameter to a Three.js `Object3D`'s world-space transform; HRTF on browsers that support it, amplitude panning + distance attenuation universally.
|
||
3. **`<ProximityVoiceChat>`** — wraps a `ProximityVoicePlugin` instance; routes a participant's voice stream through it with distance-based attenuation against the active listener.
|
||
4. **`<MusicLayerMixer>`** — wraps a `MusicLayerMixerPlugin`; binds the layer-weight map to a React-controlled `gameState` prop (e.g. `{ tension, exploration, combat, stealth }`) that updates `AmbientState.extension.gameState` on every change with smooth crossfade.
|
||
5. **`<VoiceTransform>`** — wraps a `VoiceTransformPlugin` for character voice morphing on NPC dialogue lines.
|
||
6. **`<OcclusionProvider>`** — uses `Raycaster` against scene meshes to compute per-source attenuation when geometry sits between source and listener; pushes occlusion factor into the spatial plugin via its `occlusionDb` parameter.
|
||
7. **A typed hook layer** — `useAudioListener()`, `usePositionalAudio(ref)`, `useMusicLayerWeights(weights)`, `useProximityVoice(participantId)`, `useOcclusion(source, target)`.
|
||
|
||
The package contains **no DSP, no audio context, no plugin instance state** — every audio surface is delegated to `@suaveplan/audio-plugins-fx`. R3F components own the scene-graph-to-ambient-state bridge and React lifecycle; the plugins own the audio.
|
||
|
||
---
|
||
|
||
## Current-state audit (genesis)
|
||
|
||
The package does not exist on genesis yet — fresh build replacing the obsolete pre-tier 2026-04-24 draft. The prior upstream `packages/gaming/effects/r3f-audio/` is built on raw Web Audio `PannerNode` and `AudioListener`, with no integration into the universal `AmbientState` system or the game-audio building blocks; this rewrite consumes the building blocks instead so all audio behaviours are universal and the R3F surface is purely the scene-graph bridge.
|
||
|
||
---
|
||
|
||
## Parity Checklist (vs upstream)
|
||
|
||
| Capability | Upstream | Migrated | Notes |
|
||
|---|:---:|:---:|---|
|
||
| `AudioListener` component | ✅ | ⬆️ | Now pushes listener pose into `AmbientState.listener` for every chain in the active provider, not just one panner. |
|
||
| `PositionalAudio` component | ✅ | ⬆️ | Now wraps `SpatialPositionalPlugin` (which supports both HRTF and universal amplitude panning). |
|
||
| Pool of `AudioSource` objects | ✅ | ✅ | Internal voice pooling; instance reuse across mount/unmount. |
|
||
| Occlusion estimator | ✅ | ⬆️ | Now writes into the plugin's declared `occlusionDb` parameter via `setParameterCurve` for smooth transitions. |
|
||
| Dynamic music layers | – | ⬆️ | NEW: `<MusicLayerMixer>` over `MusicLayerMixerPlugin` with React-controlled `gameState`. |
|
||
| Proximity voice chat | – | ⬆️ | NEW: `<ProximityVoiceChat>` over `ProximityVoicePlugin`. |
|
||
| Character voice transformation | – | ⬆️ | NEW: `<VoiceTransform>` over `VoiceTransformPlugin`. |
|
||
| OTel active-source gauge | ✅ | ✅ | Preserved from upstream additions. |
|
||
| Structured errors | ⚠ partial | ✅ | `R3F_AUDIO.*` codes via `defineCodes`. |
|
||
| Coverage | ~78 % | ✅ | 100/100/100/100. |
|
||
|
||
No 🚫 rows.
|
||
|
||
---
|
||
|
||
## Available workspace deps on genesis
|
||
|
||
| Package | Role |
|
||
|---|---|
|
||
| `@suaveplan/audio-plugins-fx` (`workspace:^`) | Four game-audio building blocks consumed as plugins |
|
||
| `@suaveplan/react-audio` (`workspace:^`) | `AudioProvider` + hooks for chain / plugin / transport access |
|
||
| `@suaveplan/audio-transport` (`workspace:^`) | Transport reference for tempo-synced music layers |
|
||
| `@suaveplan/error` (`workspace:^`) | `defineCodes` + `AudioError` |
|
||
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Vec3`, `Quat` |
|
||
| `@suaveplan/opentelemetry` (`workspace:^`) | OTel gauges |
|
||
| `react` (`peerDependency`) | React 19 peer |
|
||
| `@react-three/fiber` (`peerDependency`) | R3F peer |
|
||
| `three` (`peerDependency`) | Three.js peer |
|
||
| `zod` (`catalog:forms`) | Options validation |
|
||
|
||
---
|
||
|
||
## File layout
|
||
|
||
```text
|
||
packages/gaming/r3f-audio/
|
||
src/
|
||
index.ts
|
||
error-codes.ts (+ .test.ts .md)
|
||
components/
|
||
audio-listener/ (.tsx .md .types.ts .test.tsx .stories.tsx)
|
||
positional-audio/ ...
|
||
proximity-voice-chat/ ...
|
||
music-layer-mixer/ ...
|
||
voice-transform/ ...
|
||
occlusion-provider/ ...
|
||
hooks/
|
||
use-audio-listener/ (.ts .md .test.tsx)
|
||
use-positional-audio/ ...
|
||
use-music-layer-weights/ ...
|
||
use-proximity-voice/ ...
|
||
use-occlusion/ ...
|
||
util/
|
||
pose-to-ambient/ (.ts .md .types.ts .test.ts)
|
||
raycaster-pool/ ...
|
||
__benchmarks__/
|
||
pose-sync.bench.ts
|
||
occlusion.bench.ts
|
||
tests/
|
||
integration.test.tsx
|
||
purity.test.ts
|
||
exports.test.ts
|
||
e2e/
|
||
spatial.spec.ts
|
||
```
|
||
|
||
---
|
||
|
||
## Impact
|
||
|
||
**Breaking changes**: none — fresh build.
|
||
|
||
**Prerequisites**: `audio-plugins-fx`, `react-audio`, `audio-transport`, `error`, `types`, `opentelemetry`.
|
||
|
||
### Affected dependents
|
||
|
||
No dependents in this batch (verified by `bunx turbo ls --filter=...@suaveplan/r3f-audio`). Future game packages will consume this surface and SHALL run its tests on change per `openspec/AGENTS.md` §13 step 10.
|