Initial Commit
This commit is contained in:
151
openspec/changes/enterprise-gaming-r3f-audio/design.md
Normal file
151
openspec/changes/enterprise-gaming-r3f-audio/design.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Design — enterprise-gaming-r3f-audio
|
||||
|
||||
`@suaveplan/r3f-audio` is a **bridge package**. Its only job is to translate Three.js scene-graph state (camera pose, object pose, raycaster results) into the universal `AmbientState` and into parameter writes on game-audio plugins. All audio behaviour — distance attenuation, HRTF panning, music-layer crossfading, voice transformation, sidechain ducking — lives in `@suaveplan/audio-plugins-fx` and runs identically without R3F (in tests, in headless renders, in non-R3F games). R3F is the **input source**, not the audio.
|
||||
|
||||
Cross-cutting audio architecture: see `openspec/changes/enterprise-audio-plugins-fx/design.md`. This document covers the R3F-specific bridge.
|
||||
|
||||
---
|
||||
|
||||
## 1. Position in the audio family
|
||||
|
||||
```
|
||||
R3F game scene
|
||||
│
|
||||
▼
|
||||
@suaveplan/r3f-audio ← this package
|
||||
(scene-graph → AmbientState bridge,
|
||||
ref-to-plugin parameter wiring)
|
||||
│
|
||||
▼
|
||||
@suaveplan/audio-plugins-fx
|
||||
spatial-positional + proximity-voice
|
||||
+ music-layer-mixer + voice-transform
|
||||
│
|
||||
▼
|
||||
@suaveplan/audio-engine-browser
|
||||
(rendering)
|
||||
```
|
||||
|
||||
The package contains **no `BaseAudioContext` reference**. It does not construct, own, or dispose audio objects. Every audio behaviour is delegated to plugins; this package only writes the plugins' inputs every frame.
|
||||
|
||||
---
|
||||
|
||||
## 2. The scene-graph-to-ambient bridge
|
||||
|
||||
Three.js gives us a transform per `Object3D` updated every render frame. The universal `AmbientState.listener` carries `{ position, orientation, velocity }`. The bridge is:
|
||||
|
||||
```ts
|
||||
function useAudioListener(objectRef: RefObject<Object3D>) {
|
||||
const { engine } = useAudioEngine()
|
||||
useFrame(() => {
|
||||
const obj = objectRef.current
|
||||
if (!obj) return
|
||||
const pose: ListenerState = {
|
||||
position: vec3FromThree(obj.getWorldPosition(scratchVec)),
|
||||
orientation: quatFromThree(obj.getWorldQuaternion(scratchQuat)),
|
||||
velocity: computeVelocity(obj, scratchVec, scratchPrev), // discrete diff per frame
|
||||
}
|
||||
engine.updateAmbient({ listener: pose })
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
- `useFrame` runs once per R3F render tick (typically 60 Hz).
|
||||
- The bridge calls `engine.updateAmbient({ listener })`, which the engine forwards to every active chain. Every chain emits a single `ambient-changed` event; plugins read the new listener on their next `process()`.
|
||||
- Scratch vectors and quaternions are allocated once at hook construction (no per-frame allocation).
|
||||
|
||||
`<AudioListener>` is a thin component that mounts `useAudioListener` against `useThree().camera` by default. Users can opt into a different `Object3D` (e.g. first-person controller) by passing `target`.
|
||||
|
||||
---
|
||||
|
||||
## 3. `<PositionalAudio>` — plugin per source
|
||||
|
||||
Each `<PositionalAudio>` mounts one `SpatialPositionalPlugin` instance via the provider's `PluginRegistry`, inserts it into a designated `PluginChain` (default: a project-level "spatial sources" chain owned by the provider), and writes its `position`/`orientation` parameters from a Three.js `Object3D` ref every frame:
|
||||
|
||||
```tsx
|
||||
function PositionalAudio({ src, position, audio, options }: PositionalAudioProps) {
|
||||
const ref = useRef<Group>(null)
|
||||
const plugin = useSpatialPositionalPlugin(audio, options)
|
||||
useFrame(() => {
|
||||
const obj = ref.current
|
||||
if (!obj || !plugin) return
|
||||
plugin.setParameter('positionX', obj.position.x)
|
||||
plugin.setParameter('positionY', obj.position.y)
|
||||
plugin.setParameter('positionZ', obj.position.z)
|
||||
})
|
||||
return <group ref={ref} position={position} />
|
||||
}
|
||||
```
|
||||
|
||||
Distance attenuation, HRTF panning, occlusion compensation, and (browser tier) the `PannerNode` wiring all happen inside the plugin. The component owns only the pose write loop.
|
||||
|
||||
---
|
||||
|
||||
## 4. Occlusion
|
||||
|
||||
`<OcclusionProvider>` exposes a per-source occlusion factor by raycasting from listener to source against a configurable list of occluder meshes:
|
||||
|
||||
```ts
|
||||
const hits = raycaster.intersectObjects(occluders, true)
|
||||
const occlusion = hits.length > 0 ? sumAttenuation(hits) : 0 // dB
|
||||
plugin.setParameterCurve('occlusionDb', { type: 'linear', value: occlusion, duration: 0.05 })
|
||||
```
|
||||
|
||||
The 50 ms `setParameterCurve` smooths discrete frame-to-frame occlusion changes into an audible curve — without it, walking past a wall produces audible stepping. The raycaster pool reuses one `Raycaster` instance across all positional-audio sources to avoid per-frame allocation.
|
||||
|
||||
---
|
||||
|
||||
## 5. `<MusicLayerMixer>` — game-state to layer weights
|
||||
|
||||
`MusicLayerMixerPlugin` reads `AmbientState.extension.gameState`. The R3F component binds a React-controlled prop to that ambient slot:
|
||||
|
||||
```tsx
|
||||
function MusicLayerMixer({ layers, gameState }: MusicLayerMixerProps) {
|
||||
const { engine } = useAudioEngine()
|
||||
useEffect(() => {
|
||||
engine.updateAmbient({ extension: { gameState } })
|
||||
}, [engine, gameState])
|
||||
// mounts/loads each layer once via plugin.loadLayer
|
||||
}
|
||||
```
|
||||
|
||||
A consumer renders `<MusicLayerMixer gameState={{ tension: 0.8, exploration: 0.1, combat: 0.6 }} />` and the plugin's crossfade lands on the next process block. The crossfade duration is set on the plugin (`transitionMs`), not the component.
|
||||
|
||||
---
|
||||
|
||||
## 6. `<ProximityVoiceChat>` and `<VoiceTransform>`
|
||||
|
||||
Voice plugins wrap inputs from a `MediaStream` (peer audio over WebRTC, captured mic, etc.). The R3F components handle:
|
||||
|
||||
- attaching the stream as a source to the plugin (via the plugin's `setSourceStream` API),
|
||||
- writing the participant's world-space position to the plugin's `voicePosition` parameter every frame (proximity voice),
|
||||
- writing transform parameters from React props (voice transform).
|
||||
|
||||
The streaming itself is owned by upstream packages (`react-audio-stream` for WebRTC, browser `getUserMedia` for local capture).
|
||||
|
||||
---
|
||||
|
||||
## 7. Voice pooling and lifecycle
|
||||
|
||||
`<PositionalAudio>` mounts and unmounts as the React tree changes. To avoid plugin churn (constructor + destructor per dialogue line is expensive), the provider exposes a **voice pool** for short-lived sources:
|
||||
|
||||
```ts
|
||||
const handle = pool.acquire('spatial-positional') // gets a pooled plugin instance
|
||||
playSound(handle, buffer)
|
||||
pool.release(handle) // returns to pool, resets state
|
||||
```
|
||||
|
||||
Long-lived sources (continuous emitters, music layers) bypass the pool and mount their plugin for the component's lifetime.
|
||||
|
||||
---
|
||||
|
||||
## 8. Decisions and trade-offs
|
||||
|
||||
- **No new audio surface in this package.** Everything is in `audio-plugins-fx`. R3F is the input source.
|
||||
- **One bridge per concept**: listener → ambient, positional source → plugin parameter writes, music layers → ambient extension, occlusion → parameter curve.
|
||||
- **Scene-graph reads via `useFrame`**, never via React state. React renders are too slow for 60 Hz pose updates.
|
||||
- **All vector/quaternion math uses scratch allocations** captured at hook construction. Per-frame allocation is the easiest way to tank a 60 fps game.
|
||||
- **Voice pool for short-lived sources, dedicated mount for long-lived ones.** Construction cost is real for FFT-based plugins.
|
||||
- **Occlusion is smoothed via `setParameterCurve`, not raw writes.** Discrete frame-to-frame stepping is audible; 50 ms ramps are not.
|
||||
- **No coupling to a specific game state shape**: `gameState` is a generic `Record<string, number>` written into `AmbientState.extension.gameState`. The plugin reads what it knows; unknown keys are ignored.
|
||||
- **R3F is a peer dep, not a runtime dep.** The package never imports R3F transitively — it only imports types — so non-R3F games that consume the same building blocks pay zero R3F cost.
|
||||
118
openspec/changes/enterprise-gaming-r3f-audio/proposal.md
Normal file
118
openspec/changes/enterprise-gaming-r3f-audio/proposal.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# OpenSpec Change: enterprise-gaming-r3f-audio
|
||||
|
||||
- **Change ID**: enterprise-gaming-r3f-audio
|
||||
- **Package**: `@suaveplan/r3f-audio`
|
||||
- **Category**: gaming
|
||||
- **Target path**: `packages/gaming/r3f-audio`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-15
|
||||
- **Design**: `./design.md`
|
||||
- **Spec delta**: `./specs/r3f-audio/spec.md`
|
||||
|
||||
> Replaces the prior 2026-04-24 draft that was flagged `NEEDS-REWRITE`. See `../REACT_AUDIO_WAVE_REWRITE_COMPLETE.md`.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Deliver `@suaveplan/r3f-audio` on genesis from scratch at enterprise quality. The package is the React-Three-Fiber surface for the unified audio architecture: it composes the four game-audio building blocks shipped by `@suaveplan/audio-plugins-fx` (`SpatialPositionalPlugin`, `ProximityVoicePlugin`, `MusicLayerMixerPlugin`, `VoiceTransformPlugin`) into R3F components that bridge the Three.js scene graph to the universal `AmbientState` system:
|
||||
|
||||
1. **`<AudioListener>`** — attaches to a Three.js camera (or any `Object3D`) and pushes its world-space position / orientation / velocity into the active `PluginChain`'s `AmbientState.listener` every frame, so every plugin in every chain reads consistent listener state.
|
||||
2. **`<PositionalAudio>`** — wraps a `SpatialPositionalPlugin` instance and syncs its `position` parameter to a Three.js `Object3D`'s world-space transform; HRTF on browsers that support it, amplitude panning + distance attenuation universally.
|
||||
3. **`<ProximityVoiceChat>`** — wraps a `ProximityVoicePlugin` instance; routes a participant's voice stream through it with distance-based attenuation against the active listener.
|
||||
4. **`<MusicLayerMixer>`** — wraps a `MusicLayerMixerPlugin`; binds the layer-weight map to a React-controlled `gameState` prop (e.g. `{ tension, exploration, combat, stealth }`) that updates `AmbientState.extension.gameState` on every change with smooth crossfade.
|
||||
5. **`<VoiceTransform>`** — wraps a `VoiceTransformPlugin` for character voice morphing on NPC dialogue lines.
|
||||
6. **`<OcclusionProvider>`** — uses `Raycaster` against scene meshes to compute per-source attenuation when geometry sits between source and listener; pushes occlusion factor into the spatial plugin via its `occlusionDb` parameter.
|
||||
7. **A typed hook layer** — `useAudioListener()`, `usePositionalAudio(ref)`, `useMusicLayerWeights(weights)`, `useProximityVoice(participantId)`, `useOcclusion(source, target)`.
|
||||
|
||||
The package contains **no DSP, no audio context, no plugin instance state** — every audio surface is delegated to `@suaveplan/audio-plugins-fx`. R3F components own the scene-graph-to-ambient-state bridge and React lifecycle; the plugins own the audio.
|
||||
|
||||
---
|
||||
|
||||
## Current-state audit (genesis)
|
||||
|
||||
The package does not exist on genesis yet — fresh build replacing the obsolete pre-tier 2026-04-24 draft. The prior upstream `packages/gaming/effects/r3f-audio/` is built on raw Web Audio `PannerNode` and `AudioListener`, with no integration into the universal `AmbientState` system or the game-audio building blocks; this rewrite consumes the building blocks instead so all audio behaviours are universal and the R3F surface is purely the scene-graph bridge.
|
||||
|
||||
---
|
||||
|
||||
## Parity Checklist (vs upstream)
|
||||
|
||||
| Capability | Upstream | Migrated | Notes |
|
||||
|---|:---:|:---:|---|
|
||||
| `AudioListener` component | ✅ | ⬆️ | Now pushes listener pose into `AmbientState.listener` for every chain in the active provider, not just one panner. |
|
||||
| `PositionalAudio` component | ✅ | ⬆️ | Now wraps `SpatialPositionalPlugin` (which supports both HRTF and universal amplitude panning). |
|
||||
| Pool of `AudioSource` objects | ✅ | ✅ | Internal voice pooling; instance reuse across mount/unmount. |
|
||||
| Occlusion estimator | ✅ | ⬆️ | Now writes into the plugin's declared `occlusionDb` parameter via `setParameterCurve` for smooth transitions. |
|
||||
| Dynamic music layers | – | ⬆️ | NEW: `<MusicLayerMixer>` over `MusicLayerMixerPlugin` with React-controlled `gameState`. |
|
||||
| Proximity voice chat | – | ⬆️ | NEW: `<ProximityVoiceChat>` over `ProximityVoicePlugin`. |
|
||||
| Character voice transformation | – | ⬆️ | NEW: `<VoiceTransform>` over `VoiceTransformPlugin`. |
|
||||
| OTel active-source gauge | ✅ | ✅ | Preserved from upstream additions. |
|
||||
| Structured errors | ⚠ partial | ✅ | `R3F_AUDIO.*` codes via `defineCodes`. |
|
||||
| Coverage | ~78 % | ✅ | 100/100/100/100. |
|
||||
|
||||
No 🚫 rows.
|
||||
|
||||
---
|
||||
|
||||
## Available workspace deps on genesis
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@suaveplan/audio-plugins-fx` (`workspace:^`) | Four game-audio building blocks consumed as plugins |
|
||||
| `@suaveplan/react-audio` (`workspace:^`) | `AudioProvider` + hooks for chain / plugin / transport access |
|
||||
| `@suaveplan/audio-transport` (`workspace:^`) | Transport reference for tempo-synced music layers |
|
||||
| `@suaveplan/error` (`workspace:^`) | `defineCodes` + `AudioError` |
|
||||
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Vec3`, `Quat` |
|
||||
| `@suaveplan/opentelemetry` (`workspace:^`) | OTel gauges |
|
||||
| `react` (`peerDependency`) | React 19 peer |
|
||||
| `@react-three/fiber` (`peerDependency`) | R3F peer |
|
||||
| `three` (`peerDependency`) | Three.js peer |
|
||||
| `zod` (`catalog:forms`) | Options validation |
|
||||
|
||||
---
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/gaming/r3f-audio/
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts (+ .test.ts .md)
|
||||
components/
|
||||
audio-listener/ (.tsx .md .types.ts .test.tsx .stories.tsx)
|
||||
positional-audio/ ...
|
||||
proximity-voice-chat/ ...
|
||||
music-layer-mixer/ ...
|
||||
voice-transform/ ...
|
||||
occlusion-provider/ ...
|
||||
hooks/
|
||||
use-audio-listener/ (.ts .md .test.tsx)
|
||||
use-positional-audio/ ...
|
||||
use-music-layer-weights/ ...
|
||||
use-proximity-voice/ ...
|
||||
use-occlusion/ ...
|
||||
util/
|
||||
pose-to-ambient/ (.ts .md .types.ts .test.ts)
|
||||
raycaster-pool/ ...
|
||||
__benchmarks__/
|
||||
pose-sync.bench.ts
|
||||
occlusion.bench.ts
|
||||
tests/
|
||||
integration.test.tsx
|
||||
purity.test.ts
|
||||
exports.test.ts
|
||||
e2e/
|
||||
spatial.spec.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
**Breaking changes**: none — fresh build.
|
||||
|
||||
**Prerequisites**: `audio-plugins-fx`, `react-audio`, `audio-transport`, `error`, `types`, `opentelemetry`.
|
||||
|
||||
### Affected dependents
|
||||
|
||||
No dependents in this batch (verified by `bunx turbo ls --filter=...@suaveplan/r3f-audio`). Future game packages will consume this surface and SHALL run its tests on change per `openspec/AGENTS.md` §13 step 10.
|
||||
@@ -0,0 +1,147 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: r3f-audio (enterprise-gaming-r3f-audio)
|
||||
|
||||
**Package**: `@suaveplan/r3f-audio`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Bridge-tier purity — no audio context, no DSP
|
||||
|
||||
`@suaveplan/r3f-audio` SHALL import zero browser audio APIs (`AudioContext`, `OfflineAudioContext`, `AudioNode`, `AudioWorklet`) and zero Node-only APIs. Every audio behaviour SHALL be delegated to plugins from `@suaveplan/audio-plugins-fx`; the package SHALL be a pure scene-graph-to-`AmbientState` bridge plus thin component wrappers around those plugins. The package's runtime `dependencies` SHALL be limited to `@suaveplan/audio-plugins-fx`, `@suaveplan/react-audio`, `@suaveplan/audio-transport`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`, `zod`, and `tslib`. `react`, `@react-three/fiber`, and `three` SHALL be peer dependencies only.
|
||||
|
||||
#### Scenario: built dist references no audio context
|
||||
|
||||
- **WHEN** the built `dist/` is grep'd
|
||||
- **THEN** it MUST NOT reference `AudioContext`, `OfflineAudioContext`, `AudioNode`, or `AudioWorklet`
|
||||
|
||||
**Verified by:** `tests/purity.test.ts::"dist references no audio context"` AND `tests/purity.test.ts::"runtime dependency set matches the allowlist"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AudioListener pushes camera pose into AmbientState every frame
|
||||
|
||||
`<AudioListener target?>` SHALL on every R3F render tick (via `useFrame`) read the world-space position, orientation, and velocity (computed as a per-frame discrete diff) of the target `Object3D` (default: the active R3F camera) and call `engine.updateAmbient({ listener: { position, orientation, velocity } })`. Vector and quaternion math SHALL use scratch instances captured at mount; the per-frame loop SHALL allocate zero new objects. The component SHALL be safe to nest under multiple `AudioProvider`s (each gets its own listener bridge).
|
||||
|
||||
#### Scenario: per-frame loop allocates no new objects
|
||||
|
||||
- **WHEN** `<AudioListener>` runs for 600 frames against an instrumented allocator
|
||||
- **THEN** the count of vector / quaternion allocations attributable to the listener bridge equals zero after the initial mount
|
||||
|
||||
#### Scenario: listener pose propagates to every chain
|
||||
|
||||
- **WHEN** the camera moves and the next R3F frame ticks
|
||||
- **THEN** every active `PluginChain`'s `ChainContext.ambient.listener.position` equals the camera's world-space position on the next processed audio block
|
||||
|
||||
**Verified by:** `src/components/audio-listener/audio-listener.test.tsx::"per-frame loop allocates no new objects"` AND `tests/integration.test.tsx::"camera move propagates to every chain ambient.listener"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: PositionalAudio wraps SpatialPositionalPlugin
|
||||
|
||||
`<PositionalAudio src position rolloff? maxDistance?>` SHALL on mount acquire a `SpatialPositionalPlugin` instance from the provider's voice pool (or instantiate via `PluginRegistry.create('spatial-positional')` if pooling is disabled), insert it into the configured `PluginChain`, and load the audio source. On every R3F render tick the component SHALL write the wrapped `Object3D`'s world-space position into the plugin's `positionX`/`positionY`/`positionZ` parameters. On unmount the plugin instance SHALL be released back to the pool (state reset) or disposed if pooling is disabled.
|
||||
|
||||
#### Scenario: position write reaches the plugin on the next frame
|
||||
|
||||
- **WHEN** the `Object3D` is moved to `(10, 0, 0)` and the next frame ticks
|
||||
- **THEN** `plugin.getParameter('positionX') === 10` (within float tolerance)
|
||||
|
||||
#### Scenario: unmount returns the plugin to the pool
|
||||
|
||||
- **WHEN** `<PositionalAudio>` unmounts under a provider with pooling enabled
|
||||
- **THEN** the plugin instance is returned to the pool with its state reset, and re-mounting the same `src` reuses the pooled instance rather than constructing a fresh one
|
||||
|
||||
**Verified by:** `src/components/positional-audio/positional-audio.test.tsx::"position write reaches the plugin on the next frame"` AND `src/components/positional-audio/positional-audio.test.tsx::"unmount returns the plugin to the pool"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OcclusionProvider smooths raycast results into a parameter curve
|
||||
|
||||
`<OcclusionProvider occluders raycastRate? smoothingMs?>` SHALL register a per-frame (rate-limited to `raycastRate`, default 30 Hz) `Raycaster` walk from each active `<PositionalAudio>`'s position to the listener position, sum attenuation across hits, and write the resulting dB value via `plugin.setParameterCurve('occlusionDb', { type: 'linear', value, duration: smoothingMs / 1000 })`. A single `Raycaster` instance SHALL be reused across all sources via a pool (no per-source allocation). The component SHALL handle the listener-or-source-absent case by writing `occlusionDb = 0` for stale sources.
|
||||
|
||||
#### Scenario: smooths occlusion change into a curve
|
||||
|
||||
- **WHEN** an occluder is moved between listener and source between two frames
|
||||
- **THEN** the spatial plugin observes `setParameterCurve('occlusionDb', { duration: smoothingMs / 1000 })` rather than `setParameter`
|
||||
|
||||
#### Scenario: raycaster is pooled across sources
|
||||
|
||||
- **WHEN** 20 `<PositionalAudio>` sources are active under one `<OcclusionProvider>`
|
||||
- **THEN** exactly one `Raycaster` instance is constructed and reused for every source's per-frame walk
|
||||
|
||||
**Verified by:** `src/components/occlusion-provider/occlusion-provider.test.tsx::"smooths occlusion change into a parameter curve"` AND `src/components/occlusion-provider/occlusion-provider.test.tsx::"raycaster is pooled across sources"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MusicLayerMixer binds gameState to ambient extension
|
||||
|
||||
`<MusicLayerMixer layers gameState transitionMs?>` SHALL on mount acquire a `MusicLayerMixerPlugin` instance, call `plugin.loadLayer(id, buffer)` for each entry in `layers`, and insert it as the chain's source plugin (index 0). When the `gameState` prop changes between renders, the component SHALL call `engine.updateAmbient({ extension: { gameState: nextGameState } })`. The crossfade duration SHALL be the plugin's configured `transitionMs` (set once at mount via the `transitionMs` prop); per-render-cycle prop changes do not need to re-configure the plugin.
|
||||
|
||||
#### Scenario: gameState change writes to ambient extension
|
||||
|
||||
- **WHEN** the consumer changes `gameState` from `{ tension: 0 }` to `{ tension: 1 }`
|
||||
- **THEN** `engine.updateAmbient` is called once with `{ extension: { gameState: { tension: 1 } } }`
|
||||
|
||||
**Verified by:** `src/components/music-layer-mixer/music-layer-mixer.test.tsx::"gameState prop change writes the new value to ambient extension"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: ProximityVoiceChat and VoiceTransform thin wrappers
|
||||
|
||||
`<ProximityVoiceChat participantId stream position>` SHALL wrap a `ProximityVoicePlugin` instance: attach `stream` as its source, write `participant`'s world-space position into the plugin's `voicePosition` parameter every frame, and (on unmount) detach the stream and release the plugin to the pool. `<VoiceTransform character preset stream>` SHALL wrap a `VoiceTransformPlugin` instance: attach `stream`, apply the requested preset on mount and on `preset` prop change, and release on unmount.
|
||||
|
||||
#### Scenario: ProximityVoiceChat attenuates with distance
|
||||
|
||||
- **WHEN** a `<ProximityVoiceChat>` participant moves from 1 m to 50 m from the listener with the proximity plugin's `rolloffFactor = 1`
|
||||
- **THEN** the plugin's output level decreases monotonically per the rolloff model
|
||||
|
||||
#### Scenario: VoiceTransform applies preset changes
|
||||
|
||||
- **WHEN** the consumer changes the `preset` prop from `'robot'` to `'monster'`
|
||||
- **THEN** the plugin's `applyPreset('monster')` is called once and the output reflects the new transform on the next block
|
||||
|
||||
**Verified by:** `src/components/proximity-voice-chat/proximity-voice-chat.test.tsx::"output level decreases monotonically with distance"` AND `src/components/voice-transform/voice-transform.test.tsx::"preset prop change applies a new preset"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry active-source gauge
|
||||
|
||||
The package SHALL register an `@suaveplan/opentelemetry` gauge `r3f_audio_active_sources` reporting the count of currently mounted `<PositionalAudio>` + `<ProximityVoiceChat>` + `<VoiceTransform>` instances per provider. The gauge SHALL be inert when no exporter is registered.
|
||||
|
||||
#### Scenario: gauge reflects mount and unmount
|
||||
|
||||
- **WHEN** four `<PositionalAudio>` components mount, then two unmount
|
||||
- **THEN** the gauge reads 4 after the mounts and 2 after the unmounts
|
||||
|
||||
**Verified by:** `src/components/positional-audio/positional-audio.test.tsx::"OTel active-source gauge tracks mount and unmount"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Structured error codes
|
||||
|
||||
The package SHALL declare its codes via `defineCodes("R3F_AUDIO", [...])` from `@suaveplan/error`, producing the dotted codes `R3F_AUDIO.NO_PROVIDER`, `R3F_AUDIO.NO_AUDIO_LISTENER`, `R3F_AUDIO.PLUGIN_POOL_EXHAUSTED`, and `R3F_AUDIO.INVALID_OPTIONS`. The package SHALL export `R3fAudioErrorCode`, `R3F_AUDIO_ERROR_METADATA`, `registerR3fAudioErrorCodes()`, and an `R3fAudioError` class extending `AudioError` from `@suaveplan/audio-core`. Every error thrown SHALL carry one of these codes; every registered code SHALL be reachable from a real call site.
|
||||
|
||||
#### Scenario: every registered code is reachable
|
||||
|
||||
- **WHEN** the test suite runs
|
||||
- **THEN** each code in `R3F_AUDIO_ERROR_METADATA` is asserted thrown by at least one test
|
||||
|
||||
**Verified by:** `src/error-codes.test.ts::"every R3F_AUDIO code is reachable from a call site"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath exports per surface
|
||||
|
||||
`package.json` SHALL declare one `exports` entry per surface: `.`, `./components`, `./components/<name>`, `./hooks`, `./hooks/<name>`, `./util`. `sideEffects` SHALL be `false`. Importing one component SHALL NOT pull in others.
|
||||
|
||||
#### Scenario: importing one component does not pull in others
|
||||
|
||||
- **WHEN** a consumer imports `@suaveplan/r3f-audio/components/audio-listener` only
|
||||
- **THEN** the tree-shaken bundle contains zero bytes from `src/components/positional-audio/`
|
||||
|
||||
**Verified by:** `tests/exports.test.ts::"every surface has a subpath export entry"` AND `tests/bundle-size/listener-only.size.test.ts::"importing audio-listener does not bundle other components"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
77
openspec/changes/enterprise-gaming-r3f-audio/tasks.md
Normal file
77
openspec/changes/enterprise-gaming-r3f-audio/tasks.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Tasks: enterprise-gaming-r3f-audio
|
||||
|
||||
**Change ID**: enterprise-gaming-r3f-audio
|
||||
**Package**: `@suaveplan/r3f-audio`
|
||||
**Target path**: `packages/gaming/r3f-audio`
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/audio-plugins-fx`, `@suaveplan/react-audio`, `@suaveplan/audio-transport`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`
|
||||
|
||||
---
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [ ] 1.1 Create directory tree per `proposal.md` file layout.
|
||||
- [ ] 1.2 Write `package.json` at `0.1.0`, `"sideEffects": false`, full `exports` map (`.`, `./components`, `./components/<name>`, `./hooks`, `./hooks/<name>`, `./util`). React + R3F + three as peer deps.
|
||||
- [ ] 1.3 Write `tsconfig.json`, `vitest.config.ts` (`happy-dom`).
|
||||
|
||||
## 2. Error codes
|
||||
|
||||
- [ ] 2.1 `src/error-codes.ts` — `defineCodes("R3F_AUDIO", [...])`, `R3fAudioError extends AudioError`.
|
||||
- [ ] 2.2 `error-codes.test.ts`.
|
||||
|
||||
## 3. Util
|
||||
|
||||
- [ ] 3.1 `src/util/pose-to-ambient/` — `Vec3`/`Quat` conversions, scratch-vector helpers; no allocation per call.
|
||||
- [ ] 3.2 `src/util/raycaster-pool/` — shared raycaster instance + per-source scoreboard.
|
||||
|
||||
## 4. Hooks
|
||||
|
||||
- [ ] 4.1 `use-audio-listener` — binds an `Object3D` ref to `engine.updateAmbient({ listener })` per frame.
|
||||
- [ ] 4.2 `use-positional-audio` — acquires `SpatialPositionalPlugin`, writes pose per frame, releases on unmount.
|
||||
- [ ] 4.3 `use-music-layer-weights` — writes `gameState` into `AmbientState.extension.gameState`.
|
||||
- [ ] 4.4 `use-proximity-voice` — acquires `ProximityVoicePlugin`, attaches stream, writes position per frame.
|
||||
- [ ] 4.5 `use-occlusion` — runs the raycaster pool against occluders, writes smoothed `occlusionDb`.
|
||||
|
||||
## 5. Components (each: `.tsx`, `.md`, `.types.ts`, `.test.tsx`, `.stories.tsx`)
|
||||
|
||||
- [ ] 5.1 `audio-listener/` — default target = active camera.
|
||||
- [ ] 5.2 `positional-audio/`.
|
||||
- [ ] 5.3 `proximity-voice-chat/`.
|
||||
- [ ] 5.4 `music-layer-mixer/`.
|
||||
- [ ] 5.5 `voice-transform/`.
|
||||
- [ ] 5.6 `occlusion-provider/`.
|
||||
|
||||
## 6. OTel
|
||||
|
||||
- [ ] 6.1 Register `r3f_audio_active_sources` gauge; wire mount/unmount to its `add`/`subtract`.
|
||||
|
||||
## 7. Benchmarks
|
||||
|
||||
- [ ] 7.1 `src/__benchmarks__/pose-sync.bench.ts` — 60 fps pose write loop over 1024 sources.
|
||||
- [ ] 7.2 `src/__benchmarks__/occlusion.bench.ts` — raycast pool over 256 sources against a 5 k-tri scene.
|
||||
|
||||
## 8. Integration + purity + bundle-size
|
||||
|
||||
- [ ] 8.1 `tests/integration.test.tsx` — full R3F scene + AudioProvider + listener + 4 positional sources + occlusion + music layer mixer; verify ambient state writes land on chain.
|
||||
- [ ] 8.2 `tests/purity.test.ts` — built `dist/` references no audio context.
|
||||
- [ ] 8.3 `tests/exports.test.ts` — every surface has subpath export; no `export *` in feature barrels.
|
||||
- [ ] 8.4 `tests/bundle-size/listener-only.size.test.ts`.
|
||||
|
||||
## 9. Docs + verify
|
||||
|
||||
- [ ] 9.1 All co-located `.md` files (≥200 words).
|
||||
- [ ] 9.2 `README.md` at package root.
|
||||
- [ ] 9.3 `bunx turbo lint typecheck test --filter=@suaveplan/r3f-audio` green.
|
||||
- [ ] 9.4 `bunx turbo test:coverage --filter=@suaveplan/r3f-audio` 100/100/100/100.
|
||||
- [ ] 9.5 Banned-pattern + runner-import + file-size + missing-`.md` sweeps empty.
|
||||
|
||||
## 10. Spec reconciliation + archive
|
||||
|
||||
- [ ] 10.1 Walk every `### Requirement` and `**Verified by:**`.
|
||||
- [ ] 10.2 Fold spec delta into `openspec/specs/r3f-audio/spec.md`.
|
||||
- [ ] 10.3 Move change folder to archive.
|
||||
Reference in New Issue
Block a user