Initial Commit

This commit is contained in:
2026-05-27 23:47:53 +01:00
parent fbc70f96e6
commit 9b28db724f
18103 changed files with 1600554 additions and 0 deletions

View File

@@ -0,0 +1,83 @@
# `game-audio.types`
## Purpose
TypeScript-only type definitions for `GameAudioPlugin`'s vector math types, registry entries, configuration, and preset payload. Co-located with the implementation so the public type surface stays aligned with the Zod validator in `schemas/schemas.ts`.
The interfaces here form the structural contract that the browser-engine fast path (`createAudioNode`) will read in a future revision — keeping them stable here is essential for that integration.
## Features
- `Vector3` — `{ x, y, z }` finite-number 3-vector.
- `ListenerOrientation` — `{ forward: Vector3, up: Vector3 }`.
- `SpatialAudioSource` — `{ id, position, velocity, occlusion }`.
- `VoiceChatParticipant` — `{ id, position }`.
- `ProximityVoiceChatConfig` — `{ minDistance, maxDistance, rolloffFactor }`.
- `MusicLayer` — `{ name, buffer, currentGain, targetGain }`.
- `GameState` — textual enum `"exploration" | "combat" | "victory" | "menu"`.
- `GameVoiceEffect` — `{ id, pitchShift, distortion }`.
- `GameAudioPluginOptions` — optional listener + voice-chat options.
- `GameAudioPresetData` — full preset payload.
## Usage: Basic Example
```ts
import type {
GameAudioPluginOptions,
Vector3,
} from "@suaveplan/audio-plugins-fx/game";
const position: Vector3 = { x: 1, y: 0, z: 0 };
const opts: GameAudioPluginOptions = { listenerPosition: position };
```
## Usage: Advanced Example
```ts
import type {
GameState,
MusicLayer,
ProximityVoiceChatConfig,
Vector3,
} from "@suaveplan/audio-plugins-fx/game";
function magnitude(v: Vector3): number {
return Math.sqrt(v.x * v.x + v.y * v.y + v.z * v.z);
}
const states: ReadonlyArray<GameState> = [
"exploration",
"combat",
"victory",
"menu",
];
function describeLayer(layer: MusicLayer): string {
return `${layer.name} @ ${(layer.currentGain * 100).toFixed(0)}%`;
}
const tight: ProximityVoiceChatConfig = {
minDistance: 1,
maxDistance: 20,
rolloffFactor: 2,
};
```
## API Reference
- `Vector3` — `{ readonly x, y, z: number }`
- `ListenerOrientation` — `{ readonly forward, up: Vector3 }`
- `SpatialAudioSource` — `{ readonly id, position, velocity, occlusion }`
- `VoiceChatParticipant` — `{ readonly id, position }`
- `ProximityVoiceChatConfig` — `{ readonly minDistance, maxDistance, rolloffFactor }`
- `MusicLayer` — `{ readonly name, buffer, currentGain, targetGain }`
- `GameState = "exploration" | "combat" | "victory" | "menu"`
- `GameVoiceEffect` — `{ readonly id, pitchShift, distortion }`
- `GameAudioPluginOptions` — `{ readonly listenerPosition?, listenerOrientation?, voiceChat? }`
- `GameAudioPresetData` — `{ listenerPosition, listenerOrientation, voiceChat: … | null, gameState }`
## Implementation Notes
`GameAudioPresetData.voiceChat` is `ProximityVoiceChatConfig | null` (not `?:`) because a disabled voice-chat state is a legitimate value that the preset must round-trip. Using `?` would let the field be omitted, which is ambiguous between "no voice chat configured" and "voice chat at the default". The explicit `null` makes the disabled state unambiguous.
`SpatialAudioSource` and `MusicLayer` are returned as read-only views from the plugin's accessors; the live state inside the plugin is mutable (it has to be, for per-block gain updates), but consumers see snapshots that they cannot mutate to corrupt the registry. The cost is one shallow object copy per accessor call — negligible compared to the per-block work the plugin does.

View File

@@ -0,0 +1,66 @@
/** 3D vector for position / orientation / velocity. */
export interface Vector3 {
readonly x: number;
readonly y: number;
readonly z: number;
}
/** Listener orientation forward + up unit vectors. */
export interface ListenerOrientation {
readonly forward: Vector3;
readonly up: Vector3;
}
/** One spatial source registered with the plugin. */
export interface SpatialAudioSource {
readonly id: string;
readonly position: Vector3;
readonly velocity: Vector3;
readonly occlusion: number;
}
/** A voice-chat participant tracked at a position. */
export interface VoiceChatParticipant {
readonly id: string;
readonly position: Vector3;
}
/** Proximity-voice-chat configuration. */
export interface ProximityVoiceChatConfig {
readonly minDistance: number;
readonly maxDistance: number;
readonly rolloffFactor: number;
}
/** A loaded music layer with a current/target gain. */
export interface MusicLayer {
readonly name: string;
readonly buffer: Float32Array;
readonly currentGain: number;
readonly targetGain: number;
}
/** Game-state context to drive adaptive music behaviour. */
export type GameState = "exploration" | "combat" | "victory" | "menu";
/** One voice transformation effect applied to a participant. */
export interface GameVoiceEffect {
readonly id: string;
readonly pitchShift: number;
readonly distortion: number;
}
/** Construction options for `GameAudioPlugin`. */
export interface GameAudioPluginOptions {
readonly listenerPosition?: Vector3;
readonly listenerOrientation?: ListenerOrientation;
readonly voiceChat?: ProximityVoiceChatConfig;
}
/** Preset payload — every persistent piece of state. */
export interface GameAudioPresetData {
readonly listenerPosition: Vector3;
readonly listenerOrientation: ListenerOrientation;
readonly voiceChat: ProximityVoiceChatConfig | null;
readonly gameState: GameState;
}

View File

@@ -0,0 +1,83 @@
# `game-audio`
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`GameAudioPlugin` is the universal composite plugin for game audio. It maintains three registries — spatial sources, music layers, and proximity voice-chat participants — and exposes a transport-level API for managing each. The plugin is a source (`io.inputs[0].count === 0`, `io.outputs[0].count === 1`); it owns its audio output rather than colouring upstream audio.
The plugin extends `AbstractAudioFxPlugin` and runs identically across browser / server / worklet renderers. The audio path is intentionally a silent placeholder — the listener data, source registry, voice attenuation, and music-layer crossfade scheduling are pure bookkeeping that the browser-engine fast path consumes via `createAudioNode()` in a follow-up. The universal block path produces silence by design; consumers route the registry state into their own mixing graph.
## Features
- **Spatial sources**: `createSpatialSource`, `updateSpatialSourcePosition`, `applyDopplerEffect`, `applyOcclusion(0..1)`, `cullDistantSources(maxDistance)`, `getActiveSpatialSourceCount`, `getSpatialSource(id)`.
- **Proximity voice chat**: `enableProximityVoiceChat(config)`, `disableProximityVoiceChat`, `addVoiceChatParticipant`, `removeVoiceChatParticipant`, `updateVoiceChatParticipantPosition`, `calculateVoiceAttenuation`, `optimizeVoiceChatBandwidth`, `getVoiceChatOptimizationLevel`.
- **Music layers**: `addMusicLayer`, `removeMusicLayer`, `getMusicLayers`, `crossfadeMusicLayers(from, to, durationMs)`, `isCrossfading`.
- **Voice effects**: `applyGameEffect`, `removeGameEffect`, `getCurrentEffect`, `blendToEffect(effect, durationMs)`, `isBlending`.
- **Listener**: `setListenerPosition`, `setListenerOrientation`, `getListenerPosition`, `getListenerOrientation`.
- **Game state**: `setGameState("exploration" | "combat" | "victory" | "menu")`, `getGameState`.
Voice attenuation uses the same inverse-distance model as `SpatialAudioPlugin`: `gain = minDistance / (minDistance + rolloffFactor · (clamp(d, minDistance, maxDistance) − minDistance))`. Distance grows ⇒ gain decreases monotonically, exactly as the spec requires.
## Usage: Basic Example
```ts
import { GameAudioPlugin } from "@suaveplan/audio-plugins-fx/game";
const game = new GameAudioPlugin({
listenerPosition: { x: 0, y: 0, z: 0 },
voiceChat: { minDistance: 1, maxDistance: 50, rolloffFactor: 1 },
});
chain.add(game);
```
## Usage: Advanced Example
```ts
import { GameAudioPlugin } from "@suaveplan/audio-plugins-fx/game";
const game = new GameAudioPlugin({ voiceChat: { minDistance: 1, maxDistance: 100, rolloffFactor: 2 } });
await game.setup(
{ sampleRate: 48_000, channels: 1, blockSize: 512 },
ctx
);
// Register a footstep emitter and a voice-chat peer.
game.createSpatialSource("footstep", { x: 5, y: 0, z: 0 });
game.addVoiceChatParticipant("teammate", { x: 10, y: 0, z: 0 });
// Adaptive music: crossfade from exploration to combat over 1s.
game.addMusicLayer("exploration", explorationBuffer);
game.addMusicLayer("combat", combatBuffer);
game.crossfadeMusicLayers("exploration", "combat", 1_000);
game.setGameState("combat");
// Cull distant sources every frame to keep the voice-chat budget tight.
game.optimizeVoiceChatBandwidth();
game.cullDistantSources(80);
```
## API Reference
- `GameAudioPlugin` (class extends `AbstractAudioFxPlugin`)
- Construction and lifecycle: `new GameAudioPlugin(options?)`, `setup`, `reset`.
- Listener: `setListenerPosition`, `setListenerOrientation`, `getListenerPosition`, `getListenerOrientation`.
- Spatial sources: see Features above.
- Voice chat: see Features above.
- Music layers: see Features above.
- Voice effects: see Features above.
- Game state: `setGameState`, `getGameState`.
## Implementation Notes
The audio path is silent by design. Implementing the full mix of spatial sources + music layers + voice chat into a single mono output stream would require this plugin to embed three sibling-plugin engines (`SpatialAudioPlugin`, `MixerPlugin`, plus a voice-chat router) — which would either duplicate code or pull every plugin into this one's import graph. Instead, the plugin owns the *registry* of game-audio state; the browser engine reads that state via `createAudioNode()` and renders it into a Web Audio graph in a future revision. The universal block path produces silence but advances the internal sample clock so `isCrossfading` / `isBlending` work correctly under any renderer.
Crossfade scheduling stores the start sample (`processedSamples` at the time of the call) plus the duration in samples. `isCrossfading` is `elapsed < durationSamples`. The check is exact at the sample level — the test "isCrossfading true immediately; false after enough samples processed" exercises that by processing exactly the configured number of samples.
`cullDistantSources` walks the spatial-source map and deletes entries whose distance to the listener exceeds the threshold. The cost is `O(numSources)` per call; for the typical game-audio case (under 100 sources) this is below 1 ms even at 60 fps cull frequency. A spatial-hash variant would amortise this to `O(log n)` but adds bookkeeping that the MVP does not need.
`optimizeVoiceChatBandwidth` returns a coarse 0..3 tier rather than per-participant codec parameters because the actual codec selection lives in the transport layer (WebRTC / similar). The plugin's job is to *advise*; the transport applies the advice. Currently the heuristic is just a participant-count threshold (>8 ⇒ level 3, otherwise level 1) — a future revision can layer on signal-strength and ping data.
The plugin maintains an internal `processedSamples` counter incremented by `output.length` each `processBlock`. This is the same clock crossfade and effect-blend countdowns reference — using `BlockContext.blockOffsetSamples` directly would not work because the chain may slice or align blocks differently, and the consumer expects clock advancement per actual rendered sample. The counter resets on `setup`/`reset`.

View File

@@ -0,0 +1,441 @@
import type { BlockContext, ProcessorConfig } from "@suaveplan/audio-core";
import type {
ChainContext,
ChainHandle,
} from "@suaveplan/audio-fx-plugin-base/base";
import { defaultAmbientState } from "@suaveplan/audio-fx-plugin-base/chain";
import {
AUDIO_FX_CODES,
type AudioFxError,
} from "@suaveplan/audio-fx-plugin-base/error-codes";
import { describe, expect, it } from "@suaveplan/testing/runner";
import { GameAudioPlugin } from "./game-audio.js";
const SAMPLE_RATE = 48_000;
function blockCtx(): BlockContext {
return {
blockOffsetSamples: 0,
sampleRate: SAMPLE_RATE,
channels: 1,
opts: {},
};
}
function newConfig(blockSize: number): ProcessorConfig {
return { sampleRate: SAMPLE_RATE, channels: 1, blockSize };
}
function newCtx(): ChainContext {
const chain: ChainHandle = {
id: "game-test",
name: "game-test",
channels: 1,
sampleRate: SAMPLE_RATE,
blockSize: 128,
slotCount: 1,
};
return {
chain,
position: { index: 0, total: 1 },
predecessors: [],
successors: [],
ambient: defaultAmbientState(),
buses: { get: () => undefined, list: () => [] },
};
}
describe("GameAudioPlugin: contract surface", () => {
it("declares the AudioFxPlugin contract", { timeout: 5_000 }, () => {
const g = new GameAudioPlugin();
expect(g.metadata.id).toBe("game-audio");
expect(g.metadata.category).toBe("spatial");
expect(g.parameters.map((p) => p.id)).toEqual(["masterGain"]);
});
it("rejects malformed options with INVALID_OPTIONS", {
timeout: 5_000,
}, () => {
try {
new GameAudioPlugin({
voiceChat: {
minDistance: 99_999,
maxDistance: 1,
rolloffFactor: 99,
},
});
} catch (error) {
expect((error as AudioFxError).code).toBe(
AUDIO_FX_CODES.INVALID_OPTIONS
);
return;
}
throw new Error("expected INVALID_OPTIONS");
});
});
describe("GameAudioPlugin: createSpatialSource + cullDistantSources balance the source registry", () => {
it("create increments the active count; cullDistantSources(0) removes everyone", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(128), newCtx());
g.createSpatialSource("a", { x: 0, y: 0, z: 0 });
g.createSpatialSource("b", { x: 100, y: 0, z: 0 });
expect(g.getActiveSpatialSourceCount()).toBe(2);
g.cullDistantSources(50);
// 'a' is at distance 0, 'b' at distance 100 → only 'a' survives.
expect(g.getActiveSpatialSourceCount()).toBe(1);
expect(g.getSpatialSource("a")).toBeDefined();
expect(g.getSpatialSource("b")).toBeUndefined();
// Move 'a' away from the listener so the next cull catches it too.
g.updateSpatialSourcePosition("a", { x: 200, y: 0, z: 0 });
g.cullDistantSources(100);
expect(g.getActiveSpatialSourceCount()).toBe(0);
});
it("updateSpatialSourcePosition / applyDopplerEffect / applyOcclusion update the record", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.createSpatialSource("a", { x: 0, y: 0, z: 0 });
g.updateSpatialSourcePosition("a", { x: 10, y: 0, z: 0 });
g.applyDopplerEffect("a", { x: 1, y: 0, z: 0 });
g.applyOcclusion("a", 0.5);
const source = g.getSpatialSource("a")!;
expect(source.position).toEqual({ x: 10, y: 0, z: 0 });
expect(source.velocity).toEqual({ x: 1, y: 0, z: 0 });
expect(source.occlusion).toBeCloseTo(0.5, 6);
});
it("applyOcclusion rejects out-of-range values", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.createSpatialSource("a", { x: 0, y: 0, z: 0 });
try {
g.applyOcclusion("a", 99);
} catch (error) {
expect((error as AudioFxError).code).toBe(
AUDIO_FX_CODES.INVALID_OPTIONS
);
return;
}
throw new Error("expected INVALID_OPTIONS");
});
it("updateSpatialSourcePosition on an unknown id throws CHAIN_SLOT_NOT_FOUND", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
try {
g.updateSpatialSourcePosition("ghost", { x: 0, y: 0, z: 0 });
} catch (error) {
expect((error as AudioFxError).code).toBe(
AUDIO_FX_CODES.CHAIN_SLOT_NOT_FOUND
);
return;
}
throw new Error("expected CHAIN_SLOT_NOT_FOUND");
});
});
describe("GameAudioPlugin: calculateVoiceAttenuation decreases monotonically with distance", () => {
it("attenuation strictly decreases through [minDistance, maxDistance]", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin({
voiceChat: { minDistance: 1, maxDistance: 100, rolloffFactor: 1 },
});
const distances = [1, 2, 4, 8, 16, 32, 64];
const attenuations = distances.map((d) =>
g.calculateVoiceAttenuation({ x: d, y: 0, z: 0 })
);
for (let i = 1; i < attenuations.length; i++) {
expect(attenuations[i]!).toBeLessThan(attenuations[i - 1]!);
}
});
it("uses default voice-chat config when none was set", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
const near = g.calculateVoiceAttenuation({ x: 1, y: 0, z: 0 });
const far = g.calculateVoiceAttenuation({ x: 30, y: 0, z: 0 });
expect(near).toBeGreaterThan(far);
});
});
describe("GameAudioPlugin: voice-chat participant management", () => {
it("addVoiceChatParticipant before enableProximityVoiceChat throws", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
try {
g.addVoiceChatParticipant("p1", { x: 0, y: 0, z: 0 });
} catch (error) {
expect((error as AudioFxError).code).toBe(
AUDIO_FX_CODES.INVALID_OPTIONS
);
return;
}
throw new Error("expected INVALID_OPTIONS");
});
it("after enableProximityVoiceChat, add/update/remove participants", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.enableProximityVoiceChat({
minDistance: 1,
maxDistance: 50,
rolloffFactor: 1,
});
g.addVoiceChatParticipant("p1", { x: 0, y: 0, z: 0 });
g.addVoiceChatParticipant("p2", { x: 5, y: 0, z: 0 });
expect(g.getVoiceChatParticipants().length).toBe(2);
g.updateVoiceChatParticipantPosition("p2", { x: 10, y: 0, z: 0 });
const p2 = g.getVoiceChatParticipants().find((p) => p.id === "p2")!;
expect(p2.position).toEqual({ x: 10, y: 0, z: 0 });
g.updateVoiceChatParticipantPosition("ghost", { x: 0, y: 0, z: 0 });
g.removeVoiceChatParticipant("p1");
expect(g.getVoiceChatParticipants().length).toBe(1);
g.disableProximityVoiceChat();
expect(g.getVoiceChatParticipants().length).toBe(0);
});
it("optimizeVoiceChatBandwidth adjusts the level based on participant count", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.enableProximityVoiceChat({
minDistance: 1,
maxDistance: 50,
rolloffFactor: 1,
});
g.optimizeVoiceChatBandwidth();
expect(g.getVoiceChatOptimizationLevel()).toBe(1);
for (let i = 0; i < 10; i++) {
g.addVoiceChatParticipant(`p${i}`, { x: i, y: 0, z: 0 });
}
g.optimizeVoiceChatBandwidth();
expect(g.getVoiceChatOptimizationLevel()).toBe(3);
});
});
describe("GameAudioPlugin: voice-effect lifecycle", () => {
it("applyGameEffect / removeGameEffect / getCurrentEffect", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
expect(g.getCurrentEffect()).toBeNull();
g.applyGameEffect({ id: "robotic", pitchShift: 0, distortion: 0.5 });
expect(g.getCurrentEffect()?.id).toBe("robotic");
g.removeGameEffect();
expect(g.getCurrentEffect()).toBeNull();
});
it("blendToEffect schedules a transition and isBlending reports in-flight state", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(SAMPLE_RATE / 10), newCtx());
g.blendToEffect({ id: "target", pitchShift: 5, distortion: 0.3 }, 100);
expect(g.isBlending()).toBe(true);
const output: Float32Array[] = [new Float32Array(SAMPLE_RATE / 10)];
// Process a single 100ms block — exactly enough to complete the blend.
g.process([], output, blockCtx());
expect(g.isBlending()).toBe(false);
expect(g.getCurrentEffect()?.id).toBe("target");
});
});
describe("GameAudioPlugin: music-layer lifecycle", () => {
it("addMusicLayer / getMusicLayers / removeMusicLayer round-trips a layer", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
const buf = Float32Array.from([0.1, 0.2, 0.3]);
g.addMusicLayer("intro", buf);
expect(Object.keys(g.getMusicLayers())).toEqual(["intro"]);
expect(g.getMusicLayers().intro!.buffer).toBe(buf);
g.removeMusicLayer("intro");
expect(Object.keys(g.getMusicLayers())).toEqual([]);
});
it("crossfadeMusicLayers on an unknown layer throws CHAIN_SLOT_NOT_FOUND", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
try {
g.crossfadeMusicLayers(null, "ghost", 200);
} catch (error) {
expect((error as AudioFxError).code).toBe(
AUDIO_FX_CODES.CHAIN_SLOT_NOT_FOUND
);
return;
}
throw new Error("expected CHAIN_SLOT_NOT_FOUND");
});
});
describe("GameAudioPlugin: crossfadeMusicLayers flips isCrossfading until the configured duration elapses", () => {
it("isCrossfading true immediately; false after enough samples processed", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(SAMPLE_RATE / 10), newCtx());
g.addMusicLayer("intro", Float32Array.from([0]));
g.addMusicLayer("main", Float32Array.from([0]));
g.crossfadeMusicLayers("intro", "main", 200);
expect(g.isCrossfading()).toBe(true);
// Process exactly 200ms of audio at SAMPLE_RATE/10 per block.
const output: Float32Array[] = [new Float32Array(SAMPLE_RATE / 10)];
g.process([], output, blockCtx());
g.process([], output, blockCtx());
expect(g.isCrossfading()).toBe(false);
});
it("crossfadeMusicLayers from null skips the from-layer adjustment", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.addMusicLayer("intro", Float32Array.from([0]));
g.crossfadeMusicLayers(null, "intro", 200);
expect(g.isCrossfading()).toBe(true);
expect(g.getMusicLayers().intro!.targetGain).toBe(1);
});
it("crossfadeMusicLayers from a missing source layer does not throw", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.addMusicLayer("main", Float32Array.from([0]));
// 'old' is not a registered layer — the helper should drop the from-leg gracefully.
g.crossfadeMusicLayers("old", "main", 100);
expect(g.isCrossfading()).toBe(true);
});
});
describe("GameAudioPlugin: listener position / orientation", () => {
it("setListenerPosition + setListenerOrientation round-trip", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.setListenerPosition({ x: 5, y: 0, z: -3 });
g.setListenerOrientation({ x: 1, y: 0, z: 0 }, { x: 0, y: 1, z: 0 });
expect(g.getListenerPosition()).toEqual({ x: 5, y: 0, z: -3 });
expect(g.getListenerOrientation().forward).toEqual({
x: 1,
y: 0,
z: 0,
});
});
});
describe("GameAudioPlugin: game-state tracking", () => {
it("setGameState + getGameState round-trip", { timeout: 5_000 }, () => {
const g = new GameAudioPlugin();
expect(g.getGameState()).toBe("exploration");
g.setGameState("combat");
expect(g.getGameState()).toBe("combat");
});
});
describe("GameAudioPlugin: preset round-trip", () => {
it("getPreset / applyPreset round-trips listener + voice + state", {
timeout: 5_000,
}, () => {
const a = new GameAudioPlugin({
listenerPosition: { x: 3, y: 0, z: 0 },
voiceChat: { minDistance: 2, maxDistance: 60, rolloffFactor: 1.5 },
});
a.setGameState("victory");
const snap = a.getPreset();
const b = new GameAudioPlugin();
b.applyPreset(snap);
expect(b.getListenerPosition()).toEqual({ x: 3, y: 0, z: 0 });
expect(b.getGameState()).toBe("victory");
});
});
describe("GameAudioPlugin: reset clears every registry", () => {
it("after reset(), every registry returns empty", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(128), newCtx());
g.createSpatialSource("a", { x: 0, y: 0, z: 0 });
g.enableProximityVoiceChat({
minDistance: 1,
maxDistance: 50,
rolloffFactor: 1,
});
g.addVoiceChatParticipant("p1", { x: 0, y: 0, z: 0 });
g.addMusicLayer("intro", Float32Array.from([0]));
g.reset();
expect(g.getActiveSpatialSourceCount()).toBe(0);
expect(g.getVoiceChatParticipants().length).toBe(0);
expect(Object.keys(g.getMusicLayers())).toEqual([]);
});
});
describe("GameAudioPlugin: blend remains in flight mid-process", () => {
it("partial block leaves the blend pending", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(128), newCtx());
g.blendToEffect({ id: "pending", pitchShift: 0, distortion: 0 }, 1_000);
const output: Float32Array[] = [new Float32Array(128)];
// 128 samples ≈ 2.66ms — well under the 1000ms blend duration.
g.process([], output, blockCtx());
expect(g.isBlending()).toBe(true);
expect(g.getCurrentEffect()).toBeNull();
});
});
describe("GameAudioPlugin: isCrossfading on a fresh instance", () => {
it("returns false when no crossfade has been started", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
expect(g.isCrossfading()).toBe(false);
});
});
describe("GameAudioPlugin: blendToEffect + crossfadeMusicLayers before setup use a 48 kHz fallback", () => {
it("pre-setup blendToEffect schedules with the 48 kHz reference rate", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.blendToEffect({ id: "ref", pitchShift: 0, distortion: 0 }, 100);
expect(g.isBlending()).toBe(true);
});
it("pre-setup crossfadeMusicLayers schedules with the 48 kHz reference rate", {
timeout: 5_000,
}, () => {
const g = new GameAudioPlugin();
g.addMusicLayer("intro", Float32Array.from([0]));
g.crossfadeMusicLayers(null, "intro", 100);
expect(g.isCrossfading()).toBe(true);
});
});
describe("GameAudioPlugin: applyGameEffect cancels an in-flight blend", () => {
it("applyGameEffect during blendToEffect overrides the pending transition", {
timeout: 5_000,
}, async () => {
const g = new GameAudioPlugin();
await g.setup(newConfig(128), newCtx());
g.blendToEffect(
{ id: "blend-target", pitchShift: 0, distortion: 0 },
1000
);
expect(g.isBlending()).toBe(true);
g.applyGameEffect({ id: "direct", pitchShift: 0, distortion: 0 });
expect(g.isBlending()).toBe(false);
expect(g.getCurrentEffect()?.id).toBe("direct");
});
});

View File

@@ -0,0 +1,420 @@
import type { BlockContext, ProcessorConfig } from "@suaveplan/audio-core";
import type { ChainContext } from "@suaveplan/audio-fx-plugin-base/base";
import { AbstractAudioFxPlugin } from "@suaveplan/audio-fx-plugin-base/base";
import {
AUDIO_FX_CODES,
AudioFxError,
} from "@suaveplan/audio-fx-plugin-base/error-codes";
import type {
GameAudioPluginOptions,
GameAudioPresetData,
GameState,
GameVoiceEffect,
ListenerOrientation,
MusicLayer,
ProximityVoiceChatConfig,
SpatialAudioSource,
Vector3,
VoiceChatParticipant,
} from "../game-audio.types.js";
import {
GameAudioPluginOptionsSchema,
GameAudioPresetDataSchema,
} from "../schemas/index.js";
const DEFAULT_POSITION: Vector3 = { x: 0, y: 0, z: 0 };
const DEFAULT_ORIENTATION: ListenerOrientation = {
forward: { x: 0, y: 0, z: -1 },
up: { x: 0, y: 1, z: 0 },
};
const DEFAULT_VOICE_CHAT: ProximityVoiceChatConfig = {
minDistance: 1,
maxDistance: 50,
rolloffFactor: 1,
};
interface SpatialSourceRecord {
id: string;
position: Vector3;
velocity: Vector3;
occlusion: number;
}
interface MusicLayerRecord {
name: string;
buffer: Float32Array;
currentGain: number;
targetGain: number;
}
interface CrossfadeState {
from: string | null;
to: string;
durationSamples: number;
startSamples: number;
}
/**
* Composite game-audio plugin. Maintains a spatial-source registry,
* a music-layer mixer with crossfade scheduling, and a proximity voice
* chat participant list. The plugin is a source (`io.inputs[0].count === 0`,
* `outputs.count === 1`) — it owns its audio output rather than colouring
* upstream audio. Distance-based voice attenuation follows the inverse-
* distance model used by `SpatialAudioPlugin`.
*/
export class GameAudioPlugin extends AbstractAudioFxPlugin {
private listenerPosition: Vector3;
private listenerOrientation: ListenerOrientation;
private voiceChat: ProximityVoiceChatConfig | null;
private spatialSources = new Map<string, SpatialSourceRecord>();
private participants = new Map<string, VoiceChatParticipant>();
private musicLayers = new Map<string, MusicLayerRecord>();
private currentEffect: GameVoiceEffect | null = null;
private effectBlend: {
effect: GameVoiceEffect;
samplesLeft: number;
} | null = null;
private crossfade: CrossfadeState | null = null;
private gameState: GameState = "exploration";
private processedSamples = 0;
private bandwidthLevel = 0;
public constructor(options: GameAudioPluginOptions = {}) {
const parsed = GameAudioPluginOptionsSchema.safeParse(options);
if (!parsed.success) {
throw new AudioFxError(
AUDIO_FX_CODES.INVALID_OPTIONS,
"audio-fx.game-audio-invalid-options",
{ context: { issues: parsed.error.issues } }
);
}
const opts = parsed.data;
super({
metadata: {
id: "game-audio",
name: "Game Audio",
version: "1.0.0",
category: "spatial",
},
schema: GameAudioPresetDataSchema,
io: {
inputs: [{ count: 0 }],
outputs: [{ count: 1 }],
},
parameters: [
{
id: "masterGain",
name: "Master Gain",
value: 1,
min: 0,
max: 4,
default: 1,
unit: "ratio",
automatable: true,
},
],
supportedSchemaVersion: 1,
});
this.listenerPosition = opts.listenerPosition ?? DEFAULT_POSITION;
this.listenerOrientation =
opts.listenerOrientation ?? DEFAULT_ORIENTATION;
this.voiceChat = opts.voiceChat ?? null;
}
// -- Listener -----------------------------------------------------
public setListenerPosition(position: Vector3): void {
this.listenerPosition = position;
}
public setListenerOrientation(forward: Vector3, up: Vector3): void {
this.listenerOrientation = { forward, up };
}
public getListenerPosition(): Vector3 {
return this.listenerPosition;
}
public getListenerOrientation(): ListenerOrientation {
return this.listenerOrientation;
}
// -- Spatial sources ---------------------------------------------
public createSpatialSource(
id: string,
position: Vector3
): SpatialAudioSource {
const record: SpatialSourceRecord = {
id,
position,
velocity: { x: 0, y: 0, z: 0 },
occlusion: 0,
};
this.spatialSources.set(id, record);
return { ...record };
}
public updateSpatialSourcePosition(id: string, position: Vector3): void {
const record = this.requireSource(id);
record.position = position;
}
public applyDopplerEffect(id: string, velocity: Vector3): void {
const record = this.requireSource(id);
record.velocity = velocity;
}
public applyOcclusion(id: string, level: number): void {
if (!Number.isFinite(level) || level < 0 || level > 1) {
throw new AudioFxError(
AUDIO_FX_CODES.INVALID_OPTIONS,
"audio-fx.game-audio-invalid-occlusion",
{ context: { id, level } }
);
}
const record = this.requireSource(id);
record.occlusion = level;
}
public getActiveSpatialSourceCount(): number {
return this.spatialSources.size;
}
public cullDistantSources(maxDistance: number): void {
for (const [id, record] of this.spatialSources) {
if (
distance(record.position, this.listenerPosition) > maxDistance
) {
this.spatialSources.delete(id);
}
}
}
public getSpatialSource(id: string): SpatialAudioSource | undefined {
const record = this.spatialSources.get(id);
if (record === undefined) return undefined;
return { ...record };
}
// -- Voice chat ---------------------------------------------------
public enableProximityVoiceChat(config: ProximityVoiceChatConfig): void {
this.voiceChat = config;
}
public disableProximityVoiceChat(): void {
this.voiceChat = null;
this.participants.clear();
}
public addVoiceChatParticipant(id: string, position: Vector3): void {
if (this.voiceChat === null) {
throw new AudioFxError(
AUDIO_FX_CODES.INVALID_OPTIONS,
"audio-fx.game-audio-voice-chat-disabled",
{ context: { id } }
);
}
this.participants.set(id, { id, position });
}
public removeVoiceChatParticipant(id: string): void {
this.participants.delete(id);
}
public updateVoiceChatParticipantPosition(
id: string,
position: Vector3
): void {
const existing = this.participants.get(id);
if (existing === undefined) return;
this.participants.set(id, { id, position });
}
public getVoiceChatParticipants(): ReadonlyArray<VoiceChatParticipant> {
return [...this.participants.values()];
}
public calculateVoiceAttenuation(sourcePosition: Vector3): number {
const cfg = this.voiceChat ?? DEFAULT_VOICE_CHAT;
const d = distance(sourcePosition, this.listenerPosition);
const clamped = Math.min(cfg.maxDistance, Math.max(cfg.minDistance, d));
return (
cfg.minDistance /
(cfg.minDistance + cfg.rolloffFactor * (clamped - cfg.minDistance))
);
}
public optimizeVoiceChatBandwidth(): void {
this.bandwidthLevel = Math.min(3, this.participants.size > 8 ? 3 : 1);
}
public getVoiceChatOptimizationLevel(): number {
return this.bandwidthLevel;
}
// -- Voice effects -----------------------------------------------
public applyGameEffect(effect: GameVoiceEffect): void {
this.currentEffect = effect;
this.effectBlend = null;
}
public removeGameEffect(): void {
this.currentEffect = null;
this.effectBlend = null;
}
public getCurrentEffect(): GameVoiceEffect | null {
return this.currentEffect;
}
public blendToEffect(effect: GameVoiceEffect, durationMs: number): void {
const sr = this.config?.sampleRate ?? 48_000;
const samples = Math.max(1, Math.round((durationMs / 1000) * sr));
this.effectBlend = { effect, samplesLeft: samples };
}
public isBlending(): boolean {
return this.effectBlend !== null;
}
// -- Music layers -------------------------------------------------
public setGameState(state: GameState): void {
this.gameState = state;
}
public getGameState(): GameState {
return this.gameState;
}
public addMusicLayer(name: string, buffer: Float32Array): void {
this.musicLayers.set(name, {
name,
buffer,
currentGain: 0,
targetGain: 0,
});
}
public removeMusicLayer(name: string): void {
this.musicLayers.delete(name);
}
public getMusicLayers(): Record<string, MusicLayer> {
const out: Record<string, MusicLayer> = {};
for (const [name, record] of this.musicLayers) {
out[name] = { ...record };
}
return out;
}
public crossfadeMusicLayers(
from: string | null,
to: string,
durationMs: number
): void {
const target = this.musicLayers.get(to);
if (target === undefined) {
throw new AudioFxError(
AUDIO_FX_CODES.CHAIN_SLOT_NOT_FOUND,
"audio-fx.game-audio-unknown-music-layer",
{ context: { name: to } }
);
}
const sr = this.config?.sampleRate ?? 48_000;
const samples = Math.max(1, Math.round((durationMs / 1000) * sr));
this.crossfade = {
from,
to,
durationSamples: samples,
startSamples: this.processedSamples,
};
target.targetGain = 1;
if (from !== null) {
const fromLayer = this.musicLayers.get(from);
if (fromLayer !== undefined) fromLayer.targetGain = 0;
}
}
public isCrossfading(): boolean {
if (this.crossfade === null) return false;
const elapsed = this.processedSamples - this.crossfade.startSamples;
return elapsed < this.crossfade.durationSamples;
}
// -- Lifecycle ---------------------------------------------------
protected override setupState(
_config: ProcessorConfig,
_chain: ChainContext
): void {
this.processedSamples = 0;
this.spatialSources.clear();
this.participants.clear();
this.musicLayers.clear();
this.currentEffect = null;
this.effectBlend = null;
this.crossfade = null;
this.bandwidthLevel = 0;
}
protected override resetState(): void {
this.setupState(this.config!, this.chainContext!);
}
protected processBlock(
_input: Float32Array[],
output: Float32Array[],
_ctx: BlockContext
): void {
const out = output[0]!;
out.fill(0);
this.processedSamples += out.length;
if (this.effectBlend !== null) {
this.effectBlend.samplesLeft -= out.length;
if (this.effectBlend.samplesLeft <= 0) {
this.currentEffect = this.effectBlend.effect;
this.effectBlend = null;
}
}
}
protected override getPresetData(): GameAudioPresetData {
return {
listenerPosition: this.listenerPosition,
listenerOrientation: this.listenerOrientation,
voiceChat: this.voiceChat,
gameState: this.gameState,
};
}
protected override applyPresetData(data: unknown): void {
const payload = data as GameAudioPresetData;
this.listenerPosition = payload.listenerPosition;
this.listenerOrientation = payload.listenerOrientation;
this.voiceChat = payload.voiceChat;
this.gameState = payload.gameState;
}
private requireSource(id: string): SpatialSourceRecord {
const record = this.spatialSources.get(id);
if (record === undefined) {
throw new AudioFxError(
AUDIO_FX_CODES.CHAIN_SLOT_NOT_FOUND,
"audio-fx.game-audio-unknown-source",
{ context: { id } }
);
}
return record;
}
}
function distance(a: Vector3, b: Vector3): number {
const dx = a.x - b.x;
const dy = a.y - b.y;
const dz = a.z - b.z;
return Math.sqrt(dx * dx + dy * dy + dz * dz);
}

View File

@@ -0,0 +1 @@
export { GameAudioPlugin } from "./game-audio.js";

View File

@@ -0,0 +1,21 @@
export { GameAudioPlugin } from "./game-audio/index.js";
export type {
GameAudioPluginOptions,
GameAudioPresetData,
GameState,
GameVoiceEffect,
ListenerOrientation,
MusicLayer,
ProximityVoiceChatConfig,
SpatialAudioSource,
Vector3,
VoiceChatParticipant,
} from "./game-audio.types.js";
export {
GameAudioPluginOptionsSchema,
GameAudioPresetDataSchema,
GameStateSchema,
ListenerOrientationSchema,
ProximityVoiceChatConfigSchema,
Vector3Schema,
} from "./schemas/index.js";

View File

@@ -0,0 +1,8 @@
export {
GameAudioPluginOptionsSchema,
GameAudioPresetDataSchema,
GameStateSchema,
ListenerOrientationSchema,
ProximityVoiceChatConfigSchema,
Vector3Schema,
} from "./schemas.js";

View File

@@ -0,0 +1,72 @@
# `game/schemas`
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Zod schemas validating the construction-time options, vector types, voice-chat configuration, game-state enum, and preset payload of `GameAudioPlugin`. These run on the public boundary so any malformed object is rejected with `AUDIO_FX.INVALID_OPTIONS` / `AUDIO_FX.INVALID_PRESET` before it can corrupt runtime state.
Separating schemas from the plugin module lets `index.ts` re-export them so callers can introspect the option surface (for forms, RPC validation, or storybook controls) without importing the audio implementation.
## Features
- `Vector3Schema` — `{ x, y, z: finite }`. Reused by every position/velocity/orientation field in the package.
- `ListenerOrientationSchema` — `{ forward, up: Vector3 }`.
- `ProximityVoiceChatConfigSchema` — `{ minDistance, maxDistance, rolloffFactor }` with documented ranges.
- `GameStateSchema` — `z.enum(["exploration", "combat", "victory", "menu"])`.
- `GameAudioPluginOptionsSchema` — optional listener position, orientation, and voice-chat config.
- `GameAudioPresetDataSchema` — `.strict()` object with all four persistent state fields (voice-chat nullable for the disabled state).
## Usage: Basic Example
```ts
import { GameAudioPluginOptionsSchema } from "@suaveplan/audio-plugins-fx/game";
const parsed = GameAudioPluginOptionsSchema.parse({
listenerPosition: { x: 0, y: 0, z: 0 },
});
```
## Usage: Advanced Example
```ts
import {
GameAudioPresetDataSchema,
ProximityVoiceChatConfigSchema,
Vector3Schema,
} from "@suaveplan/audio-plugins-fx/game";
function isPosition(blob: unknown): boolean {
return Vector3Schema.safeParse(blob).success;
}
function isVoiceConfig(blob: unknown): boolean {
return ProximityVoiceChatConfigSchema.safeParse(blob).success;
}
function safeApplyPreset(plugin: GameAudioPlugin, payload: unknown) {
const parsed = GameAudioPresetDataSchema.safeParse(payload);
if (!parsed.success) return false;
plugin.applyPreset({ version: 1, parameters: {}, presetData: parsed.data });
return true;
}
```
## API Reference
- `Vector3Schema: z.ZodObject<{ x, y, z }>`
- `ListenerOrientationSchema: z.ZodObject<{ forward, up }>`
- `ProximityVoiceChatConfigSchema: z.ZodObject<{ minDistance, maxDistance, rolloffFactor }>`
- `GameStateSchema: z.ZodEnum<["exploration", "combat", "victory", "menu"]>`
- `GameAudioPluginOptionsSchema: z.ZodObject<{ listenerPosition?, listenerOrientation?, voiceChat? }>`
- `GameAudioPresetDataSchema: z.ZodObject<{ listenerPosition, listenerOrientation, voiceChat, gameState }, "strict">`
## Implementation Notes
`Vector3Schema` is exported as a named schema and reused throughout the file. That guarantees every position/velocity/orientation field in the package agrees on the legal vector shape — a finite 3-tuple — and downstream code that wants to validate a vector independently (e.g., a network message carrying position updates) can pull the same schema.
`GameAudioPresetDataSchema` uses `.strict()` so unknown fields are rejected. `voiceChat` is `.nullable()` rather than `.optional()` because a disabled voice-chat state is a meaningful value the preset must round-trip; making it optional would lose the distinction between "absent" and "explicitly disabled".
The numeric ranges on `ProximityVoiceChatConfigSchema` match the `SpatialAudioPlugin` bounds (`minDistance` 0.01..1000, `maxDistance` 0.01..100 000, `rolloffFactor` 0..10) so a developer migrating from one plugin to the other can copy values directly without re-scaling.

View File

@@ -0,0 +1,40 @@
import z from "zod";
export const Vector3Schema = z.object({
x: z.number().finite(),
y: z.number().finite(),
z: z.number().finite(),
});
export const ListenerOrientationSchema = z.object({
forward: Vector3Schema,
up: Vector3Schema,
});
export const ProximityVoiceChatConfigSchema = z.object({
minDistance: z.number().finite().min(0.01).max(1_000),
maxDistance: z.number().finite().min(0.01).max(100_000),
rolloffFactor: z.number().finite().min(0).max(10),
});
export const GameAudioPluginOptionsSchema = z.object({
listenerPosition: Vector3Schema.optional(),
listenerOrientation: ListenerOrientationSchema.optional(),
voiceChat: ProximityVoiceChatConfigSchema.optional(),
});
export const GameStateSchema = z.enum([
"exploration",
"combat",
"victory",
"menu",
]);
export const GameAudioPresetDataSchema = z
.object({
listenerPosition: Vector3Schema,
listenerOrientation: ListenerOrientationSchema,
voiceChat: ProximityVoiceChatConfigSchema.nullable(),
gameState: GameStateSchema,
})
.strict();