131 lines
8.7 KiB
Markdown
131 lines
8.7 KiB
Markdown
# Design: enterprise-audio-sprite-player
|
|
|
|
Non-normative. Captures rationale, architecture, algorithms, and test strategy for `@suaveplan/audio-sprite-player`. Normative requirements live in `./specs/audio-sprite-player/spec.md`.
|
|
|
|
---
|
|
|
|
## 1. Why a separate package vs. extending `audio-timeline`
|
|
|
|
`@suaveplan/audio-timeline` models multi-track, multi-clip arrangements with tempo, transport, automation, and a deterministic render graph. It is the right tool for music, podcasts, and any "edit then render" workflow where clips are stateful arrangement participants.
|
|
|
|
Sprite playback is the **opposite shape**:
|
|
|
|
- **Fire-and-forget.** A trigger fires (UI click, game footstep, hit confirmation) and the resulting voice is a transient with no place in a persistent arrangement.
|
|
- **No clip / track model.** Sprite entries are flat named regions inside one blob, not arrangeable clips with positions and lengths the user edits.
|
|
- **Latency-sensitive, scheduler-light.** A game wants `play("hit")` to take microseconds; it does not want to instantiate a transport, place a clip, advance a playhead, and tear it down.
|
|
- **Voice-stealing is core**, not an afterthought. Timelines allocate every voice they need; sprite players cap voices and drop the oldest when over budget.
|
|
|
|
Forcing sprites through the timeline model would degrade timeline (it would gain a "transient voice" concept that doesn't belong) and bloat sprites (each one-shot allocates a track and a clip). Keeping them separate keeps each package coherent.
|
|
|
|
The two layers do compose downstream: a timeline track of triggered SFX can call into a `SpritePlayer` per trigger. The composition lives in the consumer, not in this package.
|
|
|
|
---
|
|
|
|
## 2. Plugin-slot pattern (mirrors `TimelineAdapter`)
|
|
|
|
Universal-tier purity demands the package never touch `AudioContext`, `decodeAudioData`, native audio bindings, `Buffer`, `node:fs`, or any other platform global. The package therefore owns **only** orchestration; everything that has to talk to real audio hardware is reached through the injected `SpritePlaybackAdapter`.
|
|
|
|
```ts
|
|
interface SpritePlaybackAdapter {
|
|
init(sprite: AudioSprite, signal?: AbortSignal): Promise<void>
|
|
play(entry: AudioSpriteEntry, opts: ResolvedPlayOptions, signal?: AbortSignal): Promise<SpritePlayHandle>
|
|
dispose(): Promise<void>
|
|
}
|
|
```
|
|
|
|
Responsibility split:
|
|
|
|
| Concern | Universal `SpritePlayer` | Engine `SpritePlaybackAdapter` |
|
|
|---|---|---|
|
|
| Manifest Zod validation | ✅ | — |
|
|
| Entry lookup by name | ✅ | — |
|
|
| Option defaulting + Zod validation | ✅ | — |
|
|
| Voice manager + voice-stealing | ✅ | — |
|
|
| Sequence and random orchestration | ✅ | — |
|
|
| OTel span/counter/gauge/histogram | ✅ | — |
|
|
| Typed event emission | ✅ | — |
|
|
| Abort propagation | ✅ (passes signal through) | ✅ (must honour signal during init/play) |
|
|
| `decodeAudioData` / native decode | — | ✅ |
|
|
| Actual gain ramps (fadeIn/fadeOut) | — | ✅ (player passes ms request) |
|
|
| Pitch / playback-rate scheduling | — | ✅ |
|
|
| Hardware mixer routing | — | ✅ |
|
|
|
|
The adapter is the **only** seam the player has into the environment. `audio-engine-browser` will provide one implementation; `audio-engine-server` another. Tests in this package use `NullSpritePlaybackAdapter`.
|
|
|
|
This mirrors `TimelineAdapter` in `@suaveplan/audio-timeline`: universal package owns orchestration, environment-specific work is delegated through one interface.
|
|
|
|
---
|
|
|
|
## 3. Voice manager + voice-stealing
|
|
|
|
`VoiceManager` tracks active `SpritePlayHandle`s. On every `play` call:
|
|
|
|
1. If `voices.size < maxConcurrentVoices`, allocate and run.
|
|
2. Otherwise, select the **oldest** active voice by `startedAt` (ties broken by allocation order), call `voice.stop(0)`, emit `voice-stolen` with `{ stolenId, replacedBy: newId, entryName }`, then run the new voice.
|
|
3. The newly stolen handle's `isPlaying()` returns `false` immediately; its `stop()` becomes idempotent.
|
|
|
|
The algorithm is intentionally simple — oldest-first — because that is what sprite-driven consumers (games, UI sound layers) expect. Priority-based stealing is **out of scope**; it can be layered in a follow-up by giving entries a priority field on the consumer side and filtering before calling `play`. The cap is configurable (`maxConcurrentVoices`, default 32) and capped at a hard ceiling validated by the Zod boundary schema (≤ 256) so a misconfiguration cannot exhaust memory.
|
|
|
|
`voice-stolen` is a distinct event from `stop` so consumers can distinguish a user-driven stop from voice-pressure-driven stop. Both share the same underlying mechanism so consumer cleanup logic can listen to either.
|
|
|
|
---
|
|
|
|
## 4. Sequence and random playback semantics
|
|
|
|
### `playSequence(entryNames, opts)`
|
|
|
|
Plays entries strictly in order, each starting `previousEntry.durationMs + gapMs` after the previous one began (using `clock.now()` + adapter `when`). Returns a `Promise<SpritePlayHandle[]>` that resolves once **all** voices have been started (not when they have completed); waiting on completion is the caller's job via the `complete` event. If any single `play` throws, the sequence aborts and any already-started voices are stopped with `fadeOutMs = 0`; the rejection wraps the underlying error with `AUDIO_SPRITE_PLAYER.PLAYBACK_FAILED` and includes the failed entry name and sequence index in the error context.
|
|
|
|
Sequence honours `AbortSignal`: aborting between two entries stops the in-flight voices and rejects with `AUDIO_SPRITE_PLAYER.ABORTED`.
|
|
|
|
### `playRandom(filterFn?, opts)`
|
|
|
|
Uses `Math.random()` only when no `Rng` is injected via `SpritePlayerOptions`. (The package accepts an optional `Rng` from `@suaveplan/types` — TODO promoted into the spec — so determinism in tests does not depend on stubbing globals.) `filterFn` runs against every entry; if it returns `false` for all entries, the call rejects with `AUDIO_SPRITE_PLAYER.UNKNOWN_ENTRY` (filter ruled out everything is observably the same as no matching entry). With no filter, every entry has equal weight.
|
|
|
|
---
|
|
|
|
## 5. Fade-in / fade-out
|
|
|
|
The player specifies fade requests in **milliseconds** on the `PlayOptions` passed to the adapter. **The player does not implement fades.** It cannot — fades need access to a real gain node / amplitude envelope, which is engine-tier.
|
|
|
|
Contract: when the player tells the adapter to play with `fadeInMs: 100`, the adapter MUST ramp from `0 → volume` over 100 ms; on `stop(fadeOutMs)`, the adapter MUST ramp from current `→ 0` over that many milliseconds and only then disconnect.
|
|
|
|
This keeps the universal layer pure while still expressing the user's intent through the public surface.
|
|
|
|
---
|
|
|
|
## 6. AbortSignal propagation
|
|
|
|
Every async public method accepts `AbortSignal`:
|
|
|
|
- `load(sprite, signal?)` — aborts before / during `adapter.init`; rejects with `AUDIO_SPRITE_PLAYER.ABORTED` and disposes the adapter.
|
|
- `play(entryName, opts)` — `opts.signal` propagates into `adapter.play`; aborting after the voice starts triggers `handle.stop(0)`.
|
|
- `playSequence(entryNames, opts)` — see §4. Abort between entries stops in-flight voices.
|
|
- `playRandom(filterFn, opts)` — same as `play`.
|
|
|
|
If the signal is already aborted on entry, the method rejects synchronously with `AUDIO_SPRITE_PLAYER.ABORTED` before any work is started.
|
|
|
|
---
|
|
|
|
## 7. Test strategy with `NullSpritePlaybackAdapter`
|
|
|
|
`NullSpritePlaybackAdapter` is a recording adapter: every call to `init`, `play`, and `dispose` appends an entry to a public `calls` array with the timestamp from a `FakeClock`. `play` returns a `SpritePlayHandle` whose `stop` / `setVolume` / `isPlaying` calls are also recorded. This makes every spec scenario verifiable without a real audio backend:
|
|
|
|
- Voice-stealing is observable by asserting on the `calls` order and the `voice-stolen` event payload.
|
|
- Abort propagation is observable by asserting that `init` / `play` saw the signal in its aborted state.
|
|
- Fade requests are observable by asserting the `ResolvedPlayOptions` passed to `adapter.play`.
|
|
- OTel instrumentation is observable through `@opentelemetry/api`'s in-memory exporter wired in the test setup.
|
|
|
|
The null adapter is exported from the package so downstream packages writing real adapters (engine-browser, engine-server) can also use it as a baseline contract test and as a default in their own tests.
|
|
|
|
---
|
|
|
|
## 8. Out of scope for this change
|
|
|
|
- `BrowserSpritePlaybackAdapter` (separate change against `@suaveplan/audio-engine-browser`).
|
|
- `ServerSpritePlaybackAdapter` (separate change against `@suaveplan/audio-engine-server`).
|
|
- Priority-based voice-stealing (consumer concern; can be layered).
|
|
- 3D spatial panning / HRTF (engine concern; out of the universal contract).
|
|
- Streaming sprite descriptors (the format is a single concatenated blob by design).
|
|
- Hot-reloading a sprite while voices are playing (call `dispose` then construct a new player).
|