Initial Commit
This commit is contained in:
@@ -0,0 +1,234 @@
|
||||
# Design: enterprise-audio-analysis-beat
|
||||
|
||||
**Change ID**: enterprise-audio-analysis-beat
|
||||
**Package**: `@suaveplan/audio-analysis-beat`
|
||||
|
||||
`@suaveplan/audio-analysis-beat` is a universal-tier Analysis Op plugin for the `@suaveplan/audio-*`
|
||||
family. It performs tempo (BPM) detection, beat-grid construction, and onset detection over
|
||||
`AudioDocument` streams. This document carries the **package-specific design** — algorithm
|
||||
rationale, code sketches, file layout, integration notes, and trade-offs. It deliberately does not
|
||||
restate the cross-cutting ecosystem model.
|
||||
|
||||
---
|
||||
|
||||
## 1. Relationship to the ecosystem architecture
|
||||
|
||||
The cross-cutting `@suaveplan/audio-*` architecture — the three-layer separation, the six plugin
|
||||
slots, the zero-duplication policy over genesis primitives, and the universal-purity rule — is
|
||||
defined once in `enterprise-audio-core/design.md` §1–§4 and is **referenced, not duplicated** here.
|
||||
|
||||
Two facts from that document anchor this package:
|
||||
|
||||
- **Analysis Op is plugin slot 3.** Per `enterprise-audio-core/design.md` §1.3, an Analysis Op
|
||||
"reads audio and emits features (BPM, key, spectrum); runs as a sidecar scan, not part of the
|
||||
render pipeline." `BeatDetector` is exactly such a sidecar: it never registers in an `OpRegistry`
|
||||
plan or process slot, never mutates the `AudioDocument`, and runs concurrently with playback or
|
||||
as a standalone batch scan.
|
||||
- **The `AnalysisOp` interface and `AnalysisResult` shape** are defined by the audio ecosystem and
|
||||
consumed here unchanged. `BeatDetector implements AnalysisOp`; `BeatResult extends
|
||||
AnalysisResult`.
|
||||
|
||||
Everything below is specific to beat/tempo/onset detection.
|
||||
|
||||
---
|
||||
|
||||
## 2. Detection pipeline
|
||||
|
||||
Beat detection is a three-stage pipeline regardless of which algorithm is selected:
|
||||
|
||||
```text
|
||||
AudioDocument ──► [1] Onset Detection Function ──► [2] Tempo Estimation ──► [3] Beat Tracking
|
||||
(per-block ODF curve) (dominant period) (phase + grid)
|
||||
```
|
||||
|
||||
### 2.1. Stage 1 — Onset Detection Function (ODF)
|
||||
|
||||
The ODF is a 1-D curve, one value per analysis block, that peaks where the signal's energy or
|
||||
spectral content rises sharply — i.e. at note onsets. Two ODF algorithms ship; a third algorithm
|
||||
(`comb`) reuses the energy ODF and differs only in stage 2.
|
||||
|
||||
**`energy` (default).** The cheapest ODF. It consumes the per-block energy statistics already
|
||||
produced by `@suaveplan/audio-core` and takes the half-wave-rectified first difference:
|
||||
|
||||
```ts
|
||||
// energy-odf.ts — sketch
|
||||
export function energyOdf(blockEnergies: Float64Array): Float64Array {
|
||||
const odf = new Float64Array(blockEnergies.length)
|
||||
for (let i = 1; i < blockEnergies.length; i++) {
|
||||
const d = blockEnergies[i] - blockEnergies[i - 1]
|
||||
odf[i] = d > 0 ? d : 0 // half-wave rectify: onsets are energy increases
|
||||
}
|
||||
return odf
|
||||
}
|
||||
```
|
||||
|
||||
It is fast and allocation-light but blurs onsets that are loud in timbre yet quiet in total energy
|
||||
(e.g. a hi-hat over a sustained pad).
|
||||
|
||||
**`spectral-flux`.** Higher precision and the right default for streaming. A hand-rolled real FFT
|
||||
(no `AnalyserNode`, no Node bindings — universal purity, see the spec delta) produces a magnitude
|
||||
spectrum per frame; the ODF is the sum of positive bin-to-bin differences:
|
||||
|
||||
```ts
|
||||
// spectral-flux.ts — sketch
|
||||
export function spectralFlux(frame: Float32Array, prevMag: Float32Array, fft: RealFft): number {
|
||||
const mag = fft.magnitude(frame) // |X[k]|
|
||||
let flux = 0
|
||||
for (let k = 0; k < mag.length; k++) {
|
||||
const d = mag[k] - prevMag[k]
|
||||
if (d > 0) flux += d // only rising bins count toward an onset
|
||||
prevMag[k] = mag[k]
|
||||
}
|
||||
return flux
|
||||
}
|
||||
```
|
||||
|
||||
Spectral flux catches timbral onsets the energy ODF misses, at the cost of an FFT per frame.
|
||||
|
||||
In both cases, peaks in the ODF above an adaptive threshold (a moving median plus a fixed margin)
|
||||
are reported as `onsets` — timestamps in seconds, strictly increasing.
|
||||
|
||||
### 2.2. Stage 2 — Tempo estimation
|
||||
|
||||
Given the ODF, the dominant periodicity is the tempo. Two strategies:
|
||||
|
||||
- **Autocorrelation** (used by `energy` and `spectral-flux`). Autocorrelate the ODF; the lag of the
|
||||
strongest peak whose implied BPM falls inside `[minBpm, maxBpm]` is the period. Octave errors
|
||||
(detecting half- or double-tempo) are resolved by preferring the candidate closest to the
|
||||
perceptual centre (~120 BPM) when two peaks are within a small ratio.
|
||||
- **Comb filter** (the `comb` algorithm). A bank of comb filters, one per candidate BPM in the
|
||||
range, is convolved with the ODF; the filter with the highest resonance output wins. This is more
|
||||
robust on syncopated material than raw autocorrelation but costs `O(range × odfLength)`.
|
||||
|
||||
```ts
|
||||
// comb-tempo.ts — sketch
|
||||
export function combTempo(odf: Float64Array, sr: number, hop: number, lo: number, hi: number) {
|
||||
let bestBpm = lo, bestScore = -Infinity
|
||||
for (let bpm = lo; bpm <= hi; bpm++) {
|
||||
const period = Math.round((60 / bpm) * (sr / hop)) // ODF samples per beat
|
||||
let score = 0
|
||||
for (let i = period; i < odf.length; i++) score += odf[i] * odf[i - period]
|
||||
if (score > bestScore) { bestScore = score; bestBpm = bpm }
|
||||
}
|
||||
return { bpm: bestBpm, score: bestScore }
|
||||
}
|
||||
```
|
||||
|
||||
### 2.3. Stage 3 — Beat tracking and grid construction
|
||||
|
||||
With a period known, `beat-grid.ts` finds the phase: the offset that, when a `60 / bpm`-spaced grid
|
||||
is laid down, minimises total onset-to-nearest-gridline distance. The output `beats` array is that
|
||||
phase-aligned grid clipped to the document duration. `confidence` is derived from how tightly the
|
||||
detected onsets cluster around the grid (a sharp cluster → high confidence; scattered onsets → low).
|
||||
|
||||
---
|
||||
|
||||
## 3. The three entry points
|
||||
|
||||
`BeatDetector` exposes the full `AnalysisOp` surface; each entry point reuses the same three-stage
|
||||
pipeline at a different granularity.
|
||||
|
||||
- **`analyze(doc, opts?)`** — batch scan. Walks the whole document through stages 1–3 and returns a
|
||||
single settled `BeatResult`. Wrapped in the `audio.analysis.beat` OTel span.
|
||||
- **`analyzeStream(doc, opts?)`** — incremental scan. Runs stage 1 block-by-block and re-runs
|
||||
stages 2–3 on the ODF accumulated so far, yielding a refining `BeatResult` after each window. For
|
||||
a stable-tempo signal, `confidence` is non-decreasing across yields and the final yield's `bpm`
|
||||
equals `analyze`'s.
|
||||
- **`analyzeBlock(block, ctx)`** — real-time. Computes the ODF value for a single block, updates a
|
||||
running tempo estimate, and returns synchronously. Allocation-light enough to call once per
|
||||
audio block from a visualiser's render loop.
|
||||
|
||||
```ts
|
||||
// beat-detector.ts — analyze sketch
|
||||
async analyze(doc: AudioDocument, opts?: BeatDetectorOptions): Promise<BeatResult> {
|
||||
const o = BeatDetectorOptionsSchema.parse({ ...this.opts, ...opts })
|
||||
if (o.signal?.aborted) throw new AudioError('AUDIO_ANALYSIS_BEAT.ABORTED')
|
||||
if (doc.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.EMPTY_DOCUMENT')
|
||||
return tracer.startActiveSpan('audio.analysis.beat', async (span) => {
|
||||
const odf = await this.buildOdf(doc, o) // stage 1
|
||||
const onsets = pickPeaks(odf, doc.sampleRate)
|
||||
if (onsets.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED')
|
||||
const { bpm } = this.estimateTempo(odf, o) // stage 2
|
||||
const { beats, confidence } = buildBeatGrid(onsets, bpm, doc.duration) // stage 3
|
||||
runsCounter.add(1); durationHist.record(span.duration)
|
||||
return { op: 'beat-detector', timestamp: 0, duration: doc.duration, bpm, beats, onsets, confidence }
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. File layout
|
||||
|
||||
```text
|
||||
packages/media/audio/audio-analysis-beat/
|
||||
package.json, tsconfig.json, vitest.config.ts
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts error-codes.test.ts
|
||||
detector/
|
||||
beat-detector.ts (+ .md .types.ts .test.ts) · beat-detector.schemas.ts · index.ts
|
||||
algorithms/
|
||||
energy-odf.ts (+ .md .test.ts)
|
||||
spectral-flux.ts (+ .md .test.ts)
|
||||
comb-tempo.ts (+ .md .test.ts)
|
||||
beat-grid.ts (+ .md .test.ts)
|
||||
index.ts
|
||||
__benchmarks__/ beat.bench.ts
|
||||
tests/
|
||||
integration.test.ts property.test.ts purity.test.ts exports.test.ts
|
||||
```
|
||||
|
||||
`detector/` owns the `BeatDetector` class, its `BeatDetectorOptions` / `BeatResult` types, and the
|
||||
Zod schemas. `algorithms/` holds the four pure numeric modules — each independently importable via a
|
||||
subpath export so a consumer that only wants the comb-filter estimator does not pull the whole
|
||||
package.
|
||||
|
||||
---
|
||||
|
||||
## 5. Integration notes
|
||||
|
||||
- **`@suaveplan/audio-core`** — `BeatDetector` consumes `AudioDocument` (page reads via the core's
|
||||
streaming API, never direct page poking), `BlockContext` for `analyzeBlock`, and the `AnalysisOp`
|
||||
interface. The `energy` ODF reuses the core's per-block energy statistics rather than recomputing
|
||||
them. The package never registers an op in an `OpRegistry`.
|
||||
- **`@suaveplan/error`** — `AudioError` is the base for every throw; the `AUDIO_ANALYSIS_*` codes
|
||||
are registered through `@suaveplan/error`'s `defineCodes` registry.
|
||||
- **`@suaveplan/opentelemetry`** — supplies the tracer, counter, and histogram. Instrumentation is
|
||||
inert without an exporter and never runs at import time.
|
||||
- **`@suaveplan/types`** — `LoggerLike` for the optional injected logger.
|
||||
- **Downstream consumers** — application/timeline code registers `BeatDetector` as a sidecar
|
||||
analysis op; it is not depended on by any other package in this 13-change batch.
|
||||
|
||||
---
|
||||
|
||||
## 6. Benchmarks
|
||||
|
||||
- `beat.bench.ts` — `analyze` over a synthetic 1-minute 48 kHz stereo track, once per algorithm.
|
||||
- `beat.bench.ts` — `analyze` over a synthetic 10-minute track to confirm linear scaling and bound
|
||||
per-minute scan cost.
|
||||
|
||||
---
|
||||
|
||||
## 7. Decisions and trade-offs
|
||||
|
||||
- **`energy` is the default, not `spectral-flux`.** The energy ODF reuses statistics `audio-core`
|
||||
already computes, so the default path adds almost no cost over a plain document scan. Callers who
|
||||
need timbral-onset precision opt into `spectral-flux` explicitly. This keeps the common case
|
||||
cheap and the precise case available.
|
||||
- **Hand-rolled FFT, no `AnalyserNode`.** The raw source material suggested a browser
|
||||
`AnalyserNode` for the FFT. That would break universal-tier purity (the package must run
|
||||
identically under Bun, Node, Deno, and browsers). The `spectral-flux` algorithm therefore ships a
|
||||
hand-rolled real FFT over `Float32Array` — slightly slower in a browser than the native node, but
|
||||
one code path everywhere and zero platform branching.
|
||||
- **Comb filter is a selectable algorithm, not the default.** Comb-filter tempo estimation is more
|
||||
robust on syncopated material but costs `O(bpmRange × odfLength)`. It is offered as
|
||||
`algorithm: 'comb'` rather than forced on every caller.
|
||||
- **No-beats is an error, not a zero-confidence result.** A scan that finds no onsets above
|
||||
threshold throws `AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED` rather than returning `{ bpm: 0, confidence:
|
||||
0 }`. A silent or non-rhythmic input is a caller-visible condition, and a thrown code is
|
||||
unmissable where a zero-confidence struct is easy to ignore.
|
||||
- **Octave-error resolution favours the perceptual centre.** Autocorrelation routinely detects
|
||||
half- or double-tempo. When two candidate periods are within a small ratio, the one whose BPM is
|
||||
closest to ~120 wins — a cheap heuristic that matches human tapping far more often than picking
|
||||
the raw strongest peak.
|
||||
Reference in New Issue
Block a user