# Design: enterprise-audio-analysis-beat **Change ID**: enterprise-audio-analysis-beat **Package**: `@suaveplan/audio-analysis-beat` `@suaveplan/audio-analysis-beat` is a universal-tier Analysis Op plugin for the `@suaveplan/audio-*` family. It performs tempo (BPM) detection, beat-grid construction, and onset detection over `AudioDocument` streams. This document carries the **package-specific design** — algorithm rationale, code sketches, file layout, integration notes, and trade-offs. It deliberately does not restate the cross-cutting ecosystem model. --- ## 1. Relationship to the ecosystem architecture The cross-cutting `@suaveplan/audio-*` architecture — the three-layer separation, the six plugin slots, the zero-duplication policy over genesis primitives, and the universal-purity rule — is defined once in `enterprise-audio-core/design.md` §1–§4 and is **referenced, not duplicated** here. Two facts from that document anchor this package: - **Analysis Op is plugin slot 3.** Per `enterprise-audio-core/design.md` §1.3, an Analysis Op "reads audio and emits features (BPM, key, spectrum); runs as a sidecar scan, not part of the render pipeline." `BeatDetector` is exactly such a sidecar: it never registers in an `OpRegistry` plan or process slot, never mutates the `AudioDocument`, and runs concurrently with playback or as a standalone batch scan. - **The `AnalysisOp` interface and `AnalysisResult` shape** are defined by the audio ecosystem and consumed here unchanged. `BeatDetector implements AnalysisOp`; `BeatResult extends AnalysisResult`. Everything below is specific to beat/tempo/onset detection. --- ## 2. Detection pipeline Beat detection is a three-stage pipeline regardless of which algorithm is selected: ```text AudioDocument ──► [1] Onset Detection Function ──► [2] Tempo Estimation ──► [3] Beat Tracking (per-block ODF curve) (dominant period) (phase + grid) ``` ### 2.1. Stage 1 — Onset Detection Function (ODF) The ODF is a 1-D curve, one value per analysis block, that peaks where the signal's energy or spectral content rises sharply — i.e. at note onsets. Two ODF algorithms ship; a third algorithm (`comb`) reuses the energy ODF and differs only in stage 2. **`energy` (default).** The cheapest ODF. It consumes the per-block energy statistics already produced by `@suaveplan/audio-core` and takes the half-wave-rectified first difference: ```ts // energy-odf.ts — sketch export function energyOdf(blockEnergies: Float64Array): Float64Array { const odf = new Float64Array(blockEnergies.length) for (let i = 1; i < blockEnergies.length; i++) { const d = blockEnergies[i] - blockEnergies[i - 1] odf[i] = d > 0 ? d : 0 // half-wave rectify: onsets are energy increases } return odf } ``` It is fast and allocation-light but blurs onsets that are loud in timbre yet quiet in total energy (e.g. a hi-hat over a sustained pad). **`spectral-flux`.** Higher precision and the right default for streaming. A hand-rolled real FFT (no `AnalyserNode`, no Node bindings — universal purity, see the spec delta) produces a magnitude spectrum per frame; the ODF is the sum of positive bin-to-bin differences: ```ts // spectral-flux.ts — sketch export function spectralFlux(frame: Float32Array, prevMag: Float32Array, fft: RealFft): number { const mag = fft.magnitude(frame) // |X[k]| let flux = 0 for (let k = 0; k < mag.length; k++) { const d = mag[k] - prevMag[k] if (d > 0) flux += d // only rising bins count toward an onset prevMag[k] = mag[k] } return flux } ``` Spectral flux catches timbral onsets the energy ODF misses, at the cost of an FFT per frame. In both cases, peaks in the ODF above an adaptive threshold (a moving median plus a fixed margin) are reported as `onsets` — timestamps in seconds, strictly increasing. ### 2.2. Stage 2 — Tempo estimation Given the ODF, the dominant periodicity is the tempo. Two strategies: - **Autocorrelation** (used by `energy` and `spectral-flux`). Autocorrelate the ODF; the lag of the strongest peak whose implied BPM falls inside `[minBpm, maxBpm]` is the period. Octave errors (detecting half- or double-tempo) are resolved by preferring the candidate closest to the perceptual centre (~120 BPM) when two peaks are within a small ratio. - **Comb filter** (the `comb` algorithm). A bank of comb filters, one per candidate BPM in the range, is convolved with the ODF; the filter with the highest resonance output wins. This is more robust on syncopated material than raw autocorrelation but costs `O(range × odfLength)`. ```ts // comb-tempo.ts — sketch export function combTempo(odf: Float64Array, sr: number, hop: number, lo: number, hi: number) { let bestBpm = lo, bestScore = -Infinity for (let bpm = lo; bpm <= hi; bpm++) { const period = Math.round((60 / bpm) * (sr / hop)) // ODF samples per beat let score = 0 for (let i = period; i < odf.length; i++) score += odf[i] * odf[i - period] if (score > bestScore) { bestScore = score; bestBpm = bpm } } return { bpm: bestBpm, score: bestScore } } ``` ### 2.3. Stage 3 — Beat tracking and grid construction With a period known, `beat-grid.ts` finds the phase: the offset that, when a `60 / bpm`-spaced grid is laid down, minimises total onset-to-nearest-gridline distance. The output `beats` array is that phase-aligned grid clipped to the document duration. `confidence` is derived from how tightly the detected onsets cluster around the grid (a sharp cluster → high confidence; scattered onsets → low). --- ## 3. The three entry points `BeatDetector` exposes the full `AnalysisOp` surface; each entry point reuses the same three-stage pipeline at a different granularity. - **`analyze(doc, opts?)`** — batch scan. Walks the whole document through stages 1–3 and returns a single settled `BeatResult`. Wrapped in the `audio.analysis.beat` OTel span. - **`analyzeStream(doc, opts?)`** — incremental scan. Runs stage 1 block-by-block and re-runs stages 2–3 on the ODF accumulated so far, yielding a refining `BeatResult` after each window. For a stable-tempo signal, `confidence` is non-decreasing across yields and the final yield's `bpm` equals `analyze`'s. - **`analyzeBlock(block, ctx)`** — real-time. Computes the ODF value for a single block, updates a running tempo estimate, and returns synchronously. Allocation-light enough to call once per audio block from a visualiser's render loop. ```ts // beat-detector.ts — analyze sketch async analyze(doc: AudioDocument, opts?: BeatDetectorOptions): Promise { const o = BeatDetectorOptionsSchema.parse({ ...this.opts, ...opts }) if (o.signal?.aborted) throw new AudioError('AUDIO_ANALYSIS_BEAT.ABORTED') if (doc.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.EMPTY_DOCUMENT') return tracer.startActiveSpan('audio.analysis.beat', async (span) => { const odf = await this.buildOdf(doc, o) // stage 1 const onsets = pickPeaks(odf, doc.sampleRate) if (onsets.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED') const { bpm } = this.estimateTempo(odf, o) // stage 2 const { beats, confidence } = buildBeatGrid(onsets, bpm, doc.duration) // stage 3 runsCounter.add(1); durationHist.record(span.duration) return { op: 'beat-detector', timestamp: 0, duration: doc.duration, bpm, beats, onsets, confidence } }) } ``` --- ## 4. File layout ```text packages/media/audio/audio-analysis-beat/ package.json, tsconfig.json, vitest.config.ts src/ index.ts error-codes.ts error-codes.test.ts detector/ beat-detector.ts (+ .md .types.ts .test.ts) · beat-detector.schemas.ts · index.ts algorithms/ energy-odf.ts (+ .md .test.ts) spectral-flux.ts (+ .md .test.ts) comb-tempo.ts (+ .md .test.ts) beat-grid.ts (+ .md .test.ts) index.ts __benchmarks__/ beat.bench.ts tests/ integration.test.ts property.test.ts purity.test.ts exports.test.ts ``` `detector/` owns the `BeatDetector` class, its `BeatDetectorOptions` / `BeatResult` types, and the Zod schemas. `algorithms/` holds the four pure numeric modules — each independently importable via a subpath export so a consumer that only wants the comb-filter estimator does not pull the whole package. --- ## 5. Integration notes - **`@suaveplan/audio-core`** — `BeatDetector` consumes `AudioDocument` (page reads via the core's streaming API, never direct page poking), `BlockContext` for `analyzeBlock`, and the `AnalysisOp` interface. The `energy` ODF reuses the core's per-block energy statistics rather than recomputing them. The package never registers an op in an `OpRegistry`. - **`@suaveplan/error`** — `AudioError` is the base for every throw; the `AUDIO_ANALYSIS_*` codes are registered through `@suaveplan/error`'s `defineCodes` registry. - **`@suaveplan/opentelemetry`** — supplies the tracer, counter, and histogram. Instrumentation is inert without an exporter and never runs at import time. - **`@suaveplan/types`** — `LoggerLike` for the optional injected logger. - **Downstream consumers** — application/timeline code registers `BeatDetector` as a sidecar analysis op; it is not depended on by any other package in this 13-change batch. --- ## 6. Benchmarks - `beat.bench.ts` — `analyze` over a synthetic 1-minute 48 kHz stereo track, once per algorithm. - `beat.bench.ts` — `analyze` over a synthetic 10-minute track to confirm linear scaling and bound per-minute scan cost. --- ## 7. Decisions and trade-offs - **`energy` is the default, not `spectral-flux`.** The energy ODF reuses statistics `audio-core` already computes, so the default path adds almost no cost over a plain document scan. Callers who need timbral-onset precision opt into `spectral-flux` explicitly. This keeps the common case cheap and the precise case available. - **Hand-rolled FFT, no `AnalyserNode`.** The raw source material suggested a browser `AnalyserNode` for the FFT. That would break universal-tier purity (the package must run identically under Bun, Node, Deno, and browsers). The `spectral-flux` algorithm therefore ships a hand-rolled real FFT over `Float32Array` — slightly slower in a browser than the native node, but one code path everywhere and zero platform branching. - **Comb filter is a selectable algorithm, not the default.** Comb-filter tempo estimation is more robust on syncopated material but costs `O(bpmRange × odfLength)`. It is offered as `algorithm: 'comb'` rather than forced on every caller. - **No-beats is an error, not a zero-confidence result.** A scan that finds no onsets above threshold throws `AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED` rather than returning `{ bpm: 0, confidence: 0 }`. A silent or non-rhythmic input is a caller-visible condition, and a thrown code is unmissable where a zero-confidence struct is easy to ignore. - **Octave-error resolution favours the perceptual centre.** Autocorrelation routinely detects half- or double-tempo. When two candidate periods are within a small ratio, the one whose BPM is closest to ~120 wins — a cheap heuristic that matches human tapping far more often than picking the raw strongest peak.