Design: enterprise-audio-analysis-beat

Change ID: enterprise-audio-analysis-beat Package: @suaveplan/audio-analysis-beat

@suaveplan/audio-analysis-beat is a universal-tier Analysis Op plugin for the @suaveplan/audio-* family. It performs tempo (BPM) detection, beat-grid construction, and onset detection over AudioDocument streams. This document carries the package-specific design — algorithm rationale, code sketches, file layout, integration notes, and trade-offs. It deliberately does not restate the cross-cutting ecosystem model.

1. Relationship to the ecosystem architecture

The cross-cutting @suaveplan/audio-* architecture — the three-layer separation, the six plugin slots, the zero-duplication policy over genesis primitives, and the universal-purity rule — is defined once in enterprise-audio-core/design.md §1–§4 and is referenced, not duplicated here.

Two facts from that document anchor this package:

Analysis Op is plugin slot 3. Per enterprise-audio-core/design.md §1.3, an Analysis Op "reads audio and emits features (BPM, key, spectrum); runs as a sidecar scan, not part of the render pipeline." BeatDetector is exactly such a sidecar: it never registers in an OpRegistry plan or process slot, never mutates the AudioDocument, and runs concurrently with playback or as a standalone batch scan.
The AnalysisOp interface and AnalysisResult shape are defined by the audio ecosystem and consumed here unchanged. BeatDetector implements AnalysisOp; BeatResult extends AnalysisResult.

Everything below is specific to beat/tempo/onset detection.

2. Detection pipeline

Beat detection is a three-stage pipeline regardless of which algorithm is selected:

AudioDocument ──► [1] Onset Detection Function ──► [2] Tempo Estimation ──► [3] Beat Tracking
                      (per-block ODF curve)          (dominant period)        (phase + grid)

2.1. Stage 1 — Onset Detection Function (ODF)

The ODF is a 1-D curve, one value per analysis block, that peaks where the signal's energy or spectral content rises sharply — i.e. at note onsets. Two ODF algorithms ship; a third algorithm (comb) reuses the energy ODF and differs only in stage 2.

energy (default). The cheapest ODF. It consumes the per-block energy statistics already produced by @suaveplan/audio-core and takes the half-wave-rectified first difference:

// energy-odf.ts — sketch
export function energyOdf(blockEnergies: Float64Array): Float64Array {
  const odf = new Float64Array(blockEnergies.length)
  for (let i = 1; i < blockEnergies.length; i++) {
    const d = blockEnergies[i] - blockEnergies[i - 1]
    odf[i] = d > 0 ? d : 0 // half-wave rectify: onsets are energy increases
  }
  return odf
}

It is fast and allocation-light but blurs onsets that are loud in timbre yet quiet in total energy (e.g. a hi-hat over a sustained pad).

spectral-flux. Higher precision and the right default for streaming. A hand-rolled real FFT (no AnalyserNode, no Node bindings — universal purity, see the spec delta) produces a magnitude spectrum per frame; the ODF is the sum of positive bin-to-bin differences:

// spectral-flux.ts — sketch
export function spectralFlux(frame: Float32Array, prevMag: Float32Array, fft: RealFft): number {
  const mag = fft.magnitude(frame) // |X[k]|
  let flux = 0
  for (let k = 0; k < mag.length; k++) {
    const d = mag[k] - prevMag[k]
    if (d > 0) flux += d // only rising bins count toward an onset
    prevMag[k] = mag[k]
  }
  return flux
}

Spectral flux catches timbral onsets the energy ODF misses, at the cost of an FFT per frame.

In both cases, peaks in the ODF above an adaptive threshold (a moving median plus a fixed margin) are reported as onsets — timestamps in seconds, strictly increasing.

2.2. Stage 2 — Tempo estimation

Given the ODF, the dominant periodicity is the tempo. Two strategies:

Autocorrelation (used by energy and spectral-flux). Autocorrelate the ODF; the lag of the strongest peak whose implied BPM falls inside [minBpm, maxBpm] is the period. Octave errors (detecting half- or double-tempo) are resolved by preferring the candidate closest to the perceptual centre (~120 BPM) when two peaks are within a small ratio.
Comb filter (the comb algorithm). A bank of comb filters, one per candidate BPM in the range, is convolved with the ODF; the filter with the highest resonance output wins. This is more robust on syncopated material than raw autocorrelation but costs O(range × odfLength).

// comb-tempo.ts — sketch
export function combTempo(odf: Float64Array, sr: number, hop: number, lo: number, hi: number) {
  let bestBpm = lo, bestScore = -Infinity
  for (let bpm = lo; bpm <= hi; bpm++) {
    const period = Math.round((60 / bpm) * (sr / hop)) // ODF samples per beat
    let score = 0
    for (let i = period; i < odf.length; i++) score += odf[i] * odf[i - period]
    if (score > bestScore) { bestScore = score; bestBpm = bpm }
  }
  return { bpm: bestBpm, score: bestScore }
}

2.3. Stage 3 — Beat tracking and grid construction

With a period known, beat-grid.ts finds the phase: the offset that, when a 60 / bpm-spaced grid is laid down, minimises total onset-to-nearest-gridline distance. The output beats array is that phase-aligned grid clipped to the document duration. confidence is derived from how tightly the detected onsets cluster around the grid (a sharp cluster → high confidence; scattered onsets → low).

3. The three entry points

BeatDetector exposes the full AnalysisOp surface; each entry point reuses the same three-stage pipeline at a different granularity.

analyze(doc, opts?) — batch scan. Walks the whole document through stages 1–3 and returns a single settled BeatResult. Wrapped in the audio.analysis.beat OTel span.
analyzeStream(doc, opts?) — incremental scan. Runs stage 1 block-by-block and re-runs stages 2–3 on the ODF accumulated so far, yielding a refining BeatResult after each window. For a stable-tempo signal, confidence is non-decreasing across yields and the final yield's bpm equals analyze's.
analyzeBlock(block, ctx) — real-time. Computes the ODF value for a single block, updates a running tempo estimate, and returns synchronously. Allocation-light enough to call once per audio block from a visualiser's render loop.

// beat-detector.ts — analyze sketch
async analyze(doc: AudioDocument, opts?: BeatDetectorOptions): Promise<BeatResult> {
  const o = BeatDetectorOptionsSchema.parse({ ...this.opts, ...opts })
  if (o.signal?.aborted) throw new AudioError('AUDIO_ANALYSIS_BEAT.ABORTED')
  if (doc.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.EMPTY_DOCUMENT')
  return tracer.startActiveSpan('audio.analysis.beat', async (span) => {
    const odf = await this.buildOdf(doc, o)             // stage 1
    const onsets = pickPeaks(odf, doc.sampleRate)
    if (onsets.length === 0) throw new AudioError('AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED')
    const { bpm } = this.estimateTempo(odf, o)          // stage 2
    const { beats, confidence } = buildBeatGrid(onsets, bpm, doc.duration) // stage 3
    runsCounter.add(1); durationHist.record(span.duration)
    return { op: 'beat-detector', timestamp: 0, duration: doc.duration, bpm, beats, onsets, confidence }
  })
}

4. File layout

packages/media/audio/audio-analysis-beat/
  package.json, tsconfig.json, vitest.config.ts
  src/
    index.ts
    error-codes.ts  error-codes.test.ts
    detector/
      beat-detector.ts (+ .md .types.ts .test.ts) · beat-detector.schemas.ts · index.ts
    algorithms/
      energy-odf.ts (+ .md .test.ts)
      spectral-flux.ts (+ .md .test.ts)
      comb-tempo.ts (+ .md .test.ts)
      beat-grid.ts (+ .md .test.ts)
      index.ts
    __benchmarks__/  beat.bench.ts
  tests/
    integration.test.ts  property.test.ts  purity.test.ts  exports.test.ts

detector/ owns the BeatDetector class, its BeatDetectorOptions / BeatResult types, and the Zod schemas. algorithms/ holds the four pure numeric modules — each independently importable via a subpath export so a consumer that only wants the comb-filter estimator does not pull the whole package.

5. Integration notes

@suaveplan/audio-core — BeatDetector consumes AudioDocument (page reads via the core's streaming API, never direct page poking), BlockContext for analyzeBlock, and the AnalysisOp interface. The energy ODF reuses the core's per-block energy statistics rather than recomputing them. The package never registers an op in an OpRegistry.
@suaveplan/error — AudioError is the base for every throw; the AUDIO_ANALYSIS_* codes are registered through @suaveplan/error's defineCodes registry.
@suaveplan/opentelemetry — supplies the tracer, counter, and histogram. Instrumentation is inert without an exporter and never runs at import time.
@suaveplan/types — LoggerLike for the optional injected logger.
Downstream consumers — application/timeline code registers BeatDetector as a sidecar analysis op; it is not depended on by any other package in this 13-change batch.

6. Benchmarks

beat.bench.ts — analyze over a synthetic 1-minute 48 kHz stereo track, once per algorithm.
beat.bench.ts — analyze over a synthetic 10-minute track to confirm linear scaling and bound per-minute scan cost.

7. Decisions and trade-offs

energy is the default, not spectral-flux. The energy ODF reuses statistics audio-core already computes, so the default path adds almost no cost over a plain document scan. Callers who need timbral-onset precision opt into spectral-flux explicitly. This keeps the common case cheap and the precise case available.
Hand-rolled FFT, no AnalyserNode. The raw source material suggested a browser AnalyserNode for the FFT. That would break universal-tier purity (the package must run identically under Bun, Node, Deno, and browsers). The spectral-flux algorithm therefore ships a hand-rolled real FFT over Float32Array — slightly slower in a browser than the native node, but one code path everywhere and zero platform branching.
Comb filter is a selectable algorithm, not the default. Comb-filter tempo estimation is more robust on syncopated material but costs O(bpmRange × odfLength). It is offered as algorithm: 'comb' rather than forced on every caller.
No-beats is an error, not a zero-confidence result. A scan that finds no onsets above threshold throws AUDIO_ANALYSIS_BEAT.NO_BEATS_DETECTED rather than returning { bpm: 0, confidence: 0 }. A silent or non-rhythmic input is a caller-visible condition, and a thrown code is unmissable where a zero-confidence struct is easy to ignore.
Octave-error resolution favours the perceptual centre. Autocorrelation routinely detects half- or double-tempo. When two candidate periods are within a small ratio, the one whose BPM is closest to ~120 wins — a cheap heuristic that matches human tapping far more often than picking the raw strongest peak.

11 KiB Raw Blame History Unescape Escape