Initial Commit
This commit is contained in:
@@ -0,0 +1,209 @@
|
||||
# Design: enterprise-audio-analysis-pitch
|
||||
|
||||
**Change ID**: enterprise-audio-analysis-pitch
|
||||
**Package**: `@suaveplan/audio-analysis-pitch`
|
||||
|
||||
`@suaveplan/audio-analysis-pitch` is a universal-tier Analysis Op package in the `@suaveplan/audio-*`
|
||||
family. It detects fundamental frequency (pitch), musical key, and chord progressions from
|
||||
`AudioDocument` streams. This document carries the **package-specific design**: algorithm rationale,
|
||||
code sketches, file layout, and trade-offs.
|
||||
|
||||
The **cross-cutting architecture** for the whole audio ecosystem — the three-layer separation, the
|
||||
six plugin slots, the `AnalysisOp` contract, genesis-package reuse, and the `AudioError` hierarchy —
|
||||
lives in `enterprise-audio-core/design.md` §1–§4 and is **referenced, not duplicated** here. In
|
||||
particular: an Analysis Op is **plugin slot 3** — a sidecar processor that reads audio and emits
|
||||
features, running concurrently with playback or as a standalone scan, never part of the render
|
||||
pipeline and never mutating the `AudioDocument`.
|
||||
|
||||
> **Scope note.** Pitch *detection / analysis* lives here. Pitch *shifting* — DSP that changes the
|
||||
> pitch of audio — is **not** in this package; it ships as the `PitchShifterPlugin` in
|
||||
> `@suaveplan/audio-plugins-fx` (an FX plugin, slot 6). This package only measures pitch; it never
|
||||
> alters it.
|
||||
|
||||
---
|
||||
|
||||
## 1. Package responsibilities
|
||||
|
||||
Three Analysis Op plugins, each implementing the `AnalysisOp` interface from `@suaveplan/audio-core`:
|
||||
|
||||
| Plugin | `name` | Output | Core algorithms |
|
||||
|---|---|---|---|
|
||||
| `PitchDetector` | `pitch-detector` | `PitchResult` (`pitch`, `midiNote`, `confidence`) | YIN, McLeod (MPM), cepstrum |
|
||||
| `KeyDetector` | `key-detector` | `KeyResult` (`key`, `confidence`, `chromagram`) | chromagram + Krumhansl-Schmuckler / Temperley |
|
||||
| `ChordDetector` | `chord-detector` | `ChordResult` (`chords: Chord[]`) | per-frame chromagram + template matching + smoothing |
|
||||
|
||||
All three support the full `AnalysisOp` triad where it makes sense: `analyze` (whole-document,
|
||||
blocking), `analyzeStream` (async generator, refines as audio is consumed), and — for
|
||||
`PitchDetector` — `analyzeBlock` (synchronous, single block, for real-time visualisation). Key and
|
||||
chord estimation need a longer observation window than a single block to be meaningful, so
|
||||
`KeyDetector` and `ChordDetector` expose `analyze` + `analyzeStream` only.
|
||||
|
||||
---
|
||||
|
||||
## 2. Pitch detection algorithms
|
||||
|
||||
### 2.1. YIN (default)
|
||||
|
||||
YIN (de Cheveigné & Kawahara, 2002) is the default because it is robust, has low octave-error rates,
|
||||
and needs no FFT. The pipeline over a frame of samples:
|
||||
|
||||
1. **Difference function** `d(τ) = Σ (x[j] − x[j+τ])²` for lag `τ` in the range implied by
|
||||
`[minFreq, maxFreq]`.
|
||||
2. **Cumulative mean normalised difference** `d'(τ) = d(τ) / ((1/τ) Σ d(j))`, with `d'(0) = 1`.
|
||||
3. **Absolute threshold** — pick the first `τ` where `d'(τ)` drops below ~0.1; if none, take the
|
||||
global minimum.
|
||||
4. **Parabolic interpolation** around the chosen `τ` for sub-sample precision.
|
||||
5. `pitch = sampleRate / τ`; `confidence = 1 − d'(τ)`.
|
||||
|
||||
```ts
|
||||
function yinPitch(frame: Float32Array, sampleRate: number, minF: number, maxF: number): PitchEstimate {
|
||||
const tauMin = Math.floor(sampleRate / maxF)
|
||||
const tauMax = Math.min(Math.ceil(sampleRate / minF), frame.length >> 1)
|
||||
const dprime = cumulativeMeanNormalisedDifference(frame, tauMin, tauMax)
|
||||
const tau = absoluteThreshold(dprime, 0.1)
|
||||
if (tau < 0) return { pitch: 0, confidence: 0 }
|
||||
const refined = parabolicInterpolate(dprime, tau)
|
||||
return { pitch: sampleRate / refined, confidence: 1 - dprime[tau] }
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2. McLeod Pitch Method (MPM)
|
||||
|
||||
MPM uses the normalised square difference function (NSDF) and key-maxima picking. It is stronger than
|
||||
YIN on signals with strong harmonics and slightly better at clean musical instrument tones, at higher
|
||||
cost. Selected via `{ algorithm: "mcleod" }`.
|
||||
|
||||
### 2.3. Cepstrum
|
||||
|
||||
The cepstrum method takes the inverse FFT of the log magnitude spectrum and finds the quefrency peak.
|
||||
It is cheap when an FFT is already on hand and resilient to formant structure (useful for voice), but
|
||||
has coarser resolution than YIN/MPM. Selected via `{ algorithm: "cepstrum" }`. The package ships its
|
||||
own small radix-2 real FFT for this path so it carries **no hard dependency** on
|
||||
`@suaveplan/audio-analysis-spectrum`.
|
||||
|
||||
### 2.4. MIDI mapping and the no-pitch case
|
||||
|
||||
`midiNote = round(69 + 12 * log2(pitch / 440))`. When no periodicity clears the algorithm's
|
||||
confidence floor (silence, broadband noise), the detector returns `pitch: 0`, `midiNote: -1`,
|
||||
`confidence: 0` rather than guessing.
|
||||
|
||||
---
|
||||
|
||||
## 3. Key detection
|
||||
|
||||
### 3.1. Chromagram
|
||||
|
||||
A chromagram folds spectral energy into 12 pitch classes. Per analysis frame: window → real FFT →
|
||||
magnitude spectrum → map each bin's frequency to a pitch class via
|
||||
`pitchClass = round(12 * log2(freq / C0)) mod 12` → accumulate magnitude into the bin. Frames are
|
||||
summed (or averaged) across the analysed range to produce one aggregate 12-bin `Float32Array`.
|
||||
|
||||
### 3.2. Krumhansl-Schmuckler key-finding
|
||||
|
||||
The aggregate chromagram is correlated against all 24 major/minor **key profiles** (each profile is a
|
||||
12-vector of perceptual pitch-class weights, rotated to each tonic). The default profile set is
|
||||
Krumhansl-Schmuckler; `{ profile: "temperley" }` swaps in Temperley's revised weights, which reduce
|
||||
relative-major/minor confusion. The best Pearson correlation wins; `confidence` is the normalised
|
||||
margin between the top score and the runner-up.
|
||||
|
||||
```ts
|
||||
function detectKey(chroma: Float32Array, profiles: KeyProfileSet): KeyEstimate {
|
||||
let best = { key: "", score: -Infinity }
|
||||
let second = -Infinity
|
||||
for (const { name, vector } of profiles.all()) {
|
||||
const score = pearson(chroma, vector)
|
||||
if (score > best.score) { second = best.score; best = { key: name, score } }
|
||||
else if (score > second) { second = score }
|
||||
}
|
||||
return { key: best.key, confidence: clamp01((best.score - second) / Math.max(best.score, 1e-6)) }
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3. Chord progression
|
||||
|
||||
`ChordDetector` computes a chromagram **per frame** (no aggregation), template-matches each frame
|
||||
against major / minor / diminished / augmented triad templates rotated to each root (48 templates),
|
||||
and applies temporal smoothing — a median/Viterbi-style pass over the frame labels — so a single
|
||||
noisy frame cannot spawn a spurious chord. Runs of identical labels are merged into one `Chord`
|
||||
record `{ timestamp, duration, root, quality, confidence }`; no emitted chord is shorter than the
|
||||
smoothing window.
|
||||
|
||||
---
|
||||
|
||||
## 4. Streaming refinement
|
||||
|
||||
`analyzeStream` for all three detectors consumes `doc.stream()` block-by-block. Pitch yields a result
|
||||
per frame. Key and chord detectors maintain a running chromagram (or running frame buffer) and re-run
|
||||
the estimator on each yield, so `confidence` is non-decreasing in expectation as more audio arrives.
|
||||
An `AbortSignal` is threaded through the options object: pre-aborted → throw `AUDIO_ANALYSIS_PITCH.ABORTED`
|
||||
before any block work; aborted mid-stream → throw at the next block boundary. This mirrors the
|
||||
`AbortSignal` contract `audio-core` defines for `AudioDocument.stream()`.
|
||||
|
||||
---
|
||||
|
||||
## 5. File layout
|
||||
|
||||
```text
|
||||
packages/media/audio/audio-analysis-pitch/
|
||||
package.json, tsconfig.json, vitest.config.ts
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts (+ .md .test.ts)
|
||||
detector/
|
||||
pitch-detector.ts (+ .md .types.ts .schemas.ts .test.ts)
|
||||
key-detector.ts (+ .md .types.ts .schemas.ts .test.ts)
|
||||
chord-detector.ts (+ .md .types.ts .schemas.ts .test.ts)
|
||||
index.ts
|
||||
algorithms/
|
||||
yin.ts (+ .md .test.ts) — YIN + McLeod + cepstrum estimators
|
||||
chromagram.ts (+ .md .test.ts) — pitch-class folding + small real FFT
|
||||
key-finding.ts (+ .md .test.ts) — Krumhansl-Schmuckler + Temperley profiles
|
||||
chord-matching.ts (+ .md .test.ts) — triad templates + temporal smoothing
|
||||
index.ts
|
||||
__benchmarks__/
|
||||
pitch.bench.ts
|
||||
tests/
|
||||
integration.test.ts property.test.ts purity.test.ts exports.test.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Benchmarks
|
||||
|
||||
- `pitch.bench.ts` — `PitchDetector.analyze` over a 1-minute 48 kHz mono track (all three
|
||||
algorithms), and `KeyDetector.analyze` + `ChordDetector.analyze` over the same track.
|
||||
|
||||
---
|
||||
|
||||
## 7. Integration notes
|
||||
|
||||
- **`@suaveplan/audio-core`** — provides `AudioDocument`, the `AnalysisOp` / `AnalysisResult`
|
||||
contract, `BlockContext`, and the `AudioError` base. `PitchResult`, `KeyResult`, and `ChordResult`
|
||||
all extend `AnalysisResult`.
|
||||
- **`@suaveplan/audio-analysis-spectrum`** — *optional* peer. When present, callers may feed its FFT
|
||||
output in; this package never imports it at runtime and ships its own FFT so it stays a leaf.
|
||||
- **`@suaveplan/opentelemetry`** — spans, counters, and histograms per detector run.
|
||||
- **Consumers** — application/timeline code registers these detectors as analysis ops; e.g.
|
||||
`audio-plugins-fx` auto-tune reads `PitchResult`, a React player surfaces `KeyResult.key`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Decisions and trade-offs
|
||||
|
||||
- **YIN is the default, three algorithms ship.** YIN is the best general-purpose default (no FFT,
|
||||
low octave error). MPM and cepstrum are offered because they win on specific material
|
||||
(harmonically rich instruments, voice). Per-call selection avoids forcing one trade-off on every
|
||||
caller.
|
||||
- **Own FFT, no hard `audio-analysis-spectrum` dependency.** A pitch package that could not run
|
||||
without the spectrum package would couple two leaves unnecessarily. The cepstrum path and the
|
||||
chromagram need only a small radix-2 real FFT, which this package vendors. `audio-analysis-spectrum`
|
||||
stays a *sibling*, not a *prerequisite*.
|
||||
- **Pitch shift is explicitly out of scope.** Detection and shifting are different concerns in
|
||||
different plugin slots. `PitchShifterPlugin` (phase-vocoder, stateful) belongs in
|
||||
`audio-plugins-fx`; this package is read-only over audio.
|
||||
- **Chord detection ships here, not in a separate package.** Key and chord estimation share the
|
||||
chromagram machinery and pitch-class theory; splitting them would duplicate that core. They live
|
||||
together behind separate `AnalysisOp` plugins.
|
||||
- **Temporal smoothing on chords is mandatory, not optional.** Raw per-frame chord labels are too
|
||||
noisy to be useful; the smoothing pass is part of the algorithm, not a tuning knob, so every
|
||||
caller gets a usable progression.
|
||||
@@ -0,0 +1,171 @@
|
||||
# OpenSpec Change: enterprise-audio-analysis-pitch
|
||||
|
||||
- **Change ID**: enterprise-audio-analysis-pitch
|
||||
- **Package**: `@suaveplan/audio-analysis-pitch`
|
||||
- **Category**: universal
|
||||
- **Target path**: `packages/media/audio/audio-analysis-pitch`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-14
|
||||
- **Design**: `./design.md`
|
||||
- **Spec delta**: `./specs/audio-analysis-pitch/spec.md`
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Deliver `@suaveplan/audio-analysis-pitch` on the genesis branch from scratch at enterprise quality. Pitch, key, and chord detection plugin for the universal audio analysis framework. Detects fundamental frequency, MIDI note numbers, musical key, and chord progressions from `AudioDocument` streams using YIN/McLeod pitch methods and chromagram-based key-finding.
|
||||
|
||||
Runs identically in browser and server via `BlockProcessor` or direct stream iteration.
|
||||
|
||||
---
|
||||
|
||||
## Current-state audit (genesis)
|
||||
|
||||
The package does not exist on genesis yet — this is a fresh build. No prior source, no test results, no coverage data.
|
||||
|
||||
---
|
||||
|
||||
## Available workspace deps on genesis
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@suaveplan/audio-core` (`workspace:^`) | `AudioDocument`, `BlockProcessor`, `BlockContext` |
|
||||
| `@suaveplan/audio-analysis-spectrum` (`workspace:^`) | FFT, chromagram (optional — can include own FFT) |
|
||||
| `@suaveplan/error` (`workspace:^`) | `AudioError` base class; `AUDIO_ANALYSIS.*` codes |
|
||||
| `@suaveplan/types` (`workspace:^`) | `LoggerLike` |
|
||||
| `@suaveplan/opentelemetry` (`workspace:^`) | OTel spans, counters, histograms |
|
||||
| `zod` (`catalog:forms`) | Zod schemas for option types |
|
||||
|
||||
Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
|
||||
|
||||
---
|
||||
|
||||
## Public API
|
||||
|
||||
### Core types
|
||||
|
||||
```ts
|
||||
interface PitchDetectorOptions {
|
||||
algorithm?: 'yin' | 'mcleod' | 'cepstrum' // default 'yin'
|
||||
minFreq?: number // default 20
|
||||
maxFreq?: number // default 20000
|
||||
}
|
||||
|
||||
interface PitchResult extends AnalysisResult {
|
||||
pitch: number // fundamental frequency (Hz)
|
||||
midiNote: number // MIDI note number
|
||||
confidence: number // 0..1
|
||||
}
|
||||
|
||||
interface KeyDetectorOptions {
|
||||
profile?: 'krumhansl-schmuckler' | 'temperley' // default 'krumhansl-schmuckler'
|
||||
}
|
||||
|
||||
interface KeyResult extends AnalysisResult {
|
||||
key: string // e.g. "C major", "F# minor"
|
||||
confidence: number
|
||||
chromagram: Float32Array // 12-bin pitch class distribution
|
||||
}
|
||||
|
||||
interface Chord {
|
||||
timestamp: number
|
||||
duration: number
|
||||
root: string
|
||||
quality: string // "major", "minor", "dim", etc.
|
||||
confidence: number
|
||||
}
|
||||
|
||||
interface ChordResult extends AnalysisResult {
|
||||
chords: Chord[]
|
||||
}
|
||||
|
||||
class PitchDetector implements AnalysisOp {
|
||||
readonly name = 'pitch-detector'
|
||||
analyze(doc: AudioDocument, opts?: PitchDetectorOptions): Promise<PitchResult>
|
||||
analyzeStream(doc: AudioDocument, opts?: PitchDetectorOptions): AsyncGenerator<PitchResult>
|
||||
analyzeBlock(block: Float32Array[], ctx: BlockContext): PitchResult
|
||||
}
|
||||
|
||||
class KeyDetector implements AnalysisOp {
|
||||
readonly name = 'key-detector'
|
||||
analyze(doc: AudioDocument, opts?: KeyDetectorOptions): Promise<KeyResult>
|
||||
analyzeStream(doc: AudioDocument, opts?: KeyDetectorOptions): AsyncGenerator<KeyResult>
|
||||
}
|
||||
|
||||
class ChordDetector implements AnalysisOp {
|
||||
readonly name = 'chord-detector'
|
||||
analyze(doc: AudioDocument): Promise<ChordResult>
|
||||
analyzeStream(doc: AudioDocument): AsyncGenerator<ChordResult>
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New additions for genesis
|
||||
|
||||
1. **Three pitch algorithms** — YIN, McLeod pitch method, cepstrum; selectable per detection call.
|
||||
2. **Key detection** — Krumhansl-Schmuckler and Temperley profiles on aggregated chromagram.
|
||||
3. **Chord progression** — template matching on per-frame chromagram with smoothing.
|
||||
4. **Streaming refinement** — pitch/key confidence improves as more audio is processed.
|
||||
5. **OTel instrumentation** — spans per detector, counters, histograms.
|
||||
6. **Property-based tests** — sine waves at known frequencies yield exact pitch, silence yields no pitch.
|
||||
7. **Benchmark suite** — analyze 1-minute track for pitch, key, and chords.
|
||||
|
||||
---
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/media/audio/audio-analysis-pitch/
|
||||
package.json
|
||||
tsconfig.json
|
||||
vitest.config.ts
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts
|
||||
detector/
|
||||
pitch-detector.ts
|
||||
pitch-detector.md
|
||||
pitch-detector.types.ts
|
||||
pitch-detector.test.ts
|
||||
key-detector.ts
|
||||
key-detector.md
|
||||
key-detector.types.ts
|
||||
key-detector.test.ts
|
||||
chord-detector.ts
|
||||
chord-detector.md
|
||||
chord-detector.types.ts
|
||||
chord-detector.test.ts
|
||||
index.ts
|
||||
algorithms/
|
||||
yin.ts
|
||||
yin.md
|
||||
yin.test.ts
|
||||
chromagram.ts
|
||||
chromagram.md
|
||||
chromagram.test.ts
|
||||
key-finding.ts
|
||||
key-finding.md
|
||||
key-finding.test.ts
|
||||
chord-matching.ts
|
||||
chord-matching.md
|
||||
chord-matching.test.ts
|
||||
index.ts
|
||||
__benchmarks__/
|
||||
pitch.bench.ts
|
||||
tests/
|
||||
integration.test.ts
|
||||
property.test.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
**Breaking changes**: none — fresh build, package does not yet exist on genesis.
|
||||
|
||||
**Prerequisites**: `@suaveplan/audio-core`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry` published to Verdaccio.
|
||||
|
||||
### Affected dependents
|
||||
|
||||
No dependents in this batch (verified by `bunx turbo ls --filter=...@suaveplan/audio-analysis-pitch`). `@suaveplan/audio-analysis-pitch` is a leaf Analysis Op plugin — no genesis package depends on it at the time of this change. It is consumed at runtime by application/timeline code registering it as an analysis op (e.g. `@suaveplan/audio-plugins-fx` auto-tune reading `PitchResult`, a React player surfacing `KeyResult.key`, `@suaveplan/audio-cli` batch analysis); those consumers land in later changes and are created against this package's spec delta, not retrofitted.
|
||||
@@ -0,0 +1,183 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: audio-analysis-pitch (audio-analysis-pitch)
|
||||
|
||||
**Package**: `@suaveplan/audio-analysis-pitch`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Universal-tier purity
|
||||
|
||||
The `@suaveplan/audio-analysis-pitch` package SHALL import zero browser-only APIs (`window`, `document`, `navigator`, `AudioContext`, `AudioBuffer`, `AnalyserNode`) and zero Node-only APIs (`node:fs`, `node:path`, `process`, `Buffer`). All environment-specific behaviour SHALL be reached only through `AudioDocument` and injected adapters. The package's `dependencies` SHALL be limited to `@suaveplan/audio-core`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`, `tslib`, and `zod`. The package SHALL ship its own pitch-domain FFT/autocorrelation so it has no hard dependency on `@suaveplan/audio-analysis-spectrum`.
|
||||
|
||||
#### Scenario: Built dist references no platform globals
|
||||
|
||||
- **WHEN** the built `dist/` is grep'd
|
||||
- **THEN** it MUST NOT reference `window`, `document`, `navigator`, `AudioContext`, `AnalyserNode`, `node:fs`, `node:path`, `process`, or `Buffer`
|
||||
|
||||
#### Scenario: Tests pass under both runtimes
|
||||
|
||||
- **WHEN** the suite runs under `bun test` (server) and under vitest with a `happy-dom` shim (browser)
|
||||
- **THEN** both runs pass with identical results
|
||||
|
||||
**Verified by:** `tests/purity.test.ts::"dist references no browser or Node globals"` AND `tests/purity.test.ts::"dependency set is limited to the declared workspace deps and zod"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: PitchDetector implements the AnalysisOp contract
|
||||
|
||||
The package SHALL export a `PitchDetector` class implementing the `AnalysisOp` interface from `@suaveplan/audio-core` (plugin slot 3, Analysis Op). It SHALL expose a readonly `name` of `"pitch-detector"`, a readonly `description`, and the three methods `analyze(doc, opts?)`, `analyzeStream(doc, opts?)`, and `analyzeBlock(block, ctx)`. `analyze` SHALL return a `Promise<PitchResult>` aggregated over the whole document; `analyzeStream` SHALL be an `AsyncGenerator<PitchResult>` whose `confidence` field is non-decreasing in expectation as more audio is consumed; `analyzeBlock` SHALL return a synchronous `PitchResult` for a single `Float32Array[]` block suitable for real-time visualisation. As an Analysis Op the detector SHALL run as a sidecar scan and SHALL NOT mutate the `AudioDocument` or participate in its render pipeline.
|
||||
|
||||
#### Scenario: analyze returns an aggregate PitchResult
|
||||
|
||||
- **WHEN** a caller `await new PitchDetector().analyze(doc)` on a document containing a steady tone
|
||||
- **THEN** the resolved `PitchResult` carries `op === "pitch-detector"`, a `pitch` in Hz, a `midiNote`, a `confidence` in `[0, 1]`, and `timestamp`/`duration` covering the analysed range
|
||||
|
||||
#### Scenario: analyzeBlock is synchronous and non-mutating
|
||||
|
||||
- **WHEN** a caller calls `analyzeBlock(block, ctx)` on a single block
|
||||
- **THEN** a `PitchResult` is returned synchronously, the input `block` arrays are unmodified, and no `AudioDocument` state is touched
|
||||
|
||||
**Verified by:** `src/detector/pitch-detector.test.ts::"analyze returns an aggregate PitchResult for a steady tone"` AND `src/detector/pitch-detector.test.ts::"analyzeBlock returns a synchronous result without mutating input"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Fundamental-frequency estimation algorithms
|
||||
|
||||
`PitchDetector` SHALL accept `PitchDetectorOptions` with `algorithm` (`"yin" | "mcleod" | "cepstrum"`, default `"yin"`), `minFreq` (default `20`), and `maxFreq` (default `20000`). It SHALL estimate the fundamental frequency of the dominant periodic component within `[minFreq, maxFreq]` and report it on `PitchResult.pitch`, with the corresponding `midiNote` derived as `69 + 12 * log2(pitch / 440)` rounded to the nearest integer. When the input has no detectable periodicity above the algorithm's confidence floor (e.g. silence or broadband noise), `pitch` SHALL be `0`, `midiNote` SHALL be `-1`, and `confidence` SHALL be `0`.
|
||||
|
||||
#### Scenario: Known sine frequency is estimated within tolerance
|
||||
|
||||
- **WHEN** `analyze` runs on a synthesised 440 Hz sine at 48 kHz with the default `"yin"` algorithm
|
||||
- **THEN** `pitch` is within ±1 Hz of `440`, `midiNote` is `69`, and `confidence` is greater than `0.9`
|
||||
|
||||
#### Scenario: Silence yields no pitch
|
||||
|
||||
- **WHEN** `analyze` runs on a fully silent document
|
||||
- **THEN** `pitch` is `0`, `midiNote` is `-1`, and `confidence` is `0`
|
||||
|
||||
#### Scenario: Algorithm selection is honoured
|
||||
|
||||
- **WHEN** `analyze` is called with `{ algorithm: "mcleod" }` and again with `{ algorithm: "cepstrum" }` on the same 440 Hz tone
|
||||
- **THEN** each run estimates `pitch` within ±2 Hz of `440` using the selected estimator
|
||||
|
||||
**Verified by:** `src/algorithms/yin.test.ts::"estimates a 440 Hz sine within one hertz"` AND `src/detector/pitch-detector.test.ts::"silence yields zero pitch and zero confidence"` AND `src/detector/pitch-detector.test.ts::"honours the algorithm option for mcleod and cepstrum"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: KeyDetector implements the AnalysisOp contract
|
||||
|
||||
The package SHALL export a `KeyDetector` class implementing the `AnalysisOp` interface (plugin slot 3, Analysis Op). It SHALL expose a readonly `name` of `"key-detector"`, a readonly `description`, and the methods `analyze(doc, opts?)` and `analyzeStream(doc, opts?)`. `analyze` SHALL return a `Promise<KeyResult>` and `analyzeStream` SHALL be an `AsyncGenerator<KeyResult>` whose estimate refines as more audio is consumed. The detector SHALL accumulate a 12-bin chromagram over the analysed range and SHALL NOT mutate the `AudioDocument`.
|
||||
|
||||
#### Scenario: analyze returns a KeyResult with a 12-bin chromagram
|
||||
|
||||
- **WHEN** a caller `await new KeyDetector().analyze(doc)` on a tonal document
|
||||
- **THEN** the resolved `KeyResult` carries `op === "key-detector"`, a `key` string, a `confidence` in `[0, 1]`, and a `chromagram` `Float32Array` of length `12`
|
||||
|
||||
**Verified by:** `src/detector/key-detector.test.ts::"analyze returns a KeyResult with a length-12 chromagram"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Key and chord estimation outputs
|
||||
|
||||
`KeyDetector` SHALL accept `KeyDetectorOptions` with `profile` (`"krumhansl-schmuckler" | "temperley"`, default `"krumhansl-schmuckler"`). It SHALL correlate the aggregated chromagram against all 24 major/minor key profiles for the selected weighting and report the best-scoring key on `KeyResult.key` as a pitch-class name plus mode (e.g. `"C major"`, `"F# minor"`), with the normalised correlation margin on `KeyResult.confidence`. The package SHALL additionally export a `ChordDetector` `AnalysisOp` (`name === "chord-detector"`) whose `analyze` returns a `Promise<ChordResult>` carrying a time-ordered `chords` array of `Chord` records (`{ timestamp, duration, root, quality, confidence }`) produced by template-matching the per-frame chromagram against major/minor/diminished/augmented templates with temporal smoothing so transient frames do not produce spurious chords.
|
||||
|
||||
#### Scenario: A C-major signal is identified as C major
|
||||
|
||||
- **WHEN** `KeyDetector.analyze` runs on a document built from a C-major triad/scale with the default profile
|
||||
- **THEN** `key` equals `"C major"` and `confidence` is greater than `0.6`
|
||||
|
||||
#### Scenario: Key profile selection is honoured
|
||||
|
||||
- **WHEN** `analyze` is called with `{ profile: "temperley" }`
|
||||
- **THEN** the Temperley weighting is used to score the chromagram and the returned `key` reflects that profile
|
||||
|
||||
#### Scenario: Chord progression is time-ordered and smoothed
|
||||
|
||||
- **WHEN** `ChordDetector.analyze` runs on a document with a C-major then G-major passage
|
||||
- **THEN** `chords` is ordered by ascending `timestamp`, each `Chord.root`/`quality` names the matched chord, no `Chord` is shorter than the smoothing window, and adjacent identical chords are merged
|
||||
|
||||
**Verified by:** `src/algorithms/key-finding.test.ts::"identifies a C-major signal as C major"` AND `src/detector/key-detector.test.ts::"honours the temperley profile option"` AND `src/algorithms/chord-matching.test.ts::"emits a time-ordered smoothed chord progression"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Streaming refinement and AbortSignal propagation
|
||||
|
||||
`PitchDetector.analyzeStream`, `KeyDetector.analyzeStream`, and `ChordDetector.analyzeStream` SHALL accept an `AbortSignal` via their options object and yield progressively refined `AnalysisResult` values as document blocks are consumed. An already-aborted signal SHALL cause the generator to throw `AudioAnalysisError` with code `AUDIO_ANALYSIS_PITCH.ABORTED` before any block work; a signal aborted mid-stream SHALL terminate the generator at the next block boundary with the same error. `analyze` SHALL likewise honour an `AbortSignal` and reject with `AUDIO_ANALYSIS_PITCH.ABORTED`.
|
||||
|
||||
#### Scenario: Pre-aborted signal short-circuits the stream
|
||||
|
||||
- **WHEN** `analyzeStream` is called with an already-aborted signal
|
||||
- **THEN** it throws `AudioAnalysisError` whose `code === "AUDIO_ANALYSIS_PITCH.ABORTED"` before yielding any result
|
||||
|
||||
#### Scenario: Mid-stream abort terminates at the next block boundary
|
||||
|
||||
- **WHEN** a signal is aborted after the first partial result is yielded
|
||||
- **THEN** the generator throws `AUDIO_ANALYSIS_PITCH.ABORTED` at the next block boundary and yields nothing further
|
||||
|
||||
**Verified by:** `src/detector/pitch-detector.test.ts::"analyzeStream throws AUDIO_ANALYSIS_PITCH.ABORTED for a pre-aborted signal"` AND `src/detector/key-detector.test.ts::"analyzeStream honours a mid-stream abort"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Zod boundary validation
|
||||
|
||||
Every public options object — `PitchDetectorOptions` and `KeyDetectorOptions` — SHALL have a Zod schema exported from a sibling `*.schemas.ts`, and `PitchResult`, `KeyResult`, and `ChordResult` SHALL each have a Zod schema for output validation. Every public entry point (`analyze`, `analyzeStream`, `analyzeBlock`) SHALL validate its options argument against the schema before use. A validation failure SHALL throw `AudioAnalysisError` with code `AUDIO_ANALYSIS_PITCH.INVALID_OPTIONS` carrying the Zod issue list in `context`.
|
||||
|
||||
#### Scenario: Invalid options are rejected at the boundary
|
||||
|
||||
- **WHEN** `PitchDetector.analyze` is called with `{ minFreq: -1 }` or an unknown `algorithm`
|
||||
- **THEN** it throws `AudioAnalysisError` whose `code === "AUDIO_ANALYSIS_PITCH.INVALID_OPTIONS"` with the Zod issues in `context`
|
||||
|
||||
**Verified by:** `src/detector/pitch-detector.test.ts::"rejects invalid options with AUDIO_ANALYSIS_PITCH.INVALID_OPTIONS"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Structured error codes
|
||||
|
||||
The package SHALL export `AudioAnalysisPitchErrorCode`, `AUDIO_ANALYSIS_PITCH_ERROR_METADATA`, and `registerAudioAnalysisPitchErrorCodes()`, registering the codes `AUDIO_ANALYSIS_PITCH.INVALID_OPTIONS`, `AUDIO_ANALYSIS_PITCH.ABORTED`, `AUDIO_ANALYSIS_PITCH.UNSUPPORTED_FORMAT`, and `AUDIO_ANALYSIS_PITCH.INSUFFICIENT_DATA` with `@suaveplan/error`. Every error thrown by the package SHALL be an `AudioAnalysisError` extending `SuaveplanError` and carry one of these codes, `cause`, and `context`. `AUDIO_ANALYSIS_PITCH.UNSUPPORTED_FORMAT` SHALL be thrown when a document's channel layout or sample rate cannot be analysed; `AUDIO_ANALYSIS_PITCH.INSUFFICIENT_DATA` SHALL be thrown when the analysed range is shorter than the minimum window an algorithm requires. Every registered code SHALL be reachable from a real call site exercised by tests.
|
||||
|
||||
#### Scenario: Every registered code is thrown by a call site
|
||||
|
||||
- **WHEN** the test suite runs
|
||||
- **THEN** each code in `AUDIO_ANALYSIS_PITCH_ERROR_METADATA` is asserted thrown by at least one test
|
||||
|
||||
#### Scenario: Too-short input throws AUDIO_ANALYSIS_PITCH.INSUFFICIENT_DATA
|
||||
|
||||
- **WHEN** `PitchDetector.analyze` runs on a document shorter than one analysis window
|
||||
- **THEN** it throws `AudioAnalysisError` whose `code === "AUDIO_ANALYSIS_PITCH.INSUFFICIENT_DATA"`
|
||||
|
||||
**Verified by:** `src/error-codes.test.ts::"every AUDIO_ANALYSIS code is reachable from a call site"` AND `src/detector/pitch-detector.test.ts::"throws AUDIO_ANALYSIS_PITCH.INSUFFICIENT_DATA for a too-short document"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry instrumentation
|
||||
|
||||
The package SHALL instrument its public entry points via `@suaveplan/opentelemetry`: a span per detector run (`audio.analysis.pitch`, `audio.analysis.key`, `audio.analysis.chord`), a counter `audio_analysis_runs_total` tagged with `op`, and histograms `audio_analysis_duration_ms` and `audio_analysis_block_duration_ms` tagged with `op`. Instrumentation SHALL be a no-op when no exporter is registered and SHALL NOT run as an import-time side effect.
|
||||
|
||||
#### Scenario: Instrumentation is inert without an exporter
|
||||
|
||||
- **WHEN** the package is imported and used with no OTel exporter registered
|
||||
- **THEN** no error is thrown and no global state is mutated at import time
|
||||
|
||||
#### Scenario: A detector run records a span and increments the counter
|
||||
|
||||
- **WHEN** `PitchDetector.analyze` runs with an exporter registered
|
||||
- **THEN** an `audio.analysis.pitch` span is recorded, `audio_analysis_runs_total{op="pitch-detector"}` is incremented, and `audio_analysis_duration_ms` observes the run
|
||||
|
||||
**Verified by:** `tests/purity.test.ts::"importing the package mutates no global state"` AND `src/detector/pitch-detector.test.ts::"analyze records a span, counter, and histogram when an exporter is registered"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath exports per feature module
|
||||
|
||||
`package.json` SHALL declare one `exports` entry per feature module (`./detector/pitch-detector`, `./detector/key-detector`, `./detector/chord-detector`, `./algorithms/yin`, `./algorithms/chromagram`, `./algorithms/key-finding`, `./algorithms/chord-matching`) in addition to the root `.` barrel, each resolving to a typed `import` condition. `sideEffects` SHALL be `false`.
|
||||
|
||||
#### Scenario: Each feature module is independently importable
|
||||
|
||||
- **WHEN** a consumer imports `@suaveplan/audio-analysis-pitch/algorithms/yin`
|
||||
- **THEN** the import resolves to the YIN module without pulling the whole package barrel
|
||||
|
||||
**Verified by:** `tests/exports.test.ts::"every feature module has a subpath export entry"`
|
||||
@@ -0,0 +1,73 @@
|
||||
# Tasks: enterprise-audio-analysis-pitch
|
||||
|
||||
**Change ID**: enterprise-audio-analysis-pitch
|
||||
**Package**: `@suaveplan/audio-analysis-pitch`
|
||||
**Target path**: `packages/media/audio/audio-analysis-pitch`
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/audio-core`, `@suaveplan/error`, `@suaveplan/types`
|
||||
|
||||
---
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [ ] Create directory `packages/media/audio/audio-analysis-pitch/` with `src/`, `tests/`, `src/__benchmarks__/`, and all feature subdirectories per the layout in `proposal.md`.
|
||||
- [ ] Write `package.json` at version `0.1.0` with `"type": "module"`, `"sideEffects": false`, full `exports` map, deps: `@suaveplan/audio-core workspace:^`, `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/opentelemetry workspace:^`, `zod catalog:forms`.
|
||||
- [ ] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`.
|
||||
- [ ] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config`.
|
||||
- [ ] Add `lint`, `typecheck`, `test`, `test:coverage`, `build`, `docs:check` scripts.
|
||||
|
||||
---
|
||||
|
||||
## 2. Types + errors
|
||||
|
||||
- [ ] Write `src/error-codes.ts` — `PITCHErrorCode`, `AUDIO_ANALYSIS_PITCH_ERROR_METADATA`, `registerAudioAnalysisErrorCodes()`.
|
||||
- [ ] Write Zod schemas: `PitchDetectorOptionsSchema`, `PitchResultSchema`, `KeyDetectorOptionsSchema`, `KeyResultSchema`, `ChordResultSchema`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Algorithms
|
||||
|
||||
- [ ] Implement `yin.ts` — YIN pitch detection algorithm.
|
||||
- [ ] Write `yin.test.ts` and `yin.md` (≥200 words).
|
||||
- [ ] Implement `chromagram.ts` — 12-bin pitch class profile from magnitude spectrum.
|
||||
- [ ] Write `chromagram.test.ts` and `chromagram.md` (≥200 words).
|
||||
- [ ] Implement `key-finding.ts` — Krumhansl-Schmuckler and Temperley key profiles.
|
||||
- [ ] Write `key-finding.test.ts` and `key-finding.md` (≥200 words).
|
||||
- [ ] Implement `chord-matching.ts` — template matching for chord recognition.
|
||||
- [ ] Write `chord-matching.test.ts` and `chord-matching.md` (≥200 words).
|
||||
|
||||
---
|
||||
|
||||
## 4. Detectors
|
||||
|
||||
- [ ] Implement `PitchDetector` class with `analyze()`, `analyzeStream()`, `analyzeBlock()`.
|
||||
- [ ] Write `pitch-detector.test.ts` and `pitch-detector.md` (≥200 words).
|
||||
- [ ] Implement `KeyDetector` class with `analyze()`, `analyzeStream()`.
|
||||
- [ ] Write `key-detector.test.ts` and `key-detector.md` (≥200 words).
|
||||
- [ ] Implement `ChordDetector` class with `analyze()`, `analyzeStream()`.
|
||||
- [ ] Write `chord-detector.test.ts` and `chord-detector.md` (≥200 words).
|
||||
- [ ] Wire OTel: spans, counters, histograms per detector.
|
||||
|
||||
---
|
||||
|
||||
## 5. Benchmarks
|
||||
|
||||
- [ ] Write `src/__benchmarks__/pitch.bench.ts` — pitch detect 1-minute track, key detect 1-minute track.
|
||||
|
||||
---
|
||||
|
||||
## 6. Tests
|
||||
|
||||
- [ ] Write `tests/integration.test.ts` — full pitch/key/chord round-trip on real audio fixture.
|
||||
- [ ] Write `tests/property.test.ts` (fast-check) — sine wave frequency → exact pitch, confidence in [0,1].
|
||||
|
||||
---
|
||||
|
||||
## 7. Docs + verify + publish
|
||||
|
||||
- [ ] All co-located `.md` files (≥200 words each).
|
||||
- [ ] Write `README.md` at package root.
|
||||
- [ ] `bunx turbo lint typecheck test --filter=@suaveplan/audio-analysis-pitch` — green.
|
||||
- [ ] `bunx turbo test:coverage --filter=@suaveplan/audio-analysis-pitch` — 100%.
|
||||
- [ ] `bun scripts/publish.ts --package @suaveplan/audio-analysis-pitch`.
|
||||
- [ ] Fold `specs/audio-analysis-pitch/spec.md` into `openspec/specs/audio-analysis-pitch/spec.md`.
|
||||
- [ ] Archive to `openspec/archive/2026-05-14-enterprise-audio-analysis-pitch/`.
|
||||
Reference in New Issue
Block a user