165 lines
5.3 KiB
Markdown
165 lines
5.3 KiB
Markdown
# OpenSpec Change: enterprise-audio-analysis-spectrum
|
|
|
|
- **Change ID**: enterprise-audio-analysis-spectrum
|
|
- **Package**: `@suaveplan/audio-analysis-spectrum`
|
|
- **Category**: universal
|
|
- **Target path**: `packages/media/audio/audio-analysis-spectrum`
|
|
- **Status**: PROPOSED
|
|
- **Date**: 2026-05-14
|
|
- **Design**: `./design.md`
|
|
- **Spec delta**: `./specs/audio-analysis-spectrum/spec.md`
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Deliver `@suaveplan/audio-analysis-spectrum` on the genesis branch from scratch at enterprise quality. Spectrum analysis plugin for the universal audio analysis framework. Computes magnitude spectra, spectrograms, MFCCs, spectral centroid, rolloff, and flux. Powers visualizations, feature extraction, and downstream analysis (beat, pitch, loudness).
|
|
|
|
Runs identically in browser and server via `BlockProcessor` or direct stream iteration.
|
|
|
|
---
|
|
|
|
## Current-state audit (genesis)
|
|
|
|
The package does not exist on genesis yet — this is a fresh build. No prior source, no test results, no coverage data.
|
|
|
|
---
|
|
|
|
## Available workspace deps on genesis
|
|
|
|
| Package | Role |
|
|
|---|---|
|
|
| `@suaveplan/audio-core` (`workspace:^`) | `AudioDocument`, `BlockProcessor`, `BlockContext` |
|
|
| `@suaveplan/error` (`workspace:^`) | `AudioError` base class; `AUDIO_ANALYSIS.*` codes |
|
|
| `@suaveplan/types` (`workspace:^`) | `LoggerLike` |
|
|
| `@suaveplan/opentelemetry` (`workspace:^`) | OTel spans, counters, histograms |
|
|
| `zod` (`catalog:forms`) | Zod schemas for option types |
|
|
|
|
Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
|
|
|
|
---
|
|
|
|
## Public API
|
|
|
|
### Core types
|
|
|
|
```ts
|
|
interface SpectrumAnalyzerOptions {
|
|
bins?: number // default 128
|
|
frameSize?: number // default 2048
|
|
hopSize?: number // default 512
|
|
window?: 'hann' | 'hamming' | 'blackman' // default 'hann'
|
|
}
|
|
|
|
interface SpectrumResult extends AnalysisResult {
|
|
spectrum: Float32Array // magnitude per bin
|
|
centroid: number
|
|
rolloff: number
|
|
flux: number
|
|
}
|
|
|
|
interface SpectrogramResult extends AnalysisResult {
|
|
spectrogram: Float32Array[] // time-frequency matrix
|
|
}
|
|
|
|
interface MFCCResult extends AnalysisResult {
|
|
mfcc: Float32Array[] // MFCC coefficients per frame
|
|
}
|
|
|
|
class SpectrumAnalyzer implements AnalysisOp {
|
|
readonly name = 'spectrum-analyzer'
|
|
analyze(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): Promise<SpectrumResult>
|
|
analyzeStream(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): AsyncGenerator<SpectrumResult>
|
|
analyzeBlock(block: Float32Array[], ctx: BlockContext): SpectrumResult
|
|
}
|
|
|
|
class Spectrogram implements AnalysisOp {
|
|
readonly name = 'spectrogram'
|
|
analyze(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): Promise<SpectrogramResult>
|
|
analyzeStream(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): AsyncGenerator<SpectrogramResult>
|
|
}
|
|
|
|
class MFCC implements AnalysisOp {
|
|
readonly name = 'mfcc'
|
|
analyze(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): Promise<MFCCResult>
|
|
analyzeStream(doc: AudioDocument, opts?: SpectrumAnalyzerOptions): AsyncGenerator<MFCCResult>
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## New additions for genesis
|
|
|
|
1. **Real FFT** — hand-rolled real FFT for universal use; on browser can delegate to `AnalyserNode` if available.
|
|
2. **Mel-spectrum** — triangular mel-scale filterbank with configurable bin count.
|
|
3. **MFCC** — DCT-II on log mel-spectrum, configurable coefficient count.
|
|
4. **Spectral features** — centroid, rolloff, flux computed per frame.
|
|
5. **Streaming analysis** — `analyzeStream()` yields per-frame results for real-time visualization.
|
|
6. **OTel instrumentation** — spans, counters, histograms per analyzer.
|
|
7. **Property-based tests** — sine wave → single peak, white noise → flat spectrum.
|
|
8. **Benchmark suite** — spectrogram 1-minute track at 2048/512.
|
|
|
|
---
|
|
|
|
## File layout
|
|
|
|
```text
|
|
packages/media/audio/audio-analysis-spectrum/
|
|
package.json
|
|
tsconfig.json
|
|
vitest.config.ts
|
|
src/
|
|
index.ts
|
|
error-codes.ts
|
|
analyzer/
|
|
spectrum-analyzer.ts
|
|
spectrum-analyzer.md
|
|
spectrum-analyzer.types.ts
|
|
spectrum-analyzer.test.ts
|
|
spectrogram.ts
|
|
spectrogram.md
|
|
spectrogram.types.ts
|
|
spectrogram.test.ts
|
|
mfcc.ts
|
|
mfcc.md
|
|
mfcc.types.ts
|
|
mfcc.test.ts
|
|
index.ts
|
|
algorithms/
|
|
fft.ts
|
|
fft.md
|
|
fft.test.ts
|
|
mel-scale.ts
|
|
mel-scale.md
|
|
mel-scale.test.ts
|
|
window.ts
|
|
window.md
|
|
window.test.ts
|
|
dct.ts
|
|
dct.md
|
|
dct.test.ts
|
|
index.ts
|
|
__benchmarks__/
|
|
spectrum.bench.ts
|
|
tests/
|
|
integration.test.ts
|
|
property.test.ts
|
|
```
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
**Breaking changes**: none — fresh build, package does not yet exist on genesis.
|
|
|
|
**Prerequisites**: `@suaveplan/audio-core`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry` published to Verdaccio.
|
|
|
|
### Affected dependents
|
|
|
|
`@suaveplan/audio-analysis-spectrum` is a leaf analysis plugin: No dependents in this batch
|
|
(verified by `bunx turbo ls --filter=...@suaveplan/audio-analysis-spectrum`). Consumed at runtime
|
|
by application/visualization code registering it as an analysis op. Sibling analysis packages that
|
|
will read its surface in later waves (`@suaveplan/audio-analysis-beat` for the spectral-flux signal,
|
|
`@suaveplan/audio-analysis-pitch` for the chromagram) do not exist on genesis yet and are each
|
|
created against this package's spec delta, not retrofitted.
|