Initial Commit
This commit is contained in:
@@ -0,0 +1,114 @@
|
||||
# Design — enterprise-react-audio-fingerprint
|
||||
|
||||
`@suaveplan/react-audio-fingerprint` is a focused package: client-side audio fingerprinting + matching, exposed as React hooks and components. Its job is to take a `MediaSource | URL | ArrayBuffer | AudioBuffer`, compute a constellation fingerprint, and either store / match it against a corpus. Every piece of generic audio handling — decode, FFT, spectrogram — is delegated to existing audio-family packages so this package only owns the fingerprinting algorithm and its UI surface.
|
||||
|
||||
---
|
||||
|
||||
## 1. The constellation algorithm
|
||||
|
||||
The package implements the Shazam-style algorithm:
|
||||
|
||||
1. **Decode** the input audio to PCM via `audio-engine-browser`'s decode path.
|
||||
2. **Window + FFT** through `@suaveplan/audio-analysis-spectrum`'s FFT primitives (4096-point windows, Hann window, 75 % overlap by default).
|
||||
3. **2D local-peak detection** in the resulting spectrogram (per-frame top-N peaks above a noise floor; configurable `peaksPerFrame` and `noiseFloorDb`).
|
||||
4. **Constellation pair hashing**: for each peak, look at the next `fanout` peaks within a `targetZoneSec` time window; emit a hash `(f1, f2, dt)` quantised into a 32-bit integer.
|
||||
5. **Output** is a sorted list of `(hash: number, timeOffsetSec: number)` pairs — the `Fingerprint` shape.
|
||||
|
||||
This sits in `src/fingerprint/audio-fingerprinter.ts` and `src/fingerprint/constellation.ts`. Tests verify deterministic output for identical inputs, partial-match resilience under noise / pitch-shift / time-stretch (within tolerance), and stable output across browsers (FFT result equivalence guaranteed by `audio-analysis-spectrum`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Matching
|
||||
|
||||
`FingerprintMatcher` exposes:
|
||||
|
||||
```ts
|
||||
class FingerprintMatcher {
|
||||
match(query: Fingerprint, corpus: Iterable<{ id: string; fingerprint: Fingerprint }>): MatchResult[]
|
||||
matchOne(query: Fingerprint, candidate: Fingerprint): { score: number; offsetSec: number }
|
||||
}
|
||||
```
|
||||
|
||||
The basic match uses **time-offset histogramming**: for each (query, candidate) pair, compute the time-offset for each shared hash, bin into 10 ms buckets, and the maximum bucket count is the match score. The offset of that bucket gives the time alignment. This is robust to partial matches (the query is a clip of the full reference) and outperforms naive Jaccard.
|
||||
|
||||
`matchOne` is exposed for direct two-fingerprint comparison; `match` iterates the corpus. For corpora over ~10k, the package recommends the optional `python-audio-fingerprint` backend via the injectable `FingerprintBackend` interface — the same `match()` API, different storage and indexing internals.
|
||||
|
||||
---
|
||||
|
||||
## 3. Corpus storage
|
||||
|
||||
The package ships two corpus implementations:
|
||||
|
||||
- **`MemoryCorpus`** — `Map<string, Fingerprint>` for tests / small corpora.
|
||||
- **`IndexedDbCorpus`** — IndexedDB-backed for browser sessions; persists across page reloads.
|
||||
|
||||
Both implement `Corpus`:
|
||||
|
||||
```ts
|
||||
interface Corpus {
|
||||
add(id: string, fingerprint: Fingerprint): Promise<void>
|
||||
remove(id: string): Promise<void>
|
||||
get(id: string): Promise<Fingerprint | undefined>
|
||||
values(): AsyncIterable<{ id: string; fingerprint: Fingerprint }>
|
||||
size(): Promise<number>
|
||||
}
|
||||
```
|
||||
|
||||
Consumers can implement their own (Postgres-via-API, S3, etc.).
|
||||
|
||||
---
|
||||
|
||||
## 4. Backend abstraction
|
||||
|
||||
The `FingerprintBackend` interface lets consumers swap out the local fingerprinting algorithm for a remote service:
|
||||
|
||||
```ts
|
||||
interface FingerprintBackend {
|
||||
fingerprint(source: AudioBuffer): Promise<Fingerprint>
|
||||
match(query: Fingerprint, corpusRef: string): Promise<MatchResult[]>
|
||||
}
|
||||
```
|
||||
|
||||
`LocalBackend` (default) runs the algorithm in-browser. `PythonBackend` (optional subpath; requires `@suaveplan/python-audio-fingerprint` peer) delegates both `fingerprint` and `match` to a backend service over HTTP/WebSocket. Hooks accept a `backend?: FingerprintBackend` prop and default to `LocalBackend`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Hooks
|
||||
|
||||
```ts
|
||||
useAudioFingerprint(source: AudioSource, opts?: { backend?: FingerprintBackend, signal?: AbortSignal }): {
|
||||
fingerprint: Fingerprint | undefined
|
||||
state: 'idle' | 'decoding' | 'fingerprinting' | 'done' | 'error'
|
||||
error: ReactAudioFingerprintError | undefined
|
||||
progress: number // 0..1
|
||||
}
|
||||
|
||||
useFingerprintMatcher(corpus: Corpus, opts?: { backend?: FingerprintBackend }): {
|
||||
match(query: Fingerprint): Promise<MatchResult[]>
|
||||
matching: boolean
|
||||
lastResults: MatchResult[]
|
||||
}
|
||||
```
|
||||
|
||||
`useAudioFingerprint` is concurrent-rendering safe (`useSyncExternalStore`-backed). It uses `AbortSignal` so navigating away cancels long-running fingerprints cleanly.
|
||||
|
||||
---
|
||||
|
||||
## 6. Components
|
||||
|
||||
- `<AudioFingerprintUploader onFingerprint={fp => ...}>` — accepts drag-drop, file picker, URL paste. Renders progress via `useAudioFingerprint`. Errors surface in an accessible toast.
|
||||
- `<FingerprintMatchDisplay results>` — ranked list of `{ id, score, offsetSec }` with confidence bars (score normalised). Click-to-play if the corpus entry provides a playable URL.
|
||||
- `<FingerprintVisualizer fingerprint>` — overlays constellation peaks on a spectrogram (via `audio-analysis-spectrum`'s spectrogram render). Useful for debugging / explainability.
|
||||
|
||||
All three are keyboard-accessible per the `react-audio` accessibility requirement.
|
||||
|
||||
---
|
||||
|
||||
## 7. Decisions and trade-offs
|
||||
|
||||
- **FFT is delegated, not duplicated.** One FFT implementation across the audio family.
|
||||
- **Time-offset histogram for matching, not raw Jaccard.** Robust to partial matches; matches the published Shazam method.
|
||||
- **Backend abstraction.** Local default; server-delegation opt-in via subpath. Same hook API both ways.
|
||||
- **Two corpus implementations + an injectable interface.** Memory for tests, IndexedDB for sessions, custom for production.
|
||||
- **AbortSignal everywhere.** Long-running fingerprints don't leak on unmount.
|
||||
- **Visualizer reuses spectrogram render from `audio-analysis-spectrum`.** No new render path.
|
||||
@@ -0,0 +1,116 @@
|
||||
# OpenSpec Change: enterprise-react-audio-fingerprint
|
||||
|
||||
- **Change ID**: enterprise-react-audio-fingerprint
|
||||
- **Package**: `@suaveplan/react-audio-fingerprint`
|
||||
- **Category**: react
|
||||
- **Target path**: `packages/ui/react/react-audio-fingerprint`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-15
|
||||
- **Design**: `./design.md`
|
||||
- **Spec delta**: `./specs/react-audio-fingerprint/spec.md`
|
||||
|
||||
> Replaces the prior 2026-04-24 draft flagged `NEEDS-REWRITE`. See `../REACT_AUDIO_WAVE_REWRITE_COMPLETE.md`.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Deliver `@suaveplan/react-audio-fingerprint` on genesis from scratch at enterprise quality. The package provides client-side audio fingerprinting (Shazam-style constellation hashing) and identity matching as React hooks + components, built on the unified architecture:
|
||||
|
||||
1. **`AudioFingerprinter`** — a universal class (no React) implementing chromagram FFT + 2D local-peak detection + constellation pair hashing per the classic algorithm; produces a deterministic `Fingerprint` (a sorted list of `(hash, timeOffsetSec)` pairs). Sources audio via the `@suaveplan/audio-analysis-spectrum` package's FFT primitives so there is **one FFT implementation across the audio family**, not a duplicate.
|
||||
2. **`FingerprintMatcher`** — universal Jaccard-similarity matcher that scores a query fingerprint against a corpus and returns ranked matches with confidence scores.
|
||||
3. **Hooks**: `useAudioFingerprint(source)`, `useFingerprintMatcher(corpus)`, `useFingerprintProgress(hashHandle)`.
|
||||
4. **Components**: `<AudioFingerprintUploader>` (file drop + URL paste), `<FingerprintMatchDisplay>` (ranked results with thumbnails + confidence bars), `<FingerprintVisualizer>` (constellation plot over the spectrogram).
|
||||
5. **OTel instrumentation** (fingerprint duration, match duration, corpus size gauge).
|
||||
6. **Integration with `@suaveplan/python-audio-fingerprint`** (optional server-side high-accuracy backend) via an injectable `FingerprintBackend` interface — local algorithm is the default; server delegation is opt-in for large corpora.
|
||||
|
||||
The package owns **no DSP outside the local fingerprinting algorithm itself** — FFT primitives come from `audio-analysis-spectrum`, audio loading comes from `audio-engine-browser`'s decode path, persistence and corpus storage are injectable.
|
||||
|
||||
---
|
||||
|
||||
## Current-state audit (genesis)
|
||||
|
||||
The package does not exist on genesis yet — fresh build replacing the obsolete pre-tier 2026-04-24 draft.
|
||||
|
||||
---
|
||||
|
||||
## Parity Checklist (vs upstream)
|
||||
|
||||
| Capability | Upstream | Migrated | Notes |
|
||||
|---|:---:|:---:|---|
|
||||
| `AudioFingerprinter` (chromagram + peak detect + constellation) | ✅ | ⬆️ | FFT delegated to `audio-analysis-spectrum`; one implementation across audio family. |
|
||||
| `FingerprintMatcher` (Jaccard similarity) | ✅ | ✅ | Pure universal; no DOM deps. |
|
||||
| `AudioLoader` | ✅ | ⬆️ | Replaced by `audio-engine-browser` decode + WebCodecs path; consistent across all audio surfaces. |
|
||||
| `useAudioFingerprint` hook | ✅ | ✅ | Suspense-friendly via `useSyncExternalStore`. |
|
||||
| `useFingerprintMatcher` hook | ✅ | ✅ | Subscribes to corpus changes. |
|
||||
| `<AudioFingerprintUploader>` component | ✅ | ✅ | DnD + URL + file picker; accessible. |
|
||||
| `<FingerprintMatchDisplay>` component | ✅ | ✅ | Ranked results + confidence bars. |
|
||||
| `<FingerprintVisualizer>` component | ✅ | ✅ | Constellation overlay on spectrogram. |
|
||||
| Server backend integration (NEW) | – | ⬆️ | Optional `python-audio-fingerprint` backend via injectable `FingerprintBackend`. |
|
||||
| Storybook coverage | partial | ✅ | Default + interactive + edge. |
|
||||
| Coverage | ~72 % | ✅ | 100/100/100/100. |
|
||||
|
||||
No 🚫 rows.
|
||||
|
||||
---
|
||||
|
||||
## Available workspace deps on genesis
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@suaveplan/react-audio` (`workspace:^`) | Hooks + provider |
|
||||
| `@suaveplan/audio-analysis-spectrum` (`workspace:^`) | FFT + spectrogram primitives |
|
||||
| `@suaveplan/audio-engine-browser` (`workspace:^`) | Source decode + buffer load |
|
||||
| `@suaveplan/python-audio-fingerprint` (`workspace:^`, optional peer for backend gating) | Optional server backend |
|
||||
| `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`, `@suaveplan/events` | foundations |
|
||||
| `react` (peer), `zustand` (`catalog:state`), `framer-motion` (`catalog:animation`), `lucide-react` (`catalog:ui`), `zod` (`catalog:forms`) |
|
||||
|
||||
---
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/ui/react/react-audio-fingerprint/
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts
|
||||
fingerprint/
|
||||
audio-fingerprinter/ (.ts .md .types.ts .schemas.ts .test.ts)
|
||||
fingerprint-matcher/ ...
|
||||
constellation/ ...
|
||||
peak-detector/ ...
|
||||
backends/
|
||||
local-backend/ ...
|
||||
python-backend/ ← optional subpath
|
||||
hooks/
|
||||
use-audio-fingerprint/ (.ts .md .test.tsx)
|
||||
use-fingerprint-matcher/ ...
|
||||
use-fingerprint-progress/ ...
|
||||
components/
|
||||
audio-fingerprint-uploader/ (.tsx .md .types.ts .test.tsx .stories.tsx)
|
||||
fingerprint-match-display/ ...
|
||||
fingerprint-visualizer/ ...
|
||||
corpus/
|
||||
memory-corpus/ ...
|
||||
indexeddb-corpus/ ...
|
||||
__benchmarks__/
|
||||
fingerprint.bench.ts
|
||||
match.bench.ts
|
||||
tests/
|
||||
integration.test.tsx
|
||||
purity.test.ts
|
||||
exports.test.ts
|
||||
property.test.ts ← fingerprint(A) ≠ fingerprint(B) for distinct inputs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
**Breaking changes**: none — fresh build.
|
||||
|
||||
**Prerequisites**: `react-audio`, `audio-analysis-spectrum`, `audio-engine-browser`, foundations. `python-audio-fingerprint` optional.
|
||||
|
||||
### Affected dependents
|
||||
|
||||
No dependents in this batch (verified by `bunx turbo ls --filter=...@suaveplan/react-audio-fingerprint`).
|
||||
@@ -0,0 +1,181 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: react-audio-fingerprint (enterprise-react-audio-fingerprint)
|
||||
|
||||
**Package**: `@suaveplan/react-audio-fingerprint`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Fingerprint-tier purity — no FFT, no audio context
|
||||
|
||||
`@suaveplan/react-audio-fingerprint` SHALL NOT implement its own FFT or spectrogram — those are delegated to `@suaveplan/audio-analysis-spectrum`. The package SHALL NOT reference `AudioContext`, `OfflineAudioContext`, `AudioNode`, or `AudioWorklet`. The package's runtime `dependencies` SHALL be limited to `@suaveplan/react-audio`, `@suaveplan/audio-analysis-spectrum`, `@suaveplan/audio-engine-browser`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`, `@suaveplan/events`, `zustand`, `framer-motion`, `lucide-react`, `zod`, `tslib`. `react` and `@suaveplan/python-audio-fingerprint` SHALL be peer dependencies; the latter SHALL only be required by the `./backends/python-backend` subpath.
|
||||
|
||||
#### Scenario: built dist contains no FFT implementation
|
||||
|
||||
- **WHEN** the source tree is grep'd for FFT-implementation hallmarks (Cooley-Tukey, twiddle-factor tables, radix-2/4 butterfly inlines) inside `src/fingerprint/`
|
||||
- **THEN** no implementation is present; the spectrogram is reached only through imports of `@suaveplan/audio-analysis-spectrum`
|
||||
|
||||
**Verified by:** `tests/purity.test.ts::"no FFT is implemented in src/; spectrogram is reached via audio-analysis-spectrum"` AND `tests/purity.test.ts::"runtime dependency set matches the allowlist"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AudioFingerprinter constellation algorithm
|
||||
|
||||
The package SHALL export `AudioFingerprinter` from `./fingerprint/audio-fingerprinter` whose `fingerprint(source: AudioBuffer, opts?): Promise<Fingerprint>` SHALL: (1) compute the spectrogram via `audio-analysis-spectrum` using a default 4096-point Hann window with 75 % overlap (configurable), (2) detect 2D local peaks per frame above a `noiseFloorDb` (default −60 dBFS), capped at `peaksPerFrame` (default 5), (3) for each peak emit constellation pair hashes against the next `fanout` (default 5) peaks within `targetZoneSec` (default 5.0), and (4) return a sorted-by-time `Fingerprint = ReadonlyArray<{ hash: number; timeOffsetSec: number }>`. Output SHALL be deterministic: identical input bytes produce a byte-equal fingerprint.
|
||||
|
||||
#### Scenario: identical input produces identical fingerprint
|
||||
|
||||
- **WHEN** the same `AudioBuffer` is fingerprinted twice
|
||||
- **THEN** the two fingerprints are byte-equal
|
||||
|
||||
#### Scenario: noise-tolerant partial match
|
||||
|
||||
- **WHEN** a 10-second clip is extracted from a 30-second source, white-noised at −20 dB SNR, and fingerprinted
|
||||
- **THEN** `FingerprintMatcher.matchOne(clipFingerprint, fullFingerprint)` returns a score whose histogram peak is ≥ 30 % of `clip.length`, and the `offsetSec` matches the clip's position within ±50 ms
|
||||
|
||||
**Verified by:** `src/fingerprint/audio-fingerprinter.test.ts::"identical input produces identical fingerprint"` AND `tests/property.test.ts::"noisy clip matches its source within histogram tolerance"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: FingerprintMatcher time-offset histogram
|
||||
|
||||
The package SHALL export `FingerprintMatcher` from `./fingerprint/fingerprint-matcher` whose `match(query, corpus): MatchResult[]` SHALL use time-offset histogramming (10 ms bin width) to score each candidate, and return results sorted by descending score. `matchOne(query, candidate): { score: number; offsetSec: number }` SHALL return the maximum bucket count and the offset of that bucket. `MatchResult` SHALL carry `{ id: string; score: number; offsetSec: number; matchedHashCount: number }`.
|
||||
|
||||
#### Scenario: match returns results sorted by descending score
|
||||
|
||||
- **WHEN** `match(query, corpusOfThree)` is called against fingerprints with known descending similarity to the query
|
||||
- **THEN** the result array is ordered by descending `score` and `length === 3`
|
||||
|
||||
**Verified by:** `src/fingerprint/fingerprint-matcher.test.ts::"match returns results sorted by descending score"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: useAudioFingerprint hook
|
||||
|
||||
`useAudioFingerprint(source, opts?)` SHALL return `{ fingerprint, state, error, progress }` where `state ∈ 'idle' | 'decoding' | 'fingerprinting' | 'done' | 'error'`, `progress` is a `0..1` float updated at frame rate via `useSyncExternalStore`, and `error` is a `ReactAudioFingerprintError` when state is `'error'`. The hook SHALL accept `opts.signal: AbortSignal`; on unmount or signal abort, the in-flight fingerprint SHALL be cancelled and `state` SHALL settle to `'idle'`. The hook SHALL be SSR-safe: `state === 'idle'` and `fingerprint === undefined` on the first server render.
|
||||
|
||||
#### Scenario: unmount cancels in-flight fingerprint
|
||||
|
||||
- **WHEN** the consumer component using `useAudioFingerprint(longSource)` unmounts while `state === 'fingerprinting'`
|
||||
- **THEN** the underlying fingerprint task's `AbortSignal` fires, no further state updates occur, and no error is logged
|
||||
|
||||
#### Scenario: progress advances monotonically to 1
|
||||
|
||||
- **WHEN** a 30-second fingerprint runs to completion
|
||||
- **THEN** `progress` increases monotonically from 0 toward 1 and reaches `1.0` exactly when `state` transitions to `'done'`
|
||||
|
||||
**Verified by:** `src/hooks/use-audio-fingerprint/use-audio-fingerprint.test.tsx::"unmount cancels in-flight fingerprint"` AND `src/hooks/use-audio-fingerprint/use-audio-fingerprint.test.tsx::"progress advances monotonically to 1"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: useFingerprintMatcher with corpus subscription
|
||||
|
||||
`useFingerprintMatcher(corpus, opts?)` SHALL return `{ match(query), matching, lastResults }`. The hook SHALL subscribe to the corpus's `'changed'` event so adding / removing entries between matches updates the relevant render. `lastResults` SHALL be cleared when the corpus identity changes (different `Corpus` instance prop) but SHALL persist across `add`/`remove` of the same corpus.
|
||||
|
||||
#### Scenario: corpus add re-renders consumers that read size or lastResults relevance
|
||||
|
||||
- **WHEN** a consumer reading `useFingerprintMatcher(corpus)` triggers `corpus.add(id, fingerprint)`
|
||||
- **THEN** the consumer re-renders once with the corpus change visible
|
||||
|
||||
**Verified by:** `src/hooks/use-fingerprint-matcher/use-fingerprint-matcher.test.tsx::"corpus add re-renders consumers"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Backend abstraction
|
||||
|
||||
The package SHALL export a `FingerprintBackend` interface with `fingerprint(source): Promise<Fingerprint>` and `match(query, corpusRef): Promise<MatchResult[]>`. A `LocalBackend` (default, in-browser algorithm) SHALL be the default; a `PythonBackend` (optional, gated to `./backends/python-backend` subpath) SHALL delegate to `@suaveplan/python-audio-fingerprint` over HTTP. Hooks SHALL accept `opts.backend?: FingerprintBackend` and SHALL default to `LocalBackend` when absent.
|
||||
|
||||
#### Scenario: hook honours injected backend
|
||||
|
||||
- **WHEN** `useAudioFingerprint(source, { backend: customBackend })` runs
|
||||
- **THEN** `customBackend.fingerprint` is called and the local algorithm is not exercised
|
||||
|
||||
#### Scenario: python-backend is gated to its subpath
|
||||
|
||||
- **WHEN** a consumer imports the package root without `./backends/python-backend`
|
||||
- **THEN** the tree-shaken bundle contains zero bytes from `@suaveplan/python-audio-fingerprint`
|
||||
|
||||
**Verified by:** `src/hooks/use-audio-fingerprint/use-audio-fingerprint.test.tsx::"hook honours injected backend"` AND `tests/bundle-size/root-only.size.test.ts::"root barrel does not bundle python-audio-fingerprint"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Corpus implementations
|
||||
|
||||
The package SHALL export `MemoryCorpus` (`Map`-backed, synchronous wrapped in `Promise`) and `IndexedDbCorpus` (IndexedDB-backed, persistent across page reloads) from `./corpus/memory-corpus` and `./corpus/indexeddb-corpus`. Both SHALL implement the `Corpus` interface (`add`, `remove`, `get`, `values`, `size`) and emit a typed `'changed'` event on mutation. `IndexedDbCorpus.add` SHALL persist the fingerprint such that a fresh `IndexedDbCorpus` instance constructed in a later session can read it back. Custom corpus implementations are first-class consumers; nothing in the package presumes which corpus is in use.
|
||||
|
||||
#### Scenario: IndexedDb corpus persists across instances
|
||||
|
||||
- **WHEN** a caller adds an entry via one `IndexedDbCorpus` instance, then constructs a fresh instance with the same `dbName`/`storeName`
|
||||
- **THEN** the second instance's `get(id)` returns the same fingerprint
|
||||
|
||||
**Verified by:** `src/corpus/indexeddb-corpus/indexeddb-corpus.test.ts::"persists across instances with the same dbName"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AudioFingerprintUploader component
|
||||
|
||||
`<AudioFingerprintUploader onFingerprint label?>` SHALL accept input via file drop, file picker (click), and URL paste, decode the source via `audio-engine-browser`'s decode path, run the fingerprint, and call `onFingerprint(fp)` on success. Errors SHALL surface as an accessible toast carrying the `ReactAudioFingerprintError`'s `code`. The component SHALL render a progress bar driven by `useAudioFingerprint.progress` and SHALL be keyboard-accessible (Enter / Space to open the file picker, `role="button"` on the drop zone, `aria-busy` while fingerprinting).
|
||||
|
||||
#### Scenario: drop file → fingerprint → onFingerprint fires
|
||||
|
||||
- **WHEN** a user drops an audio file onto `<AudioFingerprintUploader>`
|
||||
- **THEN** the component decodes, fingerprints, and calls `onFingerprint(fp)` exactly once with the resulting fingerprint
|
||||
|
||||
**Verified by:** `src/components/audio-fingerprint-uploader/audio-fingerprint-uploader.test.tsx::"drop file calls onFingerprint once"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: FingerprintVisualizer constellation overlay
|
||||
|
||||
`<FingerprintVisualizer fingerprint spectrogram?>` SHALL render a `<canvas>` displaying the input spectrogram (via `audio-analysis-spectrum`'s spectrogram render path) overlaid with the constellation peaks from `fingerprint`. When `spectrogram` is not supplied, the component SHALL render only the constellation (sparse plot). Hovering or focusing a peak SHALL show its `{ frequencyHz, timeSec, hash }` in a tooltip / aria-described region.
|
||||
|
||||
#### Scenario: peaks render at the expected canvas coordinates
|
||||
|
||||
- **WHEN** a fingerprint with a known peak at `(timeSec: 1.5, frequencyHz: 440)` is rendered against a 1024×512 canvas spanning 0..3 s × 0..22050 Hz
|
||||
- **THEN** the rendered peak's canvas coordinates are within ±2 px of `(512, 502)`
|
||||
|
||||
**Verified by:** `src/components/fingerprint-visualizer/fingerprint-visualizer.test.tsx::"peaks render at the expected canvas coordinates"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry instrumentation
|
||||
|
||||
The package SHALL register: a histogram `react_audio_fingerprint_fingerprint_duration_ms{backend}`, a histogram `react_audio_fingerprint_match_duration_ms{backend}`, and a gauge `react_audio_fingerprint_corpus_size{corpus}`. Instrumentation SHALL be inert when no exporter is registered.
|
||||
|
||||
#### Scenario: corpus size gauge tracks add and remove
|
||||
|
||||
- **WHEN** five entries are added to a corpus then two removed
|
||||
- **THEN** `react_audio_fingerprint_corpus_size` reads 3
|
||||
|
||||
**Verified by:** `src/corpus/memory-corpus/memory-corpus.test.ts::"OTel corpus size gauge tracks add and remove"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Structured error codes
|
||||
|
||||
The package SHALL declare its codes via `defineCodes("REACT_AUDIO_FINGERPRINT", [...])` producing the dotted codes `REACT_AUDIO_FINGERPRINT.DECODE_FAILED`, `REACT_AUDIO_FINGERPRINT.FINGERPRINT_FAILED`, `REACT_AUDIO_FINGERPRINT.MATCH_FAILED`, `REACT_AUDIO_FINGERPRINT.CORPUS_FAILED`, `REACT_AUDIO_FINGERPRINT.ABORTED`, and `REACT_AUDIO_FINGERPRINT.INVALID_OPTIONS`. Export `ReactAudioFingerprintErrorCode`, `REACT_AUDIO_FINGERPRINT_ERROR_METADATA`, `registerReactAudioFingerprintErrorCodes()`, and a `ReactAudioFingerprintError` class extending `AudioError`. Every registered code SHALL be reachable from a real call site.
|
||||
|
||||
#### Scenario: every registered code is reachable
|
||||
|
||||
- **WHEN** the test suite runs
|
||||
- **THEN** each code in `REACT_AUDIO_FINGERPRINT_ERROR_METADATA` is asserted thrown by at least one test
|
||||
|
||||
**Verified by:** `src/error-codes.test.ts::"every REACT_AUDIO_FINGERPRINT code is reachable from a call site"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath exports per surface
|
||||
|
||||
`package.json` SHALL declare one `exports` entry per surface: `.`, `./fingerprint`, `./hooks`, `./hooks/<name>`, `./components`, `./components/<name>`, `./corpus`, `./corpus/<name>`, `./backends`, `./backends/local-backend`, `./backends/python-backend`. `sideEffects` SHALL be `false`.
|
||||
|
||||
#### Scenario: importing the matcher does not pull in components
|
||||
|
||||
- **WHEN** a consumer imports `@suaveplan/react-audio-fingerprint/fingerprint/fingerprint-matcher`
|
||||
- **THEN** the tree-shaken bundle contains zero bytes from `src/components/`
|
||||
|
||||
**Verified by:** `tests/exports.test.ts::"every surface has a subpath export entry"` AND `tests/bundle-size/matcher-only.size.test.ts::"matcher subpath does not bundle components"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,75 @@
|
||||
# Tasks: enterprise-react-audio-fingerprint
|
||||
|
||||
**Change ID**: enterprise-react-audio-fingerprint
|
||||
**Package**: `@suaveplan/react-audio-fingerprint`
|
||||
**Target path**: `packages/ui/react/react-audio-fingerprint`
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/react-audio`, `@suaveplan/audio-analysis-spectrum`, `@suaveplan/audio-engine-browser`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/opentelemetry`, `@suaveplan/events`
|
||||
|
||||
---
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [ ] 1.1 Directory tree per proposal.
|
||||
- [ ] 1.2 `package.json` at `0.1.0`, `"sideEffects": false`, full subpath exports. React 19 peer; `python-audio-fingerprint` optional peer gated to `./backends/python-backend`.
|
||||
- [ ] 1.3 `tsconfig.json`, `vitest.config.ts` (`happy-dom`).
|
||||
|
||||
## 2. Error codes
|
||||
|
||||
- [ ] 2.1 `src/error-codes.ts` — `defineCodes("REACT_AUDIO_FINGERPRINT", [...])`, `ReactAudioFingerprintError extends AudioError`.
|
||||
- [ ] 2.2 `error-codes.test.ts` — every code reachable.
|
||||
|
||||
## 3. Fingerprint algorithm
|
||||
|
||||
- [ ] 3.1 `src/fingerprint/peak-detector.ts` — 2D local-peak detection above noise floor with cap.
|
||||
- [ ] 3.2 `src/fingerprint/constellation.ts` — pair hashing within target-zone window; 32-bit quantisation.
|
||||
- [ ] 3.3 `src/fingerprint/audio-fingerprinter.ts` — composes peak-detector + constellation against `audio-analysis-spectrum` FFT/spectrogram.
|
||||
- [ ] 3.4 `audio-fingerprinter.test.ts` — determinism, peak count cap, noise-floor behaviour.
|
||||
|
||||
## 4. Matcher
|
||||
|
||||
- [ ] 4.1 `src/fingerprint/fingerprint-matcher.ts` — time-offset histogramming.
|
||||
- [ ] 4.2 `fingerprint-matcher.test.ts` — sort order, partial match resilience.
|
||||
|
||||
## 5. Corpus
|
||||
|
||||
- [ ] 5.1 `src/corpus/memory-corpus/` — `Map`-backed, `'changed'` event.
|
||||
- [ ] 5.2 `src/corpus/indexeddb-corpus/` — IDB-backed, persistence-across-instances test.
|
||||
- [ ] 5.3 OTel `corpus_size` gauge wiring.
|
||||
|
||||
## 6. Backends
|
||||
|
||||
- [ ] 6.1 `src/backends/local-backend/` — default; runs the local algorithm.
|
||||
- [ ] 6.2 `src/backends/python-backend/` — optional subpath delegating to `python-audio-fingerprint` over HTTP.
|
||||
|
||||
## 7. Hooks
|
||||
|
||||
- [ ] 7.1 `use-audio-fingerprint` — `useSyncExternalStore`-backed; `AbortSignal` cancellation; SSR-safe initial state.
|
||||
- [ ] 7.2 `use-fingerprint-matcher` — subscribes to corpus `'changed'`; clears `lastResults` on corpus identity change.
|
||||
- [ ] 7.3 `use-fingerprint-progress` — convenience progress accessor.
|
||||
|
||||
## 8. Components (each: `.tsx`, `.md`, `.types.ts`, `.test.tsx`, `.stories.tsx`)
|
||||
|
||||
- [ ] 8.1 `audio-fingerprint-uploader/` — DnD + file picker + URL paste; progress bar; accessible.
|
||||
- [ ] 8.2 `fingerprint-match-display/` — ranked list with confidence bars.
|
||||
- [ ] 8.3 `fingerprint-visualizer/` — constellation overlay on spectrogram; tooltip on peak focus.
|
||||
|
||||
## 9. Benchmarks
|
||||
|
||||
- [ ] 9.1 `src/__benchmarks__/fingerprint.bench.ts` — fingerprint a 60 s reference; report MB/s.
|
||||
- [ ] 9.2 `src/__benchmarks__/match.bench.ts` — match a 10 s query against a 10 k-entry corpus.
|
||||
|
||||
## 10. Tests
|
||||
|
||||
- [ ] 10.1 `tests/integration.test.tsx` — uploader → fingerprint → match against memory corpus → display.
|
||||
- [ ] 10.2 `tests/purity.test.ts` — no FFT in `src/`; dependency-set allowlist.
|
||||
- [ ] 10.3 `tests/property.test.ts` (fast-check) — noisy clip matches its source; distinct sources distinct fingerprints.
|
||||
- [ ] 10.4 `tests/exports.test.ts` — every surface subpath; python gated.
|
||||
- [ ] 10.5 `tests/bundle-size/{root-only,matcher-only}.size.test.ts`.
|
||||
|
||||
## 11. Docs + verify + archive
|
||||
|
||||
- [ ] 11.1 All co-located `.md` (≥200 words).
|
||||
- [ ] 11.2 `README.md` at package root with quick-start, corpus example, backend swap example.
|
||||
- [ ] 11.3 `bunx turbo lint typecheck test --filter=@suaveplan/react-audio-fingerprint` green; coverage 100/100/100/100.
|
||||
- [ ] 11.4 Walk every `### Requirement` and `**Verified by:**`.
|
||||
- [ ] 11.5 Fold spec delta; move change folder to archive.
|
||||
Reference in New Issue
Block a user