skills/plugins/document-navigator/docs/ui/react/react-audio-fingerprint/README.md

@suaveplan/react-audio-fingerprint - React 19 surface for client-side audio fingerprinting (Shazam-style constellation algorithm) and identity matching. Consumes FFT/spectrogram primitives from @suaveplan/audio-analysis-spectrum so the audio family has a single FFT implementation. Universal fingerprint + matcher cores; React hook + accessible component on top.

React 19 surface for client-side audio fingerprinting and identity matching. The package implements the classic Shazam-style constellation algorithm (2D-peak detection + pair hashing + time-offset histogram matching) on top of the FFT/spectrogram primitives from @suaveplan/audio-analysis-spectrum — there is exactly one FFT implementation across the audio family.

Documentation tree

• local-backend — LocalBackend is the default implementation of the FingerprintBackend contract that hooks and components depend on.…
  • local-backend — LocalBackend is the default implementation of the FingerprintBackend contract that hooks and components depend on.…
• fingerprint-match-display — <FingerprintMatchDisplay> renders the output of FingerprintMatcher.match as an accessible ranked list. Each row sho…
  • fingerprint-match-display — <FingerprintMatchDisplay> renders the output of FingerprintMatcher.match as an accessible ranked list. Each row sho…
• memory-corpus — MemoryCorpus is the default Corpus implementation: a Map<string, Fingerprint> wrapped with a Promise-returning…
  • memory-corpus — MemoryCorpus is the default Corpus implementation: a Map<string, Fingerprint> wrapped with a Promise-returning…
• error-codes — This module declares every error code that @suaveplan/react-audio-fingerprint can throw. Each code is registered thro…
  • error-codes — This module declares every error code that @suaveplan/react-audio-fingerprint can throw. Each code is registered thro…
• audio-fingerprinter — AudioFingerprinter is the entry point that turns an AudioDocument into a deterministic constellation fingerprint. I…
  • audio-fingerprinter — AudioFingerprinter is the entry point that turns an AudioDocument into a deterministic constellation fingerprint. I…
• constellation — The constellation module converts a list of spectrogram peaks into a stable, hashable fingerprint by pairing each ancho…
  • constellation — The constellation module converts a list of spectrogram peaks into a stable, hashable fingerprint by pairing each ancho…
• fingerprint-matcher — FingerprintMatcher scores a query fingerprint against a corpus of candidate fingerprints using the classic Shazam tim…
  • fingerprint-matcher — FingerprintMatcher scores a query fingerprint against a corpus of candidate fingerprints using the classic Shazam tim…
• peak-detector — The peak detector consumes a spectrogram (a sequence of FFT magnitude frames produced by `@suaveplan/audio-analysis-spe…
  • peak-detector — The peak detector consumes a spectrogram (a sequence of FFT magnitude frames produced by `@suaveplan/audio-analysis-spe…
• use-audio-fingerprint — useAudioFingerprint(source, options?) drives a FingerprintBackend off a React-managed AudioDocument and returns t…
  • use-audio-fingerprint — useAudioFingerprint(source, options?) drives a FingerprintBackend off a React-managed AudioDocument and returns t…
• test-setup — This file is loaded by vitest.config.ts via setupFiles so every test in the package runs against a consistent jest-…

Status

bootstrap landing — initial slice focused on:

• Six error codes via defineCodes("REACT_AUDIO_FINGERPRINT", …).
• peak-detector, constellation, audio-fingerprinter.
• fingerprint-matcher (time-offset histogramming).
• memory-corpus (Map-backed, async-shaped, subscribable).
• local-backend (default FingerprintBackend running in-process).
• useAudioFingerprint hook (driver with abort + progress).
• <FingerprintMatchDisplay> component + stories.

Deferred to follow-up: IndexedDbCorpus, <AudioFingerprintUploader>, <FingerprintVisualizer>, python-backend, __benchmarks__, integration + purity + property + bundle-size tests.

Quick Start

import { decode } from "@suaveplan/audio-engine-browser/decode";
import {
    LocalBackend,
    MemoryCorpus,
    FingerprintMatchDisplay,
    useAudioFingerprint,
} from "@suaveplan/react-audio-fingerprint";

function MatchSurface({ file }: { file: File }) {
    const [doc, setDoc] = useState<AudioDocument | undefined>(undefined);
    useEffect(() => {
        decode(file).then(setDoc);
    }, [file]);
    const { state, fingerprint } = useAudioFingerprint(doc);
    return state === "done"
        ? <FingerprintMatchDisplay results={[]} />
        : <progress />;
}

Algorithm

AudioFingerprinter composes three deterministic stages:

1. Spectrogram (delegated to @suaveplan/audio-analysis-spectrum).
2. 2D peak detection (peak-detector) — local-max sweep above the noiseFloor, capped at peaksPerFrame per frame.
3. Constellation pair hashing (constellation) — 32-bit packed binA[12] | binB[12] | dt[8] over each anchor's fanout next peaks inside a targetZoneSec window.

Matching is identical to the original Wang (2003) paper: build a 10 ms-wide histogram of (candidateOffset − queryOffset) per shared hash, return the heaviest bin's count as the score and its midpoint as the alignment offset.

License

MIT — see LICENSE at the repo root.