# Capability Spec Delta: audio-fingerprint (enterprise-audio-fingerprint-python) **Package**: `@suaveplan/audio-fingerprint` **Status**: PROPOSED > Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. ## ADDED Requirements ### Requirement: Chromagram-based fingerprinter The package SHALL expose a Python class `AudioFingerprinter` whose constructor accepts `sample_rate`, `n_fft`, `hop_length`, and `n_chroma` and persists each as a public attribute. The instance MUST extract chromagrams from monaural audio using librosa's `chroma_stft` over a `librosa.stft` magnitude spectrogram, returning an `(n_chroma, n_frames)` float matrix whose minimum value is non-negative. #### Scenario: default construction exposes documented defaults - **WHEN** `AudioFingerprinter()` is instantiated with no arguments - **THEN** `sample_rate == 22050`, `n_fft == 2048`, `hop_length == 512`, `n_chroma == 12` #### Scenario: chromagram has correct shape and non-negative values - **WHEN** `extract_chromagram` is called on a 2-second 440 Hz sine wave - **THEN** the returned array has shape `(n_chroma, frames>0)` and `min() >= 0` **Verified by:** `tests/test_fingerprint.py::test_fingerprinter_initialization` AND `tests/test_fingerprint.py::test_fingerprinter_custom_parameters` AND `tests/test_fingerprint.py::test_extract_chromagram` --- ### Requirement: Audio loading from disk `AudioFingerprinter.load_audio` SHALL load an audio file from disk via librosa, downmix to mono, resample to the configured `sample_rate`, and return a `float32` ndarray. It MUST raise `FileNotFoundError` with the message `"Audio file not found: "` when the path does not exist and MUST raise `RuntimeError` wrapping the underlying cause when librosa itself fails. #### Scenario: loads a real WAV - **WHEN** `load_audio` is called on a 2-second WAV - **THEN** the result is a `float32` ndarray with `len > 0` whose values lie in `[-1.0, 1.0]` #### Scenario: missing file raises FileNotFoundError - **WHEN** `load_audio("/nonexistent/file.wav")` is called - **THEN** `FileNotFoundError` is raised matching `Audio file not found` **Verified by:** `tests/test_fingerprint.py::test_load_audio` AND `tests/test_fingerprint.py::test_load_audio_nonexistent_file` --- ### Requirement: 2D spectral peak detection `AudioFingerprinter.find_peaks` SHALL detect local maxima on the normalised chromagram using `scipy.ndimage.maximum_filter` with a `(3, 5)` neighborhood. A coordinate `(t, f)` SHALL be returned only when its normalised amplitude equals the local maximum AND is greater-than-or-equal-to the caller-supplied `threshold`. When the chromagram's maximum is zero the function MUST return an empty list. Returned peaks MUST be `list[tuple[int, int, float]]` of `(time_frame, chroma_bin, amplitude)`. #### Scenario: higher threshold filters out smaller peaks - **WHEN** `find_peaks` is called on the same chromagram with thresholds 0.3 and 0.7 - **THEN** `len(peaks_low) >= len(peaks_high)` #### Scenario: zero-magnitude chromagram returns empty list - **WHEN** `find_peaks` is called on a chromagram whose max is 0 - **THEN** an empty list is returned **Verified by:** `tests/test_fingerprint.py::test_find_peaks` AND `tests/test_fingerprint.py::test_find_peaks_threshold` AND `tests/test_fingerprint.py::test_find_peaks_zero_magnitude` --- ### Requirement: Constellation-map fingerprint hashing `AudioFingerprinter.generate_fingerprint` SHALL pair each peak with the next `fan_value` peaks in time-sorted order and emit `(hash, time_offset, time_delta)` tuples where `hash` is the deterministic hash of `(frequency1, frequency2, time_delta)`. The returned `set` MUST contain only non-negative `time_offset` and `time_delta` values, and MUST grow monotonically as `fan_value` increases. #### Scenario: higher fan value produces at least as many fingerprints - **WHEN** `generate_fingerprint` is called with `fan_value=5` and `fan_value=20` on the same peak list - **THEN** `len(fps_large) >= len(fps_small)` **Verified by:** `tests/test_fingerprint.py::test_generate_fingerprint` AND `tests/test_fingerprint.py::test_generate_fingerprint_fan_value` --- ### Requirement: Full-pipeline file fingerprinting `AudioFingerprinter.fingerprint_file` SHALL run the full pipeline (`load_audio` → `extract_chromagram` → `find_peaks` → `generate_fingerprint`) and return a dict with `fingerprints` (set), `duration` (seconds, float), `peaks_count` (int), and `fingerprints_count` (int) where `fingerprints_count == len(fingerprints)`. #### Scenario: pipeline yields consistent counts - **WHEN** `fingerprint_file` is called on a 2-second WAV - **THEN** the result contains all four keys and `fingerprints_count == len(fingerprints) > 0` **Verified by:** `tests/test_fingerprint.py::test_fingerprint_file` --- ### Requirement: Jaccard-based fingerprint matching with time offset `AudioFingerprinter.match_fingerprints` SHALL compare two fingerprint sets by hash, return Jaccard similarity (`|intersection| / |union|`) as `confidence` (zero when both sides are empty), report `matched = (match_count >= min_matches)`, and estimate `time_offset` as the median per-hash `(reference_time - query_time)` only when `match_count >= min_matches`. When `match_count < min_matches`, `time_offset` MUST default to `0`. #### Scenario: identical inputs yield high confidence and near-zero offset - **WHEN** the same audio is fingerprinted twice and `match_fingerprints` is called - **THEN** `matched is True`, `confidence > 0.9`, `abs(time_offset) <= 5` #### Scenario: impossible threshold forces matched = False - **WHEN** `min_matches=999999` on identical fingerprint sets - **THEN** `matched is False` #### Scenario: empty-on-empty input yields zero confidence and zero offset - **WHEN** `match_fingerprints(set(), set())` is called - **THEN** `match_count == 0`, `confidence == 0`, `time_offset == 0`, `matched is False` **Verified by:** `tests/test_fingerprint.py::test_match_fingerprints_identical` AND `tests/test_fingerprint.py::test_match_fingerprints_min_matches` AND `tests/test_fingerprint.py::test_match_fingerprints_empty` --- ### Requirement: FastAPI health endpoint The FastAPI application SHALL expose `GET /` returning `{"status": "ok", "service": "audio-fingerprinting"}` with HTTP 200. #### Scenario: health check returns the documented payload - **WHEN** the FastAPI TestClient calls `GET /` - **THEN** status is 200 and the JSON body equals `{"status": "ok", "service": "audio-fingerprinting"}` **Verified by:** `tests/test_api.py::test_root_endpoint` --- ### Requirement: FastAPI fingerprint endpoint The FastAPI application SHALL expose `POST /fingerprint` accepting a multipart `file` upload with optional `peak_threshold` (float) and `fan_value` (int) form fields. The response body MUST include `filename`, `duration`, `peaks_count`, `fingerprints_count`, and `fingerprints` (array of `{hash, time, delta}`). The handler MUST persist the upload to a `NamedTemporaryFile`, process it, and unlink the file in a `finally` block. The handler MUST raise HTTP 500 with detail `"Failed to process audio file: "` when the underlying pipeline raises. Posting without a file MUST return HTTP 422. #### Scenario: happy-path returns fingerprints - **WHEN** a WAV file is posted to `/fingerprint` - **THEN** status is 200, `fingerprints_count > 0`, and each `fingerprints[i]` contains `hash`, `time`, `delta` #### Scenario: custom parameters propagate - **WHEN** form data `peak_threshold=0.6 fan_value=20` is posted with a WAV - **THEN** the handler returns 200 with `fingerprints_count > 0` #### Scenario: missing file returns 422 - **WHEN** `POST /fingerprint` is called with no body - **THEN** status is 422 #### Scenario: pipeline failure surfaces as HTTP 500 - **WHEN** a non-audio file is posted to `/fingerprint` - **THEN** status is 500 with `detail` starting with `Failed to process audio file:` **Verified by:** `tests/test_api.py::test_fingerprint_endpoint` AND `tests/test_api.py::test_fingerprint_endpoint_with_parameters` AND `tests/test_api.py::test_fingerprint_endpoint_no_file` AND `tests/test_api.py::test_fingerprint_endpoint_invalid_audio` AND `tests/test_api.py::test_fingerprint_endpoint_async` --- ### Requirement: FastAPI match endpoint The FastAPI application SHALL expose `POST /match` accepting `query_file` and `reference_file` multipart uploads with optional `peak_threshold`, `fan_value`, `min_matches` form fields. The response body MUST include `query_file`, `reference_file`, `query_duration`, `reference_duration`, `matched`, `match_count`, `confidence`, and `time_offset`. The handler MUST persist both uploads to `NamedTemporaryFile`s and unlink them in `finally`. The handler MUST raise HTTP 500 with detail `"Failed to match audio files: "` on pipeline failure. Posting without both files MUST return HTTP 422. #### Scenario: matching identical files returns matched=True with high confidence - **WHEN** the same WAV is posted as `query_file` and `reference_file` - **THEN** status is 200, `matched is True`, `confidence > 0.9`, `abs(time_offset) <= 5` #### Scenario: custom parameters propagate - **WHEN** `peak_threshold=0.6 fan_value=20 min_matches=5` is posted with identical query and reference WAVs - **THEN** status is 200 and `matched is True` #### Scenario: missing files return 422 - **WHEN** `POST /match` is called with no body - **THEN** status is 422 #### Scenario: pipeline failure surfaces as HTTP 500 - **WHEN** a non-audio file is posted as one side of `/match` - **THEN** status is 500 with `detail` starting with `Failed to match audio files:` **Verified by:** `tests/test_api.py::test_match_endpoint` AND `tests/test_api.py::test_match_endpoint_with_parameters` AND `tests/test_api.py::test_match_endpoint_different_files` AND `tests/test_api.py::test_match_endpoint_missing_files` AND `tests/test_api.py::test_match_endpoint_invalid_audio` AND `tests/test_api.py::test_match_endpoint_async` --- ### Requirement: CORS configured for local dev The FastAPI application SHALL respond to preflight `OPTIONS` requests from `http://localhost:3000` and `http://localhost:5173` with `Access-Control-Allow-Origin` echoed back. #### Scenario: preflight from localhost:3000 - **WHEN** the TestClient sends `OPTIONS /fingerprint` with `Origin: http://localhost:3000` - **THEN** the response carries an `access-control-allow-origin` header **Verified by:** `tests/test_api.py::test_cors_headers` --- ### Requirement: TypeScript HTTP client surface The package SHALL export a `PythonAudioService` class (TS) whose constructor accepts an optional `AudioFingerprintConfig` (`baseUrl`, `timeout`) defaulting to `"http://localhost:8000"` and `30000` ms respectively. The class MUST expose `healthCheck()`, `fingerprint(req)`, and `match(req)` methods returning the documented response shapes. Requests MUST be cancelled via `AbortController` after `timeout` ms, raising `Error("Request timeout after : ")`. Error responses MUST be parsed via `await response.json()` and the `detail` field used as the thrown message; if parsing fails the client MUST fall back to `HTTP : `. Non-`Error` thrown values MUST surface as `Error("Unknown error occurred: ")`. #### Scenario: healthCheck returns the parsed body - **WHEN** `client.healthCheck()` is invoked against a stub that returns `{ status: "ok", service: "audio-fingerprinting" }` - **THEN** the resolved value equals that object #### Scenario: fingerprint posts the file and optional params via FormData - **WHEN** `client.fingerprint({ file, peak_threshold: 0.6, fan_value: 20 })` is invoked - **THEN** the request is `POST /fingerprint` with a multipart body whose `file` entry equals the input and whose `peak_threshold` / `fan_value` entries equal `"0.6"` / `"20"` #### Scenario: match posts both files and all optional params - **WHEN** `client.match({ query_file, reference_file, peak_threshold, fan_value, min_matches })` is invoked - **THEN** the request is `POST /match` and every supplied option appears as a string FormData entry #### Scenario: timeout aborts the request - **WHEN** the server fails to respond within `timeout` ms - **THEN** the client rejects with `Error("Request timeout after Nms: ")` #### Scenario: error body without parseable JSON falls back to status text - **WHEN** the server returns HTTP 500 with a body that is not valid JSON - **THEN** the client rejects with `Error("HTTP 500: ")` #### Scenario: error body with `detail` field surfaces the detail - **WHEN** the server returns HTTP 500 with `{ "detail": "Failed to process audio file: bad" }` - **THEN** the client rejects with `Error("Failed to process audio file: bad")` #### Scenario: non-Error thrown by fetch surfaces as Unknown error - **WHEN** the underlying fetch throws a string - **THEN** the client rejects with `Error("Unknown error occurred: ")` **Verified by:** `tests/python-audio-service.test.ts::"healthCheck returns the parsed body"` AND `tests/python-audio-service.test.ts::"fingerprint posts the file and optional params via FormData"` AND `tests/python-audio-service.test.ts::"match posts both files and all optional params"` AND `tests/python-audio-service.test.ts::"timeout aborts the request"` AND `tests/python-audio-service.test.ts::"non-json error body falls back to status text"` AND `tests/python-audio-service.test.ts::"json error body surfaces the detail field"` AND `tests/python-audio-service.test.ts::"non-Error thrown by fetch surfaces as Unknown error"` --- ### Requirement: TypeScript types match Python wire format The package SHALL export TypeScript types `HealthCheckResponse`, `AudioFingerprint`, `FingerprintRequest`, `FingerprintResponse`, `MatchRequest`, `MatchResponse`, `ErrorResponse`, and `AudioFingerprintConfig` whose shapes match the JSON the Python service emits and consumes. The package SHALL provide subpath exports `./client` and `./types` so consumers can pick the wire types without pulling in the client implementation. #### Scenario: subpath imports resolve - **WHEN** a consumer imports `PythonAudioService` from `@suaveplan/audio-fingerprint/client` and types from `@suaveplan/audio-fingerprint/types` - **THEN** both resolve and pass the package's own typecheck pass **Verified by:** `tests/exports.test.ts::"barrel re-exports the client and types"` AND `tests/exports.test.ts::"client subpath re-exports PythonAudioService"` --- ### Requirement: Python coverage and TS coverage at 100% The package SHALL ship with a `pyproject.toml` configuring `pytest --cov=src --cov-branch --cov-fail-under=100` and a `vitest.config.ts` enforcing 100/100/100/100 istanbul thresholds on the TS surface. CI invocations of `pytest` and `bunx turbo test:coverage --filter=@suaveplan/audio-fingerprint` MUST hard-fail below 100%. #### Scenario: pytest enforces 100% line+branch coverage - **WHEN** the test command `micromamba run -n suaveplan-audio-fingerprint pytest --cov=src --cov-branch --cov-fail-under=100` is run from the package directory - **THEN** it exits 0 only when total coverage is 100% #### Scenario: vitest enforces 100% line+branch coverage on the TS bridge - **WHEN** `bunx turbo test:coverage --filter=@suaveplan/audio-fingerprint` is run - **THEN** it exits 0 only when statements, branches, functions, and lines all reach 100% **Verified by:** `pyproject.toml::[tool.coverage.report]` AND `vitest.config.ts::thresholds`