first commit

docs/media/audio/audio-fingerprint/README.md

@@ -0,0 +1,88 @@
+# @suaveplan/audio-fingerprint - Python-tier high-accuracy audio fingerprinting service (librosa chromagram + scipy 2D peak detection + constellation hashing + Jaccard matching), exposed as a FastAPI REST API, with a typed TypeScript HTTP client (PythonAudioService) for browser/Node consumers.
+
+High-accuracy audio fingerprinting service. The package is dual-runtime: a Python FastAPI backend (`src/fingerprint.py` + `src/api/app.py`) implements chromagram-based Shazam-style fingerprinting on top of librosa and scipy, and a TypeScript HTTP client (`src/client/python-audio-service.ts`) exposes that service to browser and Node consumers. It is the high-accuracy backend behind the browser-tier `@suaveplan/react-audio-fingerprint` React surface.
+
+<!-- begin:auto-docs -->
+## Documentation tree
+
+- [api](./src/api/app.md) — `api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript…
+  - [app](./src/api/app.md) — `api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript…
+- [client](./src/client/python-audio-service.md) — `PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio finge…
+  - [python-audio-service](./src/client/python-audio-service.md) — `PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio finge…
+- [fingerprint](./src/fingerprint.md) — The `AudioFingerprinter` class is the Python core of `@suaveplan/audio-fingerprint`. It implements the full Shazam-styl…
+<!-- end:auto-docs -->
+
+## Install
+
+```bash
+# TypeScript client side
+bun add @suaveplan/audio-fingerprint
+
+# Python service side (creates the suaveplan-audio-fingerprint micromamba env)
+cd packages/media/audio/audio-fingerprint
+micromamba create -f environment.yml -y
+```
+
+## Quick start
+
+```bash
+# Start the FastAPI service on port 8000
+micromamba run -n suaveplan-audio-fingerprint \
+  uvicorn src.api.app:app --host 0.0.0.0 --port 8000
+```
+
+```ts
+import { PythonAudioService } from "@suaveplan/audio-fingerprint";
+
+const client = new PythonAudioService({ baseUrl: "http://localhost:8000" });
+const health = await client.healthCheck();
+const fingerprint = await client.fingerprint({ file: someAudioFile });
+const match = await client.match({
+    query_file: clipFile,
+    reference_file: corpusFile,
+    min_matches: 20,
+});
+```
+
+## Architecture
+
+The package is split into two runtimes that share the wire format documented in `src/types/audio-fingerprint.types.ts`:
+
+- **Python tier** — `src/fingerprint.py` implements the `AudioFingerprinter` class (chromagram extraction via librosa, 2D peak detection via `scipy.ndimage.maximum_filter`, constellation-map hashing, Jaccard matching with median time-offset estimation). `src/api/app.py` wraps it in a FastAPI service with `GET /`, `POST /fingerprint`, and `POST /match` endpoints, plus CORS for `localhost:3000` and `localhost:5173`.
+- **TypeScript tier** — `src/client/python-audio-service.ts` is the typed HTTP client. It depends only on the platform `fetch`, `FormData`, and `AbortController` APIs and runs unchanged in browsers and modern Node runtimes.
+
+## Algorithm
+
+1. **Audio loading** — mono downmix and resample to 22050 Hz (configurable).
+2. **STFT** — `n_fft=2048`, `hop_length=512` (configurable).
+3. **Chromagram** — 12-bin pitch-class profile (configurable) via `librosa.feature.chroma_stft`.
+4. **Peak detection** — 2D local maxima on the normalised chromagram using a `(3, 5)` neighbourhood maximum filter; peaks below `peak_threshold` are discarded.
+5. **Constellation map** — each peak is paired with the next `fan_value` peaks in time-sorted order.
+6. **Hashing** — each pair becomes a `(hash(f1, f2, dt), t1, dt)` tuple.
+7. **Matching** — set intersection on hashes, Jaccard similarity for confidence, median per-hash time offset for alignment.
+
+## Modules
+
+- [`src/fingerprint.py`](./src/fingerprint.py) — `AudioFingerprinter` core. Docs: [`src/fingerprint.md`](./src/fingerprint.md).
+- [`src/api/app.py`](./src/api/app.py) — FastAPI service. Docs: [`src/api/app.md`](./src/api/app.md).
+- [`src/client/python-audio-service.ts`](./src/client/python-audio-service.ts) — typed HTTP client. Docs: [`src/client/python-audio-service.md`](./src/client/python-audio-service.md).
+- [`src/types/audio-fingerprint.types.ts`](./src/types/audio-fingerprint.types.ts) — wire-format type definitions (re-exported from `@suaveplan/audio-fingerprint/types`).
+
+## Testing
+
+```bash
+# TypeScript bridge tests + 100/100/100/100 istanbul coverage
+bunx turbo test:coverage --filter=@suaveplan/audio-fingerprint
+
+# Python service tests + 100% line+branch pytest coverage
+cd packages/media/audio/audio-fingerprint
+micromamba run -n suaveplan-audio-fingerprint pytest
+```
+
+Both gates hard-fail below 100% coverage. The Python test suite is not currently wired into `turbo`; it must be invoked standalone via `micromamba`. See  for the rationale and the proposed follow-up to wire a `turbo` task once the broader Python tooling story lands.
+
+## See also
+
+- `@suaveplan/react-audio-fingerprint` — the React 19 sibling that runs the browser-tier fingerprinter and, via this package's `python-backend` integration, consults this service for high-accuracy matching.
+- `@suaveplan/audio-analysis-spectrum` — the universal FFT/spectrogram primitives the browser-tier fingerprinter is built on.
+

docs/media/audio/audio-fingerprint/src/api/app.md

@@ -0,0 +1,83 @@
+# `api/app.py` — FastAPI service
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+`api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript `PythonAudioService` client talks to from browsers (via the React-tier sibling) and from server-side JavaScript runtimes. The application exposes a health-check endpoint and two operational endpoints (`POST /fingerprint`, `POST /match`) that accept multipart file uploads and return JSON. It is intentionally a thin REST shell: every business decision (chromagram parameters, peak threshold, fan value, min matches) is driven by client-supplied form fields.
+
+The service is deployment-tier code — it runs under `uvicorn` and is intended to live behind a reverse proxy in production. CORS is pre-configured for the two local development origins (`http://localhost:3000` and `http://localhost:5173`); add production origins via your deployment-time `CORSMiddleware` override.
+
+## Features
+
+- `GET /` health-check returning `{"status": "ok", "service": "audio-fingerprinting"}`.
+- `POST /fingerprint` accepting an audio file, optional `peak_threshold` and `fan_value`, returning fingerprint hashes and statistics.
+- `POST /match` accepting two audio files, optional `peak_threshold`, `fan_value`, `min_matches`, returning the match decision, count, confidence, and time offset.
+- Temporary-file lifecycle: each upload is persisted to `NamedTemporaryFile`, processed, and unlinked in a `finally` block — including on the failure path.
+- Uniform HTTP-500 error envelope: pipeline exceptions are caught and re-raised as `HTTPException(status_code=500, detail="Failed to ... : <message>")`.
+- CORS pre-configured for `localhost:3000` and `localhost:5173`.
+
+## Usage: Basic
+
+```bash
+# Start the service
+micromamba run -n suaveplan-audio-fingerprint \
+  uvicorn src.api.app:app --host 0.0.0.0 --port 8000
+
+# Health check
+curl http://localhost:8000/
+
+# Fingerprint a single file
+curl -X POST http://localhost:8000/fingerprint -F "file=@track.mp3"
+```
+
+## Usage: Advanced
+
+```bash
+# Tune the algorithm parameters at request time
+curl -X POST http://localhost:8000/fingerprint \
+  -F "file=@track.flac" \
+  -F "peak_threshold=0.6" \
+  -F "fan_value=20"
+
+# Match two files with a stricter min_matches threshold
+curl -X POST http://localhost:8000/match \
+  -F "query_file=@query.wav" \
+  -F "reference_file=@reference.wav" \
+  -F "peak_threshold=0.5" \
+  -F "fan_value=15" \
+  -F "min_matches=25"
+```
+
+```python
+# Programmatic in-process testing via httpx ASGITransport
+from httpx import ASGITransport, AsyncClient
+from src.api.app import app
+
+async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
+    response = await ac.get("/")
+    assert response.json() == {"status": "ok", "service": "audio-fingerprinting"}
+```
+
+## API Reference
+
+| Method | Path | Form fields | Response |
+|---|---|---|---|
+| `GET` | `/` | — | `{"status": "ok", "service": "audio-fingerprinting"}` |
+| `POST` | `/fingerprint` | `file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15) | `{filename, duration, peaks_count, fingerprints_count, fingerprints[{hash,time,delta}]}` |
+| `POST` | `/match` | `query_file` (UploadFile, required), `reference_file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15), `min_matches` (int, default 10) | `{query_file, reference_file, query_duration, reference_duration, matched, match_count, confidence, time_offset}` |
+
+### Failure paths
+
+- Missing required upload(s) -> HTTP 422 (FastAPI validation error envelope).
+- Pipeline failure (unreadable audio, decode error) -> HTTP 500 with `detail` starting with `Failed to process audio file:` (for `/fingerprint`) or `Failed to match audio files:` (for `/match`).
+
+## Implementation Notes
+
+- A single module-level `AudioFingerprinter()` instance is reused across requests. It has no mutable per-request state; only its analysis parameters are shared, and those are configuration-stable.
+- `tempfile.NamedTemporaryFile(delete=False)` is used so the descriptor can be closed while the file is still on disk for librosa to open by path. The unlink lives in a `finally` clause and uses `missing_ok=True` so the cleanup is idempotent even if librosa already moved or removed the file.
+- The `match` endpoint persists both uploads inside a single nested `with` block; this guarantees both temp paths are bound before either pipeline call runs, simplifying the cleanup in the `finally` clause.
+- `CORSMiddleware` is wired with `allow_credentials=True`, `allow_methods=["*"]`, `allow_headers=["*"]`. The TestClient suite exercises the preflight (`OPTIONS`) path for `http://localhost:3000` to ensure the configuration is live.
+- The `version="0.1.0"` field on the `FastAPI(...)` constructor matches the `pyproject.toml` and `package.json` versions; bump all three in lockstep at graduation to `1.0.0`.

docs/media/audio/audio-fingerprint/src/client/python-audio-service.md

@@ -0,0 +1,109 @@
+# `client/python-audio-service.ts` — `PythonAudioService`
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+`PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio fingerprinting service. It wraps `fetch`, handles multipart `FormData` uploads, propagates `AbortController`-based timeouts, and translates the FastAPI error envelope (`{detail: string}`) into well-typed JavaScript `Error` instances with informative messages. The class is the only runtime export of the `@suaveplan/audio-fingerprint/client` subpath, and it is also re-exported from the package root.
+
+The client deliberately depends on nothing beyond the platform `fetch`, `FormData`, and `AbortController` APIs. Browsers (Chrome, Firefox, Safari 14+, Edge) and modern server runtimes (Node 18+, Bun, Deno) all expose these, so the same code runs unchanged in every consumer tier. There is no axios, no isomorphic-fetch polyfill, and no transitive dependency closure to audit.
+
+## Features
+
+- Configurable `baseUrl` (default `http://localhost:8000`) and `timeout` in ms (default `30000`).
+- `healthCheck()` — `GET /`, returns the typed `HealthCheckResponse`.
+- `fingerprint(req)` — `POST /fingerprint` with `file` plus optional `peak_threshold` / `fan_value`.
+- `match(req)` — `POST /match` with `query_file`, `reference_file`, plus optional `peak_threshold` / `fan_value` / `min_matches`.
+- `AbortController`-based timeouts; on expiry the client rejects with `Error("Request timeout after Nms: <endpoint>")`.
+- Error parsing: HTTP-non-OK responses are parsed as JSON; if the body is `{detail: "..."}`, that string becomes the `Error.message`. Otherwise the client falls back to `HTTP <status>: <statusText>`.
+- Non-`Error` thrown values from `fetch` are normalised to `Error("Unknown error occurred: <string>")`.
+
+## Usage: Basic
+
+```ts
+import { PythonAudioService } from "@suaveplan/audio-fingerprint/client";
+
+const client = new PythonAudioService();
+const health = await client.healthCheck();
+// { status: "ok", service: "audio-fingerprinting" }
+```
+
+## Usage: Advanced
+
+```ts
+import { PythonAudioService } from "@suaveplan/audio-fingerprint";
+import type {
+    FingerprintResponse,
+    MatchResponse,
+} from "@suaveplan/audio-fingerprint/types";
+
+const client = new PythonAudioService({
+    baseUrl: "https://audio-fp.internal.example.com",
+    timeout: 60_000,
+});
+
+async function identify(query: File, reference: File): Promise<MatchResponse> {
+    try {
+        const queryFingerprint: FingerprintResponse = await client.fingerprint({
+            file: query,
+            peak_threshold: 0.55,
+            fan_value: 18,
+        });
+        // ... persist queryFingerprint to corpus if needed
+        return await client.match({
+            query_file: query,
+            reference_file: reference,
+            peak_threshold: 0.55,
+            fan_value: 18,
+            min_matches: 25,
+        });
+    } catch (error) {
+        // The thrown message is either the FastAPI `detail`, a status-text
+        // fallback, or the timeout / unknown-error envelope.
+        console.error("identification failed:", (error as Error).message);
+        throw error;
+    }
+}
+```
+
+## API Reference
+
+### Constructor
+
+```ts
+constructor(config?: AudioFingerprintConfig)
+```
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `config.baseUrl` | `string` | `"http://localhost:8000"` | Base URL of the FastAPI service. |
+| `config.timeout` | `number` | `30000` | Per-request timeout in milliseconds. |
+
+### Methods
+
+| Method | Signature | Throws |
+|---|---|---|
+| `healthCheck` | `() => Promise<HealthCheckResponse>` | `Error` on HTTP error, timeout, or network failure. |
+| `fingerprint` | `(req: FingerprintRequest) => Promise<FingerprintResponse>` | `Error` on HTTP error, timeout, or network failure. |
+| `match` | `(req: MatchRequest) => Promise<MatchResponse>` | `Error` on HTTP error, timeout, or network failure. |
+
+### Error envelope
+
+| Cause | `Error.message` |
+|---|---|
+| Timeout | `Request timeout after <Nms>: <endpoint>` |
+| HTTP non-OK with parseable `{detail: string}` body | `<detail>` |
+| HTTP non-OK with non-JSON or `detail`-missing body | `HTTP <status>: <statusText>` |
+| `fetch` throws a non-`Error` value | `Unknown error occurred: <stringified value>` |
+| `fetch` throws a non-Abort `Error` | the original error is re-raised verbatim |
+
+## Implementation Notes
+
+- The timeout is implemented with a `setTimeout` that calls `controller.abort()`; the timer is cleared both on success and on failure (try/catch + redundant `clearTimeout` after the `await`). This avoids leaking timers in long-lived browsers tabs.
+- `AbortError` is detected by `error.name === "AbortError"` rather than by `instanceof DOMException` to remain runtime-agnostic (DOMException is browser-only).
+- Optional form fields are only appended when the caller supplies them; the FastAPI defaults remain authoritative.
+- The error-body JSON parse is wrapped in a try/catch with no rethrow inside the catch — a parse failure should NOT mask the original HTTP error, only fall through to the status-text fallback.
+- Type-only imports from `../types/audio-fingerprint.types.js` are used so the client carries zero runtime cost from the type definitions.
+- The client has no peer dependency on any framework; consumers wrap it in their own React/Vue/Svelte hooks as appropriate. The sibling `@suaveplan/react-audio-fingerprint` package's `python-backend` subpath is the canonical React adapter.

docs/media/audio/audio-fingerprint/src/fingerprint.md

@@ -0,0 +1,64 @@
+# `fingerprint.py` — `AudioFingerprinter`
+
+## Purpose
+
+The `AudioFingerprinter` class is the Python core of `@suaveplan/audio-fingerprint`. It implements the full Shazam-style audio identification pipeline: load and downmix the source audio with librosa, run a Short-Time Fourier Transform, distil that magnitude spectrogram into a 12-bin chromagram (pitch-class profile), detect 2D local maxima in the chromagram with `scipy.ndimage.maximum_filter`, hash each peak with its `fan_value` nearest neighbours into a constellation map, and match two fingerprint sets using Jaccard similarity plus median time-offset estimation. It is the high-accuracy backend the React-tier `@suaveplan/react-audio-fingerprint` package is designed to call through the FastAPI bridge.
+
+The pipeline is deliberately chromagram-based rather than raw-magnitude based. Chromagrams fold pitch into 12 bins, so the fingerprint is robust against octave shifts, compression artefacts, mild noise, EQ, and re-encoding — at the cost of being unable to distinguish two different pieces in the same key with the same rhythm. For corpus-scale broadcast monitoring or sample identification, this trade is the right one.
+
+## Features
+
+- Configurable `sample_rate`, `n_fft`, `hop_length`, and `n_chroma` (defaults match the industry-standard 22050/2048/512/12).
+- librosa-backed audio loader with mono downmix, target-rate resampling, structured `FileNotFoundError` / `RuntimeError` failures.
+- 2D local-maximum peak detection on the normalised chromagram via `scipy.ndimage.maximum_filter` with a `(3, 5)` neighbourhood (3 frequency bins by 5 time frames).
+- Constellation-map fingerprint hashing with a configurable `fan_value`.
+- O(n+m) Jaccard matcher with median time-offset estimation, defensive against empty inputs and partial overlaps.
+- Full pipeline orchestrator `fingerprint_file` that returns duration plus peak and fingerprint counts.
+
+## Usage: Basic
+
+```python
+from src.fingerprint import AudioFingerprinter
+
+fp = AudioFingerprinter()
+result = fp.fingerprint_file("track.mp3", peak_threshold=0.5, fan_value=15)
+print(result["duration"], result["peaks_count"], result["fingerprints_count"])
+```
+
+## Usage: Advanced
+
+```python
+from src.fingerprint import AudioFingerprinter
+
+fp = AudioFingerprinter(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)
+
+audio = fp.load_audio("query.wav")
+chromagram = fp.extract_chromagram(audio)  # shape (12, n_frames)
+peaks = fp.find_peaks(chromagram, threshold=0.5)
+query_fingerprints = fp.generate_fingerprint(peaks, fan_value=15)
+
+reference = fp.fingerprint_file("reference.flac")
+match = fp.match_fingerprints(query_fingerprints, reference["fingerprints"], min_matches=20)
+print(match["matched"], match["confidence"], match["time_offset"])
+```
+
+## API Reference
+
+| Method | Signature | Description |
+|---|---|---|
+| `__init__` | `(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)` | Persists the analysis parameters as public attributes. |
+| `load_audio` | `(file_path: str \| Path) -> NDArray[float32]` | Loads, downmixes to mono, resamples to `sample_rate`. Raises `FileNotFoundError` when the file is missing; raises `RuntimeError("Failed to load audio file: ...")` on librosa decoder failures. |
+| `extract_chromagram` | `(audio: NDArray) -> NDArray[float32]` | Computes the chromagram from the STFT magnitude. Output shape: `(n_chroma, n_frames)`. |
+| `find_peaks` | `(chromagram: NDArray, threshold=0.5) -> list[tuple[int,int,float]]` | Detects 2D local maxima exceeding `threshold` on the normalised chromagram. Returns `[]` when the chromagram is all-zero. |
+| `generate_fingerprint` | `(peaks, fan_value=15) -> set[tuple[int,int,int]]` | Constellation-map hashing; each tuple is `(hash, time_offset, time_delta)`. |
+| `fingerprint_file` | `(file_path, peak_threshold=0.5, fan_value=15) -> dict` | Runs the full pipeline. Returns `fingerprints`, `duration`, `peaks_count`, `fingerprints_count`. |
+| `match_fingerprints` | `(query, reference, min_matches=10) -> dict` | Returns `matched`, `match_count`, `confidence` (Jaccard), `time_offset` (median frame offset, 0 when no positive match). |
+
+## Implementation Notes
+
+- The chromagram normalisation is done in `find_peaks` rather than `extract_chromagram` so the threshold semantics are stable across signals of differing absolute amplitude.
+- The `(3, 5)` neighbourhood is intentionally asymmetric: time has more frames than there are pitch classes, so we accept a wider time window before considering a point a local maximum.
+- Constellation hashing uses Python's built-in `hash((f1, f2, dt))`; the values are not cryptographically stable across interpreters but they are stable within a single run, which is all the matcher needs.
+- The matcher builds an O(m) `hash -> reference_time` lookup table before iterating the query set; this turns the original upstream's nested O(n*m) loop into O(n + m) while preserving the median-of-offsets semantics.
+- An empty union of hashes is treated as a Jaccard denominator of `max(union_size, 1)`, so the empty/empty case returns confidence 0 cleanly rather than dividing by zero.
+- The `if match_count >= min_matches and matching_hashes:` guard catches the edge case `min_matches=0` with two empty inputs, preventing `np.median([])` from raising.