Move docs/ package mirror into document-navigator plugin

Relocate the ~6k-file docs tree from the repo root into
plugins/document-navigator/docs/ so the plugin is self-contained and
portable. docs/superpowers/ (specs and plans) stays at the repo root.

Moved via cp+rm rather than git mv because directory rename() fails with
EPERM on this WSL /mnt DrvFs mount; git rename-detection pairs the
identical-content files at commit time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-30 01:10:00 +01:00
parent fe3fedf42e
commit 9bf56a806b
6022 changed files with 12437 additions and 0 deletions

View File

@@ -0,0 +1,83 @@
# `api/app.py` — FastAPI service
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript `PythonAudioService` client talks to from browsers (via the React-tier sibling) and from server-side JavaScript runtimes. The application exposes a health-check endpoint and two operational endpoints (`POST /fingerprint`, `POST /match`) that accept multipart file uploads and return JSON. It is intentionally a thin REST shell: every business decision (chromagram parameters, peak threshold, fan value, min matches) is driven by client-supplied form fields.
The service is deployment-tier code — it runs under `uvicorn` and is intended to live behind a reverse proxy in production. CORS is pre-configured for the two local development origins (`http://localhost:3000` and `http://localhost:5173`); add production origins via your deployment-time `CORSMiddleware` override.
## Features
- `GET /` health-check returning `{"status": "ok", "service": "audio-fingerprinting"}`.
- `POST /fingerprint` accepting an audio file, optional `peak_threshold` and `fan_value`, returning fingerprint hashes and statistics.
- `POST /match` accepting two audio files, optional `peak_threshold`, `fan_value`, `min_matches`, returning the match decision, count, confidence, and time offset.
- Temporary-file lifecycle: each upload is persisted to `NamedTemporaryFile`, processed, and unlinked in a `finally` block — including on the failure path.
- Uniform HTTP-500 error envelope: pipeline exceptions are caught and re-raised as `HTTPException(status_code=500, detail="Failed to ... : <message>")`.
- CORS pre-configured for `localhost:3000` and `localhost:5173`.
## Usage: Basic
```bash
# Start the service
micromamba run -n suaveplan-audio-fingerprint \
uvicorn src.api.app:app --host 0.0.0.0 --port 8000
# Health check
curl http://localhost:8000/
# Fingerprint a single file
curl -X POST http://localhost:8000/fingerprint -F "file=@track.mp3"
```
## Usage: Advanced
```bash
# Tune the algorithm parameters at request time
curl -X POST http://localhost:8000/fingerprint \
-F "file=@track.flac" \
-F "peak_threshold=0.6" \
-F "fan_value=20"
# Match two files with a stricter min_matches threshold
curl -X POST http://localhost:8000/match \
-F "query_file=@query.wav" \
-F "reference_file=@reference.wav" \
-F "peak_threshold=0.5" \
-F "fan_value=15" \
-F "min_matches=25"
```
```python
# Programmatic in-process testing via httpx ASGITransport
from httpx import ASGITransport, AsyncClient
from src.api.app import app
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
response = await ac.get("/")
assert response.json() == {"status": "ok", "service": "audio-fingerprinting"}
```
## API Reference
| Method | Path | Form fields | Response |
|---|---|---|---|
| `GET` | `/` | — | `{"status": "ok", "service": "audio-fingerprinting"}` |
| `POST` | `/fingerprint` | `file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15) | `{filename, duration, peaks_count, fingerprints_count, fingerprints[{hash,time,delta}]}` |
| `POST` | `/match` | `query_file` (UploadFile, required), `reference_file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15), `min_matches` (int, default 10) | `{query_file, reference_file, query_duration, reference_duration, matched, match_count, confidence, time_offset}` |
### Failure paths
- Missing required upload(s) -> HTTP 422 (FastAPI validation error envelope).
- Pipeline failure (unreadable audio, decode error) -> HTTP 500 with `detail` starting with `Failed to process audio file:` (for `/fingerprint`) or `Failed to match audio files:` (for `/match`).
## Implementation Notes
- A single module-level `AudioFingerprinter()` instance is reused across requests. It has no mutable per-request state; only its analysis parameters are shared, and those are configuration-stable.
- `tempfile.NamedTemporaryFile(delete=False)` is used so the descriptor can be closed while the file is still on disk for librosa to open by path. The unlink lives in a `finally` clause and uses `missing_ok=True` so the cleanup is idempotent even if librosa already moved or removed the file.
- The `match` endpoint persists both uploads inside a single nested `with` block; this guarantees both temp paths are bound before either pipeline call runs, simplifying the cleanup in the `finally` clause.
- `CORSMiddleware` is wired with `allow_credentials=True`, `allow_methods=["*"]`, `allow_headers=["*"]`. The TestClient suite exercises the preflight (`OPTIONS`) path for `http://localhost:3000` to ensure the configuration is live.
- The `version="0.1.0"` field on the `FastAPI(...)` constructor matches the `pyproject.toml` and `package.json` versions; bump all three in lockstep at graduation to `1.0.0`.

View File

@@ -0,0 +1,109 @@
# `client/python-audio-service.ts` — `PythonAudioService`
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio fingerprinting service. It wraps `fetch`, handles multipart `FormData` uploads, propagates `AbortController`-based timeouts, and translates the FastAPI error envelope (`{detail: string}`) into well-typed JavaScript `Error` instances with informative messages. The class is the only runtime export of the `@suaveplan/audio-fingerprint/client` subpath, and it is also re-exported from the package root.
The client deliberately depends on nothing beyond the platform `fetch`, `FormData`, and `AbortController` APIs. Browsers (Chrome, Firefox, Safari 14+, Edge) and modern server runtimes (Node 18+, Bun, Deno) all expose these, so the same code runs unchanged in every consumer tier. There is no axios, no isomorphic-fetch polyfill, and no transitive dependency closure to audit.
## Features
- Configurable `baseUrl` (default `http://localhost:8000`) and `timeout` in ms (default `30000`).
- `healthCheck()` — `GET /`, returns the typed `HealthCheckResponse`.
- `fingerprint(req)` — `POST /fingerprint` with `file` plus optional `peak_threshold` / `fan_value`.
- `match(req)` — `POST /match` with `query_file`, `reference_file`, plus optional `peak_threshold` / `fan_value` / `min_matches`.
- `AbortController`-based timeouts; on expiry the client rejects with `Error("Request timeout after Nms: <endpoint>")`.
- Error parsing: HTTP-non-OK responses are parsed as JSON; if the body is `{detail: "..."}`, that string becomes the `Error.message`. Otherwise the client falls back to `HTTP <status>: <statusText>`.
- Non-`Error` thrown values from `fetch` are normalised to `Error("Unknown error occurred: <string>")`.
## Usage: Basic
```ts
import { PythonAudioService } from "@suaveplan/audio-fingerprint/client";
const client = new PythonAudioService();
const health = await client.healthCheck();
// { status: "ok", service: "audio-fingerprinting" }
```
## Usage: Advanced
```ts
import { PythonAudioService } from "@suaveplan/audio-fingerprint";
import type {
FingerprintResponse,
MatchResponse,
} from "@suaveplan/audio-fingerprint/types";
const client = new PythonAudioService({
baseUrl: "https://audio-fp.internal.example.com",
timeout: 60_000,
});
async function identify(query: File, reference: File): Promise<MatchResponse> {
try {
const queryFingerprint: FingerprintResponse = await client.fingerprint({
file: query,
peak_threshold: 0.55,
fan_value: 18,
});
// ... persist queryFingerprint to corpus if needed
return await client.match({
query_file: query,
reference_file: reference,
peak_threshold: 0.55,
fan_value: 18,
min_matches: 25,
});
} catch (error) {
// The thrown message is either the FastAPI `detail`, a status-text
// fallback, or the timeout / unknown-error envelope.
console.error("identification failed:", (error as Error).message);
throw error;
}
}
```
## API Reference
### Constructor
```ts
constructor(config?: AudioFingerprintConfig)
```
| Field | Type | Default | Description |
|---|---|---|---|
| `config.baseUrl` | `string` | `"http://localhost:8000"` | Base URL of the FastAPI service. |
| `config.timeout` | `number` | `30000` | Per-request timeout in milliseconds. |
### Methods
| Method | Signature | Throws |
|---|---|---|
| `healthCheck` | `() => Promise<HealthCheckResponse>` | `Error` on HTTP error, timeout, or network failure. |
| `fingerprint` | `(req: FingerprintRequest) => Promise<FingerprintResponse>` | `Error` on HTTP error, timeout, or network failure. |
| `match` | `(req: MatchRequest) => Promise<MatchResponse>` | `Error` on HTTP error, timeout, or network failure. |
### Error envelope
| Cause | `Error.message` |
|---|---|
| Timeout | `Request timeout after <Nms>: <endpoint>` |
| HTTP non-OK with parseable `{detail: string}` body | `<detail>` |
| HTTP non-OK with non-JSON or `detail`-missing body | `HTTP <status>: <statusText>` |
| `fetch` throws a non-`Error` value | `Unknown error occurred: <stringified value>` |
| `fetch` throws a non-Abort `Error` | the original error is re-raised verbatim |
## Implementation Notes
- The timeout is implemented with a `setTimeout` that calls `controller.abort()`; the timer is cleared both on success and on failure (try/catch + redundant `clearTimeout` after the `await`). This avoids leaking timers in long-lived browsers tabs.
- `AbortError` is detected by `error.name === "AbortError"` rather than by `instanceof DOMException` to remain runtime-agnostic (DOMException is browser-only).
- Optional form fields are only appended when the caller supplies them; the FastAPI defaults remain authoritative.
- The error-body JSON parse is wrapped in a try/catch with no rethrow inside the catch — a parse failure should NOT mask the original HTTP error, only fall through to the status-text fallback.
- Type-only imports from `../types/audio-fingerprint.types.js` are used so the client carries zero runtime cost from the type definitions.
- The client has no peer dependency on any framework; consumers wrap it in their own React/Vue/Svelte hooks as appropriate. The sibling `@suaveplan/react-audio-fingerprint` package's `python-backend` subpath is the canonical React adapter.

View File

@@ -0,0 +1,64 @@
# `fingerprint.py` — `AudioFingerprinter`
## Purpose
The `AudioFingerprinter` class is the Python core of `@suaveplan/audio-fingerprint`. It implements the full Shazam-style audio identification pipeline: load and downmix the source audio with librosa, run a Short-Time Fourier Transform, distil that magnitude spectrogram into a 12-bin chromagram (pitch-class profile), detect 2D local maxima in the chromagram with `scipy.ndimage.maximum_filter`, hash each peak with its `fan_value` nearest neighbours into a constellation map, and match two fingerprint sets using Jaccard similarity plus median time-offset estimation. It is the high-accuracy backend the React-tier `@suaveplan/react-audio-fingerprint` package is designed to call through the FastAPI bridge.
The pipeline is deliberately chromagram-based rather than raw-magnitude based. Chromagrams fold pitch into 12 bins, so the fingerprint is robust against octave shifts, compression artefacts, mild noise, EQ, and re-encoding — at the cost of being unable to distinguish two different pieces in the same key with the same rhythm. For corpus-scale broadcast monitoring or sample identification, this trade is the right one.
## Features
- Configurable `sample_rate`, `n_fft`, `hop_length`, and `n_chroma` (defaults match the industry-standard 22050/2048/512/12).
- librosa-backed audio loader with mono downmix, target-rate resampling, structured `FileNotFoundError` / `RuntimeError` failures.
- 2D local-maximum peak detection on the normalised chromagram via `scipy.ndimage.maximum_filter` with a `(3, 5)` neighbourhood (3 frequency bins by 5 time frames).
- Constellation-map fingerprint hashing with a configurable `fan_value`.
- O(n+m) Jaccard matcher with median time-offset estimation, defensive against empty inputs and partial overlaps.
- Full pipeline orchestrator `fingerprint_file` that returns duration plus peak and fingerprint counts.
## Usage: Basic
```python
from src.fingerprint import AudioFingerprinter
fp = AudioFingerprinter()
result = fp.fingerprint_file("track.mp3", peak_threshold=0.5, fan_value=15)
print(result["duration"], result["peaks_count"], result["fingerprints_count"])
```
## Usage: Advanced
```python
from src.fingerprint import AudioFingerprinter
fp = AudioFingerprinter(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)
audio = fp.load_audio("query.wav")
chromagram = fp.extract_chromagram(audio) # shape (12, n_frames)
peaks = fp.find_peaks(chromagram, threshold=0.5)
query_fingerprints = fp.generate_fingerprint(peaks, fan_value=15)
reference = fp.fingerprint_file("reference.flac")
match = fp.match_fingerprints(query_fingerprints, reference["fingerprints"], min_matches=20)
print(match["matched"], match["confidence"], match["time_offset"])
```
## API Reference
| Method | Signature | Description |
|---|---|---|
| `__init__` | `(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)` | Persists the analysis parameters as public attributes. |
| `load_audio` | `(file_path: str \| Path) -> NDArray[float32]` | Loads, downmixes to mono, resamples to `sample_rate`. Raises `FileNotFoundError` when the file is missing; raises `RuntimeError("Failed to load audio file: ...")` on librosa decoder failures. |
| `extract_chromagram` | `(audio: NDArray) -> NDArray[float32]` | Computes the chromagram from the STFT magnitude. Output shape: `(n_chroma, n_frames)`. |
| `find_peaks` | `(chromagram: NDArray, threshold=0.5) -> list[tuple[int,int,float]]` | Detects 2D local maxima exceeding `threshold` on the normalised chromagram. Returns `[]` when the chromagram is all-zero. |
| `generate_fingerprint` | `(peaks, fan_value=15) -> set[tuple[int,int,int]]` | Constellation-map hashing; each tuple is `(hash, time_offset, time_delta)`. |
| `fingerprint_file` | `(file_path, peak_threshold=0.5, fan_value=15) -> dict` | Runs the full pipeline. Returns `fingerprints`, `duration`, `peaks_count`, `fingerprints_count`. |
| `match_fingerprints` | `(query, reference, min_matches=10) -> dict` | Returns `matched`, `match_count`, `confidence` (Jaccard), `time_offset` (median frame offset, 0 when no positive match). |
## Implementation Notes
- The chromagram normalisation is done in `find_peaks` rather than `extract_chromagram` so the threshold semantics are stable across signals of differing absolute amplitude.
- The `(3, 5)` neighbourhood is intentionally asymmetric: time has more frames than there are pitch classes, so we accept a wider time window before considering a point a local maximum.
- Constellation hashing uses Python's built-in `hash((f1, f2, dt))`; the values are not cryptographically stable across interpreters but they are stable within a single run, which is all the matcher needs.
- The matcher builds an O(m) `hash -> reference_time` lookup table before iterating the query set; this turns the original upstream's nested O(n*m) loop into O(n + m) while preserving the median-of-offsets semantics.
- An empty union of hashes is treated as a Jaccard denominator of `max(union_size, 1)`, so the empty/empty case returns confidence 0 cleanly rather than dividing by zero.
- The `if match_count >= min_matches and matching_hashes:` guard catches the edge case `min_matches=0` with two empty inputs, preventing `np.median([])` from raising.