SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
admin
2026-05-27 23:47:53 +01:00
parent fbc70f96e6
commit 9b28db724f
18103 changed files with 1600554 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
packages/media/audio/audio-fingerprint/.gitignore vendored Normal file
View File

@@ -0,0 +1,11 @@
dist/
node_modules/
coverage/
htmlcov/
.coverage
*.tsbuildinfo
__pycache__/
*.pyc
.pytest_cache/
.mypy_cache/
.ruff_cache/

88
packages/media/audio/audio-fingerprint/README.md Normal file
View File

@@ -0,0 +1,88 @@
# @suaveplan/audio-fingerprint - Python-tier high-accuracy audio fingerprinting service (librosa chromagram + scipy 2D peak detection + constellation hashing + Jaccard matching), exposed as a FastAPI REST API, with a typed TypeScript HTTP client (PythonAudioService) for browser/Node consumers.
High-accuracy audio fingerprinting service. The package is dual-runtime: a Python FastAPI backend (`src/fingerprint.py` + `src/api/app.py`) implements chromagram-based Shazam-style fingerprinting on top of librosa and scipy, and a TypeScript HTTP client (`src/client/python-audio-service.ts`) exposes that service to browser and Node consumers. It is the high-accuracy backend behind the browser-tier `@suaveplan/react-audio-fingerprint` React surface.
<!-- begin:auto-docs -->
## Documentation tree
- [api](./src/api/app.md) — `api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript…
- [app](./src/api/app.md) — `api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript…
- [client](./src/client/python-audio-service.md) — `PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio finge…
- [python-audio-service](./src/client/python-audio-service.md) — `PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio finge…
- [fingerprint](./src/fingerprint.md) — The `AudioFingerprinter` class is the Python core of `@suaveplan/audio-fingerprint`. It implements the full Shazam-styl…
<!-- end:auto-docs -->
## Install
```bash
# TypeScript client side
bun add @suaveplan/audio-fingerprint
# Python service side (creates the suaveplan-audio-fingerprint micromamba env)
cd packages/media/audio/audio-fingerprint
micromamba create -f environment.yml -y
```
## Quick start
```bash
# Start the FastAPI service on port 8000
micromamba run -n suaveplan-audio-fingerprint \
uvicorn src.api.app:app --host 0.0.0.0 --port 8000
```
```ts
import { PythonAudioService } from "@suaveplan/audio-fingerprint";
const client = new PythonAudioService({ baseUrl: "http://localhost:8000" });
const health = await client.healthCheck();
const fingerprint = await client.fingerprint({ file: someAudioFile });
const match = await client.match({
query_file: clipFile,
reference_file: corpusFile,
min_matches: 20,
});
```
## Architecture
The package is split into two runtimes that share the wire format documented in `src/types/audio-fingerprint.types.ts`:
- **Python tier** — `src/fingerprint.py` implements the `AudioFingerprinter` class (chromagram extraction via librosa, 2D peak detection via `scipy.ndimage.maximum_filter`, constellation-map hashing, Jaccard matching with median time-offset estimation). `src/api/app.py` wraps it in a FastAPI service with `GET /`, `POST /fingerprint`, and `POST /match` endpoints, plus CORS for `localhost:3000` and `localhost:5173`.
- **TypeScript tier** — `src/client/python-audio-service.ts` is the typed HTTP client. It depends only on the platform `fetch`, `FormData`, and `AbortController` APIs and runs unchanged in browsers and modern Node runtimes.
## Algorithm
1. **Audio loading** — mono downmix and resample to 22050 Hz (configurable).
2. **STFT** — `n_fft=2048`, `hop_length=512` (configurable).
3. **Chromagram** — 12-bin pitch-class profile (configurable) via `librosa.feature.chroma_stft`.
4. **Peak detection** — 2D local maxima on the normalised chromagram using a `(3, 5)` neighbourhood maximum filter; peaks below `peak_threshold` are discarded.
5. **Constellation map** — each peak is paired with the next `fan_value` peaks in time-sorted order.
6. **Hashing** — each pair becomes a `(hash(f1, f2, dt), t1, dt)` tuple.
7. **Matching** — set intersection on hashes, Jaccard similarity for confidence, median per-hash time offset for alignment.
## Modules
- [`src/fingerprint.py`](./src/fingerprint.py) — `AudioFingerprinter` core. Docs: [`src/fingerprint.md`](./src/fingerprint.md).
- [`src/api/app.py`](./src/api/app.py) — FastAPI service. Docs: [`src/api/app.md`](./src/api/app.md).
- [`src/client/python-audio-service.ts`](./src/client/python-audio-service.ts) — typed HTTP client. Docs: [`src/client/python-audio-service.md`](./src/client/python-audio-service.md).
- [`src/types/audio-fingerprint.types.ts`](./src/types/audio-fingerprint.types.ts) — wire-format type definitions (re-exported from `@suaveplan/audio-fingerprint/types`).
## Testing
```bash
# TypeScript bridge tests + 100/100/100/100 istanbul coverage
bunx turbo test:coverage --filter=@suaveplan/audio-fingerprint
# Python service tests + 100% line+branch pytest coverage
cd packages/media/audio/audio-fingerprint
micromamba run -n suaveplan-audio-fingerprint pytest
```
Both gates hard-fail below 100% coverage. The Python test suite is not currently wired into `turbo`; it must be invoked standalone via `micromamba`. See for the rationale and the proposed follow-up to wire a `turbo` task once the broader Python tooling story lands.
## See also
- `@suaveplan/react-audio-fingerprint` — the React 19 sibling that runs the browser-tier fingerprinter and, via this package's `python-backend` integration, consults this service for high-accuracy matching.
- `@suaveplan/audio-analysis-spectrum` — the universal FFT/spectrogram primitives the browser-tier fingerprinter is built on.

20
packages/media/audio/audio-fingerprint/environment.yml Normal file
View File

@@ -0,0 +1,20 @@
name: suaveplan-audio-fingerprint
channels:
- conda-forge
dependencies:
- python=3.11
- numpy>=1.24.0,<2.0.0
- scipy>=1.10.0,<2.0.0
- pip
- pip:
- librosa>=0.10.0,<1.0.0
- soundfile>=0.12.0
- fastapi>=0.104.0
- uvicorn>=0.24.0
- python-multipart>=0.0.6
- pytest>=7.4.0
- pytest-cov>=4.1.0
- pytest-asyncio>=0.21.0
- httpx>=0.25.0
- ruff>=0.1.0
- mypy>=1.7.0

78
packages/media/audio/audio-fingerprint/package.json Normal file
View File

@@ -0,0 +1,78 @@
{
"name": "@suaveplan/audio-fingerprint",
"version": "0.1.0",
"type": "module",
"sideEffects": false,
"description": "Python-tier high-accuracy audio fingerprinting service (librosa chromagram + scipy 2D peak detection + constellation hashing + Jaccard matching), exposed as a FastAPI REST API, with a typed TypeScript HTTP client (PythonAudioService) for browser/Node consumers.",
"license": "MIT",
"author": "Suaveplan Team",
"main": "./dist/index.js",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
"./package.json": "./package.json",
".": {
"import": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"./client": {
"import": {
"types": "./dist/client/index.d.ts",
"default": "./dist/client/index.js"
}
},
"./client/python-audio-service": {
"import": {
"types": "./dist/client/python-audio-service.d.ts",
"default": "./dist/client/python-audio-service.js"
}
},
"./types": {
"import": {
"types": "./dist/types/index.d.ts",
"default": "./dist/types/index.js"
}
}
},
"scripts": {
"build": "vite build",
"test": "vitest run --passWithNoTests",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage --passWithNoTests",
"test:python": "micromamba run -n suaveplan-audio-fingerprint pytest",
"test:python:install": "micromamba create -f environment.yml -y",
"lint": "bunx biome check src/ tests/",
"lint:fix": "bunx biome lint --fix --unsafe --max-diagnostics 500 src/ tests/",
"format": "bunx biome format --write --max-diagnostics 500 src/ tests/",
"check": "bunx biome check --write --unsafe --max-diagnostics 500 src/ tests/",
"typecheck": "bunx tsc --noEmit --composite false -p tsconfig.lib.json",
"docs:check": "bun run ../../../../scripts/docs-check.ts .",
"dev": "micromamba run -n suaveplan-audio-fingerprint uvicorn src.api.app:app --reload --host 0.0.0.0 --port 8000",
"start": "micromamba run -n suaveplan-audio-fingerprint uvicorn src.api.app:app --host 0.0.0.0 --port 8000"
},
"dependencies": {
"tslib": "catalog:build"
},
"devDependencies": {
"@suaveplan/testing": "workspace:^",
"@types/node": "catalog:",
"@vitest/coverage-istanbul": "catalog:testing",
"typescript": "catalog:",
"vite": "catalog:build",
"vite-plugin-dts": "catalog:build",
"vitest": "catalog:testing"
},
"publishConfig": {
"registry": "https://npmjs.fq.io/",
"access": "public"
},
"files": [
"dist",
"src",
"environment.yml",
"pyproject.toml",
"README.md"
]
}

92
packages/media/audio/audio-fingerprint/pyproject.toml Normal file
View File

@@ -0,0 +1,92 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "suaveplan-audio-fingerprint"
version = "0.1.0"
description = "High-accuracy audio fingerprinting and matching service (chromagram + constellation hashing) for Suaveplan genesis."
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
authors = [{name = "Suaveplan", email = "hello@suaveplan.com"}]
keywords = ["audio", "fingerprinting", "librosa", "chromagram", "fastapi", "shazam"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
dependencies = [
"numpy>=1.24.0",
"librosa>=0.10.0",
"soundfile>=0.12.0",
"scipy>=1.10.0",
"fastapi>=0.104.0",
"uvicorn[standard]>=0.24.0",
"python-multipart>=0.0.6",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"pytest-asyncio>=0.21.0",
"pytest-cov>=4.1.0",
"httpx>=0.25.0",
"ruff>=0.1.0",
"mypy>=1.7.0",
]
[tool.setuptools.packages.find]
where = ["."]
include = ["src*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
asyncio_mode = "auto"
addopts = [
"-ra",
"--strict-markers",
"--strict-config",
"--cov=src",
"--cov-branch",
"--cov-report=term-missing",
"--cov-fail-under=100",
]
[tool.coverage.run]
source = ["src"]
branch = true
omit = [
"*/tests/*",
"*/__pycache__/*",
"*/.venv/*",
]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"raise NotImplementedError",
"if TYPE_CHECKING:",
]
precision = 2
show_missing = true
fail_under = 100
[tool.ruff]
target-version = "py311"
line-length = 100
indent-width = 4
[tool.ruff.lint]
select = ["E", "W", "F", "I", "N", "UP", "B", "C4", "SIM", "TCH"]
ignore = ["E501"]
[tool.mypy]
python_version = "3.11"
strict = true
ignore_missing_imports = true

5
packages/media/audio/audio-fingerprint/src/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Suaveplan audio-fingerprint Python package."""
from .fingerprint import AudioFingerprinter
__all__ = ["AudioFingerprinter"]

5
packages/media/audio/audio-fingerprint/src/api/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""FastAPI app for the audio-fingerprinting service."""
from .app import app
__all__ = ["app"]

83
packages/media/audio/audio-fingerprint/src/api/app.md Normal file
View File

@@ -0,0 +1,83 @@
# `api/app.py` — FastAPI service
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`api/app.py` wraps the `AudioFingerprinter` core into a FastAPI HTTP service. It is the network surface the TypeScript `PythonAudioService` client talks to from browsers (via the React-tier sibling) and from server-side JavaScript runtimes. The application exposes a health-check endpoint and two operational endpoints (`POST /fingerprint`, `POST /match`) that accept multipart file uploads and return JSON. It is intentionally a thin REST shell: every business decision (chromagram parameters, peak threshold, fan value, min matches) is driven by client-supplied form fields.
The service is deployment-tier code — it runs under `uvicorn` and is intended to live behind a reverse proxy in production. CORS is pre-configured for the two local development origins (`http://localhost:3000` and `http://localhost:5173`); add production origins via your deployment-time `CORSMiddleware` override.
## Features
- `GET /` health-check returning `{"status": "ok", "service": "audio-fingerprinting"}`.
- `POST /fingerprint` accepting an audio file, optional `peak_threshold` and `fan_value`, returning fingerprint hashes and statistics.
- `POST /match` accepting two audio files, optional `peak_threshold`, `fan_value`, `min_matches`, returning the match decision, count, confidence, and time offset.
- Temporary-file lifecycle: each upload is persisted to `NamedTemporaryFile`, processed, and unlinked in a `finally` block — including on the failure path.
- Uniform HTTP-500 error envelope: pipeline exceptions are caught and re-raised as `HTTPException(status_code=500, detail="Failed to ... : <message>")`.
- CORS pre-configured for `localhost:3000` and `localhost:5173`.
## Usage: Basic
```bash
# Start the service
micromamba run -n suaveplan-audio-fingerprint \
uvicorn src.api.app:app --host 0.0.0.0 --port 8000
# Health check
curl http://localhost:8000/
# Fingerprint a single file
curl -X POST http://localhost:8000/fingerprint -F "file=@track.mp3"
```
## Usage: Advanced
```bash
# Tune the algorithm parameters at request time
curl -X POST http://localhost:8000/fingerprint \
-F "file=@track.flac" \
-F "peak_threshold=0.6" \
-F "fan_value=20"
# Match two files with a stricter min_matches threshold
curl -X POST http://localhost:8000/match \
-F "query_file=@query.wav" \
-F "reference_file=@reference.wav" \
-F "peak_threshold=0.5" \
-F "fan_value=15" \
-F "min_matches=25"
```
```python
# Programmatic in-process testing via httpx ASGITransport
from httpx import ASGITransport, AsyncClient
from src.api.app import app
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
response = await ac.get("/")
assert response.json() == {"status": "ok", "service": "audio-fingerprinting"}
```
## API Reference
| Method | Path | Form fields | Response |
|---|---|---|---|
| `GET` | `/` | — | `{"status": "ok", "service": "audio-fingerprinting"}` |
| `POST` | `/fingerprint` | `file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15) | `{filename, duration, peaks_count, fingerprints_count, fingerprints[{hash,time,delta}]}` |
| `POST` | `/match` | `query_file` (UploadFile, required), `reference_file` (UploadFile, required), `peak_threshold` (float, default 0.5), `fan_value` (int, default 15), `min_matches` (int, default 10) | `{query_file, reference_file, query_duration, reference_duration, matched, match_count, confidence, time_offset}` |
### Failure paths
- Missing required upload(s) -> HTTP 422 (FastAPI validation error envelope).
- Pipeline failure (unreadable audio, decode error) -> HTTP 500 with `detail` starting with `Failed to process audio file:` (for `/fingerprint`) or `Failed to match audio files:` (for `/match`).
## Implementation Notes
- A single module-level `AudioFingerprinter()` instance is reused across requests. It has no mutable per-request state; only its analysis parameters are shared, and those are configuration-stable.
- `tempfile.NamedTemporaryFile(delete=False)` is used so the descriptor can be closed while the file is still on disk for librosa to open by path. The unlink lives in a `finally` clause and uses `missing_ok=True` so the cleanup is idempotent even if librosa already moved or removed the file.
- The `match` endpoint persists both uploads inside a single nested `with` block; this guarantees both temp paths are bound before either pipeline call runs, simplifying the cleanup in the `finally` clause.
- `CORSMiddleware` is wired with `allow_credentials=True`, `allow_methods=["*"]`, `allow_headers=["*"]`. The TestClient suite exercises the preflight (`OPTIONS`) path for `http://localhost:3000` to ensure the configuration is live.
- The `version="0.1.0"` field on the `FastAPI(...)` constructor matches the `pyproject.toml` and `package.json` versions; bump all three in lockstep at graduation to `1.0.0`.

125
packages/media/audio/audio-fingerprint/src/api/app.py Normal file
View File

@@ -0,0 +1,125 @@
"""FastAPI application for the Suaveplan audio-fingerprinting service."""
import tempfile
from pathlib import Path
from typing import Any
from fastapi import FastAPI, File, HTTPException, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from ..fingerprint import AudioFingerprinter
app = FastAPI(
title="Suaveplan Audio Fingerprinting API",
description="High-accuracy audio fingerprinting and matching service",
version="0.1.0",
)
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:3000", "http://localhost:5173"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
fingerprinter = AudioFingerprinter()
@app.get("/")
async def root() -> dict[str, str]:
"""Health-check endpoint."""
return {"status": "ok", "service": "audio-fingerprinting"}
@app.post("/fingerprint")
async def create_fingerprint(
file: UploadFile = File(...),
peak_threshold: float = 0.5,
fan_value: int = 15,
) -> dict[str, Any]:
"""Generate a fingerprint from an uploaded audio file."""
suffix = Path(file.filename).suffix if file.filename else ""
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
result = fingerprinter.fingerprint_file(
tmp_path, peak_threshold=peak_threshold, fan_value=fan_value
)
fingerprints_list = [
{"hash": fp[0], "time": fp[1], "delta": fp[2]}
for fp in result["fingerprints"]
]
return {
"filename": file.filename,
"duration": result["duration"],
"peaks_count": result["peaks_count"],
"fingerprints_count": result["fingerprints_count"],
"fingerprints": fingerprints_list,
}
except Exception as exc:
raise HTTPException(
status_code=500,
detail=f"Failed to process audio file: {exc}",
) from exc
finally:
Path(tmp_path).unlink(missing_ok=True)
@app.post("/match")
async def match_audio(
query_file: UploadFile = File(...),
reference_file: UploadFile = File(...),
peak_threshold: float = 0.5,
fan_value: int = 15,
min_matches: int = 10,
) -> dict[str, Any]:
"""Match two uploaded audio files."""
query_suffix = Path(query_file.filename).suffix if query_file.filename else ""
ref_suffix = (
Path(reference_file.filename).suffix if reference_file.filename else ""
)
with tempfile.NamedTemporaryFile(
delete=False, suffix=query_suffix
) as query_tmp, tempfile.NamedTemporaryFile(
delete=False, suffix=ref_suffix
) as ref_tmp:
query_tmp.write(await query_file.read())
ref_tmp.write(await reference_file.read())
query_path = query_tmp.name
ref_path = ref_tmp.name
try:
query_result = fingerprinter.fingerprint_file(
query_path, peak_threshold=peak_threshold, fan_value=fan_value
)
ref_result = fingerprinter.fingerprint_file(
ref_path, peak_threshold=peak_threshold, fan_value=fan_value
)
match_result = fingerprinter.match_fingerprints(
query_result["fingerprints"],
ref_result["fingerprints"],
min_matches=min_matches,
)
return {
"query_file": query_file.filename,
"reference_file": reference_file.filename,
"query_duration": query_result["duration"],
"reference_duration": ref_result["duration"],
"matched": match_result["matched"],
"match_count": match_result["match_count"],
"confidence": match_result["confidence"],
"time_offset": match_result["time_offset"],
}
except Exception as exc:
raise HTTPException(
status_code=500,
detail=f"Failed to match audio files: {exc}",
) from exc
finally:
Path(query_path).unlink(missing_ok=True)
Path(ref_path).unlink(missing_ok=True)

1
packages/media/audio/audio-fingerprint/src/client/index.ts Normal file
View File

@@ -0,0 +1 @@
export { PythonAudioService } from "./python-audio-service.js";

109
packages/media/audio/audio-fingerprint/src/client/python-audio-service.md Normal file
View File

@@ -0,0 +1,109 @@
# `client/python-audio-service.ts` — `PythonAudioService`
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`PythonAudioService` is the typed HTTP client TypeScript and browser callers use to talk to the Python-tier audio fingerprinting service. It wraps `fetch`, handles multipart `FormData` uploads, propagates `AbortController`-based timeouts, and translates the FastAPI error envelope (`{detail: string}`) into well-typed JavaScript `Error` instances with informative messages. The class is the only runtime export of the `@suaveplan/audio-fingerprint/client` subpath, and it is also re-exported from the package root.
The client deliberately depends on nothing beyond the platform `fetch`, `FormData`, and `AbortController` APIs. Browsers (Chrome, Firefox, Safari 14+, Edge) and modern server runtimes (Node 18+, Bun, Deno) all expose these, so the same code runs unchanged in every consumer tier. There is no axios, no isomorphic-fetch polyfill, and no transitive dependency closure to audit.
## Features
- Configurable `baseUrl` (default `http://localhost:8000`) and `timeout` in ms (default `30000`).
- `healthCheck()` — `GET /`, returns the typed `HealthCheckResponse`.
- `fingerprint(req)` — `POST /fingerprint` with `file` plus optional `peak_threshold` / `fan_value`.
- `match(req)` — `POST /match` with `query_file`, `reference_file`, plus optional `peak_threshold` / `fan_value` / `min_matches`.
- `AbortController`-based timeouts; on expiry the client rejects with `Error("Request timeout after Nms: <endpoint>")`.
- Error parsing: HTTP-non-OK responses are parsed as JSON; if the body is `{detail: "..."}`, that string becomes the `Error.message`. Otherwise the client falls back to `HTTP <status>: <statusText>`.
- Non-`Error` thrown values from `fetch` are normalised to `Error("Unknown error occurred: <string>")`.
## Usage: Basic
```ts
import { PythonAudioService } from "@suaveplan/audio-fingerprint/client";
const client = new PythonAudioService();
const health = await client.healthCheck();
// { status: "ok", service: "audio-fingerprinting" }
```
## Usage: Advanced
```ts
import { PythonAudioService } from "@suaveplan/audio-fingerprint";
import type {
FingerprintResponse,
MatchResponse,
} from "@suaveplan/audio-fingerprint/types";
const client = new PythonAudioService({
baseUrl: "https://audio-fp.internal.example.com",
timeout: 60_000,
});
async function identify(query: File, reference: File): Promise<MatchResponse> {
try {
const queryFingerprint: FingerprintResponse = await client.fingerprint({
file: query,
peak_threshold: 0.55,
fan_value: 18,
});
// ... persist queryFingerprint to corpus if needed
return await client.match({
query_file: query,
reference_file: reference,
peak_threshold: 0.55,
fan_value: 18,
min_matches: 25,
});
} catch (error) {
// The thrown message is either the FastAPI `detail`, a status-text
// fallback, or the timeout / unknown-error envelope.
console.error("identification failed:", (error as Error).message);
throw error;
}
}
```
## API Reference
### Constructor
```ts
constructor(config?: AudioFingerprintConfig)
```
| Field | Type | Default | Description |
|---|---|---|---|
| `config.baseUrl` | `string` | `"http://localhost:8000"` | Base URL of the FastAPI service. |
| `config.timeout` | `number` | `30000` | Per-request timeout in milliseconds. |
### Methods
| Method | Signature | Throws |
|---|---|---|
| `healthCheck` | `() => Promise<HealthCheckResponse>` | `Error` on HTTP error, timeout, or network failure. |
| `fingerprint` | `(req: FingerprintRequest) => Promise<FingerprintResponse>` | `Error` on HTTP error, timeout, or network failure. |
| `match` | `(req: MatchRequest) => Promise<MatchResponse>` | `Error` on HTTP error, timeout, or network failure. |
### Error envelope
| Cause | `Error.message` |
|---|---|
| Timeout | `Request timeout after <Nms>: <endpoint>` |
| HTTP non-OK with parseable `{detail: string}` body | `<detail>` |
| HTTP non-OK with non-JSON or `detail`-missing body | `HTTP <status>: <statusText>` |
| `fetch` throws a non-`Error` value | `Unknown error occurred: <stringified value>` |
| `fetch` throws a non-Abort `Error` | the original error is re-raised verbatim |
## Implementation Notes
- The timeout is implemented with a `setTimeout` that calls `controller.abort()`; the timer is cleared both on success and on failure (try/catch + redundant `clearTimeout` after the `await`). This avoids leaking timers in long-lived browsers tabs.
- `AbortError` is detected by `error.name === "AbortError"` rather than by `instanceof DOMException` to remain runtime-agnostic (DOMException is browser-only).
- Optional form fields are only appended when the caller supplies them; the FastAPI defaults remain authoritative.
- The error-body JSON parse is wrapped in a try/catch with no rethrow inside the catch — a parse failure should NOT mask the original HTTP error, only fall through to the status-text fallback.
- Type-only imports from `../types/audio-fingerprint.types.js` are used so the client carries zero runtime cost from the type definitions.
- The client has no peer dependency on any framework; consumers wrap it in their own React/Vue/Svelte hooks as appropriate. The sibling `@suaveplan/react-audio-fingerprint` package's `python-backend` subpath is the canonical React adapter.

128
packages/media/audio/audio-fingerprint/src/client/python-audio-service.ts Normal file
View File

@@ -0,0 +1,128 @@
/**
* HTTP client for the Suaveplan audio-fingerprint Python service.
*
* Provides a typed interface for `GET /`, `POST /fingerprint`, and
* `POST /match` on the FastAPI surface defined in `src/api/app.py`.
*/
import type {
AudioFingerprintConfig,
ErrorResponse,
FingerprintRequest,
FingerprintResponse,
HealthCheckResponse,
MatchRequest,
MatchResponse,
} from "../types/audio-fingerprint.types.js";
const DEFAULT_BASE_URL = "http://localhost:8000";
const DEFAULT_TIMEOUT_MS = 30_000;
/**
* Typed client for the Python audio-fingerprinting FastAPI service.
*
* @example
* ```ts
* const client = new PythonAudioService({ baseUrl: "http://localhost:8000" });
* const health = await client.healthCheck();
* const result = await client.fingerprint({ file });
* const match = await client.match({ query_file, reference_file });
* ```
*/
export class PythonAudioService {
private readonly baseUrl: string;
private readonly timeout: number;
constructor(config: AudioFingerprintConfig = {}) {
this.baseUrl = config.baseUrl ?? DEFAULT_BASE_URL;
this.timeout = config.timeout ?? DEFAULT_TIMEOUT_MS;
}
/** Calls `GET /`. Returns `{ status: 'ok', service: 'audio-fingerprinting' }`. */
async healthCheck(): Promise<HealthCheckResponse> {
return this.request<HealthCheckResponse>("/");
}
/** Calls `POST /fingerprint` with the audio file and optional parameters. */
async fingerprint(req: FingerprintRequest): Promise<FingerprintResponse> {
const formData = new FormData();
formData.append("file", req.file);
if (req.peak_threshold !== undefined) {
formData.append("peak_threshold", String(req.peak_threshold));
}
if (req.fan_value !== undefined) {
formData.append("fan_value", String(req.fan_value));
}
return this.request<FingerprintResponse>("/fingerprint", {
method: "POST",
body: formData,
});
}
/** Calls `POST /match` with the query and reference files and options. */
async match(req: MatchRequest): Promise<MatchResponse> {
const formData = new FormData();
formData.append("query_file", req.query_file);
formData.append("reference_file", req.reference_file);
if (req.peak_threshold !== undefined) {
formData.append("peak_threshold", String(req.peak_threshold));
}
if (req.fan_value !== undefined) {
formData.append("fan_value", String(req.fan_value));
}
if (req.min_matches !== undefined) {
formData.append("min_matches", String(req.min_matches));
}
return this.request<MatchResponse>("/match", {
method: "POST",
body: formData,
});
}
/**
* Internal fetch wrapper. Adds `AbortController`-based timeouts and a
* uniform error surface: HTTP errors carry the FastAPI `detail` body if
* present, otherwise fall back to `HTTP <status>: <statusText>`.
*/
private async request<T>(endpoint: string, init?: RequestInit): Promise<T> {
const url = `${this.baseUrl}${endpoint}`;
const controller = new AbortController();
const timeoutId = setTimeout(() => {
controller.abort();
}, this.timeout);
try {
const response = await fetch(url, {
...init,
signal: controller.signal,
});
clearTimeout(timeoutId);
if (!response.ok) {
let errorMessage = `HTTP ${response.status}: ${response.statusText}`;
try {
const errorData = (await response.json()) as ErrorResponse;
if (errorData.detail) {
errorMessage = errorData.detail;
}
} catch {
// JSON parse failure -> fall through to HTTP status text.
}
throw new Error(errorMessage);
}
return (await response.json()) as T;
} catch (error) {
clearTimeout(timeoutId);
if (error instanceof Error) {
if (error.name === "AbortError") {
throw new Error(
`Request timeout after ${this.timeout}ms: ${endpoint}`
);
}
throw error;
}
throw new Error(`Unknown error occurred: ${String(error)}`);
}
}
}

64
packages/media/audio/audio-fingerprint/src/fingerprint.md Normal file
View File

@@ -0,0 +1,64 @@
# `fingerprint.py` — `AudioFingerprinter`
## Purpose
The `AudioFingerprinter` class is the Python core of `@suaveplan/audio-fingerprint`. It implements the full Shazam-style audio identification pipeline: load and downmix the source audio with librosa, run a Short-Time Fourier Transform, distil that magnitude spectrogram into a 12-bin chromagram (pitch-class profile), detect 2D local maxima in the chromagram with `scipy.ndimage.maximum_filter`, hash each peak with its `fan_value` nearest neighbours into a constellation map, and match two fingerprint sets using Jaccard similarity plus median time-offset estimation. It is the high-accuracy backend the React-tier `@suaveplan/react-audio-fingerprint` package is designed to call through the FastAPI bridge.
The pipeline is deliberately chromagram-based rather than raw-magnitude based. Chromagrams fold pitch into 12 bins, so the fingerprint is robust against octave shifts, compression artefacts, mild noise, EQ, and re-encoding — at the cost of being unable to distinguish two different pieces in the same key with the same rhythm. For corpus-scale broadcast monitoring or sample identification, this trade is the right one.
## Features
- Configurable `sample_rate`, `n_fft`, `hop_length`, and `n_chroma` (defaults match the industry-standard 22050/2048/512/12).
- librosa-backed audio loader with mono downmix, target-rate resampling, structured `FileNotFoundError` / `RuntimeError` failures.
- 2D local-maximum peak detection on the normalised chromagram via `scipy.ndimage.maximum_filter` with a `(3, 5)` neighbourhood (3 frequency bins by 5 time frames).
- Constellation-map fingerprint hashing with a configurable `fan_value`.
- O(n+m) Jaccard matcher with median time-offset estimation, defensive against empty inputs and partial overlaps.
- Full pipeline orchestrator `fingerprint_file` that returns duration plus peak and fingerprint counts.
## Usage: Basic
```python
from src.fingerprint import AudioFingerprinter
fp = AudioFingerprinter()
result = fp.fingerprint_file("track.mp3", peak_threshold=0.5, fan_value=15)
print(result["duration"], result["peaks_count"], result["fingerprints_count"])
```
## Usage: Advanced
```python
from src.fingerprint import AudioFingerprinter
fp = AudioFingerprinter(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)
audio = fp.load_audio("query.wav")
chromagram = fp.extract_chromagram(audio) # shape (12, n_frames)
peaks = fp.find_peaks(chromagram, threshold=0.5)
query_fingerprints = fp.generate_fingerprint(peaks, fan_value=15)
reference = fp.fingerprint_file("reference.flac")
match = fp.match_fingerprints(query_fingerprints, reference["fingerprints"], min_matches=20)
print(match["matched"], match["confidence"], match["time_offset"])
```
## API Reference
| Method | Signature | Description |
|---|---|---|
| `__init__` | `(sample_rate=22050, n_fft=2048, hop_length=512, n_chroma=12)` | Persists the analysis parameters as public attributes. |
| `load_audio` | `(file_path: str \| Path) -> NDArray[float32]` | Loads, downmixes to mono, resamples to `sample_rate`. Raises `FileNotFoundError` when the file is missing; raises `RuntimeError("Failed to load audio file: ...")` on librosa decoder failures. |
| `extract_chromagram` | `(audio: NDArray) -> NDArray[float32]` | Computes the chromagram from the STFT magnitude. Output shape: `(n_chroma, n_frames)`. |
| `find_peaks` | `(chromagram: NDArray, threshold=0.5) -> list[tuple[int,int,float]]` | Detects 2D local maxima exceeding `threshold` on the normalised chromagram. Returns `[]` when the chromagram is all-zero. |
| `generate_fingerprint` | `(peaks, fan_value=15) -> set[tuple[int,int,int]]` | Constellation-map hashing; each tuple is `(hash, time_offset, time_delta)`. |
| `fingerprint_file` | `(file_path, peak_threshold=0.5, fan_value=15) -> dict` | Runs the full pipeline. Returns `fingerprints`, `duration`, `peaks_count`, `fingerprints_count`. |
| `match_fingerprints` | `(query, reference, min_matches=10) -> dict` | Returns `matched`, `match_count`, `confidence` (Jaccard), `time_offset` (median frame offset, 0 when no positive match). |
## Implementation Notes
- The chromagram normalisation is done in `find_peaks` rather than `extract_chromagram` so the threshold semantics are stable across signals of differing absolute amplitude.
- The `(3, 5)` neighbourhood is intentionally asymmetric: time has more frames than there are pitch classes, so we accept a wider time window before considering a point a local maximum.
- Constellation hashing uses Python's built-in `hash((f1, f2, dt))`; the values are not cryptographically stable across interpreters but they are stable within a single run, which is all the matcher needs.
- The matcher builds an O(m) `hash -> reference_time` lookup table before iterating the query set; this turns the original upstream's nested O(n*m) loop into O(n + m) while preserving the median-of-offsets semantics.
- An empty union of hashes is treated as a Jaccard denominator of `max(union_size, 1)`, so the empty/empty case returns confidence 0 cleanly rather than dividing by zero.
- The `if match_count >= min_matches and matching_hashes:` guard catches the edge case `min_matches=0` with two empty inputs, preventing `np.median([])` from raising.

220
packages/media/audio/audio-fingerprint/src/fingerprint.py Normal file
View File

@@ -0,0 +1,220 @@
"""Audio fingerprinting utilities using chromagram + constellation hashing."""
from pathlib import Path
from typing import Any
import librosa
import numpy as np
from numpy.typing import NDArray
from scipy.ndimage import maximum_filter
class AudioFingerprinter:
"""High-accuracy audio fingerprinting for track identification.
Uses chromagram-based fingerprinting with 2D peak detection and
constellation-map hashing for robust audio identification even with
noise, compression, or modifications. The matcher uses Jaccard
similarity over hashed peak pairs and reports median time offset.
"""
def __init__(
self,
sample_rate: int = 22050,
n_fft: int = 2048,
hop_length: int = 512,
n_chroma: int = 12,
) -> None:
"""Initialise fingerprinter with audio-analysis parameters.
Args:
sample_rate: Target sample rate for audio processing.
n_fft: FFT window size for frequency analysis.
hop_length: Number of samples between successive frames.
n_chroma: Number of chroma bins (12 for standard pitch classes).
"""
self.sample_rate = sample_rate
self.n_fft = n_fft
self.hop_length = hop_length
self.n_chroma = n_chroma
def load_audio(self, file_path: str | Path) -> NDArray[np.float32]:
"""Load and preprocess audio file to mono at the target sample rate.
Args:
file_path: Path to audio file (WAV, MP3, OGG, FLAC, etc.).
Returns:
Mono audio signal resampled to ``self.sample_rate``.
Raises:
FileNotFoundError: If the file does not exist.
RuntimeError: If librosa fails to decode the file.
"""
path = Path(file_path)
if not path.exists():
raise FileNotFoundError(f"Audio file not found: {file_path}")
try:
audio, _ = librosa.load(str(path), sr=self.sample_rate, mono=True)
return audio
except Exception as exc:
raise RuntimeError(f"Failed to load audio file: {exc}") from exc
def extract_chromagram(
self, audio: NDArray[np.float32]
) -> NDArray[np.float32]:
"""Extract chromagram (pitch-class profile) from a mono signal.
Args:
audio: Mono audio signal.
Returns:
Chromagram matrix of shape ``(n_chroma, n_frames)``.
"""
stft = librosa.stft(
audio, n_fft=self.n_fft, hop_length=self.hop_length
)
chromagram = librosa.feature.chroma_stft(
S=np.abs(stft), sr=self.sample_rate, n_chroma=self.n_chroma
)
return chromagram
def find_peaks(
self,
chromagram: NDArray[np.float32],
threshold: float = 0.5,
) -> list[tuple[int, int, float]]:
"""Find 2D local maxima in the chromagram.
Uses ``scipy.ndimage.maximum_filter`` with a ``(3, 5)`` neighbourhood
and keeps only points whose normalised amplitude equals the local
maximum and exceeds ``threshold``.
Args:
chromagram: Chromagram from :meth:`extract_chromagram`.
threshold: Minimum peak amplitude after normalisation (0.0-1.0).
Returns:
List of ``(time_frame, chroma_bin, amplitude)`` tuples.
"""
max_val = float(np.max(chromagram))
if max_val == 0:
return []
normalized = chromagram / max_val
neighborhood_size = (3, 5)
local_max = maximum_filter(normalized, size=neighborhood_size)
peaks_mask = (normalized == local_max) & (normalized >= threshold)
peak_coords = np.argwhere(peaks_mask)
return [
(int(t), int(f), float(normalized[f, t])) for f, t in peak_coords
]
def generate_fingerprint(
self,
peaks: list[tuple[int, int, float]],
fan_value: int = 15,
) -> set[tuple[int, int, int]]:
"""Generate constellation-map fingerprint hashes from spectral peaks.
Pairs each peak with the next ``fan_value`` peaks in time-sorted
order and emits ``(hash(f1, f2, dt), t1, dt)`` tuples.
Args:
peaks: Peaks from :meth:`find_peaks`.
fan_value: Number of future peaks to pair with each peak.
Returns:
Set of ``(fingerprint_hash, t1, time_delta)`` tuples.
"""
sorted_peaks = sorted(peaks, key=lambda x: x[0])
fingerprints: set[tuple[int, int, int]] = set()
for i, (t1, f1, _) in enumerate(sorted_peaks):
for j in range(i + 1, min(i + fan_value + 1, len(sorted_peaks))):
t2, f2, _ = sorted_peaks[j]
time_delta = t2 - t1
fingerprint_hash = hash((f1, f2, time_delta))
fingerprints.add((fingerprint_hash, t1, time_delta))
return fingerprints
def fingerprint_file(
self,
file_path: str | Path,
peak_threshold: float = 0.5,
fan_value: int = 15,
) -> dict[str, Any]:
"""Run the full fingerprinting pipeline from a file path.
Args:
file_path: Path to audio file.
peak_threshold: Minimum peak amplitude (0.0-1.0).
fan_value: Number of peaks to pair for hashing.
Returns:
Dict with ``fingerprints`` (set), ``duration`` (s), ``peaks_count``
(int), ``fingerprints_count`` (int).
"""
audio = self.load_audio(file_path)
chromagram = self.extract_chromagram(audio)
peaks = self.find_peaks(chromagram, threshold=peak_threshold)
fingerprints = self.generate_fingerprint(peaks, fan_value=fan_value)
duration = len(audio) / self.sample_rate
return {
"fingerprints": fingerprints,
"duration": duration,
"peaks_count": len(peaks),
"fingerprints_count": len(fingerprints),
}
def match_fingerprints(
self,
query_fingerprints: set[tuple[int, int, int]],
reference_fingerprints: set[tuple[int, int, int]],
min_matches: int = 10,
) -> dict[str, Any]:
"""Match a query fingerprint set against a reference fingerprint set.
Args:
query_fingerprints: Fingerprints from the query audio.
reference_fingerprints: Fingerprints from the reference audio.
min_matches: Minimum overlap for a positive match.
Returns:
Dict with ``matched`` (bool), ``match_count`` (int),
``confidence`` (Jaccard similarity, 0.0-1.0), ``time_offset``
(median frame offset; 0 when no positive match).
"""
query_hashes = {fp[0] for fp in query_fingerprints}
reference_hashes = {fp[0] for fp in reference_fingerprints}
matching_hashes = query_hashes & reference_hashes
match_count = len(matching_hashes)
union_size = len(query_hashes | reference_hashes)
confidence = match_count / max(union_size, 1)
time_offset = 0
if match_count >= min_matches and matching_hashes:
# Build a hash -> reference-time lookup once, then walk the query
# set computing per-hash time deltas. The previous nested O(n*m)
# loop is replaced with O(n+m).
reference_time_by_hash: dict[int, int] = {}
for rfp in reference_fingerprints:
reference_time_by_hash.setdefault(rfp[0], rfp[1])
offsets: list[int] = [
reference_time_by_hash[qfp[0]] - qfp[1]
for qfp in query_fingerprints
if qfp[0] in matching_hashes
]
time_offset = int(np.median(offsets))
return {
"matched": match_count >= min_matches,
"match_count": match_count,
"confidence": confidence,
"time_offset": time_offset,
}

11
packages/media/audio/audio-fingerprint/src/index.ts Normal file
View File

@@ -0,0 +1,11 @@
export { PythonAudioService } from "./client/python-audio-service.js";
export type {
AudioFingerprint,
AudioFingerprintConfig,
ErrorResponse,
FingerprintRequest,
FingerprintResponse,
HealthCheckResponse,
MatchRequest,
MatchResponse,
} from "./types/audio-fingerprint.types.js";

71
packages/media/audio/audio-fingerprint/src/types/audio-fingerprint.types.ts Normal file
View File

@@ -0,0 +1,71 @@
/**
* TypeScript type definitions for the Suaveplan audio-fingerprint Python
* service. These types correspond exactly to the wire shapes returned by
* the FastAPI endpoints defined in `src/api/app.py`.
*/
/** Health-check response from the `/` endpoint. */
export interface HealthCheckResponse {
status: "ok";
service: "audio-fingerprinting";
}
/** Individual fingerprint hash with timing information. */
export interface AudioFingerprint {
/** Hash of `(frequency1, frequency2, time_delta)`. */
hash: number;
/** Time position in the audio, in STFT frame indices. */
time: number;
/** Time delta between the paired peaks, in STFT frame indices. */
delta: number;
}
/** Response payload for the `/fingerprint` endpoint. */
export interface FingerprintResponse {
filename: string;
duration: number;
peaks_count: number;
fingerprints_count: number;
fingerprints: AudioFingerprint[];
}
/** Response payload for the `/match` endpoint. */
export interface MatchResponse {
query_file: string;
reference_file: string;
query_duration: number;
reference_duration: number;
matched: boolean;
match_count: number;
/** Jaccard similarity score in the range [0.0, 1.0]. */
confidence: number;
/** Median frame offset between matched sections. */
time_offset: number;
}
/** Request body for `/fingerprint`. The file is sent as multipart form data. */
export interface FingerprintRequest {
file: File | Blob;
peak_threshold?: number;
fan_value?: number;
}
/** Request body for `/match`. Files are sent as multipart form data. */
export interface MatchRequest {
query_file: File | Blob;
reference_file: File | Blob;
peak_threshold?: number;
fan_value?: number;
min_matches?: number;
}
/** Error response shape (matches FastAPI's `HTTPException` JSON). */
export interface ErrorResponse {
detail: string;
}
/** Configuration accepted by `PythonAudioService`. */
export interface AudioFingerprintConfig {
baseUrl?: string;
timeout?: number;
}

10
packages/media/audio/audio-fingerprint/src/types/index.ts Normal file
View File

@@ -0,0 +1,10 @@
export type {
AudioFingerprint,
AudioFingerprintConfig,
ErrorResponse,
FingerprintRequest,
FingerprintResponse,
HealthCheckResponse,
MatchRequest,
MatchResponse,
} from "./audio-fingerprint.types.js";

0
packages/media/audio/audio-fingerprint/tests/__init__.py Normal file
View File

36
packages/media/audio/audio-fingerprint/tests/conftest.py Normal file
View File

@@ -0,0 +1,36 @@
"""Shared pytest fixtures for audio-fingerprint tests."""
from __future__ import annotations
import sys
import tempfile
from pathlib import Path
import numpy as np
import pytest
import soundfile as sf
# Ensure the package source is importable as `src.*` when pytest is invoked
# from the package directory without an installed wheel.
_PKG_DIR = Path(__file__).resolve().parents[1]
if str(_PKG_DIR) not in sys.path:
sys.path.insert(0, str(_PKG_DIR))
@pytest.fixture
def sample_audio() -> tuple[np.ndarray, int]:
"""Generate a 2-second 440 Hz sine for tests."""
sample_rate = 22050
duration = 2.0
t = np.linspace(0, duration, int(sample_rate * duration))
audio = 0.5 * np.sin(2 * np.pi * 440.0 * t)
return audio.astype(np.float32), sample_rate
@pytest.fixture
def audio_file(sample_audio: tuple[np.ndarray, int]) -> Path:
"""Persist the sample audio to a temp WAV path."""
audio, sample_rate = sample_audio
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
sf.write(tmp.name, audio, sample_rate)
return Path(tmp.name)

24
packages/media/audio/audio-fingerprint/tests/exports.test.ts Normal file
View File

@@ -0,0 +1,24 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import * as clientBarrel from "../src/client/index.js";
import * as root from "../src/index.js";
import * as typesBarrel from "../src/types/index.js";
describe("audio-fingerprint exports", () => {
it("barrel re-exports the client and types", { timeout: 5000 }, () => {
expect(typeof root.PythonAudioService).toBe("function");
});
it("client subpath re-exports PythonAudioService", {
timeout: 5000,
}, () => {
expect(clientBarrel.PythonAudioService).toBe(root.PythonAudioService);
});
it("types subpath barrel is loadable (no runtime exports)", {
timeout: 5000,
}, () => {
// Types are erased at runtime; the barrel module exists but has
// no own enumerable runtime keys.
expect(Object.keys(typesBarrel)).toEqual([]);
});
});

306
packages/media/audio/audio-fingerprint/tests/python-audio-service.test.ts Normal file
View File

@@ -0,0 +1,306 @@
import {
afterEach,
beforeEach,
describe,
expect,
it,
} from "@suaveplan/testing/runner";
import { PythonAudioService } from "../src/client/python-audio-service.js";
import type {
FingerprintResponse,
HealthCheckResponse,
MatchResponse,
} from "../src/types/audio-fingerprint.types.js";
type FetchHandler = (
url: string,
init?: RequestInit
) => Promise<Response> | Response;
const originalFetch = globalThis.fetch;
function installFetch(handler: FetchHandler): {
calls: Array<{ url: string; init?: RequestInit }>;
} {
const calls: Array<{ url: string; init?: RequestInit }> = [];
globalThis.fetch = (async (
input: RequestInfo | URL,
init?: RequestInit
) => {
const url =
typeof input === "string"
? input
: input instanceof URL
? input.toString()
: input.url;
calls.push({ url, init });
return handler(url, init);
}) as typeof fetch;
return { calls };
}
function jsonResponse(body: unknown, init: ResponseInit = {}): Response {
return new Response(JSON.stringify(body), {
status: 200,
headers: { "content-type": "application/json" },
...init,
});
}
async function formDataEntries(
init: RequestInit | undefined
): Promise<Array<[string, FormDataEntryValue]>> {
if (!init || !(init.body instanceof FormData)) {
return [];
}
return Array.from(init.body.entries());
}
describe("PythonAudioService", () => {
afterEach(() => {
globalThis.fetch = originalFetch;
});
describe("constructor defaults", () => {
it("uses the documented baseUrl and timeout when no config is supplied", {
timeout: 5000,
}, async () => {
const { calls } = installFetch(() =>
jsonResponse({
status: "ok",
service: "audio-fingerprinting",
})
);
const client = new PythonAudioService();
await client.healthCheck();
expect(calls[0]?.url).toBe("http://localhost:8000/");
});
});
describe("healthCheck", () => {
it("healthCheck returns the parsed body", {
timeout: 5000,
}, async () => {
const payload: HealthCheckResponse = {
status: "ok",
service: "audio-fingerprinting",
};
installFetch(() => jsonResponse(payload));
const client = new PythonAudioService({
baseUrl: "http://example.test",
});
const result = await client.healthCheck();
expect(result).toEqual(payload);
});
});
describe("fingerprint", () => {
it("fingerprint posts the file and optional params via FormData", {
timeout: 5000,
}, async () => {
const payload: FingerprintResponse = {
filename: "test.wav",
duration: 1.0,
peaks_count: 1,
fingerprints_count: 1,
fingerprints: [{ hash: 1, time: 0, delta: 0 }],
};
const { calls } = installFetch(() => jsonResponse(payload));
const client = new PythonAudioService();
const file = new Blob(["RIFF...."], { type: "audio/wav" });
const result = await client.fingerprint({
file,
peak_threshold: 0.6,
fan_value: 20,
});
expect(result).toEqual(payload);
const entries = await formDataEntries(calls[0]?.init);
const keys = entries.map(([k]) => k);
expect(keys).toContain("file");
expect(entries).toContainEqual(["peak_threshold", "0.6"]);
expect(entries).toContainEqual(["fan_value", "20"]);
expect(calls[0]?.url).toBe("http://localhost:8000/fingerprint");
});
it("fingerprint omits optional params when undefined", {
timeout: 5000,
}, async () => {
const { calls } = installFetch(() =>
jsonResponse({
filename: "x",
duration: 0,
peaks_count: 0,
fingerprints_count: 0,
fingerprints: [],
})
);
const client = new PythonAudioService();
await client.fingerprint({ file: new Blob(["a"]) });
const entries = await formDataEntries(calls[0]?.init);
const keys = entries.map(([k]) => k);
expect(keys).toEqual(["file"]);
});
});
describe("match", () => {
it("match posts both files and all optional params", {
timeout: 5000,
}, async () => {
const payload: MatchResponse = {
query_file: "q.wav",
reference_file: "r.wav",
query_duration: 1.0,
reference_duration: 1.0,
matched: true,
match_count: 10,
confidence: 0.9,
time_offset: 0,
};
const { calls } = installFetch(() => jsonResponse(payload));
const client = new PythonAudioService();
const queryFile = new Blob(["q"], { type: "audio/wav" });
const referenceFile = new Blob(["r"], { type: "audio/wav" });
const result = await client.match({
query_file: queryFile,
reference_file: referenceFile,
peak_threshold: 0.6,
fan_value: 20,
min_matches: 5,
});
expect(result).toEqual(payload);
const entries = await formDataEntries(calls[0]?.init);
const keys = entries.map(([k]) => k);
expect(keys).toContain("query_file");
expect(keys).toContain("reference_file");
expect(entries).toContainEqual(["peak_threshold", "0.6"]);
expect(entries).toContainEqual(["fan_value", "20"]);
expect(entries).toContainEqual(["min_matches", "5"]);
});
it("match omits optional params when undefined", {
timeout: 5000,
}, async () => {
const { calls } = installFetch(() =>
jsonResponse({
query_file: "",
reference_file: "",
query_duration: 0,
reference_duration: 0,
matched: false,
match_count: 0,
confidence: 0,
time_offset: 0,
})
);
const client = new PythonAudioService();
await client.match({
query_file: new Blob(["a"]),
reference_file: new Blob(["b"]),
});
const entries = await formDataEntries(calls[0]?.init);
const keys = entries.map(([k]) => k);
expect(keys).toEqual(["query_file", "reference_file"]);
});
});
describe("error and timeout handling", () => {
let realSetTimeout: typeof setTimeout;
beforeEach(() => {
realSetTimeout = globalThis.setTimeout;
});
afterEach(() => {
globalThis.setTimeout = realSetTimeout;
});
it("timeout aborts the request", { timeout: 5000 }, async () => {
installFetch(
(_, init) =>
new Promise<Response>((_resolve, reject) => {
const signal = init?.signal;
if (!signal) {
throw new Error("missing AbortSignal");
}
signal.addEventListener("abort", () => {
const err = new Error("aborted");
err.name = "AbortError";
reject(err);
});
})
);
const client = new PythonAudioService({ timeout: 5 });
await expect(client.healthCheck()).rejects.toThrow(
/Request timeout after 5ms: \//
);
});
it("non-json error body falls back to status text", {
timeout: 5000,
}, async () => {
installFetch(
() =>
new Response("not-json", {
status: 500,
statusText: "Internal Server Error",
headers: { "content-type": "text/plain" },
})
);
const client = new PythonAudioService();
await expect(client.healthCheck()).rejects.toThrow(
"HTTP 500: Internal Server Error"
);
});
it("json error body surfaces the detail field", {
timeout: 5000,
}, async () => {
installFetch(() =>
jsonResponse(
{ detail: "Failed to process audio file: bad" },
{ status: 500, statusText: "Internal Server Error" }
)
);
const client = new PythonAudioService();
await expect(client.healthCheck()).rejects.toThrow(
"Failed to process audio file: bad"
);
});
it("json error body without detail falls back to status text", {
timeout: 5000,
}, async () => {
installFetch(() =>
jsonResponse(
{ error: "missing-detail-field" },
{ status: 500, statusText: "Internal Server Error" }
)
);
const client = new PythonAudioService();
await expect(client.healthCheck()).rejects.toThrow(
"HTTP 500: Internal Server Error"
);
});
it("non-Error thrown by fetch surfaces as Unknown error", {
timeout: 5000,
}, async () => {
globalThis.fetch = (() => {
throw "raw-string-throw";
}) as typeof fetch;
const client = new PythonAudioService();
await expect(client.healthCheck()).rejects.toThrow(
"Unknown error occurred: raw-string-throw"
);
});
it("non-AbortError thrown by fetch is re-raised verbatim", {
timeout: 5000,
}, async () => {
globalThis.fetch = (() => {
throw new Error("network down");
}) as typeof fetch;
const client = new PythonAudioService();
await expect(client.healthCheck()).rejects.toThrow("network down");
});
});
});

224
packages/media/audio/audio-fingerprint/tests/test_api.py Normal file
View File

@@ -0,0 +1,224 @@
"""Tests for the FastAPI audio-fingerprinting service."""
from __future__ import annotations
import tempfile
from pathlib import Path
import numpy as np
import pytest
import soundfile as sf
from fastapi.testclient import TestClient
from httpx import ASGITransport, AsyncClient
from src.api.app import app
@pytest.fixture
def client() -> TestClient:
return TestClient(app)
def _make_bad_audio_file() -> Path:
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False, mode="wb") as tmp:
tmp.write(b"not an audio file at all")
return Path(tmp.name)
def _make_different_audio() -> Path:
sample_rate = 22050
duration = 2.0
t = np.linspace(0, duration, int(sample_rate * duration))
audio = 0.5 * np.sin(2 * np.pi * 880.0 * t)
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
sf.write(tmp.name, audio.astype(np.float32), sample_rate)
return Path(tmp.name)
def test_root_endpoint(client: TestClient) -> None:
response = client.get("/")
assert response.status_code == 200
data = response.json()
assert data["status"] == "ok"
assert data["service"] == "audio-fingerprinting"
def test_fingerprint_endpoint(client: TestClient, audio_file: Path) -> None:
with open(audio_file, "rb") as f:
response = client.post(
"/fingerprint", files={"file": ("test.wav", f, "audio/wav")}
)
assert response.status_code == 200
data = response.json()
assert data["filename"] == "test.wav"
assert data["duration"] > 0.0
assert data["peaks_count"] > 0
assert data["fingerprints_count"] > 0
assert isinstance(data["fingerprints"], list)
assert len(data["fingerprints"]) > 0
for fp in data["fingerprints"]:
assert "hash" in fp
assert "time" in fp
assert "delta" in fp
def test_fingerprint_endpoint_with_parameters(
client: TestClient, audio_file: Path
) -> None:
with open(audio_file, "rb") as f:
response = client.post(
"/fingerprint",
files={"file": ("test.wav", f, "audio/wav")},
data={"peak_threshold": 0.6, "fan_value": 20},
)
assert response.status_code == 200
assert response.json()["fingerprints_count"] > 0
def test_fingerprint_endpoint_no_file(client: TestClient) -> None:
response = client.post("/fingerprint")
assert response.status_code == 422
def test_fingerprint_endpoint_invalid_audio(client: TestClient) -> None:
bad = _make_bad_audio_file()
try:
with open(bad, "rb") as f:
response = client.post(
"/fingerprint",
files={"file": ("bad.wav", f, "audio/wav")},
)
assert response.status_code == 500
assert response.json()["detail"].startswith("Failed to process audio file:")
finally:
bad.unlink(missing_ok=True)
def test_match_endpoint(client: TestClient, audio_file: Path) -> None:
with open(audio_file, "rb") as q, open(audio_file, "rb") as r:
response = client.post(
"/match",
files={
"query_file": ("q.wav", q, "audio/wav"),
"reference_file": ("r.wav", r, "audio/wav"),
},
)
assert response.status_code == 200
data = response.json()
for key in (
"query_file",
"reference_file",
"query_duration",
"reference_duration",
"matched",
"match_count",
"confidence",
"time_offset",
):
assert key in data
assert data["matched"] is True
assert data["confidence"] > 0.9
assert abs(data["time_offset"]) <= 5
def test_match_endpoint_with_parameters(
client: TestClient, audio_file: Path
) -> None:
with open(audio_file, "rb") as q, open(audio_file, "rb") as r:
response = client.post(
"/match",
files={
"query_file": ("q.wav", q, "audio/wav"),
"reference_file": ("r.wav", r, "audio/wav"),
},
data={"peak_threshold": 0.6, "fan_value": 20, "min_matches": 5},
)
assert response.status_code == 200
assert response.json()["matched"] is True
def test_match_endpoint_different_files(
client: TestClient, audio_file: Path
) -> None:
different = _make_different_audio()
try:
with open(audio_file, "rb") as q, open(different, "rb") as r:
response = client.post(
"/match",
files={
"query_file": ("q.wav", q, "audio/wav"),
"reference_file": ("r.wav", r, "audio/wav"),
},
)
assert response.status_code == 200
data = response.json()
assert "confidence" in data
assert "matched" in data
finally:
different.unlink(missing_ok=True)
def test_match_endpoint_missing_files(client: TestClient) -> None:
response = client.post("/match")
assert response.status_code == 422
def test_match_endpoint_invalid_audio(
client: TestClient, audio_file: Path
) -> None:
bad = _make_bad_audio_file()
try:
with open(audio_file, "rb") as q, open(bad, "rb") as r:
response = client.post(
"/match",
files={
"query_file": ("q.wav", q, "audio/wav"),
"reference_file": ("bad.wav", r, "audio/wav"),
},
)
assert response.status_code == 500
assert response.json()["detail"].startswith("Failed to match audio files:")
finally:
bad.unlink(missing_ok=True)
def test_cors_headers(client: TestClient) -> None:
response = client.options(
"/fingerprint",
headers={
"Origin": "http://localhost:3000",
"Access-Control-Request-Method": "POST",
},
)
assert response.status_code == 200
assert "access-control-allow-origin" in response.headers
@pytest.mark.asyncio
async def test_fingerprint_endpoint_async(audio_file: Path) -> None:
async with AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
with open(audio_file, "rb") as f:
response = await ac.post(
"/fingerprint", files={"file": ("test.wav", f, "audio/wav")}
)
assert response.status_code == 200
assert response.json()["fingerprints_count"] > 0
@pytest.mark.asyncio
async def test_match_endpoint_async(audio_file: Path) -> None:
async with AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
with open(audio_file, "rb") as q, open(audio_file, "rb") as r:
response = await ac.post(
"/match",
files={
"query_file": ("q.wav", q, "audio/wav"),
"reference_file": ("r.wav", r, "audio/wav"),
},
)
assert response.status_code == 200
assert response.json()["matched"] is True

229
packages/media/audio/audio-fingerprint/tests/test_fingerprint.py Normal file
View File

@@ -0,0 +1,229 @@
"""Tests for the AudioFingerprinter chromagram + constellation pipeline."""
from __future__ import annotations
import tempfile
from pathlib import Path
import numpy as np
import pytest
import soundfile as sf
from src.fingerprint import AudioFingerprinter
@pytest.fixture
def fingerprinter() -> AudioFingerprinter:
return AudioFingerprinter()
def test_fingerprinter_initialization() -> None:
fp = AudioFingerprinter()
assert fp.sample_rate == 22050
assert fp.n_fft == 2048
assert fp.hop_length == 512
assert fp.n_chroma == 12
def test_fingerprinter_custom_parameters() -> None:
fp = AudioFingerprinter(
sample_rate=44100, n_fft=4096, hop_length=1024, n_chroma=24
)
assert fp.sample_rate == 44100
assert fp.n_fft == 4096
assert fp.hop_length == 1024
assert fp.n_chroma == 24
def test_load_audio(fingerprinter: AudioFingerprinter, audio_file: Path) -> None:
audio = fingerprinter.load_audio(audio_file)
assert isinstance(audio, np.ndarray)
assert audio.dtype == np.float32
assert len(audio) > 0
assert -1.0 <= audio.min() <= 1.0
assert -1.0 <= audio.max() <= 1.0
def test_load_audio_nonexistent_file(fingerprinter: AudioFingerprinter) -> None:
with pytest.raises(FileNotFoundError, match="Audio file not found"):
fingerprinter.load_audio("/nonexistent/file.wav")
def test_load_audio_invalid_data_raises_runtime_error(
fingerprinter: AudioFingerprinter,
) -> None:
"""A file that exists but isn't decodable must surface as RuntimeError."""
with tempfile.NamedTemporaryFile(
suffix=".wav", delete=False, mode="wb"
) as tmp:
tmp.write(b"this is not a wav file at all")
bad_path = Path(tmp.name)
try:
with pytest.raises(RuntimeError, match="Failed to load audio file"):
fingerprinter.load_audio(bad_path)
finally:
bad_path.unlink(missing_ok=True)
def test_extract_chromagram(
fingerprinter: AudioFingerprinter,
sample_audio: tuple[np.ndarray, int],
) -> None:
audio, _ = sample_audio
chromagram = fingerprinter.extract_chromagram(audio)
assert isinstance(chromagram, np.ndarray)
assert chromagram.shape[0] == fingerprinter.n_chroma
assert chromagram.shape[1] > 0
assert chromagram.min() >= 0.0
def test_find_peaks(
fingerprinter: AudioFingerprinter,
sample_audio: tuple[np.ndarray, int],
) -> None:
audio, _ = sample_audio
chromagram = fingerprinter.extract_chromagram(audio)
peaks = fingerprinter.find_peaks(chromagram, threshold=0.5)
assert isinstance(peaks, list)
assert len(peaks) > 0
for time, chroma, amplitude in peaks:
assert isinstance(time, int)
assert isinstance(chroma, int)
assert isinstance(amplitude, float)
assert 0 <= chroma < fingerprinter.n_chroma
assert 0.0 <= amplitude <= 1.0
def test_find_peaks_threshold(
fingerprinter: AudioFingerprinter,
sample_audio: tuple[np.ndarray, int],
) -> None:
audio, _ = sample_audio
chromagram = fingerprinter.extract_chromagram(audio)
peaks_low = fingerprinter.find_peaks(chromagram, threshold=0.3)
peaks_high = fingerprinter.find_peaks(chromagram, threshold=0.7)
assert len(peaks_low) >= len(peaks_high)
def test_find_peaks_zero_magnitude(fingerprinter: AudioFingerprinter) -> None:
"""A zero chromagram must return an empty peak list."""
zero_chroma = np.zeros((fingerprinter.n_chroma, 50), dtype=np.float32)
assert fingerprinter.find_peaks(zero_chroma) == []
def test_generate_fingerprint(
fingerprinter: AudioFingerprinter,
sample_audio: tuple[np.ndarray, int],
) -> None:
audio, _ = sample_audio
chromagram = fingerprinter.extract_chromagram(audio)
peaks = fingerprinter.find_peaks(chromagram)
fingerprints = fingerprinter.generate_fingerprint(peaks, fan_value=15)
assert isinstance(fingerprints, set)
assert len(fingerprints) > 0
for fp_hash, time, delta in fingerprints:
assert isinstance(fp_hash, int)
assert isinstance(time, int)
assert isinstance(delta, int)
assert time >= 0
assert delta >= 0
def test_generate_fingerprint_fan_value(
fingerprinter: AudioFingerprinter,
sample_audio: tuple[np.ndarray, int],
) -> None:
audio, _ = sample_audio
chromagram = fingerprinter.extract_chromagram(audio)
peaks = fingerprinter.find_peaks(chromagram)
fps_small = fingerprinter.generate_fingerprint(peaks, fan_value=5)
fps_large = fingerprinter.generate_fingerprint(peaks, fan_value=20)
assert len(fps_large) >= len(fps_small)
def test_fingerprint_file(
fingerprinter: AudioFingerprinter, audio_file: Path
) -> None:
result = fingerprinter.fingerprint_file(audio_file)
assert isinstance(result, dict)
for key in ("fingerprints", "duration", "peaks_count", "fingerprints_count"):
assert key in result
assert isinstance(result["fingerprints"], set)
assert len(result["fingerprints"]) > 0
assert result["duration"] > 0.0
assert result["peaks_count"] > 0
assert result["fingerprints_count"] == len(result["fingerprints"])
def test_match_fingerprints_identical(
fingerprinter: AudioFingerprinter, audio_file: Path
) -> None:
result1 = fingerprinter.fingerprint_file(audio_file)
result2 = fingerprinter.fingerprint_file(audio_file)
match = fingerprinter.match_fingerprints(
result1["fingerprints"], result2["fingerprints"], min_matches=10
)
assert match["matched"] is True
assert match["match_count"] > 0
assert match["confidence"] > 0.9
assert abs(match["time_offset"]) <= 5
def test_match_fingerprints_min_matches(
fingerprinter: AudioFingerprinter, audio_file: Path
) -> None:
result1 = fingerprinter.fingerprint_file(audio_file)
result2 = fingerprinter.fingerprint_file(audio_file)
match_reasonable = fingerprinter.match_fingerprints(
result1["fingerprints"], result2["fingerprints"], min_matches=10
)
match_impossible = fingerprinter.match_fingerprints(
result1["fingerprints"], result2["fingerprints"], min_matches=999999
)
assert match_reasonable["matched"] is True
assert match_impossible["matched"] is False
assert match_impossible["time_offset"] == 0
def test_match_fingerprints_empty(fingerprinter: AudioFingerprinter) -> None:
"""Empty inputs must yield zero match_count, zero confidence, zero offset."""
match = fingerprinter.match_fingerprints(set(), set(), min_matches=10)
assert match["match_count"] == 0
assert match["confidence"] == 0
assert match["time_offset"] == 0
assert match["matched"] is False
def test_match_fingerprints_with_offset(
fingerprinter: AudioFingerprinter,
) -> None:
"""When matches exist, the median time offset between hashes is reported."""
query = {(1001, 0, 3), (1002, 10, 4), (1003, 20, 5)}
reference = {(1001, 100, 3), (1002, 110, 4), (1003, 120, 5)}
match = fingerprinter.match_fingerprints(query, reference, min_matches=2)
assert match["matched"] is True
assert match["match_count"] == 3
assert match["time_offset"] == 100
def test_match_fingerprints_partial_overlap(
fingerprinter: AudioFingerprinter,
) -> None:
"""Query hashes not present in reference must be skipped without crashing."""
query = {(1001, 0, 3), (9999, 10, 4), (1003, 20, 5)}
reference = {(1001, 50, 3), (1003, 70, 5), (4242, 999, 9)}
match = fingerprinter.match_fingerprints(query, reference, min_matches=2)
assert match["match_count"] == 2
assert match["matched"] is True
# Offsets are 50 and 50 -> median 50.
assert match["time_offset"] == 50
def test_match_fingerprints_zero_threshold_empty_inputs(
fingerprinter: AudioFingerprinter,
) -> None:
"""min_matches=0 with empty inputs must not attempt median on empty list."""
match = fingerprinter.match_fingerprints(set(), set(), min_matches=0)
assert match["match_count"] == 0
assert match["matched"] is True # 0 >= 0
assert match["time_offset"] == 0

17
packages/media/audio/audio-fingerprint/tsconfig.lib.json Normal file
View File

@@ -0,0 +1,17 @@
{
"$schema": "https://json.schemastore.org/tsconfig",
"extends": "../../../foundation/core/typescript-config/library.json",
"compilerOptions": {
"rootDir": "./src",
"outDir": "./dist",
"types": ["node"]
},
"include": ["src/**/*.ts"],
"exclude": [
"src/**/*.test.ts",
"src/**/*.bench.ts",
"tests/**",
"dist",
"node_modules"
]
}

38
packages/media/audio/audio-fingerprint/vite.config.ts Normal file
View File

@@ -0,0 +1,38 @@
import { defineConfig } from "vite";
import dts from "vite-plugin-dts";
export default defineConfig({
root: __dirname,
cacheDir: "../../../../node_modules/.vite/packages/audio-fingerprint",
plugins: [
dts({
entryRoot: "src",
tsconfigPath: "./tsconfig.lib.json",
insertTypesEntry: true,
}),
],
build: {
outDir: "./dist",
emptyOutDir: true,
reportCompressedSize: true,
sourcemap: true,
lib: {
entry: {
index: "src/index.ts",
"client/index": "src/client/index.ts",
"client/python-audio-service": "src/client/python-audio-service.ts",
"types/index": "src/types/index.ts",
},
formats: ["es"],
},
rollupOptions: {
treeshake: true,
output: {
preserveModules: true,
preserveModulesRoot: "src",
entryFileNames: "[name].js",
},
external: ["tslib", /^node:/],
},
},
});

34
packages/media/audio/audio-fingerprint/vitest.config.ts Normal file
View File

@@ -0,0 +1,34 @@
import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
globals: false,
environment: "node",
pool: "forks",
setupFiles: ["@suaveplan/testing/polyfills"],
include: ["src/**/*.test.ts", "tests/**/*.test.ts"],
exclude: ["dist/**", "node_modules/**", "src/__benchmarks__/**"],
testTimeout: 30000,
hookTimeout: 30000,
coverage: {
provider: "istanbul",
reporter: ["text", "html", "lcov"],
include: ["src/**/*.ts"],
exclude: [
"src/**/*.test.ts",
"src/**/*.bench.ts",
"src/**/*.schemas.ts",
"src/**/*.types.ts",
"src/**/index.ts",
"src/__benchmarks__/**",
"tests/**",
],
thresholds: {
statements: 100,
branches: 100,
functions: 100,
lines: 100,
},
},
},
});