215 lines
9.8 KiB
Markdown
215 lines
9.8 KiB
Markdown
# Design: enterprise-audio-codec-ffmpeg-native
|
||
|
||
Non-normative architecture, algorithms, and trade-offs for
|
||
`@suaveplan/audio-codec-ffmpeg-native`. Normative requirements live in
|
||
`./specs/audio-codec-ffmpeg-native/spec.md`.
|
||
|
||
---
|
||
|
||
## 1. Why subprocess, not native bindings
|
||
|
||
There are three credible ways to wrap ffmpeg in Node:
|
||
|
||
1. **Subprocess via `fluent-ffmpeg` + a bundled binary** (this package).
|
||
2. **Native libav bindings** via `beamcoder` or `node-libav` (linked C addon).
|
||
3. **Compile ffmpeg to WASM** via `ffmpeg-wasm` and run in-process.
|
||
|
||
We picked subprocess for four concrete reasons:
|
||
|
||
- **Durability across Node versions.** Native addons require N-API rebuilds for every
|
||
Node major; a static binary is a self-contained artefact that survives version churn.
|
||
- **Licensing posture.** ffmpeg is GPL/LGPL with codec-specific encumbrances. Shipping
|
||
a *separate binary* (via `@ffmpeg-installer/ffmpeg`) keeps the LGPL boundary clean —
|
||
we do not statically link, we exec. Consumers may swap the binary at runtime for a
|
||
build matching their licence needs.
|
||
- **Parallel scaling.** The OS scheduler runs N independent `ffmpeg` processes across
|
||
cores for free; native bindings serialise on libav's internal threading model and
|
||
contend with Node's libuv pool.
|
||
- **Failure isolation.** A segfault in ffmpeg crashes a child process, not the host.
|
||
The semaphore + abort path turns codec faults into recoverable errors.
|
||
|
||
The cost is process-spawn latency (~5–20 ms cold) and stdin/stdout serialisation
|
||
overhead. For the >2 GB streaming use case the overhead is negligible against the
|
||
decode itself; for tiny clips, the WASM-backed sibling codecs in `audio-codecs` are the
|
||
right call. The README documents this trade explicitly.
|
||
|
||
---
|
||
|
||
## 2. Subprocess lifecycle
|
||
|
||
Every decode/encode call follows a fixed lifecycle managed by the `runner` module:
|
||
|
||
1. **Acquire** a semaphore slot (default `concurrencyLimit = 4`).
|
||
2. **Resolve** the binary path (see §5).
|
||
3. **Spawn** via `fluent-ffmpeg(binaryPath)`. Configure input/output codecs, sample
|
||
format (`f32le` for decode output, format-specific for encode input/output), and
|
||
the `-hide_banner -loglevel error` flags.
|
||
4. **Pipe** the source bytes into ffmpeg's `stdin` as a `Readable` derived from the
|
||
input `Uint8Array` / `ReadableStream`.
|
||
5. **Drain** `stdout` into a `Writable` that accumulates `Float32Array` frames (decode)
|
||
or `Uint8Array` chunks (encode). Output is **chunk-by-chunk**; we never `Buffer.concat`
|
||
the full payload.
|
||
6. **Capture** `stderr` into a bounded ring buffer (last 64 KiB) used only to enrich
|
||
error context on failure; in the happy path it is discarded.
|
||
7. **Resolve** on `exit` code 0 + clean stream-end. Reject on non-zero exit
|
||
(`_DECODE_FAILED` / `_ENCODE_FAILED`), spawn failure (`_SPAWN_FAILED`), timeout
|
||
(`_TIMEOUT`), abort (`_ABORTED`), or unsupported codec mapping (`_UNSUPPORTED_CODEC`).
|
||
8. **Release** the semaphore slot in a `finally` block — never leak slots on error.
|
||
|
||
The runner module is ~150 lines and the only place `child_process` / `fluent-ffmpeg`
|
||
is imported. Every other module talks to it through `runDecode(...)` / `runEncode(...)`
|
||
function calls. This concentrates the subprocess surface in one well-tested file.
|
||
|
||
---
|
||
|
||
## 3. Concurrency limit — in-package semaphore
|
||
|
||
We do **not** pull in `p-limit` or `async-sema`. The behaviour we need is ~30 lines:
|
||
|
||
```ts
|
||
class Semaphore {
|
||
private active = 0;
|
||
private readonly queue: Array<() => void> = [];
|
||
constructor(private readonly max: number) {}
|
||
|
||
async acquire(): Promise<void> {
|
||
if (this.active < this.max) { this.active++; return; }
|
||
await new Promise<void>((resolve) => this.queue.push(resolve));
|
||
this.active++;
|
||
}
|
||
release(): void {
|
||
this.active--;
|
||
const next = this.queue.shift();
|
||
if (next) next();
|
||
}
|
||
}
|
||
```
|
||
|
||
The semaphore is owned by the `FfmpegNativeCodec` instance — one per codec, shared
|
||
across all its decode/encode calls. A fresh codec instance per request would defeat
|
||
the limit, so the `register(registry, options)` helper constructs one codec and
|
||
registers it. The module-level singleton lives inside the consumer's registry, not
|
||
inside this package.
|
||
|
||
---
|
||
|
||
## 4. Streaming design
|
||
|
||
Decode and encode both go chunk-by-chunk; the spec forbids buffering full files.
|
||
|
||
### Decode pipeline (`decode/decode.ts` and `stream/stream-decoder.ts`)
|
||
|
||
```text
|
||
Uint8Array | ReadableStream<Uint8Array>
|
||
→ stdin (ffmpeg child, args: -i pipe:0 -f f32le -acodec pcm_f32le pipe:1)
|
||
→ stdout chunks (raw float32 little-endian, interleaved)
|
||
→ interleave splitter (chunks → Float32Array[] per channel)
|
||
→ DecodeResult { channelData, sampleRate, format }
|
||
```
|
||
|
||
For the streaming variant `createDecoder()` returns a `StreamingDecoder` whose
|
||
`write(chunk)` forwards to stdin and whose `flush()` ends stdin and resolves with
|
||
the trailing decoded frames. Internally this is a `@suaveplan/streams`
|
||
`TransformStream<Uint8Array, AudioPage>`, so consumers compose it into
|
||
`pipeThrough` without an adapter.
|
||
|
||
### Encode pipeline (`encode/encode.ts` and `stream/stream-encoder.ts`)
|
||
|
||
```text
|
||
Float32Array[] (per-channel PCM)
|
||
→ interleaver (per-channel → interleaved Float32 buffer)
|
||
→ stdin (ffmpeg child, args: -f f32le -ar <sr> -ac <ch> -i pipe:0 -f <fmt> pipe:1)
|
||
→ stdout chunks (encoded format bytes)
|
||
→ Uint8Array (decode) | TransformStream<Float32Array[], Uint8Array> (streaming)
|
||
```
|
||
|
||
No intermediate `Buffer.concat` on the hot path. The interleaver writes directly into
|
||
a fixed-size 64 KiB working buffer and flushes to stdin when full.
|
||
|
||
---
|
||
|
||
## 5. Binary discovery order
|
||
|
||
The `binary/binary.ts` module resolves the binary path in this exact order:
|
||
|
||
1. `options.binaryProvider?.()` — synchronous test override.
|
||
2. `options.binaryPath` — explicit caller-supplied absolute path.
|
||
3. `require("@ffmpeg-installer/ffmpeg").path` — bundled static binary
|
||
(declared in `optionalDependencies` so installation never fails if the OS lacks
|
||
a prebuilt). Loaded via dynamic `import()` inside the resolver, NOT at module top.
|
||
4. `throw new AudioCodecFfmpegNativeError("AUDIO_CODEC_FFMPEG_NATIVE.BINARY_MISSING")`.
|
||
|
||
Resolution is cached on the codec instance after first success. The cache is cleared
|
||
on `register()` reconstruction, so tests can rebind a different provider per case.
|
||
|
||
---
|
||
|
||
## 6. Format support matrix
|
||
|
||
| Format | Decode | Encode | Streaming decode | Streaming encode | Notes |
|
||
|--------------|:------:|:------:|:----------------:|:----------------:|-------|
|
||
| `wav` | ✅ | ✅ | ✅ | ✅ | PCM 8/16/24/32-bit + float |
|
||
| `mp3` | ✅ | ✅ | ✅ | ✅ | libmp3lame for encode |
|
||
| `ogg` | ✅ | ✅ | ✅ | ✅ | Vorbis in Ogg container |
|
||
| `opus` | ✅ | ✅ | ✅ | ✅ | Opus in Ogg or WebM container — declared `opus` accepts either; encode emits Ogg/Opus |
|
||
| `flac` | ✅ | ✅ | ✅ | ✅ | FLAC native |
|
||
| `aac` / `m4a`| ✅ | ✅ | ✅ | ✅ | aac in MP4/M4A container; encode requires `libfdk_aac` or `aac` codec — runner detects and surfaces `_UNSUPPORTED_CODEC` if neither is in the binary |
|
||
| `webm` | ✅ | ✅ | ✅ | ✅ | Matroska/WebM container, Opus or Vorbis audio |
|
||
| `wma` | ✅ | 🚫 | ✅ | 🚫 | Decode only — most ffmpeg builds ship `wmav2` decoder but no encoder; encode raises `_UNSUPPORTED_CODEC` |
|
||
|
||
`canDecode(bytes)` delegates magic-byte detection to `@suaveplan/audio-metadata`'s
|
||
detector and answers `true` iff the detected format is in the supported list above.
|
||
|
||
---
|
||
|
||
## 7. Trade-offs vs alternatives
|
||
|
||
- **vs `beamcoder` (libav bindings).** beamcoder gives lower latency and finer-grained
|
||
control over individual codec frames, but it ships prebuilt addons that lag Node
|
||
releases by months, locks the licence boundary (statically links libav), and
|
||
serialises across libav's global state. We accept ~10 ms of spawn latency in
|
||
exchange for upgrade safety, licence clarity, and free per-process parallelism.
|
||
- **vs `ffmpeg-wasm`.** WASM runs in-process and is cross-platform without a binary
|
||
install, but it is 3–10× slower than native ffmpeg and has the same single-threaded
|
||
serialisation issue as beamcoder. For the universal/browser tier we already ship
|
||
WASM codecs from `@suaveplan/audio-codecs` (Mp3Codec, FlacCodec, …). This package
|
||
exists specifically because servers should not pay the WASM tax for batch jobs.
|
||
|
||
---
|
||
|
||
## 8. AbortSignal propagation
|
||
|
||
Every public method accepts a `signal?: AbortSignal`. The `abort/abort.ts` module
|
||
wires it into the subprocess:
|
||
|
||
1. On signal abort, send `SIGTERM` to the child.
|
||
2. Start a 1-second grace timer.
|
||
3. If the child has not exited when the timer fires, send `SIGKILL`.
|
||
4. Reject the call with `_ABORTED` carrying the abort `reason` in `context`.
|
||
5. Release the semaphore slot in `finally`.
|
||
|
||
A signal already aborted when the call begins short-circuits before spawn — no child
|
||
process is created. This matches the `audio-codecs` `_STREAM_ERROR` semantics from the
|
||
upstream contract.
|
||
|
||
---
|
||
|
||
## 9. OpenTelemetry
|
||
|
||
- Spans: `audio-codec-ffmpeg-native.decode`, `audio-codec-ffmpeg-native.encode`,
|
||
carrying `format`, `sampleRate`, `channels`, and `byteLength` as attributes.
|
||
- Counter: `audio_codec_ffmpeg_native_invocations_total` with `op=decode|encode` and
|
||
`format=<fmt>` labels.
|
||
- Histogram: `audio_codec_ffmpeg_native_duration_ms` with the same labels.
|
||
- Gauge: `audio_codec_ffmpeg_native_inflight` tracking active subprocess count.
|
||
|
||
Instrumentation is inert without a registered exporter (no import-time side effects).
|
||
|
||
---
|
||
|
||
## 10. File layout
|
||
|
||
See `proposal.md` § "Module layout". The five-file pattern (`feature.ts`,
|
||
`feature.test.ts`, `feature.types.ts`, `feature.md`, `index.ts`) applies to every
|
||
feature directory; barrels are explicit named re-exports per §6a.
|