Initial Commit
This commit is contained in:
@@ -0,0 +1,214 @@
|
||||
# Design: enterprise-audio-codec-ffmpeg-native
|
||||
|
||||
Non-normative architecture, algorithms, and trade-offs for
|
||||
`@suaveplan/audio-codec-ffmpeg-native`. Normative requirements live in
|
||||
`./specs/audio-codec-ffmpeg-native/spec.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why subprocess, not native bindings
|
||||
|
||||
There are three credible ways to wrap ffmpeg in Node:
|
||||
|
||||
1. **Subprocess via `fluent-ffmpeg` + a bundled binary** (this package).
|
||||
2. **Native libav bindings** via `beamcoder` or `node-libav` (linked C addon).
|
||||
3. **Compile ffmpeg to WASM** via `ffmpeg-wasm` and run in-process.
|
||||
|
||||
We picked subprocess for four concrete reasons:
|
||||
|
||||
- **Durability across Node versions.** Native addons require N-API rebuilds for every
|
||||
Node major; a static binary is a self-contained artefact that survives version churn.
|
||||
- **Licensing posture.** ffmpeg is GPL/LGPL with codec-specific encumbrances. Shipping
|
||||
a *separate binary* (via `@ffmpeg-installer/ffmpeg`) keeps the LGPL boundary clean —
|
||||
we do not statically link, we exec. Consumers may swap the binary at runtime for a
|
||||
build matching their licence needs.
|
||||
- **Parallel scaling.** The OS scheduler runs N independent `ffmpeg` processes across
|
||||
cores for free; native bindings serialise on libav's internal threading model and
|
||||
contend with Node's libuv pool.
|
||||
- **Failure isolation.** A segfault in ffmpeg crashes a child process, not the host.
|
||||
The semaphore + abort path turns codec faults into recoverable errors.
|
||||
|
||||
The cost is process-spawn latency (~5–20 ms cold) and stdin/stdout serialisation
|
||||
overhead. For the >2 GB streaming use case the overhead is negligible against the
|
||||
decode itself; for tiny clips, the WASM-backed sibling codecs in `audio-codecs` are the
|
||||
right call. The README documents this trade explicitly.
|
||||
|
||||
---
|
||||
|
||||
## 2. Subprocess lifecycle
|
||||
|
||||
Every decode/encode call follows a fixed lifecycle managed by the `runner` module:
|
||||
|
||||
1. **Acquire** a semaphore slot (default `concurrencyLimit = 4`).
|
||||
2. **Resolve** the binary path (see §5).
|
||||
3. **Spawn** via `fluent-ffmpeg(binaryPath)`. Configure input/output codecs, sample
|
||||
format (`f32le` for decode output, format-specific for encode input/output), and
|
||||
the `-hide_banner -loglevel error` flags.
|
||||
4. **Pipe** the source bytes into ffmpeg's `stdin` as a `Readable` derived from the
|
||||
input `Uint8Array` / `ReadableStream`.
|
||||
5. **Drain** `stdout` into a `Writable` that accumulates `Float32Array` frames (decode)
|
||||
or `Uint8Array` chunks (encode). Output is **chunk-by-chunk**; we never `Buffer.concat`
|
||||
the full payload.
|
||||
6. **Capture** `stderr` into a bounded ring buffer (last 64 KiB) used only to enrich
|
||||
error context on failure; in the happy path it is discarded.
|
||||
7. **Resolve** on `exit` code 0 + clean stream-end. Reject on non-zero exit
|
||||
(`_DECODE_FAILED` / `_ENCODE_FAILED`), spawn failure (`_SPAWN_FAILED`), timeout
|
||||
(`_TIMEOUT`), abort (`_ABORTED`), or unsupported codec mapping (`_UNSUPPORTED_CODEC`).
|
||||
8. **Release** the semaphore slot in a `finally` block — never leak slots on error.
|
||||
|
||||
The runner module is ~150 lines and the only place `child_process` / `fluent-ffmpeg`
|
||||
is imported. Every other module talks to it through `runDecode(...)` / `runEncode(...)`
|
||||
function calls. This concentrates the subprocess surface in one well-tested file.
|
||||
|
||||
---
|
||||
|
||||
## 3. Concurrency limit — in-package semaphore
|
||||
|
||||
We do **not** pull in `p-limit` or `async-sema`. The behaviour we need is ~30 lines:
|
||||
|
||||
```ts
|
||||
class Semaphore {
|
||||
private active = 0;
|
||||
private readonly queue: Array<() => void> = [];
|
||||
constructor(private readonly max: number) {}
|
||||
|
||||
async acquire(): Promise<void> {
|
||||
if (this.active < this.max) { this.active++; return; }
|
||||
await new Promise<void>((resolve) => this.queue.push(resolve));
|
||||
this.active++;
|
||||
}
|
||||
release(): void {
|
||||
this.active--;
|
||||
const next = this.queue.shift();
|
||||
if (next) next();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The semaphore is owned by the `FfmpegNativeCodec` instance — one per codec, shared
|
||||
across all its decode/encode calls. A fresh codec instance per request would defeat
|
||||
the limit, so the `register(registry, options)` helper constructs one codec and
|
||||
registers it. The module-level singleton lives inside the consumer's registry, not
|
||||
inside this package.
|
||||
|
||||
---
|
||||
|
||||
## 4. Streaming design
|
||||
|
||||
Decode and encode both go chunk-by-chunk; the spec forbids buffering full files.
|
||||
|
||||
### Decode pipeline (`decode/decode.ts` and `stream/stream-decoder.ts`)
|
||||
|
||||
```text
|
||||
Uint8Array | ReadableStream<Uint8Array>
|
||||
→ stdin (ffmpeg child, args: -i pipe:0 -f f32le -acodec pcm_f32le pipe:1)
|
||||
→ stdout chunks (raw float32 little-endian, interleaved)
|
||||
→ interleave splitter (chunks → Float32Array[] per channel)
|
||||
→ DecodeResult { channelData, sampleRate, format }
|
||||
```
|
||||
|
||||
For the streaming variant `createDecoder()` returns a `StreamingDecoder` whose
|
||||
`write(chunk)` forwards to stdin and whose `flush()` ends stdin and resolves with
|
||||
the trailing decoded frames. Internally this is a `@suaveplan/streams`
|
||||
`TransformStream<Uint8Array, AudioPage>`, so consumers compose it into
|
||||
`pipeThrough` without an adapter.
|
||||
|
||||
### Encode pipeline (`encode/encode.ts` and `stream/stream-encoder.ts`)
|
||||
|
||||
```text
|
||||
Float32Array[] (per-channel PCM)
|
||||
→ interleaver (per-channel → interleaved Float32 buffer)
|
||||
→ stdin (ffmpeg child, args: -f f32le -ar <sr> -ac <ch> -i pipe:0 -f <fmt> pipe:1)
|
||||
→ stdout chunks (encoded format bytes)
|
||||
→ Uint8Array (decode) | TransformStream<Float32Array[], Uint8Array> (streaming)
|
||||
```
|
||||
|
||||
No intermediate `Buffer.concat` on the hot path. The interleaver writes directly into
|
||||
a fixed-size 64 KiB working buffer and flushes to stdin when full.
|
||||
|
||||
---
|
||||
|
||||
## 5. Binary discovery order
|
||||
|
||||
The `binary/binary.ts` module resolves the binary path in this exact order:
|
||||
|
||||
1. `options.binaryProvider?.()` — synchronous test override.
|
||||
2. `options.binaryPath` — explicit caller-supplied absolute path.
|
||||
3. `require("@ffmpeg-installer/ffmpeg").path` — bundled static binary
|
||||
(declared in `optionalDependencies` so installation never fails if the OS lacks
|
||||
a prebuilt). Loaded via dynamic `import()` inside the resolver, NOT at module top.
|
||||
4. `throw new AudioCodecFfmpegNativeError("AUDIO_CODEC_FFMPEG_NATIVE.BINARY_MISSING")`.
|
||||
|
||||
Resolution is cached on the codec instance after first success. The cache is cleared
|
||||
on `register()` reconstruction, so tests can rebind a different provider per case.
|
||||
|
||||
---
|
||||
|
||||
## 6. Format support matrix
|
||||
|
||||
| Format | Decode | Encode | Streaming decode | Streaming encode | Notes |
|
||||
|--------------|:------:|:------:|:----------------:|:----------------:|-------|
|
||||
| `wav` | ✅ | ✅ | ✅ | ✅ | PCM 8/16/24/32-bit + float |
|
||||
| `mp3` | ✅ | ✅ | ✅ | ✅ | libmp3lame for encode |
|
||||
| `ogg` | ✅ | ✅ | ✅ | ✅ | Vorbis in Ogg container |
|
||||
| `opus` | ✅ | ✅ | ✅ | ✅ | Opus in Ogg or WebM container — declared `opus` accepts either; encode emits Ogg/Opus |
|
||||
| `flac` | ✅ | ✅ | ✅ | ✅ | FLAC native |
|
||||
| `aac` / `m4a`| ✅ | ✅ | ✅ | ✅ | aac in MP4/M4A container; encode requires `libfdk_aac` or `aac` codec — runner detects and surfaces `_UNSUPPORTED_CODEC` if neither is in the binary |
|
||||
| `webm` | ✅ | ✅ | ✅ | ✅ | Matroska/WebM container, Opus or Vorbis audio |
|
||||
| `wma` | ✅ | 🚫 | ✅ | 🚫 | Decode only — most ffmpeg builds ship `wmav2` decoder but no encoder; encode raises `_UNSUPPORTED_CODEC` |
|
||||
|
||||
`canDecode(bytes)` delegates magic-byte detection to `@suaveplan/audio-metadata`'s
|
||||
detector and answers `true` iff the detected format is in the supported list above.
|
||||
|
||||
---
|
||||
|
||||
## 7. Trade-offs vs alternatives
|
||||
|
||||
- **vs `beamcoder` (libav bindings).** beamcoder gives lower latency and finer-grained
|
||||
control over individual codec frames, but it ships prebuilt addons that lag Node
|
||||
releases by months, locks the licence boundary (statically links libav), and
|
||||
serialises across libav's global state. We accept ~10 ms of spawn latency in
|
||||
exchange for upgrade safety, licence clarity, and free per-process parallelism.
|
||||
- **vs `ffmpeg-wasm`.** WASM runs in-process and is cross-platform without a binary
|
||||
install, but it is 3–10× slower than native ffmpeg and has the same single-threaded
|
||||
serialisation issue as beamcoder. For the universal/browser tier we already ship
|
||||
WASM codecs from `@suaveplan/audio-codecs` (Mp3Codec, FlacCodec, …). This package
|
||||
exists specifically because servers should not pay the WASM tax for batch jobs.
|
||||
|
||||
---
|
||||
|
||||
## 8. AbortSignal propagation
|
||||
|
||||
Every public method accepts a `signal?: AbortSignal`. The `abort/abort.ts` module
|
||||
wires it into the subprocess:
|
||||
|
||||
1. On signal abort, send `SIGTERM` to the child.
|
||||
2. Start a 1-second grace timer.
|
||||
3. If the child has not exited when the timer fires, send `SIGKILL`.
|
||||
4. Reject the call with `_ABORTED` carrying the abort `reason` in `context`.
|
||||
5. Release the semaphore slot in `finally`.
|
||||
|
||||
A signal already aborted when the call begins short-circuits before spawn — no child
|
||||
process is created. This matches the `audio-codecs` `_STREAM_ERROR` semantics from the
|
||||
upstream contract.
|
||||
|
||||
---
|
||||
|
||||
## 9. OpenTelemetry
|
||||
|
||||
- Spans: `audio-codec-ffmpeg-native.decode`, `audio-codec-ffmpeg-native.encode`,
|
||||
carrying `format`, `sampleRate`, `channels`, and `byteLength` as attributes.
|
||||
- Counter: `audio_codec_ffmpeg_native_invocations_total` with `op=decode|encode` and
|
||||
`format=<fmt>` labels.
|
||||
- Histogram: `audio_codec_ffmpeg_native_duration_ms` with the same labels.
|
||||
- Gauge: `audio_codec_ffmpeg_native_inflight` tracking active subprocess count.
|
||||
|
||||
Instrumentation is inert without a registered exporter (no import-time side effects).
|
||||
|
||||
---
|
||||
|
||||
## 10. File layout
|
||||
|
||||
See `proposal.md` § "Module layout". The five-file pattern (`feature.ts`,
|
||||
`feature.test.ts`, `feature.types.ts`, `feature.md`, `index.ts`) applies to every
|
||||
feature directory; barrels are explicit named re-exports per §6a.
|
||||
Reference in New Issue
Block a user