suaveplan/openspec/archive/2026-05-23-enterprise-audio-codec-ffmpeg-native/design.md

Design: enterprise-audio-codec-ffmpeg-native

Non-normative architecture, algorithms, and trade-offs for @suaveplan/audio-codec-ffmpeg-native. Normative requirements live in ./specs/audio-codec-ffmpeg-native/spec.md.

---

1. Why subprocess, not native bindings

There are three credible ways to wrap ffmpeg in Node:

1. Subprocess via fluent-ffmpeg + a bundled binary (this package).
2. Native libav bindings via beamcoder or node-libav (linked C addon).
3. Compile ffmpeg to WASM via ffmpeg-wasm and run in-process.

We picked subprocess for four concrete reasons:

• Durability across Node versions. Native addons require N-API rebuilds for every Node major; a static binary is a self-contained artefact that survives version churn.
• Licensing posture. ffmpeg is GPL/LGPL with codec-specific encumbrances. Shipping a separate binary (via @ffmpeg-installer/ffmpeg) keeps the LGPL boundary clean — we do not statically link, we exec. Consumers may swap the binary at runtime for a build matching their licence needs.
• Parallel scaling. The OS scheduler runs N independent ffmpeg processes across cores for free; native bindings serialise on libav's internal threading model and contend with Node's libuv pool.
• Failure isolation. A segfault in ffmpeg crashes a child process, not the host. The semaphore + abort path turns codec faults into recoverable errors.

The cost is process-spawn latency (~5–20 ms cold) and stdin/stdout serialisation overhead. For the >2 GB streaming use case the overhead is negligible against the decode itself; for tiny clips, the WASM-backed sibling codecs in audio-codecs are the right call. The README documents this trade explicitly.

---

2. Subprocess lifecycle

Every decode/encode call follows a fixed lifecycle managed by the runner module:

1. Acquire a semaphore slot (default concurrencyLimit = 4).
2. Resolve the binary path (see §5).
3. Spawn via fluent-ffmpeg(binaryPath). Configure input/output codecs, sample format (f32le for decode output, format-specific for encode input/output), and the -hide_banner -loglevel error flags.
4. Pipe the source bytes into ffmpeg's stdin as a Readable derived from the input Uint8Array / ReadableStream.
5. Drain stdout into a Writable that accumulates Float32Array frames (decode) or Uint8Array chunks (encode). Output is chunk-by-chunk; we never Buffer.concat the full payload.
6. Capture stderr into a bounded ring buffer (last 64 KiB) used only to enrich error context on failure; in the happy path it is discarded.
7. Resolve on exit code 0 + clean stream-end. Reject on non-zero exit (_DECODE_FAILED / _ENCODE_FAILED), spawn failure (_SPAWN_FAILED), timeout (_TIMEOUT), abort (_ABORTED), or unsupported codec mapping (_UNSUPPORTED_CODEC).
8. Release the semaphore slot in a finally block — never leak slots on error.

The runner module is ~150 lines and the only place child_process / fluent-ffmpeg is imported. Every other module talks to it through runDecode(...) / runEncode(...) function calls. This concentrates the subprocess surface in one well-tested file.

---

3. Concurrency limit — in-package semaphore

We do not pull in p-limit or async-sema. The behaviour we need is ~30 lines:

class Semaphore {
  private active = 0;
  private readonly queue: Array<() => void> = [];
  constructor(private readonly max: number) {}

  async acquire(): Promise<void> {
    if (this.active < this.max) { this.active++; return; }
    await new Promise<void>((resolve) => this.queue.push(resolve));
    this.active++;
  }
  release(): void {
    this.active--;
    const next = this.queue.shift();
    if (next) next();
  }
}

The semaphore is owned by the FfmpegNativeCodec instance — one per codec, shared across all its decode/encode calls. A fresh codec instance per request would defeat the limit, so the register(registry, options) helper constructs one codec and registers it. The module-level singleton lives inside the consumer's registry, not inside this package.

---

4. Streaming design

Decode and encode both go chunk-by-chunk; the spec forbids buffering full files.

Decode pipeline (decode/decode.ts and stream/stream-decoder.ts)

Uint8Array | ReadableStream<Uint8Array>
   → stdin (ffmpeg child, args: -i pipe:0 -f f32le -acodec pcm_f32le pipe:1)
   → stdout chunks (raw float32 little-endian, interleaved)
   → interleave splitter (chunks → Float32Array[] per channel)
   → DecodeResult { channelData, sampleRate, format }

For the streaming variant createDecoder() returns a StreamingDecoder whose write(chunk) forwards to stdin and whose flush() ends stdin and resolves with the trailing decoded frames. Internally this is a @suaveplan/streams TransformStream<Uint8Array, AudioPage>, so consumers compose it into pipeThrough without an adapter.

Encode pipeline (encode/encode.ts and stream/stream-encoder.ts)

Float32Array[] (per-channel PCM)
   → interleaver (per-channel → interleaved Float32 buffer)
   → stdin (ffmpeg child, args: -f f32le -ar <sr> -ac <ch> -i pipe:0 -f <fmt> pipe:1)
   → stdout chunks (encoded format bytes)
   → Uint8Array (decode) | TransformStream<Float32Array[], Uint8Array> (streaming)

No intermediate Buffer.concat on the hot path. The interleaver writes directly into a fixed-size 64 KiB working buffer and flushes to stdin when full.

---

5. Binary discovery order

The binary/binary.ts module resolves the binary path in this exact order:

1. options.binaryProvider?.() — synchronous test override.
2. options.binaryPath — explicit caller-supplied absolute path.
3. require("@ffmpeg-installer/ffmpeg").path — bundled static binary (declared in optionalDependencies so installation never fails if the OS lacks a prebuilt). Loaded via dynamic import() inside the resolver, NOT at module top.
4. throw new AudioCodecFfmpegNativeError("AUDIO_CODEC_FFMPEG_NATIVE.BINARY_MISSING").

Resolution is cached on the codec instance after first success. The cache is cleared on register() reconstruction, so tests can rebind a different provider per case.

---

6. Format support matrix

Format	Decode	Encode	Streaming decode	Streaming encode	Notes
wav	✅	✅	✅	✅	PCM 8/16/24/32-bit + float
mp3	✅	✅	✅	✅	libmp3lame for encode
ogg	✅	✅	✅	✅	Vorbis in Ogg container
opus	✅	✅	✅	✅	Opus in Ogg or WebM container — declared opus accepts either; encode emits Ogg/Opus
flac	✅	✅	✅	✅	FLAC native
aac / m4a	✅	✅	✅	✅	aac in MP4/M4A container; encode requires libfdk_aac or aac codec — runner detects and surfaces _UNSUPPORTED_CODEC if neither is in the binary
webm	✅	✅	✅	✅	Matroska/WebM container, Opus or Vorbis audio
wma	✅	🚫	✅	🚫	Decode only — most ffmpeg builds ship wmav2 decoder but no encoder; encode raises _UNSUPPORTED_CODEC

canDecode(bytes) delegates magic-byte detection to @suaveplan/audio-metadata's detector and answers true iff the detected format is in the supported list above.

---

7. Trade-offs vs alternatives

• vs beamcoder (libav bindings). beamcoder gives lower latency and finer-grained control over individual codec frames, but it ships prebuilt addons that lag Node releases by months, locks the licence boundary (statically links libav), and serialises across libav's global state. We accept ~10 ms of spawn latency in exchange for upgrade safety, licence clarity, and free per-process parallelism.
• vs ffmpeg-wasm. WASM runs in-process and is cross-platform without a binary install, but it is 3–10× slower than native ffmpeg and has the same single-threaded serialisation issue as beamcoder. For the universal/browser tier we already ship WASM codecs from @suaveplan/audio-codecs (Mp3Codec, FlacCodec, …). This package exists specifically because servers should not pay the WASM tax for batch jobs.

---

8. AbortSignal propagation

Every public method accepts a signal?: AbortSignal. The abort/abort.ts module wires it into the subprocess:

1. On signal abort, send SIGTERM to the child.
2. Start a 1-second grace timer.
3. If the child has not exited when the timer fires, send SIGKILL.
4. Reject the call with _ABORTED carrying the abort reason in context.
5. Release the semaphore slot in finally.

A signal already aborted when the call begins short-circuits before spawn — no child process is created. This matches the audio-codecs _STREAM_ERROR semantics from the upstream contract.

---

9. OpenTelemetry

• Spans: audio-codec-ffmpeg-native.decode, audio-codec-ffmpeg-native.encode, carrying format, sampleRate, channels, and byteLength as attributes.
• Counter: audio_codec_ffmpeg_native_invocations_total with op=decode|encode and format=<fmt> labels.
• Histogram: audio_codec_ffmpeg_native_duration_ms with the same labels.
• Gauge: audio_codec_ffmpeg_native_inflight tracking active subprocess count.

Instrumentation is inert without a registered exporter (no import-time side effects).

---

10. File layout

See proposal.md § "Module layout". The five-file pattern (feature.ts, feature.test.ts, feature.types.ts, feature.md, index.ts) applies to every feature directory; barrels are explicit named re-exports per §6a.