# Design: enterprise-audio-codecs **Change ID**: enterprise-audio-codecs **Package**: `@suaveplan/audio-codecs` `@suaveplan/audio-codecs` is the universal format decode/encode layer of the `@suaveplan/audio-*` family. It turns any audio source — file path, URL, buffer, or byte stream — into an `AudioDocument`, and turns an `AudioDocument` back into encoded bytes in any supported format. This document carries the **package-specific design** for the codec layer. --- ## 1. Relationship to the ecosystem architecture The cross-cutting `@suaveplan/audio-*` architecture — the three-layer separation (data model / timeline-transport / rendering engine), the six plugin slots, the "zero duplication of genesis primitives" rule, and the universal-tier purity rule — lives in `enterprise-audio-core/design.md` §1–§4. It is **referenced here, not duplicated**. `audio-codecs` occupies **plugin slot 4 (Codec)** from that architecture: it decodes / encodes a specific format and registers codecs in a `CodecRegistry`. It sits directly above `audio-core` in the dependency graph and below every rendering engine and CLI. Its output is the universal `AudioDocument` defined by `audio-core` — the codec layer never owns its own audio container. The `Segment` shape referenced anywhere in this package is the single family-wide definition from `enterprise-audio-core/design.md` §2.2: `{ from, count, to, rate?, ref?, interp? }`. Codecs do not introduce segment variants — they produce `AudioPage` PCM, and editing/segmentation is `audio-core`'s job. --- ## 2. Decode model ### 2.1. Source resolution A `Source` is one of `string | URL | ArrayBuffer | Uint8Array | ReadableStream`. Resolution rules: - `string` starting with `http`/`https`/`data`/`blob`, or any `URL` → `fetch()`. - `string` starting with `file:` → dynamic `import('node:fs/promises')`. - other bare `string` → `fs/promises.readFile` on a server runtime, `fetch()` in a browser. - `ArrayBuffer` / `Uint8Array` / `ReadableStream` → consumed directly, no platform call. The `node:fs/promises` import is **always** inside the runtime-gated branch — never a top-level import — so the package stays universal-tier per `enterprise-audio-core/design.md` §1.5. ### 2.2. Format detection ```ts const HEADER_SIZE = 256 * 1024 async function detectSource(source: Source) { if (typeof source === 'string' && !source.startsWith('http')) { const { open } = await import('node:fs/promises') const fh = await open(source, 'r') const hdr = new Uint8Array(HEADER_SIZE) await fh.read(hdr, 0, HEADER_SIZE, 0) await fh.close() return { format: detectFormat(hdr), path: source } } const bytes = await resolveToUint8Array(source) return { format: detectFormat(bytes.subarray(0, HEADER_SIZE)), bytes } } ``` `detectFormat` delegates to `@suaveplan/audio-metadata`'s magic-byte sniffing. Detection never throws for unknown content — it returns `format: undefined`, and the caller falls back to an explicit `opts.format` hint or throws `AUDIO_CODECS.UNKNOWN_FORMAT`. Reading only the first 256 KiB keeps detection cheap even for multi-gigabyte files. ### 2.3. Progressive streaming decode The decoder fills pages progressively so the document is editable before decode completes: ```ts async function progressiveDecode(source, opts, registry) { const { format, reader, bytes } = await detectSource(source) const codec = registry.get(format ?? opts.format) const dec = codec.createDecoder?.() ?? null const acc = pageAccumulator({ pages: opts.pages ?? [], notify: opts.notify, ondata: opts.ondata, }) if (reader) { for await (const chunk of reader) { const result = dec ? await dec.write(chunk) : await codec.decode(chunk) if (result?.channelData.length) acc.push(result.channelData, result.sampleRate) } if (dec) { const flushed = await dec.flush() if (flushed?.channelData.length) acc.push(flushed.channelData, flushed.sampleRate) } } const final = acc.done() return createDocument(acc.pages, acc.sampleRate, acc.channels, final.length, opts, final.stats) } ``` **Key property:** the returned `AudioDocument` is usable immediately. Because `audio-core` edits are metadata (`enterprise-audio-core/design.md` §2.1), an edit chained onto the partial document applies to whatever pages exist and extends automatically as the accumulator pushes more pages. `notify()` fires when the first page is ready; `ondata(delta)` fires per decoded block with running stats. ### 2.4. Memory budget When `opts.budget` is set, `decodeSource` / `progressiveDecode` project required memory as `sampleRate × channels × durationSamples × 4` (bytes per `Float32` sample) and throw `AUDIO_CODECS.BUDGET_EXCEEDED` before allocating any page when the projection exceeds the budget. The projected and budgeted byte counts go into `error.context` so the caller can size a retry. --- ## 3. Encode model `encodeTo(doc, destination, format, opts?)` renders the document's PCM through `audio-core`, encodes it via the codec registered for `format`, then writes the bytes to the `Destination`: - `string` file path → dynamic `import('node:fs/promises')`, `writeFile`. - `WritableStream` → obtain a writer, write, close. - `(chunk: Uint8Array) => void` callback → invoke per chunk (single chunk for non-streaming codecs). Streaming encode is exposed two ways: `CodecRegistry.encodeStream(doc, format)` returns a `ReadableStream`, and `createEncodeTransform(format)` returns a `@suaveplan/streams` `TransformStream` so a codec stage drops into a stream pipeline with native backpressure. --- ## 4. Codec plugin contract Every codec implements `AudioCodec`: ```ts interface AudioCodec { readonly name: string readonly extensions: string[] readonly mimeTypes: string[] canDecode(bytes: Uint8Array): boolean decode(bytes: Uint8Array | ArrayBuffer): Promise encode(pcm: Float32Array[], opts: EncodeOpts): Promise createDecoder?(): StreamingDecoder createEncoder?(): StreamingEncoder } ``` The `CodecRegistry` indexes each codec by `name`, every `extension`, and every `mimeType`. `detect(bytes)` returns the first codec whose `canDecode` accepts the header. Codecs that do not support encoding throw `AUDIO_CODECS.NOT_SUPPORTED` from `encode`; codecs that do not support streaming **omit** `createDecoder`/`createEncoder` rather than ship a non-functional stub — the registry checks for the factory's presence before offering a streaming path. ### 4.1. Built-in codecs | Format | Decode | Encode | Streaming | Backing | |---|---|---|---|---| | WAV | yes | yes | yes | native, no deps | | AIFF | yes | no | no | native, no deps | | CAF | yes | no | no | native, no deps | | MP3 | yes | yes | yes | WASM `@audio/decode-mp3` | | FLAC | yes | yes | yes | WASM `@audio/decode-flac` | | Ogg Vorbis | yes | no | yes | WASM | | Opus | yes | yes | yes | WASM | | AAC | yes | no | no | WASM | | WebM | yes | no | no | WASM | | QOA | yes | yes | yes | WASM | Native codecs (WAV/AIFF/CAF) parse their container chunk structures directly and require zero external dependencies. WASM codecs wrap a WebAssembly module loaded via dynamic `import()` of an `optionalDependency`, so consumers install only the formats they use. The WASM module loads lazily on first `decode`/`encode`/`createDecoder` — never at package import time — and a missing optional dependency throws `AUDIO_CODECS.NOT_SUPPORTED` naming the absent module. ### 4.2. `@suaveplan/streams` integration ```ts import { TransformStream } from '@suaveplan/streams' function createDecodeTransform(format: string): TransformStream { const codec = registry.get(format) const dec = codec.createDecoder()! return new TransformStream({ async transform(chunk, controller) { const result = await dec.write(chunk) if (result?.channelData.length) controller.enqueue(createPage(result.channelData)) }, async flush(controller) { const result = await dec.flush() if (result?.channelData.length) controller.enqueue(createPage(result.channelData)) }, }) } ``` --- ## 5. Error hierarchy `audio-codecs` registers its own `AUDIO_CODEC_*` namespace with `@suaveplan/error`, parallel to `audio-core`'s `AUDIO_CORE_*` namespace: ```ts defineCodes('AUDIO_CODEC', [ { code: 'AUDIO_CODECS.UNKNOWN_FORMAT', severity: 'error', retryable: false }, { code: 'AUDIO_CODECS.DECODE_FAILED', severity: 'error', retryable: false }, { code: 'AUDIO_CODECS.ENCODE_FAILED', severity: 'error', retryable: false }, { code: 'AUDIO_CODECS.STREAM_ERROR', severity: 'warning', retryable: true }, { code: 'AUDIO_CODECS.NOT_SUPPORTED', severity: 'error', retryable: false }, { code: 'AUDIO_CODECS.INVALID_SOURCE', severity: 'error', retryable: false }, { code: 'AUDIO_CODECS.BUDGET_EXCEEDED', severity: 'error', retryable: false }, ]) ``` Every code is reachable from a real call site exercised by tests — `error-codes.test.ts` asserts each metadata entry is thrown somewhere. --- ## 6. File layout ```text packages/media/audio/audio-codecs/ package.json, tsconfig.json, vitest.config.ts src/ index.ts error-codes.ts registry/ registry.ts .md .types.ts .test.ts index.ts decode/ decode-source.ts .md .types.ts .test.ts index.ts encode/ encode-to.ts .md .types.ts .test.ts index.ts detect/ detect-format.ts .md .test.ts index.ts stream/ stream-decoder.ts (+ .md .test.ts) · stream-encoder.ts (+ .md .test.ts) · index.ts codecs/ wav.ts (+ .md .test.ts) · aiff.ts · caf.ts · mp3.ts (+ .md .test.ts) · flac.ts (+ .md .test.ts) · ogg-vorbis.ts · opus.ts · aac.ts · webm.ts · qoa.ts · index.ts __benchmarks__/ decode.bench.ts encode.bench.ts stream-decode.bench.ts tests/ integration.test.ts property.test.ts purity.test.ts exports.test.ts ``` --- ## 7. Benchmarks - `decode.bench.ts` — decode a 5-minute stereo WAV (native) and a 5-minute MP3 (WASM). - `encode.bench.ts` — encode 5 minutes of stereo PCM to WAV and to FLAC. - `stream-decode.bench.ts` — progressive stream decode of a 5-minute MP3, measuring time-to-first-page. --- ## 8. Decisions and trade-offs - **Codecs are plugins, not a fixed enum.** A `CodecRegistry` rather than a hardcoded `switch` means consumers can register custom or proprietary codecs, and the built-in set can shrink to just the native WAV/AIFF/CAF codecs for a minimal install. This matches plugin slot 4 in the ecosystem architecture. - **WASM codecs are `optionalDependencies`.** Bundling every WASM decoder would bloat every install with formats most consumers never touch. Declaring them optional and loading via dynamic `import()` keeps the base package small and lets the dependency graph stay universal-tier — the import is lazy and runtime-gated, so a browser build that never decodes FLAC never pulls the FLAC WASM. - **`node:fs` is dynamic-import-only.** The package must run in browsers. Resolving a file path is the one server-only operation, so it lives behind a runtime-gated `import('node:fs/promises')` rather than a top-level import — preserving the universal-tier purity rule from `enterprise-audio-core/design.md` §1.5. - **Progressive decode returns the document early.** Because `audio-core` edits are metadata, the document can be handed back before decode finishes and edits chained onto it extend as pages arrive. This is the whole reason the codec layer produces `AudioPage` PCM rather than a monolithic buffer — it lets a UI start editing a long file the instant the first page lands. - **Memory budget is opt-in.** Projecting and checking allocation size on every decode would add overhead to the common small-file case. `budget` is opt-in for callers decoding untrusted or unbounded sources, where an OOM would otherwise crash the host. - **The architecture doc is not re-stated here.** Per the same convention `audio-core` uses, the cross-cutting model lives once in `enterprise-audio-core/design.md` §1–§4 and sibling proposals reference it. Duplicating it would create drift.