Initial Commit
This commit is contained in:
274
openspec/changes/enterprise-audio-codecs/design.md
Normal file
274
openspec/changes/enterprise-audio-codecs/design.md
Normal file
@@ -0,0 +1,274 @@
|
||||
# Design: enterprise-audio-codecs
|
||||
|
||||
**Change ID**: enterprise-audio-codecs
|
||||
**Package**: `@suaveplan/audio-codecs`
|
||||
|
||||
`@suaveplan/audio-codecs` is the universal format decode/encode layer of the `@suaveplan/audio-*`
|
||||
family. It turns any audio source — file path, URL, buffer, or byte stream — into an
|
||||
`AudioDocument`, and turns an `AudioDocument` back into encoded bytes in any supported format. This
|
||||
document carries the **package-specific design** for the codec layer.
|
||||
|
||||
---
|
||||
|
||||
## 1. Relationship to the ecosystem architecture
|
||||
|
||||
The cross-cutting `@suaveplan/audio-*` architecture — the three-layer separation (data model /
|
||||
timeline-transport / rendering engine), the six plugin slots, the "zero duplication of genesis
|
||||
primitives" rule, and the universal-tier purity rule — lives in
|
||||
`enterprise-audio-core/design.md` §1–§4. It is **referenced here, not duplicated**.
|
||||
|
||||
`audio-codecs` occupies **plugin slot 4 (Codec)** from that architecture: it decodes / encodes a
|
||||
specific format and registers codecs in a `CodecRegistry`. It sits directly above `audio-core` in
|
||||
the dependency graph and below every rendering engine and CLI. Its output is the universal
|
||||
`AudioDocument` defined by `audio-core` — the codec layer never owns its own audio container.
|
||||
|
||||
The `Segment` shape referenced anywhere in this package is the single family-wide definition from
|
||||
`enterprise-audio-core/design.md` §2.2: `{ from, count, to, rate?, ref?, interp? }`. Codecs do not
|
||||
introduce segment variants — they produce `AudioPage` PCM, and editing/segmentation is `audio-core`'s
|
||||
job.
|
||||
|
||||
---
|
||||
|
||||
## 2. Decode model
|
||||
|
||||
### 2.1. Source resolution
|
||||
|
||||
A `Source` is one of `string | URL | ArrayBuffer | Uint8Array | ReadableStream<Uint8Array>`.
|
||||
Resolution rules:
|
||||
|
||||
- `string` starting with `http`/`https`/`data`/`blob`, or any `URL` → `fetch()`.
|
||||
- `string` starting with `file:` → dynamic `import('node:fs/promises')`.
|
||||
- other bare `string` → `fs/promises.readFile` on a server runtime, `fetch()` in a browser.
|
||||
- `ArrayBuffer` / `Uint8Array` / `ReadableStream<Uint8Array>` → consumed directly, no platform call.
|
||||
|
||||
The `node:fs/promises` import is **always** inside the runtime-gated branch — never a top-level
|
||||
import — so the package stays universal-tier per `enterprise-audio-core/design.md` §1.5.
|
||||
|
||||
### 2.2. Format detection
|
||||
|
||||
```ts
|
||||
const HEADER_SIZE = 256 * 1024
|
||||
|
||||
async function detectSource(source: Source) {
|
||||
if (typeof source === 'string' && !source.startsWith('http')) {
|
||||
const { open } = await import('node:fs/promises')
|
||||
const fh = await open(source, 'r')
|
||||
const hdr = new Uint8Array(HEADER_SIZE)
|
||||
await fh.read(hdr, 0, HEADER_SIZE, 0)
|
||||
await fh.close()
|
||||
return { format: detectFormat(hdr), path: source }
|
||||
}
|
||||
const bytes = await resolveToUint8Array(source)
|
||||
return { format: detectFormat(bytes.subarray(0, HEADER_SIZE)), bytes }
|
||||
}
|
||||
```
|
||||
|
||||
`detectFormat` delegates to `@suaveplan/audio-metadata`'s magic-byte sniffing. Detection never
|
||||
throws for unknown content — it returns `format: undefined`, and the caller falls back to an
|
||||
explicit `opts.format` hint or throws `AUDIO_CODECS.UNKNOWN_FORMAT`. Reading only the first 256 KiB
|
||||
keeps detection cheap even for multi-gigabyte files.
|
||||
|
||||
### 2.3. Progressive streaming decode
|
||||
|
||||
The decoder fills pages progressively so the document is editable before decode completes:
|
||||
|
||||
```ts
|
||||
async function progressiveDecode(source, opts, registry) {
|
||||
const { format, reader, bytes } = await detectSource(source)
|
||||
const codec = registry.get(format ?? opts.format)
|
||||
const dec = codec.createDecoder?.() ?? null
|
||||
|
||||
const acc = pageAccumulator({
|
||||
pages: opts.pages ?? [],
|
||||
notify: opts.notify,
|
||||
ondata: opts.ondata,
|
||||
})
|
||||
|
||||
if (reader) {
|
||||
for await (const chunk of reader) {
|
||||
const result = dec ? await dec.write(chunk) : await codec.decode(chunk)
|
||||
if (result?.channelData.length) acc.push(result.channelData, result.sampleRate)
|
||||
}
|
||||
if (dec) {
|
||||
const flushed = await dec.flush()
|
||||
if (flushed?.channelData.length) acc.push(flushed.channelData, flushed.sampleRate)
|
||||
}
|
||||
}
|
||||
|
||||
const final = acc.done()
|
||||
return createDocument(acc.pages, acc.sampleRate, acc.channels, final.length, opts, final.stats)
|
||||
}
|
||||
```
|
||||
|
||||
**Key property:** the returned `AudioDocument` is usable immediately. Because `audio-core` edits are
|
||||
metadata (`enterprise-audio-core/design.md` §2.1), an edit chained onto the partial document applies
|
||||
to whatever pages exist and extends automatically as the accumulator pushes more pages. `notify()`
|
||||
fires when the first page is ready; `ondata(delta)` fires per decoded block with running stats.
|
||||
|
||||
### 2.4. Memory budget
|
||||
|
||||
When `opts.budget` is set, `decodeSource` / `progressiveDecode` project required memory as
|
||||
`sampleRate × channels × durationSamples × 4` (bytes per `Float32` sample) and throw
|
||||
`AUDIO_CODECS.BUDGET_EXCEEDED` before allocating any page when the projection exceeds the budget. The
|
||||
projected and budgeted byte counts go into `error.context` so the caller can size a retry.
|
||||
|
||||
---
|
||||
|
||||
## 3. Encode model
|
||||
|
||||
`encodeTo(doc, destination, format, opts?)` renders the document's PCM through `audio-core`, encodes
|
||||
it via the codec registered for `format`, then writes the bytes to the `Destination`:
|
||||
|
||||
- `string` file path → dynamic `import('node:fs/promises')`, `writeFile`.
|
||||
- `WritableStream<Uint8Array>` → obtain a writer, write, close.
|
||||
- `(chunk: Uint8Array) => void` callback → invoke per chunk (single chunk for non-streaming codecs).
|
||||
|
||||
Streaming encode is exposed two ways: `CodecRegistry.encodeStream(doc, format)` returns a
|
||||
`ReadableStream<Uint8Array>`, and `createEncodeTransform(format)` returns a `@suaveplan/streams`
|
||||
`TransformStream<Float32Array[], Uint8Array>` so a codec stage drops into a stream pipeline with
|
||||
native backpressure.
|
||||
|
||||
---
|
||||
|
||||
## 4. Codec plugin contract
|
||||
|
||||
Every codec implements `AudioCodec`:
|
||||
|
||||
```ts
|
||||
interface AudioCodec {
|
||||
readonly name: string
|
||||
readonly extensions: string[]
|
||||
readonly mimeTypes: string[]
|
||||
canDecode(bytes: Uint8Array): boolean
|
||||
decode(bytes: Uint8Array | ArrayBuffer): Promise<DecodeResult>
|
||||
encode(pcm: Float32Array[], opts: EncodeOpts): Promise<Uint8Array>
|
||||
createDecoder?(): StreamingDecoder
|
||||
createEncoder?(): StreamingEncoder
|
||||
}
|
||||
```
|
||||
|
||||
The `CodecRegistry` indexes each codec by `name`, every `extension`, and every `mimeType`.
|
||||
`detect(bytes)` returns the first codec whose `canDecode` accepts the header. Codecs that do not
|
||||
support encoding throw `AUDIO_CODECS.NOT_SUPPORTED` from `encode`; codecs that do not support
|
||||
streaming **omit** `createDecoder`/`createEncoder` rather than ship a non-functional stub — the
|
||||
registry checks for the factory's presence before offering a streaming path.
|
||||
|
||||
### 4.1. Built-in codecs
|
||||
|
||||
| Format | Decode | Encode | Streaming | Backing |
|
||||
|---|---|---|---|---|
|
||||
| WAV | yes | yes | yes | native, no deps |
|
||||
| AIFF | yes | no | no | native, no deps |
|
||||
| CAF | yes | no | no | native, no deps |
|
||||
| MP3 | yes | yes | yes | WASM `@audio/decode-mp3` |
|
||||
| FLAC | yes | yes | yes | WASM `@audio/decode-flac` |
|
||||
| Ogg Vorbis | yes | no | yes | WASM |
|
||||
| Opus | yes | yes | yes | WASM |
|
||||
| AAC | yes | no | no | WASM |
|
||||
| WebM | yes | no | no | WASM |
|
||||
| QOA | yes | yes | yes | WASM |
|
||||
|
||||
Native codecs (WAV/AIFF/CAF) parse their container chunk structures directly and require zero
|
||||
external dependencies. WASM codecs wrap a WebAssembly module loaded via dynamic `import()` of an
|
||||
`optionalDependency`, so consumers install only the formats they use. The WASM module loads lazily
|
||||
on first `decode`/`encode`/`createDecoder` — never at package import time — and a missing optional
|
||||
dependency throws `AUDIO_CODECS.NOT_SUPPORTED` naming the absent module.
|
||||
|
||||
### 4.2. `@suaveplan/streams` integration
|
||||
|
||||
```ts
|
||||
import { TransformStream } from '@suaveplan/streams'
|
||||
|
||||
function createDecodeTransform(format: string): TransformStream<Uint8Array, AudioPage> {
|
||||
const codec = registry.get(format)
|
||||
const dec = codec.createDecoder()!
|
||||
return new TransformStream({
|
||||
async transform(chunk, controller) {
|
||||
const result = await dec.write(chunk)
|
||||
if (result?.channelData.length) controller.enqueue(createPage(result.channelData))
|
||||
},
|
||||
async flush(controller) {
|
||||
const result = await dec.flush()
|
||||
if (result?.channelData.length) controller.enqueue(createPage(result.channelData))
|
||||
},
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Error hierarchy
|
||||
|
||||
`audio-codecs` registers its own `AUDIO_CODEC_*` namespace with `@suaveplan/error`, parallel to
|
||||
`audio-core`'s `AUDIO_CORE_*` namespace:
|
||||
|
||||
```ts
|
||||
defineCodes('AUDIO_CODEC', [
|
||||
{ code: 'AUDIO_CODECS.UNKNOWN_FORMAT', severity: 'error', retryable: false },
|
||||
{ code: 'AUDIO_CODECS.DECODE_FAILED', severity: 'error', retryable: false },
|
||||
{ code: 'AUDIO_CODECS.ENCODE_FAILED', severity: 'error', retryable: false },
|
||||
{ code: 'AUDIO_CODECS.STREAM_ERROR', severity: 'warning', retryable: true },
|
||||
{ code: 'AUDIO_CODECS.NOT_SUPPORTED', severity: 'error', retryable: false },
|
||||
{ code: 'AUDIO_CODECS.INVALID_SOURCE', severity: 'error', retryable: false },
|
||||
{ code: 'AUDIO_CODECS.BUDGET_EXCEEDED', severity: 'error', retryable: false },
|
||||
])
|
||||
```
|
||||
|
||||
Every code is reachable from a real call site exercised by tests — `error-codes.test.ts` asserts
|
||||
each metadata entry is thrown somewhere.
|
||||
|
||||
---
|
||||
|
||||
## 6. File layout
|
||||
|
||||
```text
|
||||
packages/media/audio/audio-codecs/
|
||||
package.json, tsconfig.json, vitest.config.ts
|
||||
src/
|
||||
index.ts
|
||||
error-codes.ts
|
||||
registry/ registry.ts .md .types.ts .test.ts index.ts
|
||||
decode/ decode-source.ts .md .types.ts .test.ts index.ts
|
||||
encode/ encode-to.ts .md .types.ts .test.ts index.ts
|
||||
detect/ detect-format.ts .md .test.ts index.ts
|
||||
stream/ stream-decoder.ts (+ .md .test.ts) · stream-encoder.ts (+ .md .test.ts) · index.ts
|
||||
codecs/ wav.ts (+ .md .test.ts) · aiff.ts · caf.ts · mp3.ts (+ .md .test.ts) · flac.ts (+ .md .test.ts) · ogg-vorbis.ts · opus.ts · aac.ts · webm.ts · qoa.ts · index.ts
|
||||
__benchmarks__/ decode.bench.ts encode.bench.ts stream-decode.bench.ts
|
||||
tests/
|
||||
integration.test.ts property.test.ts purity.test.ts exports.test.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Benchmarks
|
||||
|
||||
- `decode.bench.ts` — decode a 5-minute stereo WAV (native) and a 5-minute MP3 (WASM).
|
||||
- `encode.bench.ts` — encode 5 minutes of stereo PCM to WAV and to FLAC.
|
||||
- `stream-decode.bench.ts` — progressive stream decode of a 5-minute MP3, measuring time-to-first-page.
|
||||
|
||||
---
|
||||
|
||||
## 8. Decisions and trade-offs
|
||||
|
||||
- **Codecs are plugins, not a fixed enum.** A `CodecRegistry` rather than a hardcoded `switch`
|
||||
means consumers can register custom or proprietary codecs, and the built-in set can shrink to just
|
||||
the native WAV/AIFF/CAF codecs for a minimal install. This matches plugin slot 4 in the ecosystem
|
||||
architecture.
|
||||
- **WASM codecs are `optionalDependencies`.** Bundling every WASM decoder would bloat every install
|
||||
with formats most consumers never touch. Declaring them optional and loading via dynamic `import()`
|
||||
keeps the base package small and lets the dependency graph stay universal-tier — the import is
|
||||
lazy and runtime-gated, so a browser build that never decodes FLAC never pulls the FLAC WASM.
|
||||
- **`node:fs` is dynamic-import-only.** The package must run in browsers. Resolving a file path is
|
||||
the one server-only operation, so it lives behind a runtime-gated `import('node:fs/promises')`
|
||||
rather than a top-level import — preserving the universal-tier purity rule from
|
||||
`enterprise-audio-core/design.md` §1.5.
|
||||
- **Progressive decode returns the document early.** Because `audio-core` edits are metadata, the
|
||||
document can be handed back before decode finishes and edits chained onto it extend as pages
|
||||
arrive. This is the whole reason the codec layer produces `AudioPage` PCM rather than a monolithic
|
||||
buffer — it lets a UI start editing a long file the instant the first page lands.
|
||||
- **Memory budget is opt-in.** Projecting and checking allocation size on every decode would add
|
||||
overhead to the common small-file case. `budget` is opt-in for callers decoding untrusted or
|
||||
unbounded sources, where an OOM would otherwise crash the host.
|
||||
- **The architecture doc is not re-stated here.** Per the same convention `audio-core` uses, the
|
||||
cross-cutting model lives once in `enterprise-audio-core/design.md` §1–§4 and sibling proposals
|
||||
reference it. Duplicating it would create drift.
|
||||
Reference in New Issue
Block a user