441 lines
14 KiB
Markdown
441 lines
14 KiB
Markdown
# OpenSpec Change: enterprise-audio-core
|
||
|
||
- **Change ID**: enterprise-audio-core
|
||
- **Package**: `@suaveplan/audio-core`
|
||
- **Category**: universal
|
||
- **Target path**: `packages/audio/audio-core/`
|
||
- **Status**: PROPOSED
|
||
- **Date**: 2026-05-14
|
||
- **Design**: `./design.md` (also carries the cross-cutting `@suaveplan/audio-*` ecosystem architecture)
|
||
- **Spec delta**: `./specs/audio-core/spec.md`
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
Deliver `@suaveplan/audio-core` on the genesis branch from scratch at enterprise quality. The package provides the universal audio data model and processing primitives for the genesis monorepo: a paged audio container (`AudioDocument`), non-destructive edit compilation (`PlanEngine`), block-level DSP pipeline (`ProcessPipeline`), and typed operation registry (`OpRegistry`). Zero browser or server dependencies — runs identically in Bun, Node.js, Deno, and browsers. This is the lowest-level audio package in the ecosystem; every other audio package depends on it.
|
||
|
||
---
|
||
|
||
## Current-state audit (genesis)
|
||
|
||
The package does not exist on genesis yet — this is a fresh build. No prior source, no test results, no coverage data.
|
||
|
||
---
|
||
|
||
## Available workspace deps on genesis
|
||
|
||
| Package | Role |
|
||
|---|---|
|
||
| `@suaveplan/error` (`workspace:^`) | Base class for `AudioError`; `AUDIO_CORE.*` codes |
|
||
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `IdGenerator`, `Rng` |
|
||
| `@suaveplan/events` (`workspace:^`) | Typed event bus for `AudioDocument` lifecycle events |
|
||
| `@suaveplan/streams` (`workspace:^`) | WHATWG `TransformStream` for audio stream transforms |
|
||
| `@suaveplan/serialize` (`workspace:^`) | Structured serialization for `AudioDocument.toJSON()` |
|
||
| `@suaveplan/audio-metadata` (`workspace:^`) | Supplies the `AudioMetadata` type carried on `AudioDocument.metadata` (type-only consumption) |
|
||
| `zod` (`catalog:forms`) | Zod schemas for all public option types |
|
||
| `@opentelemetry/api` (`catalog:observability`) | Spans + counters on compile, process, and stream ops |
|
||
|
||
Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
|
||
|
||
---
|
||
|
||
## Public API
|
||
|
||
### Core types
|
||
|
||
```ts
|
||
class AudioError extends SuaveplanError
|
||
|
||
interface AudioDocumentOptions {
|
||
id?: string
|
||
sampleRate: number
|
||
channels: number
|
||
sourceLength: number
|
||
pages?: AudioPage[]
|
||
edits?: AudioEdit[]
|
||
source?: string | URL
|
||
metadata?: AudioMetadata
|
||
}
|
||
|
||
interface AudioPage {
|
||
readonly channelData: Float32Array[]
|
||
readonly startSample: number
|
||
}
|
||
|
||
interface AudioEdit {
|
||
type: string
|
||
opts: Record<string, unknown>
|
||
}
|
||
|
||
interface AudioStats {
|
||
peak: number
|
||
rms: number
|
||
lufs?: number
|
||
dcOffset: number
|
||
}
|
||
|
||
interface StreamOptions {
|
||
blockSize?: number
|
||
startSample?: number
|
||
endSample?: number
|
||
}
|
||
|
||
interface AudioDocumentEventMap {
|
||
'edit': (e: { edit: AudioEdit; version: number }) => void
|
||
'undo': (e: { edit: AudioEdit; version: number }) => void
|
||
'plugin:error': (e: { plugin: string; error: unknown }) => void
|
||
'page:loaded': (e: { page: AudioPage }) => void
|
||
}
|
||
|
||
class AudioDocument {
|
||
constructor(opts: AudioDocumentOptions)
|
||
|
||
readonly id: string
|
||
readonly pages: AudioPage[]
|
||
readonly sampleRate: number
|
||
readonly channels: number
|
||
readonly sourceLength: number
|
||
|
||
get length(): number
|
||
get duration(): number
|
||
|
||
edits: AudioEdit[]
|
||
version: number
|
||
stats?: AudioStats
|
||
source?: string | URL
|
||
metadata?: AudioMetadata
|
||
|
||
run(...edits: AudioEdit[]): this
|
||
undo(n?: number): AudioEdit | AudioEdit[] | null
|
||
clone(): AudioDocument
|
||
toJSON(): AudioDocumentJSON
|
||
|
||
read(offset?: number, duration?: number): Promise<Float32Array[]>
|
||
stream(opts?: StreamOptions): AsyncGenerator<Float32Array[]>
|
||
|
||
push(data: Float32Array[] | Float32Array, fmt?: PCMFormat): this
|
||
stop(): this
|
||
|
||
on<K extends keyof AudioDocumentEventMap>(event: K, listener: AudioDocumentEventMap[K]): void
|
||
off<K extends keyof AudioDocumentEventMap>(event: K, listener: AudioDocumentEventMap[K]): void
|
||
}
|
||
```
|
||
|
||
### Plan engine
|
||
|
||
```ts
|
||
interface CompiledPlan {
|
||
segs: Segment[]
|
||
pipeline: AudioEdit[]
|
||
totalLen: number
|
||
sampleRate: number
|
||
limit: number
|
||
}
|
||
|
||
class PlanEngine {
|
||
constructor(registry: OpRegistry)
|
||
|
||
compile(doc: AudioDocument, final?: boolean): CompiledPlan
|
||
readPlan(doc: AudioDocument, plan: CompiledPlan, offset?: number, duration?: number): Float32Array[]
|
||
streamPlan(doc: AudioDocument, plan: CompiledPlan, opts?: StreamOptions): AsyncGenerator<Float32Array[]>
|
||
ensurePages(doc: AudioDocument, plan: CompiledPlan, offset?: number, duration?: number): Promise<void>
|
||
}
|
||
|
||
interface Segment {
|
||
from: number // source sample start
|
||
count: number // source sample count
|
||
to: number // output sample start
|
||
rate?: number // playback rate (1 = normal)
|
||
ref?: AudioDocument | null // external source reference (null = silence)
|
||
interp?: Interpolator // custom resampler for non-unity rate
|
||
}
|
||
|
||
interface PlanContext {
|
||
totalLen: number
|
||
sampleRate: number
|
||
offset: number
|
||
length: number
|
||
opts: Record<string, unknown>
|
||
}
|
||
|
||
type PlanFn = (segs: Segment[], ctx: PlanContext) => Segment[]
|
||
type ResolveFn = (ctx: { stats?: AudioStats; sampleRate: number; channels: number; opts: Record<string, unknown> }) => AudioEdit[] | false | void
|
||
type SampleRateTransformFn = (sr: number, opts: Record<string, unknown>) => number | void
|
||
type ChannelTransformFn = (ch: number, opts: Record<string, unknown>) => number | void
|
||
```
|
||
|
||
### Process pipeline
|
||
|
||
```ts
|
||
interface ProcessState {
|
||
op: AudioOpDescriptor
|
||
params: Record<string, unknown>
|
||
buffers: Float32Array[]
|
||
}
|
||
|
||
interface BlockContext {
|
||
blockOffsetSamples: number
|
||
sampleRate: number
|
||
channels: number
|
||
}
|
||
|
||
class ProcessPipeline {
|
||
constructor(registry: OpRegistry)
|
||
|
||
init(pipeline: AudioEdit[], totalDuration: number, sampleRate: number, channels: number): ProcessState[]
|
||
patch(state: ProcessState[], pipeline: AudioEdit[]): void
|
||
process(
|
||
input: Float32Array[],
|
||
output: Float32Array[],
|
||
state: ProcessState[],
|
||
blockOffsetSamples: number,
|
||
sampleRate: number
|
||
): Float32Array[]
|
||
}
|
||
```
|
||
|
||
### Operation registry
|
||
|
||
```ts
|
||
interface AudioOpDescriptor {
|
||
readonly name: string
|
||
readonly params?: string[]
|
||
readonly resolve?: ResolveFn
|
||
readonly plan?: PlanFn
|
||
readonly process?: ProcessFn
|
||
readonly sr?: SampleRateTransformFn
|
||
readonly ch?: ChannelTransformFn
|
||
}
|
||
|
||
type ProcessFn = (
|
||
input: Float32Array[],
|
||
output: Float32Array[],
|
||
ctx: BlockContext
|
||
) => void
|
||
|
||
class OpRegistry {
|
||
register(descriptor: AudioOpDescriptor): void
|
||
unregister(name: string): void
|
||
get(name: string): AudioOpDescriptor | undefined
|
||
list(): AudioOpDescriptor[]
|
||
|
||
static createWithDefaults(): OpRegistry
|
||
}
|
||
```
|
||
|
||
### Block processor interface
|
||
|
||
```ts
|
||
interface ProcessorConfig {
|
||
sampleRate: number
|
||
channels: number
|
||
blockSize: number
|
||
}
|
||
|
||
interface BlockProcessor {
|
||
readonly name: string
|
||
setup(config: ProcessorConfig): void
|
||
process(input: Float32Array[], output: Float32Array[], ctx: BlockContext): void
|
||
reset(): void
|
||
dispose(): void
|
||
}
|
||
```
|
||
|
||
### Paged memory primitives
|
||
|
||
```ts
|
||
const PAGE_SIZE = 1024 * 1024 // samples (~23MB stereo @ 48kHz)
|
||
const BLOCK_SIZE = 1024 // samples per process block
|
||
|
||
function walkPages(
|
||
doc: AudioDocument,
|
||
channel: number,
|
||
startSample: number,
|
||
count: number,
|
||
visitor: (page: AudioPage, offset: number, length: number) => void
|
||
): void
|
||
|
||
function copyPages(
|
||
doc: AudioDocument,
|
||
channel: number,
|
||
startSample: number,
|
||
count: number,
|
||
target: Float32Array,
|
||
targetOffset: number
|
||
): void
|
||
|
||
function readPages(doc: AudioDocument, offset?: number, duration?: number): Float32Array[]
|
||
```
|
||
|
||
### Built-in operations
|
||
|
||
Plan ops: `trim`, `crop`, `speed`, `stretch`, `reverse`, `insert`, `remove`, `repeat`, `pad`, `resample`.
|
||
Process ops: `gain`, `normalize`, `fade`, `pan`, `mix`, `dither`, `transform`.
|
||
|
||
### Error codes
|
||
|
||
```ts
|
||
type AudioCoreErrorCode =
|
||
| 'AUDIO_PLUGIN_NOT_FOUND'
|
||
| 'AUDIO_INVALID_EDIT'
|
||
| 'AUDIO_STREAM_ABORTED'
|
||
| 'AUDIO_PAGE_OUT_OF_RANGE'
|
||
| 'AUDIO_PROCESS_ERROR'
|
||
|
||
const AUDIO_CORE_ERROR_METADATA: Readonly<Record<AudioCoreErrorCode, CodeMetadata>>
|
||
function registerAudioCoreErrorCodes(): void
|
||
```
|
||
|
||
### Zod schemas
|
||
|
||
```ts
|
||
AudioDocumentOptionsSchema
|
||
AudioPageSchema
|
||
AudioEditSchema
|
||
AudioStatsSchema
|
||
StreamOptionsSchema
|
||
CompiledPlanSchema
|
||
AudioOpDescriptorSchema
|
||
ProcessorConfigSchema
|
||
```
|
||
|
||
---
|
||
|
||
## New additions for genesis
|
||
|
||
1. **Zod boundary validation** — every public options object validated at entry.
|
||
2. **OTel instrumentation** — spans on `plan.compile`, `process.block`, `stream.read`; counters for `audio_core_edits_total`, `audio_core_stream_blocks_total`; histograms for `audio_core_compile_duration_ms`, `audio_core_process_duration_ms`.
|
||
3. **Error isolation** — crashing process ops emit `plugin:error` and pass input through unmodified without corrupting the stream.
|
||
4. **AbortSignal propagation** — threaded through `read()`, `stream()`, `ensurePages()`.
|
||
5. **Page pool allocator** — reusable `Float32Array` pages to reduce GC pressure during long streams.
|
||
6. **Property-based tests** — plan compilation is deterministic and reversible for identity edits; stream yields correct total sample count; clone produces equivalent `toJSON()`.
|
||
7. **Benchmark suite** — `plan.bench.ts` (100 edits × 1000 iterations), `stream.bench.ts` (5 min stereo through 10 ops), `process.bench.ts` (normalize + fade + pan over 1M samples).
|
||
|
||
---
|
||
|
||
## File layout
|
||
|
||
```text
|
||
packages/audio/audio-core/
|
||
package.json
|
||
tsconfig.json
|
||
vitest.config.ts
|
||
src/
|
||
index.ts
|
||
error-codes.ts
|
||
document/
|
||
document.ts
|
||
document.md
|
||
document.types.ts
|
||
document.test.ts
|
||
index.ts
|
||
page/
|
||
page.ts
|
||
page.md
|
||
page.types.ts
|
||
page.test.ts
|
||
index.ts
|
||
plan/
|
||
plan-engine.ts
|
||
plan-engine.md
|
||
plan-engine.types.ts
|
||
plan-engine.test.ts
|
||
segment.ts
|
||
segment.md
|
||
segment.types.ts
|
||
segment.test.ts
|
||
index.ts
|
||
process/
|
||
process-pipeline.ts
|
||
process-pipeline.md
|
||
process-pipeline.types.ts
|
||
process-pipeline.test.ts
|
||
block-processor.ts
|
||
block-processor.md
|
||
block-processor.types.ts
|
||
block-processor.test.ts
|
||
index.ts
|
||
registry/
|
||
op-registry.ts
|
||
op-registry.md
|
||
op-registry.types.ts
|
||
op-registry.test.ts
|
||
built-in/
|
||
trim.ts
|
||
crop.ts
|
||
speed.ts
|
||
stretch.ts
|
||
reverse.ts
|
||
insert.ts
|
||
remove.ts
|
||
repeat.ts
|
||
pad.ts
|
||
resample.ts
|
||
gain.ts
|
||
normalize.ts
|
||
fade.ts
|
||
pan.ts
|
||
mix.ts
|
||
dither.ts
|
||
transform.ts
|
||
index.ts
|
||
stream/
|
||
stream.ts
|
||
stream.md
|
||
stream.types.ts
|
||
stream.test.ts
|
||
index.ts
|
||
render/
|
||
render.ts
|
||
render.md
|
||
render.test.ts
|
||
index.ts
|
||
stats/
|
||
stats.ts
|
||
stats.md
|
||
stats.types.ts
|
||
stats.test.ts
|
||
index.ts
|
||
__benchmarks__/
|
||
plan.bench.ts
|
||
stream.bench.ts
|
||
process.bench.ts
|
||
tests/
|
||
integration.test.ts
|
||
```
|
||
|
||
---
|
||
|
||
## Impact
|
||
|
||
**Breaking changes**: none — fresh build, package does not yet exist on genesis.
|
||
|
||
**Prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/streams`, `@suaveplan/serialize`, `@suaveplan/audio-metadata` published to Verdaccio.
|
||
|
||
**No dependents may import browser or server APIs through this package.**
|
||
|
||
### Affected dependents
|
||
|
||
`@suaveplan/audio-core` is the foundation of the audio family — every other audio package depends
|
||
on it. At the time of this change none of the dependents exist on genesis yet (they land in later
|
||
changes in this same batch), so there is no existing dependent surface to test or migrate. Each
|
||
dependent is created against this package's spec delta, not retrofitted.
|
||
|
||
- `@suaveplan/audio-codecs` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created against this spec in `enterprise-audio-codecs`)
|
||
- `@suaveplan/audio-analysis-beat` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-analysis-beat`)
|
||
- `@suaveplan/audio-analysis-pitch` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-analysis-pitch`)
|
||
- `@suaveplan/audio-analysis-loudness` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-analysis-loudness`)
|
||
- `@suaveplan/audio-analysis-spectrum` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-analysis-spectrum`)
|
||
- `@suaveplan/audio-timeline` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-timeline`)
|
||
- `@suaveplan/audio-transport` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-transport`)
|
||
- `@suaveplan/audio-engine-browser` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-engine-browser`)
|
||
- `@suaveplan/audio-engine-server` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-engine-server`)
|
||
- `@suaveplan/audio-plugins-fx` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-plugins-fx`)
|
||
- `@suaveplan/audio-controllers` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-controllers`)
|
||
- `@suaveplan/audio-cli` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason: does not exist yet; created in `enterprise-audio-cli`)
|
||
|
||
Once landed, `@suaveplan/audio-core` is a foundation package: any subsequent change to its public
|
||
surface MUST run `bunx turbo test --filter=...@suaveplan/audio-core` per `openspec/AGENTS.md` §13
|
||
step 10.
|