Initial Commit

openspec/archive/2026-05-02-voice/proposal.md

@@ -0,0 +1,270 @@
+# OpenSpec Change: voice
+
+- **Change ID**: voice
+- **Package**: `@suaveplan/voice`
+- **Category**: universal
+- **Target path**: `universal/media/voice`
+- **Status**: PROPOSED
+- **Date**: 2026-04-24
+- **Reference spec**: none — new capability
+
+---
+
+## Summary
+
+Deliver `@suaveplan/voice` on the genesis branch from scratch at enterprise quality. The package is the provider-agnostic voice core: abstract interfaces, shared types, Zod schemas, and a structured error hierarchy for speech synthesis (TTS), speech recognition (STT), voice management, sound-effect generation, and voice isolation. No provider-specific code, no HTTP client, no API keys. Concrete provider adapters (ElevenLabs, OpenAI, Google Cloud, Azure, Amazon Polly) live in sibling `voice-*` packages.
+
+This package enables any downstream package — API plugins, React components, browser audio engines, or game narrative systems — to depend on voice contracts without pulling in provider SDKs or networking libraries.
+
+---
+
+## Current-state audit (genesis)
+
+The package does not exist on genesis yet — this is a fresh build.
+
+---
+
+## Available workspace deps on genesis
+
+| Package | Role |
+|---|---|
+| `@suaveplan/error` (`workspace:^`) | `VoiceError`; `VOICE.*` error codes |
+| `@suaveplan/types` (`workspace:^`) | `Clock`, `LoggerLike`, `CancellableOptions` |
+| `@suaveplan/streams` (`workspace:^`) | `ReadableStream` / `WritableStream` helpers for audio streaming |
+| `zod` (`catalog:forms`) | Schema validation |
+| `tslib` (`catalog:build`) | Runtime helpers |
+
+Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check` (all `catalog:testing`).
+
+---
+
+## Public API surface
+
+### Error hierarchy
+
+`VoiceError extends SuaveplanError` with codes:
+
+- `VOICE.TTS.SYNTHESIS_FAILED`
+- `VOICE.TTS.STREAM_INTERRUPTED`
+- `VOICE.STT.TRANSCRIPTION_FAILED`
+- `VOICE.STT.UNSUPPORTED_FORMAT`
+- `VOICE.VOICE.NOT_FOUND`
+- `VOICE.VOICE.CLONE_REJECTED`
+- `VOICE.EFFECTS.GENERATION_FAILED`
+- `VOICE.ISOLATION.FAILED`
+- `VOICE.PROVIDER.NOT_CONFIGURED`
+- `VOICE.PROVIDER.RATE_LIMITED`
+- `VOICE.VALIDATION.INVALID_REQUEST`
+- `VOICE.AUDIO.FORMAT_UNSUPPORTED`
+
+### Shared types (schemas + TypeScript)
+
+```ts
+// types/common
+AudioFormat: 'mp3_22050_32' | 'mp3_44100_128' | 'mp3_44100_192' | 'pcm_16000' | 'pcm_22050' | 'pcm_44100' | 'ulaw_8000' | 'wav_44100' | 'opus_48000' | 'aac_44100' | 'flac_44100'
+LatencyMode: 0 | 1 | 2 | 3 | 4
+VoiceCategory: 'premade' | 'cloned' | 'generated' | 'professional'
+VoiceGender: 'male' | 'female' | 'neutral'
+VoiceAge: 'young' | 'middle' | 'old'
+
+VoiceSettings {
+  stability?: number        // 0–1
+  similarityBoost?: number  // 0–1
+  style?: number            // 0–1
+  speakerBoost?: boolean
+}
+
+Voice {
+  id: string
+  name: string
+  category: VoiceCategory
+  gender?: VoiceGender
+  accent?: string
+  age?: VoiceAge
+  languageCodes?: string[]   // ISO 639-1
+  description?: string
+  previewUrl?: string
+  settings?: VoiceSettings
+  metadata?: Record<string, unknown>
+}
+
+ListVoicesOptions {
+  category?: VoiceCategory
+  languageCode?: string
+  search?: string
+  pageToken?: string
+  pageSize?: number          // 1–100
+}
+
+VoiceList {
+  voices: Voice[]
+  nextPageToken?: string
+  hasMore: boolean
+}
+
+AudioSource = Uint8Array | ReadableStream<Uint8Array> | string // URL or file path (provider decides)
+```
+
+### TTS engine interface
+
+```ts
+interface VoiceSynthesizer {
+  readonly providerName: string
+  synthesize(options: SynthesisRequest): Promise<SynthesisResult>
+  synthesizeStream(options: SynthesisRequest): ReadableStream<Uint8Array>
+}
+
+SynthesisRequest {
+  text: string
+  voiceId: string
+  modelId?: string
+  languageCode?: string
+  voiceSettings?: VoiceSettings
+  outputFormat?: AudioFormat
+  latencyMode?: LatencyMode
+  previousText?: string
+  nextText?: string
+  seed?: number
+  pronunciationDictionaryIds?: string[]
+  signal?: AbortSignal
+}
+
+SynthesisResult {
+  audio: Uint8Array
+  format: AudioFormat
+  durationMs?: number
+  charactersUsed?: number
+  requestId?: string
+}
+```
+
+### STT engine interface
+
+```ts
+interface TranscriptionEngine {
+  readonly providerName: string
+  transcribe(options: TranscriptionRequest): Promise<TranscriptionResult>
+}
+
+TranscriptionRequest {
+  audio: AudioSource
+  modelId?: string
+  languageCode?: string
+  tagAudioEvents?: boolean
+  numSpeakers?: number       // 1–32
+  timestampsGranularity?: 'word' | 'character'
+  diarize?: boolean
+  diarizationThreshold?: number
+  useMultiChannel?: boolean
+  entityDetection?: string[] | 'all'
+  entityRedaction?: string[] | 'all'
+  keyterms?: string[]
+  temperature?: number       // 0–2
+  seed?: number
+  signal?: AbortSignal
+}
+
+WordTimestamp {
+  word: string
+  startMs: number
+  endMs: number
+  speakerId?: string
+  channelIndex?: number
+  confidence?: number
+}
+
+SpeakerSegment {
+  speakerId: string
+  startMs: number
+  endMs: number
+}
+
+DetectedEntity {
+  text: string
+  type: string
+  startChar: number
+  endChar: number
+}
+
+TranscriptionResult {
+  text: string
+  words?: WordTimestamp[]
+  speakers?: SpeakerSegment[]
+  entities?: DetectedEntity[]
+  language?: string
+  durationMs?: number
+  requestId?: string
+}
+```
+
+### Voice repository interface
+
+```ts
+interface VoiceRepository {
+  readonly providerName: string
+  list(options?: ListVoicesOptions): Promise<VoiceList>
+  get(voiceId: string): Promise<Voice | null>
+  // Cloning is intentionally NOT in the base interface;
+  // provider-specific clone APIs vary too widely.
+}
+```
+
+### Sound-effect engine interface
+
+```ts
+interface SoundEffectEngine {
+  readonly providerName: string
+  generate(options: SoundEffectRequest): Promise<SoundEffectResult>
+}
+
+SoundEffectRequest {
+  text: string
+  durationSeconds?: number   // 0.5–30
+  loop?: boolean
+  promptInfluence?: number   // 0–1
+  modelId?: string
+  outputFormat?: AudioFormat
+  signal?: AbortSignal
+}
+
+SoundEffectResult {
+  audio: Uint8Array
+  format: AudioFormat
+  durationMs?: number
+  requestId?: string
+}
+```
+
+### Voice-isolation engine interface
+
+```ts
+interface VoiceIsolationEngine {
+  readonly providerName: string
+  isolate(options: IsolationRequest): Promise<IsolationResult>
+}
+
+IsolationRequest {
+  audio: AudioSource
+  outputFormat?: AudioFormat
+  signal?: AbortSignal
+}
+
+IsolationResult {
+  audio: Uint8Array
+  format: AudioFormat
+  requestId?: string
+}
+```
+
+---
+
+## Design decisions
+
+1. **No provider-specific types leak into the core.** `AudioSource` is intentionally loose (`Uint8Array | ReadableStream | string`) so providers can accept URLs, file paths, or raw bytes. The provider validates and narrows at runtime.
+2. **Streaming is TTS-only in the base interface.** STT streaming, real-time conversational AI, and duplex WebSocket audio are provider-specific extensions that will ship in per-provider `/streaming` subpaths, not in the core contract.
+3. **Voice cloning is excluded from `VoiceRepository`.** Clone APIs (sample upload, verification, fine-tuning status) vary radically across providers. A `VoiceCloner` interface may be added later in a provider-specific subpath or a future core revision.
+4. **Voice changing is excluded from the core.** It is an ElevenLabs-specific feature with no clear generic equivalent in other providers.
+5. **Conversational AI agents are excluded from the core.** They are a product-level abstraction above raw TTS/STT and will ship as a separate package if needed.
+6. **`@suaveplan/types` stays pure.** Voice types live in `@suaveplan/voice/types/common` so API plugins can depend on the voice contract without circular issues, while `@suaveplan/types` remains a zero-runtime foundation.
+7. **All async entrypoints accept `AbortSignal`.** Every engine method propagates cancellation to provider adapters.
+8. **All engines expose `readonly providerName`.** This enables observability tags and debugging without `instanceof` checks.