Initial Commit
This commit is contained in:
224
openspec/archive/2026-05-22-third-party-elevenlabs/proposal.md
Normal file
224
openspec/archive/2026-05-22-third-party-elevenlabs/proposal.md
Normal file
@@ -0,0 +1,224 @@
|
||||
# OpenSpec Change: third-party-elevenlabs
|
||||
|
||||
- **Change ID**: third-party-elevenlabs
|
||||
- **Package**: `@suaveplan/third-party-elevenlabs`
|
||||
- **Category**: universal
|
||||
- **Target path**: `packages/audio/third-party-elevenlabs`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-02
|
||||
- **Reference spec**: none — new capability (genesis-original)
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Deliver `@suaveplan/third-party-elevenlabs` on the genesis branch from scratch at enterprise quality. The package is a **typed thin client for the entire ElevenLabs platform API** — REST and WebSocket. It owns HTTP plumbing (auth, base URL, fetch wrapper), retry + circuit breaker via `@suaveplan/retry`, rate-limit-header parsing, structured error mapping, OpenTelemetry instrumentation, and Zod-validated request/response DTOs for every documented endpoint.
|
||||
|
||||
It is **not** a voice abstraction. It does not implement `@suaveplan/voice` interfaces. It is the substrate that domain adapter packages — `@suaveplan/voice-elevenlabs`, future `@suaveplan/conversational-elevenlabs`, `@suaveplan/music-elevenlabs`, `@suaveplan/dubbing-elevenlabs` — depend on. Domain adapters consume DTOs from this package and re-shape them into provider-agnostic domain types.
|
||||
|
||||
This split exists because the ElevenLabs API surface spans far beyond TTS/STT — Music, Dubbing, Studio, Audio Native, Conversational AI agents, History, Pronunciation Dictionaries, Workspace administration. Fusing HTTP plumbing into a single voice adapter would force every future surface to reinvent auth, retry, rate-limiting, OTel, and error mapping.
|
||||
|
||||
---
|
||||
|
||||
## Why a separate package
|
||||
|
||||
1. **DRY HTTP plumbing.** Auth (`xi-api-key` header), base URL handling, idempotent retry, `Retry-After` and `X-RateLimit-Remaining` parsing, 401/422/429/5xx → `SuaveplanError` mapping, OTel spans/counters/histograms — all live in one place.
|
||||
2. **Layering matches the genesis precedent.** `payment` + `payment-providers`, `currency` + `currency-providers`, `sms` + `sms-providers` already separate domain abstraction from provider HTTP. ElevenLabs is the inverse — the *vendor* surfaces multiple domains, so the vendor client is the shared layer.
|
||||
3. **Independent versioning of vendor changes.** ElevenLabs ships rapid API changes (new models, new headers, new endpoints). The thin client absorbs them; downstream adapters only update when domain semantics shift.
|
||||
4. **Reusable across workspaces.** The same client backs voice, music, dubbing, conversational-AI, and any future surface. No two adapters re-implement the same `fetch` wrapper.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Provider-agnostic abstractions. The client returns ElevenLabs-shaped DTOs (validated via Zod). Mapping to `@suaveplan/voice` types happens in `@suaveplan/voice-elevenlabs`.
|
||||
- Voice-domain factories or composite engines. Those live in `@suaveplan/voice-elevenlabs`.
|
||||
- API key management or secret storage. The caller injects `apiKey` at construction time; this package never reads from `process.env`.
|
||||
- Rate-limit *enforcement*. The client surfaces rate-limit metadata via headers + errors; quota planning is the caller's responsibility.
|
||||
|
||||
---
|
||||
|
||||
## Available workspace deps on genesis
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@suaveplan/error` (`workspace:^`) | `SuaveplanError` base + `defineCodes` for `THIRD_PARTY_ELEVENLABS.*` codes |
|
||||
| `@suaveplan/types` (`workspace:^`) | `Clock`, `LoggerLike`, `EventBusLike`, `CancellableOptions` |
|
||||
| `@suaveplan/streams` (`workspace:^`) | `toBytes` for buffering `ReadableStream` audio uploads |
|
||||
| `@suaveplan/retry` (`workspace:^`) | Retry policies, circuit breaker, backoff with jitter |
|
||||
| `@suaveplan/logger` (`workspace:^`) | Optional structured logging hook |
|
||||
| `@suaveplan/serialize` (`workspace:^`) | Multipart form-data helpers and binary-safe encoders if needed |
|
||||
| `zod` (`catalog:forms`) | DTO schemas for every endpoint |
|
||||
| `@opentelemetry/api` (`catalog:observability`) | Spans, counters, histograms |
|
||||
| `tslib` (`catalog:build`) | Runtime helpers |
|
||||
|
||||
Dev deps: `@suaveplan/testing` (`workspace:^`, dev-only), `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
|
||||
|
||||
---
|
||||
|
||||
## Public API surface
|
||||
|
||||
### Errors
|
||||
|
||||
```ts
|
||||
class ElevenLabsApiError extends SuaveplanError<ElevenLabsErrorCode> {}
|
||||
```
|
||||
|
||||
Codes (under namespace `THIRD_PARTY_ELEVENLABS`):
|
||||
|
||||
- `AUTH.UNAUTHORIZED` — 401, invalid/missing API key
|
||||
- `AUTH.FORBIDDEN` — 403, key valid but lacks scope/quota
|
||||
- `REQUEST.INVALID` — 4xx (excluding 401/403/404/422/429), client-side request shape error
|
||||
- `REQUEST.VALIDATION_FAILED` — 422, ElevenLabs validation rejection
|
||||
- `REQUEST.NOT_FOUND` — 404
|
||||
- `RATE_LIMIT.EXCEEDED` — 429 (with `retryAfterMs` context)
|
||||
- `RATE_LIMIT.QUOTA_EXHAUSTED` — quota header indicates exhaustion (subclass for caller branch)
|
||||
- `SERVER.TRANSIENT` — 5xx, retryable
|
||||
- `SERVER.UNAVAILABLE` — 503 / circuit-open
|
||||
- `NETWORK.TIMEOUT` — request timeout
|
||||
- `NETWORK.ABORTED` — caller-supplied `AbortSignal` aborted
|
||||
- `NETWORK.UNREACHABLE` — DNS / connect failure
|
||||
- `RESPONSE.MALFORMED` — JSON parse failure or Zod schema mismatch
|
||||
- `RESPONSE.UNSUPPORTED_FORMAT` — server returned content-type the client cannot handle
|
||||
- `STREAM.INTERRUPTED` — server closed the streaming response mid-flight
|
||||
- `WEBSOCKET.CONNECTION_FAILED` — handshake failure
|
||||
- `WEBSOCKET.PROTOCOL` — server sent unexpected frame / close code
|
||||
|
||||
Each code has a typed subclass (`UnauthorizedError`, `RateLimitExceededError`, etc.) so callers can branch with `instanceof`.
|
||||
|
||||
### Client
|
||||
|
||||
```ts
|
||||
class ElevenLabsClient {
|
||||
constructor(options: ElevenLabsClientOptions)
|
||||
|
||||
fetchJson<TBody>(req: JsonRequest, schema: ZodSchema<TBody>): Promise<TBody>
|
||||
fetchBinary(req: BinaryRequest): Promise<BinaryResponse>
|
||||
fetchStream(req: StreamRequest): Promise<ReadableStream<Uint8Array>>
|
||||
fetchMultipart<TBody>(req: MultipartRequest, schema: ZodSchema<TBody>): Promise<TBody>
|
||||
|
||||
openSocket(req: SocketRequest): Promise<ElevenLabsSocket>
|
||||
}
|
||||
|
||||
interface ElevenLabsClientOptions {
|
||||
apiKey: string
|
||||
baseUrl?: string
|
||||
timeoutMs?: number
|
||||
retryPolicy?: RetryPolicy
|
||||
circuitBreaker?: CircuitBreaker
|
||||
fetchImpl?: typeof fetch
|
||||
webSocketImpl?: WebSocketImpl
|
||||
clock?: Clock
|
||||
logger?: LoggerLike
|
||||
events?: EventBusLike<ElevenLabsClientEvents>
|
||||
userAgent?: string
|
||||
defaultHeaders?: Record<string, string>
|
||||
}
|
||||
```
|
||||
|
||||
### Feature modules (REST)
|
||||
|
||||
Each ElevenLabs API surface gets its own subpath module. Every module exports a class taking `ElevenLabsClient` in its constructor; Zod schemas in `*.schemas.ts`; TypeScript types inferred from Zod; sibling `.md`.
|
||||
|
||||
| Module | Subpath | ElevenLabs endpoints |
|
||||
|---|---|---|
|
||||
| `tts` | `./tts/tts` | `POST /v1/text-to-speech/{voice_id}`, `/stream`, `/with-timestamps`, `/stream/with-timestamps` |
|
||||
| `tts-websocket` | `./tts-websocket/tts-websocket` | `wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/stream-input` |
|
||||
| `stt` | `./stt/stt` | `POST /v1/speech-to-text` (multipart) |
|
||||
| `voices` | `./voices/voices` | `GET /v1/voices`, `/v1/voices/{id}`, `/v1/shared-voices`, settings get/edit, default settings |
|
||||
| `voice-cloning` | `./voice-cloning/voice-cloning` | instant clone, edit, delete, noise-reduced add |
|
||||
| `voice-design` | `./voice-design/voice-design` | generate-voice, create-voice, design, create-from-design |
|
||||
| `speech-to-speech` | `./speech-to-speech/speech-to-speech` | speech-to-speech (sync + stream) |
|
||||
| `sound-generation` | `./sound-generation/sound-generation` | `POST /v1/sound-generation` |
|
||||
| `audio-isolation` | `./audio-isolation/audio-isolation` | `POST /v1/audio-isolation`, `/stream` |
|
||||
| `music` | `./music/music` | music, music/stream, composition-plan, music/detailed |
|
||||
| `dubbing` | `./dubbing/dubbing` | project, asset, transcript, language audio, dubbing-resource CRUD |
|
||||
| `studio` | `./studio/studio` | projects + chapters + snapshots |
|
||||
| `audio-native` | `./audio-native/audio-native` | settings, content-update |
|
||||
| `history` | `./history/history` | list, get, audio, delete, download |
|
||||
| `user` | `./user/user` | user info, subscription |
|
||||
| `workspace` | `./workspace/workspace` | invites, members, groups, secrets, webhooks |
|
||||
| `pronunciation-dictionaries` | `./pronunciation-dictionaries/pronunciation-dictionaries` | CRUD + version listing |
|
||||
| `conversational-ai` | `./conversational-ai/conversational-ai` | agents, conversations, knowledge bases, tools, phone numbers, secrets, batch calls, dashboard |
|
||||
| `models` | `./models/models` | `GET /v1/models` |
|
||||
|
||||
### WebSocket transport
|
||||
|
||||
```ts
|
||||
interface ElevenLabsSocket {
|
||||
readonly url: string
|
||||
readonly readyState: 'connecting' | 'open' | 'closing' | 'closed'
|
||||
send(frame: SocketFrame): void
|
||||
close(code?: number, reason?: string): void
|
||||
onMessage(handler: (frame: SocketFrame) => void): () => void
|
||||
onClose(handler: (event: SocketCloseEvent) => void): () => void
|
||||
onError(handler: (err: ElevenLabsApiError) => void): () => void
|
||||
}
|
||||
```
|
||||
|
||||
WebSocket modules (`tts-websocket`, `conversational-ai`) expose typed wrappers that emit/parse domain frames over this transport. The transport is responsible for ping/pong, surfacing protocol errors as `WEBSOCKET.PROTOCOL` errors, and clean shutdown.
|
||||
|
||||
---
|
||||
|
||||
## Design decisions
|
||||
|
||||
1. **Native `fetch` only.** No `axios`, `node-fetch`, `undici` direct dep. Bun's `fetch` supports streaming bodies, multipart uploads, `AbortSignal`, and `ReadableStream` responses out of the box.
|
||||
2. **DTOs validated at the boundary.** Every JSON response is parsed through a Zod schema before reaching the caller. Malformed responses throw `RESPONSE.MALFORMED` with the raw body in `context` (truncated for logging).
|
||||
3. **Retry only on documented retryable codes.** 429, 502, 503, 504 and `NETWORK.UNREACHABLE` retry by default. 5xx that aren't gateway/throttling errors do not auto-retry — domain code decides whether to escalate.
|
||||
4. **`Retry-After` strict parsing.** Both seconds and HTTP-date formats. Surface in `RateLimitExceededError.context.retryAfterMs`.
|
||||
5. **`X-RateLimit-Remaining` and `X-Character-Cost` exposed via response metadata.** Every successful call returns a `metadata` envelope alongside the body so adapters can attribute usage.
|
||||
6. **Streams are never buffered server-side.** `fetchStream` returns `response.body` as `ReadableStream<Uint8Array>` directly. Callers can pipe into Web Audio, MediaSource, or buffer on demand via `@suaveplan/streams`.
|
||||
7. **Multipart uses `FormData`.** Audio uploads accept `Uint8Array | Blob | ReadableStream | string` (URL). `ReadableStream` is buffered to bytes via `@suaveplan/streams.toBytes` because ElevenLabs STT requires a complete file. URLs are fetched first.
|
||||
8. **OTel on every call.** Span name `elevenlabs.<feature>.<operation>`, attributes `elevenlabs.endpoint`, `elevenlabs.method`, `http.response.status_code`, `elevenlabs.model_id` (where present). Counter `third_party_elevenlabs.requests_total{feature, operation, status}`. Histogram `third_party_elevenlabs.request_duration_ms{feature, operation}`.
|
||||
9. **Versioned base path.** Each module is constructed with the `ElevenLabsClient`; the path version (`/v1/`) is owned by the module so a v2 migration touches only the affected module.
|
||||
10. **No singleton state.** No global registry. Every feature class is constructed per-`ElevenLabsClient` instance.
|
||||
11. **Structured error context preservation.** Every error carries `context.requestId`, `context.endpoint`, `context.statusCode` (where applicable), and `context.retryAfterMs` (for 429). The Zod-parsed body is carried under `context.body` for `VALIDATION_FAILED` only.
|
||||
12. **No conditional `process.env` reads.** This package is universal-tier and must work in browsers and edge runtimes; environment variables are the caller's concern.
|
||||
|
||||
---
|
||||
|
||||
## Implementation phases
|
||||
|
||||
The full ElevenLabs surface is large. Implementation lands in waves; the package ships at version `0.1.0` between waves.
|
||||
|
||||
| Wave | Scope | Status target |
|
||||
|---|---|---|
|
||||
| **Wave 1 — voice surface** | client, errors, schemas, OTel, retry, `tts`, `stt`, `voices`, `sound-generation`, `audio-isolation`, `speech-to-speech`, `models` | **This OpenSpec change ships Wave 1**; required by `@suaveplan/voice-elevenlabs` v1 |
|
||||
| Wave 2 — voice extensions | `tts-websocket`, `voice-cloning`, `voice-design`, `pronunciation-dictionaries` | Follow-up `third-party-elevenlabs-voice-extensions` |
|
||||
| Wave 3 — Conversational AI | `conversational-ai`, agent WebSocket frames | Follow-up `third-party-elevenlabs-conversational` |
|
||||
| Wave 4 — content ops | `music`, `dubbing`, `studio`, `audio-native` | Follow-up `third-party-elevenlabs-content-ops` |
|
||||
| Wave 5 — admin + history | `history`, `user`, `workspace` | Follow-up `third-party-elevenlabs-admin` |
|
||||
|
||||
Each subsequent wave gets its own `openspec/changes/<id>/` folder with delta `## ADDED Requirements` against `openspec/specs/third-party-elevenlabs/spec.md`.
|
||||
|
||||
---
|
||||
|
||||
## Parity Checklist
|
||||
|
||||
Genesis-original package — no upstream `@suaveplan/third-party-elevenlabs` exists in suaveplan main. Reference for behaviour: ElevenLabs API documentation at <https://third-party-elevenlabs.io/docs/api-reference/>.
|
||||
|
||||
| Capability | Upstream | This proposal | Notes |
|
||||
|---|---|---|---|
|
||||
| Typed REST client | n/a | ✅ Wave 1 | |
|
||||
| Streaming TTS | n/a | ✅ Wave 1 | `ReadableStream<Uint8Array>` |
|
||||
| WebSocket transport | n/a | ⏳ Wave 2 + Wave 3 | Plumbing in Wave 2 (tts-websocket); agents in Wave 3 |
|
||||
| Retry/backoff/circuit breaker | n/a | ✅ Wave 1 | via `@suaveplan/retry` |
|
||||
| OTel instrumentation | n/a | ✅ Wave 1 | spans + counters + histograms |
|
||||
| Rate-limit header parsing | n/a | ✅ Wave 1 | `Retry-After`, `X-RateLimit-Remaining`, `X-Character-Cost` |
|
||||
| All ElevenLabs endpoints | n/a | ✅ planned across waves 1–5 | this proposal lists every documented endpoint |
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
This is a new package — no existing dependents. After Wave 1 ships, the only direct consumer will be `@suaveplan/voice-elevenlabs` (rebuilt as a thin adapter under change `voice-elevenlabs`).
|
||||
|
||||
- `@suaveplan/voice-elevenlabs` — [x] tests run / [x] docs reviewed / [ ] no action needed (the rebuild lands together; see change `voice-elevenlabs`)
|
||||
|
||||
Future changes that consume this package: `@suaveplan/conversational-elevenlabs`, `@suaveplan/music-elevenlabs`, `@suaveplan/dubbing-elevenlabs`, `@suaveplan/studio-elevenlabs` — each will land via its own OpenSpec change.
|
||||
|
||||
### Migration
|
||||
|
||||
This is the first version of the package; no migration notes apply. When endpoints change across ElevenLabs API revisions, follow-up OpenSpec changes will document the migration in their `## Migration` sections.
|
||||
@@ -0,0 +1,168 @@
|
||||
# Capability Spec Delta: third-party-elevenlabs
|
||||
|
||||
This delta is the first revision of `openspec/specs/third-party-elevenlabs/spec.md`. On archival of change `third-party-elevenlabs` (Wave 1), the `## ADDED Requirements` block below is folded into a fresh canonical spec at `openspec/specs/third-party-elevenlabs/spec.md`.
|
||||
|
||||
Subsequent waves (`tts-websocket`, `voice-cloning`, `voice-design`, `pronunciation-dictionaries`, `conversational-ai`, `music`, `dubbing`, `studio`, `audio-native`, `history`, `user`, `workspace`) ship their own delta blocks against this canonical spec.
|
||||
|
||||
---
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Typed thin client for ElevenLabs platform API
|
||||
|
||||
The package SHALL provide a typed HTTP client (`ElevenLabsClient`) for the ElevenLabs platform API at `https://api.elevenlabs.io`, owning auth, retry, rate-limit-header parsing, error mapping, OpenTelemetry instrumentation, and Zod-validated request/response DTOs.
|
||||
|
||||
#### Scenario: Client constructs with API key
|
||||
|
||||
- **GIVEN** a caller with a valid ElevenLabs API key
|
||||
- **WHEN** the caller constructs `new ElevenLabsClient({ apiKey })`
|
||||
- **THEN** the client retains the key, applies a default base URL of `https://api.elevenlabs.io`, a default `30_000` ms timeout, and a default retry policy of three exponential-backoff attempts on retryable status codes
|
||||
- **AND** every outbound request automatically includes the header `xi-api-key: <apiKey>`
|
||||
|
||||
#### Scenario: Client must not read API key from environment
|
||||
|
||||
- **GIVEN** a caller that omits `apiKey` in the options object
|
||||
- **WHEN** they construct `new ElevenLabsClient({} as ElevenLabsClientOptions)`
|
||||
- **THEN** the constructor throws `RequestInvalidError` (`THIRD_PARTY_ELEVENLABS.REQUEST.INVALID`)
|
||||
- **AND** the package never reads `process.env.ELEVENLABS_API_KEY` (or any environment variable) at any point
|
||||
|
||||
### Requirement: Structured error mapping per HTTP status
|
||||
|
||||
The client SHALL map every non-2xx HTTP response to a typed `ElevenLabsApiError` subclass under namespace `THIRD_PARTY_ELEVENLABS`. The mapping MUST be total and disjoint over the `[400, 600)` range.
|
||||
|
||||
#### Scenario: 401 maps to UnauthorizedError
|
||||
|
||||
- **GIVEN** the server responds with HTTP `401`
|
||||
- **WHEN** the client awaits `fetchJson`/`fetchBinary`/`fetchStream`/`fetchMultipart`
|
||||
- **THEN** the client throws `UnauthorizedError` with code `THIRD_PARTY_ELEVENLABS.AUTH.UNAUTHORIZED`
|
||||
- **AND** `error.context.statusCode === 401` and `error.context.endpoint` is the requested path
|
||||
|
||||
#### Scenario: 422 maps to RequestValidationFailedError with body context
|
||||
|
||||
- **GIVEN** the server responds with HTTP `422` and a JSON error body
|
||||
- **WHEN** the client awaits the call
|
||||
- **THEN** the client throws `RequestValidationFailedError` (`REQUEST.VALIDATION_FAILED`)
|
||||
- **AND** the parsed JSON body is preserved on `error.context.body` (truncated to 4 KB if larger)
|
||||
|
||||
#### Scenario: 429 with Retry-After (seconds) sets retryAfterMs
|
||||
|
||||
- **GIVEN** the server responds with HTTP `429` and header `Retry-After: 5`
|
||||
- **WHEN** the client awaits the call
|
||||
- **THEN** the client throws `RateLimitExceededError` (`RATE_LIMIT.EXCEEDED`)
|
||||
- **AND** `error.context.retryAfterMs === 5000`
|
||||
|
||||
#### Scenario: 429 with Retry-After (HTTP-date) sets retryAfterMs
|
||||
|
||||
- **GIVEN** the server responds with HTTP `429` and header `Retry-After: <HTTP-date>` exactly 12 seconds in the future according to the injected `Clock`
|
||||
- **WHEN** the client awaits the call
|
||||
- **THEN** the client throws `RateLimitExceededError` with `error.context.retryAfterMs` between `11_500` and `12_500`
|
||||
|
||||
#### Scenario: Retryable 5xx is automatically retried
|
||||
|
||||
- **GIVEN** the server responds with HTTP `503` then HTTP `200`
|
||||
- **AND** the default retry policy of three attempts is in effect
|
||||
- **WHEN** the client awaits the call
|
||||
- **THEN** the client retries once, observes the `200`, and resolves with the parsed body
|
||||
- **AND** an OTel span event `retry.attempt` is emitted with attempt index `1`
|
||||
|
||||
#### Scenario: Non-retryable 5xx surfaces ServerTransientError
|
||||
|
||||
- **GIVEN** the server responds with HTTP `500` (not in the retryable set `{502, 503, 504}`)
|
||||
- **WHEN** the client awaits the call
|
||||
- **THEN** the client throws `ServerTransientError` (`SERVER.TRANSIENT`) without retrying
|
||||
- **AND** the caller is responsible for any further retry decision
|
||||
|
||||
### Requirement: Streaming responses are passed through unbuffered
|
||||
|
||||
The `fetchStream` API SHALL return `response.body` as `ReadableStream<Uint8Array>` directly, without buffering server-side bytes into memory.
|
||||
|
||||
#### Scenario: Stream body is consumed lazily
|
||||
|
||||
- **GIVEN** the server returns HTTP `200` with `Content-Type: audio/mpeg` and a chunked body
|
||||
- **WHEN** the caller awaits `client.fetchStream(req)` and pulls one chunk
|
||||
- **THEN** only that chunk has been consumed from the wire
|
||||
- **AND** the caller may pipe the stream into Web Audio, MediaSource, or buffer on demand
|
||||
|
||||
### Requirement: Multipart uploads accept all four file shapes
|
||||
|
||||
The `fetchMultipart` API SHALL accept `MultipartFile = Uint8Array | Blob | ReadableStream<Uint8Array> | string` (where `string` is interpreted as a URL).
|
||||
|
||||
#### Scenario: Uint8Array attaches without copying
|
||||
|
||||
- **GIVEN** an upload with `audio: new Uint8Array([0x01, 0x02, 0x03])`
|
||||
- **WHEN** the client builds the multipart body
|
||||
- **THEN** the resulting `FormData` part contains exactly those three bytes
|
||||
|
||||
#### Scenario: ReadableStream is buffered before upload
|
||||
|
||||
- **GIVEN** an upload with `audio` as a `ReadableStream` of two chunks
|
||||
- **WHEN** the client builds the multipart body
|
||||
- **THEN** the stream is fully drained via `@suaveplan/streams.toBytes` before sending
|
||||
- **AND** the resulting `FormData` part contains the concatenated bytes
|
||||
|
||||
#### Scenario: URL is fetched first then uploaded
|
||||
|
||||
- **GIVEN** an upload with `audio: 'https://example.com/sample.wav'`
|
||||
- **AND** that URL returns HTTP `200` with bytes `B`
|
||||
- **WHEN** the client builds the multipart body
|
||||
- **THEN** the URL is fetched first via the injected `fetchImpl`
|
||||
- **AND** the resulting `FormData` part contains exactly `B`
|
||||
|
||||
### Requirement: Rate-limit metadata is exposed on success
|
||||
|
||||
Every successful response from `fetchJson`/`fetchBinary`/`fetchStream`/`fetchMultipart` SHALL return both the parsed body and a `metadata` envelope containing `requestId`, `rateLimitRemaining` (when present), and `characterCost` (when present).
|
||||
|
||||
#### Scenario: Metadata is populated from response headers
|
||||
|
||||
- **GIVEN** the server returns `X-RateLimit-Remaining: 100` and `X-Character-Cost: 42`
|
||||
- **WHEN** the caller awaits `fetchJson`
|
||||
- **THEN** the resolved value is `{ body, metadata: { rateLimitRemaining: 100, characterCost: 42, requestId: <string|undefined> } }`
|
||||
|
||||
### Requirement: OpenTelemetry instrumentation per call
|
||||
|
||||
The client SHALL emit, for every public call:
|
||||
|
||||
- A span named `elevenlabs.<feature>.<operation>` with attributes `elevenlabs.endpoint`, `elevenlabs.method`, `http.response.status_code`, and `elevenlabs.model_id` (when present).
|
||||
- An increment of counter `third_party_elevenlabs.requests_total` tagged by `feature`, `operation`, and `status` (`ok`, `client_error`, `server_error`, `network_error`, `cancelled`).
|
||||
- A histogram observation on `third_party_elevenlabs.request_duration_ms` tagged by `feature` and `operation`.
|
||||
|
||||
#### Scenario: OTel span is emitted on success
|
||||
|
||||
- **GIVEN** a configured tracer
|
||||
- **WHEN** the caller awaits `client.fetchJson` and the call returns `200`
|
||||
- **THEN** exactly one span named `elevenlabs.<feature>.<operation>` is emitted with status `ok`
|
||||
- **AND** the counter `third_party_elevenlabs.requests_total{status=ok}` is incremented exactly once
|
||||
|
||||
### Requirement: AbortSignal is honoured end-to-end
|
||||
|
||||
Every public call SHALL accept an optional `AbortSignal` (via the request envelope) and merge it with the internal timeout signal. Aborting either signal SHALL cause the call to throw `NetworkAbortedError` or `NetworkTimeoutError` respectively.
|
||||
|
||||
#### Scenario: Caller AbortSignal cancels in-flight request
|
||||
|
||||
- **GIVEN** a caller-supplied `AbortController` and an in-flight `fetchJson` call
|
||||
- **WHEN** the caller invokes `controller.abort()`
|
||||
- **THEN** the call throws `NetworkAbortedError` (`NETWORK.ABORTED`)
|
||||
- **AND** no retry is attempted (abort is not a retryable code)
|
||||
- **AND** the OTel counter records `status=cancelled`
|
||||
|
||||
### Requirement: WebSocket transport surfaces typed protocol errors
|
||||
|
||||
`openSocket` SHALL return an `ElevenLabsSocket` that surfaces handshake failures as `WebSocketConnectionFailedError` and unexpected frames or abnormal close codes as `WebSocketProtocolError`.
|
||||
|
||||
#### Scenario: Handshake failure surfaces ConnectionFailedError
|
||||
|
||||
- **GIVEN** an injected `WebSocketImpl` that synthetically fails the handshake with code `4001`
|
||||
- **WHEN** the caller awaits `client.openSocket(req)`
|
||||
- **THEN** the call rejects with `WebSocketConnectionFailedError`
|
||||
- **AND** `error.context.code === 4001`
|
||||
|
||||
### Requirement: Wave 1 feature modules cover the voice surface
|
||||
|
||||
The package SHALL expose, in Wave 1, feature modules `tts`, `stt`, `voices`, `sound-generation`, `audio-isolation`, `speech-to-speech`, and `models`. Each module SHALL be subpath-exported per genesis convention and depend solely on `ElevenLabsClient` for transport.
|
||||
|
||||
#### Scenario: Voice surface modules are independently importable
|
||||
|
||||
- **GIVEN** a consumer that only needs TTS
|
||||
- **WHEN** they import `import { ElevenLabsTts } from '@suaveplan/third-party-elevenlabs/tts/tts'`
|
||||
- **THEN** the import succeeds without pulling in `stt`, `voices`, `sound-generation`, `audio-isolation`, `speech-to-speech`, or `models` modules into the bundle
|
||||
- **AND** tree-shaking removes the unused modules
|
||||
220
openspec/archive/2026-05-22-third-party-elevenlabs/tasks.md
Normal file
220
openspec/archive/2026-05-22-third-party-elevenlabs/tasks.md
Normal file
@@ -0,0 +1,220 @@
|
||||
# Tasks: third-party-elevenlabs
|
||||
|
||||
**Change ID**: third-party-elevenlabs
|
||||
**Package**: `@suaveplan/third-party-elevenlabs`
|
||||
**Target path**: `packages/audio/third-party-elevenlabs`
|
||||
**Wave**: 1 (voice surface)
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/streams`, `@suaveplan/retry`, `@suaveplan/logger`
|
||||
|
||||
> Wave 1 only. Subsequent waves (`tts-websocket`, `voice-cloning`, `voice-design`, `pronunciation-dictionaries`, `conversational-ai`, `music`, `dubbing`, `studio`, `audio-native`, `history`, `user`, `workspace`) ship under their own change IDs.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [x] Create `packages/audio/third-party-elevenlabs/` with `src/`, `tests/`, `src/__benchmarks__/`.
|
||||
- [x] Write `package.json` at version `0.1.0`, `"type": "module"`, `"sideEffects": false`, full subpath `exports` map per feature module, `dependencies` and `devDependencies` via catalogs only (no bare versions).
|
||||
- [x] Write `tsconfig.lib.json` extending `../../../tsconfig.base.json`.
|
||||
- [x] Write `vite.config.ts` with multi-entry build for every feature module.
|
||||
- [x] Write `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config` with `pool: 'forks'`, coverage thresholds 100/100/100/100, `provider: 'istanbul'`.
|
||||
- [x] Add `lint`, `lint:fix`, `format`, `check`, `typecheck`, `test`, `test:coverage`, `build`, `docs:check` scripts.
|
||||
- [x] Update `scripts/folder-reorg/assignment.yml` to include `audio/third-party-elevenlabs` and regenerate `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`.
|
||||
- [x] Update root `CLAUDE.md` package listing.
|
||||
|
||||
---
|
||||
|
||||
## 2. Errors
|
||||
|
||||
- [x] `src/errors/errors.ts` — define `ELEVENLABS_CODES` via `defineCodes('THIRD_PARTY_ELEVENLABS', [...])` covering 17 codes from proposal §Errors; export `ElevenLabsApiError` base + 17 typed subclasses (`UnauthorizedError`, `ForbiddenError`, `RequestInvalidError`, `RequestValidationFailedError`, `RequestNotFoundError`, `RateLimitExceededError`, `RateLimitQuotaExhaustedError`, `ServerTransientError`, `ServerUnavailableError`, `NetworkTimeoutError`, `NetworkAbortedError`, `NetworkUnreachableError`, `ResponseMalformedError`, `ResponseUnsupportedFormatError`, `StreamInterruptedError`, `WebSocketConnectionFailedError`, `WebSocketProtocolError`).
|
||||
- [x] `src/errors/errors.md` — full API reference with throw-site examples.
|
||||
- [x] `src/errors/errors.test.ts` — instantiation, code mapping, context preservation, JSON round-trip via `SuaveplanError.toJSON()`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Shared types
|
||||
|
||||
- [x] `src/types/types.ts` — `JsonRequest`, `BinaryRequest`, `StreamRequest`, `MultipartRequest`, `SocketRequest`, `BinaryResponse`, `RequestMetadata` (`requestId`, `rateLimitRemaining`, `characterCost`), `ElevenLabsClientEvents` (request-start, request-end, retry-attempt, rate-limit-hit), `RetryDecision`, `MultipartFile` (`Uint8Array | Blob | ReadableStream | string`).
|
||||
- [x] `src/types/types.md`
|
||||
- [x] `src/types/types.test.ts` — Zod schemas for runtime-checked subset.
|
||||
|
||||
---
|
||||
|
||||
## 4. Client
|
||||
|
||||
- [x] `src/client/client.ts` — `ElevenLabsClient` class.
|
||||
- Constructor: validate options (Zod), default base URL `https://api.elevenlabs.io`, default timeout `30_000`, default retry policy (exponential backoff with jitter, 3 attempts, retryable codes only).
|
||||
- `fetchJson(req, schema)` — JSON in, Zod-validated JSON out, returns `{ body, metadata }`.
|
||||
- `fetchBinary(req)` — JSON in, binary `Uint8Array` out + headers metadata.
|
||||
- `fetchStream(req)` — JSON or multipart in, returns `{ stream, metadata }` with `ReadableStream<Uint8Array>` body.
|
||||
- `fetchMultipart(req, schema)` — `FormData` upload (string URL → fetched first; `ReadableStream` → buffered via `@suaveplan/streams.toBytes`; `Uint8Array`/`Blob` → direct), Zod-validated JSON out.
|
||||
- `openSocket(req)` — `WebSocket` handshake with `xi-api-key` header, returns `ElevenLabsSocket`.
|
||||
- HTTP status mapping per proposal §Errors.
|
||||
- `Retry-After` parsing (seconds + HTTP-date).
|
||||
- `X-RateLimit-Remaining` + `X-Character-Cost` header surfacing in metadata.
|
||||
- OTel span / counter / histogram per call.
|
||||
- `AbortSignal` plumbed end-to-end (caller signal merged with internal timeout signal).
|
||||
- Circuit breaker integration (opt-in via options).
|
||||
- [x] `src/client/client.schemas.ts` — Zod for `ElevenLabsClientOptions`, request envelopes, metadata.
|
||||
- [x] `src/client/client.md`
|
||||
- [x] `src/client/client.test.ts` — unit tests with `fetchImpl` injection: 200 happy path, 401, 403, 404, 422, 429 with `Retry-After: 5`, 429 with HTTP-date, 500 retried then succeeds, 503 retried then fails, timeout, abort, `RESPONSE.MALFORMED` on Zod mismatch, multipart with `Uint8Array`, multipart with `ReadableStream`, multipart with URL fetched first, stream returns body unmodified, OTel span emitted, counter incremented, histogram recorded.
|
||||
|
||||
---
|
||||
|
||||
## 5. WebSocket transport
|
||||
|
||||
- [x] `src/socket/socket.ts` — `ElevenLabsSocket` over injected `WebSocketImpl`. Handles ping/pong, abnormal close → `WebSocketProtocolError`, handshake failure → `WebSocketConnectionFailedError`, send buffering, ready-state transitions.
|
||||
- [x] `src/socket/socket.schemas.ts` — generic `SocketFrame` discriminated-union schema (typed per-feature in feature modules).
|
||||
- [x] `src/socket/socket.md`
|
||||
- [x] `src/socket/socket.test.ts` — fake `WebSocket` implementation: connect/send/close/error paths, ping/pong, protocol error mapping.
|
||||
|
||||
---
|
||||
|
||||
## 6. Feature modules — Wave 1
|
||||
|
||||
Every feature module follows the same shape: class taking `ElevenLabsClient`, Zod schemas in `*.schemas.ts`, sibling `.md`, `*.test.ts` with mocked `fetchImpl`, OTel attributes per call.
|
||||
|
||||
### 6.1 `tts`
|
||||
|
||||
- [x] `src/tts/tts.ts` — `ElevenLabsTts` class.
|
||||
- `synthesize(voiceId, body)` → `BinaryResponse` via `POST /v1/text-to-speech/{voiceId}`
|
||||
- `synthesizeWithTimestamps(voiceId, body)` → `JsonResponse<TtsTimestampsBody>` via `POST /v1/text-to-speech/{voiceId}/with-timestamps`
|
||||
- `synthesizeStream(voiceId, body, options?)` → `StreamResponse` via `POST /v1/text-to-speech/{voiceId}/stream`
|
||||
- `synthesizeStreamWithTimestamps(voiceId, body, options?)` → `StreamResponse<JsonLine>` via `POST /v1/text-to-speech/{voiceId}/stream/with-timestamps`
|
||||
- [x] `src/tts/tts.schemas.ts` — full request schema (`text`, `model_id`, `language_code`, `voice_settings`, `previous_text`, `next_text`, `seed`, `pronunciation_dictionary_locators`, `apply_text_normalization`, `apply_language_text_normalization`, `output_format`, `optimize_streaming_latency`).
|
||||
- [x] `src/tts/tts.md`
|
||||
- [x] `src/tts/tts.test.ts`
|
||||
|
||||
### 6.2 `stt`
|
||||
|
||||
- [x] `src/stt/stt.ts` — `ElevenLabsStt` class.
|
||||
- `transcribe(body)` → `JsonResponse<SttTranscript>` via `POST /v1/speech-to-text` (multipart).
|
||||
- [x] `src/stt/stt.schemas.ts` — `model_id`, `language_code`, `tag_audio_events`, `num_speakers`, `timestamps_granularity`, `diarize`, `diarization_threshold`, `additional_formats`, `file`, `cloud_storage_url`, `webhook` request shape; `language_code`, `language_probability`, `text`, `words[]` (with `start`, `end`, `speaker_id`, `type`), `additional_formats[]` response.
|
||||
- [x] `src/stt/stt.md`
|
||||
- [x] `src/stt/stt.test.ts`
|
||||
|
||||
### 6.3 `voices`
|
||||
|
||||
- [x] `src/voices/voices.ts` — `ElevenLabsVoices` class.
|
||||
- `list(params)` — `GET /v1/voices` with `show_legacy`, `category`, `voice_type`, `search`, `next_page_token`, `page_size`
|
||||
- `listShared(params)` — `GET /v1/shared-voices`
|
||||
- `get(voiceId, params?)` — `GET /v1/voices/{id}`
|
||||
- `getSettings(voiceId)` — `GET /v1/voices/{id}/settings`
|
||||
- `editSettings(voiceId, body)` — `POST /v1/voices/{id}/settings/edit`
|
||||
- `getDefaultSettings()` — `GET /v1/voices/settings/default`
|
||||
- [x] `src/voices/voices.schemas.ts`
|
||||
- [x] `src/voices/voices.md`
|
||||
- [x] `src/voices/voices.test.ts`
|
||||
|
||||
### 6.4 `sound-generation`
|
||||
|
||||
- [x] `src/sound-generation/sound-generation.ts` — `ElevenLabsSoundGeneration` class.
|
||||
- `generate(body)` — `POST /v1/sound-generation` returning binary audio.
|
||||
- [x] `src/sound-generation/sound-generation.schemas.ts` — `text`, `duration_seconds` (0.5–30), `loop`, `prompt_influence` (0–1), `model_id`, `output_format`.
|
||||
- [x] `src/sound-generation/sound-generation.md`
|
||||
- [x] `src/sound-generation/sound-generation.test.ts`
|
||||
|
||||
### 6.5 `audio-isolation`
|
||||
|
||||
- [x] `src/audio-isolation/audio-isolation.ts` — `ElevenLabsAudioIsolation` class.
|
||||
- `isolate(body)` — `POST /v1/audio-isolation` (multipart) returning binary audio.
|
||||
- `isolateStream(body)` — `POST /v1/audio-isolation/stream` returning stream.
|
||||
- [x] `src/audio-isolation/audio-isolation.schemas.ts` — `audio` (multipart file), `file_format`.
|
||||
- [x] `src/audio-isolation/audio-isolation.md`
|
||||
- [x] `src/audio-isolation/audio-isolation.test.ts`
|
||||
|
||||
### 6.6 `speech-to-speech`
|
||||
|
||||
- [x] `src/speech-to-speech/speech-to-speech.ts` — `ElevenLabsSpeechToSpeech` class.
|
||||
- `convert(voiceId, body)` — `POST /v1/speech-to-speech/{voiceId}` (multipart) returning binary audio.
|
||||
- `convertStream(voiceId, body)` — `POST /v1/speech-to-speech/{voiceId}/stream` returning stream.
|
||||
- [x] `src/speech-to-speech/speech-to-speech.schemas.ts` — `audio`, `model_id`, `voice_settings`, `seed`, `remove_background_noise`, `output_format`.
|
||||
- [x] `src/speech-to-speech/speech-to-speech.md`
|
||||
- [x] `src/speech-to-speech/speech-to-speech.test.ts`
|
||||
|
||||
### 6.7 `models`
|
||||
|
||||
- [x] `src/models/models.ts` — `ElevenLabsModels` class.
|
||||
- `list()` — `GET /v1/models` returning model registry.
|
||||
- [x] `src/models/models.schemas.ts` — `model_id`, `name`, `can_be_finetuned`, `can_do_text_to_speech`, `can_do_voice_conversion`, `can_use_style`, `can_use_speaker_boost`, `serves_pro_voices`, `token_cost_factor`, `description`, `requires_alpha_access`, `max_characters_request_free_user`, `max_characters_request_subscribed_user`, `maximum_text_length_per_request`, `languages[]`, `model_rates`.
|
||||
- [x] `src/models/models.md`
|
||||
- [x] `src/models/models.test.ts`
|
||||
|
||||
---
|
||||
|
||||
## 7. Index + composite factory
|
||||
|
||||
- [x] `src/index.ts` — barrel re-export of `ElevenLabsClient` + every feature class + every error subclass + every TS type.
|
||||
- [x] `src/factory/factory.ts` — `createElevenLabsApi(options)` returning `{ client, tts, stt, voices, soundGeneration, audioIsolation, speechToSpeech, models }`.
|
||||
- [x] `src/factory/factory.md`
|
||||
- [x] `src/factory/factory.test.ts` — smoke test that factory returns working instances.
|
||||
|
||||
---
|
||||
|
||||
## 8. Property-based tests (`tests/property.test.ts`)
|
||||
|
||||
- [x] `Retry-After` parser: arbitrary integer seconds → `retryAfterMs = seconds * 1000`.
|
||||
- [x] `Retry-After` parser: HTTP-date (RFC 7231) → positive `retryAfterMs` relative to `clock.now()`.
|
||||
- [x] Multipart builder: any `MultipartFile` shape produces a `FormData` whose blob bytes equal the input bytes.
|
||||
- [x] HTTP status → error code mapping is total and disjoint over `[400, 600)`.
|
||||
- [x] `OutputFormat` round-trip: every documented format string serialises and parses to itself via Zod.
|
||||
|
||||
---
|
||||
|
||||
## 9. Benchmarks (`src/__benchmarks__/third-party-elevenlabs.bench.ts`)
|
||||
|
||||
- [x] `client.fetchJson` mock-server round-trip.
|
||||
- [x] `client.fetchStream` mock-server round-trip (1 MB body).
|
||||
- [x] `tts.synthesize` mock-server round-trip.
|
||||
|
||||
---
|
||||
|
||||
## 10. Spec deltas
|
||||
|
||||
- [x] Verify `openspec/changes/third-party-elevenlabs/specs/third-party-elevenlabs/spec.md` covers every Wave 1 requirement from the proposal.
|
||||
- [x] Confirm 100% coverage thresholds enforced in `vitest.config.ts`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Pre-publish gate
|
||||
|
||||
- [x] `bunx turbo lint typecheck test --filter=@suaveplan/third-party-elevenlabs` — green
|
||||
- [x] `bunx turbo test:coverage --filter=@suaveplan/third-party-elevenlabs` — 100/100/100/100
|
||||
- [x] `bun run docs:check --filter=@suaveplan/third-party-elevenlabs` — ≥ 90%
|
||||
- [x] Banned-pattern scan (`@deprecated`, `TODO`, `FIXME`, `ts-ignore`, `ts-expect-error`, `.skip`, `.only`, `\bany\b`, `console.log`, `biome-ignore`) — zero hits in `src/`.
|
||||
- [x] `README.md` exists at package root with install + quick-start + Modules section.
|
||||
- [x] Subpath exports count matches feature module count.
|
||||
- [x] No bare versions in `package.json`.
|
||||
- [x] `bunx turbo test --filter=...@suaveplan/third-party-elevenlabs` — dependents pass (only `voice-elevenlabs` after its rebuild).
|
||||
|
||||
---
|
||||
|
||||
## 12. Archive
|
||||
|
||||
- [x] Fold spec deltas into `openspec/specs/third-party-elevenlabs/spec.md`.
|
||||
- [x] Move `openspec/changes/third-party-elevenlabs/` → `openspec/archive/2026-05-02-third-party-elevenlabs/`.
|
||||
- [x] Publish to Verdaccio: `bun scripts/publish.ts --package @suaveplan/third-party-elevenlabs`.
|
||||
- [x] Update `STATUS.md`.
|
||||
|
||||
---
|
||||
|
||||
## Audit Note (2026-05-02)
|
||||
|
||||
**Status**: DEFERRED.
|
||||
|
||||
State at audit:
|
||||
|
||||
- Package scaffold exists at `packages/audio/third-party-elevenlabs/` with all 11 feature directories (`tts`, `stt`, `voices`, `sound-generation`, `audio-isolation`, `speech-to-speech`, `models`, `client`, `errors`, `types`, `factory`).
|
||||
- **Typecheck fixed in this audit**: `src/errors/errors.ts` switched 15 `ctx.<prop> = ...` writes (banned by TS4111 under index-signature `Record<string, unknown>`) to bracket access (`ctx["<prop>"] = ...`); `src/types/types.ts` `ElevenLabsClientEvents` gained a `readonly [event: string]: unknown` index signature so it satisfies the `EventBusLike<TMap extends Record<string, unknown>>` constraint. Typecheck is now green.
|
||||
- **Build fails**: there is no `src/index.ts` barrel — `vite build` errors with `Could not resolve entry module "src/index.ts"`. Task 7 ("Index + composite factory") is unchecked.
|
||||
- No `tests/property.test.ts` (Section 8) and no `src/__benchmarks__/` (Section 9).
|
||||
- Package `README.md` not present at root (Section 11).
|
||||
- Spec deltas live under `specs/third-party-elevenlabs/` but have not been folded.
|
||||
|
||||
Resuming work:
|
||||
|
||||
1. Author `src/index.ts` re-exporting `ElevenLabsClient`, every feature class, every error subclass, every TS type.
|
||||
2. Implement `src/factory/factory.ts` (exists as folder) and `createElevenLabsApi(options)`.
|
||||
3. Add `tests/property.test.ts` covering Retry-After parsing, multipart builder, status→code mapping totality, OutputFormat round-trip.
|
||||
4. Add `src/__benchmarks__/third-party-elevenlabs.bench.ts` (mock-server fetchJson / fetchStream / tts.synthesize).
|
||||
5. Author root `README.md`.
|
||||
6. Confirm `bunx turbo lint typecheck test:coverage --filter=@suaveplan/third-party-elevenlabs` is 100/100/100/100, fold deltas, archive.
|
||||
Reference in New Issue
Block a user