# Design — enterprise-react-audio-stream

`@suaveplan/react-audio-stream` provides real-time audio streaming over the unified architecture. Its core insight: an incoming peer audio stream is just another **source plugin** on a dedicated chain, indistinguishable to the rest of the system from a local `AudioPlayerPlugin`. That uniformity gets us FX chains per peer, sidechain ducking from peers, analyser readouts per peer, and snapshot/restore of "studio sessions" for free — without the streaming code having to know about any of it.

---

## 1. Position in the audio family

```
        consumer React app
                │
                ▼
        react-audio-stream         ← this package
          (StreamSession + per-peer source plugin + transports)
                │
                ▼
        react-audio                (provider + hooks)
                │
                ▼
        audio-engine-browser       (TimelineAdapter + chains)
```

Per-peer sources are inserted on dedicated tracks in the engine's timeline. Track ids follow the convention `peer:<participantId>`. The chain owns FX; the source plugin owns playback of the incoming `MediaStream`.

---

## 2. `StreamSession` — transport-agnostic core

```ts
class StreamSession {
  constructor(opts: StreamSessionOptions)

  connect(roomId: string): Promise<void>
  join(): Promise<void>
  leave(): Promise<void>
  disconnect(): Promise<void>

  readonly state: SessionState                // FSM via @suaveplan/fsm
  readonly peers: ReadonlyMap<string, PeerChannel>

  on(event: SessionEvent, listener): UnsubscribeFn
  // 'state-changed' | 'peer-joined' | 'peer-left' | 'peer-stream-added'
  // | 'peer-stream-removed' | 'transport-share-request' | 'transport-share-granted'
  // | 'interruption' | 'reconnect-attempt' | 'reconnect-failed'
}
```

The session does not speak any specific transport. It delegates negotiation, ICE, signalling, and stream attachment to an injected `StreamTransport` adapter (`WebRTCTransport`, `WebSocketTransport`, `TrysteroTransport`). The same session can run identically against any transport.

---

## 3. Per-peer source plugin

When a peer adds a stream, the session emits `'peer-stream-added'` with `{ participantId, stream: MediaStream }`. The provider listens, instantiates a `StreamSourcePlugin` (an `AudioFxPlugin` source — `io.inputs[0].count === 0`, captures from a `MediaStream`), inserts it into the chain of a new `peer:<participantId>` track, and exposes it via `usePeerStreams()`.

```ts
class StreamSourcePlugin implements AudioFxPlugin {
  readonly io = { inputs: [{ count: 0 }], outputs: [{ count: 2, layout: 'stereo' }] }
  // browser path: createAudioNode wraps MediaStreamSourceNode
  // server path: not applicable (server doesn't have MediaStream)
  setSourceStream(stream: MediaStream): void
  process(_in, output, ctx) {
    // browser worklet path mirrors AudioNode path's behaviour
  }
}
```

The plugin is just like every other source plugin — `<PluginChainView>` renders it generically, `useParameter` writes its volume, the chain can have FX after it, the bus registry can read its post-FX output.

---

## 4. FSM session lifecycle

`@suaveplan/fsm` drives the session through:

```
idle → connecting → connected (idle in room) → joining → joined (streaming)
                                  │                  │
                                  └─→ disconnecting ─┴─→ disconnected
                                                          ↑
                                          (reconnect attempt brings us back to connecting)
```

Transitions:

- `connect(roomId)` from `idle` → `connecting` → `connected`
- `join()` from `connected` → `joining` → `joined`
- `leave()` from `joined` → `connected`
- `disconnect()` from any → `disconnecting` → `disconnected`
- Network failure from `connected`/`joined` → `disconnected` (auto-reconnect triggers)

Illegal transitions throw `AudioError` with code `REACT_AUDIO_STREAM.INVALID_STATE`.

---

## 5. Auto-reconnect

Backed by `@suaveplan/retry` (exponential back-off with jitter, capped at 30 s). On disconnect, the session classifies the interruption (see §6), and if the class is `network` or `signalling` it auto-reconnects unless `autoReconnect: false`. `media-permission` and explicit-disconnect do not auto-reconnect. Each attempt emits `'reconnect-attempt'`; success emits `'state-changed'`; final failure emits `'reconnect-failed'`.

---

## 6. Interruption classification

When a stream drops, classify to one of:

- `network` — ICE failure, DNS fail, no route.
- `media-permission` — `getUserMedia` revoked, OS audio device removed.
- `signalling` — signalling-server disconnect.
- `peer` — peer left explicitly, or peer's media track ended.
- `unknown` — anything else.

The classifier reads the underlying error / event and applies a rule set. Classification feeds both the reconnect decision and the UI toast.

---

## 7. Transport-share mode

In a collaborative listening session a designated peer drives transport state for everyone:

- Designation happens via `session.requestTransportShare(peerId)` from any client; the target peer accepts with `session.grantTransportShare()`.
- Once granted, the local transport's `play`/`pause`/`seek`/`setTempo` events are mirrored to a control channel on each peer's `PeerChannel`.
- Each non-driving peer's local transport receives the events and applies them; their UI controls become advisory (no-op `play`/`pause` calls).
- A driver disconnect transfers control back to "local"; UI surfaces this via toast.

---

## 8. WebSocket transport (server-relay fallback)

WebRTC is the default, but corporate networks and embedded contexts often need WebSocket-relayed audio. The `WebSocketTransport` packs raw PCM frames (or opus-encoded frames if `@suaveplan/audio-codec-ffmpeg-wasm` is available) and ships them through the `network-transport-websockets`. Latency is higher than WebRTC but it works through NATs that block UDP.

The transport is interchangeable; the session and source plugin do not care which.

---

## 9. Decisions and trade-offs

- **Per-peer source plugin = uniform consumer experience.** No special-case streaming code in `<PluginChainView>`, `useParameter`, `usePreset`, etc.
- **Transport-agnostic session.** Same code drives WebRTC, WebSocket, or Trystero.
- **FSM session lifecycle.** Avoids race conditions from imperative `if (state === 'X') …` checks; the FSM enforces legal transitions.
- **Interruption classification before reconnect decision.** Re-permission prompts are user-driven; we don't endlessly retry through them.
- **Transport-share via control channel, not engine state.** The driving peer's `Transport.on('play')` listener fires a control-channel message; receivers apply it locally. No special "shared transport" type — it's the local transport, just driven remotely.