# Drive · Storage · Network — How They Compose

This is the integration document. It exists to answer one question: *given storage, drive, and network as three separate libraries, what does a complete deployment look like and how does a single byte travel through it?*

If you want depth on any one layer, read its folder:

- [`docs/storage/`](./storage/) — backend layer.
- [`docs/drive/`](./drive/) — file layer.
- [`docs/network/`](./network/) — mesh + failover layer.
- [`docs/system-review.md`](./system-review.md) — 16 numbered findings against the design (2 blockers, 9 majors, 5 minors) — read this before implementing.
- [`docs/drive/mesh-protocol.md`](./drive/mesh-protocol.md) — the chunk-request wire protocol (three forms, bitpacker bitsets, response piggyback).
- [`docs/drive/dehydrated-drives.md`](./drive/dehydrated-drives.md) — server-signed templates that pre-configure drives with policy flags + CDN reversed-ID URL fallback + retroactive key grants.

This document is the map.

---

## The three layers, one paragraph

**Storage** is `key → bytes`. It knows nothing else.
**Drive** is `path → file`, with plugins for chunking, signing, encryption, sharing, and mesh sync. It uses storage as its substrate.
**Network** is the resilient tiered mesh that lets every drive talk to every other drive. Drives plug into network via the `drive-mesh` and `drive-transport-*` plugins, communicating only through `NetworkBus` events.

The three layers compose strictly: drive depends on storage and network; network depends on neither; storage depends on neither. There is exactly one bridging type package — `@suaveplan/types` — that holds the cross-cutting interfaces (`StorageLike`, `ChunkSource`, `ChunkRef`, `DrivePlugin`, `MiddlewareHandler`, `HealthStatus`) so no concrete library has to import another concrete library.

---

## The dependency graph (concrete)

```text
@suaveplan/types  ◄──────────────── (everything depends on this; types has zero runtime code)
   ▲           ▲
   │           │
@suaveplan/storage           @suaveplan/network
   ▲              ▲              ▲          ▲
   │              │              │          │
@suaveplan/storage-s3        @suaveplan/network-plugin-protocol
@suaveplan/storage-browser   @suaveplan/network-plugin-registry
@suaveplan/storage-...       @suaveplan/network-transport-webrtc
                             @suaveplan/network-plugin-...
                                            ▲
                                            │
   ┌────────────────────────────────────────┘
   │
@suaveplan/drive  ──────► uses storage as substrate
   ▲
   │
@suaveplan/drive-plugin-chunks   ──► uses crypto, packer, bitpacker
@suaveplan/drive-sharing         ──► uses crypto, drive
@suaveplan/drive-mesh            ──► uses drive, network (via bus only)
@suaveplan/drive-transport-https ──► uses drive
@suaveplan/drive-transport-webrtc ─► uses drive, network-transport-webrtc (via bus only)
@suaveplan/drive-react-sharing   ──► uses drive-sharing
@suaveplan/drive-webrtc          ──► pre-packaged: chunks + transport-webrtc + signalling
```

The "via bus only" qualifier is structural: `drive-mesh` doesn't import `network-plugin-content-delivery`. They communicate through bus topics. This is the genesis discipline that broke the circular-deps mess on main.

---

## The file write — end to end

A user clicks "save" on a 50 MiB photo. The drive is configured with `OpfsStorage` locally, `S3` for cold storage, the chunks plugin, the sharing plugin, the mesh plugin, and a WebRTC transport. Every numbered step is a real boundary in the code.

```text
1. App  →  drive.write("/photos/wedding/IMG_0001.jpg", bytes)

2. Drive middleware chain (request side, in registration order):
   2a. logging  → record { op:'write', path, size }
   2b. chunks   → start a TransformStream:
                     bytes ──┐
                             ├─ chop into 65,408-byte plaintext chunks
                             │  (chunkSize 65,536 minus 164-byte chunk header+sig)
                             └─ for each:
                                  nonce  = HKDF(fileKey, fileId ‖ chunkId)
                                  cipher = XChaCha20-Poly1305(fileKey, nonce, plaintext, AAD=metadata)
                                  sig    = Ed25519-sign(driveSecretKey, metadata ‖ aeadTag)
                                  emit chunk = metadata(100) ‖ sig(64) ‖ ciphertext
   2c. sharing  → wraps fileKey with the partition KEK; appends wrap blob to manifest
   2d. mesh     → no-op on write, but listens for the post-write hook
   2e. transport-webrtc → no-op on write

3. Storage selection per chunk:
   3a. storage.capabilities.partialWrites === true (OPFS) ? → sparse-file layout
   3b. storage.setPartial("files/<fileId>", chunkId * chunkSize, chunk)
   3c. for cold copy in S3: storage.set("files/<fileId>/<chunkId>", chunk)
       (S3 has partialWrites=false; the chunks plugin auto-picks chunk-keys layout for it)

4. Manifest assembly:
   4a. manifest = { fileId, totalChunks=800, version=1, entries: [
                     { chunkId:0, hash:blake3(chunk0), payloadLen },
                     ... 799 more ...
                   ]}
   4b. manifestSig = Ed25519-sign(driveSecretKey, canonicalSerialize(manifest))
   4c. storage.set("manifests/<fileId>-v1", encodeManifest(manifest, manifestSig))

5. Drive index update (CAS-versioned):
   5a. read DirIndex; insert DirEntry { name:"IMG_0001.jpg", type:"file", fileLocId:"L-..." }
       under /photos/wedding/. Increment version.
   5b. read FileLocationIndex; insert FileLocation {
            fileId, version:1, size:52,428,800, checksum:"blake3:...",
            storageHints: [
              { kind:"sparse-file", storageId:"opfs", key:"files/<fileId>", chunkSize:65536 },
              { kind:"chunk-keys",  storageId:"s3",   manifestKey:"manifests/<fileId>-v1" },
              { kind:"mesh",        manifestKey:"manifests/<fileId>-v1" },
            ]
          }
   5c. CAS-write both indexes; on conflict, retry with rebased version.

6. Drive middleware chain (response side, reverse order):
   6e. transport-webrtc → no-op
   6d. mesh             → emit `cd:advertise { fileId, chunkBitset }` on NetworkBus
                          tells peers we now have all 800 chunks of this file
   6c. sharing          → strip wrap blob from response (already stored)
   6b. chunks           → return manifest summary
   6a. logging          → record { op:'write', path, size, durationMs }

7. Network propagation (asynchronous, fire-and-forget):
   7a. network-plugin-content-delivery hears `cd:advertise` and updates its gossip index
   7b. network-plugin-content-delivery may push `cd:advertise` to siblings on the relay
       backbone, depending on QoS class
   7c. peer drives subscribed to this drive's advertisements update their own
       per-fileId chunk bitsets

→ drive.write resolves. The user's UI updates "saving..." → "saved".
```

Every byte that hit storage, every byte that hit the network, was AEAD-encrypted ciphertext. The plaintext lived only in step 2b's TransformStream and never on disk or wire.

---

## The file read — end to end (with one chunk missing locally)

A second user (Bob's drive) wants to read the photo. Bob has the file's location entry but is missing chunks 200–250 locally (e.g., he just synced this drive partially).

```text
1. App → drive.read("/photos/wedding/IMG_0001.jpg")

2. Drive index resolve:
   2a. walk DirIndex from root → DirEntry { fileLocId:"L-..." }
   2b. lookup FileLocation by fileLocId → 800 chunks, manifest in OPFS, mesh-known

3. Drive middleware (request side):
   3a. logging  → start span { op:'read' }
   3b. chunks   → load manifest, parse 800 entries
   3c. sharing  → unwrap fileKey via partition KEK in our KeyRing
   3d. mesh     → for each chunkId in 0..799, check if available locally
                  result: 0..199 ✓, 200..250 ✗, 251..799 ✓
   3e. transport-webrtc → ensure peer connections to known content holders are live

4. Per-chunk fetch loop:
   For chunkIds 0..199 and 251..799:
     - storage.getPartial("files/<fileId>", chunkId * chunkSize, +chunkSize)
       returns the 65,536-byte chunk (header + sig + ciphertext)

   For chunkIds 200..250 (missing locally):
     - drive-mesh emits `cd:request { fileId, chunkId, requesterId }` per chunk
     - network-plugin-content-delivery picks top-N candidate peers from gossip index
     - races them via networkNode.send() → drive-transport-webrtc on each peer
     - peer receives `cd:fetch`, runs their drive's storage.getPartial(...),
       sends back `cd:response { fileId, chunkId, bytes }`
     - first verifying response wins; lagging responses dropped
     - emits `cd:response` on requester's bus
     - drive-mesh takes the bytes, hands to chunks plugin

5. Per-chunk verify+decrypt (chunks plugin):
   5a. parse 164-byte header → metadata, sig, AAD
   5b. Ed25519-verify(driveId.pubkey, sig, metadata ‖ aeadTag)
       → if fail: drop chunk, blacklist source peer (set score to -Infinity), retry
   5c. XChaCha20-Poly1305-decrypt(fileKey, nonce, ciphertext, AAD=metadata)
       → if AEAD tag fails: drop chunk, retry from a different source

6. Reassemble:
   6a. concat all 800 plaintext chunks in chunkId order
   6b. blake3(concatenated) === manifest.checksum ?
       → if not: hard error, manifest is wrong or chunks are corrupted

7. Drive middleware (response side):
   7e. transport-webrtc → no-op
   7d. mesh             → emit `cd:advertise` (we now have chunks 200..250)
                          peers learn we are a new source
   7c. sharing          → no-op on read
   7b. chunks           → return Uint8Array(52,428,800)
   7a. logging          → record { op:'read', durationMs, peerFetches:51 }

→ drive.read resolves with the full photo. Bob's drive is now a holder for chunks
  200..250; the next time another peer needs them, they may fetch from Bob.
```

The user gets the photo. Three peers cooperated to deliver it. Nobody had the whole file from the source; each peer served the chunks they had cached.

---

## What happens when the central server vanishes mid-stream

A real scenario: 3 minutes into Bob's read, the central server drops off. Result:

- Storage layer: untouched. Bob's local `OpfsStorage` reads and the WebRTC chunk fetches don't go through the server anyway.
- Drive layer: untouched. Every operation continues — every chunk verify uses the writer's per-peer Ed25519 pubkey, which is in Bob's local registry.
- Network layer:
  - `network-plugin-server-bridge` fires `server:disconnected` on the bus.
  - `network-plugin-failover` keeps running heartbeats among peers; mesh continues.
  - `network-plugin-governance` won't trigger a server-replacement election in `SERVER_BASED` mode (mesh stops); in `SERVER_TO_CLIENT_FAILOVER` mode it elects an acting Tier 0 from Tier 1 within seconds.
  - Bob's read completes normally — the chunks are coming from peer drives, not the server.
  - Any *new* file Bob writes is written locally (signed by Bob's drive identity), advertised to the mesh, and queued in `network-plugin-server-bridge`'s buffer for replay when the server returns.
  - Token grace periods extend; Bob's `CapabilityToken` doesn't expire mid-outage.
  - On server reconnect: buffered audit-log entries replay, the acting Tier 0 demotes, the mesh reconciles. Bob's new file becomes server-known.

This is the "if the server goes down, it's all in place with meshed pockets of users with their nominated hosts" part of the user's brief. It works because:

1. Every byte is signed by its writer; no server is needed to verify.
2. Every chunk is content-addressed; no server is needed to discover them.
3. Topology, election, partition handling, and circuit breaking are all peer-side.
4. The server is a coordinator and an authority for *new-resource issuance*, not a relay for the data plane.

---

## How n drives find each other

A drive's identity is its Ed25519 public key — its `driveId`. Two drives find each other through one of three routes:

1. **Direct address** — you give one drive the address of another (hostname/port/peer-id). They handshake and join.
2. **Discovery plugin** — `network-discovery-trystero` (or libp2p / GunDB / WebTorrent / hybrid) gives each drive a way to find peers without knowing them in advance. The plugin emits `peer:discovered`; `network-plugin-registry` records them; `drive-mesh` starts gossiping.
3. **Server-mediated** — a deployment with a central server can have the server return peer lists on join. This is the cheap default for `SERVER_TO_CLIENT_FAILOVER` mode.

Once two drives know about each other, they handshake (Ed25519 challenge-response, NET-30), exchange capability bitmaps, register each other in `network-plugin-registry`, and begin announcing chunk inventories on the bus.

---

## What lives where — quick reference

| Question | Layer |
| --- | --- |
| What bytes are at this key? | Storage |
| What backend is this storage? | Storage |
| Can this backend do partial writes? | Storage (`capabilities`) |
| What's in /photos/2026/? | Drive (DirIndex walk) |
| Where are the chunks of file X? | Drive (FileLocation `storageHints`) |
| Encrypt/sign these bytes | Drive (chunks plugin) |
| Verify a peer's signature on a chunk | Drive (chunks plugin), peer pubkey from network registry |
| Who can read this file? | Drive (sharing plugin partition) |
| Who's online right now? | Network (registry plugin) |
| Where can I get chunk Y? | Drive (mesh plugin requests; network content-delivery responds) |
| Is the server up? | Network (server-bridge plugin) |
| Who's the host of this mesh? | Network (governance + topology plugins) |
| Has there been a partition? | Network (partition plugin) |
| Can mesh A talk to mesh B? | Network (auth plugin trust validator + relay plugin) |

---

## The migration timeline (executive summary)

The genesis branch is mid-port from `main`. The shipped/pending split (full detail in each layer's `implementation-status.md`):

| Layer | Shipped on genesis | Most-pressing pending |
| --- | --- | --- |
| Storage | `storage`, `storage-s3` | `storage-browser-opfs`, `storage-browser`, `storage-file` (port from main) |
| Drive | `drive`, `drive-plugin-chunks` | `drive-mesh`, `drive-transport-https`, `drive-transport-webrtc` (write fresh on genesis); `drive-sharing`, `drive-react-sharing`, `drive-webrtc` (port from main) |
| Network | `network` (core), `network-plugin-protocol`, `network-plugin-registry`, `network-transport-webrtc`, `network-failover`, `network-governance`, `network-topology`, `network-transport-websockets`, `network-testing` | 14 of the 21 `network-plugin-*` packages (write fresh on genesis); discovery + libp2p + hyperswarm transports (port from main) |

The blocking spec change is `enterprise-types-cross-cutting` — it adds the canonical interfaces (`ChunkSource`, `ChunkRef`, `DrivePlugin`, `MiddlewareHandler`) to `@suaveplan/types`. Already implemented; awaits archive. Until it's archived, downstream packages keep redeclaring local copies of these types, which is the cause of the "every AI has its own ideas" feeling. **Land it first.**

The blocking architectural decision is `enterprise-drive` (monolith) vs `drive-port` (split). The migrated code follows `drive-port`; archive `enterprise-drive` and re-propose its CoW + snapshot ideas as their own plugins.

Both decisions are zero-code: open the change folders, write archive notes, fold the deltas into the canonical specs. Once that's done, every downstream package gets unblocked.

---

## Checklists for common deployments

### Single-server file storage (e.g., a SaaS backend)

```ts
const drive = new Drive(new S3({ ... }))
  .usePlugin(chunksPlugin({ chunkSize: 65536, signing: { secretKey } }))
  .usePlugin(sharingPlugin({ ... }));
```

No network plugin. Storage is one S3 bucket. Encrypted-at-rest, signed by the server's drive identity. Used as the persistent layer for an API server or the receiving end of a `drive-transport-https` upload.

### Browser drive with offline-first, cloud sync

```ts
const drive = new Drive()
  .use("local",  new OpfsStorage())
  .use("remote", new S3({ ... }))
  .usePlugin(chunksPlugin({ ... }))
  .usePlugin(sharingPlugin({ ... }))
  .usePlugin(meshPlugin({ networkNode }))
  .usePlugin(transportWebrtcPlugin({ signaller }))
  .usePlugin(transportHttpsPlugin({ baseUrl: "https://api.example.com" }));

const networkNode = new NetworkNode()
  .use(protocolPlugin())
  .use(registryPlugin())
  .use(webrtcPlugin({ ... }))
  .use(governancePlugin())
  .use(topologyPlugin())
  .use(serverBridgePlugin({ url: "wss://api.example.com/mesh", model: "SERVER_TO_CLIENT_FAILOVER" }))
  .use(authPlugin())
  .use(cryptoMessagesPlugin());
```

Browser drive that works offline, syncs to peers when online, falls back to S3 via HTTPS, and survives server outages.

### Pure-peer mesh (no server)

```ts
const networkNode = new NetworkNode()
  .use(protocolPlugin())
  .use(registryPlugin())
  .use(webrtcPlugin({ ... }))
  .use(discoveryTrysteroPlugin({ ... }))   // peer discovery
  .use(governancePlugin())                  // peer-elects host
  .use(topologyPlugin())
  .use(authPlugin({ mode: 'PURE_CLIENT' })) // no server tokens
  .use(cryptoMessagesPlugin());

const drive = new Drive(new OpfsStorage())
  .usePlugin(chunksPlugin({ ... }))
  .usePlugin(meshPlugin({ networkNode }))
  .usePlugin(transportWebrtcPlugin({ ... }));
```

Two phones in the same room. Trystero-discovery finds each other (BitTorrent / Nostr / MQTT). Governance elects a host. Drives share files end-to-end-encrypted. No server anywhere.

### Cross-mesh corporate federation

Add `relayPlugin()` and `crossMeshTrustValidator()` to each mesh's `NetworkNode`, exchange `MeshTrustToken`s out of band, and the `HostBackboneTransport` between Tier 0 hosts of each mesh lets envelopes cross. A user in mesh A can read a file from a drive in mesh B; the chunks travel across the relay backbone, unwrapped only by the authorised peers.

---

## What this design buys you

| Need | Mechanism |
| --- | --- |
| Storage anywhere (memory / fs / db / browser / S3 / FAT image) | Storage backend plugins; capability matrix routes around limitations. |
| Encrypt / sign / decrypt / verify on the fly | Drive chunks plugin: streaming AEAD + Ed25519 + BLAKE3. |
| Files navigable by path | Drive directory and file-location indexes (bitpacker-encoded). |
| Files shareable between drives, with revocation | Drive sharing plugin: partition KEKs, X25519 sealed-box wrapping. |
| n drives that find each other | Discovery plugins + registry plugin. |
| Peer-to-peer chunk transfer | drive-mesh + network-plugin-content-delivery + drive-transport-{webrtc,https}. |
| Resilient tiered network | Tier 0/1/2 topology + governance + failover + relay. |
| Server-down survival | server-bridge offline mode; signed-log buffering; replay on reconnect. |
| Cross-mesh interop | Relay plugin + auth plugin trust validators. |
| Verifiable audit trail | Audit-mesh signed log chains + replication + compaction. |
| Multi-tenant isolation | NamespacedStorage + sharing partitions + per-file fileKey. |
| End-to-end encryption with peer relay | Peers serve ciphertext bytes they cannot themselves read. |
| Test-driven development | testing-integration fixtures, MemoryTransport, FakeClock, no shared state. |

This is what "enterprise-quality" looks like in this design: every concern owned by exactly one plugin, every cross-plugin link mediated by a typed event, every byte ciphertext-signed-on-disk-and-on-wire, every operation backed by a canonical OpenSpec capability requirement.

---

## How to keep the water clear

The user's complaint was: *"I feel because of how complex it is, and multiple AI having their own ideas, it's muddied the water."* Three rules that prevent that going forward:

1. **Specs are canonical, proposals are drafts.** [`openspec/specs/{drive,storage,network}/spec.md`](../openspec/specs/) are the source of truth. Anything in `openspec/changes/` is a proposal in flight; if two proposals conflict, archive one before any code lands. The `enterprise-drive` vs `drive-port` conflict is the cardinal example: archive `enterprise-drive`, keep `drive-port`, re-propose its unique deltas separately.
2. **Cross-cutting types live in one place.** `enterprise-types-cross-cutting` was implemented but not archived; until it lands in `openspec/specs/types/spec.md`, every plugin keeps redeclaring local copies. Land it.
3. **No plugin imports another plugin.** Cross-plugin communication is `NetworkBus` (network) or `drive:hook` (drive) topics only. `drive-network-integration`'s 5-line responsibility table is the rulebook; treat it as the authoritative interpretation of "who owns what."

Every time a future AI suggests adding a cross-plugin import, point them at the bus. Every time a future AI proposes a fat package, point them at the split. Every time a future AI invents new types in a leaf package, point them at `@suaveplan/types`.

That keeps the water clear.