142 lines
8.1 KiB
Markdown
142 lines
8.1 KiB
Markdown
# Drive Encryption — Encrypt / Sign / Decrypt / Verify on the Fly
|
|
|
|
The user's question: *"to be able to encrypt / sign / decrypt / verify on-the-fly."*
|
|
|
|
The drive layer does this through `@suaveplan/drive-plugin-chunks`, which streams a file through a five-stage pipeline on the way down to storage and reverses it on the way up. Storage never sees plaintext.
|
|
|
|
Canonical specs:
|
|
|
|
- [`openspec/specs/packer/spec.md`](../../openspec/specs/packer/spec.md) — chunked AEAD format.
|
|
- [`openspec/changes/drive-plugin-chunks-port/specs/drive-plugin-chunks/spec.md`](../../openspec/changes/drive-plugin-chunks-port/specs/drive-plugin-chunks/spec.md) — drive-side requirements.
|
|
|
|
## The chunk binary format (locked, v0x0001)
|
|
|
|
Every chunk on disk is exactly:
|
|
|
|
```text
|
|
metadata(100) ‖ sig(64) ‖ ciphertext(variable)
|
|
```
|
|
|
|
| Field | Size | Description |
|
|
| --- | --- | --- |
|
|
| version | 3 | `0x0001` (locked v1; `0x0002+` reserved for future formats) |
|
|
| fileId | 32 | 32-byte stable file identifier |
|
|
| chunkId | 4 | 0-indexed chunk number within the file |
|
|
| totalChunks | 4 | Total chunks in the file |
|
|
| fileSize | 8 | Total file size in bytes |
|
|
| chunkSize | 4 | Configured chunk size (e.g. 65536) |
|
|
| payloadLen | 4 | Actual ciphertext length for this chunk |
|
|
| flags | 2 | Reserved (compression, alg agility — currently 0) |
|
|
| nonce | 24 | XChaCha20 nonce, deterministic from `(fileId, chunkId)` |
|
|
| aeadTag | 16 | Poly1305 authentication tag |
|
|
| **sig** | **64** | **Ed25519 signature over `metadata ‖ aeadTag`** |
|
|
| ciphertext | `payloadLen` | XChaCha20-Poly1305 encrypted payload |
|
|
|
|
Header total: 100 bytes. Signature: 64 bytes. Header + sig = **164 bytes per chunk** of overhead, regardless of chunk size.
|
|
|
|
Non-final chunks are exactly `chunkSize - 164` plaintext bytes. The final chunk is naturally short (no zero padding).
|
|
|
|
## The pipeline
|
|
|
|
### Write side
|
|
|
|
```text
|
|
plaintext stream
|
|
│
|
|
├── split into chunks of (chunkSize - 164) plaintext bytes (chop)
|
|
│
|
|
├── for each chunk:
|
|
│ ├── nonce = HKDF(fileKey, "chunk-nonce", fileId ‖ chunkId)
|
|
│ ├── ciphertext, aeadTag = XChaCha20-Poly1305-encrypt(fileKey, nonce, plaintext, AAD=metadata)
|
|
│ ├── sig = Ed25519-sign(driveSecretKey, metadata ‖ aeadTag)
|
|
│ └── chunk = metadata ‖ sig ‖ ciphertext (164 + payload)
|
|
│
|
|
└── store each chunk:
|
|
if storage.capabilities.partialWrites:
|
|
storage.setPartial(fileKey, chunkId * chunkSize, chunk) // sparse-file layout
|
|
else:
|
|
storage.set(`${fileKey}/${chunkId}`, chunk) // chunk-keys layout
|
|
│
|
|
├── manifest = { fileId, totalChunks, version, entries: [{ chunkId, hash, payloadLen }, ...] }
|
|
├── manifestSig = Ed25519-sign(driveSecretKey, canonicalSerialize(manifest))
|
|
└── store manifest at `manifests/${fileId}-v${version}`
|
|
```
|
|
|
|
### Read side
|
|
|
|
```text
|
|
storage read of one chunk
|
|
│
|
|
├── parse 164-byte header (extract metadata, sig, AAD)
|
|
│
|
|
├── verify Ed25519 sig over (metadata ‖ aeadTag) using driveId's pubkey
|
|
│ → reject with CHUNKS.SIG_INVALID if it doesn't match
|
|
│
|
|
├── decrypt ciphertext with XChaCha20-Poly1305(fileKey, nonce, AAD=metadata)
|
|
│ → reject with CHUNKS.AEAD_FAILED if the tag doesn't match
|
|
│
|
|
├── reassemble plaintext from contiguous chunkIds
|
|
│
|
|
└── verify whole-file BLAKE3 hash matches the manifest's checksum
|
|
→ reject with CHUNKS.INTEGRITY_FAILED otherwise
|
|
```
|
|
|
|
The verify-then-decrypt order is intentional: the signature is over the metadata + AEAD tag, *not* the plaintext, so signature verification can happen on bytes the storage layer just produced — no plaintext needed. If the signature is wrong (tampered chunk, wrong drive), we reject before spending CPU on decrypt.
|
|
|
|
## Where the keys come from
|
|
|
|
Two key materials per file:
|
|
|
|
1. **`fileKey`** — symmetric AEAD key. 32 bytes. Used for XChaCha20-Poly1305. Never leaves the drive in plaintext.
|
|
2. **`driveSecretKey`** — Ed25519 signing key. The drive's identity. Used to sign every chunk and every manifest.
|
|
|
|
Where they live:
|
|
|
|
- `driveSecretKey` is stored in the **vault** (`@suaveplan/crypto-vault`). The vault is unlocked with a BIP39 mnemonic + Argon2id-hardened KDF; on lock or process exit, all derived keys are zeroized. The public key — `driveId` — is derived from this and is what every other drive uses to verify signatures.
|
|
- `fileKey` is per-file, generated fresh on first write. It is **wrapped** with the partition KEK (key-encryption-key) by `drive-sharing` and stored alongside the manifest. Authorized peers unwrap it; unauthorized peers see only ciphertext.
|
|
|
|
For shared files (multiple drives can read), `drive-sharing` rewraps the `fileKey` with each authorized drive's public key (X25519 + sealed-box). Revocation rotates `fileKey` on the next write.
|
|
|
|
## End-to-end encryption properties
|
|
|
|
| Property | Mechanism |
|
|
| --- | --- |
|
|
| Confidentiality at rest | XChaCha20-Poly1305 ciphertext is what storage stores. Any backend (S3, OPFS, FAT) sees only ciphertext. |
|
|
| Confidentiality in transit | Same ciphertext flows through `drive-transport-*`. Network plugins never see plaintext. |
|
|
| Authenticity | Per-chunk Ed25519 sig — only the drive that wrote the chunk could have produced it. |
|
|
| Integrity | (a) AEAD tag per chunk, (b) BLAKE3 over the whole file in the manifest, (c) Ed25519 over the manifest. Three layers; any one catches tampering. |
|
|
| Anti-substitution | Manifest signature pins the exact `(chunkId, hash, payloadLen)` tuple. A peer can't swap chunk 3's content for an older chunk 3 of the same length. |
|
|
| Forward secrecy on revocation | Rotating `fileKey` on the next write means revoked peers can decrypt only what they've already cached, not future content. |
|
|
| Replay resistance | Manifest version monotonically increases; old manifests are rejected once a newer one is signed by the same drive. |
|
|
|
|
## The on-the-fly part
|
|
|
|
Both directions are streaming. The chunks plugin uses `@suaveplan/streams` (which uses WHATWG `TransformStream`). A 1 GiB file never fully lands in memory:
|
|
|
|
- **Write** — `createWriteStream(path)` returns a `WritableStream` that the plugin pipes through `chop → encrypt+sign → store`.
|
|
- **Read** — `createReadStream(path)` returns a `ReadableStream` that pulls chunks lazily, verifies and decrypts each, and yields plaintext bytes.
|
|
|
|
The pipeline is backpressure-safe end-to-end: if the consumer slows down, the storage reads stop; if the producer is slow, the encrypt stage idles.
|
|
|
|
## What the storage layer sees
|
|
|
|
A 1 GiB file written through this pipeline, with `chunkSize = 65536`:
|
|
|
|
- Sparse-file layout: one storage key (`files/<fileId>`), 1 GiB of opaque ciphertext, plus one key (`manifests/<fileId>-v<version>`) of ~16 KiB.
|
|
- Chunk-keys layout: 16,384 storage keys (`files/<fileId>/<chunkId>`), each ~64 KiB ciphertext, plus the manifest key.
|
|
|
|
In **neither** case can the storage backend reconstruct the plaintext, the file size in plaintext, the file path, the owner, or the file's relationship to other files. Storage is a black box that holds ciphertext.
|
|
|
|
## Why this fits enterprise scenarios
|
|
|
|
- **Encrypted backups** — set the drive's storage to S3, configure server-side encryption *also* if you want defence-in-depth. The bytes hitting S3 are already AEAD-sealed; SSE is icing.
|
|
- **Browser drives without trust in the storage origin** — OPFS in a third-party context or IndexedDB in a shared browser is fine: the bytes are ciphertext.
|
|
- **Multi-tenant servers** — `NamespacedStorage` per tenant, plus `drive-sharing` partitions, plus per-file `fileKey` — three layers of isolation.
|
|
- **Peer-to-peer chunk distribution** — peers serve ciphertext bytes they cannot themselves read (unless they hold the wrapped `fileKey`). The mesh forwards verifiable bytes; only authorised consumers decrypt.
|
|
|
|
## Reading further
|
|
|
|
- [`plugins.md`](./plugins.md) — where the chunks plugin sits in the chain.
|
|
- [`mesh-and-sharing.md`](./mesh-and-sharing.md) — partition-based key wrapping and grant/revoke.
|
|
- [`../network/security-and-tokens.md`](../network/security-and-tokens.md) — message-level signing and the mesh/session token model.
|