Files

2026-05-27 23:29:53 +01:00

18 KiB

Raw Blame History

System Review — Drive · Storage · Network

A critical pass over the design captured in docs/storage/, docs/drive/, docs/network/, and docs/drive-storage-network.md. Identifies concrete flaws, proposes fixes, and aligns the system with the in-conversation requirements: bitpacker-encoded chunk-state bitsets, batched mesh requests, dehydrated drive templates, CDN fallback with reversed-ID URL layout, per-file policy flags, and lock-icon UX.

The review is structured as 16 numbered findings, each with: diagnosis (the gap), severity, and fix (the concrete change). Where the fix has a corresponding new document, that doc is referenced and contains the formal spec.

Severity legend

Blocker — without this, a real deployment will fail. Must land before any drive/network code ships.
Major — produces incorrect behavior, wasted bandwidth, or UX gaps that users will notice. Land early.
Minor — design hygiene, future-proofing, or convention. Land opportunistically.

§1 — Chunk request granularity is wasteful — Major

Diagnosis. The current spec (drive-mesh-port, network-plugin-content-delivery) describes cd:request as one bus event per chunk. A 1,000-chunk file with 800 missing chunks generates 800 envelopes, 800 round-trips, 800 routing decisions. Even with the 13-byte protocol header, the framing overhead is ~16 KiB of pure noise per file — and the worst case (every peer requesting the same 800 chunks) saturates the gossip layer.

Fix. Define a unified ChunkRequestEnvelope with three forms:

{ kind: 'whole' } — request everything the holder has (replaces empty-array ambiguity).
{ kind: 'specific'; chunkIds: number[] } — sparse selection (use when ≤ 64 chunks).
{ kind: 'bitset'; bytes: Uint8Array } — dense selection (use when > 64 chunks; bytes encoded via @suaveplan/bitpacker-compressed for sparse runs).

The selection threshold (64) matches the 8-byte break-even where bitpacker-compressed outperforms a chunkIds array.

Spec'd in: drive/mesh-protocol.md.

§2 — Bitset advertisements not standardized to bitpacker — Major

Diagnosis. MESH-GOSSIP-01 says "gzipped bitset advertisements" but does not pin a schema or codec. With 10 bitpacker packages already shipped (@suaveplan/bitpacker, bitpacker-compressed, bitpacker-network, bitpacker-schema, bitpacker-streaming, bitpacker-strategies, bitpacker-delta, bitpacker-game, bitpacker-validation, bitpacker-crypto), every mesh implementer would otherwise pick a different encoding and fragment the protocol.

Fix. Pin the canonical advertisement format:

Codec: @suaveplan/bitpacker-compressed for the bitset payload (RLE + gzip — typical 800-chunk presence map encodes to <100 bytes when most chunks are present).
Schema: @suaveplan/bitpacker-schema for envelope validation.
Wire transport: @suaveplan/bitpacker-network for the on-wire framing.

Spec'd in: drive/mesh-protocol.md §3.

§3 — No "dehydrated drive" concept — Blocker

Diagnosis. The current docs assume drives are populated by writes. They have no concept of "here is the structure of the drive (folders, files, sizes, hashes) — go fetch the bytes from peers/CDN as needed." This is a real production scenario: a game ships with a drive template describing 2 GB of assets across 5,000 files; the user starts the game and the drive lazily populates from mesh peers, falling back to a CDN, while showing per-file progress in the UI.

Fix. Introduce DriveTemplate — a server-signed, schema-versioned config containing:

Folder structure (recursive FolderTemplate).
Per-file FileTemplate stubs with fileId, size, chunkCount, checksum, and policy flags.
Optional inline-wrapped fileKey for immediate decryption, OR a keyAuthority peer ID for retroactive grant.
Server signature over canonical serialization, with expiresAt and a 16-byte nonce for replay protection.

Drive.hydrateFrom(template) instantiates the DirIndex and FileLocationIndex from the template; per-file bytes arrive over the mesh/CDN later.

Spec'd in: drive/dehydrated-drives.md.

§4 — CDN fallback URL convention unspecified — Major

Diagnosis. drive-transport-https is described as "server fallback when WebRTC isn't available" but no URL layout is defined. Without a convention, every deployment writes its own — and the chunks plugin can't fetch from a CDN it doesn't know how to address.

Fix. Adopt the reversed-ID URL scheme for CDN-backed chunk pools:

<base-url>/<reversed[0]>/<reversed[1]>/<reversed[2]>/<fileId>_<chunkIdPadded>.chunk

Where reversed = fileId.split('').reverse().join('') and chunkIdPadded is the chunk ID zero-padded to 9 digits. Example with fileId = "abcdef1234567890...":

/0/9/8/abcdef1234567890.../000000001.chunk

Reversing avoids hot directories when fileIds share prefixes (sequential timestamps, f- prefixes, etc.). Three directory levels × 16-character hex alphabet = 4,096 leaf directories — distributes 1 M files at ~250 per directory.

DriveTemplate.cdnFallback carries { baseUrl, levels, padding } so the layout is declarative, not hardcoded.

Spec'd in: drive/dehydrated-drives.md §4.

§5 — Per-file policy flags missing — Blocker

Diagnosis. The drive layer has no per-file controls for the three policies users care about:

autoDownload — should this file be eagerly fetched, or wait until first read?
meshAllowed — may we request this from peers, or must we hit the server only?
cdnAllowed — may we fall back to the CDN, or only the mesh / only local?

Today these are global drive settings or implicit. The user's brief calls for them to be per-file, signed in the template, and enforced at runtime.

Fix. Add PerFilePolicy to FileLocation and FileTemplate:

interface PerFilePolicy {
  autoDownload: boolean;
  meshAllowed: boolean;
  cdnAllowed: boolean;
  retention?: 'permanent' | 'lru' | { maxAgeMs: number };
}

The drive enforces these in the read middleware:

autoDownload=true triggers a fetch on Drive.hydrateFrom() not on first read.
meshAllowed=false skips the cd:request bus emit.
cdnAllowed=false skips the drive-transport-https fallback.
All-disabled means the file is local-only — read() throws DRIVE.UNAVAILABLE if the bytes aren't present.

Spec'd in: drive/dehydrated-drives.md §6.

§6 — No `getKeyStatus()` / no lock-icon UX contract — Major

Diagnosis. The user's brief calls for visual lock states (locked, unlocked-with-key, unencrypted) per file. The drive surface has no method exposing this — the chunks plugin knows about encryption, the sharing plugin knows about KEKs, but Drive.list() returns DirEntries that don't aggregate the answer.

Fix. Add Drive.getKeyStatus(path): KeyStatus returning:

type KeyStatus = 'unencrypted' | 'encrypted-locked' | 'encrypted-unlocked';

And include it on every DirEntry returned by list():

interface DirEntry {
  // ... existing fields ...
  keyStatus?: KeyStatus;  // present for type === 'file'
  presence?: ChunkPresence;  // see §7
}

UX maps directly: 🔒 for encrypted-locked, 🔓 for encrypted-unlocked, no icon for unencrypted.

Spec'd in: drive/architecture.md (will be updated).

§7 — `Drive.list()` lacks presence/progress data — Major

Diagnosis. The user wants ███░░░░░░░ 33% next to file listings. Currently Drive.list() returns DirEntries with size and mtime but no chunksPresent / chunksTotal. The mesh plugin tracks this internally but doesn't expose it.

Fix. Add ChunkPresence to every file DirEntry:

interface ChunkPresence {
  chunksPresent: number;     // count of chunks locally available
  chunksTotal: number;       // total chunks in the file
  sourceHints: SourceHint[]; // ordered: where missing chunks could come from
}

type SourceHint = 'local' | 'mesh' | 'cdn';

Renderable as a progress bar in any UI. Updated lazily — list() is allowed to return a snapshot from the last gossip cycle, not a fresh probe.

Spec'd in: drive/architecture.md update.

§8 — Retroactive key distribution undefined — Major

Diagnosis. The user's brief: "optional decryption keys could be included as part of the signed config, or would have to be retro-actively added to the drive for files to be unlocked." The current sharing plugin handles partition KEKs but not per-file fileKey retro-grants from arbitrary peers.

Fix. Two distribution paths, both first-class:

Inline grant (FileTemplate.encryption.keyDistribution.kind === 'inline') — the wrapped fileKey is in the signed template. Drive can decrypt immediately on hydrateFrom().
Retroactive grant (FileTemplate.encryption.keyDistribution.kind === 'retroactive') — template names a keyAuthority (peer pubkey). The peer issues a KeyGrantEnvelope whenever access is granted:

interface KeyGrantEnvelope extends MessageEnvelope {
  type: 'key:grant';
  payload: {
    fileId: string;
    wrappedKey: Uint8Array;     // wrapped to recipient X25519 pubkey
    issuer: string;              // peer pubkey
    issuerSignature: Uint8Array; // Ed25519 over (fileId, wrappedKey, recipient, expiresAt)
    expiresAt: number;
  };
}

On receipt, the drive validates the issuer signature against the template's keyAuthority, unwraps, and stores the fileKey in its KeyRing. The lock icon flips from 🔒 to 🔓.

Spec'd in: drive/dehydrated-drives.md §7.

§9 — No cache retention strategy — Major

Diagnosis. A drive that auto-downloads everything will fill the disk forever. The current docs don't address GC, LRU, or retention windows. Users in dehydrated-drive scenarios (game assets, media libraries) need predictable storage budgets.

Fix. Add three retention modes to PerFilePolicy.retention:

'permanent' — never evict (default for explicit user-saved files).
'lru' — drive maintains a global byte budget; LRU evicts chunks (not whole files; preserves manifest+index for re-download).
{ maxAgeMs } — chunks older than N ms eligible for eviction.

Plus DriveOptions.maxLocalCacheBytes for the global budget. Eviction is at chunk granularity; the file remains visible (with reduced chunksPresent) and re-fetches on read.

Spec'd in: drive/dehydrated-drives.md §6.

§10 — Mesh advertisement trust unbounded — Major

Diagnosis. MESH-GOSSIP-01 says peers advertise their chunk inventory. A malicious or buggy peer can advertise chunks it doesn't have, wasting our request budget. The current spec has MESH-SCORE-01 (bad-sig sets score to -Infinity), but only after we've already paid the round-trip.

Fix. Probabilistic verification + adaptive trust:

On a peer's first N requests for a given file, accept their advertisements as truth.
After the first M responses, compute their "advertised-but-missing" rate. If > 5%, demote to "unverified" — their advertisements are still listened to, but probabilistically skipped (P(skip) ∝ miss rate).
Bad sig immediately drops to -Infinity (existing MESH-SCORE-01).
Trust score per (peerId, fileId) so a peer good for one file doesn't get blanket trust.

Spec'd in: drive/mesh-protocol.md §6.

§11 — Chunk response doesn't piggyback presence — Minor

Diagnosis. The current cd:response returns chunk bytes only. But the responding peer is in the perfect position to tell us its full presence bitset for this file — saving us a separate cd:advertise round-trip in many cases.

Fix. Add presence: Uint8Array (bitpacker-compressed bitset) to every cd:response payload. Receiver updates its peer-index for that (peerId, fileId) for free.

Spec'd in: drive/mesh-protocol.md §4.

§12 — CDN write-side authentication unspecified — Major

Diagnosis. The CDN URL convention (§4) covers reads. Writes are unspecified — anyone with the URL would otherwise be able to upload chunks, poisoning the cache. The current drive-transport-https proposal doesn't address this.

Fix. CDN writes use server-signed upload tokens, not direct PUT:

The drive POSTs a chunk to <base-url>/upload with a CdnUploadToken (server-signed, 60-second expiry).
Token contains (fileId, chunkId, chunkHash, uploaderPubkey). CDN validates the token, validates the chunk against the hash, and stores at the reversed-ID URL.
Ed25519 sig from the chunk's writer is preserved end-to-end; readers verify it when fetching.

CDNs that don't want to validate signatures can decline writes (read-only CDN); the drive falls back to peer storage in that case.

Spec'd in: drive/dehydrated-drives.md §5.

§13 — Drive template lacks replay protection — Major

Diagnosis. A DriveTemplate signed once is valid forever unless its expiresAt is hit. An attacker could re-issue an old (less-restrictive) template they captured. There's no nonce or sequence number making each issuance unique.

Fix. Required nonce: Uint8Array(16) and issuedAt: number on every template. Signature covers both. Drives reject templates with:

nonce already seen (16-byte LRU of recent nonces, ~1 KiB cost).
issuedAt more than 24h in the past.
expiresAt in the past (existing).

Spec'd in: drive/dehydrated-drives.md §3.

§14 — `kind: 'inline'` files miss signature scaffolding — Minor

Diagnosis. The FileLocation.storageHints kind: 'inline' case (small files <1 KiB stored directly in the FileLocationIndex) skips the chunks-plugin pipeline. That means no per-chunk Ed25519 sig and no manifest sig. For internal indexes and tiny configs this is fine, but if an inline file is shared, the integrity story breaks.

Fix. Inline files must still carry a signature: { kind: 'inline'; bytes: Uint8Array; sig: Uint8Array } where sig = Ed25519-sign(driveSecretKey, fileId ‖ bytes). Verify on read. The cost is +64 bytes per inline file; trivial.

Spec'd in: drive/index-and-paths.md (will be updated).

§15 — No "stuck" state surface — Minor

Diagnosis. When a chunk is unavailable from local + mesh + CDN, the drive read currently throws DRIVE.UNAVAILABLE. But the UI wants to know before the user clicks: "this file is partially-stuck — 12 chunks unfetchable." Today there's no signal.

Fix. Add ChunkPresence.stuckChunks: number (chunks tried-and-failed across all configured sources within the last N minutes). Reset on retry-success or peer-population. UI can show ⚠ next to a file with stuckChunks > 0.

Spec'd in: drive/architecture.md update.

§16 — Folder template recursion unbounded — Minor

Diagnosis. FolderTemplate.children: (FolderTemplate | FileTemplate)[] is recursive with no depth cap. A maliciously-crafted template could blow the stack on validation.

Fix. Cap recursion at depth 32 (conservative; real filesystems rarely exceed 16). Validate during template parsing; reject deeper. The depth cap is in DriveTemplateSchema (Zod).

Spec'd in: drive/dehydrated-drives.md §3.

Summary by severity

Severity	Count	Findings
Blocker	2	§3 (dehydrated drive), §5 (per-file policy flags)
Major	9	§1, §2, §4, §6, §7, §8, §9, §10, §12, §13
Minor	5	§11, §14, §15, §16

The two Blockers are about features the user explicitly asked for and the system does not yet have. They produce two new docs:

drive/dehydrated-drives.md — DriveTemplate, policy flags, CDN convention, key distribution.
drive/mesh-protocol.md — chunk request envelope forms, bitset advertisements, response piggyback.

The Major items either land inside those two docs or as additive sections to existing docs (drive/architecture.md gains getKeyStatus/getPresence; drive/index-and-paths.md gets the inline-sig fix).

The Minor items are hardening; they ride on the same change set without their own structural impact.

Implementation order

Land enterprise-types-cross-cutting — already archived 2026-05-02. ✓
Resolve enterprise-drive vs drive-port — already SUPERSEDED 2026-05-02. ✓
Author new spec changes for the two new docs:
- drive-mesh-protocol-v1 — formalize the request/response/advertise envelopes (§1, §2, §10, §11).
- drive-template-v1 — formalize DriveTemplate, policy flags, key distribution, CDN convention (§3, §4, §5, §8, §9, §12, §13, §16).
Update existing capability specs — openspec/specs/drive/spec.md adds getKeyStatus / getPresence requirements (§6, §7); openspec/specs/network/spec.md references the new mesh-protocol spec (§1, §2, §10, §11).
Land drive-port + drive-plugin-chunks-port + drive-mesh-port + drive-transport-https-port in dependency order, each conforming to the new specs.
Land network-plugin-content-delivery consuming the mesh-protocol envelopes.
Port the discovery + transport plugins from main.

What this review does NOT touch

Three areas where the existing design is sound and needs no change:

Storage capability matrix. Five flags (partialReads, partialWrites, nativeStreams, atomicWrites, contentAddressable) cover the access patterns; no new flag needed for any finding here.
Tier model & failover. Tier 0/1/2 + four hosting models + sibling-pocket structure already accommodates the user's mesh/online-player scenarios. The mesh-protocol changes plug in without altering topology.
Cryptographic primitives. AEAD + Ed25519 + BLAKE3 + X25519 sealed-box is sufficient. No new primitives proposed.

The stack as documented is a sound foundation. The findings above harden it for the production scenarios the user described — they are sharp edges to round, not load-bearing rewrites.

18 KiB Raw Blame History Unescape Escape