suaveplan/docs/storage/architecture.md

Storage Architecture

Canonical capability spec: openspec/specs/storage/spec.md (STORAGE-01..STORAGE-04). This document explains the contract; the spec defines it.

The structural contract: StorageLike

The minimum any storage adapter must satisfy lives in @suaveplan/types/storage-like so that consumers (drive, plugins, tests) can accept any conformant implementation without taking a runtime dependency on @suaveplan/storage. This is the wire that breaks the circular-dep risk between drive and storage.

interface StorageLike {
  get(key: string): Promise<Uint8Array | undefined>;
  set(key: string, value: Uint8Array): Promise<void>;
  delete(key: string): Promise<boolean>;
  exists(key: string): Promise<boolean>;
  list(opts?: { prefix?: string; limit?: number }): Promise<{
    keys: string[]; total: number; hasMore: boolean; continuationToken?: string;
  }>;
  clear(): Promise<void>;
  // optional batch surface
  getMany?(keys: readonly string[]): Promise<ReadonlyMap<string, Uint8Array | undefined>>;
  setMany?(entries: readonly (readonly [string, Uint8Array])[]): Promise<void>;
  deleteMany?(keys: readonly string[]): Promise<number>;
}

The concrete base class: AbstractStorage

AbstractStorage (in @suaveplan/storage) extends StorageLike with the rest of the platform surface. Every concrete backend extends AbstractStorage.

Method	Purpose
get(key) / set(key, value, options?)	Whole-blob read/write. options carries TTL and metadata.
setPartial(key, offset, bytes, signal?)	Sparse byte-range write. See STORAGE-01..03 below.
getPartial(key, start, end)	Byte-range read for resume-friendly downloads.
delete(key) / has(key) / clear(pattern?)	Lifecycle.
list(pattern?, options?)	Glob-prefixed enumeration.
batch(operations)	Multi-op grouping (atomic only when the backend says so).
createReadStream(key) / createWriteStream(key)	WHATWG streams; falls back to whole-blob when nativeStreams=false.
namespace(prefix)	Returns a NamespacedStorage view.
capabilities	Readonly record describing what the backend can do natively.

The capability matrix (STORAGE-04)

Every backend declares a capabilities record so the drive layer can pick the right strategy without runtime probing.

interface StorageCapabilities {
  partialReads: boolean;        // getPartial(key, start, end) is native
  partialWrites: boolean;       // setPartial(key, offset, bytes) is native (no RMW)
  nativeStreams: boolean;       // createReadStream/createWriteStream are native (no buffering)
  atomicWrites: boolean;        // set is atomic across crash
  contentAddressable: boolean;  // key is the content hash; same-content writes coalesce
}

Defaults are all false (DEFAULT_STORAGE_CAPABILITIES). Subclasses opt in only for what they truly support.

The drive-plugin-chunks layer reads these flags directly to choose its layout (see CHUNKS-LAYOUT-AUTO-01):

• partialWrites === true → use the sparse-file layout (one key per file, byte ranges).
• partialWrites === false → use the chunk-keys layout (one key per chunk).

setPartial semantics (STORAGE-01..03)

setPartial is the load-bearing method that lets large files survive a backend that does not support random-access writes. The contract is:

1. Always available (STORAGE-01). AbstractStorage.setPartial is abstract; every concrete subclass must provide one. defaultSetPartial is a protected RMW fallback that subclasses can call when capabilities.partialWrites === false.
2. Sparse extension (STORAGE-02). Writing to an offset past the current end zero-fills the gap on subsequent reads. This lets the chunks plugin pre-allocate a sparse file and fill chunks in arbitrary order.
3. In-place overwrite (STORAGE-03). Writing into an existing range overwrites only the addressed bytes; everything outside [offset, offset+bytes.length) is preserved exactly. Zero-length writes are a no-op.

When a backend genuinely cannot support partial writes (S3), setPartial falls back to defaultSetPartial, which downloads the full object, splices the new bytes in, and re-uploads — correct but expensive. The drive layer reads partialWrites and routes around this when it can.

NamespacedStorage (scoping)

storage.namespace("drive/users/alice/") returns a NamespacedStorage that prefixes every key with "drive/users/alice/" before delegating to the inner storage. The wrapper inherits the inner storage's capabilities exactly — namespace scoping is purely a key transform. This is how drive, drive-plugin-chunks, and drive-sharing give each tenant/file/partition its own keyspace without dragging tenancy concerns down into every backend.

What lives above storage, not in it

Concern	Where it lives	Why
Paths, directories, .. resolution	@suaveplan/drive	Storage keys are opaque strings.
File metadata (size, mtime, owner)	@suaveplan/drive	Stat goes through the drive plugin chain.
Chunk headers, signatures, AEAD tags	@suaveplan/drive-plugin-chunks	Storage stores ciphertext bytes, never plaintext.
Manifests, BLAKE3/SHA-256 integrity	@suaveplan/drive-plugin-chunks	Storage does not verify content.
Replication, redundancy, peer fetch	@suaveplan/drive-mesh + @suaveplan/network-plugin-content-delivery	Storage is single-replica by definition.
Encryption keys, vault, KDF	@suaveplan/crypto-vault, @suaveplan/keys	Storage never sees a key.

This separation is what lets the storage layer accept any backend — local fs, S3, OPFS, IndexedDB, an embedded FAT image, a Garage cluster — and have the drive layer treat them all the same.

OTel and errors

Every concrete subclass instruments storage.ops (counter), storage.op.duration_ms (histogram), and storage.value.bytes (histogram). Errors are StorageError (and the four typed subclasses StorageNotFoundError, StorageQuotaError, StorageInvalidKeyError, StorageBatchError) with codes in the frozen STORAGE_CODES registry. Backends MAY add their own namespaced registries (S3.S3_CODES) but MUST extend the base error class.

Reading further

• backends.md — every concrete backend.
• implementation-status.md — what's shipped, what's pending.
• ../drive/architecture.md — the consumer.