suaveplan/docs/storage/architecture.md

# Storage Architecture

Canonical capability spec: [`openspec/specs/storage/spec.md`](../../openspec/specs/storage/spec.md) (`STORAGE-01`..`STORAGE-04`). This document explains the contract; the spec defines it.

## The structural contract: `StorageLike`

The minimum any storage adapter must satisfy lives in `@suaveplan/types/storage-like` so that consumers (drive, plugins, tests) can accept any conformant implementation without taking a runtime dependency on `@suaveplan/storage`. This is the wire that breaks the circular-dep risk between drive and storage.

```ts
interface StorageLike {
  get(key: string): Promise<Uint8Array | undefined>;
  set(key: string, value: Uint8Array): Promise<void>;
  delete(key: string): Promise<boolean>;
  exists(key: string): Promise<boolean>;
  list(opts?: { prefix?: string; limit?: number }): Promise<{
    keys: string[]; total: number; hasMore: boolean; continuationToken?: string;
  }>;
  clear(): Promise<void>;
  // optional batch surface
  getMany?(keys: readonly string[]): Promise<ReadonlyMap<string, Uint8Array | undefined>>;
  setMany?(entries: readonly (readonly [string, Uint8Array])[]): Promise<void>;
  deleteMany?(keys: readonly string[]): Promise<number>;
}
```

## The concrete base class: `AbstractStorage`

`AbstractStorage` (in `@suaveplan/storage`) extends `StorageLike` with the rest of the platform surface. Every concrete backend extends `AbstractStorage`.

| Method | Purpose |
| --- | --- |
| `get(key)` / `set(key, value, options?)` | Whole-blob read/write. `options` carries TTL and metadata. |
| `setPartial(key, offset, bytes, signal?)` | **Sparse byte-range write.** See `STORAGE-01..03` below. |
| `getPartial(key, start, end)` | Byte-range read for resume-friendly downloads. |
| `delete(key)` / `has(key)` / `clear(pattern?)` | Lifecycle. |
| `list(pattern?, options?)` | Glob-prefixed enumeration. |
| `batch(operations)` | Multi-op grouping (atomic only when the backend says so). |
| `createReadStream(key)` / `createWriteStream(key)` | WHATWG streams; falls back to whole-blob when `nativeStreams=false`. |
| `namespace(prefix)` | Returns a `NamespacedStorage` view. |
| `capabilities` | Readonly record describing what the backend can do natively. |

## The capability matrix (`STORAGE-04`)

Every backend declares a `capabilities` record so the drive layer can pick the right strategy without runtime probing.

```ts
interface StorageCapabilities {
  partialReads: boolean;        // getPartial(key, start, end) is native
  partialWrites: boolean;       // setPartial(key, offset, bytes) is native (no RMW)
  nativeStreams: boolean;       // createReadStream/createWriteStream are native (no buffering)
  atomicWrites: boolean;        // set is atomic across crash
  contentAddressable: boolean;  // key is the content hash; same-content writes coalesce
}
```

Defaults are all `false` (`DEFAULT_STORAGE_CAPABILITIES`). Subclasses opt in only for what they truly support.

The drive-plugin-chunks layer reads these flags directly to choose its layout (see `CHUNKS-LAYOUT-AUTO-01`):

- `partialWrites === true` → use the **sparse-file** layout (one key per file, byte ranges).
- `partialWrites === false` → use the **chunk-keys** layout (one key per chunk).

## `setPartial` semantics (`STORAGE-01..03`)

`setPartial` is the load-bearing method that lets large files survive a backend that does not support random-access writes. The contract is:

1. **Always available** (`STORAGE-01`). `AbstractStorage.setPartial` is abstract; every concrete subclass must provide one. `defaultSetPartial` is a protected RMW fallback that subclasses can call when `capabilities.partialWrites === false`.
2. **Sparse extension** (`STORAGE-02`). Writing to an offset past the current end zero-fills the gap on subsequent reads. This lets the chunks plugin pre-allocate a sparse file and fill chunks in arbitrary order.
3. **In-place overwrite** (`STORAGE-03`). Writing into an existing range overwrites only the addressed bytes; everything outside `[offset, offset+bytes.length)` is preserved exactly. Zero-length writes are a no-op.

When a backend genuinely cannot support partial writes (`S3`), `setPartial` falls back to `defaultSetPartial`, which downloads the full object, splices the new bytes in, and re-uploads — correct but expensive. The drive layer reads `partialWrites` and routes around this when it can.

## `NamespacedStorage` (scoping)

`storage.namespace("drive/users/alice/")` returns a `NamespacedStorage` that prefixes every key with `"drive/users/alice/"` before delegating to the inner storage. The wrapper inherits the inner storage's `capabilities` exactly — namespace scoping is purely a key transform. This is how `drive`, `drive-plugin-chunks`, and `drive-sharing` give each tenant/file/partition its own keyspace without dragging tenancy concerns down into every backend.

## What lives above storage, not in it

| Concern | Where it lives | Why |
| --- | --- | --- |
| Paths, directories, `..` resolution | `@suaveplan/drive` | Storage keys are opaque strings. |
| File metadata (size, mtime, owner) | `@suaveplan/drive` | Stat goes through the drive plugin chain. |
| Chunk headers, signatures, AEAD tags | `@suaveplan/drive-plugin-chunks` | Storage stores ciphertext bytes, never plaintext. |
| Manifests, BLAKE3/SHA-256 integrity | `@suaveplan/drive-plugin-chunks` | Storage does not verify content. |
| Replication, redundancy, peer fetch | `@suaveplan/drive-mesh` + `@suaveplan/network-plugin-content-delivery` | Storage is single-replica by definition. |
| Encryption keys, vault, KDF | `@suaveplan/crypto-vault`, `@suaveplan/keys` | Storage never sees a key. |

This separation is what lets the storage layer accept **any** backend — local fs, S3, OPFS, IndexedDB, an embedded FAT image, a Garage cluster — and have the drive layer treat them all the same.

## OTel and errors

Every concrete subclass instruments `storage.ops` (counter), `storage.op.duration_ms` (histogram), and `storage.value.bytes` (histogram). Errors are `StorageError` (and the four typed subclasses `StorageNotFoundError`, `StorageQuotaError`, `StorageInvalidKeyError`, `StorageBatchError`) with codes in the frozen `STORAGE_CODES` registry. Backends MAY add their own namespaced registries (`S3.S3_CODES`) but MUST extend the base error class.

## Reading further

- [`backends.md`](./backends.md) — every concrete backend.
- [`implementation-status.md`](./implementation-status.md) — what's shipped, what's pending.
- [`../drive/architecture.md`](../drive/architecture.md) — the consumer.