157 lines
7.5 KiB
Markdown
157 lines
7.5 KiB
Markdown
# Drive Architecture
|
|
|
|
Canonical capability spec: [`openspec/specs/drive/spec.md`](../../openspec/specs/drive/spec.md) (`DRIVE-01`..`DRIVE-09`).
|
|
|
|
## The shape genesis chose
|
|
|
|
Two competing drive designs exist in the OpenSpec change folder:
|
|
|
|
| Design | Folder | Stance |
|
|
| --- | --- | --- |
|
|
| **Monolith** | `enterprise-drive` | Drive is a fat package — chunks, CoW, dedup, snapshots, encryption, transforms, plugins all inside. |
|
|
| **Split** | `drive-port` *(genesis-canonical)* | Drive is a thin VFS core. Chunks, mesh, transports, sharing live in sibling packages. |
|
|
|
|
The genesis branch follows the **split**. The reason is mechanical: the canonical `openspec/specs/drive/spec.md` is the split-design spec (`DRIVE-01..09`), the migrated `@suaveplan/drive` package on genesis already implements the thin core, and the chunks layer is its own migrated package (`@suaveplan/drive-plugin-chunks`). The `enterprise-drive` proposal should be archived and the few unique requirements it carries (CoW, snapshots) re-proposed as their own plugin packages.
|
|
|
|
## The core class
|
|
|
|
```ts
|
|
class Drive<TStorageMap extends Record<string, DriveStorage> = {}> {
|
|
constructor();
|
|
constructor(defaultStorage: DriveStorage, options?: DriveOptions);
|
|
constructor(options: DriveOptions & { defaultStorage?: DriveStorage; plugins?: DrivePlugin[] });
|
|
|
|
use<K extends string, S extends DriveStorage>(id: K, storage: S): Drive<TStorageMap & Record<K, S>>;
|
|
usePlugin(plugin: DrivePlugin): this;
|
|
useMiddleware(mw: MiddlewareHandler): this;
|
|
|
|
initialize(): Promise<void>;
|
|
dispose(): Promise<void>;
|
|
|
|
read(path: string, opts?: ReadOptions): Promise<Uint8Array>;
|
|
write(path: string, bytes: Uint8Array, opts?: WriteOptions): Promise<void>;
|
|
delete(path: string): Promise<boolean>;
|
|
stat(path: string): Promise<DriveStats>;
|
|
list(prefix: string, opts?: ListOptions): Promise<DriveEntry[]>;
|
|
createReadStream(path: string): ReadableStream<Uint8Array>;
|
|
createWriteStream(path: string): WritableStream<Uint8Array>;
|
|
|
|
getStorage<K extends keyof TStorageMap>(id: K): TStorageMap[K];
|
|
}
|
|
```
|
|
|
|
Five things are worth understanding.
|
|
|
|
### 1. `DriveStorage` is the narrowed view of `AbstractStorage` (DRIVE-01)
|
|
|
|
```ts
|
|
interface DriveStorage extends AbstractStorage {
|
|
readonly id: string;
|
|
onDriveAttached?(drive: Drive): Promise<void>;
|
|
onDriveDetached?(drive: Drive): Promise<void>;
|
|
}
|
|
```
|
|
|
|
There is no separate `StorageAdapter` shim. The drive consumes any `AbstractStorage` directly; it just needs an `id` so the drive can address it inside a multi-storage map and (optionally) lifecycle hooks. This is the line that breaks the temptation to wrap storage in another interface.
|
|
|
|
### 2. Lifecycle is ownership-tracking (DRIVE-02, DRIVE-03)
|
|
|
|
```text
|
|
initialize() → for each storage, call connect() iff !isConnected(); record in connectedByMe: WeakSet
|
|
dispose() → for each storage in connectedByMe, call close()
|
|
→ leave pre-connected storages alone
|
|
```
|
|
|
|
A drive that didn't open the connection doesn't close it. This matters for shared backends (e.g., a server with one S3 client serving many drives): the first drive opens it, the others reuse it, and the *last* drive to be disposed is the one that actually closes the connection (or none of them, if the client owns the lifetime).
|
|
|
|
### 3. Operations after `dispose()` throw (DRIVE-04)
|
|
|
|
`read`, `write`, `delete`, `stat`, `list`, `createReadStream`, `createWriteStream` all throw `DriveError(DRIVE.DISPOSED)` after `dispose()`, and `DriveError(DRIVE.NOT_INITIALIZED)` before `initialize()`. The drive is a state machine: `pending → initialized → disposed`, and the storage layer below it is left in a known state.
|
|
|
|
### 4. Plugin middleware order is well-defined (DRIVE-06)
|
|
|
|
```text
|
|
drive.usePlugin(A);
|
|
drive.usePlugin(B);
|
|
drive.useMiddleware(M);
|
|
```
|
|
|
|
Outbound (request path): `A → B → M → core operation`.
|
|
Inbound (response path): `core operation → M → B → A`.
|
|
|
|
`plugin.dispose()` is called in **reverse** registration order, so the most-recently-installed plugin gets to clean up first. Plugin install/dispose throws are wrapped in `DriveError(DRIVE.PLUGIN_FAILED, { cause })` so consumers see one error type at the boundary.
|
|
|
|
### 5. Capability fallbacks remove runtime probing (DRIVE-07)
|
|
|
|
```text
|
|
createReadStream(path):
|
|
if storage.capabilities.nativeStreams === true:
|
|
return storage.createReadStream(...)
|
|
else:
|
|
return AbstractStorage.defaultCreateReadStream(...) // chunked read of get()
|
|
```
|
|
|
|
Same rule for `createWriteStream`. The drive never inspects the backend at runtime to decide whether to stream — it reads `capabilities` once and routes accordingly. This is why the storage layer's capability matrix is load-bearing.
|
|
|
|
## Plugin and middleware contracts
|
|
|
|
Both contracts live in `@suaveplan/types` (the cross-cutting types package) so plugins and middleware never import drive directly:
|
|
|
|
```ts
|
|
interface DrivePlugin {
|
|
readonly name: string;
|
|
readonly hooks: Record<string, PluginHook>; // e.g. { 'drive:read': fn, 'drive:write': fn }
|
|
readonly priority?: number;
|
|
}
|
|
|
|
type PluginHook = (context: unknown, next: NextFunction) => Promise<unknown>;
|
|
type NextFunction = () => Promise<void>;
|
|
|
|
interface MiddlewareHandler {
|
|
(context: MiddlewareContext, next: NextFunction): Promise<void>;
|
|
}
|
|
|
|
interface MiddlewareContext {
|
|
operation: string; // 'read' | 'write' | 'delete' | 'stat' | 'list'
|
|
path: string;
|
|
[key: string]: unknown; // open index — drive attaches { drive, args, ... }
|
|
}
|
|
```
|
|
|
|
The `[key: string]: unknown` index signature is deliberate: it lets `Drive` attach domain-specific fields to the context (`{ drive: this, args: [path, bytes] }`) without forcing `@suaveplan/types` to import `Drive`. That is the wire that breaks the otherwise-circular dependency.
|
|
|
|
## Generic storage typing (DRIVE-08)
|
|
|
|
```ts
|
|
const drive = new Drive()
|
|
.use("local", new MemoryStorage())
|
|
.use("remote", new S3({ ... }));
|
|
|
|
drive.getStorage("local"); // typed as MemoryStorage
|
|
drive.getStorage("remote"); // typed as S3
|
|
drive.getStorage("nope"); // TS error
|
|
```
|
|
|
|
`Drive<TStorageMap>` narrows on each `use<K, S>(id, storage)` call so `getStorage(id)` returns the exact backend type, not a widened `DriveStorage`. Plugins that need to call backend-specific APIs (e.g., the chunks plugin asking S3 for a presigned URL) get type-safe access without casts.
|
|
|
|
## Constructor overloads (DRIVE-09)
|
|
|
|
```ts
|
|
new Drive() // no default storage; must add via .use()
|
|
new Drive(memoryStorage) // registers as id "default"
|
|
new Drive(memoryStorage, { plugins: [chunks] }) // default + options
|
|
new Drive({ defaultStorage: memoryStorage, plugins: [chunks] }) // bag-of-options form
|
|
```
|
|
|
|
The bare-storage form is a thin sugar over `{ defaultStorage }` — registers the storage under id `"default"`. Every other form is just option assembly.
|
|
|
|
## Why this shape
|
|
|
|
The narrow core is what makes "n drives" work. Every feature you might want — chunking, encryption, sharing, mesh sync, CoW snapshots, dedup — is a plugin. A drive that doesn't need the network skips the network plugins. A drive in a browser without OPFS uses `IndexedDbStorage` and the chunks plugin's `chunk-keys` layout. A drive on a server with petabytes of S3 uses the `sparse-file` layout and skips drive-mesh because there's only one replica anyway. None of those configurations require code changes in `Drive` itself.
|
|
|
|
## Reading further
|
|
|
|
- [`index-and-paths.md`](./index-and-paths.md) — how paths become storage keys.
|
|
- [`plugins.md`](./plugins.md) — what each plugin contributes.
|
|
- [`../storage/architecture.md`](../storage/architecture.md) — the layer below.
|