199 lines
17 KiB
Markdown
199 lines
17 KiB
Markdown
# Design — `complete-drive-uri`
|
|
|
|
Non-normative. Normative requirements live in `./specs/drive-uri/spec.md` and `./specs/drive/spec.md`.
|
|
|
|
## Goals
|
|
|
|
1. Land every URI operation (`get`, `set`, `delete`, `stat`, `exists`, `list`, `createReadStream`, `createWriteStream`) on every scheme (`drive://`, `drive+mesh://`, `drive+template://`) with first-class semantics.
|
|
2. Thread `AbortSignal` through every buffered op (today's spec footnote becomes a spec requirement).
|
|
3. Land `{ offset, length }` range reads end-to-end (resolver → adapter → `Drive.createReadStream` → chunks-plugin middleware → mesh `ChunkRequest.specific|bitset` pivot).
|
|
4. Per-scheme query-string options (`?advertise`, `?nonce`, `?signal-timeout-ms`) with explicit redaction policy.
|
|
5. Comprehensive `DRIVE_URI.*` error registry (14 codes total — every code reachable from a real call site).
|
|
6. Re-run `runUriPluginConformance` over the full op matrix for all three plugins.
|
|
|
|
## Per-scheme operation matrix (canonical)
|
|
|
|
| Op | `drive://` | `drive+mesh://` | `drive+template://` |
|
|
|---------------------|----------------------------------|----------------------------------------------------------|------------------------------------------------|
|
|
| `get(key, signal?)` | `Drive.read(key, { signal })` | same | hydrate (memoised) → `Drive.read(key, { signal })` |
|
|
| `set(key, v, sig?)` | `Drive.write(k, v, { signal })` | `Drive.write` + opt-in `MeshAdvertiseHook.advertise(...)` on `?advertise=true` | throw `TEMPLATE_READ_ONLY` |
|
|
| `delete(key, sig?)` | `Drive.delete(k, signal)` | `Drive.delete` + opt-in `MeshAdvertiseHook.unadvertise(...)` | throw `TEMPLATE_READ_ONLY` |
|
|
| `stat(key, sig?)` | `Drive.stat(k, signal)` → `StatResult` | same; cache-aware (the post-hydrate `Drive.stat` is cheap) | post-hydrate `Drive.stat` |
|
|
| `exists(key, sig?)` | `Drive.getPresence(k).chunksPresent > 0` → fallback `stat !== null` | same | post-hydrate; `getKeyStatus()` reports `'encrypted-unlocked'` for present-and-decryptable |
|
|
| `list(prefix, opts)`| `Drive.list(prefix, opts)` → re-serialised via `UriPlugin.build` | same | post-hydrate |
|
|
| `createReadStream(k, { offset?, length?, signal? })` | `Drive.createReadStream(k, opts)` | same; chunks-plugin honours `ctx.range`; mesh pivots between `ChunkRequest.specific` (≤64 chunks) and `bitset` (>64) | post-hydrate |
|
|
| `createWriteStream(k, body, sig?)` | pipe `body` into `Drive.createWriteStream(k, sig)` | same | throw `TEMPLATE_READ_ONLY` |
|
|
| `clear()` | throw `WRITE_UNSUPPORTED` (footgun) | same | same |
|
|
|
|
## Error code registry — 14 codes, every code reachable
|
|
|
|
| Code | Call site |
|
|
|-------------------------------|---------------------------------------------------------------------------------------------|
|
|
| `INVALID_URI` | every plugin's `parse` (malformed authority/path/scheme). |
|
|
| `DRIVE_ID_MISMATCH` | `driveUriPlugin.parse` when URI authority ≠ `options.driveId`. |
|
|
| `TEMPLATE_NOT_AVAILABLE` | `driveTemplateUriPlugin.beforeOp` when `templateProvider.fetchTemplate` rejects. |
|
|
| `TEMPLATE_INVALID` | `driveTemplateUriPlugin.beforeOp` when `Drive.hydrateFrom` rejects. |
|
|
| `TEMPLATE_READ_ONLY` | `driveTemplateUriPlugin` adapter's `set`/`delete`/`createWriteStream`. |
|
|
| `WRITE_UNSUPPORTED` | `DriveStorageLikeAdapter.clear()` only (narrowed). |
|
|
| `CAPABILITY_MISSING` | construction-time validation: missing `drive`/`driveId`/`templateProvider`/`advertise` hook.|
|
|
| `ABORTED` | pre-aborted `AbortSignal` short-circuit on any op; `?signal-timeout-ms` upper bound elapsed. |
|
|
| `RANGE_INVALID` | adapter `createReadStream` when `Drive.createReadStream` throws `DRIVE.RANGE_INVALID`. |
|
|
| `RANGE_OUT_OF_BOUNDS` | adapter `createReadStream` when `Drive.createReadStream` throws `DRIVE.RANGE_OUT_OF_BOUNDS`. |
|
|
| `MESH_PROPAGATION_TIMEOUT` | `driveMeshUriPlugin` adapter when opt-in advertise hook rejects or exceeds its budget. |
|
|
| `STAT_UNAVAILABLE` | adapter `getMetadata` when `Drive.stat` cannot answer for a key the URI surface needs. |
|
|
| `LIST_UNSUPPORTED` | adapter `list` when the bound `Drive`'s storage lacks `list`. |
|
|
| `INVALID_QUERY` | query parser when an unknown key or malformed value (`?advertise=maybe`) is encountered. |
|
|
|
|
Every code lives in `defineCodes("DRIVE_URI", [...] as const)`. No code is "declared but unused" (CLAUDE.md §2c).
|
|
|
|
## AbortSignal threading model
|
|
|
|
The `StorageLike` interface (current shape on `@suaveplan/types`) does not declare `signal` on `get`/`set`/`delete`/`exists`/`list`. The drive-uri adapter widens those signatures as a tolerated extension and forwards to the corresponding `Drive` methods, all of which already accept `signal?` (via `ReadOptions`, `WriteOptions`, the bare arg, etc.). The widening is documented on `drive-storage-like.md` and is opaque to the resolver — `StorageUriResolver` already passes its own signal into the buffered `get`/`set`/`delete` paths today; the adapter simply gains the ability to honour it.
|
|
|
|
Pre-aborted signal handling: every op checks `signal?.aborted === true` BEFORE any `Drive` call; on hit, throws `DriveUriError(DRIVE_URI.ABORTED, …, { scheme, path: key })`. The check sits in `InstrumentedDriveAdapter` so all three plugins inherit it.
|
|
|
|
`?signal-timeout-ms=<int>` parses to a per-op `AbortSignal.timeout(ms)` linked with the caller signal via `AbortSignal.any([callerSignal, timeoutSignal])`. On elapse, the op rejects with `DRIVE_URI.ABORTED` and a `reason: "signal-timeout-ms"` discriminant.
|
|
|
|
## Range threading model
|
|
|
|
```text
|
|
resolver.open(uri, { offset, length, signal })
|
|
→ openViaPlugin(plugin, uri, key, { offset, length, signal }) # storage-uri (existing)
|
|
→ plugin.adapter.createReadStream(key, { offset, length, signal })
|
|
→ Drive.createReadStream(path, { offset, length, signal }) # NEW overload
|
|
→ file-operations/streams.ts → defaultCreateReadStream / native pass-through
|
|
→ DriveStorage.createReadStream(path, { offset, length, signal }) if storage.capabilities.nativeRange
|
|
→ OR slicing transform around defaultCreateReadStream
|
|
→ middleware ctx.range = { offset, length } # populated by Drive
|
|
→ chunksPlugin's read middleware reads ctx.range
|
|
→ chunksPlugin.#reassemble(fileId, range) emits only chunks ∩ [offset, end)
|
|
→ over mesh: parallelPuller(range) builds ChunkRequest.specific (span ≤ 64) | bitset (span > 64)
|
|
```
|
|
|
|
`StorageUriResolver.open(uri, { offset, length, signal })` reaches the adapter's `createReadStream` via the existing `openViaPlugin` indirection. The resolver already calls `adapter.createReadStream(key, signal)`; we widen it to accept an options bag (matches the `StreamingStorageExtensions` interface signature, which is already `signal?: AbortSignal` — extended in lock-step to `signal | { offset?, length?, signal? }` per the same `signalOrOptions` discriminator used inside `Drive`).
|
|
|
|
## Mesh re-advertise opt-in
|
|
|
|
Default behaviour (no `?advertise`): write completes when local persistence is durable; no gossip; `drive_uri.mesh_write_local_only` span event records the contract holds. This matches direct-`Drive` behaviour today.
|
|
|
|
Opt-in (`?advertise=true`): after the local write resolves, the plugin invokes `options.advertise.advertise(fileId, bitset, totalChunks, signal)`. The hook is supplied by the consumer at construction; its absence at `parse` time when `?advertise=true` is requested raises `DRIVE_URI.CAPABILITY_MISSING` with `reason: "advertise"`. The hook's rejection (any throw, including timeout) is re-wrapped as `DRIVE_URI.MESH_PROPAGATION_TIMEOUT` with the original on `cause`.
|
|
|
|
`fileId`, `bitset`, and `totalChunks` are sourced from the freshly-written file: `fileId` from the chunks-plugin via `drive.getStorage("default")` lookup (the chunks plugin's `pathToFileId` map is internal — we surface it via a structural `ChunkSource`-bearing hook), `bitset` and `totalChunks` from the plugin's manifest. Consumers wiring `?advertise=true` MUST be running with `chunksPlugin`; if not, the hook hand-rolls the values from `Drive.stat`. The `drive-uri` plugin holds no chunks-plugin dependency — the lookup goes through a structural duck-type.
|
|
|
|
`?advertise=false` (explicit): identical to the default; documented so the URI is self-describing.
|
|
|
|
`?advertise` on `drive://` or `drive+template://`: parse error → `DRIVE_URI.INVALID_QUERY`.
|
|
|
|
## Query-string parsing & redaction
|
|
|
|
Parser: every plugin's `parse` invokes `parseQuery(parts.query)` (existing `parts.query` on `ParsedUri` from `storage-uri`'s `parseUri`). Recognised keys per scheme:
|
|
|
|
| Scheme | Recognised keys |
|
|
|---------------------|------------------------------------------------|
|
|
| `drive://` | `signal-timeout-ms` |
|
|
| `drive+mesh://` | `signal-timeout-ms`, `advertise` |
|
|
| `drive+template://` | `signal-timeout-ms`, `nonce` |
|
|
|
|
Unknown keys raise `DRIVE_URI.INVALID_QUERY` with `key` and `value` on the error.
|
|
|
|
Redaction: `redactUri` (already strips `?` and `#`) is applied to:
|
|
|
|
- every `drive.uri` span attribute,
|
|
- every emitted EventBus event's `uri` field,
|
|
- every `DriveUriError.uri` field carried in serialised wire-format errors (`SuaveplanError.toWire()` adapter).
|
|
|
|
Sensitive keys (`nonce`, `auth=…` future) are NEVER stored in cleartext on a span, log, or event. The parsed value is held in process memory only for the duration of the dispatched call.
|
|
|
|
## EventBus emission
|
|
|
|
The wrapped `Drive`'s `events: EventBusLike | undefined` is used. When present:
|
|
|
|
| Topic | Payload |
|
|
|----------------------|--------------------------------------------------------------------------------------------------------|
|
|
| `drive_uri:write` | `{ scheme, key, uri (redacted), outcome: "success"|"error", code?, durationMs, fileId?, range? }` |
|
|
| `drive_uri:delete` | `{ scheme, key, uri (redacted), outcome, code?, durationMs, fileId? }` |
|
|
| `drive_uri:mesh_advertised` | `{ scheme: "drive+mesh", fileId, totalChunks, durationMs }` — emitted on successful opt-in advertise |
|
|
|
|
Downstream consumers (`@suaveplan/vault-sync`, `drive-sharing`, future audit-log) subscribe at the bus level. Absence of `Drive.events` (the default) is a silent no-op.
|
|
|
|
## File layout
|
|
|
|
```text
|
|
packages/drive/drive-uri/src/
|
|
adapter/
|
|
drive-storage-like.ts # MODIFIED — every op gains signal?; new createWriteStream + getMetadata
|
|
drive-storage-like.md # MODIFIED
|
|
drive-storage-like.test.ts # MODIFIED — abort threading, write-stream, getMetadata
|
|
error-codes/
|
|
error-codes.ts # MODIFIED — 14 codes
|
|
error-codes.md # MODIFIED
|
|
error-codes.test.ts # MODIFIED — exhaustive registry shape + reachability assertions
|
|
events/
|
|
events.ts # ADDED — typed topic constants + emit helper
|
|
events.md # ADDED
|
|
events.test.ts # ADDED
|
|
query/
|
|
parse-query.ts # ADDED — per-scheme query parser
|
|
parse-query.md # ADDED
|
|
parse-query.test.ts # ADDED
|
|
otel/
|
|
otel.ts # MODIFIED — range attr, advertise attr, chunks histogram
|
|
otel.md # MODIFIED
|
|
otel.test.ts # MODIFIED
|
|
plugins/
|
|
drive/
|
|
drive.ts # MODIFIED — query parsing wired in
|
|
drive.streams.ts # MODIFIED — per-scheme set/delete/writeStream dispatch
|
|
drive.streams.md # MODIFIED
|
|
drive.test.ts # MODIFIED — full op matrix
|
|
drive.streams.test.ts # MODIFIED — range + signal + writeStream
|
|
drive-mesh/
|
|
drive-mesh.ts # MODIFIED — ?advertise opt-in via MeshAdvertiseHook
|
|
drive-mesh.md # MODIFIED
|
|
drive-mesh.test.ts # MODIFIED — advertise spy, full op matrix, query parsing
|
|
drive-template/
|
|
drive-template.ts # MODIFIED — set/delete/writeStream throw TEMPLATE_READ_ONLY; ?nonce forwarded
|
|
drive-template.md # MODIFIED
|
|
drive-template.test.ts # MODIFIED — full op matrix, nonce forwarding
|
|
types/
|
|
plugin-options.types.ts # MODIFIED — MeshAdvertiseHook, widened TemplateProvider signature
|
|
plugin-options.types.md # MODIFIED
|
|
conformance/
|
|
conformance.test.ts # MODIFIED — full read/write/delete/stat/list/exists/range matrix
|
|
tests/integration/
|
|
drive-roundtrip-write.test.ts # ADDED
|
|
drive-mesh-roundtrip-write.test.ts # ADDED — advertise opt-in spy
|
|
template-write-rejection.test.ts # ADDED
|
|
drive-range-roundtrip.test.ts # ADDED — read bytes [60_000, 70_000) of a 200 KiB file
|
|
drive-stat-list-exists.test.ts # ADDED — full read-side matrix on a real Drive
|
|
|
|
packages/drive/drive/src/
|
|
drive/drive.ts # MODIFIED — createReadStream overload, range plumbing
|
|
file-operations/streams.ts # MODIFIED — overload signatures, slicing transform
|
|
file-operations/streams.range.ts # ADDED — pure slicing transform
|
|
file-operations/streams.range.md # ADDED
|
|
file-operations/streams.range.test.ts # ADDED — boundary table
|
|
types/drive.types.ts # MODIFIED — RangeReadOptions, DriveStorage.capabilities.nativeRange
|
|
error-codes.ts # MODIFIED — RANGE_INVALID, RANGE_OUT_OF_BOUNDS
|
|
```
|
|
|
|
Every file stays inside the 300-line source / 500-line test budgets. The largest existing file (`drive.streams.ts` at 148 lines) gains ~80 lines of per-scheme dispatch — under budget.
|
|
|
|
## Trade-offs considered
|
|
|
|
- **One change vs. two.** Combining the two narrow follow-ups removes a redundant conformance / docs / dependent cycle and lets the package land at a coherent surface. Cost: a larger PR. Mitigation: the per-scheme matrix is the same in either factoring; the file count is identical; verification scope only widens by the integration tests that would have been added in either change separately.
|
|
- **Per-scheme `if`-branch vs. polymorphic subclass for `set`/`delete`/`createWriteStream`.** Chose per-scheme branch — only one scheme has the constraint and a subclass would duplicate the instrumented method body. If a fourth scheme appears, revisit.
|
|
- **Auto-advertise vs. opt-in on mesh write.** Opt-in via `?advertise=true`. Auto-advertise would surprise consumers managing `AdvertiseThrottle`/`GenerationTracker` lifecycles themselves; the opt-in surfaces the contract on the URI itself (self-describing).
|
|
- **`MeshAdvertiseHook` in `drive-uri` vs. importing `@suaveplan/drive-mesh`.** Hook lives in `drive-uri`. Importing the mesh package would couple the URI tier to a concrete transport and pull a heavy graph into a leaf-ish adapter.
|
|
- **EventBus on `Drive.events` vs. dedicated bus in `drive-uri`.** Reuse `Drive.events`. A dedicated bus would force two subscriptions for every consumer that already listens to `Drive`-tier events.
|
|
- **`?signal-timeout-ms` per-op vs. construction-time.** Per-op (URI carries it). Construction-time would force one timeout policy across every URI a plugin handles; per-op lets callers vary by request.
|
|
- **Reusing `WRITE_UNSUPPORTED` for templates vs. distinct `TEMPLATE_READ_ONLY`.** Distinct. `WRITE_UNSUPPORTED` was a *temporary* state code; templates are *permanently* read-only. Distinct codes let consumers branch correctly.
|
|
- **`{ offset, length }` vs. `{ start, end }` for range.** `offset`/`length` — matches HTTP `Range: bytes=offset-(offset+length-1)` and avoids the inclusive-vs-exclusive ambiguity that bites every range API at least once.
|
|
- **Throw vs. clamp on range overflow.** Throw `RANGE_OUT_OF_BOUNDS`. Clamping hides upstream bugs; throwing matches the rest of `DRIVE_*` / `DRIVE_URI.*` explicit-failure style.
|
|
|
|
## Cross-reference
|
|
|
|
This change consolidates the two superseded deferrals: `_superseded-narrow-deferrals-2026-05-17/add-drive-uri-writes` and `…/add-drive-range-reads`. The reasoning and design notes from both are folded into this single change; the superseded folders are kept as research history.
|
|
|
|
`DriveLike` hoist to `@suaveplan/types` is an independent change (`hoist-drivelike-to-types`) — landing order is irrelevant; this change consumes the structural shape from whichever has merged first.
|