# Add byte-range reads to Drive.createReadStream and `drive*://` URI plugins **Packages**: `@suaveplan/drive`, `@suaveplan/drive-uri` **Status**: PROPOSED **Type**: capability-extension **Defers from**: `openspec/archive/2026-05-16-add-drive-uri/proposal.md` § "Out of scope → Range reads on `drive+mesh://`" - **Design**: ./design.md - **Spec deltas**: - ./specs/drive/spec.md - ./specs/drive-uri/spec.md ## Why `add-drive-uri` shipped `resolver.read("drive+mesh://swarm-x/clip.mov")` as a whole-file fetch only, explicitly carving out byte-range reads: > **Range reads on `drive+mesh://`.** `drive.createReadStream` already streams chunk-by-chunk via the chunks plugin; byte-range slicing on top requires a future `Drive.createReadStream({ offset, length })` extension — tracked separately. Range reads are the single most-asked-for missing capability for the drive surface — every consumer that wants HTTP `Range:` header support (audio/video seek, virtual scrolling over chunked logs, partial-image decode, resume-on-reconnect, S3-style multipart staging) is currently forced to fetch the full file and slice client-side. Over `drive+mesh://` this is catastrophic: a 4 GiB media file pulled to seek the last 1 MiB downloads the entire swarm payload. The chunked layout in `drive-plugin-chunks` already addresses every chunk by `(fileId, chunkId)` with a known fixed `chunkSize` per manifest, so range → chunk math is deterministic; the missing piece is plumbing the offset/length pair from `Drive.createReadStream` through the chunks-plugin middleware so only the chunks intersecting the range are fetched, decrypted, and emitted. This change adds the `{ offset, length }` options to `Drive.createReadStream`, extends the chunks plugin's `#reassemble` to honour them (computing first/last chunk indices, slicing the boundary chunks), threads range hints through the `drive-uri` adapters, and — over `drive+mesh://` — emits a `ChunkRequest.specific` (or `bitset` for large contiguous ranges) listing only the in-range chunkIds rather than `ChunkRequest.whole`. The local-storage and template paths get the same offset/length math for free because they go through the same chunks-plugin middleware. ## What changes ### `@suaveplan/drive` - MODIFIED: `Drive.createReadStream(path, signal?)` → `Drive.createReadStream(path, options?: RangeReadOptions | AbortSignal)`. The new positional-options overload accepts `{ offset?: number; length?: number; signal?: AbortSignal }`. The bare-`AbortSignal` form is preserved via overload so all existing call sites compile unchanged. - ADDED: `RangeReadOptions` interface in `src/types/drive.types.ts` — `offset` defaults to `0`, `length` defaults to "to EOF". Both MUST be non-negative integers when present; violations raise `DRIVE.RANGE_INVALID` synchronously (before stream construction). - ADDED: `DRIVE.RANGE_INVALID` and `DRIVE.RANGE_OUT_OF_BOUNDS` codes in `DRIVE_CODES`. `RANGE_INVALID` covers negative / non-integer / NaN inputs. `RANGE_OUT_OF_BOUNDS` covers `offset > size` and `offset + length > size` where `size` is known at stream-start (e.g. via the chunks-plugin manifest); when size is unknown at start-time, the stream end-event MUST surface `RANGE_OUT_OF_BOUNDS` if the underlying source closed before `length` bytes were emitted. - MODIFIED: `defaultCreateReadStream` and the storage-capability passthrough in `src/file-operations/streams.ts` honour `RangeReadOptions`. The default buffered path slices the loaded `Uint8Array`; the native-streams path forwards `{ offset, length }` to `storage.createReadStream` when `storage.capabilities?.nativeRange === true`, otherwise wraps the native stream in a slicing transform that drops bytes before `offset` and stops after `length`. - ADDED: `DriveStorage` capability flag `capabilities.nativeRange?: boolean` and an extended `createReadStream(path, options?: { offset?: number; length?: number; signal?: AbortSignal })` signature on the storage layer. Storages that already accept a range (e.g. file with `fs.createReadStream({ start, end })`) opt in by setting the flag; existing storages are unaffected. - ADDED: Middleware context carries `range?: { offset: number; length?: number }`. The chunks-plugin middleware reads it and short-circuits its `#reassemble` to the intersecting chunk window. ### `@suaveplan/drive-plugin-chunks` - MODIFIED: `ChunksPluginHandle.#reassemble(fileId, range?)` accepts an optional range and emits only the chunks intersecting `[offset, offset + length)`. First and last chunks are sliced; interior chunks are emitted whole. Algorithm in `design.md` § "Range → chunk math". - MODIFIED: The `read` middleware forwards `ctx.range` (when present) to `#reassemble` and to a sibling `#reassembleAsStream` used by `createReadStream`. The middleware's `read(path)` buffered call buffers only the in-range bytes. - ADDED: `CHUNKS.RANGE_OUT_OF_BOUNDS` code — raised when the requested range exceeds `manifest.fileSize`. Surfaced on stream `error`. ### `@suaveplan/drive-mesh-protocol` - MODIFIED: `drive-mesh`'s pull path (consumer of this package; no protocol wire change) constructs `ChunkRequest.specific { chunkIds }` for ranges spanning ≤ 64 chunks and `ChunkRequest.bitset { bytes }` for larger contiguous spans. No new wire fields — both forms already exist in `ChunkRequestPayload`. - No `DMP_*` registry changes. ### `@suaveplan/drive-uri` - MODIFIED: `DriveStorageLikeAdapter.createReadStream(key, options?)` accepts and forwards `{ offset?, length?, signal? }`. The three plugins (`driveUriPlugin`, `driveMeshUriPlugin`, `driveTemplateUriPlugin`) advertise `capabilities.nativeRange = true` on their adapter so `StorageUriResolver` issues a single range-aware `createReadStream` call instead of a whole-file `get` + client-side slice. - ADDED: `DRIVE_URI.RANGE_UNSUPPORTED` is **not** introduced — every `drive*://` URI now supports ranges because the underlying chunks plugin does. The resolver surfaces upstream `DRIVE.RANGE_INVALID` / `DRIVE.RANGE_OUT_OF_BOUNDS` directly under the existing pass-through rule. - ADDED: OTel attribute `drive.uri.range = "{offset}-{end}"` on `createReadStream` spans (omitted when no range supplied). Histogram `drive_uri.range_chunks_fetched` tags `{scheme}` and records the count of chunks actually pulled — measurable savings vs. whole-file fetch. ## Out of scope - **Multi-range reads** (RFC 7233 `multipart/byteranges`). One range per call. A caller wanting N disjoint ranges issues N `createReadStream` calls. - **`Drive.read(path, { offset, length })`** buffered range read. Achievable today via `createReadStream({ offset, length })` + buffer-collect; adding it to the buffered path would duplicate the chunk-math without changing observable behaviour. Hoist if a real consumer needs it. - **Range writes / partial uploads.** Writes remain whole-file. Chunk-level patch semantics belong in a future `add-drive-patch` change. - **Range hints on `drive`-tier `read()`** for non-chunked storages without `nativeRange`. Falls back to the slicing transform; performance characteristics are documented but no new code path is added. - **Backpressure-aware peer scheduling** for large bitset requests. The mesh plugin's existing `ParallelPuller` window applies unchanged; range simply narrows the chunk set it pulls from. ## Source mapping This is a **genesis-original** extension; deferral source is `openspec/archive/2026-05-16-add-drive-uri/proposal.md`. Surface anchors: - `Drive.createReadStream` — `packages/drive/drive/src/drive/drive.ts:222-230`, `packages/drive/drive/src/file-operations/streams.ts`. - Chunks plugin `#reassemble` — `packages/drive/drive-plugin-chunks/src/plugin.ts:218-251`; chunkSize lives on the manifest (`packages/drive/drive-plugin-chunks/src/manifest/build.ts:11,30,74`), default 65536 (`packages/drive/drive-plugin-chunks/src/plugin.ts:78`). - `ChunkRequest` union — `packages/drive/drive-mesh-protocol/src/types/protocol.types.ts:38-42`; both `specific` and `bitset` already encodable. - `drive-uri` adapter — `packages/drive/drive-uri/src/plugins/drive-mesh/drive-mesh.ts` extends `InstrumentedDriveAdapter`. ### Requirements with no source line | Requirement | Justification | |---|---| | Overload `createReadStream(path, signal)` AND `createReadStream(path, options)` rather than breaking the single-signal form | Every existing consumer in the monorepo (chunks plugin, drive-uri adapters, drive integration tests) passes the bare `AbortSignal`. A breaking signature change for an additive feature is unjustified when overloads cost nothing. | | Switch `ChunkRequest.specific` ↔ `ChunkRequest.bitset` at 64 chunks | `MAX_CHUNKS_DEFAULT = 64` in `drive-mesh-protocol`; beyond that, the JSON `chunkIds` array out-weighs a bitset that addresses every chunk in two bytes per 16. The 64-chunk pivot matches the wire-cap and minimises bytes-on-wire for the common case (4 MiB range over 64 KiB chunks). | | `RANGE_OUT_OF_BOUNDS` may fire at start OR at stream-end | When `size` is known (chunks-plugin manifest loaded, storage `stat` populated), reject synchronously. When unknown (raw `defaultCreateReadStream` over a non-stat-able storage), accept the range, emit what's available, then surface the code on `close` if shortfall is detected. Two paths because the size-known/size-unknown split is a real storage capability variance, not a defect. | | Zero-length range is a valid request | `length: 0` returns an immediately-closed empty stream. Useful as a probe-for-existence pattern; rejecting it would surprise consumers who compute ranges from upstream API responses. | | `nativeRange` storage capability flag rather than always-slicing in drive | `bun:sqlite`-backed and `fs`-backed storages can deliver true range reads with no read-amplification. Always wrapping in a slicing transform would discard that win for the storages that have it. The flag is opt-in so adding it is purely additive on every existing storage. | ## Migration This is a purely additive change for every existing call site: - **Old**: `drive.createReadStream("file.bin", abortSignal)` → still compiles, still returns whole-file stream. - **New**: `drive.createReadStream("file.bin", { offset: 1024, length: 4096, signal: abortSignal })` → emits exactly bytes [1024, 5120). Over `drive+mesh://`, requests at most ⌈(4096 + 1024 % chunkSize) / chunkSize⌉ chunks from peers. No consumer must change. Consumers that want range behaviour migrate at their leisure. Search command for opt-in adoption: ```bash grep -rn "createReadStream" packages/ | grep -v "/dist/\|/coverage/" ``` Recommended replacement when range-aware: pass `{ offset, length, signal }` instead of the bare `signal`. ## Impact ### Affected dependents Run `bunx turbo ls --filter=...@suaveplan/drive` to enumerate. The blast radius is "every package that calls `Drive.createReadStream`," which today is: - `@suaveplan/drive-uri` — [x] tests run / [x] docs reviewed / [ ] no action needed (adapter `createReadStream` extended to forward range; new requirement added to spec delta) - `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed / [ ] no action needed (middleware honours range; `#reassemble` extended) - `@suaveplan/drive-mesh` — [x] tests run / [x] docs reviewed / [ ] no action needed (pull path constructs `ChunkRequest.specific` / `bitset` from range) - `@suaveplan/drive-transport-https` — [ ] tests run / [x] docs reviewed / [x] no action needed (transport carries chunk envelopes opaquely; no surface change) - `@suaveplan/drive-transport-webrtc` — [ ] tests run / [x] docs reviewed / [x] no action needed (same as above) - `@suaveplan/drive-template` — [ ] tests run / [x] docs reviewed / [x] no action needed (hydration unchanged; range applies post-hydration) - `@suaveplan/drive-sharing` — [ ] tests run / [x] docs reviewed / [x] no action needed (calls `Drive.read`, not `createReadStream`) - `@suaveplan/drive-webrtc` — [ ] tests run / [x] docs reviewed / [x] no action needed (browser-tier transport, unaffected) ### Affected upstreams (none modified) | Package | Action | Breaking? | |---|---|---| | `@suaveplan/storage` | Optional: extend `AbstractStorage.createReadStream` with `{ offset, length }` options. Existing storages opt in via `capabilities.nativeRange`. | no | | `@suaveplan/storage-uri` | None — already routes `createReadStream` through `UriPlugin.adapter`. | no | | `@suaveplan/types` | None — `RangeReadOptions` lives in `@suaveplan/drive` until a second consumer emerges. | no | ### Ordering Unblocked. All packages exist on genesis today; the change is additive.