suaveplan/openspec/changes/_superseded-narrow-deferrals-2026-05-17/add-drive-range-reads/proposal.md

Add byte-range reads to Drive.createReadStream and drive*:// URI plugins

Packages: @suaveplan/drive, @suaveplan/drive-uri Status: PROPOSED Type: capability-extension Defers from: openspec/archive/2026-05-16-add-drive-uri/proposal.md § "Out of scope → Range reads on drive+mesh://"

• Design: ./design.md
• Spec deltas:
  • ./specs/drive/spec.md
  • ./specs/drive-uri/spec.md

Why

add-drive-uri shipped resolver.read("drive+mesh://swarm-x/clip.mov") as a whole-file fetch only, explicitly carving out byte-range reads:

Range reads on drive+mesh://. drive.createReadStream already streams chunk-by-chunk via the chunks plugin; byte-range slicing on top requires a future Drive.createReadStream({ offset, length }) extension — tracked separately.

Range reads are the single most-asked-for missing capability for the drive surface — every consumer that wants HTTP Range: header support (audio/video seek, virtual scrolling over chunked logs, partial-image decode, resume-on-reconnect, S3-style multipart staging) is currently forced to fetch the full file and slice client-side. Over drive+mesh:// this is catastrophic: a 4 GiB media file pulled to seek the last 1 MiB downloads the entire swarm payload. The chunked layout in drive-plugin-chunks already addresses every chunk by (fileId, chunkId) with a known fixed chunkSize per manifest, so range → chunk math is deterministic; the missing piece is plumbing the offset/length pair from Drive.createReadStream through the chunks-plugin middleware so only the chunks intersecting the range are fetched, decrypted, and emitted.

This change adds the { offset, length } options to Drive.createReadStream, extends the chunks plugin's #reassemble to honour them (computing first/last chunk indices, slicing the boundary chunks), threads range hints through the drive-uri adapters, and — over drive+mesh:// — emits a ChunkRequest.specific (or bitset for large contiguous ranges) listing only the in-range chunkIds rather than ChunkRequest.whole. The local-storage and template paths get the same offset/length math for free because they go through the same chunks-plugin middleware.

What changes

@suaveplan/drive

• MODIFIED: Drive.createReadStream(path, signal?) → Drive.createReadStream(path, options?: RangeReadOptions | AbortSignal). The new positional-options overload accepts { offset?: number; length?: number; signal?: AbortSignal }. The bare-AbortSignal form is preserved via overload so all existing call sites compile unchanged.
• ADDED: RangeReadOptions interface in src/types/drive.types.ts — offset defaults to 0, length defaults to "to EOF". Both MUST be non-negative integers when present; violations raise DRIVE.RANGE_INVALID synchronously (before stream construction).
• ADDED: DRIVE.RANGE_INVALID and DRIVE.RANGE_OUT_OF_BOUNDS codes in DRIVE_CODES. RANGE_INVALID covers negative / non-integer / NaN inputs. RANGE_OUT_OF_BOUNDS covers offset > size and offset + length > size where size is known at stream-start (e.g. via the chunks-plugin manifest); when size is unknown at start-time, the stream end-event MUST surface RANGE_OUT_OF_BOUNDS if the underlying source closed before length bytes were emitted.
• MODIFIED: defaultCreateReadStream and the storage-capability passthrough in src/file-operations/streams.ts honour RangeReadOptions. The default buffered path slices the loaded Uint8Array; the native-streams path forwards { offset, length } to storage.createReadStream when storage.capabilities?.nativeRange === true, otherwise wraps the native stream in a slicing transform that drops bytes before offset and stops after length.
• ADDED: DriveStorage capability flag capabilities.nativeRange?: boolean and an extended createReadStream(path, options?: { offset?: number; length?: number; signal?: AbortSignal }) signature on the storage layer. Storages that already accept a range (e.g. file with fs.createReadStream({ start, end })) opt in by setting the flag; existing storages are unaffected.
• ADDED: Middleware context carries range?: { offset: number; length?: number }. The chunks-plugin middleware reads it and short-circuits its #reassemble to the intersecting chunk window.

@suaveplan/drive-plugin-chunks

• MODIFIED: ChunksPluginHandle.#reassemble(fileId, range?) accepts an optional range and emits only the chunks intersecting [offset, offset + length). First and last chunks are sliced; interior chunks are emitted whole. Algorithm in design.md § "Range → chunk math".
• MODIFIED: The read middleware forwards ctx.range (when present) to #reassemble and to a sibling #reassembleAsStream used by createReadStream. The middleware's read(path) buffered call buffers only the in-range bytes.
• ADDED: CHUNKS.RANGE_OUT_OF_BOUNDS code — raised when the requested range exceeds manifest.fileSize. Surfaced on stream error.

@suaveplan/drive-mesh-protocol

• MODIFIED: drive-mesh's pull path (consumer of this package; no protocol wire change) constructs ChunkRequest.specific { chunkIds } for ranges spanning ≤ 64 chunks and ChunkRequest.bitset { bytes } for larger contiguous spans. No new wire fields — both forms already exist in ChunkRequestPayload.
• No DMP_* registry changes.

@suaveplan/drive-uri

• MODIFIED: DriveStorageLikeAdapter.createReadStream(key, options?) accepts and forwards { offset?, length?, signal? }. The three plugins (driveUriPlugin, driveMeshUriPlugin, driveTemplateUriPlugin) advertise capabilities.nativeRange = true on their adapter so StorageUriResolver issues a single range-aware createReadStream call instead of a whole-file get + client-side slice.
• ADDED: DRIVE_URI.RANGE_UNSUPPORTED is not introduced — every drive*:// URI now supports ranges because the underlying chunks plugin does. The resolver surfaces upstream DRIVE.RANGE_INVALID / DRIVE.RANGE_OUT_OF_BOUNDS directly under the existing pass-through rule.
• ADDED: OTel attribute drive.uri.range = "{offset}-{end}" on createReadStream spans (omitted when no range supplied). Histogram drive_uri.range_chunks_fetched tags {scheme} and records the count of chunks actually pulled — measurable savings vs. whole-file fetch.

Out of scope

• Multi-range reads (RFC 7233 multipart/byteranges). One range per call. A caller wanting N disjoint ranges issues N createReadStream calls.
• Drive.read(path, { offset, length }) buffered range read. Achievable today via createReadStream({ offset, length }) + buffer-collect; adding it to the buffered path would duplicate the chunk-math without changing observable behaviour. Hoist if a real consumer needs it.
• Range writes / partial uploads. Writes remain whole-file. Chunk-level patch semantics belong in a future add-drive-patch change.
• Range hints on drive-tier read() for non-chunked storages without nativeRange. Falls back to the slicing transform; performance characteristics are documented but no new code path is added.
• Backpressure-aware peer scheduling for large bitset requests. The mesh plugin's existing ParallelPuller window applies unchanged; range simply narrows the chunk set it pulls from.

Source mapping

This is a genesis-original extension; deferral source is openspec/archive/2026-05-16-add-drive-uri/proposal.md. Surface anchors:

• Drive.createReadStream — packages/drive/drive/src/drive/drive.ts:222-230, packages/drive/drive/src/file-operations/streams.ts.
• Chunks plugin #reassemble — packages/drive/drive-plugin-chunks/src/plugin.ts:218-251; chunkSize lives on the manifest (packages/drive/drive-plugin-chunks/src/manifest/build.ts:11,30,74), default 65536 (packages/drive/drive-plugin-chunks/src/plugin.ts:78).
• ChunkRequest union — packages/drive/drive-mesh-protocol/src/types/protocol.types.ts:38-42; both specific and bitset already encodable.
• drive-uri adapter — packages/drive/drive-uri/src/plugins/drive-mesh/drive-mesh.ts extends InstrumentedDriveAdapter.

Requirements with no source line

Requirement	Justification
Overload createReadStream(path, signal) AND createReadStream(path, options) rather than breaking the single-signal form	Every existing consumer in the monorepo (chunks plugin, drive-uri adapters, drive integration tests) passes the bare AbortSignal. A breaking signature change for an additive feature is unjustified when overloads cost nothing.
Switch ChunkRequest.specific ↔ ChunkRequest.bitset at 64 chunks	MAX_CHUNKS_DEFAULT = 64 in drive-mesh-protocol; beyond that, the JSON chunkIds array out-weighs a bitset that addresses every chunk in two bytes per 16. The 64-chunk pivot matches the wire-cap and minimises bytes-on-wire for the common case (4 MiB range over 64 KiB chunks).
RANGE_OUT_OF_BOUNDS may fire at start OR at stream-end	When size is known (chunks-plugin manifest loaded, storage stat populated), reject synchronously. When unknown (raw defaultCreateReadStream over a non-stat-able storage), accept the range, emit what's available, then surface the code on close if shortfall is detected. Two paths because the size-known/size-unknown split is a real storage capability variance, not a defect.
Zero-length range is a valid request	length: 0 returns an immediately-closed empty stream. Useful as a probe-for-existence pattern; rejecting it would surprise consumers who compute ranges from upstream API responses.
nativeRange storage capability flag rather than always-slicing in drive	bun:sqlite-backed and fs-backed storages can deliver true range reads with no read-amplification. Always wrapping in a slicing transform would discard that win for the storages that have it. The flag is opt-in so adding it is purely additive on every existing storage.

Migration

This is a purely additive change for every existing call site:

• Old: drive.createReadStream("file.bin", abortSignal) → still compiles, still returns whole-file stream.
• New: drive.createReadStream("file.bin", { offset: 1024, length: 4096, signal: abortSignal }) → emits exactly bytes [1024, 5120). Over drive+mesh://, requests at most ⌈(4096 + 1024 % chunkSize) / chunkSize⌉ chunks from peers.

No consumer must change. Consumers that want range behaviour migrate at their leisure. Search command for opt-in adoption:

grep -rn "createReadStream" packages/ | grep -v "/dist/\|/coverage/"

Recommended replacement when range-aware: pass { offset, length, signal } instead of the bare signal.

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/drive to enumerate. The blast radius is "every package that calls Drive.createReadStream," which today is:

• @suaveplan/drive-uri — [x] tests run / [x] docs reviewed / [ ] no action needed (adapter createReadStream extended to forward range; new requirement added to spec delta)
• @suaveplan/drive-plugin-chunks — [x] tests run / [x] docs reviewed / [ ] no action needed (middleware honours range; #reassemble extended)
• @suaveplan/drive-mesh — [x] tests run / [x] docs reviewed / [ ] no action needed (pull path constructs ChunkRequest.specific / bitset from range)
• @suaveplan/drive-transport-https — [ ] tests run / [x] docs reviewed / [x] no action needed (transport carries chunk envelopes opaquely; no surface change)
• @suaveplan/drive-transport-webrtc — [ ] tests run / [x] docs reviewed / [x] no action needed (same as above)
• @suaveplan/drive-template — [ ] tests run / [x] docs reviewed / [x] no action needed (hydration unchanged; range applies post-hydration)
• @suaveplan/drive-sharing — [ ] tests run / [x] docs reviewed / [x] no action needed (calls Drive.read, not createReadStream)
• @suaveplan/drive-webrtc — [ ] tests run / [x] docs reviewed / [x] no action needed (browser-tier transport, unaffected)

Affected upstreams (none modified)

Package	Action	Breaking?
@suaveplan/storage	Optional: extend AbstractStorage.createReadStream with { offset, length } options. Existing storages opt in via capabilities.nativeRange.	no
@suaveplan/storage-uri	None — already routes createReadStream through UriPlugin.adapter.	no
@suaveplan/types	None — RangeReadOptions lives in @suaveplan/drive until a second consumer emerges.	no

Ordering

Unblocked. All packages exist on genesis today; the change is additive.