Initial Commit
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-05-19
|
||||
131
openspec/changes/add-drive-storage-archive/design.md
Normal file
131
openspec/changes/add-drive-storage-archive/design.md
Normal file
@@ -0,0 +1,131 @@
|
||||
## Context
|
||||
|
||||
`@suaveplan/drive` consumes any object satisfying the structural `DriveStorage` view (read/write/delete/list/stat/streams + capabilities flags). Existing storage adapters (`storage-file`, `storage-http`, `storage-cdn-*`, `storage-browser-*`) all expose flat byte stores. None of them treat their content as a structured archive whose entries should themselves appear as paths in the drive.
|
||||
|
||||
Compressed archives have format-specific quirks that prevent a single monolithic adapter:
|
||||
|
||||
- **Container shape**: ZIP and 7z have a trailing central directory (random-access friendly); tar is purely sequential; gzip wraps a single stream with no directory at all.
|
||||
- **Compression coupling**: tar separates archiving from compression (`.tar.gz`, `.tar.xz`, `.tar.zst`); zip/7z couple them per-entry; rar couples them per-volume.
|
||||
- **Mutability**: zip and 7z support entry-level update via central-directory rewrite; tar requires full rewrite for deletes; gzip is single-stream and must be re-encoded entirely on any change; rar is patent-encumbered for writing.
|
||||
- **Splitting**: zip uses `.z01`..`.zNN`+`.zip`; rar uses `.partN.rar` (RAR5) or `.r00`..`.r99` (RAR4); 7z uses `.7z.NNN`. Each has a different sibling-discovery and ordering rule.
|
||||
- **Extension lies**: a `.zip` may be a 7z stream; a `.bin` may be tgz. The host can only trust magic bytes.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- One installable package per archive format (zip, tar, gz, tgz, rar, 7z) plus one shared `drive-storage-archive-core`. Each is a separately versioned library.
|
||||
- Each adapter implements `DriveStorage` so `new Drive(adapter)` works with zero drive-side changes.
|
||||
- Content-sniff format from magic bytes; extension is advisory only.
|
||||
- First-class split-archive support with sibling-volume resolution callback so the host storage (file, http, indexeddb) decides how to fetch siblings.
|
||||
- Three explicit modes: `read-only`, `copy-on-write` (in-memory overlay + `flush()`), `rewrite` (write-through via host storage).
|
||||
- Streaming reads (`ReadableStream<Uint8Array>`) for entries larger than memory.
|
||||
- Password/encryption hook so credentials never live in the adapter — host supplies via callback.
|
||||
- Conformance harness in `drive-storage-archive-core/testing` that every per-format adapter passes.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Writing RAR — patent-encumbered, deliberately read-only.
|
||||
- New compression formats outside the six listed (no `.lz4`, `.zst` standalone, `.bz2` standalone in this change — tgz codec slot will accept them later).
|
||||
- Replacing `drive-plugin-chunks` chunk addressing; archive entries are not chunked at the drive level.
|
||||
- Mounting archives inside archives transparently (nested mount is a host concern: `openArchiveStorage(openArchiveStorage(...))`).
|
||||
- A unified `@suaveplan/storage-archive` umbrella package. Each format ships standalone (matches user directive).
|
||||
|
||||
## Decisions
|
||||
|
||||
### One package per format, plus a shared core
|
||||
|
||||
**Choice:** Six per-format packages (`drive-storage-{zip,tar,gz,tgz,rar,7z}`) + `drive-storage-archive-core` for shared primitives.
|
||||
|
||||
**Alternative considered:** Single `drive-storage-archive` umbrella package with internal format modules.
|
||||
|
||||
**Rationale:** Per-format isolates large codec dependencies (LZMA SDK is ~150KB, RAR unpacker is server-only) and lets consumers depend only on what they need. Matches existing repo pattern (`storage-cdn-bunnycdn`, `storage-cdn-rackspace`, etc.). User explicitly requested separate libraries.
|
||||
|
||||
### Format detection via magic bytes, extension as hint only
|
||||
|
||||
**Choice:** `sniffFormat(bytes: Uint8Array): ArchiveFormat | null` in `drive-storage-archive-core` inspects the first 512 bytes against a magic-byte registry: `PK\x03\x04` (zip), `Rar!\x1A\x07\x00` / `Rar!\x1A\x07\x01\x00` (rar4/5), `7z\xBC\xAF\x27\x1C` (7z), `\x1F\x8B` (gzip), and content-based tar detection (block 0 offset 257 = `"ustar"` for POSIX, or full-block heuristic for legacy v7).
|
||||
|
||||
**Rationale:** Users routinely mislabel archives. Extension drives default codec selection but is overridden by sniff result with a logged `ARCHIVE_FORMAT_MISMATCH` warning.
|
||||
|
||||
### Split-volume resolution via sibling-resolver callback
|
||||
|
||||
**Choice:** `openArchiveStorage(source, { resolveSibling: (basename, index) => Promise<Uint8Array | ReadableStream> })`.
|
||||
|
||||
**Alternative considered:** Hard-code sibling lookup against the host storage path.
|
||||
|
||||
**Rationale:** Host storage might be HTTP (siblings on different URLs), IndexedDB (siblings in different rows), or a P2P drive (siblings discovered via gossip). The adapter never assumes a filesystem. Each format ships a `defaultResolver(hostStorage, primaryPath)` helper using its own naming convention (`name.z01..zNN`, `name.partN.rar`, `name.7z.NNN`).
|
||||
|
||||
### Three explicit mutation modes
|
||||
|
||||
**Choice:** `mode: 'read-only' | 'copy-on-write' | 'rewrite'`.
|
||||
|
||||
**Rationale:**
|
||||
|
||||
- `read-only` is the safe default. RAR forces this.
|
||||
- `copy-on-write` keeps the original archive bytes intact until `await adapter.flush()`. Mutations land in an in-memory overlay (entry → new bytes or tombstone). Suitable for batch edits.
|
||||
- `rewrite` writes through on every mutation by re-encoding the archive into the host storage. Suitable for occasional single edits. Each format documents the cost: O(archive size) per mutation.
|
||||
|
||||
### Streaming entries via Web Streams
|
||||
|
||||
**Choice:** `getEntryStream(path): ReadableStream<Uint8Array>` returns a Web Streams `ReadableStream`. Bulk reads via `getEntry(path): Promise<Uint8Array>` are a convenience wrapper.
|
||||
|
||||
**Rationale:** Archives often hold multi-GB entries (DB dumps, media). The `drive` `createReadStream`/`createWriteStream` operations already negotiate native vs. polyfill — this slots in cleanly.
|
||||
|
||||
### Codec abstraction for tgz family
|
||||
|
||||
**Choice:** `@suaveplan/drive-storage-tgz` accepts a `codec: ArchiveCodec` parameter. Ships with `gzipCodec` from `@suaveplan/drive-storage-gz`. Bzip2/xz/zstd codecs are pluggable (out of scope for this change; slot exists).
|
||||
|
||||
**Rationale:** Avoids forcing every consumer to install xz/zstd deps when they only need `.tar.gz`.
|
||||
|
||||
### Error taxonomy
|
||||
|
||||
**Choice:** Each package owns its `*_CODES` registry via `defineCodes` (`error` package convention). `drive-storage-archive-core` owns `ARCHIVE_FORMAT_UNKNOWN`, `ARCHIVE_FORMAT_MISMATCH`, `ARCHIVE_SPLIT_MISSING_VOLUME`, `ARCHIVE_PASSWORD_REQUIRED`, `ARCHIVE_PASSWORD_INVALID`, `ARCHIVE_MODE_READ_ONLY`, `ARCHIVE_FLUSH_REQUIRED`, `ARCHIVE_CORRUPT`, `ARCHIVE_UNSUPPORTED_FEATURE`, `ARCHIVE_CODEC_MISSING`. Per-format packages add format-specific codes (e.g., `ZIP_ZIP64_REQUIRED`, `TAR_PAX_HEADER_INVALID`, `RAR_VERSION_UNSUPPORTED`, `SEVENZ_BCJ_FILTER_UNSUPPORTED`).
|
||||
|
||||
### Universal vs. server tier
|
||||
|
||||
- `drive-storage-archive-core`, `drive-storage-zip`, `drive-storage-tar`, `drive-storage-gz`, `drive-storage-tgz` → **universal** (pure-TS codecs: `fflate` for deflate, hand-rolled tar block parser, pure-TS gzip via `fflate`).
|
||||
- `drive-storage-7z` → **universal** via wasm LZMA (preferred) or **server** via `lzma-native` binding (decided per perf test in tasks).
|
||||
- `drive-storage-rar` → **server-only** (no royalty-free pure-TS RAR5 unpacker exists with full filter support; uses `node-unrar-js` wasm or `unrar` native binding). Browser builds resolve to a stub that throws `RAR_UNAVAILABLE_IN_BROWSER`.
|
||||
|
||||
### Public factory shape
|
||||
|
||||
```ts
|
||||
import { openArchiveStorage } from "@suaveplan/drive-storage-archive-core";
|
||||
|
||||
const archiveStorage = await openArchiveStorage(hostBytes, {
|
||||
format: "auto", // 'auto' | 'zip' | 'tar' | ...
|
||||
mode: "copy-on-write",
|
||||
password: async () => promptUser(),
|
||||
resolveSibling: async (basename, index) => hostStorage.read(`${basename}.z${String(index).padStart(2, "0")}`),
|
||||
});
|
||||
|
||||
const drive = new Drive(archiveStorage);
|
||||
await drive.initialize();
|
||||
await drive.write("inside-archive/notes.txt", new TextEncoder().encode("hello"));
|
||||
await archiveStorage.flush(); // for copy-on-write
|
||||
```
|
||||
|
||||
The factory does the sniff, picks the per-format adapter via dynamic import (so unused codecs aren't bundled), and returns the constructed `DriveStorage`. Each per-format package also exports its constructor directly (`ZipArchiveStorage`, etc.) for callers who don't need sniffing.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **RAR licensing** → Mitigation: read-only, vendored via permissive-license wasm unrar; license note in package README. Block writes at type level (`mode: 'read-only'` only).
|
||||
- **Rewrite cost on tar/gz** → Mitigation: document O(archive size) cost in README; recommend `copy-on-write` + single `flush()` for batches; expose `pendingMutationCount` getter.
|
||||
- **Password-encrypted entries leak metadata** → Mitigation: document that entry names + sizes are unencrypted in zip (by spec) and 7z header-encryption mode is supported as opt-in. RAR encrypted-headers supported.
|
||||
- **Split-volume ordering ambiguity** → Mitigation: each format's `defaultResolver` enforces canonical ordering (`.z01..zNN..zip` final; `.partN.rar` ascending; `.7z.001..NNN`). Adapter validates volume signatures match before assembly; mismatched volumes → `ARCHIVE_SPLIT_MISSING_VOLUME` with which-volume detail.
|
||||
- **Browser memory pressure on large archives** → Mitigation: lazy entry parsing (central directory only on open); stream-only API for >100MB entries; warn via logger if archive exceeds a configurable threshold.
|
||||
- **Format sniff false positives** → Mitigation: magic-byte match plus structural validation (zip → can locate EOCD; tar → header checksum matches; gz → trailing CRC matches). Sniff returns `null` if structural check fails, falling back to extension.
|
||||
- **Nested archives** → Mitigation: not solved here. Host composes adapters explicitly; documented as a recipe in `drive-storage-archive-core` README.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
- New packages — no existing consumer migration required.
|
||||
- Each package ships at `0.1.0`.
|
||||
- Drive consumers gain archive support opportunistically by passing an archive storage where they previously passed a flat storage.
|
||||
- Rollback: each package is independently removable. No drive-side changes mean no rollback risk to existing storage adapters.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- 7z wasm LZMA performance vs. native: settle in tasks via benchmark; if wasm <50% of native throughput on a 100MB archive, ship server-only and provide a wasm browser variant later.
|
||||
- Whether `drive-storage-archive-core` should expose a `suaveplan-archive` CLI bin (list/extract/test) or whether that belongs in a separate `archive-cli` package. Default: defer CLI to a follow-up change to keep this scope tight.
|
||||
- ZIP AES encryption (WinZip AE-2) — implement now or follow-up? Default: implement read in v0.1.0, write in v0.2.0.
|
||||
41
openspec/changes/add-drive-storage-archive/proposal.md
Normal file
41
openspec/changes/add-drive-storage-archive/proposal.md
Normal file
@@ -0,0 +1,41 @@
|
||||
## Why
|
||||
|
||||
The `drive` VFS already abstracts file operations behind a structural `DriveStorage` view, but there is no way to mount a compressed archive (zip, tar, gz, tgz, rar, 7z) as a drive. Users routinely receive bundles whose extension lies about the real compression (a `.zip` that is actually a 7z stream, a `.tar.gz` renamed `.bin`) or whose contents are split across multi-volume sets (`.part1.rar`/`.r00`–`.r99`, `.7z.001`–`.7z.NNN`, multi-file split-zip). Treating those archives as first-class drives lets every existing drive consumer (sharing, hydrate-from, file-operations, plugins) browse, read, and (where the format permits) write archive contents without ad-hoc extraction.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a shared `@suaveplan/drive-storage-archive-core` package: format sniffing by magic bytes (extension is advisory only), split-volume reassembly, central-directory caching, password/encryption hook contract, and a streaming `EntryReader` abstraction so per-format adapters only implement the format-specific decode path.
|
||||
- Add six per-format storage adapters, each implementing the `DriveStorage` contract over an underlying `AbstractStorage` (or raw byte source) holding the archive bytes:
|
||||
- `@suaveplan/drive-storage-zip` — ZIP / ZIP64 / split-zip (`.z01`–`.zNN` + `.zip`), read-write with rewrite-on-flush.
|
||||
- `@suaveplan/drive-storage-tar` — POSIX/USTAR/PAX, sparse files, long-name extensions; read-write (append + rewrite on delete).
|
||||
- `@suaveplan/drive-storage-gz` — single-stream gzip wrapping one logical entry; read-write.
|
||||
- `@suaveplan/drive-storage-tgz` — composition of `tar` over `gz` (and `.tar.bz2`, `.tar.xz`, `.tar.zst` via pluggable codec); read-mostly, write via full rewrite.
|
||||
- `@suaveplan/drive-storage-rar` — RAR4 + RAR5 + multi-volume (`.partN.rar`, legacy `.r00`–`.r99`); **read-only** (writing RAR is patent-encumbered).
|
||||
- `@suaveplan/drive-storage-7z` — 7z LZMA/LZMA2/BCJ filters + multi-volume (`.7z.001`–`.7z.NNN`); read-write via rewrite-on-flush.
|
||||
- Public factory `openArchiveStorage(source, { format?, password?, splitParts?, mode? })` that auto-detects format from magic bytes, discovers split volumes through a sibling-resolver callback on the host storage, and returns a `DriveStorage` ready to hand to `new Drive(...)`.
|
||||
- Mode flags: `read-only` | `copy-on-write` (in-memory overlay, materialise on `flush()`) | `rewrite` (write directly back through host storage on each mutation). RAR is locked to `read-only`.
|
||||
- Error codes namespaced per package (`ARCHIVE_*`, `ZIP_*`, `TAR_*`, `GZ_*`, `RAR_*`, `SEVENZ_*`) registered in each package's error registry.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `drive-storage-archive-core`: shared format sniffing, split-volume reassembly, mode/flush contract, password hook, codec registry, error taxonomy, and conformance harness all per-format adapters run against.
|
||||
- `drive-storage-zip`: ZIP/ZIP64/split-zip adapter exposing archive entries as a `DriveStorage`.
|
||||
- `drive-storage-tar`: POSIX/USTAR/PAX tar adapter exposing tar entries as a `DriveStorage`.
|
||||
- `drive-storage-gz`: single-entry gzip adapter exposing the wrapped stream as a `DriveStorage`.
|
||||
- `drive-storage-tgz`: composed tar-over-codec adapter (gzip/bzip2/xz/zstd) exposing entries as a `DriveStorage`.
|
||||
- `drive-storage-rar`: read-only RAR4/RAR5 + multi-volume adapter exposing rar entries as a `DriveStorage`.
|
||||
- `drive-storage-7z`: 7z + multi-volume adapter exposing 7z entries as a `DriveStorage`.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
_None — drive's `DriveStorage` view is already satisfied; no requirement changes to existing specs._
|
||||
|
||||
## Impact
|
||||
|
||||
- New packages under `packages/drive/drive-storage-{zip,tar,gz,tgz,rar,7z}` plus `packages/drive/drive-storage-archive-core` (seven total). All universal-tier where the codec is pure-TS, server-tier where native bindings are needed (RAR, optionally 7z LZMA SDK).
|
||||
- New dependencies on pure-TS codec libraries via Bun catalog (`fflate` for deflate, `pako` fallback, `lzma-native` or wasm `lzma`, `xz-decompress`, `@zip.js/zip.js` is **not** used — we implement directly for streaming + split support).
|
||||
- Drive consumers (`drive-sharing`, `drive-template`, `drive-react-sharing`) gain transparent archive browsing with no API changes.
|
||||
- Bin: optional `suaveplan-archive` CLI in `drive-storage-archive-core` for listing/extracting any supported archive from the shell. (Decision deferred to design.md.)
|
||||
- Test surface: `runArchiveConformance` harness in `drive-storage-archive-core/testing` exercised by all six adapters.
|
||||
@@ -0,0 +1,65 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: 7z adapter exposes 7z entries as DriveStorage
|
||||
|
||||
The package SHALL export `SevenZipArchiveStorage` implementing the structural `DriveStorage` view. Entry paths inside the archive MUST appear as drive paths with directory separators normalised to `/`.
|
||||
|
||||
#### Scenario: Drive lists 7z entries
|
||||
|
||||
- **WHEN** a 7z archive containing two entries is wrapped and `drive.list()` is called
|
||||
- **THEN** both entry paths are returned
|
||||
|
||||
### Requirement: LZMA, LZMA2, and BCJ filter support
|
||||
|
||||
The adapter SHALL support LZMA and LZMA2 compression codecs and the BCJ, BCJ2, Delta, and ARM/X86/SPARC preprocessor filters as composed in 7z `Folder` records. Unsupported codec IDs MUST raise `SEVENZ_UNSUPPORTED_CODEC`. Unsupported filter chains MUST raise `SEVENZ_BCJ_FILTER_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: LZMA2 + BCJ executable decodes
|
||||
|
||||
- **WHEN** a 7z archive whose folder chain is LZMA2 → BCJ over an executable entry is read
|
||||
- **THEN** the decoded bytes match the original executable
|
||||
|
||||
#### Scenario: Unknown codec errors clearly
|
||||
|
||||
- **WHEN** an archive references a codec ID not implemented by the adapter
|
||||
- **THEN** the read throws `SEVENZ_UNSUPPORTED_CODEC` with the codec ID in details
|
||||
|
||||
### Requirement: Multi-volume support
|
||||
|
||||
The adapter SHALL support multi-volume 7z archives using `.7z.001`, `.7z.002`, …, `.7z.NNN` naming. Sibling discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. The adapter MUST concatenate volumes in ascending index order before parsing the trailing header.
|
||||
|
||||
#### Scenario: Three-volume 7z assembles
|
||||
|
||||
- **WHEN** the primary `archive.7z.001` is opened with a `resolveSibling` returning `archive.7z.002` and `archive.7z.003`
|
||||
- **THEN** all entries are listed and read
|
||||
|
||||
#### Scenario: Out-of-order volume rejected
|
||||
|
||||
- **WHEN** `resolveSibling` returns volume 1 at index 2 (i.e. mis-ordered)
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` with both expected and received indices
|
||||
|
||||
### Requirement: Header encryption and per-entry encryption
|
||||
|
||||
The adapter SHALL support reading 7z archives with encrypted headers (`7zAES` over the `EncodedHeader` block) and per-entry-encrypted content. The password hook from `drive-storage-archive-core` MUST be invoked lazily on first decryption need.
|
||||
|
||||
#### Scenario: Encrypted-header archive opens
|
||||
|
||||
- **WHEN** a 7z archive with encrypted headers is opened with a correct-password hook
|
||||
- **THEN** entry listing and reading succeed
|
||||
|
||||
### Requirement: Read-write with copy-on-write or rewrite
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Writes MUST produce a standards-compliant 7z file readable by reference 7-Zip implementations. Mutations re-encode the affected folder(s) and rewrite the trailing header.
|
||||
|
||||
#### Scenario: Round-trip write then read
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes `inside/n.txt`, flushes, then a new adapter is opened over the flushed bytes
|
||||
- **THEN** `inside/n.txt` is listed and reads with the original content
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `SEVENZ_HEADER_INVALID`, `SEVENZ_UNSUPPORTED_CODEC`, `SEVENZ_BCJ_FILTER_UNSUPPORTED`, `SEVENZ_CRC_MISMATCH`, `SEVENZ_CORRUPT`.
|
||||
|
||||
#### Scenario: Bad signature errors
|
||||
|
||||
- **WHEN** the host bytes do not begin with `7z\xBC\xAF\x27\x1C`
|
||||
- **THEN** open throws `SEVENZ_HEADER_INVALID`
|
||||
@@ -0,0 +1,95 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Format detection from magic bytes
|
||||
|
||||
The package SHALL expose a `sniffFormat(bytes: Uint8Array): ArchiveFormat | null` function that identifies archive format by inspecting magic bytes in the first 512 bytes, treating any file-name extension as advisory only. Detection MUST cover ZIP, RAR4, RAR5, 7z, GZIP, and POSIX/USTAR tar. When magic bytes match a known format, a structural validation step MUST also succeed before the format is returned.
|
||||
|
||||
#### Scenario: Sniff matches when extension lies
|
||||
|
||||
- **WHEN** `sniffFormat` is called with bytes whose first four bytes are `PK\x03\x04` but whose source filename was `bundle.bin`
|
||||
- **THEN** the function returns `"zip"` and the source filename is not consulted
|
||||
|
||||
#### Scenario: Mismatch between extension and content emits warning
|
||||
|
||||
- **WHEN** `openArchiveStorage` is called with a source named `archive.zip` whose magic bytes match `7z\xBC\xAF\x27\x1C`
|
||||
- **THEN** an `ARCHIVE_FORMAT_MISMATCH` warning is logged with both detected and declared formats, and the detected format is used
|
||||
|
||||
#### Scenario: Unknown bytes return null
|
||||
|
||||
- **WHEN** `sniffFormat` is called with bytes matching no known magic-byte signature and no structural fallback succeeds
|
||||
- **THEN** the function returns `null`
|
||||
|
||||
### Requirement: openArchiveStorage factory
|
||||
|
||||
The package SHALL expose `openArchiveStorage(source, options)` that returns a `DriveStorage` view backed by the appropriate per-format adapter. Options MUST include `format`, `mode`, `password`, and `resolveSibling`. The factory MUST dynamic-import the per-format adapter so unused format codecs are not bundled into consumers that do not need them.
|
||||
|
||||
#### Scenario: Auto-detect and construct adapter
|
||||
|
||||
- **WHEN** `openArchiveStorage(bytes, { format: "auto", mode: "read-only" })` is called with valid zip bytes
|
||||
- **THEN** the function returns a `DriveStorage` instance from `@suaveplan/drive-storage-zip` constructed in read-only mode
|
||||
|
||||
#### Scenario: Explicit format overrides sniff
|
||||
|
||||
- **WHEN** `openArchiveStorage(bytes, { format: "tar" })` is called
|
||||
- **THEN** the factory loads the tar adapter without invoking `sniffFormat`
|
||||
|
||||
### Requirement: Split-volume sibling resolution
|
||||
|
||||
The package SHALL define a `resolveSibling(basename: string, index: number): Promise<Uint8Array | ReadableStream<Uint8Array>>` callback contract that per-format adapters use to fetch additional volumes. The host application owns sibling discovery; the adapter never assumes a filesystem layout.
|
||||
|
||||
#### Scenario: Multi-volume archive triggers sibling fetch
|
||||
|
||||
- **WHEN** a zip adapter opens a primary volume whose EOCD references volumes 0..3
|
||||
- **THEN** the adapter invokes `resolveSibling(basename, 0)`, `resolveSibling(basename, 1)`, `resolveSibling(basename, 2)` and uses the returned bytes in volume order
|
||||
|
||||
#### Scenario: Missing sibling raises typed error
|
||||
|
||||
- **WHEN** `resolveSibling` rejects or returns `null` for a required volume
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` whose details include the missing volume index and expected basename
|
||||
|
||||
### Requirement: Mutation mode contract
|
||||
|
||||
The package SHALL define three modes — `read-only`, `copy-on-write`, `rewrite` — and a `flush(): Promise<void>` method on the returned `DriveStorage`. In `read-only` mode any write/delete operation MUST throw `ARCHIVE_MODE_READ_ONLY`. In `copy-on-write` mode mutations MUST land in an in-memory overlay and be materialised to host storage only on `flush()`. In `rewrite` mode each mutation MUST re-encode the archive into host storage before returning.
|
||||
|
||||
#### Scenario: Write in read-only mode throws
|
||||
|
||||
- **WHEN** any mutation is attempted on an adapter constructed with `mode: "read-only"`
|
||||
- **THEN** the call throws `ARCHIVE_MODE_READ_ONLY` and the host archive bytes are unchanged
|
||||
|
||||
#### Scenario: Copy-on-write defers materialisation
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode receives a write, the host archive bytes have not changed
|
||||
- **AND WHEN** `flush()` is awaited
|
||||
- **THEN** the host archive bytes contain the new entry and `flush()` is idempotent for a no-op state
|
||||
|
||||
### Requirement: Password hook contract
|
||||
|
||||
The package SHALL accept `password: () => Promise<string>` and invoke it lazily — only when an encrypted entry is encountered. Passwords MUST NOT be stored on the adapter instance beyond the lifetime of the active decrypt operation. An incorrect password MUST raise `ARCHIVE_PASSWORD_INVALID`; a missing hook on encrypted content MUST raise `ARCHIVE_PASSWORD_REQUIRED`.
|
||||
|
||||
#### Scenario: Password requested only on encrypted read
|
||||
|
||||
- **WHEN** an archive contains both unencrypted and encrypted entries and the consumer reads only unencrypted entries
|
||||
- **THEN** the `password` hook is never invoked
|
||||
|
||||
#### Scenario: Wrong password reported
|
||||
|
||||
- **WHEN** the `password` hook returns a value that fails authenticated decryption on an encrypted entry
|
||||
- **THEN** the read throws `ARCHIVE_PASSWORD_INVALID` and the password value is not retained
|
||||
|
||||
### Requirement: Conformance harness
|
||||
|
||||
The package SHALL expose a `runArchiveConformance(adapterFactory, opts)` test harness via the `./testing` subpath export that exercises a fixed set of behaviours every per-format adapter MUST satisfy: open, list, read, stat, password-failure path, mode enforcement, split-volume reassembly (where applicable), and round-trip of one binary entry through write/flush/read (where the format permits writes).
|
||||
|
||||
#### Scenario: Per-format package consumes harness
|
||||
|
||||
- **WHEN** a per-format package's test suite calls `runArchiveConformance(() => new ZipArchiveStorage(...))`
|
||||
- **THEN** the harness executes the full behaviour set and any failure surfaces as a typed assertion identifying the failed behaviour
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register the following stable error codes via `defineCodes` from `@suaveplan/error`: `ARCHIVE_FORMAT_UNKNOWN`, `ARCHIVE_FORMAT_MISMATCH`, `ARCHIVE_SPLIT_MISSING_VOLUME`, `ARCHIVE_PASSWORD_REQUIRED`, `ARCHIVE_PASSWORD_INVALID`, `ARCHIVE_MODE_READ_ONLY`, `ARCHIVE_FLUSH_REQUIRED`, `ARCHIVE_CORRUPT`, `ARCHIVE_UNSUPPORTED_FEATURE`, `ARCHIVE_CODEC_MISSING`. Each MUST have a stable string identifier never reused after removal.
|
||||
|
||||
#### Scenario: Code identifiers stable across releases
|
||||
|
||||
- **WHEN** the package is published at any version `0.x.y`
|
||||
- **THEN** every code in the registry resolves to the same string identifier as in version `0.1.0`
|
||||
@@ -0,0 +1,60 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: GZ adapter exposes the wrapped stream as one drive entry
|
||||
|
||||
The package SHALL export `GzipArchiveStorage` implementing the structural `DriveStorage` view. Because gzip wraps a single logical stream, the adapter MUST present exactly one entry whose path defaults to the original filename from the gzip `FNAME` field, or to a configurable fallback path when `FNAME` is absent.
|
||||
|
||||
#### Scenario: FNAME header used as entry path
|
||||
|
||||
- **WHEN** a gzip stream whose `FNAME` field equals `report.csv` is wrapped in `GzipArchiveStorage`
|
||||
- **THEN** `drive.list()` returns exactly one entry at `report.csv`
|
||||
|
||||
#### Scenario: Fallback path used when FNAME absent
|
||||
|
||||
- **WHEN** a gzip stream without an `FNAME` field is opened with `fallbackPath: "data.bin"`
|
||||
- **THEN** the single entry appears at `data.bin`
|
||||
|
||||
### Requirement: Multi-member gzip streams
|
||||
|
||||
The adapter SHALL transparently concatenate sequential gzip members (per RFC 1952 §2.2) when present, returning their decompressed bytes as a single logical entry stream.
|
||||
|
||||
#### Scenario: Two concatenated members read as one stream
|
||||
|
||||
- **WHEN** a gzip file containing two valid members each holding half of a logical file is read
|
||||
- **THEN** the returned stream contains the concatenation of both members' decompressed bytes
|
||||
|
||||
### Requirement: Read-write with full re-encode
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Because gzip is a single stream, any write MUST fully re-encode the host bytes. The adapter MUST document this cost.
|
||||
|
||||
#### Scenario: Write replaces entire host bytes
|
||||
|
||||
- **WHEN** an adapter in `rewrite` mode receives a write to the single entry
|
||||
- **THEN** the host bytes are replaced by a freshly encoded gzip stream containing the new content
|
||||
|
||||
### Requirement: CRC32 and ISIZE validation
|
||||
|
||||
The adapter SHALL validate the trailing CRC32 and ISIZE fields on read. A mismatch MUST raise `GZ_CRC_MISMATCH` or `GZ_ISIZE_MISMATCH` respectively.
|
||||
|
||||
#### Scenario: Truncated stream errors
|
||||
|
||||
- **WHEN** the host bytes end before a valid CRC32+ISIZE trailer
|
||||
- **THEN** the read throws `GZ_TRUNCATED`
|
||||
|
||||
### Requirement: Codec export for tgz composition
|
||||
|
||||
The package SHALL export a `gzipCodec: ArchiveCodec` object that `@suaveplan/drive-storage-tgz` composes over a tar adapter. The codec MUST expose `compress(stream)` and `decompress(stream)` returning `ReadableStream<Uint8Array>`.
|
||||
|
||||
#### Scenario: tgz consumes gzipCodec
|
||||
|
||||
- **WHEN** `@suaveplan/drive-storage-tgz` constructs an adapter with `codec: gzipCodec`
|
||||
- **THEN** decompression of a `.tar.gz` host file produces the same entry listing as decompressing the gz then opening the tar
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `GZ_HEADER_INVALID`, `GZ_CRC_MISMATCH`, `GZ_ISIZE_MISMATCH`, `GZ_TRUNCATED`, `GZ_UNSUPPORTED_FLAG`.
|
||||
|
||||
#### Scenario: Bad gzip magic errors
|
||||
|
||||
- **WHEN** the first two bytes are not `\x1F\x8B`
|
||||
- **THEN** the open throws `GZ_HEADER_INVALID`
|
||||
@@ -0,0 +1,75 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: RAR adapter exposes rar entries as DriveStorage (read-only)
|
||||
|
||||
The package SHALL export `RarArchiveStorage` implementing the structural `DriveStorage` view in read-only mode only. Construction with any mode other than `read-only` MUST throw `ARCHIVE_MODE_READ_ONLY` immediately.
|
||||
|
||||
#### Scenario: Construction rejects writable mode
|
||||
|
||||
- **WHEN** `new RarArchiveStorage(bytes, { mode: "rewrite" })` is called
|
||||
- **THEN** the constructor throws `ARCHIVE_MODE_READ_ONLY` and does not allocate any decoder state
|
||||
|
||||
#### Scenario: Drive lists rar entries
|
||||
|
||||
- **WHEN** a RAR5 archive containing two entries is wrapped and `drive.list()` is called
|
||||
- **THEN** both entry paths are returned
|
||||
|
||||
### Requirement: RAR4 and RAR5 format support
|
||||
|
||||
The adapter SHALL parse both RAR4 (signature `Rar!\x1A\x07\x00`) and RAR5 (signature `Rar!\x1A\x07\x01\x00`) container formats. Format detection MUST happen via magic bytes, and unsupported RAR3-or-older signatures MUST raise `RAR_VERSION_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: RAR4 archive opens
|
||||
|
||||
- **WHEN** a RAR4 archive is opened
|
||||
- **THEN** entries are listed correctly
|
||||
|
||||
#### Scenario: RAR5 archive opens
|
||||
|
||||
- **WHEN** a RAR5 archive is opened
|
||||
- **THEN** entries are listed correctly
|
||||
|
||||
### Requirement: Multi-volume support
|
||||
|
||||
The adapter SHALL support RAR5 multi-volume archives using `.partN.rar` naming (e.g. `name.part1.rar`, `name.part2.rar`) and legacy RAR4 multi-volume using `.r00`, `.r01`, …, `.r99` extensions. Sibling discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. Each subsequent volume's signature and volume number MUST validate before its bytes are consumed.
|
||||
|
||||
#### Scenario: RAR5 partN volumes assemble
|
||||
|
||||
- **WHEN** the primary `archive.part1.rar` is opened with a `resolveSibling` that returns `archive.part2.rar` and `archive.part3.rar`
|
||||
- **THEN** entries spanning all three volumes are listed and read
|
||||
|
||||
#### Scenario: RAR4 r00..r02 volumes assemble
|
||||
|
||||
- **WHEN** the primary `archive.rar` is opened with a `resolveSibling` returning `archive.r00`, `archive.r01`
|
||||
- **THEN** entries spanning all volumes are read
|
||||
|
||||
#### Scenario: Volume sequence gap errors
|
||||
|
||||
- **WHEN** `resolveSibling` returns volume 2 with a header indicating volume number 3
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME`
|
||||
|
||||
### Requirement: Encrypted header and encrypted entry support
|
||||
|
||||
The adapter SHALL support reading entries protected by per-entry passwords and full encrypted-header archives. The password hook from `drive-storage-archive-core` MUST be invoked lazily on first encrypted-content access.
|
||||
|
||||
#### Scenario: Encrypted-header archive
|
||||
|
||||
- **WHEN** an archive with encrypted headers is opened and the password hook returns the correct passphrase
|
||||
- **THEN** entry listing succeeds
|
||||
|
||||
### Requirement: Browser builds raise typed error
|
||||
|
||||
The package SHALL be server-tier. The `browser` field in `package.json` MUST resolve to a stub module whose entry points throw `RAR_UNAVAILABLE_IN_BROWSER` on any call.
|
||||
|
||||
#### Scenario: Browser bundle throws on use
|
||||
|
||||
- **WHEN** a browser bundle that imports `RarArchiveStorage` calls the constructor
|
||||
- **THEN** it throws `RAR_UNAVAILABLE_IN_BROWSER`
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `RAR_VERSION_UNSUPPORTED`, `RAR_FILTER_UNSUPPORTED`, `RAR_UNAVAILABLE_IN_BROWSER`, `RAR_CORRUPT`, `RAR_CRC_MISMATCH`.
|
||||
|
||||
#### Scenario: CRC mismatch surfaces
|
||||
|
||||
- **WHEN** an entry's stored CRC does not match the decoded bytes
|
||||
- **THEN** the read throws `RAR_CRC_MISMATCH` with the entry path in details
|
||||
@@ -0,0 +1,56 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TAR adapter exposes tar entries as DriveStorage
|
||||
|
||||
The package SHALL export `TarArchiveStorage` implementing the structural `DriveStorage` view. Entry paths inside the tar MUST appear as drive paths with directory separators normalised to `/`.
|
||||
|
||||
#### Scenario: Drive lists tar entries
|
||||
|
||||
- **WHEN** a tar archive containing `etc/conf` and `etc/hosts` is wrapped in `TarArchiveStorage` and `drive.list("etc/")` is called
|
||||
- **THEN** the result contains both `etc/conf` and `etc/hosts`
|
||||
|
||||
### Requirement: POSIX, USTAR, and PAX header support
|
||||
|
||||
The adapter SHALL parse legacy v7, USTAR, and POSIX.1-2001 PAX extended headers. Long filenames exceeding 100 bytes MUST be reconstructed from PAX `path` records or GNU `L` long-link entries. PAX size records MUST override the 12-octet octal size field.
|
||||
|
||||
#### Scenario: PAX long filename round-trips
|
||||
|
||||
- **WHEN** a tar entry whose path is 300 bytes long is read
|
||||
- **THEN** the entry path is returned unchanged from `list()` and `stat()`
|
||||
|
||||
#### Scenario: PAX size record honoured
|
||||
|
||||
- **WHEN** an entry's PAX `size` record specifies a value larger than 8 GiB
|
||||
- **THEN** the adapter reads the full byte count without truncation
|
||||
|
||||
### Requirement: Sparse file extension
|
||||
|
||||
The adapter SHALL parse GNU sparse 1.0 extension records and present sparse regions as zero bytes in the reconstructed entry stream. Sparse 0.0/0.1 variants MAY be supported but MUST raise `TAR_SPARSE_VARIANT_UNSUPPORTED` if unsupported rather than silently mis-reading.
|
||||
|
||||
#### Scenario: Sparse 1.0 entry reads as zero-filled
|
||||
|
||||
- **WHEN** a sparse 1.0 entry with one data region of 1 KiB and one hole of 1 GiB is read
|
||||
- **THEN** the returned stream contains the data region followed by 1 GiB of zero bytes
|
||||
|
||||
### Requirement: Read-write with append and rewrite
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Tar's sequential format means deletes require full rewrite; the adapter MUST document this cost and expose `pendingMutationCount`. Pure-append writes MAY append blocks before the two-block zero terminator and rewrite only the terminator.
|
||||
|
||||
#### Scenario: Append avoids full rewrite
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes one new entry and calls `flush()`
|
||||
- **THEN** the host bytes are appended in place plus a new zero terminator, not fully re-encoded
|
||||
|
||||
#### Scenario: Delete triggers full rewrite
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode deletes one existing entry and calls `flush()`
|
||||
- **THEN** `flush()` writes a complete new archive omitting the deleted entry
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `TAR_HEADER_CHECKSUM_INVALID`, `TAR_PAX_HEADER_INVALID`, `TAR_SPARSE_VARIANT_UNSUPPORTED`, `TAR_BLOCK_ALIGNMENT_INVALID`.
|
||||
|
||||
#### Scenario: Bad header checksum errors
|
||||
|
||||
- **WHEN** a tar header's checksum field does not match the sum of its octets
|
||||
- **THEN** the read throws `TAR_HEADER_CHECKSUM_INVALID` with the byte offset in details
|
||||
@@ -0,0 +1,42 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TGZ adapter composes tar over a codec
|
||||
|
||||
The package SHALL export `TgzArchiveStorage` implementing the structural `DriveStorage` view by composing a tar adapter over a pluggable codec. The default constructor MUST accept `codec: ArchiveCodec` and SHALL ship a convenience `openTarGz(source, opts)` that defaults to `gzipCodec` from `@suaveplan/drive-storage-gz`.
|
||||
|
||||
#### Scenario: Default tar.gz opens with gzip codec
|
||||
|
||||
- **WHEN** `openTarGz(bytes, { mode: "read-only" })` is called on valid `.tar.gz` bytes
|
||||
- **THEN** entries from the inner tar are listed and read correctly
|
||||
|
||||
### Requirement: Codec slot for tar.bz2, tar.xz, tar.zst
|
||||
|
||||
The package SHALL accept any object satisfying the `ArchiveCodec` interface (`compress`, `decompress`). Codecs for bzip2, xz, and zstd are NOT shipped in v0.1.0 but the slot MUST be functional so consumers can plug them in. Construction with no codec MUST raise `ARCHIVE_CODEC_MISSING`.
|
||||
|
||||
#### Scenario: Custom xz codec works
|
||||
|
||||
- **WHEN** `new TgzArchiveStorage(bytes, { codec: customXzCodec })` is constructed and the bytes are a valid `.tar.xz`
|
||||
- **THEN** entries are listed and read correctly using the custom codec's `decompress`
|
||||
|
||||
#### Scenario: Missing codec rejects clearly
|
||||
|
||||
- **WHEN** `new TgzArchiveStorage(bytes, {})` is constructed without a codec
|
||||
- **THEN** construction throws `ARCHIVE_CODEC_MISSING`
|
||||
|
||||
### Requirement: Read-mostly with full re-encode on write
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Writes MUST re-encode the full tar and then the full codec output. The cost MUST be documented and `pendingMutationCount` exposed.
|
||||
|
||||
#### Scenario: Round-trip write
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode adds one entry and calls `flush()`
|
||||
- **THEN** the host bytes contain a valid codec-wrapped tar whose entry list contains the new entry
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `TGZ_CODEC_FAILED`, `TGZ_INNER_TAR_INVALID`. Format-specific errors from the underlying tar and codec packages MUST be propagated unchanged.
|
||||
|
||||
#### Scenario: Inner tar corruption surfaces
|
||||
|
||||
- **WHEN** the codec decompresses successfully but the resulting tar bytes fail header checksum validation
|
||||
- **THEN** the error chain contains both `TGZ_INNER_TAR_INVALID` and the underlying `TAR_HEADER_CHECKSUM_INVALID`
|
||||
@@ -0,0 +1,60 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: ZIP adapter exposes archive entries as DriveStorage
|
||||
|
||||
The package SHALL export `ZipArchiveStorage` implementing the structural `DriveStorage` view so that `new Drive(new ZipArchiveStorage(...))` works without drive-side changes. Entry paths inside the archive MUST appear as drive paths preserving the archive's internal directory separators (normalised to `/`).
|
||||
|
||||
#### Scenario: Drive lists archive entries
|
||||
|
||||
- **WHEN** a zip archive containing `docs/a.txt` and `docs/b.txt` is wrapped in `ZipArchiveStorage` and handed to `new Drive(...)` and `drive.list("docs/")` is called
|
||||
- **THEN** the result contains both `docs/a.txt` and `docs/b.txt`
|
||||
|
||||
### Requirement: ZIP64 support
|
||||
|
||||
The adapter SHALL transparently parse ZIP64 central directory extensions when the EOCD `total entries`, `central directory size`, or `central directory offset` fields equal their 32-bit sentinel value `0xFFFFFFFF`. ZIP64 archives MUST be read without consumer opt-in.
|
||||
|
||||
#### Scenario: ZIP64 archive opens
|
||||
|
||||
- **WHEN** an archive larger than 4 GiB with valid ZIP64 EOCD locator is opened
|
||||
- **THEN** all entries are listed and read without error
|
||||
|
||||
### Requirement: Split-zip volume reassembly
|
||||
|
||||
The adapter SHALL support split-zip archives whose volumes follow the canonical naming `name.z01`, `name.z02`, …, `name.zNN`, `name.zip` (final volume holds EOCD). Volume discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. Volume signatures MUST be validated; mismatched volumes raise `ARCHIVE_SPLIT_MISSING_VOLUME`.
|
||||
|
||||
#### Scenario: Three-volume zip opens via sibling resolver
|
||||
|
||||
- **WHEN** the primary `archive.zip` is opened with a `resolveSibling` callback that returns `archive.z01` and `archive.z02` for indices 0 and 1
|
||||
- **THEN** the adapter lists entries spanning all three volumes and reads each correctly
|
||||
|
||||
#### Scenario: Missing middle volume errors clearly
|
||||
|
||||
- **WHEN** `resolveSibling` returns `null` for volume index 1
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` with `{ basename: "archive", missingIndex: 1 }` in the error details
|
||||
|
||||
### Requirement: Read-write with copy-on-write or rewrite mode
|
||||
|
||||
The adapter SHALL implement `read-only`, `copy-on-write`, and `rewrite` modes. In write modes the adapter MUST rebuild a valid EOCD on `flush()` (copy-on-write) or after each mutation (rewrite). The host archive bytes MUST remain readable by any standards-compliant zip tool after flush.
|
||||
|
||||
#### Scenario: Round-trip write then read
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes `inside/notes.txt`, calls `flush()`, then a new adapter is opened over the flushed bytes
|
||||
- **THEN** `inside/notes.txt` is listed and read with the original byte content
|
||||
|
||||
### Requirement: AES-encrypted entries
|
||||
|
||||
The adapter SHALL support reading WinZip AE-2 AES-256 encrypted entries using the password hook from `drive-storage-archive-core`. Writing AES-encrypted entries is out of scope for v0.1.0 and SHALL raise `ARCHIVE_UNSUPPORTED_FEATURE` until v0.2.0.
|
||||
|
||||
#### Scenario: Encrypted entry decrypts with correct password
|
||||
|
||||
- **WHEN** an AES-256 encrypted entry is read with a `password` hook returning the correct passphrase
|
||||
- **THEN** the decrypted bytes are returned and the password value is not retained on the adapter
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `ZIP_EOCD_NOT_FOUND`, `ZIP_ZIP64_REQUIRED`, `ZIP_CENTRAL_DIR_CORRUPT`, `ZIP_CRC_MISMATCH`, `ZIP_UNSUPPORTED_COMPRESSION`, `ZIP_AES_WRITE_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: CRC mismatch raises typed error
|
||||
|
||||
- **WHEN** an entry's decompressed CRC does not match the central directory CRC
|
||||
- **THEN** the read throws `ZIP_CRC_MISMATCH` with the entry path in details
|
||||
124
openspec/changes/add-drive-storage-archive/tasks.md
Normal file
124
openspec/changes/add-drive-storage-archive/tasks.md
Normal file
@@ -0,0 +1,124 @@
|
||||
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. drive-storage-archive-core (foundation)
|
||||
|
||||
- [x] 1.1 Scaffold `packages/drive/drive-storage-archive-core/` with `package.json`, vite/tsup config, vitest config via `createTestConfig()`, README, biome-clean tree
|
||||
- [x] 1.2 Define `ArchiveFormat` union, `ArchiveMode`, `ArchiveCodec`, `ResolveSibling`, `PasswordHook`, `OpenArchiveStorageOptions` types in `src/types/`
|
||||
- [x] 1.3 Implement `sniffFormat(bytes)` with magic-byte registry + structural validators (EOCD locate for zip, header checksum for tar, CRC trailer for gz, etc.)
|
||||
- [x] 1.4 Implement `openArchiveStorage(source, opts)` factory with dynamic `import()` of per-format packages (no eager bundling)
|
||||
- [x] 1.5 Implement `ARCHIVE_*` error code registry via `defineCodes` from `@suaveplan/error`
|
||||
- [x] 1.6 Implement `runArchiveConformance(adapterFactory, opts)` harness exported via `./testing` subpath
|
||||
- [x] 1.7 Implement default sibling-resolver helpers per format (`zipResolver`, `rarResolver`, `sevenZipResolver`) over a `DriveStorage`-shaped host
|
||||
- [x] 1.8 Co-locate `.md` for every implementation file (Purpose, Features, Usage Basic/Advanced, API Reference, Implementation Notes — ≥200 words, real-import examples)
|
||||
- [x] 1.9 100% Vitest coverage on `drive-storage-archive-core` via `bunx turbo test:coverage --filter=@suaveplan/drive-storage-archive-core`
|
||||
- [x] 1.10 Green `bunx turbo lint typecheck test --filter=@suaveplan/drive-storage-archive-core`
|
||||
- [x] 1.11 Update `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
|
||||
## 2. drive-storage-gz (foundation for tgz)
|
||||
|
||||
- [ ] 2.1 Scaffold `packages/drive/drive-storage-gz/`
|
||||
- [ ] 2.2 Implement gzip member parser (RFC 1952), multi-member concatenation, FNAME extraction
|
||||
- [ ] 2.3 Implement `GzipArchiveStorage` `DriveStorage` view with single-entry presentation
|
||||
- [ ] 2.4 Implement `gzipCodec: ArchiveCodec` export for tgz composition
|
||||
- [ ] 2.5 Implement write paths for `copy-on-write` and `rewrite` modes (full re-encode)
|
||||
- [ ] 2.6 Implement CRC32 + ISIZE validation
|
||||
- [ ] 2.7 Register `GZ_*` error codes
|
||||
- [ ] 2.8 Run `runArchiveConformance` from `drive-storage-archive-core/testing`
|
||||
- [ ] 2.9 Co-located `.md` docs per implementation file
|
||||
- [ ] 2.10 100% coverage + lint + typecheck green
|
||||
- [ ] 2.11 Regenerate MANIFEST
|
||||
|
||||
## 3. drive-storage-tar
|
||||
|
||||
- [ ] 3.1 Scaffold `packages/drive/drive-storage-tar/`
|
||||
- [ ] 3.2 Implement tar block reader (512-byte alignment, two-block terminator)
|
||||
- [ ] 3.3 Implement legacy v7, USTAR, POSIX.1-2001 PAX header parsing
|
||||
- [ ] 3.4 Implement GNU long-name (`L`) and long-link (`K`) reconstruction
|
||||
- [ ] 3.5 Implement PAX extended header records (path, size, mtime, atime, ctime, uid, gid)
|
||||
- [ ] 3.6 Implement GNU sparse 1.0 reconstruction with zero-fill
|
||||
- [ ] 3.7 Implement `TarArchiveStorage` `DriveStorage` view
|
||||
- [ ] 3.8 Implement append-write fast path (before terminator) and full-rewrite path for deletes
|
||||
- [ ] 3.9 Implement header-checksum validation, raise `TAR_HEADER_CHECKSUM_INVALID`
|
||||
- [ ] 3.10 Register `TAR_*` error codes
|
||||
- [ ] 3.11 Run `runArchiveConformance`
|
||||
- [ ] 3.12 Co-located `.md` docs per implementation file
|
||||
- [ ] 3.13 100% coverage + lint + typecheck green
|
||||
- [ ] 3.14 Regenerate MANIFEST
|
||||
|
||||
## 4. drive-storage-tgz
|
||||
|
||||
- [ ] 4.1 Scaffold `packages/drive/drive-storage-tgz/`
|
||||
- [ ] 4.2 Implement `TgzArchiveStorage` composing `TarArchiveStorage` over `ArchiveCodec`
|
||||
- [ ] 4.3 Implement `openTarGz(source, opts)` convenience defaulting to `gzipCodec` from `drive-storage-gz`
|
||||
- [ ] 4.4 Validate `ARCHIVE_CODEC_MISSING` on construction without codec
|
||||
- [ ] 4.5 Implement write path (codec-encode tar bytes back to host)
|
||||
- [ ] 4.6 Register `TGZ_*` error codes, propagate inner tar/codec errors with chain
|
||||
- [ ] 4.7 Run `runArchiveConformance` with `gzipCodec`
|
||||
- [ ] 4.8 Co-located `.md` docs per implementation file
|
||||
- [ ] 4.9 100% coverage + lint + typecheck green
|
||||
- [ ] 4.10 Regenerate MANIFEST
|
||||
|
||||
## 5. drive-storage-zip
|
||||
|
||||
- [x] 5.1 Scaffold `packages/drive/drive-storage-zip/`
|
||||
- [x] 5.2 Implement EOCD + Central Directory parser, ZIP64 EOCD locator/record handling
|
||||
- [x] 5.3 Implement Local File Header reader, deflate decode via `fflate`
|
||||
- [x] 5.4 Implement WinZip AE-2 AES-256 decrypt with PBKDF2 key derivation
|
||||
- [x] 5.5 Implement split-zip volume reassembly via `resolveSibling`, validate volume signatures
|
||||
- [x] 5.6 Implement `ZipArchiveStorage` `DriveStorage` view
|
||||
- [x] 5.7 Implement write path: append local headers + rebuild central directory + EOCD on flush
|
||||
- [x] 5.8 Block AES write with `ZIP_AES_WRITE_UNSUPPORTED` (deferred to v0.2.0)
|
||||
- [x] 5.9 Implement CRC32 validation, raise `ZIP_CRC_MISMATCH`
|
||||
- [x] 5.10 Register `ZIP_*` error codes
|
||||
- [x] 5.11 Run `runArchiveConformance`
|
||||
- [x] 5.12 Co-located `.md` docs per implementation file
|
||||
- [x] 5.13 100% coverage + lint + typecheck green
|
||||
- [x] 5.14 Regenerate MANIFEST
|
||||
|
||||
## 6. drive-storage-7z
|
||||
|
||||
- [ ] 6.1 Scaffold `packages/drive/drive-storage-7z/`
|
||||
- [ ] 6.2 Benchmark wasm LZMA vs `lzma-native` on 100MB archive; record decision in package README
|
||||
- [ ] 6.3 Implement 7z trailing header parser, `EncodedHeader` decode path
|
||||
- [ ] 6.4 Implement LZMA + LZMA2 decompression
|
||||
- [ ] 6.5 Implement BCJ, BCJ2, Delta, ARM/X86/SPARC filter chain
|
||||
- [ ] 6.6 Implement multi-volume reassembly (`.7z.NNN`) via `resolveSibling`
|
||||
- [ ] 6.7 Implement `7zAES` header + content encryption with password hook
|
||||
- [ ] 6.8 Implement `SevenZipArchiveStorage` `DriveStorage` view
|
||||
- [ ] 6.9 Implement write path: re-encode affected folders + rewrite trailing header
|
||||
- [ ] 6.10 Register `SEVENZ_*` error codes
|
||||
- [ ] 6.11 Run `runArchiveConformance`
|
||||
- [ ] 6.12 Co-located `.md` docs per implementation file
|
||||
- [ ] 6.13 100% coverage + lint + typecheck green
|
||||
- [ ] 6.14 Regenerate MANIFEST
|
||||
|
||||
## 7. drive-storage-rar (read-only, server-tier)
|
||||
|
||||
- [ ] 7.1 Scaffold `packages/drive/drive-storage-rar/` with `engines.node` + browser stub
|
||||
- [ ] 7.2 Vendor wasm unrar (permissive license) or wire native binding; record license attribution in README
|
||||
- [ ] 7.3 Implement RAR4 + RAR5 signature detection and version routing
|
||||
- [ ] 7.4 Implement multi-volume reassembly: RAR5 `.partN.rar` + RAR4 `.r00`..`.r99` via `resolveSibling`
|
||||
- [ ] 7.5 Implement encrypted-header and per-entry-encrypted reads with password hook
|
||||
- [ ] 7.6 Implement `RarArchiveStorage` `DriveStorage` view; reject any non-`read-only` mode in constructor
|
||||
- [ ] 7.7 Implement browser stub that throws `RAR_UNAVAILABLE_IN_BROWSER` for every entry point
|
||||
- [ ] 7.8 Register `RAR_*` error codes
|
||||
- [ ] 7.9 Run `runArchiveConformance` (read-only subset)
|
||||
- [ ] 7.10 Co-located `.md` docs per implementation file
|
||||
- [ ] 7.11 100% coverage + lint + typecheck green
|
||||
- [ ] 7.12 Regenerate MANIFEST
|
||||
|
||||
## 8. Integration & cross-cutting
|
||||
|
||||
- [ ] 8.1 Wire `openArchiveStorage` end-to-end test fixture using a real `.zip`, `.tar`, `.gz`, `.tar.gz`, `.rar`, `.7z` and a split-zip set
|
||||
- [ ] 8.2 Add an integration test demonstrating `new Drive(openArchiveStorage(...))` for each format
|
||||
- [ ] 8.3 Update `openspec/specs/drive/spec.md` only if drive-side contract clarification is needed (likely no change — `DriveStorage` view already satisfied)
|
||||
- [ ] 8.4 Run `bunx turbo lint typecheck test` across all seven new packages and ensure green
|
||||
- [ ] 8.5 Run `bunx turbo test:coverage` across all seven packages and confirm 100% coverage on touched modules
|
||||
- [ ] 8.6 Final `MANIFEST.md` regeneration + commit
|
||||
- [ ] 8.7 Archive change to `openspec/archive/<yyyy-mm-dd>-add-drive-storage-archive/` per repo workflow
|
||||
Reference in New Issue
Block a user