SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
admin
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
openspec/archive/2026-04-28-packer-port/proposal.md Normal file
View File

@@ -0,0 +1,164 @@
# OpenSpec Change: packer-port
- **Change ID**: packer-port
- **Package**: `@suaveplan/packer` (new)
- **Status**: PROPOSED
- **Date**: 2026-04-28
## Summary
Port `@suaveplan/packer` from `../suaveplan/packages/universal/packer` into genesis. Two modes ship as subpaths:
- **`@suaveplan/packer`** — whole-file pack/unpack: gzip + AEAD + ed25519 sign in one shot, schema-driven via bitpacker. Self-contained pack-and-go for small assets. Carried over with minor cleanup.
- **`@suaveplan/packer/chunked`** — chunk-addressed streaming pack/unpack/verify emitting the locked **v1 byte format** specced in `drive-plugin-chunks-port`. Partial rewrite: keeps source's pipeline (streaming source normalisation, AEAD streaming, per-chunk signing, verifier) but drops bitpacker on the chunked path in favour of a byte-aligned fixed-layout codec.
Three structural changes versus source:
1. **Chunked wire format = locked v1** (164-byte fixed header + ciphertext). Source's u8/u64/u40 bitpacker schema is replaced. Whole-file mode keeps bitpacker (no fixed-size constraint there).
2. **No signature recovery.** Source uses `signWithRecovery`/`verifyWithRecovery` (1-bit recovery flag); v1 is plain `ed25519_sign` because every consumer knows the server pubkey out-of-band. Saves a bit, simplifies verify path.
3. **Streaming-first chunked pipeline.** Encryption runs through a streaming AEAD; the ciphertext stream is chopped at `payload_per_chunk = chunkSize - 164` byte offsets. Last chunk is naturally short (no zero-padding). Header `payloadLen` is authoritative.
## Source-side scope
`../suaveplan/packages/universal/packer/` v2.0.0 — 37 `.ts` files, ~3 460 src + ~4 997 test LOC.
```
src/
├── abstract/ — AbstractPacker base (compression / AEAD / signing helpers)
├── streaming/ — normalizeSource(Uint8Array | ReadableStream | AsyncIterable)
├── packer/ — Packer (whole-file)
├── unpacker/ — Unpacker (whole-file)
├── chunked/
│ ├── packer/ — ChunkedPacker
│ ├── unpacker/ — ChunkedUnpacker
│ └── verifier/ — ChunkVerifier
├── types/
└── integration/
```
Source whole-file public surface: `Packer`, `Unpacker`, `signedPacketSchema`. Source chunked surface: `ChunkedPacker`, `ChunkedUnpacker`, `ChunkVerifier`, `chunkedPacketSchema` (replaced by v1 codec).
## Dependencies
All present in genesis:
- `@suaveplan/bitpacker` (`packages/primitives/bitpacker`) — used by whole-file mode
- `@suaveplan/crypto` (`packages/crypto/crypto`) — `encrypt`/`decrypt`, AEAD streaming, ed25519 sign/verify, hashing
- `@suaveplan/events` (`packages/events`) — `EventEmitter` base for progress events
- `@suaveplan/mime` (`packages/primitives/mime`) — MIME detection on pack input
- `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/streams`, `@opentelemetry/api`, `zod`
No new packages are required. No source-side blockers.
## What changes (port-time)
### 1. Layout
```
packages/data/packer/
├── package.json # @suaveplan/packer @ 0.0.0
├── tsconfig.json # extends @suaveplan/typescript-config/library
├── src/
│ ├── abstract/ # ported with minor cleanup
│ ├── streaming/ # ported as-is
│ ├── packer/ # whole-file Packer (bitpacker schema)
│ ├── unpacker/ # whole-file Unpacker
│ ├── chunked/
│ │ ├── codec/ # NEW: v1 byte-format header encode/decode
│ │ ├── packer/ # ChunkedPacker — emits v1 chunks via streaming pipeline
│ │ ├── unpacker/ # ChunkedUnpacker — consumes v1 chunks
│ │ └── verifier/ # ChunkVerifier — sig + AEAD verify, no decrypt
│ ├── error-codes.ts # PACKER_CODES via defineCodes
│ ├── schemas/ # Zod boundary schemas for options
│ └── index.ts # barrel — whole-file surface
└── src/chunked/index.ts # barrel — chunked surface (subpath ./chunked)
```
### 2. Subpath exports
```jsonc
"exports": {
".": { "import": "./dist/index.js", "types": "./dist/index.d.ts" },
"./chunked": { "import": "./dist/chunked/index.js", "types": "./dist/chunked/index.d.ts" },
"./streaming":{ "import": "./dist/streaming/index.js", "types": "./dist/streaming/index.d.ts" },
"./errors": { "import": "./dist/error-codes.js", "types": "./dist/error-codes.d.ts" }
}
```
### 3. Chunked v1 codec (fixed-layout, byte-aligned, big-endian)
Header layout per `drive-plugin-chunks-port` `CHUNKS-FORMAT-V1-01`. Repeated here for completeness:
```
metadata(100 B):
0..2 version uint16 = 0x0001
2..34 fileId bytes(32) // content hash of plaintext
34..38 chunkId uint32
38..42 totalChunks uint32
42..50 fileSize uint64
50..54 chunkSize uint32 // budget for non-final chunks
54..58 payloadLen uint32
58..60 flags uint16 // v1 MUST be 0x0000
60..84 nonce bytes(24) // XChaCha20 nonce
84..100 aeadTag bytes(16) // Poly1305 tag
sig(64 B):
100..164 ed25519 signature over metadata[0..100] ‖ ciphertext[0..payloadLen]
ciphertext(payloadLen B):
164..(164+payloadLen)
```
Last chunk: `payloadLen = fileSize − chunkId * (chunkSize − 164)`. Non-final chunks: `payloadLen = chunkSize − 164`. No zero padding.
### 4. Streaming chunked pipeline (write path)
```
PackSource ─► normalizeSource ─► AsyncIterable<Uint8Array>
─► hash(plaintext) → fileId (sha256, 32 B)
─► aead-encrypt-stream (XChaCha20-Poly1305, file key + per-chunk nonce)
─► chop at (chunkSize − 164) byte boundaries
─► for each slice:
derive nonce = baseNonce ⊕ (chunkId as 8 B little-endian, zero-extended to 24 B)
build metadata (100 B)
ed25519-sign(metadata ‖ ciphertext) → 64 B sig
emit metadata ‖ sig ‖ ciphertext
─► finally: build manifest (server-signed, fileId-addressed)
```
End-to-end streaming. No whole-file buffering.
### 5. Streaming chunked pipeline (read/verify path)
```
chunk-bytes ─► parseHeader (structural validation: version, flags, payloadLen invariants)
─► verifySig (metadata ‖ ciphertext) — reject on fail
─► aead-decrypt(ciphertext, nonce, aeadTag, fileKey)
─► emit plaintext slice
```
`ChunkVerifier` performs structural-parse + sig-verify only (no decrypt) — used by mesh peers that forward chunks without holding the file key.
### 6. Drops
- `signWithRecovery`/`verifyWithRecovery` calls on the chunked path — replaced with plain `ed25519_sign`/`ed25519_verify`. Whole-file mode keeps recovery semantics if the source had them (audit during port).
- Bitpacker schema for chunks (`chunkedPacketSchema`) — replaced by the v1 byte codec.
- Source's `EventEmitter` -based progress events — replaced by genesis `@suaveplan/events` `EventBus<TMap>` typed events.
### 7. Error codes
`PACKER_CODES = defineCodes("PACKER", [INVALID_SOURCE, COMPRESSION_FAILED, ENCRYPTION_FAILED, SIGNATURE_FAILED, VERIFICATION_FAILED, INVALID_FORMAT, VERSION_UNSUPPORTED, FLAGS_RESERVED_NONZERO, PAYLOAD_LEN_INVALID, ABORTED] as const)`. `PackerError extends SuaveplanError<PackerErrorCode>`. Frozen.
## Impact
- New package `packages/data/packer/` at `0.1.0`.
- New OpenSpec capability `packer` at `openspec/specs/packer/spec.md`.
- Unblocks `drive-plugin-chunks-port` (which depends on the chunked subpath).
- Source-side compatibility: **wire-format break** on the chunked side. Whole-file packed data from source v2.0.0 is unaffected (whole-file format unchanged).
## Out of scope
- Source's `integration/integration.test.ts` end-to-end suite — re-authored as conformance tests against the new format, not ported byte-for-byte.
- Compression on chunked path — v1 reserved `flags` bit 0 for `compressed`; not implemented in v1.
- Algorithm agility — v1 hardwires XChaCha20-Poly1305 + ed25519. Algo selector via `flags` bit 1 reserved for v2.
- Browser-specific `packer-browser` — separate change after this lands.

105
openspec/archive/2026-04-28-packer-port/specs/packer/spec.md Normal file
View File

@@ -0,0 +1,105 @@
# Packer — Spec Delta (packer-port)
## ADDED Requirements
### PACKER-01 — Whole-file pack/unpack round-trip
`Packer.pack(data, filename?)` followed by `Unpacker.unpack(packed, key, pubKey?)` MUST yield bytes equal to the original `data`. Optional MIME detection MUST surface in the returned metadata.
**Verified by:** `src/packer/packer.spec-conformance.test.ts` (`PACKER-01`).
### PACKER-02 — Whole-file gzip compression
When `compressionLevel > 0`, `pack` MUST gzip the payload before AEAD; `unpack` MUST gunzip after AEAD. Any compression failure MUST throw `PackerError(PACKER.COMPRESSION_FAILED)`.
**Verified by:** `src/packer/packer.spec-conformance.test.ts` (`PACKER-02`).
### PACKER-03 — Whole-file signing
When a `SignKey` is provided to `pack`, the produced `PackedData` MUST include an ed25519 signature over the canonical packed bytes. `unpack` with a corresponding `PublicKey` MUST verify the signature and throw `PackerError(PACKER.VERIFICATION_FAILED)` on mismatch.
**Verified by:** `src/packer/packer.spec-conformance.test.ts` (`PACKER-03`).
### PACKER-04 — Whole-file event lifecycle
`Packer` MUST emit typed events via `EventBus<PackerEventMap>`: `pack:start`, `pack:complete`, `pack:error`, with corresponding `unpack:*` events on the unpacker. Events MUST fire in deterministic FIFO order on completion.
**Verified by:** `src/packer/packer.spec-conformance.test.ts` (`PACKER-04`).
### PACKER-05 — Chunked v1 byte layout
A v1 chunk produced by `ChunkedPacker` MUST be the byte concatenation `metadata(100) ‖ sig(64) ‖ ciphertext(payloadLen)`, total `164 + payloadLen` bytes, big-endian. Field offsets within metadata MUST be exactly: `version` 0–2, `fileId` 2–34, `chunkId` 34–38, `totalChunks` 38–42, `fileSize` 42–50, `chunkSize` 50–54, `payloadLen` 54–58, `flags` 58–60, `nonce` 60–84, `aeadTag` 84–100. Sig occupies 100–164. Ciphertext starts at 164.
**Verified by:** `src/chunked/codec/chunk.spec-conformance.test.ts` (`PACKER-05`).
### PACKER-06 — `flags` reserved in v1
`flags` MUST be `0x0000` in v1. `parseHeader` MUST reject non-zero `flags` with `PackerError(PACKER.FLAGS_RESERVED_NONZERO)` before any further parse work.
**Verified by:** `src/chunked/codec/chunk.spec-conformance.test.ts` (`PACKER-06`).
### PACKER-07 — `payloadLen` invariants
For non-final chunks (`chunkId < totalChunks - 1`), `payloadLen` MUST equal `chunkSize - 164`. For the final chunk, `payloadLen` MUST equal `fileSize - chunkId * (chunkSize - 164)`. Mismatch MUST be rejected with `PackerError(PACKER.PAYLOAD_LEN_INVALID)` before sig verification.
**Verified by:** `src/chunked/codec/chunk.spec-conformance.test.ts` (`PACKER-07`).
### PACKER-08 — Streaming pipeline produces v1 chunks
`ChunkedPacker.pack(source, fileKey, signKey, options)` MUST stream-process the input via `normalizeSource → hash → aead-encrypt-stream → chop → per-chunk sign → emit`. The output MUST be an `AsyncIterable<{ chunkId, bytes }>` where each `bytes` is a v1-format chunk. The full source MUST NOT be buffered in memory during pack.
**Verified by:** `src/chunked/packer/packer.spec-conformance.test.ts` (`PACKER-08`).
### PACKER-09 — Non-final chunks are exactly `chunkSize` bytes
Every emitted chunk with `chunkId < totalChunks - 1` MUST be exactly `chunkSize` bytes on the wire (default `chunkSize = 1 MiB → payloadLen = 1048412`). No zero padding is added.
**Verified by:** `src/chunked/packer/packer.spec-conformance.test.ts` (`PACKER-09`).
### PACKER-10 — Final chunk is naturally short
The final chunk MUST be `164 + lastPayloadLen` bytes, where `lastPayloadLen = fileSize - (totalChunks - 1) * (chunkSize - 164)`. The final chunk MUST NOT be zero-padded to `chunkSize`.
**Verified by:** `src/chunked/packer/packer.spec-conformance.test.ts` (`PACKER-10`).
### PACKER-11 — Round-trip parity
`ChunkedUnpacker.unpack(chunks, fileKey, serverPubKey)` over the chunks emitted by `ChunkedPacker.pack(...)` MUST yield a plaintext stream byte-equal to the original input. Order MUST be `chunkId` ascending; out-of-order delivery is the consumer's responsibility to reorder before passing to unpacker.
**Verified by:** `src/chunked/unpacker/unpacker.spec-conformance.test.ts` (`PACKER-11`).
### PACKER-12 — Mutation detection
Any mutation to any signed byte (metadata field or any byte of ciphertext) MUST cause `ChunkedUnpacker.unpack` to throw `PackerError(PACKER.VERIFICATION_FAILED)` from the per-chunk sig-verify step before AEAD decryption is attempted.
**Verified by:** `src/chunked/unpacker/unpacker.spec-conformance.test.ts` (`PACKER-12`).
### PACKER-13 — Verifier without file key
`ChunkVerifier.verify(chunkBytes, serverPubKey)` MUST validate the structural invariants and the ed25519 signature without the file decryption key. It MUST return `{ ok: true, metadata }` for valid chunks and `{ ok: false, code }` for invalid chunks. It MUST NOT attempt AEAD decryption.
**Verified by:** `src/chunked/verifier/chunk-verifier.spec-conformance.test.ts` (`PACKER-13`).
### PACKER-14 — Verifier rejects forged sig
`ChunkVerifier.verify` MUST return `{ ok: false, code: "PACKER.VERIFICATION_FAILED" }` for a chunk whose signature does not validate under `serverPubKey`. Verifier MUST NOT consult the chunks plugin or any persistence layer.
**Verified by:** `src/chunked/verifier/chunk-verifier.spec-conformance.test.ts` (`PACKER-14`).
### PACKER-15 — Manifest signature
The manifest produced by `buildManifest(...)` MUST be ed25519-signed via `signManifest(manifest, signKey)`. `verifyManifest(manifest, serverPubKey)` MUST return `false` for any manifest whose signature does not validate; mutation of any signed field MUST cause verification failure.
**Verified by:** `src/chunked/manifest/manifest.spec-conformance.test.ts` (`PACKER-15`).
### PACKER-16 — Manifest matches chunk metadata
The manifest produced by a successful `ChunkedPacker.pack(...)` MUST satisfy: `manifest.fileId == chunkMeta[i].fileId` for all `i`, `manifest.totalChunks == chunkCount`, `manifest.chunkSize == options.chunkSize`, `manifest.fileSize == sum(payloadLen)`, and `manifest.chunks[i].payloadLen == chunkMeta[i].payloadLen` for all `i`.
**Verified by:** `src/chunked/manifest/manifest.spec-conformance.test.ts` (`PACKER-16`).
### PACKER-17 — Error registry namespaced and frozen
`PACKER_CODES` MUST be declared via `defineCodes("PACKER", [...])` from `@suaveplan/error`. `Object.isFrozen(PACKER_CODES)` MUST be `true`. Every code value MUST start with `PACKER.`. `PackerError` MUST extend `SuaveplanError<PackerErrorCode>`.
**Verified by:** `src/error-codes.spec-conformance.test.ts` (`PACKER-17`).

78
openspec/archive/2026-04-28-packer-port/tasks.md Normal file
View File

@@ -0,0 +1,78 @@
# Tasks: packer-port
## 1. Scaffold
- [ ] 1.1 Create `packages/data/packer/` with `package.json` (`@suaveplan/packer@0.0.0`), `tsconfig.json` extending `@suaveplan/typescript-config/library.json`, `vitest.config.ts` via `createTestConfig()`, `README.md`.
- [ ] 1.2 Workspace deps: `@suaveplan/bitpacker`, `@suaveplan/crypto`, `@suaveplan/events`, `@suaveplan/mime`, `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/streams`, `@opentelemetry/api`, `zod`. All `workspace:^`.
- [ ] 1.3 Subpath exports: `.`, `./chunked`, `./streaming`, `./errors`.
## 2. Foundations
- [ ] 2.1 `src/error-codes.ts` — `defineCodes("PACKER", [...])`. Sibling `PackerError`.
- [ ] 2.2 `src/schemas/packer.schemas.ts` — Zod boundary schemas for `PackerOptions`, `ChunkedPackerOptions`. Marker `// schema: boundary-only`.
- [ ] 2.3 `src/types/packer.types.ts` — `PackSource`, `PackerOptions`, `PackedData`, `ChunkedPackerOptions`, `ChunkMetadata`, `Manifest`, `EncryptionAlgorithm`.
## 3. Streaming source normaliser
- [ ] 3.1 Port `src/streaming/normalize-source.ts` from source. Accepts `Uint8Array | ReadableStream<Uint8Array> | AsyncIterable<Uint8Array> | Blob`. Returns `AsyncIterable<Uint8Array>`.
- [ ] 3.2 Sibling `normalize-source.test.ts`, `normalize-source.md`, `normalize-source.types.ts`.
## 4. Abstract base
- [ ] 4.1 `src/abstract/abstract.ts` — `AbstractPacker` extending `EventBus<PackerEventMap>`-backed emitter. Compression (gzip via `node:zlib`), AEAD wrappers, sign helpers.
- [ ] 4.2 Replace source's untyped `EventEmitter` with typed `EventBus<PackerEventMap>` from `@suaveplan/events`.
## 5. Whole-file mode (preserves bitpacker schema)
- [ ] 5.1 `src/packer/packer.ts` — `Packer extends AbstractPacker`. `pack(data, filename?): Promise<PackedData>` with optional gzip + AEAD + ed25519 sign. Bitpacker schema retained.
- [ ] 5.2 `src/unpacker/unpacker.ts` — `Unpacker extends AbstractPacker`. `unpack(packed, key, pubKey?): Promise<{ data, metadata }>`.
- [ ] 5.3 Sibling tests + `.md` for each.
## 6. Chunked v1 codec
- [ ] 6.1 `src/chunked/codec/header.ts` — `encodeHeader(metadata): Uint8Array(100)`, `parseHeader(buf): ChunkMetadata` with structural validation. Big-endian DataView reads/writes at locked offsets.
- [ ] 6.2 `src/chunked/codec/chunk.ts` — `encodeChunk({ metadata, sig, ciphertext })`, `parseChunk(buf)` with length checks.
- [ ] 6.3 `src/chunked/codec/chunk.spec-conformance.test.ts` — pin every byte offset against `CHUNKS-FORMAT-V1-01`.
## 7. Chunked packer
- [ ] 7.1 `src/chunked/packer/packer.ts` — `ChunkedPacker extends AbstractPacker`. `pack(source: PackSource, fileKey: EncryptKey, signKey: SignKey, opts?: ChunkedPackerOptions): AsyncIterable<{ chunkId, bytes }>`.
- [ ] 7.2 Pipeline: `normalizeSource → hash → aead-encrypt-stream → chop at (chunkSize - 164) → per-chunk sign → emit`.
- [ ] 7.3 `pack` resolves with the assembled manifest (server-signed, fileId-addressed) at end of stream.
- [ ] 7.4 Per-chunk nonce derivation: `nonce = baseNonce ⊕ chunkId-as-LE-bytes` (24 B).
## 8. Chunked unpacker
- [ ] 8.1 `src/chunked/unpacker/unpacker.ts` — `ChunkedUnpacker`. `unpack(chunks: AsyncIterable<Uint8Array>, fileKey: EncryptKey, serverPubKey: PublicKey): AsyncIterable<Uint8Array>` (plaintext stream).
- [ ] 8.2 Per-chunk: structural-parse → flags check → payloadLen invariants → sig-verify → aead-decrypt → emit `payloadLen` plaintext bytes.
## 9. Chunk verifier (no decrypt)
- [ ] 9.1 `src/chunked/verifier/chunk-verifier.ts` — `ChunkVerifier`. `verify(chunkBytes: Uint8Array, serverPubKey: PublicKey): VerifyResult` with `{ ok: true, metadata }` or `{ ok: false, code: PackerErrorCode }`.
- [ ] 9.2 Used by mesh peers that forward chunks without holding the file decryption key.
## 10. Manifest
- [ ] 10.1 `src/chunked/manifest/manifest.ts` — `buildManifest(metadata[], encryption, fileKey, signKey): Manifest`, `signManifest(manifest, signKey)`, `verifyManifest(manifest, serverPubKey)`.
- [ ] 10.2 Zod schema for manifest in `src/schemas/manifest.schema.ts`.
## 11. Spec
- [ ] 11.1 `openspec/specs/packer/spec.md` — `PACKER-WHOLE-01` … `PACKER-WHOLE-04`, `PACKER-CHUNK-01` … `PACKER-CHUNK-12`, `PACKER-CODES-01`. Each with `**Verified by:**` line.
## 12. Conformance tests
- [ ] 12.1 `src/packer/packer.spec-conformance.test.ts` — whole-file pack/unpack round-trip, signed and unsigned, with and without gzip (`PACKER-WHOLE-01` … `-04`).
- [ ] 12.2 `src/chunked/codec/chunk.spec-conformance.test.ts` — exact byte-offset pinning (`PACKER-CHUNK-01` v1 layout, `PACKER-CHUNK-02` flags=0 reject, `PACKER-CHUNK-03` payloadLen invariants).
- [ ] 12.3 `src/chunked/packer/packer.spec-conformance.test.ts` — streaming pipeline end-to-end (`PACKER-CHUNK-04`); fixed non-final size (`PACKER-CHUNK-05`); short final chunk (`PACKER-CHUNK-06`).
- [ ] 12.4 `src/chunked/unpacker/unpacker.spec-conformance.test.ts` — chunks-out → plaintext-in matches original (`PACKER-CHUNK-07`); rejects mutated chunks at sig stage (`PACKER-CHUNK-08`).
- [ ] 12.5 `src/chunked/verifier/chunk-verifier.spec-conformance.test.ts` — verifies without file key (`PACKER-CHUNK-09`); rejects bad sig (`PACKER-CHUNK-10`).
- [ ] 12.6 `src/chunked/manifest/manifest.spec-conformance.test.ts` — manifest sign + verify (`PACKER-CHUNK-11`); manifest matches chunk metadata (`PACKER-CHUNK-12`).
- [ ] 12.7 `src/error-codes.spec-conformance.test.ts` — frozen registry, `PACKER.*` namespace (`PACKER-CODES-01`).
## 13. Verify
- [ ] 13.1 `bunx turbo lint typecheck test --filter=@suaveplan/packer` green.
- [ ] 13.2 `bunx turbo test:coverage --filter=@suaveplan/packer` 100/100/100/100.
- [ ] 13.3 `bun scripts/openspec-conformance.ts @suaveplan/packer` PASS.
- [ ] 13.4 Sibling `.md` exists for every non-test, non-types, non-barrel `.ts` in `src/`.