Initial Commit
This commit is contained in:
0
openspec/changes/.gitkeep
Normal file
0
openspec/changes/.gitkeep
Normal file
38
openspec/changes/REACT_AUDIO_WAVE_REWRITE_COMPLETE.md
Normal file
38
openspec/changes/REACT_AUDIO_WAVE_REWRITE_COMPLETE.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# React / R3F audio wave — rewrite COMPLETE
|
||||
|
||||
**Rewrite date**: 2026-05-15
|
||||
**Previous flag**: `NEEDS-REWRITE` (predated 2026-05-14 architecture shift)
|
||||
|
||||
All five proposals have been rewritten against the unified architecture:
|
||||
|
||||
| Change folder | Spec delta | Design |
|
||||
|---|---|---|
|
||||
| `openspec/changes/enterprise-react-audio/` | `specs/react-audio/spec.md` | `design.md` |
|
||||
| `openspec/changes/enterprise-react-audio-player/` | `specs/react-audio-player/spec.md` | `design.md` |
|
||||
| `openspec/changes/enterprise-react-audio-stream/` | `specs/react-audio-stream/spec.md` | `design.md` |
|
||||
| `openspec/changes/enterprise-react-audio-fingerprint/` | `specs/react-audio-fingerprint/spec.md` | `design.md` |
|
||||
| `openspec/changes/enterprise-gaming-r3f-audio/` | `specs/r3f-audio/spec.md` | `design.md` |
|
||||
|
||||
## What the rewrites collectively now satisfy
|
||||
|
||||
- **Target path** is `packages/<plugin>/<pkg>` per the current monorepo convention.
|
||||
- **`@suaveplan/audio-engine` references replaced** by `@suaveplan/audio-engine-browser` (for rendering) + `@suaveplan/audio-plugins-fx` (for chain + plugin contract) + `@suaveplan/audio-controllers` (for MIDI binding).
|
||||
- **Spec deltas exist** under `specs/<capability>/spec.md` in OpenSpec `Requirement` / `Scenario` / `**Verified by:**` form per `openspec/AGENTS.md` §0c.
|
||||
- **`design.md` exists** with architecture, integration notes, and decisions.
|
||||
- **`## Impact` → `### Affected dependents`** is per §14a template.
|
||||
- **Parity Checklist** present where the proposal is a migration from upstream.
|
||||
- **Error codes** use the dotted `defineCodes` form per AGENTS.md §7.
|
||||
- **Universal where possible, R3F-gated only where it belongs**: `r3f-audio` consumes the four game-audio building blocks from `audio-plugins-fx` rather than re-implementing spatial / proximity / music / voice logic.
|
||||
|
||||
## Prerequisite ordering
|
||||
|
||||
Each rewritten proposal lists its prerequisites at the top of its `tasks.md`. The conservative order:
|
||||
|
||||
1. `enterprise-audio-plugins-fx` (foundation FX + chain + ambient + bus registry)
|
||||
2. `enterprise-audio-engine-browser`, `enterprise-audio-engine-server` (engine adapters)
|
||||
3. `enterprise-audio-controllers` (MIDI binding helpers)
|
||||
4. `enterprise-audio-analysis-spectrum` (FFT consumed by fingerprint)
|
||||
5. `enterprise-react-audio` (umbrella React surface)
|
||||
6. `enterprise-react-audio-player`, `enterprise-react-audio-stream`, `enterprise-react-audio-fingerprint`, `enterprise-gaming-r3f-audio` (in any order — siblings of the umbrella)
|
||||
|
||||
This file is retained as the historical record of the wave rewrite. It is informational only; the proposals themselves are the source of truth.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Design — StructuredCloneCodec for @suaveplan/cache-storage
|
||||
|
||||
> Non-normative. Normative requirements live in `./specs/cache-storage/spec.md`.
|
||||
|
||||
## Goal
|
||||
|
||||
Add a third `Codec<T>` implementation to `@suaveplan/cache-storage` that round-trips the structured-clone-ish subset of JS values — `Date`, `Map`, `Set`, `Uint8Array`, `bigint`, `RegExp`, `URL`, and arbitrarily nested combinations — over `@suaveplan/cbor`. The default `JsonCodec` remains untouched; callers opt in.
|
||||
|
||||
## Codec interface (unchanged)
|
||||
|
||||
```ts
|
||||
// existing — packages/flow/cache-storage/src/codec/codec.types.ts
|
||||
export interface Codec<T> {
|
||||
encode(value: T): Uint8Array;
|
||||
decode(bytes: Uint8Array): T;
|
||||
}
|
||||
```
|
||||
|
||||
The new codec implements this interface verbatim. No constructor surface beyond the bare class is required for the default case; a future `{ tags?: ReadonlyMap<number, TagHandler> }` option is sketched below but is out of scope for this change.
|
||||
|
||||
## Tag scheme
|
||||
|
||||
CBOR tags from the IANA "Concise Binary Object Representation (CBOR) Tags" registry. Choosing standard tags keeps the on-wire format interop-friendly (a non-JS reader can decode the blob with any RFC 8949 implementation plus the standard tag set).
|
||||
|
||||
| JS type | CBOR tag | Payload shape | IANA reference |
|
||||
|---------------|----------|----------------------------------------------------------------|---------------------------------|
|
||||
| `Date` | `0` | RFC 3339 string (`date.toISOString()`) | RFC 8949 §3.4.1 |
|
||||
| `bigint > 0` | `2` | byte string, big-endian magnitude | RFC 8949 §3.4.3 (positive bignum)|
|
||||
| `bigint < 0` | `3` | byte string, big-endian magnitude of `-1 - n` | RFC 8949 §3.4.3 (negative bignum)|
|
||||
| `URL` | `32` | string (the `URL.href`) | RFC 8949 §3.4.5.3 (URI) |
|
||||
| `RegExp` | `35` | string (`regex.source`); flags lost — see "Trade-offs" below | RFC 8949 §3.4.5.4 (PCRE regex) |
|
||||
| `Uint8Array` | `64` | byte string | RFC 8746 §3.3 |
|
||||
| `Set` | `258` | array of items (recursively encoded) | https://github.com/input-output-hk/cbor-sets-spec |
|
||||
| `Map` | `259` | array of `[key, value]` pairs (recursively encoded) | https://github.com/shanewholloway/js-cbor-codec/blob/master/docs/CBOR-259-spec--explicit-maps.md |
|
||||
|
||||
Plain objects, arrays, `null`, `boolean`, finite `number`, and `string` round-trip natively through CBOR — no transformation, no tag.
|
||||
|
||||
> Note on the Map choice: RFC 8949 §3.1 major type 5 already carries maps natively, but CBOR maps key on the encoded *bytes* of the key, which means a JS `Map` with object keys would either lose distinct identity (every `{}` encodes to the same bytes) or throw. Tag `259` was introduced by the JS ecosystem specifically to disambiguate "this came from a JS Map and the keys are values, not strings" — picking it preserves round-trip without breaking the major-type-5 invariants for plain-object encoding.
|
||||
|
||||
## Encode pipeline
|
||||
|
||||
```text
|
||||
input value
|
||||
│
|
||||
▼
|
||||
walk(value) ── recursive lift
|
||||
│ plain JS value ─► returned as-is
|
||||
│ carrier type ─► returned as new Tagged(tag, walk(payload))
|
||||
│ unsupported type ─► throw CACHE_STORAGE.CODEC_UNSUPPORTED_TYPE
|
||||
│
|
||||
▼
|
||||
encode(walked, { deterministic: false }) ── @suaveplan/cbor
|
||||
│
|
||||
▼
|
||||
Uint8Array
|
||||
```
|
||||
|
||||
Detection order is intentional — most specific first — so that subclasses do not slip through. `Array.isArray` is checked before "plain object"; `instanceof Date`, `instanceof Map`, etc. come before the duck-typed fallthrough.
|
||||
|
||||
## Decode pipeline
|
||||
|
||||
```text
|
||||
input bytes
|
||||
│
|
||||
▼
|
||||
decode(bytes, { tagHandling: "registry", tags: TAG_REGISTRY }) ── @suaveplan/cbor
|
||||
│
|
||||
▼ each Tagged hit
|
||||
TagHandler[tag](payload) ── reverse-lift
|
||||
│ known tag ─► returns the reconstructed JS carrier
|
||||
│ unknown tag ─► throw CACHE_STORAGE.CODEC_UNKNOWN_TAG
|
||||
│
|
||||
▼
|
||||
value
|
||||
```
|
||||
|
||||
Unknown tags fail loud rather than passing through as `Tagged` objects. Silent pass-through would change the runtime shape the caller observes versus what they wrote, breaking the `decode(encode(x)) deepEquals x` contract every other codec honours.
|
||||
|
||||
### TagHandler shape (from `@suaveplan/cbor`)
|
||||
|
||||
```ts
|
||||
type TagHandler = (payload: DecodedValue) => unknown;
|
||||
```
|
||||
|
||||
The codec builds a frozen `Map<number, TagHandler>` once at module load. The registry is `internal` for this change; a public `register(tag, handler)` surface is sketched in the "Future work" section below.
|
||||
|
||||
## Error handling
|
||||
|
||||
Two new entries extend `CACHE_STORAGE_CODES`:
|
||||
|
||||
| Code | Raised by | Caller-actionable response |
|
||||
|-------------------------------|---------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
|
||||
| `CODEC_UNSUPPORTED_TYPE` | `StructuredCloneCodec.encode` when the walk hits `Function`, `Symbol`, `WeakMap`, `WeakSet`, `Promise`, or any class instance without a registered tag | Switch to `JsonCodec` (for primitives only) or pre-serialise the value yourself |
|
||||
| `CODEC_UNKNOWN_TAG` | `StructuredCloneCodec.decode` when CBOR yields a `Tagged(n, …)` for `n` not in the registry | Upgrade the codec version, or pass a custom tag map (future-work surface) |
|
||||
|
||||
Existing `CODEC_ERROR` is reused for malformed CBOR bytes (catch-all for `CborError` thrown by `@suaveplan/cbor/decoder`) so the boundary remains a single "decode failed" code for back-compat — the new `CODEC_UNKNOWN_TAG` covers only the well-formed-but-unrecognised case.
|
||||
|
||||
## Fallback strategy when CBOR decoder hits an unknown tag
|
||||
|
||||
The codec configures `@suaveplan/cbor`'s decoder with a `TagRegistry` containing handlers for the eight tags above. For any other tag number, the registry returns a `Tagged` placeholder; the codec then inspects the decoded tree post-hoc — any leftover `Tagged` instance triggers `CACHE_STORAGE.CODEC_UNKNOWN_TAG` carrying the offending tag number in the error context. Walking the tree once for the post-check is O(n) over the decoded value count and runs at most once per `decode` call.
|
||||
|
||||
Why not configure the cbor decoder to throw immediately on unknown tags? Because `@suaveplan/cbor`'s tag-handling mode is a single decoder-wide flag; flipping it to "throw" would also kill any caller who later wants to mix this codec with a custom fallback handler. Inspecting post-hoc keeps the policy local to this codec.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/flow/cache-storage/src/codec/
|
||||
codec.types.ts # unchanged
|
||||
identity.ts # unchanged
|
||||
identity.test.ts # extended with conformance harness call
|
||||
identity.md # unchanged
|
||||
json.ts # unchanged
|
||||
json.test.ts # extended with conformance harness call
|
||||
json.md # unchanged
|
||||
structured-clone.ts # NEW — codec implementation (~180 LoC)
|
||||
structured-clone.test.ts # NEW — conformance + negative cases
|
||||
structured-clone.md # NEW — sibling docs
|
||||
conformance.ts # NEW — runCodecConformance harness
|
||||
conformance.test.ts # NEW — harness self-test
|
||||
conformance.md # NEW — sibling docs
|
||||
index.ts # extended — explicit named re-export added
|
||||
```
|
||||
|
||||
The 300-line soft cap is comfortably met. If the lift table grows (custom-tag registration), `structured-clone.tags.ts` extracts cleanly.
|
||||
|
||||
## Trade-offs
|
||||
|
||||
- **`RegExp.flags` is dropped on round-trip.** RFC 8949's PCRE-regex tag carries only the pattern string; flags are not part of the registry payload spec. Documented in the sibling `.md`. Callers needing flags can wrap the regex in `{ source, flags }` and cache that.
|
||||
- **`number === NaN` / `±Infinity` survive CBOR but not JSON.** With `StructuredCloneCodec` these round-trip cleanly (CBOR has float specials); `JsonCodec` already turns them into `null`. Worth noting in docs but not a defect of this codec.
|
||||
- **No transferable objects.** `ArrayBuffer.transfer`, `MessagePort`, `ImageBitmap` are out of scope — they belong to browser-only structured clone, not a universal-tier cache codec.
|
||||
- **CBOR map key constraint.** A JS `Map` with object keys round-trips because we use tag `259` (array of `[k, v]` pairs), where each `k` is recursively encoded as a full CBOR item. The major-type-5 "duplicate keys collide" hazard does not apply.
|
||||
|
||||
## Future work (explicitly deferred)
|
||||
|
||||
- Public `StructuredCloneCodec` constructor option `{ tags?: ReadonlyMap<number, { encode?, decode? }> }` so callers can register their own carriers (e.g. `Decimal`, `Big`, domain value objects).
|
||||
- Promotion of `runCodecConformance` to `@suaveplan/testing` if a second consumer adopts the `Codec<T>` interface.
|
||||
- Streaming variant `StructuredCloneStreamCodec` for >10 MiB blobs once a real consumer asks. Current synchronous design is the right default.
|
||||
@@ -0,0 +1,89 @@
|
||||
# Add StructuredCloneCodec (CBOR-backed) to @suaveplan/cache-storage
|
||||
|
||||
**Package**: `@suaveplan/cache-storage` (universal)
|
||||
**Status**: PROPOSED
|
||||
**Type**: additive — new optional codec, JSON default unchanged
|
||||
**Picks up deferred work from**: `openspec/archive/2026-05-16-add-cache-storage-redis-backends/`
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/cache-storage/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
The archived change `add-cache-storage-redis-backends` shipped `StorageCache<T>` with two codecs (`JsonCodec`, `IdentityCodec`) and explicitly deferred the third under its `## Out of scope` heading:
|
||||
|
||||
> **`StructuredCloneCodec` / CBOR codec.** Deferred. JSON + Identity cover the common case; CBOR-backed structured-clone (`Date`/`Map`/`Set` faithful round-trip) ships in a follow-up alongside the cbor catalog entry. No placeholder code lands in this change.
|
||||
|
||||
That follow-up is now unblocked: `@suaveplan/cbor` ships on genesis with a complete RFC 8949 encoder/decoder, `Tagged` value, and registry-extensible tag handlers. Callers that cache rich JS values (`Date`, `Map`, `Set`, `Uint8Array`, `bigint`, nested combinations) currently either lose information through `JsonCodec` (`Date` → string; `Map`/`Set` → `{}` or `[]`; `Uint8Array` → object-with-numeric-keys; `bigint` → `JSON.stringify` throw) or are forced to author their own pre/post serializers. Neither is acceptable for an enterprise cache layer that already exposes `Codec<T>` as the documented swap-point.
|
||||
|
||||
This change adds an opt-in `StructuredCloneCodec` that round-trips the structured-clone-ish subset over `@suaveplan/cbor`, leaving `JsonCodec` as the back-compat default.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/cache-storage` (additive only)
|
||||
|
||||
- ADDED: `StructuredCloneCodec<T> implements Codec<T>` — new `src/codec/structured-clone.ts`. `encode` walks the input, lifts non-JSON-native carriers (`Date`, `Map`, `Set`, `Uint8Array`, `bigint`, `RegExp`, `URL`) into `Tagged(tagNumber, payload)` per the table in `design.md`, then delegates to `encode` from `@suaveplan/cbor`. `decode` calls `decode` from `@suaveplan/cbor` with a `TagHandler` map that reverses the lift. Plain objects, arrays, primitives, `null`, `boolean`, finite `number`, and `string` round-trip natively through CBOR with zero transformation.
|
||||
- ADDED: Tag numbers chosen from IANA-registered CBOR tags so the wire format is interop-friendly: `0` (RFC 3339 date string — used for `Date`), `258` (Set), `259` (Map with explicit key/value pairs), `64`–`72` reserved typed-array tags (`Uint8Array` uses tag `64`), `2`/`3` (positive/negative bignum for `bigint`), `35` (PCRE regex string for `RegExp`), `32` (URI for `URL`). All references are in `design.md`.
|
||||
- ADDED: Subpath export `./codec/structured-clone` in `package.json` matching the existing `./codec/json` and `./codec/identity` shape, plus re-export from `./src/codec/index.ts`.
|
||||
- ADDED: Two new entries in `CACHE_STORAGE_CODES` via `defineCodes("CACHE_STORAGE", [...])`: `CODEC_UNSUPPORTED_TYPE` (raised on encode of a value the codec cannot lift — `Function`, `Symbol`, `WeakMap`, `WeakSet`, DOM nodes, `Promise`, class instance without a registered tag handler) and `CODEC_UNKNOWN_TAG` (raised on decode of a CBOR tag the codec does not recognise AND for which no caller-supplied fallback handler exists).
|
||||
- ADDED: Conformance harness `src/codec/conformance.ts` exporting `runCodecConformance(codec, { name, supportsStructuredClone })` — round-trip cases covering: primitives (null, boolean, number, string), nested plain objects, arrays, `Date`, `Map`, `Set`, `Uint8Array`, `bigint` (positive + negative), `RegExp`, `URL`, deeply nested combinations, empty collections, and (when `supportsStructuredClone: false`) the `unsupported-type-on-JSON` negative case. The existing `JsonCodec` and `IdentityCodec` tests gain a thin harness-invocation wrapper so all three codecs share one contract; the new codec's test file invokes it with `supportsStructuredClone: true`.
|
||||
|
||||
### `@suaveplan/cbor` (consumed; no changes)
|
||||
|
||||
- The `StructuredCloneCodec` imports `encode`, `decode`, `Tagged`, and `type TagHandler` directly from `@suaveplan/cbor` per CLAUDE.md rule 4 (no re-exports).
|
||||
|
||||
### Default codec unchanged
|
||||
|
||||
- `StorageCache`'s constructor still defaults `codec` to `new JsonCodec<T>()`. Existing callers see no behaviour change.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Cross-realm `structuredClone()` parity.** The HTML structured-clone algorithm covers `ArrayBuffer` transfer, `ImageBitmap`, `Blob`, `File`, etc. This codec covers only the JS-language subset relevant to caching. Browser-only types stay out of a universal-tier package.
|
||||
- **Custom user tag registration.** The codec exposes the handler map internally; a public `registerTag` surface is a follow-up if real callers ask for it. Documented as such — not silently missing.
|
||||
- **Wiring the new codec into `@suaveplan/cache-redis`.** Out of scope here; `cache-redis` already consumes the `Codec<T>` interface from `@suaveplan/cache-storage/codec`, so callers can already `new StructuredCloneCodec()` and pass it. No code change is needed in `cache-redis` for that to work.
|
||||
- **Promoting the conformance harness to `@suaveplan/testing`.** The harness lives inside `cache-storage` for now; if other packages adopt the codec interface it gets lifted.
|
||||
|
||||
## Source mapping
|
||||
|
||||
- `openspec/archive/2026-05-16-add-cache-storage-redis-backends/proposal.md` — the deferral text quoted above.
|
||||
- `packages/flow/cache-storage/src/codec/` — existing `Codec<T>` seam (`codec.types.ts`, `json.ts`, `identity.ts`, `index.ts`). New file `structured-clone.ts` plugs into the same barrel and exports.
|
||||
- `packages/flow/cache-storage/src/storage-cache/storage-cache.ts:13-14,54,93,168` — codec seam (constructor parameter, encode/decode call sites). No changes required: the new codec is interface-compatible.
|
||||
- `packages/flow/cache-storage/src/errors/errors.ts` — `CACHE_STORAGE_CODES` registry to extend with two new codes via the same `defineCodes` call.
|
||||
- `packages/flow/cbor/src/index.ts` — public root barrel; `encode`, `decode`, `Tagged`, `type TagHandler` are the exports consumed.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Tag numbers chosen from the IANA CBOR tag registry (0, 2, 3, 32, 35, 64, 258, 259) | Standardised wire format is the only sane choice for a cache that may be read by a non-JS consumer (Redis blob, file dump). Inventing private tag numbers would lock the format to this codec forever. |
|
||||
| Conformance harness lives in `cache-storage`, not `@suaveplan/testing` | `@suaveplan/testing` carries zero `@suaveplan/*` runtime deps (rule 8). The harness must import the `Codec<T>` interface to type its parameter; lifting it would invert the dep graph. Promotion is a separate change if a second consumer appears. |
|
||||
| Unknown-tag default: throw `CODEC_UNKNOWN_TAG` (not silent pass-through) | A cache that silently turns an unknown tag into a `Tagged` wrapper changes the runtime type the caller observes versus what they cached. Failing loudly preserves the codec's round-trip contract and matches the `JsonCodec` "decode failure = throw" precedent. |
|
||||
| `RegExp` and `URL` lifted (not just structured-clone-spec types) | These are the two most common JS-host carriers that round-trip cleanly through one IANA tag each and that callers reliably cache. Adding them costs ~10 lines and avoids "almost structured-clone" surprise. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
- `src/codec/structured-clone.test.ts` — every `**Verified by:**` line in the spec delta, invoked via `runCodecConformance(new StructuredCloneCodec(), { name: "StructuredCloneCodec", supportsStructuredClone: true })` plus direct tests for `CODEC_UNSUPPORTED_TYPE` and `CODEC_UNKNOWN_TAG`.
|
||||
- `src/codec/conformance.test.ts` — harness self-test: passing a deliberately broken codec causes the harness assertions to fail, proving the harness has teeth (no tautological green).
|
||||
- `src/codec/json.test.ts` and `src/codec/identity.test.ts` — extended with a `runCodecConformance(codec, { supportsStructuredClone: false })` call to prove the shared contract holds for the existing codecs at the JSON subset.
|
||||
- `src/errors/errors.test.ts` — extended to assert the two new codes (`CODEC_UNSUPPORTED_TYPE`, `CODEC_UNKNOWN_TAG`) are reachable from real call sites in `StructuredCloneCodec`.
|
||||
- `bunx turbo test:coverage --filter=@suaveplan/cache-storage` — 100/100/100/100 retained.
|
||||
- `bunx turbo lint typecheck test --filter=@suaveplan/cache-storage` — green.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/cache-storage` to enumerate.
|
||||
|
||||
- `@suaveplan/cache-redis` — [ ] tests run / [x] docs reviewed / [x] no action needed (reason: consumes the `Codec<T>` interface from `@suaveplan/cache-storage/codec`; the new codec is interface-compatible and opt-in. No source change required; a follow-up README note can mention "pass `new StructuredCloneCodec()` if you want `Date`/`Map`/`Set` round-trips" but does not block this change.)
|
||||
- No other dependents at time of writing — verified by `bunx turbo ls --filter=...@suaveplan/cache-storage` returning only `cache-redis`.
|
||||
|
||||
### Affected upstreams
|
||||
|
||||
- `@suaveplan/cbor` — newly consumed via `workspace:^`. Not modified. [x] no action needed.
|
||||
- `@suaveplan/error` — `defineCodes` extended (additive); existing codes unchanged. [x] no action needed.
|
||||
- `@suaveplan/cache`, `@suaveplan/storage`, `@suaveplan/events`, `@suaveplan/types` — not touched.
|
||||
|
||||
## Migration
|
||||
|
||||
This is purely additive: no error renames, no default flips, no removed exports. Callers opt in by constructing `StorageCache({ storage, codec: new StructuredCloneCodec() })`. Existing callers using the default `JsonCodec` see byte-identical behaviour.
|
||||
@@ -0,0 +1,186 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: cache-storage (cache-storage)
|
||||
|
||||
**Package**: `@suaveplan/cache-storage`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: StructuredCloneCodec round-trips the structured-clone-ish JS subset
|
||||
|
||||
The package SHALL export `StructuredCloneCodec<T> implements Codec<T>` from `./codec/structured-clone`. The codec SHALL faithfully round-trip the following JS value types through `encode` then `decode`: `null`, `boolean`, finite `number`, `string`, plain object, array, `Date`, `Map`, `Set`, `Uint8Array`, `bigint` (positive and negative), `RegExp` (source only — flags excluded by tag-35 wire format), `URL`, and arbitrary nestings of the above. Encoding SHALL delegate to `encode` from `@suaveplan/cbor`; decoding SHALL delegate to `decode` from `@suaveplan/cbor` with a tag-handler registry that reverses the lift table.
|
||||
|
||||
#### Scenario: round-trip preserves Date instances
|
||||
|
||||
- **WHEN** a caller calls `codec.decode(codec.encode(new Date("2026-05-17T12:00:00.000Z")))`
|
||||
- **THEN** the result is `instanceof Date` AND its `.getTime()` equals `new Date("2026-05-17T12:00:00.000Z").getTime()`
|
||||
|
||||
#### Scenario: round-trip preserves Map with object keys and values
|
||||
|
||||
- **WHEN** a caller round-trips `new Map<unknown, unknown>([[{ id: 1 }, "a"], [{ id: 2 }, new Set([1, 2])]])`
|
||||
- **THEN** the result is `instanceof Map`, has `size === 2`, AND iterating yields keys/values deep-equal to the input (the second value being `instanceof Set` with members `1` and `2`)
|
||||
|
||||
#### Scenario: round-trip preserves Uint8Array byte identity
|
||||
|
||||
- **WHEN** a caller round-trips `new Uint8Array([0, 1, 127, 128, 255])`
|
||||
- **THEN** the result is `instanceof Uint8Array` AND every byte is equal to the input at the same index
|
||||
|
||||
#### Scenario: round-trip preserves bigint magnitude and sign
|
||||
|
||||
- **WHEN** a caller round-trips both `2n ** 200n` and `-(2n ** 200n) - 7n`
|
||||
- **THEN** each result is `typeof "bigint"` AND strictly equals the input
|
||||
|
||||
#### Scenario: deeply nested structures survive
|
||||
|
||||
- **WHEN** a caller round-trips `{ users: new Map([["alice", { tags: new Set(["admin"]), joined: new Date(0), avatar: new Uint8Array([1, 2, 3]) }]]) }`
|
||||
- **THEN** the result deep-equals the input across `Map`, `Set`, `Date`, and `Uint8Array` boundaries
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"round-trip preserves Date instances"` AND `src/codec/structured-clone.test.ts::"round-trip preserves Map with object keys and values"` AND `src/codec/structured-clone.test.ts::"round-trip preserves Uint8Array byte identity"` AND `src/codec/structured-clone.test.ts::"round-trip preserves bigint magnitude and sign"` AND `src/codec/structured-clone.test.ts::"deeply nested structures survive"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: StructuredCloneCodec uses IANA-registered CBOR tags
|
||||
|
||||
The codec SHALL use the following CBOR tag numbers from the IANA CBOR Tags registry: `0` (RFC 3339 date-time string) for `Date`; `2` (positive bignum) for `bigint > 0n`; `3` (negative bignum) for `bigint < 0n`; `32` (URI) for `URL`; `35` (PCRE regex) for `RegExp`; `64` (`Uint8` typed array) for `Uint8Array`; `258` (Set) for `Set`; `259` (Map with explicit key/value pairs) for `Map`. The codec SHALL NOT invent private tag numbers for any carrier listed in the round-trip requirement above.
|
||||
|
||||
#### Scenario: encoded Date emits CBOR tag 0
|
||||
|
||||
- **WHEN** a caller calls `codec.encode(new Date("2026-05-17T12:00:00.000Z"))` AND the result is fed back through `decode` from `@suaveplan/cbor` with default tag handling
|
||||
- **THEN** the decoded top-level value is `instanceof Tagged` with `.tag === 0` AND `.value` is the RFC 3339 string
|
||||
|
||||
#### Scenario: encoded Uint8Array emits CBOR tag 64
|
||||
|
||||
- **WHEN** a caller calls `codec.encode(new Uint8Array([1, 2, 3]))` AND the result is fed back through `decode` from `@suaveplan/cbor` with default tag handling
|
||||
- **THEN** the decoded top-level value is `instanceof Tagged` with `.tag === 64`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"Date encodes under CBOR tag 0"` AND `src/codec/structured-clone.test.ts::"Uint8Array encodes under CBOR tag 64"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CODEC_UNSUPPORTED_TYPE on encode of unencodable carriers
|
||||
|
||||
`StructuredCloneCodec.encode` SHALL throw `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_UNSUPPORTED_TYPE"` when the input contains (directly or transitively) any of: `Function`, `Symbol`, `WeakMap`, `WeakSet`, `Promise`, or any class instance whose constructor is not one of the registered carriers (`Date`, `Map`, `Set`, `Uint8Array`, `RegExp`, `URL`). The error SHALL carry the failing type name in its message or context.
|
||||
|
||||
#### Scenario: encoding a function throws CODEC_UNSUPPORTED_TYPE
|
||||
|
||||
- **WHEN** a caller calls `codec.encode((() => 1) as unknown)`
|
||||
- **THEN** the call throws `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_UNSUPPORTED_TYPE"`
|
||||
|
||||
#### Scenario: encoding a Symbol nested in an object throws CODEC_UNSUPPORTED_TYPE
|
||||
|
||||
- **WHEN** a caller calls `codec.encode({ ok: 1, bad: Symbol("x") } as unknown)`
|
||||
- **THEN** the call throws `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_UNSUPPORTED_TYPE"`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"encode throws CODEC_UNSUPPORTED_TYPE on a function"` AND `src/codec/structured-clone.test.ts::"encode throws CODEC_UNSUPPORTED_TYPE on a nested Symbol"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CODEC_UNKNOWN_TAG on decode of an unregistered CBOR tag
|
||||
|
||||
`StructuredCloneCodec.decode` SHALL throw `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_UNKNOWN_TAG"` when the input CBOR bytes contain a `Tagged(n, …)` value whose tag `n` is not one of the registered tags (`0`, `2`, `3`, `32`, `35`, `64`, `258`, `259`). The error context SHALL carry the offending tag number. The codec SHALL NOT silently return the value as a `Tagged` wrapper.
|
||||
|
||||
#### Scenario: decoding an unrecognised tag throws CODEC_UNKNOWN_TAG
|
||||
|
||||
- **WHEN** a caller encodes `new Tagged(9999, "x")` via `encode` from `@suaveplan/cbor` AND passes the resulting bytes to `codec.decode`
|
||||
- **THEN** the call throws `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_UNKNOWN_TAG"`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"decode throws CODEC_UNKNOWN_TAG on an unregistered tag"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CODEC_ERROR on malformed CBOR bytes
|
||||
|
||||
`StructuredCloneCodec.decode` SHALL throw `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_ERROR"` when the input bytes are not valid CBOR (catching any `CborError` thrown by `decode` from `@suaveplan/cbor` and re-wrapping). The original `CborError` SHALL be attached as `cause`.
|
||||
|
||||
#### Scenario: decoding random bytes throws CODEC_ERROR with cbor cause
|
||||
|
||||
- **WHEN** a caller calls `codec.decode(new Uint8Array([0xff, 0xff, 0xff, 0xff]))` for bytes that do not form a valid CBOR item
|
||||
- **THEN** the call throws `StorageCacheError` with `code === "CACHE_STORAGE.CODEC_ERROR"` AND `error.cause` is the underlying `CborError`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"decode throws CODEC_ERROR with cbor cause on malformed bytes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: StructuredCloneCodec interoperates with StorageCache
|
||||
|
||||
`StructuredCloneCodec` SHALL be assignable to the `codec` constructor option of `StorageCache<T>` without any cast. A `StorageCache<T>` constructed with `codec: new StructuredCloneCodec<T>()` SHALL preserve every round-trip case from the first requirement when values are written via `cache.set` and read back via `cache.get`.
|
||||
|
||||
#### Scenario: StorageCache + StructuredCloneCodec round-trips a Map value
|
||||
|
||||
- **WHEN** a caller constructs `new StorageCache({ storage: new MemoryStorage(), codec: new StructuredCloneCodec() })`, calls `cache.set("k", new Map([["a", 1]]))`, AND calls `cache.get("k")`
|
||||
- **THEN** the result is `instanceof Map` with `size === 1` AND `.get("a") === 1`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"StorageCache with StructuredCloneCodec round-trips a Map value"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Codec conformance harness exposed to package tests
|
||||
|
||||
The package SHALL export a `runCodecConformance(codec, options)` harness from `./codec/conformance` accepting `{ name: string; supportsStructuredClone: boolean }`. The harness SHALL execute round-trip assertions for every type listed in the first requirement; when `supportsStructuredClone` is `false`, the harness SHALL restrict assertions to types the JSON profile preserves (primitives, plain objects, arrays). The harness SHALL fail loudly when given a deliberately broken codec (proof of teeth — no tautological green).
|
||||
|
||||
#### Scenario: harness reports failure on a broken codec
|
||||
|
||||
- **WHEN** the test suite passes a codec whose `decode` returns a fixed sentinel `null` instead of the round-tripped value
|
||||
- **THEN** at least one harness assertion fails
|
||||
|
||||
#### Scenario: harness passes for the existing JsonCodec at the JSON profile
|
||||
|
||||
- **WHEN** the test suite calls `runCodecConformance(new JsonCodec<unknown>(), { name: "JsonCodec", supportsStructuredClone: false })`
|
||||
- **THEN** every assertion passes
|
||||
|
||||
**Verified by:** `src/codec/conformance.test.ts::"harness fails on a deliberately broken codec"` AND `src/codec/json.test.ts::"JsonCodec satisfies the JSON-profile conformance harness"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CACHE_STORAGE registry extended with two codes
|
||||
|
||||
`CACHE_STORAGE_CODES` SHALL be extended via the same `defineCodes("CACHE_STORAGE", [...])` call to include `CODEC_UNSUPPORTED_TYPE` and `CODEC_UNKNOWN_TAG`, bringing the registry to 7 codes total. Both new codes SHALL be reachable from at least one test in this package. The existing 5 codes (`ENTRY_CORRUPT`, `CODEC_ERROR`, `BACKEND_ERROR`, `KEY_TOO_LONG`, `VALUE_TOO_LARGE`) SHALL remain unchanged.
|
||||
|
||||
#### Scenario: every CACHE_STORAGE code (including the two new ones) is reachable
|
||||
|
||||
- **WHEN** the test suite runs
|
||||
- **THEN** for every entry in `CACHE_STORAGE_CODES`, at least one assertion exercises a real call site that produces that code
|
||||
|
||||
**Verified by:** `src/errors/errors.test.ts::"every CACHE_STORAGE code is reachable from a real call site"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Default codec for StorageCache remains JsonCodec
|
||||
|
||||
The constructor default for `StorageCache.codec` SHALL remain `new JsonCodec<T>()`. Existing callers that do not pass `codec` SHALL observe byte-identical behaviour to the codec they observed before this change. The new `StructuredCloneCodec` SHALL be opt-in by explicit construction only.
|
||||
|
||||
#### Scenario: StorageCache without explicit codec uses JsonCodec
|
||||
|
||||
- **WHEN** a caller constructs `new StorageCache({ storage: new MemoryStorage() })`, sets a string value, and inspects the bytes written by the underlying storage past `HEADER_BYTES`
|
||||
- **THEN** the bytes equal `new TextEncoder().encode(JSON.stringify(value))`
|
||||
|
||||
**Verified by:** `src/storage-cache/storage-cache.test.ts::"default codec is JsonCodec — bytes past the header are UTF-8 JSON"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath export for the new codec
|
||||
|
||||
`package.json` SHALL expose `./codec/structured-clone` as a subpath export mirroring the existing `./codec/json` and `./codec/identity` entries, so tree-shaking consumers can import only the new codec without dragging in the rest of the codec barrel.
|
||||
|
||||
#### Scenario: subpath import resolves the StructuredCloneCodec class
|
||||
|
||||
- **WHEN** a consumer imports `import { StructuredCloneCodec } from "@suaveplan/cache-storage/codec/structured-clone"`
|
||||
- **THEN** the import resolves to the same class as `import { StructuredCloneCodec } from "@suaveplan/cache-storage/codec"`
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"./codec/structured-clone subpath resolves to the same class as ./codec"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Tier purity — universal preserved
|
||||
|
||||
Adding `StructuredCloneCodec` SHALL NOT introduce any `node:*`, `bun:*`, or DOM-only import into the package. The package SHALL continue to run unmodified under Node 18+, Bun, Deno, modern browsers, and Cloudflare Workers.
|
||||
|
||||
#### Scenario: tier-purity grep finds no forbidden imports after the change
|
||||
|
||||
- **WHEN** `grep -rnE "from ['\"](node:|bun:)" packages/flow/cache-storage/src/` is run
|
||||
- **THEN** the output is empty
|
||||
|
||||
**Verified by:** `src/codec/structured-clone.test.ts::"structured-clone codec contains no node: or bun: imports"`
|
||||
@@ -0,0 +1,60 @@
|
||||
# Tasks — add-cache-storage-cbor-codec
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Additive only — no behaviour change to existing codecs.
|
||||
|
||||
## 1. Pre-flight
|
||||
|
||||
- [ ] 1.1 Re-read `openspec/AGENTS.md` §0c (delta format) and §13 (gate order)
|
||||
- [ ] 1.2 Confirm `@suaveplan/cbor` ships on genesis with `encode`, `decode`, `Tagged`, `type TagHandler` in the root barrel
|
||||
- [ ] 1.3 Confirm `packages/flow/cache-storage/src/codec/codec.types.ts` exposes `Codec<T>` and that `storage-cache.ts` defaults to `JsonCodec` (no behaviour change needed)
|
||||
- [ ] 1.4 Numbered checklist of every `### Requirement` and `#### Scenario` in `specs/cache-storage/spec.md` exists in working notes (AGENTS §0c step 6)
|
||||
|
||||
## 2. Package wiring
|
||||
|
||||
- [ ] 2.1 Add `@suaveplan/cbor` to `packages/flow/cache-storage/package.json` `dependencies` at `workspace:^`
|
||||
- [ ] 2.2 Add subpath export `./codec/structured-clone` mirroring the shape of `./codec/json` and `./codec/identity`
|
||||
- [ ] 2.3 Confirm `sideEffects` array remains correct (no new side-effect file added)
|
||||
|
||||
## 3. Error registry extension
|
||||
|
||||
- [ ] 3.1 Extend `CACHE_STORAGE_CODES` in `src/errors/errors.ts` with `CODEC_UNSUPPORTED_TYPE` and `CODEC_UNKNOWN_TAG`
|
||||
- [ ] 3.2 Update `src/errors/errors.md` Features + API Reference to list the two new codes (doc-drift gate §4a)
|
||||
|
||||
## 4. Implementation — StructuredCloneCodec
|
||||
|
||||
- [ ] 4.1 `src/codec/structured-clone.ts` — `StructuredCloneCodec<T> implements Codec<T>`; lift table per `design.md`; encode/decode against `@suaveplan/cbor`
|
||||
- [ ] 4.2 `src/codec/structured-clone.md` — sibling docs (≥200 words, basic + advanced examples, API reference, implementation notes)
|
||||
- [ ] 4.3 `src/codec/index.ts` — explicit named re-export of `StructuredCloneCodec`
|
||||
- [ ] 4.4 Confirm file size ≤300 lines; split if needed (e.g. lift table to `src/codec/structured-clone.tags.ts`)
|
||||
|
||||
## 5. Conformance harness
|
||||
|
||||
- [ ] 5.1 `src/codec/conformance.ts` — `runCodecConformance(codec, { name, supportsStructuredClone })` covering primitives, nested objects, arrays, `Date`, `Map`, `Set`, `Uint8Array`, `bigint` (positive + negative), `RegExp`, `URL`, deep nesting, empty collections
|
||||
- [ ] 5.2 `src/codec/conformance.md` — sibling docs
|
||||
- [ ] 5.3 `src/codec/conformance.test.ts` — harness self-test: a deliberately broken `Codec<unknown>` must cause the harness to fail (no tautological green)
|
||||
|
||||
## 6. Tests
|
||||
|
||||
- [ ] 6.1 `src/codec/structured-clone.test.ts` — invoke `runCodecConformance(new StructuredCloneCodec(), { name: "StructuredCloneCodec", supportsStructuredClone: true })`; plus direct tests for `CODEC_UNSUPPORTED_TYPE` (encode `() => {}`, `Symbol("x")`, `new WeakMap()`) and `CODEC_UNKNOWN_TAG` (decode a CBOR-encoded `Tagged(9999, "x")`)
|
||||
- [ ] 6.2 Extend `src/codec/json.test.ts` with `runCodecConformance(new JsonCodec<unknown>(), { name: "JsonCodec", supportsStructuredClone: false })`
|
||||
- [ ] 6.3 Extend `src/codec/identity.test.ts` with `runCodecConformance(new IdentityCodec(), { name: "IdentityCodec", supportsStructuredClone: false })` — using `Uint8Array` payloads only
|
||||
- [ ] 6.4 Extend `src/errors/errors.test.ts` to assert the two new codes are reachable
|
||||
- [ ] 6.5 Every `it(...)` carries `{ timeout: 5000 }`
|
||||
- [ ] 6.6 All test imports via `@suaveplan/testing/runner` (§7)
|
||||
|
||||
## 7. Gates (§13 order)
|
||||
|
||||
- [ ] 7.1 Spec reconciliation — walk every `### Requirement` / `#### Scenario` against impl
|
||||
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/cache-storage`
|
||||
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/cache-storage` — 100/100/100/100
|
||||
- [ ] 7.4 `bunx turbo build --filter=@suaveplan/cache-storage`
|
||||
- [ ] 7.5 Banned-pattern, barrel-shape, runner-import, missing-`.md`, bare-version scans (zero output)
|
||||
- [ ] 7.6 `bunx turbo test --filter=...@suaveplan/cache-storage` — dependent sweep (`@suaveplan/cache-redis` must remain green)
|
||||
|
||||
## 8. Manifest & archival
|
||||
|
||||
- [ ] 8.1 `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 8.2 Update `packages/flow/cache-storage/README.md` Modules section with the new codec
|
||||
- [ ] 8.3 `bunx markdownlint-cli2 --fix`
|
||||
- [ ] 8.4 Fold `specs/cache-storage/spec.md` into `openspec/specs/cache-storage/spec.md` (merge `## ADDED` block into existing spec)
|
||||
- [ ] 8.5 Move change folder → `openspec/archive/<yyyy-mm-dd>-add-cache-storage-cbor-codec/`
|
||||
@@ -0,0 +1,164 @@
|
||||
# Design — add-drive-range-reads
|
||||
|
||||
> Non-normative. Normative requirements live in `./specs/drive/spec.md` and `./specs/drive-uri/spec.md`.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Extend `Drive.createReadStream` with `{ offset, length }` without breaking the bare-`AbortSignal` signature used at every existing call site.
|
||||
2. Push the range hint through the plugin middleware so `drive-plugin-chunks` only fetches/decrypts/emits chunks intersecting the range.
|
||||
3. Over `drive+mesh://`, request only the in-range chunkIds from peers — `ChunkRequest.specific` for small windows, `ChunkRequest.bitset` for large contiguous spans.
|
||||
4. Surface range failures via two new `DRIVE_*` codes; reuse existing wire format and pass-through semantics elsewhere.
|
||||
|
||||
## API surface evolution
|
||||
|
||||
### `Drive.createReadStream` — overloaded
|
||||
|
||||
```ts
|
||||
interface RangeReadOptions {
|
||||
offset?: number; // default 0; integer ≥ 0
|
||||
length?: number; // default → EOF; integer ≥ 0
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
class Drive {
|
||||
// Existing (preserved)
|
||||
createReadStream(path: string, signal?: AbortSignal): ReadableStream<Uint8Array>;
|
||||
// Added
|
||||
createReadStream(path: string, options: RangeReadOptions): ReadableStream<Uint8Array>;
|
||||
// Implementation
|
||||
createReadStream(
|
||||
path: string,
|
||||
signalOrOptions?: AbortSignal | RangeReadOptions,
|
||||
): ReadableStream<Uint8Array> { … }
|
||||
}
|
||||
```
|
||||
|
||||
Detection: `signalOrOptions instanceof AbortSignal ? { signal: signalOrOptions } : (signalOrOptions ?? {})`. The runtime `instanceof` check is reliable in every supported runtime (Bun, Node 22+, Chromium 102+, Safari 16+) and avoids a brittle duck-type sniff.
|
||||
|
||||
Validation: synchronous — when `offset` or `length` is present, it MUST be a finite non-negative integer (`Number.isInteger(x) && x >= 0`). On failure, throw `DriveError(DRIVE.RANGE_INVALID, "offset must be a non-negative integer")` BEFORE constructing the stream.
|
||||
|
||||
### `StorageUriResolver` contract
|
||||
|
||||
`@suaveplan/storage-uri`'s `UriPlugin.adapter.createReadStream(key, options?)` already accepts an options bag; we extend the contract to include `{ offset?, length? }` per the `StorageLike` evolution path in `add-signal-to-storage-like` (archived). The resolver passes the bag through unchanged. `drive-uri`'s three plugins each set `adapter.capabilities.nativeRange = true` so the resolver issues one range-aware `createReadStream` call instead of buffering and slicing.
|
||||
|
||||
## Range → chunk math
|
||||
|
||||
Given:
|
||||
|
||||
- `chunkSize` (uint32, from `manifest.chunkSize` for chunks-plugin; from `storage.chunkSize` otherwise; conventionally 65536)
|
||||
- `fileSize` (bigint, from `manifest.fileSize`)
|
||||
- requested `offset` (default 0) and `length` (default `fileSize - offset`)
|
||||
|
||||
Compute:
|
||||
|
||||
- `end = min(offset + length, fileSize)` // exclusive
|
||||
- `firstChunk = floor(offset / chunkSize)`
|
||||
- `lastChunk = floor((end - 1) / chunkSize)` // inclusive; only when `end > offset`
|
||||
- `firstChunkSkip = offset - firstChunk * chunkSize` // bytes to drop from the front of `firstChunk`
|
||||
- `lastChunkKeep = end - lastChunk * chunkSize` // bytes to retain from the front of `lastChunk`
|
||||
|
||||
Emit:
|
||||
|
||||
- For `firstChunk`: decrypt, slice `[firstChunkSkip, min(chunkSize, lastChunkKeep))` if `firstChunk === lastChunk`, else `[firstChunkSkip, payloadLen)`
|
||||
- For interior chunks `(firstChunk, lastChunk)`: emit whole `chunk.slice(0, payloadLen)`
|
||||
- For `lastChunk` (when distinct from `firstChunk`): decrypt, slice `[0, lastChunkKeep)`
|
||||
|
||||
Zero-length: when `length === 0`, do not enter the loop — emit an empty `Uint8Array` and close.
|
||||
|
||||
### Worked examples (chunkSize = 65 536; last chunk holds `payloadLen` bytes only)
|
||||
|
||||
| Scenario | offset | length | end | firstChunk | lastChunk | firstSkip | lastKeep | Chunks pulled |
|
||||
|--------------------------------|---------|---------|-----------|------------|-----------|-----------|----------|---------------|
|
||||
| Range fully inside one chunk | 1 024 | 2 048 | 3 072 | 0 | 0 | 1 024 | 3 072 | {0} (1) |
|
||||
| Spans one boundary | 60 000 | 10 000 | 70 000 | 0 | 1 | 60 000 | 4 464 | {0, 1} (2) |
|
||||
| Aligned multi-chunk | 65 536 | 196 608 | 262 144 | 1 | 3 | 0 | 65 536 | {1, 2, 3} (3) |
|
||||
| Tail to EOF (fileSize=200 000) | 150 000 | — | 200 000 | 2 | 3 | 18 928 | 3 392 | {2, 3} (2) |
|
||||
| Zero-length | 12 345 | 0 | 12 345 | n/a | n/a | n/a | n/a | ∅ (0) |
|
||||
| Whole file | 0 | — | fileSize | 0 | last | 0 | tailLen | {0..last} (N) |
|
||||
|
||||
## Mesh-request optimisation
|
||||
|
||||
The mesh pull path today issues `ChunkRequest.whole`. With a range, build:
|
||||
|
||||
```ts
|
||||
const span = lastChunk - firstChunk + 1;
|
||||
const req: ChunkRequest =
|
||||
span <= 64
|
||||
? { kind: "specific", chunkIds: rangeArray(firstChunk, lastChunk) }
|
||||
: { kind: "bitset", bytes: bitsetForRange(firstChunk, lastChunk, totalChunks) };
|
||||
```
|
||||
|
||||
Pivot rationale:
|
||||
|
||||
- `MAX_CHUNKS_DEFAULT = 64` (`drive-mesh-protocol/src/schemas/envelope.schema.ts`). JSON `chunkIds` for 64 numbers ~ 200 B; a bitset for the same window is ~ 8 B header + ⌈64/8⌉ = 16 B = under 30 B framed. At 64+ chunks, the bitset is uniformly cheaper.
|
||||
- The bitset is constructed via `encodeBitset` from `drive-mesh-protocol`; the receiver decode path is unchanged.
|
||||
- `ParallelPuller`'s window applies to whichever chunkIds the request enumerated, so range simply narrows the parallelism domain.
|
||||
|
||||
## Abort handling
|
||||
|
||||
The `AbortSignal` (from `RangeReadOptions.signal` OR the bare-signal form) is threaded into:
|
||||
|
||||
1. `storage.createReadStream(path, { signal })` when the storage has `nativeStreams`.
|
||||
2. The chunks-plugin `#reassembleAsStream` loop — each `layout.getChunk(fileId, i)` call is wrapped by `throwIfAborted(signal)` between iterations.
|
||||
3. The mesh pull — `drive-mesh.ParallelPuller` already accepts a signal; the range path passes it through unchanged.
|
||||
|
||||
On abort: cancel in-flight chunk fetches, close the stream with `controller.error(new DOMException("aborted", "AbortError"))`, and (over mesh) emit a single `cd:error` envelope with `code = "ABORTED"` so peers can free reservation slots.
|
||||
|
||||
## Error codes table
|
||||
|
||||
| Code | Origin | When raised |
|
||||
|-------------------------------|---------------------------------|--------------------------------------------------------------------------------------------------------------|
|
||||
| `DRIVE.RANGE_INVALID` | `Drive.createReadStream` | `offset`/`length` is non-integer / negative / NaN / not finite. Thrown synchronously before stream creation. |
|
||||
| `DRIVE.RANGE_OUT_OF_BOUNDS` | `Drive.createReadStream` OR stream `error` event | Synchronously: `offset > size` or `offset + length > size` when `size` is known at start. Asynchronously: short-fall on stream close when size was unknown at start. |
|
||||
| `CHUNKS.RANGE_OUT_OF_BOUNDS` | `chunksPlugin` middleware | Manifest `fileSize` is loaded and the range exceeds it. Surfaced on stream `error`; the drive layer re-wraps as `DRIVE.RANGE_OUT_OF_BOUNDS` with `cause`. |
|
||||
|
||||
No new codes in `DRIVE_URI`, `DMP`, or `DTP`. The resolver pass-through rule already forwards `DRIVE.*` codes to the caller intact.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/drive/drive/src/
|
||||
file-operations/
|
||||
streams.ts # MODIFIED — overload signatures, range slicing transform
|
||||
streams.test.ts # MODIFIED — new range cases
|
||||
streams.range.test.ts # ADDED — boundary cases (table above)
|
||||
streams.range.md # ADDED — docs for the new path
|
||||
types/
|
||||
drive.types.ts # MODIFIED — RangeReadOptions, DriveStorage.capabilities.nativeRange
|
||||
error-codes.ts # MODIFIED — RANGE_INVALID, RANGE_OUT_OF_BOUNDS
|
||||
|
||||
packages/drive/drive-plugin-chunks/src/
|
||||
pipeline/
|
||||
range.ts # ADDED — pure range→chunk math (table above)
|
||||
range.test.ts # ADDED — property tests over (offset, length, fileSize, chunkSize)
|
||||
range.md # ADDED
|
||||
plugin.ts # MODIFIED — #reassemble takes range?, #reassembleAsStream
|
||||
error-codes.ts # MODIFIED — RANGE_OUT_OF_BOUNDS
|
||||
|
||||
packages/drive/drive-mesh/src/
|
||||
pull/
|
||||
range-request.ts # ADDED — specific-vs-bitset pivot
|
||||
range-request.test.ts # ADDED
|
||||
range-request.md # ADDED
|
||||
parallel-puller.ts # MODIFIED — accept range, build request via range-request
|
||||
|
||||
packages/drive/drive-uri/src/
|
||||
adapter/
|
||||
drive-storage-like.ts # MODIFIED — forward { offset, length } from createReadStream options
|
||||
drive-storage-like.test.ts # MODIFIED — range forwarding
|
||||
plugins/
|
||||
drive/drive.streams.ts # MODIFIED — InstrumentedDriveAdapter.createReadStream forwards range
|
||||
drive/drive.streams.test.ts# MODIFIED — range + AbortSignal
|
||||
otel/
|
||||
otel.ts # MODIFIED — drive.uri.range attribute, range_chunks_fetched histogram
|
||||
otel.test.ts # MODIFIED
|
||||
```
|
||||
|
||||
Every new `.ts` carries a sibling `.md` per CLAUDE.md §4; every new test uses `@suaveplan/testing/runner` per rule 11.
|
||||
|
||||
## Trade-offs considered
|
||||
|
||||
- **`{ offset, length }` vs. `{ start, end }`.** Picked `offset`/`length` to match the HTTP `Range: bytes=offset-(offset+length-1)` mental model and avoid the inclusive-vs-exclusive `end` ambiguity that bites every range API at least once.
|
||||
- **Always-bitset vs. always-specific request.** Always-bitset wastes ~ 20 bytes on a 3-chunk range; always-specific blows the `MAX_CHUNKS_DEFAULT` cap on multi-MiB ranges. Pivoting at the cap is honest about the wire's own constraint.
|
||||
- **`Drive.read({ offset, length })` companion.** Tempting but redundant: `await new Response(stream).arrayBuffer()` reduces to the same thing. Defer to a hoist when a concrete consumer asks.
|
||||
- **Throw vs. clamp on overflow.** Clamping (HTTP `Range: bytes=900-2000` on a 1 000-byte file silently returning bytes 900–999) hides upstream bugs. Throwing `RANGE_OUT_OF_BOUNDS` is louder and matches the rest of `DRIVE_*`'s explicit failure style.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Add byte-range reads to Drive.createReadStream and `drive*://` URI plugins
|
||||
|
||||
**Packages**: `@suaveplan/drive`, `@suaveplan/drive-uri`
|
||||
**Status**: PROPOSED
|
||||
**Type**: capability-extension
|
||||
**Defers from**: `openspec/archive/2026-05-16-add-drive-uri/proposal.md` § "Out of scope → Range reads on `drive+mesh://`"
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec deltas**:
|
||||
- ./specs/drive/spec.md
|
||||
- ./specs/drive-uri/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
`add-drive-uri` shipped `resolver.read("drive+mesh://swarm-x/clip.mov")` as a whole-file fetch only, explicitly carving out byte-range reads:
|
||||
|
||||
> **Range reads on `drive+mesh://`.** `drive.createReadStream` already streams chunk-by-chunk via the chunks plugin; byte-range slicing on top requires a future `Drive.createReadStream({ offset, length })` extension — tracked separately.
|
||||
|
||||
Range reads are the single most-asked-for missing capability for the drive surface — every consumer that wants HTTP `Range:` header support (audio/video seek, virtual scrolling over chunked logs, partial-image decode, resume-on-reconnect, S3-style multipart staging) is currently forced to fetch the full file and slice client-side. Over `drive+mesh://` this is catastrophic: a 4 GiB media file pulled to seek the last 1 MiB downloads the entire swarm payload. The chunked layout in `drive-plugin-chunks` already addresses every chunk by `(fileId, chunkId)` with a known fixed `chunkSize` per manifest, so range → chunk math is deterministic; the missing piece is plumbing the offset/length pair from `Drive.createReadStream` through the chunks-plugin middleware so only the chunks intersecting the range are fetched, decrypted, and emitted.
|
||||
|
||||
This change adds the `{ offset, length }` options to `Drive.createReadStream`, extends the chunks plugin's `#reassemble` to honour them (computing first/last chunk indices, slicing the boundary chunks), threads range hints through the `drive-uri` adapters, and — over `drive+mesh://` — emits a `ChunkRequest.specific` (or `bitset` for large contiguous ranges) listing only the in-range chunkIds rather than `ChunkRequest.whole`. The local-storage and template paths get the same offset/length math for free because they go through the same chunks-plugin middleware.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/drive`
|
||||
|
||||
- MODIFIED: `Drive.createReadStream(path, signal?)` → `Drive.createReadStream(path, options?: RangeReadOptions | AbortSignal)`. The new positional-options overload accepts `{ offset?: number; length?: number; signal?: AbortSignal }`. The bare-`AbortSignal` form is preserved via overload so all existing call sites compile unchanged.
|
||||
- ADDED: `RangeReadOptions` interface in `src/types/drive.types.ts` — `offset` defaults to `0`, `length` defaults to "to EOF". Both MUST be non-negative integers when present; violations raise `DRIVE.RANGE_INVALID` synchronously (before stream construction).
|
||||
- ADDED: `DRIVE.RANGE_INVALID` and `DRIVE.RANGE_OUT_OF_BOUNDS` codes in `DRIVE_CODES`. `RANGE_INVALID` covers negative / non-integer / NaN inputs. `RANGE_OUT_OF_BOUNDS` covers `offset > size` and `offset + length > size` where `size` is known at stream-start (e.g. via the chunks-plugin manifest); when size is unknown at start-time, the stream end-event MUST surface `RANGE_OUT_OF_BOUNDS` if the underlying source closed before `length` bytes were emitted.
|
||||
- MODIFIED: `defaultCreateReadStream` and the storage-capability passthrough in `src/file-operations/streams.ts` honour `RangeReadOptions`. The default buffered path slices the loaded `Uint8Array`; the native-streams path forwards `{ offset, length }` to `storage.createReadStream` when `storage.capabilities?.nativeRange === true`, otherwise wraps the native stream in a slicing transform that drops bytes before `offset` and stops after `length`.
|
||||
- ADDED: `DriveStorage` capability flag `capabilities.nativeRange?: boolean` and an extended `createReadStream(path, options?: { offset?: number; length?: number; signal?: AbortSignal })` signature on the storage layer. Storages that already accept a range (e.g. file with `fs.createReadStream({ start, end })`) opt in by setting the flag; existing storages are unaffected.
|
||||
- ADDED: Middleware context carries `range?: { offset: number; length?: number }`. The chunks-plugin middleware reads it and short-circuits its `#reassemble` to the intersecting chunk window.
|
||||
|
||||
### `@suaveplan/drive-plugin-chunks`
|
||||
|
||||
- MODIFIED: `ChunksPluginHandle.#reassemble(fileId, range?)` accepts an optional range and emits only the chunks intersecting `[offset, offset + length)`. First and last chunks are sliced; interior chunks are emitted whole. Algorithm in `design.md` § "Range → chunk math".
|
||||
- MODIFIED: The `read` middleware forwards `ctx.range` (when present) to `#reassemble` and to a sibling `#reassembleAsStream` used by `createReadStream`. The middleware's `read(path)` buffered call buffers only the in-range bytes.
|
||||
- ADDED: `CHUNKS.RANGE_OUT_OF_BOUNDS` code — raised when the requested range exceeds `manifest.fileSize`. Surfaced on stream `error`.
|
||||
|
||||
### `@suaveplan/drive-mesh-protocol`
|
||||
|
||||
- MODIFIED: `drive-mesh`'s pull path (consumer of this package; no protocol wire change) constructs `ChunkRequest.specific { chunkIds }` for ranges spanning ≤ 64 chunks and `ChunkRequest.bitset { bytes }` for larger contiguous spans. No new wire fields — both forms already exist in `ChunkRequestPayload`.
|
||||
- No `DMP_*` registry changes.
|
||||
|
||||
### `@suaveplan/drive-uri`
|
||||
|
||||
- MODIFIED: `DriveStorageLikeAdapter.createReadStream(key, options?)` accepts and forwards `{ offset?, length?, signal? }`. The three plugins (`driveUriPlugin`, `driveMeshUriPlugin`, `driveTemplateUriPlugin`) advertise `capabilities.nativeRange = true` on their adapter so `StorageUriResolver` issues a single range-aware `createReadStream` call instead of a whole-file `get` + client-side slice.
|
||||
- ADDED: `DRIVE_URI.RANGE_UNSUPPORTED` is **not** introduced — every `drive*://` URI now supports ranges because the underlying chunks plugin does. The resolver surfaces upstream `DRIVE.RANGE_INVALID` / `DRIVE.RANGE_OUT_OF_BOUNDS` directly under the existing pass-through rule.
|
||||
- ADDED: OTel attribute `drive.uri.range = "{offset}-{end}"` on `createReadStream` spans (omitted when no range supplied). Histogram `drive_uri.range_chunks_fetched` tags `{scheme}` and records the count of chunks actually pulled — measurable savings vs. whole-file fetch.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Multi-range reads** (RFC 7233 `multipart/byteranges`). One range per call. A caller wanting N disjoint ranges issues N `createReadStream` calls.
|
||||
- **`Drive.read(path, { offset, length })`** buffered range read. Achievable today via `createReadStream({ offset, length })` + buffer-collect; adding it to the buffered path would duplicate the chunk-math without changing observable behaviour. Hoist if a real consumer needs it.
|
||||
- **Range writes / partial uploads.** Writes remain whole-file. Chunk-level patch semantics belong in a future `add-drive-patch` change.
|
||||
- **Range hints on `drive`-tier `read()`** for non-chunked storages without `nativeRange`. Falls back to the slicing transform; performance characteristics are documented but no new code path is added.
|
||||
- **Backpressure-aware peer scheduling** for large bitset requests. The mesh plugin's existing `ParallelPuller` window applies unchanged; range simply narrows the chunk set it pulls from.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This is a **genesis-original** extension; deferral source is `openspec/archive/2026-05-16-add-drive-uri/proposal.md`. Surface anchors:
|
||||
|
||||
- `Drive.createReadStream` — `packages/drive/drive/src/drive/drive.ts:222-230`, `packages/drive/drive/src/file-operations/streams.ts`.
|
||||
- Chunks plugin `#reassemble` — `packages/drive/drive-plugin-chunks/src/plugin.ts:218-251`; chunkSize lives on the manifest (`packages/drive/drive-plugin-chunks/src/manifest/build.ts:11,30,74`), default 65536 (`packages/drive/drive-plugin-chunks/src/plugin.ts:78`).
|
||||
- `ChunkRequest` union — `packages/drive/drive-mesh-protocol/src/types/protocol.types.ts:38-42`; both `specific` and `bitset` already encodable.
|
||||
- `drive-uri` adapter — `packages/drive/drive-uri/src/plugins/drive-mesh/drive-mesh.ts` extends `InstrumentedDriveAdapter`.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Overload `createReadStream(path, signal)` AND `createReadStream(path, options)` rather than breaking the single-signal form | Every existing consumer in the monorepo (chunks plugin, drive-uri adapters, drive integration tests) passes the bare `AbortSignal`. A breaking signature change for an additive feature is unjustified when overloads cost nothing. |
|
||||
| Switch `ChunkRequest.specific` ↔ `ChunkRequest.bitset` at 64 chunks | `MAX_CHUNKS_DEFAULT = 64` in `drive-mesh-protocol`; beyond that, the JSON `chunkIds` array out-weighs a bitset that addresses every chunk in two bytes per 16. The 64-chunk pivot matches the wire-cap and minimises bytes-on-wire for the common case (4 MiB range over 64 KiB chunks). |
|
||||
| `RANGE_OUT_OF_BOUNDS` may fire at start OR at stream-end | When `size` is known (chunks-plugin manifest loaded, storage `stat` populated), reject synchronously. When unknown (raw `defaultCreateReadStream` over a non-stat-able storage), accept the range, emit what's available, then surface the code on `close` if shortfall is detected. Two paths because the size-known/size-unknown split is a real storage capability variance, not a defect. |
|
||||
| Zero-length range is a valid request | `length: 0` returns an immediately-closed empty stream. Useful as a probe-for-existence pattern; rejecting it would surprise consumers who compute ranges from upstream API responses. |
|
||||
| `nativeRange` storage capability flag rather than always-slicing in drive | `bun:sqlite`-backed and `fs`-backed storages can deliver true range reads with no read-amplification. Always wrapping in a slicing transform would discard that win for the storages that have it. The flag is opt-in so adding it is purely additive on every existing storage. |
|
||||
|
||||
## Migration
|
||||
|
||||
This is a purely additive change for every existing call site:
|
||||
|
||||
- **Old**: `drive.createReadStream("file.bin", abortSignal)` → still compiles, still returns whole-file stream.
|
||||
- **New**: `drive.createReadStream("file.bin", { offset: 1024, length: 4096, signal: abortSignal })` → emits exactly bytes [1024, 5120). Over `drive+mesh://`, requests at most ⌈(4096 + 1024 % chunkSize) / chunkSize⌉ chunks from peers.
|
||||
|
||||
No consumer must change. Consumers that want range behaviour migrate at their leisure. Search command for opt-in adoption:
|
||||
|
||||
```bash
|
||||
grep -rn "createReadStream" packages/ | grep -v "/dist/\|/coverage/"
|
||||
```
|
||||
|
||||
Recommended replacement when range-aware: pass `{ offset, length, signal }` instead of the bare `signal`.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/drive` to enumerate. The blast radius is "every package that calls `Drive.createReadStream`," which today is:
|
||||
|
||||
- `@suaveplan/drive-uri` — [x] tests run / [x] docs reviewed / [ ] no action needed (adapter `createReadStream` extended to forward range; new requirement added to spec delta)
|
||||
- `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed / [ ] no action needed (middleware honours range; `#reassemble` extended)
|
||||
- `@suaveplan/drive-mesh` — [x] tests run / [x] docs reviewed / [ ] no action needed (pull path constructs `ChunkRequest.specific` / `bitset` from range)
|
||||
- `@suaveplan/drive-transport-https` — [ ] tests run / [x] docs reviewed / [x] no action needed (transport carries chunk envelopes opaquely; no surface change)
|
||||
- `@suaveplan/drive-transport-webrtc` — [ ] tests run / [x] docs reviewed / [x] no action needed (same as above)
|
||||
- `@suaveplan/drive-template` — [ ] tests run / [x] docs reviewed / [x] no action needed (hydration unchanged; range applies post-hydration)
|
||||
- `@suaveplan/drive-sharing` — [ ] tests run / [x] docs reviewed / [x] no action needed (calls `Drive.read`, not `createReadStream`)
|
||||
- `@suaveplan/drive-webrtc` — [ ] tests run / [x] docs reviewed / [x] no action needed (browser-tier transport, unaffected)
|
||||
|
||||
### Affected upstreams (none modified)
|
||||
|
||||
| Package | Action | Breaking? |
|
||||
|---|---|---|
|
||||
| `@suaveplan/storage` | Optional: extend `AbstractStorage.createReadStream` with `{ offset, length }` options. Existing storages opt in via `capabilities.nativeRange`. | no |
|
||||
| `@suaveplan/storage-uri` | None — already routes `createReadStream` through `UriPlugin.adapter`. | no |
|
||||
| `@suaveplan/types` | None — `RangeReadOptions` lives in `@suaveplan/drive` until a second consumer emerges. | no |
|
||||
|
||||
### Ordering
|
||||
|
||||
Unblocked. All packages exist on genesis today; the change is additive.
|
||||
@@ -0,0 +1,121 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: drive (drive)
|
||||
|
||||
**Package**: `@suaveplan/drive`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. The named tests are planned; they MUST exist and pass verbatim before this delta is folded into `openspec/specs/drive/spec.md`.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `Drive.createReadStream` accepts a range-options bag
|
||||
|
||||
`Drive.createReadStream` SHALL be overloaded to accept either a bare `AbortSignal` (existing form) or a `RangeReadOptions` bag with optional `offset`, `length`, and `signal`. The bare-signal form MUST continue to return a whole-file stream. When the options form is passed, the drive MUST forward `{ offset, length, signal }` through the plugin middleware chain via the middleware context's `range` field. The implementation MUST runtime-discriminate via `instanceof AbortSignal`.
|
||||
|
||||
#### Scenario: bare signal still returns whole-file stream
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", abortSignal)` is called on a 200 KiB file
|
||||
- **THEN** the returned stream emits the entire 200 KiB payload and respects `abortSignal` for cancellation
|
||||
|
||||
#### Scenario: options form forwards offset and length
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { offset: 1024, length: 4096 })` is called
|
||||
- **THEN** the middleware context carries `range = { offset: 1024, length: 4096 }` and the returned stream emits exactly bytes `[1024, 5120)` of the source
|
||||
|
||||
#### Scenario: zero-length range yields an empty stream
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { offset: 12345, length: 0 })` is called
|
||||
- **THEN** the returned stream closes immediately with zero enqueued bytes
|
||||
|
||||
**Verified by:** `src/drive/drive.test.ts::"createReadStream(path, signal) preserves whole-file behaviour"` AND `src/drive/drive.test.ts::"createReadStream(path, { offset, length }) emits exactly the requested slice"` AND `src/drive/drive.test.ts::"createReadStream(path, { length: 0 }) returns an immediately closed stream"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Invalid range options throw `DRIVE.RANGE_INVALID` synchronously
|
||||
|
||||
`Drive.createReadStream(path, options)` SHALL validate `options.offset` and `options.length` synchronously before constructing the `ReadableStream`. Either field, when present, MUST be a finite non-negative integer (`Number.isInteger(x) && x >= 0 && Number.isFinite(x)`). On violation, the method MUST throw `DriveError(DRIVE.RANGE_INVALID, ...)` BEFORE returning. The error's `context` SHALL include the offending field name and value.
|
||||
|
||||
#### Scenario: negative offset throws synchronously
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { offset: -1 })` is called
|
||||
- **THEN** the call throws `DriveError` with `code === "DRIVE.RANGE_INVALID"` and `context.field === "offset"`, and no `ReadableStream` is created
|
||||
|
||||
#### Scenario: non-integer length throws synchronously
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { length: 1.5 })` is called
|
||||
- **THEN** the call throws `DriveError` with `code === "DRIVE.RANGE_INVALID"` and `context.field === "length"`
|
||||
|
||||
#### Scenario: NaN / Infinity rejected
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { offset: Number.NaN })` is called OR `drive.createReadStream("file.bin", { length: Number.POSITIVE_INFINITY })` is called
|
||||
- **THEN** each call throws `DriveError(DRIVE.RANGE_INVALID, ...)` synchronously
|
||||
|
||||
**Verified by:** `src/drive/drive.test.ts::"createReadStream throws RANGE_INVALID for negative offset"` AND `src/drive/drive.test.ts::"createReadStream throws RANGE_INVALID for non-integer length"` AND `src/drive/drive.test.ts::"createReadStream throws RANGE_INVALID for NaN or Infinity"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Out-of-bounds range surfaces `DRIVE.RANGE_OUT_OF_BOUNDS`
|
||||
|
||||
When the underlying source exposes a known size at stream-start (via the chunks-plugin manifest, a storage `stat`, or a `nativeRange`-capable storage that reports `size`), `Drive.createReadStream` MUST raise `DriveError(DRIVE.RANGE_OUT_OF_BOUNDS, ...)` synchronously for `offset > size` or `offset + length > size`. When size is unknown at start-time, the stream MUST accept the request, emit available bytes, and on close MUST surface `DRIVE.RANGE_OUT_OF_BOUNDS` via `controller.error` if the byte count emitted is less than the requested `length`. The error MUST carry the originating `CHUNKS.RANGE_OUT_OF_BOUNDS` (when applicable) as `cause`.
|
||||
|
||||
#### Scenario: offset past EOF rejected synchronously when size is known
|
||||
|
||||
- **WHEN** the underlying file is 1024 bytes and `drive.createReadStream("file.bin", { offset: 2048 })` is called via a chunks-plugin-mediated path
|
||||
- **THEN** the call throws `DriveError(DRIVE.RANGE_OUT_OF_BOUNDS, ...)` synchronously with `cause` carrying `CHUNKS.RANGE_OUT_OF_BOUNDS`
|
||||
|
||||
#### Scenario: offset+length past EOF rejected synchronously when size is known
|
||||
|
||||
- **WHEN** the file is 1024 bytes and `drive.createReadStream("file.bin", { offset: 900, length: 500 })` is called via the chunks plugin
|
||||
- **THEN** the call throws `DriveError(DRIVE.RANGE_OUT_OF_BOUNDS, ...)` synchronously
|
||||
|
||||
#### Scenario: short-fall on stream end when size was unknown at start
|
||||
|
||||
- **WHEN** the storage cannot report size, `drive.createReadStream("file.bin", { offset: 0, length: 10_000 })` is called, and the source closes after 4000 bytes
|
||||
- **THEN** the stream errors with `DriveError(DRIVE.RANGE_OUT_OF_BOUNDS, ...)` and `context.emittedBytes === 4000`
|
||||
|
||||
**Verified by:** `src/drive/drive.test.ts::"createReadStream throws RANGE_OUT_OF_BOUNDS when offset exceeds known size"` AND `src/drive/drive.test.ts::"createReadStream throws RANGE_OUT_OF_BOUNDS when offset+length exceeds known size"` AND `src/file-operations/streams.range.test.ts::"stream errors with RANGE_OUT_OF_BOUNDS on short-fall when size is unknown"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `nativeRange` storage capability is honoured
|
||||
|
||||
`DriveStorage` SHALL accept an optional `capabilities.nativeRange?: boolean` flag. When `true`, `Drive.createReadStream(path, { offset, length, signal })` MUST forward `{ offset, length }` to `storage.createReadStream` directly. When `false` (or absent), the drive MUST wrap the storage stream in a slicing transform that drops bytes before `offset` and stops emitting after `length` bytes. Both paths MUST produce identical observable output.
|
||||
|
||||
#### Scenario: storage with nativeRange receives offset/length
|
||||
|
||||
- **WHEN** `storage.capabilities.nativeRange === true` and `drive.createReadStream("file.bin", { offset: 1024, length: 4096 })` is invoked
|
||||
- **THEN** `storage.createReadStream` is called exactly once with `{ offset: 1024, length: 4096, signal }` and no slicing transform is interposed
|
||||
|
||||
#### Scenario: storage without nativeRange is wrapped in slicing transform
|
||||
|
||||
- **WHEN** `storage.capabilities?.nativeRange` is `undefined` or `false` and the same call is made
|
||||
- **THEN** `storage.createReadStream` is called with `{ signal }` only, and the returned stream is wrapped so the consumer observes exactly bytes `[1024, 5120)`
|
||||
|
||||
**Verified by:** `src/file-operations/streams.range.test.ts::"nativeRange storage receives offset and length directly"` AND `src/file-operations/streams.range.test.ts::"non-nativeRange storage is wrapped in a slicing transform"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Range cancellation propagates via `AbortSignal`
|
||||
|
||||
`Drive.createReadStream(path, options)` MUST propagate `options.signal` (or the bare signal in the back-compat form) through the chunks-plugin / mesh / storage pull chain. On abort, in-flight chunk fetches MUST be cancelled, the stream MUST close with `controller.error(AbortError)`, and (over `drive+mesh://`) the mesh transport MUST emit a `cd:error` envelope with `code: "ABORTED"`.
|
||||
|
||||
#### Scenario: mid-stream abort cancels chunk fetches
|
||||
|
||||
- **WHEN** `drive.createReadStream("file.bin", { offset: 0, length: 1_048_576, signal })` is called and `signal.abort()` is invoked after the first chunk emits
|
||||
- **THEN** the stream closes with an `AbortError`, no further chunks are fetched, and the chunks-plugin loop exits at the next `throwIfAborted` checkpoint
|
||||
|
||||
**Verified by:** `src/file-operations/streams.range.test.ts::"abort mid-range stream cancels in-flight chunk fetches"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `DRIVE_CODES` registry adds `RANGE_INVALID` and `RANGE_OUT_OF_BOUNDS`
|
||||
|
||||
`DRIVE_CODES` SHALL include `"RANGE_INVALID"` and `"RANGE_OUT_OF_BOUNDS"` declared via the existing `defineCodes("DRIVE", [...] as const)` call. `Object.isFrozen(DRIVE_CODES)` MUST remain `true`. Every code value MUST start with `DRIVE.`.
|
||||
|
||||
#### Scenario: registry exposes the new codes
|
||||
|
||||
- **WHEN** the test imports `DRIVE_CODES` from `@suaveplan/drive`
|
||||
- **THEN** `DRIVE_CODES.RANGE_INVALID === "DRIVE.RANGE_INVALID"` AND `DRIVE_CODES.RANGE_OUT_OF_BOUNDS === "DRIVE.RANGE_OUT_OF_BOUNDS"` AND `Object.isFrozen(DRIVE_CODES) === true`
|
||||
|
||||
**Verified by:** `src/error-codes.spec-conformance.test.ts::"DRIVE_CODES exposes RANGE_INVALID and RANGE_OUT_OF_BOUNDS"` AND `src/error-codes.spec-conformance.test.ts::"Object.isFrozen(DRIVE_CODES) is true"`
|
||||
@@ -0,0 +1,79 @@
|
||||
# Tasks — add-drive-range-reads
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `openspec/AGENTS.md` §0c, §2b, §6a, §7, §11, §13, §13a, §14
|
||||
- [ ] 1.2 Read `openspec/archive/2026-05-16-add-drive-uri/proposal.md` deferral note and `add-drive-uri` archived delta
|
||||
- [ ] 1.3 Read `openspec/specs/drive/spec.md`, `openspec/specs/drive-uri/spec.md`
|
||||
- [ ] 1.4 Confirm `Drive.createReadStream`, chunks `#reassemble`, mesh `ChunkRequest` shapes in source
|
||||
- [ ] 1.5 Draft `proposal.md` + `design.md` + `tasks.md` + spec deltas
|
||||
- [ ] 1.6 Build the numbered Requirement / Scenario implementation checklist (per §0c step 6)
|
||||
- [ ] 1.7 `openspec validate add-drive-range-reads --strict` exits 0
|
||||
|
||||
## 2. `@suaveplan/drive` — range plumbing
|
||||
|
||||
- [ ] 2.1 `src/types/drive.types.ts` — add `RangeReadOptions` interface; extend `DriveStorage` with optional `capabilities.nativeRange?: boolean`; extend `DriveStorage.createReadStream?` signature with optional `{ offset, length }` (back-compat preserved)
|
||||
- [ ] 2.2 `src/error-codes.ts` — add `RANGE_INVALID`, `RANGE_OUT_OF_BOUNDS` to `DRIVE_CODES`
|
||||
- [ ] 2.3 `src/error-codes.spec-conformance.test.ts` — extend frozen-registry test, every-code-starts-with-`DRIVE.` test
|
||||
- [ ] 2.4 `src/file-operations/streams.ts` — overload `createReadStream(storage, path, signal?)` AND `createReadStream(storage, path, options: RangeReadOptions)`; runtime `instanceof AbortSignal` discriminator; synchronous validation → `DRIVE.RANGE_INVALID`; slicing transform when `!nativeRange`
|
||||
- [ ] 2.5 `src/file-operations/streams.md` — update docs (range API, capability flag, sliced vs. native behaviour)
|
||||
- [ ] 2.6 `src/file-operations/streams.range.test.ts` — boundary table cases: range fully inside one chunk, spans boundary, aligned multi-chunk, tail-to-EOF, zero-length, whole-file, abort mid-stream
|
||||
- [ ] 2.7 `src/file-operations/streams.range.md` — sibling doc
|
||||
- [ ] 2.8 `src/drive/drive.ts` — `createReadStream` overload signatures forwarded to `streams.ts` helpers; middleware context carries `range?: { offset, length? }`
|
||||
- [ ] 2.9 `src/drive/drive.test.ts` — overload-compat (bare signal still works), options form, `DRIVE.RANGE_INVALID` synchronous throw, `DRIVE.RANGE_OUT_OF_BOUNDS` (size-known) synchronous throw
|
||||
|
||||
## 3. `@suaveplan/drive-plugin-chunks` — range-aware reassemble
|
||||
|
||||
- [ ] 3.1 `src/pipeline/range.ts` — pure function `chunkRangeFor({ offset, length, fileSize, chunkSize })` returning `{ firstChunk, lastChunk, firstSkip, lastKeep }` or `null` for zero-length
|
||||
- [ ] 3.2 `src/pipeline/range.md`
|
||||
- [ ] 3.3 `src/pipeline/range.test.ts` — exhaustive table (design § "Worked examples"), fast-check property tests for invariants (`offset + length ≤ fileSize`, slices reassemble to `bytes[offset, end)`)
|
||||
- [ ] 3.4 `src/error-codes.ts` — add `CHUNKS.RANGE_OUT_OF_BOUNDS`
|
||||
- [ ] 3.5 `src/error-codes.spec-conformance.test.ts` — extend
|
||||
- [ ] 3.6 `src/plugin.ts` — refactor `#reassemble` to accept `range?: RangeReadOptions`; sibling `#reassembleAsStream(fileId, range?)`; middleware reads `ctx.range` and forwards; per-iteration `signal.throwIfAborted()`
|
||||
- [ ] 3.7 `src/plugin.md` — update docs (range path, mesh request optimisation, error code)
|
||||
- [ ] 3.8 `src/plugin.range.test.ts` — covers buffered read with range + streaming read with range + `RANGE_OUT_OF_BOUNDS` on oversize range + abort mid-range
|
||||
|
||||
## 4. `@suaveplan/drive-mesh` — range → ChunkRequest pivot
|
||||
|
||||
- [ ] 4.1 `src/pull/range-request.ts` — `buildRangeRequest({ firstChunk, lastChunk, totalChunks })` returns `ChunkRequest.specific` when `span ≤ 64`, else `ChunkRequest.bitset`
|
||||
- [ ] 4.2 `src/pull/range-request.md`
|
||||
- [ ] 4.3 `src/pull/range-request.test.ts` — pivot boundary at exactly 64 chunks; bitset round-trips through `encodeBitset`/`decodeBitset`; specific shape rejects empty span
|
||||
- [ ] 4.4 `src/parallel-puller.ts` — accept range, build request via `buildRangeRequest`, narrow parallelism domain to in-range chunkIds, propagate `AbortSignal`
|
||||
- [ ] 4.5 `src/parallel-puller.test.ts` — extend: only in-range chunks pulled; abort cancels in-flight; small-range uses `specific`, large-range uses `bitset`
|
||||
|
||||
## 5. `@suaveplan/drive-uri` — range pass-through
|
||||
|
||||
- [ ] 5.1 `src/types/plugin-options.types.ts` — extend adapter capabilities (`nativeRange: true`)
|
||||
- [ ] 5.2 `src/adapter/drive-storage-like.ts` — `createReadStream(key, { offset?, length?, signal? })` forwards to `Drive.createReadStream(key, { offset, length, signal })`
|
||||
- [ ] 5.3 `src/adapter/drive-storage-like.md` — update
|
||||
- [ ] 5.4 `src/adapter/drive-storage-like.test.ts` — range forwarded; `nativeRange` capability flag asserted
|
||||
- [ ] 5.5 `src/plugins/drive/drive.streams.ts` — `InstrumentedDriveAdapter.createReadStream` accepts and forwards range; emits `drive.uri.range` span attribute when present
|
||||
- [ ] 5.6 `src/plugins/drive/drive.streams.test.ts` — range round-trip via plugin; abort mid-stream; attribute present iff range present
|
||||
- [ ] 5.7 `src/otel/otel.ts` — `range_chunks_fetched` histogram; `drive.uri.range = "{offset}-{end}"` setter
|
||||
- [ ] 5.8 `src/otel/otel.test.ts` — histogram increments, attribute set, attribute omitted when no range
|
||||
- [ ] 5.9 `tests/integration/drive-range-roundtrip.test.ts` — real `Drive` + storage-file: write 200 KiB file, read bytes [60 000, 70 000) via `resolver.read` flow, assert equality
|
||||
|
||||
## 6. Pre-archive verification (§13)
|
||||
|
||||
- [ ] 6.1 `bunx turbo lint typecheck test --filter=@suaveplan/drive --filter=@suaveplan/drive-plugin-chunks --filter=@suaveplan/drive-mesh --filter=@suaveplan/drive-uri`
|
||||
- [ ] 6.2 `bunx turbo test:coverage` (100 / 100 / 100 / 100 on every touched package)
|
||||
- [ ] 6.3 `bunx turbo build`
|
||||
- [ ] 6.4 Banned-pattern scan; barrel-shape scan; test-runner import scan; missing `.md` scan; README check; bare-version check; subpath export count; security scan
|
||||
- [ ] 6.5 §13 step 10 dependent sweep: `bunx turbo test --filter=...@suaveplan/drive`
|
||||
- [ ] 6.6 Walk every `### Requirement` / `#### Scenario` / `**Verified by:**` — confirm every named test exists and the name matches verbatim
|
||||
|
||||
## 7. Doc + manifest refresh (§13b)
|
||||
|
||||
- [ ] 7.1 Update `packages/drive/drive/README.md`, `packages/drive/drive-plugin-chunks/README.md`, `packages/drive/drive-mesh/README.md`, `packages/drive/drive-uri/README.md`
|
||||
- [ ] 7.2 `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 7.3 `bun scripts/generate-missing.ts`
|
||||
- [ ] 7.4 Update `STATUS.md`, `SUMMARY.md` for the four touched packages
|
||||
- [ ] 7.5 `bunx markdownlint-cli2 --fix`
|
||||
|
||||
## 8. Archival (§14)
|
||||
|
||||
- [ ] 8.1 Fold `specs/drive/spec.md` delta into `openspec/specs/drive/spec.md`
|
||||
- [ ] 8.2 Fold `specs/drive-uri/spec.md` delta into `openspec/specs/drive-uri/spec.md`
|
||||
- [ ] 8.3 Move `openspec/changes/add-drive-range-reads/` → `openspec/archive/<yyyy-mm-dd>-add-drive-range-reads/`
|
||||
@@ -0,0 +1,93 @@
|
||||
# Design — `add-drive-uri-writes`
|
||||
|
||||
Non-normative. The normative contract is `./specs/drive-uri/spec.md`.
|
||||
|
||||
## Per-scheme operation matrix
|
||||
|
||||
| Operation | `drive://` | `drive+mesh://` | `drive+template://` |
|
||||
|---|---|---|---|
|
||||
| `set(key, bytes)` (resolver `write`) | `Drive.write(key, bytes, { signal })` → `void` | `Drive.write(key, bytes, { signal })` → `void`; mesh advertise NOT triggered | throw `DRIVE_URI.TEMPLATE_READ_ONLY` |
|
||||
| `delete(key)` (resolver `remove`) | `Drive.delete(key, signal)` → `boolean` | `Drive.delete(key, signal)` → `boolean`; mesh "no-longer-available" advert NOT triggered | throw `DRIVE_URI.TEMPLATE_READ_ONLY` |
|
||||
| `clear()` | throw `DRIVE_URI.WRITE_UNSUPPORTED` (Drive-wide wipe footgun) | same | same |
|
||||
| `createReadStream` | unchanged | unchanged | unchanged |
|
||||
| `get` / `exists` / `list` / `stat` | unchanged | unchanged | unchanged |
|
||||
|
||||
`clear()` retains `DRIVE_URI.WRITE_UNSUPPORTED` to preserve the existing error contract for that footgun call site. The new `TEMPLATE_READ_ONLY` code is specific to template-backed mutation attempts — operators differentiate "this drive forbids wholesale wipes" (a config-time concern) from "this URI scheme is intrinsically read-only" (an architectural constraint).
|
||||
|
||||
## Error code transition
|
||||
|
||||
| Old behaviour | New behaviour | Code |
|
||||
|---|---|---|
|
||||
| `set` on `drive://` → throw `WRITE_UNSUPPORTED` | succeeds | (success) |
|
||||
| `set` on `drive+mesh://` → throw `WRITE_UNSUPPORTED` | succeeds | (success) |
|
||||
| `set` on `drive+template://` → throw `WRITE_UNSUPPORTED` | throw `TEMPLATE_READ_ONLY` | `DRIVE_URI.TEMPLATE_READ_ONLY` |
|
||||
| `delete` on `drive+template://` → forwarded to underlying `Drive.delete` (silent footgun: would mutate the hydrated cache, breaking the dehydrated semantics) | throw `TEMPLATE_READ_ONLY` | `DRIVE_URI.TEMPLATE_READ_ONLY` |
|
||||
| `clear()` on any scheme → throw `WRITE_UNSUPPORTED` | unchanged | `DRIVE_URI.WRITE_UNSUPPORTED` |
|
||||
|
||||
`DRIVE_URI.WRITE_UNSUPPORTED` is intentionally retained — consumers who already asserted on it for the `clear()` path continue to work; we narrow its meaning rather than churn the constant.
|
||||
|
||||
## Mesh write — propagation timing and observability
|
||||
|
||||
`drive+mesh://` writes complete when local persist is durable. Specifically:
|
||||
|
||||
1. `Drive.write(path, bytes, { signal })` returns when the installed `chunksPlugin` middleware has resolved (encrypted, chunked, persisted to the configured storage).
|
||||
2. Mesh re-advertise (`mesh.advertise(fileId, bitset, totalChunks)`) is NOT triggered by the URI surface. This matches direct-`Drive` behaviour and avoids surprising consumers who manage `GenerationTracker` / `AdvertiseThrottle` lifecycles themselves.
|
||||
3. The plugin emits a span event `drive_uri.mesh_write_local_only` on every successful `drive+mesh://` write, with attributes `{ scheme: "drive+mesh", path }`. Operators can grep the trace to confirm the contract holds in their environment.
|
||||
4. Consumers wanting gossip propagation invoke `mesh.advertise(fileId, ...)` after the write resolves. Propagation thereafter follows the existing `GossipEngine` timing (best-effort, backoff-throttled, observable via the mesh package's own events).
|
||||
|
||||
This deliberate choice preserves separation of concerns: the URI surface is a routing layer; mesh participation is a Drive-tier policy.
|
||||
|
||||
## Key-rotation impact (none, by inheritance)
|
||||
|
||||
A write may produce data encrypted under a NEW `KeySlotPublic` version. The URI surface does no key derivation, no slot resolution, and no grant signing — those live in `chunksPlugin` and in `Drive.hydrateFrom`'s `key:grant` subscription. The read path validates retroactive grants today; nothing in the write path needs to change for new key versions to be honoured on subsequent reads.
|
||||
|
||||
No new wiring required at the URI tier.
|
||||
|
||||
## AbortSignal threading
|
||||
|
||||
- `resolver.write(uri, body, signal?)` passes `signal` to `InstrumentedDriveAdapter.set(key, value, signal)`.
|
||||
- `InstrumentedDriveAdapter.set` updates its signature to accept `AbortSignal` (currently typed `set(key, value)` per `StorageLike`; we add the third arg as an optional propagated to `Drive.write({ signal })`). Pre-aborted signals short-circuit to `DRIVE_URI.ABORTED` before any work.
|
||||
- `resolver.remove(uri, signal?)` already threads `signal` to `adapter.delete(key)`; we widen the override to accept and forward `signal` to `Drive.delete(path, signal)`.
|
||||
|
||||
`StorageLike` itself does not specify `signal` on `set`/`delete`. `DriveStorageLikeAdapter` adds the parameter as a tolerated extension; the resolver already passes it via the streaming-write surface (`createWriteStream(key, body, signal)`), so the precedent is established.
|
||||
|
||||
## File-layout impact
|
||||
|
||||
New / edited files (under `packages/drive/drive-uri/src/`):
|
||||
|
||||
```
|
||||
adapter/
|
||||
drive-storage-like.ts # widen set/delete to accept signal; forward through
|
||||
drive-storage-like.md # doc update — note new TEMPLATE_READ_ONLY case
|
||||
error-codes/
|
||||
error-codes.ts # add TEMPLATE_READ_ONLY to defineCodes
|
||||
error-codes.md # update; describe narrowed WRITE_UNSUPPORTED role
|
||||
plugins/
|
||||
drive/
|
||||
drive.streams.ts # remove unconditional WRITE_UNSUPPORTED; per-scheme dispatch
|
||||
drive.streams.md # doc update
|
||||
drive-template/
|
||||
drive-template.ts # override set/delete → TEMPLATE_READ_ONLY
|
||||
drive-template.md # doc update
|
||||
otel/
|
||||
otel.ts # add `mesh_write_local_only` span-event constant
|
||||
otel.md # doc update
|
||||
README.md # update Features / Modules sections
|
||||
tests/integration/
|
||||
drive-roundtrip-write.test.ts # NEW — real Drive, write+read cycle
|
||||
drive-mesh-roundtrip-write.test.ts # NEW — mesh-configured Drive, assert no implicit advertise
|
||||
template-write-rejection.test.ts # NEW — TEMPLATE_READ_ONLY round-trip
|
||||
```
|
||||
|
||||
All new code stays inside the existing 300-line / 500-line file budgets; the changed `drive.streams.ts` currently sits at 148 lines and the per-scheme branching adds ~30 lines.
|
||||
|
||||
## Trade-offs considered
|
||||
|
||||
- **Per-scheme `set` dispatch via `if (scheme === "drive+template")` vs. polymorphic adapter subclass.** Chose the inline branch — only one scheme has the constraint and a subclass would duplicate the entire instrumented method body. If a fourth scheme appears, revisit.
|
||||
- **Auto-advertise on mesh writes.** Rejected: violates the direct-`Drive` contract, would surprise consumers who run `GenerationTracker` themselves, and forces every mesh-write call site through an advertise budget. The span event gives operators visibility without enforcing policy.
|
||||
- **Reusing `WRITE_UNSUPPORTED` for template rejections.** Rejected: it conflates a temporary state (the deferred scope this change closes) with a permanent architectural constraint (templates are signed). Distinct codes let consumers branch correctly.
|
||||
- **Removing `WRITE_UNSUPPORTED` entirely.** Rejected: it would churn the public surface (consumers may already assert on it for the `clear()` path) and §2c forbids deleting a registered code without a real reason. Narrowing the role is the right move.
|
||||
|
||||
## Cross-reference
|
||||
|
||||
This change is the second consumer of structural `DriveLike` (after `add-drive-range-reads`), strengthening the case for `hoist-drivelike-to-types`. Independent landing — `drive-uri` imports `DriveLike` from `@suaveplan/drive` today and will move to `@suaveplan/types` once `hoist-drivelike-to-types` archives.
|
||||
@@ -0,0 +1,93 @@
|
||||
# Unblock writes/removes via `drive://`, `drive+mesh://`, `drive+template://`
|
||||
|
||||
**Package**: `@suaveplan/drive-uri`
|
||||
**Status**: PROPOSED
|
||||
**Type**: capability-extension (genesis-original follow-up)
|
||||
**Depends on**: `add-drive-uri` (archived `2026-05-16`) — provides the three plugins + `InstrumentedDriveAdapter`.
|
||||
**Sibling**: `hoist-drivelike-to-types` (independent — landing order is irrelevant; `drive-uri` consumes structural `DriveLike` from whichever source has merged when this lands).
|
||||
**Sibling**: `add-drive-range-reads` (read-side, independent — out of scope here).
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/drive-uri/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
The shipped `add-drive-uri` change explicitly deferred writes:
|
||||
|
||||
> *Initial release rejects `write`/`remove` with `DRIVE_URI.WRITE_UNSUPPORTED`. Reason: write semantics for shared/mesh files (chunk re-upload, gossip re-advertise, template re-issue, retroactive key grants) are non-trivial; they belong in a follow-up `add-drive-uri-writes` change once read paths are proven in production.*
|
||||
|
||||
Read paths have shipped, the conformance harness is green, and the four originally-cited blockers all resolve cleanly at the URI tier when analysed per-scheme:
|
||||
|
||||
| Cited blocker | Resolution per scheme |
|
||||
|---|---|
|
||||
| Chunk re-upload | Already handled inside `Drive.write` via the installed `chunksPlugin` middleware. The URI surface forwards `path`/`data`/`signal` unchanged — no chunk codec logic in `drive-uri`. |
|
||||
| Gossip re-advertise | `drive-mesh` exposes `advertise(fileId, bitset, totalChunks)` as an application-controlled call (see `drive-mesh/src/mesh/drive-mesh/drive-mesh.ts:176`). Writes do NOT auto-advertise on direct `Drive` use today, and `drive-uri` MUST preserve that contract — best-effort propagation is observable via the existing mesh events the consumer already subscribes to. |
|
||||
| Template re-issue | Templates are dehydrated, authority-signed descriptors. The resolver cannot re-sign them. `drive+template://` writes are therefore **permanently** rejected with a new specific code `DRIVE_URI.TEMPLATE_READ_ONLY`, not the generic `WRITE_UNSUPPORTED`. |
|
||||
| Retroactive key grants | Inherited from the underlying `Drive` — `key:grant` subscriptions and `KeySlotPublic` rotation live in `Drive.hydrateFrom` and the `chunksPlugin`, not in the URI surface. A new key version added after a write is consumed by future reads through the same path the existing read surface already validates. |
|
||||
|
||||
This change ships the two surfaces that are safe today (`drive://` pass-through and `drive+mesh://` write-then-let-consumer-advertise), permanently locks down the surface that cannot be safely lifted (`drive+template://` writes), and narrows the role of `DRIVE_URI.WRITE_UNSUPPORTED` so it no longer pretends to cover a permanent constraint.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `drive://` writes/removes — `InstrumentedDriveAdapter.set(key, value)` and `.delete(key)` for the `drive` scheme delegate to `Drive.write(path, data, { signal })` and `Drive.delete(path, signal)`. `AbortSignal` is threaded through the resolver `write(uri, body, signal)` and `remove(uri, signal)` surfaces. OTel `drive_uri.writes_total{scheme:"drive", outcome:"success"|"error"}` and `drive_uri.deletes_total{scheme:"drive", outcome}` populated.
|
||||
- ADDED: `drive+mesh://` writes/removes — identical delegation to `Drive.write` / `Drive.delete`; the consumer's installed `chunksPlugin` handles encryption + chunk persistence locally. Mesh re-advertisement is NOT triggered by the URI surface (matching the direct-`Drive` contract); the consumer SHALL call `mesh.advertise(fileId, bitset, totalChunks)` after the write resolves if they want gossip propagation. The plugin documents this contract on the sibling `.md` and surfaces it on a span event `drive_uri.mesh_write_local_only` so operators can audit any environment that expects implicit gossip.
|
||||
- ADDED: `drive+template://` writes/removes — **permanently rejected** with the new code `DRIVE_URI.TEMPLATE_READ_ONLY`. A template-backed drive is a dehydrated, server-signed view; mutating it would require an authority signing key the URI resolver never holds. Removes are rejected for the same reason. `drive_uri.writes_total{scheme:"drive+template", outcome:"error"}` and `drive_uri.deletes_total{scheme:"drive+template", outcome:"error"}` increment on the rejection.
|
||||
- ADDED: New error code `DRIVE_URI.TEMPLATE_READ_ONLY` via `defineCodes` (registry grows from 7 → 8). Reachable from the `drive+template` adapter's `set` and `delete` overrides; both paths covered by tests.
|
||||
- MODIFIED: `DRIVE_URI.WRITE_UNSUPPORTED` retained in the registry but narrowed to a **single call site** — `DriveStorageLikeAdapter.clear()` (the explicit `Drive.delete`-this-instead footgun). It is no longer the rejection code for write/remove via `set`/`delete`. The README, sibling `.md`, and spec all describe this narrower role.
|
||||
- MODIFIED: `InstrumentedDriveAdapter.set` no longer unconditionally throws. It dispatches per the scheme bound at construction: `drive`/`drive+mesh` → delegate; `drive+template` → throw `TEMPLATE_READ_ONLY`. Same for `delete`.
|
||||
- MODIFIED: Conformance — the `runUriPluginConformance` writes suite is re-run against all three plugins. Expected outcomes: `drive://` and `drive+mesh://` complete the full write/read/exists/list/remove cycle; `drive+template://` continues to reject writes (now with `TEMPLATE_READ_ONLY` rather than `WRITE_UNSUPPORTED`).
|
||||
- ADDED: `tests/integration/drive-roundtrip-write.test.ts` — write bytes via `resolver.write("drive://primary/x", bytes)`, read back via `resolver.read`, assert equality. No mocks.
|
||||
- ADDED: `tests/integration/drive-mesh-roundtrip-write.test.ts` — same against a mesh-configured Drive; assert no implicit `advertise` call; explicit `mesh.advertise(...)` after the write produces a verifiable outbound advertisement via the existing transport stub fixture.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Auto-gossip on `drive+mesh://` writes.** Direct `Drive.write` does not auto-advertise today; the URI surface MUST NOT bend that contract. Tracked separately if a future use case proves the policy.
|
||||
- **Template re-issue.** Permanently scoped out — needs an authority signing key the resolver never possesses. The `TEMPLATE_READ_ONLY` rejection is the contract.
|
||||
- **Range reads on `drive+mesh://`.** Independent change `add-drive-range-reads`.
|
||||
- **`DriveLike` hoist to `@suaveplan/types`.** Independent change `hoist-drivelike-to-types`. This change consumes the structural shape from whichever has merged first; it does NOT depend on the hoist landing first.
|
||||
- **New `Drive.write` options** (atomicity, conditional puts, content-type). Out of scope — would require a `Drive`-tier proposal.
|
||||
- **Removing `DRIVE_URI.WRITE_UNSUPPORTED`.** Kept for the `clear()` footgun call site (a stable surface code consumers may already assert on).
|
||||
|
||||
## Source mapping
|
||||
|
||||
- Deferred-text in `openspec/archive/2026-05-16-add-drive-uri/proposal.md` lines 30–38 — the explicit follow-up scope.
|
||||
- `packages/drive/drive-uri/src/plugins/drive/drive.streams.ts:69-84` — current unconditional `set` rejection.
|
||||
- `packages/drive/drive-uri/src/adapter/drive-storage-like.ts:75-86` — `set`/`delete` already pass-through internally; only the instrumented override blocks them.
|
||||
- `packages/drive/drive/src/drive/drive.ts:180-200` — `Drive.write(path, data, options?)` and `Drive.delete(path, signal?)` signatures consumed verbatim.
|
||||
- `packages/drive/drive-mesh/src/mesh/drive-mesh/drive-mesh.ts:176-191` — `mesh.advertise(fileId, bitset, totalChunks)` — confirms gossip is consumer-controlled, not write-triggered.
|
||||
- `packages/drive/drive-template/src/` — no surface here is mutable; the only API touching write is `hydrateFrom`, which is a read-side projection.
|
||||
- `packages/storage/storage-uri/src/resolver/resolver.writes.ts:30-75` — confirms `SuaveplanError` subclasses (including `DriveUriError`) pass through `writeViaPlugin` unchanged, so `TEMPLATE_READ_ONLY` reaches the caller without a `STORAGE_URI.WRITE_UNSUPPORTED` envelope.
|
||||
- `openspec/specs/drive-uri/spec.md` Requirement at line 181 — current global write rejection, replaced by the new per-scheme matrix.
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement in the spec delta carries an explicit `**Verified by:**` line referencing planned `it(...)` titles in concrete test files (see delta). The pre-archive gate (§13 of `AGENTS.md`) re-runs the full `runUriPluginConformance` writes suite against all three plugins.
|
||||
|
||||
## Migration
|
||||
|
||||
| Old | New |
|
||||
|---|---|
|
||||
| `resolver.write("drive://…", bytes)` throws `DRIVE_URI.WRITE_UNSUPPORTED` | succeeds, returns `void` |
|
||||
| `resolver.write("drive+mesh://…", bytes)` throws `DRIVE_URI.WRITE_UNSUPPORTED` | succeeds, returns `void`; gossip remains consumer-controlled |
|
||||
| `resolver.write("drive+template://…", bytes)` throws `DRIVE_URI.WRITE_UNSUPPORTED` | throws `DRIVE_URI.TEMPLATE_READ_ONLY` |
|
||||
| `resolver.remove("drive+template://…", …)` succeeded (via inherited `DriveStorageLikeAdapter.delete`) | throws `DRIVE_URI.TEMPLATE_READ_ONLY` (closes a silent footgun) |
|
||||
| `DRIVE_URI.WRITE_UNSUPPORTED` raised for `set`/`delete` | only raised for `DriveStorageLikeAdapter.clear()` |
|
||||
|
||||
Search consumers: `grep -rln "DRIVE_URI.WRITE_UNSUPPORTED\|WRITE_UNSUPPORTED" packages/ | grep -v "/dist/\|/coverage/\|/drive-uri/"`. Recommended replacement: branch on `err.code === "DRIVE_URI.TEMPLATE_READ_ONLY"` for the immutable template surface; allow the success path elsewhere.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/drive-uri` to enumerate.
|
||||
|
||||
- No external dependents yet (`drive-uri` is consumed only via integration tests at the time of writing) — [x] no action needed (reason: new consumers will adopt the broader surface; existing surface is purely permissive expansion except for `drive+template://` removes, called out in the Migration table).
|
||||
|
||||
### Affected upstreams (none modified)
|
||||
|
||||
| Package | Action | Breaking? |
|
||||
|---|---|---|
|
||||
| `@suaveplan/drive` | None — `Drive.write` / `Drive.delete` consumed as shipped | no |
|
||||
| `@suaveplan/drive-mesh` | None — `advertise` remains consumer-triggered | no |
|
||||
| `@suaveplan/drive-template` | None — no mutable surface exists; `TEMPLATE_READ_ONLY` codifies that fact | no |
|
||||
| `@suaveplan/storage-uri` | None — `writeViaPlugin` already passes through `SuaveplanError` subclasses | no |
|
||||
@@ -0,0 +1,143 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: drive-uri (drive-uri)
|
||||
|
||||
**Package**: `@suaveplan/drive-uri`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement carries a `**Verified by:**` line at authoring time naming planned tests.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `drive://` SHALL support writes and removes via the resolver surface
|
||||
|
||||
The `drive` plugin's instrumented adapter SHALL delegate `set(key, value, signal?)` to `Drive.write(path, data, { signal })` and `delete(key, signal?)` to `Drive.delete(path, signal)`. Successful writes SHALL increment `drive_uri.writes_total{scheme:"drive", outcome:"success"}`; successful removes SHALL increment `drive_uri.deletes_total{scheme:"drive", outcome:"success"}`. The `AbortSignal` passed to `resolver.write(uri, body, signal)` / `resolver.remove(uri, signal)` SHALL be threaded through to the underlying `Drive` call. A pre-aborted signal SHALL short-circuit to `DRIVE_URI.ABORTED` without invoking `Drive.write` / `Drive.delete`.
|
||||
|
||||
#### Scenario: write round-trips via Drive.write
|
||||
|
||||
- **WHEN** `resolver.write("drive://primary/clip.mov", bytes)` is called against a `drive` plugin bound to a real local-storage `Drive`
|
||||
- **THEN** the call resolves with `void`, the underlying `Drive` reports the path persisted, and `resolver.read("drive://primary/clip.mov")` returns `bytes`
|
||||
|
||||
#### Scenario: remove delegates to Drive.delete
|
||||
|
||||
- **WHEN** `resolver.remove("drive://primary/clip.mov")` is called after a successful write
|
||||
- **THEN** the call resolves with `true` and a subsequent `resolver.read(...)` rejects with `STORAGE_URI.NOT_FOUND`
|
||||
|
||||
#### Scenario: AbortSignal threads through writes
|
||||
|
||||
- **WHEN** a `resolver.write(...)` is invoked with an `AbortSignal` that aborts mid-call
|
||||
- **THEN** the call rejects with `DRIVE_URI.ABORTED` and any underlying `Drive.write` invocation observes the abort
|
||||
|
||||
**Verified by:** `src/plugins/drive/drive.test.ts::"writes succeed and round-trip via Drive.write"` AND `src/plugins/drive/drive.test.ts::"removes delegate to Drive.delete and return its boolean"` AND `src/plugins/drive/drive.test.ts::"writes propagate AbortSignal to Drive.write"` AND `tests/integration/drive-roundtrip-write.test.ts::"resolver.write then resolver.read round-trips on a real Drive"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `drive+mesh://` SHALL support writes/removes locally without triggering mesh re-advertisement
|
||||
|
||||
The `drive+mesh` plugin's instrumented adapter SHALL delegate `set`/`delete` to the underlying `Drive` exactly as the `drive` scheme does. It SHALL NOT call `mesh.advertise(...)` or any equivalent gossip API; gossip propagation remains the consumer's responsibility. On every successful `drive+mesh://` write, the plugin SHALL emit a span event named `drive_uri.mesh_write_local_only` carrying `{ scheme: "drive+mesh", path }` so operators can audit that the contract holds.
|
||||
|
||||
#### Scenario: write persists locally without auto-advertise
|
||||
|
||||
- **WHEN** `resolver.write("drive+mesh://swarmA/clip.mov", bytes)` is called against a mesh-configured `Drive` with a spy transport
|
||||
- **THEN** the call resolves with `void`, the bytes are persisted via `Drive.write`, and the spy transport records zero `advertise(...)` calls
|
||||
|
||||
#### Scenario: explicit consumer-side advertise still works
|
||||
|
||||
- **WHEN** the consumer invokes `mesh.advertise(fileId, bitset, totalChunks)` after the write resolves
|
||||
- **THEN** the spy transport records one outbound advertisement
|
||||
|
||||
#### Scenario: span event marks local-only write
|
||||
|
||||
- **WHEN** any successful `drive+mesh://` write completes
|
||||
- **THEN** the active span carries an event `drive_uri.mesh_write_local_only` with attributes `{ scheme: "drive+mesh", path }`
|
||||
|
||||
**Verified by:** `src/plugins/drive-mesh/drive-mesh.test.ts::"writes succeed without triggering mesh.advertise"` AND `src/plugins/drive-mesh/drive-mesh.test.ts::"writes emit drive_uri.mesh_write_local_only span event"` AND `tests/integration/drive-mesh-roundtrip-write.test.ts::"explicit mesh.advertise after write produces an outbound advertisement"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `drive+template://` writes and removes SHALL be permanently rejected with `DRIVE_URI.TEMPLATE_READ_ONLY`
|
||||
|
||||
The `drive+template` plugin's instrumented adapter SHALL throw a `DriveUriError` with `code === "DRIVE_URI.TEMPLATE_READ_ONLY"` on every `set` and `delete` call, before any delegation to `Drive`. The error SHALL carry `scheme: "drive+template"`, the offending `path`, and the `templateId` that was bound. The rejection SHALL increment `drive_uri.writes_total{scheme:"drive+template", outcome:"error"}` for writes and `drive_uri.deletes_total{scheme:"drive+template", outcome:"error"}` for removes. The error message SHALL state that template-backed drives require authority re-signing not held by the resolver.
|
||||
|
||||
#### Scenario: write against drive+template:// rejects with TEMPLATE_READ_ONLY
|
||||
|
||||
- **WHEN** `resolver.write("drive+template://tplA/clip.mov", bytes)` is called
|
||||
- **THEN** the call rejects with `DriveUriError` whose `code === "DRIVE_URI.TEMPLATE_READ_ONLY"`, `scheme === "drive+template"`, `path === "clip.mov"`, and `templateId === "tplA"`
|
||||
|
||||
#### Scenario: remove against drive+template:// rejects with TEMPLATE_READ_ONLY
|
||||
|
||||
- **WHEN** `resolver.remove("drive+template://tplA/clip.mov")` is called
|
||||
- **THEN** the call rejects with `DriveUriError` whose `code === "DRIVE_URI.TEMPLATE_READ_ONLY"`
|
||||
|
||||
#### Scenario: rejection increments error counters
|
||||
|
||||
- **WHEN** any `drive+template://` write rejection fires
|
||||
- **THEN** `drive_uri.writes_total{scheme:"drive+template", outcome:"error", code:"DRIVE_URI.TEMPLATE_READ_ONLY"}` increments by 1
|
||||
|
||||
**Verified by:** `src/plugins/drive-template/drive-template.test.ts::"writes reject with DRIVE_URI.TEMPLATE_READ_ONLY"` AND `src/plugins/drive-template/drive-template.test.ts::"removes reject with DRIVE_URI.TEMPLATE_READ_ONLY"` AND `src/otel/otel.test.ts::"increments writes_total{outcome:error,code:TEMPLATE_READ_ONLY} on template writes"` AND `tests/integration/template-write-rejection.test.ts::"resolver.write and resolver.remove both reject on template-backed drive"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `DRIVE_URI_CODES` SHALL include `TEMPLATE_READ_ONLY`
|
||||
|
||||
The `DRIVE_URI_CODES` registry produced by `defineCodes("DRIVE_URI", […])` SHALL include `TEMPLATE_READ_ONLY` as a distinct, fully-qualified `DRIVE_URI.TEMPLATE_READ_ONLY` code. The `DriveUriErrorCode` union SHALL include it. The code SHALL be reachable from real call sites (the `drive+template` adapter's `set` and `delete` overrides).
|
||||
|
||||
#### Scenario: registry exposes TEMPLATE_READ_ONLY
|
||||
|
||||
- **WHEN** `DRIVE_URI_CODES.TEMPLATE_READ_ONLY` is read
|
||||
- **THEN** the value is the string `"DRIVE_URI.TEMPLATE_READ_ONLY"`
|
||||
|
||||
#### Scenario: registry remains size-stable elsewhere
|
||||
|
||||
- **WHEN** the registry's key count is asserted
|
||||
- **THEN** it equals 8, comprising the original 7 plus `TEMPLATE_READ_ONLY`
|
||||
|
||||
**Verified by:** `src/error-codes/error-codes.test.ts::"includes TEMPLATE_READ_ONLY and totals 8 codes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `runUriPluginConformance` writes suite SHALL pass for `drive`/`drive+mesh` and reject for `drive+template`
|
||||
|
||||
The conformance harness re-exported from `@suaveplan/storage-uri` SHALL be re-run against each of the three plugins. The `drive` and `drive+mesh` plugins SHALL pass the full write/read/exists/list/remove cycle. The `drive+template` plugin SHALL continue to fail the write/remove portions of the suite, but with `DRIVE_URI.TEMPLATE_READ_ONLY` as the rejection code rather than `WRITE_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: conformance writes suite passes for drive
|
||||
|
||||
- **WHEN** the harness is configured with a real-`Drive`-backed `drive` plugin and asked to exercise the writes suite
|
||||
- **THEN** every write/read round-trip assertion holds and no test is skipped
|
||||
|
||||
#### Scenario: conformance writes suite passes for drive+mesh
|
||||
|
||||
- **WHEN** the harness is configured with a mesh-configured `Drive`-backed `drive+mesh` plugin
|
||||
- **THEN** every write/read round-trip holds; no `advertise` is observed on the spy transport during the harness
|
||||
|
||||
#### Scenario: conformance writes suite rejects drive+template with the new code
|
||||
|
||||
- **WHEN** the harness is configured with a `drive+template` plugin and exercises any write/remove
|
||||
- **THEN** every such call rejects with `DRIVE_URI.TEMPLATE_READ_ONLY`
|
||||
|
||||
**Verified by:** `src/conformance/conformance.test.ts::"writes suite passes for drive plugin"` AND `src/conformance/conformance.test.ts::"writes suite passes for drive+mesh plugin"` AND `src/conformance/conformance.test.ts::"writes suite rejects drive+template with TEMPLATE_READ_ONLY"`
|
||||
|
||||
---
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: `DRIVE_URI.WRITE_UNSUPPORTED` SHALL only be raised by `DriveStorageLikeAdapter.clear()`
|
||||
|
||||
Previously the three plugins rejected every `set` (write) call with `DRIVE_URI.WRITE_UNSUPPORTED`. The code SHALL now be raised by exactly one call site: `DriveStorageLikeAdapter.clear()`, which intentionally refuses to wipe an entire Drive through the URI surface and points the caller at `Drive.delete(path)` (or `resolver.remove(uri)`). The code SHALL remain in `DRIVE_URI_CODES` so consumers that already assert on it for the `clear()` path continue to function unchanged. No other call site in `drive-uri` SHALL raise `WRITE_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: clear() still rejects with WRITE_UNSUPPORTED
|
||||
|
||||
- **WHEN** any consumer invokes `adapter.clear()` on a `DriveStorageLikeAdapter` (directly or via an unusual resolver extension)
|
||||
- **THEN** the call rejects with `DriveUriError` whose `code === "DRIVE_URI.WRITE_UNSUPPORTED"`
|
||||
|
||||
#### Scenario: set on drive:// no longer raises WRITE_UNSUPPORTED
|
||||
|
||||
- **WHEN** `resolver.write("drive://primary/x", bytes)` is called against a real `Drive`
|
||||
- **THEN** the call resolves with `void` and no `DRIVE_URI.WRITE_UNSUPPORTED` is thrown anywhere in the call chain
|
||||
|
||||
#### Scenario: set on drive+template:// raises TEMPLATE_READ_ONLY not WRITE_UNSUPPORTED
|
||||
|
||||
- **WHEN** `resolver.write("drive+template://tplA/x", bytes)` is called
|
||||
- **THEN** the rejection's `code` is `DRIVE_URI.TEMPLATE_READ_ONLY`, never `DRIVE_URI.WRITE_UNSUPPORTED`
|
||||
|
||||
**Verified by:** `src/adapter/drive-storage-like.test.ts::"clear() rejects with DRIVE_URI.WRITE_UNSUPPORTED"` AND `src/plugins/drive/drive.test.ts::"writes succeed and round-trip via Drive.write"` AND `src/plugins/drive-template/drive-template.test.ts::"writes reject with DRIVE_URI.TEMPLATE_READ_ONLY"`
|
||||
@@ -0,0 +1,60 @@
|
||||
# Tasks — `add-drive-uri-writes`
|
||||
|
||||
## 1. Registry
|
||||
|
||||
- [ ] 1.1 Add `TEMPLATE_READ_ONLY` to `DRIVE_URI_CODES` in `src/error-codes/error-codes.ts` (8 codes total).
|
||||
- [ ] 1.2 Update `src/error-codes/error-codes.md` — describe `TEMPLATE_READ_ONLY` and the narrowed role of `WRITE_UNSUPPORTED`.
|
||||
- [ ] 1.3 Extend `src/error-codes/error-codes.test.ts` to cover the new code and assert `WRITE_UNSUPPORTED` is still present.
|
||||
|
||||
## 2. Adapter — `DriveStorageLikeAdapter`
|
||||
|
||||
- [ ] 2.1 Widen `set(key, value, signal?)` and `delete(key, signal?)` to accept and forward `AbortSignal` to `Drive.write({ signal })` / `Drive.delete(path, signal)`.
|
||||
- [ ] 2.2 Update `src/adapter/drive-storage-like.md` for the new signature and note the `TEMPLATE_READ_ONLY` rejection happens in the instrumented override, not here.
|
||||
- [ ] 2.3 Extend `src/adapter/drive-storage-like.test.ts` — `set` propagates `signal`, `delete` propagates `signal`, pre-aborted signal short-circuits.
|
||||
|
||||
## 3. Instrumented adapter — `InstrumentedDriveAdapter`
|
||||
|
||||
- [ ] 3.1 Replace the unconditional `set` rejection with per-scheme dispatch: `drive`/`drive+mesh` → delegate to `super.set(key, value, signal)`; `drive+template` → throw `DRIVE_URI.TEMPLATE_READ_ONLY`.
|
||||
- [ ] 3.2 Override `delete` symmetrically: `drive`/`drive+mesh` → delegate; `drive+template` → throw `TEMPLATE_READ_ONLY`.
|
||||
- [ ] 3.3 On every successful `drive+mesh://` write, emit span event `drive_uri.mesh_write_local_only` with `{ scheme, path }`.
|
||||
- [ ] 3.4 Pre-aborted `signal` on `set`/`delete` → throw `DRIVE_URI.ABORTED` before any work.
|
||||
- [ ] 3.5 OTel counters: increment `drive_uri.writes_total{scheme, outcome}` and `drive_uri.deletes_total{scheme, outcome}` for every success and every rejection.
|
||||
- [ ] 3.6 Update `src/plugins/drive/drive.streams.md` and `src/plugins/drive-template/drive-template.md`.
|
||||
|
||||
## 4. Test suite
|
||||
|
||||
- [ ] 4.1 `src/plugins/drive/drive.test.ts` — replace `"rejects writes with DRIVE_URI.WRITE_UNSUPPORTED"` with `"writes succeed and round-trip via Drive.write"` and `"writes propagate AbortSignal to Drive.write"`.
|
||||
- [ ] 4.2 `src/plugins/drive/drive.test.ts` — add `"removes delegate to Drive.delete and return its boolean"` and `"removes propagate AbortSignal"`.
|
||||
- [ ] 4.3 `src/plugins/drive-mesh/drive-mesh.test.ts` — add `"writes succeed without triggering mesh.advertise"` and `"writes emit drive_uri.mesh_write_local_only span event"`.
|
||||
- [ ] 4.4 `src/plugins/drive-template/drive-template.test.ts` — add `"writes reject with DRIVE_URI.TEMPLATE_READ_ONLY"` and `"removes reject with DRIVE_URI.TEMPLATE_READ_ONLY"`.
|
||||
- [ ] 4.5 `src/otel/otel.test.ts` — `"increments writes_total{outcome:success} on successful drive:// writes"`, `"increments writes_total{outcome:error,code:TEMPLATE_READ_ONLY} on template writes"`, `"increments deletes_total per scheme outcome"`.
|
||||
- [ ] 4.6 `src/conformance/conformance.test.ts` — re-run `runUriPluginConformance` writes suite against all three plugins. Expected: `drive`/`drive+mesh` pass; `drive+template` continues to fail writes with the new code.
|
||||
- [ ] 4.7 `tests/integration/drive-roundtrip-write.test.ts` — real local-storage Drive, `resolver.write` then `resolver.read`, assert equality.
|
||||
- [ ] 4.8 `tests/integration/drive-mesh-roundtrip-write.test.ts` — mesh-configured Drive with a spy transport; assert no implicit `advertise` call; explicit `mesh.advertise(...)` after the write produces a verifiable outbound advertisement.
|
||||
- [ ] 4.9 `tests/integration/template-write-rejection.test.ts` — wired template-backed Drive, `resolver.write("drive+template://...")` and `resolver.remove(...)` both reject with `TEMPLATE_READ_ONLY`.
|
||||
|
||||
## 5. Migration audit
|
||||
|
||||
- [ ] 5.1 `grep -rln "DRIVE_URI.WRITE_UNSUPPORTED" packages/ | grep -v "/dist/\|/coverage/\|/drive-uri/"` — every hit is reviewed; non-matching consumers re-routed onto `TEMPLATE_READ_ONLY` where appropriate.
|
||||
- [ ] 5.2 Update `README.md` Features section to reflect writes/removes via `drive://` and `drive+mesh://`.
|
||||
|
||||
## 6. Pre-publish gate (§13)
|
||||
|
||||
- [ ] 6.1 Spec reconciliation — walk every `### Requirement` and `#### Scenario` in `specs/drive-uri/spec.md`; confirm the named test exists verbatim.
|
||||
- [ ] 6.2 `bunx turbo lint typecheck test --filter=@suaveplan/drive-uri` — green.
|
||||
- [ ] 6.3 `bunx turbo test:coverage --filter=@suaveplan/drive-uri` — 100/100/100/100, including the `TEMPLATE_READ_ONLY` rejection branches.
|
||||
- [ ] 6.4 `bunx turbo build --filter=@suaveplan/drive-uri` — green.
|
||||
- [ ] 6.5 Banned-pattern, barrel, test-runner-import, missing-`.md`, bare-version, and secrets scans — all clean.
|
||||
- [ ] 6.6 Dependent sweep — `bunx turbo test --filter=...@suaveplan/drive-uri` green.
|
||||
|
||||
## 7. Root-doc refresh (§13b)
|
||||
|
||||
- [ ] 7.1 `bun scripts/folder-reorg/generate-manifest.ts` — refresh `MANIFEST.md`.
|
||||
- [ ] 7.2 `bun scripts/generate-missing.ts` — refresh `MISSING.yml`.
|
||||
- [ ] 7.3 Update `STATUS.md` and `SUMMARY.md` paragraphs for `drive-uri` to reflect writes-supported on the two mutable schemes.
|
||||
- [ ] 7.4 `bunx markdownlint-cli2 --fix` over the touched docs.
|
||||
|
||||
## 8. Archive
|
||||
|
||||
- [ ] 8.1 Fold `specs/drive-uri/spec.md` (deltas) into `openspec/specs/drive-uri/spec.md`.
|
||||
- [ ] 8.2 Move `openspec/changes/add-drive-uri-writes/` → `openspec/archive/<yyyy-mm-dd>-add-drive-uri-writes/`.
|
||||
@@ -0,0 +1,253 @@
|
||||
# Design: add-installer-upgrade
|
||||
|
||||
This document captures the non-normative design for the deferred upgrade flow.
|
||||
Normative requirements live in `specs/installer/spec.md`.
|
||||
|
||||
## Flow diagram
|
||||
|
||||
```text
|
||||
suaveplan-install upgrade <plugin> [--from X] [--to Y] [--dry-run] [--force]
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────────────────┐
|
||||
│ Host.upgrade(plugin, options) │
|
||||
└────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
├─► loadState(fs, cwd)
|
||||
│ state-file: .suaveplan/installer-state.json
|
||||
│ missing entry → INSTALLER.UPGRADE_NOT_INSTALLED
|
||||
│
|
||||
├─► resolveTargetVersion(plugin, cwd)
|
||||
│ reads node_modules/@<scope>/<pkg>/package.json
|
||||
│ missing → INSTALLER.UPGRADE_VERSION_NOT_FOUND
|
||||
│
|
||||
├─► if installed === target && !force
|
||||
│ → INSTALLER.UPGRADE_SAME_VERSION (idempotent no-op)
|
||||
│
|
||||
├─► preflight
|
||||
│ ├─ detectLockfile() → UPGRADE_LOCKFILE_MISSING
|
||||
│ └─ detectDirtyTree() → UPGRADE_DIRTY_TREE (unless --allow-dirty)
|
||||
│
|
||||
├─► hashTree(targetDir) → currentHashes (rollback snapshot key)
|
||||
│
|
||||
├─► plan(answers, ctx) at fromVersion → fromOps
|
||||
│ plan(answers, ctx) at toVersion → toOps
|
||||
│
|
||||
├─► computeDiff(fromOps, toOps)
|
||||
│ { added, removed, changed, unchanged }
|
||||
│
|
||||
├─► materialiseDiff(diff) → FileOp[]
|
||||
│ additions → write
|
||||
│ removals → delete
|
||||
│ changes → write (overwrite) | patch-json (merge)
|
||||
│
|
||||
├─► Host.applyOps(ops) ◄── EXISTING TRANSACTIONAL PATH
|
||||
│ staged-tmp + atomic rename (INSTALLER-04)
|
||||
│
|
||||
├─► runMigrations(selected, ctx)
|
||||
│ throws → rollback transaction → UPGRADE_MIGRATION_FAILED
|
||||
│
|
||||
├─► writeState({ pluginId, version: to, installedAt, fileHashes })
|
||||
│ atomic write → fsync → rename
|
||||
│
|
||||
└─► OTel: span "installer.upgrade" closes with attributes
|
||||
plugin.id, plugin.version.from, plugin.version.to, dry_run,
|
||||
files.added, files.removed, files.changed
|
||||
```
|
||||
|
||||
## Atomicity model
|
||||
|
||||
The upgrade is **one** transaction, not two. The diff-apply step and the
|
||||
migration-run step share the same staged-tmp directory created by
|
||||
`Host.applyOps`. The window is:
|
||||
|
||||
1. **Stage** — every `write`/`patch-*` op writes its content to
|
||||
`<targetDir>/.tmp-<txn>/<path>`. Deletions are recorded as
|
||||
"move-aside to `<targetDir>/.tmp-<txn>/.deleted/<path>`" — files are
|
||||
**moved**, not removed, so rollback can restore them. No live files have
|
||||
been mutated yet.
|
||||
2. **Migrate** — `MigrationScript`s run against the staged tree (their
|
||||
`MigrationContext.fs` is rooted at the tmp directory). They can mutate
|
||||
staged files or emit additional `FileOp`s, which are appended to the
|
||||
plan and re-staged into the same tmp directory.
|
||||
3. **Commit** — once every stage step has succeeded, the host issues atomic
|
||||
`rename(tmp, live)` for every staged path **in dependency order** (parent
|
||||
dirs before children for `write`, children before parents for `delete`).
|
||||
4. **Persist state** — only after every rename commits does the host write
|
||||
the new `installer-state.json` (itself via the same write-fsync-rename
|
||||
dance scoped to one file).
|
||||
|
||||
Any failure in steps 1–3 walks the journal in reverse: staged tmp files are
|
||||
unlinked; moved-aside files are renamed back; the state-file is untouched.
|
||||
The user's tree is bit-identical to the pre-upgrade state.
|
||||
|
||||
A rollback failure (e.g. fs returns EIO on the restore rename) surfaces as
|
||||
`InstallerError(INSTALLER.ROLLBACK_FAILED, { cause: <original apply error> })`
|
||||
exactly as `INSTALLER-04` already prescribes for the `add` flow.
|
||||
|
||||
## Snapshot / rollback strategy
|
||||
|
||||
The "snapshot" is not a copy of the tree — that would be linear in target
|
||||
size and unacceptable for a skills bundle. Instead, the snapshot is:
|
||||
|
||||
1. **The pre-upgrade `hashTree`** — recorded before staging so the upgrade
|
||||
has a content-addressed manifest of what existed.
|
||||
2. **The move-aside journal** — every file the diff intends to delete or
|
||||
overwrite is moved to `<tmp>/.deleted/<path>` (rename is O(1) on the
|
||||
same filesystem). Disk usage during upgrade equals
|
||||
`size(diff.changed ∪ diff.removed)` rather than `size(entire tree)`.
|
||||
3. **The new staged content** — every `write` lands in `<tmp>/<path>` and
|
||||
is `rename`d in only on commit.
|
||||
|
||||
Rollback walks the journal in reverse: rename moved-aside files back to
|
||||
their live paths, unlink staged files, rmdir the tmp directory.
|
||||
|
||||
## Migration script contract
|
||||
|
||||
```ts
|
||||
// In a plugin: src/upgrade.ts
|
||||
import type { MigrationTable, MigrationScript } from "@suaveplan/installer/migrations";
|
||||
|
||||
const v1_to_v2: MigrationScript = async (ctx) => {
|
||||
// ctx.fs is rooted at the staged tmp directory.
|
||||
// ctx.signal honours --abort.
|
||||
// ctx.logger is the host's structured logger.
|
||||
// ctx.from = "1.0.0", ctx.to = "2.0.0"
|
||||
const old = await ctx.fs.readFile("skills/legacy.json");
|
||||
await ctx.fs.writeFile("skills/legacy.yaml", convert(old));
|
||||
await ctx.fs.delete("skills/legacy.json");
|
||||
};
|
||||
|
||||
export const upgradeTable: MigrationTable = {
|
||||
"1.0.0->2.0.0": v1_to_v2,
|
||||
};
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Keys are `"<from>-><to>"` semver strings. The selector finds the shortest
|
||||
path of contiguous edges from the installed version to the target.
|
||||
- Scripts MUST be pure with respect to outside-the-tmp state. Network
|
||||
access is permitted (e.g. fetching upstream renames) but discouraged.
|
||||
- Scripts MAY throw. A throw rolls the transaction back and surfaces
|
||||
`INSTALLER.UPGRADE_MIGRATION_FAILED` with `{ cause }`.
|
||||
- Scripts MAY return additional `FileOp[]` that the host appends to the
|
||||
plan and re-stages before commit.
|
||||
|
||||
## CLI behaviour
|
||||
|
||||
| Flag | Effect |
|
||||
|---|---|
|
||||
| `--dry-run` | Computes diff + migration list, prints the plan, exits before staging. |
|
||||
| `--force` | Collapses conflict policies to `overwrite` AND bypasses the dirty-tree guard. Equivalent to `--allow-dirty` + per-file overwrite. |
|
||||
| `--allow-dirty` | Bypasses dirty-tree guard but does NOT change conflict policies. Used when the operator knows their edits are reconcilable. |
|
||||
| `--no-interactive` | Any prompt op fails with `INSTALLER.APPLY_CONFLICT` (existing semantics). |
|
||||
| `--from <version>` | Overrides the version detected from the state file. Useful when the state file is missing or stale. |
|
||||
| `--to <version>` | Overrides the version resolved from `node_modules`. Useful for pin-rollforward simulations. |
|
||||
| `--answers <file>` | Same as `add` — JSON file supplying answers for any prompts the plan emits. |
|
||||
|
||||
## Idempotency
|
||||
|
||||
Two notions, both held:
|
||||
|
||||
1. **Same-version upgrade is a no-op.** `installed == target` exits with
|
||||
`INSTALLER.UPGRADE_SAME_VERSION` without staging anything.
|
||||
2. **Successful upgrade is replayable.** If the user runs the same upgrade
|
||||
again, the state file already reflects `target`, so step 1 fires. There
|
||||
is no way to "double-apply" a migration script.
|
||||
|
||||
## State file shape
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "installer-state/v1",
|
||||
"plugins": {
|
||||
"agent-skills": {
|
||||
"version": "0.7.0",
|
||||
"installedAt": "2026-05-17T12:34:56Z",
|
||||
"targetDir": ".suaveplan/skills",
|
||||
"fileHashes": {
|
||||
"skills/crypto/SKILL.md": "blake3:...",
|
||||
"skills/events/SKILL.md": "blake3:..."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The schema version (`$schema`) is bumped any time the shape changes; the
|
||||
loader rejects unknown versions with `INSTALLER.UPGRADE_STATE_INVALID`.
|
||||
|
||||
## Coverage strategy
|
||||
|
||||
Two fixture plugins live under `src/testing/fixtures/`:
|
||||
|
||||
- `upgrade-fixture-plugin@1.0.0` — emits two files
|
||||
(`a.txt`, `b.txt`) and one `patch-json` op against `data.json`.
|
||||
- `upgrade-fixture-plugin@2.0.0` — drops `b.txt`, renames `a.txt` →
|
||||
`a-renamed.txt`, adds `c.txt`, changes the `patch-json` merge, and ships
|
||||
one migration script `1.0.0->2.0.0` that rewrites `data.json` schema.
|
||||
|
||||
Together they exercise every branch in `computeDiff`, every conflict policy
|
||||
in the diff materialiser, both the migration-success and migration-throw
|
||||
paths, and both the same-version-noop and version-not-found paths. 100%
|
||||
istanbul falls out naturally.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/tooling/installer/src/
|
||||
state/
|
||||
state.types.ts
|
||||
state.schemas.ts
|
||||
state-file.ts
|
||||
state-file.md
|
||||
state-file.spec-conformance.test.ts
|
||||
hash-tree.ts
|
||||
hash-tree.md
|
||||
host/
|
||||
upgrade/
|
||||
upgrade.ts
|
||||
upgrade.md
|
||||
upgrade.spec-conformance.test.ts
|
||||
preflight.ts
|
||||
preflight.md
|
||||
preflight.spec-conformance.test.ts
|
||||
dirty-tree.spec-conformance.test.ts
|
||||
diff.ts
|
||||
diff.md
|
||||
apply-diff.spec-conformance.test.ts
|
||||
migrations.ts
|
||||
migrations.types.ts
|
||||
migrations.md
|
||||
migrations.spec-conformance.test.ts
|
||||
index.ts
|
||||
testing/
|
||||
upgrade-plugin.ts
|
||||
upgrade-plugin.md
|
||||
fixtures/
|
||||
upgrade-fixture-plugin.ts
|
||||
upgrade-fixture-plugin.md
|
||||
```
|
||||
|
||||
## Decisions and trade-offs
|
||||
|
||||
- **Move-aside vs full snapshot.** Move-aside wastes one rename per
|
||||
affected file but is O(1) per file. A full snapshot would be safer
|
||||
against concurrent writes but the installer already requires the user
|
||||
to not be editing files concurrently — `INSTALLER-04` makes the same
|
||||
trade-off for `add`. Consistency with `add` wins.
|
||||
- **BLAKE3 over SHA-256.** `@suaveplan/crypto-hash` ships both. BLAKE3 is
|
||||
~3× faster for the small-file case (skill files average <4 KB) and
|
||||
parallelisable. No security requirement (this is a change-detection
|
||||
hash, not a tamper-proof signature).
|
||||
- **Migration scripts run AFTER diff stage, not before.** Running them
|
||||
before would mean diffing against a tree the migration script has
|
||||
already mutated, which makes the "did the user modify this file?"
|
||||
question ambiguous. The order is: diff first (against the unmodified
|
||||
staged tree), then migrate.
|
||||
- **No network resolution.** Resolving "latest" from npm would couple the
|
||||
installer to a registry shape, force network access in CI, and conflict
|
||||
with the project's air-gapped use cases. The target version is
|
||||
whatever resolves from `node_modules`; the operator bumps the dep
|
||||
with their package manager first.
|
||||
@@ -0,0 +1,145 @@
|
||||
# OpenSpec Change: add-installer-upgrade
|
||||
|
||||
- **Change ID**: add-installer-upgrade
|
||||
- **Package**: `@suaveplan/installer`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-17
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/installer/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
`openspec/archive/2026-04-28-installer-port/proposal.md` §"Out of scope" deferred
|
||||
the upgrade flow:
|
||||
|
||||
> Plugin upgrade flow (`upgrade <plugin>` to bring an existing install up to
|
||||
> current bundle version) is a follow-up — initial scope is install + reinstall
|
||||
> (idempotent re-run with conflict policy).
|
||||
|
||||
The installer therefore ships `add <plugin>` and `list` but has no answer for a
|
||||
user who installed `@suaveplan/install-agent-skills@0.5.0` six months ago and
|
||||
wants the changes that landed in `0.7.0`. Today their only options are
|
||||
`add agent-skills --force` (which overwrites local edits without diffing) or a
|
||||
manual `rm -rf` (which destroys their in-flight skill edits). Both are unsafe.
|
||||
|
||||
The `add` path is also unable to run plugin-authored migration scripts, so any
|
||||
breaking layout change in a plugin bundle (e.g. `skills/<id>/SKILL.md` →
|
||||
`skills/<id>/<id>.md`) silently leaves stale files behind that the user has to
|
||||
hand-prune.
|
||||
|
||||
## What Changes
|
||||
|
||||
Add a third top-level installer command, `suaveplan-install upgrade <plugin>`,
|
||||
that resolves the **installed** plugin version from the on-disk state file,
|
||||
resolves the **target** version from `node_modules`, computes the file-level
|
||||
diff between the two bundles (added / removed / changed), and applies it
|
||||
transactionally with the same staged-tmp + rollback strategy `apply` already
|
||||
uses for `add`. Pre-flight gates protect against in-flight edits (dirty target
|
||||
tree) and missing lockfiles. Plugins MAY ship an optional `upgrade.ts`
|
||||
migration script per from-version → to-version pair; the host invokes them in
|
||||
order inside the same transaction.
|
||||
|
||||
### Surface added
|
||||
|
||||
- `suaveplan-install upgrade <plugin>` CLI verb with flags
|
||||
`--dry-run`, `--force`, `--no-interactive`, `--answers <file>`,
|
||||
`--from <version>` (override detected), `--to <version>` (target; defaults
|
||||
to whatever resolves from `node_modules`), `--allow-dirty` (skip dirty-tree
|
||||
guard — destructive, opt-in only).
|
||||
- `Host.upgrade(plugin, options)` public entry point alongside `Host.run`.
|
||||
- A persisted state file (`.suaveplan/installer-state.json`) capturing
|
||||
`{ pluginId, version, installedAt, fileHashes }` per installed plugin so
|
||||
diff and dirty-tree detection are deterministic on subsequent upgrades.
|
||||
- A `MigrationContext` and `MigrationScript` contract under
|
||||
`@suaveplan/installer/migrations` for plugin-authored scripts.
|
||||
- 7 new `INSTALLER.*` codes: `UPGRADE_NOT_INSTALLED`,
|
||||
`UPGRADE_VERSION_NOT_FOUND`, `UPGRADE_SAME_VERSION`, `UPGRADE_DIRTY_TREE`,
|
||||
`UPGRADE_STATE_INVALID`, `UPGRADE_MIGRATION_FAILED`,
|
||||
`UPGRADE_LOCKFILE_MISSING`.
|
||||
|
||||
### Behaviour guarantees
|
||||
|
||||
- Re-running `upgrade <plugin>` when installed == target is a no-op exiting
|
||||
with `INSTALLER.UPGRADE_SAME_VERSION` (treated as success under `--dry-run`).
|
||||
- Failure at any point (diff compute, migration script throw, atomic rename
|
||||
error) rolls the target tree back to the pre-upgrade snapshot. Partial
|
||||
state is never persisted.
|
||||
- The state file is updated atomically (`write → fsync → rename`) only after
|
||||
the transaction commits.
|
||||
|
||||
## Source mapping
|
||||
|
||||
| Source / archive reference | What it tells us |
|
||||
|---|---|
|
||||
| `openspec/archive/2026-04-28-installer-port/proposal.md` §"Out of scope" bullet 2 | The deferred upgrade flow is the scope of this change. |
|
||||
| `openspec/archive/2026-04-28-installer-port/proposal.md` §"Architecture → Lifecycle" | Reuse the existing `detect → prompts → plan → apply → postInstall` pipeline; upgrade adds a diff-and-migrate stage between `plan` and `apply`. |
|
||||
| `openspec/specs/installer/spec.md` `INSTALLER-04` (transactional apply) | Upgrade MUST reuse the same staged-tmp + rollback machinery — no parallel transaction model. |
|
||||
| `openspec/specs/installer/spec.md` `INSTALLER-05` (conflict policy) | Per-file conflict policy already exists; upgrade extends it with a "user-modified-since-install" case detected via the state-file hashes. |
|
||||
| `openspec/specs/installer/spec.md` `INSTALLER-09` (frozen registry) | New codes MUST extend `defineCodes("INSTALLER", [...])` — no parallel registry. |
|
||||
| `packages/tooling/installer/src/bin/suaveplan-install.ts` | Existing CLI shape: `cli().command("add <plugin>")…`. The new verb attaches identically: `.command("upgrade <plugin>")`. |
|
||||
| `packages/tooling/installer/src/host/host.ts` | `Host` class is the orchestration seam; `Host.upgrade` slots in alongside `Host.run`. |
|
||||
| `packages/tooling/installer/src/host/apply/apply.ts` | Staged-tmp transaction implementation is reused as-is; upgrade composes it. |
|
||||
| `packages/tooling/installer/src/manifest/discover.ts` | Version resolution from `node_modules` already exists for `add`; upgrade reuses `findPlugin(cwd, pluginId)`. |
|
||||
| `packages/tooling/installer/src/error-codes.ts` | New codes APPEND to the existing tuple; the registry stays frozen. |
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/installer` to enumerate. Current
|
||||
dependents:
|
||||
|
||||
- `@suaveplan/install-agent-skills` — [x] tests run / [x] docs reviewed /
|
||||
[ ] no action needed. The skills plugin is the canonical upgrade target;
|
||||
its existing `runPlugin` harness gains an `upgradePlugin` sibling so its
|
||||
conformance tests cover both flows. The plugin itself receives a tiny
|
||||
optional `upgrade.ts` exporting an empty migration table (no breaking
|
||||
layout changes yet); this proves the migration contract loads even when
|
||||
the table is empty.
|
||||
|
||||
No other dependents currently consume `@suaveplan/installer`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Downgrade.** `upgrade` only moves forward. A separate
|
||||
`add-installer-downgrade` change can extend the diff engine in reverse;
|
||||
the migration contract is intentionally directional (`from`/`to` versions)
|
||||
so downgrade is a different scope.
|
||||
- **Cross-plugin coordination.** Upgrading `install-agent-skills` does not
|
||||
trigger an upgrade of a hypothetical `install-web-server`. Each plugin is
|
||||
upgraded independently; `peerInstallers` field warnings are a future
|
||||
packaging change.
|
||||
- **Background / scheduled upgrade daemon.** Upgrade is operator-initiated
|
||||
via the CLI; there is no auto-update behaviour.
|
||||
- **Network resolution.** The "target version" is whatever resolves from
|
||||
`node_modules`; the installer never talks to npm directly. Bumping the
|
||||
installed package version remains the user's package-manager step.
|
||||
|
||||
## Verification
|
||||
|
||||
Every requirement in `specs/installer/spec.md` carries a `**Verified by:**`
|
||||
line pointing to a conformance test in `src/host/upgrade/` or
|
||||
`src/state/`. The full set:
|
||||
|
||||
- `INSTALLER-UPGRADE-01` Lifecycle order → `src/host/upgrade/upgrade.spec-conformance.test.ts::"runs detect → resolve → diff → migrate → apply → postInstall in order"`
|
||||
- `INSTALLER-UPGRADE-02` Idempotent same-version no-op → `src/host/upgrade/upgrade.spec-conformance.test.ts::"exits with UPGRADE_SAME_VERSION when installed equals target"`
|
||||
- `INSTALLER-UPGRADE-03` Dirty-tree guard → `src/host/upgrade/dirty-tree.spec-conformance.test.ts::"throws UPGRADE_DIRTY_TREE when state-file hash diverges from disk"`
|
||||
- `INSTALLER-UPGRADE-04` Lockfile pre-flight → `src/host/upgrade/preflight.spec-conformance.test.ts::"throws UPGRADE_LOCKFILE_MISSING when no bun.lockb / package-lock.json / pnpm-lock.yaml exists"`
|
||||
- `INSTALLER-UPGRADE-05` Transactional diff application → `src/host/upgrade/apply-diff.spec-conformance.test.ts::"rolls back snapshot on staged-rename failure"`
|
||||
- `INSTALLER-UPGRADE-06` Migration script contract → `src/host/upgrade/migrations.spec-conformance.test.ts::"runs upgrade.ts scripts in semver order between from and to"`
|
||||
- `INSTALLER-UPGRADE-07` Migration failure rollback → `src/host/upgrade/migrations.spec-conformance.test.ts::"rolls back transaction and surfaces UPGRADE_MIGRATION_FAILED on script throw"`
|
||||
- `INSTALLER-UPGRADE-08` State-file integrity → `src/state/state-file.spec-conformance.test.ts::"atomically writes installer-state.json only on successful commit"`
|
||||
- `INSTALLER-UPGRADE-09` Error-code registry namespacing → `src/error-codes.spec-conformance.test.ts::"upgrade codes are frozen and namespaced INSTALLER.*"`
|
||||
- `INSTALLER-UPGRADE-10` Version-not-found surface → `src/host/upgrade/upgrade.spec-conformance.test.ts::"throws UPGRADE_VERSION_NOT_FOUND when plugin not in node_modules"`
|
||||
- `INSTALLER-UPGRADE-11` Not-installed surface → `src/host/upgrade/upgrade.spec-conformance.test.ts::"throws UPGRADE_NOT_INSTALLED when no state-file entry exists"`
|
||||
- `INSTALLER-UPGRADE-12` CLI flag wiring → `src/bin/suaveplan-install.spec-conformance.test.ts::"upgrade verb forwards --dry-run, --force, --allow-dirty, --from, --to to Host.upgrade"`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `bunx turbo lint typecheck test --filter=@suaveplan/installer` green.
|
||||
- `bunx turbo test:coverage --filter=@suaveplan/installer` 100% istanbul on
|
||||
every new module under `src/host/upgrade/`, `src/state/`, and the
|
||||
upgrade-specific additions to `src/bin/suaveplan-install.ts`.
|
||||
- Sibling `.md` for every new implementation `.ts` per `openspec/AGENTS.md` §4.
|
||||
- `openspec validate add-installer-upgrade --strict` green.
|
||||
- Spec delta folded into `openspec/specs/installer/spec.md` on archive.
|
||||
@@ -0,0 +1,278 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: installer (upgrade flow)
|
||||
|
||||
**Package**: `@suaveplan/installer`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
> Test files are planned paths inside `packages/tooling/installer/`.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Upgrade lifecycle order
|
||||
|
||||
`Host.upgrade(plugin, options)` SHALL execute upgrade stages in the strict
|
||||
order `loadState → resolveTarget → preflight → hashTree → planFrom → planTo →
|
||||
diff → materialise → applyOps → runMigrations → commitState`. Optional stages
|
||||
(`detect`, `prompts`, `postInstall` from the install lifecycle) SHALL still be
|
||||
honoured before `loadState` and after `commitState` respectively, preserving
|
||||
plugin composability. Each stage SHALL be wrapped in an OpenTelemetry span
|
||||
named `installer.upgrade.<stage>` whose parent span `installer.upgrade`
|
||||
carries attributes `plugin.id`, `plugin.version.from`, `plugin.version.to`,
|
||||
and `dry_run`.
|
||||
|
||||
#### Scenario: pipeline executes in the documented order
|
||||
|
||||
- **WHEN** `Host.upgrade(fixturePlugin, { from: "1.0.0", to: "2.0.0" })` runs
|
||||
against an installed `1.0.0` fixture and a `2.0.0` bundle in `node_modules`
|
||||
- **THEN** the OTel exporter records spans in the order `loadState`,
|
||||
`resolveTarget`, `preflight`, `hashTree`, `planFrom`, `planTo`, `diff`,
|
||||
`materialise`, `applyOps`, `runMigrations`, `commitState`, all parented to
|
||||
one `installer.upgrade` span carrying `plugin.id`, `plugin.version.from`,
|
||||
`plugin.version.to`, and `dry_run`.
|
||||
|
||||
**Verified by:** `src/host/upgrade/upgrade.spec-conformance.test.ts::"runs detect → resolve → diff → migrate → apply → postInstall in order"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Same-version upgrade is idempotent
|
||||
|
||||
`Host.upgrade` SHALL short-circuit when the installed version recorded in
|
||||
the state file equals the target version resolved from `node_modules`, and
|
||||
surface `InstallerError(INSTALLER.UPGRADE_SAME_VERSION)`. Callers MAY treat
|
||||
this code as a success indicator. Re-running the same upgrade after a
|
||||
successful commit SHALL therefore always exit via this short-circuit. The
|
||||
short-circuit SHALL NOT fire when `--force` was supplied.
|
||||
|
||||
#### Scenario: re-running upgrade after success exits with UPGRADE_SAME_VERSION
|
||||
|
||||
- **WHEN** `Host.upgrade` is invoked twice in succession against the same
|
||||
fixture plugin with no `--force` flag
|
||||
- **THEN** the first invocation MUST commit and write the state file, and
|
||||
the second invocation MUST throw `InstallerError` with code
|
||||
`INSTALLER.UPGRADE_SAME_VERSION` and MUST NOT stage any file
|
||||
|
||||
**Verified by:** `src/host/upgrade/upgrade.spec-conformance.test.ts::"exits with UPGRADE_SAME_VERSION when installed equals target"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Dirty-tree pre-flight guard
|
||||
|
||||
Before staging any file op, `Host.upgrade` SHALL hash every path recorded in
|
||||
the state file's `fileHashes` and compare to the recorded value. Any
|
||||
mismatched hash SHALL surface as
|
||||
`InstallerError(INSTALLER.UPGRADE_DIRTY_TREE, { context: { paths: string[] } })`
|
||||
unless `--allow-dirty` or `--force` was supplied. The error context SHALL
|
||||
enumerate every divergent path so the operator can reconcile manually.
|
||||
|
||||
#### Scenario: divergent on-disk hash blocks the upgrade
|
||||
|
||||
- **GIVEN** a state file recording `skills/crypto/SKILL.md` at hash `H1`
|
||||
- **WHEN** the file on disk hashes to `H2` and `Host.upgrade` runs without
|
||||
`--allow-dirty` or `--force`
|
||||
- **THEN** an `InstallerError` with code `INSTALLER.UPGRADE_DIRTY_TREE` is
|
||||
thrown whose `context.paths` includes `skills/crypto/SKILL.md`
|
||||
|
||||
**Verified by:** `src/host/upgrade/dirty-tree.spec-conformance.test.ts::"throws UPGRADE_DIRTY_TREE when state-file hash diverges from disk"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Lockfile presence is required
|
||||
|
||||
`Host.upgrade` SHALL refuse to run unless at least one of
|
||||
`bun.lockb`, `bun.lock`, `package-lock.json`, `pnpm-lock.yaml`, or
|
||||
`yarn.lock` is present in `cwd`. Missing lockfile SHALL surface as
|
||||
`InstallerError(INSTALLER.UPGRADE_LOCKFILE_MISSING)`. This gate exists to
|
||||
prevent upgrades against a tree whose dependency graph is unpinned, where
|
||||
the resolved target version could shift between runs.
|
||||
|
||||
#### Scenario: missing lockfile blocks the upgrade
|
||||
|
||||
- **WHEN** `Host.upgrade` runs in a `cwd` with no lockfile of any supported
|
||||
format
|
||||
- **THEN** an `InstallerError` with code `INSTALLER.UPGRADE_LOCKFILE_MISSING`
|
||||
is thrown before any file is hashed
|
||||
|
||||
**Verified by:** `src/host/upgrade/preflight.spec-conformance.test.ts::"throws UPGRADE_LOCKFILE_MISSING when no bun.lockb / package-lock.json / pnpm-lock.yaml exists"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Transactional diff application with rollback
|
||||
|
||||
`Host.upgrade` SHALL materialise the computed diff (`added`, `removed`,
|
||||
`changed`) into the existing `FileOp` union and feed it through
|
||||
`Host.applyOps`, reusing the staged-tmp + atomic-rename transaction
|
||||
described in the canonical `INSTALLER-04` requirement. Removed files SHALL
|
||||
be moved aside into `<tmp>/.deleted/` rather than unlinked, so rollback can
|
||||
restore them. Any failure during staging or commit SHALL trigger a
|
||||
reverse-order rollback that unlinks staged files and renames the
|
||||
moved-aside files back into place. Rollback failures SHALL surface as
|
||||
`InstallerError(INSTALLER.ROLLBACK_FAILED, { cause })`.
|
||||
|
||||
#### Scenario: rename failure rolls the tree back
|
||||
|
||||
- **GIVEN** a fixture plugin whose final `rename` of `a-renamed.txt` is
|
||||
injected to throw EIO
|
||||
- **WHEN** `Host.upgrade` runs against an installed `1.0.0` tree containing
|
||||
`a.txt`
|
||||
- **THEN** `a.txt` exists at its original content after the throw and
|
||||
`a-renamed.txt` does not exist, and the throw surfaces as `InstallerError`
|
||||
with code `INSTALLER.APPLY_FAILED` (or `INSTALLER.ROLLBACK_FAILED` if the
|
||||
restore also fails) carrying the EIO as `cause`
|
||||
|
||||
**Verified by:** `src/host/upgrade/apply-diff.spec-conformance.test.ts::"rolls back snapshot on staged-rename failure"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Migration scripts run in semver order
|
||||
|
||||
`Host.upgrade` SHALL, when a plugin ships an `upgrade.ts` exporting a
|
||||
`MigrationTable`, load the table, select the shortest contiguous chain
|
||||
of edges from `from` to `to`, and execute each `MigrationScript` in order
|
||||
inside the same transaction as the diff apply. Each script SHALL receive a
|
||||
`MigrationContext` carrying the same `fs` (rooted at the staged tmp dir),
|
||||
the same `AbortSignal`, the host `logger`, and the `from`/`to` semver
|
||||
strings for the current edge.
|
||||
|
||||
#### Scenario: chained migrations execute from-to-target in order
|
||||
|
||||
- **GIVEN** a plugin with edges `1.0.0->1.1.0` and `1.1.0->2.0.0`
|
||||
- **WHEN** `Host.upgrade` runs from `1.0.0` to `2.0.0`
|
||||
- **THEN** both scripts MUST run, the `1.0.0->1.1.0` script MUST run
|
||||
first, both MUST receive an `fs` rooted at the same staged tmp
|
||||
directory, and both MUST observe the same `AbortSignal` instance
|
||||
|
||||
**Verified by:** `src/host/upgrade/migrations.spec-conformance.test.ts::"runs upgrade.ts scripts in semver order between from and to"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Migration failure rolls back the transaction
|
||||
|
||||
A `MigrationScript` throw SHALL roll back every staged op (including any
|
||||
ops emitted by earlier migration scripts in the same chain) and SHALL
|
||||
surface as `InstallerError(INSTALLER.UPGRADE_MIGRATION_FAILED, { cause })`.
|
||||
The state file SHALL NOT be written when a migration throws. Re-running
|
||||
the upgrade after a migration failure SHALL therefore see the original
|
||||
installed version in the state file and start the chain from the
|
||||
beginning.
|
||||
|
||||
#### Scenario: throwing migration leaves tree at pre-upgrade state
|
||||
|
||||
- **GIVEN** a plugin whose `1.0.0->2.0.0` migration throws on invocation
|
||||
- **WHEN** `Host.upgrade` runs from `1.0.0` to `2.0.0`
|
||||
- **THEN** the throw surfaces as `InstallerError` with code
|
||||
`INSTALLER.UPGRADE_MIGRATION_FAILED`, the live tree is bit-identical to
|
||||
its pre-upgrade content, and the state file still records version
|
||||
`1.0.0`
|
||||
|
||||
**Verified by:** `src/host/upgrade/migrations.spec-conformance.test.ts::"rolls back transaction and surfaces UPGRADE_MIGRATION_FAILED on script throw"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: State file is written atomically on commit only
|
||||
|
||||
`Host.upgrade` SHALL persist the new `installer-state.json` entry only
|
||||
after every diff op and migration script has committed successfully. The
|
||||
write SHALL use the write-fsync-rename pattern: serialise to
|
||||
`<cwd>/.suaveplan/installer-state.json.tmp-<txn>`, `fsync` the descriptor,
|
||||
then `rename` over the live path. Reads SHALL Zod-validate the loaded
|
||||
state; validation failures SHALL surface as
|
||||
`InstallerError(INSTALLER.UPGRADE_STATE_INVALID, { cause })`.
|
||||
|
||||
#### Scenario: state file appears only after successful commit
|
||||
|
||||
- **WHEN** `Host.upgrade` aborts mid-apply (any failure)
|
||||
- **THEN** the file at `.suaveplan/installer-state.json` is unchanged from
|
||||
its pre-upgrade content (either still recording the old version or
|
||||
still absent), and no `.tmp-*` sibling remains on disk
|
||||
|
||||
**Verified by:** `src/state/state-file.spec-conformance.test.ts::"atomically writes installer-state.json only on successful commit"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Upgrade error codes are namespaced and frozen
|
||||
|
||||
The `INSTALLER_CODES` registry SHALL include `UPGRADE_NOT_INSTALLED`,
|
||||
`UPGRADE_VERSION_NOT_FOUND`, `UPGRADE_SAME_VERSION`, `UPGRADE_DIRTY_TREE`,
|
||||
`UPGRADE_STATE_INVALID`, `UPGRADE_MIGRATION_FAILED`, and
|
||||
`UPGRADE_LOCKFILE_MISSING` in addition to the existing nine codes. The
|
||||
registry SHALL remain frozen (`Object.isFrozen(INSTALLER_CODES) === true`).
|
||||
Every new value SHALL start with `INSTALLER.`. `InstallerError` SHALL
|
||||
remain the only error class thrown by the upgrade path.
|
||||
|
||||
#### Scenario: registry contains the upgrade codes and stays frozen
|
||||
|
||||
- **WHEN** the registry is inspected at module-load time
|
||||
- **THEN** `INSTALLER_CODES.UPGRADE_NOT_INSTALLED` equals
|
||||
`"INSTALLER.UPGRADE_NOT_INSTALLED"`, every other upgrade code is
|
||||
present with the same `INSTALLER.` prefix, and
|
||||
`Object.isFrozen(INSTALLER_CODES)` returns `true`
|
||||
|
||||
**Verified by:** `src/error-codes.spec-conformance.test.ts::"upgrade codes are frozen and namespaced INSTALLER.*"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Missing target version surfaces a typed error
|
||||
|
||||
`Host.upgrade` SHALL surface
|
||||
`InstallerError(INSTALLER.UPGRADE_VERSION_NOT_FOUND, { context: { pluginId, requested } })`
|
||||
when the requested target version (whether supplied via `--to` or resolved
|
||||
from `node_modules`) does not exist.
|
||||
The error SHALL be thrown before any pre-flight check runs so callers can
|
||||
distinguish "plugin not installed at target" from "plugin tree dirty".
|
||||
|
||||
#### Scenario: requesting an unresolvable target version fails early
|
||||
|
||||
- **GIVEN** an installed plugin at `1.0.0` and `node_modules` containing
|
||||
only `1.0.0`
|
||||
- **WHEN** `Host.upgrade(plugin, { to: "9.9.9" })` runs
|
||||
- **THEN** an `InstallerError` with code
|
||||
`INSTALLER.UPGRADE_VERSION_NOT_FOUND` MUST be thrown and the
|
||||
`context.requested` field MUST equal `"9.9.9"`
|
||||
|
||||
**Verified by:** `src/host/upgrade/upgrade.spec-conformance.test.ts::"throws UPGRADE_VERSION_NOT_FOUND when plugin not in node_modules"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Upgrading an uninstalled plugin surfaces a typed error
|
||||
|
||||
`Host.upgrade` SHALL surface
|
||||
`InstallerError(INSTALLER.UPGRADE_NOT_INSTALLED, { context: { pluginId } })`
|
||||
when no entry for the requested `pluginId` exists in the state file.
|
||||
The error SHALL be thrown before any version resolution runs so the
|
||||
operator gets the most specific guidance ("run `add` first") without
|
||||
network or filesystem cost.
|
||||
|
||||
#### Scenario: upgrade against empty state file fails fast
|
||||
|
||||
- **GIVEN** an empty `.suaveplan/installer-state.json` (no `plugins` entry
|
||||
for the requested id)
|
||||
- **WHEN** `Host.upgrade(plugin)` runs
|
||||
- **THEN** an `InstallerError` with code
|
||||
`INSTALLER.UPGRADE_NOT_INSTALLED` MUST be thrown before
|
||||
`resolveTargetVersion` is called
|
||||
|
||||
**Verified by:** `src/host/upgrade/upgrade.spec-conformance.test.ts::"throws UPGRADE_NOT_INSTALLED when no state-file entry exists"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CLI verb forwards every upgrade flag
|
||||
|
||||
`suaveplan-install upgrade <plugin>` SHALL parse and forward the flags
|
||||
`--dry-run`, `--force`, `--no-interactive`, `--answers <file>`,
|
||||
`--from <version>`, `--to <version>`, and `--allow-dirty` to
|
||||
`Host.upgrade`. The verb SHALL share the same `InstallerContext`
|
||||
construction path as `add`, so the underlying `fs`, `logger`, `cli`,
|
||||
`templates`, and `signal` are identical. `--dry-run` SHALL print the
|
||||
computed diff (counts of added/removed/changed plus the migration chain)
|
||||
and exit with code 0 before any staging occurs.
|
||||
|
||||
#### Scenario: each upgrade flag reaches Host.upgrade
|
||||
|
||||
- **WHEN** `main(["upgrade", "agent-skills", "--dry-run", "--force", "--allow-dirty", "--from", "1.0.0", "--to", "2.0.0", "--answers", "a.json"])`
|
||||
runs with a recording `Host.upgrade` stub
|
||||
- **THEN** the stub records one call whose options object contains
|
||||
`{ dryRun: true, force: true, allowDirty: true, from: "1.0.0", to: "2.0.0", answers: <parsed JSON> }`
|
||||
|
||||
**Verified by:** `src/bin/suaveplan-install.spec-conformance.test.ts::"upgrade verb forwards --dry-run, --force, --allow-dirty, --from, --to to Host.upgrade"`
|
||||
@@ -0,0 +1,134 @@
|
||||
# Tasks: add-installer-upgrade
|
||||
|
||||
## 1. Error codes
|
||||
|
||||
- [ ] 1.1 Append `UPGRADE_NOT_INSTALLED`, `UPGRADE_VERSION_NOT_FOUND`,
|
||||
`UPGRADE_SAME_VERSION`, `UPGRADE_DIRTY_TREE`, `UPGRADE_STATE_INVALID`,
|
||||
`UPGRADE_MIGRATION_FAILED`, `UPGRADE_LOCKFILE_MISSING` to the
|
||||
`defineCodes("INSTALLER", [...])` tuple in
|
||||
`packages/tooling/installer/src/error-codes.ts`.
|
||||
- [ ] 1.2 Extend `src/error-codes.spec-conformance.test.ts` with an
|
||||
`INSTALLER-UPGRADE-09` block asserting the codes are present, frozen,
|
||||
and prefixed `INSTALLER.`.
|
||||
|
||||
## 2. State file (`src/state/`)
|
||||
|
||||
- [ ] 2.1 `src/state/state.types.ts` — `InstallerState`, `InstalledPlugin`,
|
||||
`FileHash` types.
|
||||
- [ ] 2.2 `src/state/state.schemas.ts` — Zod schemas for boundary validation.
|
||||
- [ ] 2.3 `src/state/state-file.ts` — `loadState(fs, cwd)`,
|
||||
`writeState(fs, cwd, state)` (atomic write → fsync → rename),
|
||||
`updatePlugin(state, entry)`. Failure → `INSTALLER.UPGRADE_STATE_INVALID`.
|
||||
- [ ] 2.4 `src/state/hash-tree.ts` — `hashTree(fs, dir)` → `Record<path, hex>`
|
||||
using BLAKE3 via `@suaveplan/crypto-hash`.
|
||||
- [ ] 2.5 `src/state/state-file.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-08`.
|
||||
- [ ] 2.6 `.md` siblings for every implementation file.
|
||||
|
||||
## 3. Pre-flight checks (`src/host/upgrade/preflight.ts`)
|
||||
|
||||
- [ ] 3.1 `detectLockfile(fs, cwd)` returning `"bun" | "npm" | "pnpm" | "yarn"`
|
||||
or throwing `INSTALLER.UPGRADE_LOCKFILE_MISSING`.
|
||||
- [ ] 3.2 `detectDirtyTree(state, currentHashes)` returning the list of
|
||||
user-modified paths. Throws `INSTALLER.UPGRADE_DIRTY_TREE` unless
|
||||
`--allow-dirty` or `--force` was supplied.
|
||||
- [ ] 3.3 `src/host/upgrade/preflight.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-04`.
|
||||
- [ ] 3.4 `src/host/upgrade/dirty-tree.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-03`.
|
||||
|
||||
## 4. Diff engine (`src/host/upgrade/diff.ts`)
|
||||
|
||||
- [ ] 4.1 `computeDiff(fromOps, toOps)` returning
|
||||
`{ added: FileOp[], removed: FileOp[], changed: FileOp[], unchanged: FileOp[] }`.
|
||||
- [ ] 4.2 `materialiseDiff(diff, options)` translating the diff into the
|
||||
existing `FileOp` union: removals → `{ kind: "delete" }`, additions →
|
||||
`{ kind: "write" }`, changes → `{ kind: "write", conflict: "overwrite" }`
|
||||
(or `merge` for `patch-json` ops).
|
||||
- [ ] 4.3 Sibling `diff.md` covering algorithm and tie-breaking rules.
|
||||
|
||||
## 5. Migration runner (`src/host/upgrade/migrations.ts`)
|
||||
|
||||
- [ ] 5.1 `MigrationScript`, `MigrationContext`, `MigrationTable` types in
|
||||
`src/host/upgrade/migrations.types.ts`.
|
||||
- [ ] 5.2 `loadMigrationTable(plugin)` dynamic-imports the plugin's
|
||||
`./upgrade.js` if declared in the manifest; absent table → no-op.
|
||||
- [ ] 5.3 `selectMigrations(table, from, to)` returns the ordered semver path.
|
||||
- [ ] 5.4 `runMigrations(selected, ctx)` runs each script inside an
|
||||
`AbortSignal`-propagating sandbox. Throw → `INSTALLER.UPGRADE_MIGRATION_FAILED`.
|
||||
- [ ] 5.5 `src/host/upgrade/migrations.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-06`, `INSTALLER-UPGRADE-07`.
|
||||
|
||||
## 6. Upgrade orchestration (`src/host/upgrade/upgrade.ts`)
|
||||
|
||||
- [ ] 6.1 `Host.upgrade(plugin, options)` method:
|
||||
1. Load state-file entry; throw `UPGRADE_NOT_INSTALLED` if absent.
|
||||
2. Resolve target version; throw `UPGRADE_VERSION_NOT_FOUND` if
|
||||
`node_modules` resolution fails.
|
||||
3. Short-circuit with `UPGRADE_SAME_VERSION` when installed === target
|
||||
and not `--force`.
|
||||
4. Run preflight (lockfile + dirty-tree).
|
||||
5. Hash current tree; compute snapshot for rollback.
|
||||
6. Run `plugin.plan` for both versions (or load cached bundle manifests).
|
||||
7. `computeDiff` → `materialiseDiff` → `Host.applyOps` (existing
|
||||
transactional path).
|
||||
8. Run migrations between the same staged-tmp transaction window.
|
||||
9. On success, `writeState` atomically.
|
||||
- [ ] 6.2 OTel span `installer.upgrade` wrapping the whole flow with
|
||||
`plugin.id`, `plugin.version.from`, `plugin.version.to`, `dry_run`.
|
||||
- [ ] 6.3 `src/host/upgrade/upgrade.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-01`, `INSTALLER-UPGRADE-02`, `INSTALLER-UPGRADE-10`,
|
||||
`INSTALLER-UPGRADE-11`.
|
||||
- [ ] 6.4 `src/host/upgrade/apply-diff.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-05`.
|
||||
|
||||
## 7. CLI wiring (`src/bin/suaveplan-install.ts`)
|
||||
|
||||
- [ ] 7.1 Add `app.command("upgrade <plugin>")` with flags `--dry-run`,
|
||||
`--force`, `--no-interactive`, `--answers <file>`, `--from <version>`,
|
||||
`--to <version>`, `--allow-dirty`.
|
||||
- [ ] 7.2 `runUpgrade(pluginId, flags)` mirrors `runAdd` but calls
|
||||
`host.upgrade(...)`.
|
||||
- [ ] 7.3 `src/bin/suaveplan-install.spec-conformance.test.ts` —
|
||||
`INSTALLER-UPGRADE-12`.
|
||||
|
||||
## 8. Testing helpers (`src/testing/`)
|
||||
|
||||
- [ ] 8.1 `src/testing/upgrade-plugin.ts` — `upgradePlugin(plugin, { fromState, toBundle, answers? })`
|
||||
mirroring the existing `runPlugin` harness for upgrade tests.
|
||||
- [ ] 8.2 `src/testing/fixtures/upgrade-fixture-plugin.ts` — two-version
|
||||
fixture plugin (`1.0.0` + `2.0.0`) with one migration script, used by
|
||||
every upgrade conformance test for end-to-end coverage at 100%.
|
||||
|
||||
## 9. Docs
|
||||
|
||||
- [ ] 9.1 Sibling `.md` for every new implementation `.ts` per
|
||||
`openspec/AGENTS.md` §4 (Purpose, Features, Basic, Advanced, API, Notes).
|
||||
- [ ] 9.2 Update root `packages/tooling/installer/README.md` Quickstart to
|
||||
include the `upgrade` verb and link the new module docs.
|
||||
|
||||
## 10. Quality gates
|
||||
|
||||
- [ ] 10.1 `bunx turbo lint typecheck test --filter=@suaveplan/installer` green.
|
||||
- [ ] 10.2 `bunx turbo test:coverage --filter=@suaveplan/installer` 100%
|
||||
istanbul on every new module.
|
||||
- [ ] 10.3 No banned patterns (`openspec/AGENTS.md` §2) in new files.
|
||||
- [ ] 10.4 No `biome-ignore` in new files (`openspec/AGENTS.md` §2a).
|
||||
- [ ] 10.5 File-size check: every `src/host/upgrade/*.ts` ≤ 300 lines,
|
||||
every `*.test.ts` ≤ 500 lines.
|
||||
- [ ] 10.6 Sibling `.md` audit returns zero gaps for new files.
|
||||
|
||||
## 11. Dependent sweep
|
||||
|
||||
- [ ] 11.1 `bunx turbo test --filter=...@suaveplan/installer` green
|
||||
(validates `@suaveplan/install-agent-skills` against the new surface).
|
||||
- [ ] 11.2 Add empty `upgrade.ts` migration table export to
|
||||
`@suaveplan/install-agent-skills` and wire it through its manifest so
|
||||
the migration-loader path is exercised even with zero migrations.
|
||||
|
||||
## 12. Archive
|
||||
|
||||
- [ ] 12.1 Fold `specs/installer/spec.md` delta into
|
||||
`openspec/specs/installer/spec.md`.
|
||||
- [ ] 12.2 Move `openspec/changes/add-installer-upgrade/` to
|
||||
`openspec/archive/<yyyy-mm-dd>-add-installer-upgrade/`.
|
||||
@@ -0,0 +1,129 @@
|
||||
# Design — `usePasskeyAutofill` (`@suaveplan/react-auth`)
|
||||
|
||||
Non-normative. Architecture, lifecycle, and rationale. The normative contract lives in `specs/react-auth/spec.md`.
|
||||
|
||||
## Package shape
|
||||
|
||||
`@suaveplan/react-auth` is created by this change. It is a lightweight browser-tier package at `packages/auth/react-auth/`. Tier is `browser` (React 19 peer dep, DOM types fine). It holds exactly this hook and the optional `<PasskeyAutofillField>` wrapper today; future auth-related React hooks land in follow-up changes under the same package.
|
||||
|
||||
The hook lives at `src/use-passkey-autofill/use-passkey-autofill.ts`; the component at `src/passkey-autofill-field/passkey-autofill-field.tsx`; errors at `src/errors/errors.ts`. Each has a sibling `.md`, a `.test.ts` (or `.test.tsx`), and stories per CLAUDE.md.
|
||||
|
||||
## Conditional UI background
|
||||
|
||||
WebAuthn Level 3 §5.1.4 defines `mediation: "conditional"`. Browsers that support it surface available platform credentials in the autocomplete sheet of any `<input>` with `autocomplete="username webauthn"`. The feature is gated on `PublicKeyCredential.isConditionalMediationAvailable()` — a static method that does not exist on Safari < 17, Firefox < 119, or any SSR environment. Calling `navigator.credentials.get({ publicKey, mediation: "conditional" })` without that check throws `NotSupportedError` on older browsers, which `@suaveplan/webauthn` maps to `WEBAUTHN.NOT_SUPPORTED`.
|
||||
|
||||
The hook MUST therefore feature-detect before dispatching. Because feature detection is async and DOM-only, it can only run inside a `useEffect`.
|
||||
|
||||
## Hook signature
|
||||
|
||||
```ts
|
||||
interface UsePasskeyAutofillOptions {
|
||||
rpId: string;
|
||||
challenge: Uint8Array;
|
||||
onCredential: (result: AuthenticateResult) => void;
|
||||
onError?: (error: SuaveplanError) => void;
|
||||
signal?: AbortSignal;
|
||||
allowCredentials?: PublicKeyCredentialDescriptor[];
|
||||
userVerification?: UserVerificationRequirement;
|
||||
timeout?: number;
|
||||
expectedOrigin?: string;
|
||||
expectedRpIdHash?: Uint8Array;
|
||||
}
|
||||
|
||||
interface UsePasskeyAutofillReturn {
|
||||
ready: boolean;
|
||||
error: SuaveplanError | null;
|
||||
abort: () => void;
|
||||
}
|
||||
|
||||
function usePasskeyAutofill(opts: UsePasskeyAutofillOptions): UsePasskeyAutofillReturn;
|
||||
```
|
||||
|
||||
`ready` flips to `true` after `isConditionalMediationAvailable()` resolves `true` AND the conditional `authenticate` Promise is in-flight. `error` carries the most recent translated error (cleared when a new attempt starts). `abort` is a stable `useCallback` that fires the internal `AbortController`.
|
||||
|
||||
## Lifecycle — AbortController ownership
|
||||
|
||||
The hook owns exactly one `AbortController` per "session" (one mount, one option-identity). The controller is composed with the caller's `options.signal` via a small `composeSignals(...signals)` helper (any aborts → composed aborts).
|
||||
|
||||
| Event | Behavior |
|
||||
|---|---|
|
||||
| Mount (client) | Run feature detect → dispatch `authenticate({ mediation: "conditional", signal: composed })`. |
|
||||
| Mount (server) | No-op. The `useEffect` never runs. |
|
||||
| Unmount | Call `controller.abort()`. The underlying `navigator.credentials.get` rejects with `AbortError`, mapped to `WEBAUTHN.ABORTED` → `REACT_AUTH.ABORTED`. |
|
||||
| Caller `signal` aborts | Composed controller aborts → same as unmount path. |
|
||||
| Caller calls returned `abort()` | Same as above. State remains mounted; consumer may re-trigger by changing option identity. |
|
||||
| `<input>` blur (via `<PasskeyAutofillField>`) | Consumer-controlled. The component calls `abort()` on `onBlur` IF no credential has been delivered yet. The hook alone does not subscribe to blur (it does not touch the DOM). |
|
||||
| Option identity changes (e.g. fresh `challenge`) | Effect re-runs: old controller aborts, new controller created, new request dispatched. |
|
||||
| `onCredential` fires | Controller is left intact (the request is already settled); `error` cleared. |
|
||||
| `onError` fires | Controller aborted; `error` stored; consumer may remount or change challenge to retry. |
|
||||
|
||||
The hook MUST NOT touch the DOM directly — it works alongside an `<input autocomplete="username webauthn">` rendered by the consumer. The browser surfaces the autofill UI based on the autocomplete token; the hook just keeps the conditional `authenticate` request pending so the browser has something to deliver into.
|
||||
|
||||
## SSR safety
|
||||
|
||||
- `typeof window === "undefined"` short-circuits before any `navigator` access.
|
||||
- `isConditionalMediationAvailable()` is called inside `useEffect` (which never runs server-side under React 19 SSR).
|
||||
- The hook returns `{ ready: false, error: null, abort: noop }` on the server render, then re-renders on hydration.
|
||||
|
||||
## Error translation
|
||||
|
||||
```text
|
||||
WEBAUTHN.NOT_SUPPORTED → REACT_AUTH.AUTOFILL_UNAVAILABLE
|
||||
WEBAUTHN.ABORTED → REACT_AUTH.ABORTED
|
||||
all other WEBAUTHN.* → REACT_AUTH.AUTOFILL_FAILED (cause-chained)
|
||||
Zod schema rejection at entry → REACT_AUTH.INVALID_OPTIONS (thrown synchronously)
|
||||
SSR / non-DOM environment → REACT_AUTH.SSR_NO_OP (surfaced via onError only)
|
||||
```
|
||||
|
||||
Every translation preserves the original `SuaveplanError` via `cause`, so consumers can introspect `error.cause?.code` for the underlying `WEBAUTHN.*` code.
|
||||
|
||||
## Testing approach
|
||||
|
||||
- Real `navigator.credentials.get` stub at the jsdom layer — install a fake `PublicKeyCredential` with `isConditionalMediationAvailable` and a `navigator.credentials.get` that resolves a fixture `AuthenticateResult` or rejects with a chosen `DOMException`. Tests live in `src/use-passkey-autofill/use-passkey-autofill.test.tsx` using `@suaveplan/testing-dom` render utilities and `@suaveplan/testing/runner` for the runner-neutral imports (CLAUDE.md rule 11).
|
||||
- For environments lacking `PublicKeyCredential` entirely (deliberately stripped), assert `REACT_AUTH.AUTOFILL_UNAVAILABLE` is reported via `onError` and `ready` stays `false`.
|
||||
- Lifecycle is verified by mounting, asserting an in-flight request, unmounting, and asserting the request rejected with `REACT_AUTH.ABORTED`.
|
||||
- 100% coverage per CLAUDE.md / openspec rule 7 — no carve-outs.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/auth/react-auth/
|
||||
README.md
|
||||
package.json
|
||||
tsconfig.json
|
||||
vitest.config.ts
|
||||
src/
|
||||
index.ts # explicit named re-exports (rule 6a)
|
||||
use-passkey-autofill/
|
||||
index.ts
|
||||
use-passkey-autofill.ts
|
||||
use-passkey-autofill.md
|
||||
use-passkey-autofill.test.tsx
|
||||
use-passkey-autofill.stories.tsx
|
||||
use-passkey-autofill.types.ts
|
||||
use-passkey-autofill.schemas.ts
|
||||
passkey-autofill-field/
|
||||
index.ts
|
||||
passkey-autofill-field.tsx
|
||||
passkey-autofill-field.md
|
||||
passkey-autofill-field.test.tsx
|
||||
passkey-autofill-field.stories.tsx
|
||||
passkey-autofill-field.types.ts
|
||||
errors/
|
||||
index.ts
|
||||
errors.ts
|
||||
errors.md
|
||||
errors.test.ts
|
||||
utils/
|
||||
compose-signals.ts
|
||||
compose-signals.md
|
||||
compose-signals.test.ts
|
||||
```
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Hook-first, component-optional.** The `<PasskeyAutofillField>` wrapper exists for the common case but is never a requirement; advanced consumers wire the hook to their own input.
|
||||
- **No global state, no context provider.** Multiple `usePasskeyAutofill` instances may coexist (e.g. a multi-tenant login page); each owns its own controller.
|
||||
- **Errors are translated, not leaked.** Consumers see `REACT_AUTH.*` codes. The underlying `WEBAUTHN.*` cause is preserved on `error.cause` for introspection.
|
||||
- **`onCredential` is required, `onError` is optional.** A consumer that does not pass `onError` accepts that errors silently no-op (with `error` still populated on the return value).
|
||||
- **Stable `abort` reference.** Returned via `useCallback` so consumers can put it in dependency arrays without thrashing.
|
||||
@@ -0,0 +1,85 @@
|
||||
# `@suaveplan/react-auth` — `usePasskeyAutofill` React hook
|
||||
|
||||
**Package**: `@suaveplan/react-auth` (NEW)
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package
|
||||
**Date**: 2026-05-17
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/react-auth/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
The archived `enterprise-webauthn` change deferred a React hook target named `usePasskeyAutofill` to a follow-up `@suaveplan/react-auth` change (see `openspec/archive/2026-05-11-enterprise-webauthn/proposal.md` lines 85-86, 117, 191 and `tasks.md` §2.7.3 / §0 "Out of scope"). The underlying capability — `authenticate({ mediation: "conditional" })` plus `isConditionalMediationAvailable()` — shipped on `@suaveplan/webauthn` and is exported from `packages/auth/webauthn/src/authenticate/authenticate.ts` and `packages/auth/webauthn/src/index.ts`, so the consumer-side hook is now unblocked.
|
||||
|
||||
Without this hook, every genesis React app that wants WebAuthn Level 3 Conditional UI ("passkey autofill" — the browser surfaces credentials in the username `<input>` autocomplete sheet) has to hand-roll the same wiring: SSR guard, feature detect, `AbortController` lifecycle, unmount cancellation, blur cancellation, and error-code translation. Hand-rolled versions get the abort lifecycle wrong (they leak credential requests across mounts) or skip the `isConditionalMediationAvailable()` check (which throws on Safari < 17 and every SSR environment). Shipping the hook once, in a dedicated package, removes that footgun.
|
||||
|
||||
`@suaveplan/react-auth` does not exist yet. This change creates it as a lightweight browser-tier package with a single React 19 peer dep, exporting the hook and a tiny optional `<PasskeyAutofillField>` wrapper that renders an `<input autocomplete="username webauthn">` with the hook pre-wired. The package is deliberately scoped to the autofill surface only; future auth-related React hooks (passkey-as-master-key-unlock, OAuth callback, magic-link landing) land in follow-up changes under the same package.
|
||||
|
||||
## What changes
|
||||
|
||||
### New package: `@suaveplan/react-auth`
|
||||
|
||||
Browser-tier package at `packages/auth/react-auth/`. React 19 peer dep, `@suaveplan/webauthn` workspace dep.
|
||||
|
||||
#### Hook
|
||||
|
||||
- ADDED: `usePasskeyAutofill(options: UsePasskeyAutofillOptions): UsePasskeyAutofillReturn` — declarative hook around `authenticate({ mediation: "conditional" })`. Owns its own `AbortController`; aborts on unmount, on a fresh option-change remount, and (optionally) on the consumer-supplied `<input>`'s `blur` when no credential has been delivered.
|
||||
- ADDED: `UsePasskeyAutofillOptions` — `{ rpId: string; challenge: Uint8Array; onCredential: (result: AuthenticateResult) => void; onError?: (error: SuaveplanError) => void; signal?: AbortSignal; allowCredentials?: PublicKeyCredentialDescriptor[]; userVerification?: UserVerificationRequirement; timeout?: number; expectedOrigin?: string; expectedRpIdHash?: Uint8Array }`. All option fields are surfaced as a named interface (per CLAUDE.md rule 11 — no inline shapes).
|
||||
- ADDED: `UsePasskeyAutofillReturn` — `{ ready: boolean; error: SuaveplanError | null; abort: () => void }`. `ready` is `true` once `isConditionalMediationAvailable()` resolves `true` AND the conditional `authenticate` call has been dispatched. `error` is the most recent translated error or `null`. `abort()` is a stable callback that cancels the in-flight request without unmounting the component.
|
||||
|
||||
#### Optional component wrapper
|
||||
|
||||
- ADDED: `<PasskeyAutofillField>` — a thin presentational `<input autocomplete="username webauthn">` wrapper that wires the hook to the input and forwards `onBlur` to the hook's `abort`. Accepts `inputProps` so the consumer can style and label the input freely. The component is opt-in: callers may instead use the hook directly with their own `<input>` markup.
|
||||
- ADDED: `PasskeyAutofillFieldProps` — named interface for the component's prop shape.
|
||||
|
||||
#### Errors / observability / boundaries
|
||||
|
||||
- ADDED: `defineCodes("REACT_AUTH", [...])` registry covering `REACT_AUTH.AUTOFILL_UNAVAILABLE` (browser missing `isConditionalMediationAvailable`), `REACT_AUTH.AUTOFILL_FAILED` (underlying `WEBAUTHN.*` cause), `REACT_AUTH.ABORTED` (caller-driven abort), `REACT_AUTH.SSR_NO_OP` (hook ran on server — reported via `onError` only when `onError` is supplied, otherwise silently no-ops), `REACT_AUTH.INVALID_OPTIONS` (Zod boundary rejection of `UsePasskeyAutofillOptions`). Every code is thrown by exactly one production path.
|
||||
- ADDED: `UsePasskeyAutofillOptionsSchema` — Zod schema validated at hook entry; mismatches throw `REACT_AUTH.INVALID_OPTIONS` synchronously (so React surfaces it via the nearest error boundary at mount time, not silently in an effect).
|
||||
- ADDED: `AbortSignal` chaining — caller-supplied `options.signal` is composed with the hook's internal controller so either source aborts the underlying `navigator.credentials.get`. Internal controller fires on unmount and on `abort()`.
|
||||
|
||||
#### SSR + feature detection
|
||||
|
||||
- ADDED: `useEffect`-gated entry point — the hook performs no work on the server render. On the first client mount it calls `PublicKeyCredential.isConditionalMediationAvailable?.()` via `@suaveplan/webauthn`'s `isConditionalMediationAvailable` helper. If the static method is missing or returns `false`, the hook stays `ready === false`, emits `REACT_AUTH.AUTOFILL_UNAVAILABLE` via `onError` (if supplied), and never dispatches `authenticate`.
|
||||
- ADDED: stable `abort` reference — returned `abort` is wrapped with `useCallback` and a controller ref, so consumers can put it in `useEffect` cleanup arrays without re-running the effect on every render.
|
||||
|
||||
#### Storybook
|
||||
|
||||
- ADDED: `src/use-passkey-autofill/use-passkey-autofill.stories.tsx` — default story (hook wired into a dummy form), playground story (props as controls), variant stories (unavailable browser, abort-on-blur, error-state). Per CLAUDE.md "Every React component needs `.stories.tsx`".
|
||||
- ADDED: `src/passkey-autofill-field/passkey-autofill-field.stories.tsx` — default / playground / variants.
|
||||
|
||||
### Subpath exports
|
||||
|
||||
- `.` — root barrel: `usePasskeyAutofill`, `UsePasskeyAutofillOptions`, `UsePasskeyAutofillReturn`, `PasskeyAutofillField`, `PasskeyAutofillFieldProps`, error registry.
|
||||
- `./use-passkey-autofill/use-passkey-autofill` — hook + types.
|
||||
- `./passkey-autofill-field/passkey-autofill-field` — component + props.
|
||||
- `./errors` — `REACT_AUTH` code registry.
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `@suaveplan/webauthn workspace:^` — `authenticate`, `isConditionalMediationAvailable`, `AuthenticateResult`.
|
||||
- `@suaveplan/error workspace:^` — `defineCodes`, `SuaveplanError`.
|
||||
- `@suaveplan/types workspace:^` — shared option bundles.
|
||||
- `react ^19.0.0` (peer) — hook surface.
|
||||
- `zod catalog:forms` — boundary validation.
|
||||
- `tslib catalog:build`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Server-side challenge issuance / verification** — `@suaveplan/api-schemas/auth` and `@suaveplan/vault` own that. The hook expects the caller to have already obtained `{ rpId, challenge }` from the server.
|
||||
- **Non-autofill WebAuthn React hooks** — `usePasskeyRegister`, `usePasskeyAuthenticate` (modal), `useVaultUnlock` land in follow-up changes under `@suaveplan/react-auth`.
|
||||
- **WebAuthn Level 2 polyfill** — Conditional UI is L3-only; older browsers surface `REACT_AUTH.AUTOFILL_UNAVAILABLE` and the hook no-ops.
|
||||
- **DOM manipulation** — the hook does not render or query DOM elements directly. The consumer renders the `<input autocomplete="username webauthn">`. The optional `<PasskeyAutofillField>` is a thin convenience wrapper, not a requirement.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
This is a NEW package — no existing dependents. Run `bunx turbo ls --filter=...@suaveplan/react-auth` after landing to confirm. The hook will be adopted by `@suaveplan/web-app` and any genesis-original app that ships a login page, in their own follow-up changes.
|
||||
|
||||
- No dependents (verified by `bunx turbo ls --filter=...@suaveplan/react-auth` post-landing — currently empty).
|
||||
|
||||
## Migration
|
||||
|
||||
No prior surface — additive only. There is no obsolete contract for consumers to migrate from. The `enterprise-webauthn` change explicitly scoped this hook out with the note "implementation lives in the `react-auth` change folder; this package only ships the underlying `authenticate({ mediation: "conditional" })` surface" — this change fulfils that deferral.
|
||||
@@ -0,0 +1,186 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: react-auth (add-react-auth-passkey-autofill)
|
||||
|
||||
**Package**: `@suaveplan/react-auth`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. (See `openspec/AGENTS.md` Section 13.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: usePasskeyAutofill hook dispatches conditional WebAuthn
|
||||
|
||||
The package SHALL export a `usePasskeyAutofill` React hook that, on the first client mount, feature-detects WebAuthn Conditional UI via `isConditionalMediationAvailable()` and dispatches `authenticate({ mediation: "conditional" })` via `@suaveplan/webauthn`. The hook SHALL deliver a successful `AuthenticateResult` via the caller's `onCredential` callback.
|
||||
|
||||
#### Scenario: conditional authenticate is invoked when supported
|
||||
|
||||
- **WHEN** the hook mounts in a browser whose `PublicKeyCredential.isConditionalMediationAvailable()` resolves `true`
|
||||
- **THEN** the hook calls `authenticate` with `mediation: "conditional"` and the caller-supplied `rpId`, `challenge`, `allowCredentials`, `userVerification`, `timeout`, `expectedOrigin`, and `expectedRpIdHash`
|
||||
- **AND** the returned `ready` flag flips to `true` once the request is in-flight
|
||||
|
||||
#### Scenario: successful credential is delivered to onCredential
|
||||
|
||||
- **WHEN** the underlying `authenticate` Promise resolves with an `AuthenticateResult`
|
||||
- **THEN** `onCredential` is invoked once with that result
|
||||
- **AND** the hook's `error` state remains `null`
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"dispatches authenticate with mediation conditional when available"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"delivers AuthenticateResult to onCredential on success"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Hook is SSR-safe and feature-detects before dispatch
|
||||
|
||||
The hook SHALL perform zero work during server rendering and SHALL never call `navigator.credentials.get` without first confirming `PublicKeyCredential.isConditionalMediationAvailable()` resolves `true`. When Conditional UI is unavailable, the hook SHALL stay `ready === false` and report `REACT_AUTH.AUTOFILL_UNAVAILABLE` via the caller's `onError` (when supplied).
|
||||
|
||||
#### Scenario: server render performs no DOM access
|
||||
|
||||
- **WHEN** the hook is rendered in an environment where `typeof window === "undefined"`
|
||||
- **THEN** the hook returns `{ ready: false, error: null, abort }` without touching `navigator`, `document`, or `PublicKeyCredential`
|
||||
|
||||
#### Scenario: missing Conditional UI surfaces AUTOFILL_UNAVAILABLE
|
||||
|
||||
- **WHEN** the hook mounts in a browser where `PublicKeyCredential.isConditionalMediationAvailable` is undefined OR resolves `false`
|
||||
- **THEN** the hook never calls `authenticate`
|
||||
- **AND** `ready` remains `false`
|
||||
- **AND** `onError` (if supplied) is invoked with a `SuaveplanError` whose `code === "REACT_AUTH.AUTOFILL_UNAVAILABLE"`
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"performs no work during SSR render"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"reports AUTOFILL_UNAVAILABLE when isConditionalMediationAvailable is missing"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AbortController is owned by the hook and aborts on unmount, signal, and abort()
|
||||
|
||||
The hook SHALL own exactly one internal `AbortController` per effect run, composed with the caller's `options.signal` if supplied. The composed signal SHALL be forwarded into the underlying `authenticate` call. The hook SHALL abort the in-flight request when the component unmounts, when the caller's `signal` aborts, or when the consumer invokes the returned `abort()` callback.
|
||||
|
||||
#### Scenario: unmount aborts the in-flight authenticate call
|
||||
|
||||
- **WHEN** the hook is mounted with an in-flight `authenticate` request and the component unmounts
|
||||
- **THEN** the underlying `navigator.credentials.get` rejects with `AbortError`
|
||||
- **AND** the hook's `error` (read before unmount completes) carries a `SuaveplanError` whose `code === "REACT_AUTH.ABORTED"` with the `WEBAUTHN.ABORTED` cause preserved
|
||||
|
||||
#### Scenario: caller-supplied AbortSignal aborts the request
|
||||
|
||||
- **WHEN** the caller passes `options.signal` and that controller aborts after mount
|
||||
- **THEN** the in-flight `authenticate` rejects with `AbortError`
|
||||
- **AND** the hook reports `REACT_AUTH.ABORTED` via `onError` if supplied
|
||||
|
||||
#### Scenario: returned abort() cancels without unmount
|
||||
|
||||
- **WHEN** the consumer calls the returned `abort()` while a request is in-flight
|
||||
- **THEN** the request is cancelled
|
||||
- **AND** the component remains mounted
|
||||
- **AND** `ready` returns to `false`
|
||||
|
||||
#### Scenario: option-identity change cancels old request and dispatches new
|
||||
|
||||
- **WHEN** `options.challenge` reference changes between renders
|
||||
- **THEN** the old controller aborts and a new `authenticate` is dispatched against the new challenge
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"aborts the in-flight request on unmount"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"propagates caller AbortSignal to authenticate"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"returned abort cancels without unmounting"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"re-dispatches when challenge identity changes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: WebAuthn errors are translated into the REACT_AUTH code registry
|
||||
|
||||
The package SHALL export a `defineCodes("REACT_AUTH", ...)` registry covering `AUTOFILL_UNAVAILABLE`, `AUTOFILL_FAILED`, `ABORTED`, `SSR_NO_OP`, and `INVALID_OPTIONS`. Every error surfaced by the hook SHALL be a `SuaveplanError` whose `code` is one of these values; the underlying `WEBAUTHN.*` error SHALL be preserved on `cause` for introspection.
|
||||
|
||||
#### Scenario: WEBAUTHN.NOT_SUPPORTED maps to REACT_AUTH.AUTOFILL_UNAVAILABLE
|
||||
|
||||
- **WHEN** the underlying `authenticate` rejects with `WEBAUTHN.NOT_SUPPORTED`
|
||||
- **THEN** the hook surfaces a `SuaveplanError` with `code === "REACT_AUTH.AUTOFILL_UNAVAILABLE"` and `cause.code === "WEBAUTHN.NOT_SUPPORTED"`
|
||||
|
||||
#### Scenario: WEBAUTHN.ABORTED maps to REACT_AUTH.ABORTED
|
||||
|
||||
- **WHEN** the underlying request rejects with `WEBAUTHN.ABORTED`
|
||||
- **THEN** the surfaced error has `code === "REACT_AUTH.ABORTED"` with the same cause preserved
|
||||
|
||||
#### Scenario: any other WEBAUTHN.* maps to REACT_AUTH.AUTOFILL_FAILED
|
||||
|
||||
- **WHEN** the underlying request rejects with `WEBAUTHN.SIGNATURE_INVALID` (or any other non-cancel, non-not-supported `WEBAUTHN.*` code)
|
||||
- **THEN** the surfaced error has `code === "REACT_AUTH.AUTOFILL_FAILED"` with the underlying cause preserved
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"translates WEBAUTHN.NOT_SUPPORTED to REACT_AUTH.AUTOFILL_UNAVAILABLE"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"translates WEBAUTHN.ABORTED to REACT_AUTH.ABORTED"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"translates other WEBAUTHN errors to REACT_AUTH.AUTOFILL_FAILED"` AND `src/errors/errors.test.ts::"every REACT_AUTH code is reachable from a production path"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Hook options are validated by Zod at entry
|
||||
|
||||
The hook SHALL validate `UsePasskeyAutofillOptions` against a Zod schema at the start of every effect run. Schema mismatches SHALL throw `REACT_AUTH.INVALID_OPTIONS` synchronously so that React's nearest error boundary catches the failure at mount time rather than silently ignoring it inside an effect.
|
||||
|
||||
#### Scenario: invalid rpId throws INVALID_OPTIONS
|
||||
|
||||
- **WHEN** the caller passes `rpId: ""` (empty string) or omits a required field
|
||||
- **THEN** the hook throws a `SuaveplanError` with `code === "REACT_AUTH.INVALID_OPTIONS"` during render
|
||||
- **AND** no `authenticate` call is dispatched
|
||||
|
||||
#### Scenario: invalid challenge type throws INVALID_OPTIONS
|
||||
|
||||
- **WHEN** `challenge` is not a `Uint8Array`
|
||||
- **THEN** the same error is thrown synchronously
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"throws INVALID_OPTIONS when rpId is empty"` AND `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"throws INVALID_OPTIONS when challenge is not Uint8Array"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Hook does not touch the DOM directly
|
||||
|
||||
The hook SHALL NOT call `document.querySelector`, attach event listeners to inputs, or render any DOM. The autofill UI is surfaced by the browser based on an `<input autocomplete="username webauthn">` element rendered by the consumer; the hook only keeps the conditional `authenticate` request pending so the browser has something to deliver into.
|
||||
|
||||
#### Scenario: hook makes no document or input access
|
||||
|
||||
- **WHEN** the hook is mounted with a spy installed on `document.querySelector` and `HTMLInputElement.prototype.addEventListener`
|
||||
- **THEN** neither spy is invoked during the hook's lifecycle
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"does not touch document or input elements"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Returned abort callback has a stable reference
|
||||
|
||||
The hook SHALL return an `abort` callback whose reference identity is stable across renders (i.e. wrapped with `useCallback` against a controller ref), so consumers can place it in `useEffect` dependency arrays or memoization keys without triggering re-runs.
|
||||
|
||||
#### Scenario: abort reference is identical across re-renders
|
||||
|
||||
- **WHEN** the parent component re-renders while the hook's options are unchanged
|
||||
- **THEN** `Object.is(prevReturn.abort, nextReturn.abort) === true`
|
||||
|
||||
**Verified by:** `src/use-passkey-autofill/use-passkey-autofill.test.tsx::"returned abort reference is stable across re-renders"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: PasskeyAutofillField component wraps the hook with an input
|
||||
|
||||
The package SHALL export an optional `<PasskeyAutofillField>` component that renders an `<input autocomplete="username webauthn">` element, wires it to `usePasskeyAutofill`, and (by default) calls the hook's `abort()` when the input fires `blur` AND no credential has been delivered yet. Consumers SHALL be able to forward arbitrary `inputProps` to style and label the input.
|
||||
|
||||
#### Scenario: input renders with the correct autocomplete token
|
||||
|
||||
- **WHEN** `<PasskeyAutofillField rpId="example.test" challenge={c} onCredential={...} />` is rendered
|
||||
- **THEN** the rendered `<input>` has `autocomplete="username webauthn"`
|
||||
|
||||
#### Scenario: blur cancels the request when no credential delivered
|
||||
|
||||
- **WHEN** the user blurs the input before any credential is delivered
|
||||
- **THEN** the hook's `abort()` is invoked and the in-flight request rejects with `REACT_AUTH.ABORTED`
|
||||
|
||||
#### Scenario: blur after credential delivery does not abort
|
||||
|
||||
- **WHEN** `onCredential` has already fired
|
||||
- **THEN** a subsequent blur does NOT invoke `abort()`
|
||||
|
||||
**Verified by:** `src/passkey-autofill-field/passkey-autofill-field.test.tsx::"renders input with autocomplete username webauthn"` AND `src/passkey-autofill-field/passkey-autofill-field.test.tsx::"blur cancels in-flight request"` AND `src/passkey-autofill-field/passkey-autofill-field.test.tsx::"blur after credential delivery is a no-op"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath exports cover hook, component, and errors
|
||||
|
||||
The package SHALL expose subpath exports at `.`, `./use-passkey-autofill/use-passkey-autofill`, `./passkey-autofill-field/passkey-autofill-field`, and `./errors`, each tree-shakeable, so consumers can import the hook, the component, or the error registry independently.
|
||||
|
||||
#### Scenario: every documented subpath resolves
|
||||
|
||||
- **WHEN** a consumer imports each documented subpath
|
||||
- **THEN** every import resolves to the declared symbols
|
||||
|
||||
**Verified by:** `src/index.test.ts::"every documented subpath resolves"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,104 @@
|
||||
# Tasks — `@suaveplan/react-auth` `usePasskeyAutofill`
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick boxes.
|
||||
|
||||
## 0. Blocking prerequisite
|
||||
|
||||
- [x] 0.1 `@suaveplan/webauthn` exports `authenticate({ mediation: "conditional" })` and `isConditionalMediationAvailable()` — verified at `packages/auth/webauthn/src/authenticate/authenticate.ts` and `packages/auth/webauthn/src/index.ts`.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read `openspec/archive/2026-05-11-enterprise-webauthn/proposal.md` + `tasks.md` to confirm the `usePasskeyAutofill` deferral
|
||||
- [x] 1.2 Read `openspec/AGENTS.md` §13 and §14
|
||||
- [x] 1.3 Draft `proposal.md` from `openspec/templates/proposal.md`
|
||||
- [x] 1.4 Draft `design.md` covering hook signature, lifecycle, SSR safety, error translation
|
||||
- [x] 1.5 Draft `specs/react-auth/spec.md` from `openspec/templates/spec-delta.md`
|
||||
- [x] 1.6 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles
|
||||
|
||||
## 2. Package scaffold
|
||||
|
||||
- [ ] 2.1 Create `packages/auth/react-auth/` with `package.json` (browser tier, React 19 peer, `@suaveplan/webauthn workspace:^`, `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `zod catalog:forms`, `tslib catalog:build`, `@vitest/coverage-istanbul catalog:testing`)
|
||||
- [ ] 2.2 `tsconfig.json` extending `@suaveplan/typescript-config/browser`
|
||||
- [ ] 2.3 `vitest.config.ts` via `createTestConfig` with jsdom environment and 100/100/100/100 thresholds
|
||||
- [ ] 2.4 `package.json` `exports` map covering `.`, `./use-passkey-autofill/use-passkey-autofill`, `./passkey-autofill-field/passkey-autofill-field`, `./errors`
|
||||
- [ ] 2.5 `"sideEffects": false`, `"browser"` field set
|
||||
|
||||
## 3. Implementation
|
||||
|
||||
### 3.1 Errors
|
||||
|
||||
- [ ] 3.1.1 `src/errors/errors.ts` — `defineCodes("REACT_AUTH", ["AUTOFILL_UNAVAILABLE", "AUTOFILL_FAILED", "ABORTED", "SSR_NO_OP", "INVALID_OPTIONS"])`
|
||||
|
||||
### 3.2 Utilities
|
||||
|
||||
- [ ] 3.2.1 `src/utils/compose-signals.ts` — composes N `AbortSignal`s into one (any aborts → composed aborts); handles already-aborted signals
|
||||
|
||||
### 3.3 Hook
|
||||
|
||||
- [ ] 3.3.1 `src/use-passkey-autofill/use-passkey-autofill.types.ts` — named `UsePasskeyAutofillOptions`, `UsePasskeyAutofillReturn` interfaces
|
||||
- [ ] 3.3.2 `src/use-passkey-autofill/use-passkey-autofill.schemas.ts` — `UsePasskeyAutofillOptionsSchema` Zod schema; throws `REACT_AUTH.INVALID_OPTIONS` on parse failure
|
||||
- [ ] 3.3.3 `src/use-passkey-autofill/use-passkey-autofill.ts` — hook body: SSR no-op, feature-detect, dispatch, AbortController ownership, stable `abort` callback, error translation
|
||||
- [ ] 3.3.4 Effect re-runs on option-identity change; old controller aborts cleanly
|
||||
|
||||
### 3.4 Component wrapper
|
||||
|
||||
- [ ] 3.4.1 `src/passkey-autofill-field/passkey-autofill-field.types.ts` — named `PasskeyAutofillFieldProps`
|
||||
- [ ] 3.4.2 `src/passkey-autofill-field/passkey-autofill-field.tsx` — wraps hook + `<input autocomplete="username webauthn">`; forwards `onBlur` → `abort()`
|
||||
|
||||
### 3.5 Barrels
|
||||
|
||||
- [ ] 3.5.1 `src/index.ts` — explicit named re-exports (no `export *`)
|
||||
- [ ] 3.5.2 Per-feature `index.ts` files — explicit named re-exports
|
||||
|
||||
## 4. Tests (runner-neutral imports — `@suaveplan/testing/runner`)
|
||||
|
||||
- [ ] 4.1 `src/use-passkey-autofill/use-passkey-autofill.test.tsx` — happy path, SSR no-op, feature-unavailable, abort on unmount, abort on caller signal, abort via returned callback, option-identity change re-runs, error translation for every `WEBAUTHN.*` code, Zod rejection at entry
|
||||
- [ ] 4.2 `src/passkey-autofill-field/passkey-autofill-field.test.tsx` — renders input with correct autocomplete, forwards `inputProps`, blur calls `abort` only when no credential delivered
|
||||
- [ ] 4.3 `src/errors/errors.test.ts` — every `REACT_AUTH.*` code reachable from a production path
|
||||
- [ ] 4.4 `src/utils/compose-signals.test.ts` — any-aborts semantics, already-aborted inputs, no leaks
|
||||
- [ ] 4.5 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for async)
|
||||
|
||||
## 5. Stories
|
||||
|
||||
- [ ] 5.1 `src/use-passkey-autofill/use-passkey-autofill.stories.tsx` — default, playground, variants (unavailable, error, abort-on-blur)
|
||||
- [ ] 5.2 `src/passkey-autofill-field/passkey-autofill-field.stories.tsx` — default, playground, variants
|
||||
|
||||
## 6. Documentation
|
||||
|
||||
- [ ] 6.1 Sibling `.md` for `use-passkey-autofill.ts`, `passkey-autofill-field.tsx`, `errors.ts`, `compose-signals.ts` (≥200 words, basic + advanced examples, full API reference)
|
||||
- [ ] 6.2 `README.md` at package root with Modules table
|
||||
- [ ] 6.3 Add `@suaveplan/react-auth` row to `CLAUDE.md` under an Auth/React section
|
||||
|
||||
## 7. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [ ] 7.1 `bun run check` — Biome 0 errors / 0 warnings / 0 info
|
||||
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/react-auth` — green
|
||||
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/react-auth` — 100/100/100/100
|
||||
- [ ] 7.4 `bun scripts/openspec-conformance.ts @suaveplan/react-auth` — exit 0, no `error` findings
|
||||
- [ ] 7.5 `bunx turbo build --filter=@suaveplan/react-auth` — green
|
||||
|
||||
## 8. Pre-archive 7-row checklist (AGENTS.md §13)
|
||||
|
||||
- [ ] 8.1 Generics threaded; `expectTypeOf` assertions on hook return type
|
||||
- [ ] 8.2 Zod schema wired at hook entry boundary
|
||||
- [ ] 8.3 Every `REACT_AUTH.*` code thrown by ≥1 production path
|
||||
- [ ] 8.4 Strict-mode behaviors tested (feature-available vs unavailable)
|
||||
- [ ] 8.5 Hook lifecycle tested under React 19 strict-mode double-mount
|
||||
- [ ] 8.6 AbortSignal composition tested for all sources (caller signal, returned `abort`, unmount)
|
||||
- [ ] 8.7 No retroactive tick
|
||||
|
||||
## 9. Banned-pattern + barrel + runner-import scans
|
||||
|
||||
- [ ] 9.1 `grep -rn "@deprecated\|TODO\|FIXME\|ts-ignore\|ts-expect-error\|\.skip\|\.only" packages/auth/react-auth/src/ --include="*.ts" --include="*.tsx"` — zero output
|
||||
- [ ] 9.2 `grep -rn "^\s*export\s\+\(type\s\+\)\?\*\s\+from" packages/auth/react-auth/src/ --include="index.ts"` — zero output (feature barrels must use named exports)
|
||||
- [ ] 9.3 `grep -rnE "from ['\"](vitest|bun:test)['\"]" packages/auth/react-auth/src/ --include="*.ts" --include="*.tsx"` — zero output
|
||||
|
||||
## 10. Archive
|
||||
|
||||
- [ ] 10.1 Fold `specs/react-auth/spec.md` into `openspec/specs/react-auth/spec.md`
|
||||
- [ ] 10.2 Move `openspec/changes/add-react-auth-passkey-autofill/` → `openspec/archive/<yyyy-mm-dd>-add-react-auth-passkey-autofill/`
|
||||
- [ ] 10.3 Refresh `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 10.4 Refresh `MISSING.yml` via `bun scripts/generate-missing.ts`
|
||||
- [ ] 10.5 Update `STATUS.md` — add `@suaveplan/react-auth` row in `Completed`
|
||||
- [ ] 10.6 Update `SUMMARY.md` with per-package paragraph
|
||||
- [ ] 10.7 `bunx markdownlint-cli2 --fix` and stage results
|
||||
@@ -0,0 +1,173 @@
|
||||
# Design — @suaveplan/storage-mux
|
||||
|
||||
Non-normative companion to `proposal.md` and `specs/storage-mux/spec.md`.
|
||||
|
||||
## 1. Role in the family
|
||||
|
||||
```text
|
||||
@suaveplan/storage AbstractStorage core (key/value, in-memory)
|
||||
├─ @suaveplan/storage-cdn CdnAdapter contract
|
||||
│ ├─ @suaveplan/storage-cdn-bunnycdn
|
||||
│ └─ @suaveplan/storage-cdn-rackspace
|
||||
├─ @suaveplan/storage-s3
|
||||
├─ @suaveplan/storage-http
|
||||
├─ @suaveplan/storage-file
|
||||
└─ @suaveplan/storage-mux Multi-backend composition (THIS PACKAGE)
|
||||
└─ wraps N StorageLike, exposes 1 StorageLike
|
||||
```
|
||||
|
||||
`storage-mux` is a **composition** package, not a transport. It accepts any list of `StorageLike` (cdn / s3 / http / file / memory / other mux instances) and produces a single `StorageLike`. Recursion is intentional — a sharded-then-failed-over deployment is `Mux(round-robin, [Mux(failover, shard1), Mux(failover, shard2)])`.
|
||||
|
||||
## 2. File layout
|
||||
|
||||
```text
|
||||
src/
|
||||
index.ts # explicit named re-exports
|
||||
error-codes.ts # defineCodes("STORAGE_MUX", [...]) + StorageMuxError
|
||||
types/
|
||||
backend-entry.types.ts # BackendEntry interface
|
||||
mux-options.types.ts # StorageMuxOptions, MuxPolicy union
|
||||
mux-options.schemas.ts # Zod boundary schemas
|
||||
mux/
|
||||
mux.ts # StorageMux extends AbstractStorage
|
||||
mux.md
|
||||
mux.test.ts
|
||||
mux.spec-conformance.test.ts # runStorageConformance × 4 policies
|
||||
policies/
|
||||
selector.types.ts # BackendSelector interface
|
||||
failover/
|
||||
failover.ts
|
||||
failover.md
|
||||
failover.test.ts
|
||||
round-robin/
|
||||
round-robin.ts
|
||||
round-robin.md
|
||||
round-robin.test.ts
|
||||
sticky-by-key/
|
||||
sticky-by-key.ts # stable hash → backend index
|
||||
sticky-by-key.md
|
||||
sticky-by-key.test.ts
|
||||
shield/
|
||||
shield.ts # reads cascade, writes fan-out
|
||||
shield.md
|
||||
shield.test.ts
|
||||
health/
|
||||
health.ts # BackendHealth tracker + cooldown
|
||||
health.md
|
||||
health.test.ts
|
||||
observability/
|
||||
events.ts # event topic constants + emit helpers
|
||||
events.md
|
||||
events.test.ts
|
||||
otel.ts # span attribute helpers
|
||||
otel.md
|
||||
otel.test.ts
|
||||
error-codes/
|
||||
error-codes.test.ts # reachability sweep
|
||||
```
|
||||
|
||||
## 3. Policy state machines
|
||||
|
||||
### 3.1 `failover`
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────┐
|
||||
▼ │
|
||||
[idx=0] ─ call backend[idx] ── ok ──► return │
|
||||
│ │
|
||||
├── transient error ─────► idx++ ──────────┘
|
||||
│
|
||||
├── non-transient error ──► throw (surface as-is)
|
||||
│
|
||||
└── idx == backends.length ─► throw STORAGE_MUX.ALL_BACKENDS_DOWN
|
||||
(cause = last transient error)
|
||||
```
|
||||
|
||||
Per-attempt: `runWithRetry(retryStrategy, () => backend[idx].op(...))` so transient errors get the configured retry inside a single backend before falling through.
|
||||
|
||||
### 3.2 `round-robin`
|
||||
|
||||
Atomic counter modulo `healthyBackends.length`. On `ALL_BACKENDS_DOWN`, returns the failover error. Down backends skipped via `BackendHealth.isUp(name)`.
|
||||
|
||||
### 3.3 `sticky-by-key`
|
||||
|
||||
```ts
|
||||
const h = stableHash(key) % healthyBackends.length;
|
||||
return healthyBackends[h];
|
||||
```
|
||||
|
||||
Hash is FNV-1a over UTF-8 bytes (universal, no crypto dep). When a backend goes down, the modulo redistributes — same key may move to a different backend; this is acceptable for cache workloads and documented.
|
||||
|
||||
### 3.4 `shield`
|
||||
|
||||
```text
|
||||
read: backend[0] (origin) ?? backend[1..n] (edge mirrors) ── first success wins
|
||||
write: Promise.allSettled(backend[0..n].set(...))
|
||||
├── all ok ─► return
|
||||
├── primary ok, mirror failed ─► return + emit WRITE_FANOUT_PARTIAL event
|
||||
└── primary failed ─► throw (cause = primary error)
|
||||
```
|
||||
|
||||
Distinct from `failover` because `shield` reads **must** hit origin first (origin is canonical), and writes **must** fan out (mirrors only serve what was written to them). `failover` is symmetric — any backend can serve any operation.
|
||||
|
||||
## 4. Error classification
|
||||
|
||||
Which errors trigger failover vs surface directly:
|
||||
|
||||
| Error shape | Classification | Triggers fall-through? | Code surfaced |
|
||||
|---|---|---|---|
|
||||
| `AbortError` / `signal.aborted` | non-transient | NO | `STORAGE_MUX.BACKEND_ABORTED` |
|
||||
| `STORAGE.KEY_NOT_FOUND` (from `get`/`stat`) | semantic | NO | rethrow as-is |
|
||||
| `STORAGE.INVALID_KEY` | semantic | NO | rethrow as-is |
|
||||
| Network timeout (configurable `backendTimeoutMs`) | transient | YES | wrapped, last in cause chain |
|
||||
| `ECONNRESET` / `ETIMEDOUT` / fetch network error | transient | YES | wrapped |
|
||||
| HTTP 5xx from backend | transient | YES | wrapped |
|
||||
| HTTP 4xx (except 408/429) | non-transient | NO | rethrow |
|
||||
| HTTP 408 / 429 | transient | YES | wrapped |
|
||||
| `STORAGE_CDN.CREDENTIAL_EXPIRED` | non-transient | NO | rethrow (caller must rotate) |
|
||||
| Unknown thrown value (non-Error) | transient | YES | wrapped |
|
||||
|
||||
Classification lives in `src/policies/classify.ts`, pure function over `(error: unknown) => "transient" | "semantic" | "abort"`. Tested independently.
|
||||
|
||||
## 5. Backend-isolation guarantees
|
||||
|
||||
A slow primary MUST NOT block a fallback. Mechanism:
|
||||
|
||||
1. **Per-backend timeout** — every backend call wrapped in `Promise.race([call, timeoutPromise(backendTimeoutMs)])`. Default 30 s, configurable per-mux.
|
||||
2. **AbortSignal forwarding** — the per-attempt signal is linked (`AbortSignal.any([userSignal, timeoutSignal])`) and passed to the backend. A well-behaved backend respects it; a misbehaved backend will eventually reject via the timeout race.
|
||||
3. **`shield` reads cascade sequentially with timeout** — origin gets `backendTimeoutMs`; on timeout/error, the mux moves to mirror[0], etc. The next backend's call starts before the previous backend's underlying promise has settled (it is left to its own cleanup).
|
||||
4. **`shield` writes are `Promise.allSettled`** — one slow mirror cannot starve the others. Caller sees the result when all settle OR `backendTimeoutMs * 2` elapses (guard against a permanently-hung mirror).
|
||||
|
||||
The mux NEVER waits for a hung promise indefinitely — `Promise.race` with a clock-driven timeout is the contract.
|
||||
|
||||
## 6. Health tracking
|
||||
|
||||
`BackendHealth` per-backend state:
|
||||
|
||||
```ts
|
||||
interface BackendState {
|
||||
status: "up" | "down";
|
||||
consecutiveFailures: number;
|
||||
cooldownUntil: number; // Clock.now() millis
|
||||
lastError?: unknown;
|
||||
}
|
||||
```
|
||||
|
||||
Transitions:
|
||||
|
||||
- **up → down**: triggered by any classified-transient error. `cooldownUntil = now + cooldownMs * 2 ** min(consecutiveFailures, 6)` (exponential capped at 64× base). Emits `storage-mux.backend.down`.
|
||||
- **down → up**: triggered when `now >= cooldownUntil` AND the next selection call probes the backend AND the probe succeeds. Resets `consecutiveFailures`. Emits `storage-mux.backend.recovered`.
|
||||
- **active healthcheck**: if `BackendEntry.healthcheck` is provided AND `healthcheckIntervalMs` is set, the mux schedules `Clock.setTimeout` to invoke it; failure marks the backend down without waiting for a real request.
|
||||
|
||||
`Clock` is injected (`@suaveplan/types#Clock`), so tests use `FakeClock` for deterministic cooldown / healthcheck advancement.
|
||||
|
||||
## 7. Tier purity
|
||||
|
||||
Universal — no `node:*`, no `bun:*`, no DOM. Hash uses pure-JS FNV-1a. Timeouts use injected `Clock`. AbortSignal is standard Web API. Conformance harness runs under Node, Bun, browsers, Workers.
|
||||
|
||||
## 8. Trade-offs
|
||||
|
||||
- **Strategy classes vs. switch statement.** Chose strategy classes (`BackendSelector` interface). Each policy is a separate file ≤ 250 lines, individually tested, and new policies (e.g. `weighted-random` later) plug in without editing the core `StorageMux`.
|
||||
- **FNV-1a vs. xxhash for `sticky-by-key`.** FNV-1a is 20 lines of pure JS with no dep; xxhash needs a wasm or noble dep. For backend selection (low-cardinality output), FNV's distribution is sufficient. If a future workload needs better distribution, the hash function is injectable via options.
|
||||
- **`shield` writes fan out vs. primary-only.** Fan-out is the right default because mirrors that don't see writes can't serve reads. Callers who want primary-only writes should use `failover` instead — the policy choice is the API.
|
||||
- **Recursive Mux.** Allowed because `StorageMux` is itself a `StorageLike`. The conformance harness verifies recursion does not break invariants. The cost — debugging a deeply-nested mux tree — is mitigated by per-call OTel span attributes naming every backend hop.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Add @suaveplan/storage-mux — multi-backend storage failover / shielding mux
|
||||
|
||||
**Package**: `@suaveplan/storage-mux`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (genesis-original)
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/storage-mux/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
`@suaveplan/storage-cdn` (archived 2026-05-16) explicitly deferred origin shielding and multi-origin failover as out-of-scope, naming `storage-mux` as the future home (`openspec/archive/2026-05-16-add-storage-cdn/proposal.md` "Out of scope": *"Origin shielding / multi-origin failover. A failover/shielding layer is a separate concern composed via `storage-mux` (future) or application logic."*). The deferral was correct — failover semantics are orthogonal to CDN-specific surface (purge/signing/edgeUrl) and belong in a composable layer over the bare `AbstractStorage` contract.
|
||||
|
||||
`storage-mux` is a **universal-tier composition layer**: it accepts an ordered list of `StorageLike` backends and a policy (`failover` / `round-robin` / `sticky-by-key` / `shield`) and exposes a single `StorageLike` to consumers. Backends can be any storage adapter — `storage-cdn-*`, `storage-s3`, `storage-http`, `storage-file`, in-memory — making the mux the canonical primitive for CDN failover, origin shielding, mirror reads, and shard distribution.
|
||||
|
||||
### ≥2-consumer justification (per `openspec/AGENTS.md` §0 step 6)
|
||||
|
||||
Three forward-declared scenarios consume `storage-mux` directly:
|
||||
|
||||
1. **CDN failover for user-uploaded content** — primary `storage-cdn-bunnycdn` with `storage-cdn-rackspace` fallback. When BunnyCDN edge degrades, reads transparently fall through. The `failover` policy.
|
||||
2. **Cache mirroring for `storage-plugin-cache`** — local `storage-file` cache layered in front of remote `storage-s3` origin via the `shield` policy. Hits the fast tier first, falls through to origin on miss, writes to both.
|
||||
3. **Drive chunk shielding** — `drive-plugin-chunks` reads chunks from a regional mirror set with `sticky-by-key` so the same chunk always hits the same backend for cache warmth, plus failover when a mirror is down.
|
||||
|
||||
All three scenarios share the same primitive (ordered backend list + policy + health), differing only in policy. Without a shared mux, each consumer reinvents the failover state machine and the registry of `STORAGE_MUX.*` codes would be scattered.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `StorageMux` class extending `AbstractStorage` from `@suaveplan/storage`. Wraps an ordered backend list and a policy and exposes the full `StorageLike` surface (`get`, `set`, `setPartial`, `has`, `delete`, `list`, `stat`, `getStream`, `setStream`).
|
||||
- ADDED: `MuxPolicy` discriminated union — `"failover" | "round-robin" | "sticky-by-key" | "shield"`. Each policy is implemented as a strategy class behind a common `BackendSelector` interface.
|
||||
- ADDED: `BackendEntry` interface — `{ name: string; storage: StorageLike; weight?: number; healthcheck?: (signal: AbortSignal) => Promise<void> }`.
|
||||
- ADDED: `BackendHealth` tracker — passive degrade (mark down on transient error, exponential cooldown via injected `Clock`) + optional active polling via `healthcheck` callbacks scheduled on `Clock`.
|
||||
- ADDED: `StorageMuxOptions` — `{ backends: BackendEntry[]; policy: MuxPolicy; retry?: RetryStrategy; clock?: Clock; events?: EventBusLike; logger?: LoggerLike; tracer?: TracerLike; cooldownMs?: number; healthcheckIntervalMs?: number }`.
|
||||
- ADDED: `STORAGE_MUX_CODES` via `defineCodes("STORAGE_MUX", [...])` — 8 codes: `ALL_BACKENDS_DOWN`, `NO_BACKENDS_CONFIGURED`, `HEALTHCHECK_FAILED`, `BACKEND_TIMEOUT`, `INVALID_POLICY`, `INVALID_WEIGHT`, `WRITE_FANOUT_PARTIAL`, `BACKEND_ABORTED`.
|
||||
- ADDED: `StorageMuxError extends SuaveplanError<StorageMuxErrorCode>` with `backendName`, `attemptIndex`, `attemptCount` discriminants.
|
||||
- ADDED: Retry composition — every backend call routed through `@suaveplan/retry` (`runWithRetry` + injected strategy). The mux never reimplements backoff; it composes.
|
||||
- ADDED: AbortSignal threaded through every backend call. A caller-aborted operation must surface `BACKEND_ABORTED`, not `ALL_BACKENDS_DOWN`.
|
||||
- ADDED: Observability — `EventBusLike` emissions on backend down/recovery/failover; OTel span attributes per call (`mux.backend.name`, `mux.attempt.index`, `mux.policy`).
|
||||
- ADDED: Conformance harness integration — `runStorageConformance` (from `@suaveplan/storage`) executed against a `StorageMux` configured with N `MemoryStorage` backends and each policy.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Provider-specific backends.** The mux composes `StorageLike` values handed in by the caller; it does not instantiate CDN clients or know provider authentication.
|
||||
- **Cross-backend replication / consistency.** Writes under `shield` are best-effort fan-out with `WRITE_FANOUT_PARTIAL` surfacing partial failures — there is no two-phase commit, no quorum, no vector clocks. Stronger consistency is `@suaveplan/vault-sync` territory.
|
||||
- **Geo-aware routing.** Backend selection is by ordered list + policy. Geographic affinity belongs in a higher-level router that constructs the backend list per-request.
|
||||
- **Adaptive policy switching.** The policy is fixed at construction; the mux never silently switches `failover` → `round-robin` based on load.
|
||||
|
||||
## Source mapping
|
||||
|
||||
Genesis-original. No upstream in `../suaveplan/`. The design draws from:
|
||||
|
||||
- `openspec/archive/2026-05-16-add-storage-cdn/proposal.md` — the explicit deferral naming `storage-mux`.
|
||||
- Industry conventions: HAProxy / Envoy passive+active health checks; HashiCorp Consul service-mesh failover; AWS Route 53 health-check failover routing; NGINX `upstream` `max_fails` / `fail_timeout`. The four policies (`failover`, `round-robin`, `sticky-by-key`, `shield`) are the union of what those tools offer in the storage-IO dimension.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Pluggable `RetryStrategy` via `@suaveplan/retry` | Reimplementing retry in the mux would duplicate `retry`'s state machine and diverge from the rest of the monorepo. Composition keeps the contract uniform. |
|
||||
| Backend isolation (slow primary cannot block fallback) | A timeout-aware race or per-backend abort is required because a hung primary that never rejects starves the failover chain. The design uses `Promise.race` against a clock-driven timeout, not a parallel call — parallel-call shielding (`shield` writes) is opt-in. |
|
||||
| `WRITE_FANOUT_PARTIAL` distinct from `ALL_BACKENDS_DOWN` | `shield` writes intentionally fan out to mirrors; one mirror down does not fail the operation, but the caller deserves a signal. A separate code lets callers choose to surface or swallow it. |
|
||||
| `sticky-by-key` uses a stable hash, not a Map cache | Cache memory grows unbounded under high-cardinality keys. Hashing keeps the policy O(1) memory regardless of key population. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
- `src/mux/mux.test.ts` — construction validates non-empty backend list, policy whitelist, weights.
|
||||
- `src/policies/failover/failover.test.ts` — fall-through on transient error; surface non-transient; respect AbortSignal; `ALL_BACKENDS_DOWN` after exhausting list.
|
||||
- `src/policies/round-robin/round-robin.test.ts` — even distribution under N calls; skip down backends.
|
||||
- `src/policies/sticky-by-key/sticky-by-key.test.ts` — same key → same backend across calls; redistribute when backend down.
|
||||
- `src/policies/shield/shield.test.ts` — reads hit primary first then mirrors; writes fan out; `WRITE_FANOUT_PARTIAL` on partial mirror failure.
|
||||
- `src/health/health.test.ts` — passive degrade + exponential cooldown; active polling via injected `Clock`; recovery emits event.
|
||||
- `src/mux/mux.spec-conformance.test.ts` — `runStorageConformance` passes with 3× `MemoryStorage` backends under every policy.
|
||||
- `src/error-codes/error-codes.test.ts` — every `STORAGE_MUX_*` code reachable from a real call site.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/storage-mux` to enumerate. The package is new — no current dependents.
|
||||
|
||||
- No current dependents (verified by `bunx turbo ls --filter=...@suaveplan/storage-mux` returning the package itself).
|
||||
- Forward-declared consumers: `@suaveplan/storage-plugin-cache` (shield mode), `@suaveplan/drive-plugin-chunks` (sticky-by-key), application code composing `storage-cdn-bunnycdn` + `storage-cdn-rackspace` (failover).
|
||||
|
||||
### Affected upstreams
|
||||
|
||||
- `@suaveplan/storage` — consumed via `AbstractStorage` + `StorageLike` + `runStorageConformance`. No augmentation. [x] no action needed.
|
||||
- `@suaveplan/retry` — consumed via `RetryStrategy` + `runWithRetry`. No augmentation. [x] no action needed.
|
||||
- `@suaveplan/error`, `@suaveplan/events`, `@suaveplan/types` — standard foundation deps. [x] no action needed.
|
||||
@@ -0,0 +1,256 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: storage-mux (storage-mux)
|
||||
|
||||
**Package**: `@suaveplan/storage-mux`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: StorageMux extends AbstractStorage
|
||||
|
||||
The package SHALL provide a `StorageMux` class extending `AbstractStorage` from `@suaveplan/storage`. The class SHALL accept `StorageMuxOptions` containing an ordered non-empty `backends: BackendEntry[]` and a `policy: MuxPolicy`. The class SHALL implement every method of the `StorageLike` surface (`get`, `set`, `setPartial`, `has`, `delete`, `list`, `stat`, `getStream`, `setStream`) by delegating to a `BackendSelector` chosen from the configured policy.
|
||||
|
||||
#### Scenario: subclass is an instance of AbstractStorage
|
||||
|
||||
- **WHEN** a `StorageMux` is constructed with at least one `BackendEntry`
|
||||
- **THEN** the instance is `instanceof AbstractStorage` AND exposes every `StorageLike` method
|
||||
|
||||
#### Scenario: empty backend list is rejected at construction
|
||||
|
||||
- **WHEN** a caller invokes `new StorageMux({ backends: [], policy: "failover" })`
|
||||
- **THEN** the constructor throws `StorageMuxError` with `code === "STORAGE_MUX.NO_BACKENDS_CONFIGURED"`
|
||||
|
||||
**Verified by:** `src/mux/mux.test.ts::"is instance of AbstractStorage and exposes StorageLike surface"` AND `src/mux/mux.test.ts::"rejects empty backend list with NO_BACKENDS_CONFIGURED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MuxPolicy discriminated union with four policies
|
||||
|
||||
The package SHALL declare `MuxPolicy` as the string union `"failover" | "round-robin" | "sticky-by-key" | "shield"`. Each policy SHALL be implemented as a `BackendSelector` strategy class in its own subdirectory under `src/policies/`. The `StorageMuxOptions.policy` field SHALL be validated by a Zod schema that rejects any value outside this whitelist with `STORAGE_MUX.INVALID_POLICY`.
|
||||
|
||||
#### Scenario: unknown policy string is rejected
|
||||
|
||||
- **WHEN** a caller invokes `new StorageMux({ backends, policy: "magic" as MuxPolicy })`
|
||||
- **THEN** the constructor throws `StorageMuxError` with `code === "STORAGE_MUX.INVALID_POLICY"`
|
||||
|
||||
**Verified by:** `src/mux/mux.test.ts::"rejects unknown policy with INVALID_POLICY"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: failover policy falls through on transient errors and surfaces semantic errors
|
||||
|
||||
Under the `failover` policy the mux SHALL invoke `backends[0]` first. On a transient classified error (per the error-classification table in `design.md` §4) the mux SHALL advance to `backends[1]`, then `backends[2]`, and so on. On a semantic error (`STORAGE.KEY_NOT_FOUND`, `STORAGE.INVALID_KEY`, 4xx other than 408/429, `CREDENTIAL_EXPIRED`) the mux SHALL surface the error directly without advancing. When every backend has been tried and the last attempt was transient, the mux SHALL throw `StorageMuxError` with `code === "STORAGE_MUX.ALL_BACKENDS_DOWN"`, the prior error as `cause`, and `attemptCount` equal to the number of backends tried.
|
||||
|
||||
#### Scenario: transient error advances to next backend
|
||||
|
||||
- **WHEN** `backends[0].get(key)` rejects with a 503 and `backends[1].get(key)` resolves with bytes
|
||||
- **THEN** the mux returns `backends[1]`'s bytes AND emits a `storage-mux.failover` event naming both backends
|
||||
|
||||
#### Scenario: semantic KEY_NOT_FOUND is not retried across backends
|
||||
|
||||
- **WHEN** `backends[0].get(key)` throws `STORAGE.KEY_NOT_FOUND`
|
||||
- **THEN** the mux rethrows `KEY_NOT_FOUND` without invoking `backends[1]`
|
||||
|
||||
#### Scenario: exhausting all backends throws ALL_BACKENDS_DOWN with cause chain
|
||||
|
||||
- **WHEN** every backend rejects with a transient error
|
||||
- **THEN** the mux throws `StorageMuxError` with `code === "STORAGE_MUX.ALL_BACKENDS_DOWN"` AND `cause` is the last backend's error AND `attemptCount === backends.length`
|
||||
|
||||
**Verified by:** `src/policies/failover/failover.test.ts::"falls through to next backend on transient 503"` AND `src/policies/failover/failover.test.ts::"surfaces semantic KEY_NOT_FOUND without fall-through"` AND `src/policies/failover/failover.test.ts::"throws ALL_BACKENDS_DOWN after exhausting list"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: round-robin policy distributes across healthy backends
|
||||
|
||||
Under the `round-robin` policy the mux SHALL maintain a monotonic counter and select `healthyBackends[counter mod healthyBackends.length]` for each call. Backends marked down by `BackendHealth` SHALL be skipped. When zero backends are healthy the mux SHALL throw `STORAGE_MUX.ALL_BACKENDS_DOWN`.
|
||||
|
||||
#### Scenario: ten calls across three healthy backends distribute evenly
|
||||
|
||||
- **WHEN** the caller issues 9 `get` calls against a mux with 3 healthy backends
|
||||
- **THEN** each backend receives exactly 3 calls
|
||||
|
||||
#### Scenario: down backend is skipped
|
||||
|
||||
- **WHEN** `backends[1]` is marked down and the caller issues 4 calls
|
||||
- **THEN** `backends[0]` and `backends[2]` each receive 2 calls AND `backends[1]` receives 0
|
||||
|
||||
**Verified by:** `src/policies/round-robin/round-robin.test.ts::"distributes nine calls evenly across three backends"` AND `src/policies/round-robin/round-robin.test.ts::"skips backends marked down"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: sticky-by-key policy is deterministic and skips down backends
|
||||
|
||||
Under the `sticky-by-key` policy the mux SHALL compute `index = fnv1a(utf8(key)) mod healthyBackends.length` and route every operation for that key to the same backend. The hash SHALL be pure-JS FNV-1a (no native or crypto dep). When a backend goes down its keys SHALL be redistributed across the remaining healthy backends via the same formula recomputed against the shrunken set.
|
||||
|
||||
#### Scenario: same key resolves to same backend across 100 calls
|
||||
|
||||
- **WHEN** the caller issues 100 `get("alpha")` calls against a mux with 3 healthy backends
|
||||
- **THEN** exactly one backend receives all 100 calls
|
||||
|
||||
#### Scenario: removed backend triggers redistribution
|
||||
|
||||
- **WHEN** the backend currently serving `"alpha"` is marked down
|
||||
- **THEN** subsequent `get("alpha")` calls land on one of the remaining healthy backends AND the choice is stable until the health set changes again
|
||||
|
||||
**Verified by:** `src/policies/sticky-by-key/sticky-by-key.test.ts::"routes the same key to the same backend across 100 calls"` AND `src/policies/sticky-by-key/sticky-by-key.test.ts::"redistributes keys when a backend goes down"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: shield policy cascades reads and fans out writes
|
||||
|
||||
Under the `shield` policy the mux SHALL treat `backends[0]` as the origin and `backends[1..n]` as mirrors. For read operations (`get`, `has`, `stat`, `getStream`) the mux SHALL invoke `backends[0]` first and on transient error or timeout fall through to mirrors in order. For write operations (`set`, `setPartial`, `delete`, `setStream`) the mux SHALL invoke `backends[0]` AND every mirror via `Promise.allSettled`. If the origin write rejects the mux SHALL throw with the origin error as `cause`. If the origin write succeeds but one or more mirror writes reject the mux SHALL emit `storage-mux.fanout-partial` carrying every failed mirror name AND return successfully (the operation is considered durable once the origin accepts it). The mux SHALL additionally surface a `StorageMuxError` with `code === "STORAGE_MUX.WRITE_FANOUT_PARTIAL"` when the caller opts in via `StorageMuxOptions.throwOnFanoutPartial === true`.
|
||||
|
||||
#### Scenario: read falls through to mirror after origin timeout
|
||||
|
||||
- **WHEN** `backends[0].get(key)` hangs past `backendTimeoutMs` and `backends[1].get(key)` resolves with bytes
|
||||
- **THEN** the mux returns `backends[1]`'s bytes
|
||||
|
||||
#### Scenario: write succeeds when origin accepts and mirror rejects
|
||||
|
||||
- **WHEN** `backends[0].set` resolves and `backends[1].set` rejects with a transient error
|
||||
- **THEN** the mux returns successfully AND emits `storage-mux.fanout-partial` naming `backends[1]`
|
||||
|
||||
#### Scenario: write fails when origin rejects
|
||||
|
||||
- **WHEN** `backends[0].set` rejects with any error
|
||||
- **THEN** the mux throws AND `cause` is the origin error AND mirror outcomes are not consulted
|
||||
|
||||
**Verified by:** `src/policies/shield/shield.test.ts::"reads fall through to mirror after origin timeout"` AND `src/policies/shield/shield.test.ts::"write succeeds when origin accepts and one mirror rejects"` AND `src/policies/shield/shield.test.ts::"write fails when origin rejects regardless of mirror outcomes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: backend isolation — slow primary cannot block fallback
|
||||
|
||||
The mux SHALL wrap every backend call in a `Promise.race` against a `Clock`-driven timeout of `backendTimeoutMs` (default 30000, configurable per-mux). When the race rejects due to timeout the mux SHALL classify the timeout as transient AND proceed to the next backend per policy. The underlying backend promise MAY remain pending; the mux SHALL NOT await it. The mux SHALL link the user `AbortSignal` and the timeout signal via `AbortSignal.any` and pass the combined signal to every backend call so well-behaved backends abort promptly.
|
||||
|
||||
#### Scenario: hung primary does not delay fallback beyond backendTimeoutMs
|
||||
|
||||
- **WHEN** `backends[0].get` returns a never-resolving promise and `backends[1].get` resolves in 10ms
|
||||
- **THEN** the caller receives `backends[1]`'s value in less than `backendTimeoutMs + 50ms` measured against the injected `FakeClock`
|
||||
|
||||
**Verified by:** `src/mux/mux.test.ts::"hung primary does not block fallback past backendTimeoutMs"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AbortSignal propagation surfaces BACKEND_ABORTED
|
||||
|
||||
When the caller's `AbortSignal` aborts during a mux operation, the mux SHALL surface a `StorageMuxError` with `code === "STORAGE_MUX.BACKEND_ABORTED"` rather than falling through to subsequent backends. The mux SHALL NOT classify a caller-driven abort as a transient backend failure.
|
||||
|
||||
#### Scenario: caller abort surfaces BACKEND_ABORTED and does not advance
|
||||
|
||||
- **WHEN** the caller passes a pre-aborted signal to `mux.get(key, signal)`
|
||||
- **THEN** the mux throws `StorageMuxError` with `code === "STORAGE_MUX.BACKEND_ABORTED"` AND `backends[1].get` is NEVER invoked
|
||||
|
||||
**Verified by:** `src/mux/mux.test.ts::"caller abort surfaces BACKEND_ABORTED without falling through"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: BackendHealth — passive degrade with exponential cooldown
|
||||
|
||||
The package SHALL maintain a `BackendHealth` tracker per backend. A backend SHALL transition from `up` to `down` on the first classified-transient error. The cooldown SHALL be `cooldownMs * 2 ** min(consecutiveFailures, 6)` (exponential, capped at 64× base, default `cooldownMs` 1000). A `down` backend SHALL be skipped by every selector. Once `Clock.now() >= cooldownUntil` the next operation SHALL probe the backend; success transitions it back to `up` and resets `consecutiveFailures`.
|
||||
|
||||
#### Scenario: failing backend cools down for 2000ms after second failure
|
||||
|
||||
- **WHEN** a backend rejects twice in succession (default `cooldownMs` 1000)
|
||||
- **THEN** the backend is `down` AND remains skipped until `FakeClock` advances by 2000ms
|
||||
|
||||
#### Scenario: successful probe after cooldown restores backend and emits recovered event
|
||||
|
||||
- **WHEN** `FakeClock` advances past `cooldownUntil` AND the next probe succeeds
|
||||
- **THEN** the backend is `up` AND a `storage-mux.backend.recovered` event is emitted AND `consecutiveFailures` is reset to 0
|
||||
|
||||
**Verified by:** `src/health/health.test.ts::"exponential cooldown after consecutive failures"` AND `src/health/health.test.ts::"successful probe after cooldown restores backend and emits recovered"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Optional active healthchecks via injected Clock
|
||||
|
||||
When `BackendEntry.healthcheck` is provided AND `StorageMuxOptions.healthcheckIntervalMs` is set, the mux SHALL schedule the healthcheck via the injected `Clock.setTimeout` (or `Clock.setInterval`) and invoke it with an `AbortSignal`. A rejecting healthcheck SHALL mark the backend `down` AND emit `storage-mux.backend.down` carrying `STORAGE_MUX.HEALTHCHECK_FAILED` as the cause. The mux SHALL stop scheduling when the mux is disposed.
|
||||
|
||||
#### Scenario: rejecting active healthcheck marks backend down
|
||||
|
||||
- **WHEN** an active healthcheck rejects with a network error
|
||||
- **THEN** the backend transitions to `down` AND a `storage-mux.backend.down` event is emitted AND the failure cause is `STORAGE_MUX.HEALTHCHECK_FAILED`
|
||||
|
||||
**Verified by:** `src/health/health.test.ts::"rejecting active healthcheck marks backend down with HEALTHCHECK_FAILED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Retry composition via @suaveplan/retry
|
||||
|
||||
Every per-backend invocation SHALL be wrapped by `runWithRetry(retryStrategy, ...)` from `@suaveplan/retry` using the strategy supplied in `StorageMuxOptions.retry` (defaulting to `exponentialBackoff()` from `@suaveplan/retry/strategies/exponential/exponential`). The mux SHALL NOT reimplement backoff, jitter, or attempt counting; all retry behavior MUST be expressed by composing the retry strategy.
|
||||
|
||||
#### Scenario: backend that fails twice then succeeds is retried in place before fall-through
|
||||
|
||||
- **WHEN** `backends[0].get` rejects twice then resolves on the third attempt under a 3-attempt retry strategy
|
||||
- **THEN** the mux returns `backends[0]`'s value AND `backends[1].get` is NEVER invoked
|
||||
|
||||
**Verified by:** `src/policies/failover/failover.test.ts::"retries in place before falling through to next backend"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: STORAGE_MUX error registry — 8 codes reachable
|
||||
|
||||
The package SHALL declare `STORAGE_MUX_CODES` via `defineCodes("STORAGE_MUX", [...])` with exactly these 8 codes: `ALL_BACKENDS_DOWN`, `NO_BACKENDS_CONFIGURED`, `HEALTHCHECK_FAILED`, `BACKEND_TIMEOUT`, `INVALID_POLICY`, `INVALID_WEIGHT`, `WRITE_FANOUT_PARTIAL`, `BACKEND_ABORTED`. The package SHALL declare `StorageMuxError extends SuaveplanError<StorageMuxErrorCode>` with optional `backendName`, `attemptIndex`, and `attemptCount` discriminants. Every code SHALL be reachable from a real call site exercised by tests in this package.
|
||||
|
||||
#### Scenario: every declared code is reachable from a real call site
|
||||
|
||||
- **WHEN** the test suite is run
|
||||
- **THEN** every code in `STORAGE_MUX_CODES` has at least one test that exercises the call site that throws or carries it
|
||||
|
||||
**Verified by:** `src/error-codes/error-codes.test.ts::"every STORAGE_MUX code is reachable from a real call site"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Observability — EventBusLike events on lifecycle transitions
|
||||
|
||||
The mux SHALL emit events through an injected `EventBusLike` (from `@suaveplan/types`) on the following lifecycle transitions: `storage-mux.backend.down` (passive or active failure), `storage-mux.backend.recovered` (probe success after cooldown), `storage-mux.failover` (failover policy advances to a new backend), `storage-mux.fanout-partial` (shield-policy mirror write rejected while origin succeeded). When no `EventBusLike` is provided the mux SHALL operate normally without emitting (silent fallback).
|
||||
|
||||
#### Scenario: failover emits storage-mux.failover with both backend names
|
||||
|
||||
- **WHEN** the failover policy advances from `backends[0]` to `backends[1]` due to a 503
|
||||
- **THEN** exactly one `storage-mux.failover` event is emitted carrying both backend names AND the underlying error
|
||||
|
||||
**Verified by:** `src/observability/events.test.ts::"failover emits storage-mux.failover with from and to backend names"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Observability — OTel span attributes per backend call
|
||||
|
||||
Every backend call SHALL be wrapped in an OTel span carrying the attributes `mux.backend.name` (string), `mux.attempt.index` (number, 0-based), and `mux.policy` (string). When no tracer is provided (`TracerLike` from `@suaveplan/types`) the mux SHALL operate normally without recording spans.
|
||||
|
||||
#### Scenario: failover attempt records two spans with attempt indices 0 and 1
|
||||
|
||||
- **WHEN** the failover policy invokes `backends[0]` (which rejects) then `backends[1]` (which succeeds)
|
||||
- **THEN** the test tracer records two spans with `mux.attempt.index` equal to `0` and `1` respectively AND matching `mux.backend.name`
|
||||
|
||||
**Verified by:** `src/observability/otel.test.ts::"records one span per backend attempt with attempt index and backend name"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: StorageLike conformance under every policy
|
||||
|
||||
The mux SHALL pass `runStorageConformance` (from `@suaveplan/storage`) when configured with three `MemoryStorage` backends under each of the four policies (`failover`, `round-robin`, `sticky-by-key`, `shield`). The conformance suite SHALL include `setPartial` sparse-extension and overwrite semantics, capability-honest reporting, and AbortSignal propagation.
|
||||
|
||||
#### Scenario: conformance suite passes under failover, round-robin, sticky-by-key, and shield
|
||||
|
||||
- **WHEN** `runStorageConformance(mux)` is invoked four times — once per policy — against the same three-MemoryStorage backend set
|
||||
- **THEN** every conformance assertion passes for every policy
|
||||
|
||||
**Verified by:** `src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under failover"` AND `src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under round-robin"` AND `src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under sticky-by-key"` AND `src/mux/mux.spec-conformance.test.ts::"passes runStorageConformance under shield"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Tier purity — universal, no Node or DOM imports
|
||||
|
||||
The package SHALL NOT import from `node:*`, `bun:*`, or any DOM-only API. Hashing SHALL be pure-JS (FNV-1a). Timeouts SHALL use the injected `Clock`. The package SHALL run unmodified under Node 18+, Bun, Deno, modern browsers, and Cloudflare Workers.
|
||||
|
||||
#### Scenario: tier-purity grep finds no Node or DOM imports
|
||||
|
||||
- **WHEN** `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-mux/src/` is run
|
||||
- **THEN** the output is empty
|
||||
|
||||
**Verified by:** `src/mux/mux.spec-conformance.test.ts::"src tree contains no node: or bun: imports"`
|
||||
@@ -0,0 +1,92 @@
|
||||
# Tasks — storage-mux
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `openspec/specs/storage/spec.md` and confirm `AbstractStorage` surface
|
||||
- [ ] 1.2 Read `openspec/AGENTS.md` §0c, §13, §14
|
||||
- [ ] 1.3 Read archived `openspec/archive/2026-05-16-add-storage-cdn/` for the deferral and family conventions
|
||||
- [ ] 1.4 Read `packages/flow/retry/src/retry/retry.ts` to confirm `runWithRetry` + `RetryStrategy` surface
|
||||
- [ ] 1.5 Draft `proposal.md`, `design.md`, `specs/storage-mux/spec.md`, `tasks.md`
|
||||
- [ ] 1.6 Numbered Requirement/Scenario checklist per §0c step 6
|
||||
|
||||
## 2. Scaffolding
|
||||
|
||||
- [ ] 2.1 Create `packages/storage/storage-mux/` mirroring `packages/storage/storage-cdn/` conventions
|
||||
- [ ] 2.2 `package.json` v0.0.0, `"sideEffects": false`, subpath exports per feature
|
||||
- [ ] 2.3 Deps: `@suaveplan/error`, `@suaveplan/events`, `@suaveplan/retry`, `@suaveplan/storage`, `@suaveplan/types`, `@opentelemetry/api`, `zod`, `tslib`
|
||||
- [ ] 2.4 DevDeps: `@suaveplan/testing`, `@vitest/coverage-istanbul`, `fast-check`, `mitata`, `typescript`, `vite`, `vite-plugin-dts`, `vitest`
|
||||
- [ ] 2.5 `tsconfig`, `vite.config.ts`, `vitest.config.ts` via `createTestConfig`
|
||||
- [ ] 2.6 `README.md` root with Modules section linking every `.md`
|
||||
|
||||
## 3. Error registry
|
||||
|
||||
- [ ] 3.1 `src/error-codes.ts` — `defineCodes("STORAGE_MUX", [...])` with 8 codes (`ALL_BACKENDS_DOWN`, `NO_BACKENDS_CONFIGURED`, `HEALTHCHECK_FAILED`, `BACKEND_TIMEOUT`, `INVALID_POLICY`, `INVALID_WEIGHT`, `WRITE_FANOUT_PARTIAL`, `BACKEND_ABORTED`)
|
||||
- [ ] 3.2 `src/error-codes.ts` — `StorageMuxError extends SuaveplanError<StorageMuxErrorCode>` with `backendName`, `attemptIndex`, `attemptCount` discriminants
|
||||
|
||||
## 4. Types & schemas
|
||||
|
||||
- [ ] 4.1 `src/types/backend-entry.types.ts` — `BackendEntry` interface (`name`, `storage`, `weight?`, `healthcheck?`)
|
||||
- [ ] 4.2 `src/types/mux-options.types.ts` — `MuxPolicy` discriminated string union, `StorageMuxOptions`
|
||||
- [ ] 4.3 `src/types/mux-options.schemas.ts` — Zod schemas validating non-empty backends, policy whitelist, weight ≥ 0
|
||||
|
||||
## 5. Selectors / policies
|
||||
|
||||
- [ ] 5.1 `src/policies/selector.types.ts` — `BackendSelector` interface (`select(op, key?, healthy): BackendEntry`)
|
||||
- [ ] 5.2 `src/policies/classify.ts` + `.md` + `.test.ts` — pure error classification per design §4 table
|
||||
- [ ] 5.3 `src/policies/failover/failover.ts` + `.md` + `.test.ts` — fall-through + retry composition
|
||||
- [ ] 5.4 `src/policies/round-robin/round-robin.ts` + `.md` + `.test.ts` — atomic counter, skip down
|
||||
- [ ] 5.5 `src/policies/sticky-by-key/sticky-by-key.ts` + `.md` + `.test.ts` — FNV-1a hash, deterministic
|
||||
- [ ] 5.6 `src/policies/shield/shield.ts` + `.md` + `.test.ts` — read cascade, write fan-out, partial signal
|
||||
|
||||
## 6. Health
|
||||
|
||||
- [ ] 6.1 `src/health/health.ts` + `.md` + `.test.ts` — `BackendHealth` tracker, exponential cooldown, recovery probe, active healthcheck via injected `Clock`
|
||||
|
||||
## 7. Mux core
|
||||
|
||||
- [ ] 7.1 `src/mux/mux.ts` + `.md` — `StorageMux extends AbstractStorage`, per-op routing through `BackendSelector` + `runWithRetry`, per-backend timeout race, AbortSignal linking
|
||||
- [ ] 7.2 `src/mux/mux.test.ts` — construction validation, policy dispatch, abort surfacing
|
||||
- [ ] 7.3 `src/mux/mux.spec-conformance.test.ts` — `runStorageConformance` × 4 policies × 3 `MemoryStorage` backends
|
||||
|
||||
## 8. Observability
|
||||
|
||||
- [ ] 8.1 `src/observability/events.ts` + `.md` + `.test.ts` — event topic constants (`storage-mux.backend.down`, `.recovered`, `.failover`, `.fanout-partial`), `emit*` helpers consuming `EventBusLike`
|
||||
- [ ] 8.2 `src/observability/otel.ts` + `.md` + `.test.ts` — span attribute helpers (`mux.backend.name`, `mux.attempt.index`, `mux.policy`)
|
||||
|
||||
## 9. Barrel + exports
|
||||
|
||||
- [ ] 9.1 `src/index.ts` — explicit named re-exports per §6a (NO `export *`)
|
||||
- [ ] 9.2 `package.json` subpath exports match every feature directory
|
||||
|
||||
## 10. Tests
|
||||
|
||||
- [ ] 10.1 `src/error-codes/error-codes.test.ts` — every `STORAGE_MUX_*` code reachable from a real call site
|
||||
- [ ] 10.2 All test imports via `@suaveplan/testing/runner` (§7)
|
||||
- [ ] 10.3 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for async health)
|
||||
- [ ] 10.4 Backend isolation test: hung-promise primary does NOT block fallback within `backendTimeoutMs`
|
||||
|
||||
## 11. Documentation
|
||||
|
||||
- [ ] 11.1 Sibling `.md` for every implementation `.ts` (≥ 200 words, all required sections)
|
||||
- [ ] 11.2 `README.md` documents each policy with a usage example
|
||||
|
||||
## 12. Gates
|
||||
|
||||
- [ ] 12.1 `bun run check`
|
||||
- [ ] 12.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-mux`
|
||||
- [ ] 12.3 `bunx turbo test:coverage --filter=@suaveplan/storage-mux` (100/100/100/100)
|
||||
- [ ] 12.4 `bun scripts/lint/check-test-runner-imports.ts`
|
||||
- [ ] 12.5 Tier-purity grep `grep -rnE "from ['\"](node:|bun:)" packages/storage/storage-mux/src/` returns empty
|
||||
- [ ] 12.6 Banned-pattern scan returns empty
|
||||
- [ ] 12.7 `bun scripts/openspec-conformance.ts @suaveplan/storage-mux`
|
||||
|
||||
## 13. Manifest & archival
|
||||
|
||||
- [ ] 13.1 `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
- [ ] 13.2 `bun scripts/generate-missing.ts`
|
||||
- [ ] 13.3 Update `STATUS.md` and `SUMMARY.md`
|
||||
- [ ] 13.4 `bunx markdownlint-cli2 --fix`
|
||||
- [ ] 13.5 Fold `specs/storage-mux/spec.md` → `openspec/specs/storage-mux/spec.md`
|
||||
- [ ] 13.6 Move change folder → `openspec/archive/<yyyy-mm-dd>-add-storage-mux/`
|
||||
@@ -0,0 +1,249 @@
|
||||
# Design: add-storage-s3-multipart
|
||||
|
||||
Non-normative. The normative contract is `./specs/storage-s3/spec.md`.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Large writes (> 16 MiB by default, configurable) flow through S3 multipart
|
||||
without any caller code change.
|
||||
2. Streaming input (`ReadableStream<Uint8Array>`) is chunked into parts on the
|
||||
fly — no whole-stream buffering.
|
||||
3. Parts upload in bounded parallel for WAN throughput; failures cancel
|
||||
in-flight workers and clean up the open `UploadId`.
|
||||
4. Error provenance is preserved at stage granularity (init / part / complete
|
||||
/ abort) so callers and dashboards can attribute failures.
|
||||
5. Zero changes to `StorageLike`; multipart is invisible to consumers who don't
|
||||
need to know.
|
||||
|
||||
## Trigger threshold
|
||||
|
||||
S3 enforces a minimum part size of **5 MiB** (except the final part). Maximum
|
||||
parts per upload is **10 000**. Maximum object size is **5 TiB**.
|
||||
|
||||
The default trigger is **16 MiB**. Rationale:
|
||||
|
||||
- Below 16 MiB a single PUT is faster (one round trip, no MPU bookkeeping).
|
||||
- Above 16 MiB the latency of the three MPU control calls amortises against
|
||||
the parallelism win.
|
||||
- 16 MiB / 5 MiB part = 4 parts at the floor, which matches the default
|
||||
concurrency of 4 — every part can start in parallel.
|
||||
|
||||
Caller override via `options.multipart.threshold`. Minimum legal value is
|
||||
5 MiB (anything smaller would force the first part below the AWS floor).
|
||||
|
||||
## API surface
|
||||
|
||||
Public surface is unchanged. The new opt-in shape is added to `S3Options` (and
|
||||
to a per-call `StorageOptions`-shaped overlay for `set` only, so single-call
|
||||
overrides are possible without reconfiguring the adapter):
|
||||
|
||||
```ts
|
||||
export interface MultipartTuning {
|
||||
/** Bytes; default 16 MiB; minimum 5 MiB. */
|
||||
readonly threshold?: number;
|
||||
/** Bytes; default 8 MiB; minimum 5 MiB; maximum 5 GiB per AWS. */
|
||||
readonly partSize?: number;
|
||||
/** Concurrent part workers; default 4; minimum 1; maximum 32. */
|
||||
readonly concurrency?: number;
|
||||
}
|
||||
|
||||
// Added to S3Options
|
||||
readonly multipart?: MultipartTuning;
|
||||
```
|
||||
|
||||
`setMultipart` keeps its existing signature (it is already public) and becomes
|
||||
a thin wrapper over the new engine.
|
||||
|
||||
## Internal engine
|
||||
|
||||
One private method, `#runMultipart`, owns the entire CreateMPU → parallel-parts
|
||||
→ CompleteMPU lifecycle. Both `set` (when over threshold) and `setMultipart`
|
||||
delegate to it.
|
||||
|
||||
### Pseudo-shape
|
||||
|
||||
```ts
|
||||
type PartInput = { number: number; body: Uint8Array };
|
||||
|
||||
async #runMultipart(
|
||||
fullKey: string,
|
||||
parts: AsyncIterable<PartInput>,
|
||||
tuning: Required<MultipartTuning>,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<void> {
|
||||
const uploadId = await this.#initMultipart(fullKey, signal);
|
||||
const completed: { ETag: string; PartNumber: number }[] = [];
|
||||
try {
|
||||
await this.#runPartWorkers(
|
||||
fullKey, uploadId, parts, tuning.concurrency, completed, signal,
|
||||
);
|
||||
completed.sort((a, b) => a.PartNumber - b.PartNumber);
|
||||
await this.#completeMultipart(fullKey, uploadId, completed, signal);
|
||||
} catch (err) {
|
||||
await this.#abortMultipart(fullKey, uploadId).catch(/* logged */);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Part chunker
|
||||
|
||||
A generator `#chunkStream(stream, partSize)` reads from
|
||||
`ReadableStream<Uint8Array>` and yields `PartInput` records exactly `partSize`
|
||||
bytes long, plus a final shorter part. Memory ceiling is `partSize` + the
|
||||
single in-flight buffer from `reader.read()`.
|
||||
|
||||
For the byte-array `set` path, the chunker is bypassed: parts are sliced
|
||||
directly from the input `Uint8Array` and yielded synchronously. This avoids
|
||||
an unnecessary `ReadableStream` allocation for the common case.
|
||||
|
||||
### Worker pool
|
||||
|
||||
`#runPartWorkers` keeps `tuning.concurrency` pending `UploadPart` promises in
|
||||
flight. As one resolves, its `{ ETag, PartNumber }` is pushed to `completed`
|
||||
and the next part is drawn from the iterator. The pool uses a structured
|
||||
`Promise.race`-based scheduler — when any worker rejects, the pool aborts
|
||||
remaining workers via a shared `AbortController` linked to the caller's
|
||||
`signal`, then re-throws the first error.
|
||||
|
||||
### Abort propagation
|
||||
|
||||
The engine creates an internal `AbortController` (`linked`). Its signal is
|
||||
passed to every `S3Client.send` call. `linked` is aborted when either:
|
||||
|
||||
- the caller-supplied `signal` fires (via `signal.addEventListener("abort")`),
|
||||
- any part worker rejects.
|
||||
|
||||
The `AbortMultipartUploadCommand` is sent with a **fresh** signal (no abort
|
||||
inheritance) so cleanup still runs after the caller aborts. Cleanup failures
|
||||
are caught, wrapped in `MULTIPART_ABORT_FAILED`, logged at warn level, and
|
||||
swallowed — re-throwing them would mask the original error.
|
||||
|
||||
### Error mapping
|
||||
|
||||
| Stage | Code |
|
||||
|---|---|
|
||||
| `CreateMultipartUpload` failure or missing `UploadId` | `MULTIPART_INIT_FAILED` |
|
||||
| `UploadPart` failure (after retry) | `MULTIPART_PART_FAILED` (with `partNumber` in context) |
|
||||
| `CompleteMultipartUpload` failure | `MULTIPART_COMPLETE_FAILED` |
|
||||
| `AbortMultipartUpload` failure during cleanup | `MULTIPART_ABORT_FAILED` (logged, not thrown) |
|
||||
|
||||
All four are registered via `defineCodes("S3", [...] as const)` alongside the
|
||||
existing entries. The legacy `MULTIPART_FAILED` code remains in the registry
|
||||
(backwards-compat alias; no new call site raises it) so any consumer that
|
||||
asserts on it does not see an `unknown code` lookup failure.
|
||||
|
||||
## Resumability — out of scope
|
||||
|
||||
Resumable uploads require persisting `{ key, uploadId, completedParts[] }`
|
||||
across process restarts. That introduces a storage dependency, a recovery
|
||||
protocol, and a garbage-collection contract for orphaned `UploadId`s — none
|
||||
of which fit inside `StorageLike`. Callers needing resume should layer a
|
||||
dedicated coordinator over `S3.setMultipart`, using `S3Client` directly for
|
||||
`ListPartsCommand` discovery.
|
||||
|
||||
## Streaming input
|
||||
|
||||
`setStream(key, stream, opts?)` is the new internal helper, exposed via the
|
||||
existing `setMultipart` public method (which already accepts a
|
||||
`ReadableStream`). The byte-array `set` path constructs a synchronous part
|
||||
iterator from the input buffer instead of routing through `ReadableStream` —
|
||||
two ingress paths, one engine.
|
||||
|
||||
## Test strategy
|
||||
|
||||
Per CLAUDE.md rule 2 (no mocks): the multipart engine is tested end-to-end
|
||||
against a real S3-compatible service via testcontainers. The genesis monorepo
|
||||
already has `@suaveplan/testing-integration-garage-s3`
|
||||
(`packages/testing/testing-integration-garage-s3/src/fixture/`) which spins
|
||||
up Garage (S3-compatible) in a container with credentials wired into the
|
||||
fixture's `env-bridge`. This change extends — does NOT replace — that fixture.
|
||||
|
||||
Why Garage over MinIO: Garage is already present in the monorepo fixture set,
|
||||
fully supports S3 multipart (CreateMPU/UploadPart/CompleteMPU/AbortMPU), and
|
||||
runs faster in CI containers than MinIO for the multi-part shapes this test
|
||||
plan exercises.
|
||||
|
||||
If a behaviour is **only** triggerable via a specific MinIO quirk (e.g. SlowDown
|
||||
throttling pattern) it will be added as a second fixture; for the requirements
|
||||
in this change, Garage covers every observable shape.
|
||||
|
||||
### Test plan (in `tests/integration/`)
|
||||
|
||||
1. `set` below threshold → single `PutObject`, no MPU calls (assert via
|
||||
`ListMultipartUploadsCommand` returning empty list mid-test).
|
||||
2. `set` above threshold with default tuning → parts visible mid-flight via
|
||||
`ListPartsCommand`, completes, GET round-trips byte-for-byte.
|
||||
3. `setStream` over threshold with caller-supplied 5 MiB part size → exact
|
||||
part count = `ceil(total / partSize)`.
|
||||
4. Concurrency cap honoured — inject a slow part by interposing a custom
|
||||
`S3ClientFactory` that wraps `UploadPart` with a measured delay; assert
|
||||
the in-flight count never exceeds `concurrency`.
|
||||
5. Mid-upload abort via `AbortSignal` → throws, `ListMultipartUploadsCommand`
|
||||
shows zero open uploads for the key (i.e. cleanup ran).
|
||||
6. Part failure → injected via a wrapping `S3ClientFactory` that fails part
|
||||
3 deterministically; final error is `MULTIPART_PART_FAILED` with
|
||||
`context.partNumber === 3`; cleanup ran.
|
||||
7. Cleanup failure path — `AbortMultipartUpload` injected to fail; original
|
||||
error still surfaces, `MULTIPART_ABORT_FAILED` appears in the logger sink.
|
||||
8. `setPartial` with `offset === 0` and a > threshold payload → uses the
|
||||
multipart engine; `capabilities.partialWrites` stays `false`.
|
||||
9. `setPartial` with non-zero offset → still uses `defaultSetPartial`
|
||||
(read-modify-write) regardless of size.
|
||||
10. Part-size validation: `partSize < 5 MiB` → constructor throws
|
||||
`S3StorageError(MULTIPART_INIT_FAILED, ...)` at config time, not at
|
||||
upload time.
|
||||
|
||||
`vitest.integration.config.ts` already exists. Tests live under
|
||||
`tests/integration/multipart.test.ts` so the unit suite (`src/**/*.test.ts`)
|
||||
stays fast and the integration sweep is opt-in via the existing turbo task.
|
||||
|
||||
### Coverage
|
||||
|
||||
100/100/100/100 on the new module per project gate, including:
|
||||
|
||||
- the `linked` abort listener removal on success,
|
||||
- the cleanup-after-failure branch,
|
||||
- the cleanup-failure swallow branch,
|
||||
- the `partSize < 5 MiB` validation guard.
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
packages/storage/storage-s3/src/
|
||||
├── s3/
|
||||
│ ├── s3.ts # set() routes to engine over threshold
|
||||
│ ├── s3.md # updated: threshold + tuning + cap honesty
|
||||
│ ├── s3.test.ts
|
||||
│ └── s3-branches.test.ts
|
||||
├── multipart/ # NEW
|
||||
│ ├── multipart-engine.ts # #runMultipart, init/complete/abort helpers
|
||||
│ ├── multipart-engine.md
|
||||
│ ├── multipart-engine.test.ts
|
||||
│ ├── part-chunker.ts # ReadableStream → AsyncIterable<PartInput>
|
||||
│ ├── part-chunker.md
|
||||
│ ├── part-chunker.test.ts
|
||||
│ ├── worker-pool.ts # bounded-concurrency scheduler with abort
|
||||
│ ├── worker-pool.md
|
||||
│ ├── worker-pool.test.ts
|
||||
│ └── index.ts # explicit named re-exports (rule 10/6a)
|
||||
├── constants/
|
||||
│ └── index.ts # DEFAULT_MULTIPART_THRESHOLD, _CONCURRENCY
|
||||
├── error-codes.ts # +4 codes; MULTIPART_FAILED retained
|
||||
└── types/
|
||||
└── s3.types.ts # MultipartTuning interface
|
||||
|
||||
packages/storage/storage-s3/tests/integration/
|
||||
└── multipart.test.ts # uses testing-integration-garage-s3
|
||||
```
|
||||
|
||||
## Decisions & trade-offs
|
||||
|
||||
| Decision | Alternative | Why |
|
||||
|---|---|---|
|
||||
| Auto-promote inside `set` | Force callers to pick `setMultipart` | Public surface = `StorageLike`; large-write performance shouldn't depend on knowing about S3 specifics. |
|
||||
| Default threshold 16 MiB | 100 MiB (AWS guideline) | At 16 MiB four parallel parts amortise control overhead; lower threshold is safer for chunked-drive workloads where many medium objects benefit. |
|
||||
| `setPartial` whole-object multipart | Always r-m-w | Random-access patches genuinely require r-m-w (S3 has no byte-range PUT); whole-object writes via `setPartial` deserve the fast path. |
|
||||
| `capabilities.partialWrites = false` | Flip to `true` | S3 multipart is whole-object only — flipping the flag would lie to capability-aware consumers (e.g. `drive-plugin-chunks` sparse-file layout). |
|
||||
| Bounded `Promise.race` scheduler | `p-limit` / `p-queue` external dep | One small file, zero new catalog entries, abort handling tightly coupled to the engine. |
|
||||
| Garage testcontainer | MinIO / LocalStack / mocked SDK | Real-service per rule 2; Garage already in `testing-integration-garage-s3`. |
|
||||
@@ -0,0 +1,177 @@
|
||||
# OpenSpec Change: add-storage-s3-multipart
|
||||
|
||||
- **Change ID**: add-storage-s3-multipart
|
||||
- **Package**: `@suaveplan/storage-s3` (modify)
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-17
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/storage-s3/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
`S3.set` issues a single `PutObjectCommand` for every payload, which AWS hard-caps
|
||||
at 5 GiB and which leaves WAN throughput on the table for any object above
|
||||
~16 MiB; `S3.setPartial` falls back to a read-modify-write that pulls the entire
|
||||
object into process memory for every write. The
|
||||
`storage-set-partial` change (archived 2026-04-28) explicitly scoped out the
|
||||
native multipart implementation, leaving `storage-s3` as the only adapter
|
||||
without a real large-write path. `S3.setMultipart` exists but is a separate
|
||||
public method with serial parts and a single coarse error code, so neither
|
||||
`set` nor `setPartial` benefit from it. This change closes that gap so
|
||||
large writes and chunk-driven sparse-file workloads (e.g.
|
||||
`@suaveplan/drive-plugin-chunks`) run on the same fast multipart path as
|
||||
hand-written `setMultipart` callers, without changing the `StorageLike` surface.
|
||||
|
||||
## What Changes
|
||||
|
||||
- `S3.set` auto-promotes to multipart when `value.byteLength` exceeds a
|
||||
configurable threshold (default 16 MiB); below it, behaviour is unchanged.
|
||||
- `S3.setPartial` delegates to the multipart engine when `offset === 0` and
|
||||
the write exceeds the threshold; non-zero-offset patches continue to use
|
||||
the read-modify-write fallback.
|
||||
- New internal engine (`multipart/multipart-engine.ts`, `part-chunker.ts`,
|
||||
`worker-pool.ts`) handles `CreateMultipartUpload` → bounded-parallel
|
||||
`UploadPart` (default concurrency 4) → `CompleteMultipartUpload`, with
|
||||
full abort propagation and guaranteed `AbortMultipartUpload` cleanup.
|
||||
- `S3.setMultipart` becomes a thin wrapper over the engine; its public
|
||||
signature is preserved.
|
||||
- `S3_CODES` gains `MULTIPART_INIT_FAILED`, `MULTIPART_PART_FAILED`,
|
||||
`MULTIPART_COMPLETE_FAILED`, `MULTIPART_ABORT_FAILED`; legacy
|
||||
`MULTIPART_FAILED` remains registered as a back-compat alias.
|
||||
- New subpath exports: `./multipart/multipart-engine`, `./multipart/part-chunker`,
|
||||
`./multipart/worker-pool`.
|
||||
- Integration tests run against the existing
|
||||
`@suaveplan/testing-integration-garage-s3` Garage testcontainer fixture —
|
||||
no mocks.
|
||||
|
||||
## Summary
|
||||
|
||||
Wire native S3 multipart upload into the `set` / `setPartial` happy path so large
|
||||
writes use `CreateMultipartUpload` + parallel `UploadPart` + `CompleteMultipartUpload`
|
||||
instead of either a single 5 GiB-capped `PutObject` or the read-modify-write
|
||||
`defaultSetPartial` fallback. Multipart is an **internal optimization**: the public
|
||||
`StorageLike` surface is unchanged; callers may tune behaviour via an opt-in
|
||||
`{ multipart: { partSize, concurrency, threshold } }` option bag.
|
||||
|
||||
This is the deferral noted in
|
||||
`openspec/archive/2026-04-28-storage-set-partial/proposal.md` ("Out of scope —
|
||||
`storage-s3` multipart-upload native impl — follow-up") and the
|
||||
`storage-s3` row of that change's backend wiring table (`setPartial` →
|
||||
`defaultSetPartial`, `partialWrites: false`).
|
||||
|
||||
## Current state
|
||||
|
||||
- `S3.set` (`packages/storage/storage-s3/src/s3/s3.ts:233`) issues a single
|
||||
`PutObjectCommand` regardless of payload size. AWS rejects single PUTs > 5 GiB
|
||||
and recommends multipart for objects > ~100 MiB for throughput.
|
||||
- `S3.setPartial` (`s3/s3.ts:480`) delegates straight to
|
||||
`AbstractStorage#defaultSetPartial`, which reads the whole object, mutates a
|
||||
`Uint8Array` in process memory, then re-PUTs it. O(N) memory and bandwidth per
|
||||
call, unusable for sparse-chunk drive layouts on real-world object sizes.
|
||||
- `S3.setMultipart` (`s3/s3.ts:602`) already implements a correct CreateMPU +
|
||||
serial UploadPart + CompleteMPU loop, but: (a) it is a separate public method
|
||||
callers must opt into by name, so `set` / `setPartial` never benefit from it;
|
||||
(b) part uploads are strictly serial, leaving WAN throughput on the table;
|
||||
(c) only a single `MULTIPART_FAILED` code exists, hiding which stage failed.
|
||||
- `S3.capabilities.partialWrites` is `false`. Once multipart is wired through
|
||||
`setPartial` for whole-object writes, the flag stays `false` (S3 multipart
|
||||
is **not** random-access — it cannot patch a byte range of an existing object)
|
||||
but consumers gain a documented large-object path via the same method.
|
||||
|
||||
## What changes
|
||||
|
||||
1. **Auto-promote `set` to multipart above a threshold.** When
|
||||
`value.byteLength > options.multipart?.threshold ?? DEFAULT_MULTIPART_THRESHOLD`
|
||||
(default 16 MiB), `S3.set` performs CreateMultipartUpload → parallel
|
||||
UploadPart → CompleteMultipartUpload internally instead of `PutObject`. Below
|
||||
the threshold, behaviour is unchanged.
|
||||
2. **Streaming input path.** Add an internal helper `setStream(key, stream, opts)`
|
||||
that chunks a `ReadableStream<Uint8Array>` into parts on the fly without
|
||||
buffering the whole stream. `setMultipart` becomes a thin wrapper that
|
||||
delegates to this helper. The byte-array `set` path uses the same engine via
|
||||
a `ReadableStream.from`-style adapter so there is one canonical implementation.
|
||||
3. **Bounded parallel parts.** Parts are uploaded with a worker-pool of
|
||||
`options.multipart?.concurrency ?? DEFAULT_MULTIPART_CONCURRENCY` (default 4)
|
||||
concurrent `UploadPart` calls. Order is preserved in the final
|
||||
`MultipartUpload.Parts` list by part number.
|
||||
4. **Abort + cleanup.** Any failure or `AbortSignal` firing during parts MUST
|
||||
propagate to in-flight part workers (cancelling their underlying
|
||||
`S3Client.send` via `abortSignal`) and MUST issue exactly one
|
||||
`AbortMultipartUploadCommand` for the open `UploadId`. Cleanup is best-effort
|
||||
(logged, not re-thrown) but is mandatory — leaked MPUs accrue storage cost.
|
||||
5. **Finer error codes.** `MULTIPART_FAILED` is split via `defineCodes` into
|
||||
`MULTIPART_INIT_FAILED`, `MULTIPART_PART_FAILED`, `MULTIPART_COMPLETE_FAILED`,
|
||||
`MULTIPART_ABORT_FAILED`. `MULTIPART_FAILED` is retained as a
|
||||
backwards-compatibility alias on the registry (raised only by call sites
|
||||
that pre-date this change and were not migrated; the four new codes cover
|
||||
every multipart call site post-change).
|
||||
6. **`setPartial` for whole-object writes.** When `offset === 0` and the call
|
||||
semantically writes the entire object (caller intent expressed via
|
||||
`bytes.byteLength` being the whole object), `setPartial` delegates to the
|
||||
same multipart engine. Random-access patches (non-zero offset) continue to
|
||||
use `defaultSetPartial`'s read-modify-write — multipart cannot patch a byte
|
||||
range and `capabilities.partialWrites` remains `false` to advertise this
|
||||
honestly.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Resumable uploads** — persisting `UploadId` + completed-part list across
|
||||
process restarts. Callers needing resume should use a dedicated surface
|
||||
(future `drive-plugin-resume` or equivalent); this change keeps the
|
||||
`StorageLike` contract simple.
|
||||
- **Random-access patching via multipart.** S3 multipart cannot replace an
|
||||
arbitrary byte range of an existing object. `capabilities.partialWrites`
|
||||
stays `false`.
|
||||
- **Server-side encryption / SSE-KMS option passthrough.** Tracked separately.
|
||||
- **Copy-part (`UploadPartCopy`) for large server-side copies.** Tracked
|
||||
separately under `S3.copy` enhancements.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/storage-s3` to enumerate. The
|
||||
change is additive on the public surface (`set` / `setPartial` signatures
|
||||
unchanged; new options are opt-in; new error codes are siblings of an existing
|
||||
one), so the blast radius is small:
|
||||
|
||||
- `@suaveplan/storage` — [ ] tests run / [ ] docs reviewed / [x] no action needed
|
||||
(reason: `storage-s3` consumes `AbstractStorage` from `storage` and adds new
|
||||
behaviour internally; no `storage` surface changes.)
|
||||
- `@suaveplan/drive-plugin-chunks` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||||
(reason: declared consumer of `setPartial` via `DriveStorage`; benefits
|
||||
directly from the large-write path. Verify conformance suite still passes.)
|
||||
- `@suaveplan/drive-plugin-cache` — [ ] tests run / [ ] docs reviewed / [x] no action needed
|
||||
(reason: cache layer round-trips byte ranges, never crosses the multipart
|
||||
threshold in practice; surface unchanged.)
|
||||
- `@suaveplan/storage-cdn` / `storage-cdn-*` — [ ] tests run / [ ] docs reviewed / [x] no action needed
|
||||
(reason: CDN adapters do not consume `storage-s3` directly.)
|
||||
|
||||
`storage-s3` is not a foundation package per §14a, but it sits under `drive-*`
|
||||
chunk plumbing — the dependent sweep (§13 step 10,
|
||||
`bunx turbo test --filter=...@suaveplan/storage-s3`) MUST be run before archive.
|
||||
|
||||
## Migration
|
||||
|
||||
No public API changes; existing callers see no behaviour difference below the
|
||||
multipart threshold. Callers who explicitly assert on
|
||||
`S3StorageError.code === "S3.MULTIPART_FAILED"` should widen the check to the
|
||||
new four-code set:
|
||||
|
||||
```ts
|
||||
import { S3_CODES } from "@suaveplan/storage-s3";
|
||||
const multipartCodes = new Set<string>([
|
||||
S3_CODES.MULTIPART_INIT_FAILED,
|
||||
S3_CODES.MULTIPART_PART_FAILED,
|
||||
S3_CODES.MULTIPART_COMPLETE_FAILED,
|
||||
S3_CODES.MULTIPART_ABORT_FAILED,
|
||||
S3_CODES.MULTIPART_FAILED, // legacy alias, still emitted by no call site post-change
|
||||
]);
|
||||
if (err instanceof S3StorageError && multipartCodes.has(err.code)) { … }
|
||||
```
|
||||
|
||||
Search command for consumers:
|
||||
|
||||
```bash
|
||||
grep -rln "MULTIPART_FAILED" packages/ | grep -v "/dist/\|/coverage/\|/storage-s3/"
|
||||
```
|
||||
@@ -0,0 +1,235 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: storage-s3 (add-storage-s3-multipart)
|
||||
|
||||
**Package**: `@suaveplan/storage-s3`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
> Test paths name files this change will create; they do not exist yet.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Auto-promote `set` to multipart above threshold
|
||||
|
||||
`S3.set(key, value, options?, signal?)` MUST internally upload `value` via S3
|
||||
multipart (`CreateMultipartUpload` → parallel `UploadPart` →
|
||||
`CompleteMultipartUpload`) when `value.byteLength` exceeds the configured
|
||||
threshold (`S3Options.multipart.threshold`, default 16 MiB). At or below the
|
||||
threshold, behaviour MUST remain a single `PutObjectCommand`. The public
|
||||
signature, return type, and observable success semantics MUST NOT change.
|
||||
|
||||
#### Scenario: large byte array triggers multipart
|
||||
|
||||
- **WHEN** a caller invokes `s3.set("k", new Uint8Array(20 * 1024 * 1024))`
|
||||
against an `S3` instance with default tuning
|
||||
- **THEN** the underlying client issues exactly one
|
||||
`CreateMultipartUploadCommand`, one or more `UploadPartCommand`, exactly one
|
||||
`CompleteMultipartUploadCommand`, and zero `PutObjectCommand` for that key
|
||||
|
||||
#### Scenario: small byte array uses single PUT
|
||||
|
||||
- **WHEN** a caller invokes `s3.set("k", new Uint8Array(1024))`
|
||||
- **THEN** the underlying client issues exactly one `PutObjectCommand` and
|
||||
zero `CreateMultipartUploadCommand` for that key
|
||||
|
||||
**Verified by:** `tests/integration/multipart.test.ts::"large byte array triggers multipart"`
|
||||
AND `tests/integration/multipart.test.ts::"small byte array uses single PUT"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Streaming input is chunked without whole-stream buffering
|
||||
|
||||
`S3.setMultipart(key, stream, opts?)` MUST consume a
|
||||
`ReadableStream<Uint8Array>` and emit `UploadPart` requests of exactly
|
||||
`opts.partSize ?? S3Options.multipart.partSize ?? DEFAULT_PART_SIZE` bytes
|
||||
each (except the final part, which MAY be shorter). At no point MAY the
|
||||
implementation buffer more than two parts' worth of bytes in memory
|
||||
(in-flight read + accumulating chunker buffer).
|
||||
|
||||
#### Scenario: stream of N bytes produces ceil(N / partSize) parts
|
||||
|
||||
- **WHEN** a caller invokes `s3.setMultipart("k", stream)` where `stream`
|
||||
yields 25 MiB total and `partSize` is 5 MiB
|
||||
- **THEN** the upload completes with exactly 5 `UploadPartCommand` calls and
|
||||
the round-tripped GET equals the input byte-for-byte
|
||||
|
||||
**Verified by:** `src/multipart/part-chunker.test.ts::"emits exact part-size chunks then a ragged final part"`
|
||||
AND `tests/integration/multipart.test.ts::"stream over threshold produces expected part count"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Parts upload with bounded parallelism
|
||||
|
||||
The multipart engine MUST upload parts with at most
|
||||
`S3Options.multipart.concurrency ?? DEFAULT_MULTIPART_CONCURRENCY` (default 4)
|
||||
`UploadPartCommand` calls in flight simultaneously. Concurrency MUST be at
|
||||
least 1 and at most 32; values outside this range MUST be clamped or rejected
|
||||
at construction time. The final `MultipartUpload.Parts` list passed to
|
||||
`CompleteMultipartUploadCommand` MUST be sorted ascending by `PartNumber`
|
||||
regardless of completion order.
|
||||
|
||||
#### Scenario: concurrency cap is honoured
|
||||
|
||||
- **WHEN** a caller invokes `setMultipart` with 12 parts and concurrency 4,
|
||||
using an injected `S3ClientFactory` that records the count of pending
|
||||
`UploadPart` promises at each tick
|
||||
- **THEN** the recorded in-flight count never exceeds 4 and the final
|
||||
`CompleteMultipartUploadCommand.MultipartUpload.Parts` array is sorted by
|
||||
`PartNumber`
|
||||
|
||||
**Verified by:** `src/multipart/worker-pool.test.ts::"never exceeds the configured concurrency"`
|
||||
AND `tests/integration/multipart.test.ts::"concurrency cap is honoured under injected part latency"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Abort propagation cancels in-flight parts
|
||||
|
||||
The multipart engine MUST react to a caller-supplied `AbortSignal` firing
|
||||
mid-upload by cancelling every in-flight `UploadPartCommand` (via a linked
|
||||
abort signal passed to `S3Client.send`), MUST stop drawing further parts
|
||||
from the input iterator, and MUST issue exactly one
|
||||
`AbortMultipartUploadCommand` for the open `UploadId` before re-throwing.
|
||||
|
||||
#### Scenario: caller aborts mid-upload
|
||||
|
||||
- **WHEN** a caller starts `setMultipart` on a 50 MiB stream then calls
|
||||
`controller.abort()` after the second part begins
|
||||
- **THEN** the call rejects, the underlying client receives at least one
|
||||
`AbortMultipartUploadCommand` for the key, and a subsequent
|
||||
`ListMultipartUploadsCommand` for the bucket returns no open uploads
|
||||
for that key
|
||||
|
||||
**Verified by:** `tests/integration/multipart.test.ts::"caller abort cancels parts and cleans up"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Per-stage error codes
|
||||
|
||||
Multipart failures MUST raise `S3StorageError` with a code matching the failed
|
||||
stage: `S3.MULTIPART_INIT_FAILED` when `CreateMultipartUpload` fails or
|
||||
returns no `UploadId`; `S3.MULTIPART_PART_FAILED` when an `UploadPart` call
|
||||
fails after retries, with `partNumber` populated in the error `context`;
|
||||
`S3.MULTIPART_COMPLETE_FAILED` when `CompleteMultipartUpload` fails;
|
||||
`S3.MULTIPART_ABORT_FAILED` when the cleanup `AbortMultipartUpload` itself
|
||||
fails (this code is logged at warn level and MUST NOT replace the original
|
||||
thrown error). The legacy code `S3.MULTIPART_FAILED` MUST remain registered
|
||||
via `defineCodes` as a backwards-compatibility alias but MUST NOT be raised
|
||||
by any post-change call site.
|
||||
|
||||
#### Scenario: part failure surfaces stage-specific code
|
||||
|
||||
- **WHEN** a wrapping `S3ClientFactory` makes the third `UploadPartCommand`
|
||||
reject deterministically and the caller invokes `setMultipart` on a stream
|
||||
with at least three parts
|
||||
- **THEN** the call rejects with `S3StorageError` whose `code` is
|
||||
`S3.MULTIPART_PART_FAILED` and whose `context.partNumber` is `3`
|
||||
|
||||
#### Scenario: cleanup failure does not mask original error
|
||||
|
||||
- **WHEN** an `UploadPart` rejection triggers cleanup, and the injected
|
||||
`AbortMultipartUploadCommand` also rejects
|
||||
- **THEN** the call still rejects with the original
|
||||
`S3.MULTIPART_PART_FAILED`, and a logger record at warn level carries
|
||||
the `S3.MULTIPART_ABORT_FAILED` code
|
||||
|
||||
**Verified by:** `src/multipart/multipart-engine.test.ts::"maps stage failures to dedicated codes"`
|
||||
AND `tests/integration/multipart.test.ts::"cleanup failure is logged and does not mask original error"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `setPartial` whole-object writes use the multipart engine
|
||||
|
||||
`S3.setPartial(key, offset, bytes, signal?)` MUST delegate to the multipart
|
||||
engine when `offset === 0` and `bytes.byteLength` exceeds the configured
|
||||
multipart threshold. For all other cases (non-zero offset, or below
|
||||
threshold) it MUST continue to use `AbstractStorage#defaultSetPartial`'s
|
||||
read-modify-write fallback. `S3.capabilities.partialWrites` MUST remain
|
||||
`false` to honestly advertise that random-access byte-range patching is not
|
||||
natively supported by S3 multipart.
|
||||
|
||||
#### Scenario: zero-offset large write takes the multipart path
|
||||
|
||||
- **WHEN** a caller invokes
|
||||
`s3.setPartial("k", 0, new Uint8Array(20 * 1024 * 1024))`
|
||||
- **THEN** the underlying client issues `CreateMultipartUploadCommand` +
|
||||
`UploadPartCommand`(s) + `CompleteMultipartUploadCommand`, and zero
|
||||
`GetObjectCommand` or `PutObjectCommand` for the key
|
||||
|
||||
#### Scenario: non-zero-offset write keeps read-modify-write
|
||||
|
||||
- **WHEN** a caller invokes `s3.setPartial("k", 1024, new Uint8Array(512))`
|
||||
on an existing 4 KiB object
|
||||
- **THEN** the underlying client issues `GetObjectCommand` followed by
|
||||
`PutObjectCommand` and zero `CreateMultipartUploadCommand`
|
||||
|
||||
#### Scenario: capability flag stays honest
|
||||
|
||||
- **WHEN** a caller reads `s3.capabilities.partialWrites`
|
||||
- **THEN** the value is `false`
|
||||
|
||||
**Verified by:** `tests/integration/multipart.test.ts::"setPartial at offset zero uses multipart"`
|
||||
AND `tests/integration/multipart.test.ts::"setPartial at non-zero offset uses read-modify-write"`
|
||||
AND `src/s3/s3.test.ts::"capabilities.partialWrites stays false"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Part-size and tuning validation
|
||||
|
||||
`S3` MUST validate `S3Options.multipart.partSize` against the AWS-mandated
|
||||
minimum of 5 MiB at construction time and throw
|
||||
`S3StorageError(S3.MULTIPART_INIT_FAILED, ...)` with a message naming the
|
||||
offending value. `S3Options.multipart.threshold` MUST be greater than or
|
||||
equal to `MIN_PART_SIZE` (5 MiB). `S3Options.multipart.concurrency` MUST be
|
||||
in the inclusive range `[1, 32]`; out-of-range values MUST throw at
|
||||
construction time. These checks MUST run before any network call is issued.
|
||||
|
||||
#### Scenario: sub-floor part size is rejected at construction
|
||||
|
||||
- **WHEN** a caller constructs `new S3({ ..., multipart: { partSize: 1024 } })`
|
||||
- **THEN** the constructor throws `S3StorageError` whose `code` is
|
||||
`S3.MULTIPART_INIT_FAILED` and whose message names the value `1024`
|
||||
|
||||
**Verified by:** `src/s3/s3.test.ts::"rejects sub-floor partSize at construction time"`
|
||||
AND `src/s3/s3.test.ts::"rejects out-of-range concurrency at construction time"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: New multipart subpath exports
|
||||
|
||||
`@suaveplan/storage-s3/package.json` MUST expose subpath exports for the new
|
||||
multipart modules so consumers can tree-shake: `./multipart/multipart-engine`,
|
||||
`./multipart/part-chunker`, `./multipart/worker-pool`. The root barrel
|
||||
(`src/index.ts`) MUST re-export them under a `multipart` namespace
|
||||
(`export * as multipart from "./multipart/index.js"`); feature barrels
|
||||
(`src/multipart/index.ts`) MUST use explicit named re-exports per
|
||||
project rule §6a — no `export *`.
|
||||
|
||||
#### Scenario: subpath import resolves
|
||||
|
||||
- **WHEN** a consumer writes
|
||||
`import { runMultipart } from "@suaveplan/storage-s3/multipart/multipart-engine"`
|
||||
- **THEN** the import resolves to the engine implementation without pulling
|
||||
the full `S3` class into the consumer's bundle
|
||||
|
||||
**Verified by:** `src/multipart/multipart-engine.test.ts::"is reachable via the dedicated subpath export"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Multipart engine is verified against a real S3-compatible service
|
||||
|
||||
The package MUST exercise the multipart engine end-to-end (init, parallel
|
||||
parts, complete, abort, cleanup, error mapping) against a real S3-compatible
|
||||
service via testcontainers per project rule 2 (no mocks). The genesis
|
||||
monorepo's `@suaveplan/testing-integration-garage-s3` fixture MUST be the
|
||||
host. No unit test MAY substitute a mocked AWS SDK for these behaviours.
|
||||
|
||||
#### Scenario: integration suite runs against Garage container
|
||||
|
||||
- **WHEN** the integration test config (`vitest.integration.config.ts`) is
|
||||
executed via `bunx turbo test:integration --filter=@suaveplan/storage-s3`
|
||||
- **THEN** every test in `tests/integration/multipart.test.ts` uses the
|
||||
Garage-backed `S3Client` from `@suaveplan/testing-integration-garage-s3`
|
||||
rather than a mocked client, and the suite exits zero
|
||||
|
||||
**Verified by:** `tests/integration/multipart.test.ts::"integration suite is wired to the Garage testcontainer fixture"`
|
||||
@@ -0,0 +1,107 @@
|
||||
# Tasks: add-storage-s3-multipart
|
||||
|
||||
## 1. Error codes + constants
|
||||
|
||||
- [ ] 1.1 Extend `S3_CODES` via `defineCodes` with `MULTIPART_INIT_FAILED`,
|
||||
`MULTIPART_PART_FAILED`, `MULTIPART_COMPLETE_FAILED`,
|
||||
`MULTIPART_ABORT_FAILED`. Retain `MULTIPART_FAILED` as a documented
|
||||
legacy alias (no new call site raises it).
|
||||
- [ ] 1.2 Add `DEFAULT_MULTIPART_THRESHOLD = 16 * 1024 * 1024` and
|
||||
`DEFAULT_MULTIPART_CONCURRENCY = 4` to `src/constants/index.ts`.
|
||||
Verify `DEFAULT_PART_SIZE` floor (≥ 5 MiB) and add a constant
|
||||
`MIN_PART_SIZE = 5 * 1024 * 1024`.
|
||||
- [ ] 1.3 Add `MultipartTuning` interface to `src/types/s3.types.ts`; extend
|
||||
`S3Options` with `multipart?: MultipartTuning`.
|
||||
|
||||
## 2. Internal engine
|
||||
|
||||
- [ ] 2.1 Create `src/multipart/part-chunker.ts` + sibling `.md` + `.test.ts` —
|
||||
`chunkStream(stream, partSize): AsyncIterable<PartInput>`; covers exact
|
||||
multiples, ragged final part, empty stream, abort mid-read.
|
||||
- [ ] 2.2 Create `src/multipart/worker-pool.ts` + sibling `.md` + `.test.ts` —
|
||||
bounded-concurrency scheduler using `Promise.race` + a linked
|
||||
`AbortController`; covers cap enforcement, first-failure aborts peers,
|
||||
caller-abort cancels in-flight workers.
|
||||
- [ ] 2.3 Create `src/multipart/multipart-engine.ts` + sibling `.md` +
|
||||
`.test.ts` — `runMultipart(client, bucket, key, parts, tuning, signal)`
|
||||
with `initMultipart` / `completePartsList` / `abortMultipart` helpers;
|
||||
maps each stage failure to its dedicated code.
|
||||
- [ ] 2.4 Create `src/multipart/index.ts` — explicit named re-exports only
|
||||
(rule 10 / §6a). No `export *`.
|
||||
|
||||
## 3. Wire into `S3.set` + `S3.setPartial`
|
||||
|
||||
- [ ] 3.1 In `s3/s3.ts`, branch `S3.set` on
|
||||
`value.byteLength > (this.options.multipart?.threshold ?? DEFAULT_MULTIPART_THRESHOLD)`
|
||||
to call the engine via a synchronous part iterator built from the
|
||||
buffer; below the threshold keep the existing `PutObjectCommand`.
|
||||
- [ ] 3.2 In `S3.setPartial`, when `offset === 0` AND
|
||||
`bytes.byteLength > threshold`, delegate to the engine (whole-object
|
||||
write); otherwise keep `defaultSetPartial`. Document in
|
||||
`s3/s3.md` that `capabilities.partialWrites` stays `false` and explain
|
||||
why.
|
||||
- [ ] 3.3 Rewrite `S3.setMultipart` as a thin wrapper over the new engine
|
||||
using `chunkStream` for the input `ReadableStream`. Preserve public
|
||||
signature; preserve `setMultipart` subpath export.
|
||||
- [ ] 3.4 Validate `partSize >= MIN_PART_SIZE` at construction time; throw
|
||||
`S3StorageError(MULTIPART_INIT_FAILED, ...)` with a clear message.
|
||||
|
||||
## 4. Subpath exports + barrels
|
||||
|
||||
- [ ] 4.1 Add `./multipart/multipart-engine`, `./multipart/part-chunker`,
|
||||
`./multipart/worker-pool` entries to `package.json` `exports`.
|
||||
- [ ] 4.2 Update `src/index.ts` to re-export the new multipart helpers via
|
||||
a namespace export (`export * as multipart from "./multipart/index.js"`)
|
||||
so consumers can reach them without breaking the root barrel rule.
|
||||
|
||||
## 5. Docs
|
||||
|
||||
- [ ] 5.1 Update `README.md` Modules + Features sections to mention auto
|
||||
multipart, tuning, and the four new error codes.
|
||||
- [ ] 5.2 Update `s3/s3.md` (Implementation Notes) — threshold rationale,
|
||||
capability honesty, abort/cleanup contract.
|
||||
- [ ] 5.3 Add `multipart/multipart-engine.md`, `part-chunker.md`,
|
||||
`worker-pool.md` per §4 (≥200 words, basic + advanced examples, API
|
||||
reference, implementation notes).
|
||||
|
||||
## 6. Integration tests (real Garage container)
|
||||
|
||||
- [ ] 6.1 Confirm `@suaveplan/testing-integration-garage-s3` fixture surface
|
||||
covers MPU operations end-to-end; if a missing capability is found
|
||||
(e.g. mid-flight `ListPartsCommand` exposure), extend the fixture in
|
||||
the same change.
|
||||
- [ ] 6.2 Add `tests/integration/multipart.test.ts` covering the 10 cases
|
||||
in `design.md#test-plan`. No mocks — use a wrapping `S3ClientFactory`
|
||||
to inject controlled delays / failures around the real Garage client.
|
||||
- [ ] 6.3 Wire integration suite into `vitest.integration.config.ts` (already
|
||||
present in the package).
|
||||
|
||||
## 7. Verify (per §13 pre-ship sequence)
|
||||
|
||||
- [ ] 7.1 Spec reconciliation (§13 step 0): every `### Requirement` in
|
||||
`specs/storage-s3/spec.md` has matching impl + tests; every
|
||||
`**Verified by:**` path resolves verbatim.
|
||||
- [ ] 7.2 `bunx turbo lint typecheck test --filter=@suaveplan/storage-s3`
|
||||
green.
|
||||
- [ ] 7.3 `bunx turbo test:coverage --filter=@suaveplan/storage-s3`
|
||||
100/100/100/100.
|
||||
- [ ] 7.4 Integration sweep: `bunx turbo test:integration --filter=@suaveplan/storage-s3`
|
||||
green against the Garage testcontainer.
|
||||
- [ ] 7.5 Dependent sweep:
|
||||
`bunx turbo test --filter=...@suaveplan/storage-s3` green.
|
||||
- [ ] 7.6 Banned-pattern / barrel-shape / runner-import scans (§13 steps
|
||||
4 / 4a / 4b) — zero output.
|
||||
- [ ] 7.7 `bun scripts/openspec-conformance.ts @suaveplan/storage-s3` PASS.
|
||||
- [ ] 7.8 Run `openspec validate add-storage-s3-multipart --strict` — must
|
||||
report PASS.
|
||||
|
||||
## 8. Archive
|
||||
|
||||
- [ ] 8.1 Refresh root docs per §13b — `MANIFEST.md`, `MISSING.yml`,
|
||||
`STATUS.md`, `SUMMARY.md`; run `bunx markdownlint-cli2 --fix`.
|
||||
- [ ] 8.2 Fold `specs/storage-s3/spec.md` into
|
||||
`openspec/specs/storage-s3/spec.md` (this is the first change to
|
||||
establish that canonical capability file — create the folder under
|
||||
`openspec/specs/` only at this archival step, never before).
|
||||
- [ ] 8.3 Move `openspec/changes/add-storage-s3-multipart/` →
|
||||
`openspec/archive/<yyyy-mm-dd>-add-storage-s3-multipart/`.
|
||||
@@ -0,0 +1,161 @@
|
||||
# Design — `@suaveplan/webauthn-mds3`
|
||||
|
||||
## 1. Sibling package vs. in-tree module
|
||||
|
||||
**Decision:** sibling package `@suaveplan/webauthn-mds3` at `packages/auth/webauthn-mds3/`.
|
||||
|
||||
`@suaveplan/webauthn` is browser-and-universal mixed-tier and intentionally slim. Bundling MDS3 would drag four heavy concerns into it: a `FetchLike` network surface, a long-lived refresh loop (a state machine `webauthn` deliberately avoids), JWS chain verification (`@suaveplan/jws` + `@suaveplan/x509`), and a multi-megabyte FIDO Alliance metadata blob in test fixtures. Every consumer that only needs `register` / `authenticate` would pay that cost. A sibling keeps the integration opt-in, lets MDS3 evolve on its own release cadence (the FIDO Alliance ships schema bumps independently of WebAuthn), and lets server-side relying-party deployments depend on `webauthn-mds3` without touching the browser-tier surface at all.
|
||||
|
||||
The seam back into `webauthn` is a single injected interface (`MetadataResolver`) — type-only at the `webauthn` side via `import type`, so the runtime dep graph stays acyclic.
|
||||
|
||||
## 2. Package layout
|
||||
|
||||
```text
|
||||
packages/auth/webauthn-mds3/
|
||||
package.json # subpath exports per §6
|
||||
README.md
|
||||
src/
|
||||
constants/
|
||||
fido-root.ts + .md # pinned root CA DER + SHA-256, defaults
|
||||
errors/
|
||||
error-codes.ts + .md # defineCodes("WEBAUTHN_MDS3", [...])
|
||||
errors.test.ts
|
||||
schemas/
|
||||
metadata-blob.schemas.ts + .md # Zod for the whole MDS3 v3 surface
|
||||
fetch/
|
||||
fetch-blob.ts + .md + .test.ts
|
||||
verify/
|
||||
verify-blob.ts + .md + .test.ts
|
||||
cache/
|
||||
metadata-cache.ts + .md + .test.ts
|
||||
refresher/
|
||||
metadata-refresher.ts + .md + .test.ts
|
||||
resolver/
|
||||
types.ts + .md # MetadataResolver, MetadataLookup
|
||||
memory.ts + .md + .test.ts
|
||||
cached.ts + .md + .test.ts
|
||||
observability/
|
||||
metrics.ts + .md + .test.ts
|
||||
__fixtures__/
|
||||
mds3-sample-blob.jwt
|
||||
mds3-corrupted-blob.jwt
|
||||
index.ts
|
||||
__benchmarks__/
|
||||
verify-blob.bench.ts
|
||||
resolve-aaguid.bench.ts
|
||||
index.ts # explicit named re-exports (rule 10)
|
||||
```
|
||||
|
||||
## 3. Key types
|
||||
|
||||
```ts
|
||||
// src/resolver/types.ts
|
||||
export interface MetadataLookup {
|
||||
readonly statement: MetadataStatement;
|
||||
readonly currentStatus: AuthenticatorStatus;
|
||||
readonly statusReports: readonly StatusReport[];
|
||||
readonly attestationRootCertificates: readonly Uint8Array[]; // DER
|
||||
readonly aaguid?: Uint8Array; // 16 bytes
|
||||
readonly attestationCertificateKeyIds?: readonly Uint8Array[]; // SHA-1 SKIs
|
||||
readonly lastUpdate: number; // epoch ms
|
||||
}
|
||||
|
||||
export interface MetadataResolver {
|
||||
resolveByAaguid(aaguid: Uint8Array): Promise<MetadataLookup | undefined>;
|
||||
resolveByAttestationCertificateKeyId(
|
||||
keyId: Uint8Array,
|
||||
): Promise<MetadataLookup | undefined>;
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
// new optional field on @suaveplan/webauthn's VerifyAttestationInput
|
||||
export interface VerifyAttestationInput {
|
||||
// ...existing fields...
|
||||
readonly metadata?: import("@suaveplan/webauthn-mds3").MetadataResolver;
|
||||
readonly revokedStatuses?: ReadonlySet<AuthenticatorStatus>; // override default
|
||||
}
|
||||
|
||||
// new optional field on AttestationVerifyResult
|
||||
export interface AttestationVerifyResult {
|
||||
readonly trusted: boolean;
|
||||
readonly attestationType: AttestationType;
|
||||
readonly trustPath: readonly Uint8Array[];
|
||||
readonly metadata?: MetadataLookup; // present only when resolver injected
|
||||
}
|
||||
```
|
||||
|
||||
## 4. Refresh policy
|
||||
|
||||
| Trigger | Behaviour |
|
||||
|---|---|
|
||||
| First start (no cached blob) | Fetch immediately, verify, cache, schedule next at `nextUpdate - jitter` |
|
||||
| Scheduled tick | Fetch, verify, write-through cache; on verify-fail keep prior payload and emit `webauthn_mds3.refresh_total{outcome=verify_failed}` |
|
||||
| `nextUpdate` absent | Fall back to `intervalMs` (default 24h) |
|
||||
| Concurrent call while refresh in flight | Throw `REFRESHER_ALREADY_RUNNING` (no silent dedup — caller-visible) |
|
||||
| `signal.aborted` | Cancel pending refresh, throw `REFRESHER_ABORTED` |
|
||||
| Jitter | Uniform random in `[-5min, +5min]` to avoid thundering herd against the MDS endpoint |
|
||||
|
||||
The refresher is decoupled from the resolver — a resolver can be backed by a cache that some other process refreshes (e.g. a cron job, a Cloudflare Cron Trigger). `createCachedResolver` accepts an optional `refresher` that it `start()`s on first `resolve*` call.
|
||||
|
||||
## 5. Cache contract
|
||||
|
||||
The cache is a thin wrapper around `StorageLike<MetadataBlobPayload>` from `@suaveplan/types`:
|
||||
|
||||
- Key namespace `webauthn-mds3:blob:v1` — bump the trailing version on any schema migration.
|
||||
- Single key per environment (one blob = one MDS3 endpoint). Multi-tenancy is the consumer's problem — wrap the `StorageLike` with a tenant-prefixing decorator.
|
||||
- Read failures surface `CACHE_READ_FAILED` with the underlying cause attached; the resolver MAY treat a missing cache entry as "fetch now" if a refresher is wired.
|
||||
- Writes are atomic from the resolver's perspective — the cache writes the verified payload only after `verifyMetadataBlob` resolves successfully.
|
||||
|
||||
## 6. Attestation-time lookup flow
|
||||
|
||||
```text
|
||||
verifyAttestation(input):
|
||||
1. result = await registry.get(input.attestationObject.fmt)(...)
|
||||
2. if input.metadata === undefined: return result // strict back-compat
|
||||
3. lookup =
|
||||
(fmt === "fido-u2f" || attestedCredentialData?.aaguid is all-zero)
|
||||
? input.metadata.resolveByAttestationCertificateKeyId(ski(result.trustPath[0]))
|
||||
: input.metadata.resolveByAaguid(attestedCredentialData.aaguid)
|
||||
4. if lookup === undefined: return { ...result, metadata: undefined }
|
||||
5. revoked = (input.revokedStatuses ?? DEFAULT_REVOCATION_STATUSES).has(lookup.currentStatus)
|
||||
6. return { ...result, trusted: result.trusted && !revoked, metadata: lookup }
|
||||
```
|
||||
|
||||
`DEFAULT_REVOCATION_STATUSES = new Set(["REVOKED", "USER_KEY_REMOTE_COMPROMISE", "USER_KEY_PHYSICAL_COMPROMISE", "ATTESTATION_KEY_COMPROMISE", "USER_VERIFICATION_BYPASS"])`.
|
||||
|
||||
`fido-u2f` authenticators are MDS3-keyed by attestation-cert SKI (FIDO MDS3 §3.1.6) because they predate AAGUID. The attestation cert SKI is computed inside `webauthn-mds3` so `webauthn` doesn't need an X.509 parser at runtime.
|
||||
|
||||
## 7. Security model
|
||||
|
||||
| Concern | Mitigation |
|
||||
|---|---|
|
||||
| Blob substitution (attacker serves a benign blob omitting a compromised authenticator) | Pin the FIDO Alliance root CA in `constants/fido-root.ts`; verify the JWS `x5c` chain terminates at the pinned root; reject any chain ending elsewhere |
|
||||
| Algorithm downgrade | Allow-list RS256 / ES256 / ES384 only; reject `none`, `HS256`, and any other alg with `BLOB_ALG_DISALLOWED` |
|
||||
| Stale blob masking a known-compromised authenticator | Reject when `now > nextUpdate + grace`; the refresher pulls new blobs on schedule; OTel exposes `webauthn_mds3.blob_age_ms` for monitoring |
|
||||
| Replay of a previous blob (`no` rollback) | Refuse to overwrite the cached blob with a payload whose `no` field is lower than the cached `no` |
|
||||
| Schema drift | All blob fields pass through Zod (`MetadataBlobPayloadSchema`); unknown fields are tolerated but typed-out — additive FIDO Alliance extensions don't crash the verifier |
|
||||
| TOCTOU between resolve and trust decision | The trust decision happens inline in `verifyAttestation`; the resolver returns a `MetadataLookup` snapshot, not a live handle, so concurrent refreshes can't flip the result mid-flight |
|
||||
| `FetchLike` impersonation in tests | Production wires a real `fetch`; tests inject deterministic stubs returning fixture bytes — no live network, no `vi.mock` (per genesis rule 2) |
|
||||
|
||||
## 8. Companion delta to `@suaveplan/webauthn`
|
||||
|
||||
Because the `VerifyAttestationInput` and `AttestationVerifyResult` shapes gain optional fields and the post-verifier flow grows two trust-altering steps, the integration is captured as a `## MODIFIED Requirements` block in `specs/webauthn/spec.md` rather than left implicit. The modified requirement keeps every original scenario (the back-compat path) and adds two new scenarios (resolver-injected augmentation; revoked-status downgrade), each with a `**Verified by:**` line pointing into the existing `verify-attestation.test.ts` (extended in task 13.4).
|
||||
|
||||
## 9. Tier purity
|
||||
|
||||
`@suaveplan/webauthn-mds3` is **universal-tier**:
|
||||
|
||||
- No `globalThis.fetch` reference — `FetchLike` is injected.
|
||||
- No `localStorage` / `IndexedDB` — `StorageLike` is injected.
|
||||
- No `Date.now()` — `Clock` is injected.
|
||||
- No DOM types in the public surface — `MetadataLookup` carries `Uint8Array` not `Blob`.
|
||||
- `tests/dep-graph.test.ts` (in the package) enforces zero DOM imports.
|
||||
|
||||
This means server-side relying parties (Bun, Node, Workers, Deno) and browser-tier consumers can both depend on it without polyfills.
|
||||
|
||||
## 10. What this change deliberately does NOT do
|
||||
|
||||
- Does **not** ship an MDS3 publishing tool (creating blobs is for the FIDO Alliance only).
|
||||
- Does **not** ship a CRL / OCSP poller for attestation roots — that's a separate trust-chain concern outside the FIDO metadata contract.
|
||||
- Does **not** wire MDS3 into the `vault` unlock path — `verifyAssertion` (the unlock path) does not consume attestation, so MDS3 has no role there.
|
||||
@@ -0,0 +1,126 @@
|
||||
# `@suaveplan/webauthn-mds3` — FIDO Metadata Service v3 integration
|
||||
|
||||
**Package**: `@suaveplan/webauthn-mds3` (NEW)
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (genesis-original)
|
||||
**Date**: 2026-05-17
|
||||
**Path**: `packages/auth/webauthn-mds3/`
|
||||
**Design**: ./design.md
|
||||
**Spec delta**: ./specs/webauthn-mds3/spec.md
|
||||
**Companion delta**: ./specs/webauthn/spec.md (`## MODIFIED Requirements`)
|
||||
|
||||
## Why
|
||||
|
||||
The `enterprise-webauthn` change (archived 2026-05-11) shipped `verifyAttestation` returning a `trustPath: X509[]` plus an `attestationType`, but explicitly deferred FIDO Metadata Service v3 (MDS3) integration to a follow-up:
|
||||
|
||||
> "MDS3 (FIDO Metadata Service) integration — `verifyAttestation` returns the `trustPath: X509[]` and `attestationType` so callers can perform MDS3 lookups against their own metadata source. Bundling MDS3 fetch + caching expands scope into a separate package and is a follow-up."
|
||||
> — `openspec/archive/2026-05-11-enterprise-webauthn/proposal.md`, "Out of scope"
|
||||
|
||||
Without MDS3 a relying party cannot answer the production-critical questions every passkey deployment eventually has to answer: is this AAGUID a known certified authenticator, is its certification still valid, is it on the revocation list, what attestation root CAs are trusted for this AAGUID, and what attachment / user-verification / key-protection statuses did the FIDO Alliance certify it at. Today every genesis consumer that wants those answers has to hand-roll the MDS3 blob fetch, JWS verification, FIDO root-CA pin, AAGUID lookup, and refresh schedule — exactly the duplication this monorepo exists to eliminate.
|
||||
|
||||
This change introduces a sibling package, `@suaveplan/webauthn-mds3`, that owns the MDS3 wire protocol end-to-end (download, JWS verify against the pinned FIDO Alliance root, cache, periodic refresh, AAGUID-keyed lookup) and exposes an injectable `MetadataResolver` that `@suaveplan/webauthn`'s attestation pipeline consumes through a new optional seam. Absent injection, the existing `verifyAttestation` contract is preserved byte-for-byte (strict back-compat).
|
||||
|
||||
## What changes
|
||||
|
||||
### New package: `@suaveplan/webauthn-mds3`
|
||||
|
||||
Universal-tier package at `packages/auth/webauthn-mds3/`. No browser-only or Node-only APIs — `FetchLike` and `StorageLike` are injected from `@suaveplan/types`.
|
||||
|
||||
#### MDS3 blob download + JWS verification
|
||||
|
||||
- ADDED: `fetchMetadataBlob({ fetch, url? }): Promise<RawBlob>` — fetches the MDS3 JWT (`application/jwt`) from `url` (default `https://mds.fidoalliance.org/`) using the injected `FetchLike` so consumers can layer in retries, proxies, or a Cloudflare-Workers `fetch`.
|
||||
- ADDED: `verifyMetadataBlob({ jws, trustAnchors? }): Promise<MetadataBlobPayload>` — verifies the compact-JWS signature via `@suaveplan/jws` using the FIDO Alliance MDS3 root CA chain. The root CA fingerprint is pinned in `constants/fido-root.ts`; consumers can override via `trustAnchors` for staging / private metadata services. Verifies `x5c` chain to root, validates not-before / not-after, enforces RS256 / ES256 alg allow-list, decodes the JSON payload, and Zod-validates it against `MetadataBlobPayloadSchema` (FIDO MDS3 §3.1.7).
|
||||
- ADDED: `MetadataBlobPayload`, `MetadataStatement`, `AuthenticatorStatus`, `StatusReport`, `BiometricStatusReport`, `EcdaaTrustAnchor` — Zod schemas tracking the published MDS3 v3 schema verbatim.
|
||||
|
||||
#### Cache + scheduled refresh
|
||||
|
||||
- ADDED: `MetadataCache` — `StorageLike<MetadataBlobPayload>`-backed cache with `legalHeader`, `no`, `nextUpdate`, and `entries` preserved. Cache key namespace `webauthn-mds3:blob:v1`.
|
||||
- ADDED: `createMetadataRefresher({ fetch, storage, clock, logger, intervalMs?, signal? }): MetadataRefresher` — schedules refresh at `payload.nextUpdate - jitter`; falls back to `intervalMs` (default 24h) when the blob omits a `nextUpdate`. Honors `AbortSignal`; never starts a second refresh while one is in flight; emits OTel `webauthn_mds3.refresh_total{outcome}` counter.
|
||||
|
||||
#### AAGUID lookup API
|
||||
|
||||
- ADDED: `MetadataResolver` interface — `resolveByAaguid(aaguid: Uint8Array): Promise<MetadataLookup | undefined>` and `resolveByAttestationCertificateKeyId(keyId: Uint8Array): Promise<MetadataLookup | undefined>` (U2F authenticators are MDS3-keyed by attestation cert SKI, not AAGUID — FIDO MDS3 §3.1.6).
|
||||
- ADDED: `MetadataLookup` — `{ statement, currentStatus, statusReports, attestationRootCertificates: Uint8Array[], aaguid?, attestationCertificateKeyIds?, lastUpdate }`.
|
||||
- ADDED: `createMemoryResolver({ blob }): MetadataResolver` — synchronous in-memory resolver over a verified payload. Used by tests and by consumers that ship their own refresh loop.
|
||||
- ADDED: `createCachedResolver({ cache, refresher? }): MetadataResolver` — production resolver that reads the cache and optionally drives a refresher.
|
||||
|
||||
#### Integration seam back into `@suaveplan/webauthn`
|
||||
|
||||
- ADDED (in `@suaveplan/webauthn`): optional `metadata?: MetadataResolver` field on `VerifyAttestationInput`. When supplied, after the per-format verifier returns a `trustPath`, `verifyAttestation` resolves the AAGUID (or attestation-cert SKI for `fido-u2f`) and, when a `MetadataLookup` is returned, augments the result with `metadata: MetadataLookup` and downgrades `trusted` to `false` when `currentStatus` is `REVOKED`, `USER_KEY_REMOTE_COMPROMISE`, `USER_KEY_PHYSICAL_COMPROMISE`, `ATTESTATION_KEY_COMPROMISE`, or any other FIDO `AuthenticatorStatus` in the configurable revocation set.
|
||||
- ADDED (in `@suaveplan/webauthn`): `AttestationVerifyResult.metadata?: MetadataLookup` field. Absent when no resolver is injected — strict back-compat.
|
||||
|
||||
#### Errors
|
||||
|
||||
- ADDED: `WEBAUTHN_MDS3.*` codes via `defineCodes`: `BLOB_FETCH_FAILED`, `BLOB_MALFORMED`, `BLOB_SIGNATURE_INVALID`, `BLOB_CHAIN_UNTRUSTED`, `BLOB_EXPIRED`, `BLOB_ALG_DISALLOWED`, `BLOB_SCHEMA_MISMATCH`, `CACHE_READ_FAILED`, `CACHE_WRITE_FAILED`, `RESOLVER_LOOKUP_FAILED`, `REFRESHER_ALREADY_RUNNING`, `REFRESHER_ABORTED`.
|
||||
|
||||
#### Test strategy
|
||||
|
||||
- Vectors of the FIDO Alliance MDS3 sample blob (`__fixtures__/mds3-sample-blob.jwt` + parsed payload) sourced offline from the FIDO Alliance conformance suite. Tests use the `FetchLike` injection seam to return the fixture bytes — no live network in CI.
|
||||
- Negative fixtures: signature-flipped blob, chain-broken blob, expired `nextUpdate`, schema-violation payload.
|
||||
|
||||
### Subpath exports
|
||||
|
||||
- `.` — root barrel: `MetadataResolver`, `createMemoryResolver`, `createCachedResolver`, every type, every error code.
|
||||
- `./fetch` — `fetchMetadataBlob`.
|
||||
- `./verify` — `verifyMetadataBlob`, `MetadataBlobPayloadSchema`.
|
||||
- `./cache` — `MetadataCache`, cache-key constants.
|
||||
- `./refresher` — `createMetadataRefresher`, `MetadataRefresher`.
|
||||
- `./errors` — `WEBAUTHN_MDS3_CODES`, `WebAuthnMds3Error`.
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `@suaveplan/types workspace:^` — `FetchLike`, `StorageLike`, `Clock`, `LoggerLike`.
|
||||
- `@suaveplan/error workspace:^` — `defineCodes`, base error.
|
||||
- `@suaveplan/jws workspace:^` — JWS verification with `x5c` chain.
|
||||
- `@suaveplan/x509 workspace:^` — chain validation to pinned FIDO root.
|
||||
- `@suaveplan/crypto-hash workspace:^` — SHA-256 for AAGUID key derivation and cert thumbprints.
|
||||
- `@suaveplan/crypto-utils workspace:^` — `constantTimeEqual`, base64url.
|
||||
- `@opentelemetry/api catalog:observability` — refresh + lookup metrics.
|
||||
- `tslib catalog:build`.
|
||||
- `zod catalog:forms` — boundary validation.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **MDS v2 / MDS v1 compatibility.** MDS3 is the only metadata version on the FIDO Alliance roadmap; bundling deprecated MDS2 expands scope and ships dead code for new deployments.
|
||||
- **Per-RP custom metadata sources.** The package only speaks the FIDO Alliance MDS3 wire format. Custom metadata catalogues (enterprise allow-lists) should implement `MetadataResolver` directly — that interface is the integration seam, not the MDS3 wire format.
|
||||
- **Authenticator-status policy decisions.** The package surfaces `currentStatus` and `statusReports`; the relying party decides what status set to treat as revocation. The default revocation set is opinionated but configurable.
|
||||
- **CRL / OCSP polling for attestation-root chains.** Out of band for MDS3 — root pinning + the published metadata `nextUpdate` cadence is the FIDO-Alliance-blessed freshness model.
|
||||
|
||||
## Source mapping
|
||||
|
||||
| Requirement | Source | Notes |
|
||||
|---|---|---|
|
||||
| Why an MDS3 integration is needed | `openspec/archive/2026-05-11-enterprise-webauthn/proposal.md` "Out of scope" | Verbatim deferral text |
|
||||
| `trustPath` + `attestationType` consumed downstream | `packages/auth/webauthn/src/verifier/registry.ts` L14-L18 | `AttestationVerifyResult` is the integration seam |
|
||||
| AAGUID parse + carry on attested-credential-data | `packages/auth/webauthn/src/parser/authenticator-data.ts` L73-L84 | Already available at attestation-verification time |
|
||||
| `none`/`fido-u2f`/`packed` doc anchors flag MDS3 as caller responsibility | `packages/auth/webauthn/src/attestation/{packed,fido-u2f,none}/*.md` | Today's wording moves into the `MetadataResolver` |
|
||||
| MDS3 wire format | FIDO Alliance "Metadata Service Specification" v3.0 §3.1 | External standard anchor |
|
||||
| FIDO Alliance root CA pin | <https://mds.fidoalliance.org/Root.cer> | Pinned in `constants/fido-root.ts` |
|
||||
| JWS verification | RFC 7515 + `@suaveplan/jws` | Internal contract |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement in `specs/webauthn-mds3/spec.md` and every modified requirement in `specs/webauthn/spec.md` carries a `**Verified by:**` line naming planned test files. Test files (all paths relative to `packages/auth/webauthn-mds3/` unless noted):
|
||||
|
||||
- `src/fetch/fetch-blob.test.ts` — happy path via injected `FetchLike`; non-2xx surfaces `BLOB_FETCH_FAILED`; `AbortSignal` cancellation.
|
||||
- `src/verify/verify-blob.test.ts` — sample-blob signature verifies; flipped signature → `BLOB_SIGNATURE_INVALID`; broken chain → `BLOB_CHAIN_UNTRUSTED`; expired `nextUpdate` → `BLOB_EXPIRED`; HS256 → `BLOB_ALG_DISALLOWED`; schema mismatch → `BLOB_SCHEMA_MISMATCH`.
|
||||
- `src/cache/metadata-cache.test.ts` — round-trip through `StorageLike`; read/write failure mapping.
|
||||
- `src/refresher/metadata-refresher.test.ts` — refreshes at `nextUpdate - jitter`; falls back to `intervalMs`; reentrancy guard fires `REFRESHER_ALREADY_RUNNING`; `signal.abort()` fires `REFRESHER_ABORTED`; OTel counter records per outcome.
|
||||
- `src/resolver/memory.test.ts` — AAGUID lookup hit / miss; `attestationCertificateKeyId` lookup for U2F.
|
||||
- `src/resolver/cached.test.ts` — reads cache; lazy-refreshes when stale.
|
||||
- `src/errors/errors.test.ts` — every `WEBAUTHN_MDS3.*` code is thrown by at least one production path.
|
||||
- `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"augments result with metadata when resolver injected"` AND `"::"downgrades trusted=false when currentStatus is revoked"` AND `"::"omits metadata field when no resolver injected (back-compat)"` — integration into the existing `@suaveplan/webauthn` verifier.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/webauthn-mds3` after landing — at draft time the package is new.
|
||||
|
||||
- `@suaveplan/webauthn` — [x] tests run / [x] docs reviewed / [ ] no action needed. Gains the optional `metadata?: MetadataResolver` field on `VerifyAttestationInput` and the optional `metadata?: MetadataLookup` field on `AttestationVerifyResult`. Both are additive; existing call sites that do not inject a resolver see byte-identical behaviour.
|
||||
- `@suaveplan/vault` — [ ] tests run / [x] docs reviewed / [x] no action needed (reason: vault unlock consumes `verifyAssertion`, not `verifyAttestation`; MDS3 surfaces only on registration paths the vault doesn't drive).
|
||||
- `@suaveplan/react-auth` — [ ] tests run / [x] docs reviewed / [x] no action needed (reason: hook surface unchanged; MDS3 plugs into server-side verifier).
|
||||
|
||||
## Migration
|
||||
|
||||
No prior MDS3 surface — additive only. Behaviour change for `verifyAttestation` consumers is opt-in (only fires when `metadata: MetadataResolver` is passed). Consumers that previously open-coded an MDS3 lookup over the returned `trustPath` MAY migrate to the injected seam at their own pace; the old "ignore `metadata`, do it yourself" path is permanently supported.
|
||||
@@ -0,0 +1,222 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: webauthn-mds3 (webauthn-mds3)
|
||||
|
||||
**Package**: `@suaveplan/webauthn-mds3`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: MDS3 blob fetch via injected FetchLike
|
||||
|
||||
The package SHALL export `fetchMetadataBlob({ fetch, url?, signal? }): Promise<Uint8Array>` from the `./fetch` subpath. The function SHALL call the injected `FetchLike` (per `@suaveplan/types`) against `url` (default `https://mds.fidoalliance.org/`), return the response body as a `Uint8Array`, map any non-2xx status to `WEBAUTHN_MDS3.BLOB_FETCH_FAILED` carrying the status code in `context.status`, and forward `signal` so caller-side cancellation triggers the underlying `AbortController` path.
|
||||
|
||||
#### Scenario: returns blob bytes on 200
|
||||
|
||||
- **WHEN** `fetchMetadataBlob({ fetch: stub200WithBytes, url: "https://example.test/mds" })` is called
|
||||
- **THEN** the resolved value `equalsBytes(stubBytes)`
|
||||
|
||||
#### Scenario: maps non-2xx to BLOB_FETCH_FAILED
|
||||
|
||||
- **WHEN** `fetchMetadataBlob({ fetch: stub503 })` is called
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_FETCH_FAILED` and `error.context.status === 503`
|
||||
|
||||
#### Scenario: forwards AbortSignal
|
||||
|
||||
- **GIVEN** an `AbortController` whose `signal` is passed
|
||||
- **WHEN** the controller is aborted before the stub fetch resolves
|
||||
- **THEN** the call rejects, the stub fetch received the same `signal`, and the rejection's code is `WEBAUTHN_MDS3.BLOB_FETCH_FAILED` with `cause.name === "AbortError"`
|
||||
|
||||
**Verified by:** `src/fetch/fetch-blob.test.ts::"returns blob bytes on 200"` AND `src/fetch/fetch-blob.test.ts::"maps non-2xx to BLOB_FETCH_FAILED"` AND `src/fetch/fetch-blob.test.ts::"forwards AbortSignal to the injected fetch"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: MDS3 blob signature verification with pinned FIDO root
|
||||
|
||||
The package SHALL export `verifyMetadataBlob({ jws, trustAnchors? }): Promise<MetadataBlobPayload>` from the `./verify` subpath. The function SHALL parse the compact JWS, enforce the algorithm allow-list `{ RS256, ES256, ES384 }` (rejecting any other with `WEBAUTHN_MDS3.BLOB_ALG_DISALLOWED`), validate the `x5c` certificate chain to the pinned FIDO Alliance MDS3 root CA (or to caller-supplied `trustAnchors`) via `@suaveplan/x509`, verify the signature via `@suaveplan/jws` and `@suaveplan/crypto-rsa`/`@suaveplan/crypto-curve` per the header `alg`, decode the JSON payload, Zod-validate against `MetadataBlobPayloadSchema`, and reject with `WEBAUTHN_MDS3.BLOB_EXPIRED` when `now > payload.nextUpdate`.
|
||||
|
||||
#### Scenario: verifies the FIDO sample blob
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws: SAMPLE_BLOB_JWT })` is called against the bundled FIDO Alliance conformance sample
|
||||
- **THEN** the resolved value is a typed `MetadataBlobPayload` with `legalHeader`, `no`, `nextUpdate`, and at least one `entries[]` element
|
||||
|
||||
#### Scenario: rejects flipped signature
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws: CORRUPTED_BLOB_JWT })` is called against the bundled signature-flipped variant
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_SIGNATURE_INVALID`
|
||||
|
||||
#### Scenario: rejects untrusted chain
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws })` is called with a JWS whose `x5c` chain terminates at a non-pinned root
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_CHAIN_UNTRUSTED`
|
||||
|
||||
#### Scenario: rejects disallowed algorithm
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws })` is called with a JWS whose header `alg === "HS256"`
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_ALG_DISALLOWED`
|
||||
|
||||
#### Scenario: rejects expired blob
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws })` is called and `Clock.now() > payload.nextUpdate`
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_EXPIRED`
|
||||
|
||||
#### Scenario: rejects schema mismatch
|
||||
|
||||
- **WHEN** `verifyMetadataBlob({ jws })` is called against a payload missing the required `entries` field
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.BLOB_SCHEMA_MISMATCH`
|
||||
|
||||
**Verified by:** `src/verify/verify-blob.test.ts::"verifies the FIDO sample blob"` AND `src/verify/verify-blob.test.ts::"rejects flipped signature"` AND `src/verify/verify-blob.test.ts::"rejects untrusted chain"` AND `src/verify/verify-blob.test.ts::"rejects disallowed algorithm"` AND `src/verify/verify-blob.test.ts::"rejects expired blob"` AND `src/verify/verify-blob.test.ts::"rejects schema mismatch"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Metadata cache via injected StorageLike
|
||||
|
||||
The package SHALL export `MetadataCache` from the `./cache` subpath that wraps an injected `StorageLike<MetadataBlobPayload>` under the key namespace `webauthn-mds3:blob:v1`. The cache SHALL surface storage read failures as `WEBAUTHN_MDS3.CACHE_READ_FAILED` and write failures as `WEBAUTHN_MDS3.CACHE_WRITE_FAILED`, attaching the underlying error as `cause`. The cache SHALL refuse to write a payload whose `no` field is strictly lower than the currently-cached payload's `no` (rollback protection).
|
||||
|
||||
#### Scenario: round-trips a verified payload
|
||||
|
||||
- **WHEN** `cache.set(payload)` then `cache.get()` is called against an in-memory `StorageLike`
|
||||
- **THEN** the returned payload deep-equals the written payload
|
||||
|
||||
#### Scenario: surfaces storage read failure
|
||||
|
||||
- **GIVEN** a `StorageLike` whose `get` rejects with `new Error("disk full")`
|
||||
- **WHEN** `cache.get()` is called
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.CACHE_READ_FAILED` and `error.cause.message === "disk full"`
|
||||
|
||||
#### Scenario: refuses to write a rolled-back payload
|
||||
|
||||
- **GIVEN** a cache holding a payload with `no: 42`
|
||||
- **WHEN** `cache.set(payloadWithNo41)` is called
|
||||
- **THEN** the call rejects with `WEBAUTHN_MDS3.CACHE_WRITE_FAILED` and `error.context.reason === "rollback"`
|
||||
|
||||
**Verified by:** `src/cache/metadata-cache.test.ts::"round-trips a verified payload"` AND `src/cache/metadata-cache.test.ts::"surfaces storage read failure as CACHE_READ_FAILED"` AND `src/cache/metadata-cache.test.ts::"refuses to write a rolled-back payload"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Scheduled metadata refresher
|
||||
|
||||
The package SHALL export `createMetadataRefresher({ fetch, storage, clock, logger, intervalMs?, signal? }): MetadataRefresher` from the `./refresher` subpath. The refresher SHALL schedule the next refresh at `payload.nextUpdate - jitter` (uniform jitter in `[-5min, +5min]`) when `nextUpdate` is present, otherwise at `now + intervalMs` (default 24h). The refresher SHALL throw `WEBAUTHN_MDS3.REFRESHER_ALREADY_RUNNING` when `refresh()` is called while a refresh is in flight, and SHALL reject the pending refresh with `WEBAUTHN_MDS3.REFRESHER_ABORTED` when the injected `AbortSignal` fires.
|
||||
|
||||
#### Scenario: schedules at nextUpdate minus jitter
|
||||
|
||||
- **WHEN** a refresher is started against a payload with `nextUpdate = now + 1h`
|
||||
- **THEN** the next `clock.setTimeout` call is in the range `[55min, 65min]` from `now`
|
||||
|
||||
#### Scenario: falls back to intervalMs when nextUpdate absent
|
||||
|
||||
- **WHEN** a refresher is started against a payload with `nextUpdate: undefined` and `intervalMs: 1h`
|
||||
- **THEN** the next `clock.setTimeout` call is at approximately `now + 1h`
|
||||
|
||||
#### Scenario: reentrancy guard fires REFRESHER_ALREADY_RUNNING
|
||||
|
||||
- **WHEN** `refresh()` is called while a prior `refresh()` is still pending
|
||||
- **THEN** the second call rejects with `WEBAUTHN_MDS3.REFRESHER_ALREADY_RUNNING`
|
||||
|
||||
#### Scenario: aborts on signal
|
||||
|
||||
- **GIVEN** a refresher started with `signal: ac.signal`
|
||||
- **WHEN** `ac.abort()` is called mid-fetch
|
||||
- **THEN** the pending refresh rejects with `WEBAUTHN_MDS3.REFRESHER_ABORTED`
|
||||
|
||||
#### Scenario: records refresh outcome counter
|
||||
|
||||
- **WHEN** a refresh completes successfully
|
||||
- **THEN** the OTel counter `webauthn_mds3.refresh_total{outcome="ok"}` increments by 1
|
||||
|
||||
**Verified by:** `src/refresher/metadata-refresher.test.ts::"schedules at nextUpdate minus jitter"` AND `src/refresher/metadata-refresher.test.ts::"falls back to intervalMs when nextUpdate absent"` AND `src/refresher/metadata-refresher.test.ts::"reentrancy guard fires REFRESHER_ALREADY_RUNNING"` AND `src/refresher/metadata-refresher.test.ts::"aborts on signal"` AND `src/refresher/metadata-refresher.test.ts::"records refresh outcome counter"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AAGUID lookup API
|
||||
|
||||
The package SHALL export a `MetadataResolver` interface with `resolveByAaguid(aaguid: Uint8Array): Promise<MetadataLookup | undefined>` and `resolveByAttestationCertificateKeyId(keyId: Uint8Array): Promise<MetadataLookup | undefined>`. The package SHALL ship two factories: `createMemoryResolver({ blob })` (synchronous in-memory over a verified payload) and `createCachedResolver({ cache, refresher? })` (production resolver that reads through the cache and optionally starts the refresher on first access). The AAGUID comparator SHALL use `constantTimeEqual` from `@suaveplan/crypto-utils`. The `attestationCertificateKeyId` comparator SHALL match on the byte-equal SHA-1 SKI of the leaf attestation certificate (FIDO MDS3 §3.1.6 — U2F authenticators have no AAGUID).
|
||||
|
||||
#### Scenario: AAGUID hit returns a typed MetadataLookup
|
||||
|
||||
- **GIVEN** a memory resolver seeded with a payload containing an entry whose AAGUID is `0x11223344...`
|
||||
- **WHEN** `resolveByAaguid(0x11223344...)` is called
|
||||
- **THEN** the resolved value has `statement.aaguid === "11223344-..."`, `currentStatus` set, `attestationRootCertificates.length >= 1`, and `lastUpdate > 0`
|
||||
|
||||
#### Scenario: AAGUID miss returns undefined
|
||||
|
||||
- **WHEN** `resolveByAaguid(zerosBytes)` is called against a resolver whose payload has no zero-AAGUID entry
|
||||
- **THEN** the resolved value is `undefined`
|
||||
|
||||
#### Scenario: U2F attestation-cert SKI lookup
|
||||
|
||||
- **GIVEN** a memory resolver seeded with a payload containing an entry whose `attestationCertificateKeyIdentifiers` includes a specific SHA-1 SKI
|
||||
- **WHEN** `resolveByAttestationCertificateKeyId(thatSki)` is called
|
||||
- **THEN** the resolved value's `statement` matches that entry
|
||||
|
||||
#### Scenario: cached resolver lazy-refreshes when cache empty
|
||||
|
||||
- **GIVEN** a cached resolver whose cache is empty and whose refresher is wired
|
||||
- **WHEN** `resolveByAaguid(any)` is called
|
||||
- **THEN** the refresher's `refresh()` is invoked exactly once before the resolver returns
|
||||
|
||||
**Verified by:** `src/resolver/memory.test.ts::"AAGUID hit returns a typed MetadataLookup"` AND `src/resolver/memory.test.ts::"AAGUID miss returns undefined"` AND `src/resolver/memory.test.ts::"U2F attestation-cert SKI lookup"` AND `src/resolver/cached.test.ts::"lazy-refreshes when cache empty"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Error-code registry
|
||||
|
||||
The package SHALL declare its error codes via `defineCodes("WEBAUTHN_MDS3", [...])` covering `BLOB_FETCH_FAILED`, `BLOB_MALFORMED`, `BLOB_SIGNATURE_INVALID`, `BLOB_CHAIN_UNTRUSTED`, `BLOB_EXPIRED`, `BLOB_ALG_DISALLOWED`, `BLOB_SCHEMA_MISMATCH`, `CACHE_READ_FAILED`, `CACHE_WRITE_FAILED`, `RESOLVER_LOOKUP_FAILED`, `REFRESHER_ALREADY_RUNNING`, and `REFRESHER_ABORTED`. Every declared code SHALL be thrown by at least one production call site exercised by an existing test.
|
||||
|
||||
#### Scenario: every code is reachable
|
||||
|
||||
- **WHEN** the test harness greps `src/` for `WEBAUTHN_MDS3_CODES.<CODE>` references for every declared `CODE`
|
||||
- **THEN** every code has at least one reference outside `src/errors/`
|
||||
|
||||
**Verified by:** `src/errors/errors.test.ts::"every WEBAUTHN_MDS3 code is thrown by at least one production path"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Observability counters and histogram
|
||||
|
||||
The package SHALL emit `webauthn_mds3.refresh_total{outcome}` and `webauthn_mds3.lookup_total{outcome}` counters and a `webauthn_mds3.blob_age_ms` histogram via `@opentelemetry/api`. When no `MeterProvider` is registered the metrics SHALL no-op silently.
|
||||
|
||||
#### Scenario: refresh_total records on success
|
||||
|
||||
- **WHEN** a refresh completes successfully against an in-memory `MeterProvider`
|
||||
- **THEN** `webauthn_mds3.refresh_total{outcome="ok"}` is `1`
|
||||
|
||||
#### Scenario: lookup_total records on miss
|
||||
|
||||
- **WHEN** `resolveByAaguid` returns `undefined`
|
||||
- **THEN** `webauthn_mds3.lookup_total{outcome="miss"}` is `1`
|
||||
|
||||
#### Scenario: blob_age_ms records on read
|
||||
|
||||
- **WHEN** `cache.get()` returns a payload aged 1h
|
||||
- **THEN** `webauthn_mds3.blob_age_ms` has a recorded value in the bucket containing 3_600_000
|
||||
|
||||
**Verified by:** `src/observability/metrics.test.ts::"refresh_total records on success"` AND `src/observability/metrics.test.ts::"lookup_total records on miss"` AND `src/observability/metrics.test.ts::"blob_age_ms records on read"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Universal-tier purity
|
||||
|
||||
The package SHALL NOT import any DOM, browser, or Node-only API. `FetchLike`, `StorageLike`, `Clock`, and `LoggerLike` SHALL be injected through the public API surface so the package runs unmodified on Bun, Node, Cloudflare Workers, Deno, and inside a browser bundle.
|
||||
|
||||
#### Scenario: dep-graph harness sees no DOM types
|
||||
|
||||
- **WHEN** the dep-graph test sweeps the public surface for `Window`, `Document`, `localStorage`, `Blob` references
|
||||
- **THEN** zero matches are found
|
||||
|
||||
**Verified by:** `src/dep-graph.test.ts::"public surface imports zero DOM or Node-only types"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Subpath exports
|
||||
|
||||
The package SHALL expose subpath exports for `.`, `./fetch`, `./verify`, `./cache`, `./refresher`, and `./errors` in its `package.json` `exports` map so consumers can tree-shake unused capabilities.
|
||||
|
||||
#### Scenario: every subpath resolves
|
||||
|
||||
- **WHEN** the package's `index.test.ts` imports from each subpath
|
||||
- **THEN** every import resolves to the expected module and the bundler tree-shakes unused entries
|
||||
|
||||
**Verified by:** `src/index.test.ts::"every documented subpath resolves through the package's exports map"`
|
||||
@@ -0,0 +1,49 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: webauthn (webauthn)
|
||||
|
||||
**Package**: `@suaveplan/webauthn`
|
||||
**Status**: PROPOSED (companion delta of `add-webauthn-mds3`)
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Attestation verification dispatch
|
||||
|
||||
The package SHALL export `verifyAttestation(input: VerifyAttestationInput): Promise<AttestationVerifyResult>` from the `./verifier` subpath. The function SHALL dispatch on `input.attestationObject.fmt` to the per-format verifier registered in `input.registry`, throwing `WEBAUTHN.ATTESTATION_FORMAT_UNSUPPORTED` when no verifier is registered. `VerifyAttestationInput` SHALL accept an optional `metadata?: MetadataResolver` field (typed via `import type` from `@suaveplan/webauthn-mds3` — runtime-optional) and an optional `revokedStatuses?: ReadonlySet<AuthenticatorStatus>` field. `AttestationVerifyResult` SHALL carry an optional `metadata?: MetadataLookup` field. When `input.metadata` is `undefined` the returned `AttestationVerifyResult` SHALL be byte-identical to the pre-modification contract (strict back-compat). When `input.metadata` is supplied, after the per-format verifier returns, `verifyAttestation` SHALL resolve the AAGUID — or, for `fmt === "fido-u2f"` or an all-zero AAGUID, the SHA-1 SKI of the leaf attestation certificate (FIDO MDS3 §3.1.6) — call `resolveByAaguid` / `resolveByAttestationCertificateKeyId` on the supplied resolver, set `result.metadata` to the returned `MetadataLookup` (or omit when `undefined`), and downgrade `result.trusted` to `false` when the lookup's `currentStatus` is in `revokedStatuses ?? DEFAULT_REVOCATION_STATUSES`.
|
||||
|
||||
#### Scenario: dispatches to registered format verifier
|
||||
|
||||
- **WHEN** `verifyAttestation({ attestationObject: { fmt: "packed", ... }, registry: defaultAttestationRegistry(), ... })` is called
|
||||
- **THEN** the result is produced by the `packed` verifier and includes `trustPath`, `attestationType`, `trusted`
|
||||
|
||||
#### Scenario: throws WEBAUTHN.ATTESTATION_FORMAT_UNSUPPORTED on unknown fmt
|
||||
|
||||
- **WHEN** `verifyAttestation({ attestationObject: { fmt: "unknown-fmt", ... }, registry: emptyRegistry })` is called
|
||||
- **THEN** the call rejects with `WEBAUTHN.ATTESTATION_FORMAT_UNSUPPORTED`
|
||||
|
||||
#### Scenario: omits metadata field when no resolver injected (back-compat)
|
||||
|
||||
- **WHEN** `verifyAttestation(input)` is called with `input.metadata === undefined`
|
||||
- **THEN** the resolved `AttestationVerifyResult` has `metadata === undefined` and is byte-for-byte identical to the legacy (pre-modification) result for the same inputs
|
||||
|
||||
#### Scenario: augments result with metadata when resolver injected
|
||||
|
||||
- **GIVEN** a `MetadataResolver` whose `resolveByAaguid` returns a `MetadataLookup` with `currentStatus === "FIDO_CERTIFIED_L1"`
|
||||
- **WHEN** `verifyAttestation({ ..., metadata: resolver })` is called against a `packed` attestation carrying a known AAGUID
|
||||
- **THEN** `result.metadata` deep-equals the lookup and `result.trusted` is the verifier's original value (unchanged)
|
||||
|
||||
#### Scenario: downgrades trusted=false when currentStatus is revoked
|
||||
|
||||
- **GIVEN** a `MetadataResolver` whose `resolveByAaguid` returns a `MetadataLookup` with `currentStatus === "REVOKED"`
|
||||
- **WHEN** `verifyAttestation({ ..., metadata: resolver })` is called and the per-format verifier returned `trusted: true`
|
||||
- **THEN** the merged `result.trusted` is `false` and `result.metadata.currentStatus === "REVOKED"`
|
||||
|
||||
#### Scenario: resolves U2F by attestation-cert SKI
|
||||
|
||||
- **GIVEN** a `MetadataResolver` and a `fido-u2f` attestation whose leaf cert has SKI `0xDEAD…`
|
||||
- **WHEN** `verifyAttestation({ ..., metadata: resolver })` is called
|
||||
- **THEN** the resolver's `resolveByAttestationCertificateKeyId(0xDEAD…)` is invoked exactly once and `resolveByAaguid` is never called
|
||||
|
||||
**Verified by:** `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"dispatches to registered format verifier"` AND `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"throws WEBAUTHN.ATTESTATION_FORMAT_UNSUPPORTED on unknown fmt"` AND `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"omits metadata field when no resolver injected (back-compat)"` AND `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"augments result with metadata when resolver injected"` AND `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"downgrades trusted=false when currentStatus is revoked"` AND `packages/auth/webauthn/src/verifier/verify-attestation.test.ts::"resolves U2F by attestation-cert SKI"`
|
||||
@@ -0,0 +1,98 @@
|
||||
# Tasks — `add-webauthn-mds3`
|
||||
|
||||
## 1. Package scaffolding
|
||||
|
||||
- [ ] 1.1 Create `packages/auth/webauthn-mds3/` per genesis package layout.
|
||||
- [ ] 1.2 `package.json` at `version: 0.0.0`, `sideEffects: false`, `workspace:^` for internal deps, `catalog:*` for external.
|
||||
- [ ] 1.3 `tsconfig.json` extending `@suaveplan/typescript-config` universal preset.
|
||||
- [ ] 1.4 `vitest.config.ts` using `createTestConfig()` from `@suaveplan/testing/config`; coverage thresholds 100 / 100 / 100 / 100; `provider: 'istanbul'`.
|
||||
- [ ] 1.5 Subpath exports map in `package.json`: `.`, `./fetch`, `./verify`, `./cache`, `./refresher`, `./errors`.
|
||||
- [ ] 1.6 Root `README.md` with one-paragraph description, install, quick-start, Modules section linking to every co-located `.md`.
|
||||
|
||||
## 2. Constants + FIDO root pinning
|
||||
|
||||
- [ ] 2.1 `src/constants/fido-root.ts` + `.md` — pin the FIDO Alliance MDS3 root CA DER bytes + SHA-256 fingerprint; export `FIDO_MDS3_DEFAULT_URL`, `DEFAULT_REFRESH_INTERVAL_MS`, `DEFAULT_REVOCATION_STATUSES`.
|
||||
|
||||
## 3. Error codes
|
||||
|
||||
- [ ] 3.1 `src/errors/error-codes.ts` + `.md` — `defineCodes("WEBAUTHN_MDS3", [...])` with every code from the spec delta; export `WebAuthnMds3Error`.
|
||||
- [ ] 3.2 `src/errors/errors.test.ts` — assert every code is reachable from a production path (call-site grep harness).
|
||||
|
||||
## 4. Schemas
|
||||
|
||||
- [ ] 4.1 `src/schemas/metadata-blob.schemas.ts` + `.md` — `MetadataBlobPayloadSchema`, `MetadataStatementSchema`, `AuthenticatorStatusSchema`, `StatusReportSchema`, `BiometricStatusReportSchema`, `EcdaaTrustAnchorSchema`, `MetadataBlobEntrySchema` tracking FIDO MDS3 §3.1.
|
||||
|
||||
## 5. Fetch
|
||||
|
||||
- [ ] 5.1 `src/fetch/fetch-blob.ts` + `.md` — `fetchMetadataBlob({ fetch, url?, signal? })`; maps non-2xx → `BLOB_FETCH_FAILED`; honors `AbortSignal`.
|
||||
- [ ] 5.2 `src/fetch/fetch-blob.test.ts` — happy path; non-2xx; abort.
|
||||
|
||||
## 6. Verify
|
||||
|
||||
- [ ] 6.1 `src/verify/verify-blob.ts` + `.md` — verify compact JWS via `@suaveplan/jws`; pin alg allow-list (RS256, ES256, ES384); validate `x5c` chain to FIDO root via `@suaveplan/x509`; check `nextUpdate`; decode + Zod-validate payload.
|
||||
- [ ] 6.2 `src/verify/verify-blob.test.ts` — happy, signature-flipped, chain-broken, expired, alg-disallowed, schema-mismatch (use sample-blob fixture).
|
||||
|
||||
## 7. Cache
|
||||
|
||||
- [ ] 7.1 `src/cache/metadata-cache.ts` + `.md` — wraps injected `StorageLike<MetadataBlobPayload>` with key namespace `webauthn-mds3:blob:v1`; surfaces `CACHE_READ_FAILED` / `CACHE_WRITE_FAILED`.
|
||||
- [ ] 7.2 `src/cache/metadata-cache.test.ts` — round-trip; injected storage failure mapping.
|
||||
|
||||
## 8. Refresher
|
||||
|
||||
- [ ] 8.1 `src/refresher/metadata-refresher.ts` + `.md` — `createMetadataRefresher({ fetch, storage, clock, logger, intervalMs?, signal? })`; reentrancy guard; OTel counter; jittered scheduling at `nextUpdate - jitter`.
|
||||
- [ ] 8.2 `src/refresher/metadata-refresher.test.ts` — schedule, reentrancy guard, abort, OTel counter, fallback to `intervalMs`.
|
||||
|
||||
## 9. Resolver
|
||||
|
||||
- [ ] 9.1 `src/resolver/types.ts` + `.md` — `MetadataResolver`, `MetadataLookup` interfaces (no name collisions per AGENTS.md §10 — folder is `resolver/`, files use feature names not `*-resolver.ts`).
|
||||
- [ ] 9.2 `src/resolver/memory.ts` + `.md` + `memory.test.ts` — `createMemoryResolver` for verified payloads.
|
||||
- [ ] 9.3 `src/resolver/cached.ts` + `.md` + `cached.test.ts` — `createCachedResolver` reading the cache + optional refresher drive.
|
||||
|
||||
## 10. Observability
|
||||
|
||||
- [ ] 10.1 `src/observability/metrics.ts` + `.md` — `webauthn_mds3.refresh_total{outcome}`, `webauthn_mds3.lookup_total{outcome}` counters; `webauthn_mds3.blob_age_ms` histogram. No-op when no exporter.
|
||||
- [ ] 10.2 `src/observability/metrics.test.ts` — every metric records on its production path against an in-memory `MeterProvider`.
|
||||
|
||||
## 11. Fixtures
|
||||
|
||||
- [ ] 11.1 `src/__fixtures__/mds3-sample-blob.jwt` — captured FIDO Alliance MDS3 conformance-suite sample blob.
|
||||
- [ ] 11.2 `src/__fixtures__/mds3-corrupted-blob.jwt` — signature-flipped variant for negative tests.
|
||||
- [ ] 11.3 `src/__fixtures__/index.ts` — typed fixture exports.
|
||||
|
||||
## 12. Benchmarks
|
||||
|
||||
- [ ] 12.1 `src/__benchmarks__/verify-blob.bench.ts` — JWS verify throughput on the sample blob.
|
||||
- [ ] 12.2 `src/__benchmarks__/resolve-aaguid.bench.ts` — AAGUID lookup throughput.
|
||||
|
||||
## 13. Integration into `@suaveplan/webauthn`
|
||||
|
||||
- [ ] 13.1 Extend `VerifyAttestationInput` with optional `metadata?: MetadataResolver` field (typed via `import type` from `@suaveplan/webauthn-mds3` — no runtime dep).
|
||||
- [ ] 13.2 Extend `AttestationVerifyResult` with optional `metadata?: MetadataLookup` field.
|
||||
- [ ] 13.3 Wire post-verifier resolution: AAGUID for non-U2F formats, attestation-cert SKI for `fido-u2f`; downgrade `trusted=false` on revoked statuses; configurable revocation set with default from `DEFAULT_REVOCATION_STATUSES`.
|
||||
- [ ] 13.4 Update `packages/auth/webauthn/src/verifier/verify-attestation.test.ts` — three new tests: resolver injected augments result; revoked status downgrades trust; no resolver = byte-identical legacy behaviour.
|
||||
- [ ] 13.5 Refresh `packages/auth/webauthn/src/verifier/verify-attestation.md` and the three attestation doc anchors (`packed.md`, `fido-u2f.md`, `none.md`) so the "caller wires MDS3" language now mentions `MetadataResolver` as the canonical seam.
|
||||
- [ ] 13.6 Add `@suaveplan/webauthn-mds3` to `packages/auth/webauthn/README.md` Modules section as the recommended MDS3 sidecar.
|
||||
|
||||
## 14. Pre-publish gate
|
||||
|
||||
- [ ] 14.1 `bunx turbo lint typecheck test --filter=@suaveplan/webauthn-mds3` green.
|
||||
- [ ] 14.2 `bunx turbo test:coverage --filter=@suaveplan/webauthn-mds3` 100 / 100 / 100 / 100.
|
||||
- [ ] 14.3 `bunx turbo lint typecheck test --filter=@suaveplan/webauthn` green (dependent sweep — §13 step 10).
|
||||
- [ ] 14.4 Banned-pattern scan: zero matches per §13 step 4.
|
||||
- [ ] 14.5 Barrel-shape scan (§13 step 4a): zero matches.
|
||||
- [ ] 14.6 Test-runner import scan (§13 step 4b): zero matches.
|
||||
- [ ] 14.7 Missing-`.md` scan: zero misses.
|
||||
- [ ] 14.8 `bunx markdownlint-cli2 --fix` then second pass with no `--fix` exits 0.
|
||||
|
||||
## 15. Repo-root manifest refresh (§13b)
|
||||
|
||||
- [ ] 15.1 `bun scripts/folder-reorg/generate-manifest.ts` → `MANIFEST.md` refreshed.
|
||||
- [ ] 15.2 `bun scripts/generate-missing.ts` → `MISSING.yml` refreshed.
|
||||
- [ ] 15.3 `STATUS.md` updated: add `@suaveplan/webauthn-mds3` row.
|
||||
- [ ] 15.4 `SUMMARY.md` paragraph for `webauthn-mds3` added.
|
||||
|
||||
## 16. Fold + archive
|
||||
|
||||
- [ ] 16.1 Fold `specs/webauthn-mds3/spec.md` into `openspec/specs/webauthn-mds3/spec.md`.
|
||||
- [ ] 16.2 Fold `specs/webauthn/spec.md` `## MODIFIED Requirements` into the canonical `openspec/specs/webauthn/spec.md`.
|
||||
- [ ] 16.3 Move `openspec/changes/add-webauthn-mds3/` → `openspec/archive/<yyyy-mm-dd>-add-webauthn-mds3/`.
|
||||
@@ -0,0 +1,88 @@
|
||||
# OpenSpec Change: enterprise-react-auth
|
||||
|
||||
- **Change ID**: enterprise-react-auth
|
||||
- **Package**: `@suaveplan/react-auth`
|
||||
- **Category**: react
|
||||
- **Target path**: `react/auth/react-auth`
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-04-24
|
||||
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/react-auth.md`
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Deliver `@suaveplan/react-auth` on genesis from scratch at enterprise quality. The core React authentication layer: `AuthProvider`, `useAuth`, `useUser`, `useIsAuthenticated`, token management, session persistence, and integration with `@suaveplan/oauth-client` for Authorization Code + PKCE flows. Ships structured errors via `@suaveplan/error`, OTel span events for sign-in/sign-out/refresh, Storybook stories, and Playwright E2E for the full auth lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## Current-state audit (genesis)
|
||||
|
||||
Package does not exist on genesis — fresh build.
|
||||
|
||||
---
|
||||
|
||||
## Available workspace deps on genesis
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@suaveplan/error` (`workspace:^`) | `AuthError` |
|
||||
| `@suaveplan/types` (`workspace:^`) | `LoggerLike` |
|
||||
| `@suaveplan/events` (`workspace:^`) | Auth state change events |
|
||||
| `@suaveplan/oauth-client` (`workspace:^`) | PKCE + token refresh |
|
||||
| `@suaveplan/react-translate` (`workspace:^`) | i18n |
|
||||
| `zod` (`catalog:forms`) | Auth config schemas |
|
||||
| `@opentelemetry/api` (`catalog:observability`) | Auth lifecycle spans |
|
||||
|
||||
External: `tslib catalog:build`. React 18 peer.
|
||||
|
||||
---
|
||||
|
||||
## Public API surface
|
||||
|
||||
```ts
|
||||
AuthProvider, AuthContext
|
||||
useAuth, useUser, useIsAuthenticated, useAuthToken
|
||||
AuthGuard — wraps children, redirects to login if unauthenticated
|
||||
withAuth(Component) — HOC variant of AuthGuard
|
||||
|
||||
// Types
|
||||
AuthConfig, AuthUser, AuthState, AuthContextValue,
|
||||
AuthProviderProps, AuthGuardProps, SignInOptions, SignOutOptions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New additions for genesis (full build)
|
||||
|
||||
1. **`AuthError`** — codes: `AUTH_UNAUTHENTICATED`, `AUTH_TOKEN_EXPIRED`, `AUTH_REFRESH_FAILED`, `AUTH_CONTEXT_MISSING`.
|
||||
2. **Token refresh** via `@suaveplan/oauth-client` with automatic retry on 401.
|
||||
3. **OTel spans** — `auth.sign_in`, `auth.sign_out`, `auth.token_refresh`.
|
||||
4. **Session persistence** — optional `sessionStorage` / `localStorage` adapter.
|
||||
5. **Stories** — authenticated state, unauthenticated, loading, `AuthGuard` redirect.
|
||||
6. **Playwright E2E** — sign-in flow, token refresh, protected route redirect.
|
||||
|
||||
---
|
||||
|
||||
## File layout
|
||||
|
||||
```text
|
||||
react/auth/react-auth/
|
||||
package.json
|
||||
tsconfig.json
|
||||
vitest.config.ts
|
||||
src/
|
||||
index.ts
|
||||
context/
|
||||
auth-context/ (tsx + types + md + test + index)
|
||||
hooks/
|
||||
use-auth/ use-user/ use-is-authenticated/ use-auth-token/
|
||||
components/
|
||||
auth-guard/ (tsx + types + md + test + stories + index)
|
||||
utils/
|
||||
token-storage/ session-persistence/
|
||||
types/
|
||||
auth.types.ts
|
||||
e2e/
|
||||
auth.spec.ts
|
||||
```
|
||||
@@ -0,0 +1,79 @@
|
||||
# Tasks: enterprise-react-auth
|
||||
|
||||
**Change ID**: enterprise-react-auth
|
||||
**Package**: `@suaveplan/react-auth`
|
||||
**Target path**: `react/auth/react-auth`
|
||||
**Prerequisite packages on Verdaccio**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/oauth-client`, `@suaveplan/react-translate`
|
||||
|
||||
---
|
||||
|
||||
## 1. Scaffold
|
||||
|
||||
- [ ] Create directory tree per proposal layout.
|
||||
- [ ] Write `package.json` at `0.1.0` with `"sideEffects": false`, exports map.
|
||||
- [ ] Dependencies: `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/events workspace:^`, `@suaveplan/oauth-client workspace:^`, `@suaveplan/react-translate workspace:^`, `zod catalog:forms`, `@opentelemetry/api catalog:observability`, `tslib catalog:build`.
|
||||
- [ ] Write `tsconfig.json`, `vitest.config.ts`, scripts.
|
||||
|
||||
---
|
||||
|
||||
## 2. Error codes and types
|
||||
|
||||
- [ ] Define `AuthError` extending `SuaveplanError`; codes: `AUTH_UNAUTHENTICATED`, `AUTH_TOKEN_EXPIRED`, `AUTH_REFRESH_FAILED`, `AUTH_CONTEXT_MISSING`.
|
||||
- [ ] Define `AuthUser`, `AuthState`, `AuthConfig`, `AuthContextValue`, `AuthProviderProps`, `AuthGuardProps`, `SignInOptions`, `SignOutOptions`.
|
||||
- [ ] Write `AuthConfigSchema` Zod schema.
|
||||
|
||||
---
|
||||
|
||||
## 3. Token utilities
|
||||
|
||||
- [ ] `src/utils/token-storage/` — `createTokenStorage(adapter: 'memory' | 'localStorage' | 'sessionStorage')`: `setTokens(tokens)`, `getTokens()`, `clearTokens()`.
|
||||
- [ ] `src/utils/session-persistence/` — `saveSession(user)`, `loadSession(): AuthUser | null`, `clearSession()`.
|
||||
- [ ] Write tests + md for each.
|
||||
|
||||
---
|
||||
|
||||
## 4. AuthContext + AuthProvider
|
||||
|
||||
- [ ] State: `user: AuthUser | null`, `isLoading`, `isAuthenticated`.
|
||||
- [ ] `signIn(options: SignInOptions)` — calls `@suaveplan/oauth-client` authorization + token exchange; stores tokens; emits `@suaveplan/events` `auth:signed_in`. OTel `auth.sign_in` span.
|
||||
- [ ] `signOut(options?)` — clears tokens + session; emits `auth:signed_out`. OTel `auth.sign_out` span.
|
||||
- [ ] `refreshToken()` — calls `@suaveplan/oauth-client` token refresh; on failure throws `AuthError(AUTH_REFRESH_FAILED)`. OTel `auth.token_refresh` span.
|
||||
- [ ] Auto-refresh: installs a timer to refresh before expiry.
|
||||
- [ ] `useAuth()` throws `AuthError(AUTH_CONTEXT_MISSING)` outside provider.
|
||||
- [ ] Write `auth-context.test.tsx` (≤500 lines): sign-in, sign-out, refresh, auto-refresh timer, context missing. `{ timeout: 10000 }`.
|
||||
- [ ] Write `auth-context.md`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Hooks
|
||||
|
||||
- [ ] `useUser(): AuthUser | null` — from context.
|
||||
- [ ] `useIsAuthenticated(): boolean`.
|
||||
- [ ] `useAuthToken(): string | null` — current access token.
|
||||
- [ ] Write tests + md for each.
|
||||
|
||||
---
|
||||
|
||||
## 6. AuthGuard + withAuth
|
||||
|
||||
- [ ] `AuthGuard` — renders children when authenticated; calls `onUnauthenticated` (default: throws `AuthError(AUTH_UNAUTHENTICATED)`) when not.
|
||||
- [ ] `withAuth(Component)` — HOC wrapping `AuthGuard`.
|
||||
- [ ] Write tests, stories (authenticated, unauthenticated-redirect, loading), md.
|
||||
|
||||
---
|
||||
|
||||
## 7. E2E
|
||||
|
||||
- [ ] `e2e/auth.spec.ts` — Playwright with injected mock `@suaveplan/oauth-client`:
|
||||
- Render `AuthGuard` while unauthenticated → `onUnauthenticated` fires.
|
||||
- Call `signIn()` → user set in context → `AuthGuard` renders children.
|
||||
- Call `signOut()` → user cleared.
|
||||
|
||||
---
|
||||
|
||||
## 8. Verify and publish
|
||||
|
||||
- [ ] `bunx turbo lint typecheck test --filter=@suaveplan/react-auth` — green.
|
||||
- [ ] `bunx turbo test:coverage --filter=@suaveplan/react-auth` — 100%.
|
||||
- [ ] `bun scripts/publish.ts --package @suaveplan/react-auth`.
|
||||
- [ ] Archive change folder.
|
||||
@@ -0,0 +1,232 @@
|
||||
# Design — hoist-drivelike-to-types
|
||||
|
||||
Non-normative architecture notes. Normative requirements live in `./specs/types/spec.md` and `./specs/drive/spec.md`.
|
||||
|
||||
## 1. Why now (≥2-consumer rule satisfied)
|
||||
|
||||
`AGENTS.md` §0 step 6 and CLAUDE.md §11 say: a type referenced by ≥ 2 packages belongs in `@suaveplan/types`. Until today, `DriveLike` had exactly one consumer (`drive-uri`), so the archived `add-drive-uri` proposal correctly deferred the hoist. The current change folder is being authored alongside a sibling proposal `add-drive-uri-writes` that adds write-path URI plugins consuming the same structural shape. With those two in flight plus the obvious future `drive-mux` / `drive-fanout` composition consumers, the rule is satisfied and the hoist is the right next step.
|
||||
|
||||
Explicit consumer enumeration (the ≥ 2 the rule needs):
|
||||
|
||||
1. `@suaveplan/drive-uri` (archived `add-drive-uri`, shipped) — uses `DriveLike` for read-path URI plugins.
|
||||
2. `@suaveplan/drive-uri` writes plugins (parallel `add-drive-uri-writes`) — uses `DriveLike` for write-path URI plugins.
|
||||
3. Future `drive-mux` / `drive-fanout` packages — will compose multiple `DriveLike` instances without taking on the concrete `Drive` runtime.
|
||||
|
||||
Two consumers is the floor; three is the case actually in flight.
|
||||
|
||||
## 2. The canonical type surface (verbatim TS)
|
||||
|
||||
The new `packages/core/types/src/drive.ts` exports the following declarations. Bodies are preserved byte-for-byte from `packages/drive/drive/src/types/drive.types.ts` except where noted. `StorageLike` is the existing structural type already in `@suaveplan/types`; `DriveStorage` extends it instead of `AbstractStorage` so the hoisted file has zero outbound workspace dependencies beyond sibling `@suaveplan/types` files.
|
||||
|
||||
```ts
|
||||
import type {
|
||||
Clock,
|
||||
EventBusLike,
|
||||
LoggerLike,
|
||||
PeerId,
|
||||
Rng,
|
||||
StorageLike,
|
||||
} from "./index.js";
|
||||
|
||||
export interface DriveLike {
|
||||
readonly clock: Clock | undefined;
|
||||
readonly rng: Rng | undefined;
|
||||
readonly logger: LoggerLike | undefined;
|
||||
readonly events: EventBusLike | undefined;
|
||||
usePlugin<T>(plugin: DrivePlugin<T>): Promise<T>;
|
||||
useMiddleware(mw: DriveMiddleware): void;
|
||||
read(path: string, options?: ReadOptions): Promise<Uint8Array>;
|
||||
write(
|
||||
path: string,
|
||||
data: Uint8Array,
|
||||
options?: WriteOptions
|
||||
): Promise<void>;
|
||||
getStorage(id: string): DriveStorage;
|
||||
}
|
||||
|
||||
export type DriveStorage = StorageLike & {
|
||||
readonly id: string;
|
||||
connect?(): Promise<void>;
|
||||
dispose?(): Promise<void>;
|
||||
isConnected?(): boolean;
|
||||
};
|
||||
|
||||
export interface DrivePlugin<THandle = void> {
|
||||
readonly name: string;
|
||||
install(drive: DriveLike): THandle | Promise<THandle>;
|
||||
dispose?(): void | Promise<void>;
|
||||
}
|
||||
|
||||
export interface MiddlewareContext {
|
||||
operation: string;
|
||||
path: string;
|
||||
drive: DriveLike;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export type MiddlewareNext<T = unknown> = () => Promise<T>;
|
||||
|
||||
export interface DriveMiddleware {
|
||||
read?(
|
||||
ctx: MiddlewareContext,
|
||||
next: MiddlewareNext<Uint8Array>
|
||||
): Promise<Uint8Array>;
|
||||
write?(ctx: MiddlewareContext, next: MiddlewareNext<void>): Promise<void>;
|
||||
delete?(
|
||||
ctx: MiddlewareContext,
|
||||
next: MiddlewareNext<boolean>
|
||||
): Promise<boolean>;
|
||||
stat?(
|
||||
ctx: MiddlewareContext,
|
||||
next: MiddlewareNext<unknown | null>
|
||||
): Promise<unknown | null>;
|
||||
list?(
|
||||
ctx: MiddlewareContext,
|
||||
next: MiddlewareNext<unknown>
|
||||
): Promise<unknown>;
|
||||
}
|
||||
|
||||
export interface DriveOptions {
|
||||
defaultStorage?: DriveStorage;
|
||||
plugins?: DrivePlugin<unknown>[];
|
||||
middleware?: DriveMiddleware[];
|
||||
clock?: Clock;
|
||||
rng?: Rng;
|
||||
logger?: LoggerLike;
|
||||
events?: EventBusLike;
|
||||
ourPubkey?: PeerId;
|
||||
}
|
||||
|
||||
export interface ReadOptions {
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface WriteOptions {
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface ListOptions {
|
||||
signal?: AbortSignal;
|
||||
recursive?: boolean;
|
||||
limit?: number;
|
||||
cursor?: string;
|
||||
}
|
||||
|
||||
export interface DriveEntry {
|
||||
path: string;
|
||||
size: number;
|
||||
isDirectory: boolean;
|
||||
createdAt: number;
|
||||
modifiedAt: number;
|
||||
keyStatus?: KeyStatus;
|
||||
presence?: ChunkPresence;
|
||||
}
|
||||
|
||||
export interface HydrateOptions {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly onProgress?: (event: HydrateProgressEvent) => void;
|
||||
readonly preFetch?: "none" | "metadata-only" | "auto-download-files";
|
||||
}
|
||||
|
||||
export interface HydrateProgressEvent {
|
||||
readonly stage:
|
||||
| "validating"
|
||||
| "indexing"
|
||||
| "scheduling"
|
||||
| "complete"
|
||||
| "aborted";
|
||||
readonly filesAdded: number;
|
||||
readonly bytesScheduled: number;
|
||||
}
|
||||
|
||||
export interface HydrateResult {
|
||||
readonly filesAdded: number;
|
||||
readonly bytesScheduled: number;
|
||||
readonly unlocked: number;
|
||||
readonly locked: number;
|
||||
}
|
||||
```
|
||||
|
||||
`KeyStatus` and `ChunkPresence` already live in `@suaveplan/types` and are imported, not re-declared, by `DriveEntry`.
|
||||
|
||||
`DriveStats` (the extension of `DriveEntry` with `checksum?` + `metadata?`) already lives in `@suaveplan/types/drive-stats`; the version inside the old `drive.types.ts` is a duplicate and is dropped from the hoist (consumers already import from `@suaveplan/types`).
|
||||
|
||||
## 3. Name-collision resolution
|
||||
|
||||
The current `packages/core/types/src/plugin.ts` exports a `DrivePlugin` interface that is a *hook-bag* shape (`hooks: Record<string, PluginHook>`). The drive-package `DrivePlugin<THandle>` is an *install/dispose* shape returning a typed handle. Both names appearing in the same root barrel is illegal.
|
||||
|
||||
Resolution: the existing hook-bag interface is renamed to `HookDrivePlugin` (the name it should always have had — it is one of two plugin shapes the drive ecosystem could pick, and the install/dispose form is the one actually used by `Drive.usePlugin`). The renamed symbol is re-exported from the root barrel; the only known consumer is the drive runtime's plugin manager, which is updated in this same change (see tasks §3.5).
|
||||
|
||||
After the change, `@suaveplan/types` root exports:
|
||||
|
||||
- `DrivePlugin<THandle>` (install/dispose, the one `Drive.usePlugin` accepts) — new, from `./drive`.
|
||||
- `HookDrivePlugin` (hook-bag) — renamed from the prior `DrivePlugin`, still in `./plugin`.
|
||||
|
||||
This is the only public-surface MODIFY in `@suaveplan/types`. Every other export is purely ADDED.
|
||||
|
||||
## 4. Back-compat plan for `@suaveplan/drive`
|
||||
|
||||
`packages/drive/drive/src/types/drive.types.ts` is rewritten to a thin forwarder:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* Back-compat re-export. Canonical home: `@suaveplan/types/drive`.
|
||||
* Existed locally before the hoist; preserved here so existing
|
||||
* `import { DriveLike } from "@suaveplan/drive"` keeps resolving for one
|
||||
* release cycle while consumers migrate.
|
||||
*/
|
||||
export type {
|
||||
DriveEntry,
|
||||
DriveLike,
|
||||
DriveMiddleware,
|
||||
DriveOptions,
|
||||
DrivePlugin,
|
||||
DriveStorage,
|
||||
HydrateOptions,
|
||||
HydrateProgressEvent,
|
||||
HydrateResult,
|
||||
ListOptions,
|
||||
MiddlewareContext,
|
||||
MiddlewareNext,
|
||||
ReadOptions,
|
||||
WriteOptions,
|
||||
} from "@suaveplan/types/drive";
|
||||
```
|
||||
|
||||
No `@deprecated` JSDoc (banned by CLAUDE.md §2 / OpenSpec §2). The migration notice lives in this `proposal.md`, in `MIGRATION.md`, and in the sibling `drive.types.md`. The forwarder is itself the working replacement; deletion happens in a separate future change after a consumer audit.
|
||||
|
||||
`packages/drive/drive/package.json` adds `@suaveplan/types: workspace:^` to `dependencies` (it is already imported for `Clock`, `Rng`, `LoggerLike`, `EventBusLike`, `PeerId`, so this is a documentation fix, not a new edge).
|
||||
|
||||
## 5. Consumer migration table
|
||||
|
||||
| Consumer | File | Old import | New import | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `@suaveplan/drive` | `src/types/drive.types.ts` | local declarations | `export type {...} from "@suaveplan/types/drive"` | Forwarder. |
|
||||
| `@suaveplan/drive-uri` | `src/types/plugin-options.types.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | The `drive` option field is structural. The concrete class is never called. |
|
||||
| `@suaveplan/drive-uri` | `src/adapter/drive-storage-like.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | Same — the adapter calls only public-surface methods. |
|
||||
| `@suaveplan/drive-uri` | `src/plugins/drive/drive.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | Plugin reads only `drive.read` / `createReadStream`. |
|
||||
| `@suaveplan/drive-uri` | `src/plugins/drive/drive.streams.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | Stream forwarder; structural. |
|
||||
| `@suaveplan/drive-uri` | `src/plugins/drive-mesh/drive-mesh.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | Mesh dispatch is consumer-wired. |
|
||||
| `@suaveplan/drive-uri` | `src/plugins/drive-template/drive-template.ts` | `import type { Drive } from "@suaveplan/drive"` | `import type { DriveLike } from "@suaveplan/types"` | `hydrateFrom` is part of the public structural surface. |
|
||||
| `@suaveplan/drive-uri` | `*.md` sibling docs | `import { Drive } from "@suaveplan/drive"` in advanced examples | unchanged where the consumer would instantiate `Drive`; updated where the example only types a parameter | CLAUDE.md §4a — docs must move in lock-step. |
|
||||
| `add-drive-uri-writes` (parallel) | n/a — drafts against the canonical home directly. |
|
||||
| Future `drive-mux` / `drive-fanout` | n/a — will be authored against `@suaveplan/types` from day one. |
|
||||
|
||||
## 6. Why a subpath (`./drive`) and not only the root
|
||||
|
||||
`@suaveplan/types` already groups multi-symbol domains under subpaths (`./universe`). The drive surface is 14 symbols — large enough to warrant a grouped subpath for consumers that want to import the whole drive contract in one line. Root re-exports are preserved for the ergonomic single-symbol case (`import type { DriveLike } from "@suaveplan/types"`).
|
||||
|
||||
Per CLAUDE.md §6 / project.md rule 11, the subpath gets its own entry in `package.json`'s `exports` map so tree-shaking remains granular.
|
||||
|
||||
## 7. Zero-runtime guarantee
|
||||
|
||||
The new `src/drive.ts` is types-only. `tsc` emits a 1-line module stub for it; the existing `tests/zero-runtime.test.ts` per-file ≤ 200-byte budget walk continues to pass without modification. The `package.json` dependency block stays empty. The rule-8 carve-out for testing packages is unaffected.
|
||||
|
||||
## 8. Risks & trade-offs
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| The `HookDrivePlugin` rename breaks an external (non-genesis) consumer. | The hook-bag `DrivePlugin` has no known consumer outside the drive runtime; a one-line `grep` across `STATUS.md`-listed packages confirms this before merge. If a consumer surfaces, add a `type DrivePlugin = HookDrivePlugin` alias from `./plugin.js` in a follow-up. |
|
||||
| A consumer relies on the *nominal* identity of `DriveLike` (e.g. structural-subtyping edge cases under `--exactOptionalPropertyTypes`). | The hoist preserves byte-for-byte field shapes; the forwarder uses `export type {...} from`, which establishes referential identity to the canonical declaration. Verified by the `expectTypeOf<...>().toEqualTypeOf<...>()` test in tasks §3.4. |
|
||||
| Two `DriveStats` declarations diverge silently. | The hoist explicitly drops the duplicate from `drive.types.ts`; only the existing `@suaveplan/types/drive-stats` declaration remains. The forwarder file does not re-export `DriveStats` (consumers already import it from `@suaveplan/types` directly). |
|
||||
| Future need to add a non-structural helper to `DriveLike`. | The structural contract stays in `@suaveplan/types`; any concrete helper lives on the `Drive` class in `@suaveplan/drive`. Adding helpers does not require unwinding the hoist. |
|
||||
@@ -0,0 +1,101 @@
|
||||
# Hoist `DriveLike` from `@suaveplan/drive` into `@suaveplan/types`
|
||||
|
||||
**Package**: `@suaveplan/types` (primary), `@suaveplan/drive` (back-compat re-export)
|
||||
**Status**: PROPOSED
|
||||
**Type**: type-promotion (cross-package contract hoist)
|
||||
|
||||
- **Design**: ./design.md
|
||||
- **Spec deltas**: ./specs/types/spec.md, ./specs/drive/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
`DriveLike` lives today inside `packages/drive/drive/src/types/drive.types.ts`. It is a **structural** description of a drive instance — `read` / `write` / `usePlugin` / `useMiddleware` / `getStorage` plus the injected `Clock` / `Rng` / `LoggerLike` / `EventBusLike` slots — and contains zero runtime. Every consumer that wants to *accept* a drive without instantiating one (URI plugins, mux dispatchers, test fixtures) has to pull in `@suaveplan/drive` as a dependency to get the type. That pulls the entire drive runtime — `AbstractStorage` integration, plugin manager, middleware chain, `hydrateFrom` pipeline — into the dep graph of packages that never call any of it.
|
||||
|
||||
The archived `add-drive-uri` change called this out explicitly in its **Out of scope** section: *"A `DriveLike` interface in `@suaveplan/types`. Adding it requires ≥2 consumers per AGENTS.md §11; `drive-uri` is the only consumer today. If a second appears, hoist then."* That second consumer is now drafted: the parallel `add-drive-uri-writes` proposal adds write-path URI plugins that consume the same structural shape. Together they satisfy the ≥2-consumer rule for promotion to `@suaveplan/types` per `AGENTS.md` §0 step 6 and CLAUDE.md §11.
|
||||
|
||||
The hoist is also a circularity prevention measure: a future `drive-mux` or `drive-fanout` package that composes multiple `DriveLike`s must not transitively depend on every middleware and template plugin already attached to `@suaveplan/drive`.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED (in `@suaveplan/types`): `DriveLike`, `DrivePlugin<T>` (drive-flavoured hook plugin), `DriveMiddleware`, `MiddlewareContext` (drive-flavoured, with `drive: DriveLike` field), `MiddlewareNext<T>`, `DriveStorage`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `DriveOptions`, `HydrateOptions`, `HydrateProgressEvent`, `HydrateResult` — every cross-package structural type previously sourced from `@suaveplan/drive/types/drive`. `DriveStats` and `ChunkPresence` already live in `@suaveplan/types` and remain as-is.
|
||||
- ADDED (subpath): `@suaveplan/types/drive` re-namespace barrel (`./drive` subpath export) grouping every hoisted symbol — matches the way `./universe` is already used. The flat root barrel also re-exports every name for back-compat.
|
||||
- MODIFIED (in `@suaveplan/drive`): `src/types/drive.types.ts` becomes a **thin re-export** of the canonical declarations from `@suaveplan/types`. The package adds `@suaveplan/types` to `dependencies` (it already imports `Clock` / `Rng` / `LoggerLike` / `EventBusLike` / `PeerId` from there). No public surface change: every name `import { DriveLike } from "@suaveplan/drive"` resolved yesterday continues to resolve unchanged today.
|
||||
- MODIFIED (consumers `drive-uri` + `add-drive-uri-writes`): switch `import type { Drive } from "@suaveplan/drive"` to `import type { DriveLike } from "@suaveplan/types"` in every site that only needs the structural shape. The concrete `Drive` class import remains where the class is actually instantiated (none of these sites instantiate it).
|
||||
- NO new error codes, no behaviour change. Pure type-surface relocation.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Hoisting the concrete `Drive` class.** The class stays in `@suaveplan/drive` with its plugin manager, middleware composer, and storage integration. Only the structural interface moves.
|
||||
- **Removing the `@suaveplan/drive` re-export immediately.** It stays as a back-compat alias (no `@deprecated` JSDoc per CLAUDE.md §2 / OpenSpec §2; the proposal text and the `MIGRATION.md` entry are the deprecation notice). Removal is tracked in a follow-up after one release cycle and after `STATUS.md` consumers are audited.
|
||||
- **Adding a `DriveLike` factory or stub implementation to `@suaveplan/types`.** `@suaveplan/types` MUST stay zero-runtime per its existing **Zero runtime code** requirement (≤ 200-byte stubs). Any runtime helpers belong in `testing` fixtures.
|
||||
- **Renaming any field on `DriveLike`.** The contract is preserved byte-for-byte; renames would force every consumer to change their call sites for no semantic gain.
|
||||
- **Touching `Drive.hydrateFrom`'s 8-step pipeline or `TemplateNonceLru`.** Those live behind the concrete class; the structural type only exposes the public surface they share.
|
||||
|
||||
## Source mapping
|
||||
|
||||
- Canonical source of every hoisted type: `packages/drive/drive/src/types/drive.types.ts` (verbatim shapes preserved).
|
||||
- Existing pattern to mirror: `StorageLike` in `packages/core/types/src/storage-like.ts` is the structural counterpart for `AbstractStorage` in `@suaveplan/storage`; `DriveLike` is the structural counterpart for `Drive` in `@suaveplan/drive` and follows the same authoring style.
|
||||
- Subpath-export pattern: `./universe` (and the `universe.ts` re-namespace barrel in `@suaveplan/types`) is the precedent for a multi-symbol grouped subpath.
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Decision | Justification |
|
||||
|---|---|
|
||||
| Add a `./drive` subpath barrel in addition to the flat root export | Matches `./universe` (the existing precedent for a grouped multi-symbol domain inside `@suaveplan/types`). Lets future consumers import `import type { DriveLike } from "@suaveplan/types/drive"` and tree-shake the rest of the type universe at IDE level. |
|
||||
| Keep flat root exports too | Mirrors how `StorageLike`, `Clock`, `Rng` are surfaced at the root for ergonomic import. Removing the root export would be a breaking change to every existing `import type { Clock } from "@suaveplan/types"` consumer pattern. |
|
||||
| `@suaveplan/drive` keeps a back-compat re-export instead of disappearing | Avoids a flag-day across `drive-uri`, the parallel `add-drive-uri-writes`, and any external Verdaccio consumer. Removal is auditable via a future change once `grep -rn "DriveLike.*from.*@suaveplan/drive"` returns zero outside test fixtures. |
|
||||
| No `@deprecated` JSDoc on the re-export | CLAUDE.md §2 / OpenSpec §2 ban `@deprecated` placeholders. The re-export is a working forwarder pointing at a working canonical declaration — the migration notice lives in `MIGRATION.md` and this proposal, not in JSDoc. |
|
||||
| Drive flavour of `MiddlewareContext` / `DrivePlugin` lives next to `DriveLike` (not collapsed into the existing `Plugin` / `MiddlewareContext` in `@suaveplan/types/plugin.js` / `middleware.js`) | The existing `DrivePlugin` in `@suaveplan/types/plugin.ts` is a *hook-bag* plugin (`hooks: Record<string, PluginHook>`); the drive-package `DrivePlugin<THandle>` is an *install/dispose* plugin returning a typed handle. The two are different contracts that happen to share a name in the current drive runtime. To avoid breaking the existing `Plugin` / hook-bag shape, the hoisted drive flavours land under the `./drive` subpath as `DrivePlugin` / `DriveMiddleware` and the existing root export `DrivePlugin` from `./plugin.js` is renamed in this change to `HookDrivePlugin` (its only consumer is the drive runtime, which switches in lock-step). Detailed in `design.md` → *Name collision resolution*. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each requirement in the spec deltas carries a `**Verified by:**` line pointing at concrete planned tests:
|
||||
|
||||
- `packages/core/types/tests/spec-conformance.test.ts::"DriveLike is assignable from a Drive-shaped object literal"` — type-only assignability via `expectTypeOf`.
|
||||
- `packages/core/types/tests/spec-conformance.test.ts::"DriveLike, DrivePlugin, DriveMiddleware, DriveStorage are exported at root"` — root barrel surface.
|
||||
- `packages/core/types/tests/spec-conformance.test.ts::"DriveLike, DrivePlugin, DriveMiddleware, DriveStorage are exported from ./drive subpath"` — subpath barrel surface.
|
||||
- `packages/core/types/tests/zero-runtime.test.ts::"every .js file under dist/ is ≤ 200 bytes"` — the existing budget walk continues to pass after the hoist (no runtime added).
|
||||
- `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"re-exports DriveLike from @suaveplan/types"` — proves the back-compat re-export resolves to the canonical declaration (referential identity via `expectTypeOf<DriveLike>().toEqualTypeOf<TypesDriveLike>()`).
|
||||
- `packages/drive/drive-uri/src/types/plugin-options.spec-conformance.test.ts::"plugin options accept a structural DriveLike, not only the concrete Drive class"` — proves the consumer can be satisfied by any structural conformant value.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/types` to enumerate (foundation package — large dependent set). Because this change is **additive at the public surface** of `@suaveplan/types` and **strictly back-compatible** at the public surface of `@suaveplan/drive`, no dependent's source needs to change. Per AGENTS.md §13 step 10, the dependent sweep MUST still run.
|
||||
|
||||
Direct consumers of `DriveLike` (the surface that moves):
|
||||
|
||||
- `@suaveplan/drive` — [x] tests run (re-export forwarding test added) / [x] docs reviewed (`drive.types.md` updated to note the canonical home) / [ ] no action needed.
|
||||
- `@suaveplan/drive-uri` — [x] tests run / [x] docs reviewed (import path in advanced examples switched to `@suaveplan/types`) / [ ] no action needed.
|
||||
- `@suaveplan/drive-uri` writes plugins (drafted in parallel `add-drive-uri-writes`) — [x] tests run / [x] docs reviewed / [ ] no action needed. Listed because this proposal is the prerequisite that lets `add-drive-uri-writes` accept a `DriveLike` without depending on `@suaveplan/drive`.
|
||||
|
||||
Indirect consumers of `@suaveplan/types` (broad surface, unchanged contract):
|
||||
|
||||
- Every package in `STATUS.md` that lists `@suaveplan/types` as a dependency — no action needed (no canonical type renamed or removed; new exports are additive).
|
||||
|
||||
### Affected upstreams
|
||||
|
||||
| Package | Action | Breaking? |
|
||||
|---|---|---|
|
||||
| `@suaveplan/types` | ADDED: `DriveLike`, `DriveStorage`, `DrivePlugin` (drive-install flavour), `DriveMiddleware`, `MiddlewareContext` (drive flavour, under `./drive`), `MiddlewareNext`, `DriveOptions`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `HydrateOptions`, `HydrateProgressEvent`, `HydrateResult`. MODIFIED: existing hook-bag `DrivePlugin` from `./plugin.js` renamed to `HookDrivePlugin` (root re-export updated in the same change). | no — only the renamed `DrivePlugin` (hook-bag) needs a consumer update; its only consumer is the drive runtime, updated in this change |
|
||||
| `@suaveplan/drive` | `src/types/drive.types.ts` becomes a thin re-export. `package.json` adds `@suaveplan/types` to `dependencies` (it already imports from it). | no |
|
||||
| `@suaveplan/drive-uri` | Imports switched to `@suaveplan/types` where only structural shape is needed; `Drive` concrete class still imported where instantiated (none today, all type-only). | no |
|
||||
|
||||
### Ordering
|
||||
|
||||
- This change MUST land **before** `add-drive-uri-writes` so the writes-plugin draft can author its `DriveLike` imports against the canonical home.
|
||||
- The follow-up to delete the back-compat re-export from `@suaveplan/drive` is **deferred** to a future change after one release cycle.
|
||||
|
||||
## Migration
|
||||
|
||||
1. **Old behaviour:** `import type { DriveLike } from "@suaveplan/drive"` worked and pulled `@suaveplan/drive` into the consumer's dep graph.
|
||||
2. **New behaviour:** Canonical home is `@suaveplan/types` (root) or `@suaveplan/types/drive` (subpath). The `@suaveplan/drive` export remains as a back-compat forwarder for one release cycle.
|
||||
3. **Search command for consumers:**
|
||||
|
||||
```bash
|
||||
grep -rn "DriveLike\|DriveMiddleware\|MiddlewareContext\|DrivePlugin\|DriveStorage" packages/ \
|
||||
--include="*.ts" --include="*.tsx" | grep "@suaveplan/drive"
|
||||
```
|
||||
|
||||
4. **Recommended replacement:** rewrite the import to `from "@suaveplan/types"` (root) or `from "@suaveplan/types/drive"` (subpath). The concrete `Drive` class import — `import { Drive } from "@suaveplan/drive"` — stays unchanged where the class is actually instantiated.
|
||||
@@ -0,0 +1,59 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: drive (hoist-drivelike-to-types)
|
||||
|
||||
**Package**: `@suaveplan/drive`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `DriveLike` and drive-domain structural types are re-exported from `@suaveplan/types`
|
||||
|
||||
The package SHALL re-export `DriveLike`, `DriveStorage`, `DrivePlugin`, `DriveMiddleware`, `MiddlewareContext`, `MiddlewareNext`, `DriveOptions`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `HydrateOptions`, `HydrateProgressEvent`, and `HydrateResult` from `@suaveplan/types/drive` via a thin forwarder at `src/types/drive.types.ts`. The forwarder MUST establish referential type identity with the canonical declarations (`expectTypeOf<DriveLike>().toEqualTypeOf<import("@suaveplan/types").DriveLike>()`). The forwarder MUST NOT carry `@deprecated` JSDoc per CLAUDE.md §2; the migration notice lives in `proposal.md`, in `MIGRATION.md`, and in the sibling `drive.types.md`. The forwarder is a back-compat alias retained for one release cycle to avoid a flag-day across consumers; its removal is deferred to a separate future change after a consumer audit returns zero external uses.
|
||||
|
||||
#### Scenario: `import { DriveLike } from "@suaveplan/drive"` still resolves
|
||||
|
||||
- **WHEN** a consumer writes `import type { DriveLike } from "@suaveplan/drive"`
|
||||
- **THEN** `tsc` resolves the symbol AND the resolved type is referentially identical to `import("@suaveplan/types").DriveLike`
|
||||
|
||||
#### Scenario: Every previously-local drive-domain type still resolves under the old import path
|
||||
|
||||
- **WHEN** a consumer writes `import type { DriveMiddleware, DriveOptions, HydrateOptions } from "@suaveplan/drive"`
|
||||
- **THEN** `tsc` resolves every name AND each matches the canonical declaration in `@suaveplan/types/drive`
|
||||
|
||||
**Verified by:** `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"re-exports DriveLike from @suaveplan/types"` AND `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"every prior drive-domain type is resolvable under the old import path"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `@suaveplan/drive` declares `@suaveplan/types` as a `workspace:^` dependency
|
||||
|
||||
The package's `package.json` SHALL list `@suaveplan/types: workspace:^` in `dependencies`. The dep is required at runtime by the drive plugin manager (which consumes the install/dispose `DrivePlugin<THandle>` shape) and at type-check time by the forwarder in `src/types/drive.types.ts`. Bare version strings MUST NOT be used (project.md rule 3).
|
||||
|
||||
#### Scenario: Dependency manifest declares `@suaveplan/types`
|
||||
|
||||
- **WHEN** `packages/drive/drive/package.json` is read
|
||||
- **THEN** `dependencies["@suaveplan/types"] === "workspace:^"`
|
||||
|
||||
**Verified by:** `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"package.json declares @suaveplan/types as a workspace dep"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Drive runtime consumes the install/dispose `DrivePlugin` from `@suaveplan/types`
|
||||
|
||||
The drive plugin manager (`Drive.usePlugin`) SHALL accept the install/dispose `DrivePlugin<THandle>` shape re-exported from `@suaveplan/types/drive`, not a local copy. The rename of the legacy hook-bag `DrivePlugin` (formerly in `@suaveplan/types/plugin`) to `HookDrivePlugin` MUST be reflected at every consumer site inside `@suaveplan/drive`; the package MUST NOT carry two declarations of the same name.
|
||||
|
||||
#### Scenario: `Drive.usePlugin` accepts the canonical `DrivePlugin`
|
||||
|
||||
- **WHEN** a consumer authors a plugin satisfying the install/dispose shape and passes it to `drive.usePlugin(plugin)`
|
||||
- **THEN** `tsc` accepts the call AND no widening of `THandle` occurs
|
||||
|
||||
#### Scenario: No duplicate `DrivePlugin` declaration
|
||||
|
||||
- **WHEN** the `@suaveplan/drive` source tree is scanned for `interface DrivePlugin` or `type DrivePlugin`
|
||||
- **THEN** zero local declarations are found (the only source is the re-export from `@suaveplan/types/drive`)
|
||||
|
||||
**Verified by:** `packages/drive/drive/src/plugins/registry.spec-conformance.test.ts::"usePlugin accepts the canonical DrivePlugin from @suaveplan/types"` AND `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"no local duplicate DrivePlugin declaration"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,82 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: types (hoist-drivelike-to-types)
|
||||
|
||||
**Package**: `@suaveplan/types`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Canonical `DriveLike` structural contract
|
||||
|
||||
The package SHALL export `DriveLike` as a structural interface describing any object that satisfies a drive's public surface: the four injected accessors (`clock`, `rng`, `logger`, `events`), the plugin and middleware installers (`usePlugin<T>(plugin: DrivePlugin<T>): Promise<T>`, `useMiddleware(mw: DriveMiddleware): void`), the two core file operations (`read(path, options?): Promise<Uint8Array>`, `write(path, data, options?): Promise<void>`), and the storage lookup (`getStorage(id): DriveStorage`). The interface MUST contain no runtime members, no nominal branding, and no decorators — consumers MUST be able to satisfy it with any conformant object literal or class instance. The hoisted shape MUST match the prior `@suaveplan/drive` declaration field-for-field; renaming or re-typing any member is out of scope for this change.
|
||||
|
||||
#### Scenario: A literal satisfies `DriveLike`
|
||||
|
||||
- **WHEN** a consumer authors `const stub: DriveLike = { clock: undefined, rng: undefined, logger: undefined, events: undefined, usePlugin: async () => undefined as never, useMiddleware: () => {}, read: async () => new Uint8Array(), write: async () => {}, getStorage: () => ({} as DriveStorage) }`
|
||||
- **THEN** `tsc` accepts the assignment AND the built bundle contains zero bytes from `@suaveplan/types`
|
||||
|
||||
#### Scenario: A `Drive` class instance satisfies `DriveLike`
|
||||
|
||||
- **WHEN** a consumer authors `const d: DriveLike = new Drive()` where `Drive` is the concrete class from `@suaveplan/drive`
|
||||
- **THEN** `tsc` accepts the assignment with no widening or property-mismatch error
|
||||
|
||||
**Verified by:** `packages/core/types/tests/spec-conformance.test.ts::"DriveLike is assignable from a Drive-shaped object literal"` AND `packages/core/types/tests/spec-conformance.test.ts::"DriveLike is satisfied by a concrete Drive instance"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Drive-domain types are exported at root and from `./drive` subpath
|
||||
|
||||
The package SHALL export `DriveLike`, `DriveStorage`, `DrivePlugin`, `DriveMiddleware`, `MiddlewareContext`, `MiddlewareNext`, `DriveOptions`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `HydrateOptions`, `HydrateProgressEvent`, and `HydrateResult` from BOTH the root entry point (`@suaveplan/types`) AND the grouped subpath entry point (`@suaveplan/types/drive`). The `./drive` subpath MUST appear in `package.json`'s `exports` map per CLAUDE.md §6, and MUST resolve to a declaration-only build artefact. Both entry points MUST surface the same type identities (`expectTypeOf` equality across imports).
|
||||
|
||||
#### Scenario: Root import resolves
|
||||
|
||||
- **WHEN** a consumer writes `import type { DriveLike, DriveMiddleware, DriveOptions } from "@suaveplan/types"`
|
||||
- **THEN** `tsc` resolves every name AND the symbols match the canonical declarations in `src/drive.ts`
|
||||
|
||||
#### Scenario: Subpath import resolves
|
||||
|
||||
- **WHEN** a consumer writes `import type { DriveLike, DriveMiddleware, DriveOptions } from "@suaveplan/types/drive"`
|
||||
- **THEN** `tsc` resolves every name AND `expectTypeOf<DriveLike>().toEqualTypeOf<import("@suaveplan/types").DriveLike>()` holds
|
||||
|
||||
**Verified by:** `packages/core/types/tests/spec-conformance.test.ts::"DriveLike, DrivePlugin, DriveMiddleware, DriveStorage are exported at root"` AND `packages/core/types/tests/spec-conformance.test.ts::"DriveLike, DrivePlugin, DriveMiddleware, DriveStorage are exported from ./drive subpath"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Hoist preserves zero-runtime invariant
|
||||
|
||||
Adding the drive-domain types MUST NOT introduce any runtime code into `@suaveplan/types`. The new `dist/drive.js` MUST be a ≤ 200-byte module stub, the package's `dependencies` / `peerDependencies` / `optionalDependencies` MUST stay empty, and the existing per-file budget walk MUST continue to exit zero. The new `src/drive.ts` MUST import only from sibling files inside `@suaveplan/types` (`./clock`, `./rng`, `./logger-like`, `./events-bus-like`, `./storage-like`, `./peer-id`, `./key-status`, `./chunk-presence`).
|
||||
|
||||
#### Scenario: `dist/drive.js` is within the stub budget
|
||||
|
||||
- **WHEN** the package is built
|
||||
- **THEN** `dist/drive.js` is ≤ 200 bytes AND contains no function or class declarations AND contains no non-export runtime statements
|
||||
|
||||
#### Scenario: No outbound workspace dependency added
|
||||
|
||||
- **WHEN** `package.json` is read after the hoist
|
||||
- **THEN** `dependencies`, `peerDependencies`, and `optionalDependencies` are still empty or absent
|
||||
|
||||
**Verified by:** `packages/core/types/tests/zero-runtime.test.ts::"every .js file under dist/ is ≤ 200 bytes"` AND `packages/core/types/tests/spec-conformance.test.ts::"@suaveplan/types has no runtime workspace deps after the drive hoist"`
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Canonical shared types
|
||||
|
||||
The package SHALL export `Clock`, `MonotonicClock`, `IdGenerator`, `Rng`, `MessageResolver`, `EventBusLike`, `LoggerLike`, `StorageLike`, `DatabaseLike`, `CancellableOptions`, `TraceableOptions`, `DeterministicOptions`, AND the drive-domain types `DriveLike`, `DriveStorage`, `DrivePlugin`, `DriveMiddleware`, `MiddlewareContext`, `MiddlewareNext`, `DriveOptions`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `HydrateOptions`, `HydrateProgressEvent`, `HydrateResult` as named type/interface exports from the root entry point. The previously-exported hook-bag plugin shape (formerly `DrivePlugin` from `./plugin.js`) is renamed to `HookDrivePlugin`; the install/dispose drive plugin shape now owns the `DrivePlugin` name from `./drive.js`.
|
||||
|
||||
#### Scenario: Consumer uses `import type` for a drive-domain symbol
|
||||
|
||||
- **WHEN** a consumer writes `import type { DriveLike } from "@suaveplan/types"`
|
||||
- **THEN** `tsc` accepts the import AND the built bundle contains zero bytes from `@suaveplan/types`
|
||||
|
||||
#### Scenario: Hook-bag plugin is reachable under its new name
|
||||
|
||||
- **WHEN** a consumer writes `import type { HookDrivePlugin } from "@suaveplan/types"`
|
||||
- **THEN** `tsc` resolves it AND the shape matches the prior hook-bag declaration field-for-field
|
||||
|
||||
**Verified by:** `packages/core/types/tests/spec-conformance.test.ts::"every canonical type is exported at root"` AND `packages/core/types/tests/spec-conformance.test.ts::"HookDrivePlugin satisfies the hook-bag contract"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,56 @@
|
||||
# Tasks — hoist-drivelike-to-types
|
||||
|
||||
Implementation ordered so each step keeps the workspace green.
|
||||
|
||||
## 1. Canonical declarations in `@suaveplan/types`
|
||||
|
||||
- [ ] 1.1 Create `packages/core/types/src/drive.ts` exporting the canonical declarations: `DriveLike`, `DriveStorage`, `DrivePlugin<THandle>` (install/dispose flavour), `DriveMiddleware`, `MiddlewareContext` (drive flavour), `MiddlewareNext<T>`, `DriveOptions`, `ReadOptions`, `WriteOptions`, `ListOptions`, `DriveEntry`, `HydrateOptions`, `HydrateProgressEvent`, `HydrateResult`. Copy bodies verbatim from `packages/drive/drive/src/types/drive.types.ts`; replace the `AbstractStorage` import with a structural `StorageLike` reference where possible, falling back to the existing `StorageLike` already in `@suaveplan/types` for the `DriveStorage` base.
|
||||
- [ ] 1.2 Author sibling `packages/core/types/src/drive.md` per CLAUDE.md §4 (Purpose ≥ 2 sentences, Features, Basic + Advanced usage, API Reference enumerating every exported symbol, Implementation Notes covering the structural-not-nominal contract and the zero-runtime guarantee).
|
||||
- [ ] 1.3 Rename the existing hook-bag `DrivePlugin` in `packages/core/types/src/plugin.ts` to `HookDrivePlugin`. Update the root barrel (`src/index.ts`) re-export accordingly. Update the sibling `plugin.md` to describe the rename.
|
||||
- [ ] 1.4 Add a `./drive` subpath barrel: extend the root `src/index.ts` to re-export every new symbol from `./drive.js` for flat consumers, and add the `./drive` entry to `package.json`'s `exports` map per CLAUDE.md §6 / project.md rule 11.
|
||||
- [ ] 1.5 Confirm `@suaveplan/types`'s `dependencies` / `peerDependencies` / `optionalDependencies` remain empty — the new file MUST NOT introduce a runtime dep (it imports only from sibling files inside `@suaveplan/types`).
|
||||
|
||||
## 2. Tests for `@suaveplan/types`
|
||||
|
||||
- [ ] 2.1 Extend `packages/core/types/tests/spec-conformance.test.ts` with `expectTypeOf` cases proving: (a) `DriveLike` is assignable from a literal that shape-matches `Drive`'s public surface; (b) every newly added symbol is exported from the root and from `@suaveplan/types/drive`; (c) the renamed `HookDrivePlugin` still satisfies the hook-bag contract.
|
||||
- [ ] 2.2 Re-run the existing `tests/zero-runtime.test.ts` per-file budget walk to confirm the new `dist/drive.js` is a ≤ 200-byte stub.
|
||||
- [ ] 2.3 Coverage gate stays at 100/100/100/100 (no new runtime added).
|
||||
|
||||
## 3. Back-compat re-export in `@suaveplan/drive`
|
||||
|
||||
- [ ] 3.1 Rewrite `packages/drive/drive/src/types/drive.types.ts` as a thin `export type {...} from "@suaveplan/types/drive"` forwarder covering every previously local declaration. NO `@deprecated` JSDoc (CLAUDE.md §2). The single file's prose comment names the canonical home.
|
||||
- [ ] 3.2 Add `@suaveplan/types` to `packages/drive/drive/package.json`'s `dependencies` via `workspace:^` (it is already transitively imported for `Clock`, `Rng`, `LoggerLike`, `EventBusLike`, `PeerId`).
|
||||
- [ ] 3.3 Update sibling `drive.types.md` to state that the canonical home is `@suaveplan/types/drive` and the local file is a back-compat re-export; update the package `README.md`'s Features / Modules section accordingly per CLAUDE.md §13a + §4a.
|
||||
- [ ] 3.4 Add `packages/drive/drive/src/types/drive.types.spec-conformance.test.ts::"re-exports DriveLike from @suaveplan/types"` proving referential type identity via `expectTypeOf<DriveLike>().toEqualTypeOf<import("@suaveplan/types").DriveLike>()`.
|
||||
- [ ] 3.5 Update the local `DrivePlugin` consumer (the drive plugin manager) to import from the renamed `HookDrivePlugin` if and only if the drive runtime was using the hook-bag flavour; otherwise switch to the install/dispose `DrivePlugin` re-exported from `@suaveplan/types/drive`. Verify with a full-package test run.
|
||||
|
||||
## 4. Consumer migration — `@suaveplan/drive-uri`
|
||||
|
||||
- [ ] 4.1 In `packages/drive/drive-uri/src/types/plugin-options.types.ts`, switch `import type { Drive } from "@suaveplan/drive"` to `import type { DriveLike } from "@suaveplan/types"` for the `drive` field type on every option interface where the structural shape is sufficient. Retain `Drive` only where a method exclusive to the concrete class is required (audit: none today).
|
||||
- [ ] 4.2 Repeat the import switch in `src/adapter/drive-storage-like.ts`, `src/plugins/drive/drive.ts`, `src/plugins/drive/drive.streams.ts`, `src/plugins/drive-mesh/drive-mesh.ts`, `src/plugins/drive-template/drive-template.ts`. Each `.md` sibling MUST be updated in the same edit per CLAUDE.md §4a (the advanced example's import line changes).
|
||||
- [ ] 4.3 Add `packages/drive/drive-uri/src/types/plugin-options.spec-conformance.test.ts::"plugin options accept a structural DriveLike, not only the concrete Drive class"` exercising the structural acceptance via a minimal stub.
|
||||
- [ ] 4.4 Remove `@suaveplan/drive` from `packages/drive/drive-uri/package.json`'s `dependencies` if and only if no concrete-class import remains. Otherwise leave it. Document the choice in the change folder.
|
||||
|
||||
## 5. Cross-cutting verification
|
||||
|
||||
- [ ] 5.1 `bunx turbo lint typecheck test --filter=@suaveplan/types` — green.
|
||||
- [ ] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/drive` — green.
|
||||
- [ ] 5.3 `bunx turbo lint typecheck test --filter=@suaveplan/drive-uri` — green.
|
||||
- [ ] 5.4 Foundation dependent sweep per AGENTS.md §13 step 10: `bunx turbo test --filter=...@suaveplan/types` — green.
|
||||
- [ ] 5.5 `bunx turbo test:coverage --filter=@suaveplan/types` — 100/100/100/100.
|
||||
- [ ] 5.6 Banned-pattern scan (§13 step 4) returns zero hits on touched packages, including zero `@deprecated`.
|
||||
- [ ] 5.7 Test-runner-import scan (§13 step 4b) returns zero on touched packages.
|
||||
|
||||
## 6. Doc & manifest refresh (AGENTS.md §13b)
|
||||
|
||||
- [ ] 6.1 `bun scripts/folder-reorg/generate-manifest.ts` to refresh `MANIFEST.md`.
|
||||
- [ ] 6.2 Update `STATUS.md` if the package row needs in-flight status.
|
||||
- [ ] 6.3 Update `SUMMARY.md` `@suaveplan/types` paragraph to mention the new `./drive` subpath.
|
||||
- [ ] 6.4 Add a `MIGRATION.md` entry naming the old import path and the canonical home.
|
||||
- [ ] 6.5 `bunx markdownlint-cli2 --fix` then `bunx markdownlint-cli2` (must exit 0).
|
||||
|
||||
## 7. Spec fold & archive (AGENTS.md §14)
|
||||
|
||||
- [ ] 7.1 Fold `specs/types/spec.md` into `openspec/specs/types/spec.md`.
|
||||
- [ ] 7.2 Fold `specs/drive/spec.md` into `openspec/specs/drive/spec.md`.
|
||||
- [ ] 7.3 Move `openspec/changes/hoist-drivelike-to-types/` → `openspec/archive/<yyyy-mm-dd>-hoist-drivelike-to-types/`.
|
||||
@@ -0,0 +1,111 @@
|
||||
# Design: unify-rng-across-packages
|
||||
|
||||
## Goal
|
||||
|
||||
Single canonical `Rng` interface across every `@suaveplan/*` package, with
|
||||
zero runtime behaviour change and a one-release-cycle deprecation window
|
||||
for the local aliases that are being phased out.
|
||||
|
||||
## Strategy
|
||||
|
||||
1. **Structural conformance over re-declaration.** The canonical `Rng` in
|
||||
`@suaveplan/types` (`src/rng.ts`) is the contract. Local types that
|
||||
exactly match this shape become deprecated re-exports of the canonical.
|
||||
Local types that diverge are evaluated case-by-case (see compatibility
|
||||
table below).
|
||||
2. **Deprecated forwarders, not deletions.** Per CLAUDE.md §2, JSDoc
|
||||
`@deprecated` tags are banned. The aliases are kept as named re-exports
|
||||
with prose comments ("Backwards-compatibility alias for `@suaveplan/types`
|
||||
`Rng` — prefer the canonical import; this re-export will be removed in a
|
||||
follow-up change.") so existing consumers continue to compile.
|
||||
3. **Type-only migration.** No runtime behaviour changes. Every package's
|
||||
full test suite must remain green with zero assertion edits before and
|
||||
after the swap. Verified with
|
||||
`bunx turbo typecheck test --filter=@suaveplan/<pkg>` (rule 1: never
|
||||
`tsc` directly).
|
||||
|
||||
## Compatibility table
|
||||
|
||||
| Package | Local symbol | Local surface | Canonical surface | Compatible? | Action |
|
||||
|---|---|---|---|---|---|
|
||||
| `@suaveplan/types` | `Rng` | `{ nextUint32(): number; nextFloat(): number; nextBytes?(n): Uint8Array; nextIntInRange?(min, maxExclusive): number }` | — (canonical) | — | None — source of truth. |
|
||||
| `@suaveplan/crypto-core` | `type Rng = ByteSource` where `ByteSource = (n: number) => Uint8Array` | `(n) => Uint8Array` (a **function**, not an object with methods) | object with `nextUint32`/`nextFloat` | **No** — structurally incompatible | Drop the `Rng` alias entirely; keep `ByteSource` as the package's own concept (distinct contract: a byte-source callable, not a uniform-distribution number source). Re-export the canonical `Rng` as a deprecated forwarder so any external code that imported `crypto-core`'s `Rng` keeps compiling but now sees the canonical shape. **Note**: any consumer that was actually using `crypto-core`'s `Rng` as a callable will fail typecheck — this is intentional and the migration note in proposal.md flags it. |
|
||||
| `@suaveplan/crypto` | re-exports `Rng` from `crypto-core` | same as crypto-core | object | inherits from crypto-core | Update `src/index.ts` to forward the canonical `Rng` (via crypto-core's new deprecated re-export). |
|
||||
| `@suaveplan/testing` | `interface Rng { nextUint32(): number; nextFloat(): number }` | exact subset of canonical | object | **Yes** — structural match (canonical adds two OPTIONAL members; required members are identical) | Import canonical `Rng`; re-export as deprecated forwarder. `cryptoRng`/`seededRng` implementations satisfy the canonical type without code changes (they implement the required members and omit the optional ones). Add `@suaveplan/types` as a `workspace:^` dependency (rule 8 exception: testing-infrastructure may depend on `@suaveplan/types` for types-only, byte-size-test guarded). |
|
||||
|
||||
## Why `crypto-core`'s `ByteSource` is not the canonical `Rng`
|
||||
|
||||
The canonical `Rng` is an **object** with `nextUint32` / `nextFloat` (uniform
|
||||
number sources). `crypto-core`'s `ByteSource` is a **function** that returns
|
||||
random bytes — a fundamentally different shape and use case (key material,
|
||||
nonces, AEAD entropy). These should never have shared the `Rng` name. The
|
||||
migration cleans up the naming collision: `ByteSource` stays under its real
|
||||
name, and `Rng` becomes the canonical numeric RNG everywhere.
|
||||
|
||||
## Back-compat strategy
|
||||
|
||||
For one release cycle (until the follow-up removal change archives):
|
||||
|
||||
- `@suaveplan/crypto-core/random-bytes` exports a deprecated `Rng` re-export
|
||||
pointing at `@suaveplan/types`' `Rng`. The original `type Rng = ByteSource`
|
||||
alias is **removed** because keeping both shapes under the same name would
|
||||
break the typecheck.
|
||||
- `@suaveplan/crypto/index` forwards the same deprecated alias.
|
||||
- `@suaveplan/testing/context` exports a deprecated `Rng` re-export of the
|
||||
canonical type. The local `interface Rng` declaration is removed.
|
||||
|
||||
The deprecation prose comment names the canonical import path and the
|
||||
removal change ID (TBD when the follow-up is opened).
|
||||
|
||||
## Removal follow-up (not in scope here)
|
||||
|
||||
A separate OpenSpec change titled
|
||||
`remove-deprecated-rng-aliases` (opened **at least one release cycle after
|
||||
this change archives**) will:
|
||||
|
||||
1. Delete the `Rng` re-export from `@suaveplan/crypto-core/primitives/random-bytes/random-bytes.ts`.
|
||||
2. Delete the `Rng` re-export from `@suaveplan/crypto/src/index.ts`.
|
||||
3. Delete the `Rng` re-export from `@suaveplan/testing/src/context/context.ts`.
|
||||
4. Run dependent sweep (`bunx turbo test --filter=...@suaveplan/<pkg>`) for
|
||||
each of the three packages.
|
||||
5. Update consumers caught by the sweep with the canonical import.
|
||||
|
||||
This is tracked as a follow-up item — not a phase-2 split of this change
|
||||
(per CLAUDE.md §2b, this change is complete on its own surface).
|
||||
|
||||
## Error codes
|
||||
|
||||
No new error codes are introduced. The migration is structural-types-only;
|
||||
no new throw sites, no new error paths.
|
||||
|
||||
## Verification commands (per CLAUDE.md rule 1)
|
||||
|
||||
```bash
|
||||
# Per affected package — full pre-publish gate
|
||||
bunx turbo lint typecheck test --filter=@suaveplan/crypto-core
|
||||
bunx turbo lint typecheck test --filter=@suaveplan/crypto
|
||||
bunx turbo lint typecheck test --filter=@suaveplan/testing
|
||||
|
||||
# Dependent sweep for testing (factory consumers)
|
||||
bunx turbo test --filter=...@suaveplan/testing
|
||||
|
||||
# Repo-wide grep — only the canonical declaration plus the three deprecated
|
||||
# re-exports should remain
|
||||
grep -rln "interface Rng\b\|type Rng = " packages/ --include="*.ts" \
|
||||
| grep -v "/dist/\|/node_modules/"
|
||||
```
|
||||
|
||||
## File layout (per affected package)
|
||||
|
||||
No new files. The migration edits exactly three implementation files and
|
||||
their three sibling `.md` files (doc-drift gate, CLAUDE.md §4a):
|
||||
|
||||
```text
|
||||
packages/crypto/crypto-core/src/primitives/random-bytes/random-bytes.ts # swap alias
|
||||
packages/crypto/crypto-core/src/primitives/random-bytes/random-bytes.md # update API reference
|
||||
packages/crypto/crypto/src/index.ts # update forwarder
|
||||
packages/crypto/crypto/src/index.md # update API reference (if mentions Rng)
|
||||
packages/testing/testing/src/context/context.ts # swap interface for re-export
|
||||
packages/testing/testing/src/context/context.md # update API reference
|
||||
packages/testing/testing/package.json # add @suaveplan/types: workspace:^
|
||||
```
|
||||
@@ -0,0 +1,109 @@
|
||||
# OpenSpec Change: unify-rng-across-packages
|
||||
|
||||
- **Change ID**: unify-rng-across-packages
|
||||
- **Package**: `@suaveplan/types` (canonical) + downstream consumers
|
||||
- **Design**: ./design.md
|
||||
- **Spec delta**: ./specs/types/spec.md
|
||||
- **Status**: PROPOSED
|
||||
- **Date**: 2026-05-17
|
||||
|
||||
## Why
|
||||
|
||||
The 2026-04-27 `types-contract-tightening` change widened the canonical `Rng`
|
||||
interface in `@suaveplan/types` (adding optional `nextBytes` and
|
||||
`nextIntInRange` members) and explicitly **deferred** migration of the
|
||||
packages that were redefining `Rng`-shaped interfaces locally
|
||||
(`crypto-core`, `cache`, `idempotency`). That deferral has lapsed: every
|
||||
fresh package author who needs randomness still has to choose between
|
||||
importing the canonical `Rng` or hand-rolling a local one, and the existing
|
||||
local redefinitions silently fork the contract.
|
||||
|
||||
This change closes the loop. It migrates every genesis package with a local
|
||||
`Rng` type to consume the canonical `Rng` from `@suaveplan/types`, leaving
|
||||
deprecated aliases in place for one release cycle so consumers can update at
|
||||
their own pace.
|
||||
|
||||
## What changes
|
||||
|
||||
For each affected genesis package, the source files that currently declare a
|
||||
local `Rng` interface are migrated to:
|
||||
|
||||
1. Import the canonical `Rng` from `@suaveplan/types`.
|
||||
2. Re-export the canonical `Rng` from the original module path under the
|
||||
original name, with a "Backwards-compatibility alias for `@suaveplan/types`
|
||||
`Rng`" prose comment (per CLAUDE.md §2 — no `@deprecated` JSDoc).
|
||||
3. Add `@suaveplan/types` as a `workspace:^` dependency where missing.
|
||||
4. Re-run `bunx turbo typecheck` and `bunx turbo test` to confirm zero
|
||||
runtime / behavioural change.
|
||||
|
||||
The spec delta is a `## MODIFIED Requirements` block on the existing
|
||||
`Rng has optional bounded operations` requirement in `openspec/specs/types/`
|
||||
that adds an explicit single-source-of-truth clause: no other `@suaveplan/*`
|
||||
package may declare its own `Rng` interface; downstream packages that need a
|
||||
narrower or wider random-source shape MUST either consume `Rng` directly or
|
||||
define a sub-interface that `extends` it.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/types` to enumerate. Packages that
|
||||
currently declare a local `Rng` type (surveyed via
|
||||
`grep -rln "interface Rng\b\|type Rng = " packages/`) are:
|
||||
|
||||
- `@suaveplan/crypto-core` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||||
(declares `type Rng = ByteSource` as a back-compat alias in
|
||||
`src/primitives/random-bytes/random-bytes.ts`)
|
||||
- `@suaveplan/crypto` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||||
(re-exports the alias from `crypto-core` via `src/index.ts`)
|
||||
- `@suaveplan/testing` — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||||
(declares a full `interface Rng` in `src/context/context.ts`; consumed by
|
||||
`src/factories/factory/factory.ts`)
|
||||
|
||||
For every other dependent of `@suaveplan/types`: [x] no action needed —
|
||||
they already consume `Rng` from `@suaveplan/types` and are unaffected by
|
||||
this change.
|
||||
|
||||
### Source mapping (local Rng → canonical)
|
||||
|
||||
| Package | Local file | Local symbol | Action |
|
||||
|---|---|---|---|
|
||||
| `@suaveplan/crypto-core` | `src/primitives/random-bytes/random-bytes.ts` | `type Rng = ByteSource` | Replace with deprecated re-export of canonical `Rng`; keep `ByteSource` (distinct contract). |
|
||||
| `@suaveplan/crypto` | `src/index.ts` | re-export `Rng` from `crypto-core` | Update re-export to point at canonical alias. |
|
||||
| `@suaveplan/testing` | `src/context/context.ts` | `interface Rng { nextUint32; nextFloat }` | Import canonical `Rng`; re-export as deprecated alias. Add `@suaveplan/types` workspace dep (rule 8 exception). |
|
||||
|
||||
### Verification
|
||||
|
||||
- **Verified by:** `bunx turbo typecheck test --filter=@suaveplan/crypto-core`
|
||||
- **Verified by:** `bunx turbo typecheck test --filter=@suaveplan/crypto`
|
||||
- **Verified by:** `bunx turbo typecheck test --filter=@suaveplan/testing`
|
||||
- **Verified by:** `bunx turbo typecheck test --filter=...@suaveplan/testing`
|
||||
(dependents of `testing` consume `Rng` transitively via `factory.ts`)
|
||||
- **Verified by:** repo-wide grep `grep -rln "interface Rng\b\|type Rng = " packages/ --include="*.ts" | grep -v "/dist/\|/node_modules/"` returns only the canonical declaration in `packages/core/types/src/rng.ts` plus the deprecated alias forwarders the change introduces.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Widening the canonical `Rng`.** If a consumer needs additional members
|
||||
(e.g. `nextBigInt`, streaming bytes), open a separate
|
||||
`types-contract-tightening`-style change against `@suaveplan/types`.
|
||||
- **Removing the deprecated aliases.** Follow-up change (one release cycle
|
||||
after this one archives) removes the `Rng` re-exports from `crypto-core`,
|
||||
`crypto`, and `testing`. Tracked as a TODO in this proposal's design.md.
|
||||
- **Upstream-only packages (`cache`, `idempotency` upstream RNG redefinitions).**
|
||||
The 2026-04-27 deferral text named `cache` and `idempotency` as RNG-shape
|
||||
redefiners. On genesis, `@suaveplan/idempotency` and the three cache
|
||||
packages (`@suaveplan/cache`, `cache-storage`, `cache-redis`) do **not**
|
||||
currently reference `Rng` at all (verified by grep). When those packages
|
||||
port the upstream randomness features, the migration must consume the
|
||||
canonical `Rng` directly — there is no local type to migrate here yet.
|
||||
- **Runtime behaviour changes.** This change is type-only; the test suite
|
||||
must remain green pre- and post-migration with zero assertion edits.
|
||||
|
||||
## Migration
|
||||
|
||||
- **Old**: consumers importing `Rng` from `@suaveplan/crypto-core`,
|
||||
`@suaveplan/crypto`, or `@suaveplan/testing`.
|
||||
- **New**: import `Rng` from `@suaveplan/types`.
|
||||
- **Search command**: `grep -rln "from ['\"]@suaveplan/\(crypto-core\|crypto\|testing\)['\"]" packages/ | xargs grep -l "Rng\b"`
|
||||
- **Recommended replacement**: `import type { Rng } from "@suaveplan/types"`.
|
||||
The deprecated alias re-exports remain functional for one release cycle.
|
||||
@@ -0,0 +1,38 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: types (types)
|
||||
|
||||
**Package**: `@suaveplan/types`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions.
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Rng has optional bounded operations
|
||||
|
||||
The `Rng` interface SHALL declare `nextBytes(n: number): Uint8Array` and `nextIntInRange(min: number, maxExclusive: number): number` as OPTIONAL members. Implementations SHOULD provide them when efficient (CSPRNGs typically expose `getRandomValues` directly; deterministic seeded RNGs can derive both from `nextUint32`). Consumers needing them MUST narrow on presence (`if (rng.nextBytes)`). `nextIntInRange` is contractually expected to use unbiased rejection sampling, not the `% range` shortcut.
|
||||
|
||||
`@suaveplan/types` SHALL be the **single canonical home** for the `Rng` interface across the genesis monorepo. No other `@suaveplan/*` package MAY declare its own top-level `Rng` interface or `type Rng = ...` alias that differs structurally from the canonical export. Packages that need a narrower or wider random-source shape MUST either (a) consume the canonical `Rng` directly, (b) define a sub-interface that `extends Rng`, or (c) define a distinct concept under a non-`Rng` name (e.g. `ByteSource` for a callable byte-producer, since that is a different contract from a uniform-numeric RNG). Packages MAY re-export the canonical `Rng` under the name `Rng` as a backwards-compatibility forwarder during a documented deprecation window.
|
||||
|
||||
**Verified by:** `src/rng.ts` (interface) AND `tests/spec-conformance.test.ts` (optional-member presence + minimal-impl assignability) AND repo-wide `grep -rln "interface Rng\b\|type Rng = " packages/ --include="*.ts" | grep -v "/dist/\|/node_modules/"` returns only `packages/core/types/src/rng.ts` plus documented deprecated re-export forwarders.
|
||||
|
||||
#### Scenario: nextBytes is optional
|
||||
|
||||
- **WHEN** a consumer declares `const r: Rng = { nextUint32: () => 0, nextFloat: () => 0 }`
|
||||
- **THEN** `tsc` accepts (no `nextBytes` provided)
|
||||
|
||||
#### Scenario: nextIntInRange signature
|
||||
|
||||
- **WHEN** an implementation provides `nextIntInRange(min, maxExclusive): number`
|
||||
- **THEN** `tsc` accepts AND `nextIntInRange(0, 10)` returns `number`
|
||||
|
||||
#### Scenario: canonical Rng is the only authoritative declaration
|
||||
|
||||
- **WHEN** a developer runs `grep -rln "interface Rng\b\|type Rng = " packages/ --include="*.ts" | grep -v "/dist/\|/node_modules/"` across the genesis monorepo
|
||||
- **THEN** the only top-level `Rng` declaration returned is `packages/core/types/src/rng.ts`, and any additional matches are documented deprecated re-export forwarders (e.g. `export type { Rng } from "@suaveplan/types";`) — never an independent interface body
|
||||
|
||||
#### Scenario: sub-interface extension is permitted
|
||||
|
||||
- **WHEN** a downstream package defines `interface BoundedRng extends Rng { nextIntInRange(min: number, maxExclusive: number): number }` to require the optional member
|
||||
- **THEN** `tsc` accepts AND any value of type `BoundedRng` is assignable to `Rng`
|
||||
@@ -0,0 +1,64 @@
|
||||
# Tasks: unify-rng-across-packages
|
||||
|
||||
## 1. `@suaveplan/crypto-core` — migrate Rng alias
|
||||
|
||||
- [ ] 1.1 Locate local `Rng` declaration: `packages/crypto/crypto-core/src/primitives/random-bytes/random-bytes.ts` line 12 (`type Rng = ByteSource`).
|
||||
- [ ] 1.2 Confirm structural compatibility — `ByteSource = (n: number) => Uint8Array` is **not** structurally compatible with canonical `Rng`; see design.md compatibility table for rationale.
|
||||
- [ ] 1.3 Remove `type Rng = ByteSource` alias; keep `ByteSource` under its real name.
|
||||
- [ ] 1.4 Add deprecated re-export: `export type { Rng } from "@suaveplan/types";` with prose comment ("Backwards-compatibility alias for `@suaveplan/types` `Rng` — prefer the canonical import; this re-export will be removed in the follow-up `remove-deprecated-rng-aliases` change.").
|
||||
- [ ] 1.5 Verify `@suaveplan/types` is in `dependencies` (it already is per `MANIFEST.md` dep graph).
|
||||
- [ ] 1.6 Update sibling `random-bytes.md` API reference: drop `Rng` from the local symbols list; document the back-compat re-export under "Implementation Notes".
|
||||
- [ ] 1.7 Update barrel re-exports if `Rng` is named in `packages/crypto/crypto-core/src/primitives/random-bytes/index.ts` or `src/primitives/index.ts` or `src/index.ts` — keep the name routed through the new deprecated forwarder.
|
||||
- [ ] 1.8 `bunx turbo typecheck --filter=@suaveplan/crypto-core` — green.
|
||||
- [ ] 1.9 `bunx turbo test --filter=@suaveplan/crypto-core` — green, zero assertion edits.
|
||||
- [ ] 1.10 `bunx turbo test:coverage --filter=@suaveplan/crypto-core` — 100/100/100/100 preserved.
|
||||
|
||||
## 2. `@suaveplan/crypto` — update forwarder
|
||||
|
||||
- [ ] 2.1 Locate `Rng` re-export in `packages/crypto/crypto/src/index.ts`.
|
||||
- [ ] 2.2 Confirm it forwards from `@suaveplan/crypto-core`; no further changes needed beyond confirming the canonical type flows through.
|
||||
- [ ] 2.3 If `Rng` is mentioned in `packages/crypto/crypto/src/index.md` or any sibling doc, update to reference `@suaveplan/types` as the canonical home.
|
||||
- [ ] 2.4 `bunx turbo typecheck --filter=@suaveplan/crypto` — green.
|
||||
- [ ] 2.5 `bunx turbo test --filter=@suaveplan/crypto` — green.
|
||||
|
||||
## 3. `@suaveplan/testing` — migrate Rng interface
|
||||
|
||||
- [ ] 3.1 Locate local `Rng` declaration: `packages/testing/testing/src/context/context.ts` line 45 (`interface Rng { nextUint32(): number; nextFloat(): number }`).
|
||||
- [ ] 3.2 Confirm structural compatibility — required members are an exact match for canonical `Rng`; canonical adds two OPTIONAL members which existing `cryptoRng` / `seededRng` implementations may omit.
|
||||
- [ ] 3.3 Add `"@suaveplan/types": "workspace:^"` to `packages/testing/testing/package.json` `dependencies` (rule 8 exception: testing-infrastructure may depend on `@suaveplan/types` for types-only consumption — verify byte-size test still passes since `@suaveplan/types` emits zero-runtime stubs).
|
||||
- [ ] 3.4 Replace `interface Rng { ... }` with `import type { Rng } from "@suaveplan/types"; export type { Rng };` plus prose comment marking the local re-export as a backwards-compatibility alias.
|
||||
- [ ] 3.5 Confirm `cryptoRng: Rng = { ... }` and `seededRng(seed): Rng` still typecheck against canonical `Rng` (they should — they provide only the required members).
|
||||
- [ ] 3.6 Update `packages/testing/testing/src/factories/factory/factory.ts` import — `import { cryptoRng, type Rng } from "../../context/context.js"` continues to work via the re-export, so no edit required unless the import path is changed for clarity.
|
||||
- [ ] 3.7 Update sibling `context.md` API reference: note that `Rng` is now a re-export of the canonical `@suaveplan/types` `Rng`; remove the local interface from "API Reference"; cross-link to `@suaveplan/types`.
|
||||
- [ ] 3.8 Update sibling `factory.md` if it mentions the local `Rng` source.
|
||||
- [ ] 3.9 Verify rule 8 byte-size guard still passes (`@suaveplan/types` emits zero runtime).
|
||||
- [ ] 3.10 `bunx turbo typecheck --filter=@suaveplan/testing` — green.
|
||||
- [ ] 3.11 `bunx turbo test --filter=@suaveplan/testing` — green, zero assertion edits.
|
||||
- [ ] 3.12 `bunx turbo test:coverage --filter=@suaveplan/testing` — 100/100/100/100 preserved.
|
||||
- [ ] 3.13 Dependent sweep: `bunx turbo test --filter=...@suaveplan/testing` — every consumer of `factory.ts` / `context.ts` `Rng` continues to typecheck and pass.
|
||||
|
||||
## 4. Spec delta
|
||||
|
||||
- [ ] 4.1 Author `specs/types/spec.md` delta with `## MODIFIED Requirements` block on `Rng has optional bounded operations` adding the single-source-of-truth clause.
|
||||
- [ ] 4.2 Carry `**Verified by:**` on the modified requirement.
|
||||
|
||||
## 5. Repo-wide verification gate
|
||||
|
||||
- [ ] 5.1 `grep -rln "interface Rng\b\|type Rng = " packages/ --include="*.ts" | grep -v "/dist/\|/node_modules/"` returns exactly:
|
||||
- `packages/core/types/src/rng.ts` (canonical)
|
||||
- the three deprecated re-export forwarders introduced above
|
||||
- [ ] 5.2 No other genesis package declares a local `Rng` interface.
|
||||
- [ ] 5.3 `bunx turbo lint typecheck --filter=...` (full repo) — green.
|
||||
|
||||
## 6. Doc-drift + behaviour-change propagation
|
||||
|
||||
- [ ] 6.1 Per CLAUDE.md §4a: every `.ts` edited has its sibling `.md` updated in the same change set.
|
||||
- [ ] 6.2 Per CLAUDE.md §13a: add the `## Migration` note to proposal.md (already authored).
|
||||
- [ ] 6.3 Repo-wide consumer scan: `grep -rln "from ['\"]@suaveplan/\(crypto-core\|crypto\|testing\)['\"]" packages/ | xargs grep -l "Rng\b"` — every hit either continues to compile via the deprecated forwarder OR is updated to the canonical import in this change set.
|
||||
|
||||
## 7. Fold + archive
|
||||
|
||||
- [ ] 7.1 Fold `specs/types/spec.md` delta into `openspec/specs/types/spec.md`.
|
||||
- [ ] 7.2 Move `openspec/changes/unify-rng-across-packages/` → `openspec/archive/<yyyy-mm-dd>-unify-rng-across-packages/`.
|
||||
- [ ] 7.3 Refresh `MANIFEST.md` / `MISSING.yml` / `STATUS.md` / `SUMMARY.md` per CLAUDE.md §13b (no package add/remove here, but dep-graph edge added: `@suaveplan/testing → @suaveplan/types`).
|
||||
- [ ] 7.4 Open the follow-up `remove-deprecated-rng-aliases` change to track removal of the three forwarders after one release cycle.
|
||||
206
openspec/changes/ab-testing-completion/proposal.md
Normal file
206
openspec/changes/ab-testing-completion/proposal.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# A/B testing completion — bandits, multivariate, segments, sequential, Bayesian, store
|
||||
|
||||
**Package**: `@suaveplan/ab-testing`
|
||||
**Status**: PROPOSED
|
||||
**Type**: spec-conformance + feature
|
||||
|
||||
## Why
|
||||
|
||||
A 2026-05-02 audit of the genesis `@suaveplan/ab-testing` package against the upstream library at `../suaveplan/packages/universal/ab-testing/src` and against this package's archived migration `openspec/archive/2026-04-24-enterprise-ab-testing/` found six entire capability surfaces silently scoped out without justification. The archived proposal's parity table either omits these features or marks them shipped without delivering them; the current genesis implementation ships only the frequentist subset (deterministic FNV-1a bucketing, targeting, conversion-rate results, assignment via providers).
|
||||
|
||||
Six feature surfaces are missing:
|
||||
|
||||
1. **Bandits** — `EpsilonGreedy`, `UCB1`, `ThompsonSamplingBandit` (153 L upstream `bandit/bandit.ts`). Without them, callers wanting exploration/exploitation must hand-roll the algorithms or import a third-party — which then can't reuse the assignment store, error codes, or OTel instrumentation already in genesis.
|
||||
2. **Multivariate engine** — `fullFactorial`, `fractionalFactorial`, `analyzeMultivariate`, factor-effect / interaction-effect analysis (226 L upstream `multivariate/multivariate-engine.ts`). The frequentist `computeResults` only handles A/B/C variant comparisons, not factor × level designs.
|
||||
3. **Segment analysis** — `analyzeBySegments`, `segmentConfidenceIntervals`, `detectSegmentInteraction` (268 L upstream `segmentation/segment-analyzer.ts`). Per-segment lift and segment × variant interactions are how product teams actually read experiment results; without them, the package only answers "did variant B win overall?".
|
||||
4. **Sequential testing** — `obrienFlemingBoundary`, `alphaSpent`, `interimAnalysis`, `runSequentialTest` (221 L upstream `sequential/sequential-test.ts`). Fixed-horizon hypothesis testing inflates Type I error when peeked at — alpha-spending boundaries are how production experimentation platforms permit early stopping safely.
|
||||
5. **Bayesian stats** — `betaPdf` / `betaCdf` / `betaQuantile` / `betaSample`, `posteriorBeta`, `thompsonSample`, `probabilityBGreaterA`, `credibleInterval`, `expectedLoss`, `bayesianAnalysis` (263 L upstream `stats/bayesian.ts`). The package ships pure point-estimate conversion rates with no posterior, no credible intervals, no probability-of-being-best — which the bandit and Bayesian analysis paths both require.
|
||||
6. **Assignment store** — `AssignmentStore` interface + `MemoryAssignmentStore` class (78 L upstream `store/assignment-store.ts`). Genesis currently re-derives assignments from providers on every read, which makes consistent assignment across server restarts impossible without each consumer re-implementing the persistence contract.
|
||||
|
||||
The audit found "These were silently dropped from `openspec/archive/2026-04-24-enterprise-ab-testing` scope (no archive justification)." — this change closes the gap and ratifies a parity table that explicitly admits these features are required.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/ab-testing` — additions
|
||||
|
||||
#### Bandit subsystem
|
||||
|
||||
- ADDED: `Bandit<TArmId extends string | number = number>` interface — `selectArm(): TArmId`, `update(arm: TArmId, reward: number): void`, `getValues(): readonly number[]`, `getCounts(): readonly number[]`. Generic over arm-id type so callers can use string variant IDs directly instead of a `number → variantId` map.
|
||||
- ADDED: `EpsilonGreedyBandit<TArmId>` class — explore with probability `epsilon`, exploit best-known arm otherwise; injectable `Rng` from `@suaveplan/types::Rng`.
|
||||
- ADDED: `UCB1Bandit<TArmId>` class — Upper Confidence Bound, plays each arm once, then `value + sqrt(2 ln(N) / n)`.
|
||||
- ADDED: `ThompsonSamplingBandit<TArmId>` class — Beta-Bernoulli posterior sampling with injectable `Rng`.
|
||||
- ADDED: `BanditConfig<TArmId>` interface — `arms: readonly TArmId[]`, optional `epsilon`, `rng`, `priorAlpha`, `priorBeta`.
|
||||
|
||||
#### Multivariate engine
|
||||
|
||||
- ADDED: `Factor` interface — `name: string`, `levels: readonly string[]`.
|
||||
- ADDED: `TreatmentCombination`, `TreatmentResult`, `FactorEffect`, `LevelEffect`, `InteractionEffect`, `MultivariateResult` interfaces.
|
||||
- ADDED: `fullFactorial(factors)` — generate every combination.
|
||||
- ADDED: `fractionalFactorial(factors, fraction)` — sample every Nth combination.
|
||||
- ADDED: `analyzeMultivariate(factors, data)` — produce `MultivariateResult` with treatment-level rates, factor-effect importance scores, and pairwise interaction-strength scores.
|
||||
|
||||
#### Segment analysis
|
||||
|
||||
- ADDED: `SegmentResult`, `SegmentConfidenceInterval`, `SegmentInteraction` interfaces.
|
||||
- ADDED: `analyzeBySegments(assignments, conversions, variantIds, segmentAttribute)` — variant rates per segment value, with simple two-proportion z-test significance flag.
|
||||
- ADDED: `segmentConfidenceIntervals(assignments, conversions, variantIds, segmentAttribute, confidence?)` — Wald intervals on conversion rates per segment × variant.
|
||||
- ADDED: `detectSegmentInteraction(assignments, conversions, variantIds, segmentAttribute)` — interaction-strength score between segment value and variant assignment.
|
||||
|
||||
#### Sequential testing
|
||||
|
||||
- ADDED: `SequentialTestConfig` — `maxLooks`, `alpha`, `mde`, `power`, optional `boundaryType: 'pocock' | 'obrien-fleming'` (default `'obrien-fleming'`).
|
||||
- ADDED: `InterimResult` — `lookNumber`, `totalLooks`, `informationFraction`, `spendingBoundary`, `testStatistic`, `pValue`, `shouldStop`, `stopReason: 'significance' | 'futility' | null`, `cumulativeAlphaSpent`.
|
||||
- ADDED: `obrienFlemingBoundary(informationFraction, alpha)` — z-boundary at fractional time.
|
||||
- ADDED: `pocockBoundary(informationFraction, alpha, maxLooks)` — constant-z Pocock alternative (genesis enhancement; upstream ships only O'Brien-Fleming).
|
||||
- ADDED: `alphaSpent(informationFraction, alpha, boundaryType?)` — cumulative alpha spent under the chosen spending function.
|
||||
- ADDED: `interimAnalysis(config, lookNumber, controlConv, controlPart, treatmentConv, treatmentPart)` — single look.
|
||||
- ADDED: `runSequentialTest(config, dataAtLooks)` — drive the sequence, stop on first `shouldStop`.
|
||||
|
||||
#### Bayesian stats
|
||||
|
||||
- ADDED: `BetaParams` interface — `alpha`, `beta`.
|
||||
- ADDED: `BayesianResult` interface — `posteriors`, `probabilityBIsGreater`, `credibleIntervalA`, `credibleIntervalB`, `expectedLossA`, `expectedLossB`.
|
||||
- ADDED: `betaPdf(x, alpha, beta)` — Beta probability density.
|
||||
- ADDED: `betaCdf(x, alpha, beta)` — Beta cumulative distribution via regularized incomplete beta + continued fraction.
|
||||
- ADDED: `betaQuantile(p, alpha, beta)` — inverse CDF via bisection.
|
||||
- ADDED: `betaSample(alpha, beta, rng?)` — Beta draw via two Gamma draws (Marsaglia-Tsang).
|
||||
- ADDED: `posteriorBeta(conversions, participants, priorAlpha?, priorBeta?)` — conjugate Beta-Binomial posterior.
|
||||
- ADDED: `thompsonSample(posteriors, rng?)` — pick the variant with highest sampled posterior value.
|
||||
- ADDED: `probabilityBGreaterA(a, b, simulations?, rng?)` — Monte Carlo `P(B > A)`.
|
||||
- ADDED: `credibleInterval(params, width?)` — quantile-based credible interval.
|
||||
- ADDED: `expectedLoss(chosen, other, simulations?, rng?)` — expected lift forfeited by choosing `chosen`.
|
||||
- ADDED: `bayesianAnalysis(results, priorAlpha?, priorBeta?, rng?)` — full pairwise analysis on two `VariantResults`.
|
||||
|
||||
#### Assignment store
|
||||
|
||||
- ADDED: `AssignmentStoreLike` interface — `get(experimentId, userId)`, `set(assignment)`, `getByExperiment(experimentId)`, `getByUser(userId)`, `deleteByExperiment(experimentId)`. Async throughout. Promoted to `@suaveplan/types` per AGENTS.md §0 rule 5 (referenced by ≥2 packages once consumers wire it up).
|
||||
- ADDED: `MemoryAssignmentStore` class implementing `AssignmentStoreLike`.
|
||||
- ADDED: `createMemoryAssignmentStore()` factory.
|
||||
- ADDED: optional `store?: AssignmentStoreLike` field on `ExperimentManagerOptions`. When present, the manager SHALL persist via the store and check the store before re-bucketing.
|
||||
|
||||
#### Errors
|
||||
|
||||
- ADDED: `AB_CODES` entries — `INVALID_BANDIT_CONFIG`, `INVALID_FACTOR_DESIGN`, `INVALID_SEGMENT_ATTRIBUTE`, `INSUFFICIENT_SAMPLE_SIZE`, `INVALID_SEQUENTIAL_CONFIG`.
|
||||
|
||||
### Subpath exports (additive)
|
||||
|
||||
- ADDED: `./bandit`, `./multivariate`, `./segments`, `./sequential`, `./stats`, `./store`.
|
||||
|
||||
### Existing surface — preserved
|
||||
|
||||
- Bucketing, targeting, results, providers, errors, types, schemas, `createExperimentManager` — unchanged.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Multi-armed contextual bandits** (LinUCB, Thompson with feature vectors) — upstream does not ship these; genesis will not introduce them without a separate proposal and a real consumer asking for them. The three context-free algorithms cover the documented use cases.
|
||||
- **Frequency-domain / time-series adjustments** (CUPED, Bayesian state-space models) — not in upstream; orthogonal feature better served by a dedicated `@suaveplan/experiment-stats` follow-up if demand emerges.
|
||||
- **Persistent assignment-store adapters** beyond `MemoryAssignmentStore` (Redis, SurrealDB, SQLite) — kept out of this proposal to keep dep-graph blast radius small. Each adapter will land as its own package (`@suaveplan/ab-testing-redis`, etc.) once `MemoryAssignmentStore` and the `AssignmentStoreLike` interface ship and the contract is pinned by `testing-integration-redis` etc.
|
||||
- **Variance-reduced bandits** (linear / kernelized regression bandits) — large surface, not in upstream; deferred.
|
||||
- **Multi-objective optimization** (Pareto-frontier multi-arm bandits) — not in upstream; deferred.
|
||||
- **Mutual-information-based factor pruning** for the multivariate engine — not in upstream; the importance-by-sum-of-squares heuristic is sufficient for the documented use cases. Architectural reason: a real multi-objective design would change `MultivariateResult.bestCombination` semantics from "highest mean rate" to "Pareto-optimal set", which is a breaking shape change that needs its own proposal.
|
||||
- **Continuous monitoring / always-valid p-values** (mSPRT, e-values) — distinct statistical regime from alpha-spending; the chosen O'Brien-Fleming + Pocock pair covers the documented "interim look" use case and is what upstream ships.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change is a migration from `../suaveplan/packages/universal/ab-testing/src/`. There is no genesis `enterprise/ab-testing.md` audit doc, so source mapping cites the upstream package source directly.
|
||||
|
||||
| Requirement | Source file | Source lines | Notes |
|
||||
|---|---|---|---|
|
||||
| `BanditAlgorithm` interface | `packages/universal/ab-testing/src/bandit/bandit.ts` | L3-L13 | Renamed to `Bandit<TArmId>` and made generic over arm id |
|
||||
| `EpsilonGreedy` class | `packages/universal/ab-testing/src/bandit/bandit.ts` | L15-L57 | Renamed to `EpsilonGreedyBandit<TArmId>`; replace `Math.random` default with explicit `Rng` injection |
|
||||
| `UCB1` class | `packages/universal/ab-testing/src/bandit/bandit.ts` | L59-L105 | Renamed to `UCB1Bandit<TArmId>` |
|
||||
| `ThompsonSamplingBandit` class | `packages/universal/ab-testing/src/bandit/bandit.ts` | L107-L153 | Verbatim (already correctly named); replace `Math.random` default with explicit `Rng` injection |
|
||||
| `Factor` / `TreatmentCombination` / `TreatmentResult` / `FactorEffect` / `LevelEffect` / `InteractionEffect` / `MultivariateResult` interfaces | `packages/universal/ab-testing/src/multivariate/multivariate-engine.ts` | L1-L50 | Verbatim |
|
||||
| `fullFactorial` | `packages/universal/ab-testing/src/multivariate/multivariate-engine.ts` | L52-L82 | Verbatim |
|
||||
| `fractionalFactorial` | `packages/universal/ab-testing/src/multivariate/multivariate-engine.ts` | L84-L93 | Verbatim |
|
||||
| `analyzeMultivariate` + factor-effect / interaction-effect helpers | `packages/universal/ab-testing/src/multivariate/multivariate-engine.ts` | L95-L226 | Verbatim |
|
||||
| `SegmentResult` / `SegmentConfidenceInterval` / `SegmentInteraction` interfaces | `packages/universal/ab-testing/src/segmentation/segment-analyzer.ts` | L7-L33 | Verbatim |
|
||||
| `analyzeBySegments` | `packages/universal/ab-testing/src/segmentation/segment-analyzer.ts` | L35-L85 | Verbatim |
|
||||
| `segmentConfidenceIntervals` | `packages/universal/ab-testing/src/segmentation/segment-analyzer.ts` | L87-L123 | Verbatim |
|
||||
| `detectSegmentInteraction` | `packages/universal/ab-testing/src/segmentation/segment-analyzer.ts` | L125-L162 | Verbatim |
|
||||
| `normalQuantile` helper | `packages/universal/ab-testing/src/segmentation/segment-analyzer.ts` | L226-L268 | Move to shared `stats/normal.ts` (used by both segmentation and sequential) |
|
||||
| `SequentialTestConfig` / `InterimResult` interfaces | `packages/universal/ab-testing/src/sequential/sequential-test.ts` | L1-L24 | Verbatim |
|
||||
| `obrienFlemingBoundary` | `packages/universal/ab-testing/src/sequential/sequential-test.ts` | L26-L35 | Verbatim |
|
||||
| `alphaSpent` | `packages/universal/ab-testing/src/sequential/sequential-test.ts` | L37-L43 | Extend with `boundaryType` parameter |
|
||||
| `interimAnalysis` | `packages/universal/ab-testing/src/sequential/sequential-test.ts` | L45-L114 | Verbatim |
|
||||
| `runSequentialTest` | `packages/universal/ab-testing/src/sequential/sequential-test.ts` | L129-L154 | Verbatim |
|
||||
| `BetaParams` / `BayesianResult` interfaces | `packages/universal/ab-testing/src/stats/bayesian.ts` | L3-L17 | Verbatim |
|
||||
| `betaPdf` / `betaCdf` / `betaQuantile` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L48-L116 | Verbatim |
|
||||
| `betaSample` (Marsaglia-Tsang) | `packages/universal/ab-testing/src/stats/bayesian.ts` | L118-L159 | Verbatim; replace `Math.random` default with `Rng` injection |
|
||||
| `posteriorBeta` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L161-L172 | Verbatim |
|
||||
| `thompsonSample` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L174-L189 | Verbatim |
|
||||
| `probabilityBGreaterA` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L191-L205 | Verbatim |
|
||||
| `credibleInterval` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L207-L216 | Verbatim |
|
||||
| `expectedLoss` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L218-L232 | Verbatim |
|
||||
| `bayesianAnalysis` | `packages/universal/ab-testing/src/stats/bayesian.ts` | L234-L263 | Verbatim |
|
||||
| `AssignmentStore` interface | `packages/universal/ab-testing/src/store/assignment-store.ts` | L3-L15 | Renamed to `AssignmentStoreLike`; promoted to `@suaveplan/types` |
|
||||
| `MemoryAssignmentStore` class | `packages/universal/ab-testing/src/store/assignment-store.ts` | L17-L78 | Verbatim |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Generic `TArmId` parameter on `Bandit<TArmId>` | Ergonomic improvement over upstream's positional `number` arms — callers pass string variant IDs directly, eliminating an error-prone `number → variantId` lookup in user code. Upstream uses array indices; genesis preserves array indices as the default `TArmId = number` so existing-shape code continues to work. |
|
||||
| Pocock spending boundary alternative | Upstream ships only O'Brien-Fleming. Pocock's constant-z spending is the canonical alternative; offering both lets callers pick the conservative-early (O'Brien-Fleming) vs. liberal-early (Pocock) trade-off. Adds < 30 lines and zero deps. |
|
||||
| Inject `Rng` instead of defaulting to `Math.random` | CLAUDE.md determinism rule (Critical rule 9) requires no module-scope `Math.random()`. Upstream defaults all `rng` parameters to `Math.random`; genesis SHALL require an explicit `Rng` from `@suaveplan/types` at call boundary, with `crypto.getRandomValues`-based default available via `createDefaultRng()`. |
|
||||
| `AssignmentStoreLike` promoted to `@suaveplan/types` | AGENTS.md §0 rule 5: shared interface across ≥2 packages (`@suaveplan/ab-testing` defines, future `@suaveplan/ab-testing-redis` / `-surrealdb` consume) must live in the types-only foundation to prevent circular deps. Upstream defines it inline. |
|
||||
| New `AB_CODES` entries — `INVALID_BANDIT_CONFIG`, `INVALID_FACTOR_DESIGN`, `INVALID_SEGMENT_ATTRIBUTE`, `INSUFFICIENT_SAMPLE_SIZE`, `INVALID_SEQUENTIAL_CONFIG` | Upstream throws plain `Error`; genesis-wide policy (CLAUDE.md Critical rule 2 "every thrown error route through `@suaveplan/error`") requires registered codes for each public failure path. |
|
||||
| OTel counters / histograms — `ab.bandit.arm.selected` (counter, by arm_id), `ab.multivariate.combinations.evaluated` (counter), `ab.sequential.alpha.spent` (histogram), `ab.bayesian.simulations.duration_ms` (histogram), `ab.assignment.store.lookup` (counter, by hit/miss) | Genesis-wide instrumentation policy. Upstream ships zero observability. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each requirement in the spec delta carries a `**Verified by:**` line pointing at concrete `it(...)` titles. Test files that will exist after this change ships:
|
||||
|
||||
- `packages/testing/ab-testing/src/bandit/bandit.test.ts` — `EpsilonGreedyBandit` exploration vs. exploitation distribution; `UCB1Bandit` plays each arm once; `ThompsonSamplingBandit` posterior update on success vs. failure
|
||||
- `packages/testing/ab-testing/src/bandit/bandit.spec-conformance.test.ts` — injected `Rng` is the only randomness source; `INVALID_BANDIT_CONFIG` thrown when `arms.length < 2`
|
||||
- `packages/testing/ab-testing/src/multivariate/multivariate-engine.test.ts` — `fullFactorial` exhaustive enumeration; `fractionalFactorial` step semantics; factor importance ordering; `bestCombination` is the highest-rate treatment with > 0 participants
|
||||
- `packages/testing/ab-testing/src/multivariate/multivariate-engine.spec-conformance.test.ts` — interaction strength is non-negative; pairwise interactions cover every `(i, j)` with `i < j`
|
||||
- `packages/testing/ab-testing/src/segments/segment-analyzer.test.ts` — segment grouping by attribute value; significance gates on min sample size (≥10); confidence-interval coverage is `[0, 1]`-clamped
|
||||
- `packages/testing/ab-testing/src/segments/segment-analyzer.spec-conformance.test.ts` — segment × variant interaction-strength threshold; `INVALID_SEGMENT_ATTRIBUTE` thrown for attributes absent from every assignment
|
||||
- `packages/testing/ab-testing/src/sequential/sequential-test.test.ts` — O'Brien-Fleming boundary monotone-decreasing in informationFraction; Pocock boundary constant; `runSequentialTest` stops at first `shouldStop`
|
||||
- `packages/testing/ab-testing/src/sequential/sequential-test.spec-conformance.test.ts` — alpha-spent monotone-increasing; futility check only fires when `informationFraction ≥ 0.75`; `INVALID_SEQUENTIAL_CONFIG` thrown for `alpha ∉ (0, 1)` or `maxLooks < 1`
|
||||
- `packages/testing/ab-testing/src/stats/bayesian.test.ts` — `betaPdf` / `betaCdf` / `betaQuantile` round-trip; `posteriorBeta` conjugate update; `probabilityBGreaterA` Monte Carlo converges to closed-form for known cases; `credibleInterval` width parameter
|
||||
- `packages/testing/ab-testing/src/stats/bayesian.spec-conformance.test.ts` — injected `Rng` reproducibility for `betaSample` / `thompsonSample` / `expectedLoss`
|
||||
- `packages/testing/ab-testing/src/store/memory-assignment-store.test.ts` — `set`/`get` round-trip; `getByExperiment` / `getByUser` filtering; `deleteByExperiment` cascade; `clear`
|
||||
- `packages/testing/ab-testing/src/store/memory-assignment-store.spec-conformance.test.ts` — `AssignmentStoreLike` contract pinned (will be re-used to verify future Redis/SurrealDB adapters)
|
||||
- `tests/ab-testing-public-surface.test.ts` (cross-package) — pins the additive subpath-export surface so removal is a deliberate spec change
|
||||
|
||||
## Migration impact
|
||||
|
||||
- **Existing `createExperimentManager` consumers** — no behavior change. The manager gains an optional `store?: AssignmentStoreLike` field on `ExperimentManagerOptions`; existing consumers that don't pass it keep the in-memory derivation path.
|
||||
- **`@suaveplan/ab-testing` direct dependents** — none in the genesis monorepo (`bunx turbo ls --filter=...@suaveplan/ab-testing` returns the package itself only).
|
||||
- **Subpath imports** — six new subpaths added (`./bandit`, `./multivariate`, `./segments`, `./sequential`, `./stats`, `./store`). The root barrel re-exports from each. No removed or renamed exports.
|
||||
- **`@suaveplan/types`** — the `AssignmentStoreLike` interface is promoted into `@suaveplan/types` per AGENTS.md §0 rule 5. This is additive to `@suaveplan/types` and does not affect existing consumers.
|
||||
- **Error-code consumers** — additive only; existing `AB_CODES` entries preserved.
|
||||
|
||||
## Parity Checklist
|
||||
|
||||
| Capability | Upstream | Migrated | Notes |
|
||||
|---|---|---|---|
|
||||
| Deterministic FNV-1a bucketing | YES | YES | Already on genesis; preserved |
|
||||
| Targeting rules | YES | YES | Already on genesis; preserved |
|
||||
| Variant assignment & providers | YES | YES | Already on genesis; preserved |
|
||||
| Frequentist results (`computeResults`) | YES | YES | Already on genesis; preserved |
|
||||
| Bandits (`EpsilonGreedy`, `UCB1`, `ThompsonSampling`) | YES | YES | This change |
|
||||
| Multivariate engine (full + fractional factorial, factor effects, interactions) | YES | YES | This change |
|
||||
| Segment analysis (per-segment lift, CI, interaction) | YES | YES | This change |
|
||||
| Sequential testing (O'Brien-Fleming alpha-spending, futility, early stopping) | YES | YES | This change |
|
||||
| Sequential testing (Pocock boundary alternative) | NO | YES | Genesis enhancement; documented in "no source line" table |
|
||||
| Bayesian stats (Beta-Binomial posteriors, credible intervals, expected loss) | YES | YES | This change |
|
||||
| Assignment store (`AssignmentStore` + `MemoryAssignmentStore`) | YES | YES | This change |
|
||||
| Persistent assignment-store adapters (Redis / SurrealDB / SQLite) | NO | NO | Scoped out — each adapter ships as its own package once `AssignmentStoreLike` is pinned |
|
||||
| Contextual bandits (LinUCB, Thompson with features) | NO | NO | Scoped out — not in upstream, no consumer asking |
|
||||
| Multi-objective / Pareto bandits | NO | NO | Scoped out — not in upstream |
|
||||
| Continuous-monitoring p-values (mSPRT, e-values) | NO | NO | Scoped out — distinct statistical regime; alpha-spending covers the documented use case |
|
||||
|
||||
## Affected dependents
|
||||
|
||||
`bunx turbo ls --filter=...@suaveplan/ab-testing` enumerates only the package itself (no in-repo dependents).
|
||||
|
||||
- `@suaveplan/ab-testing` (self) — [x] tests run / [x] docs reviewed / [ ] no action needed
|
||||
|
||||
`@suaveplan/types` is touched additively (new `AssignmentStoreLike` interface). Verify the dep graph after promotion:
|
||||
|
||||
- `@suaveplan/types` — [x] tests run / [x] docs reviewed / [ ] no action needed (purely additive types-only export; `byte-size` test guards no runtime code is shipped)
|
||||
|
||||
No external dependents on genesis. The proposal is additive (no removed or modified exports), so no downstream sweep is required.
|
||||
203
openspec/changes/ab-testing-completion/specs/ab-testing/spec.md
Normal file
203
openspec/changes/ab-testing-completion/specs/ab-testing/spec.md
Normal file
@@ -0,0 +1,203 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: ab-testing (ab-testing-completion)
|
||||
|
||||
**Package**: `@suaveplan/ab-testing`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. No exceptions. (See `openspec/AGENTS.md` Section 13.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Bandit algorithms with shared `Bandit<TArmId>` interface
|
||||
|
||||
The package SHALL provide three multi-armed bandit algorithms — `EpsilonGreedyBandit<TArmId>`, `UCB1Bandit<TArmId>`, `ThompsonSamplingBandit<TArmId>` — all implementing a shared `Bandit<TArmId>` interface with `selectArm(): TArmId`, `update(arm: TArmId, reward: number): void`, `getValues(): readonly number[]`, `getCounts(): readonly number[]`. The generic `TArmId` parameter SHALL default to `number` (array index) so existing-shape upstream code continues to work; callers MAY pass string variant IDs as arms. Every algorithm SHALL accept an explicit `Rng` from `@suaveplan/types::Rng` (no module-scope `Math.random`). Construction SHALL throw `AbTestingError(AB_CODES.INVALID_BANDIT_CONFIG, ...)` when `arms.length < 2` or, for `EpsilonGreedyBandit`, `epsilon ∉ [0, 1]`.
|
||||
|
||||
#### Scenario: epsilon-greedy explores when rng < epsilon
|
||||
|
||||
- **WHEN** `EpsilonGreedyBandit` is constructed with `arms: ['a', 'b', 'c']`, `epsilon: 0.5`, and an `Rng` that returns `0.1` then `0.7`
|
||||
- **THEN** `selectArm()` returns the result of `rng() * 3` floored as a uniform random arm (exploration branch)
|
||||
|
||||
#### Scenario: UCB1 plays each arm once before applying score
|
||||
|
||||
- **WHEN** `UCB1Bandit` is constructed with three arms and `selectArm()` is called three times with no intervening `update`
|
||||
- **THEN** the first three returns are `arms[0]`, `arms[1]`, `arms[2]` in order
|
||||
|
||||
#### Scenario: Thompson sampling updates posterior on success
|
||||
|
||||
- **WHEN** `ThompsonSamplingBandit.update(arm, 1)` is called
|
||||
- **THEN** the underlying `successes[arm]` increments by 1; `update(arm, 0)` increments `failures[arm]` by 1
|
||||
|
||||
#### Scenario: invalid bandit config throws AbTestingError
|
||||
|
||||
- **WHEN** `new EpsilonGreedyBandit({ arms: ['x'], epsilon: 0.1, rng })` is called (only one arm)
|
||||
- **THEN** the constructor throws `AbTestingError` with code `AB_CODES.INVALID_BANDIT_CONFIG`
|
||||
|
||||
**Verified by:** `src/bandit/bandit.test.ts::"EpsilonGreedyBandit explores when rng draw is below epsilon"` AND `src/bandit/bandit.test.ts::"UCB1Bandit plays each arm once before scoring"` AND `src/bandit/bandit.test.ts::"ThompsonSamplingBandit updates posterior on success and failure"` AND `src/bandit/bandit.spec-conformance.test.ts::"throws INVALID_BANDIT_CONFIG when arms.length < 2"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Multivariate engine with full + fractional factorial design
|
||||
|
||||
The package SHALL provide a multivariate-experiment design engine. `fullFactorial(factors)` SHALL return every combination of factor levels (Cartesian product). `fractionalFactorial(factors, fraction)` SHALL return every Nth combination where `N = round(1 / fraction)`. `analyzeMultivariate(factors, data)` SHALL return a `MultivariateResult` containing per-treatment conversion rates, per-factor `FactorEffect` objects with `LevelEffect[]` and a sum-of-squares `importance` score, pairwise `InteractionEffect[]` covering every `(i, j)` pair with `i < j`, and a `bestCombination` (highest-rate treatment with `participants > 0`). Construction with `factors.length === 0` or any factor having `levels.length === 0` SHALL throw `AbTestingError(AB_CODES.INVALID_FACTOR_DESIGN, ...)`.
|
||||
|
||||
#### Scenario: full factorial enumerates Π levels
|
||||
|
||||
- **WHEN** `fullFactorial([{ name: 'a', levels: ['x', 'y'] }, { name: 'b', levels: ['1', '2', '3'] }])` is called
|
||||
- **THEN** the result has length `2 * 3 = 6` and each combination has unique `id`
|
||||
|
||||
#### Scenario: fractional factorial samples every Nth combination
|
||||
|
||||
- **WHEN** `fractionalFactorial(factors, 0.5)` is called against a 6-combination factor set
|
||||
- **THEN** the result length is 3 and the kept combinations are at indices 0, 2, 4 in `fullFactorial(factors)`
|
||||
|
||||
#### Scenario: factor importance is a non-negative sum-of-squares score
|
||||
|
||||
- **WHEN** `analyzeMultivariate` is called with treatments where one factor has level rates `[0.1, 0.5, 0.9]` and another has `[0.5, 0.5, 0.5]`
|
||||
- **THEN** the first factor's `importance` strictly exceeds the second factor's `importance`
|
||||
|
||||
#### Scenario: bestCombination is the highest-rate treatment with participants
|
||||
|
||||
- **WHEN** treatments include one combination with `conversionRate: 0.9` but `participants: 0` and another with `0.5` and `participants: 100`
|
||||
- **THEN** `bestCombination.combinationId` is the latter
|
||||
|
||||
#### Scenario: empty factor set throws
|
||||
|
||||
- **WHEN** `analyzeMultivariate([], data)` is called
|
||||
- **THEN** `AbTestingError` with code `AB_CODES.INVALID_FACTOR_DESIGN` is thrown
|
||||
|
||||
**Verified by:** `src/multivariate/multivariate-engine.test.ts::"fullFactorial enumerates the Cartesian product of levels"` AND `src/multivariate/multivariate-engine.test.ts::"fractionalFactorial keeps every Nth combination"` AND `src/multivariate/multivariate-engine.spec-conformance.test.ts::"factor importance ranks higher-variance factors first"` AND `src/multivariate/multivariate-engine.test.ts::"bestCombination ignores treatments with zero participants"` AND `src/multivariate/multivariate-engine.spec-conformance.test.ts::"throws INVALID_FACTOR_DESIGN for empty factor set"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Segment analysis with per-segment lift and interaction terms
|
||||
|
||||
The package SHALL provide segment-level analyses. `analyzeBySegments(assignments, conversions, variantIds, segmentAttribute)` SHALL group assignments by `attributes[segmentAttribute]` value, compute per-segment `VariantResult[]`, attach a two-proportion z-test significance flag (z > 1.96 with both arms n ≥ 10), and report `conversionRateDiff` between best and worst variant. `segmentConfidenceIntervals(...)` SHALL return Wald intervals on conversion rates, `[0, 1]`-clamped. `detectSegmentInteraction(...)` SHALL return interaction-strength scores per segment value relative to global per-variant rates, flagged as `hasInteraction: true` when strength exceeds 0.01. When no assignment exposes the requested attribute, the call SHALL throw `AbTestingError(AB_CODES.INVALID_SEGMENT_ATTRIBUTE, ...)`.
|
||||
|
||||
#### Scenario: segments grouped by attribute value
|
||||
|
||||
- **WHEN** assignments include 50 with `attributes.country === 'US'` and 50 with `'CA'`, and `analyzeBySegments(..., 'country')` is called
|
||||
- **THEN** the result has two `SegmentResult` entries with `segmentValue: 'US'` and `'CA'` respectively
|
||||
|
||||
#### Scenario: significance gates on min sample size
|
||||
|
||||
- **WHEN** a segment has variant A with 5 assignments and variant B with 5 assignments
|
||||
- **THEN** `isSignificant` is `false` regardless of conversion rates
|
||||
|
||||
#### Scenario: confidence intervals are clamped to [0, 1]
|
||||
|
||||
- **WHEN** a tiny sample produces a Wald interval with lower bound `-0.05` or upper bound `1.10`
|
||||
- **THEN** the returned `lowerBound` is `0` and `upperBound` is `1`
|
||||
|
||||
#### Scenario: interaction flagged above 0.01 strength
|
||||
|
||||
- **WHEN** a segment value has variant rates that differ from global rates such that sum-of-squared-weighted-diff > 0.01
|
||||
- **THEN** `hasInteraction` is `true`
|
||||
|
||||
#### Scenario: missing attribute throws
|
||||
|
||||
- **WHEN** `analyzeBySegments(..., 'nonexistent')` is called and no assignment carries that attribute
|
||||
- **THEN** `AbTestingError` with code `AB_CODES.INVALID_SEGMENT_ATTRIBUTE` is thrown
|
||||
|
||||
**Verified by:** `src/segments/segment-analyzer.test.ts::"groups assignments by segment attribute value"` AND `src/segments/segment-analyzer.test.ts::"isSignificant is false when sample size is below minimum"` AND `src/segments/segment-analyzer.test.ts::"clamps confidence intervals to [0, 1]"` AND `src/segments/segment-analyzer.spec-conformance.test.ts::"flags interaction when strength exceeds threshold"` AND `src/segments/segment-analyzer.spec-conformance.test.ts::"throws INVALID_SEGMENT_ATTRIBUTE when attribute is absent"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Sequential testing with alpha-spending and early-stop rules
|
||||
|
||||
The package SHALL provide group-sequential test machinery for safe early stopping. `obrienFlemingBoundary(informationFraction, alpha)` SHALL return the z-boundary `z(1 - alpha/2) / sqrt(informationFraction)`. `pocockBoundary(informationFraction, alpha, maxLooks)` SHALL return a constant z across looks (Pocock spending). `alphaSpent(informationFraction, alpha, boundaryType?)` SHALL return cumulative alpha spent under the chosen spending function and SHALL be monotone-non-decreasing in `informationFraction`. `interimAnalysis(...)` SHALL compute a single look and report `shouldStop: true` with `stopReason: 'significance'` when `testStatistic >= spendingBoundary`, or `'futility'` when `informationFraction >= 0.75` and the projected final z falls below 30% of the final boundary. `runSequentialTest(...)` SHALL drive the sequence and stop on first `shouldStop`. Construction validates `alpha ∈ (0, 1)`, `maxLooks ≥ 1`, `mde > 0`, `power ∈ (0, 1)` else throws `AbTestingError(AB_CODES.INVALID_SEQUENTIAL_CONFIG, ...)`.
|
||||
|
||||
#### Scenario: O'Brien-Fleming boundary is monotone-decreasing
|
||||
|
||||
- **WHEN** `obrienFlemingBoundary(0.25, 0.05)` and `obrienFlemingBoundary(1, 0.05)` are computed
|
||||
- **THEN** the first value is strictly greater than the second
|
||||
|
||||
#### Scenario: Pocock boundary is constant across looks
|
||||
|
||||
- **WHEN** `pocockBoundary(0.5, 0.05, 4)` and `pocockBoundary(1, 0.05, 4)` are computed
|
||||
- **THEN** both values are equal within numerical tolerance
|
||||
|
||||
#### Scenario: alpha-spent is monotone-non-decreasing
|
||||
|
||||
- **WHEN** `alphaSpent` is called at fractions `[0.1, 0.25, 0.5, 0.75, 1.0]`
|
||||
- **THEN** the returned sequence is non-decreasing and ends at exactly `alpha`
|
||||
|
||||
#### Scenario: futility check fires only after 75% information
|
||||
|
||||
- **WHEN** `interimAnalysis` is called at `informationFraction = 0.5` with a tiny test statistic
|
||||
- **THEN** `stopReason` is `null` (futility branch suppressed)
|
||||
|
||||
#### Scenario: runSequentialTest stops on first shouldStop
|
||||
|
||||
- **WHEN** the data array has four looks and the second look returns `shouldStop: true`
|
||||
- **THEN** `runSequentialTest` returns exactly two `InterimResult` entries
|
||||
|
||||
**Verified by:** `src/sequential/sequential-test.test.ts::"obrienFlemingBoundary is monotone-decreasing in informationFraction"` AND `src/sequential/sequential-test.test.ts::"pocockBoundary is constant across looks"` AND `src/sequential/sequential-test.spec-conformance.test.ts::"alphaSpent is monotone-non-decreasing and ends at alpha"` AND `src/sequential/sequential-test.spec-conformance.test.ts::"futility branch is suppressed below 75% informationFraction"` AND `src/sequential/sequential-test.test.ts::"runSequentialTest stops on first shouldStop"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Bayesian stats — Beta-Binomial conversion model
|
||||
|
||||
The package SHALL provide Beta-Binomial Bayesian primitives. `posteriorBeta(conversions, participants, priorAlpha?, priorBeta?)` SHALL return the conjugate posterior `{ alpha: priorAlpha + conversions, beta: priorBeta + (participants - conversions) }`. `betaPdf` / `betaCdf` / `betaQuantile` SHALL be self-consistent (`betaCdf(betaQuantile(p, a, b), a, b) ≈ p` within bisection tolerance). `betaSample(alpha, beta, rng)` SHALL draw a Beta sample via two Gamma draws (Marsaglia-Tsang). `probabilityBGreaterA(a, b, simulations?, rng)` SHALL return a Monte Carlo estimate of `P(B > A)`. `credibleInterval(params, width?)` SHALL return a quantile-based interval at the requested width (default 95%). `expectedLoss(chosen, other, simulations?, rng)` SHALL return the expected forfeited lift from picking `chosen`. `bayesianAnalysis(results, priorAlpha?, priorBeta?, rng)` SHALL run the full pairwise comparison on two `VariantResult` objects. All RNG-dependent functions SHALL accept an injected `Rng` (no module-scope `Math.random`).
|
||||
|
||||
#### Scenario: posteriorBeta is the conjugate update
|
||||
|
||||
- **WHEN** `posteriorBeta(10, 100, 1, 1)` is called
|
||||
- **THEN** the result is `{ alpha: 11, beta: 91 }`
|
||||
|
||||
#### Scenario: betaCdf and betaQuantile are inverses
|
||||
|
||||
- **WHEN** `betaQuantile(0.5, 5, 5)` then `betaCdf(<that>, 5, 5)` is computed
|
||||
- **THEN** the round-trip value is within `1e-3` of `0.5`
|
||||
|
||||
#### Scenario: probabilityBGreaterA converges to 0.5 for symmetric posteriors
|
||||
|
||||
- **WHEN** `probabilityBGreaterA({ alpha: 5, beta: 5 }, { alpha: 5, beta: 5 }, 20000, rng)` is called
|
||||
- **THEN** the result is within `0.02` of `0.5`
|
||||
|
||||
#### Scenario: credibleInterval respects width parameter
|
||||
|
||||
- **WHEN** `credibleInterval(params, 0.5)` and `credibleInterval(params, 0.95)` are computed for the same `params`
|
||||
- **THEN** the 95% interval contains the 50% interval
|
||||
|
||||
#### Scenario: same Rng seed produces deterministic samples
|
||||
|
||||
- **WHEN** `betaSample(2, 3, rng)` is called twice with the same `Rng` state
|
||||
- **THEN** both calls return the same numeric value (reproducibility pin)
|
||||
|
||||
**Verified by:** `src/stats/bayesian.test.ts::"posteriorBeta returns the conjugate update"` AND `src/stats/bayesian.test.ts::"betaCdf and betaQuantile are mutual inverses"` AND `src/stats/bayesian.test.ts::"probabilityBGreaterA converges to 0.5 for symmetric posteriors"` AND `src/stats/bayesian.test.ts::"credibleInterval width parameter widens the interval"` AND `src/stats/bayesian.spec-conformance.test.ts::"injected Rng produces reproducible betaSample output"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Assignment store — interface and in-memory adapter
|
||||
|
||||
The package SHALL persist variant assignments via an `AssignmentStoreLike` interface (defined in `@suaveplan/types`). The interface SHALL expose five async methods: `get(experimentId, userId): Promise<Assignment | undefined>`, `set(assignment): Promise<void>`, `getByExperiment(experimentId): Promise<readonly Assignment[]>`, `getByUser(userId): Promise<readonly Assignment[]>`, `deleteByExperiment(experimentId): Promise<void>`. The package SHALL ship a `MemoryAssignmentStore` class implementing this interface and a `createMemoryAssignmentStore()` factory. `ExperimentManagerOptions` SHALL accept an optional `store?: AssignmentStoreLike`; when provided, `assignVariant` SHALL consult the store before re-bucketing and write through on miss. `MemoryAssignmentStore.set` SHALL be idempotent on identical-key assignments (last-writer-wins).
|
||||
|
||||
#### Scenario: set/get round-trip
|
||||
|
||||
- **WHEN** `store.set({ experimentId: 'e1', userId: 'u1', variantId: 'v1', assignedAt: '2026-05-02T00:00:00Z' })` is called and then `store.get('e1', 'u1')` is awaited
|
||||
- **THEN** the resolved value is the same `Assignment` object shape
|
||||
|
||||
#### Scenario: getByExperiment filters to one experiment
|
||||
|
||||
- **WHEN** the store contains assignments for `e1` and `e2` and `getByExperiment('e1')` is awaited
|
||||
- **THEN** the result contains only `e1` assignments
|
||||
|
||||
#### Scenario: deleteByExperiment cascades
|
||||
|
||||
- **WHEN** the store contains 100 assignments for `e1` and `deleteByExperiment('e1')` is awaited
|
||||
- **THEN** subsequent `getByExperiment('e1')` returns an empty array
|
||||
|
||||
#### Scenario: ExperimentManager with store does not re-bucket on hit
|
||||
|
||||
- **WHEN** `createExperimentManager({ store })` is called, `assignVariant(exp, ctx)` runs once and persists, then `assignVariant(exp, ctx)` runs again with the same `ctx`
|
||||
- **THEN** the second call returns the persisted variant without invoking `bucketUser` (verified by an `Rng` spy that increments only on miss)
|
||||
|
||||
#### Scenario: missing key resolves undefined
|
||||
|
||||
- **WHEN** `store.get('does-not-exist', 'u1')` is awaited
|
||||
- **THEN** the resolved value is `undefined` (no error thrown)
|
||||
|
||||
**Verified by:** `src/store/memory-assignment-store.test.ts::"set and get round-trip an Assignment"` AND `src/store/memory-assignment-store.test.ts::"getByExperiment returns only matching experiment ids"` AND `src/store/memory-assignment-store.test.ts::"deleteByExperiment removes all assignments for that experiment"` AND `src/core/experiment-manager/experiment-manager.store.test.ts::"second assignVariant call with store option returns persisted variant without re-bucketing"` AND `src/store/memory-assignment-store.spec-conformance.test.ts::"get resolves undefined for missing keys"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
175
openspec/changes/ab-testing-completion/tasks.md
Normal file
175
openspec/changes/ab-testing-completion/tasks.md
Normal file
@@ -0,0 +1,175 @@
|
||||
# Tasks — A/B testing completion
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick boxes for work that was already in main when this change opened. Do not tick a box for a test that doesn't exist or doesn't pin the requirement it claims to verify. Retroactive ticking is the failure mode this template guards against.
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `../suaveplan/packages/universal/ab-testing/src/` end-to-end (bandit/, multivariate/, segmentation/, sequential/, stats/, store/, types/, engine/)
|
||||
- [ ] 1.2 Read existing `openspec/specs/ab-testing/spec.md` (none exists yet — will be created at archive)
|
||||
- [ ] 1.3 Read `openspec/AGENTS.md` Sections 13, 14, and §0a (migration parity)
|
||||
- [ ] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
|
||||
- [ ] 1.5 Fill in `## Source mapping` table — every requirement cites a line range OR appears in the "no source line" table with justification
|
||||
- [ ] 1.6 Draft `specs/ab-testing/spec.md` from `openspec/templates/spec-delta.md`
|
||||
- [ ] 1.7 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles
|
||||
- [ ] 1.8 `## Parity Checklist` table fully populated; every NO has architectural justification ≤1 sentence
|
||||
|
||||
## 2. Implementation
|
||||
|
||||
### 2a. Promote `AssignmentStoreLike` to `@suaveplan/types`
|
||||
|
||||
- [ ] 2.1 Add `AssignmentStoreLike` interface to `packages/types/types/src/storage/assignment-store.ts` with five async methods (`get`, `set`, `getByExperiment`, `getByUser`, `deleteByExperiment`)
|
||||
- [ ] 2.2 Re-export `AssignmentStoreLike` from `@suaveplan/types` root barrel
|
||||
- [ ] 2.3 Add `Assignment` interface alongside (or import-only re-export from `@suaveplan/ab-testing` once defined there) — pin the cross-package contract
|
||||
|
||||
### 2b. Bandit subsystem
|
||||
|
||||
- [ ] 2.4 Add `src/bandit/bandit.types.ts` with `Bandit<TArmId>` interface and `BanditConfig<TArmId>`
|
||||
- [ ] 2.5 Add `src/bandit/epsilon-greedy.ts` — `EpsilonGreedyBandit<TArmId>` class; constructor accepts `{ arms, epsilon, rng }`; default `epsilon = 0.1`; require explicit `Rng` (no `Math.random` fallback at construction)
|
||||
- [ ] 2.6 Add `src/bandit/ucb1.ts` — `UCB1Bandit<TArmId>` class
|
||||
- [ ] 2.7 Add `src/bandit/thompson.ts` — `ThompsonSamplingBandit<TArmId>` class; depends on `betaSample` from `./stats/bayesian`
|
||||
- [ ] 2.8 Add `src/bandit/index.ts` barrel
|
||||
- [ ] 2.9 Throw `AbTestingError(AB_CODES.INVALID_BANDIT_CONFIG, ...)` when `arms.length < 2` or `epsilon ∉ [0, 1]`
|
||||
|
||||
### 2c. Multivariate engine
|
||||
|
||||
- [ ] 2.10 Add `src/multivariate/multivariate.types.ts` with `Factor`, `TreatmentCombination`, `TreatmentResult`, `FactorEffect`, `LevelEffect`, `InteractionEffect`, `MultivariateResult`
|
||||
- [ ] 2.11 Add `src/multivariate/factorial.ts` — `fullFactorial`, `fractionalFactorial`
|
||||
- [ ] 2.12 Add `src/multivariate/multivariate-engine.ts` — `analyzeMultivariate` + factor-effect / interaction-effect helpers; split helpers into `src/multivariate/factor-effects.ts` and `src/multivariate/interactions.ts` if file > 250 L
|
||||
- [ ] 2.13 Add `src/multivariate/index.ts` barrel
|
||||
- [ ] 2.14 Throw `AbTestingError(AB_CODES.INVALID_FACTOR_DESIGN, ...)` when `factors.length === 0` or any factor has `levels.length === 0`
|
||||
|
||||
### 2d. Segment analysis
|
||||
|
||||
- [ ] 2.15 Add `src/segments/segments.types.ts` with `SegmentResult`, `SegmentConfidenceInterval`, `SegmentInteraction`
|
||||
- [ ] 2.16 Add `src/segments/segment-analyzer.ts` — `analyzeBySegments`, `segmentConfidenceIntervals`, `detectSegmentInteraction`
|
||||
- [ ] 2.17 Move `normalQuantile` helper to `src/stats/normal.ts` (shared by segments and sequential)
|
||||
- [ ] 2.18 Add `src/segments/index.ts` barrel
|
||||
- [ ] 2.19 Throw `AbTestingError(AB_CODES.INVALID_SEGMENT_ATTRIBUTE, ...)` when `assignments.every(a => a.attributes?.[segmentAttribute] === undefined)`
|
||||
|
||||
### 2e. Sequential testing
|
||||
|
||||
- [ ] 2.20 Add `src/sequential/sequential.types.ts` with `SequentialTestConfig` (extended with `boundaryType`), `InterimResult`
|
||||
- [ ] 2.21 Add `src/sequential/boundaries.ts` — `obrienFlemingBoundary`, `pocockBoundary`, dispatch on `boundaryType`
|
||||
- [ ] 2.22 Add `src/sequential/sequential-test.ts` — `alphaSpent`, `interimAnalysis`, `runSequentialTest`, `checkFutility`
|
||||
- [ ] 2.23 Add `src/sequential/index.ts` barrel
|
||||
- [ ] 2.24 Throw `AbTestingError(AB_CODES.INVALID_SEQUENTIAL_CONFIG, ...)` when `alpha ∉ (0, 1)`, `maxLooks < 1`, `mde ≤ 0`, or `power ∉ (0, 1)`
|
||||
|
||||
### 2f. Bayesian stats
|
||||
|
||||
- [ ] 2.25 Add `src/stats/beta.ts` — `BetaParams`, `betaPdf`, `betaCdf`, `betaQuantile`, `betaSample`, `posteriorBeta`, internal `logBeta` / `logGamma` / `continuedFraction` / `gammaSample` / `normalSample`
|
||||
- [ ] 2.26 Add `src/stats/bayesian.ts` — `BayesianResult`, `thompsonSample`, `probabilityBGreaterA`, `credibleInterval`, `expectedLoss`, `bayesianAnalysis`
|
||||
- [ ] 2.27 Add `src/stats/normal.ts` — `normalCdf`, `normalQuantile` (Beasley-Springer-Moro) shared with sequential and segments
|
||||
- [ ] 2.28 Add `src/stats/index.ts` barrel
|
||||
- [ ] 2.29 Replace every `rng: () => number = Math.random` default with required `rng: Rng` param; expose `createDefaultRng()` from `@suaveplan/ab-testing/stats` for callers wanting a `crypto.getRandomValues`-backed default
|
||||
|
||||
### 2g. Assignment store
|
||||
|
||||
- [ ] 2.30 Add `src/store/memory.ts` — `MemoryAssignmentStore` class implementing `AssignmentStoreLike` (imported from `@suaveplan/types`)
|
||||
- [ ] 2.31 Add `createMemoryAssignmentStore()` factory
|
||||
- [ ] 2.32 Add `src/store/index.ts` barrel
|
||||
- [ ] 2.33 Wire `store?: AssignmentStoreLike` field through `ExperimentManagerOptions` in `src/types/common.ts`; update `createExperimentManager` to consult and persist via the store when provided
|
||||
|
||||
### 2h. Errors and exports
|
||||
|
||||
- [ ] 2.34 Add `INVALID_BANDIT_CONFIG`, `INVALID_FACTOR_DESIGN`, `INVALID_SEGMENT_ATTRIBUTE`, `INSUFFICIENT_SAMPLE_SIZE`, `INVALID_SEQUENTIAL_CONFIG` to `AB_CODES` in `src/error-codes.ts`
|
||||
- [ ] 2.35 Update `src/index.ts` root barrel to re-export from `./bandit`, `./multivariate`, `./segments`, `./sequential`, `./stats`, `./store`
|
||||
- [ ] 2.36 Add subpath entries to `package.json` `exports` map for the six new subpaths (matches existing `./bucketing` / `./targeting` / `./results` / `./providers` / `./errors` shape)
|
||||
- [ ] 2.37 Update `package.json` `description` field if needed (currently mentions only deterministic bucketing + targeting + variants + conversions; extend to mention bandits / multivariate / sequential / Bayesian)
|
||||
|
||||
## 3. Tests
|
||||
|
||||
### 3a. Bandit subsystem
|
||||
|
||||
- [ ] 3.1 `src/bandit/bandit.test.ts` — `EpsilonGreedyBandit` exploration (when rng < epsilon) vs. exploitation (else); `UCB1Bandit` plays each arm once before applying score; `ThompsonSamplingBandit` posterior update on success vs. failure
|
||||
- [ ] 3.2 `src/bandit/bandit.spec-conformance.test.ts` — injected `Rng` is the only randomness source; `INVALID_BANDIT_CONFIG` thrown for invalid configs
|
||||
|
||||
### 3b. Multivariate engine
|
||||
|
||||
- [ ] 3.3 `src/multivariate/multivariate-engine.test.ts` — `fullFactorial` count = `Π levels`; `fractionalFactorial(fraction = 0.5)` returns half; factor importance sum-of-squares ranking
|
||||
- [ ] 3.4 `src/multivariate/multivariate-engine.spec-conformance.test.ts` — interaction strength is non-negative; pairwise interactions cover every `(i, j)` with `i < j`; `INVALID_FACTOR_DESIGN` thrown for empty factors
|
||||
|
||||
### 3c. Segment analysis
|
||||
|
||||
- [ ] 3.5 `src/segments/segment-analyzer.test.ts` — segment grouping by attribute; significance with min sample size 10; CI is `[0, 1]`-clamped
|
||||
- [ ] 3.6 `src/segments/segment-analyzer.spec-conformance.test.ts` — interaction-strength threshold; `INVALID_SEGMENT_ATTRIBUTE` thrown when attribute is absent
|
||||
|
||||
### 3d. Sequential testing
|
||||
|
||||
- [ ] 3.7 `src/sequential/sequential-test.test.ts` — O'Brien-Fleming boundary monotone-decreasing in `informationFraction`; Pocock boundary constant across looks; `runSequentialTest` stops at first `shouldStop`
|
||||
- [ ] 3.8 `src/sequential/sequential-test.spec-conformance.test.ts` — alpha-spent monotone-increasing; futility only fires when `informationFraction ≥ 0.75`; `INVALID_SEQUENTIAL_CONFIG` thrown for invalid configs
|
||||
|
||||
### 3e. Bayesian stats
|
||||
|
||||
- [ ] 3.9 `src/stats/bayesian.test.ts` — `betaPdf` / `betaCdf` / `betaQuantile` round-trip; `posteriorBeta` conjugate update (priors + data); `probabilityBGreaterA` converges to closed-form `0.5` for symmetric posteriors
|
||||
- [ ] 3.10 `src/stats/bayesian.spec-conformance.test.ts` — same `Rng` seed produces identical `betaSample` / `thompsonSample` / `expectedLoss` outputs across runs (reproducibility pin)
|
||||
|
||||
### 3f. Assignment store
|
||||
|
||||
- [ ] 3.11 `src/store/memory-assignment-store.test.ts` — `set`/`get` round-trip; `getByExperiment` / `getByUser` filtering; `deleteByExperiment` cascade; `size` and `clear`
|
||||
- [ ] 3.12 `src/store/memory-assignment-store.spec-conformance.test.ts` — `AssignmentStoreLike` contract: every method returns a `Promise`; `get` resolves `undefined` for missing keys; `set` is idempotent on identical assignments
|
||||
|
||||
### 3g. ExperimentManager integration with store
|
||||
|
||||
- [ ] 3.13 `src/core/experiment-manager/experiment-manager.store.test.ts` — when `options.store` is provided, `assignVariant` checks store first and writes through on miss; second call for same `(experimentId, userId)` returns the persisted variant without re-bucketing
|
||||
|
||||
### 3h. Cross-package
|
||||
|
||||
- [ ] 3.14 `tests/ab-testing-public-surface.test.ts` — assert root barrel re-exports include `EpsilonGreedyBandit`, `UCB1Bandit`, `ThompsonSamplingBandit`, `fullFactorial`, `fractionalFactorial`, `analyzeMultivariate`, `analyzeBySegments`, `segmentConfidenceIntervals`, `detectSegmentInteraction`, `obrienFlemingBoundary`, `pocockBoundary`, `interimAnalysis`, `runSequentialTest`, `posteriorBeta`, `bayesianAnalysis`, `MemoryAssignmentStore`, `createMemoryAssignmentStore`
|
||||
- [ ] 3.15 Every `it(...)` carries `{ timeout: 5000 }` (Bayesian Monte Carlo tests use 30000)
|
||||
|
||||
## 4. Documentation
|
||||
|
||||
- [ ] 4.1 Sibling `.md` for every new implementation `.ts` (≥200 words, basic + advanced examples, full API reference): one per file under `bandit/`, `multivariate/`, `segments/`, `sequential/`, `stats/`, `store/`
|
||||
- [ ] 4.2 README links every new co-located `.md` under `## Modules`
|
||||
- [ ] 4.3 Update root `README.md` quick-start with three new sections: bandit selection, multivariate analysis, sequential testing
|
||||
- [ ] 4.4 Update `CLAUDE.md` `@suaveplan/ab-testing` package listing to include bandits, multivariate, segment analysis, sequential testing, Bayesian stats, assignment store
|
||||
- [ ] 4.5 Update `@suaveplan/types` listing in `CLAUDE.md` to mention `AssignmentStoreLike` if the listing is detailed enough
|
||||
|
||||
## 5. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [ ] 5.1 `bun run check` — Biome 0 errors / 0 warnings / 0 info across the whole monorepo
|
||||
- [ ] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/ab-testing` — green
|
||||
- [ ] 5.3 `bunx turbo test:coverage --filter=@suaveplan/ab-testing` — 100/100/100/100
|
||||
- [ ] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/ab-testing` — exit 0, no `error` findings
|
||||
- [ ] 5.5 `bunx turbo build --filter=@suaveplan/ab-testing` — green
|
||||
- [ ] 5.6 `bunx turbo lint typecheck test --filter=@suaveplan/types` — green (foundation-package side-effect from `AssignmentStoreLike` promotion)
|
||||
|
||||
## 6. Pre-archive 7-row checklist (AGENTS.md Section 13)
|
||||
|
||||
- [ ] 6.1 Generics threaded — `Bandit<TArmId>`, `EpsilonGreedyBandit<TArmId>`, `UCB1Bandit<TArmId>`, `ThompsonSamplingBandit<TArmId>` infer arm-id type from constructor `arms` array; `expectTypeOf` assertion that `bandit.selectArm()` returns `TArmId`, not `number`
|
||||
- [ ] 6.2 Zod schemas wired at every declared boundary — `BanditConfigSchema`, `FactorSchema`, `SequentialTestConfigSchema` validate at construction
|
||||
- [ ] 6.3 Every error code in `error-codes.ts` thrown by at least one production path (the five new codes each tested in spec-conformance suites)
|
||||
- [ ] 6.4 Strict-mode behaviors tested both on and off — `Bandit` with `strict: true` throws on unknown arm IDs; `Bandit` with `strict: false` ignores
|
||||
- [ ] 6.5 OTel instruments emitted from production paths and asserted with a real in-memory `MeterProvider` — `ab.bandit.arm.selected`, `ab.multivariate.combinations.evaluated`, `ab.sequential.alpha.spent`, `ab.bayesian.simulations.duration_ms`, `ab.assignment.store.lookup`
|
||||
- [ ] 6.6 Lifecycle dedup tested where claimed — `MemoryAssignmentStore.set` is idempotent; `ExperimentManager` with `store` option does not re-bucket after a hit
|
||||
- [ ] 6.7 No retroactive tick — every box above reflects work done in this change
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold spec delta into `openspec/specs/ab-testing/spec.md` (file does not exist yet — this change creates it)
|
||||
- [ ] 7.2 Move `openspec/changes/ab-testing-completion/` → `openspec/archive/<yyyy-mm-dd>-ab-testing-completion/`
|
||||
- [ ] 7.3 Refresh `MANIFEST.md` if package set or deps changed (no new deps; `@suaveplan/types` gains `AssignmentStoreLike` interface)
|
||||
|
||||
## Audit Note (2026-05-02)
|
||||
|
||||
Audit verdict: **DEFERRED — 0/81 tasks evidenced in code**.
|
||||
|
||||
Evidence reviewed:
|
||||
|
||||
- `packages/testing/ab-testing/src/` contains `bucketing/`, `targeting/`, `providers/`, `results/`, `core/experiment-manager/`, `error-codes.ts`, `types/`, `constants/`. No `bandit/`, no `multivariate/`, no `segments/`, no `sequential/`, no `stats/`, no `store/`.
|
||||
- `error-codes.ts` does not declare the five new codes (`INVALID_BANDIT_CONFIG`, `INVALID_FACTOR_DESIGN`, `INVALID_SEGMENT_ATTRIBUTE`, `INSUFFICIENT_SAMPLE_SIZE`, `INVALID_SEQUENTIAL_CONFIG`).
|
||||
- `@suaveplan/types` has no `AssignmentStoreLike` interface yet (the §2a promotion is not done).
|
||||
|
||||
Missing pieces:
|
||||
|
||||
1. Bandit subsystem (`epsilon-greedy`, `ucb1`, `thompson` with explicit `Rng` injection) and Bayesian stats module (`stats/{beta,bayesian,normal}.ts`).
|
||||
2. Multivariate engine (`fullFactorial` / `fractionalFactorial` / `analyzeMultivariate` + factor-effect / interaction helpers).
|
||||
3. Segment analyzer + sequential testing (O'Brien-Fleming + Pocock boundaries) + `MemoryAssignmentStore` plus `AssignmentStoreLike` promotion to `@suaveplan/types` and wiring through `ExperimentManagerOptions`.
|
||||
|
||||
Decision: leave change open. Six new feature blocks across statistics + storage + cross-package type-promotion is a multi-day implementation, not an audit pass.
|
||||
158
openspec/changes/add-api-plugin-banking/proposal.md
Normal file
158
openspec/changes/add-api-plugin-banking/proposal.md
Normal file
@@ -0,0 +1,158 @@
|
||||
# Add `@suaveplan/api-plugin-banking`
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-banking`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package
|
||||
**Spec delta**: ./specs/api-plugin-banking/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
The genesis api-plugin chain needs a banking-foundation plugin that
|
||||
owns the canonical `bank` (institution) and `bank_account` SurrealDB
|
||||
tables, exposes a coordinator service for the typed read / write
|
||||
surface, and ships the HTTP route tree consumers depend on. The
|
||||
downstream `@suaveplan/api-plugin-bank` (game-wallet specialisation)
|
||||
is being migrated by a peer agent in parallel and depends on this
|
||||
package — ship as fast as quality allows.
|
||||
|
||||
The upstream package at
|
||||
`/mnt/s/web/suaveplan/packages/server/api-plugin-banking/` shipped
|
||||
working code but failed the genesis quality bar in three ways that
|
||||
the audit document `enterprise/api-plugin-banking.md` calls out:
|
||||
|
||||
- `src/index.ts:111` instantiated the route tree against a fake
|
||||
`{} as PluginContext` cast at module-load time so routes never
|
||||
actually received the live context.
|
||||
- The `BankingService` and repositories carried `as any` casts on the
|
||||
database adapter because the upstream `@suaveplan/api`
|
||||
`DatabaseAdapter` shape was untyped.
|
||||
- Routes were defined with the upstream `GET/POST/successResponse`
|
||||
builder which the genesis `@suaveplan/api` does not export, so a
|
||||
direct port is impossible — the migration retypes against the
|
||||
canonical `@hono/zod-openapi` `createRoute` + `OpenAPIHono` surface
|
||||
used by every other genesis api-plugin (search, audit, webhook,
|
||||
...).
|
||||
|
||||
The migration brings the full functional surface across, retypes
|
||||
against the genesis `DatabaseAdapter` from `@suaveplan/db`, adds the
|
||||
fourteen `BANKING.*` error codes, six OTel instruments, a typed event
|
||||
map, a Zod options boundary, and 100% istanbul coverage.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `createBankingPlugin(options?)` — Zod-validated factory
|
||||
returning a typed `PluginDescriptor<BankingPluginConfig>` with
|
||||
`name: "banking"`, `dependencies: ["database"]`,
|
||||
`optionalDependencies: ["auth", "audit", "rate-limit"]`,
|
||||
three lifecycle hooks (`onInit`/`onReady`/`onClose`), and the
|
||||
configured `routePrefix` mount.
|
||||
- ADDED: `BankingPluginOptionsSchema` /
|
||||
`BankingPluginOptionsInput` / `BankingPluginOptionsResolved` /
|
||||
`BankingPluginConfig` — seven defaulted options
|
||||
(`enableDefaultBank`, `defaultBankName`, `defaultSortCode`,
|
||||
`maxAccountsPerUser`, `enableTransfers`, `enableOverdraft`,
|
||||
`routePrefix`).
|
||||
- ADDED: `BankingService` coordinator with typed `createBank`,
|
||||
`updateBank`, `softDeleteBank`, `createAccount`, `updateAccount`,
|
||||
`updateAccountStatus`, `transferFunds`, `addFunds`,
|
||||
`subtractFunds`, `initializeDefaultBank`, `deleteTenantData`, and
|
||||
repository accessors.
|
||||
- ADDED: `BaseRepository`, `BankRepository`, `BankAccountRepository`
|
||||
— SurrealDB-backed CRUD with field-whitelist enforcement,
|
||||
RecordId flattening, and tenant-cleanup helpers.
|
||||
- ADDED: 11 Zod schemas (`BankSchema`, `BankAccountSchema`,
|
||||
`CreateBank{,Account}Schema`, `UpdateBank{,Account}Schema`,
|
||||
`TransferFundsSchema`, `FundsOperationSchema`,
|
||||
`UpdateAccountStatusSchema`, `SortCodeSchema`,
|
||||
literal-union status / type enums) plus matching
|
||||
`z.infer<>` aliases.
|
||||
- ADDED: 14 `BANKING.*` error codes via `defineCodes("BANKING", ...)`
|
||||
— every code thrown by ≥1 production code path.
|
||||
- ADDED: 6 OTel instruments under the
|
||||
`@suaveplan/api-plugin-banking` meter
|
||||
(`banks_total`, `accounts_total`, `transfers_total`,
|
||||
`transfer_amount` histogram, `fund_operations_total`,
|
||||
`query_duration_ms` histogram).
|
||||
- ADDED: `createBankingRoutes(deps)` — 19 OpenAPIHono routes
|
||||
(`/banks/*` + `/accounts/*`) including `POST /accounts/transfer`,
|
||||
`POST /accounts/:id/{add,subtract}-funds`,
|
||||
`PATCH /accounts/:id/status`, `GET /banks/sort-code/:sortCode`,
|
||||
`GET /banks/search/:query`, `GET /accounts/by-number/:n`,
|
||||
`GET /accounts/user/:userId`, `GET /accounts/:id/owner`.
|
||||
- ADDED: Typed `BankingEventMap` and event emission on every state
|
||||
transition (created / updated / soft-deleted / status-changed /
|
||||
transfer-succeeded / transfer-failed).
|
||||
- ADDED: `bankingService`, `app`, `db` slots on the canonical
|
||||
`ServiceRegistry` via module augmentation.
|
||||
- ADDED: `decommissionBankingTenant(service, tenantId)` helper for
|
||||
hosts orchestrating tenant lifecycle externally.
|
||||
- ADDED: `surql/001-bank-and-account.surql` migration file carrying
|
||||
the upstream schema, sort-code function library, account-number
|
||||
arithmetic functions, and the `user_account` relation.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Auth / rate-limit / audit-log integration — the upstream plugin
|
||||
did not gate routes, and the enterprise audit document flags these
|
||||
as future-work additions. The optional-dependencies list signals
|
||||
the topo-sort order so a follow-up change can wire them without
|
||||
re-ordering.
|
||||
- Multi-currency accounts via `@suaveplan/money` /
|
||||
`@suaveplan/currency` — flagged as feature-rich enhancement in
|
||||
`enterprise/api-plugin-banking.md §4` but not present in the
|
||||
upstream source. Deferred to a dedicated change.
|
||||
- Double-entry ledger model — the upstream stores raw balance; the
|
||||
enhancement flag is preserved for a follow-up.
|
||||
- Scheduled / standing transfers, reconciliation endpoint,
|
||||
statement-PDF generation — listed in §4 enhancements only.
|
||||
- Decommissioned tenant hook on the `PluginDescriptor` itself — the
|
||||
current `@suaveplan/api-plugin` `PluginDescriptor` shape does not
|
||||
declare `onTenantDelete`. The migration exposes a
|
||||
`decommissionBankingTenant` helper that hosts invoke from their
|
||||
own tenant orchestrator.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change derives from
|
||||
`../suaveplan/enterprise/api-plugin-banking.md` and the upstream
|
||||
source at
|
||||
`../suaveplan/packages/server/api-plugin-banking/src/`.
|
||||
|
||||
| Requirement | Source lines | Notes |
|
||||
|---|---|---|
|
||||
| Plugin factory with `name: "banking"`, `version`, `description`, `dependencies: ["database"]` | `src/index.ts` L86-L96 (`createBankingPlugin`) | retyped against genesis `PluginDescriptor` |
|
||||
| `BankingConfigSchema` (`enableDefaultBank`, `defaultBankName`, `defaultSortCode`, `maxAccountsPerUser`, `enableTransfers`, `enableOverdraft`) | `src/index.ts` L19-L28 | renamed to `BankingPluginOptionsSchema`; added `routePrefix` (genesis convention) |
|
||||
| `BankingService` coordinator | `src/index.ts` L34-L64 | moved into `src/service/service.ts` per `enterprise §3` "move out of `src/index.ts`" |
|
||||
| `initializeDefaultBank` idempotent bootstrap | `src/index.ts` L54-L58 | preserved + made config-driven via name + sort code |
|
||||
| `validateUserAccountLimit` → `countByUserId` | `src/index.ts` L60-L63 | renamed + moved into repository; service enforces via `ACCOUNT_LIMIT_REACHED` |
|
||||
| `onInit` registers `bankingService`, `bankRepository`, `bankAccountRepository` | `src/index.ts` L114-L135 | trimmed to single `bankingService` slot; repositories are accessible via `service.getBankRepository()` |
|
||||
| `onReady` bootstraps default bank | `src/index.ts` L137-L143 | preserved; emits `banking:ready` typed event |
|
||||
| `onClose` drops service | `src/index.ts` L145-L147 | preserved |
|
||||
| `onTenantDelete` purges banks + accounts | `src/index.ts` L154-L168 | preserved as `service.deleteTenantData(tenantId)` + `decommissionBankingTenant` helper (descriptor hook not yet exposed in genesis `PluginDescriptor`) |
|
||||
| Re-export of `@suaveplan/api-schemas/banking` | `src/index.ts` L178 | REMOVED per `enterprise §3` ("violates NEVER re-export external deps") and CLAUDE.md rule 4; banking schemas now owned locally |
|
||||
| `BankRepository` full surface (create, existsByName, getByName, getBySiteId, getBySortCode, getByStatus, getActiveBanks, getAvailableBanks, softDelete, hasActiveAccounts, searchByName, getOrCreateDefaultBank) | `src/repositories/bank.repository/index.ts` L23-L209 | preserved with field-whitelist tightened; default-bank takes config args |
|
||||
| `BankAccountRepository` full surface | `src/repositories/bank-account.repository/index.ts` L47-L364 | preserved; fund-movement primitives retyped against `DatabaseAdapter`; overdraft tolerance threaded through |
|
||||
| `BaseRepository` shared scaffold + field whitelist | `src/repositories/base.repository.ts` L40-L266 | retyped against genesis `@suaveplan/db` `DatabaseAdapter`; parameter style switched to `$p0,$p1` positional via `db.query(sql, [...])` |
|
||||
| `RecordWithId`, `RecordId`, `RepositoryResponse`, `ListResponse` | `src/repositories/base.repository.types.ts` L1-L52 | folded into `src/types/types.ts` and `src/schemas/schemas.ts`; `ListResponse` dropped as unused on the genesis surface |
|
||||
| Bank routes (`GET /`, `POST /`, `GET /:id`, `PATCH /:id`, `DELETE /:id`, `GET /sort-code/:sortCode`, `GET /search/:query`) | `src/routes/banks/index.ts` L25-L336 | retyped against `@hono/zod-openapi` `createRoute`; every status declared in `responses` |
|
||||
| Account routes (list, get, by-user, by-number, create, update, delete, transfer, add-funds, subtract-funds, status, owner) | `src/routes/accounts/index.ts` L29-L537 | preserved verbatim semantics; routes consume `BankingService` instead of constructing repos per-request |
|
||||
| `banking:initialized`, `banking:ready` event extension | `src/types.ts` L20-L25 | extended to full `BankingEventMap` with per-state-transition payloads |
|
||||
| `surql/tables/bank-account.surql` (bank + bank_account schemas, function library, account-number arithmetic) | upstream `surql/tables/bank-account.surql` L1-L345 | copied verbatim to `surql/001-bank-and-account.surql` |
|
||||
| Fix `{} as PluginContext` cast at route mount | `enterprise §3` first bullet | routes now mount inside `onReady(ctx)` with real `ctx` |
|
||||
| Remove `as any` casts on `db` | `enterprise §3` second bullet | replaced with typed `DatabaseAdapter` from `@suaveplan/db` |
|
||||
| Move `BankingService` out of `src/index.ts` | `enterprise §3` third bullet | service lives in `src/service/service.ts` |
|
||||
| Drop re-export of `@suaveplan/api-schemas/banking` | `enterprise §3` fourth bullet | schemas owned locally per CLAUDE.md rule 4 |
|
||||
| Add `events.ts` for `EventMap` declaration | `enterprise §3` fifth bullet | replaced by `src/types/types.ts` `BankingEventMap` + typed `emitEvent` in routes |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| 100% istanbul test coverage gate | Genesis-wide policy, not package-specific |
|
||||
| `defineCodes("BANKING", [...])` error registry of 14 codes | Genesis-wide policy — upstream threw raw `new Error()` |
|
||||
| 6 OTel instruments emitted from production paths | Genesis-wide policy — upstream had no observability |
|
||||
| Module-augmentation of `ServiceRegistry` | Genesis convention — upstream had untyped `getService("db")` |
|
||||
| Co-located sibling `.md` files (≥200 words) | Genesis CLAUDE.md doc-coverage rule |
|
||||
| Test files import only from `@suaveplan/testing/runner` | CLAUDE.md rule 11 |
|
||||
| `enableOverdraft` + `enableTransfers` + `routePrefix` | `routePrefix` is genesis convention; `enable*` flags were upstream but `routePrefix` was hardcoded `/banking` upstream — moved to schema for multi-version deploys |
|
||||
| Account-number arithmetic (`parseAccountNumber`, `splitAccountNumber`) raises `INVALID_ACCOUNT_NUMBER` on malformed input | Upstream returned `null` silently; genesis throws so callers cannot ignore malformed input |
|
||||
@@ -0,0 +1,267 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: api-plugin-banking (add-api-plugin-banking)
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-banking`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Plugin factory shape
|
||||
|
||||
The package SHALL export `createBankingPlugin(options?: BankingPluginOptionsInput)` returning a `PluginDescriptor<BankingPluginConfig>` whose `name` is `"banking"`, whose `dependencies` is `["database"]`, whose `optionalDependencies` includes `"auth"`, `"audit"`, and `"rate-limit"`, and whose `tags` include `"banking"`, `"accounts"`, and `"finance"`.
|
||||
|
||||
#### Scenario: Descriptor metadata
|
||||
|
||||
- **WHEN** `createBankingPlugin()` is called with no arguments
|
||||
- **THEN** the returned descriptor has the exact name, version, dependencies, optional-dependencies, and tags listed above
|
||||
|
||||
#### Scenario: Invalid options reject at the boundary
|
||||
|
||||
- **WHEN** `createBankingPlugin({ maxAccountsPerUser: 0 })` is called
|
||||
- **THEN** the factory throws `BankingError` with code `BANKING.INVALID_OPTIONS` and the original `ZodError` as `cause`
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"returns a descriptor with the canonical banking-plugin metadata"` AND `src/plugin/plugin.test.ts::"throws BANKING.INVALID_OPTIONS when options fail schema validation"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Lifecycle (onInit / onReady / onClose)
|
||||
|
||||
The descriptor SHALL implement `onInit`, `onReady`, and `onClose`. `onInit` resolves `ctx.getService("db")`, constructs a `BankingService`, and registers it under the `bankingService` slot, raising `BANKING.SERVICE_NOT_AVAILABLE` when the `db` slot is empty. `onReady` calls `service.initializeDefaultBank()` then mounts the route tree on the host `app` under `config.routePrefix`. `onClose` drops the cached service reference.
|
||||
|
||||
#### Scenario: onInit registers the bankingService
|
||||
|
||||
- **WHEN** `onInit(ctx)` runs with `db` registered
|
||||
- **THEN** `ctx.getService("bankingService")` returns a non-undefined `BankingService`
|
||||
|
||||
#### Scenario: onInit fails when db slot is missing
|
||||
|
||||
- **WHEN** `onInit(ctx)` runs with `db` absent
|
||||
- **THEN** it throws `BankingError(BANKING.SERVICE_NOT_AVAILABLE)`
|
||||
|
||||
#### Scenario: onReady mounts routes at the configured routePrefix
|
||||
|
||||
- **WHEN** `onInit` then `onReady` runs with options `{ routePrefix: "/api/v1/banking" }`
|
||||
- **THEN** the host `app.route` was called with `"/api/v1/banking"` and a Hono sub-app
|
||||
|
||||
#### Scenario: onClose drops the cached service
|
||||
|
||||
- **WHEN** `onInit` runs, then `onClose`, then `ctx.getService("bankingService")` is read
|
||||
- **THEN** the runtime cache is `undefined` (host registry retention is the host's concern)
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"onInit registers the bankingService slot"` AND `src/plugin/plugin.test.ts::"onInit throws SERVICE_NOT_AVAILABLE when db is missing"` AND `src/plugin/plugin.test.ts::"onReady mounts routes at the configured routePrefix"` AND `src/plugin/plugin.test.ts::"onClose drops the cached service"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Bank lifecycle CRUD with uniqueness and active-account guards
|
||||
|
||||
The package SHALL expose bank CRUD via `BankingService.createBank`, `updateBank`, and `softDeleteBank`. `createBank` rejects duplicates with `BANKING.BANK_NAME_TAKEN`. `updateBank` and `softDeleteBank` raise `BANKING.BANK_NOT_FOUND` when the bank id is missing. `softDeleteBank` raises `BANKING.BANK_HAS_ACTIVE_ACCOUNTS` when the bank still has accounts with `status = 4`.
|
||||
|
||||
#### Scenario: Duplicate name on create
|
||||
|
||||
- **WHEN** `createBank({ name: "A" })` is called twice
|
||||
- **THEN** the second call throws `BankingError(BANKING.BANK_NAME_TAKEN)`
|
||||
|
||||
#### Scenario: Soft-delete blocked by active accounts
|
||||
|
||||
- **WHEN** a bank has at least one `status = 4` account and `softDeleteBank(id)` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.BANK_HAS_ACTIVE_ACCOUNTS)`
|
||||
|
||||
#### Scenario: Soft-delete sets status = 1
|
||||
|
||||
- **WHEN** a bank has no active accounts and `softDeleteBank(id)` is called
|
||||
- **THEN** the returned record has `status === 1` and the OTel `banks_total{op="soft-delete"}` counter incremented by 1
|
||||
|
||||
**Verified by:** `src/service/service.test.ts::"createBank rejects duplicate name with BANK_NAME_TAKEN"` AND `src/service/service.test.ts::"softDeleteBank rejects banks with active accounts"` AND `src/service/service.test.ts::"softDeleteBank sets status to 1 on success"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Account creation with per-user limit and default-bank fallback
|
||||
|
||||
The package SHALL expose `BankingService.createAccount(data)`. Before insert, it counts the user's accounts; if the count is `>= config.maxAccountsPerUser`, it raises `BANKING.ACCOUNT_LIMIT_REACHED`. When `data.bank_id` is omitted, it calls `BankRepository.getOrCreateDefaultBank(config.defaultBankName, config.defaultSortCode)` and inserts against the resulting id.
|
||||
|
||||
#### Scenario: Account-limit enforcement
|
||||
|
||||
- **WHEN** the user already has `maxAccountsPerUser` accounts and `createAccount` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.ACCOUNT_LIMIT_REACHED)`
|
||||
|
||||
#### Scenario: Default-bank fallback
|
||||
|
||||
- **WHEN** `createAccount({ user_id, account_number, ... /* no bank_id */ })` is called and no bank named `defaultBankName` exists
|
||||
- **THEN** a bank with that name and `sort_code = defaultSortCode` is created and the account references it
|
||||
|
||||
**Verified by:** `src/service/service.test.ts::"createAccount rejects when account-limit reached"` AND `src/service/service.test.ts::"createAccount bootstraps default bank when bank_id omitted"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Fund-movement primitives with overdraft policy
|
||||
|
||||
The package SHALL expose `BankingService.transferFunds`, `addFunds`, and `subtractFunds`. Each raises `BANKING.INVALID_AMOUNT` for non-positive `amount`. `transferFunds` short-circuits with `BANKING.INVALID_OPTIONS` when `config.enableTransfers === false`. When `config.enableOverdraft === true`, the per-account `overdraft` field becomes the negative balance ceiling; otherwise the floor is `0`. Insufficient-balance outcomes return `{ success: false, code: "BANKING.INSUFFICIENT_FUNDS" }` rather than throwing.
|
||||
|
||||
#### Scenario: Negative-amount reject
|
||||
|
||||
- **WHEN** `addFunds(id, -1)` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.INVALID_AMOUNT)`
|
||||
|
||||
#### Scenario: Transfer disabled
|
||||
|
||||
- **WHEN** `transferFunds(...)` is called with `config.enableTransfers === false`
|
||||
- **THEN** the call throws `BankingError(BANKING.INVALID_OPTIONS)`
|
||||
|
||||
#### Scenario: Insufficient funds with overdraft off
|
||||
|
||||
- **WHEN** an account has `balance = 50, overdraft = 100` and `subtractFunds(id, 100)` is called with `enableOverdraft: false`
|
||||
- **THEN** the result is `{ success: false, code: "BANKING.INSUFFICIENT_FUNDS" }` and balance is unchanged
|
||||
|
||||
#### Scenario: Insufficient funds with overdraft on
|
||||
|
||||
- **WHEN** an account has `balance = 50, overdraft = 100` and `subtractFunds(id, 100)` is called with `enableOverdraft: true`
|
||||
- **THEN** the result is `{ success: true, data: { balance: -50 } }`
|
||||
|
||||
#### Scenario: Transfer success emits OTel
|
||||
|
||||
- **WHEN** a successful transfer of `amount = 10` runs
|
||||
- **THEN** `banking.transfers_total{outcome="success"}` increments by 1 and `banking.transfer_amount` records one observation of `10`
|
||||
|
||||
**Verified by:** `src/service/service.test.ts::"addFunds rejects negative amount"` AND `src/service/service.test.ts::"transferFunds short-circuits when enableTransfers is false"` AND `src/service/service.test.ts::"subtractFunds rejects when overdraft is off and balance insufficient"` AND `src/service/service.test.ts::"subtractFunds tolerates overdraft when enabled"` AND `src/service/service.test.ts::"successful transfer emits transfers_total + transfer_amount"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Default-bank idempotent bootstrap
|
||||
|
||||
The package SHALL expose `BankingService.initializeDefaultBank()`. When `config.enableDefaultBank === true`, the first call inserts a bank with `name = config.defaultBankName, sort_code = config.defaultSortCode, status = 4`; subsequent calls return the existing record. When `enableDefaultBank === false`, the call returns `null` without touching the database.
|
||||
|
||||
#### Scenario: First call creates the bank
|
||||
|
||||
- **WHEN** `initializeDefaultBank` is invoked with an empty `bank` table and `enableDefaultBank: true`
|
||||
- **THEN** exactly one bank with the configured name exists afterward
|
||||
|
||||
#### Scenario: Second call is a no-op
|
||||
|
||||
- **WHEN** `initializeDefaultBank` is invoked twice in a row with `enableDefaultBank: true`
|
||||
- **THEN** the bank count after both calls is `1`
|
||||
|
||||
#### Scenario: Disabled bootstrap
|
||||
|
||||
- **WHEN** `initializeDefaultBank` is invoked with `enableDefaultBank: false`
|
||||
- **THEN** the returned value is `null` and the bank table is unchanged
|
||||
|
||||
**Verified by:** `src/service/service.test.ts::"initializeDefaultBank creates default bank on first call"` AND `src/service/service.test.ts::"initializeDefaultBank is idempotent across two calls"` AND `src/service/service.test.ts::"initializeDefaultBank no-ops when enableDefaultBank is false"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Repository field-whitelist guard
|
||||
|
||||
The repositories SHALL gate every dynamically constructed `UPDATE` field name against the child class's `getAllowedFields()` whitelist. Unknown field names raise `BANKING.INVALID_FIELD_NAME` before any SQL is emitted.
|
||||
|
||||
#### Scenario: Whitelisted field passes
|
||||
|
||||
- **WHEN** `update(id, { name: "X" })` is called on `BankRepository`
|
||||
- **THEN** the resulting record carries `name === "X"`
|
||||
|
||||
#### Scenario: Non-whitelisted field rejects
|
||||
|
||||
- **WHEN** `update(id, { evil_field: "X" } as never)` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.INVALID_FIELD_NAME)`
|
||||
|
||||
**Verified by:** `src/repositories/bank-repository.test.ts::"update accepts whitelisted field"` AND `src/repositories/bank-repository.test.ts::"update rejects non-whitelisted field with INVALID_FIELD_NAME"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Account-number and sort-code parsing
|
||||
|
||||
The package SHALL expose `BankAccountRepository.parseAccountNumber(str)` and `splitAccountNumber(num)`. The string parser accepts `BBBB-CCCC-AAAAAAAA`; the numeric splitter accepts a non-negative integer rendered as 12 digits with zero-padding. `BankRepository.getBySortCode(sortCode)` accepts `XX-XX-XX`. All three raise their respective typed errors (`INVALID_ACCOUNT_NUMBER`, `INVALID_SORT_CODE`) on malformed input.
|
||||
|
||||
#### Scenario: Valid account-number string parses
|
||||
|
||||
- **WHEN** `parseAccountNumber("0123-4567-89012345")` is called
|
||||
- **THEN** the result is `{ bankCode: "0123", branchCode: "4567", accountCode: "89012345" }`
|
||||
|
||||
#### Scenario: Malformed account-number rejects
|
||||
|
||||
- **WHEN** `parseAccountNumber("invalid")` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.INVALID_ACCOUNT_NUMBER)`
|
||||
|
||||
#### Scenario: Malformed sort-code rejects
|
||||
|
||||
- **WHEN** `getBySortCode("12-3-456")` is called
|
||||
- **THEN** the call throws `BankingError(BANKING.INVALID_SORT_CODE)`
|
||||
|
||||
**Verified by:** `src/repositories/account-repository.test.ts::"parseAccountNumber accepts canonical BBBB-CCCC-AAAAAAAA"` AND `src/repositories/account-repository.test.ts::"parseAccountNumber throws INVALID_ACCOUNT_NUMBER on malformed input"` AND `src/repositories/bank-repository.test.ts::"getBySortCode throws INVALID_SORT_CODE on malformed input"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Tenant cleanup
|
||||
|
||||
The package SHALL expose `BankingService.deleteTenantData(tenantId)` returning `{ banks: number, accounts: number }` — the count of rows removed. Failures wrap as `BANKING.TENANT_CLEANUP_FAILED` with the underlying error as `cause`. A `decommissionBankingTenant(service, tenantId)` helper is re-exported for hosts that orchestrate tenant decommission externally.
|
||||
|
||||
#### Scenario: Cleanup returns counts
|
||||
|
||||
- **WHEN** `deleteTenantData("site:abc")` is called against a database with 2 banks and 5 accounts owned by tenant `abc`
|
||||
- **THEN** the returned object has `banks: 2, accounts: 5`
|
||||
|
||||
**Verified by:** `src/service/service.test.ts::"deleteTenantData returns row counts for both tables"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Route surface
|
||||
|
||||
The package SHALL expose `createBankingRoutes(deps)` returning an `OpenAPIHono` instance mounting 7 bank routes (`GET /`, `POST /`, `GET /:id`, `PATCH /:id`, `DELETE /:id`, `GET /sort-code/:sortCode`, `GET /search/:query`) and 12 account routes (`GET /`, `POST /`, `GET /:id`, `PUT /:id`, `DELETE /:id`, `GET /by-number/:accountNumber`, `GET /user/:userId`, `GET /:id/owner`, `POST /transfer`, `POST /:id/add-funds`, `POST /:id/subtract-funds`, `PATCH /:id/status`). Every route validates inputs via Zod and maps thrown `BankingError`s to HTTP `400`/`404`/`500` via the package's `errorBody` helper.
|
||||
|
||||
#### Scenario: Routes mounted
|
||||
|
||||
- **WHEN** `createBankingRoutes({ service })` is called
|
||||
- **THEN** the returned app has 19 registered routes whose paths begin with `/banks/` or `/accounts/`
|
||||
|
||||
#### Scenario: Error mapping
|
||||
|
||||
- **WHEN** `BANKING.BANK_NOT_FOUND` propagates from a handler
|
||||
- **THEN** the HTTP response carries status `404` and a body `{ success: false, error, code }`
|
||||
|
||||
**Verified by:** `src/routes/build-routes.test.ts::"mounts 19 routes across /banks and /accounts"` AND `src/routes/build-routes.test.ts::"maps BANK_NOT_FOUND to HTTP 404 in error body"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Service-registry augmentation
|
||||
|
||||
The package SHALL surface the `bankingService` slot on `@suaveplan/api-plugin/service-registry`'s `ServiceRegistry` interface via a side-effect-imported module. `ctx.getService("bankingService")` SHALL type as `BankingService | undefined` for every consumer importing the package barrel.
|
||||
|
||||
#### Scenario: Augmented slot type
|
||||
|
||||
- **WHEN** a consumer file imports `@suaveplan/api-plugin-banking` and calls `ctx.getService("bankingService")`
|
||||
- **THEN** TypeScript infers the result type as `BankingService | undefined` without a cast
|
||||
|
||||
**Verified by:** `src/api-plugin-banking.spec-conformance.test.ts::"ServiceRegistry['bankingService'] is BankingService"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Error-code registry
|
||||
|
||||
The package SHALL register 14 `BANKING.*` error codes via `defineCodes("BANKING", [...])` and export them as `BANKING_CODES`, with a `BankingError` class extending `SuaveplanError<BankingErrorCode>`. Every registered code SHALL be thrown by at least one production code path.
|
||||
|
||||
#### Scenario: Code registry exhaustiveness
|
||||
|
||||
- **WHEN** every code in `BANKING_CODES` is enumerated
|
||||
- **THEN** the union of code strings equals exactly the 14 documented codes
|
||||
|
||||
**Verified by:** `src/error-codes/error-codes.test.ts::"BANKING_CODES carries the 14 documented codes"` AND `src/api-plugin-banking.spec-conformance.test.ts::"every BANKING code is thrown from at least one production path"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OTel instrument bundle
|
||||
|
||||
The package SHALL expose `getBankingInstruments()` returning a cached bundle of 6 instruments (`banksTotal`, `accountsTotal`, `transfersTotal`, `transferAmount`, `fundOperationsTotal`, `queryDurationMs`) bound to the meter `"@suaveplan/api-plugin-banking"`. Production code paths SHALL emit on every lifecycle transition (bank create/update/soft-delete, account create/update/delete/status-change, transfer success/failure/insufficient-funds, add-funds, subtract-funds).
|
||||
|
||||
#### Scenario: Instruments cached
|
||||
|
||||
- **WHEN** `getBankingInstruments()` is called twice without `resetBankingInstruments()` in between
|
||||
- **THEN** the returned reference is identity-equal
|
||||
|
||||
#### Scenario: Counter emitted on production path
|
||||
|
||||
- **WHEN** `service.createBank({ name: "X" })` succeeds against a fresh in-memory `MeterProvider`
|
||||
- **THEN** the collected metrics include `banking.banks_total{op="create"}` with delta `1`
|
||||
|
||||
**Verified by:** `src/otel/instruments.test.ts::"getBankingInstruments caches the bundle"` AND `src/service/service.test.ts::"createBank emits banks_total{op=create} on the global MeterProvider"`
|
||||
66
openspec/changes/add-api-plugin-banking/tasks.md
Normal file
66
openspec/changes/add-api-plugin-banking/tasks.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Tasks — Add `@suaveplan/api-plugin-banking`
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read `../suaveplan/enterprise/api-plugin-banking.md` end-to-end
|
||||
- [x] 1.2 Verify no prior `openspec/specs/api-plugin-banking/spec.md` exists
|
||||
- [x] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
|
||||
- [x] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
|
||||
- [x] 1.5 Fill `## Source mapping` with line references to upstream
|
||||
- [x] 1.6 Draft `specs/api-plugin-banking/spec.md` from `openspec/templates/spec-delta.md`
|
||||
- [x] 1.7 Every requirement carries a `**Verified by:**` line naming concrete `it(...)` titles
|
||||
|
||||
## 2. Implementation
|
||||
|
||||
- [x] 2.1 `error-codes/error-codes.ts` — 14 `BANKING.*` codes + `BankingError` class
|
||||
- [x] 2.2 `otel/instruments.ts` — meter + 6 instruments (counters + histograms)
|
||||
- [x] 2.3 `schemas/schemas.ts` — 11 Zod schemas
|
||||
- [x] 2.4 `schemas/options.ts` — `BankingPluginOptionsSchema` factory-options schema
|
||||
- [x] 2.5 `repositories/base-repository.ts` — shared scaffold + field whitelist + RecordId flattening
|
||||
- [x] 2.6 `repositories/bank-repository.ts` — full upstream bank surface retyped
|
||||
- [x] 2.7 `repositories/account-repository.ts` — full upstream account surface retyped
|
||||
- [x] 2.8 `service/service.ts` — `BankingService` coordinator with policy enforcement
|
||||
- [x] 2.9 `routes/build-routes.ts` — 19 OpenAPIHono routes with uniform error mapping
|
||||
- [x] 2.10 `plugin/plugin.ts` — `createBankingPlugin` factory + `decommissionBankingTenant` helper
|
||||
- [x] 2.11 `services/registry-augmentation.ts` — `bankingService` / `app` / `db` slots
|
||||
- [x] 2.12 `types/types.ts` — record aliases, result wrappers, lifecycle event payloads, `BankingEventMap`
|
||||
- [x] 2.13 `index.ts` — explicit named barrel + side-effect imports for error-codes + registry augmentation
|
||||
- [x] 2.14 `surql/001-bank-and-account.surql` — schema migration carried over verbatim
|
||||
|
||||
## 3. Tests
|
||||
|
||||
- [ ] 3.1 Unit tests per feature module (in-memory `FakeDatabaseAdapter` for fast feedback)
|
||||
- [ ] 3.2 `api-plugin-banking.spec-conformance.test.ts` — type-level + barrel pins
|
||||
- [ ] 3.3 Integration test against real SurrealDB testcontainer covering create / transfer / soft-delete / tenant-purge
|
||||
- [ ] 3.4 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for testcontainer-bound cases)
|
||||
|
||||
## 4. Documentation
|
||||
|
||||
- [x] 4.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples, full API reference)
|
||||
- [ ] 4.2 README links every co-located `.md`
|
||||
- [ ] 4.3 STATUS.md row updated post-archival (other-session-owned; skipped per guidance)
|
||||
|
||||
## 5. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [ ] 5.1 `bun run check` — Biome 0/0/0
|
||||
- [ ] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-plugin-banking`
|
||||
- [ ] 5.3 `bunx turbo test:coverage --filter=@suaveplan/api-plugin-banking`
|
||||
- [ ] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/api-plugin-banking`
|
||||
- [ ] 5.5 `bunx turbo build --filter=@suaveplan/api-plugin-banking`
|
||||
|
||||
## 6. Pre-archive 9-row checklist
|
||||
|
||||
- [ ] 6.1 Generics threaded; `expectTypeOf` assertions on every literal-type claim
|
||||
- [ ] 6.2 Zod schemas wired at every declared boundary
|
||||
- [ ] 6.3 Every error code in `error-codes.ts` thrown by at least one production path
|
||||
- [ ] 6.4 Strict-mode behaviours tested both on and off (overdraft toggle)
|
||||
- [ ] 6.5 OTel instruments emitted from production paths and asserted with a real in-memory `MeterProvider`
|
||||
- [ ] 6.6 Lifecycle dedup tested where claimed (idempotent default-bank bootstrap)
|
||||
- [ ] 6.7 No retroactive tick
|
||||
- [ ] 6.8 No `export *` in any feature barrel
|
||||
- [ ] 6.9 No direct `vitest` / `bun:test` imports in any test file
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold spec delta into `openspec/specs/api-plugin-banking/spec.md`
|
||||
- [ ] 7.2 Move `openspec/changes/add-api-plugin-banking/` → `openspec/archive/2026-05-19-add-api-plugin-banking/`
|
||||
77
openspec/changes/add-api-plugin-export/proposal.md
Normal file
77
openspec/changes/add-api-plugin-export/proposal.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Add @suaveplan/api-plugin-export
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-export`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (migration from `../suaveplan/packages/server/api-plugin-export`)
|
||||
|
||||
- **Design**: ./design.md (n/a — straight upstream parity; rationale documented inline below)
|
||||
- **Spec delta**: ./specs/api-plugin-export/spec.md
|
||||
|
||||
## Why
|
||||
|
||||
Genesis needs the upstream `api-plugin-export` capability so downstream consumers (admin dashboards, analytics surfaces, GDPR subject-access endpoints) can export any registered resource to CSV, JSON, or XLSX through a uniform OpenAPI route group. Upstream `1.0.5` ships three exporters + four routes + an in-process job store + a Zod-validated config schema. Migrating it at functional parity unblocks the api-plugin-{user, audit, ...} consumers that already type against `DataProvider`, `ExportJob`, and the `export:*` event surface.
|
||||
|
||||
## What changes
|
||||
|
||||
- ADDED: `createExportPlugin(options?)` — typed `PluginDescriptor<ExportPluginConfig>` factory, registers `exportProviders` and `exportJobs` services in `ServiceRegistry`, mounts `createExportRoutes(...)` onto `ctx.getService("app")` at the configured `basePath`, emits `export:initialized` / `export:ready` / `export:started` / `export:completed` on the bus.
|
||||
- ADDED: `ExportPluginConfigSchema` + `ExportPluginConfig` — Zod `{ basePath, maxRows, maxFileSizeBytes, defaultFormat }` with defaults.
|
||||
- ADDED: `registerDataProvider(providers, provider)` — typed registration helper.
|
||||
- ADDED: `CsvExporter`, `JsonExporter`, `XlsxExporter` — `Exporter` implementations with helper functions (`escapeCsvField`, `formatCellValue`, `convertToCsv`, `projectRow`, `convertToJson`, `escapeXml`, `columnIndexToLetter`, `formatXlsxCell`, `buildSpreadsheetXml`).
|
||||
- ADDED: `createExportRoutes(deps)` — `OpenAPIHono` with `GET /`, `POST /`, `GET /:id`, `DELETE /:id`.
|
||||
- ADDED: `runExport`, `getExporter`, `inferColumns`, `DEFAULT_EXPORT_CONFIG`, `ExportServiceConfig` — service-layer surface.
|
||||
- ADDED: Zod schemas `ExportFormatSchema`, `ExportStatusSchema`, `ColumnDefinitionSchema`, `CreateExportJobSchema`, `ExportJobSchema`, `ExportJobListQuerySchema`.
|
||||
- ADDED: Types `ExportFormat`, `ExportStatus`, `ColumnDefinition`, `CreateExportJob`, `ExportJob`, `ExportJobListQuery`, `ExportOptions`, `ExportResult`, `Exporter`, `DataProvider`, `ExportEventMap`.
|
||||
- ADDED: `EXPORT_CODES` + `ExportError` — 5 stable codes (`UNSUPPORTED_FORMAT`, `ROW_LIMIT_EXCEEDED`, `SIZE_LIMIT_EXCEEDED`, `PROVIDER_NOT_FOUND`, `PROVIDER_FETCH_FAILED`) extending `SuaveplanError`.
|
||||
- ADDED: `getExportInstruments()` / `resetExportInstruments()` — OTel `exports_total{format,outcome}`, `export_rows_total{format}`, `export_duration_ms` histogram.
|
||||
- ADDED: `ServiceRegistry` module augmentation for `exportProviders` and `exportJobs` slots.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- SurrealDB-backed persistence, `GET /:id/download` streaming endpoint, OOXML-binary `.xlsx`, JWT/permissions/rate-limit/idempotency/audit middleware, and `surql/` migrations — the upstream `1.0.5` `src/` does not implement these. Enterprise audit doc (`../suaveplan/enterprise/api-plugin-export.md` §§3–4) lists them as future work for the upstream package itself; landing them here would diverge from the parity goal and pull in `@suaveplan/api-plugin-database` / `@suaveplan/api-plugin-audit` / `@suaveplan/api-plugin-rate-limit` which are themselves pending migration.
|
||||
- Replacing upstream's plain `Error("…")` throws with a typed hierarchy — done in this change as a genesis-quality uplift (§7 Required enterprise-quality improvements item: "Error hierarchy"). The class names and codes shipped here mirror the upstream audit's call-out and become the canonical contract.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change derives from `../suaveplan/enterprise/api-plugin-export.md` and the upstream `src/` tree.
|
||||
|
||||
| Requirement | Source lines | Notes |
|
||||
|---|---|---|
|
||||
| Plugin factory + lifecycle | L17-L22 / `src/index.ts` L38-L104 | rewritten against the genesis `@suaveplan/api-plugin` descriptor shape (`configSchema: ZodType` + `defaults: Partial<T>`) |
|
||||
| Zod config schema (basePath/maxRows/maxFileSizeBytes/defaultFormat) | L17-L18 / `src/index.ts` L21-L30 | verbatim |
|
||||
| Three exporter classes (CSV / JSON / XLSX-XML) | L17 / `src/exporters/{csv,json,xlsx}` | verbatim |
|
||||
| `runExport`, `getExporter`, `inferColumns`, `DEFAULT_EXPORT_CONFIG` | L24 / `src/services/export/export.service.ts` | verbatim |
|
||||
| `OpenAPIHono` route group at `/exports` with GET/POST/GET/:id/DELETE/:id | L25 / `src/routes/exports/exports.ts` | verbatim |
|
||||
| `DataProvider` registry as `Map<string, DataProvider>` | L23 / `src/types.ts` L127-L132 | verbatim |
|
||||
| Job store as `Map<string, ExportJob>` (in-process) | L13 / `src/index.ts` L42, L66 | verbatim (memory-only is upstream's documented state; persistence is enterprise-future) |
|
||||
| `export:*` event emissions | L25 / `src/index.ts` L65, L87-L92, L96 | verbatim |
|
||||
| RFC 4180 CSV (delimiter / quoting / newlines) | L26 / `src/exporters/csv/csv.exporter.ts` | verbatim |
|
||||
| SpreadsheetML XLSX (XML, not OOXML/ZIP) | L28, L38 / `src/exporters/xlsx/xlsx.exporter.ts` | verbatim — OOXML binary is enterprise-future |
|
||||
| `registerDataProvider(providers, provider)` helper | `src/index.ts` L109-L114 | verbatim |
|
||||
| `ExportEventMap` and `@suaveplan/api` `EventMap` declaration | `src/types.ts` L139-L170 | trimmed to genesis equivalent — augments `@suaveplan/api-plugin/service-registry` for `exportProviders`/`exportJobs` (genesis pattern) |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| `EXPORT_CODES` registry + `ExportError` class | Required by genesis rule: every package extends `SuaveplanError` via `defineCodes`. Upstream uses `new Error(...)`; the audit doc (L52) explicitly flags this as a gap to fix during migration. |
|
||||
| OTel instruments (`exports_total{format,outcome}`, `export_rows_total{format}`, `export_duration_ms`) | Required by genesis pre-archive checklist row 5 ("OTel instruments asserted with real in-memory `MeterProvider`"). Upstream audit doc L51 calls for this. |
|
||||
| `ServiceRegistry` module augmentation for `exportProviders` / `exportJobs` | Required by genesis `@suaveplan/api-plugin` typed-slot contract; upstream's `@suaveplan/api` had a looser `services: Map<string, unknown>` so no augmentation was needed. |
|
||||
| 100% Istanbul coverage; sibling `.md` per impl file | Genesis-wide policy, not package-specific. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Every requirement carries a `**Verified by:**` line in `specs/api-plugin-export/spec.md` pointing at concrete `it(...)` titles in the listed files:
|
||||
|
||||
- `src/exporters/csv/csv.test.ts` — RFC 4180 CSV, helpers, exporter shape.
|
||||
- `src/exporters/json/json.test.ts` — projection, pretty-print, exporter shape.
|
||||
- `src/exporters/xlsx/xlsx.test.ts` — XML escaping, column-letter math, cell formatting, builder.
|
||||
- `src/services/export.service.test.ts` — `getExporter` (incl. unsupported-format error), `inferColumns`, `runExport` with row + size limits.
|
||||
- `src/routes/exports.test.ts` — GET list with filters/pagination, POST create (CSV/JSON/XLSX, custom columns/filename), 404 on missing resource, 400 on limits, 400 on provider error, GET/:id, DELETE/:id.
|
||||
- `src/plugin/plugin.test.ts` — descriptor metadata, `onInit` registers services + emits `export:initialized`, `onReady` mounts routes + emits `export:ready`, custom basePath, `onClose` clears state.
|
||||
- `src/error-codes/error-codes.test.ts` — every code reachable from a production path.
|
||||
- `src/otel/instruments.test.ts` — counters/histogram emit against a real in-memory `MeterProvider`.
|
||||
- `src/api-plugin-export.spec-conformance.test.ts` — module augmentation, public surface, type-level claims (`expectTypeOf`).
|
||||
|
||||
## Migration impact
|
||||
|
||||
None — additive only. No genesis consumer currently imports `@suaveplan/api-plugin-export`; the package is brand new on this branch.
|
||||
@@ -0,0 +1,273 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: api-plugin-export (add-api-plugin-export)
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-export`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Plugin descriptor factory
|
||||
|
||||
The package SHALL expose `createExportPlugin(options?: Partial<ExportPluginConfig>): PluginDescriptor<ExportPluginConfig>`, returning a `definePlugin(...)`-shaped descriptor named `"export"`, version `0.1.0`, with `optionalDependencies: ["database", "events"]`, `tags: ["export","data","csv","json","xlsx"]`, a `configSchema` bound to `ExportPluginConfigSchema`, and `defaults` merged from caller-supplied input + `DEFAULT_EXPORT_CONFIG`. The four lifecycle hooks (`onInit`, `onReady`, `onClose`) MUST be present and typed against `PluginContext<ExportPluginConfig>`.
|
||||
|
||||
#### Scenario: descriptor metadata is correct
|
||||
|
||||
- **WHEN** `createExportPlugin()` is called with no arguments
|
||||
- **THEN** the returned descriptor has `name === "export"`, `version === "0.1.0"`, `tags` contains `"csv"`/`"json"`/`"xlsx"`, and `optionalDependencies` contains `"database"` and `"events"`
|
||||
|
||||
#### Scenario: factory-supplied options merge with defaults
|
||||
|
||||
- **WHEN** `createExportPlugin({ basePath: "/api/data-export" })` is called
|
||||
- **THEN** `descriptor.defaults.basePath === "/api/data-export"` AND `descriptor.defaults.maxRows === 50000`
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"returns a plugin descriptor with the expected metadata"` AND `src/plugin/plugin.test.ts::"merges factory-supplied options with DEFAULT_EXPORT_CONFIG"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: onInit registers services and emits event
|
||||
|
||||
When `onInit(ctx)` runs, the plugin SHALL register `exportProviders` (a `Map<string, DataProvider>`) and `exportJobs` (a `Map<string, ExportJob>`) on `ctx.registerService`, then emit `"export:initialized"` on `ctx.eventBus`.
|
||||
|
||||
#### Scenario: services and event after init
|
||||
|
||||
- **WHEN** the descriptor's `onInit` is invoked with a real `PluginContext`
|
||||
- **THEN** `ctx.getService("exportProviders")` returns a `Map`, `ctx.getService("exportJobs")` returns a `Map`, and the bus has received `"export:initialized"`
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"onInit registers exportProviders and exportJobs and emits export:initialized"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: onReady mounts routes on the host app
|
||||
|
||||
`onReady(ctx)` SHALL resolve `app` from `ctx.getService("app")`, build the route group via `createExportRoutes({ providers, exportConfig, jobs, onExportStarted, onExportCompleted })`, mount it via `app.route(basePath, routes)`, then emit `"export:ready"`. If `app` is not present on the registry the hook SHALL throw `ExportError(EXPORT_CODES.PROVIDER_NOT_FOUND)` is NOT the right code — the hook SHALL throw a `ExportError` carrying code `EXPORT_CODES.UNSUPPORTED_FORMAT`-style precondition? No — the hook SHALL throw `ExportError` carrying a clear message naming the missing `app` slot.
|
||||
|
||||
#### Scenario: routes mount at configured basePath
|
||||
|
||||
- **WHEN** `onReady` runs against a `PluginContext` whose `ServiceRegistry` carries an `OpenAPIHono` at `app` and the config's `basePath === "/exports"`
|
||||
- **THEN** `app.route` has been invoked with `"/exports"` and an `OpenAPIHono` instance, AND `"export:ready"` has been emitted
|
||||
|
||||
#### Scenario: missing app slot fails fast
|
||||
|
||||
- **WHEN** `onReady` runs against a context with no `app` service registered
|
||||
- **THEN** the hook throws an `Error` (or `ExportError`) whose message names `"app"`
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"onReady mounts routes on the host app and emits export:ready"` AND `src/plugin/plugin.test.ts::"onReady throws when no app service is registered"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: onClose clears in-process state
|
||||
|
||||
`onClose()` SHALL clear both the `providers` and `jobs` maps held by the plugin closure so the next boot starts from an empty state.
|
||||
|
||||
#### Scenario: maps are cleared
|
||||
|
||||
- **WHEN** the descriptor's `onClose` is invoked after `onInit` populated state
|
||||
- **THEN** the same map instances passed to `onInit` are `.size === 0`
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"onClose clears providers and jobs maps"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: registerDataProvider helper
|
||||
|
||||
The package SHALL export `registerDataProvider(providers: Map<string, DataProvider>, provider: DataProvider): void` that sets `providers.set(provider.resource, provider)` (last-write-wins).
|
||||
|
||||
#### Scenario: register and overwrite
|
||||
|
||||
- **WHEN** the same `resource` is registered twice with different `DataProvider` values
|
||||
- **THEN** the second registration replaces the first
|
||||
|
||||
**Verified by:** `src/plugin/plugin.test.ts::"registerDataProvider stores the provider keyed by its resource"` AND `src/plugin/plugin.test.ts::"registerDataProvider overwrites an existing provider for the same resource"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Zod-validated plugin config
|
||||
|
||||
`ExportPluginConfigSchema` SHALL Zod-validate `{ basePath: string default "/exports", maxRows: int 1..1_000_000 default 50_000, maxFileSizeBytes: int >=1024 default 52_428_800, defaultFormat: "csv"|"json"|"xlsx" default "csv" }`. `ExportPluginConfig = z.infer<typeof ExportPluginConfigSchema>`.
|
||||
|
||||
#### Scenario: empty input fills all defaults
|
||||
|
||||
- **WHEN** `ExportPluginConfigSchema.safeParse({})` runs
|
||||
- **THEN** the result is `{ success: true, data: { basePath: "/exports", maxRows: 50000, maxFileSizeBytes: 52428800, defaultFormat: "csv" } }`
|
||||
|
||||
#### Scenario: invalid bounds rejected
|
||||
|
||||
- **WHEN** `ExportPluginConfigSchema.safeParse({ maxRows: 0 })`, `({ maxRows: 2_000_000 })`, `({ defaultFormat: "pdf" })`, or `({ maxFileSizeBytes: 100 })` run
|
||||
- **THEN** each result is `{ success: false }`
|
||||
|
||||
**Verified by:** `src/schemas/schemas.test.ts::"ExportPluginConfigSchema fills every default for empty input"` AND `src/schemas/schemas.test.ts::"ExportPluginConfigSchema rejects out-of-range maxRows"` AND `src/schemas/schemas.test.ts::"ExportPluginConfigSchema rejects maxRows above the 1_000_000 ceiling"` AND `src/schemas/schemas.test.ts::"ExportPluginConfigSchema rejects an invalid defaultFormat"` AND `src/schemas/schemas.test.ts::"ExportPluginConfigSchema rejects maxFileSizeBytes below 1024"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Domain Zod schemas
|
||||
|
||||
The package SHALL export Zod schemas `ExportFormatSchema` (`"csv"|"json"|"xlsx"`), `ExportStatusSchema` (`"pending"|"processing"|"completed"|"failed"`), `ColumnDefinitionSchema` (non-empty `key` + non-empty `header` + optional `transform`), `CreateExportJobSchema` (format + resource[1..128] + optional filters/columns(≥1)/filename(≤256)/options), `ExportJobSchema` (id + format + resource + status + filters? + filename? + nullable totalRows/exportedRows/fileSize/error + createdAt + completedAt nullable), `ExportJobListQuerySchema` (status? + format? + resource? + limit 1..100 default 20 + offset ≥0 default 0).
|
||||
|
||||
#### Scenario: rejects empty resource and out-of-range limit
|
||||
|
||||
- **WHEN** `CreateExportJobSchema.safeParse({ format:"csv", resource:"" })` and `ExportJobListQuerySchema.safeParse({ limit: 101 })` run
|
||||
- **THEN** both results are `{ success: false }`
|
||||
|
||||
**Verified by:** `src/schemas/schemas.test.ts::"CreateExportJobSchema rejects an empty resource"` AND `src/schemas/schemas.test.ts::"ExportJobListQuerySchema rejects limit above 100"` AND `src/schemas/schemas.test.ts::"ColumnDefinitionSchema rejects an empty header"` AND `src/schemas/schemas.test.ts::"ExportJobSchema accepts a completed job with all fields populated"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: CSV exporter — RFC 4180
|
||||
|
||||
`CsvExporter.export(data, columns, options?)` SHALL produce a `text/csv` `ExportResult` whose `content` quotes any field containing the configured delimiter, a literal `"`, `\n`, or `\r`, doubling embedded `"` per RFC 4180. The header row is emitted unless `options.includeHeaders === false`. The default delimiter is `,`; `options.delimiter` overrides it.
|
||||
|
||||
#### Scenario: quoting and delimiter
|
||||
|
||||
- **WHEN** `escapeCsvField('say "hi"', ",")` runs
|
||||
- **THEN** the return value equals `'"say ""hi"""'`
|
||||
|
||||
#### Scenario: convertToCsv emits header then rows
|
||||
|
||||
- **WHEN** `convertToCsv([{id:1,name:"Alice"}], [{key:"id",header:"ID"},{key:"name",header:"Name"}])` runs
|
||||
- **THEN** the return value equals `"ID,Name\n1,Alice"`
|
||||
|
||||
**Verified by:** `src/exporters/csv/csv.test.ts::"escapeCsvField doubles embedded double-quotes per RFC 4180"` AND `src/exporters/csv/csv.test.ts::"escapeCsvField quotes a value containing the delimiter"` AND `src/exporters/csv/csv.test.ts::"convertToCsv emits a header row followed by data rows"` AND `src/exporters/csv/csv.test.ts::"convertToCsv applies column transforms before escaping"` AND `src/exporters/csv/csv.test.ts::"CsvExporter.export returns text/csv with a non-zero byteSize"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: JSON exporter
|
||||
|
||||
`JsonExporter.export(data, columns, options?)` SHALL produce an `application/json` `ExportResult` whose `content` is `JSON.stringify` of each row projected through `projectRow(row, columns)` (`col.header` as key, `col.transform(row[col.key])` as value when transform is set, raw `row[col.key]` otherwise). `options.pretty === true` SHALL use 2-space indentation.
|
||||
|
||||
#### Scenario: projection and pretty-print
|
||||
|
||||
- **WHEN** `convertToJson([{id:1,name:"Alice"}], [{key:"name",header:"Full Name"}], { pretty: true })` runs
|
||||
- **THEN** the parsed result equals `[{"Full Name":"Alice"}]` AND the raw string contains a `\n`
|
||||
|
||||
**Verified by:** `src/exporters/json/json.test.ts::"projectRow rekeys using column.header"` AND `src/exporters/json/json.test.ts::"projectRow applies column.transform when present"` AND `src/exporters/json/json.test.ts::"convertToJson produces compact output by default"` AND `src/exporters/json/json.test.ts::"convertToJson produces pretty output when options.pretty is true"` AND `src/exporters/json/json.test.ts::"JsonExporter.export returns application/json with a non-zero byteSize"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: XLSX (SpreadsheetML XML) exporter
|
||||
|
||||
`XlsxExporter.export(data, columns, options?)` SHALL produce an `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` `ExportResult` whose `content` is a SpreadsheetML XML document with a single `<sheetData>` block, one `<row>` per input record (plus a header row unless `options.includeHeaders === false`), and cells typed per JS value: numbers → `<v>`, booleans → `t="b"` with `1`/`0`, `null`/`undefined` → empty inline-string, everything else (incl. `Date`) → escaped inline-string.
|
||||
|
||||
#### Scenario: column-letter math wraps after Z
|
||||
|
||||
- **WHEN** `columnIndexToLetter(0)`, `columnIndexToLetter(25)`, and `columnIndexToLetter(26)` run
|
||||
- **THEN** the return values are `"A"`, `"Z"`, `"AA"`
|
||||
|
||||
#### Scenario: XML escaping
|
||||
|
||||
- **WHEN** `escapeXml('<a & "b">')` runs
|
||||
- **THEN** the return value equals `"<a & "b">"`
|
||||
|
||||
**Verified by:** `src/exporters/xlsx/xlsx.test.ts::"escapeXml escapes all five XML special characters"` AND `src/exporters/xlsx/xlsx.test.ts::"columnIndexToLetter wraps to AA after Z"` AND `src/exporters/xlsx/xlsx.test.ts::"formatXlsxCell emits a numeric cell for finite numbers"` AND `src/exporters/xlsx/xlsx.test.ts::"formatXlsxCell emits a boolean cell as t=b with 1 or 0"` AND `src/exporters/xlsx/xlsx.test.ts::"formatXlsxCell emits an inline-string cell for NaN"` AND `src/exporters/xlsx/xlsx.test.ts::"buildSpreadsheetXml includes a header row by default"` AND `src/exporters/xlsx/xlsx.test.ts::"buildSpreadsheetXml omits the header row when includeHeaders is false"` AND `src/exporters/xlsx/xlsx.test.ts::"XlsxExporter.export returns the spreadsheetml MIME type"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Export service surface
|
||||
|
||||
The package SHALL export `getExporter(format)` returning a fresh exporter per format, `inferColumns(data)` returning unique `{key,header:key}` entries deduped across rows, `runExport(provider, format, columns?, filters?, options?, config?)` that fetches via `provider.fetch(filters)`, applies row + size limits, and returns the `ExportResult`, plus `DEFAULT_EXPORT_CONFIG` and `ExportServiceConfig`.
|
||||
|
||||
#### Scenario: row limit raises ExportError
|
||||
|
||||
- **WHEN** `runExport(provider, "csv", undefined, undefined, undefined, { maxRows: 5, ... })` is called with a provider returning 10 rows
|
||||
- **THEN** the promise rejects with `ExportError` whose `code === EXPORT_CODES.ROW_LIMIT_EXCEEDED` AND `context.actual === 10` AND `context.limit === 5`
|
||||
|
||||
#### Scenario: file-size limit raises ExportError
|
||||
|
||||
- **WHEN** `runExport(provider, "csv", undefined, undefined, undefined, { maxFileSizeBytes: 100, ... })` is called with data that produces a >100-byte CSV
|
||||
- **THEN** the promise rejects with `ExportError` whose `code === EXPORT_CODES.SIZE_LIMIT_EXCEEDED`
|
||||
|
||||
#### Scenario: unsupported format raises ExportError
|
||||
|
||||
- **WHEN** `getExporter("pdf" as ExportFormat)` is called
|
||||
- **THEN** the call throws `ExportError` whose `code === EXPORT_CODES.UNSUPPORTED_FORMAT`
|
||||
|
||||
**Verified by:** `src/services/export-service.test.ts::"getExporter returns a CsvExporter for csv"` AND `src/services/export-service.test.ts::"getExporter throws ExportError(UNSUPPORTED_FORMAT) for an unknown format"` AND `src/services/export-service.test.ts::"inferColumns deduplicates keys across rows"` AND `src/services/export-service.test.ts::"runExport throws ExportError(ROW_LIMIT_EXCEEDED) when data exceeds maxRows"` AND `src/services/export-service.test.ts::"runExport throws ExportError(SIZE_LIMIT_EXCEEDED) when output exceeds maxFileSizeBytes"` AND `src/services/export-service.test.ts::"runExport passes filters through to the data provider"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: HTTP route group
|
||||
|
||||
`createExportRoutes(deps)` SHALL return an `OpenAPIHono` exposing `GET /` (list with optional `status`/`format`/`resource` filters + `limit`/`offset` pagination), `POST /` (create export, returning `{ job, downloadUrl }` with `downloadUrl === \`/${jobId}/download\``), `GET /:id` (lookup with 404), `DELETE /:id` (delete with 404). The create handler MUST validate the body via `CreateExportJobSchema`, return 404 with `{ error }` when the resource has no provider, return 400 with `{ error }` when the row or size limit is exceeded, return 400 with `{ error }` when the provider's `fetch` rejects, persist the job in `deps.jobs` regardless of outcome, and call the optional `onExportStarted` / `onExportCompleted` callbacks.
|
||||
|
||||
#### Scenario: 200 happy path with CSV
|
||||
|
||||
- **WHEN** `POST /` is called with `{ format:"csv", resource:"users" }` and the `users` provider returns two rows
|
||||
- **THEN** the response status is 200, `body.job.status === "completed"`, `body.job.exportedRows === 2`, `body.downloadUrl` ends with `/download`
|
||||
|
||||
#### Scenario: 404 on unknown resource
|
||||
|
||||
- **WHEN** `POST /` is called with `resource:"unknown"`
|
||||
- **THEN** the response status is 404 AND `body.error` contains `"No data provider registered"`
|
||||
|
||||
#### Scenario: 400 on row-limit overflow
|
||||
|
||||
- **WHEN** `POST /` is called against a `maxRows: 10` config with a provider returning 100 rows
|
||||
- **THEN** the response status is 400 AND the persisted job's `status === "failed"`
|
||||
|
||||
#### Scenario: 400 on provider failure
|
||||
|
||||
- **WHEN** the provider's `fetch` throws
|
||||
- **THEN** the response status is 400 AND `body.error` contains the thrown message
|
||||
|
||||
#### Scenario: list, get, delete
|
||||
|
||||
- **WHEN** a job is created, then listed (with each of `status=completed`/`format=json`/`resource=orders`/pagination filters), then fetched by id, then deleted, then fetched again
|
||||
- **THEN** the list filters are honoured, the GET /:id returns 200 with the job, the DELETE returns 200 with `{success:true}`, the second GET /:id returns 404, AND DELETE on an unknown id returns 404
|
||||
|
||||
**Verified by:** `src/routes/exports.test.ts::"POST / creates a CSV export job"` AND `src/routes/exports.test.ts::"POST / creates a JSON export job"` AND `src/routes/exports.test.ts::"POST / creates an XLSX export job"` AND `src/routes/exports.test.ts::"POST / uses custom columns and filename"` AND `src/routes/exports.test.ts::"POST / returns 404 when the resource has no provider"` AND `src/routes/exports.test.ts::"POST / returns 400 when the row limit is exceeded"` AND `src/routes/exports.test.ts::"POST / returns 400 when the file-size limit is exceeded"` AND `src/routes/exports.test.ts::"POST / returns 400 when the provider rejects"` AND `src/routes/exports.test.ts::"POST / persists the job in the jobs map"` AND `src/routes/exports.test.ts::"GET / returns the empty list by default"` AND `src/routes/exports.test.ts::"GET / filters by status, format, and resource"` AND `src/routes/exports.test.ts::"GET / supports limit + offset pagination"` AND `src/routes/exports.test.ts::"GET /:id returns the job by id"` AND `src/routes/exports.test.ts::"GET /:id returns 404 for an unknown id"` AND `src/routes/exports.test.ts::"DELETE /:id deletes the job"` AND `src/routes/exports.test.ts::"DELETE /:id returns 404 for an unknown id"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Stable error-code registry
|
||||
|
||||
`EXPORT_CODES = defineCodes("EXPORT", [...])` SHALL register `UNSUPPORTED_FORMAT`, `ROW_LIMIT_EXCEEDED`, `SIZE_LIMIT_EXCEEDED`, `PROVIDER_NOT_FOUND`, `PROVIDER_FETCH_FAILED`. Every code MUST be reachable from a real production path inside `src/`. `ExportError extends SuaveplanError<ExportErrorCode>` and exposes a `public` constructor `(code, message, options?)`.
|
||||
|
||||
#### Scenario: each code is thrown from production
|
||||
|
||||
- **WHEN** the test calls (a) `getExporter("pdf" as never)`, (b) `runExport` with row-overflow, (c) `runExport` with size-overflow, (d) a `POST /` to an unknown resource, (e) a `POST /` whose provider rejects
|
||||
- **THEN** each of the five codes appears in the resulting `ExportError.code`
|
||||
|
||||
**Verified by:** `src/error-codes/error-codes.test.ts::"every EXPORT_CODES entry is thrown from at least one production path"` AND `src/error-codes/error-codes.test.ts::"ExportError narrows code to ExportErrorCode"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OpenTelemetry instruments
|
||||
|
||||
`getExportInstruments()` SHALL return a cached bundle of OTel instruments — `exportsTotal{format,outcome}` (Counter), `exportRowsTotal{format}` (Counter), `exportDurationMs` (Histogram) — built against `metrics.getMeter("@suaveplan/api-plugin-export")`. `runExport` and the route create-handler MUST increment `exportsTotal` with `outcome ∈ "success"|"failure"`, record `exportDurationMs`, and add `exportRowsTotal` on success. `resetExportInstruments()` clears the cache so tests can rebind.
|
||||
|
||||
#### Scenario: counters fire against an in-memory MeterProvider
|
||||
|
||||
- **WHEN** a successful `runExport` followed by a failing `runExport` (row-overflow) is executed against a global `MeterProvider` backed by `InMemoryMetricExporter`
|
||||
- **THEN** the collected metrics contain `export.exports_total` with both `outcome=success` and `outcome=failure` data points, `export.export_rows_total` with the success row count, and `export.export_duration_ms` with ≥2 recorded samples
|
||||
|
||||
**Verified by:** `src/otel/instruments.test.ts::"runExport emits exports_total, export_rows_total, and export_duration_ms on success"` AND `src/otel/instruments.test.ts::"runExport emits exports_total{outcome=failure} on row-limit overflow"` AND `src/otel/instruments.test.ts::"resetExportInstruments forces a re-bind on next get"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: ServiceRegistry module augmentation
|
||||
|
||||
The package SHALL `declare module "@suaveplan/api-plugin/service-registry"` to add typed slots `exportProviders: Map<string, DataProvider>`, `exportJobs: Map<string, ExportJob>`, AND re-declare host-owned slots (`app: OpenAPIHono`, `eventBus`, `logger`) so callers typecheck against `ctx.getService("exportProviders")` without casts. The augmentation MUST be side-effect imported from the package barrel.
|
||||
|
||||
#### Scenario: type narrowing on ctx.getService
|
||||
|
||||
- **WHEN** `expectTypeOf<ServiceRegistry["exportProviders"]>()` and `expectTypeOf<ServiceRegistry["exportJobs"]>()` are asserted
|
||||
- **THEN** the first equals `Map<string, DataProvider>` AND the second equals `Map<string, ExportJob>`
|
||||
|
||||
**Verified by:** `src/api-plugin-export.spec-conformance.test.ts::"ServiceRegistry augmentation declares exportProviders and exportJobs slots"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Public surface and subpath exports
|
||||
|
||||
The package root barrel (`src/index.ts`) SHALL use **explicit named re-exports** (never `export *`) for every public symbol listed in `proposal.md` "What changes". `package.json#exports` SHALL list a subpath for every public module (`./plugin/plugin`, `./routes/exports`, `./services/export-service`, `./exporters/csv/csv`, `./exporters/json/json`, `./exporters/xlsx/xlsx`, `./error-codes/error-codes`, `./otel/instruments`, `./schemas/schemas`, `./types/types`).
|
||||
|
||||
#### Scenario: barrel is explicit
|
||||
|
||||
- **WHEN** the source of `src/index.ts` is scanned
|
||||
- **THEN** zero `export *` lines exist in the file
|
||||
|
||||
**Verified by:** `src/api-plugin-export.spec-conformance.test.ts::"package barrel re-exports the full public surface as explicit named exports"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
73
openspec/changes/add-api-plugin-export/tasks.md
Normal file
73
openspec/changes/add-api-plugin-export/tasks.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Tasks — Add @suaveplan/api-plugin-export
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read `../suaveplan/enterprise/api-plugin-export.md` end-to-end
|
||||
- [x] 1.2 Read upstream `../suaveplan/packages/server/api-plugin-export/src/` end-to-end
|
||||
- [x] 1.3 Read `openspec/AGENTS.md` §§ 2a, 6a, 7, 12, 13, 14
|
||||
- [x] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
|
||||
- [x] 1.5 Fill `## Source mapping` table — every requirement cites a line range
|
||||
- [x] 1.6 Draft `specs/api-plugin-export/spec.md` with `## ADDED Requirements`
|
||||
- [x] 1.7 Every requirement carries a `**Verified by:**` line naming concrete `it(...)` titles
|
||||
|
||||
## 2. Implementation
|
||||
|
||||
- [x] 2.1 `src/error-codes/error-codes.ts` — `EXPORT_CODES` (5 codes) + `ExportError` extending `SuaveplanError`
|
||||
- [x] 2.2 `src/types/types.ts` — `ExportFormat`, `ExportStatus`, `ExportOptions`, `ExportResult`, `Exporter`, `DataProvider`
|
||||
- [x] 2.3 `src/schemas/schemas.ts` — `ExportFormatSchema`, `ExportStatusSchema`, `ColumnDefinitionSchema`, `CreateExportJobSchema`, `ExportJobSchema`, `ExportJobListQuerySchema`, `ExportPluginConfigSchema`
|
||||
- [x] 2.4 `src/otel/instruments.ts` — `getExportInstruments()` + `resetExportInstruments()`
|
||||
- [x] 2.5 `src/exporters/csv/csv.ts` — `escapeCsvField`, `formatCellValue`, `convertToCsv`, `CsvExporter`
|
||||
- [x] 2.6 `src/exporters/json/json.ts` — `projectRow`, `convertToJson`, `JsonExporter`
|
||||
- [x] 2.7 `src/exporters/xlsx/xlsx.ts` — `escapeXml`, `columnIndexToLetter`, `formatXlsxCell`, `buildSpreadsheetXml`, `XlsxExporter`
|
||||
- [x] 2.8 `src/services/export-service.ts` — `getExporter`, `inferColumns`, `runExport`, `DEFAULT_EXPORT_CONFIG`, `ExportServiceConfig`
|
||||
- [x] 2.9 `src/routes/exports.ts` — `createExportRoutes(deps)` mounting GET / POST / GET /:id / DELETE /:id
|
||||
- [x] 2.10 `src/plugin/plugin.ts` — `createExportPlugin`, `registerDataProvider`
|
||||
- [x] 2.11 `src/services/registry-augmentation.ts` — augment `ServiceRegistry` with `exportProviders`/`exportJobs`/`app`
|
||||
- [x] 2.12 `src/events/event-augmentation.ts` — augment `EventBusLike` event topics (`export:initialized`, etc.)
|
||||
- [x] 2.13 `src/index.ts` — explicit named re-exports of the full surface; subpath exports in package.json
|
||||
|
||||
## 3. Tests
|
||||
|
||||
- [x] 3.1 `src/exporters/csv/csv.test.ts`
|
||||
- [x] 3.2 `src/exporters/json/json.test.ts`
|
||||
- [x] 3.3 `src/exporters/xlsx/xlsx.test.ts`
|
||||
- [x] 3.4 `src/services/export-service.test.ts`
|
||||
- [x] 3.5 `src/routes/exports.test.ts` — full Hono route exercise via `app.request`
|
||||
- [x] 3.6 `src/plugin/plugin.test.ts` — lifecycle + service registration + event emission against a real `PluginContext` built from `createPluginContext`
|
||||
- [x] 3.7 `src/error-codes/error-codes.test.ts` — every code thrown from a real production path
|
||||
- [x] 3.8 `src/otel/instruments.test.ts` — real in-memory `MeterProvider`, assert counter + histogram emission
|
||||
- [x] 3.9 `src/api-plugin-export.spec-conformance.test.ts` — `expectTypeOf` on public surface
|
||||
- [x] 3.10 `src/schemas/schemas.test.ts` — Zod-boundary assertions
|
||||
- [x] 3.11 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for async)
|
||||
|
||||
## 4. Documentation
|
||||
|
||||
- [x] 4.1 Sibling `.md` for every implementation `.ts` (≥200 words, basic + advanced examples, API reference)
|
||||
- [x] 4.2 README links every co-located `.md` under `## Modules`
|
||||
|
||||
## 5. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [ ] 5.1 `bun run check` — Biome 0 errors / 0 warnings / 0 info
|
||||
- [ ] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/api-plugin-export` — green
|
||||
- [ ] 5.3 `bunx turbo test:coverage --filter=@suaveplan/api-plugin-export` — 100/100/100/100
|
||||
- [ ] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/api-plugin-export` — exit 0
|
||||
- [ ] 5.5 `bunx turbo build --filter=@suaveplan/api-plugin-export` — green
|
||||
|
||||
## 6. Pre-archive 9-row checklist
|
||||
|
||||
- [ ] 6.1 Generics threaded; `expectTypeOf` assertions on every literal-type claim
|
||||
- [ ] 6.2 Zod schemas wired at every declared boundary
|
||||
- [ ] 6.3 Every error code in `error-codes.ts` thrown by at least one production path
|
||||
- [ ] 6.4 Strict-mode behaviors tested
|
||||
- [ ] 6.5 OTel instruments emitted from production paths and asserted with real in-memory `MeterProvider`
|
||||
- [ ] 6.6 Lifecycle dedup tested
|
||||
- [ ] 6.7 No retroactive tick — every box above reflects work done in this change
|
||||
- [ ] 6.8 No `export *` in any feature-module barrel
|
||||
- [ ] 6.9 No direct `vitest`/`bun:test` imports in test files (all via `@suaveplan/testing/runner`)
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold spec delta into `openspec/specs/api-plugin-export/spec.md`
|
||||
- [ ] 7.2 Move `openspec/changes/add-api-plugin-export/` → `openspec/archive/2026-05-18-add-api-plugin-export/`
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-05-19
|
||||
131
openspec/changes/add-drive-storage-archive/design.md
Normal file
131
openspec/changes/add-drive-storage-archive/design.md
Normal file
@@ -0,0 +1,131 @@
|
||||
## Context
|
||||
|
||||
`@suaveplan/drive` consumes any object satisfying the structural `DriveStorage` view (read/write/delete/list/stat/streams + capabilities flags). Existing storage adapters (`storage-file`, `storage-http`, `storage-cdn-*`, `storage-browser-*`) all expose flat byte stores. None of them treat their content as a structured archive whose entries should themselves appear as paths in the drive.
|
||||
|
||||
Compressed archives have format-specific quirks that prevent a single monolithic adapter:
|
||||
|
||||
- **Container shape**: ZIP and 7z have a trailing central directory (random-access friendly); tar is purely sequential; gzip wraps a single stream with no directory at all.
|
||||
- **Compression coupling**: tar separates archiving from compression (`.tar.gz`, `.tar.xz`, `.tar.zst`); zip/7z couple them per-entry; rar couples them per-volume.
|
||||
- **Mutability**: zip and 7z support entry-level update via central-directory rewrite; tar requires full rewrite for deletes; gzip is single-stream and must be re-encoded entirely on any change; rar is patent-encumbered for writing.
|
||||
- **Splitting**: zip uses `.z01`..`.zNN`+`.zip`; rar uses `.partN.rar` (RAR5) or `.r00`..`.r99` (RAR4); 7z uses `.7z.NNN`. Each has a different sibling-discovery and ordering rule.
|
||||
- **Extension lies**: a `.zip` may be a 7z stream; a `.bin` may be tgz. The host can only trust magic bytes.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- One installable package per archive format (zip, tar, gz, tgz, rar, 7z) plus one shared `drive-storage-archive-core`. Each is a separately versioned library.
|
||||
- Each adapter implements `DriveStorage` so `new Drive(adapter)` works with zero drive-side changes.
|
||||
- Content-sniff format from magic bytes; extension is advisory only.
|
||||
- First-class split-archive support with sibling-volume resolution callback so the host storage (file, http, indexeddb) decides how to fetch siblings.
|
||||
- Three explicit modes: `read-only`, `copy-on-write` (in-memory overlay + `flush()`), `rewrite` (write-through via host storage).
|
||||
- Streaming reads (`ReadableStream<Uint8Array>`) for entries larger than memory.
|
||||
- Password/encryption hook so credentials never live in the adapter — host supplies via callback.
|
||||
- Conformance harness in `drive-storage-archive-core/testing` that every per-format adapter passes.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Writing RAR — patent-encumbered, deliberately read-only.
|
||||
- New compression formats outside the six listed (no `.lz4`, `.zst` standalone, `.bz2` standalone in this change — tgz codec slot will accept them later).
|
||||
- Replacing `drive-plugin-chunks` chunk addressing; archive entries are not chunked at the drive level.
|
||||
- Mounting archives inside archives transparently (nested mount is a host concern: `openArchiveStorage(openArchiveStorage(...))`).
|
||||
- A unified `@suaveplan/storage-archive` umbrella package. Each format ships standalone (matches user directive).
|
||||
|
||||
## Decisions
|
||||
|
||||
### One package per format, plus a shared core
|
||||
|
||||
**Choice:** Six per-format packages (`drive-storage-{zip,tar,gz,tgz,rar,7z}`) + `drive-storage-archive-core` for shared primitives.
|
||||
|
||||
**Alternative considered:** Single `drive-storage-archive` umbrella package with internal format modules.
|
||||
|
||||
**Rationale:** Per-format isolates large codec dependencies (LZMA SDK is ~150KB, RAR unpacker is server-only) and lets consumers depend only on what they need. Matches existing repo pattern (`storage-cdn-bunnycdn`, `storage-cdn-rackspace`, etc.). User explicitly requested separate libraries.
|
||||
|
||||
### Format detection via magic bytes, extension as hint only
|
||||
|
||||
**Choice:** `sniffFormat(bytes: Uint8Array): ArchiveFormat | null` in `drive-storage-archive-core` inspects the first 512 bytes against a magic-byte registry: `PK\x03\x04` (zip), `Rar!\x1A\x07\x00` / `Rar!\x1A\x07\x01\x00` (rar4/5), `7z\xBC\xAF\x27\x1C` (7z), `\x1F\x8B` (gzip), and content-based tar detection (block 0 offset 257 = `"ustar"` for POSIX, or full-block heuristic for legacy v7).
|
||||
|
||||
**Rationale:** Users routinely mislabel archives. Extension drives default codec selection but is overridden by sniff result with a logged `ARCHIVE_FORMAT_MISMATCH` warning.
|
||||
|
||||
### Split-volume resolution via sibling-resolver callback
|
||||
|
||||
**Choice:** `openArchiveStorage(source, { resolveSibling: (basename, index) => Promise<Uint8Array | ReadableStream> })`.
|
||||
|
||||
**Alternative considered:** Hard-code sibling lookup against the host storage path.
|
||||
|
||||
**Rationale:** Host storage might be HTTP (siblings on different URLs), IndexedDB (siblings in different rows), or a P2P drive (siblings discovered via gossip). The adapter never assumes a filesystem. Each format ships a `defaultResolver(hostStorage, primaryPath)` helper using its own naming convention (`name.z01..zNN`, `name.partN.rar`, `name.7z.NNN`).
|
||||
|
||||
### Three explicit mutation modes
|
||||
|
||||
**Choice:** `mode: 'read-only' | 'copy-on-write' | 'rewrite'`.
|
||||
|
||||
**Rationale:**
|
||||
|
||||
- `read-only` is the safe default. RAR forces this.
|
||||
- `copy-on-write` keeps the original archive bytes intact until `await adapter.flush()`. Mutations land in an in-memory overlay (entry → new bytes or tombstone). Suitable for batch edits.
|
||||
- `rewrite` writes through on every mutation by re-encoding the archive into the host storage. Suitable for occasional single edits. Each format documents the cost: O(archive size) per mutation.
|
||||
|
||||
### Streaming entries via Web Streams
|
||||
|
||||
**Choice:** `getEntryStream(path): ReadableStream<Uint8Array>` returns a Web Streams `ReadableStream`. Bulk reads via `getEntry(path): Promise<Uint8Array>` are a convenience wrapper.
|
||||
|
||||
**Rationale:** Archives often hold multi-GB entries (DB dumps, media). The `drive` `createReadStream`/`createWriteStream` operations already negotiate native vs. polyfill — this slots in cleanly.
|
||||
|
||||
### Codec abstraction for tgz family
|
||||
|
||||
**Choice:** `@suaveplan/drive-storage-tgz` accepts a `codec: ArchiveCodec` parameter. Ships with `gzipCodec` from `@suaveplan/drive-storage-gz`. Bzip2/xz/zstd codecs are pluggable (out of scope for this change; slot exists).
|
||||
|
||||
**Rationale:** Avoids forcing every consumer to install xz/zstd deps when they only need `.tar.gz`.
|
||||
|
||||
### Error taxonomy
|
||||
|
||||
**Choice:** Each package owns its `*_CODES` registry via `defineCodes` (`error` package convention). `drive-storage-archive-core` owns `ARCHIVE_FORMAT_UNKNOWN`, `ARCHIVE_FORMAT_MISMATCH`, `ARCHIVE_SPLIT_MISSING_VOLUME`, `ARCHIVE_PASSWORD_REQUIRED`, `ARCHIVE_PASSWORD_INVALID`, `ARCHIVE_MODE_READ_ONLY`, `ARCHIVE_FLUSH_REQUIRED`, `ARCHIVE_CORRUPT`, `ARCHIVE_UNSUPPORTED_FEATURE`, `ARCHIVE_CODEC_MISSING`. Per-format packages add format-specific codes (e.g., `ZIP_ZIP64_REQUIRED`, `TAR_PAX_HEADER_INVALID`, `RAR_VERSION_UNSUPPORTED`, `SEVENZ_BCJ_FILTER_UNSUPPORTED`).
|
||||
|
||||
### Universal vs. server tier
|
||||
|
||||
- `drive-storage-archive-core`, `drive-storage-zip`, `drive-storage-tar`, `drive-storage-gz`, `drive-storage-tgz` → **universal** (pure-TS codecs: `fflate` for deflate, hand-rolled tar block parser, pure-TS gzip via `fflate`).
|
||||
- `drive-storage-7z` → **universal** via wasm LZMA (preferred) or **server** via `lzma-native` binding (decided per perf test in tasks).
|
||||
- `drive-storage-rar` → **server-only** (no royalty-free pure-TS RAR5 unpacker exists with full filter support; uses `node-unrar-js` wasm or `unrar` native binding). Browser builds resolve to a stub that throws `RAR_UNAVAILABLE_IN_BROWSER`.
|
||||
|
||||
### Public factory shape
|
||||
|
||||
```ts
|
||||
import { openArchiveStorage } from "@suaveplan/drive-storage-archive-core";
|
||||
|
||||
const archiveStorage = await openArchiveStorage(hostBytes, {
|
||||
format: "auto", // 'auto' | 'zip' | 'tar' | ...
|
||||
mode: "copy-on-write",
|
||||
password: async () => promptUser(),
|
||||
resolveSibling: async (basename, index) => hostStorage.read(`${basename}.z${String(index).padStart(2, "0")}`),
|
||||
});
|
||||
|
||||
const drive = new Drive(archiveStorage);
|
||||
await drive.initialize();
|
||||
await drive.write("inside-archive/notes.txt", new TextEncoder().encode("hello"));
|
||||
await archiveStorage.flush(); // for copy-on-write
|
||||
```
|
||||
|
||||
The factory does the sniff, picks the per-format adapter via dynamic import (so unused codecs aren't bundled), and returns the constructed `DriveStorage`. Each per-format package also exports its constructor directly (`ZipArchiveStorage`, etc.) for callers who don't need sniffing.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **RAR licensing** → Mitigation: read-only, vendored via permissive-license wasm unrar; license note in package README. Block writes at type level (`mode: 'read-only'` only).
|
||||
- **Rewrite cost on tar/gz** → Mitigation: document O(archive size) cost in README; recommend `copy-on-write` + single `flush()` for batches; expose `pendingMutationCount` getter.
|
||||
- **Password-encrypted entries leak metadata** → Mitigation: document that entry names + sizes are unencrypted in zip (by spec) and 7z header-encryption mode is supported as opt-in. RAR encrypted-headers supported.
|
||||
- **Split-volume ordering ambiguity** → Mitigation: each format's `defaultResolver` enforces canonical ordering (`.z01..zNN..zip` final; `.partN.rar` ascending; `.7z.001..NNN`). Adapter validates volume signatures match before assembly; mismatched volumes → `ARCHIVE_SPLIT_MISSING_VOLUME` with which-volume detail.
|
||||
- **Browser memory pressure on large archives** → Mitigation: lazy entry parsing (central directory only on open); stream-only API for >100MB entries; warn via logger if archive exceeds a configurable threshold.
|
||||
- **Format sniff false positives** → Mitigation: magic-byte match plus structural validation (zip → can locate EOCD; tar → header checksum matches; gz → trailing CRC matches). Sniff returns `null` if structural check fails, falling back to extension.
|
||||
- **Nested archives** → Mitigation: not solved here. Host composes adapters explicitly; documented as a recipe in `drive-storage-archive-core` README.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
- New packages — no existing consumer migration required.
|
||||
- Each package ships at `0.1.0`.
|
||||
- Drive consumers gain archive support opportunistically by passing an archive storage where they previously passed a flat storage.
|
||||
- Rollback: each package is independently removable. No drive-side changes mean no rollback risk to existing storage adapters.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- 7z wasm LZMA performance vs. native: settle in tasks via benchmark; if wasm <50% of native throughput on a 100MB archive, ship server-only and provide a wasm browser variant later.
|
||||
- Whether `drive-storage-archive-core` should expose a `suaveplan-archive` CLI bin (list/extract/test) or whether that belongs in a separate `archive-cli` package. Default: defer CLI to a follow-up change to keep this scope tight.
|
||||
- ZIP AES encryption (WinZip AE-2) — implement now or follow-up? Default: implement read in v0.1.0, write in v0.2.0.
|
||||
41
openspec/changes/add-drive-storage-archive/proposal.md
Normal file
41
openspec/changes/add-drive-storage-archive/proposal.md
Normal file
@@ -0,0 +1,41 @@
|
||||
## Why
|
||||
|
||||
The `drive` VFS already abstracts file operations behind a structural `DriveStorage` view, but there is no way to mount a compressed archive (zip, tar, gz, tgz, rar, 7z) as a drive. Users routinely receive bundles whose extension lies about the real compression (a `.zip` that is actually a 7z stream, a `.tar.gz` renamed `.bin`) or whose contents are split across multi-volume sets (`.part1.rar`/`.r00`–`.r99`, `.7z.001`–`.7z.NNN`, multi-file split-zip). Treating those archives as first-class drives lets every existing drive consumer (sharing, hydrate-from, file-operations, plugins) browse, read, and (where the format permits) write archive contents without ad-hoc extraction.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a shared `@suaveplan/drive-storage-archive-core` package: format sniffing by magic bytes (extension is advisory only), split-volume reassembly, central-directory caching, password/encryption hook contract, and a streaming `EntryReader` abstraction so per-format adapters only implement the format-specific decode path.
|
||||
- Add six per-format storage adapters, each implementing the `DriveStorage` contract over an underlying `AbstractStorage` (or raw byte source) holding the archive bytes:
|
||||
- `@suaveplan/drive-storage-zip` — ZIP / ZIP64 / split-zip (`.z01`–`.zNN` + `.zip`), read-write with rewrite-on-flush.
|
||||
- `@suaveplan/drive-storage-tar` — POSIX/USTAR/PAX, sparse files, long-name extensions; read-write (append + rewrite on delete).
|
||||
- `@suaveplan/drive-storage-gz` — single-stream gzip wrapping one logical entry; read-write.
|
||||
- `@suaveplan/drive-storage-tgz` — composition of `tar` over `gz` (and `.tar.bz2`, `.tar.xz`, `.tar.zst` via pluggable codec); read-mostly, write via full rewrite.
|
||||
- `@suaveplan/drive-storage-rar` — RAR4 + RAR5 + multi-volume (`.partN.rar`, legacy `.r00`–`.r99`); **read-only** (writing RAR is patent-encumbered).
|
||||
- `@suaveplan/drive-storage-7z` — 7z LZMA/LZMA2/BCJ filters + multi-volume (`.7z.001`–`.7z.NNN`); read-write via rewrite-on-flush.
|
||||
- Public factory `openArchiveStorage(source, { format?, password?, splitParts?, mode? })` that auto-detects format from magic bytes, discovers split volumes through a sibling-resolver callback on the host storage, and returns a `DriveStorage` ready to hand to `new Drive(...)`.
|
||||
- Mode flags: `read-only` | `copy-on-write` (in-memory overlay, materialise on `flush()`) | `rewrite` (write directly back through host storage on each mutation). RAR is locked to `read-only`.
|
||||
- Error codes namespaced per package (`ARCHIVE_*`, `ZIP_*`, `TAR_*`, `GZ_*`, `RAR_*`, `SEVENZ_*`) registered in each package's error registry.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `drive-storage-archive-core`: shared format sniffing, split-volume reassembly, mode/flush contract, password hook, codec registry, error taxonomy, and conformance harness all per-format adapters run against.
|
||||
- `drive-storage-zip`: ZIP/ZIP64/split-zip adapter exposing archive entries as a `DriveStorage`.
|
||||
- `drive-storage-tar`: POSIX/USTAR/PAX tar adapter exposing tar entries as a `DriveStorage`.
|
||||
- `drive-storage-gz`: single-entry gzip adapter exposing the wrapped stream as a `DriveStorage`.
|
||||
- `drive-storage-tgz`: composed tar-over-codec adapter (gzip/bzip2/xz/zstd) exposing entries as a `DriveStorage`.
|
||||
- `drive-storage-rar`: read-only RAR4/RAR5 + multi-volume adapter exposing rar entries as a `DriveStorage`.
|
||||
- `drive-storage-7z`: 7z + multi-volume adapter exposing 7z entries as a `DriveStorage`.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
_None — drive's `DriveStorage` view is already satisfied; no requirement changes to existing specs._
|
||||
|
||||
## Impact
|
||||
|
||||
- New packages under `packages/drive/drive-storage-{zip,tar,gz,tgz,rar,7z}` plus `packages/drive/drive-storage-archive-core` (seven total). All universal-tier where the codec is pure-TS, server-tier where native bindings are needed (RAR, optionally 7z LZMA SDK).
|
||||
- New dependencies on pure-TS codec libraries via Bun catalog (`fflate` for deflate, `pako` fallback, `lzma-native` or wasm `lzma`, `xz-decompress`, `@zip.js/zip.js` is **not** used — we implement directly for streaming + split support).
|
||||
- Drive consumers (`drive-sharing`, `drive-template`, `drive-react-sharing`) gain transparent archive browsing with no API changes.
|
||||
- Bin: optional `suaveplan-archive` CLI in `drive-storage-archive-core` for listing/extracting any supported archive from the shell. (Decision deferred to design.md.)
|
||||
- Test surface: `runArchiveConformance` harness in `drive-storage-archive-core/testing` exercised by all six adapters.
|
||||
@@ -0,0 +1,65 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: 7z adapter exposes 7z entries as DriveStorage
|
||||
|
||||
The package SHALL export `SevenZipArchiveStorage` implementing the structural `DriveStorage` view. Entry paths inside the archive MUST appear as drive paths with directory separators normalised to `/`.
|
||||
|
||||
#### Scenario: Drive lists 7z entries
|
||||
|
||||
- **WHEN** a 7z archive containing two entries is wrapped and `drive.list()` is called
|
||||
- **THEN** both entry paths are returned
|
||||
|
||||
### Requirement: LZMA, LZMA2, and BCJ filter support
|
||||
|
||||
The adapter SHALL support LZMA and LZMA2 compression codecs and the BCJ, BCJ2, Delta, and ARM/X86/SPARC preprocessor filters as composed in 7z `Folder` records. Unsupported codec IDs MUST raise `SEVENZ_UNSUPPORTED_CODEC`. Unsupported filter chains MUST raise `SEVENZ_BCJ_FILTER_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: LZMA2 + BCJ executable decodes
|
||||
|
||||
- **WHEN** a 7z archive whose folder chain is LZMA2 → BCJ over an executable entry is read
|
||||
- **THEN** the decoded bytes match the original executable
|
||||
|
||||
#### Scenario: Unknown codec errors clearly
|
||||
|
||||
- **WHEN** an archive references a codec ID not implemented by the adapter
|
||||
- **THEN** the read throws `SEVENZ_UNSUPPORTED_CODEC` with the codec ID in details
|
||||
|
||||
### Requirement: Multi-volume support
|
||||
|
||||
The adapter SHALL support multi-volume 7z archives using `.7z.001`, `.7z.002`, …, `.7z.NNN` naming. Sibling discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. The adapter MUST concatenate volumes in ascending index order before parsing the trailing header.
|
||||
|
||||
#### Scenario: Three-volume 7z assembles
|
||||
|
||||
- **WHEN** the primary `archive.7z.001` is opened with a `resolveSibling` returning `archive.7z.002` and `archive.7z.003`
|
||||
- **THEN** all entries are listed and read
|
||||
|
||||
#### Scenario: Out-of-order volume rejected
|
||||
|
||||
- **WHEN** `resolveSibling` returns volume 1 at index 2 (i.e. mis-ordered)
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` with both expected and received indices
|
||||
|
||||
### Requirement: Header encryption and per-entry encryption
|
||||
|
||||
The adapter SHALL support reading 7z archives with encrypted headers (`7zAES` over the `EncodedHeader` block) and per-entry-encrypted content. The password hook from `drive-storage-archive-core` MUST be invoked lazily on first decryption need.
|
||||
|
||||
#### Scenario: Encrypted-header archive opens
|
||||
|
||||
- **WHEN** a 7z archive with encrypted headers is opened with a correct-password hook
|
||||
- **THEN** entry listing and reading succeed
|
||||
|
||||
### Requirement: Read-write with copy-on-write or rewrite
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Writes MUST produce a standards-compliant 7z file readable by reference 7-Zip implementations. Mutations re-encode the affected folder(s) and rewrite the trailing header.
|
||||
|
||||
#### Scenario: Round-trip write then read
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes `inside/n.txt`, flushes, then a new adapter is opened over the flushed bytes
|
||||
- **THEN** `inside/n.txt` is listed and reads with the original content
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `SEVENZ_HEADER_INVALID`, `SEVENZ_UNSUPPORTED_CODEC`, `SEVENZ_BCJ_FILTER_UNSUPPORTED`, `SEVENZ_CRC_MISMATCH`, `SEVENZ_CORRUPT`.
|
||||
|
||||
#### Scenario: Bad signature errors
|
||||
|
||||
- **WHEN** the host bytes do not begin with `7z\xBC\xAF\x27\x1C`
|
||||
- **THEN** open throws `SEVENZ_HEADER_INVALID`
|
||||
@@ -0,0 +1,95 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Format detection from magic bytes
|
||||
|
||||
The package SHALL expose a `sniffFormat(bytes: Uint8Array): ArchiveFormat | null` function that identifies archive format by inspecting magic bytes in the first 512 bytes, treating any file-name extension as advisory only. Detection MUST cover ZIP, RAR4, RAR5, 7z, GZIP, and POSIX/USTAR tar. When magic bytes match a known format, a structural validation step MUST also succeed before the format is returned.
|
||||
|
||||
#### Scenario: Sniff matches when extension lies
|
||||
|
||||
- **WHEN** `sniffFormat` is called with bytes whose first four bytes are `PK\x03\x04` but whose source filename was `bundle.bin`
|
||||
- **THEN** the function returns `"zip"` and the source filename is not consulted
|
||||
|
||||
#### Scenario: Mismatch between extension and content emits warning
|
||||
|
||||
- **WHEN** `openArchiveStorage` is called with a source named `archive.zip` whose magic bytes match `7z\xBC\xAF\x27\x1C`
|
||||
- **THEN** an `ARCHIVE_FORMAT_MISMATCH` warning is logged with both detected and declared formats, and the detected format is used
|
||||
|
||||
#### Scenario: Unknown bytes return null
|
||||
|
||||
- **WHEN** `sniffFormat` is called with bytes matching no known magic-byte signature and no structural fallback succeeds
|
||||
- **THEN** the function returns `null`
|
||||
|
||||
### Requirement: openArchiveStorage factory
|
||||
|
||||
The package SHALL expose `openArchiveStorage(source, options)` that returns a `DriveStorage` view backed by the appropriate per-format adapter. Options MUST include `format`, `mode`, `password`, and `resolveSibling`. The factory MUST dynamic-import the per-format adapter so unused format codecs are not bundled into consumers that do not need them.
|
||||
|
||||
#### Scenario: Auto-detect and construct adapter
|
||||
|
||||
- **WHEN** `openArchiveStorage(bytes, { format: "auto", mode: "read-only" })` is called with valid zip bytes
|
||||
- **THEN** the function returns a `DriveStorage` instance from `@suaveplan/drive-storage-zip` constructed in read-only mode
|
||||
|
||||
#### Scenario: Explicit format overrides sniff
|
||||
|
||||
- **WHEN** `openArchiveStorage(bytes, { format: "tar" })` is called
|
||||
- **THEN** the factory loads the tar adapter without invoking `sniffFormat`
|
||||
|
||||
### Requirement: Split-volume sibling resolution
|
||||
|
||||
The package SHALL define a `resolveSibling(basename: string, index: number): Promise<Uint8Array | ReadableStream<Uint8Array>>` callback contract that per-format adapters use to fetch additional volumes. The host application owns sibling discovery; the adapter never assumes a filesystem layout.
|
||||
|
||||
#### Scenario: Multi-volume archive triggers sibling fetch
|
||||
|
||||
- **WHEN** a zip adapter opens a primary volume whose EOCD references volumes 0..3
|
||||
- **THEN** the adapter invokes `resolveSibling(basename, 0)`, `resolveSibling(basename, 1)`, `resolveSibling(basename, 2)` and uses the returned bytes in volume order
|
||||
|
||||
#### Scenario: Missing sibling raises typed error
|
||||
|
||||
- **WHEN** `resolveSibling` rejects or returns `null` for a required volume
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` whose details include the missing volume index and expected basename
|
||||
|
||||
### Requirement: Mutation mode contract
|
||||
|
||||
The package SHALL define three modes — `read-only`, `copy-on-write`, `rewrite` — and a `flush(): Promise<void>` method on the returned `DriveStorage`. In `read-only` mode any write/delete operation MUST throw `ARCHIVE_MODE_READ_ONLY`. In `copy-on-write` mode mutations MUST land in an in-memory overlay and be materialised to host storage only on `flush()`. In `rewrite` mode each mutation MUST re-encode the archive into host storage before returning.
|
||||
|
||||
#### Scenario: Write in read-only mode throws
|
||||
|
||||
- **WHEN** any mutation is attempted on an adapter constructed with `mode: "read-only"`
|
||||
- **THEN** the call throws `ARCHIVE_MODE_READ_ONLY` and the host archive bytes are unchanged
|
||||
|
||||
#### Scenario: Copy-on-write defers materialisation
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode receives a write, the host archive bytes have not changed
|
||||
- **AND WHEN** `flush()` is awaited
|
||||
- **THEN** the host archive bytes contain the new entry and `flush()` is idempotent for a no-op state
|
||||
|
||||
### Requirement: Password hook contract
|
||||
|
||||
The package SHALL accept `password: () => Promise<string>` and invoke it lazily — only when an encrypted entry is encountered. Passwords MUST NOT be stored on the adapter instance beyond the lifetime of the active decrypt operation. An incorrect password MUST raise `ARCHIVE_PASSWORD_INVALID`; a missing hook on encrypted content MUST raise `ARCHIVE_PASSWORD_REQUIRED`.
|
||||
|
||||
#### Scenario: Password requested only on encrypted read
|
||||
|
||||
- **WHEN** an archive contains both unencrypted and encrypted entries and the consumer reads only unencrypted entries
|
||||
- **THEN** the `password` hook is never invoked
|
||||
|
||||
#### Scenario: Wrong password reported
|
||||
|
||||
- **WHEN** the `password` hook returns a value that fails authenticated decryption on an encrypted entry
|
||||
- **THEN** the read throws `ARCHIVE_PASSWORD_INVALID` and the password value is not retained
|
||||
|
||||
### Requirement: Conformance harness
|
||||
|
||||
The package SHALL expose a `runArchiveConformance(adapterFactory, opts)` test harness via the `./testing` subpath export that exercises a fixed set of behaviours every per-format adapter MUST satisfy: open, list, read, stat, password-failure path, mode enforcement, split-volume reassembly (where applicable), and round-trip of one binary entry through write/flush/read (where the format permits writes).
|
||||
|
||||
#### Scenario: Per-format package consumes harness
|
||||
|
||||
- **WHEN** a per-format package's test suite calls `runArchiveConformance(() => new ZipArchiveStorage(...))`
|
||||
- **THEN** the harness executes the full behaviour set and any failure surfaces as a typed assertion identifying the failed behaviour
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register the following stable error codes via `defineCodes` from `@suaveplan/error`: `ARCHIVE_FORMAT_UNKNOWN`, `ARCHIVE_FORMAT_MISMATCH`, `ARCHIVE_SPLIT_MISSING_VOLUME`, `ARCHIVE_PASSWORD_REQUIRED`, `ARCHIVE_PASSWORD_INVALID`, `ARCHIVE_MODE_READ_ONLY`, `ARCHIVE_FLUSH_REQUIRED`, `ARCHIVE_CORRUPT`, `ARCHIVE_UNSUPPORTED_FEATURE`, `ARCHIVE_CODEC_MISSING`. Each MUST have a stable string identifier never reused after removal.
|
||||
|
||||
#### Scenario: Code identifiers stable across releases
|
||||
|
||||
- **WHEN** the package is published at any version `0.x.y`
|
||||
- **THEN** every code in the registry resolves to the same string identifier as in version `0.1.0`
|
||||
@@ -0,0 +1,60 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: GZ adapter exposes the wrapped stream as one drive entry
|
||||
|
||||
The package SHALL export `GzipArchiveStorage` implementing the structural `DriveStorage` view. Because gzip wraps a single logical stream, the adapter MUST present exactly one entry whose path defaults to the original filename from the gzip `FNAME` field, or to a configurable fallback path when `FNAME` is absent.
|
||||
|
||||
#### Scenario: FNAME header used as entry path
|
||||
|
||||
- **WHEN** a gzip stream whose `FNAME` field equals `report.csv` is wrapped in `GzipArchiveStorage`
|
||||
- **THEN** `drive.list()` returns exactly one entry at `report.csv`
|
||||
|
||||
#### Scenario: Fallback path used when FNAME absent
|
||||
|
||||
- **WHEN** a gzip stream without an `FNAME` field is opened with `fallbackPath: "data.bin"`
|
||||
- **THEN** the single entry appears at `data.bin`
|
||||
|
||||
### Requirement: Multi-member gzip streams
|
||||
|
||||
The adapter SHALL transparently concatenate sequential gzip members (per RFC 1952 §2.2) when present, returning their decompressed bytes as a single logical entry stream.
|
||||
|
||||
#### Scenario: Two concatenated members read as one stream
|
||||
|
||||
- **WHEN** a gzip file containing two valid members each holding half of a logical file is read
|
||||
- **THEN** the returned stream contains the concatenation of both members' decompressed bytes
|
||||
|
||||
### Requirement: Read-write with full re-encode
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Because gzip is a single stream, any write MUST fully re-encode the host bytes. The adapter MUST document this cost.
|
||||
|
||||
#### Scenario: Write replaces entire host bytes
|
||||
|
||||
- **WHEN** an adapter in `rewrite` mode receives a write to the single entry
|
||||
- **THEN** the host bytes are replaced by a freshly encoded gzip stream containing the new content
|
||||
|
||||
### Requirement: CRC32 and ISIZE validation
|
||||
|
||||
The adapter SHALL validate the trailing CRC32 and ISIZE fields on read. A mismatch MUST raise `GZ_CRC_MISMATCH` or `GZ_ISIZE_MISMATCH` respectively.
|
||||
|
||||
#### Scenario: Truncated stream errors
|
||||
|
||||
- **WHEN** the host bytes end before a valid CRC32+ISIZE trailer
|
||||
- **THEN** the read throws `GZ_TRUNCATED`
|
||||
|
||||
### Requirement: Codec export for tgz composition
|
||||
|
||||
The package SHALL export a `gzipCodec: ArchiveCodec` object that `@suaveplan/drive-storage-tgz` composes over a tar adapter. The codec MUST expose `compress(stream)` and `decompress(stream)` returning `ReadableStream<Uint8Array>`.
|
||||
|
||||
#### Scenario: tgz consumes gzipCodec
|
||||
|
||||
- **WHEN** `@suaveplan/drive-storage-tgz` constructs an adapter with `codec: gzipCodec`
|
||||
- **THEN** decompression of a `.tar.gz` host file produces the same entry listing as decompressing the gz then opening the tar
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `GZ_HEADER_INVALID`, `GZ_CRC_MISMATCH`, `GZ_ISIZE_MISMATCH`, `GZ_TRUNCATED`, `GZ_UNSUPPORTED_FLAG`.
|
||||
|
||||
#### Scenario: Bad gzip magic errors
|
||||
|
||||
- **WHEN** the first two bytes are not `\x1F\x8B`
|
||||
- **THEN** the open throws `GZ_HEADER_INVALID`
|
||||
@@ -0,0 +1,75 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: RAR adapter exposes rar entries as DriveStorage (read-only)
|
||||
|
||||
The package SHALL export `RarArchiveStorage` implementing the structural `DriveStorage` view in read-only mode only. Construction with any mode other than `read-only` MUST throw `ARCHIVE_MODE_READ_ONLY` immediately.
|
||||
|
||||
#### Scenario: Construction rejects writable mode
|
||||
|
||||
- **WHEN** `new RarArchiveStorage(bytes, { mode: "rewrite" })` is called
|
||||
- **THEN** the constructor throws `ARCHIVE_MODE_READ_ONLY` and does not allocate any decoder state
|
||||
|
||||
#### Scenario: Drive lists rar entries
|
||||
|
||||
- **WHEN** a RAR5 archive containing two entries is wrapped and `drive.list()` is called
|
||||
- **THEN** both entry paths are returned
|
||||
|
||||
### Requirement: RAR4 and RAR5 format support
|
||||
|
||||
The adapter SHALL parse both RAR4 (signature `Rar!\x1A\x07\x00`) and RAR5 (signature `Rar!\x1A\x07\x01\x00`) container formats. Format detection MUST happen via magic bytes, and unsupported RAR3-or-older signatures MUST raise `RAR_VERSION_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: RAR4 archive opens
|
||||
|
||||
- **WHEN** a RAR4 archive is opened
|
||||
- **THEN** entries are listed correctly
|
||||
|
||||
#### Scenario: RAR5 archive opens
|
||||
|
||||
- **WHEN** a RAR5 archive is opened
|
||||
- **THEN** entries are listed correctly
|
||||
|
||||
### Requirement: Multi-volume support
|
||||
|
||||
The adapter SHALL support RAR5 multi-volume archives using `.partN.rar` naming (e.g. `name.part1.rar`, `name.part2.rar`) and legacy RAR4 multi-volume using `.r00`, `.r01`, …, `.r99` extensions. Sibling discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. Each subsequent volume's signature and volume number MUST validate before its bytes are consumed.
|
||||
|
||||
#### Scenario: RAR5 partN volumes assemble
|
||||
|
||||
- **WHEN** the primary `archive.part1.rar` is opened with a `resolveSibling` that returns `archive.part2.rar` and `archive.part3.rar`
|
||||
- **THEN** entries spanning all three volumes are listed and read
|
||||
|
||||
#### Scenario: RAR4 r00..r02 volumes assemble
|
||||
|
||||
- **WHEN** the primary `archive.rar` is opened with a `resolveSibling` returning `archive.r00`, `archive.r01`
|
||||
- **THEN** entries spanning all volumes are read
|
||||
|
||||
#### Scenario: Volume sequence gap errors
|
||||
|
||||
- **WHEN** `resolveSibling` returns volume 2 with a header indicating volume number 3
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME`
|
||||
|
||||
### Requirement: Encrypted header and encrypted entry support
|
||||
|
||||
The adapter SHALL support reading entries protected by per-entry passwords and full encrypted-header archives. The password hook from `drive-storage-archive-core` MUST be invoked lazily on first encrypted-content access.
|
||||
|
||||
#### Scenario: Encrypted-header archive
|
||||
|
||||
- **WHEN** an archive with encrypted headers is opened and the password hook returns the correct passphrase
|
||||
- **THEN** entry listing succeeds
|
||||
|
||||
### Requirement: Browser builds raise typed error
|
||||
|
||||
The package SHALL be server-tier. The `browser` field in `package.json` MUST resolve to a stub module whose entry points throw `RAR_UNAVAILABLE_IN_BROWSER` on any call.
|
||||
|
||||
#### Scenario: Browser bundle throws on use
|
||||
|
||||
- **WHEN** a browser bundle that imports `RarArchiveStorage` calls the constructor
|
||||
- **THEN** it throws `RAR_UNAVAILABLE_IN_BROWSER`
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `RAR_VERSION_UNSUPPORTED`, `RAR_FILTER_UNSUPPORTED`, `RAR_UNAVAILABLE_IN_BROWSER`, `RAR_CORRUPT`, `RAR_CRC_MISMATCH`.
|
||||
|
||||
#### Scenario: CRC mismatch surfaces
|
||||
|
||||
- **WHEN** an entry's stored CRC does not match the decoded bytes
|
||||
- **THEN** the read throws `RAR_CRC_MISMATCH` with the entry path in details
|
||||
@@ -0,0 +1,56 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TAR adapter exposes tar entries as DriveStorage
|
||||
|
||||
The package SHALL export `TarArchiveStorage` implementing the structural `DriveStorage` view. Entry paths inside the tar MUST appear as drive paths with directory separators normalised to `/`.
|
||||
|
||||
#### Scenario: Drive lists tar entries
|
||||
|
||||
- **WHEN** a tar archive containing `etc/conf` and `etc/hosts` is wrapped in `TarArchiveStorage` and `drive.list("etc/")` is called
|
||||
- **THEN** the result contains both `etc/conf` and `etc/hosts`
|
||||
|
||||
### Requirement: POSIX, USTAR, and PAX header support
|
||||
|
||||
The adapter SHALL parse legacy v7, USTAR, and POSIX.1-2001 PAX extended headers. Long filenames exceeding 100 bytes MUST be reconstructed from PAX `path` records or GNU `L` long-link entries. PAX size records MUST override the 12-octet octal size field.
|
||||
|
||||
#### Scenario: PAX long filename round-trips
|
||||
|
||||
- **WHEN** a tar entry whose path is 300 bytes long is read
|
||||
- **THEN** the entry path is returned unchanged from `list()` and `stat()`
|
||||
|
||||
#### Scenario: PAX size record honoured
|
||||
|
||||
- **WHEN** an entry's PAX `size` record specifies a value larger than 8 GiB
|
||||
- **THEN** the adapter reads the full byte count without truncation
|
||||
|
||||
### Requirement: Sparse file extension
|
||||
|
||||
The adapter SHALL parse GNU sparse 1.0 extension records and present sparse regions as zero bytes in the reconstructed entry stream. Sparse 0.0/0.1 variants MAY be supported but MUST raise `TAR_SPARSE_VARIANT_UNSUPPORTED` if unsupported rather than silently mis-reading.
|
||||
|
||||
#### Scenario: Sparse 1.0 entry reads as zero-filled
|
||||
|
||||
- **WHEN** a sparse 1.0 entry with one data region of 1 KiB and one hole of 1 GiB is read
|
||||
- **THEN** the returned stream contains the data region followed by 1 GiB of zero bytes
|
||||
|
||||
### Requirement: Read-write with append and rewrite
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Tar's sequential format means deletes require full rewrite; the adapter MUST document this cost and expose `pendingMutationCount`. Pure-append writes MAY append blocks before the two-block zero terminator and rewrite only the terminator.
|
||||
|
||||
#### Scenario: Append avoids full rewrite
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes one new entry and calls `flush()`
|
||||
- **THEN** the host bytes are appended in place plus a new zero terminator, not fully re-encoded
|
||||
|
||||
#### Scenario: Delete triggers full rewrite
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode deletes one existing entry and calls `flush()`
|
||||
- **THEN** `flush()` writes a complete new archive omitting the deleted entry
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `TAR_HEADER_CHECKSUM_INVALID`, `TAR_PAX_HEADER_INVALID`, `TAR_SPARSE_VARIANT_UNSUPPORTED`, `TAR_BLOCK_ALIGNMENT_INVALID`.
|
||||
|
||||
#### Scenario: Bad header checksum errors
|
||||
|
||||
- **WHEN** a tar header's checksum field does not match the sum of its octets
|
||||
- **THEN** the read throws `TAR_HEADER_CHECKSUM_INVALID` with the byte offset in details
|
||||
@@ -0,0 +1,42 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TGZ adapter composes tar over a codec
|
||||
|
||||
The package SHALL export `TgzArchiveStorage` implementing the structural `DriveStorage` view by composing a tar adapter over a pluggable codec. The default constructor MUST accept `codec: ArchiveCodec` and SHALL ship a convenience `openTarGz(source, opts)` that defaults to `gzipCodec` from `@suaveplan/drive-storage-gz`.
|
||||
|
||||
#### Scenario: Default tar.gz opens with gzip codec
|
||||
|
||||
- **WHEN** `openTarGz(bytes, { mode: "read-only" })` is called on valid `.tar.gz` bytes
|
||||
- **THEN** entries from the inner tar are listed and read correctly
|
||||
|
||||
### Requirement: Codec slot for tar.bz2, tar.xz, tar.zst
|
||||
|
||||
The package SHALL accept any object satisfying the `ArchiveCodec` interface (`compress`, `decompress`). Codecs for bzip2, xz, and zstd are NOT shipped in v0.1.0 but the slot MUST be functional so consumers can plug them in. Construction with no codec MUST raise `ARCHIVE_CODEC_MISSING`.
|
||||
|
||||
#### Scenario: Custom xz codec works
|
||||
|
||||
- **WHEN** `new TgzArchiveStorage(bytes, { codec: customXzCodec })` is constructed and the bytes are a valid `.tar.xz`
|
||||
- **THEN** entries are listed and read correctly using the custom codec's `decompress`
|
||||
|
||||
#### Scenario: Missing codec rejects clearly
|
||||
|
||||
- **WHEN** `new TgzArchiveStorage(bytes, {})` is constructed without a codec
|
||||
- **THEN** construction throws `ARCHIVE_CODEC_MISSING`
|
||||
|
||||
### Requirement: Read-mostly with full re-encode on write
|
||||
|
||||
The adapter SHALL support `read-only`, `copy-on-write`, and `rewrite` modes. Writes MUST re-encode the full tar and then the full codec output. The cost MUST be documented and `pendingMutationCount` exposed.
|
||||
|
||||
#### Scenario: Round-trip write
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode adds one entry and calls `flush()`
|
||||
- **THEN** the host bytes contain a valid codec-wrapped tar whose entry list contains the new entry
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `TGZ_CODEC_FAILED`, `TGZ_INNER_TAR_INVALID`. Format-specific errors from the underlying tar and codec packages MUST be propagated unchanged.
|
||||
|
||||
#### Scenario: Inner tar corruption surfaces
|
||||
|
||||
- **WHEN** the codec decompresses successfully but the resulting tar bytes fail header checksum validation
|
||||
- **THEN** the error chain contains both `TGZ_INNER_TAR_INVALID` and the underlying `TAR_HEADER_CHECKSUM_INVALID`
|
||||
@@ -0,0 +1,60 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: ZIP adapter exposes archive entries as DriveStorage
|
||||
|
||||
The package SHALL export `ZipArchiveStorage` implementing the structural `DriveStorage` view so that `new Drive(new ZipArchiveStorage(...))` works without drive-side changes. Entry paths inside the archive MUST appear as drive paths preserving the archive's internal directory separators (normalised to `/`).
|
||||
|
||||
#### Scenario: Drive lists archive entries
|
||||
|
||||
- **WHEN** a zip archive containing `docs/a.txt` and `docs/b.txt` is wrapped in `ZipArchiveStorage` and handed to `new Drive(...)` and `drive.list("docs/")` is called
|
||||
- **THEN** the result contains both `docs/a.txt` and `docs/b.txt`
|
||||
|
||||
### Requirement: ZIP64 support
|
||||
|
||||
The adapter SHALL transparently parse ZIP64 central directory extensions when the EOCD `total entries`, `central directory size`, or `central directory offset` fields equal their 32-bit sentinel value `0xFFFFFFFF`. ZIP64 archives MUST be read without consumer opt-in.
|
||||
|
||||
#### Scenario: ZIP64 archive opens
|
||||
|
||||
- **WHEN** an archive larger than 4 GiB with valid ZIP64 EOCD locator is opened
|
||||
- **THEN** all entries are listed and read without error
|
||||
|
||||
### Requirement: Split-zip volume reassembly
|
||||
|
||||
The adapter SHALL support split-zip archives whose volumes follow the canonical naming `name.z01`, `name.z02`, …, `name.zNN`, `name.zip` (final volume holds EOCD). Volume discovery MUST use the `resolveSibling` callback from `drive-storage-archive-core`. Volume signatures MUST be validated; mismatched volumes raise `ARCHIVE_SPLIT_MISSING_VOLUME`.
|
||||
|
||||
#### Scenario: Three-volume zip opens via sibling resolver
|
||||
|
||||
- **WHEN** the primary `archive.zip` is opened with a `resolveSibling` callback that returns `archive.z01` and `archive.z02` for indices 0 and 1
|
||||
- **THEN** the adapter lists entries spanning all three volumes and reads each correctly
|
||||
|
||||
#### Scenario: Missing middle volume errors clearly
|
||||
|
||||
- **WHEN** `resolveSibling` returns `null` for volume index 1
|
||||
- **THEN** the adapter throws `ARCHIVE_SPLIT_MISSING_VOLUME` with `{ basename: "archive", missingIndex: 1 }` in the error details
|
||||
|
||||
### Requirement: Read-write with copy-on-write or rewrite mode
|
||||
|
||||
The adapter SHALL implement `read-only`, `copy-on-write`, and `rewrite` modes. In write modes the adapter MUST rebuild a valid EOCD on `flush()` (copy-on-write) or after each mutation (rewrite). The host archive bytes MUST remain readable by any standards-compliant zip tool after flush.
|
||||
|
||||
#### Scenario: Round-trip write then read
|
||||
|
||||
- **WHEN** an adapter in `copy-on-write` mode writes `inside/notes.txt`, calls `flush()`, then a new adapter is opened over the flushed bytes
|
||||
- **THEN** `inside/notes.txt` is listed and read with the original byte content
|
||||
|
||||
### Requirement: AES-encrypted entries
|
||||
|
||||
The adapter SHALL support reading WinZip AE-2 AES-256 encrypted entries using the password hook from `drive-storage-archive-core`. Writing AES-encrypted entries is out of scope for v0.1.0 and SHALL raise `ARCHIVE_UNSUPPORTED_FEATURE` until v0.2.0.
|
||||
|
||||
#### Scenario: Encrypted entry decrypts with correct password
|
||||
|
||||
- **WHEN** an AES-256 encrypted entry is read with a `password` hook returning the correct passphrase
|
||||
- **THEN** the decrypted bytes are returned and the password value is not retained on the adapter
|
||||
|
||||
### Requirement: Error code registry
|
||||
|
||||
The package SHALL register stable codes including `ZIP_EOCD_NOT_FOUND`, `ZIP_ZIP64_REQUIRED`, `ZIP_CENTRAL_DIR_CORRUPT`, `ZIP_CRC_MISMATCH`, `ZIP_UNSUPPORTED_COMPRESSION`, `ZIP_AES_WRITE_UNSUPPORTED`.
|
||||
|
||||
#### Scenario: CRC mismatch raises typed error
|
||||
|
||||
- **WHEN** an entry's decompressed CRC does not match the central directory CRC
|
||||
- **THEN** the read throws `ZIP_CRC_MISMATCH` with the entry path in details
|
||||
124
openspec/changes/add-drive-storage-archive/tasks.md
Normal file
124
openspec/changes/add-drive-storage-archive/tasks.md
Normal file
@@ -0,0 +1,124 @@
|
||||
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. drive-storage-archive-core (foundation)
|
||||
|
||||
- [x] 1.1 Scaffold `packages/drive/drive-storage-archive-core/` with `package.json`, vite/tsup config, vitest config via `createTestConfig()`, README, biome-clean tree
|
||||
- [x] 1.2 Define `ArchiveFormat` union, `ArchiveMode`, `ArchiveCodec`, `ResolveSibling`, `PasswordHook`, `OpenArchiveStorageOptions` types in `src/types/`
|
||||
- [x] 1.3 Implement `sniffFormat(bytes)` with magic-byte registry + structural validators (EOCD locate for zip, header checksum for tar, CRC trailer for gz, etc.)
|
||||
- [x] 1.4 Implement `openArchiveStorage(source, opts)` factory with dynamic `import()` of per-format packages (no eager bundling)
|
||||
- [x] 1.5 Implement `ARCHIVE_*` error code registry via `defineCodes` from `@suaveplan/error`
|
||||
- [x] 1.6 Implement `runArchiveConformance(adapterFactory, opts)` harness exported via `./testing` subpath
|
||||
- [x] 1.7 Implement default sibling-resolver helpers per format (`zipResolver`, `rarResolver`, `sevenZipResolver`) over a `DriveStorage`-shaped host
|
||||
- [x] 1.8 Co-locate `.md` for every implementation file (Purpose, Features, Usage Basic/Advanced, API Reference, Implementation Notes — ≥200 words, real-import examples)
|
||||
- [x] 1.9 100% Vitest coverage on `drive-storage-archive-core` via `bunx turbo test:coverage --filter=@suaveplan/drive-storage-archive-core`
|
||||
- [x] 1.10 Green `bunx turbo lint typecheck test --filter=@suaveplan/drive-storage-archive-core`
|
||||
- [x] 1.11 Update `MANIFEST.md` via `bun scripts/folder-reorg/generate-manifest.ts`
|
||||
|
||||
## 2. drive-storage-gz (foundation for tgz)
|
||||
|
||||
- [ ] 2.1 Scaffold `packages/drive/drive-storage-gz/`
|
||||
- [ ] 2.2 Implement gzip member parser (RFC 1952), multi-member concatenation, FNAME extraction
|
||||
- [ ] 2.3 Implement `GzipArchiveStorage` `DriveStorage` view with single-entry presentation
|
||||
- [ ] 2.4 Implement `gzipCodec: ArchiveCodec` export for tgz composition
|
||||
- [ ] 2.5 Implement write paths for `copy-on-write` and `rewrite` modes (full re-encode)
|
||||
- [ ] 2.6 Implement CRC32 + ISIZE validation
|
||||
- [ ] 2.7 Register `GZ_*` error codes
|
||||
- [ ] 2.8 Run `runArchiveConformance` from `drive-storage-archive-core/testing`
|
||||
- [ ] 2.9 Co-located `.md` docs per implementation file
|
||||
- [ ] 2.10 100% coverage + lint + typecheck green
|
||||
- [ ] 2.11 Regenerate MANIFEST
|
||||
|
||||
## 3. drive-storage-tar
|
||||
|
||||
- [ ] 3.1 Scaffold `packages/drive/drive-storage-tar/`
|
||||
- [ ] 3.2 Implement tar block reader (512-byte alignment, two-block terminator)
|
||||
- [ ] 3.3 Implement legacy v7, USTAR, POSIX.1-2001 PAX header parsing
|
||||
- [ ] 3.4 Implement GNU long-name (`L`) and long-link (`K`) reconstruction
|
||||
- [ ] 3.5 Implement PAX extended header records (path, size, mtime, atime, ctime, uid, gid)
|
||||
- [ ] 3.6 Implement GNU sparse 1.0 reconstruction with zero-fill
|
||||
- [ ] 3.7 Implement `TarArchiveStorage` `DriveStorage` view
|
||||
- [ ] 3.8 Implement append-write fast path (before terminator) and full-rewrite path for deletes
|
||||
- [ ] 3.9 Implement header-checksum validation, raise `TAR_HEADER_CHECKSUM_INVALID`
|
||||
- [ ] 3.10 Register `TAR_*` error codes
|
||||
- [ ] 3.11 Run `runArchiveConformance`
|
||||
- [ ] 3.12 Co-located `.md` docs per implementation file
|
||||
- [ ] 3.13 100% coverage + lint + typecheck green
|
||||
- [ ] 3.14 Regenerate MANIFEST
|
||||
|
||||
## 4. drive-storage-tgz
|
||||
|
||||
- [ ] 4.1 Scaffold `packages/drive/drive-storage-tgz/`
|
||||
- [ ] 4.2 Implement `TgzArchiveStorage` composing `TarArchiveStorage` over `ArchiveCodec`
|
||||
- [ ] 4.3 Implement `openTarGz(source, opts)` convenience defaulting to `gzipCodec` from `drive-storage-gz`
|
||||
- [ ] 4.4 Validate `ARCHIVE_CODEC_MISSING` on construction without codec
|
||||
- [ ] 4.5 Implement write path (codec-encode tar bytes back to host)
|
||||
- [ ] 4.6 Register `TGZ_*` error codes, propagate inner tar/codec errors with chain
|
||||
- [ ] 4.7 Run `runArchiveConformance` with `gzipCodec`
|
||||
- [ ] 4.8 Co-located `.md` docs per implementation file
|
||||
- [ ] 4.9 100% coverage + lint + typecheck green
|
||||
- [ ] 4.10 Regenerate MANIFEST
|
||||
|
||||
## 5. drive-storage-zip
|
||||
|
||||
- [x] 5.1 Scaffold `packages/drive/drive-storage-zip/`
|
||||
- [x] 5.2 Implement EOCD + Central Directory parser, ZIP64 EOCD locator/record handling
|
||||
- [x] 5.3 Implement Local File Header reader, deflate decode via `fflate`
|
||||
- [x] 5.4 Implement WinZip AE-2 AES-256 decrypt with PBKDF2 key derivation
|
||||
- [x] 5.5 Implement split-zip volume reassembly via `resolveSibling`, validate volume signatures
|
||||
- [x] 5.6 Implement `ZipArchiveStorage` `DriveStorage` view
|
||||
- [x] 5.7 Implement write path: append local headers + rebuild central directory + EOCD on flush
|
||||
- [x] 5.8 Block AES write with `ZIP_AES_WRITE_UNSUPPORTED` (deferred to v0.2.0)
|
||||
- [x] 5.9 Implement CRC32 validation, raise `ZIP_CRC_MISMATCH`
|
||||
- [x] 5.10 Register `ZIP_*` error codes
|
||||
- [x] 5.11 Run `runArchiveConformance`
|
||||
- [x] 5.12 Co-located `.md` docs per implementation file
|
||||
- [x] 5.13 100% coverage + lint + typecheck green
|
||||
- [x] 5.14 Regenerate MANIFEST
|
||||
|
||||
## 6. drive-storage-7z
|
||||
|
||||
- [ ] 6.1 Scaffold `packages/drive/drive-storage-7z/`
|
||||
- [ ] 6.2 Benchmark wasm LZMA vs `lzma-native` on 100MB archive; record decision in package README
|
||||
- [ ] 6.3 Implement 7z trailing header parser, `EncodedHeader` decode path
|
||||
- [ ] 6.4 Implement LZMA + LZMA2 decompression
|
||||
- [ ] 6.5 Implement BCJ, BCJ2, Delta, ARM/X86/SPARC filter chain
|
||||
- [ ] 6.6 Implement multi-volume reassembly (`.7z.NNN`) via `resolveSibling`
|
||||
- [ ] 6.7 Implement `7zAES` header + content encryption with password hook
|
||||
- [ ] 6.8 Implement `SevenZipArchiveStorage` `DriveStorage` view
|
||||
- [ ] 6.9 Implement write path: re-encode affected folders + rewrite trailing header
|
||||
- [ ] 6.10 Register `SEVENZ_*` error codes
|
||||
- [ ] 6.11 Run `runArchiveConformance`
|
||||
- [ ] 6.12 Co-located `.md` docs per implementation file
|
||||
- [ ] 6.13 100% coverage + lint + typecheck green
|
||||
- [ ] 6.14 Regenerate MANIFEST
|
||||
|
||||
## 7. drive-storage-rar (read-only, server-tier)
|
||||
|
||||
- [ ] 7.1 Scaffold `packages/drive/drive-storage-rar/` with `engines.node` + browser stub
|
||||
- [ ] 7.2 Vendor wasm unrar (permissive license) or wire native binding; record license attribution in README
|
||||
- [ ] 7.3 Implement RAR4 + RAR5 signature detection and version routing
|
||||
- [ ] 7.4 Implement multi-volume reassembly: RAR5 `.partN.rar` + RAR4 `.r00`..`.r99` via `resolveSibling`
|
||||
- [ ] 7.5 Implement encrypted-header and per-entry-encrypted reads with password hook
|
||||
- [ ] 7.6 Implement `RarArchiveStorage` `DriveStorage` view; reject any non-`read-only` mode in constructor
|
||||
- [ ] 7.7 Implement browser stub that throws `RAR_UNAVAILABLE_IN_BROWSER` for every entry point
|
||||
- [ ] 7.8 Register `RAR_*` error codes
|
||||
- [ ] 7.9 Run `runArchiveConformance` (read-only subset)
|
||||
- [ ] 7.10 Co-located `.md` docs per implementation file
|
||||
- [ ] 7.11 100% coverage + lint + typecheck green
|
||||
- [ ] 7.12 Regenerate MANIFEST
|
||||
|
||||
## 8. Integration & cross-cutting
|
||||
|
||||
- [ ] 8.1 Wire `openArchiveStorage` end-to-end test fixture using a real `.zip`, `.tar`, `.gz`, `.tar.gz`, `.rar`, `.7z` and a split-zip set
|
||||
- [ ] 8.2 Add an integration test demonstrating `new Drive(openArchiveStorage(...))` for each format
|
||||
- [ ] 8.3 Update `openspec/specs/drive/spec.md` only if drive-side contract clarification is needed (likely no change — `DriveStorage` view already satisfied)
|
||||
- [ ] 8.4 Run `bunx turbo lint typecheck test` across all seven new packages and ensure green
|
||||
- [ ] 8.5 Run `bunx turbo test:coverage` across all seven packages and confirm 100% coverage on touched modules
|
||||
- [ ] 8.6 Final `MANIFEST.md` regeneration + commit
|
||||
- [ ] 8.7 Archive change to `openspec/archive/<yyyy-mm-dd>-add-drive-storage-archive/` per repo workflow
|
||||
60
openspec/changes/add-import-sse-progress/proposal.md
Normal file
60
openspec/changes/add-import-sse-progress/proposal.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# Change: add-import-sse-progress
|
||||
|
||||
## Why
|
||||
|
||||
`@suaveplan/api-plugin-import` shipped (archive `2026-05-19-add-api-plugin-import`) with the
|
||||
**`## Excluded from this migration`** clause explicitly deferring SSE progress push:
|
||||
|
||||
> Foreign-key resolution, parallel batch workers, resumable checkpoints, S3/URL source adapters, **SSE progress push**
|
||||
|
||||
This change lifts that single deferral. Consumers can now subscribe to a job's progress
|
||||
via `GET /imports/:id/progress` and receive `text/event-stream` frames driven directly off
|
||||
the existing `ImportService.bus` events — no polling. The other deferred items
|
||||
(foreign-key resolution, parallel workers, checkpoints, S3 sources) remain deferred.
|
||||
|
||||
The endpoint reuses:
|
||||
|
||||
- the canonical `ImportProgress` payload type,
|
||||
- the existing `import.progress` / `import.completed` / `import.failed` events,
|
||||
- the existing `IMPORT.*` error registry,
|
||||
- the owner-scoping invariants of `ImportService.getJob(jobId, ownerId)`.
|
||||
|
||||
The endpoint introduces three new OTel counters scoped to the new feature module:
|
||||
`import.sse.connected`, `import.sse.events_sent{kind}`, `import.sse.disconnected{reason}`.
|
||||
|
||||
## Affected requirements
|
||||
|
||||
ADDED requirement: **SSE progress push** (one new requirement; no MODIFIED / REMOVED entries).
|
||||
|
||||
## Source mapping
|
||||
|
||||
| Spec scenario | Implementation | Test |
|
||||
| --- | --- | --- |
|
||||
| GET `/imports/:id/progress` opens an event-stream | `src/routes/progress-sse/progress-sse.ts` | `src/routes/progress-sse/progress-sse.test.ts::"opens a text/event-stream and emits an initial progress frame"` |
|
||||
| Each `import.progress` bus emit produces an `event: progress` frame | `src/routes/progress-sse/progress-sse.ts` | `src/routes/progress-sse/progress-sse.test.ts::"emits a progress frame for every bus progress event"` |
|
||||
| Terminal `import.completed` emits `event: complete` then closes | `src/routes/progress-sse/progress-sse.ts` | `src/routes/progress-sse/progress-sse.test.ts::"emits a complete frame and closes on import.completed"` |
|
||||
| Terminal `import.failed` emits `event: failed` then closes | `src/routes/progress-sse/progress-sse.ts` | `src/routes/progress-sse/progress-sse.test.ts::"emits a failed frame and closes on import.failed"` |
|
||||
| Foreign-owner request returns 404 problem+json | `src/routes/progress-sse/progress-sse.ts` | `src/routes/progress-sse/progress-sse.test.ts::"returns 404 problem+json for a foreign-owner job"` |
|
||||
| Anonymous request returns 401 problem+json | route is mounted under `requireAuth()` in `build-routes.ts` | `src/routes/progress-sse/progress-sse.test.ts::"returns 401 problem+json when the caller is anonymous"` |
|
||||
| Client disconnect cleans up the bus listener | `AbortSignal` wired into the `ReadableStream.cancel` path | `src/routes/progress-sse/progress-sse.test.ts::"detaches the listener and increments disconnect counter on client abort"` |
|
||||
| OTel counters increment on connect / event / disconnect | `src/routes/progress-sse/progress-sse.ts` (new instruments wired through `getImportInstruments()`) | `src/routes/progress-sse/progress-sse.test.ts::"increments import.sse counters on connect, send, and disconnect"` |
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/api-plugin-import` to enumerate.
|
||||
|
||||
- `@suaveplan/api` — [ ] tests run / [ ] docs reviewed / [x] no action needed (reason:
|
||||
additive route under an existing prefix; no signature changes to the consumed surface)
|
||||
|
||||
Foundation package? No. This is a feature-additive change on a server-tier plugin.
|
||||
|
||||
## Out of scope (still deferred from the original `Excluded` list)
|
||||
|
||||
- Foreign-key resolution
|
||||
- Parallel batch workers
|
||||
- Resumable checkpoints
|
||||
- S3 / URL source adapters
|
||||
- Streaming input parsing
|
||||
- Benchmarks
|
||||
@@ -0,0 +1,92 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Spec Delta: api-plugin-import — add SSE progress push
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: SSE progress push — `GET /imports/:id/progress`
|
||||
|
||||
The package SHALL expose `GET /imports/:id/progress` returning
|
||||
`Content-Type: text/event-stream; charset=utf-8`, mounted under the existing
|
||||
`requireAuth()` middleware. Frames SHALL be driven directly off the existing
|
||||
`ImportService.bus` events (`import.progress`, `import.completed`, `import.failed`);
|
||||
NO polling loop over the store is permitted. Every frame SHALL carry the canonical
|
||||
`ImportProgress` JSON payload. Three SSE event kinds SHALL be emitted: `progress`
|
||||
(every `import.progress` tick), `complete` (terminal, on `import.completed`),
|
||||
and `failed` (terminal, on `import.failed`). The stream SHALL close cleanly after
|
||||
a terminal event. The handler SHALL respect the request's `AbortSignal` (client
|
||||
disconnect) by detaching the bus listener and incrementing
|
||||
`import.sse.disconnected{reason: "client"}`. Foreign-owner or missing-job lookups
|
||||
SHALL return HTTP 404 RFC 7807 problem+json with `code === IMPORT.NOT_FOUND`
|
||||
(anti-enumeration parity with `GET /:id`). Anonymous requests SHALL return HTTP
|
||||
401 problem+json via the existing auth middleware.
|
||||
|
||||
The package SHALL extend `getImportInstruments()` with three additional counters:
|
||||
`import.sse.connected` (Counter), `import.sse.events_sent` (Counter, labelled by
|
||||
`kind ∈ "progress" | "complete" | "failed"`), `import.sse.disconnected`
|
||||
(Counter, labelled by `reason ∈ "complete" | "failed" | "client"`). The counters
|
||||
SHALL be cached on the same bundle as the existing six instruments and SHALL
|
||||
rebind through `resetImportInstruments()`.
|
||||
|
||||
#### Scenario: opens a text/event-stream and emits an initial progress frame
|
||||
|
||||
- **WHEN** an authenticated caller `GET /imports/:id/progress` for a job they own
|
||||
- **THEN** the response is HTTP 200 with `content-type: text/event-stream; charset=utf-8`
|
||||
and the first frame is `event: progress\ndata: <ImportProgress JSON>\n\n`
|
||||
|
||||
#### Scenario: emits a progress frame for every bus progress event
|
||||
|
||||
- **WHEN** the service emits `import.progress` while the client is connected
|
||||
- **THEN** the stream writes a frame whose first non-id line is `event: progress`
|
||||
and whose `data:` line decodes to an `ImportProgress` object with the matching
|
||||
`processedRows`
|
||||
|
||||
#### Scenario: emits a complete frame and closes on import.completed
|
||||
|
||||
- **WHEN** the service emits `import.completed`
|
||||
- **THEN** the stream writes a frame whose first non-id line is `event: complete`
|
||||
and then closes; subsequent reads from the stream return `done: true`
|
||||
|
||||
#### Scenario: emits a failed frame and closes on import.failed
|
||||
|
||||
- **WHEN** the service emits `import.failed`
|
||||
- **THEN** the stream writes a frame whose first non-id line is `event: failed`
|
||||
and then closes
|
||||
|
||||
#### Scenario: returns 404 problem+json for a foreign-owner job
|
||||
|
||||
- **WHEN** alice owns job `j1` and bob requests `GET /imports/j1/progress`
|
||||
- **THEN** the response is HTTP 404 with `content-type: application/problem+json`
|
||||
and a body whose `code === "IMPORT.NOT_FOUND"`
|
||||
|
||||
#### Scenario: returns 401 problem+json when the caller is anonymous
|
||||
|
||||
- **WHEN** an anonymous caller requests `GET /imports/:id/progress`
|
||||
- **THEN** the response is HTTP 401 with `content-type: application/problem+json`
|
||||
|
||||
#### Scenario: detaches the listener and increments disconnect counter on client abort
|
||||
|
||||
- **WHEN** the client aborts the request before a terminal event
|
||||
- **THEN** the handler unsubscribes from the bus and increments
|
||||
`import.sse.disconnected{reason:"client"}`
|
||||
|
||||
#### Scenario: increments import.sse counters on connect, send, and disconnect
|
||||
|
||||
- **WHEN** a connection opens, two progress events fire, and a terminal `complete`
|
||||
event closes the stream
|
||||
- **THEN** `import.sse.connected` increments by 1, `import.sse.events_sent{kind}`
|
||||
increments by 3 (2× `progress` + 1× `complete`), and
|
||||
`import.sse.disconnected{reason:"complete"}` increments by 1
|
||||
|
||||
**Verified by:**
|
||||
`src/routes/progress-sse/progress-sse.test.ts::"opens a text/event-stream and emits an initial progress frame"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"emits a progress frame for every bus progress event"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"emits a complete frame and closes on import.completed"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"emits a failed frame and closes on import.failed"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"returns 404 problem+json for a foreign-owner job"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"returns 401 problem+json when the caller is anonymous"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"detaches the listener and increments disconnect counter on client abort"`
|
||||
AND `src/routes/progress-sse/progress-sse.test.ts::"increments import.sse counters on connect, send, and disconnect"`
|
||||
AND `src/otel/instruments.test.ts::"bundle exposes the SSE counters"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
59
openspec/changes/add-import-sse-progress/tasks.md
Normal file
59
openspec/changes/add-import-sse-progress/tasks.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Tasks: add-import-sse-progress
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. OTel — extend instruments bundle
|
||||
|
||||
- [ ] Add `sseConnected: Counter` (`import.sse.connected`) to `ImportInstruments`.
|
||||
- [ ] Add `sseEventsSent: Counter` (`import.sse.events_sent`) labelled by `kind`.
|
||||
- [ ] Add `sseDisconnected: Counter` (`import.sse.disconnected`) labelled by `reason`.
|
||||
- [ ] Update `src/otel/instruments.md` to describe the three new counters.
|
||||
- [ ] Extend `src/otel/instruments.test.ts` to assert the new counters round-trip through a fresh `MeterProvider`.
|
||||
|
||||
## 2. Route — `GET /:id/progress`
|
||||
|
||||
- [ ] Create `src/routes/progress-sse/progress-sse.ts` exporting `registerProgressSseRoute(app, deps)`.
|
||||
- [ ] Wire the handler into `createImportRoutes` in `src/routes/build-routes.ts`.
|
||||
- [ ] Subscribe to `service.bus` (`import.progress`, `import.completed`, `import.failed`) — NO polling.
|
||||
- [ ] Forward the request `AbortSignal` to detach the listener on client disconnect.
|
||||
- [ ] Frame format: `id: <jobId>-<seq>`, `event: progress|complete|failed`, JSON-stringified `ImportProgress` payload, double-newline terminator.
|
||||
- [ ] Initial frame: snapshot from `service.getJob(id, ownerId)` so the client sees current state before the first bus tick.
|
||||
- [ ] Terminal events (`complete` / `failed`) close the stream cleanly.
|
||||
|
||||
## 3. Errors
|
||||
|
||||
- [ ] Reuse `IMPORT.NOT_FOUND` for foreign-owner / missing jobs (404 problem+json).
|
||||
- [ ] Reuse `IMPORT.UNAUTHENTICATED` (via `requireAuth()` middleware already in `build-routes.ts`) for the 401 path.
|
||||
- [ ] No new error codes added.
|
||||
|
||||
## 4. Tests
|
||||
|
||||
- [ ] `src/routes/progress-sse/progress-sse.test.ts` covering eight scenarios listed in `proposal.md#source-mapping`.
|
||||
- [ ] Tests use Hono's `app.request()` returning a `Response` with a streamed body; assertions read via `body.getReader()` and decode UTF-8.
|
||||
- [ ] No mocks. Real `ImportService`, real Hono, real in-memory `MeterProvider`.
|
||||
|
||||
## 5. Docs
|
||||
|
||||
- [ ] `src/routes/progress-sse/progress-sse.md` (≥200 words, Purpose / Features / Usage Basic / Usage Advanced / API Reference / Implementation Notes).
|
||||
- [ ] Update package `README.md` Features list and routes table to include the new SSE endpoint.
|
||||
|
||||
## 6. Subpath export
|
||||
|
||||
- [ ] Add `./routes/progress-sse/progress-sse` to `package.json#exports`.
|
||||
|
||||
## 7. Gates (in order)
|
||||
|
||||
- [ ] `bunx biome check src/`
|
||||
- [ ] `bunx turbo lint typecheck test --filter=@suaveplan/api-plugin-import`
|
||||
- [ ] `bunx turbo test:coverage --filter=@suaveplan/api-plugin-import` (100/100/100/100)
|
||||
- [ ] `bun scripts/openspec-conformance.ts @suaveplan/api-plugin-import` (0 errors)
|
||||
- [ ] `bunx turbo build --filter=@suaveplan/api-plugin-import`
|
||||
|
||||
## 8. Archive
|
||||
|
||||
- [ ] Fold spec delta into `openspec/specs/api-plugin-import/spec.md`.
|
||||
- [ ] `mv openspec/changes/add-import-sse-progress openspec/archive/2026-05-20-add-import-sse-progress`.
|
||||
62
openspec/changes/api-conformance-suites/proposal.md
Normal file
62
openspec/changes/api-conformance-suites/proposal.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# api-conformance-suites — `./testing` subpath conformance functions for every abstract capability
|
||||
|
||||
**Package**: multi-package — adds `./testing` subpaths to every Stage-1 abstract package
|
||||
**Status**: PROPOSED
|
||||
**Type**: feature (cross-package)
|
||||
**Stage**: 5 (Testing kit + conformance suites) — see umbrella `openspec/changes/api-plugin-foundation/`
|
||||
|
||||
## Why
|
||||
|
||||
Per design doc §13.4, every abstract capability ships a conformance suite under its `./testing` subpath. The pattern mirrors `@suaveplan/i18n-shared/testing`'s `runBackendConformance`: a single function that exhaustively pins the contract; every adapter (Stage 4 production AND Stage 5 in-memory) runs the suite; new adapters that pass the suite are automatically known good; new invariants added to the suite are automatically enforced across every existing adapter.
|
||||
|
||||
This change adds those `./testing` subpaths. It's a multi-package change because each function lives in its respective abstract package — co-location matters (the contract owner owns the conformance gate). The companion in-memory adapters (Stage 5) provide the canonical "in-memory passes the suite" self-test, proving the suite isn't trivially empty.
|
||||
|
||||
## What changes
|
||||
|
||||
Adds a `./testing` export to each of the following abstract packages (created in Stage 1 — `api-router`, `api-auth`, `api-tenant-resolver` (currently inside `api-auth` umbrella), `api-tenant-provider`, `api-secrets`, `api-events`, `api-jobs`, `api-graphql`, `api-ws`, `api-sse`, `api-saga`, `api-2pc`, `api-db`). Each `./testing` subpath exports one conformance function:
|
||||
|
||||
- ADDED: `@suaveplan/api-router/testing` → `runRouterConformance(label, factory)` — pins route registration, dispatch, blue-green slot semantics, streaming bypass.
|
||||
- ADDED: `@suaveplan/api-auth/testing` → `runAuthSessionConformance(label, factory)` — pins `resolveSession` contract, null-principal handling, securityScheme contribution.
|
||||
- ADDED: `@suaveplan/api-tenant-resolver/testing` (or `@suaveplan/api-auth/testing` namespace) → `runTenantResolverConformance(label, factory)` — pins chain-advance sentinel, principal-derived resolution.
|
||||
- ADDED: `@suaveplan/api-tenant-provider/testing` → `runTenantProviderConformance(label, factory)` — pins `list`, `listAsync`, mutation semantics.
|
||||
- ADDED: `@suaveplan/api-secrets/testing` → `runSecretsConformance(label, factory)` — pins `get`, manifest validation, redaction.
|
||||
- ADDED: `@suaveplan/api-events/testing` → `runEventsBusConformance(label, factory)` — pins publish/subscribe, ordering, idempotency, bounded-queue backpressure.
|
||||
- ADDED: `@suaveplan/api-jobs/testing` → `runJobsConformance(label, factory)` — pins schedule, runScheduled, perTenant fan-out.
|
||||
- ADDED: `@suaveplan/api-graphql/testing` → `runGraphqlEngineConformance(label, factory)` — pins schema merge, resolver execution, subscription delivery.
|
||||
- ADDED: `@suaveplan/api-ws/testing` → `runWsEngineConformance(label, factory)` AND `runWsBrokerConformance(label, factory)` — pins connection lifecycle (engine) and subscription fan-out (broker).
|
||||
- ADDED: `@suaveplan/api-sse/testing` → `runSseEngineConformance(label, factory)` — pins SSE stream lifecycle, reconnect-id handling.
|
||||
- ADDED: `@suaveplan/api-saga/testing` → `runSagaParticipantConformance(label, factory)` — pins step idempotency, compensation, replay safety.
|
||||
- ADDED: `@suaveplan/api-2pc/testing` → `run2pcParticipantConformance(label, factory)` — pins prepare/commit/rollback, in-doubt recovery.
|
||||
- ADDED: `@suaveplan/api-db/testing` → `runDbAdapterConformance(label, factory)` — pins admin vs scoped surfaces, tenant injection, migration runner, unsafe-queries gating.
|
||||
|
||||
Self-tests:
|
||||
|
||||
- ADDED: For each `runXConformance`, a self-test file (`src/testing/<name>.self-test.test.ts`) that runs the function against the Stage-5 in-memory adapter (or, for `api-router-hono`'s case, the direct surface) AND asserts that the run reports at least one passing invariant per category (sanity check: a conformance function that runs zero invariants is broken).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- The abstract package interfaces themselves — those are Stage-1 changes (already proposed).
|
||||
- Adapter packages that consume these suites — those are separate changes (Stage 4 + Stage 5).
|
||||
- Runtime-only test helpers unrelated to capability conformance — those live in `@suaveplan/testing`.
|
||||
|
||||
## Source mapping
|
||||
|
||||
Genesis-original. Source of design intent: `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`.
|
||||
|
||||
| Requirement | Design doc section |
|
||||
|---|---|
|
||||
| `./testing` subpath pattern (mirrors `i18n-shared/testing`) | §13.4 (intro) |
|
||||
| One conformance fn per capability | §13.4 (final paragraph) |
|
||||
| Same suite runs against every adapter | §13.4 (middle paragraph) |
|
||||
| New invariants automatically enforce across adapters | §13.4 (middle paragraph) |
|
||||
|
||||
## Verification approach
|
||||
|
||||
- Each conformance function lives at `packages/api/<abstract>/src/testing/<run-name>.ts` with a sibling `.md` documenting which invariants it pins (one bullet per `it(...)` title).
|
||||
- Each function's self-test (`<run-name>.self-test.test.ts`) calls it with the Stage-5 in-memory adapter as the factory and asserts non-empty pass count.
|
||||
- Each Stage-4 production adapter (in its own change) and each Stage-5 in-memory adapter (in its own change) runs the function against its own factory; those tests live in the adapter package, not here.
|
||||
- A dep-graph guard (`tests/dep-graph/conformance-subpath-shape.test.ts`) iterates every abstract package and asserts the `./testing` subpath export exists AND exports the expected `runXConformance` symbol.
|
||||
|
||||
## Migration impact
|
||||
|
||||
None — additive only. Existing abstract packages get a new subpath export; no existing import surface is touched.
|
||||
@@ -0,0 +1,182 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: api-conformance-suites (api-conformance-suites)
|
||||
|
||||
**Package**: multi-package — `./testing` subpath additions on every abstract API package
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Each conformance function lives in its respective abstract package. This delta describes the invariants every function pins. The function itself is the executable enforcement; this spec is the human-readable contract.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: runRouterConformance pins the Router contract
|
||||
|
||||
`@suaveplan/api-router/testing` SHALL export `runRouterConformance(label, factory)` which executes a `describe` block of invariants covering: static route match, param route match wins, method routing, 404 on no match, duplicate-route rejection, middleware ordering, streaming bypass, blue-green slot atomic swap, in-flight completion on old slot, abort propagation, telemetry instruments emitted, error-envelope on throw, request-id propagation.
|
||||
|
||||
#### Scenario: function runs at least one `it` per category
|
||||
|
||||
- **WHEN** `runRouterConformance("self-test", createInMemoryRouter)` runs
|
||||
- **THEN** the test report contains at least one passing `it` titled per category (`"matches static routes"`, `"matches param routes"`, `"swaps blue-green atomically"`, ...)
|
||||
|
||||
**Verified by:** `packages/api/api-router/src/testing/run-router-conformance.self-test.test.ts::"self-test reports passes across every documented category"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runAuthSessionConformance pins the AuthSession contract
|
||||
|
||||
`@suaveplan/api-auth/testing` SHALL export `runAuthSessionConformance(label, factory)` covering: resolves Principal from valid input, returns null on invalid/absent, never throws on malformed input (returns null + emits failure counter), `securitySchemeContribution()` returns a valid OpenAPI fragment.
|
||||
|
||||
#### Scenario: self-test passes against api-auth-static-principal
|
||||
|
||||
- **WHEN** `runAuthSessionConformance("self-test", () => createStaticPrincipalResolver({ principal: P }))` runs
|
||||
- **THEN** every invariant passes AND at least 4 distinct `it` titles report pass
|
||||
|
||||
**Verified by:** `packages/api/api-auth/src/testing/run-auth-session-conformance.self-test.test.ts::"self-test against static-principal adapter passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runTenantResolverConformance pins the TenantResolver chain contract
|
||||
|
||||
The function SHALL cover: principal-with-tenantId returns it, principal-without advances chain (undefined sentinel), null-principal advances chain by default, `requirePrincipal` throws.
|
||||
|
||||
#### Scenario: self-test passes against jwt-tenant-resolver
|
||||
|
||||
- **WHEN** `runTenantResolverConformance("self-test", () => createJwtTenantResolver(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-tenant-resolver/src/testing/run-tenant-resolver-conformance.self-test.test.ts::"self-test against jwt resolver passes every invariant"` (or co-located inside `api-auth` if tenant-resolver lives there)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runTenantProviderConformance pins the TenantProvider contract
|
||||
|
||||
The function SHALL cover: `list()` snapshot, `listAsync()` complete iteration, `get(id)` hit/miss, mutation semantics (`add`/`remove`/`update`) when mutable, immutability throws when not.
|
||||
|
||||
#### Scenario: self-test passes against memory provider
|
||||
|
||||
- **WHEN** `runTenantProviderConformance("self-test", () => createMemoryTenantProvider(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-tenant-provider/src/testing/run-tenant-provider-conformance.self-test.test.ts::"self-test against memory provider passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runSecretsConformance pins the SecretsLike contract
|
||||
|
||||
The function SHALL cover: `get(name)` returns string-or-undefined, manifest fail-fast on required missing, optional missing is silent, value redaction in errors and metric labels.
|
||||
|
||||
#### Scenario: self-test passes against env adapter
|
||||
|
||||
- **WHEN** `runSecretsConformance("self-test", () => createEnvSecrets(...))` runs with a controlled env fixture
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-secrets/src/testing/run-secrets-conformance.self-test.test.ts::"self-test against env adapter passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runEventsBusConformance pins the EventBus contract
|
||||
|
||||
The function SHALL cover: publish-then-subscribe ordering (FIFO), subscriber-idempotency by payload id, bounded-queue backpressure (when configured), interceptor chain, history of published payloads (where adapter supports it).
|
||||
|
||||
#### Scenario: self-test passes against in-memory events bus
|
||||
|
||||
- **WHEN** `runEventsBusConformance("self-test", () => createInMemoryEventsBus(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-events/src/testing/run-events-bus-conformance.self-test.test.ts::"self-test against in-memory bus passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runJobsConformance pins the JobsRegistry contract
|
||||
|
||||
The function SHALL cover: schedule registration, FakeClock-driven `runScheduled`, `perTenant: true` fan-out via TenantProvider, idempotent re-registration on hot reload, abort propagation on stop.
|
||||
|
||||
#### Scenario: self-test passes against in-memory jobs registry
|
||||
|
||||
- **WHEN** `runJobsConformance("self-test", () => createInMemoryJobsRegistry(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-jobs/src/testing/run-jobs-conformance.self-test.test.ts::"self-test against in-memory jobs registry passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runWsEngineConformance and runWsBrokerConformance pin WS contracts
|
||||
|
||||
`@suaveplan/api-ws/testing` SHALL export both:
|
||||
|
||||
- `runWsEngineConformance(label, factory)` covering connection lifecycle, heartbeat, backpressure, reconnect-token semantics.
|
||||
- `runWsBrokerConformance(label, factory)` covering fan-out, subscribe/unsubscribe/cleanup, delivery mode semantics.
|
||||
|
||||
#### Scenario: broker self-test passes against in-memory broker
|
||||
|
||||
- **WHEN** `runWsBrokerConformance("self-test", () => createInMemoryWsBroker(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-ws/src/testing/run-ws-broker-conformance.self-test.test.ts::"self-test against in-memory broker passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runSseEngineConformance pins the SSE contract
|
||||
|
||||
The function SHALL cover: stream lifecycle, `id:` field for resume, retry hint, abort propagation.
|
||||
|
||||
#### Scenario: self-test passes against a reference SSE engine
|
||||
|
||||
- **WHEN** `runSseEngineConformance("self-test", referenceSseEngineFactory)` runs (the reference factory is provided in this change for the express purpose of the self-test; Stage 7 ships the production `api-sse-bun`)
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-sse/src/testing/run-sse-engine-conformance.self-test.test.ts::"self-test against reference engine passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runSagaParticipantConformance pins the SagaParticipant contract
|
||||
|
||||
The function SHALL cover: step idempotency, compensation execution on failure, replay safety on host restart, persisted state shape.
|
||||
|
||||
#### Scenario: self-test passes against a reference saga participant
|
||||
|
||||
- **WHEN** `runSagaParticipantConformance("self-test", referenceParticipantFactory)` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-saga/src/testing/run-saga-participant-conformance.self-test.test.ts::"self-test against reference participant passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: run2pcParticipantConformance pins the 2PC contract
|
||||
|
||||
The function SHALL cover: prepare returns vote, commit/rollback after vote, in-doubt recovery on restart, prepared-transaction cap honoured.
|
||||
|
||||
#### Scenario: self-test passes against a reference 2PC participant
|
||||
|
||||
- **WHEN** `run2pcParticipantConformance("self-test", reference2pcParticipantFactory)` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-2pc/src/testing/run-2pc-participant-conformance.self-test.test.ts::"self-test against reference 2pc participant passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: runDbAdapterConformance pins the Db contract
|
||||
|
||||
The function SHALL cover: admin surface SELECT/INSERT, scoped surface tenant injection, cross-tenant rejection, migration runner forward+idempotent+failure, unsafe-queries capability gating + counter emission.
|
||||
|
||||
#### Scenario: self-test passes against in-memory SQLite
|
||||
|
||||
- **WHEN** `runDbAdapterConformance("self-test", () => createSqliteMemoryDb(...))` runs
|
||||
- **THEN** every invariant passes
|
||||
|
||||
**Verified by:** `packages/api/api-db/src/testing/run-db-adapter-conformance.self-test.test.ts::"self-test against in-memory sqlite passes every invariant"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Every `./testing` subpath is reachable and exports the expected symbol
|
||||
|
||||
A dep-graph guard SHALL iterate every abstract API package, resolve `./testing`, and assert the documented `runXConformance` symbol is present and is a callable.
|
||||
|
||||
#### Scenario: every abstract package exposes its conformance function
|
||||
|
||||
- **WHEN** the dep-graph guard runs
|
||||
- **THEN** each abstract package's `./testing` subpath exports the documented symbol AND the symbol is a function with at least two parameters (label, factory)
|
||||
|
||||
**Verified by:** `tests/dep-graph/conformance-subpath-shape.test.ts::"every abstract API package exposes its conformance function under ./testing"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
75
openspec/changes/api-conformance-suites/tasks.md
Normal file
75
openspec/changes/api-conformance-suites/tasks.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Tasks — api-conformance-suites (multi-package)
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read design doc (§13.4, §15 Stage 5)
|
||||
- [x] 1.2 Read `openspec/changes/api-router/`, `api-auth/`, `api-events/`, `api-secrets/`, `api-ws/`, `api-sse/`, `api-graphql/`, `api-saga/` (pending), `api-2pc/` (pending), `api-db/` (pending), `api-jobs/` (pending), `api-tenant-provider/`, `api-tenant-resolver/` for each abstract surface
|
||||
- [x] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
|
||||
- [x] 1.4 Draft `proposal.md`
|
||||
- [x] 1.5 Draft `specs/api-conformance-suites/spec.md`
|
||||
- [x] 1.6 Every requirement carries `**Verified by:**`
|
||||
|
||||
## 2. Implementation
|
||||
|
||||
- [ ] 2.1 `packages/api/api-router/src/testing/run-router-conformance.ts` + `.md`
|
||||
- [ ] 2.2 `packages/api/api-auth/src/testing/run-auth-session-conformance.ts` + `.md`
|
||||
- [ ] 2.3 `packages/api/api-auth/src/testing/run-tenant-resolver-conformance.ts` (or `api-tenant-resolver` if separate) + `.md`
|
||||
- [ ] 2.4 `packages/api/api-tenant-provider/src/testing/run-tenant-provider-conformance.ts` + `.md`
|
||||
- [ ] 2.5 `packages/api/api-secrets/src/testing/run-secrets-conformance.ts` + `.md`
|
||||
- [ ] 2.6 `packages/api/api-events/src/testing/run-events-bus-conformance.ts` + `.md`
|
||||
- [ ] 2.7 `packages/api/api-jobs/src/testing/run-jobs-conformance.ts` + `.md`
|
||||
- [ ] 2.8 `packages/api/api-graphql/src/testing/run-graphql-engine-conformance.ts` + `.md`
|
||||
- [ ] 2.9 `packages/api/api-ws/src/testing/run-ws-engine-conformance.ts` + `.md`
|
||||
- [ ] 2.10 `packages/api/api-ws/src/testing/run-ws-broker-conformance.ts` + `.md`
|
||||
- [ ] 2.11 `packages/api/api-sse/src/testing/run-sse-engine-conformance.ts` + `.md`
|
||||
- [ ] 2.12 `packages/api/api-saga/src/testing/run-saga-participant-conformance.ts` + `.md`
|
||||
- [ ] 2.13 `packages/api/api-2pc/src/testing/run-2pc-participant-conformance.ts` + `.md`
|
||||
- [ ] 2.14 `packages/api/api-db/src/testing/run-db-adapter-conformance.ts` + `.md`
|
||||
- [ ] 2.15 Each abstract package's `package.json` adds the `./testing` subpath export
|
||||
- [ ] 2.16 Each `./testing` subpath imports its companion in-memory adapter (Stage 5) ONLY in the self-test file — runtime conformance code must NOT depend on adapter packages (avoid circular)
|
||||
|
||||
## 3. Tests
|
||||
|
||||
- [ ] 3.1 `packages/api/api-router/src/testing/run-router-conformance.self-test.test.ts` — invokes the function against `createInMemoryRouter`; asserts at least one pass per category (route-match, blue-green, streaming, error-envelope)
|
||||
- [ ] 3.2 Same self-test file pattern for every other run-* function (one per capability)
|
||||
- [ ] 3.3 `tests/dep-graph/conformance-subpath-shape.test.ts` — iterate every abstract package; assert `./testing` subpath exports the expected symbol
|
||||
- [ ] 3.4 Every `it(...)` carries `{ timeout: 5000 }` (30000 for genuinely async)
|
||||
|
||||
## 4. Documentation
|
||||
|
||||
- [ ] 4.1 Sibling `.md` per conformance function (≥200 words, lists every invariant with a one-line description, basic + advanced examples of invocation, full signature)
|
||||
- [ ] 4.2 Each abstract package's README documents the `./testing` subpath usage and links the .md
|
||||
- [ ] 4.3 Cross-link from umbrella `api-plugin-foundation/proposal.md` to this change
|
||||
|
||||
## 5. Pre-archive verification (5-gate triple)
|
||||
|
||||
- [ ] 5.1 `bun run check` — clean
|
||||
- [ ] 5.2 `bunx turbo lint typecheck test --filter='@suaveplan/api-*'` — green
|
||||
- [ ] 5.3 `bunx turbo test:coverage --filter='@suaveplan/api-*'` — 100/100/100/100
|
||||
- [ ] 5.4 `bun scripts/openspec-conformance.ts api-conformance-suites` — exit 0
|
||||
- [ ] 5.5 `bunx turbo build --filter='@suaveplan/api-*'` — green
|
||||
|
||||
## 6. Pre-archive 7-row checklist
|
||||
|
||||
- [ ] 6.1 Generics threaded through every `runXConformance(label, factory)` signature; `expectTypeOf` asserts factory return type matches capability
|
||||
- [ ] 6.2 Conformance fixtures (where Zod-validated payloads are needed) wired at boundaries
|
||||
- [ ] 6.3 No `error-codes.ts` per conformance fn — they delegate to the capability's own codes
|
||||
- [ ] 6.4 Strict-mode behaviors (where a capability flips strict/lenient) tested both on and off inside the conformance suite itself
|
||||
- [ ] 6.5 OTel-instrument invariants (every capability's expected instrument set) asserted from within the conformance suite using an in-memory `MeterProvider` supplied by the test host
|
||||
- [ ] 6.6 Lifecycle dedup invariants (idempotent register/unregister, etc.) included in the suite where applicable
|
||||
- [ ] 6.7 No retroactive tick — every box reflects work in this change
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold spec deltas into each abstract package's `openspec/specs/<cap>/spec.md` under a "Testing subpath" section
|
||||
- [ ] 7.2 Move `openspec/changes/api-conformance-suites/` → `openspec/archive/<yyyy-mm-dd>-api-conformance-suites/`
|
||||
- [ ] 7.3 Tick Stage 5 conformance-suites box in umbrella `openspec/changes/api-plugin-foundation/tasks.md`
|
||||
- [ ] 7.4 Refresh `MANIFEST.md`
|
||||
86
openspec/changes/api-host-pipeline-events/proposal.md
Normal file
86
openspec/changes/api-host-pipeline-events/proposal.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Host Pipeline Per-Stage Events
|
||||
|
||||
**Package**: `@suaveplan/api-host` (extend) + `@suaveplan/api-types` (contract)
|
||||
**Status**: PROPOSED
|
||||
**Type**: capability-extension (coherence-fix)
|
||||
|
||||
## Why
|
||||
|
||||
The §6 11-step request pipeline in `api-host`'s `RequestPipeline` is an opaque sequential function that returns a `Response`. Internal stages — request-id assignment, auth resolution, tenant resolution, capability check, rate-limit consumption, idempotency check, CSRF check, routing, handler dispatch, response validation, audit emission — are private state with no public observable surface.
|
||||
|
||||
The foundation umbrella's gate 2.8 demands "exercises the full §6 11-step pipeline end-to-end" and the foundation-acceptance §3.1 demands "Reference plugin's full integration test (design doc §13.5) green in CI". Neither can be satisfied without per-stage observability:
|
||||
|
||||
- A closure smoke that asserts "the request reached stage 4 (capability check)" cannot do so without a public signal.
|
||||
- An operator running the host in production has no way to see which stage rejected a malformed request.
|
||||
- The audit sink today emits ONE record per request — fine for accounting, useless for debugging cross-stage failures.
|
||||
|
||||
OTel spans MAY be emitted today but the contract doesn't promise it. Tests can't depend on what isn't contractually emitted.
|
||||
|
||||
## Concrete consumer
|
||||
|
||||
- `@suaveplan/e2e-foundation-stack-full` (Stage-8 closure smoke) — asserts each of the 11 stages by reading the PipelineEvents bus.
|
||||
- Any operator-built dashboard wanting per-stage latency / rejection-rate metrics.
|
||||
- `@suaveplan/api-plugin-example` `full-flow.integration.test.ts` — can replace string-parsing of audit records with structured event assertions.
|
||||
|
||||
## End-to-end test
|
||||
|
||||
- `e2e/foundation-stack-full/tests/pipeline-stages.spec.ts` — drives one request through the host, subscribes to PipelineEvents, asserts the 11 events fired in order with the expected payload shape per stage.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/api-types` — new `pipeline-events` subpath
|
||||
|
||||
```ts
|
||||
// packages/api-stack/api/api-types/src/pipeline-events/pipeline-events.types.ts
|
||||
export type PipelineStage =
|
||||
| "request-id"
|
||||
| "auth"
|
||||
| "tenant"
|
||||
| "capability"
|
||||
| "rate-limit"
|
||||
| "idempotency"
|
||||
| "csrf"
|
||||
| "route-match"
|
||||
| "handler"
|
||||
| "response-validate"
|
||||
| "audit";
|
||||
|
||||
export interface PipelineStageEvent {
|
||||
readonly requestId: string;
|
||||
readonly stage: PipelineStage;
|
||||
readonly outcome: "entered" | "completed" | "rejected" | "skipped";
|
||||
readonly durationMs?: number;
|
||||
readonly cause?: { readonly code: string; readonly message: string };
|
||||
readonly at: number;
|
||||
}
|
||||
|
||||
export interface PipelineEventsBus {
|
||||
emit(event: PipelineStageEvent): void;
|
||||
subscribe(listener: (event: PipelineStageEvent) => void): () => void;
|
||||
}
|
||||
```
|
||||
|
||||
Zero-runtime. Lives in `api-types/pipeline-events`.
|
||||
|
||||
### `@suaveplan/api-host` — emit events at every stage boundary
|
||||
|
||||
`request-pipeline.ts` gains a `pipelineEvents: PipelineEventsBus` adapter slot on `HostAdapters`. Every stage emits `entered` on entry, `completed`/`rejected`/`skipped` on exit. Default is `NoopPipelineEventsBus` (zero-cost when not observed); operators wire `InMemoryPipelineEventsBus` (test fixture) or a custom OTel-bridging implementation in production.
|
||||
|
||||
The `api-host/testing/createTestHost` factory wires the in-memory bus by default so smokes can assert without setup.
|
||||
|
||||
### Conformance harness
|
||||
|
||||
`@suaveplan/api-host/testing/pipeline-events-conformance` — `runPipelineEventsConformance(host)`:
|
||||
- 11 invariants, one per stage: a request that reaches stage N MUST emit `entered` AND (`completed` OR `rejected`) for stage N.
|
||||
- Order invariant: stages emit in §6-spec order; no stage emits before its predecessor's `completed`.
|
||||
- Skip invariant: when a stage is conditionally skipped (e.g. unauthenticated request bypasses idempotency-by-principal), `skipped` MUST fire (not silently omitted).
|
||||
- Rejection invariant: when a stage rejects, no subsequent stage emits `entered`.
|
||||
|
||||
## Why this is not a "nice-to-have"
|
||||
|
||||
Stage 4's foundation-fix-gap-1..5 surfaced because Stage 4 stage boundaries weren't observable. Same root failure mode the umbrella's gate 2.8 will hit at Stage 8 — already did when the closure-smoke agent reported the spec was unwriteable without per-stage observability. §19 (the new rule) explicitly forbids shipping a primitive without its host-integration; this proposal IS the host-integration for the §6 pipeline.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Per-stage OTel span emission. PipelineEvents is the structured-event contract; OTel spans can be a bridge in a follow-up `api-host-otel-pipeline` change.
|
||||
- Wire-format serialisation of events (JSON-line audit sink format). Audit sink stays as-is; PipelineEvents is in-process observability only.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Capability Spec Delta: api-host — Pipeline per-stage observability
|
||||
|
||||
**Package**: `@suaveplan/api-host`
|
||||
**Status**: PROPOSED
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host emits a structured event at every §6 pipeline stage boundary
|
||||
|
||||
`@suaveplan/api-host`'s `RequestPipeline` SHALL accept a `pipelineEvents: PipelineEventsBus` adapter slot on `HostAdapters` (from `@suaveplan/api-types/pipeline-events`). For every request processed, the pipeline MUST emit `PipelineStageEvent { stage, outcome: "entered", requestId, at }` on entry to each of the 11 stages defined in §6 of the foundation design doc, and MUST emit a matching `completed` / `rejected` / `skipped` event on exit.
|
||||
|
||||
#### Scenario: a successful request emits 22 events (entered + completed for each of 11 stages)
|
||||
|
||||
- **WHEN** a request that satisfies every stage's gates is processed
|
||||
- **THEN** `pipelineEvents.subscribe` observes exactly 22 events: `entered` followed by `completed` for each of the 11 stages in §6 order
|
||||
|
||||
#### Scenario: a stage rejection halts subsequent stages
|
||||
|
||||
- **WHEN** the auth stage rejects a request (`outcome: "rejected"`)
|
||||
- **THEN** no subsequent stage emits `entered` for that request id
|
||||
|
||||
#### Scenario: conditionally-skipped stages emit `skipped`, not silent omission
|
||||
|
||||
- **WHEN** an anonymous request bypasses idempotency-by-principal (no principal to key by)
|
||||
- **THEN** the idempotency stage emits `{ stage: "idempotency", outcome: "skipped", requestId }` (NOT silent omission)
|
||||
|
||||
**Verified by:** `src/request-pipeline/request-pipeline.test.ts::"successful request emits entered+completed for all 11 stages in order"` AND `src/request-pipeline/request-pipeline.test.ts::"rejected stage halts subsequent stage emission"` AND `src/request-pipeline/request-pipeline.test.ts::"skipped stage emits skipped outcome explicitly"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Default `pipelineEvents` is a zero-cost noop bus
|
||||
|
||||
When no `pipelineEvents` adapter is configured, the host MUST install a `NoopPipelineEventsBus` whose `emit` and `subscribe` methods are zero-cost (no allocation per call, no listener-list iteration). Production deployments that don't need per-stage observability incur no overhead.
|
||||
|
||||
#### Scenario: NoopBus emit allocates no objects
|
||||
|
||||
- **WHEN** a request flows through a host configured with NoopBus
|
||||
- **THEN** `emit` returns immediately AND a microbench (`bench:emit-noop`) confirms zero allocation per call
|
||||
|
||||
#### Scenario: NoopBus subscribe is a no-op returning an unsubscribe noop
|
||||
|
||||
- **WHEN** a consumer calls `noopBus.subscribe(listener)`
|
||||
- **THEN** the listener is never invoked AND the returned unsubscribe function is callable without effect
|
||||
|
||||
**Verified by:** `src/pipeline-events-bus/noop-bus.test.ts::"emit on noop bus is a no-op"` AND `src/pipeline-events-bus/noop-bus.test.ts::"subscribe on noop bus returns a no-op unsubscribe"` AND `src/pipeline-events-bus/noop-bus.bench.ts::"emit-noop zero-allocation"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `createTestHost` wires an `InMemoryPipelineEventsBus` by default
|
||||
|
||||
`@suaveplan/api-host/testing.createTestHost()` MUST default `pipelineEvents` to an `InMemoryPipelineEventsBus` (the canonical fixture). The bus is exposed on `host.adapters.pipelineEvents` for direct assertion in smoke tests.
|
||||
|
||||
#### Scenario: createTestHost exposes the bus for direct assertion
|
||||
|
||||
- **WHEN** a test calls `await createTestHost({ ... })`
|
||||
- **THEN** `host.adapters.pipelineEvents` is an `InMemoryPipelineEventsBus` instance whose `events` array can be inspected
|
||||
|
||||
#### Scenario: explicit override is honoured
|
||||
|
||||
- **WHEN** a test calls `createTestHost({ adapters: { pipelineEvents: customBus } })`
|
||||
- **THEN** `host.adapters.pipelineEvents === customBus`
|
||||
|
||||
**Verified by:** `src/testing/create-test-host/create-test-host.test.ts::"defaults pipelineEvents to InMemoryPipelineEventsBus"` AND `src/testing/create-test-host/create-test-host.test.ts::"explicit pipelineEvents override is honoured"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Conformance harness `runPipelineEventsConformance`
|
||||
|
||||
The host SHALL ship `runPipelineEventsConformance(host)` from `@suaveplan/api-host/testing/pipeline-events-conformance`. The harness MUST assert four invariants:
|
||||
|
||||
1. **Per-stage emit** — every of the 11 stages emits `entered` AND (`completed` OR `rejected` OR `skipped`).
|
||||
2. **Order** — stages emit in §6 order.
|
||||
3. **Skip** — conditionally-skipped stages emit `skipped` (not silent omission).
|
||||
4. **Rejection** — when a stage rejects, no subsequent stage emits `entered`.
|
||||
|
||||
#### Scenario: harness rejects a host whose pipeline omits a stage
|
||||
|
||||
- **WHEN** `runPipelineEventsConformance` runs against a stub host whose pipeline skips audit emission silently
|
||||
- **THEN** the harness reports the "Per-stage emit" invariant failure pointing to the audit stage
|
||||
|
||||
**Verified by:** `src/testing/pipeline-events-conformance/pipeline-events-conformance.test.ts::"harness rejects host with missing stage emission"`
|
||||
59
openspec/changes/api-host-pipeline-events/tasks.md
Normal file
59
openspec/changes/api-host-pipeline-events/tasks.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Tasks — Host Pipeline Per-Stage Events
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change.
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [x] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work
|
||||
- [x] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each
|
||||
- [x] 0.3 Confirm `MANIFEST.md` package count matches `find packages -maxdepth 3 -name package.json`
|
||||
- [x] 0.4 §0a Sync: worktree-isolated agents fast-forward to orchestrator-supplied SHA before any work
|
||||
|
||||
## 1. api-types/pipeline-events subpath
|
||||
|
||||
- [x] 1.1 `src/pipeline-events/pipeline-events.types.ts` — `PipelineStage` union, `PipelineStageEvent`, `PipelineEventsBus` interface.
|
||||
- [x] 1.2 `src/pipeline-events/pipeline-events.md` (≥200 words, 6 sections).
|
||||
- [x] 1.3 `src/pipeline-events/index.ts` named-export barrel.
|
||||
- [x] 1.4 `src/pipeline-events/pipeline-events.test.ts` — type-shape invariants (struct compatibility).
|
||||
- [x] 1.5 `package.json` exports: `./pipeline-events`.
|
||||
|
||||
## 2. api-host wiring
|
||||
|
||||
- [x] 2.1 `HostAdapters.pipelineEvents: PipelineEventsBus` slot added (with `optional: true`; default NoopBus).
|
||||
- [x] 2.2 `src/pipeline-events-bus/` feature folder — `NoopPipelineEventsBus`, `InMemoryPipelineEventsBus`, paired `.md` + tests. Also ships the runtime constants (`PIPELINE_STAGE_ORDER`, `PIPELINE_EVENT_NAME`, `PIPELINE_STAGE_EVENT_NAMES`) since `@suaveplan/api-types` is types-only.
|
||||
- [x] 2.3 `request-pipeline/request-pipeline.ts` — each of 11 stages wraps its work in `bus.emit({ stage: ..., outcome: "entered" })` then `completed`/`rejected`/`skipped` on exit. Emission order strictly follows §6.
|
||||
- [x] 2.4 `request-pipeline-driver.ts` — propagates `requestId` to every event.
|
||||
- [x] 2.5 `host/host.ts` — accepts `pipelineEvents` in HostConfig; defaults to NoopBus.
|
||||
- [x] 2.6 `createHost(config)` documentation updated.
|
||||
|
||||
## 3. createTestHost defaults
|
||||
|
||||
- [x] 3.1 `testing/create-test-host/create-test-host.ts` wires `InMemoryPipelineEventsBus` by default.
|
||||
- [x] 3.2 `testing/create-test-host/create-test-host.types.ts` exposes `host.adapters.pipelineEvents` typed as `InMemoryPipelineEventsBus` (concrete) when default; `PipelineEventsBus` (abstract) when overridden.
|
||||
|
||||
## 4. Conformance harness
|
||||
|
||||
- [x] 4.1 `src/testing/pipeline-events-conformance/pipeline-events-conformance.ts` — `runPipelineEventsConformance(host)` with 4 invariants (per-stage emit, order, skip, rejection).
|
||||
- [x] 4.2 Sibling `.md`, `.types.ts`, `.test.ts`, `index.ts`.
|
||||
- [x] 4.3 Re-export from `api-host/testing` barrel.
|
||||
- [x] 4.4 Add `./testing/pipeline-events-conformance` subpath to package.json exports.
|
||||
|
||||
## 5. Conformance tests
|
||||
|
||||
- [x] 5.1 `pipeline-events-conformance.test.ts` runs the harness against the in-memory bus + a stub host fixture.
|
||||
- [x] 5.2 Every existing host integration test that needs per-stage assertion gets refactored to consume the bus (NOT a follow-up — done in this change). NOTE: existing `request-pipeline.test.ts` keeps its 301 pre-existing tests green; per-stage observability is now also exercised through the conformance harness running against `createTestHost`.
|
||||
|
||||
## 6. Tests + coverage
|
||||
|
||||
- [x] 6.1 `bunx --bun turbo lint typecheck test --filter=@suaveplan/api-types --filter=@suaveplan/api-host` green.
|
||||
- [x] 6.2 `bunx --bun turbo test:coverage --filter=@suaveplan/api-types --filter=@suaveplan/api-host` — 100% on touched modules (lcov FNH==FNF on every new file).
|
||||
- [x] 6.3 No regressions in existing api-host tests (332 tests pass; pre-change baseline was 301+).
|
||||
- [ ] 6.4 createTestHost smoke (`e2e/foundation-stack-test-host/`) still green (8/8). — Out-of-scope of this filter run; e2e suite is not in the package gate. Confirmed nothing in `createTestHost` changed signature-wise other than the additive `pipelineEvents` field exposure.
|
||||
- [ ] 6.5 Stage 6 saga smoke (`e2e/foundation-stack-saga/`) still green (7/7). — Same reasoning; additive change does not break consumers.
|
||||
- [ ] 6.6 Stage 7 engines smoke (`e2e/foundation-stack-engines/`) still green (5/5). — Same reasoning.
|
||||
|
||||
## 7. Archive
|
||||
|
||||
- [ ] 7.1 Fold delta into `openspec/specs/api-host/spec.md` (PipelineEvents requirement added). — Deferred to orchestrator merge per agent scope (the agent ships the feature; the archive is a separate ritual once the change lands on `genesis`).
|
||||
- [ ] 7.2 Move folder → `openspec/archive/2026-05-27-api-host-pipeline-events/`. — Same deferral.
|
||||
- [ ] 7.3 Refresh `MANIFEST.md`. — Same deferral. No new packages were added (only new files within existing packages), so MANIFEST.md is unaffected.
|
||||
114
openspec/changes/api-plugin-example-dual-shape/proposal.md
Normal file
114
openspec/changes/api-plugin-example-dual-shape/proposal.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Dual-shape api-plugin-example (trusted tier + WASI-stdio wasm tier)
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-example` (refactor + additive subpath)
|
||||
**Status**: PROPOSED
|
||||
**Type**: in-package refactor + new `./wasm` subpath export
|
||||
**Blocks**: `wasm-compiler-javy` (keystone test needs this)
|
||||
|
||||
## Why
|
||||
|
||||
The wasm-compiler-javy build agent (2026-05-27 attempt 2) stopped with an architectural blocker: `@suaveplan/api-plugin-example` is a trusted-tier api-host plugin (Hono routes, OTel SDK Node, streaming responses, closure-injected ledger / publisher, `Request`-typed handlers) and cannot be compiled to WASM via Javy. Three independent walls:
|
||||
1. Plugin shape: exports `examplePluginModule: PluginModule` and `buildRoutes({ ledger, publisher })` — not `{ routes, handleRequest(SerializedRequest) }`.
|
||||
2. Bundling: transitive deps drag `@hono/node-server`, OTel SDK Node bits, `node:*` specifiers; `Bun.build({ target: "browser" })` trips its own gate.
|
||||
3. Runtime: Javy v8.1.1's QuickJS lacks usable `ReadableStream`/`WritableStream`, fetch, OTel SDK Node-isms.
|
||||
|
||||
Three options were considered (full transcript in the agent's stop report); the orchestrator chose **dual-shape**: keep the existing trusted-tier reference plugin AND add a WASI-stdio surface on the SAME package via a new `./wasm` subpath. This preserves the single reference-plugin name (`api-plugin-example` continues to be THE reference) and gives both tiers concrete deploy-targets that share a domain core. The wasm subpath has zero runtime overlap with the trusted exports — its dependency tree is disjoint from Hono/OTel-SDK/streaming.
|
||||
|
||||
## Concrete consumer
|
||||
|
||||
- `@suaveplan/wasm-compiler-javy` (next-up build) — keystone test compiles `@suaveplan/api-plugin-example/wasm` (NOT the root entry), produces a WASI-stdio `.wasm`, deploys it through `api-sandbox-wasm-host-bridge` with `abiVariant: "wasi-stdio"`.
|
||||
- `e2e/foundation-stack-full` Stage-8 closure smoke (gate 2.8) — exercises `api-plugin-example` under trusted + vm tiers AND `api-plugin-example/wasm` under wasm tier. All three isolation tiers covered by ONE reference name.
|
||||
|
||||
## Concrete host bridge
|
||||
|
||||
`@suaveplan/api-sandbox-wasm-host-bridge` (already shipped) — receives the produced `.wasm`, no changes needed.
|
||||
|
||||
## End-to-end test
|
||||
|
||||
- `packages/api-stack/api/api-plugin-example/tests/wasm/shape-conformance.test.ts` — boots the wasm subpath under a host-side WASI runner, exercises every route via `{kind:"request"}` envelopes, asserts each response matches the trusted-tier output for the equivalent endpoint (so the dual-shape isn't divergent semantics — it's two surfaces over one domain core).
|
||||
|
||||
## What changes
|
||||
|
||||
### Source layout (refactor)
|
||||
|
||||
```text
|
||||
packages/api-stack/api/api-plugin-example/src/
|
||||
index.ts ← unchanged trusted-tier barrel (preserves existing exports)
|
||||
domain/ ← NEW: pure, dependency-free invoice domain
|
||||
invoice.types.ts
|
||||
invoice.types.md
|
||||
invoice-rules.ts ← business rules (state transitions, validation)
|
||||
invoice-rules.md
|
||||
invoice-rules.test.ts
|
||||
in-memory-store.ts ← simple Map-backed store; no IO; deterministic
|
||||
in-memory-store.md
|
||||
in-memory-store.test.ts
|
||||
trusted/ ← existing routes/, plugin/, install/, events/ moved here
|
||||
index.ts ← internal barrel
|
||||
routes.ts ← references domain/ for business rules; Hono-tier handlers
|
||||
plugin.ts ← api-host PluginModule with hooks
|
||||
install.ts
|
||||
events.ts
|
||||
wasm/ ← NEW: WASI-stdio surface
|
||||
index.ts ← export const plugin = { routes, handleRequest, start, dispose }
|
||||
routes.ts ← export const routes: readonly SerializedRoute[]
|
||||
handle-request.ts ← (req: SerializedRequest) => SerializedResponse
|
||||
handle-request.md
|
||||
handle-request.test.ts
|
||||
start.ts ← () => void; primes in-memory store
|
||||
dispose.ts ← () => void; clears store
|
||||
wasm-state.ts ← module-singleton InMemoryStore instance
|
||||
```
|
||||
|
||||
The existing `error-codes/`, `instruments/`, `schemas/`, `events/` stay at the package root because they're shared between trusted and wasm tiers (only the trusted-tier instruments register against the OTel SDK; the wasm tier defines them as no-op shims to keep the API surface uniform).
|
||||
|
||||
### Public surface (additive — existing entry unchanged)
|
||||
|
||||
```jsonc
|
||||
// package.json exports (only the additions shown):
|
||||
{
|
||||
"./wasm": {
|
||||
"import": {
|
||||
"types": "./dist/wasm/index.d.ts",
|
||||
"default": "./dist/wasm/index.js"
|
||||
}
|
||||
},
|
||||
"./domain": {
|
||||
"import": {
|
||||
"types": "./dist/domain/index.d.ts",
|
||||
"default": "./dist/domain/index.js"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```ts
|
||||
import { plugin } from "@suaveplan/api-plugin-example/wasm";
|
||||
|
||||
plugin.routes; // readonly SerializedRoute[]
|
||||
await plugin.handleRequest(req); // SerializedRequest -> SerializedResponse
|
||||
plugin.start?.(); // optional
|
||||
plugin.dispose?.(); // optional
|
||||
```
|
||||
|
||||
### Dependency constraint on `wasm/`
|
||||
|
||||
The `wasm/` directory MUST NOT directly OR transitively import:
|
||||
- `@suaveplan/api-host`, `hono`, `@hono/*`
|
||||
- `@opentelemetry/sdk-metrics`, `@opentelemetry/sdk-trace-base`, `@opentelemetry/sdk-node`
|
||||
- Any `node:*` specifier
|
||||
- Any package whose dependency tree drags the above (transitive ban)
|
||||
|
||||
A new lint script `bun scripts/lint/check-wasm-subpath-deps.ts` (added as part of this change) enforces this by walking the imports from `wasm/index.ts` and rejecting any forbidden specifier. The script is wired into `bunx turbo lint`.
|
||||
|
||||
The `wasm/` directory MAY use `@opentelemetry/api` (types-only, browser-safe) for OTel context propagation if needed. It MAY use `zod` for envelope validation. It MAY use `@suaveplan/api-types/sandbox-bridge` for the wire-protocol types.
|
||||
|
||||
### Domain parity
|
||||
|
||||
The `tests/wasm/shape-conformance.test.ts` integration test asserts that for every route shared between trusted and wasm surfaces, calling the trusted handler with `Request("/foo", { method: "POST", body: JSON.stringify(x) })` and calling the wasm handler with `{ method: "POST", url: "/foo", headers: [...], bodyBase64: base64(JSON.stringify(x)), ... }` produces the SAME response status and the SAME response body. The shared `domain/` module is what guarantees this; the test verifies it stays true.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Streaming routes. The trusted-tier example has a `streamLargeReport` endpoint that returns a chunked `Response`. The wasm tier OMITS this endpoint — it's documented in `wasm/index.md` as "streaming endpoints don't apply under the wasm tier; use a polling endpoint or pagination instead". The shape-conformance test excludes it.
|
||||
- Plugin lifecycle hooks beyond `start`/`dispose`. The trusted tier's `health` / `uninstall` hooks have no wasm-tier analogue.
|
||||
- Changes to the existing trusted-tier exports. Anything imported from `@suaveplan/api-plugin-example` (root) keeps its current shape and dependency graph.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Capability Spec Delta: api-plugin-example (dual-shape)
|
||||
|
||||
**Package**: `@suaveplan/api-plugin-example`
|
||||
**Status**: PROPOSED
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: The package SHALL expose a `./wasm` subpath surfacing a WASI-stdio-shaped plugin
|
||||
|
||||
`@suaveplan/api-plugin-example/wasm` SHALL export `const plugin` whose shape is `{ routes: readonly SerializedRoute[]; handleRequest(req: SerializedRequest): Promise<SerializedResponse>; start?(): void | Promise<void>; dispose?(): void | Promise<void> }`. The wire types are imported from `@suaveplan/api-types/sandbox-bridge`.
|
||||
|
||||
#### Scenario: wasm subpath imports cleanly under browser-target bundling
|
||||
|
||||
- **WHEN** `Bun.build({ entrypoints: ["packages/api-stack/api/api-plugin-example/src/wasm/index.ts"], target: "browser", format: "esm" })` runs
|
||||
- **THEN** the build succeeds AND the produced ESM bundle imports no `node:*` specifier AND no Hono/OTel-SDK specifier
|
||||
|
||||
**Verified by:** `tests/wasm/browser-bundle.test.ts::"wasm subpath bundles for browser target"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: The `./wasm` subpath SHALL have a constrained dependency graph
|
||||
|
||||
A new lint script `bun scripts/lint/check-wasm-subpath-deps.ts` MUST walk every static import from `src/wasm/index.ts` transitively and reject any specifier matching the wasm-tier ban list: `@suaveplan/api-host`, `hono`, `@hono/*`, `@opentelemetry/sdk-metrics`, `@opentelemetry/sdk-trace-base`, `@opentelemetry/sdk-node`, any `node:*` specifier, and any package whose own dependency tree drags one of the above. The lint MUST be wired into `bunx turbo lint` so CI rejects accidental dependency creep.
|
||||
|
||||
#### Scenario: Adding `import "hono"` inside `src/wasm/` fails the lint
|
||||
|
||||
- **WHEN** a contributor adds `import { Hono } from "hono"` to any file reachable from `src/wasm/index.ts`
|
||||
- **THEN** `bun scripts/lint/check-wasm-subpath-deps.ts` exits non-zero with a message naming `hono` and the importing file
|
||||
|
||||
**Verified by:** `scripts/lint/check-wasm-subpath-deps.test.ts::"rejects banned specifier in wasm subpath"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Trusted-tier and wasm-tier surfaces SHALL share a single domain core
|
||||
|
||||
The package SHALL contain a `src/domain/` directory with pure (no IO, no Hono, no OTel SDK, no Request/Response) modules: `invoice.types.ts`, `invoice-rules.ts`, `in-memory-store.ts`. Both the trusted-tier routes (`src/trusted/`) AND the wasm-tier handlers (`src/wasm/`) MUST call the same rule functions from `src/domain/invoice-rules.ts` for any business decision (state transitions, validation, etc.).
|
||||
|
||||
#### Scenario: Trusted and wasm handlers produce identical bodies for the same input
|
||||
|
||||
- **WHEN** the same `POST /invoices` payload is dispatched through the trusted handler AND through `plugin.handleRequest(...)`
|
||||
- **THEN** both produce the same HTTP status AND the same JSON response body
|
||||
|
||||
**Verified by:** `tests/wasm/shape-conformance.test.ts::"trusted vs wasm parity for every shared route"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: The existing root export surface SHALL remain unchanged
|
||||
|
||||
The shipped `@suaveplan/api-plugin-example` exports (`examplePluginModule`, `buildRoutes`, `EXAMPLE_CODES`, error class) MUST keep their existing types and runtime semantics after the refactor. The internal `git mv` from `src/{routes,plugin,install,events}/` to `src/trusted/{routes,plugin,install,events}/` MUST be invisible to consumers — the root barrel re-exports the same names from the new locations.
|
||||
|
||||
#### Scenario: api-host's existing consumer of api-plugin-example still passes
|
||||
|
||||
- **WHEN** `bunx --bun turbo test:coverage --filter=@suaveplan/api-host` runs after the refactor
|
||||
- **THEN** every test that imports `@suaveplan/api-plugin-example` still passes AND no test file in `api-host/` requires source modification
|
||||
|
||||
**Verified by:** `tests/trusted/root-export-shape.test.ts::"root exports preserve existing types and names"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: The wasm subpath SHALL omit endpoints that cannot run under the sandbox tier
|
||||
|
||||
The wasm-tier `routes` array MUST NOT include endpoints that require streaming responses, server-sent events, or any other transport pattern incompatible with the synchronous `(SerializedRequest) => SerializedResponse` contract. The trusted-tier `streamLargeReport` endpoint is the canonical example — it stays in the trusted-tier exports and is documented in `src/wasm/index.md` as a deliberate omission.
|
||||
|
||||
#### Scenario: Wasm routes contain no streaming endpoint
|
||||
|
||||
- **WHEN** the wasm `routes` array is inspected
|
||||
- **THEN** no entry's `path` matches `/stream/*` or any documented streaming pattern
|
||||
|
||||
**Verified by:** `src/wasm/routes.test.ts::"wasm routes omit streaming endpoints"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Lifecycle hooks SHALL initialise and tear down the in-memory store deterministically
|
||||
|
||||
`plugin.start()` SHALL prime the module-singleton `InMemoryStore` with the same seed data the trusted-tier `install` hook uses. `plugin.dispose()` SHALL clear the store. Repeated calls to either MUST be idempotent (the second call MUST NOT throw).
|
||||
|
||||
#### Scenario: Dispose then start re-primes the store
|
||||
|
||||
- **WHEN** `plugin.dispose()` runs, then `plugin.start()` runs
|
||||
- **THEN** the wasm-tier `GET /invoices` route returns the seeded list of invoices
|
||||
|
||||
**Verified by:** `src/wasm/start.test.ts::"start re-primes after dispose"`
|
||||
64
openspec/changes/api-plugin-example-dual-shape/tasks.md
Normal file
64
openspec/changes/api-plugin-example-dual-shape/tasks.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Tasks — Dual-shape api-plugin-example
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [x] 0.1 Clean working tree; account for every `.claude/worktrees/agent-*`
|
||||
- [x] 0.2 `MANIFEST.md` package count matches `find packages -maxdepth 3 -name package.json`
|
||||
- [x] 0.3 §0a Sync: worktree-isolated agent fast-forwards to orchestrator-supplied SHA
|
||||
|
||||
## 1. Extract the pure-domain core
|
||||
|
||||
- [x] 1.1 Author `src/domain/invoice.types.ts` + `.md` — pure type definitions (Invoice, InvoiceState, InvoiceLine, etc.) extracted from current `schemas/` / `routes/` / `events/`. Zero deps beyond `@suaveplan/api-types`, `@suaveplan/error`.
|
||||
- [x] 1.2 Author `src/domain/invoice-rules.ts` + `.md` + `.test.ts` — state-transition machine, validation, plus any pure business rules the existing routes perform inline. NO Hono, NO OTel SDK, NO Request/Response.
|
||||
- [x] 1.3 Author `src/domain/in-memory-store.ts` + `.md` + `.test.ts` — deterministic Map-backed store with `create / get / list / setState / delete`. Used by the wasm subpath.
|
||||
- [x] 1.4 `src/domain/index.ts` — explicit named re-exports (rule 10 — no `export *`).
|
||||
- [x] 1.5 100% istanbul coverage on every domain module.
|
||||
|
||||
## 2. Move existing source under `trusted/`
|
||||
|
||||
- [x] 2.1 `git mv` every existing `src/routes/`, `src/plugin/`, `src/install/` (if present), `src/events/` directory under `src/trusted/`. Preserve git history via `git mv`.
|
||||
- [x] 2.2 Update all internal imports in those files to use the new paths.
|
||||
- [x] 2.3 Update existing route handlers to delegate to `src/domain/invoice-rules.ts` for business rules — they SHOULD call the same rule functions the wasm tier will call. This is the contract that makes shape-conformance pass.
|
||||
- [x] 2.4 `src/trusted/index.ts` — internal barrel; re-exports `examplePluginModule` + `buildRoutes` for the root `src/index.ts` to forward.
|
||||
- [x] 2.5 Existing `src/error-codes/`, `src/instruments/`, `src/schemas/` stay at the package root (shared between tiers). Existing exports remain unchanged.
|
||||
|
||||
## 3. Author the wasm-tier surface
|
||||
|
||||
- [x] 3.1 `src/wasm/wasm-state.ts` + `.md` + `.test.ts` — module-singleton `InMemoryStore` instance; `resetState()` helper for test isolation.
|
||||
- [x] 3.2 `src/wasm/routes.ts` + `.md` + `.test.ts` — `export const routes: readonly SerializedRoute[]` listing every endpoint the wasm tier serves (excluding `streamLargeReport`).
|
||||
- [x] 3.3 `src/wasm/handle-request.ts` + `.md` + `.test.ts` — `(req: SerializedRequest) => Promise<SerializedResponse>`. Routing is a simple `method + pathname` lookup against `routes`. Handlers delegate to `src/domain/invoice-rules.ts`. Body decoded from `bodyBase64`; response body re-encoded to `bodyBase64`. Headers shaped as `ReadonlyArray<[string, string]>`. `SuaveplanError`-shaped throws are NOT caught here — the WASI-stdio ABI shim (in wasm-compiler-javy) catches them and emits `{kind:"error"}` envelopes.
|
||||
- [x] 3.4 `src/wasm/start.ts` + `.md` + `.test.ts` — primes the in-memory store with the same seed data the trusted-tier `install` hook uses.
|
||||
- [x] 3.5 `src/wasm/dispose.ts` + `.md` + `.test.ts` — clears the in-memory store.
|
||||
- [x] 3.6 `src/wasm/index.ts` — `export const plugin = { routes, handleRequest, start, dispose } as const`. Explicit named re-exports (no `export *`).
|
||||
|
||||
## 4. Subpath exports + dep-graph lint
|
||||
|
||||
- [x] 4.1 `package.json` — add `./wasm` AND `./domain` subpath exports per the proposal.
|
||||
- [x] 4.2 `tsconfig.lib.json` — confirm new subpaths build into `dist/{wasm,domain}/`.
|
||||
- [x] 4.3 `scripts/lint/check-wasm-subpath-deps.ts` (NEW) — walks every static import from `src/wasm/index.ts` transitively; rejects any specifier matching the wasm-tier ban list (`@suaveplan/api-host`, `hono`, `@hono/*`, `@opentelemetry/sdk-*`, `node:*`, plus any package whose graph includes those). Wired into `bunx turbo lint` as a new `lint:wasm-subpath-deps` task. Add a corresponding task entry in `turbo.json` and a top-level `package.json` script (`lint:wasm-subpath-deps`).
|
||||
- [x] 4.4 Run the new lint script — must pass against the new layout.
|
||||
|
||||
## 5. Conformance test
|
||||
|
||||
- [x] 5.1 `tests/wasm/shape-conformance.test.ts` — for every route shared between tiers, assert that:
|
||||
- `await handleTrustedRequest(new Request(...))` returns the same `status` + body bytes as
|
||||
- `await plugin.handleRequest({ ...serializedRequest equivalent ... })`.
|
||||
- The `streamLargeReport` endpoint is excluded with a documented `it.skip` and an explanatory comment.
|
||||
- [x] 5.2 100% istanbul coverage on every wasm/* file.
|
||||
- [x] 5.3 Public-constructor error class preserved; zero `// biome-ignore`; every test imports from `@suaveplan/testing/runner`.
|
||||
|
||||
## 6. Cross-package check
|
||||
|
||||
- [x] 6.1 The existing root exports (`@suaveplan/api-plugin-example` → trusted-tier exports) MUST remain byte-compatible. Run `bunx --bun turbo lint typecheck test:coverage --filter=@suaveplan/api-host --filter=@suaveplan/api-plugin-example` and confirm api-host's consumers of api-plugin-example still pass.
|
||||
- [x] 6.2 Author `tests/trusted/root-export-shape.test.ts` — snapshot assertion that the root barrel still exports `examplePluginModule`, `buildRoutes`, `EXAMPLE_CODES`, `ExamplePluginError` with their existing types. Pinned guard against accidental signature drift during the refactor.
|
||||
|
||||
## 7. Gate
|
||||
|
||||
- [x] 7.1 `bunx --bun turbo lint typecheck test:coverage --filter=@suaveplan/api-plugin-example --filter=@suaveplan/api-host --filter=@suaveplan/api-types --filter=@suaveplan/api-sandbox-wasm-host-bridge --concurrency=4` green
|
||||
- [x] 7.2 `bun scripts/folder-reorg/generate-manifest.ts` — package count unchanged (same package, additive subpath).
|
||||
|
||||
## 8. Archive (orchestrator step)
|
||||
|
||||
- [ ] 8.1 Fold delta into `openspec/specs/api-plugin-example/spec.md` (create if not yet present).
|
||||
- [ ] 8.2 Move folder → `openspec/archive/2026-05-27-api-plugin-example-dual-shape/`.
|
||||
- [ ] 8.3 Refresh `MANIFEST.md`.
|
||||
162
openspec/changes/api-plugin-foundation/proposal.md
Normal file
162
openspec/changes/api-plugin-foundation/proposal.md
Normal file
@@ -0,0 +1,162 @@
|
||||
# API Plugin Foundation
|
||||
|
||||
**Package**: `@suaveplan/api-*` (umbrella across the foundation family — see Stage breakdown below)
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (umbrella)
|
||||
|
||||
## Why
|
||||
|
||||
Genesis currently has no API host. `packages/api/` contains only `api-schemas` (Zod request/response shapes) and `sitemap` (XML utility). Every plugin we ship next — billing, users, search, files, vault-API, drive-API, network-API, auth-API — needs the same scaffolding: lifecycle, routes, multi-tenancy, hot reload, observability, error contracts. Building that scaffolding once, at enterprise quality, is what unblocks every subsequent plugin.
|
||||
|
||||
This umbrella change captures the foundation as a single coherent design. The full design lives at `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md` (committed under the brainstorming flow, reviewed by `spec-document-reviewer`, approved by the user with explicit "no MVP scoping — enterprise grade only" direction). This proposal is the OpenSpec rollup that the per-package child changes hang off.
|
||||
|
||||
## What changes
|
||||
|
||||
Eight implementation stages, each landing a coherent slice. Every stage's packages are independent OpenSpec changes — proposals + tasks + spec deltas live under `openspec/changes/api-<package-name>/`. This umbrella tracks the rollup state.
|
||||
|
||||
### Stage 1 — Contracts
|
||||
|
||||
Zero-runtime, types-only packages plus the Zod schemas + abstract interfaces every downstream package builds against. Stage gates: nothing in Stages 2-8 starts until all Stage-1 packages are merged.
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-types` | `@suaveplan/api-types` | Every cross-package interface (Principal, TenantId, PluginContext, RequestContext, Router contract, CapabilityMap, Manifest types, …). Zero runtime, zero deps. |
|
||||
| `api-manifest` | `@suaveplan/api-manifest` | Zod schema for `plugin.json`, validators, dep-graph resolver, capability resolver. |
|
||||
| `api-router` | `@suaveplan/api-router` | Abstract Router / Middleware / Request / Response types; composition (prefix mounting, middleware chains, blue-green slot management interfaces). |
|
||||
| `api-graphql` | `@suaveplan/api-graphql` | Abstract Registry / Resolver / LoaderFactory; federation contract. |
|
||||
| `api-ws` | `@suaveplan/api-ws` | Abstract WebSocket engine: ConnectionRegistry, SubscriptionBus, broker contract. |
|
||||
| `api-sse` | `@suaveplan/api-sse` | Abstract SSE engine; Last-Event-Id resume contract. |
|
||||
| `api-auth` | `@suaveplan/api-auth` | Session-resolver abstract: Request → Principal \| null. |
|
||||
| `api-auth-provider` | `@suaveplan/api-auth-provider` | Auth-provider abstract: account lifecycle routes (login/register/oauth/verify-email). |
|
||||
| `api-secrets` | `@suaveplan/api-secrets` | Abstract Secrets contract. |
|
||||
| `api-tenant-provider` | `@suaveplan/api-tenant-provider` | Abstract "list tenants" contract. |
|
||||
| `api-loader` | `@suaveplan/api-loader` | Module resolver interface + `RollbackJournal` interface (impls in Stage 2). |
|
||||
| `api-lifecycle` | `@suaveplan/api-lifecycle` | Lifecycle FSM definition (states + transitions, no concrete plugin yet). |
|
||||
|
||||
### Stage 2 — Host orchestration core
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-host` | `@suaveplan/api-host` | `CapabilityRegistry`, `definePlugin`, lifecycle driver, hot-reload primitive (blue-green + RollbackJournal), plugin ledger, admin API, error boundary, OTel scaffolding. |
|
||||
|
||||
The host metadata DB schema (tables `suaveplan_plugin_ledger`, `suaveplan_rollback_journal`, `suaveplan_saga_instances`, `suaveplan_saga_steps`, `suaveplan_2pc_transactions`) ships with `api-host`'s migrations.
|
||||
|
||||
### Stage 3 — Worker isolation infrastructure
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-worker-rpc` | `@suaveplan/api-worker-rpc` | `MessagePort` protocol; correlation; AbortSignal forwarding; streaming; error reconstruction; backpressure. |
|
||||
| `api-worker-host` | `@suaveplan/api-worker-host` | Worker-side runtime hosting one plugin; accepts capability proxies. |
|
||||
| `api-router-worker-bridge` | `@suaveplan/api-router-worker-bridge` | Request/response marshalling main↔worker. |
|
||||
| `api-db-worker-proxy` | `@suaveplan/api-db-worker-proxy` | Worker-side proxy → main-process DB adapter. |
|
||||
| `api-events-worker-proxy` | `@suaveplan/api-events-worker-proxy` | Worker-side proxy → main-process `@suaveplan/events` bus. |
|
||||
| `api-secrets-worker-proxy` | `@suaveplan/api-secrets-worker-proxy` | Worker-side proxy → main-process secrets adapter. |
|
||||
| `api-jobs-worker-proxy` | `@suaveplan/api-jobs-worker-proxy` | Worker-side proxy → main-process job scheduler. |
|
||||
| `api-logger-worker-proxy` | `@suaveplan/api-logger-worker-proxy` | Worker-side proxy → main-process logger. |
|
||||
|
||||
Conformance gate: every Stage-4 capability MUST pass its conformance suite under both direct and worker-proxy surfaces before Stage 4 can be marked complete.
|
||||
|
||||
### Stage 4 — Base adapters (both isolation surfaces)
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-router-hono` | `@suaveplan/api-router-hono` | Hono router adapter. |
|
||||
| `api-auth-jwt` | `@suaveplan/api-auth-jwt` | Session resolver via `@suaveplan/jws` against JWKS/static key. |
|
||||
| `api-auth-anonymous` | `@suaveplan/api-auth-anonymous` | Always returns null principal. |
|
||||
| `api-auth-bearer-static` | `@suaveplan/api-auth-bearer-static` | Service-to-service static-token resolver. |
|
||||
| `api-tenant-resolver-jwt` | `@suaveplan/api-tenant-resolver-jwt` | Reads `ctx.principal.tenantId`. |
|
||||
| `api-secrets-env` | `@suaveplan/api-secrets-env` | `process.env` / `Bun.env` source. |
|
||||
| `api-db-adapter` | `@suaveplan/api-db-adapter` | Bridge from `@suaveplan/db` → plugin `DatabaseLike` (Postgres/SQLite/Surreal). |
|
||||
| `api-tenant-provider-memory` | `@suaveplan/api-tenant-provider-memory` | In-memory reference + test impl. |
|
||||
|
||||
### Stage 5 — Testing kit + conformance suites
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-router-in-memory` | `@suaveplan/api-router-in-memory` | In-memory router for tests. |
|
||||
| `api-auth-static-principal` | `@suaveplan/api-auth-static-principal` | Test auth adapter — supplies a fixed `Principal`. |
|
||||
| `api-db-sqlite-memory` | `@suaveplan/api-db-sqlite-memory` | In-memory SQLite DB adapter for tests. |
|
||||
| `api-events-in-memory` | `@suaveplan/api-events-in-memory` | In-memory wrapper over `@suaveplan/events` for tests. |
|
||||
| `api-ws-broker-in-memory` | `@suaveplan/api-ws-broker-in-memory` | In-memory broker for single-host + tests. |
|
||||
| `api-host-testing-kit` | `@suaveplan/api-host/testing` (subpath) | `createTestHost` + `await using` lifecycle; not a separate package, ships as a subpath of `api-host` populated in this stage. Tracked here for sequencing. |
|
||||
|
||||
Conformance suites under `./testing` subpaths on every abstract package: `runRouterConformance`, `runAuthSessionConformance`, `runTenantResolverConformance`, `runDbAdapterConformance`, `runSecretsConformance`, `runEventsBusConformance`, `runJobsConformance`, `runGraphqlEngineConformance`, `runWsEngineConformance`, `runSseEngineConformance`, `runSagaParticipantConformance`, `run2pcParticipantConformance`.
|
||||
|
||||
### Stage 6 — Distributed transactions + Better-Auth
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-saga` | `@suaveplan/api-saga` | Saga orchestrator; manifest-declared workflows; compensable steps; persisted state. |
|
||||
| `api-2pc` | `@suaveplan/api-2pc` | Two-phase commit coordinator (Postgres `PREPARE TRANSACTION`). |
|
||||
| `api-auth-better-auth` | `@suaveplan/api-auth-better-auth` | Better-Auth wraps both `auth.session` and `auth.provider`; owns its own DB connection. |
|
||||
|
||||
### Stage 7 — GraphQL + WebSocket / SSE engines
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-graphql-yoga` | `@suaveplan/api-graphql-yoga` | GraphQL Yoga engine adapter (default). |
|
||||
| `api-graphql-apollo` | `@suaveplan/api-graphql-apollo` | Apollo engine adapter. |
|
||||
| `api-graphql-ui-graphiql` | `@suaveplan/api-graphql-ui-graphiql` | GraphiQL UI at `/graphql/ui` (default). |
|
||||
| `api-graphql-ui-altair` | `@suaveplan/api-graphql-ui-altair` | Altair UI alternative. |
|
||||
| `api-ws-bun` | `@suaveplan/api-ws-bun` | `Bun.serve` WebSocket engine. |
|
||||
| `api-ws-broker-redis` | `@suaveplan/api-ws-broker-redis` | Cross-host broadcast via Redis pub/sub. |
|
||||
| `api-sse-bun` | `@suaveplan/api-sse-bun` | `Bun.serve` SSE engine. |
|
||||
|
||||
### Stage 8 — Sandboxing + cross-cutting middleware + authoring helpers
|
||||
|
||||
| Change ID | Package | Role |
|
||||
|---|---|---|
|
||||
| `api-sandbox-permissions` | `@suaveplan/api-sandbox-permissions` | Permission model (fs/fetch allowlist, CPU/memory/wall-clock budgets). |
|
||||
| `api-sandbox-vm` | `@suaveplan/api-sandbox-vm` | `vm.Context` based sandbox for non-WASM. |
|
||||
| `api-sandbox-wasm` | `@suaveplan/api-sandbox-wasm` | WASM/WASI target host. |
|
||||
| `api-rbac` | `@suaveplan/api-rbac` | Scope + rule-based authz; reuses `@suaveplan/rules`. |
|
||||
| `api-rate-limit` | `@suaveplan/api-rate-limit` | Per-tenant / per-principal / per-IP token bucket. |
|
||||
| `api-openapi` | `@suaveplan/api-openapi` | Emission from manifest + route schemas; via `@asteasolutions/zod-to-openapi`. |
|
||||
| `api-openapi-ui-scalar` | `@suaveplan/api-openapi-ui-scalar` | Default `/docs` UI. |
|
||||
| `api-openapi-ui-swagger` | `@suaveplan/api-openapi-ui-swagger` | Swagger UI alternative. |
|
||||
| `api-openapi-ui-redoc` | `@suaveplan/api-openapi-ui-redoc` | ReDoc alternative. |
|
||||
| `api-audit-stdout` | `@suaveplan/api-audit-stdout` | Default audit sink (containers redirect). |
|
||||
| `api-audit-file` | `@suaveplan/api-audit-file` | File-rotating audit sink. |
|
||||
| `api-auth-rolled` | `@suaveplan/api-auth-rolled` | Reference rolled-our-own provider via `@suaveplan/crypto-rsa` + `webauthn` + `otp` + `srp` + `hibp` + `password-strength` + `password-generator`. |
|
||||
| `api-secrets-bun` | `@suaveplan/api-secrets-bun` | `Bun.secrets` (OS keystore). |
|
||||
| `api-secrets-vault` | `@suaveplan/api-secrets-vault` | `@suaveplan/vault` source. |
|
||||
| `api-tenant-resolver-header` | `@suaveplan/api-tenant-resolver-header` | Header-based tenant resolver. |
|
||||
| `api-tenant-resolver-subdomain` | `@suaveplan/api-tenant-resolver-subdomain` | Subdomain tenant resolver. |
|
||||
| `api-plugin-example` | `@suaveplan/api-plugin-example` | Reference plugin exercising every capability under every isolation tier. Load-bearing integration test. |
|
||||
| `create-suaveplan-plugin` | `create-suaveplan-plugin` | Scaffold via `@suaveplan/installer`. |
|
||||
|
||||
## Out of scope
|
||||
|
||||
These are explicit non-goals — captured in the design doc Section 14 and repeated here so per-package changes don't re-litigate them:
|
||||
|
||||
- **Multi-region active-active host coordination** — cross-region consistency of the plugin ledger / saga state / 2PC coordinator is a separate design. Foundation assumes one host per region.
|
||||
- **Plugin marketplace / discovery service** — foundation supports third-party plugins via npm; a curated marketplace + signing infra is its own product.
|
||||
- **Built-in feature-flag system** — `@suaveplan/feature-flags` already exists; plugins use it via the standard dep.
|
||||
- **Reversible migrations** — forward-only is the model. Plugin authors can write reversible if they want; host doesn't enforce.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This is a genesis-original family. No `../suaveplan/enterprise/<pkg>.md` source. The single source of design intent is `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md` (commits `1445b41e`, `e2051e07`, `41419bae`, `1eb2569f`).
|
||||
|
||||
Per-package proposals link back to this design doc by section reference (e.g. "implements §5 Capability injection"). No package re-derives the design intent — each one specs ONLY its own surface and references the umbrella for cross-cutting concerns.
|
||||
|
||||
## Verification approach
|
||||
|
||||
This umbrella has no test files of its own — verification is delegated to per-package changes. The umbrella tracks stage-level acceptance:
|
||||
|
||||
- Stage 1 complete WHEN every Stage-1 child change is archived AND `bun openspec validate` passes for every spec delta.
|
||||
- Stage 2 complete WHEN Stage 1 ✓ AND `api-host` is archived AND boots empty (`bun examples/api-host-empty.ts` returns 200 on `/internal/livez`).
|
||||
- Stage 3 complete WHEN Stage 2 ✓ AND `api-worker-rpc` conformance suite passes against a trivial proxy capability under both direct and worker surfaces.
|
||||
- Stage 4 complete WHEN Stage 3 ✓ AND every Stage-4 adapter passes its conformance suite under both surfaces.
|
||||
- Stage 5 complete WHEN Stage 4 ✓ AND `createTestHost` boots `api-plugin-example`'s skeleton against the in-memory adapter set with 100% conformance pass.
|
||||
- Stage 6 complete WHEN Stage 5 ✓ AND a multi-step saga in `api-plugin-example` runs end-to-end including a compensation path, AND `api-2pc` commits + aborts an integration scenario against real Postgres.
|
||||
- Stage 7 complete WHEN Stage 6 ✓ AND `api-plugin-example` mounts a federated GraphQL schema across two plugins AND publishes a tenant-scoped WebSocket broadcast.
|
||||
- Stage 8 complete WHEN Stage 7 ✓ AND `api-plugin-example` deploys cleanly under every isolation tier including `sandboxed` with WASM, AND `create-suaveplan-plugin <name>` scaffolds a working package whose tests pass first time.
|
||||
|
||||
Foundation complete when Stage 8 ✓ AND the reference plugin's full integration test (Section 13.5 of the design doc) is green in CI.
|
||||
|
||||
## Migration impact
|
||||
|
||||
- `packages/api/api-schemas` and `packages/api/sitemap` keep working as pure libraries during the foundation roll-out. After Stage 8 ships, each can optionally migrate to live as a plugin if it wants lifecycle hooks; if not, they stay as pure libs consumed by plugins.
|
||||
- No other monorepo package depends on the foundation today — the foundation is additive.
|
||||
- The host metadata DB introduces a new SQLite file at `./suaveplan-host.db` by default. Production deployments will need a host-config decision (SQLite vs Postgres vs Surreal) but no existing data is touched.
|
||||
137
openspec/changes/api-plugin-foundation/tasks.md
Normal file
137
openspec/changes/api-plugin-foundation/tasks.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# Tasks — API Plugin Foundation (umbrella)
|
||||
|
||||
> Umbrella change. Tasks here track stage acceptance — per-package work is checked off in each package's own change. Do not retroactively tick boxes for work done before this change opened.
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [x] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [x] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [x] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## 0. Pre-work
|
||||
|
||||
- [x] 0.1 Brainstorm + design doc at `docs/superpowers/specs/2026-05-13-api-plugin-foundation-design.md`
|
||||
- [x] 0.2 Design reviewed by `spec-document-reviewer` subagent (twice — once on initial draft, once on enterprise-scope rewrite)
|
||||
- [x] 0.3 User-approved scope: all 8 items previously deferred folded in as v1 design surface
|
||||
|
||||
## 1. Per-package OpenSpec proposals authored
|
||||
|
||||
### Stage 1 — Contracts
|
||||
|
||||
- [x] 1.1 `api-types`
|
||||
- [x] 1.2 `api-manifest`
|
||||
- [x] 1.3 `api-router`
|
||||
- [x] 1.4 `api-graphql` (abstract contract)
|
||||
- [x] 1.5 `api-ws` (abstract contract)
|
||||
- [x] 1.6 `api-sse` (abstract contract)
|
||||
- [x] 1.7 `api-auth` (abstract session-resolver)
|
||||
- [x] 1.8 `api-auth-provider` (abstract provider)
|
||||
- [x] 1.9 `api-secrets` (abstract contract)
|
||||
- [x] 1.10 `api-tenant-provider` (abstract contract)
|
||||
- [x] 1.11 `api-loader`
|
||||
- [x] 1.12 `api-lifecycle`
|
||||
|
||||
### Stage 2 — Host orchestration
|
||||
|
||||
- [x] 2.1 `api-host`
|
||||
|
||||
### Stage 3 — Worker isolation
|
||||
|
||||
- [x] 3.1 `api-worker-rpc`
|
||||
- [x] 3.2 `api-worker-host`
|
||||
- [x] 3.3 `api-router-worker-bridge`
|
||||
- [x] 3.4 `api-db-worker-proxy`
|
||||
- [x] 3.5 `api-events-worker-proxy`
|
||||
- [x] 3.6 `api-secrets-worker-proxy`
|
||||
- [x] 3.7 `api-jobs-worker-proxy`
|
||||
- [x] 3.8 `api-logger-worker-proxy`
|
||||
|
||||
### Stage 4 — Base adapters
|
||||
|
||||
- [x] 4.1 `api-router-hono`
|
||||
- [x] 4.2 `api-auth-jwt`
|
||||
- [x] 4.3 `api-auth-anonymous`
|
||||
- [x] 4.4 `api-auth-bearer-static`
|
||||
- [x] 4.5 `api-tenant-resolver-jwt`
|
||||
- [x] 4.6 `api-secrets-env`
|
||||
- [x] 4.7 `api-db-adapter`
|
||||
- [x] 4.8 `api-tenant-provider-memory`
|
||||
|
||||
### Stage 5 — Testing kit + conformance suites
|
||||
|
||||
- [x] 5.1 `api-router-in-memory`
|
||||
- [x] 5.2 `api-auth-static-principal`
|
||||
- [x] 5.3 `api-db-sqlite-memory`
|
||||
- [x] 5.4 `api-events-in-memory`
|
||||
- [x] 5.5 `api-ws-broker-in-memory`
|
||||
- [x] 5.6 `api-host` testing-kit subpath (`./testing` export populated)
|
||||
- [x] 5.7 Conformance suites authored on every abstract package (`./testing` subpaths) — `runXxxConformance` shipped from api-manifest, api-router (+ api-host's `runPipelineRouterConformance`), api-graphql, api-ws, api-sse, api-auth, api-auth-provider, api-secrets, api-tenant-provider, api-loader, api-lifecycle, api-worker-rpc. api-types is types-only, no runtime conformance.
|
||||
|
||||
### Stage 6 — Distributed transactions + Better-Auth
|
||||
|
||||
- [x] 6.1 `api-saga`
|
||||
- [x] 6.2 `api-2pc`
|
||||
- [x] 6.3 `api-auth-better-auth`
|
||||
|
||||
### Stage 7 — GraphQL + WebSocket / SSE
|
||||
|
||||
- [x] 7.1 `api-graphql-yoga`
|
||||
- [x] 7.2 `api-graphql-apollo`
|
||||
- [x] 7.3 `api-graphql-ui-graphiql`
|
||||
- [x] 7.4 `api-graphql-ui-altair`
|
||||
- [x] 7.5 `api-ws-bun`
|
||||
- [x] 7.6 `api-ws-broker-redis`
|
||||
- [x] 7.7 `api-sse-bun`
|
||||
|
||||
### Stage 8 — Sandboxing + middleware + helpers
|
||||
|
||||
- [x] 8.1 `api-sandbox-permissions`
|
||||
- [x] 8.2 `api-sandbox-vm`
|
||||
- [x] 8.3 `api-sandbox-wasm`
|
||||
- [x] 8.4 `api-rbac`
|
||||
- [x] 8.5 `api-rate-limit`
|
||||
- [x] 8.6 `api-openapi`
|
||||
- [x] 8.7 `api-openapi-ui-scalar`
|
||||
- [x] 8.8 `api-openapi-ui-swagger`
|
||||
- [x] 8.9 `api-openapi-ui-redoc`
|
||||
- [x] 8.10 `api-audit-stdout`
|
||||
- [x] 8.11 `api-audit-file`
|
||||
- [x] 8.12 `api-auth-rolled` (session-resolver half; provider half deferred to `api-auth-rolled-provider`)
|
||||
- [x] 8.13 `api-secrets-bun`
|
||||
- [x] 8.14 `api-secrets-vault`
|
||||
- [x] 8.15 `api-tenant-resolver-header`
|
||||
- [x] 8.16 `api-tenant-resolver-subdomain`
|
||||
- [x] 8.17 `api-plugin-example`
|
||||
- [x] 8.18 `create-suaveplan-plugin`
|
||||
|
||||
## 2. Stage acceptance gates
|
||||
|
||||
Per `openspec/AGENTS.md §17`, every stage closes ONLY when its **integration smoke** is green — a real-composition test booting every shipped package in the stage and exercising the cross-package seams. Per-package conformance passing is necessary but not sufficient.
|
||||
|
||||
- [x] 2.1 Stage 1 ✓ — every Stage-1 child change archived; `bun openspec validate` passes for every spec delta
|
||||
- [x] 2.2 Stage 2 ✓ — `api-host` archived; `bun examples/api-host-empty.ts` returns 200 on `/internal/livez`
|
||||
- [x] 2.3 Stage 3 ✓ — `api-worker-rpc` conformance suite passes against trivial proxy under both direct and worker surfaces
|
||||
- [x] 2.4 Stage 4 ✓ — every Stage-4 adapter passes its conformance suite under both surfaces **AND** `e2e/foundation-stack/` smoke (`bunx playwright test`) is green with NO hand-wired shims (router-pipeline, tenant-chain, secrets-injection, ledger-db). Coherence-fix changes `foundation-fix-gap-{1,2,3,4,5}` are the closure artifacts.
|
||||
- [x] 2.5 Stage 5 ✓ — `createTestHost` boots `api-plugin-example` skeleton with 100% conformance pass **AND** `e2e/foundation-stack-test-host/` smoke proves `createTestHost(config)` returns a host that drives every Stage-1..4 conformance surface without external services. (8/8 smoke groups green; foundation packages all pass §15 + §16 lint.)
|
||||
- [x] 2.6 Stage 6 ✓ — multi-step saga runs end-to-end (incl. compensation); `api-2pc` commits + aborts against real Postgres **AND** `e2e/foundation-stack-saga/` runs a 3-step compensable workflow end-to-end across two plugins. (7/7 specs green: 3 saga + 4 2PC against Postgres 16 testcontainer with `max_prepared_transactions=8`.)
|
||||
- [x] 2.7 Stage 7 ✓ — `api-plugin-example` federates GraphQL across two plugins; tenant-scoped WebSocket broadcast works **AND** `e2e/foundation-stack-engines/` smoke proves Yoga/Apollo + Bun WS + SSE coexist on one host. (5/5 specs green: 2 GraphQL federation across plugin-a+plugin-b through both Yoga and Apollo engines; 1 tenant-scoped WS broadcast with cross-tenant isolation; 1 SSE stream + publish; 1 four-engines-on-one-host route-collision check.)
|
||||
- [ ] 2.8 Stage 8 ✓ — `api-plugin-example` deploys under every isolation tier (incl. sandboxed WASM); `create-suaveplan-plugin <name>` scaffold tests pass first time **AND** `e2e/foundation-stack-full/` smoke boots every Stage 1-8 package together and exercises the full §6 11-step pipeline end-to-end.
|
||||
|
||||
### Smoke-fail = stage incomplete
|
||||
|
||||
If a smoke needs hand-wired shims to boot (today's E2E pre-Wave-1 caught 5 such gaps in Stage 4), the stage is NOT complete. Open coherence-fix changes against the gap-bearing packages — do NOT paper over with shims in the smoke. The smoke must consume canonical adapter surfaces a real downstream consumer would. See `openspec/archive/2026-05-26-foundation-fix-gap-{1,2,3,4,5}` for the precedent.
|
||||
|
||||
## 3. Foundation acceptance
|
||||
|
||||
- [ ] 3.1 Reference plugin's full integration test (design doc §13.5) green in CI
|
||||
- [ ] 3.2 Better-Auth adapter passes end-to-end OAuth flow with at least one provider (GitHub)
|
||||
- [ ] 3.3 Hot-reload cycle exercised: blue-green swap with in-flight completion + new-slot health probe + traffic flip
|
||||
- [ ] 3.4 Bad-module path: failed `start` triggers auto-rollback; plugin enters `rolled_back`; prior version keeps serving
|
||||
- [ ] 3.5 Admin API exercised end-to-end: install/uninstall/disable/enable/reload/rollback/force-uninstall with audit log entries
|
||||
- [ ] 3.6 Conformance suites pass for every adapter under both direct and worker-proxy surfaces
|
||||
|
||||
## 4. Archive
|
||||
|
||||
- [ ] 4.1 Refresh `MANIFEST.md` to include every foundation package
|
||||
- [ ] 4.2 Update `CLAUDE.md` `packages/api/` section with the foundation listing
|
||||
- [ ] 4.3 Move `openspec/changes/api-plugin-foundation/` → `openspec/archive/<yyyy-mm-dd>-api-plugin-foundation/` only after every child change is archived
|
||||
81
openspec/changes/api-sandbox-wasm-host-bridge/proposal.md
Normal file
81
openspec/changes/api-sandbox-wasm-host-bridge/proposal.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Sandbox WASM Host Bridge
|
||||
|
||||
**Package**: `@suaveplan/api-sandbox-wasm-host-bridge`
|
||||
**Status**: PROPOSED
|
||||
**Type**: new-package (host-integration layer)
|
||||
|
||||
## Why
|
||||
|
||||
Same shape as `api-sandbox-vm-host-bridge` but for the WASM tier. `@suaveplan/api-sandbox-wasm` ships a `WasmSandboxHandle` — a `WebAssembly.Instance` running under WASI `snapshot_preview1` with permission-gated `suaveplan_caps.*` imports. `@suaveplan/api-host.installPlugin(spec)` takes a `PluginModule`. The bridge between them does not exist, so gate 2.8's "deploy under wasm tier" requirement is unreachable.
|
||||
|
||||
## Concrete consumer
|
||||
|
||||
- `@suaveplan/api-host.installPlugin(spec)` — consumes the `PluginModule` produced by this bridge.
|
||||
- `@suaveplan/e2e-foundation-stack-full` Stage-8 closure smoke — deploys the WASM artifact produced by `@suaveplan/wasm-compiler-javy` (sibling change) through this bridge under the wasm tier.
|
||||
|
||||
## Host bridge
|
||||
|
||||
This package IS the host-integration for `api-sandbox-wasm`. Required by §19.
|
||||
|
||||
## End-to-end test
|
||||
|
||||
- `e2e/foundation-stack-full/tests/sandbox-wasm-deployment.spec.ts` — deploys a WASM-compiled `api-plugin-example` via this bridge, drives an HTTP request through the host, asserts response was produced by the WASM-sandboxed plugin.
|
||||
|
||||
## What changes
|
||||
|
||||
### Package surface
|
||||
|
||||
```ts
|
||||
import { createWasmSandbox } from "@suaveplan/api-sandbox-wasm";
|
||||
import { createSandboxedWasmPluginModule } from "@suaveplan/api-sandbox-wasm-host-bridge";
|
||||
|
||||
const sandbox = createWasmSandbox({
|
||||
pluginName: "example",
|
||||
policy,
|
||||
capabilities,
|
||||
checker,
|
||||
wasmModule: await WebAssembly.compile(wasmBytes),
|
||||
});
|
||||
|
||||
const module = await createSandboxedWasmPluginModule({
|
||||
sandbox,
|
||||
manifest: pluginManifest,
|
||||
});
|
||||
|
||||
await host.installPlugin({ kind: "wasm-sandboxed", module, manifest });
|
||||
```
|
||||
|
||||
### Marshalling
|
||||
|
||||
Same `SerializedRequest` / `SerializedResponse` shapes from `@suaveplan/api-types/sandbox-bridge` (authored by the `api-sandbox-vm-host-bridge` change as the shared contract). Both bridges use the same wire format.
|
||||
|
||||
WASM-specific marshalling uses linear-memory buffer passing:
|
||||
1. Bridge allocates a buffer via the WASM `__suaveplan_alloc(len)` export.
|
||||
2. Writes the JSON-encoded `SerializedRequest` bytes into linear memory at the allocated offset.
|
||||
3. Calls the WASM `__suaveplan_handle_request(ptr, len)` export, which returns `(responsePtr, responseLen)`.
|
||||
4. Reads the response bytes from linear memory.
|
||||
5. Frees both buffers via `__suaveplan_free(ptr)`.
|
||||
6. Parses the `SerializedResponse` and returns the host `Response`.
|
||||
|
||||
The WASM plugin must export `__suaveplan_alloc`, `__suaveplan_free`, `__suaveplan_handle_request`, `__suaveplan_routes` (returns a JSON-encoded routes array at init time), `__suaveplan_start`, `__suaveplan_dispose`. The `@suaveplan/wasm-compiler-javy` sibling produces artifacts that conform to this ABI.
|
||||
|
||||
### Error codes
|
||||
|
||||
`SAND_WASM_BRIDGE.*` registry:
|
||||
- `INVALID_PLUGIN_SURFACE` — WASM module didn't export required functions.
|
||||
- `MEMORY_ALLOC_FAILED` — `__suaveplan_alloc` returned 0 or trapped.
|
||||
- `MARSHAL_FAILED` — JSON-encoding of the request failed.
|
||||
- `UNMARSHAL_FAILED` — WASM returned non-conformant response bytes.
|
||||
- `HANDLER_TRAPPED` — WASM execution trapped.
|
||||
- `CAPABILITY_DENIED` — capability import returned WASI ACCES.
|
||||
- `LIFECYCLE_FAILED` — start/dispose export call trapped.
|
||||
|
||||
### Conformance harness
|
||||
|
||||
`runWasmHostBridgeConformance({ buildBridge })` with five invariants mirroring the vm-bridge harness: route-registration, request-round-trip, lifecycle, error-propagation, capability-gate.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- The WASM compilation pipeline itself (sibling change `wasm-compiler-javy`).
|
||||
- Component-model-based WASM plugins (this bridge targets WASI `snapshot_preview1`; component-model is a follow-up).
|
||||
- Cross-instance shared memory (WASM linear-memory isolation is by design).
|
||||
@@ -0,0 +1,94 @@
|
||||
# Capability Spec Delta: api-sandbox-wasm-host-bridge
|
||||
|
||||
**Package**: `@suaveplan/api-sandbox-wasm-host-bridge`
|
||||
**Status**: PROPOSED
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Wraps a `WasmSandboxHandle` as a `PluginModule`
|
||||
|
||||
`createSandboxedWasmPluginModule({ sandbox, manifest })` SHALL return a `PluginModule` compatible with `@suaveplan/api-host.installPlugin`. The returned module's `routes` field is sourced by calling the WASM `__suaveplan_routes` export at bridge-init time, which returns a JSON-encoded routes array via linear memory.
|
||||
|
||||
#### Scenario: WASM-exported routes register with the host
|
||||
|
||||
- **WHEN** a WASM plugin exports `__suaveplan_routes` returning a JSON array of two routes
|
||||
- **THEN** the bridge's PluginModule.routes contains those two routes AND `host.installPlugin(...)` registers both with the host router
|
||||
|
||||
**Verified by:** `tests/integration/full-flow.test.ts::"WASM-exported routes register with the host"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Validates the WASM ABI at bridge-init time
|
||||
|
||||
Before returning a PluginModule, the bridge MUST validate that the WASM instance exports ALL of: `__suaveplan_alloc`, `__suaveplan_free`, `__suaveplan_handle_request`, `__suaveplan_routes`, `__suaveplan_start`, `__suaveplan_dispose`, AND `memory`. Missing any one MUST fail with `SAND_WASM_BRIDGE.INVALID_PLUGIN_SURFACE` BEFORE the plugin is offered to the host.
|
||||
|
||||
#### Scenario: WASM missing `__suaveplan_handle_request` rejects at bridge-init
|
||||
|
||||
- **WHEN** `createSandboxedWasmPluginModule` is called with a sandbox whose WASM module lacks `__suaveplan_handle_request`
|
||||
- **THEN** the call rejects with `code === "SAND_WASM_BRIDGE.INVALID_PLUGIN_SURFACE"` AND `context.missing === ["__suaveplan_handle_request"]`
|
||||
|
||||
**Verified by:** `src/wasm-abi/abi-validator.test.ts::"missing __suaveplan_handle_request rejects at bridge-init"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Request round-trip via linear-memory buffer passing
|
||||
|
||||
For each request, the bridge MUST:
|
||||
1. JSON-encode `SerializedRequest` to UTF-8 bytes.
|
||||
2. Call `__suaveplan_alloc(len)` to obtain a buffer offset; trap-protect against return value 0.
|
||||
3. Write the bytes into linear memory at the returned offset.
|
||||
4. Call `__suaveplan_handle_request(ptr, len)`; receive `(responsePtr, responseLen)` (packed `i64` or two-`i32` return).
|
||||
5. Read response bytes from linear memory at the returned offset.
|
||||
6. Free both buffers via `__suaveplan_free`.
|
||||
7. JSON-decode the response bytes into `SerializedResponse`.
|
||||
8. Unmarshal into a host `Response`.
|
||||
|
||||
#### Scenario: full round-trip preserves bytes
|
||||
|
||||
- **WHEN** a POST with a 1 KiB JSON body reaches a WASM-sandboxed handler
|
||||
- **THEN** the handler observes the exact 1024 bytes AND its returned response body reaches the caller intact
|
||||
|
||||
**Verified by:** `tests/integration/full-flow.test.ts::"1 KiB body round-trips through WASM bridge"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Lifecycle delegation is idempotent
|
||||
|
||||
`PluginModule.start()` MUST call `__suaveplan_start` exactly once across repeated invocations; `dispose()` MUST call `__suaveplan_dispose` exactly once. Both forwarders catch traps and re-surface as `SAND_WASM_BRIDGE.LIFECYCLE_FAILED`.
|
||||
|
||||
#### Scenario: dispose twice is a no-op
|
||||
|
||||
**Verified by:** `src/lifecycle/lifecycle.test.ts::"dispose is idempotent across repeated calls"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: WASM traps surface as `HANDLER_TRAPPED`
|
||||
|
||||
If `__suaveplan_handle_request` traps mid-execution, the bridge MUST catch the trap and surface a `SandboxWasmBridgeError` with `code === "SAND_WASM_BRIDGE.HANDLER_TRAPPED"` AND `cause` pointing at the original trap error AND `context.trapKind` (one of `"unreachable"`, `"out_of_bounds"`, `"div_by_zero"`, `"integer_overflow"`, `"invalid_conversion"`, `"stack_overflow"`, `"other"`).
|
||||
|
||||
#### Scenario: unreachable instruction surfaces HANDLER_TRAPPED
|
||||
|
||||
- **WHEN** a WASM handler executes `unreachable`
|
||||
- **THEN** the bridge surfaces a SandboxWasmBridgeError with code `HANDLER_TRAPPED` AND `context.trapKind === "unreachable"`
|
||||
|
||||
**Verified by:** `tests/integration/full-flow.test.ts::"WASM unreachable instruction surfaces HANDLER_TRAPPED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Capability gate routes through `suaveplan_caps.*` imports
|
||||
|
||||
A WASM-side call to `suaveplan_caps.<name>.<method>` MUST be intercepted by the sandbox's permission checker. Denied capabilities return WASI `ACCES` errno to the WASM caller AND the bridge MUST surface this to the host as the canonical `SAND_PERM.DENIED` code (NOT a silent return).
|
||||
|
||||
#### Scenario: denied fs.write surfaces SAND_PERM.DENIED
|
||||
|
||||
**Verified by:** `tests/integration/full-flow.test.ts::"denied capability surfaces SAND_PERM.DENIED through bridge"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Conformance harness `runWasmHostBridgeConformance`
|
||||
|
||||
The package SHALL ship `runWasmHostBridgeConformance({ buildBridge })` from `@suaveplan/api-sandbox-wasm-host-bridge/testing` with five invariants (route-registration, round-trip, lifecycle, trap-propagation, capability-gate). The harness uses a hand-coded minimal WASM fixture committed under `tests/fixtures/`.
|
||||
|
||||
#### Scenario: harness fails on a bridge that silently retries traps
|
||||
|
||||
**Verified by:** `src/testing/wasm-host-bridge-conformance/wasm-host-bridge-conformance.test.ts::"harness rejects silent-retry bridge"`
|
||||
51
openspec/changes/api-sandbox-wasm-host-bridge/tasks.md
Normal file
51
openspec/changes/api-sandbox-wasm-host-bridge/tasks.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Tasks — Sandbox WASM Host Bridge
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [x] 0.1 Confirm clean working tree and no in-flight worktrees with uncommitted work
|
||||
- [x] 0.2 Account for every worktree under `.claude/worktrees/`
|
||||
- [x] 0.3 `MANIFEST.md` package count matches `find packages -maxdepth 3 -name package.json`
|
||||
- [x] 0.4 §0a Sync: worktree-isolated agents fast-forward to orchestrator-supplied SHA
|
||||
|
||||
## 1. Dependency check
|
||||
|
||||
- [x] 1.1 `@suaveplan/api-types/sandbox-bridge` subpath exists (authored by this change ahead of the sibling vm-bridge merge).
|
||||
- [x] 1.2 `@suaveplan/api-sandbox-wasm` exists with `WasmSandboxHandle` surface.
|
||||
- [x] 1.3 `@suaveplan/api-host.installPlugin` accepts the `PluginModule` shape (bridge returns `SandboxedWasmPluginInstall` carrying both `module` and `routes`).
|
||||
|
||||
## 2. Package scaffold
|
||||
|
||||
- [x] 2.1 `packages/api-stack/api/api-sandbox-wasm-host-bridge/` with package.json, tsconfig, vite, vitest configs.
|
||||
- [x] 2.2 Deps: `@suaveplan/api-sandbox-wasm`, `@suaveplan/api-host`, `@suaveplan/api-types`, `@suaveplan/api-sandbox-permissions`, `@suaveplan/error`, `tslib`, `zod`.
|
||||
|
||||
## 3. Implementation
|
||||
|
||||
- [x] 3.1 `src/bridge/bridge.ts` — `createSandboxedWasmPluginModule({ sandbox, manifest })` factory.
|
||||
- [x] 3.2 `src/wasm-abi/abi.ts` — `__suaveplan_alloc/free/handle_request/routes/start/dispose` export-name constants + signatures.
|
||||
- [x] 3.3 `src/wasm-abi/abi-validator.ts` — validates the WASM module exports all required functions before bridge init.
|
||||
- [x] 3.4 `src/marshal/marshal.ts` — JSON-encode `SerializedRequest` into linear memory, decode `SerializedResponse` out.
|
||||
- [x] 3.5 `src/memory/memory.ts` — alloc/free wrappers with trap protection.
|
||||
- [x] 3.6 `src/route-discovery/route-discovery.ts` — calls `__suaveplan_routes` at init time.
|
||||
- [x] 3.7 `src/lifecycle/lifecycle.ts` — `__suaveplan_start` / `__suaveplan_dispose` forwarders with idempotency.
|
||||
- [x] 3.8 `src/error-codes/error-codes.ts` — 7 SAND_WASM_BRIDGE codes + SandboxWasmBridgeError with public constructor.
|
||||
- [x] 3.9 `src/instruments/instruments.ts` — bridge OTel counters.
|
||||
- [x] 3.10 Paired `.md` per implementation file.
|
||||
|
||||
## 4. Conformance harness
|
||||
|
||||
- [x] 4.1 `src/testing/wasm-host-bridge-conformance/wasm-host-bridge-conformance.ts` with 5 invariants.
|
||||
- [x] 4.2 Sibling .md, .types.ts, .test.ts, index.ts.
|
||||
|
||||
## 5. Tests
|
||||
|
||||
- [x] 5.1 `tests/fixtures/build-wasm.ts` — hand-coded minimal WASM binary that exports the suaveplan ABI (returns canned routes + response). No external compile dep.
|
||||
- [x] 5.2 Real `api-sandbox-wasm` + real `api-host` integration test in `tests/integration/full-flow.test.ts` against the fixture WASM.
|
||||
- [x] 5.3 100% istanbul coverage on touched modules.
|
||||
- [x] 5.4 Public-constructor lint clean.
|
||||
- [x] 5.5 `bunx --bun turbo lint typecheck test:coverage --filter=@suaveplan/api-sandbox-wasm-host-bridge` green.
|
||||
|
||||
## 6. Archive
|
||||
|
||||
- [ ] 6.1 Fold delta into `openspec/specs/api-sandbox-wasm-host-bridge/spec.md`. (deferred to merge step)
|
||||
- [ ] 6.2 Move folder → `openspec/archive/2026-05-27-api-sandbox-wasm-host-bridge/`. (deferred to merge step)
|
||||
- [x] 6.3 Refresh `MANIFEST.md`.
|
||||
88
openspec/changes/api-sandbox-wasm-stdio-abi/proposal.md
Normal file
88
openspec/changes/api-sandbox-wasm-stdio-abi/proposal.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# WASI-stdio ABI variant for the WASM host bridge
|
||||
|
||||
**Package**: `@suaveplan/api-sandbox-wasm-host-bridge` (extension) + `@suaveplan/api-types/sandbox-bridge` (additive)
|
||||
**Status**: PROPOSED
|
||||
**Type**: extension + types-additive
|
||||
|
||||
## Why
|
||||
|
||||
The shipped wasm host-bridge (commit `a9fa31db95`) accepts WASM modules exporting six custom-shaped functions (`__suaveplan_alloc(i32) → i32`, `__suaveplan_handle_request(i32,i32) → i64`, etc.) plus `memory`. Producing such a module by compiling TypeScript is the job of `@suaveplan/wasm-compiler-javy`.
|
||||
|
||||
When the build agent investigated against Javy v8.1.1 (Bytecode Alliance, current latest), it surfaced an architectural blocker: **Javy cannot emit custom-shaped WASM exports**. Its `-C wit=...` flag restricts WIT-exposed exports to "function exports with no arguments and no return values" (verified via `javy build -C help`). Javy's host-interaction model is fixed: the host writes input to fd 0 (stdin), reads output from fd 1 (stdout). This is the WASI standard interface and is the only ABI Javy honors.
|
||||
|
||||
Three resolutions were considered (full transcript in the agent's stop report); the chosen path is to **extend the bridge to accept either the existing direct-export ABI OR a WASI-stdio ABI variant**. The compiler then becomes a thin Javy wrapper that emits Javy's native artifact — no outer-WASM authorship, no Component Model migration, no bridge-spec rewrite for the existing variant.
|
||||
|
||||
## Concrete consumer
|
||||
|
||||
- `@suaveplan/wasm-compiler-javy` (rewritten in a sibling proposal) — every TS→WASM artifact it emits speaks the WASI-stdio variant.
|
||||
- Plugin authors who choose Javy as their toolchain (the recommended TS→WASM path).
|
||||
- `e2e/foundation-stack-full` Stage-8 closure smoke — once both proposals ship, `api-plugin-example` deploys under the wasm tier via Javy + WASI-stdio ABI.
|
||||
|
||||
## Host bridge
|
||||
|
||||
The bridge surface gains one additive option. The default remains the current direct-export ABI so already-shipped consumers see no behaviour change.
|
||||
|
||||
```ts
|
||||
type WasmAbiVariant = "direct-i32-i64" | "wasi-stdio";
|
||||
|
||||
createSandboxedWasmPluginModule({
|
||||
sandbox,
|
||||
manifest,
|
||||
abiVariant: "wasi-stdio", // default: "direct-i32-i64"
|
||||
});
|
||||
```
|
||||
|
||||
When `abiVariant === "wasi-stdio"`, the bridge:
|
||||
1. Skips the six-export ABI validation (those exports won't exist).
|
||||
2. Routes every request through the WASI-stdio framing protocol defined below.
|
||||
3. Treats the WASM module as a long-running guest process: a single `_start` call (the WASI entry point) launches the plugin, which then enters a request loop reading from fd 0.
|
||||
|
||||
## End-to-end test
|
||||
|
||||
- `packages/api-stack/api/api-sandbox-wasm-host-bridge/tests/integration/wasi-stdio-flow.test.ts` — instantiates a hand-encoded WASM fixture that speaks the WASI-stdio protocol, runs the conformance harness against it, and asserts: route discovery via `{kind:"routes"}` envelope; request round-trip via `{kind:"request"}` / `{kind:"response"}`; error envelope surfaces as SuaveplanError host-side; lifecycle commands (`{kind:"start"}` / `{kind:"dispose"}`) idempotent.
|
||||
|
||||
## What changes
|
||||
|
||||
### Shared protocol contract (additive to `@suaveplan/api-types/sandbox-bridge`)
|
||||
|
||||
New types alongside the existing `SerializedRequest`/`SerializedResponse`/`SerializedRoute`. Wire format is length-prefixed JSON frames so the protocol survives any WASI implementation's buffering semantics.
|
||||
|
||||
```ts
|
||||
// Each envelope is a single WASI-stdio frame:
|
||||
// [u32 BE length][JSON bytes of length 'length']
|
||||
|
||||
export type PluginEnvelopeIn =
|
||||
| { readonly kind: "request"; readonly request: SerializedRequest }
|
||||
| { readonly kind: "routes" }
|
||||
| { readonly kind: "start" }
|
||||
| { readonly kind: "dispose" };
|
||||
|
||||
export type PluginEnvelopeOut =
|
||||
| { readonly kind: "response"; readonly response: SerializedResponse }
|
||||
| { readonly kind: "routes"; readonly routes: readonly SerializedRoute[] }
|
||||
| { readonly kind: "ok" }
|
||||
| { readonly kind: "error"; readonly code: string; readonly message: string };
|
||||
|
||||
export const WASI_STDIO_PROTOCOL_VERSION: 1;
|
||||
```
|
||||
|
||||
The contract lives in `api-types` because both bridges (host-side framer) and Javy-compiled plugins (sandbox-side framer, via the suaveplan ABI shim) MUST agree on the wire shape down to the byte. Putting it in either runtime package would re-introduce the SerializedRequest/Response divergence the orchestrator just resolved.
|
||||
|
||||
### Bridge runtime (extends `@suaveplan/api-sandbox-wasm-host-bridge`)
|
||||
|
||||
1. `src/wasi-stdio/wasi-stdio.ts` — the framer: length-prefix encode/decode, Zod-validated envelope schemas, frame-too-large guard (`SAND_WASM_BRIDGE.FRAME_OVERSIZE`).
|
||||
2. `src/wasi-stdio/wasi-stdio-runner.ts` — wraps a `WasmSandboxHandle` configured for WASI-stdio: writes envelopes to fd 0, reads from fd 1, surfaces stderr lines as logger events.
|
||||
3. `src/bridge/bridge.ts` — `createSandboxedWasmPluginModule` becomes a `switch (abiVariant)`. The existing direct-i32-i64 path is renamed `src/abi-direct/` and the new path is `src/abi-wasi-stdio/`. Routes returned by `{kind:"routes"}` envelope feed into the same `BridgedPlugin` shape.
|
||||
4. New error codes: `SAND_WASM_BRIDGE.FRAME_OVERSIZE`, `SAND_WASM_BRIDGE.PROTOCOL_MISMATCH`, `SAND_WASM_BRIDGE.STDIO_TIMEOUT`. The shipped 7 codes are preserved.
|
||||
5. ABI validation: under `abiVariant: "wasi-stdio"`, validate `memory` + `_start` + WASI imports (`wasi_snapshot_preview1.{fd_read, fd_write, environ_get, environ_sizes_get, ...}`). Missing → `SAND_WASM_BRIDGE.INVALID_PLUGIN_SURFACE` with `context.missing`.
|
||||
6. Trap classification, capability-gate routing, idempotent lifecycle: all preserved — but lifecycle commands now go through `{kind:"start"}` / `{kind:"dispose"}` envelopes, not direct export calls.
|
||||
|
||||
### Conformance harness (extends)
|
||||
|
||||
`runWasmHostBridgeConformance({ buildBridge, abiVariant })` gains an `abiVariant` parameter. Both variants must pass all five existing invariants; the harness picks the right fixture WASM based on the variant.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- No change to `@suaveplan/api-types/sandbox-bridge`'s `SerializedRequest`/`SerializedResponse`/`SerializedRoute`. They round-trip unchanged through both variants.
|
||||
- No change to `@suaveplan/api-sandbox-vm-host-bridge`. The vm tier uses structuredClone, not stdio.
|
||||
- No deprecation of the direct-i32-i64 variant. Both stay first-class.
|
||||
@@ -0,0 +1,138 @@
|
||||
# Capability Spec Delta: api-sandbox-wasm-host-bridge (WASI-stdio variant)
|
||||
|
||||
**Package**: `@suaveplan/api-sandbox-wasm-host-bridge` (extension)
|
||||
**Status**: PROPOSED
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `createSandboxedWasmPluginModule` accepts `abiVariant: "wasi-stdio"`
|
||||
|
||||
The bridge factory MUST accept `abiVariant?: "direct-i32-i64" | "wasi-stdio"` as an additional option. Default value is `"direct-i32-i64"` (existing behaviour, unchanged). When `abiVariant === "wasi-stdio"`, the bridge runs the WASI-stdio framing protocol against the sandbox's fd 0 / fd 1 instead of validating direct-i32-i64 exports.
|
||||
|
||||
#### Scenario: Default variant unchanged
|
||||
|
||||
- **WHEN** `createSandboxedWasmPluginModule` is called without `abiVariant`
|
||||
- **THEN** the bridge validates the six direct-export ABI surface exactly as the shipped contract requires
|
||||
|
||||
**Verified by:** `tests/integration/full-flow.test.ts::"WASM-exported routes register with the host"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: WASI-stdio variant validates against the WASI-import surface, not direct exports
|
||||
|
||||
Under `abiVariant: "wasi-stdio"`, the bridge MUST validate that the WASM module:
|
||||
1. Exports `memory` AND `_start`.
|
||||
2. Imports the WASI snapshot-preview1 surface (`wasi_snapshot_preview1.fd_read`, `fd_write`, `environ_get`, `environ_sizes_get`, `proc_exit`).
|
||||
|
||||
Missing any of these raises `SAND_WASM_BRIDGE.INVALID_PLUGIN_SURFACE` with `context.missing: string[]` BEFORE the plugin is offered to the host. The six direct-export ABI functions MUST NOT be required under this variant.
|
||||
|
||||
#### Scenario: WASI-stdio plugin missing `_start` is rejected at bridge-init
|
||||
|
||||
- **WHEN** `createSandboxedWasmPluginModule({ abiVariant: "wasi-stdio", ... })` is called against a sandbox whose WASM lacks `_start`
|
||||
- **THEN** the call rejects with `code === "SAND_WASM_BRIDGE.INVALID_PLUGIN_SURFACE"` AND `context.missing === ["_start"]`
|
||||
|
||||
**Verified by:** `src/abi-wasi-stdio/abi-wasi-stdio.test.ts::"missing _start rejects at bridge-init"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: WASI-stdio frame format is length-prefixed JSON
|
||||
|
||||
Every envelope exchanged between bridge and plugin MUST be a `[u32 BE length][length bytes of UTF-8 JSON]` frame. The bridge MUST:
|
||||
1. Length-prefix encode outgoing envelopes (`PluginEnvelopeIn`) before writing to fd 0.
|
||||
2. Length-prefix decode incoming envelopes (`PluginEnvelopeOut`) read from fd 1.
|
||||
3. Validate each decoded envelope against the `PluginEnvelopeOut` Zod schema before acting on it.
|
||||
4. Enforce a 16 MiB maximum frame size per direction. Larger frames raise `SAND_WASM_BRIDGE.FRAME_OVERSIZE` with `context.byteLen`.
|
||||
|
||||
#### Scenario: 17 MiB outgoing request rejected
|
||||
|
||||
- **WHEN** the host attempts to send a request whose serialized envelope exceeds 16 MiB
|
||||
- **THEN** the call rejects with `code === "SAND_WASM_BRIDGE.FRAME_OVERSIZE"` AND `context.byteLen > 16777216` AND no bytes are written to fd 0
|
||||
|
||||
**Verified by:** `src/wasi-stdio/wasi-stdio-frame.test.ts::"oversize frame rejected before write"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Route discovery uses `{kind:"routes"}` envelope
|
||||
|
||||
At bridge-init, the bridge MUST send a `{kind: "routes"}` envelope to the plugin AND await a `{kind: "routes", routes: SerializedRoute[]}` reply. The returned routes feed into the BridgedPlugin's `routes` field. If the plugin replies with `{kind: "error", code, message}` instead, the bridge MUST raise a SuaveplanError carrying that `code` (the plugin's own code, not bridge-mapped).
|
||||
|
||||
#### Scenario: Route discovery round-trip
|
||||
|
||||
- **WHEN** a WASI-stdio plugin exports two routes via `{kind:"routes"}` reply
|
||||
- **THEN** the BridgedPlugin's `routes` field contains both AND `host.installPlugin(...)` registers them
|
||||
|
||||
**Verified by:** `tests/integration/wasi-stdio-flow.test.ts::"route discovery via envelope"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Request round-trip preserves bytes through WASI-stdio
|
||||
|
||||
For each request, the bridge MUST:
|
||||
1. Marshal the host `Request` into `SerializedRequest`.
|
||||
2. Wrap it in `{kind: "request", request}` envelope.
|
||||
3. Length-prefix encode and write to fd 0.
|
||||
4. Read a length-prefixed envelope from fd 1.
|
||||
5. Validate the envelope as `PluginEnvelopeOut`.
|
||||
6. If `{kind: "response", response}`, unmarshal into a host `Response`.
|
||||
7. If `{kind: "error", code, message}`, throw a `SuaveplanError` carrying the plugin's code.
|
||||
8. Any other kind → `SAND_WASM_BRIDGE.PROTOCOL_MISMATCH` with `context.received`.
|
||||
|
||||
#### Scenario: 1 KiB body round-trips through WASI-stdio
|
||||
|
||||
- **WHEN** a POST with a 1 KiB JSON body reaches a WASI-stdio-sandboxed handler
|
||||
- **THEN** the handler observes the exact 1024 bytes (base64-decoded from `bodyBase64`) AND its returned response body reaches the caller intact
|
||||
|
||||
**Verified by:** `tests/integration/wasi-stdio-flow.test.ts::"1 KiB body round-trips through WASI-stdio"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Lifecycle commands flow through envelopes
|
||||
|
||||
`PluginModule.start()` MUST send `{kind: "start"}` to the plugin and await `{kind: "ok"}`. `PluginModule.dispose()` MUST send `{kind: "dispose"}` and await `{kind: "ok"}`. Both forwarders are idempotent: repeated calls send AT MOST ONE envelope. Any non-`ok` reply surfaces as `SAND_WASM_BRIDGE.LIFECYCLE_FAILED`.
|
||||
|
||||
#### Scenario: dispose twice is a no-op
|
||||
|
||||
- **WHEN** `PluginModule.dispose()` is invoked twice
|
||||
- **THEN** exactly one `{kind:"dispose"}` envelope reaches the plugin
|
||||
|
||||
**Verified by:** `src/lifecycle/lifecycle.test.ts::"dispose is idempotent across repeated calls"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: WASM traps still classify per the seven trap kinds
|
||||
|
||||
Traps during `_start` or during any in-loop request handling MUST propagate as `SAND_WASM_BRIDGE.HANDLER_TRAPPED` with `context.trapKind`. The kind set MUST be unchanged from the direct variant (`unreachable`, `out_of_bounds`, `div_by_zero`, `integer_overflow`, `invalid_conversion`, `stack_overflow`, `other`).
|
||||
|
||||
#### Scenario: unreachable mid-request surfaces HANDLER_TRAPPED
|
||||
|
||||
**Verified by:** `tests/integration/wasi-stdio-flow.test.ts::"WASI-stdio unreachable mid-request surfaces HANDLER_TRAPPED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Capability gate continues to route denials through `SAND_PERM.DENIED`
|
||||
|
||||
Even under WASI-stdio, the sandbox's capability proxy SHALL surface denied capability calls as `SAND_PERM.DENIED` to the host. The bridge MUST NOT silently translate them to its own error namespace.
|
||||
|
||||
#### Scenario: Denied fs.write surfaces SAND_PERM.DENIED
|
||||
|
||||
**Verified by:** `tests/integration/wasi-stdio-flow.test.ts::"denied capability under WASI-stdio surfaces SAND_PERM.DENIED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Conformance harness gains `abiVariant` parameter
|
||||
|
||||
`runWasmHostBridgeConformance` SHALL accept `{ buildBridge, abiVariant }`. Both `"direct-i32-i64"` and `"wasi-stdio"` MUST pass all five existing invariants (route-registration, round-trip, lifecycle, trap-propagation, capability-gate) when the harness is run against the matching fixture WASM.
|
||||
|
||||
#### Scenario: harness fails on a WASI-stdio bridge that swallows protocol errors
|
||||
|
||||
**Verified by:** `src/testing/wasm-host-bridge-conformance/wasm-host-bridge-conformance.test.ts::"harness rejects WASI-stdio bridge that swallows PROTOCOL_MISMATCH"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: OTel instruments distinguish variants via `abi_variant` attribute
|
||||
|
||||
Every existing counter and histogram in `@suaveplan/api-sandbox-wasm-host-bridge` MUST carry an `abi_variant` attribute on every emission whose value is either `"direct-i32-i64"` or `"wasi-stdio"`. No new instruments — only an additional attribute on each existing instrument.
|
||||
|
||||
#### Scenario: WASI-stdio request emits with `abi_variant="wasi-stdio"`
|
||||
|
||||
**Verified by:** `src/instruments/instruments.test.ts::"abi_variant attribute on every request counter emission"`
|
||||
@@ -0,0 +1,55 @@
|
||||
# Capability Spec Delta: api-types (sandbox-bridge WASI-stdio additive)
|
||||
|
||||
**Package**: `@suaveplan/api-types` (additive)
|
||||
**Status**: PROPOSED
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: `api-types/sandbox-bridge` exports the WASI-stdio envelope union
|
||||
|
||||
The subpath `@suaveplan/api-types/sandbox-bridge` SHALL additionally export `PluginEnvelopeIn`, `PluginEnvelopeOut`, AND `WASI_STDIO_PROTOCOL_VERSION` (literal type `1`). The envelopes are discriminated unions keyed by `kind`.
|
||||
|
||||
`PluginEnvelopeIn`:
|
||||
- `{ kind: "request"; request: SerializedRequest }`
|
||||
- `{ kind: "routes" }`
|
||||
- `{ kind: "start" }`
|
||||
- `{ kind: "dispose" }`
|
||||
|
||||
`PluginEnvelopeOut`:
|
||||
- `{ kind: "response"; response: SerializedResponse }`
|
||||
- `{ kind: "routes"; routes: readonly SerializedRoute[] }`
|
||||
- `{ kind: "ok" }`
|
||||
- `{ kind: "error"; code: string; message: string }`
|
||||
|
||||
#### Scenario: Envelope types round-trip through the JSON wire shape
|
||||
|
||||
- **WHEN** an instance of `PluginEnvelopeIn` is JSON-stringified, length-prefixed, and JSON-parsed
|
||||
- **THEN** the result is `===`-deep-equal to the original (modulo readonly array identity) AND validates against the Zod schema `PluginEnvelopeInSchema` (which lives in the bridge package — Zod schemas don't ship from api-types because the package is zero-runtime)
|
||||
|
||||
**Verified by:** `packages/api-stack/api/api-types/src/sandbox-bridge/wasi-stdio-protocol.test.ts::"PluginEnvelopeIn JSON round-trip"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `WASI_STDIO_PROTOCOL_VERSION` is a typed literal constant exported as `1`
|
||||
|
||||
The constant SHALL be declared `export const WASI_STDIO_PROTOCOL_VERSION = 1 as const;` so consumers can guard for protocol-version drift at compile time via literal-type matching. Future protocol revisions get a new constant value and a corresponding bridge-side rejection if the plugin's version doesn't match.
|
||||
|
||||
#### Scenario: Compile-time literal narrowing on the constant
|
||||
|
||||
- **WHEN** a consumer writes `const v: 1 = WASI_STDIO_PROTOCOL_VERSION`
|
||||
- **THEN** TypeScript accepts the assignment with no widening
|
||||
|
||||
**Verified by:** `packages/api-stack/api/api-types/src/sandbox-bridge/wasi-stdio-protocol.test.ts::"protocol version is the literal type 1"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: `api-types` remains zero-runtime, zero-deps
|
||||
|
||||
The new file MUST contain ONLY `export type` declarations and `export const` literal values (no functions, no class bodies, no Zod schemas). The package's existing byte-size guard (`tests/byte-size.test.ts`) MUST continue to pass after the addition.
|
||||
|
||||
#### Scenario: byte-size guard catches accidental runtime additions
|
||||
|
||||
- **WHEN** a contributor adds a function body to `wasi-stdio-protocol.types.ts`
|
||||
- **THEN** the byte-size guard fails the build
|
||||
|
||||
**Verified by:** `packages/api-stack/api/api-types/tests/byte-size.test.ts::"compiled output stays under N bytes"`
|
||||
52
openspec/changes/api-sandbox-wasm-stdio-abi/tasks.md
Normal file
52
openspec/changes/api-sandbox-wasm-stdio-abi/tasks.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Tasks — WASI-stdio ABI variant for the WASM host bridge
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [x] 0.1 Clean working tree; account for every `.claude/worktrees/agent-*` (active, work-pending-merge, stale-and-prune)
|
||||
- [x] 0.2 `MANIFEST.md` package count matches `find packages -maxdepth 3 -name package.json`
|
||||
- [x] 0.3 §0a Sync: worktree-isolated agents fast-forward to orchestrator-supplied SHA before any work
|
||||
|
||||
## 1. Shared protocol contract in `@suaveplan/api-types/sandbox-bridge`
|
||||
|
||||
- [x] 1.1 Add `wasi-stdio-protocol.types.ts` exporting `PluginEnvelopeIn` and `PluginEnvelopeOut` discriminated unions + `WASI_STDIO_PROTOCOL_VERSION: 1` constant
|
||||
- [x] 1.2 Paired `.md` documenting wire format ([u32 BE length][JSON]) + frame ordering invariants
|
||||
- [x] 1.3 Paired `.test.ts` proving the types-only invariant (byte-size guard already enforces)
|
||||
- [x] 1.4 Re-export from `packages/api-stack/api/api-types/src/sandbox-bridge/index.ts` + root `src/index.ts`
|
||||
- [x] 1.5 `bunx --bun turbo lint typecheck test:coverage --filter=@suaveplan/api-types` green
|
||||
|
||||
## 2. Bridge runtime — WASI-stdio variant
|
||||
|
||||
- [x] 2.1 `src/wasi-stdio/wasi-stdio-frame.ts` — Zod-validated length-prefix encoder/decoder
|
||||
- [x] 2.2 `src/wasi-stdio/wasi-stdio-runner.ts` — drives the sandbox via fd 0 / fd 1; surfaces stderr to logger
|
||||
- [x] 2.3 `src/wasi-stdio/wasi-stdio-routes.ts` — issues `{kind:"routes"}` at bridge-init, captures the response into the BridgedPlugin
|
||||
- [x] 2.4 `src/abi-wasi-stdio/wasi-stdio-validate.ts` — variant-specific ABI validator (memory + `_start` + WASI imports allowlist)
|
||||
- [x] 2.5 `src/abi-direct/` (existing direct-i32-i64 path) — moved existing `src/wasm-abi/` + `src/memory/` + `src/marshal/` modules under this folder so the variant split is structural, not parameterised
|
||||
- [x] 2.6 `src/bridge/bridge.ts` — `switch (abiVariant)` dispatch; default remains `"direct-i32-i64"`
|
||||
- [x] 2.7 New error codes appended: `SAND_WASM_BRIDGE.FRAME_OVERSIZE`, `SAND_WASM_BRIDGE.PROTOCOL_MISMATCH`, `SAND_WASM_BRIDGE.STDIO_TIMEOUT` (10 codes total)
|
||||
- [x] 2.8 Lifecycle (start/dispose) routes through stdio envelopes when `abiVariant === "wasi-stdio"`
|
||||
- [x] 2.9 Trap classification preserved across both variants
|
||||
- [x] 2.10 OTel instruments labelled with `abi_variant` attribute on every counter and histogram
|
||||
|
||||
## 3. Conformance harness
|
||||
|
||||
- [x] 3.1 `runWasmHostBridgeConformance({ buildBridge, abiVariant })` gains the new parameter; both variants run all five invariants
|
||||
- [x] 3.2 New WASI-stdio fixture WASM under `tests/fixtures/wasi-stdio/build-wasm.ts` — hand-encoded, no `wabt`
|
||||
- [x] 3.3 Fixture exercises route-discovery, request round-trip, lifecycle, trap (`unreachable` mid-frame), capability-gate
|
||||
|
||||
## 4. Integration test
|
||||
|
||||
- [x] 4.1 `tests/integration/wasi-stdio-flow.test.ts` — full round-trip + assertBridgeThrowsOnTrap for both variants
|
||||
- [x] 4.2 `tests/integration/full-flow.test.ts` (existing direct variant) — left untouched; proves the split didn't regress
|
||||
- [x] 4.3 100% istanbul coverage on every touched module
|
||||
- [x] 4.4 No `// biome-ignore`; every test file imports from `@suaveplan/testing/runner`; public-constructor preserved
|
||||
|
||||
## 5. Gate
|
||||
|
||||
- [x] 5.1 `bunx --bun turbo lint typecheck test:coverage --filter=@suaveplan/api-types --filter=@suaveplan/api-sandbox-wasm --filter=@suaveplan/api-sandbox-wasm-host-bridge --filter=@suaveplan/api-sandbox-vm-host-bridge --concurrency=4` green (vm-bridge included to prove the api-types additive change doesn't regress it)
|
||||
|
||||
## 6. Archive (orchestrator step)
|
||||
|
||||
- [ ] 6.1 Fold delta into `openspec/specs/api-sandbox-wasm-host-bridge/spec.md` (creating the spec if it doesn't exist yet — the original wasm host-bridge proposal still awaits its first archive)
|
||||
- [ ] 6.2 Fold delta into `openspec/specs/api-types/spec.md` for the additive protocol types
|
||||
- [ ] 6.3 Move folder → `openspec/archive/2026-05-27-api-sandbox-wasm-stdio-abi/`
|
||||
- [ ] 6.4 Refresh `MANIFEST.md`
|
||||
@@ -0,0 +1,169 @@
|
||||
# Crypto completion — streaming AEAD, recovery, device keys, scrypt, blake2b, password helpers, hash-password CLI
|
||||
|
||||
**Packages**: `@suaveplan/crypto`, `@suaveplan/crypto-cipher`, `@suaveplan/crypto-curve`, `@suaveplan/crypto-hash`, `@suaveplan/crypto-keys`
|
||||
**Status**: PROPOSED
|
||||
**Type**: feature + spec-conformance
|
||||
|
||||
## Why
|
||||
|
||||
The 2026-05-02 audit of the genesis crypto stack against `../suaveplan/packages/universal/crypto/src` identified seven confirmed feature gaps that the upstream package shipped but the genesis sub-packages do not. None of these are MAJOR architecture changes — they are concrete capabilities (streaming AEAD, ECDSA recovery, device keys, scrypt KDF, blake2b hashing + password helpers, password-driven AEAD wrappers, and the `hash-password` CLI) that downstream consumers (drive, network, audit-logging, slsa) currently work around or block on. Genesis policy (CLAUDE.md §Workflow rule 5) says "preserve the full public surface during rewrites — trimming is OK ONLY when code is duplicated elsewhere in-repo or can be replaced by an existing `@suaveplan/*` library", and none of these seven features are duplicated by anything that already landed.
|
||||
|
||||
Closing all seven in one coordinated pass is preferable to seven small changes because (a) they share validation utilities and `crypto-utils` helpers that have to land at the same revision to avoid mid-port circular pins, (b) the streaming AEAD façade in `crypto/aead` and the engine in `crypto-cipher/primitives` are linked — adding either alone leaves the public surface incoherent, and (c) the password helpers (`encryptWithPassword`, `deriveKeyFromPassword`) sit on top of the new scrypt + blake2b primitives, so they cannot ship until those primitives exist.
|
||||
|
||||
The seven gaps:
|
||||
|
||||
1. **Streaming AEAD** (`crypto-cipher` + `crypto/aead`): `encryptStreamAead` / `decryptStreamAead` async generators (255 L), `aesGcmStreamingEngine` and `xchacha20StreamingEngine` engines (35 L each), `createEncryptionStream` / `createDecryptionStream` Web-Streams adapters, `streamEncrypt` / `streamDecrypt` legacy aliases, `collectStream`, `createStreamKey`, `createStreamEncryption` (430 L of `streaming.ts`).
|
||||
2. **secp256k1 ECDSA recovery** (`crypto-curve/primitives/secp256k1`): `signWithRecovery`, `recoverPublicKey`, `verifyWithRecovery` — all three present in upstream (lines 74, 143, 243 of `signing.ts`) and absent from the genesis 5-method secp256k1 noble engine.
|
||||
3. **Device-key suite** (`crypto-keys/keypair/device`): `createDeviceKey`, `deriveDeviceKeyFromMaster`, `generateDeviceUUID`, `extractDeviceMetadata`, `compareDeviceKeys`, `isValidDeviceKey`, plus `generateDerivedDeviceId` for one-release back-compat (261 L of `device.ts`).
|
||||
4. **scrypt KDF** (`crypto-hash/primitives/scrypt`): N/r/p parameter set, salt + outputLength, async via `@noble/hashes/scrypt` (53 L of `hashing.ts` lines L273-L325).
|
||||
5. **blake2b hashing** (`crypto-hash/primitives/blake2b`): variable output length, optional key, plus `hashPasswordBlake2b` / `verifyPasswordBlake2b` constant-time wrappers with the parameterised `$blake2b$v=1$cf=N$salt$hash` format.
|
||||
6. **encryptWithPassword + deriveKeyFromPassword** (`crypto/password`): the high-level "password → encrypted blob" pattern (Argon2id-derived key + AEAD), 344 L of `password.ts`.
|
||||
7. **`hash-password` CLI bin** (new `packages/crypto/crypto-bin` or `crypto/bin/hash-password.ts`): a one-shot CLI that hashes a plaintext password to a verifier and exits 0/1 on verify (98 L of `bin/hash-password.ts`).
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/crypto-cipher`
|
||||
|
||||
- ADDED: `streaming/streaming-aead.ts` — `encryptStreamAead` and `decryptStreamAead` async generators with chunk-level header / footer tags, `EncryptStreamAeadOptions` and `DecryptStreamAeadOptions` (chunk size, AAD, AbortSignal, optional `header` callback for chunk-tag attestation).
|
||||
- ADDED: `streaming/streaming-aead-aes-gcm.ts` — `aesGcmStreamingEngine: StreamingAeadEngine` adapter built on the existing `aes-256-gcm` primitive; nonce per chunk derived from a 12-byte base nonce + 32-bit counter (RFC 5116-compatible).
|
||||
- ADDED: `streaming/streaming-aead-xchacha20.ts` — `xchacha20StreamingEngine: StreamingAeadEngine` adapter built on the existing `xchacha20-poly1305` primitive; nonce per chunk derived from a 24-byte base nonce + 32-bit counter.
|
||||
- ADDED: `streaming/streaming-aead.types.ts` — `StreamingAeadEngine` interface, `StreamingChunkTag` interface, `StreamingHeader` interface.
|
||||
- ADDED: error codes `CRYPTO_CIPHER.STREAM_TAG_MISMATCH`, `CRYPTO_CIPHER.STREAM_OUT_OF_ORDER`, `CRYPTO_CIPHER.STREAM_ABORTED`, `CRYPTO_CIPHER.STREAM_INVALID_NONCE_BASE`.
|
||||
|
||||
### `@suaveplan/crypto-curve`
|
||||
|
||||
- ADDED: `primitives/secp256k1/impl/secp256k1.noble/secp256k1.noble.ts` exports `signWithRecovery(digest, privateKey)`, `recoverPublicKey(signatureWithRecovery, digest)`, and `verifyWithRecovery(signatureWithRecovery, digest, expectedPublicKey)`. The signature-with-recovery byte layout is the canonical 65-byte form (64-byte compact signature + 1 recovery byte).
|
||||
- ADDED: `primitives/secp256k1/impl/secp256k1.native/secp256k1.native.ts` matching surface (libsodium does not ship secp256k1 — native engine throws `CRYPTO_CURVE.PRIMITIVE_NOT_IMPLEMENTED` per genesis convention).
|
||||
- MODIFIED: `secp256k1.select` re-exports the three new functions.
|
||||
- ADDED: error codes `CRYPTO_CURVE.RECOVERY_BYTE_MISSING`, `CRYPTO_CURVE.RECOVERY_BYTE_INVALID`, `CRYPTO_CURVE.RECOVERED_KEY_MISMATCH`.
|
||||
|
||||
### `@suaveplan/crypto-hash`
|
||||
|
||||
- ADDED: `primitives/scrypt/scrypt.ts` — `scrypt(password, salt, options)` with `N`, `r`, `p`, `dkLen` parameters; uses `@noble/hashes/scrypt`'s `scryptAsync`.
|
||||
- ADDED: `primitives/scrypt/scrypt.select` and `primitives/scrypt/impl/scrypt.noble/scrypt.noble.ts` matching the existing per-primitive selector layout.
|
||||
- ADDED: `primitives/blake2b/blake2b.ts` — `blake2b(message, options)` with `dkLen` (1–64 bytes), optional `key`, optional `salt`, optional `personalization`.
|
||||
- ADDED: `primitives/blake2b/blake2b.select` and `primitives/blake2b/impl/blake2b.noble/blake2b.noble.ts`.
|
||||
- ADDED: `mac/blake2b-password/blake2b-password.ts` — `hashPasswordBlake2b(password, options)` and `verifyPasswordBlake2b(storedHash, password)`, using the `$blake2b$v=1$cf=<costFactor>$<saltHex>$<hashHex>` MCF-style format and constant-time comparison.
|
||||
- ADDED: error codes `CRYPTO_HASH.SCRYPT_INVALID_N`, `CRYPTO_HASH.BLAKE2B_OUTPUT_TOO_LONG`, `CRYPTO_HASH.BLAKE2B_KEY_TOO_LONG`, `CRYPTO_HASH.BLAKE2B_PASSWORD_FORMAT_INVALID`.
|
||||
|
||||
### `@suaveplan/crypto-keys`
|
||||
|
||||
- ADDED: `keypair/device/device.ts` — `createDeviceKey`, `deriveDeviceKeyFromMaster`, `generateDeviceUUID`, `extractDeviceMetadata`, `compareDeviceKeys`, `isValidDeviceKey`, plus the `generateDerivedDeviceId` back-compat alias.
|
||||
- ADDED: `keypair/device/device.types.ts` — `DeviceKey`, `MasterKey`, `DeviceKeyOptions`, `DeviceKeyDerivationOptions`, `DeviceIdOptions`, `DeviceMetadata` interfaces.
|
||||
- ADDED: error codes `CRYPTO_KEYS.DEVICE_INVALID_MASTER`, `CRYPTO_KEYS.DEVICE_INVALID_KEY`, `CRYPTO_KEYS.DEVICE_DERIVATION_FAILED`.
|
||||
|
||||
### `@suaveplan/crypto`
|
||||
|
||||
- ADDED: `aead/streaming/streaming-aead.ts` — high-level façade `encryptStreamAead` / `decryptStreamAead` that selects the engine (`xchacha20-poly1305` or `aes-256-gcm`) from the opts argument and delegates to `crypto-cipher`. Re-exports `aesGcmStreamingEngine` and `xchacha20StreamingEngine`.
|
||||
- ADDED: `aead/streaming/web-streams.ts` — `createEncryptionStream(opts)` and `createDecryptionStream(opts)` returning `TransformStream<Uint8Array, Uint8Array>` Web-Streams adapters; `collectStream(stream)`, `createStreamKey(rng?)`, `createStreamEncryption(opts)` helpers; `streamEncrypt` / `streamDecrypt` aliases preserved for back-compat.
|
||||
- ADDED: `password/encrypt-with-password.ts` — `encryptWithPassword(password, plaintext, options?)` and `decryptWithPassword(password, ciphertext, options?)`. Default KDF is Argon2id; opt into scrypt via `kdf: "scrypt"` in options. Default cipher is `xchacha20-poly1305`. Output is a self-describing binary header (`magic`, `kdf`, `kdfParams`, `cipher`, `salt`, `nonce`) followed by the ciphertext.
|
||||
- ADDED: `password/derive-key-from-password.ts` — `deriveKeyFromPassword(password, salt, options?)` returning a 32-byte `Uint8Array` suitable for use as a symmetric key. KDF selection (`argon2id` / `scrypt` / `pbkdf2`) is via the options bundle.
|
||||
- ADDED: error codes `CRYPTO.STREAM_AEAD_HEADER_INVALID`, `CRYPTO.PASSWORD_KDF_UNKNOWN`, `CRYPTO.PASSWORD_BLOB_HEADER_INVALID`, `CRYPTO.PASSWORD_BLOB_VERSION_UNSUPPORTED`.
|
||||
|
||||
### `@suaveplan/crypto-bin` (new package)
|
||||
|
||||
- ADDED: a thin CLI package `packages/crypto/crypto-bin` whose only purpose is to ship the `hash-password` bin. Per-Suaveplan policy CLI surfaces live in their own package so they do not leak `@suaveplan/cli` runtime deps into the universal-tier `@suaveplan/crypto`. Depends on `@suaveplan/cli`, `@suaveplan/crypto-hash`, `@suaveplan/error`. Bin: `suaveplan-hash-password`. Reads a password from stdin or `--password`, writes a verifier hash (Argon2id by default, `--algo blake2b` opt-in) to stdout; with `--verify <stored>` reads `--password` and exits `0` on match, `1` on mismatch.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Post-quantum signature primitives** (Dilithium, SPHINCS+, ML-KEM key-encapsulation) — separate `crypto-pqc-*` family of sub-packages already discussed in `STATUS.md`. Adding PQC primitives requires a separate engine adapter substrate that is larger than this entire change.
|
||||
- **FIDO2 / WebAuthn helpers** — already covered by browser-tier deps in the genesis app shells; no upstream parity gap here.
|
||||
- **HSM / KMS adapters** (CloudHSM, AWS KMS, GCP KMS) — operational concern, not a library gap; tracked separately under the audit-logging adapter family.
|
||||
- **Argon2id parameter migration helper** — upstream ships `migrateArgon2idParams(oldHash, newOptions)` for re-hashing on login. Genesis already routes Argon2id verification through `crypto-hash/primitives/argon2id` and migration belongs in a future `auth` change, not in `crypto-hash`.
|
||||
- **Modifying the existing Argon2id primitive** — the new `encryptWithPassword` defers to the existing `argon2id` primitive without changing its signature.
|
||||
- **`crypto-vault` integration** — `crypto-vault` already wraps Argon2id-hardened HD derivation; once `scrypt` lands in `crypto-hash`, a follow-up change can add a `kdf: "scrypt"` option to the vault. Out of scope here to keep this change focused on the seven enumerated gaps.
|
||||
|
||||
## Source mapping
|
||||
|
||||
Genesis `enterprise/crypto.md` is a generic enterprise-quality template (no per-feature requirements), so source mapping cites the upstream package source directly.
|
||||
|
||||
| Requirement | Source file | Source lines | Notes |
|
||||
|---|---|---|---|
|
||||
| `encryptStreamAead` / `decryptStreamAead` async generators | `packages/universal/crypto/src/symmetric/streaming/streaming-aead.ts` | L11-L255 | Verbatim port; thread `AbortSignal` through |
|
||||
| `aesGcmStreamingEngine` adapter | `packages/universal/crypto/src/symmetric/streaming/streaming-aead-aes-gcm.ts` | L9-L35 | Verbatim |
|
||||
| `xchacha20StreamingEngine` adapter | `packages/universal/crypto/src/symmetric/streaming/streaming-aead-xchacha20.ts` | L9-L35 | Verbatim |
|
||||
| `createEncryptionStream` / `createDecryptionStream` / `streamEncrypt` / `streamDecrypt` / `collectStream` / `createStreamKey` / `createStreamEncryption` | `packages/universal/crypto/src/symmetric/streaming/streaming.ts` | L23-L430 | Verbatim port; preserve `ChaCha20Stream` / `AESStream` named exports |
|
||||
| `signWithRecovery` (secp256k1) | `packages/universal/crypto/src/asymmetric/signing/signing.ts` | L74-L120 | Verbatim |
|
||||
| `recoverPublicKey` (secp256k1) | `packages/universal/crypto/src/asymmetric/signing/signing.ts` | L143-L235 | Verbatim |
|
||||
| `verifyWithRecovery` (secp256k1) | `packages/universal/crypto/src/asymmetric/signing/signing.ts` | L243-L307 | Verbatim |
|
||||
| `createDeviceKey` | `packages/universal/crypto/src/keys/device/device.ts` | L32-L49 | Verbatim |
|
||||
| `deriveDeviceKeyFromMaster` | `packages/universal/crypto/src/keys/device/device.ts` | L56-L89 | Verbatim |
|
||||
| `generateDeviceUUID` | `packages/universal/crypto/src/keys/device/device.ts` | L96-L155 | Verbatim |
|
||||
| `extractDeviceMetadata` | `packages/universal/crypto/src/keys/device/device.ts` | L157-L181 | Verbatim |
|
||||
| `isValidDeviceKey`, `compareDeviceKeys` | `packages/universal/crypto/src/keys/device/device.ts` | L182-L245 | Verbatim |
|
||||
| `generateDerivedDeviceId` back-compat alias | `packages/universal/crypto/src/keys/device/device.ts` | L249-L260 | Replace upstream `@deprecated` JSDoc with prose ("Backwards-compatibility alias for `generateDeviceUUID`.") per CLAUDE.md §2 |
|
||||
| `scrypt` async KDF | `packages/universal/crypto/src/core/hashing/hashing.ts` | L273-L325 | Verbatim (delegates to `@noble/hashes/scrypt::scryptAsync`) |
|
||||
| `blake2b` hash | `packages/universal/crypto/src/core/hashing/hashing.ts` | L1-L60 (imports + helpers used) | Re-host as a top-level primitive in `crypto-hash` |
|
||||
| `hashPasswordBlake2b` | `packages/universal/crypto/src/core/hashing/hashing.ts` | L617-L666 | Verbatim |
|
||||
| `verifyPasswordBlake2b` | `packages/universal/crypto/src/core/hashing/hashing.ts` | L672-L745 | Verbatim |
|
||||
| `encryptWithPassword` | `packages/universal/crypto/src/symmetric/password/password.ts` | L54-L113 | Verbatim port; replace `PasswordError` plain class with `SuaveplanError` code `CRYPTO.PASSWORD_*` |
|
||||
| `deriveKeyFromPassword` | `packages/universal/crypto/src/symmetric/password/password.ts` | L172-L209 | Verbatim |
|
||||
| `hash-password` CLI | `packages/universal/crypto/src/bin/hash-password.ts` | L1-L98 | Verbatim port; replace direct `process.exit` with `cli.exit(code)` from `@suaveplan/cli` |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| `StreamingAeadEngine` interface (formal) | Upstream uses a duck-typed object; genesis convention is a named interface in a `*.types.ts` file (see CLAUDE.md §11 "No nested object literals in interfaces"). |
|
||||
| Error codes mapped to `@suaveplan/error` (`CRYPTO_CIPHER.STREAM_*`, `CRYPTO_CURVE.RECOVERY_*`, `CRYPTO_KEYS.DEVICE_*`, `CRYPTO_HASH.SCRYPT_*` / `BLAKE2B_*`, `CRYPTO.PASSWORD_*`) | Genesis-wide policy: every thrown error routes through `@suaveplan/error` with a stable code. Upstream throws plain `Error` / `PasswordError`. |
|
||||
| `crypto-bin` as its own package | Universal-tier `@suaveplan/crypto` cannot depend on `@suaveplan/cli` (CLI is browser/server-tier and pulls in `@suaveplan/readline`). Splitting the CLI out keeps `crypto` zero-side-effect and tree-shakeable. |
|
||||
| `encryptWithPassword` self-describing binary header | Upstream stores `salt + ciphertext` only; consumers had to know the iteration count externally. Genesis adds a versioned 16-byte header (`magic[4] \| version[1] \| kdf[1] \| cipher[1] \| reserved[1] \| saltLen[2] \| nonceLen[2] \| kdfParamsLen[4]`) so a blob is self-descriptive and rotation-safe. |
|
||||
| `decryptWithPassword` companion to `encryptWithPassword` | Upstream's `decryptWithPassword` (L115-L160 of `password.ts`) is included. Listed here only because the proposal text earlier groups gap 6 as one verb; this row exists so reviewers can see both halves are covered. |
|
||||
| `signWithRecovery` / `recoverPublicKey` / `verifyWithRecovery` exposed at `secp256k1.noble.ts` rather than the higher-level `signing.ts` | Upstream's `signing.ts` is a multi-key façade that pulls `BaseKeySchema` plumbing genesis hasn't ported. Putting the three functions on the curve primitive (where they belong cryptographically) avoids a circular dep with `crypto-keys`. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each spec delta requirement carries a `**Verified by:**` line pointing at concrete `it(...)` titles. The test files that will exist after this change ships:
|
||||
|
||||
- `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.test.ts` — chunk round-trip per engine; tag mismatch detection; abort propagation
|
||||
- `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.spec-conformance.test.ts` — `StreamingAeadEngine` contract pinned for both adapters; nonce-counter monotonicity; out-of-order chunks emit `STREAM_OUT_OF_ORDER`
|
||||
- `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts` — extended with sign/recover/verify-with-recovery cases; recovery byte 0/1 round-trip
|
||||
- `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.spec-conformance.test.ts` — invalid recovery byte emits `RECOVERY_BYTE_INVALID`; missing recovery byte emits `RECOVERY_BYTE_MISSING`; recovered key mismatch emits `RECOVERED_KEY_MISMATCH`
|
||||
- `packages/crypto/crypto-hash/src/primitives/scrypt/scrypt.test.ts` — KAT round-trip vs upstream RFC 7914 vectors; `dkLen` 32 / 64 / 128
|
||||
- `packages/crypto/crypto-hash/src/primitives/scrypt/scrypt.spec-conformance.test.ts` — invalid `N` (non-power-of-two, < 2) emits `SCRYPT_INVALID_N`
|
||||
- `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.test.ts` — RFC 7693 KAT round-trip; output length 1..64; keyed mode
|
||||
- `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.spec-conformance.test.ts` — output > 64 emits `BLAKE2B_OUTPUT_TOO_LONG`; key > 64 emits `BLAKE2B_KEY_TOO_LONG`
|
||||
- `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.test.ts` — round-trip hash → verify; salt freshness across calls; constant-time comparison observable via timing-side-channel test
|
||||
- `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.spec-conformance.test.ts` — malformed stored hash emits `BLAKE2B_PASSWORD_FORMAT_INVALID`
|
||||
- `packages/crypto/crypto-keys/src/keypair/device/device.test.ts` — `createDeviceKey` round-trip; `deriveDeviceKeyFromMaster` deterministic; `generateDeviceUUID` valid v4; `extractDeviceMetadata` round-trip; `compareDeviceKeys` constant-time
|
||||
- `packages/crypto/crypto-keys/src/keypair/device/device.spec-conformance.test.ts` — invalid master emits `DEVICE_INVALID_MASTER`; corrupt key emits `DEVICE_INVALID_KEY`
|
||||
- `packages/crypto/crypto/src/aead/streaming/streaming-aead.test.ts` — façade selects correct engine; round-trip per `xchacha20-poly1305` and `aes-256-gcm`
|
||||
- `packages/crypto/crypto/src/aead/streaming/web-streams.test.ts` — Web-Streams round-trip; `collectStream` aggregates correctly; `streamEncrypt` / `streamDecrypt` aliases match the canonical functions
|
||||
- `packages/crypto/crypto/src/password/encrypt-with-password.test.ts` — round-trip `encrypt → decrypt`; KDF selection (`argon2id` vs `scrypt`); blob header self-descriptive (different KDF params produce different headers; reading a v1 blob with a v2 reader emits `PASSWORD_BLOB_VERSION_UNSUPPORTED`)
|
||||
- `packages/crypto/crypto/src/password/encrypt-with-password.spec-conformance.test.ts` — `kdf: "unknown"` emits `PASSWORD_KDF_UNKNOWN`; truncated blob emits `PASSWORD_BLOB_HEADER_INVALID`
|
||||
- `packages/crypto/crypto/src/password/derive-key-from-password.test.ts` — KDF selection; deterministic given fixed salt + params; output length is exactly 32 bytes
|
||||
- `packages/crypto/crypto-bin/src/hash-password/hash-password.test.ts` — Argon2id mode end-to-end (hash, then verify, exit 0); blake2b mode end-to-end; bad password verify exits 1; missing `--password` arg emits non-zero exit
|
||||
- `packages/crypto/crypto-bin/src/hash-password/hash-password.spec-conformance.test.ts` — error path emits `CRYPTO.HASH_PASSWORD_CLI_*` codes (one per known failure mode)
|
||||
|
||||
## Migration impact
|
||||
|
||||
This change is **additive** — it does not modify any existing public surface. Verification:
|
||||
|
||||
- Existing `@suaveplan/crypto` consumers (`crypto-vault`, `slsa`, `audit-logging`, `auth`, `streams`, `bitpacker-crypto`, `drive-plugin-chunks`, `data/packer`) — none use `signWithRecovery` / streaming AEAD / device keys / scrypt / blake2b / `encryptWithPassword` / hash-password CLI today (the symbols don't exist on genesis). New exports therefore have zero callers at land time.
|
||||
- Existing `crypto-cipher` callers (`crypto`, `crypto-sodium`) — only consume the three primitive selectors (`aes-256-gcm`, `chacha20-poly1305`, `xchacha20-poly1305`). The new `streaming/*` subpath is an isolated subtree.
|
||||
- Existing `crypto-curve` callers (`crypto`) — only consume the four primitive selectors. The three new `signWithRecovery` / `recoverPublicKey` / `verifyWithRecovery` exports do not collide with the existing `sign` / `verify`.
|
||||
- Existing `crypto-hash` callers (`crypto`, `crypto-vault`, `crypto-curve`) — consume `sha256`, `sha384`, `sha512`, `hmac`, `hkdf`, `pbkdf2`, `argon2id`. The new `scrypt` and `blake2b` primitives are isolated.
|
||||
- Existing `crypto-keys` callers (`crypto-vault`, `auth`) — consume `Keychain` and `keypair/{generate,validate}`. The new `keypair/device` subtree is isolated.
|
||||
|
||||
A `bunx turbo build typecheck test --filter=...@suaveplan/crypto*` sweep runs in §9 of `tasks.md` to verify all dependents compile and test green before archive.
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/crypto` to enumerate. For each dependent, check one box:
|
||||
|
||||
- `@suaveplan/crypto-sodium` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes only the existing AEAD / sign / agree façade; new streaming / password helpers are additive subpaths)
|
||||
- `@suaveplan/crypto-browser` — [ ] tests run / [ ] docs reviewed / [x] no action needed (same — consumes existing AEAD / sign / agree façade)
|
||||
- `@suaveplan/crypto-vault` — [x] tests run / [x] docs reviewed / [ ] no action needed (vault may want to opt into `scrypt` KDF in a follow-up; verifying it still typechecks is required because it imports `crypto-hash` directly)
|
||||
- `@suaveplan/crypto-keys` — [x] tests run / [x] docs reviewed / [ ] no action needed (we are *adding* `keypair/device/*` to it in this change, so its own tests must run)
|
||||
- `@suaveplan/auth` — [x] tests run / [ ] docs reviewed / [ ] no action needed (consumes `crypto-keys`, depends on us not breaking the existing `keypair/{generate,validate}` exports)
|
||||
- `@suaveplan/audit-logging` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes only the AEAD façade for log encryption — new streaming subpath does not collide)
|
||||
- `@suaveplan/slsa` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes only `crypto/sign` for in-toto attestation signing — `signWithRecovery` is a new export, not a modification)
|
||||
- `@suaveplan/streams` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes only the existing primitive selectors)
|
||||
- `@suaveplan/bitpacker-crypto` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes the AEAD façade for packed-bit encryption — new exports do not collide)
|
||||
- `@suaveplan/drive-plugin-chunks` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes the AEAD façade for at-rest chunk encryption — new streaming subpath is a candidate for a follow-up port, not a regression risk here)
|
||||
- `@suaveplan/data/packer` — [ ] tests run / [ ] docs reviewed / [x] no action needed (consumes the AEAD façade only)
|
||||
- `@suaveplan/crypto-utils` — [x] tests run / [ ] docs reviewed / [ ] no action needed (foundation — its own coverage gate must pass after this change; `crypto-utils` is unchanged but `crypto-hash` and `crypto-cipher` consume it, so its tests must still pass)
|
||||
|
||||
For the four [x] dependents above (`crypto-vault`, `crypto-keys`, `auth`, `crypto-utils`), running their full test suite is part of §9 of `tasks.md`.
|
||||
@@ -0,0 +1,64 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto-bin (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto-bin`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. This is a new package; the entire spec below is `## ADDED Requirements`.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: hash-password CLI hash mode
|
||||
|
||||
The package SHALL ship a `suaveplan-hash-password` bin that, when invoked with `--password <pw>` and no `--verify`, hashes the password using the requested algorithm (default `argon2id`, opt-in `--algo blake2b`) and writes the resulting stored-hash string to stdout, exiting with code 0. The CLI SHALL route through `@suaveplan/cli`'s arg parser (so `--help` works automatically) and SHALL NEVER call `process.exit` directly — it SHALL use `cli.exit(code)` to keep test harness compatibility.
|
||||
|
||||
#### Scenario: argon2id mode prints stored hash
|
||||
|
||||
- **WHEN** `suaveplan-hash-password --password "correct horse battery staple"` is invoked
|
||||
- **THEN** stdout contains a string starting with `"$argon2id$"` AND the exit code is 0
|
||||
|
||||
#### Scenario: blake2b mode prints stored hash
|
||||
|
||||
- **WHEN** `suaveplan-hash-password --password "..." --algo blake2b` is invoked
|
||||
- **THEN** stdout contains a string starting with `"$blake2b$"` AND the exit code is 0
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-bin/src/hash-password/hash-password.test.ts::"argon2id mode writes a $argon2id$-prefixed hash to stdout and exits 0"` AND `packages/crypto/crypto-bin/src/hash-password/hash-password.test.ts::"blake2b mode writes a $blake2b$-prefixed hash to stdout and exits 0"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: hash-password CLI verify mode
|
||||
|
||||
When invoked with `--password <pw>` and `--verify <stored>`, the bin SHALL detect the algorithm from the stored hash's prefix, run the matching verify function (`verifyPasswordArgon2id` or `verifyPasswordBlake2b`), exit with code 0 on match and code 1 on mismatch. The bin SHALL NOT print anything to stdout in verify mode (so the exit code is the only signal — suitable for use in shell pipelines).
|
||||
|
||||
#### Scenario: matching password exits 0
|
||||
|
||||
- **WHEN** the stored hash and the password agree
|
||||
- **THEN** the exit code is 0 AND stdout is empty
|
||||
|
||||
#### Scenario: non-matching password exits 1
|
||||
|
||||
- **WHEN** the stored hash and the password disagree
|
||||
- **THEN** the exit code is 1 AND stdout is empty
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-bin/src/hash-password/hash-password.test.ts::"verify mode exits 0 when password matches stored hash"` AND `packages/crypto/crypto-bin/src/hash-password/hash-password.test.ts::"verify mode exits 1 when password does not match stored hash"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: hash-password CLI argument validation
|
||||
|
||||
The bin SHALL reject malformed invocations (missing `--password`, unknown `--algo`, malformed `--verify` value) with a non-zero exit code and a `SuaveplanError` whose code is one of `CRYPTO.HASH_PASSWORD_CLI_BAD_ARGS`, `CRYPTO.HASH_PASSWORD_CLI_UNKNOWN_ALGO`, or `CRYPTO.HASH_PASSWORD_CLI_VERIFY_FAILED`. The error message SHALL be human-readable and SHALL be printed to stderr (not stdout, so it does not contaminate pipe output).
|
||||
|
||||
#### Scenario: missing --password rejected
|
||||
|
||||
- **WHEN** `suaveplan-hash-password` is invoked with no `--password`
|
||||
- **THEN** the exit code is non-zero AND stderr mentions `--password` AND the underlying error code is `CRYPTO.HASH_PASSWORD_CLI_BAD_ARGS`
|
||||
|
||||
#### Scenario: unknown algorithm rejected
|
||||
|
||||
- **WHEN** `suaveplan-hash-password --password pw --algo bcrypt` is invoked
|
||||
- **THEN** the exit code is non-zero AND the underlying error code is `CRYPTO.HASH_PASSWORD_CLI_UNKNOWN_ALGO` AND `context.algo === "bcrypt"`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-bin/src/hash-password/hash-password.spec-conformance.test.ts::"missing --password emits HASH_PASSWORD_CLI_BAD_ARGS"` AND `packages/crypto/crypto-bin/src/hash-password/hash-password.spec-conformance.test.ts::"unknown --algo emits HASH_PASSWORD_CLI_UNKNOWN_ALGO"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,95 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto-cipher (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto-cipher`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Streaming AEAD encryption / decryption
|
||||
|
||||
The package SHALL expose `encryptStreamAead(source, options)` and `decryptStreamAead(source, options)` async generators that consume a `Uint8Array` input stream chunk by chunk, attach a per-chunk authentication tag, and emit the encrypted (or decrypted) chunk stream. Each chunk SHALL use a per-chunk nonce derived deterministically from a base nonce plus a 32-bit chunk counter so the entire stream binds to a single key without nonce reuse.
|
||||
|
||||
#### Scenario: round-trip preserves bytes
|
||||
|
||||
- **WHEN** `encryptStreamAead(plaintext, { algorithm, key, baseNonce })` is consumed and its output is fed to `decryptStreamAead(ciphertext, { algorithm, key, baseNonce })`
|
||||
- **THEN** the concatenated output equals the original plaintext byte-for-byte
|
||||
|
||||
#### Scenario: tag mismatch detected on tampered chunk
|
||||
|
||||
- **WHEN** any byte of any encrypted chunk's tag is flipped before decryption
|
||||
- **THEN** `decryptStreamAead` throws a `SuaveplanError` with code `CRYPTO_CIPHER.STREAM_TAG_MISMATCH` and `context.chunkIndex` points at the tampered chunk
|
||||
|
||||
#### Scenario: chunk-counter overflow rejected
|
||||
|
||||
- **WHEN** the chunk counter reaches `2^32`
|
||||
- **THEN** the next chunk emission throws `CRYPTO_CIPHER.STREAM_OUT_OF_ORDER` with `context.counter === 2 ** 32`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.test.ts::"round-trips a multi-chunk plaintext through encryptStreamAead and decryptStreamAead"` AND `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.test.ts::"throws STREAM_TAG_MISMATCH when a chunk tag is flipped"` AND `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.spec-conformance.test.ts::"throws STREAM_OUT_OF_ORDER on chunk-counter overflow"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AbortSignal propagation through streaming AEAD
|
||||
|
||||
The package SHALL honour an optional `AbortSignal` passed via `EncryptStreamAeadOptions.signal` and `DecryptStreamAeadOptions.signal`. When the signal is aborted mid-stream the generator SHALL flush the in-flight chunk, release any pending cipher state, and throw `SuaveplanError` with code `CRYPTO_CIPHER.STREAM_ABORTED`.
|
||||
|
||||
#### Scenario: abort mid-stream rejects with STREAM_ABORTED
|
||||
|
||||
- **WHEN** an `AbortController` is signalled while `encryptStreamAead` is mid-iteration
|
||||
- **THEN** the next `await generator.next()` rejects with `CRYPTO_CIPHER.STREAM_ABORTED` and `context.chunksProcessed` reflects the count emitted before abort
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.test.ts::"aborts mid-stream and rejects with STREAM_ABORTED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AES-GCM streaming engine adapter
|
||||
|
||||
The package SHALL expose `aesGcmStreamingEngine: StreamingAeadEngine` that wraps the existing `aes-256-gcm` primitive. Per-chunk nonce SHALL be a 12-byte big-endian concatenation of an 8-byte caller-supplied prefix and a 4-byte chunk counter (RFC 5116 §3.2 compatible). The engine SHALL reject `baseNonce.length !== 8` with `CRYPTO_CIPHER.STREAM_INVALID_NONCE_BASE`.
|
||||
|
||||
#### Scenario: 8-byte base nonce accepted
|
||||
|
||||
- **WHEN** `aesGcmStreamingEngine.encrypt({ baseNonce: 8-byte prefix, key, plaintext, counter: 0 })` is called
|
||||
- **THEN** the produced nonce equals `baseNonce ‖ counter_be32(0)` (12 bytes total)
|
||||
|
||||
#### Scenario: wrong-length base nonce rejected
|
||||
|
||||
- **WHEN** `baseNonce.length === 12`
|
||||
- **THEN** the engine throws `CRYPTO_CIPHER.STREAM_INVALID_NONCE_BASE` with `context.expectedLength === 8` and `context.actualLength === 12`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-cipher/src/streaming/streaming-aead-aes-gcm/streaming-aead-aes-gcm.test.ts::"derives 12-byte nonce from 8-byte base + counter"` AND `packages/crypto/crypto-cipher/src/streaming/streaming-aead-aes-gcm/streaming-aead-aes-gcm.spec-conformance.test.ts::"rejects wrong-length base nonce with STREAM_INVALID_NONCE_BASE"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: XChaCha20-Poly1305 streaming engine adapter
|
||||
|
||||
The package SHALL expose `xchacha20StreamingEngine: StreamingAeadEngine` that wraps the existing `xchacha20-poly1305` primitive. Per-chunk nonce SHALL be a 24-byte big-endian concatenation of a 20-byte caller-supplied prefix and a 4-byte chunk counter. The engine SHALL reject `baseNonce.length !== 20` with `CRYPTO_CIPHER.STREAM_INVALID_NONCE_BASE`.
|
||||
|
||||
#### Scenario: 20-byte base nonce accepted
|
||||
|
||||
- **WHEN** `xchacha20StreamingEngine.encrypt({ baseNonce: 20-byte prefix, key, plaintext, counter: 5 })` is called
|
||||
- **THEN** the produced nonce equals `baseNonce ‖ counter_be32(5)` (24 bytes total)
|
||||
|
||||
#### Scenario: wrong-length base nonce rejected
|
||||
|
||||
- **WHEN** `baseNonce.length === 24`
|
||||
- **THEN** the engine throws `CRYPTO_CIPHER.STREAM_INVALID_NONCE_BASE` with `context.expectedLength === 20` and `context.actualLength === 24`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-cipher/src/streaming/streaming-aead-xchacha20/streaming-aead-xchacha20.test.ts::"derives 24-byte nonce from 20-byte base + counter"` AND `packages/crypto/crypto-cipher/src/streaming/streaming-aead-xchacha20/streaming-aead-xchacha20.spec-conformance.test.ts::"rejects wrong-length base nonce with STREAM_INVALID_NONCE_BASE"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: StreamingAeadEngine contract
|
||||
|
||||
The package SHALL define a `StreamingAeadEngine` interface in `streaming-aead.types.ts` with `encrypt(input: StreamingChunkInput): Promise<StreamingChunkOutput>` and `decrypt(input: StreamingChunkInput): Promise<Uint8Array>` methods, plus `nonceBaseLength: number` and `tagLength: number` constants. Both built-in engines (AES-GCM, XChaCha20-Poly1305) SHALL satisfy this interface and SHALL be substitutable for any caller-supplied engine.
|
||||
|
||||
#### Scenario: caller-supplied engine round-trips identically
|
||||
|
||||
- **WHEN** a custom `StreamingAeadEngine` implementation is passed to `encryptStreamAead` and the same instance to `decryptStreamAead`
|
||||
- **THEN** the round-trip preserves bytes byte-for-byte (validated against an in-test spy engine that records every call)
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.spec-conformance.test.ts::"StreamingAeadEngine contract is stable for custom engines"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,74 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto-curve (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto-curve`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: secp256k1 ECDSA signing with recovery byte
|
||||
|
||||
The `secp256k1` primitive SHALL expose `signWithRecovery(digest, privateKey)` returning a 65-byte signature (64-byte compact signature followed by a single recovery byte). The recovery byte SHALL be the byte returned by `secp256k1.sign(digest, privateKey).recovery` from the `@noble/curves` library; valid values are `0` or `1`.
|
||||
|
||||
#### Scenario: produces 65-byte signature
|
||||
|
||||
- **WHEN** `signWithRecovery(digest, privateKey)` is called for a valid digest and private key
|
||||
- **THEN** the result is a `Uint8Array` of length 65 AND `result[64]` is `0` or `1`
|
||||
|
||||
#### Scenario: signWithRecovery output verifies via standard verify
|
||||
|
||||
- **WHEN** the first 64 bytes of `signWithRecovery(digest, privateKey)` are passed to the existing `verify(digest, sig64, publicKey)`
|
||||
- **THEN** the signature verifies successfully
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts::"signWithRecovery returns 65-byte signature with recovery byte 0 or 1"` AND `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts::"signWithRecovery output verifies via standard verify on first 64 bytes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: secp256k1 public-key recovery
|
||||
|
||||
The `secp256k1` primitive SHALL expose `recoverPublicKey(signatureWithRecovery, digest)` returning the 33-byte compressed public key recovered from a 65-byte signature-with-recovery and the original digest. Inputs may be supplied as `Uint8Array` or hex strings.
|
||||
|
||||
#### Scenario: recovers the signing public key
|
||||
|
||||
- **WHEN** `signWithRecovery(digest, privateKey)` produces `sig` and `recoverPublicKey(sig, digest)` is called
|
||||
- **THEN** the recovered public key equals `getPublicKey(privateKey)` byte-for-byte
|
||||
|
||||
#### Scenario: missing recovery byte rejected
|
||||
|
||||
- **WHEN** a 64-byte signature (without recovery byte) is passed to `recoverPublicKey`
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_CURVE.RECOVERY_BYTE_MISSING` and `context.signatureLength === 64`
|
||||
|
||||
#### Scenario: invalid recovery byte rejected
|
||||
|
||||
- **WHEN** a 65-byte signature is passed but `signatureWithRecovery[64]` is `2`, `3`, or any value > 3
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_CURVE.RECOVERY_BYTE_INVALID` and `context.recoveryByte` reflects the offending value
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts::"recoverPublicKey returns the same key as getPublicKey for the signer"` AND `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.spec-conformance.test.ts::"throws RECOVERY_BYTE_MISSING for 64-byte signature"` AND `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.spec-conformance.test.ts::"throws RECOVERY_BYTE_INVALID when recovery byte is out of range"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: secp256k1 verify-with-recovery
|
||||
|
||||
The `secp256k1` primitive SHALL expose `verifyWithRecovery(signatureWithRecovery, digest, expectedPublicKey)` returning `true` only when the public key recovered from the signature equals `expectedPublicKey` (compared in constant time on the byte buffer). When the recovered key differs, the function SHALL return `false` (it does NOT throw on mismatch — mismatch is a routine path).
|
||||
|
||||
#### Scenario: returns true on matching key
|
||||
|
||||
- **WHEN** `verifyWithRecovery(sig, digest, expectedPublicKey)` is called with the public key that produced `sig`
|
||||
- **THEN** the result is `true`
|
||||
|
||||
#### Scenario: returns false on non-matching key
|
||||
|
||||
- **WHEN** `verifyWithRecovery(sig, digest, otherPublicKey)` is called with a public key that did not produce `sig`
|
||||
- **THEN** the result is `false` (no exception is thrown)
|
||||
|
||||
#### Scenario: comparison is constant-time
|
||||
|
||||
- **WHEN** the recovered public key and `expectedPublicKey` differ in their first byte vs differ only in their last byte
|
||||
- **THEN** the timing distribution observed across many invocations does not reveal the position of the difference (verified via a statistical timing test)
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts::"verifyWithRecovery returns true on matching public key"` AND `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.test.ts::"verifyWithRecovery returns false on non-matching public key"` AND `packages/crypto/crypto-curve/src/primitives/secp256k1/secp256k1.spec-conformance.test.ts::"verifyWithRecovery comparison is constant-time"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,79 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto-hash (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto-hash`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: scrypt KDF primitive
|
||||
|
||||
The package SHALL expose `scrypt(password, salt, options)` from `primitives/scrypt/scrypt.ts`, where `options` carries `N` (CPU/memory cost, must be a power of two ≥ 2), `r` (block size), `p` (parallelisation), and `dkLen` (output length in bytes). The implementation SHALL match RFC 7914 byte-for-byte and SHALL use the `@noble/hashes/scrypt::scryptAsync` engine in the noble selector. Output is a `Uint8Array` of length `dkLen`.
|
||||
|
||||
#### Scenario: matches RFC 7914 known-answer test
|
||||
|
||||
- **WHEN** `scrypt("password", "NaCl", { N: 1024, r: 8, p: 16, dkLen: 64 })` is called
|
||||
- **THEN** the output equals the RFC 7914 §11 KAT for those parameters byte-for-byte
|
||||
|
||||
#### Scenario: invalid N rejected
|
||||
|
||||
- **WHEN** `scrypt(pw, salt, { N: 1000, r: 8, p: 1, dkLen: 32 })` is called (1000 is not a power of two)
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_HASH.SCRYPT_INVALID_N` and `context.N === 1000`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-hash/src/primitives/scrypt/scrypt.test.ts::"matches RFC 7914 known-answer test for N=1024 r=8 p=16"` AND `packages/crypto/crypto-hash/src/primitives/scrypt/scrypt.spec-conformance.test.ts::"throws SCRYPT_INVALID_N when N is not a power of two"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: blake2b hash primitive
|
||||
|
||||
The package SHALL expose `blake2b(message, options)` from `primitives/blake2b/blake2b.ts`, where `options` carries `dkLen` (1..64), optional `key` (≤ 64 bytes), optional `salt` (16 bytes), and optional `personalization` (16 bytes). The implementation SHALL match RFC 7693 byte-for-byte.
|
||||
|
||||
#### Scenario: matches RFC 7693 known-answer test
|
||||
|
||||
- **WHEN** `blake2b("abc", { dkLen: 64 })` is called
|
||||
- **THEN** the output equals the RFC 7693 §B Appendix KAT for the input "abc" byte-for-byte
|
||||
|
||||
#### Scenario: dkLen > 64 rejected
|
||||
|
||||
- **WHEN** `blake2b(msg, { dkLen: 65 })` is called
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_HASH.BLAKE2B_OUTPUT_TOO_LONG` and `context.requestedLength === 65`
|
||||
|
||||
#### Scenario: key longer than 64 bytes rejected
|
||||
|
||||
- **WHEN** `blake2b(msg, { dkLen: 32, key: 65-byte-buffer })` is called
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_HASH.BLAKE2B_KEY_TOO_LONG` and `context.keyLength === 65`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.test.ts::"matches RFC 7693 known-answer test for input \"abc\""` AND `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.spec-conformance.test.ts::"throws BLAKE2B_OUTPUT_TOO_LONG when dkLen exceeds 64"` AND `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.spec-conformance.test.ts::"throws BLAKE2B_KEY_TOO_LONG when key exceeds 64 bytes"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: blake2b password hashing and verification
|
||||
|
||||
The package SHALL expose `hashPasswordBlake2b(password, options)` returning a stored hash in the format `$blake2b$v=1$cf=<costFactor>$<saltHex>$<hashHex>`, where `costFactor` controls iteration count (default 14, doubling each level) and a 16-byte random salt is generated when not supplied. It SHALL also expose `verifyPasswordBlake2b(storedHash, password)` returning `true` when the password verifies and `false` otherwise. Comparison SHALL be constant-time over the hash bytes.
|
||||
|
||||
#### Scenario: hash and verify round-trip
|
||||
|
||||
- **WHEN** `hashPasswordBlake2b("correct horse battery staple")` produces `stored` and `verifyPasswordBlake2b(stored, "correct horse battery staple")` is called
|
||||
- **THEN** the result is `true`
|
||||
|
||||
#### Scenario: wrong password verifies as false
|
||||
|
||||
- **WHEN** `verifyPasswordBlake2b(stored, "wrong password")` is called
|
||||
- **THEN** the result is `false` (no exception thrown)
|
||||
|
||||
#### Scenario: salt is fresh per call
|
||||
|
||||
- **WHEN** `hashPasswordBlake2b("same password")` is called twice
|
||||
- **THEN** the two stored hashes are not byte-equal (different salts → different outputs)
|
||||
|
||||
#### Scenario: malformed stored hash rejected
|
||||
|
||||
- **WHEN** `verifyPasswordBlake2b("not-a-valid-format", pw)` is called
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_HASH.BLAKE2B_PASSWORD_FORMAT_INVALID`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.test.ts::"hash and verify round-trip with same password"` AND `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.test.ts::"verify returns false for wrong password"` AND `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.test.ts::"salt is fresh per call so identical inputs hash differently"` AND `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.spec-conformance.test.ts::"throws BLAKE2B_PASSWORD_FORMAT_INVALID for malformed stored hash"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,110 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto-keys (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto-keys`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Device key creation
|
||||
|
||||
The package SHALL expose `createDeviceKey(options?)` from `keypair/device/device.ts` that generates a new `DeviceKey` carrying `BaseKeySchema` metadata (`s` secret bytes, `v` version, `m` master version, `u` user id, `d` depth, `o` key offset). Device keys SHALL always have `o === 7`. Generated keys SHALL pass `isValidDeviceKey`; an internally-failed validation SHALL throw `SuaveplanError` with code `CRYPTO_KEYS.DEVICE_INVALID_KEY`.
|
||||
|
||||
#### Scenario: round-trip valid key
|
||||
|
||||
- **WHEN** `createDeviceKey()` is called with no options
|
||||
- **THEN** the result satisfies `isValidDeviceKey(key) === true` AND `key.o === 7` AND `key.s` is a 32-byte `Uint8Array`
|
||||
|
||||
#### Scenario: caller-supplied metadata respected
|
||||
|
||||
- **WHEN** `createDeviceKey({ version: 3, masterVersion: 2, userId: 42, depth: 4 })` is called
|
||||
- **THEN** the result has `key.v === 3`, `key.m === 2`, `key.u === 42`, `key.d === 4`, `key.o === 7`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"createDeviceKey produces a valid device key with offset 7"` AND `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"createDeviceKey honors version / masterVersion / userId / depth options"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Device key derivation from master
|
||||
|
||||
The package SHALL expose `deriveDeviceKeyFromMaster(options)` that derives a per-device `DeviceKey` from a `MasterKey` plus a numeric `deviceId`. Derivation SHALL use the existing `generateDerivedKey` (from `crypto-keys/keypair/derivation`) with key offset `7` and a context derived from the device id. The same `(masterKey, deviceId)` SHALL always produce the same `DeviceKey.s` byte sequence. Invalid master keys SHALL throw `CRYPTO_KEYS.DEVICE_INVALID_MASTER`.
|
||||
|
||||
#### Scenario: deterministic derivation
|
||||
|
||||
- **WHEN** `deriveDeviceKeyFromMaster({ masterKey, deviceId: 1 })` is called twice
|
||||
- **THEN** both results have byte-equal `.s` fields
|
||||
|
||||
#### Scenario: different device ids produce different keys
|
||||
|
||||
- **WHEN** `deriveDeviceKeyFromMaster({ masterKey, deviceId: 1 })` and `deriveDeviceKeyFromMaster({ masterKey, deviceId: 2 })` are called
|
||||
- **THEN** the two results have different `.s` fields
|
||||
|
||||
#### Scenario: invalid master key rejected
|
||||
|
||||
- **WHEN** `deriveDeviceKeyFromMaster({ masterKey: { s: shortBytes } as MasterKey, deviceId: 1 })` is called with a master key that fails `validateMasterKey`
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO_KEYS.DEVICE_INVALID_MASTER`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"deriveDeviceKeyFromMaster is deterministic for fixed master and deviceId"` AND `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"different deviceIds derive different keys"` AND `packages/crypto/crypto-keys/src/keypair/device/device.spec-conformance.test.ts::"throws DEVICE_INVALID_MASTER for an invalid master key"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Device UUID generation
|
||||
|
||||
The package SHALL expose `generateDeviceUUID(options)` returning an RFC 4122-compliant UUID v4 string deterministically derived from a `DeviceKey`. The version nibble (bits 12-15 of `time_hi_and_version`) SHALL be set to `4`; the variant bits (bits 6-7 of `clock_seq_hi`) SHALL be set to `10`. The same `DeviceKey` plus context SHALL always produce the same UUID.
|
||||
|
||||
#### Scenario: UUID structure conforms to v4
|
||||
|
||||
- **WHEN** `generateDeviceUUID({ deviceMasterKey })` is called
|
||||
- **THEN** the result matches `/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/`
|
||||
|
||||
#### Scenario: deterministic for fixed inputs
|
||||
|
||||
- **WHEN** `generateDeviceUUID(opts)` is called twice with the same `opts`
|
||||
- **THEN** both calls return the same string
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"generateDeviceUUID returns an RFC 4122 v4 UUID"` AND `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"generateDeviceUUID is deterministic for fixed inputs"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Device metadata extraction and comparison
|
||||
|
||||
The package SHALL expose `extractDeviceMetadata(deviceKey)` returning `DeviceMetadata` (`version`, `masterVersion`, `userId`, `depth`, `keyOffset`) and `compareDeviceKeys(a, b)` returning `true` only when the two `DeviceKey` instances have identical `s`, `v`, `m`, `u`, `d`, `o` fields. The `s`-byte comparison SHALL be constant-time. The package SHALL also expose `isValidDeviceKey(deviceKey)` returning `true` only for keys passing structural validation (correct field types and `o === 7`).
|
||||
|
||||
#### Scenario: extractDeviceMetadata round-trips
|
||||
|
||||
- **WHEN** `extractDeviceMetadata(createDeviceKey({ version: 5, depth: 3 }))` is called
|
||||
- **THEN** the result has `version === 5` and `depth === 3`
|
||||
|
||||
#### Scenario: compareDeviceKeys equals on identical keys
|
||||
|
||||
- **WHEN** `compareDeviceKeys(k, k)` is called for any `DeviceKey` k
|
||||
- **THEN** the result is `true`
|
||||
|
||||
#### Scenario: compareDeviceKeys differs on differing s
|
||||
|
||||
- **WHEN** `compareDeviceKeys(k1, k2)` is called for keys with different `.s` bytes
|
||||
- **THEN** the result is `false`
|
||||
|
||||
#### Scenario: compareDeviceKeys is constant-time on s
|
||||
|
||||
- **WHEN** the `.s` bytes differ at byte 0 vs byte 31 (last byte)
|
||||
- **THEN** the timing distribution observed across many invocations does not reveal which byte differs (statistical timing test)
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"extractDeviceMetadata round-trips a created device key"` AND `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"compareDeviceKeys is true for identical keys"` AND `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"compareDeviceKeys is false for keys with differing s"` AND `packages/crypto/crypto-keys/src/keypair/device/device.spec-conformance.test.ts::"compareDeviceKeys runs in constant time over s"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: generateDerivedDeviceId back-compat alias
|
||||
|
||||
The package SHALL expose `generateDerivedDeviceId(options)` as a thin alias for `generateDeviceUUID(options)`, retained for upstream-API back-compat. The alias SHALL NOT use `@deprecated` JSDoc (banned in genesis per CLAUDE.md §2). Its sibling `.md` SHALL document it with prose: "Backwards-compatibility alias for `generateDeviceUUID`."
|
||||
|
||||
#### Scenario: alias produces identical output
|
||||
|
||||
- **WHEN** `generateDerivedDeviceId(opts)` and `generateDeviceUUID(opts)` are called with the same options
|
||||
- **THEN** both return the same string
|
||||
|
||||
**Verified by:** `packages/crypto/crypto-keys/src/keypair/device/device.test.ts::"generateDerivedDeviceId returns the same UUID as generateDeviceUUID"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
@@ -0,0 +1,107 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: crypto (crypto-completion)
|
||||
|
||||
**Package**: `@suaveplan/crypto`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: High-level streaming AEAD façade
|
||||
|
||||
The package SHALL expose `encryptStreamAead(opts)` and `decryptStreamAead(opts)` from `aead/streaming/streaming-aead.ts` that select the engine (`xchacha20-poly1305` or `aes-256-gcm`) from `opts.algorithm` and delegate to the corresponding `StreamingAeadEngine` in `@suaveplan/crypto-cipher`. The package SHALL re-export the named engines `aesGcmStreamingEngine` and `xchacha20StreamingEngine` so callers wanting the engine directly do not need to depend on `crypto-cipher`.
|
||||
|
||||
#### Scenario: facade routes to AES-GCM engine
|
||||
|
||||
- **WHEN** `encryptStreamAead({ algorithm: "aes-256-gcm", key, baseNonce: 8-byte, plaintext })` is called
|
||||
- **THEN** the produced ciphertext decrypts via the standalone `aesGcmStreamingEngine` exported from this package, byte-for-byte
|
||||
|
||||
#### Scenario: facade routes to XChaCha20-Poly1305 engine
|
||||
|
||||
- **WHEN** `encryptStreamAead({ algorithm: "xchacha20-poly1305", key, baseNonce: 20-byte, plaintext })` is called
|
||||
- **THEN** the produced ciphertext decrypts via the standalone `xchacha20StreamingEngine` exported from this package, byte-for-byte
|
||||
|
||||
#### Scenario: unknown algorithm rejected
|
||||
|
||||
- **WHEN** `encryptStreamAead({ algorithm: "rot13" as never, ... })` is called
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO.STREAM_AEAD_HEADER_INVALID` and `context.algorithm === "rot13"`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/aead/streaming/streaming-aead.test.ts::"facade routes aes-256-gcm to aesGcmStreamingEngine"` AND `packages/crypto/crypto/src/aead/streaming/streaming-aead.test.ts::"facade routes xchacha20-poly1305 to xchacha20StreamingEngine"` AND `packages/crypto/crypto/src/aead/streaming/streaming-aead.spec-conformance.test.ts::"throws STREAM_AEAD_HEADER_INVALID on unknown algorithm"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Web-Streams adapters for streaming AEAD
|
||||
|
||||
The package SHALL expose `createEncryptionStream(opts) → TransformStream<Uint8Array, Uint8Array>` and `createDecryptionStream(opts) → TransformStream<Uint8Array, Uint8Array>` from `aead/streaming/web-streams.ts`. These SHALL be drop-in `pipeThrough` targets for any Web-Streams `ReadableStream<Uint8Array>` source (e.g. `Blob.stream()`, `Response.body`, `fetch().body`). The package SHALL also expose `collectStream(stream) → Promise<Uint8Array>` (concatenates all chunks), `createStreamKey(rng?) → Uint8Array` (returns 32 random bytes), and `createStreamEncryption(opts)` (combined key + base-nonce + initialised stream factory).
|
||||
|
||||
#### Scenario: pipeThrough round-trips a Blob
|
||||
|
||||
- **WHEN** `blob.stream().pipeThrough(createEncryptionStream(opts)).pipeThrough(createDecryptionStream(opts))` is collected via `collectStream`
|
||||
- **THEN** the collected bytes equal the original blob bytes
|
||||
|
||||
#### Scenario: legacy aliases preserved
|
||||
|
||||
- **WHEN** `streamEncrypt`, `streamDecrypt`, `ChaCha20Stream`, `AESStream` are imported
|
||||
- **THEN** all four resolve to defined values that satisfy the documented upstream contract
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/aead/streaming/web-streams.test.ts::"pipeThrough round-trips a multi-chunk source through encrypt then decrypt"` AND `packages/crypto/crypto/src/aead/streaming/web-streams.test.ts::"streamEncrypt / streamDecrypt / ChaCha20Stream / AESStream aliases resolve and round-trip"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Password-based encryption helpers
|
||||
|
||||
The package SHALL expose `encryptWithPassword(password, plaintext, options?)` and `decryptWithPassword(password, ciphertext, options?)` from `password/encrypt-with-password.ts`. The `options.kdf` field SHALL accept `"argon2id"` (default), `"scrypt"`, or `"pbkdf2"`. The output blob SHALL begin with a versioned 16-byte self-describing header `{ magic: "SVPP", version: 1, kdf, cipher, reserved, saltLen, nonceLen, kdfParamsLen }` followed by salt, nonce, CBOR-encoded `kdfParams`, then AEAD ciphertext. `decryptWithPassword` SHALL parse the header to recover all parameters and SHALL NOT require any out-of-band metadata from the caller.
|
||||
|
||||
#### Scenario: round-trip with default Argon2id
|
||||
|
||||
- **WHEN** `encryptWithPassword("correct horse battery staple", plaintext)` produces `blob` and `decryptWithPassword("correct horse battery staple", blob)` is called
|
||||
- **THEN** the result equals `plaintext` byte-for-byte
|
||||
|
||||
#### Scenario: round-trip with scrypt KDF
|
||||
|
||||
- **WHEN** `encryptWithPassword(pw, plaintext, { kdf: "scrypt" })` produces `blob` and `decryptWithPassword(pw, blob)` is called
|
||||
- **THEN** the result equals `plaintext` byte-for-byte AND the blob's KDF byte signals scrypt
|
||||
|
||||
#### Scenario: blob header self-describes
|
||||
|
||||
- **WHEN** the same `(password, plaintext)` is encrypted twice with different `kdfParams`
|
||||
- **THEN** the two blobs have different KDF-params bytes in their headers AND both decrypt successfully without the caller passing `kdfParams` to `decryptWithPassword`
|
||||
|
||||
#### Scenario: unknown KDF rejected
|
||||
|
||||
- **WHEN** `encryptWithPassword(pw, plaintext, { kdf: "bcrypt" as never })` is called
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO.PASSWORD_KDF_UNKNOWN` and `context.kdf === "bcrypt"`
|
||||
|
||||
#### Scenario: corrupt header rejected
|
||||
|
||||
- **WHEN** `decryptWithPassword(pw, blob)` is called with a `blob` whose first four bytes are not `"SVPP"`
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO.PASSWORD_BLOB_HEADER_INVALID`
|
||||
|
||||
#### Scenario: unsupported version rejected
|
||||
|
||||
- **WHEN** `decryptWithPassword(pw, blob)` is called with a `blob` whose version byte is `99`
|
||||
- **THEN** it throws `SuaveplanError` with code `CRYPTO.PASSWORD_BLOB_VERSION_UNSUPPORTED` and `context.version === 99`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/password/encrypt-with-password.test.ts::"round-trips with default Argon2id KDF"` AND `packages/crypto/crypto/src/password/encrypt-with-password.test.ts::"round-trips with scrypt KDF"` AND `packages/crypto/crypto/src/password/encrypt-with-password.test.ts::"blob header is self-describing across kdfParams variants"` AND `packages/crypto/crypto/src/password/encrypt-with-password.spec-conformance.test.ts::"throws PASSWORD_KDF_UNKNOWN on unknown KDF name"` AND `packages/crypto/crypto/src/password/encrypt-with-password.spec-conformance.test.ts::"throws PASSWORD_BLOB_HEADER_INVALID on corrupt magic bytes"` AND `packages/crypto/crypto/src/password/encrypt-with-password.spec-conformance.test.ts::"throws PASSWORD_BLOB_VERSION_UNSUPPORTED on unknown version byte"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: deriveKeyFromPassword
|
||||
|
||||
The package SHALL expose `deriveKeyFromPassword(password, salt, options?)` from `password/derive-key-from-password.ts`. The `options` SHALL accept `kdf: "argon2id" | "scrypt" | "pbkdf2"` (default `"argon2id"`), `kdfParams: Record<string, number>` (algorithm-specific), and `dkLen: number` (default 32). The function SHALL return a `Uint8Array` of length `dkLen` suitable for direct use as a symmetric AEAD key. Identical `(password, salt, kdf, kdfParams)` SHALL produce byte-identical output.
|
||||
|
||||
#### Scenario: deterministic given fixed inputs
|
||||
|
||||
- **WHEN** `deriveKeyFromPassword(pw, salt, { kdf: "scrypt", kdfParams: { N: 1024, r: 8, p: 1 } })` is called twice
|
||||
- **THEN** both calls return byte-identical 32-byte Uint8Arrays
|
||||
|
||||
#### Scenario: dkLen honored
|
||||
|
||||
- **WHEN** `deriveKeyFromPassword(pw, salt, { kdf: "argon2id", dkLen: 64 })` is called
|
||||
- **THEN** the result is a 64-byte Uint8Array
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/password/derive-key-from-password.test.ts::"deterministic for fixed password / salt / params"` AND `packages/crypto/crypto/src/password/derive-key-from-password.test.ts::"honors dkLen option"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
181
openspec/changes/archive/2026-05-03-crypto-completion/tasks.md
Normal file
181
openspec/changes/archive/2026-05-03-crypto-completion/tasks.md
Normal file
@@ -0,0 +1,181 @@
|
||||
# Tasks — Crypto completion
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick boxes for work that was already in main when this change opened. Do not tick a box for a test that doesn't exist or doesn't pin the requirement it claims to verify.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [x] 1.1 Read `../suaveplan/packages/universal/crypto/src/symmetric/streaming/streaming-aead.ts`, `streaming.ts`, `streaming-aead-aes-gcm.ts`, `streaming-aead-xchacha20.ts` end-to-end
|
||||
- [x] 1.2 Read `../suaveplan/packages/universal/crypto/src/asymmetric/signing/signing.ts` (focus on lines L74-L307 — sign / recover / verify with recovery)
|
||||
- [x] 1.3 Read `../suaveplan/packages/universal/crypto/src/keys/device/device.ts` end-to-end
|
||||
- [x] 1.4 Read `../suaveplan/packages/universal/crypto/src/core/hashing/hashing.ts` (focus on `scrypt`, `blake2b`, `hashPasswordBlake2b`, `verifyPasswordBlake2b` — lines L273-L745)
|
||||
- [x] 1.5 Read `../suaveplan/packages/universal/crypto/src/symmetric/password/password.ts` end-to-end
|
||||
- [x] 1.6 Read `../suaveplan/packages/universal/crypto/src/bin/hash-password.ts` end-to-end
|
||||
- [x] 1.7 Read `openspec/specs/{crypto,crypto-cipher,crypto-curve,crypto-hash}/spec.md` (existing folded specs)
|
||||
- [x] 1.8 Read `openspec/AGENTS.md` Sections 13 (spec-conformance audit) and 14 (source mapping for migrations)
|
||||
- [x] 1.9 Draft `proposal.md` with explicit ADDED / MODIFIED / REMOVED lists per affected package and a `## Source mapping` table citing upstream line ranges
|
||||
- [x] 1.10 Draft spec deltas under `specs/<cap>/spec.md` for `crypto-cipher`, `crypto-curve`, `crypto-hash`, `crypto-keys`, `crypto`
|
||||
- [x] 1.11 Every requirement in every spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles
|
||||
- [x] 1.12 `openspec change validate crypto-completion` exits with no errors
|
||||
|
||||
## 2. `@suaveplan/crypto-cipher` — streaming AEAD
|
||||
|
||||
- [x] 2.1 Create `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.types.ts` defining `StreamingAeadEngine`, `StreamingChunkTag`, `StreamingHeader`, `EncryptStreamAeadOptions`, `DecryptStreamAeadOptions`
|
||||
- [x] 2.2 Port `encryptStreamAead` and `decryptStreamAead` from upstream `streaming-aead.ts` L11-L255 into `packages/crypto/crypto-cipher/src/streaming/streaming-aead/streaming-aead.ts`
|
||||
- [x] 2.3 Port `aesGcmStreamingEngine` from upstream `streaming-aead-aes-gcm.ts` into `packages/crypto/crypto-cipher/src/streaming/streaming-aead-aes-gcm/streaming-aead-aes-gcm.ts`
|
||||
- [x] 2.4 Port `xchacha20StreamingEngine` from upstream `streaming-aead-xchacha20.ts` into `packages/crypto/crypto-cipher/src/streaming/streaming-aead-xchacha20/streaming-aead-xchacha20.ts`
|
||||
- [x] 2.5 Thread `AbortSignal` through `encryptStreamAead` / `decryptStreamAead`; abort flushes in-flight chunk and rejects with `CRYPTO_CIPHER.STREAM_ABORTED`
|
||||
- [x] 2.6 Add per-chunk nonce derivation (8-byte base + 32-bit counter for AES-GCM = 12-byte nonce; 20-byte base + 32-bit counter for XChaCha20-Poly1305 = 24-byte nonce). Counter overflow at `2^32` chunks emits `CRYPTO_CIPHER.STREAM_OUT_OF_ORDER`
|
||||
- [x] 2.7 Add error codes to `packages/crypto/crypto-cipher/src/errors/codes/codes.ts`: `STREAM_TAG_MISMATCH`, `STREAM_OUT_OF_ORDER`, `STREAM_ABORTED`, `STREAM_INVALID_NONCE_BASE`
|
||||
- [x] 2.8 Wire `streaming/streaming-aead/streaming-aead.test.ts` and `streaming-aead.spec-conformance.test.ts`
|
||||
- [x] 2.9 Add subpath exports `./streaming/streaming-aead`, `./streaming/streaming-aead-aes-gcm`, `./streaming/streaming-aead-xchacha20` in `crypto-cipher/package.json`
|
||||
- [x] 2.10 Sibling `.md` per implementation file (≥200 words, basic + advanced examples, full API reference)
|
||||
|
||||
## 3. `@suaveplan/crypto-curve` — secp256k1 ECDSA recovery
|
||||
|
||||
- [x] 3.1 Add `signWithRecovery(digest, privateKey)` to `packages/crypto/crypto-curve/src/primitives/secp256k1/impl/secp256k1.noble/secp256k1.noble.ts`. Signature is 65 bytes: 64-byte compact signature + 1 recovery byte
|
||||
- [x] 3.2 Add `recoverPublicKey(signatureWithRecovery, digest)` to the same noble impl, using `secp256k1.Signature.fromBytes(sig, "compact").addRecoveryBit(rec).recoverPublicKey(digest)`
|
||||
- [x] 3.3 Add `verifyWithRecovery(signatureWithRecovery, digest, expectedPublicKey)` to the same noble impl, recovering then comparing to `expectedPublicKey` in constant time
|
||||
- [x] 3.4 Add matching stubs to `secp256k1.native/secp256k1.native.ts` that throw `PrimitiveNotImplemented` (per genesis convention — libsodium does not ship secp256k1)
|
||||
- [x] 3.5 Re-export the three new functions via `secp256k1.select/secp256k1.select.ts` (existing re-export-everything pattern picks them up automatically) and from `secp256k1.ts` / `secp256k1/index.ts`
|
||||
- [x] 3.6 Add error codes to `packages/crypto/crypto-curve/src/errors/codes/codes.ts`: `RECOVERY_BYTE_MISSING`, `RECOVERY_BYTE_INVALID`, `RECOVERED_KEY_MISMATCH`
|
||||
- [x] 3.7 Extend `secp256k1.test.ts` with sign / recover / verify-with-recovery cases and `secp256k1.spec-conformance.test.ts` with the three error-code paths
|
||||
- [x] 3.8 Update `secp256k1.md` to document the three new functions and the 65-byte signature layout
|
||||
|
||||
## 4. `@suaveplan/crypto-hash` — scrypt + blake2b + blake2b password
|
||||
|
||||
- [x] 4.1 Create `packages/crypto/crypto-hash/src/primitives/scrypt/scrypt.ts` with `scrypt(password, salt, options: { N, r, p, dkLen })`. Use `@noble/hashes/scrypt::scryptAsync`
|
||||
- [x] 4.2 Add per-primitive selector layout: `scrypt.select/scrypt.select.ts`, `impl/scrypt.noble/scrypt.noble.ts`. Match the existing `argon2id` / `pbkdf2` shape
|
||||
- [x] 4.3 Validate `N` is a power of two ≥ 2; reject with `HASH.SCRYPT_INVALID_N`
|
||||
- [x] 4.4 Create `packages/crypto/crypto-hash/src/primitives/blake2b/blake2b.ts` with `blake2b(message, options: { dkLen, key?, salt?, personalization? })`. Use `@noble/hashes/blake2b`
|
||||
- [x] 4.5 Add per-primitive selector layout: `blake2b.select/blake2b.select.ts`, `impl/blake2b.noble/blake2b.noble.ts`
|
||||
- [x] 4.6 Reject `dkLen > 64` with `HASH.BLAKE2B_OUTPUT_TOO_LONG`; reject `key.length > 64` with `HASH.BLAKE2B_KEY_TOO_LONG`
|
||||
- [x] 4.7 Create `packages/crypto/crypto-hash/src/mac/blake2b-password/blake2b-password.ts` with `hashPasswordBlake2b(password, options: { costFactor?, salt? })` and `verifyPasswordBlake2b(storedHash, password)`. Format: `$blake2b$v=1$cf=<costFactor>$<saltHex>$<hashHex>`
|
||||
- [x] 4.8 Constant-time comparison in `verifyPasswordBlake2b` using `crypto-utils::constantTimeEqual`
|
||||
- [x] 4.9 Add error codes to `packages/crypto/crypto-hash/src/errors/codes/codes.ts`: `SCRYPT_INVALID_N`, `BLAKE2B_OUTPUT_TOO_LONG`, `BLAKE2B_KEY_TOO_LONG`, `BLAKE2B_PASSWORD_FORMAT_INVALID`
|
||||
- [x] 4.10 Add subpath exports `./primitives/scrypt/scrypt`, `./primitives/blake2b/blake2b`, `./mac/blake2b-password/blake2b-password` in `crypto-hash/package.json`
|
||||
- [x] 4.11 Sibling `.md` per implementation file
|
||||
|
||||
## 5. `@suaveplan/crypto-keys` — device-key suite
|
||||
|
||||
**DEFERRED — `crypto-keys` package does not exist on genesis.** Section 5 requires a prerequisite `@suaveplan/crypto-keys` package with `BaseKeySchema` types (`DeviceKey`, `MasterKey`), `validateMasterKey`, `validateDeviceKey`, `generateDerivedKey`, and `generateDeviceKey` infrastructure. None of those exist on genesis (the upstream package was not migrated; `git status` shows `D packages/crypto/crypto-keys/package.json`). Building them from scratch would expand this change's scope from "seven feature gaps" to "one full prerequisite package + seven feature gaps", which violates the change's stated boundary. The device-key suite is **deferred to a follow-up `crypto-keys-port` change** that will (a) port the upstream `BaseKeySchema` infrastructure first, then (b) layer the device-key suite on top.
|
||||
|
||||
The crypto-keys spec delta (`specs/crypto-keys/spec.md`) is retained in the archived change folder for future reference but is not folded into `openspec/specs/crypto-keys/spec.md` (no such spec exists yet).
|
||||
|
||||
- [ ] 5.1 Create `packages/crypto/crypto-keys/src/keypair/device/device.types.ts` with `DeviceKey`, `MasterKey`, `DeviceKeyOptions`, `DeviceKeyDerivationOptions`, `DeviceIdOptions`, `DeviceMetadata` — DEFERRED (package missing)
|
||||
- [ ] 5.2 Port `createDeviceKey` — DEFERRED (package missing)
|
||||
- [ ] 5.3 Port `deriveDeviceKeyFromMaster` — DEFERRED (package missing)
|
||||
- [ ] 5.4 Port `generateDeviceUUID` — DEFERRED (package missing)
|
||||
- [ ] 5.5 Port `extractDeviceMetadata`, `isValidDeviceKey`, `compareDeviceKeys` — DEFERRED (package missing)
|
||||
- [ ] 5.6 Port `generateDerivedDeviceId` back-compat alias — DEFERRED (package missing)
|
||||
- [ ] 5.7 Add error codes `DEVICE_INVALID_MASTER`, `DEVICE_INVALID_KEY`, `DEVICE_DERIVATION_FAILED` — DEFERRED (package missing)
|
||||
- [ ] 5.8 Add subpath export `./keypair/device/device` — DEFERRED (package missing)
|
||||
- [ ] 5.9 Sibling `.md` per implementation file — DEFERRED (package missing)
|
||||
- [ ] 5.10 Wire `keypair/device/device.test.ts` and `device.spec-conformance.test.ts` — DEFERRED (package missing)
|
||||
|
||||
## 6. `@suaveplan/crypto` — streaming AEAD façade + password helpers
|
||||
|
||||
- [x] 6.1 Create `packages/crypto/crypto/src/aead/streaming/streaming-aead.ts` — high-level façade re-exporting `aesGcmStreamingEngine` and `xchacha20StreamingEngine` and providing `encryptStreamAead({ algorithm, key, baseNonce, ... })` / `decryptStreamAead({ algorithm, key, ... })` that select the engine
|
||||
- [x] 6.2 Create `packages/crypto/crypto/src/aead/streaming/web-streams.ts` — `createEncryptionStream(opts) → TransformStream<Uint8Array, Uint8Array>` and `createDecryptionStream(opts) → TransformStream<Uint8Array, Uint8Array>`. Internally drives `encryptStreamAead` / `decryptStreamAead` async generators
|
||||
- [x] 6.3 Port `collectStream`, `createStreamKey`, `createStreamEncryption` from upstream `streaming.ts` L200-L270
|
||||
- [x] 6.4 Re-export `streamEncrypt` / `streamDecrypt` aliases (matching the upstream public surface) and `ChaCha20Stream` / `AESStream` named exports
|
||||
- [x] 6.5 Create `packages/crypto/crypto/src/password/encrypt-with-password.ts` with `encryptWithPassword(password, plaintext, options?)` and `decryptWithPassword(password, ciphertext, options?)`. Default KDF Argon2id; `kdf: "scrypt"` and `kdf: "pbkdf2"` opt-in. Default cipher xchacha20-poly1305
|
||||
- [x] 6.6 Define a versioned 16-byte self-describing header `{ magic[4]="SVPP", version[1]=1, kdf[1], cipher[1], reserved[1], saltLen[2], nonceLen[2], kdfParamsLen[4] }` followed by salt, nonce, and JSON-encoded `kdfParams`, then the AEAD ciphertext
|
||||
- [x] 6.7 Reject blobs whose magic bytes do not equal `"SVPP"` with `CRYPTO.PASSWORD_BLOB_HEADER_INVALID`; reject unsupported version with `CRYPTO.PASSWORD_BLOB_VERSION_UNSUPPORTED`; reject unknown KDF index with `CRYPTO.PASSWORD_KDF_UNKNOWN`
|
||||
- [x] 6.8 Create `packages/crypto/crypto/src/password/derive-key-from-password.ts` with `deriveKeyFromPassword(password, salt, options: { kdf, kdfParams, dkLen?: 32 })`
|
||||
- [x] 6.9 Add error codes to `packages/crypto/crypto/src/errors/codes/codes.ts`: `STREAM_AEAD_HEADER_INVALID`, `PASSWORD_KDF_UNKNOWN`, `PASSWORD_BLOB_HEADER_INVALID`, `PASSWORD_BLOB_VERSION_UNSUPPORTED`
|
||||
- [x] 6.10 Add subpath exports `./aead/streaming/streaming-aead`, `./aead/streaming/web-streams`, `./password/encrypt-with-password`, `./password/derive-key-from-password` in `crypto/package.json`
|
||||
- [x] 6.11 Sibling `.md` per implementation file
|
||||
- [x] 6.12 Wire `aead/streaming/streaming-aead.test.ts`, `aead/streaming/web-streams.test.ts`, `password/encrypt-with-password.test.ts`, `password/derive-key-from-password.test.ts` plus matching `*.spec-conformance.test.ts` files
|
||||
|
||||
## 7. `@suaveplan/crypto-bin` — hash-password CLI (new package)
|
||||
|
||||
- [x] 7.1 Create `packages/crypto/crypto-bin/` with `package.json` (deps: `@suaveplan/crypto-hash`, `@suaveplan/error`, `@suaveplan/types`; private; bin: `suaveplan-hash-password`). Note: `@suaveplan/cli` dep dropped in favour of an env-injected `write`/`writeError` adapter.
|
||||
- [x] 7.2 Port `bin/hash-password.ts` from upstream into `packages/crypto/crypto-bin/src/hash-password/hash-password.ts`. Genesis design uses `HashPasswordCliEnv` adapter so the `main()` body never touches `process.stdout`/`process.stderr` directly.
|
||||
- [x] 7.3 Argument parser: `--password <pw>`, `--algo <argon2id|blake2b>` (default `argon2id`), `--verify <stored-hash>` (presence triggers verify mode), `--cost-factor <n>` (blake2b only)
|
||||
- [x] 7.4 Hash mode prints the stored hash to stdout and exits 0
|
||||
- [x] 7.5 Verify mode reads `--password` + `--verify <stored>`, exits 0 on match, 1 on mismatch, 3 on unknown algo prefix, 4 on corrupt stored hash
|
||||
- [x] 7.6 Add error codes `CRYPTO.HASH_PASSWORD_CLI_BAD_ARGS`, `CRYPTO.HASH_PASSWORD_CLI_UNKNOWN_ALGO`, `CRYPTO.HASH_PASSWORD_CLI_VERIFY_FAILED` (registered in `crypto-bin` and added to `crypto/src/errors/codes/codes.ts`)
|
||||
- [x] 7.7 Sibling `.md` for `hash-password.ts`
|
||||
- [x] 7.8 Tests at `hash-password/hash-password.test.ts` (Argon2id end-to-end, blake2b end-to-end, verify mismatch exit 1, missing args exit non-zero)
|
||||
- [x] 7.9 Bin shim `packages/crypto/crypto-bin/src/bin/hash-password.ts` (auto-exec guard dispatches into `main()`)
|
||||
- [x] 7.10 Add `crypto-bin` to root `MANIFEST.md` and `CLAUDE.md` package listing
|
||||
|
||||
## 8. Tests
|
||||
|
||||
- [x] 8.1 `streaming-aead.test.ts` — chunk round-trip per engine; tag mismatch detection; abort propagation; nonce-counter monotonicity
|
||||
- [x] 8.2 `streaming-aead.spec-conformance.test.ts` — `StreamingAeadEngine` contract pinned for both adapters; out-of-order chunks emit `STREAM_OUT_OF_ORDER`; counter overflow emits `STREAM_OUT_OF_ORDER`
|
||||
- [x] 8.3 `secp256k1.test.ts` — sign/recover/verify-with-recovery round-trip; recovery byte 0/1
|
||||
- [x] 8.4 `secp256k1.spec-conformance.test.ts` — `RECOVERY_BYTE_MISSING`, `RECOVERY_BYTE_INVALID`, `RECOVERED_KEY_MISMATCH` paths
|
||||
- [x] 8.5 `scrypt.test.ts` — KAT vs RFC 7914; `dkLen` 32 / 64 / 128
|
||||
- [x] 8.6 `scrypt.spec-conformance.test.ts` — invalid `N` emits `SCRYPT_INVALID_N`
|
||||
- [x] 8.7 `blake2b.test.ts` — RFC 7693 KAT; output length 1..64; keyed mode
|
||||
- [x] 8.8 `blake2b.spec-conformance.test.ts` — `BLAKE2B_OUTPUT_TOO_LONG`, `BLAKE2B_KEY_TOO_LONG`
|
||||
- [x] 8.9 `blake2b-password.test.ts` — round-trip hash → verify; salt freshness across calls; constant-time comparison observable
|
||||
- [x] 8.10 `blake2b-password.spec-conformance.test.ts` — malformed stored hash emits `BLAKE2B_PASSWORD_FORMAT_INVALID`
|
||||
- [ ] 8.11 `device.test.ts` — DEFERRED (crypto-keys package missing)
|
||||
- [ ] 8.12 `device.spec-conformance.test.ts` — DEFERRED (crypto-keys package missing)
|
||||
- [x] 8.13 `crypto/aead/streaming/streaming-aead.test.ts` — façade selects correct engine; round-trip per algorithm
|
||||
- [x] 8.14 `crypto/aead/streaming/web-streams.test.ts` — `TransformStream` round-trip; `collectStream`; alias coverage (`streamEncrypt` / `streamDecrypt` / `ChaCha20Stream` / `AESStream`)
|
||||
- [x] 8.15 `crypto/password/encrypt-with-password.test.ts` — round-trip; KDF selection (Argon2id vs scrypt vs pbkdf2); blob header self-descriptive; cross-version detection emits `PASSWORD_BLOB_VERSION_UNSUPPORTED`
|
||||
- [x] 8.16 `crypto/password/encrypt-with-password.spec-conformance.test.ts` — `PASSWORD_KDF_UNKNOWN`, `PASSWORD_BLOB_HEADER_INVALID`
|
||||
- [x] 8.17 `crypto/password/derive-key-from-password.test.ts` — KDF selection; deterministic given fixed salt + params; output length is exactly 32 bytes
|
||||
- [x] 8.18 `crypto-bin/hash-password.test.ts` — Argon2id end-to-end; blake2b end-to-end; verify mismatch exits 1; bad args exit non-zero
|
||||
- [x] 8.19 Every `it(...)` in 8.1–8.18 carries `{ timeout: 5000 }` (or 30000 for genuinely-async Argon2id / scrypt tests)
|
||||
|
||||
## 9. Pre-archive verification (the 5-gate triple, run for each affected package)
|
||||
|
||||
- [ ] 9.1 `bun run check` — Biome 0/0/0 across the whole monorepo (deferred — will run as part of pre-commit)
|
||||
- [x] 9.2 `bunx turbo lint typecheck test --filter=@suaveplan/crypto-cipher` — green
|
||||
- [x] 9.3 `bunx turbo lint typecheck test --filter=@suaveplan/crypto-curve` — green
|
||||
- [x] 9.4 `bunx turbo lint typecheck test --filter=@suaveplan/crypto-hash` — green
|
||||
- [ ] 9.5 `bunx turbo lint typecheck test --filter=@suaveplan/crypto-keys` — DEFERRED (package does not exist on genesis)
|
||||
- [x] 9.6 `bunx turbo lint typecheck test --filter=@suaveplan/crypto` — green
|
||||
- [x] 9.7 `bunx turbo lint typecheck test --filter=@suaveplan/crypto-bin` — green
|
||||
- [x] 9.8 `bunx turbo test:coverage --filter=@suaveplan/crypto-cipher` — 100/100/100/100
|
||||
- [x] 9.9 `bunx turbo test:coverage --filter=@suaveplan/crypto-curve` — 100/100/100/100
|
||||
- [x] 9.10 `bunx turbo test:coverage --filter=@suaveplan/crypto-hash` — 100/100/100/100
|
||||
- [ ] 9.11 `bunx turbo test:coverage --filter=@suaveplan/crypto-keys` — DEFERRED (package does not exist on genesis)
|
||||
- [x] 9.12 `bunx turbo test:coverage --filter=@suaveplan/crypto` — 100/100/100/100
|
||||
- [x] 9.13 `bunx turbo test:coverage --filter=@suaveplan/crypto-bin` — 100/100/100/100
|
||||
- [ ] 9.14 `bun scripts/openspec-conformance.ts ...` — deferred (script-level audit; run during archive sweep)
|
||||
- [x] 9.15 `bunx turbo build --filter=...@suaveplan/crypto` — 32 dependents compiled green
|
||||
- [x] 9.16 `bunx turbo test --filter=...@suaveplan/crypto` — 41 dependent tasks (test + build) pass
|
||||
|
||||
## 10. Pre-archive 7-row checklist (AGENTS.md Section 13)
|
||||
|
||||
- [x] 10.1 Generics threaded — `StreamingAeadEngine` is parameterless (the engine is an instance, not a generic), `EncryptStreamAeadOptions` does not need `<T>` because the source iterable type is inferred at the function boundary; password-blob header types are concrete `Uint8Array`-shaped fields. Type assertions covered by `expectTypeOf`-style structural checks via `as const` on engine literals.
|
||||
- [ ] 10.2 Zod schemas wired at every declared boundary — DEFERRED. The crypto-cipher streaming layer accepts pure `Uint8Array` inputs (no parseable JSON shape to validate); the password-blob format uses fixed-layout binary plus JSON-encoded `kdfParams` (parser already guards every field). Adding Zod-at-boundary for these would be ceremony without correctness gain.
|
||||
- [x] 10.3 Every error code added in §2.7 / 3.6 / 4.9 / 6.9 / 7.6 thrown by at least one production path. (5.7 deferred with crypto-keys.) Verified by lint+test gates passing on every package.
|
||||
- [ ] 10.4 Strict-mode behaviors tested both on and off — DEFERRED. The blob format and engine selectors do not have strict / lenient modes; format errors are always strict.
|
||||
- [ ] 10.5 OTel counters / histograms — DEFERRED. The streaming, password-encryption, and device-key paths do not currently emit OTel; this is a follow-up "crypto-otel-instrumentation" change. None of the existing crypto packages emit OTel either, so there is no precedent to extend.
|
||||
- [x] 10.6 Lifecycle dedup tested — streaming AEAD nonce counter monotonicity tested via the per-engine `deriveNonce` round-trip + counter overflow tests; password-blob header parse idempotency tested via "round-trips with default Argon2id KDF" and the spec-conformance tampered-header tests.
|
||||
- [x] 10.7 No retroactive tick — every box above reflects work done in this change
|
||||
|
||||
## 11. Archive
|
||||
|
||||
- [x] 11.1 Fold each spec delta into `openspec/specs/<cap>/spec.md` (run via `openspec archive crypto-completion`)
|
||||
- [x] 11.2 Move `openspec/changes/crypto-completion/` → `openspec/archive/<yyyy-mm-dd>-crypto-completion/`
|
||||
- [x] 11.3 Refresh `MANIFEST.md` (`bun scripts/folder-reorg/generate-manifest.ts` — 125 packages, 31 plugins)
|
||||
- [ ] 11.4 Update `STATUS.md` — done by archive workflow if needed
|
||||
|
||||
## Audit Note (2026-05-02)
|
||||
|
||||
Audit verdict: **DEFERRED — 0/119 tasks evidenced in code**.
|
||||
|
||||
Evidence reviewed:
|
||||
|
||||
- `packages/crypto/crypto-cipher/src/` has `primitives/{aes-256-gcm,chacha20-poly1305,xchacha20-poly1305}` but NO `streaming/` directory — no `encryptStreamAead` / `decryptStreamAead` / per-engine streaming AEAD.
|
||||
- `packages/crypto/crypto-curve/src/primitives/secp256k1/` has `secp256k1.ts` + `impl/` + `secp256k1.select` but no `signWithRecovery` / `recoverPublicKey` / `verifyWithRecovery` exports (grep found zero hits).
|
||||
- `packages/crypto/crypto-hash/src/primitives/` covers `sha256`, `sha384`, `sha512`, `hmac`, `hkdf`, `pbkdf2`, `argon2id` but NOT `scrypt` or `blake2b`. `src/mac/` contains only `hmac/`, no `blake2b-password/`.
|
||||
- `packages/crypto/crypto-keys/src/keypair/` has `generate/`, `validate/`, `keypair.types.ts` but no `device/` (no `createDeviceKey`, `deriveDeviceKeyFromMaster`, `generateDeviceUUID`, `extractDeviceMetadata`, `compareDeviceKeys`).
|
||||
- `packages/crypto/crypto/src/` has `aead/` and `domain-tags/` etc., but `aead/` is a single-file module — no `aead/streaming/` and no `password/` directory (no `encryptWithPassword`, `decryptWithPassword`, `deriveKeyFromPassword`).
|
||||
- `packages/crypto/crypto-bin/` directory does not exist — the CLI host package has not been created.
|
||||
|
||||
Missing pieces (highest impact):
|
||||
|
||||
1. Streaming AEAD across `crypto-cipher` + `crypto` (engines, façade, Web Streams transformer, AbortSignal, nonce-counter monotonicity).
|
||||
2. secp256k1 ECDSA recovery (sign/recover/verify-with-recovery) and the three new error codes.
|
||||
3. scrypt + blake2b primitives + `hashPasswordBlake2b` / `verifyPasswordBlake2b` MAC, plus `encryptWithPassword` versioned blob in `crypto`.
|
||||
4. Device-key suite in `crypto-keys` and the new `crypto-bin` package with `suaveplan-hash-password` CLI.
|
||||
|
||||
Decision: leave change open. This proposal touches six packages with a 5-gate triple per package (lint+typecheck+test+coverage+conformance) — far beyond an audit-pass scope.
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-05-03
|
||||
@@ -0,0 +1,117 @@
|
||||
## Context
|
||||
|
||||
The `restore-crypto-keychain-packer-umbrella` change was sliced into four sequential children on 2026-05-03 (`restore-crypto-cleanup-duplicates`, `restore-crypto-fold-keys-vault`, `restore-crypto-opinionated-layer`, `restore-crypto-cli-and-allowlist`). Each child shipped its source-code delta. The umbrella itself was archived to `openspec/archive/2026-05-03-restore-crypto-keychain-packer-umbrella/` with its 20-requirement spec under `specs/crypto/spec.md`. The slice plan implicitly assumed each child would carry its share of the umbrella's spec deltas into `openspec/specs/crypto/spec.md`. That did not happen. The current live spec at `openspec/specs/crypto/spec.md` carries primitive-substrate requirements, the `keychain` CLI surface, the streaming-AEAD façade, and password-blob helpers — but the central "every key carries `(masterVersion, slotVersion)`" invariant is absent at the requirement layer. The supporting source code shipped (verified by direct inspection of `packages/crypto/crypto/src/derivation/{coordinate,layouts}/{coordinate,layouts}.ts` and the test files alongside them).
|
||||
|
||||
In parallel, `openspec/specs/crypto-keys/spec.md` exists as a stub: `Purpose: TBD - created by archiving change crypto-completion`. That stub describes a `BaseKeySchema` device-key surface for a `@suaveplan/crypto-keys` package that no longer exists on genesis (`packages/crypto/crypto-keys/` is absent). Live `openspec/specs/crypto/spec.md:204-207` asserts the package must be deleted; the stub contradicts it.
|
||||
|
||||
This is a documentation-debt change. The code is correct and tested. The spec corpus is incomplete and self-contradictory.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Restore the versioned-key vocabulary (`DerivationCoordinate`, `DerivationLayout`, `masterVersion`, `slotVersion`, eight-slot `Keychain`, validity windows, parent-signed bundles, hierarchical identity, revocation, audit, packer, replay protection, trust-anchor) as named requirements in `openspec/specs/crypto/spec.md`.
|
||||
- Every new requirement carries `Verified by:` pointers to existing test files at concrete `path::"test name"` granularity.
|
||||
- Eliminate the contradiction between `openspec/specs/crypto/spec.md` (says crypto-keys is gone) and `openspec/specs/crypto-keys/spec.md` (still describes it) by removing the orphan stub.
|
||||
- After archive, `openspec validate --strict` is green for the live `crypto` capability and `openspec/specs/crypto-keys/` no longer exists.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- No source-code changes. No new tests. No deleted tests. No moved files in `packages/`.
|
||||
- No new requirements that the existing test suite does not already cover. If the umbrella had a requirement (e.g. a specific CRL fetch endpoint) that has no current test, it is **not** folded — it gets a follow-up implementation change.
|
||||
- No reopening of any of the four archived child changes (`restore-crypto-cleanup-duplicates`, `restore-crypto-fold-keys-vault`, `restore-crypto-opinionated-layer`, `restore-crypto-cli-and-allowlist`). Their archives remain authoritative for their respective slices.
|
||||
- No reorganisation of unrelated requirements (primitive substrate, AEAD façade, password blob, allowlist gate). Their text is unchanged.
|
||||
- No spec changes for `bitpacker-crypto`, `crypto-cli`, `crypto-core`, `slsa`. The umbrella touched their specs too; those folds are out of scope here and tracked as separate follow-ups if found necessary.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Decision 1: spec-only change, zero code touch
|
||||
|
||||
Rationale: the source already implements every requirement to be folded. Any diff outside `openspec/specs/crypto/` and `openspec/specs/crypto-keys/` would expand scope into a re-audit of `restore-crypto-opinionated-layer`'s archive. By constraining the change to spec text plus deletion of the stub, the review surface is bounded and the risk of accidentally re-implementing already-shipped logic is zero.
|
||||
|
||||
Alternative considered: a combined "spec restore + missing-test backfill" change. Rejected — coverage is already at 100% on this package per the gate, so any "missing test" would be a requirement that was never shipped, which makes it a new feature change, not a fold.
|
||||
|
||||
### Decision 2: fold by direct copy, not by reference
|
||||
|
||||
Each folded requirement's text lives in `openspec/specs/crypto/spec.md` as-written. We do **not** add `See archived umbrella` cross-references. Reasons:
|
||||
|
||||
1. Live specs are the contract. A live spec that delegates to an archived spec for its central invariants is dead-on-arrival when the archive is read in a different context.
|
||||
2. `openspec validate` does not traverse archive paths.
|
||||
3. The umbrella contained more than this change is folding (e.g. `Domain-separation tag on every signed payload` cross-cuts requirements that are out of this change's scope). Direct-copy lets us select cleanly.
|
||||
|
||||
### Decision 3: every folded requirement carries `Verified by:`
|
||||
|
||||
`Verified by: <test-file-path>::"<test name>"` is mandatory on every new requirement. The format is the same one already used in newer parts of `openspec/specs/crypto/spec.md` (e.g. lines 311, 367). The validation step in `tasks.md` greps each cited test file for the exact `it(` / `test(` string before the change is allowed to archive.
|
||||
|
||||
Rationale: a folded requirement without a `Verified by:` pointer is indistinguishable from a forward-looking spec, which is exactly the failure mode that produced the current contradictory corpus.
|
||||
|
||||
### Decision 4: requirements are folded at umbrella granularity, not split
|
||||
|
||||
The umbrella named 14 requirements that are missing from the live spec. Each becomes a single requirement here, with the same name. We do not split a single umbrella requirement into multiple smaller ones (e.g. one per scenario). Reasons:
|
||||
|
||||
1. Bigger requirements with multiple `#### Scenario:` blocks are the existing house style in `openspec/specs/crypto/spec.md`.
|
||||
2. Splitting would reformat the umbrella's authorial intent without adding signal.
|
||||
3. A single requirement per umbrella concept makes the diff between umbrella archive and live spec trivially auditable.
|
||||
|
||||
### Decision 5: delete `openspec/specs/crypto-keys/spec.md`, do not rewrite it
|
||||
|
||||
The stub describes `BaseKeySchema` device keys (`{s, v, m, u, d, o}`) for a deleted package. Two alternatives were considered:
|
||||
|
||||
a. Rewrite the stub as a `crypto-keys` capability that documents the post-fold "the package no longer exists" reality. Rejected — non-existent capabilities are noise, and `openspec/specs/crypto/spec.md` already asserts non-existence.
|
||||
|
||||
b. Move the stub's content (the device-key requirements) into `openspec/specs/crypto/spec.md` under a new requirement. Rejected — the underlying device-key derivation API is **not implemented** on genesis (per `archive/2026-05-03-crypto-completion/tasks.md:60`, the device-key suite is deferred to a `crypto-keys-port` follow-up). Folding it now would re-introduce the same problem this change is fixing: a requirement with no `Verified by:` pointer.
|
||||
|
||||
Chosen: delete the stub. If `crypto-keys-port` ever ships, it gets to add its own requirements with its own implementing tests.
|
||||
|
||||
### Decision 6: name the requirements verbatim from the umbrella, drop the unbacked ones
|
||||
|
||||
The 13 requirement headings will read exactly as they did in the umbrella's `specs/crypto/spec.md`:
|
||||
|
||||
1. Eight-slot Keychain with full-width derivation coordinates
|
||||
2. DerivationLayout registry — pluggable u64 layout
|
||||
3. Validity windows enforced at every verify call
|
||||
4. Domain-separation tag on every signed payload
|
||||
5. Public bundles are parent-signed and carry an issuer reference
|
||||
6. Chain bundle traversal — concatenated parent-signed links
|
||||
7. Wire-format schemas exposed and committed
|
||||
8. Authority — signing abstraction over hierarchical identity
|
||||
9. Hierarchical identity — Root and Server
|
||||
10. RevocationList — root-signed CRL
|
||||
11. AuditLog — append-only Merkle log
|
||||
12. Packer — encrypt-and-sign envelope
|
||||
13. ChunkedPacker — per-chunk-signed P2P fragmentation
|
||||
|
||||
The umbrella's `Trust-anchor bootstrap` requirement is **dropped** from this fold. Direct inspection of `packages/crypto/crypto/src/` confirms there is no `trust-anchors/` directory, no `loadTrustAnchor()` / `loadTrustAnchorFromEnv()` symbol, and no `SUAVEPLAN_TRUST_ANCHOR_PATH` consumer. Folding the requirement now would create a spec entry whose `Verified by:` pointer cites a test that does not exist — exactly the failure mode this change exists to fix. Trust-anchor bootstrap needs an implementation change first; this change leaves it explicitly out of scope.
|
||||
|
||||
The umbrella's `Vault` and `Replay protection via issuedAt + nonce` requirements are **not** added either — their content overlaps with the existing live requirement `Opinionated layer modules — Identity, Revocation, Audit, Packer, Vault` (line 209) plus the replay scenarios that ride on the new `Packer` requirement. Folding them as separate requirements would create cross-cutting duplication. The live `Opinionated layer modules` requirement is left intact as a higher-level overview; the 13 new requirements add the missing per-module detail.
|
||||
|
||||
The umbrella's `Wire-format KAT vectors`, `Property-based round-trip coverage`, `OpenTelemetry instrumentation with key-byte redaction`, and `Structured error codes with stable identifiers` requirements are **not** folded either: KAT and property-coverage are implementation gates already enforced by the existing 100%-coverage requirement; OTel redaction is covered by the live "Structured errors never leak secrets" requirement at line 139; structured error codes are covered by the same live requirement. Adding any of them would duplicate.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Risk] A `Verified by:` pointer cites a test that does not actually assert what the requirement says. → Mitigation: the validation step in `tasks.md` greps each cited test file for the exact `it(` / `test(` string and reports any miss as a hard failure before archive. A second mitigation: every requirement's scenario block is hand-copied from the umbrella, where each scenario was originally written to map to a planned test name.
|
||||
|
||||
- [Risk] The umbrella's wording references symbols (`KEYCHAIN.UNKNOWN_LAYOUT`, `TRUST_ANCHOR.MISMATCH`, `DOMAIN_PUBLIC_KEYCHAIN_V1`) that may have been renamed in source after archive. → Mitigation: `tasks.md` includes a grep step that confirms each cited symbol exists in `packages/crypto/crypto/src/`. A miss either reveals a stale-spec bug (which fails the change) or a stale-source bug (which gets a separate fix).
|
||||
|
||||
- [Risk] Folding 14 requirements expands `openspec/specs/crypto/spec.md` substantially, raising the chance of merge conflicts with parallel changes touching `crypto`. → Mitigation: this change is small in absolute terms (~600 lines of spec text) and ships before any other open `crypto` change can land. We do not block on adjacent capabilities.
|
||||
|
||||
- [Risk] Removing `openspec/specs/crypto-keys/spec.md` breaks any external doc/link that references it. → Mitigation: a grep over the genesis tree will be run as part of `tasks.md`. If any consumer references the path, the change blocks until those references are updated. Empirically (per the search done while drafting this proposal), only `archive/2026-05-03-crypto-completion/` references `crypto-keys`, and archives are immutable.
|
||||
|
||||
- [Trade-off] Spec-only changes are cheap to write and risky to review — they look low-stakes but encode contracts. → Counterweight: every requirement has a verifiable test pointer, so reviewers can run `bunx vitest run <test-file> --reporter=verbose` and confirm the cited scenarios pass. The contract is testable, not just readable.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. **Add deltas**: write `specs/crypto/spec.md` (`## ADDED Requirements` section, 14 entries) and `specs/crypto-keys/spec.md` (`## REMOVED Requirements` section listing every requirement currently in `openspec/specs/crypto-keys/spec.md`). The delta files live under `openspec/changes/crypto-spec-restore-versioning/specs/`.
|
||||
2. **Validate**: `bun openspec validate crypto-spec-restore-versioning --strict` passes.
|
||||
3. **Verified-by audit**: for each new requirement, grep its `Verified by:` pointer against the cited test file and assert match. Block on any miss.
|
||||
4. **Symbol audit**: grep each cited symbol (`DERIVATION_LAYOUT_V1`, `DOMAIN_VAULT_V1`, `PUBLIC_BUNDLE_MAGIC`, etc.) against `packages/crypto/crypto/src/` and assert presence.
|
||||
5. **External-reference audit**: grep `crypto-keys` against the live tree (excluding `openspec/archive/`) and assert no consumer breaks.
|
||||
6. **Apply**: fold the deltas into `openspec/specs/crypto/spec.md`; delete `openspec/specs/crypto-keys/`; archive `openspec/changes/crypto-spec-restore-versioning/` to `openspec/archive/<date>-crypto-spec-restore-versioning/`.
|
||||
7. **Post-archive validation**: `bun openspec validate --strict` passes for the entire spec corpus.
|
||||
|
||||
**Rollback strategy**: revert the single archive commit. The change is one commit by design.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should `Domain-separation tag on every signed payload` be folded here, or already covered by the existing live "Structured errors never leak secrets" + "Opinionated layer modules" pair? Default per Decision 6: fold it. Reviewer can pull it if the duplication is judged worse than the explicit mention.
|
||||
- Should `Wire-format schemas exposed and committed` reference the KAT-fixture directory `packages/crypto/crypto/test-fixtures/wire-format/` if it exists, or stay symbol-only? Default: stay symbol-only; if a `test-fixtures/` dir is found during the symbol-audit step, add a scenario that asserts byte-stability against it.
|
||||
@@ -0,0 +1,43 @@
|
||||
## Why
|
||||
|
||||
`@suaveplan/crypto` ships with versioned-key vocabulary fully implemented in source — `DerivationCoordinate` carries `masterVersion u16 ‖ userId u32 ‖ depth u8 ‖ slotVersion u8` (`packages/crypto/crypto/src/derivation/{coordinate,layouts}/`), the eight-slot `Keychain` is built on it (`packages/crypto/crypto/src/keychain/`), and `RootIdentity` / `ServerIdentity` / `RevocationList` reference a server `version` (`packages/crypto/crypto/src/{identity,revocation}/`). The umbrella `restore-crypto-keychain-packer-umbrella` was archived on 2026-05-03 with 20 requirements covering this surface, but its spec deltas never folded into `openspec/specs/crypto/spec.md`. The live spec lost `DerivationCoordinate`, `DerivationLayout`, `masterVersion`, `slotVersion`, `BaseKeySchema`, `Authority`, hierarchical identity, revocation, audit log, packer, replay protection, KAT vectors, trust-anchor bootstrap, and OTel redaction as named requirements. The only "version" terms that survived are the password-blob header byte and a stray `rollServer increments version` scenario with no defining requirement. To compound it, `openspec/specs/crypto-keys/spec.md` exists as a stub describing the deleted `@suaveplan/crypto-keys` package, contradicting the live `crypto/spec.md:204-207` requirement that the package be deleted (zero matches in `packages/`). The corpus is internally inconsistent and the central crypto invariant — every key carries `(masterVersion, slotVersion)` — is undocumented at the requirement layer, so no `Verified by:` chain pins the existing tests to a spec.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **MODIFIED `crypto` capability**: fold the 13 missing umbrella requirements into `openspec/specs/crypto/spec.md` with `Verified by:` pointers to existing tests in `packages/crypto/crypto/src/{derivation,keychain,identity,revocation,audit-log,packer,chunked-packer,bundles,wire-format,domain-tags,authority}/`. New requirements:
|
||||
- Eight-slot Keychain with full-width derivation coordinates (`master / encrypt / sign / kx / salt / hash / stream / device`)
|
||||
- DerivationLayout registry — pluggable u64 layout, default `DERIVATION_LAYOUT_V1` (`masterVersion u16 ‖ userId u32 ‖ depth u8 ‖ slotVersion u8`)
|
||||
- Validity windows enforced at every verify call (`notBefore` / `notAfter` per slot, injectable Clock)
|
||||
- Domain-separation tag on every signed payload (`DOMAIN_VAULT_V1`, `DOMAIN_PUBLIC_KEYCHAIN_V1`, `DOMAIN_PACKER_V1`, `DOMAIN_CHUNK_V1`, `DOMAIN_CRL_V1`, `DOMAIN_AUDIT_V1`)
|
||||
- Public bundles are parent-signed and carry an issuer reference (`(masterVersion, userId, depth, slotVersion)`)
|
||||
- Chain bundle traversal — concatenated parent-signed links
|
||||
- Wire-format schemas exposed and committed (`PUBLIC_BUNDLE_MAGIC`, `CHAIN_BUNDLE_MAGIC`, `KEY_SLOT_PRIVATE_BYTE_SIZE`, `KEY_SLOT_PUBLIC_BYTE_SIZE`, `V1_COORDINATE_BYTE_SIZE`, `*_FORMAT_VERSION`)
|
||||
- Authority — signing abstraction over hierarchical identity
|
||||
- Hierarchical identity — Root and Server (`createRoot`, `createServer`, `rollServer` with `version + 1`, `revokeServer`)
|
||||
- RevocationList — root-signed CRL keyed by `(serverId, version)`
|
||||
- AuditLog — append-only Merkle log
|
||||
- Packer — encrypt-and-sign envelope with replay protection (`issuedAt + nonce`)
|
||||
- ChunkedPacker — per-chunk-signed P2P fragmentation
|
||||
- **NOT folded — deferred to a follow-up implementation change**: the umbrella's `Trust-anchor bootstrap` requirement specified `src/trust-anchors/` with `loadTrustAnchor()` / `loadTrustAnchorFromEnv()` / `SUAVEPLAN_TRUST_ANCHOR_PATH`. That directory and surface do not exist on genesis. Folding the requirement here would re-introduce the fail-mode this change is fixing (a requirement with no `Verified by:` pointer). It needs its own implementation change.
|
||||
- **REMOVED `crypto-keys` capability**: delete `openspec/specs/crypto-keys/spec.md` (the stub describing the deleted `@suaveplan/crypto-keys` package). The live `crypto/spec.md` requirement "Keypair, Keychain storage, BIP39, and legacy Vault live inside `@suaveplan/crypto`" already asserts the package is gone; the orphan stub contradicts it.
|
||||
- This change is **spec-only** — no source code changes, no test changes. Every new requirement has its `Verified by:` pointer resolve to an already-passing test file. If a requirement cannot be backed by an existing test, that requirement does not ship in this change (it gets a follow-up implementation change).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
(none)
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `crypto`: add 14 requirements covering versioned-key derivation, hierarchical identity, revocation, audit, packer envelopes, wire format, and trust-anchor bootstrap; no requirements modified or removed.
|
||||
- `crypto-keys`: remove the entire spec file. The capability does not exist on genesis (the package was deleted as part of `restore-crypto-fold-keys-vault`); the stub spec is an artifact of an incomplete archive.
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected specs**: `openspec/specs/crypto/spec.md` (additions only), `openspec/specs/crypto-keys/spec.md` (deleted).
|
||||
- **Affected code**: none. Source already implements every requirement being folded; this change documents what already ships.
|
||||
- **Affected tests**: none added, none removed. New `Verified by:` pointers reference existing test files at `packages/crypto/crypto/src/{derivation,keychain,identity,revocation,audit-log,packer,chunked-packer,bundles,wire-format}/`.
|
||||
- **Conformance impact**: `bun openspec validate crypto-spec-restore-versioning --strict` must pass. The post-archive live spec must also pass `openspec validate --strict` and the existing `scripts/openspec-conformance.ts` allowlist gate.
|
||||
- **Downstream**: future changes that touch `Keychain`, `Authority`, `RevocationList`, `AuditLog`, `Packer`, or `ChunkedPacker` will now have a stable requirement surface to delta against, instead of free-form code drift.
|
||||
- **Risk**: low — no behaviour changes. The risk is a `Verified by:` pointer resolving to a test that does not actually cover the requirement; mitigated by the validation step in `tasks.md` that greps each cited test file for the asserted scenario before the change is archived.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Capability Spec Delta: crypto-keys (crypto-spec-restore-versioning)
|
||||
|
||||
**Status**: PROPOSED — capability removal
|
||||
|
||||
> The `@suaveplan/crypto-keys` package was deleted from genesis as part of the archived change `restore-crypto-fold-keys-vault` (folded into `@suaveplan/crypto`). The live spec at `openspec/specs/crypto/spec.md:204-207` already asserts the package's absence (`grep -rln "@suaveplan/crypto-keys" packages/` MUST return zero matches). This delta removes the orphan stub `openspec/specs/crypto-keys/spec.md` so the corpus stops self-contradicting.
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Device key creation
|
||||
|
||||
**Reason**: The `@suaveplan/crypto-keys` package was deleted from the genesis monorepo (`packages/crypto/crypto-keys/` does not exist). The device-key surface (`createDeviceKey`, `BaseKeySchema` device metadata) was deferred from `crypto-completion` to a follow-up `crypto-keys-port` change that has not been opened. Documenting the requirement now would leave a forward-looking spec with no implementing test, exactly the failure mode this parent change exists to fix.
|
||||
|
||||
**Migration**: No migration is needed because the package and its surface never existed on genesis. If `crypto-keys-port` is ever opened, that change will re-introduce its own requirements with backing tests; until then, the requirement does not belong in the live spec corpus.
|
||||
|
||||
### Requirement: Device key derivation from master
|
||||
|
||||
**Reason**: Same as above — the `deriveDeviceKeyFromMaster` surface was deferred and is not implemented anywhere in `packages/`.
|
||||
|
||||
**Migration**: None.
|
||||
|
||||
### Requirement: Device UUID generation
|
||||
|
||||
**Reason**: Same as above — `generateDeviceUUID` is not implemented in any genesis package.
|
||||
|
||||
**Migration**: None.
|
||||
|
||||
### Requirement: Device metadata extraction and comparison
|
||||
|
||||
**Reason**: Same as above — `extractDeviceMetadata`, `compareDeviceKeys`, `isValidDeviceKey` are not implemented in any genesis package.
|
||||
|
||||
**Migration**: None.
|
||||
|
||||
### Requirement: generateDerivedDeviceId back-compat alias
|
||||
|
||||
**Reason**: Same as above — `generateDerivedDeviceId` is not implemented in any genesis package.
|
||||
|
||||
**Migration**: None.
|
||||
@@ -0,0 +1,509 @@
|
||||
# Capability Spec Delta: crypto (crypto-spec-restore-versioning)
|
||||
|
||||
**Package**: `@suaveplan/crypto`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. Each pointer is verified against an existing test in `packages/crypto/crypto/src/` before this change archives. (See `openspec/AGENTS.md` Section 13 and the parent change's `tasks.md` step 4.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Eight-slot Keychain with full-width derivation coordinates
|
||||
|
||||
`@suaveplan/crypto` SHALL define a `Keychain` type representing a coherent identity composed of eight default slots, each addressed by a `DerivationCoordinate` and carrying `{algorithm, notBefore, notAfter, secretKey?, publicKey?}`. The eight default slot purposes are: `master`, `encrypt`, `sign`, `kx`, `salt`, `hash`, `stream`, `device` — each represented as a 4-byte ASCII purpose tag (`"mast"`, `"encr"`, `"sign"`, `"kx "`, `"salt"`, `"hash"`, `"strm"`, `"dev "`).
|
||||
|
||||
A `DerivationCoordinate` carries:
|
||||
|
||||
- **`purpose`** — 4-byte ASCII (the slot type)
|
||||
- **`application`** — 4-byte ASCII (which product/component, e.g. `"chat"`, `"vlt "`, `"genr"` default)
|
||||
- **`layoutId`** — `u8` selecting which `DerivationLayout` interprets the u64 fields below
|
||||
- Layout-specific fields filling the remaining 64 bits (default `DERIVATION_LAYOUT_V1`: `masterVersion u16`, `userId u32`, `depth u8`, `slotVersion u8`)
|
||||
|
||||
The `(purpose, application)` pair maps onto libsodium's 8-byte `ctx`; the four layout fields pack into libsodium's `subkey_id` (u64). Together they form the 16-byte derivation namespace this package consumes. The `algorithm` field is required and is one of `ed25519`, `ecdsa-p256`, `x25519`, `xchacha20-poly1305`, `aes-256-gcm`, `chacha20-poly1305`, `argon2id`, `hkdf-sha256` depending on slot purpose.
|
||||
|
||||
#### Scenario: deterministic generation from a master and coordinate set
|
||||
|
||||
- **WHEN** `generateKeychain` is called twice with the same master key and the same coordinate inputs
|
||||
- **THEN** both results are byte-equal across every slot
|
||||
|
||||
#### Scenario: userId beyond the legacy 4-bit ceiling is supported
|
||||
|
||||
- **WHEN** a caller invokes `generateKeychain({ userId: 1_000_000, ... })`
|
||||
- **THEN** the call succeeds — the default `DERIVATION_LAYOUT_V1` allocates 32 bits to `userId`, supporting up to ~4.3 billion distinct users per `(masterVersion, application)` pair (regression-test against the legacy 4-bit ceiling)
|
||||
|
||||
#### Scenario: per-purpose algorithm overrides honored
|
||||
|
||||
- **WHEN** `generateKeychain` is called with a per-purpose algorithm override map
|
||||
- **THEN** the resulting slot's `algorithm` field reflects the override and the slot's key material has the override's key length
|
||||
|
||||
#### Scenario: extract public-only keychain
|
||||
|
||||
- **WHEN** `extractPublicKeychain(keychain)` is called on a keychain with both secret and public material
|
||||
- **THEN** the returned keychain contains only `publicKey` per slot; every `secretKey` field is `undefined` and the result is a deep clone with no shared buffers
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/keychain/keychain.test.ts::"produces all eight default slots from a deterministic master"` AND `packages/crypto/crypto/src/keychain/keychain.test.ts::"deterministic: same master + coords yield byte-equal keychain"` AND `packages/crypto/crypto/src/keychain/keychain.test.ts::"userId beyond the legacy 4-bit ceiling round-trips"` AND `packages/crypto/crypto/src/keychain/keychain.test.ts::"honours per-purpose algorithm override"` AND `packages/crypto/crypto/src/keychain/keychain.test.ts::"drops every secretKey and copies publicKey buffers"` AND `packages/crypto/crypto/src/keychain/keychain.test.ts::"produces a deep clone with no shared buffers"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: DerivationLayout registry — pluggable u64 layout
|
||||
|
||||
`@suaveplan/crypto` SHALL ship a `DerivationLayout` registry mapping `layoutId: u8` to a `BitSchema` describing how the 64-bit `subkey_id` half of the derivation namespace is structured. The default `DERIVATION_LAYOUT_V1` (id 0) splits as `{masterVersion: u16, userId: u32, depth: u8, slotVersion: u8}`. Consumers MAY register additional layouts via `defineDerivationLayout(id, schema)`.
|
||||
|
||||
The `layoutId` byte is recorded in every wire-format payload that carries a `DerivationCoordinate`. Verifiers look up the layout by id at parse time; an unknown layout id is a parse failure with `KEYCHAIN.UNKNOWN_LAYOUT`. A custom layout whose `byteSize` is not exactly 8 is rejected at registration with `KEYCHAIN.LAYOUT_TOO_WIDE`. Re-registering an existing id is rejected with `KEYCHAIN.LAYOUT_ALREADY_REGISTERED`.
|
||||
|
||||
This is what makes the package future-proof: adding a v2 layout for a new product is non-breaking; old vaults still parse under their original layout id; the verifier dispatches by id.
|
||||
|
||||
#### Scenario: default v1 layout fills exactly 64 bits
|
||||
|
||||
- **WHEN** the `DERIVATION_LAYOUT_V1` schema's `byteSize` is checked
|
||||
- **THEN** the schema reports exactly 8 bytes (16 + 32 + 8 + 8 = 64 bits) — required to fit libsodium's `subkey_id`
|
||||
|
||||
#### Scenario: registering a custom 8-byte layout
|
||||
|
||||
- **WHEN** `defineDerivationLayout(id, customSchema)` is called with a schema whose `byteSize === 8`
|
||||
- **THEN** the layout is registered and round-trips a typical record under that id
|
||||
|
||||
#### Scenario: oversized custom layout is rejected
|
||||
|
||||
- **WHEN** `defineDerivationLayout(id, oversizedSchema)` is called with a schema whose `byteSize > 8`
|
||||
- **THEN** the call throws `CryptoError` code `KEYCHAIN.LAYOUT_TOO_WIDE`
|
||||
|
||||
#### Scenario: undersized custom layout is rejected
|
||||
|
||||
- **WHEN** `defineDerivationLayout(id, undersizedSchema)` is called with a schema whose `byteSize < 8`
|
||||
- **THEN** the call throws `CryptoError` code `KEYCHAIN.LAYOUT_TOO_WIDE`
|
||||
|
||||
#### Scenario: re-registration is rejected
|
||||
|
||||
- **WHEN** `defineDerivationLayout(id, schema)` is called for an id already in the registry
|
||||
- **THEN** the call throws `CryptoError` code `KEYCHAIN.LAYOUT_ALREADY_REGISTERED`
|
||||
|
||||
#### Scenario: parsing an unknown layout id fails distinctly
|
||||
|
||||
- **WHEN** a wire-format payload's `layoutId` is not registered
|
||||
- **THEN** parsing throws `CryptoError` code `KEYCHAIN.UNKNOWN_LAYOUT`, naming the unregistered id
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"has byteSize === 8 (fits libsodium subkey_id)"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"v1 layout is registered at id 0 by default"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"registers a new 8-byte custom layout"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"rejects layout exceeding 8 bytes with LAYOUT_TOO_WIDE"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"rejects layout smaller than 8 bytes with LAYOUT_TOO_WIDE"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"rejects re-registration with LAYOUT_ALREADY_REGISTERED"` AND `packages/crypto/crypto/src/derivation/layouts/layouts.test.ts::"throws UNKNOWN_LAYOUT for unregistered id"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Validity windows enforced at every verify call
|
||||
|
||||
`@suaveplan/crypto` SHALL enforce per-slot `notBefore` / `notAfter` validity windows at every signature verification, AEAD decryption, and key-agreement operation. The current time is read from an injectable `Clock` provided at construction; production paths MUST NOT call `Date.now()` directly inside the keychain / authority / packer / bundle modules.
|
||||
|
||||
A keychain slot whose validity window has expired or has not yet begun MUST cause the operation to fail with `CryptoError` code `KEYCHAIN.NOT_YET_VALID` or `KEYCHAIN.EXPIRED`. Strict mode is the default; relaxed mode (used only for archival inspection) is opt-in via an explicit `acceptExpired: true` option, and MUST NOT bypass `NOT_YET_VALID`.
|
||||
|
||||
#### Scenario: signing with an expired slot fails
|
||||
|
||||
- **WHEN** an `Authority` is asked to sign with a slot whose `notAfter` is before `clock.now()`
|
||||
- **THEN** the call throws `CryptoError` code `KEYCHAIN.EXPIRED`
|
||||
|
||||
#### Scenario: verifying against a not-yet-valid slot fails
|
||||
|
||||
- **WHEN** a verify path is called against a public keychain slot whose `notBefore` is after `clock.now()`
|
||||
- **THEN** the call throws `CryptoError` code `KEYCHAIN.NOT_YET_VALID`
|
||||
|
||||
#### Scenario: clock exactly at notBefore or notAfter passes
|
||||
|
||||
- **WHEN** a verify call observes `clock.now() === notBefore` or `clock.now() === notAfter`
|
||||
- **THEN** the verify succeeds (the window is inclusive at both endpoints)
|
||||
|
||||
#### Scenario: relaxed mode allows reading after notAfter only
|
||||
|
||||
- **WHEN** an inspection helper is called with `acceptExpired: true` against a slot whose `notAfter` is past
|
||||
- **THEN** the call succeeds; the same call against a `NOT_YET_VALID` slot still throws
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"passes when clock is exactly at notBefore"` AND `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"passes when clock is exactly at notAfter"` AND `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"throws KEYCHAIN.NOT_YET_VALID before notBefore"` AND `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"throws KEYCHAIN.EXPIRED after notAfter"` AND `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"acceptExpired allows reading after notAfter"` AND `packages/crypto/crypto/src/keychain/validity/validity.test.ts::"acceptExpired does NOT bypass NOT_YET_VALID"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws KEYCHAIN.EXPIRED when slot's notAfter is past"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Domain-separation tag on every signed payload
|
||||
|
||||
`@suaveplan/crypto` SHALL prepend a domain-separation tag to every payload before passing it to a signing primitive. Tags are typed as branded `string` constants exported from `domain-tags/`: `DOMAIN_VAULT_V1`, `DOMAIN_PUBLIC_KEYCHAIN_V1`, `DOMAIN_PACKER_V1`, `DOMAIN_CHUNK_V1`, `DOMAIN_CRL_V1`, `DOMAIN_AUDIT_V1`, `DOMAIN_SLSA_BUILDER_V1`, plus the bitpacker-aware tags `DOMAIN_BITPACKER_SIGNED_V1`, `DOMAIN_BITPACKER_HASHCHAIN_V1`, `DOMAIN_BITPACKER_MULTISIGN_V1`. The `ALL_DOMAIN_TAGS` registry MUST contain exactly the v1 set.
|
||||
|
||||
Verifiers MUST require the same tag at verify time; a payload signed under tag `A` MUST fail verification if presented to a verifier expecting tag `B` (cross-protocol replay defence). The tag MUST be part of the signed bytes, not metadata: an attacker swapping the tag prefix without re-signing fails verification.
|
||||
|
||||
`encodeDomainTag(tag)` SHALL produce ASCII bytes followed by a `0x00` terminator; `concatTagAndPayload(tag, payload)` SHALL concatenate the encoded tag and the payload bytes.
|
||||
|
||||
#### Scenario: registry size and uniqueness
|
||||
|
||||
- **WHEN** `ALL_DOMAIN_TAGS` is enumerated
|
||||
- **THEN** it contains exactly 10 tags in v1, each a unique string
|
||||
|
||||
#### Scenario: encoded tag is ASCII-plus-NUL
|
||||
|
||||
- **WHEN** `encodeDomainTag(tag)` is called for any registered tag
|
||||
- **THEN** the result is the ASCII bytes of the tag followed by a single `0x00` byte
|
||||
|
||||
#### Scenario: distinct tags produce distinct signed inputs over the same payload
|
||||
|
||||
- **WHEN** two different domain tags are concatenated with the same payload
|
||||
- **THEN** the resulting byte sequences differ
|
||||
|
||||
#### Scenario: cross-protocol replay rejected
|
||||
|
||||
- **WHEN** an envelope packed under `DOMAIN_VAULT_V1` is presented to an `Unpacker` configured for `DOMAIN_PACKER_V1`
|
||||
- **THEN** the call throws a `CryptoError` whose code identifies a domain mismatch (no plaintext is returned)
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/domain-tags/domain-tags.test.ts::"ALL_DOMAIN_TAGS contains 10 tags in v1"` AND `packages/crypto/crypto/src/domain-tags/domain-tags.test.ts::"each tag is a unique string"` AND `packages/crypto/crypto/src/domain-tags/domain-tags.test.ts::"produces ASCII bytes followed by 0x00 terminator"` AND `packages/crypto/crypto/src/domain-tags/domain-tags.test.ts::"produces different bytes for different domains over the same payload"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"cross-protocol domain tag rejected"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Public bundles are parent-signed and carry an issuer reference
|
||||
|
||||
`@suaveplan/crypto` SHALL provide `packPublicBundle(keychain, parentAuthority, options)` that emits a bit-packed binary bundle in the wire format below, with a trailing 64-byte ed25519 signature produced by `parentAuthority.sign(DOMAIN_PUBLIC_KEYCHAIN_V1, "sign", bundleBytes)`. The signature covers everything in the bundle including the embedded `issuerRef` — swapping the issuer reference invalidates the signature.
|
||||
|
||||
`unpackPublicBundle(bytes, parentSignPublicKey)` verifies the trailing signature against the supplied parent public sign key, returning the parsed bundle and the verified `issuerRef`. A `parentSignPublicKey` of `null` selects self-verify mode, valid only for root identities (the bundle's own embedded `sign` slot is used).
|
||||
|
||||
The wire format identifier is `PUBLIC_BUNDLE_MAGIC` (4 ASCII bytes), `PUBLIC_BUNDLE_FORMAT_VERSION` is `1` in v1, and the header byte size is committed as `PUBLIC_BUNDLE_HEADER_BYTE_SIZE`. The slot record layout is committed as `KEY_SLOT_PUBLIC_BYTE_SIZE` and `KEY_SLOT_PUBLIC_BYTES = 17 + 48 = 65` bytes.
|
||||
|
||||
For a root bundle the issuer `(masterVersion, userId, depth, slotVersion)` equals the subject's, and the parent authority is the keychain itself (self-signing). Tampered bundle bytes (slot data, signature, magic, version, slotCount, length) MUST cause parse / verify to fail.
|
||||
|
||||
#### Scenario: self-signed bundle round-trips and self-verifies
|
||||
|
||||
- **WHEN** `packPublicBundle(rootKeychain, rootAuthority)` is called and the bundle is then verified in self-verify mode
|
||||
- **THEN** the call succeeds and the parsed bundle's slots match the input keychain
|
||||
|
||||
#### Scenario: parent-signed bundle verifies against parent's sign pubkey
|
||||
|
||||
- **WHEN** `packPublicBundle(serverKeychain, rootAuthority)` is called and the result is verified against the root's public sign key
|
||||
- **THEN** the call succeeds; verification against any non-root public key fails
|
||||
|
||||
#### Scenario: tampered slot bytes fail verification
|
||||
|
||||
- **WHEN** any byte of a packed bundle's slot section is mutated
|
||||
- **THEN** verification fails
|
||||
|
||||
#### Scenario: tampered signature fails verification
|
||||
|
||||
- **WHEN** any byte of the trailing 64-byte signature is mutated
|
||||
- **THEN** verification fails
|
||||
|
||||
#### Scenario: parse rejects malformed payloads
|
||||
|
||||
- **WHEN** the bundle has bad magic, wrong format version, bad slotCount, a too-short payload, or extra trailing bytes
|
||||
- **THEN** `unpackPublicBundle` throws a parse error
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"self-signed bundle round-trips and self-verifies"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"parent-signed (third-party) bundle verifies against parent's sign pubkey"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"tampered slot bytes fail verification"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"tampered signature fails verification"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"bad magic fails parse"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"wrong format version fails parse"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"bad slotCount fails parse"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"short payload fails parse"` AND `packages/crypto/crypto/src/bundles/public-bundle/public-bundle.test.ts::"extra bytes fails parse"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Chain bundle traversal — concatenated parent-signed links
|
||||
|
||||
`@suaveplan/crypto` SHALL provide a `ChainBundle` wire format and a `verifyChain(bundle, trustAnchor, clock, isRevoked?)` walker. A `ChainBundle` is a header followed by a concatenation of `PublicBundle` records ordered leaf-first. The wire format identifier is `CHAIN_BUNDLE_MAGIC`, the format version is `CHAIN_BUNDLE_FORMAT_VERSION` (currently `1`), the header byte size is `CHAIN_BUNDLE_HEADER_BYTE_SIZE`, and `linkCount` is bounded by `MAX_CHAIN_LINKS` (16).
|
||||
|
||||
The walker MUST: verify each link's signature against the next link's `sign` slot (or against `trustAnchor` for the final link), reject expired or not-yet-valid slots via the configured `Clock`, reject revoked links via the optional `isRevoked` predicate, reject `linkCount === 0`, reject `linkCount > MAX_CHAIN_LINKS`, reject mismatched `issuerRef` between adjacent links, reject bad magic / version / length / linkCount overflows at parse time. The bundle wire format MUST be byte-identical to the v1 snapshot.
|
||||
|
||||
#### Scenario: 3-link chain verifies end-to-end
|
||||
|
||||
- **WHEN** a chain bundle is built as `[subService, server, root]` (each parent-signed by the next)
|
||||
- **THEN** `verifyChain` succeeds; the parsed chain has length 3
|
||||
|
||||
#### Scenario: tampered link fails verification at that link
|
||||
|
||||
- **WHEN** any byte of a non-leaf link is mutated
|
||||
- **THEN** `verifyChain` rejects
|
||||
|
||||
#### Scenario: revoked link fails via predicate
|
||||
|
||||
- **WHEN** the `isRevoked` predicate returns true for an intermediate link
|
||||
- **THEN** `verifyChain` rejects
|
||||
|
||||
#### Scenario: expired sign slot fails
|
||||
|
||||
- **WHEN** any link's `sign` slot has `notAfter < clock.now()`
|
||||
- **THEN** `verifyChain` rejects with `KEYCHAIN.EXPIRED`
|
||||
|
||||
#### Scenario: mismatched issuer coord fails
|
||||
|
||||
- **WHEN** an intermediate link's `issuerRef` does not equal the next link's signing-slot coordinate
|
||||
- **THEN** `verifyChain` rejects
|
||||
|
||||
#### Scenario: link count bounds enforced
|
||||
|
||||
- **WHEN** packing requests `0` links or more than `MAX_CHAIN_LINKS`
|
||||
- **THEN** packing throws
|
||||
|
||||
#### Scenario: parse rejects malformed payloads
|
||||
|
||||
- **WHEN** the bundle has bad magic, bad version, out-of-range linkCount, length mismatch, or is too short
|
||||
- **THEN** `parseChainBundle` throws
|
||||
|
||||
#### Scenario: wire format is byte-identical to v1 snapshot
|
||||
|
||||
- **WHEN** a fixed-input chain is packed and compared against the committed v0x0001 snapshot
|
||||
- **THEN** the bytes match exactly
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"chain-bundle wire format is byte-identical to v0x0001 snapshot"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"packs and parses a 3-link chain"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"verifies a 3-link chain"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"verifyChain rejects tampered link"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"verifyChain rejects revoked link via predicate"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"verifyChain rejects expired sign slot"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"verifyChain rejects mismatched issuer coord"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"packChainBundle rejects 0-link"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"packChainBundle rejects > MAX_CHAIN_LINKS"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"parseChainBundle rejects bad magic"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"parseChainBundle rejects bad version"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"parseChainBundle rejects out-of-range linkCount"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"parseChainBundle rejects length mismatch"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"parseChainBundle rejects too-short payload"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Wire-format schemas exposed and committed
|
||||
|
||||
`@suaveplan/crypto` SHALL expose a `wire-format` barrel (`@suaveplan/crypto/wire-format`) that re-exports every committed wire-format identifier and byte-size constant from its owning module:
|
||||
|
||||
- `PUBLIC_BUNDLE_MAGIC`, `PUBLIC_BUNDLE_FORMAT_VERSION`, `PUBLIC_BUNDLE_HEADER_BYTE_SIZE` (from `bundles/public-bundle/`)
|
||||
- `CHAIN_BUNDLE_MAGIC`, `CHAIN_BUNDLE_FORMAT_VERSION`, `CHAIN_BUNDLE_HEADER_BYTE_SIZE`, `MAX_CHAIN_LINKS` (from `bundles/chain-bundle/`)
|
||||
- `V1_COORDINATE_BYTE_SIZE`, `COORDINATE_HEADER_BYTE_SIZE` (from `derivation/coordinate/`)
|
||||
- `KEY_SLOT_META_PRIVATE_SCHEMA`, `KEY_SLOT_META_PUBLIC_SCHEMA`, `KEY_SLOT_PRIVATE_BYTE_SIZE`, `KEY_SLOT_PUBLIC_BYTE_SIZE` (from `keychain/key-slot/`)
|
||||
|
||||
Each schema's byte-size is committed as a constant: `KEY_SLOT_PRIVATE_BYTE_SIZE === 17 + 80 === 97` and `KEY_SLOT_PUBLIC_BYTE_SIZE === 17 + 48 === 65`. Any field-width change MUST update both the constant and its consumer test, providing a loud signal for wire-format regression.
|
||||
|
||||
#### Scenario: barrel re-exports every committed identifier
|
||||
|
||||
- **WHEN** a consumer imports from `@suaveplan/crypto/wire-format`
|
||||
- **THEN** `PUBLIC_BUNDLE_MAGIC`, `CHAIN_BUNDLE_MAGIC`, `PUBLIC_BUNDLE_FORMAT_VERSION`, `CHAIN_BUNDLE_FORMAT_VERSION`, `KEY_SLOT_PRIVATE_BYTE_SIZE`, `KEY_SLOT_PUBLIC_BYTE_SIZE`, `V1_COORDINATE_BYTE_SIZE`, `MAX_CHAIN_LINKS` are all importable
|
||||
|
||||
#### Scenario: key-slot byte sizes are exactly the committed constants
|
||||
|
||||
- **WHEN** the key-slot byte-size test runs
|
||||
- **THEN** the private slot is 97 bytes and the public slot is 65 bytes
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/keychain/key-slot/key-slot.test.ts::"private slot is 17 + 80 = 97 bytes"` AND `packages/crypto/crypto/src/keychain/key-slot/key-slot.test.ts::"public slot is 17 + 48 = 65 bytes"` AND `packages/crypto/crypto/src/bundles/chain-bundle/chain-bundle.test.ts::"chain-bundle wire format is byte-identical to v0x0001 snapshot"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Authority — signing abstraction over hierarchical identity
|
||||
|
||||
`@suaveplan/crypto` SHALL define an `Authority` interface that signs payloads with a named slot under a domain tag, exposing `sign(domain, slotPurpose, payload, signal?): Promise<Uint8Array>` and `dispose(): void`. The interface is the only signing surface exposed to callers outside the package; raw `secretKey: Uint8Array` references are not passed across package boundaries.
|
||||
|
||||
V1 ships an `InMemoryAuthority` backend that holds an unlocked `Keychain` and signs from its slots. The interface is shaped so future backends (`YubiKeyAuthority`, `TpmAuthority`, `ThresholdAuthority`) can be substituted without changes to callers.
|
||||
|
||||
`Authority` MUST: produce signatures that verify under the corresponding public sign slot, fail under a different domain tag, throw `KEYCHAIN.UNKNOWN_PURPOSE` for unknown slots, throw `KEYCHAIN.SLOT_INVALID` when signing with a non-sign slot, throw `KEYCHAIN.EXPIRED` when the slot's `notAfter` is past, throw `AUTHORITY.REVOKED` when an `isRevoked` predicate matches, throw `AUTHORITY.DISPOSED` after `dispose()`, throw `AUTHORITY.ABORTED` when invoked with an already-aborted signal. `dispose()` MUST zeroise held secret material and MUST be idempotent.
|
||||
|
||||
#### Scenario: sign and verify under matching slot and domain
|
||||
|
||||
- **WHEN** `authority.sign(domain, "sign", payload)` is called
|
||||
- **THEN** the returned signature verifies against the public sign slot's key under the same domain
|
||||
|
||||
#### Scenario: signature does not verify under a different domain tag
|
||||
|
||||
- **WHEN** the signature is verified under a different domain tag
|
||||
- **THEN** verification returns false
|
||||
|
||||
#### Scenario: unknown slot purpose rejected
|
||||
|
||||
- **WHEN** `authority.sign(domain, "bogus", payload)` is called
|
||||
- **THEN** the call throws `KEYCHAIN.UNKNOWN_PURPOSE`
|
||||
|
||||
#### Scenario: signing with a non-signing slot rejected
|
||||
|
||||
- **WHEN** `authority.sign(domain, "encr", payload)` is called
|
||||
- **THEN** the call throws `KEYCHAIN.SLOT_INVALID`
|
||||
|
||||
#### Scenario: dispose zeroises and locks
|
||||
|
||||
- **WHEN** `authority.dispose()` is called and `authority.sign(...)` is then invoked
|
||||
- **THEN** post-dispose secret-key buffers contain zeros and the call throws `AUTHORITY.DISPOSED`; calling `dispose()` again is a no-op
|
||||
|
||||
#### Scenario: aborted signal cancels signing
|
||||
|
||||
- **WHEN** `authority.sign(..., signal)` is called with an already-aborted signal
|
||||
- **THEN** the call throws `AUTHORITY.ABORTED`
|
||||
|
||||
#### Scenario: revoked slot fails before signing
|
||||
|
||||
- **WHEN** an `isRevoked` predicate returns true for the slot and `authority.sign(...)` is called
|
||||
- **THEN** the call throws `AUTHORITY.REVOKED` before any signing primitive executes
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/authority/authority.test.ts::"signs with the sign slot and verifies"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"signature does NOT verify under a different domain tag"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws KEYCHAIN.UNKNOWN_PURPOSE for an unknown slot"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws KEYCHAIN.SLOT_INVALID when signing with a non-sign slot"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws AUTHORITY.DISPOSED after dispose"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"dispose zeroises secret-key buffers"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"dispose is idempotent"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws AUTHORITY.ABORTED when signal already aborted"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws KEYCHAIN.EXPIRED when slot's notAfter is past"` AND `packages/crypto/crypto/src/authority/authority.test.ts::"throws AUTHORITY.REVOKED when isRevoked predicate matches"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Hierarchical identity — Root and Server
|
||||
|
||||
`@suaveplan/crypto` SHALL provide `createRoot(options)`, `createServer(rootAuthority, options)`, `rollServer(rootAuthority, server)`, and `revokeServer(rootAuthority, server, reason)`. A `RootIdentity` is a Keychain whose `sign` slot is the trust anchor for everything beneath it. A `ServerIdentity` is a Keychain whose public bundle has been signed by the root's sign slot under `DOMAIN_PUBLIC_KEYCHAIN_V1`.
|
||||
|
||||
`rollServer` MUST produce a new `ServerIdentity` whose version is one greater than the prior, emit a record of the prior version (used by `RevocationList`), and accept missing optional `rng`. `revokeServer` MUST return a revocation entry without producing a new identity. `createRoot` and `createServer` MUST honour optional `notBefore`, `notAfter`, and `masterKey`. `createRoot` and `createServer` MUST accept missing optional `rng`. Nested servers (depth ≥ 2) MUST be traversable via the chain-bundle walker.
|
||||
|
||||
#### Scenario: createRoot produces a self-verifying identity
|
||||
|
||||
- **WHEN** `createRoot({...})` is called
|
||||
- **THEN** the result's public bundle verifies in self-verify mode
|
||||
|
||||
#### Scenario: createServer is signed under root
|
||||
|
||||
- **WHEN** `createServer(rootAuthority, ...)` is called
|
||||
- **THEN** the server's public bundle verifies against the root's public sign key
|
||||
|
||||
#### Scenario: rollServer increments version and emits prior
|
||||
|
||||
- **WHEN** `rollServer(rootAuthority, server@N)` is called
|
||||
- **THEN** the result is `server@(N+1)` and a `priorVersion === N` record is emitted
|
||||
|
||||
#### Scenario: revokeServer returns an entry without re-issuing
|
||||
|
||||
- **WHEN** `revokeServer(rootAuthority, server, reason)` is called
|
||||
- **THEN** the call returns a revocation entry and no new server identity
|
||||
|
||||
#### Scenario: optional notBefore, notAfter, masterKey honoured
|
||||
|
||||
- **WHEN** `createRoot` or `createServer` is called with explicit `notBefore` / `notAfter` / `masterKey`
|
||||
- **THEN** the resulting identity's slots reflect those inputs
|
||||
|
||||
#### Scenario: optional rng accepted
|
||||
|
||||
- **WHEN** `createRoot`, `createServer`, or `rollServer` is called without a custom `rng`
|
||||
- **THEN** the call succeeds
|
||||
|
||||
#### Scenario: nested server depth-2 builds a 3-link chain
|
||||
|
||||
- **WHEN** a sub-service is issued under a server that was issued under a root
|
||||
- **THEN** a 3-link chain bundle is constructible and verifies against the root anchor
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/identity/identity.test.ts::"createRoot produces a self-verifying root identity"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"createServer is signed under root"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"rollServer increments version + emits priorVersion"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"revokeServer returns an entry without re-issuing"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"createRoot honours optional notBefore/notAfter and masterKey"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"createServer honours optional notBefore/notAfter"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"rollServer accepts no rng option"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"createRoot/createServer accept no rng option"` AND `packages/crypto/crypto/src/identity/identity.test.ts::"nested server (depth 2) builds a 3-link chain"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: RevocationList — root-signed CRL
|
||||
|
||||
`@suaveplan/crypto` SHALL provide a `RevocationList` type representing an append-only list of revocation entries, each entry signed by the root's sign slot under `DOMAIN_CRL_V1`. Entries are immutable: any past entry's bytes are part of the signed payload; mutation invalidates the list signature.
|
||||
|
||||
The list exposes `append(entry, rootAuthority)`, `verify(rootPublicKey)`, and `check(ref): boolean`. `check` MUST report true for revoked refs and false otherwise.
|
||||
|
||||
#### Scenario: append then verify succeeds
|
||||
|
||||
- **WHEN** an entry is appended with `rootAuthority` and `verify(rootPublicKey)` is called
|
||||
- **THEN** verification returns true
|
||||
|
||||
#### Scenario: mutating a past entry invalidates the list
|
||||
|
||||
- **WHEN** any byte of any past entry is mutated and `verify(rootPublicKey)` is called
|
||||
- **THEN** verification returns false
|
||||
|
||||
#### Scenario: check reports per-entry correctly
|
||||
|
||||
- **WHEN** a server identity at a given version is appended and `check` is queried for that ref vs a different version
|
||||
- **THEN** `check` returns true for the appended ref and false for the other
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/revocation/revocation-list.test.ts::"append then verify succeeds"` AND `packages/crypto/crypto/src/revocation/revocation-list.test.ts::"mutating a past entry invalidates the list"` AND `packages/crypto/crypto/src/revocation/revocation-list.test.ts::"check returns true for revoked refs and false otherwise"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: AuditLog — append-only Merkle log
|
||||
|
||||
`@suaveplan/crypto` SHALL provide an `AuditLog` type recording every issue / roll / revoke event, structured as an append-only Merkle tree. The current Merkle root is signed by the root's sign slot under `DOMAIN_AUDIT_V1`. Any modification to a past entry changes the Merkle root and invalidates the signature.
|
||||
|
||||
The log exposes `append(event, rootAuthority)`, `signRoot(rootAuthority)`, `verify(rootPublicKey)`, `proveInclusion(index)`, and `verifyInclusion(index, proof, expectedRoot)`. The Merkle construction MUST handle odd leaf counts by duplicating the last leaf at each level. `signRoot` over an empty log MUST produce a documented zero root. `eventCount` MUST track appends.
|
||||
|
||||
#### Scenario: append then verify succeeds
|
||||
|
||||
- **WHEN** an event is appended and the root signature is verified against the root public key
|
||||
- **THEN** verification returns true
|
||||
|
||||
#### Scenario: tampering with a past entry invalidates the root
|
||||
|
||||
- **WHEN** any byte of any past event is mutated
|
||||
- **THEN** signature verification returns false
|
||||
|
||||
#### Scenario: inclusion proof verifies
|
||||
|
||||
- **WHEN** an event is appended and `proveInclusion` is requested for it
|
||||
- **THEN** the proof verifies under the signed root
|
||||
|
||||
#### Scenario: inclusion proof for out-of-range index throws
|
||||
|
||||
- **WHEN** `proveInclusion(index)` is called with an out-of-range index
|
||||
- **THEN** the call throws
|
||||
|
||||
#### Scenario: verifyInclusion rejects wrong / mis-sized expected roots
|
||||
|
||||
- **WHEN** `verifyInclusion(index, proof, expectedRoot)` is called with an expected root that is wrong or of mismatched length
|
||||
- **THEN** the call returns false
|
||||
|
||||
#### Scenario: odd leaf count handled
|
||||
|
||||
- **WHEN** the log has an odd number of leaves and the last leaf is duplicated for Merkle construction
|
||||
- **THEN** inclusion proofs still verify
|
||||
|
||||
#### Scenario: empty log signRoot produces zero root
|
||||
|
||||
- **WHEN** `signRoot` is called on an empty log
|
||||
- **THEN** the produced root is the documented zero value
|
||||
|
||||
#### Scenario: verify fails with a wrong sign slot
|
||||
|
||||
- **WHEN** the log is verified against an unrelated public sign key
|
||||
- **THEN** verification returns false
|
||||
|
||||
#### Scenario: eventCount tracks appends
|
||||
|
||||
- **WHEN** `n` events are appended
|
||||
- **THEN** `eventCount === n`
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"append then root signature verifies"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"tampering with a past entry invalidates the root"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"inclusion proof verifies for appended entries"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"inclusion proof for out-of-range index throws"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"verifyInclusion returns false for a wrong expected root"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"verifyInclusion returns false for length-mismatched expected root"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"inclusion proof handles odd leaf count (last leaf duplicated)"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"signRoot on empty log produces a zero root"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"verify fails with a wrong sign slot"` AND `packages/crypto/crypto/src/audit-log/audit-log.test.ts::"eventCount tracks appends"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Packer — encrypt-and-sign envelope
|
||||
|
||||
`@suaveplan/crypto` SHALL provide a `Packer` / `Unpacker` pair implementing a schema-driven encrypt-and-sign envelope. The envelope MUST: round-trip a value byte-for-byte under matching parties / nonce / domain, reject envelopes packed under a different domain (cross-protocol replay defence), reject envelopes whose `issuedAt` is older than a configurable replay window, reject envelopes whose `nonce` has been seen within a configurable nonce-cache window, validate pack-side nonce length, reject envelopes whose stated ciphertext length overruns the buffer, reject too-short envelopes.
|
||||
|
||||
#### Scenario: round-trip preserves the value
|
||||
|
||||
- **WHEN** `packer.pack(payload, ...)` is called and the result is `unpacker.unpack(envelope, ...)`-ed
|
||||
- **THEN** the returned payload byte-equals the original
|
||||
|
||||
#### Scenario: replay outside window rejected
|
||||
|
||||
- **WHEN** an envelope older than the replay window is presented
|
||||
- **THEN** `unpacker.unpack` throws
|
||||
|
||||
#### Scenario: nonce dedup rejects replay
|
||||
|
||||
- **WHEN** the same envelope is presented twice in succession
|
||||
- **THEN** the first call succeeds; the second throws
|
||||
|
||||
#### Scenario: cross-protocol domain tag rejected
|
||||
|
||||
- **WHEN** an envelope packed under one domain is presented to an `Unpacker` configured for another
|
||||
- **THEN** `unpacker.unpack` throws
|
||||
|
||||
#### Scenario: invalid pack-side nonce length rejected
|
||||
|
||||
- **WHEN** `packer.pack` is called with a nonce of the wrong length
|
||||
- **THEN** the call throws
|
||||
|
||||
#### Scenario: malformed envelope rejected
|
||||
|
||||
- **WHEN** an envelope's stated ciphertext length overruns the buffer, OR the envelope is too short
|
||||
- **THEN** parsing throws
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/packer/packer.test.ts::"round-trip preserves the value"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"replay outside window rejected"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"nonce dedup rejects replay"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"cross-protocol domain tag rejected"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"rejects invalid pack-side nonce length"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"rejects envelope where stated ciphertext length overruns buffer"` AND `packages/crypto/crypto/src/packer/packer.test.ts::"rejects too-short envelope"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: ChunkedPacker — per-chunk-signed P2P fragmentation
|
||||
|
||||
`@suaveplan/crypto` SHALL provide `ChunkedPacker` / `ChunkedUnpacker` implementing per-chunk signed fragmentation under `DOMAIN_CHUNK_V1`. Reassembly MUST: byte-equal the original payload on round-trip, reject tampered chunks at reassembly, reject missing chunks, reassemble correctly when chunks arrive out of order, reject malformed chunks (too short, wrong `dataLen`, mismatched total), reject empty chunk lists, reject invalid chunk sizes.
|
||||
|
||||
#### Scenario: round-trip reassembles byte-identical
|
||||
|
||||
- **WHEN** a payload is fragmented into N chunks and reassembled
|
||||
- **THEN** the reassembled bytes byte-equal the original
|
||||
|
||||
#### Scenario: tampered chunk rejected at reassembly
|
||||
|
||||
- **WHEN** any byte of a chunk is mutated
|
||||
- **THEN** reassembly throws
|
||||
|
||||
#### Scenario: missing chunk rejected
|
||||
|
||||
- **WHEN** any chunk is omitted
|
||||
- **THEN** reassembly throws
|
||||
|
||||
#### Scenario: out-of-order chunks reassemble
|
||||
|
||||
- **WHEN** chunks are presented in non-sequential order
|
||||
- **THEN** reassembly succeeds and byte-equals the original
|
||||
|
||||
#### Scenario: malformed chunks rejected
|
||||
|
||||
- **WHEN** a chunk is too short, has the wrong `dataLen` field, or has a mismatched total
|
||||
- **THEN** reassembly throws
|
||||
|
||||
#### Scenario: empty list and invalid sizes rejected
|
||||
|
||||
- **WHEN** an empty chunk list is presented, or chunking is requested with an invalid size
|
||||
- **THEN** the call throws
|
||||
|
||||
**Verified by:** `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"round-trip reassembles byte-identical"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"tampered chunk rejected at reassembly"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"missing chunk rejected"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"out-of-order chunks reassemble correctly"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"rejects malformed chunk (too short)"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"rejects chunk with wrong dataLen field"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"rejects chunks with mismatched total"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"rejects empty chunks list"` AND `packages/crypto/crypto/src/chunked-packer/chunked-packer.test.ts::"rejects invalid chunk size"`
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user