Initial Commit

openspec/archive/2026-05-03-vault-sync/proposal.md

@@ -0,0 +1,240 @@
+# vault-sync — Domain-Specific Sync & Conflict Resolution
+
+- **Change ID**: vault-sync
+- **Package**: `@suaveplan/vault-sync`
+- **Status**: PROPOSED
+- **Type**: new-package
+- **Date**: 2026-04-29
+
+## Why
+
+The Vaulted password-manager ecosystem needs an offline-first, end-to-end-encrypted sync engine that understands vault-item semantics. Generic CRDTs (`@suaveplan/crdt`) and generic diffs (`@suaveplan/diff`) provide the primitives, but they lack domain knowledge: a password field should use last-write-wins, an audit-history field should use append-only merge, and a folder move should be atomic. `vault-sync` layers domain-specific merge strategies, JSON Patch incremental sync, durable offline queues, and encrypted payloads on top of the generic primitives.
+
+## What changes
+
+- ADDED: `mergeVaultItems(local, remote, strategy)` — CRDT merge with `"lww"`, `"append"`, and `"atomic"` strategies
+- ADDED: `diffVaultItems(before, after)` / `applyPatch(item, patch)` — JSON Patch diff/apply for vault items
+- ADDED: `SyncEngine` class — `pull()`, `push()`, `sync()` operating on in-memory snapshots
+- ADDED: `SyncQueue` — durable, serializable mutation queue with vector-clock ordering and operation collapsing
+- ADDED: `encryptSyncPayload(payload, key)` / `decryptSyncPayload(encrypted, key)` — XChaCha20-Poly1305 E2EE
+- ADDED: `VaultSyncError` with structured error codes for merge, patch, sync, queue, and crypto failures
+
+## Gap analysis: sub-standard → enterprise
+
+Most password-manager sync implementations fall well below enterprise standards:
+
+| Sub-standard practice | Enterprise replacement |
+|---|---|
+| Last-write-wins blindly applied to all fields | Domain-specific CRDT merge strategies: LWW for passwords, append-only for audit history, atomic for folder moves |
+| No offline support; sync fails when disconnected | Offline-first durable queue with vector-clock ordering and operation collapsing |
+| Plaintext or TLS-only sync payloads | End-to-end encrypted sync payloads (XChaCha20-Poly1305) with encrypted blobs on server |
+| No conflict resolution; data loss on concurrent edits | Client-side CRDT merge with commutativity, idempotency, and monotonicity guarantees |
+| No ordering guarantees; edits arrive out of order | Vector-clock-based causal ordering with explicit conflict detection |
+| No sync state observability | Structured OTel spans, safe metrics, and health checks without leaking vault data |
+| Monolithic sync engine mixing transport, crypto, and domain logic | Clean separation: `vault-sync` handles domain merge + crypto payload; transport lives in `api` |
+
+This package migrates the sync engine from a naive, lossy transport layer to an offline-first, enterprise-grade, end-to-end-encrypted synchronization system.
+
+## Security model
+
+### Threat model
+
+| Attacker | Capability | Mitigation |
+|---|---|---|
+| **Local attacker** (malware, keylogger) | Can read process memory, intercept file system | E2EE: plaintext never leaves device unencrypted; sync payloads are XChaCha20-Poly1305 ciphertext |
+| **Remote attacker** (network MITM) | Can intercept sync traffic, replay payloads | E2EE renders intercepted data meaningless; tamper detection via Poly1305 tag verification |
+| **Compromised sync server** | Can read stored blobs, reorder or drop messages | Client-side encryption; server is a dumb blob store; client-side CRDT merge is authoritative |
+| **Forensic recovery** (stolen device) | Can extract local sync queue | Queue stores only encrypted payloads; keys live in `crypto-vault` and are zeroized on lock |
+
+### Encryption boundaries
+
+- **Plaintext lives only inside** the vault memory boundary (`vault` package) before sync.
+- **Sync payload plaintext** is ephemeral: encrypted immediately by `encryptSyncPayload`, then zeroized.
+- **Keys live only inside** `crypto-vault` `SecureBuffer`s; `vault-sync` never handles raw key material.
+- The sync server stores and transports **only** ciphertext + nonce + tag; it never sees plaintext, keys, or metadata.
+
+### Key hierarchy and derivation
+
+Sync encryption uses keys derived from the standard hierarchy:
+
+```text
+AUK ──► Personal Keyset ──► Vault Key ──► Sync Encryption Key (per vault)
+```
+
+- **Sync Encryption Key**: A dedicated HKDF branch (`info = "v1/sync-enc/<vaultIdHash>"`) derived from the vault key.
+- **Sync MAC Key**: A separate HKDF branch (`info = "v1/sync-mac/<vaultIdHash>"`) for Poly1305 tag keys (or use XChaCha20-Poly1305 built-in AEAD).
+- Key rotation: deriving a new sync encryption key requires re-encrypting the sync payload header only; item data keys are unchanged.
+
+### Two-secret key derivation (2SKD) threat model
+
+`vault-sync` does not perform 2SKD itself, but it relies on the security of the vault key, which ultimately derives from the AUK (2SKD output). Therefore:
+
+- Compromise of the sync server alone reveals nothing (E2EE).
+- Compromise of the sync encryption key alone reveals only sync payload contents for that vault, not the vault key or AUK.
+- Compromise of both the sync server and the sync encryption key still requires the vault key to decrypt item contents (layered encryption).
+
+### Secure memory
+
+- All plaintext diff and patch structures are standard in-memory objects (not sensitive key material), but intermediate JSON Patch strings are zeroized after encryption.
+- Sync queue entries store only ciphertext + metadata (vector clock, strategy, timestamp).
+- No key material is ever serialized into the queue or persisted to disk unencrypted.
+
+### Side-channel resistance
+
+- Merge strategies operate on plaintext vault items (already decrypted) and do not involve secret keys.
+- XChaCha20-Poly1305 tag verification is constant-time via the underlying `@suaveplan/crypto-cipher` implementation.
+- Vector clock comparison uses integer arithmetic only; no secret-dependent branches.
+
+### Compliance
+
+- **NIST SP 800-63B**: Sync authentication relies on SRP-6a (in `api` layer); encryption relies on NIST-approved AEAD primitives (ChaCha20-Poly1305 is acceptable for TLS and data-at-rest in many NIST profiles).
+- **OWASP Cryptographic Storage Cheat Sheet**: E2EE with AEAD, unique nonces, and key separation satisfies recommended controls.
+- **GDPR Article 32**: E2EE with client-side key management provides a strong technical safeguard for personal data processing.
+
+## Performance SLOs
+
+| Metric | Target | Measurement |
+|---|---|---|
+| **Sync latency (pull+push)** | P99 ≤ 500 ms for ≤ 100 changed items | End-to-end from sync initiation to merged state |
+| **Merge latency (single item)** | P99 ≤ 5 ms | CRDT merge of two item versions |
+| **Patch apply latency** | P99 ≤ 2 ms | JSON Patch application to in-memory item |
+| **Encrypt/decrypt sync payload** | P99 ≤ 10 ms per 100 KiB payload | XChaCha20-Poly1305 AEAD |
+| **Memory bound (10K items)** | ≤ 64 MiB resident | Sync queue + in-memory snapshot |
+| **Offline queue depth** | Max 10,000 pending operations | Enforced with backpressure and structured rejection |
+| **Max sync payload size** | 10 MiB | Rejects larger payloads to prevent memory abuse |
+| **Scalability: max vault size** | 100,000 items | Vector clocks scale linearly; merge is per-item |
+| **Scalability: max attachment size** | 100 MiB | Attachments stream through `drive`; sync layer handles metadata only |
+
+## Observability plan
+
+All telemetry is **safe-by-construction**: no key material, no item content, no plaintext fields, no vault-specific identifiers, and no user data in spans, metrics, or logs.
+
+### OTel spans and metrics
+
+- `vault-sync.sync.duration` — histogram of full sync latency (pull + merge + push)
+- `vault-sync.merge.duration` — histogram of merge latency per item (no item type, no count)
+- `vault-sync.patch.apply.duration` — histogram of patch application latency
+- `vault-sync.payload.encrypt.duration` — histogram of payload encryption time (no size, no vault ID)
+- `vault-sync.queue.depth` — gauge of pending offline operations (count only)
+- `vault-sync.queue.flush.duration` — histogram of queue flush latency
+- Span names: `sync.pull`, `sync.merge`, `sync.push`, `queue.flush`, `payload.encrypt`
+
+### Structured logging
+
+Safe-to-log fields only:
+
+- `event: "sync_completed"` — `durationMs`, `success`, `conflictCount` (no vault ID, no item IDs)
+- `event: "merge_conflict_detected"` — `strategy` (e.g., `"lww"`), `resolved: boolean` (no field values)
+- `event: "queue_flush_started"` — `pendingCount`
+- `event: "queue_flush_completed"` — `processedCount`, `failedCount`
+- `event: "payload_encrypt_completed"` — `durationMs` (no size, no key info)
+
+### Error telemetry
+
+- Aggregate error counts by `VaultSyncError` code (e.g., `MERGE_STRATEGY_UNKNOWN`, `PATCH_INVALID`, `SYNC_PAYLOAD_TAMPERED`).
+- No vault ID, item ID, operation content, or vector clock values attached.
+
+### Health checks
+
+- `health.vault-sync.merge` — periodic merge of synthetic test items; fails if latency > 2× SLO.
+- `health.vault-sync.encryption` — periodic encrypt/decrypt of a test payload; fails on tamper or latency spike.
+- `health.vault-sync.queue` — verifies queue serialization and deserialization succeed.
+
+### Audit logging
+
+- `audit.vault-sync.sync_start` — `timestamp`, `deviceIdHash`, `direction` (`pull`/`push`/`bidirectional`).
+- `audit.vault-sync.sync_complete` — `timestamp`, `deviceIdHash`, `success`, `conflictCount`.
+- `audit.vault-sync.queue_flush` — `timestamp`, `deviceIdHash`, `processedCount`.
+- Retention: 90 days hot, 1 year cold.
+
+## Dependency audit
+
+| Dependency | Justification | Supply-chain risk | Tree-shakeable | License |
+|---|---|---|---|---|
+| `@suaveplan/crypto-cipher` | XChaCha20-Poly1305 encrypt/decrypt | Internal monorepo; audited in CI | Yes | MIT |
+| `@suaveplan/crypto-vault` | Sync encryption key derivation | Internal monorepo; audited in CI | Yes | MIT |
+| `@suaveplan/crdt` | Generic CRDT primitives | Internal monorepo | Yes | MIT |
+| `@suaveplan/diff` | JSON Patch diff/apply | Internal monorepo | Yes | MIT |
+| `@suaveplan/encoding` | Base64 for ciphertext transport | Internal monorepo | Yes | MIT |
+| `@suaveplan/error` | Base error class | Internal monorepo | Yes | MIT |
+
+**No external runtime dependencies.** All sync primitives are internal monorepo packages. This eliminates hidden transitive dependencies and minimizes supply-chain attack surface.
+
+## Operational readiness
+
+### Graceful degradation
+
+- If sync server is unreachable, the sync engine enters **offline mode**: mutations accumulate in the durable queue; reads continue from local snapshot.
+- If sync payload exceeds max size, the engine falls back to **chunked sync**: splits vault items into multiple encrypted payloads.
+- If merge strategy is unknown for a field, the engine falls back to **LWW** and emits a telemetry event for operator review.
+
+### Resource limits
+
+- **Max sync queue depth**: 10,000 pending operations. Beyond this, new mutations are rejected with `VaultSyncError` and the user is notified.
+- **Max sync payload size**: 10 MiB. Prevents memory exhaustion from oversized payloads.
+- **Max concurrent sync operations**: 1 per vault. Prevents race conditions and vector-clock corruption.
+
+### Error recovery
+
+- Sync failures are retried with exponential backoff (base 2, max delay 60s, max 10 retries).
+- Queue is serialized to disk (encrypted) after every mutation; crash recovery replays the queue on restart.
+- Tampered sync payloads (bad Poly1305 tag) are rejected and trigger a full re-pull from server.
+- Merge conflicts are resolved client-side and do not block subsequent operations.
+
+### Backup/restore compatibility
+
+- Sync queue serialization uses standard JSON with base64-encoded ciphertext fields — portable across platforms.
+- Vector clocks are plain `{ nodeId: counter }` maps — no proprietary formats.
+- Restore replays the queue in causal order; merging is deterministic.
+
+### Upgrade/downgrade safety
+
+- Sync payload format includes a version field (`"v1"` in the AEAD associated data). New versions can add fields; old versions ignore unknown fields.
+- Vector clock schema is stable (node ID → integer counter). No migrations required.
+- Merge strategies are versioned by field name convention (`"password:v1"`). Strategy evolution is additive.
+
+### Key rotation without full re-encryption
+
+- **Sync encryption key rotation**: Derive new sync key from vault key; re-encrypt only pending queue entries and the latest sync payload header. Historical server blobs remain readable with old key until purged.
+- **Vault key rotation**: Requires re-deriving the sync encryption key and re-encrypting all pending queue entries. Server blobs are unaffected because they are per-vault-key encrypted separately.
+
+## Out of scope
+
+- Network transport implementation — `SyncEngine` accepts snapshots and returns payloads; HTTP/WebSocket/peer-to-peer transport lives in `@suaveplan/api` and `@suaveplan/native-messaging`
+- Server-side conflict resolution — server is a dumb encrypted-blob store; all merging happens client-side
+- Real-time collaborative editing with live cursors — deferred post-MVP
+- Sync payload compression — deferred; payloads are small (individual vault items)
+
+## Verification approach
+
+- `src/merge/merge.test.ts` — strategy correctness, commutativity, idempotency, unknown-strategy rejection
+- `src/patch/patch.test.ts` — diff minimality, patch application, invalid-patch rejection, round-trip
+- `src/engine/engine.test.ts` — pull, push, bidirectional sync, tombstone propagation, conflict detection
+- `src/queue/queue.test.ts` — offline accumulation, flush ordering, redundant-op collapsing, serialization
+- `src/crypto-sync/crypto-sync.test.ts` — encrypt/decrypt round-trip, wrong-key failure, tamper detection
+- `src/error/error-codes.test.ts` — every error code is thrown by at least one production path
+- `tests/dep-graph.test.ts` — dependency invariants
+
+### Coverage and quality targets
+
+- **100% coverage target**: All source files must achieve 100% statement and branch coverage.
+- **Property-based tests**: `fast-check` properties for merge commutativity, merge idempotency, patch round-trip, vector-clock monotonicity, queue serialization equivalence, and encrypted payload tamper rejection.
+- **Load/stress tests**: 10,000-item vault sync with 1,000 concurrent modifications to verify merge latency and memory bounds; 10 MiB payload encryption/decryption to verify throughput.
+- **Security audit checklist**:
+  - [ ] All sync payloads encrypted with XChaCha20-Poly1305 before leaving device
+  - [ ] Server stores and transports only ciphertext + nonce + tag
+  - [ ] No plaintext item content in logs, errors, or telemetry
+  - [ ] Vector clocks are monotonic and causality-preserving
+  - [ ] Merge strategies are deterministic and commutative
+  - [ ] Offline queue persists only encrypted payloads
+  - [ ] Tampered payloads rejected with `SYNC_PAYLOAD_TAMPERED` error
+  - [ ] Max payload size enforced at serialization boundary
+- **Penetration test scenarios**:
+  - [ ] Intercept sync traffic — verify only ciphertext visible
+  - [ ] Modify sync payload ciphertext — verify tamper detection rejects with non-retryable error
+  - [ ] Crash during sync queue flush — verify no plaintext in crash dump or queue file
+  - [ ] Replay old sync payload — verify vector clock rejects stale data
+
+## Migration impact
+
+None — additive only. `@suaveplan/vault-sync` is a new package with no existing consumers in the monorepo.