241 lines
15 KiB
Markdown
241 lines
15 KiB
Markdown
# vault-sync — Domain-Specific Sync & Conflict Resolution
|
||
|
||
- **Change ID**: vault-sync
|
||
- **Package**: `@suaveplan/vault-sync`
|
||
- **Status**: PROPOSED
|
||
- **Type**: new-package
|
||
- **Date**: 2026-04-29
|
||
|
||
## Why
|
||
|
||
The Vaulted password-manager ecosystem needs an offline-first, end-to-end-encrypted sync engine that understands vault-item semantics. Generic CRDTs (`@suaveplan/crdt`) and generic diffs (`@suaveplan/diff`) provide the primitives, but they lack domain knowledge: a password field should use last-write-wins, an audit-history field should use append-only merge, and a folder move should be atomic. `vault-sync` layers domain-specific merge strategies, JSON Patch incremental sync, durable offline queues, and encrypted payloads on top of the generic primitives.
|
||
|
||
## What changes
|
||
|
||
- ADDED: `mergeVaultItems(local, remote, strategy)` — CRDT merge with `"lww"`, `"append"`, and `"atomic"` strategies
|
||
- ADDED: `diffVaultItems(before, after)` / `applyPatch(item, patch)` — JSON Patch diff/apply for vault items
|
||
- ADDED: `SyncEngine` class — `pull()`, `push()`, `sync()` operating on in-memory snapshots
|
||
- ADDED: `SyncQueue` — durable, serializable mutation queue with vector-clock ordering and operation collapsing
|
||
- ADDED: `encryptSyncPayload(payload, key)` / `decryptSyncPayload(encrypted, key)` — XChaCha20-Poly1305 E2EE
|
||
- ADDED: `VaultSyncError` with structured error codes for merge, patch, sync, queue, and crypto failures
|
||
|
||
## Gap analysis: sub-standard → enterprise
|
||
|
||
Most password-manager sync implementations fall well below enterprise standards:
|
||
|
||
| Sub-standard practice | Enterprise replacement |
|
||
|---|---|
|
||
| Last-write-wins blindly applied to all fields | Domain-specific CRDT merge strategies: LWW for passwords, append-only for audit history, atomic for folder moves |
|
||
| No offline support; sync fails when disconnected | Offline-first durable queue with vector-clock ordering and operation collapsing |
|
||
| Plaintext or TLS-only sync payloads | End-to-end encrypted sync payloads (XChaCha20-Poly1305) with encrypted blobs on server |
|
||
| No conflict resolution; data loss on concurrent edits | Client-side CRDT merge with commutativity, idempotency, and monotonicity guarantees |
|
||
| No ordering guarantees; edits arrive out of order | Vector-clock-based causal ordering with explicit conflict detection |
|
||
| No sync state observability | Structured OTel spans, safe metrics, and health checks without leaking vault data |
|
||
| Monolithic sync engine mixing transport, crypto, and domain logic | Clean separation: `vault-sync` handles domain merge + crypto payload; transport lives in `api` |
|
||
|
||
This package migrates the sync engine from a naive, lossy transport layer to an offline-first, enterprise-grade, end-to-end-encrypted synchronization system.
|
||
|
||
## Security model
|
||
|
||
### Threat model
|
||
|
||
| Attacker | Capability | Mitigation |
|
||
|---|---|---|
|
||
| **Local attacker** (malware, keylogger) | Can read process memory, intercept file system | E2EE: plaintext never leaves device unencrypted; sync payloads are XChaCha20-Poly1305 ciphertext |
|
||
| **Remote attacker** (network MITM) | Can intercept sync traffic, replay payloads | E2EE renders intercepted data meaningless; tamper detection via Poly1305 tag verification |
|
||
| **Compromised sync server** | Can read stored blobs, reorder or drop messages | Client-side encryption; server is a dumb blob store; client-side CRDT merge is authoritative |
|
||
| **Forensic recovery** (stolen device) | Can extract local sync queue | Queue stores only encrypted payloads; keys live in `crypto-vault` and are zeroized on lock |
|
||
|
||
### Encryption boundaries
|
||
|
||
- **Plaintext lives only inside** the vault memory boundary (`vault` package) before sync.
|
||
- **Sync payload plaintext** is ephemeral: encrypted immediately by `encryptSyncPayload`, then zeroized.
|
||
- **Keys live only inside** `crypto-vault` `SecureBuffer`s; `vault-sync` never handles raw key material.
|
||
- The sync server stores and transports **only** ciphertext + nonce + tag; it never sees plaintext, keys, or metadata.
|
||
|
||
### Key hierarchy and derivation
|
||
|
||
Sync encryption uses keys derived from the standard hierarchy:
|
||
|
||
```text
|
||
AUK ──► Personal Keyset ──► Vault Key ──► Sync Encryption Key (per vault)
|
||
```
|
||
|
||
- **Sync Encryption Key**: A dedicated HKDF branch (`info = "v1/sync-enc/<vaultIdHash>"`) derived from the vault key.
|
||
- **Sync MAC Key**: A separate HKDF branch (`info = "v1/sync-mac/<vaultIdHash>"`) for Poly1305 tag keys (or use XChaCha20-Poly1305 built-in AEAD).
|
||
- Key rotation: deriving a new sync encryption key requires re-encrypting the sync payload header only; item data keys are unchanged.
|
||
|
||
### Two-secret key derivation (2SKD) threat model
|
||
|
||
`vault-sync` does not perform 2SKD itself, but it relies on the security of the vault key, which ultimately derives from the AUK (2SKD output). Therefore:
|
||
|
||
- Compromise of the sync server alone reveals nothing (E2EE).
|
||
- Compromise of the sync encryption key alone reveals only sync payload contents for that vault, not the vault key or AUK.
|
||
- Compromise of both the sync server and the sync encryption key still requires the vault key to decrypt item contents (layered encryption).
|
||
|
||
### Secure memory
|
||
|
||
- All plaintext diff and patch structures are standard in-memory objects (not sensitive key material), but intermediate JSON Patch strings are zeroized after encryption.
|
||
- Sync queue entries store only ciphertext + metadata (vector clock, strategy, timestamp).
|
||
- No key material is ever serialized into the queue or persisted to disk unencrypted.
|
||
|
||
### Side-channel resistance
|
||
|
||
- Merge strategies operate on plaintext vault items (already decrypted) and do not involve secret keys.
|
||
- XChaCha20-Poly1305 tag verification is constant-time via the underlying `@suaveplan/crypto-cipher` implementation.
|
||
- Vector clock comparison uses integer arithmetic only; no secret-dependent branches.
|
||
|
||
### Compliance
|
||
|
||
- **NIST SP 800-63B**: Sync authentication relies on SRP-6a (in `api` layer); encryption relies on NIST-approved AEAD primitives (ChaCha20-Poly1305 is acceptable for TLS and data-at-rest in many NIST profiles).
|
||
- **OWASP Cryptographic Storage Cheat Sheet**: E2EE with AEAD, unique nonces, and key separation satisfies recommended controls.
|
||
- **GDPR Article 32**: E2EE with client-side key management provides a strong technical safeguard for personal data processing.
|
||
|
||
## Performance SLOs
|
||
|
||
| Metric | Target | Measurement |
|
||
|---|---|---|
|
||
| **Sync latency (pull+push)** | P99 ≤ 500 ms for ≤ 100 changed items | End-to-end from sync initiation to merged state |
|
||
| **Merge latency (single item)** | P99 ≤ 5 ms | CRDT merge of two item versions |
|
||
| **Patch apply latency** | P99 ≤ 2 ms | JSON Patch application to in-memory item |
|
||
| **Encrypt/decrypt sync payload** | P99 ≤ 10 ms per 100 KiB payload | XChaCha20-Poly1305 AEAD |
|
||
| **Memory bound (10K items)** | ≤ 64 MiB resident | Sync queue + in-memory snapshot |
|
||
| **Offline queue depth** | Max 10,000 pending operations | Enforced with backpressure and structured rejection |
|
||
| **Max sync payload size** | 10 MiB | Rejects larger payloads to prevent memory abuse |
|
||
| **Scalability: max vault size** | 100,000 items | Vector clocks scale linearly; merge is per-item |
|
||
| **Scalability: max attachment size** | 100 MiB | Attachments stream through `drive`; sync layer handles metadata only |
|
||
|
||
## Observability plan
|
||
|
||
All telemetry is **safe-by-construction**: no key material, no item content, no plaintext fields, no vault-specific identifiers, and no user data in spans, metrics, or logs.
|
||
|
||
### OTel spans and metrics
|
||
|
||
- `vault-sync.sync.duration` — histogram of full sync latency (pull + merge + push)
|
||
- `vault-sync.merge.duration` — histogram of merge latency per item (no item type, no count)
|
||
- `vault-sync.patch.apply.duration` — histogram of patch application latency
|
||
- `vault-sync.payload.encrypt.duration` — histogram of payload encryption time (no size, no vault ID)
|
||
- `vault-sync.queue.depth` — gauge of pending offline operations (count only)
|
||
- `vault-sync.queue.flush.duration` — histogram of queue flush latency
|
||
- Span names: `sync.pull`, `sync.merge`, `sync.push`, `queue.flush`, `payload.encrypt`
|
||
|
||
### Structured logging
|
||
|
||
Safe-to-log fields only:
|
||
|
||
- `event: "sync_completed"` — `durationMs`, `success`, `conflictCount` (no vault ID, no item IDs)
|
||
- `event: "merge_conflict_detected"` — `strategy` (e.g., `"lww"`), `resolved: boolean` (no field values)
|
||
- `event: "queue_flush_started"` — `pendingCount`
|
||
- `event: "queue_flush_completed"` — `processedCount`, `failedCount`
|
||
- `event: "payload_encrypt_completed"` — `durationMs` (no size, no key info)
|
||
|
||
### Error telemetry
|
||
|
||
- Aggregate error counts by `VaultSyncError` code (e.g., `MERGE_STRATEGY_UNKNOWN`, `PATCH_INVALID`, `SYNC_PAYLOAD_TAMPERED`).
|
||
- No vault ID, item ID, operation content, or vector clock values attached.
|
||
|
||
### Health checks
|
||
|
||
- `health.vault-sync.merge` — periodic merge of synthetic test items; fails if latency > 2× SLO.
|
||
- `health.vault-sync.encryption` — periodic encrypt/decrypt of a test payload; fails on tamper or latency spike.
|
||
- `health.vault-sync.queue` — verifies queue serialization and deserialization succeed.
|
||
|
||
### Audit logging
|
||
|
||
- `audit.vault-sync.sync_start` — `timestamp`, `deviceIdHash`, `direction` (`pull`/`push`/`bidirectional`).
|
||
- `audit.vault-sync.sync_complete` — `timestamp`, `deviceIdHash`, `success`, `conflictCount`.
|
||
- `audit.vault-sync.queue_flush` — `timestamp`, `deviceIdHash`, `processedCount`.
|
||
- Retention: 90 days hot, 1 year cold.
|
||
|
||
## Dependency audit
|
||
|
||
| Dependency | Justification | Supply-chain risk | Tree-shakeable | License |
|
||
|---|---|---|---|---|
|
||
| `@suaveplan/crypto-cipher` | XChaCha20-Poly1305 encrypt/decrypt | Internal monorepo; audited in CI | Yes | MIT |
|
||
| `@suaveplan/crypto-vault` | Sync encryption key derivation | Internal monorepo; audited in CI | Yes | MIT |
|
||
| `@suaveplan/crdt` | Generic CRDT primitives | Internal monorepo | Yes | MIT |
|
||
| `@suaveplan/diff` | JSON Patch diff/apply | Internal monorepo | Yes | MIT |
|
||
| `@suaveplan/encoding` | Base64 for ciphertext transport | Internal monorepo | Yes | MIT |
|
||
| `@suaveplan/error` | Base error class | Internal monorepo | Yes | MIT |
|
||
|
||
**No external runtime dependencies.** All sync primitives are internal monorepo packages. This eliminates hidden transitive dependencies and minimizes supply-chain attack surface.
|
||
|
||
## Operational readiness
|
||
|
||
### Graceful degradation
|
||
|
||
- If sync server is unreachable, the sync engine enters **offline mode**: mutations accumulate in the durable queue; reads continue from local snapshot.
|
||
- If sync payload exceeds max size, the engine falls back to **chunked sync**: splits vault items into multiple encrypted payloads.
|
||
- If merge strategy is unknown for a field, the engine falls back to **LWW** and emits a telemetry event for operator review.
|
||
|
||
### Resource limits
|
||
|
||
- **Max sync queue depth**: 10,000 pending operations. Beyond this, new mutations are rejected with `VaultSyncError` and the user is notified.
|
||
- **Max sync payload size**: 10 MiB. Prevents memory exhaustion from oversized payloads.
|
||
- **Max concurrent sync operations**: 1 per vault. Prevents race conditions and vector-clock corruption.
|
||
|
||
### Error recovery
|
||
|
||
- Sync failures are retried with exponential backoff (base 2, max delay 60s, max 10 retries).
|
||
- Queue is serialized to disk (encrypted) after every mutation; crash recovery replays the queue on restart.
|
||
- Tampered sync payloads (bad Poly1305 tag) are rejected and trigger a full re-pull from server.
|
||
- Merge conflicts are resolved client-side and do not block subsequent operations.
|
||
|
||
### Backup/restore compatibility
|
||
|
||
- Sync queue serialization uses standard JSON with base64-encoded ciphertext fields — portable across platforms.
|
||
- Vector clocks are plain `{ nodeId: counter }` maps — no proprietary formats.
|
||
- Restore replays the queue in causal order; merging is deterministic.
|
||
|
||
### Upgrade/downgrade safety
|
||
|
||
- Sync payload format includes a version field (`"v1"` in the AEAD associated data). New versions can add fields; old versions ignore unknown fields.
|
||
- Vector clock schema is stable (node ID → integer counter). No migrations required.
|
||
- Merge strategies are versioned by field name convention (`"password:v1"`). Strategy evolution is additive.
|
||
|
||
### Key rotation without full re-encryption
|
||
|
||
- **Sync encryption key rotation**: Derive new sync key from vault key; re-encrypt only pending queue entries and the latest sync payload header. Historical server blobs remain readable with old key until purged.
|
||
- **Vault key rotation**: Requires re-deriving the sync encryption key and re-encrypting all pending queue entries. Server blobs are unaffected because they are per-vault-key encrypted separately.
|
||
|
||
## Out of scope
|
||
|
||
- Network transport implementation — `SyncEngine` accepts snapshots and returns payloads; HTTP/WebSocket/peer-to-peer transport lives in `@suaveplan/api` and `@suaveplan/native-messaging`
|
||
- Server-side conflict resolution — server is a dumb encrypted-blob store; all merging happens client-side
|
||
- Real-time collaborative editing with live cursors — deferred post-MVP
|
||
- Sync payload compression — deferred; payloads are small (individual vault items)
|
||
|
||
## Verification approach
|
||
|
||
- `src/merge/merge.test.ts` — strategy correctness, commutativity, idempotency, unknown-strategy rejection
|
||
- `src/patch/patch.test.ts` — diff minimality, patch application, invalid-patch rejection, round-trip
|
||
- `src/engine/engine.test.ts` — pull, push, bidirectional sync, tombstone propagation, conflict detection
|
||
- `src/queue/queue.test.ts` — offline accumulation, flush ordering, redundant-op collapsing, serialization
|
||
- `src/crypto-sync/crypto-sync.test.ts` — encrypt/decrypt round-trip, wrong-key failure, tamper detection
|
||
- `src/error/error-codes.test.ts` — every error code is thrown by at least one production path
|
||
- `tests/dep-graph.test.ts` — dependency invariants
|
||
|
||
### Coverage and quality targets
|
||
|
||
- **100% coverage target**: All source files must achieve 100% statement and branch coverage.
|
||
- **Property-based tests**: `fast-check` properties for merge commutativity, merge idempotency, patch round-trip, vector-clock monotonicity, queue serialization equivalence, and encrypted payload tamper rejection.
|
||
- **Load/stress tests**: 10,000-item vault sync with 1,000 concurrent modifications to verify merge latency and memory bounds; 10 MiB payload encryption/decryption to verify throughput.
|
||
- **Security audit checklist**:
|
||
- [ ] All sync payloads encrypted with XChaCha20-Poly1305 before leaving device
|
||
- [ ] Server stores and transports only ciphertext + nonce + tag
|
||
- [ ] No plaintext item content in logs, errors, or telemetry
|
||
- [ ] Vector clocks are monotonic and causality-preserving
|
||
- [ ] Merge strategies are deterministic and commutative
|
||
- [ ] Offline queue persists only encrypted payloads
|
||
- [ ] Tampered payloads rejected with `SYNC_PAYLOAD_TAMPERED` error
|
||
- [ ] Max payload size enforced at serialization boundary
|
||
- **Penetration test scenarios**:
|
||
- [ ] Intercept sync traffic — verify only ciphertext visible
|
||
- [ ] Modify sync payload ciphertext — verify tamper detection rejects with non-retryable error
|
||
- [ ] Crash during sync queue flush — verify no plaintext in crash dump or queue file
|
||
- [ ] Replay old sync payload — verify vector clock rejects stale data
|
||
|
||
## Migration impact
|
||
|
||
None — additive only. `@suaveplan/vault-sync` is a new package with no existing consumers in the monorepo.
|