first commit
This commit is contained in:
29
docs/foundation/data/README.md
Normal file
29
docs/foundation/data/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# `data`
|
||||
|
||||
Data manipulation, masking, deduplication, diffing, and byte utilities.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Packages in `data`
|
||||
|
||||
17 packages.
|
||||
|
||||
| Package | Description |
|
||||
|---|---|
|
||||
| [`@suaveplan/bytes`](./bytes/README.md) | Low-level byte utilities for Suaveplan: lossless hex/base64/UTF-8 encoding, `Uint8Array` manipulation, and UUID conversion. No native addons; depends only on p… |
|
||||
| [`@suaveplan/clipboard`](./clipboard/README.md) | Enterprise Clipboard API abstraction for browser-tier consumers. Wraps the async Clipboard API with a textarea + `execCommand` fallback, a paste-event read fal… |
|
||||
| [`@suaveplan/data-masking`](./data-masking/README.md) | PII/sensitive data masking for objects and strings with field-level rules, nested traversal, eight masking strategies, and OpenTelemetry instrumentation. |
|
||||
| [`@suaveplan/dedup`](./dedup/README.md) | Entity deduplication engine for detecting and merging duplicate records in any data set. Given a list of objects with varying representations of the same real-… |
|
||||
| [`@suaveplan/diff`](./diff/README.md) | Zero-dependency RFC 6902 JSON Patch engine for computing, applying, and inverting structural differences between JSON documents. The `diff(a, b)` function prod… |
|
||||
| [`@suaveplan/json`](./json/README.md) | Safe, enterprise-grade JSON utilities with prototype-pollution protection, circular-reference detection, and OpenTelemetry instrumentation. |
|
||||
| [`@suaveplan/media`](./media/README.md) | Universal media detection, metadata extraction, and constraint validation. Combines a pluggable magic-byte `SignatureRegistry` with the MIME database from `@su… |
|
||||
| [`@suaveplan/mime`](./mime/README.md) | Structured MIME-type detection, parsing, and registry for the Suaveplan platform. Provides RFC 7231 Content-Type parsing, magic-byte file signature detection,… |
|
||||
| [`@suaveplan/packer`](./packer/README.md) | Binary packing/unpacking utilities. Two modes: |
|
||||
| [`@suaveplan/packer-browser`](./packer-browser/README.md) | Browser-tier runtime for `@suaveplan/packer` archives. Ships a Service Worker interceptor, a `PackedResourceManager` (LRU + Blob-URL bookkeeping + ref-count +… |
|
||||
| [`@suaveplan/pagination`](./pagination/README.md) | Universal pagination primitives for cursor-based, offset-based, and keyset-based (seek) pagination strategies. The package covers the full pagination surface:… |
|
||||
| [`@suaveplan/pwa`](./pwa/README.md) | Enterprise browser-tier PWA primitives for Suaveplan applications. |
|
||||
| [`@suaveplan/react-packer`](./react-packer/README.md) | React 19 companion for `@suaveplan/packer-browser`. Ships `usePackedResource`, `usePackedImage`, `usePackedAudio`, `usePackedVideo`, and `usePackedFont` hooks… |
|
||||
| [`@suaveplan/serialize`](./serialize/README.md) | Round-trip-preserving structured serialization for JavaScript's rich type system. The standard `JSON.stringify`/`JSON.parse` pair silently drops `Date` objects… |
|
||||
| [`@suaveplan/templates`](./templates/README.md) | A plugin-based string template engine for TypeScript. Resolves `{variable}` and `{variable:transform}` tokens in strings using a priority-sorted, regex-scan pi… |
|
||||
| [`@suaveplan/validation`](./validation/README.md) | Runtime schema validation with a Zod-compatible API. This package is a self-contained parse engine — it does not import Zod or any other validation library — s… |
|
||||
| [`@suaveplan/web-workers`](./web-workers/README.md) | Enterprise browser-tier runtime for managing Web Workers, SharedWorkers, and fixed-size worker pools with typed RPC, AbortSignal-aware cancellation, transfer-l… |
|
||||
<!-- end:auto-docs -->
|
||||
1
docs/foundation/data/_intro.md
Normal file
1
docs/foundation/data/_intro.md
Normal file
@@ -0,0 +1 @@
|
||||
Data manipulation, masking, deduplication, diffing, and byte utilities.
|
||||
95
docs/foundation/data/bytes/README.md
Normal file
95
docs/foundation/data/bytes/README.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# @suaveplan/bytes - Low-level byte utilities: hex/base64/utf8 encoding, Uint8Array helpers, UUID conversion
|
||||
|
||||
## Summary
|
||||
|
||||
Low-level byte utilities for Suaveplan: lossless hex/base64/UTF-8 encoding, `Uint8Array` manipulation, and UUID conversion. No native addons; depends only on platform globals (`TextEncoder`, `atob`/`btoa`, `crypto.randomUUID`) and `@suaveplan/error` for typed errors.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [bytes-error](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [bytes-error](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [bytes-error](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [bytes-error](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_- [encoding](./src/encoding/encoding.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_ - [bytes-error](./src/errors/bytes-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ - [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_ - [uint8](./src/uint8/uint8.md) — <!-- begin:auto-docs --> _(no submodules)_- [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_ - [uuid](./src/uuid/uuid.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/bytes
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```ts
|
||||
import {
|
||||
bytesToHex, hexToBytes,
|
||||
bytesToBase64, base64ToBytes,
|
||||
bytesToUtf8, utf8ToBytes,
|
||||
concatUint8Arrays,
|
||||
convertDataFormat,
|
||||
generateUuid, uuidToUint8Array, uint8ArrayToUuid,
|
||||
} from "@suaveplan/bytes";
|
||||
|
||||
// Encoding
|
||||
const raw = utf8ToBytes("Hello"); // Uint8Array [72,101,108,108,111]
|
||||
const hex = bytesToHex(raw); // "48656c6c6f"
|
||||
const b64 = bytesToBase64(raw); // "SGVsbG8="
|
||||
const text = bytesToUtf8(raw); // "Hello"
|
||||
|
||||
// Uint8 helpers
|
||||
const frame = concatUint8Arrays(raw, new Uint8Array([0x21]));
|
||||
|
||||
// Unified conversion
|
||||
const asBase64 = convertDataFormat("48656c6c6f", "base64", "hex"); // "SGVsbG8="
|
||||
|
||||
// UUID
|
||||
const id = generateUuid(); // "550e8400-e29b-41d4-a716-446655440000"
|
||||
const bytes = uuidToUint8Array(id); // Uint8Array (16 bytes)
|
||||
const back = uint8ArrayToUuid(bytes); // → id
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Description | Docs |
|
||||
|--------|-------------|------|
|
||||
| `@suaveplan/bytes/encoding` | Hex, base64, UTF-8 encode/decode | [encoding.md](src/encoding/encoding.md) |
|
||||
| `@suaveplan/bytes/uint8` | Uint8Array utilities and format conversion | [uint8.md](src/uint8/uint8.md) |
|
||||
| `@suaveplan/bytes/uuid` | UUID generation and binary conversion | [uuid.md](src/uuid/uuid.md) |
|
||||
| `@suaveplan/bytes/errors` | `BytesError`, `BYTES_CODES` | — |
|
||||
| `@suaveplan/bytes/schemas` | Zod schemas (HexStringSchema, etc.) | — |
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Run with `bun src/__benchmarks__/encoding.bench.ts`.
|
||||
|
||||
```text
|
||||
clk: ~2.74 GHz
|
||||
cpu: Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz
|
||||
runtime: bun 1.3.13 (x64-linux)
|
||||
|
||||
benchmark avg (min … max)
|
||||
bytesToHex (32 bytes) 9.31 µs/iter
|
||||
bytesToHex (64 KB) 5.42 ms/iter
|
||||
hexToBytes (32 bytes) 7.43 µs/iter
|
||||
bytesToBase64 (32 bytes) 4.29 µs/iter
|
||||
bytesToBase64 (64 KB) 983.62 µs/iter
|
||||
```
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Thrown by |
|
||||
|------|-----------|
|
||||
| `BYTES.ENCODE.INVALID_HEX` | `hexToBytes` — odd length or non-hex chars |
|
||||
| `BYTES.ENCODE.INVALID_BASE64` | `base64ToBytes` — invalid base64 alphabet |
|
||||
| `BYTES.ENCODE.INVALID_UTF8` | Reserved |
|
||||
| `BYTES.UINT8.INVALID_INPUT` | `inputToUint8Array` — unknown input type |
|
||||
| `BYTES.UINT8.FORMAT_MISMATCH` | `validateInputTypeFormat` — type/format clash |
|
||||
| `BYTES.UUID.INVALID_FORMAT` | `uuidToUint8Array` — malformed UUID string |
|
||||
| `BYTES.UUID.INVALID_LENGTH` | `uint8ArrayToUuid` — not 16 bytes |
|
||||
|
||||
103
docs/foundation/data/bytes/src/encoding/encoding.md
Normal file
103
docs/foundation/data/bytes/src/encoding/encoding.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# encoding
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
The `encoding` module provides lossless, bidirectional conversion between raw `Uint8Array` byte sequences and three ubiquitous text representations: lowercase hexadecimal, standard base64, and UTF-8. It is the lowest-level I/O surface in `@suaveplan/bytes` and is intentionally free of any dependency beyond `@opentelemetry/api` and the `@suaveplan/error` error substrate.
|
||||
|
||||
Every conversion function validates its input strictly and throws a typed `BytesError` on malformed data, making it safe to use directly in protocol parsers, cryptographic pipelines, and data serialisation layers without additional guard code at the call site.
|
||||
|
||||
## Features
|
||||
|
||||
- **Hex encoding/decoding** — `bytesToHex` emits lowercase two-character hex pairs; `hexToBytes` accepts both upper- and lowercase and rejects odd-length or non-hex inputs with `BYTES.ENCODE.INVALID_HEX`.
|
||||
- **Base64 encoding/decoding** — `bytesToBase64` uses the standard (non-URL-safe) alphabet; `base64ToBytes` validates the input against the standard alphabet before delegating to `atob`, rethrowing any decode error as `BYTES.ENCODE.INVALID_BASE64`.
|
||||
- **UTF-8 encoding/decoding** — `utf8ToBytes` uses `TextEncoder` and `bytesToUtf8` uses `TextDecoder`, both handling multi-byte Unicode code points correctly.
|
||||
- **OTel tracing** — every exported function wraps its body in a named active span under the `@suaveplan/bytes` tracer so encoding operations appear in distributed traces.
|
||||
- **Zod schemas** — `HexStringSchema` and `Base64StringSchema` are exported from `encoding.schemas.ts` for use in Zod-based validation pipelines.
|
||||
- **Property-tested** — fast-check verifies lossless round-trips for arbitrary byte sequences for all three codecs.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { bytesToHex, hexToBytes, bytesToBase64, base64ToBytes, utf8ToBytes, bytesToUtf8 } from "@suaveplan/bytes/encoding";
|
||||
|
||||
const raw = new Uint8Array([72, 101, 108, 108, 111]);
|
||||
|
||||
// Hex
|
||||
const hex = bytesToHex(raw); // "48656c6c6f"
|
||||
const back = hexToBytes(hex); // Uint8Array [72, 101, 108, 108, 111]
|
||||
|
||||
// Base64
|
||||
const b64 = bytesToBase64(raw); // "SGVsbG8="
|
||||
const back2 = base64ToBytes(b64); // Uint8Array [72, 101, 108, 108, 111]
|
||||
|
||||
// UTF-8
|
||||
const text = bytesToUtf8(raw); // "Hello"
|
||||
const back3 = utf8ToBytes(text); // Uint8Array [72, 101, 108, 108, 111]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { hexToBytes, bytesToBase64, BytesError } from "@suaveplan/bytes";
|
||||
|
||||
// Safely parse an untrusted hex payload from an HTTP header
|
||||
function parseHexHeader(value: string): Uint8Array {
|
||||
try {
|
||||
return hexToBytes(value.trim().toLowerCase());
|
||||
} catch (err) {
|
||||
if (err instanceof BytesError) {
|
||||
// err.code === "BYTES.ENCODE.INVALID_HEX"
|
||||
throw new Error(`Malformed X-Signature header: ${err.message}`);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
// Convert the raw bytes to base64 for JSON transport
|
||||
const signature = parseHexHeader("deadbeef");
|
||||
const forJson = bytesToBase64(signature); // "3q2+7w=="
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `bytesToHex(bytes: Uint8Array): string`
|
||||
|
||||
Converts `bytes` to a lowercase hexadecimal string. Each byte becomes exactly two hex characters. Returns `""` for an empty array.
|
||||
|
||||
### `hexToBytes(hex: string): Uint8Array`
|
||||
|
||||
Converts a hex string to a `Uint8Array`. Accepts upper- and lowercase characters. Returns an empty array for an empty string.
|
||||
|
||||
**Throws** `BytesError(BYTES.ENCODE.INVALID_HEX)` when:
|
||||
|
||||
- `hex.length` is odd.
|
||||
- `hex` contains characters outside `[0-9a-fA-F]`.
|
||||
|
||||
### `bytesToBase64(bytes: Uint8Array): string`
|
||||
|
||||
Converts `bytes` to a standard base64 string (with `=` padding). Returns `""` for an empty array.
|
||||
|
||||
### `base64ToBytes(base64: string): Uint8Array`
|
||||
|
||||
Converts a standard base64 string to a `Uint8Array`. Returns an empty array for an empty string.
|
||||
|
||||
**Throws** `BytesError(BYTES.ENCODE.INVALID_BASE64)` when `base64` contains characters outside the standard base64 alphabet or the decode otherwise fails.
|
||||
|
||||
### `bytesToUtf8(bytes: Uint8Array): string`
|
||||
|
||||
Decodes `bytes` as a UTF-8 string using `TextDecoder`. Invalid sequences are replaced with the Unicode replacement character (U+FFFD) per platform behaviour.
|
||||
|
||||
### `utf8ToBytes(str: string): Uint8Array`
|
||||
|
||||
Encodes `str` as UTF-8 bytes using `TextEncoder`. Every Unicode code point, including surrogate pairs and multi-byte characters, is encoded correctly.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The hex codec avoids regular-expression replacement in the hot encode path in favour of a simple loop over byte values with `toString(16).padStart(2, "0")`, which is both readable and cache-friendly on V8/JavaScriptCore. The decode path uses a single `HEX_RE.test()` validation pass before looping — this avoids per-character error checking while still catching all invalid inputs in one branch.
|
||||
|
||||
The base64 codec delegates to the platform `btoa`/`atob` globals (available natively in Node 18+ and Bun), avoiding any third-party encoding library. The encode path materialises a binary string from the byte array; the decode path pre-validates with a regex before calling `atob` and wraps any thrown exception in a `BytesError` so callers never see a raw `DOMException`.
|
||||
|
||||
OTel spans are started synchronously and ended in `finally` blocks, ensuring spans are always closed even when an exception propagates. The tracer is module-level so instrumentation is zero-cost when no SDK is installed (the `@opentelemetry/api` noop tracer is used automatically).
|
||||
89
docs/foundation/data/bytes/src/errors/bytes-error.md
Normal file
89
docs/foundation/data/bytes/src/errors/bytes-error.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# bytes-error
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`bytes-error.ts` defines the typed error substrate for `@suaveplan/bytes`. It registers all stable error codes under the `"BYTES"` namespace using `defineCodes` from `@suaveplan/error`, exposes the `BytesCode` union type that constrains which codes `BytesError` instances can carry, and re-exports the concrete `BytesError` class whose constructor widens visibility from `protected` to `public`.
|
||||
|
||||
By funnelling all package errors through a single typed class with a stable code registry, callers can narrow on `err instanceof BytesError` and then switch on `err.code` without any string fragility.
|
||||
|
||||
## Features
|
||||
|
||||
- **`BYTES_CODES`** — immutable record mapping human-readable dotted keys (e.g. `"ENCODE.INVALID_HEX"`) to stable prefixed string codes (`"BYTES.ENCODE.INVALID_HEX"`), produced by `defineCodes`.
|
||||
- **`BytesCode`** — TypeScript union type of all values in `BYTES_CODES`; used as the generic parameter of `BytesError`.
|
||||
- **`BytesError`** — extends `SuaveplanError<BytesCode>`; carries a stable `code`, a human-readable `message`, and an optional `cause` chain via `SuaveplanErrorOptions`.
|
||||
- **Stable code registry** — codes are registered once and never change; adding a new code is a non-breaking addition; removing one is a breaking change.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { BytesError, BYTES_CODES } from "@suaveplan/bytes/errors";
|
||||
|
||||
throw new BytesError(
|
||||
BYTES_CODES["ENCODE.INVALID_HEX"],
|
||||
"Hex string has odd length: 3 characters"
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { BytesError, BYTES_CODES } from "@suaveplan/bytes/errors";
|
||||
|
||||
function assertHex(value: string): void {
|
||||
if (value.length % 2 !== 0) {
|
||||
throw new BytesError(
|
||||
BYTES_CODES["ENCODE.INVALID_HEX"],
|
||||
`Invalid hex string: odd length (${value.length})`,
|
||||
{ cause: new RangeError("Length must be even") }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Callers can narrow:
|
||||
try {
|
||||
assertHex("abc");
|
||||
} catch (err) {
|
||||
if (err instanceof BytesError) {
|
||||
console.error(err.code); // "BYTES.ENCODE.INVALID_HEX"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `BYTES_CODES`
|
||||
|
||||
`Record<string, BytesCode>` — maps dotted keys to prefixed code strings:
|
||||
|
||||
| Key | Code |
|
||||
|-----|------|
|
||||
| `"ENCODE.INVALID_HEX"` | `"BYTES.ENCODE.INVALID_HEX"` |
|
||||
| `"ENCODE.INVALID_BASE64"` | `"BYTES.ENCODE.INVALID_BASE64"` |
|
||||
| `"ENCODE.INVALID_UTF8"` | `"BYTES.ENCODE.INVALID_UTF8"` |
|
||||
| `"UINT8.INVALID_INPUT"` | `"BYTES.UINT8.INVALID_INPUT"` |
|
||||
| `"UINT8.FORMAT_MISMATCH"` | `"BYTES.UINT8.FORMAT_MISMATCH"` |
|
||||
| `"UUID.INVALID_LENGTH"` | `"BYTES.UUID.INVALID_LENGTH"` |
|
||||
| `"UUID.INVALID_FORMAT"` | `"BYTES.UUID.INVALID_FORMAT"` |
|
||||
|
||||
### `BytesCode`
|
||||
|
||||
`type BytesCode = (typeof BYTES_CODES)[keyof typeof BYTES_CODES]`
|
||||
|
||||
Union type of all code strings. Used as the generic argument to `SuaveplanError`.
|
||||
|
||||
### `BytesError`
|
||||
|
||||
```ts
|
||||
class BytesError extends SuaveplanError<BytesCode> {
|
||||
public constructor(code: BytesCode, message: string, options?: SuaveplanErrorOptions);
|
||||
}
|
||||
```
|
||||
|
||||
Typed error for all `@suaveplan/bytes` failures. The constructor widens visibility from the protected base to public.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`defineCodes` from `@suaveplan/error` produces a frozen object whose values are the exact strings that will be serialised over the wire. The `"BYTES."` prefix guarantees no collision with codes from other packages. The `as const` assertion on the code-name array preserves tuple literal types so TypeScript can verify exhaustiveness in `switch (err.code)` statements.
|
||||
78
docs/foundation/data/bytes/src/types/types.md
Normal file
78
docs/foundation/data/bytes/src/types/types.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# types
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`types.ts` defines the shared TypeScript types used across all modules in `@suaveplan/bytes`. These types form the public contract for format identifiers and accepted input values, allowing all three modules (`encoding`, `uint8`, `uuid`) to share a consistent vocabulary without any runtime code or circular dependencies.
|
||||
|
||||
Keeping shared types in a dedicated file — rather than scattering them across implementation files — makes the public API surface explicit and gives TypeScript enough information to provide accurate autocompletion and type narrowing at call sites.
|
||||
|
||||
## Features
|
||||
|
||||
- **`InputFormat`** — string literal union identifying the four recognised input encodings: `"hex"`, `"base64"`, `"utf8"`, `"uint8array"`.
|
||||
- **`OutputFormat`** — string literal union identifying the four recognised output encodings (same set as `InputFormat`).
|
||||
- **`BytesInput`** — union type accepting either a plain `string` or a `Uint8Array`; used as the input type for `inputToUint8Array`, `detectInputFormat`, and `convertDataFormat`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import type { InputFormat, OutputFormat, BytesInput } from "@suaveplan/bytes";
|
||||
|
||||
function toBytes(input: BytesInput, fmt: InputFormat): Uint8Array {
|
||||
// TypeScript narrows input based on fmt
|
||||
if (input instanceof Uint8Array) return input;
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import type { OutputFormat, BytesInput } from "@suaveplan/bytes";
|
||||
import { convertDataFormat } from "@suaveplan/bytes";
|
||||
|
||||
// A generic transform pipeline step
|
||||
interface TransformStep {
|
||||
input: BytesInput;
|
||||
outputFormat: OutputFormat;
|
||||
inputFormat?: import("@suaveplan/bytes").InputFormat;
|
||||
}
|
||||
|
||||
function runStep(step: TransformStep): string | Uint8Array {
|
||||
return convertDataFormat(step.input, step.outputFormat, step.inputFormat);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `InputFormat`
|
||||
|
||||
```ts
|
||||
type InputFormat = "hex" | "base64" | "utf8" | "uint8array";
|
||||
```
|
||||
|
||||
Identifies the encoding of an input value passed to conversion functions. Used by `inputToUint8Array`, `detectInputFormat`, `validateInputTypeFormat`, and `convertDataFormat`.
|
||||
|
||||
### `OutputFormat`
|
||||
|
||||
```ts
|
||||
type OutputFormat = "hex" | "base64" | "utf8" | "uint8array";
|
||||
```
|
||||
|
||||
Identifies the encoding to produce from conversion functions. Used by `formatOutput` and `convertDataFormat`.
|
||||
|
||||
### `BytesInput`
|
||||
|
||||
```ts
|
||||
type BytesInput = string | Uint8Array;
|
||||
```
|
||||
|
||||
Union of the two accepted raw input types. When a `Uint8Array` is provided, no decoding step is needed. When a `string` is provided, the caller supplies an `InputFormat` or lets `detectInputFormat` infer it.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`InputFormat` and `OutputFormat` are intentionally identical sets. They are defined as separate types rather than a single alias so that function signatures self-document the intended direction of flow (`input` vs. `output`), and so that future versions can diverge — for example, adding an `"url-base64"` output-only format — without affecting the input union.
|
||||
|
||||
There is no runtime code in this file. TypeScript erases all type declarations at compile time, so this module contributes zero bytes to the final bundle.
|
||||
113
docs/foundation/data/bytes/src/uint8/uint8.md
Normal file
113
docs/foundation/data/bytes/src/uint8/uint8.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# uint8
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
The `uint8` module provides ergonomic, format-agnostic utilities for working with `Uint8Array` values: concatenation, slicing, equality testing, and format detection and conversion. It builds on the `encoding` module to offer a unified façade — `convertDataFormat` — that accepts any mix of hex, base64, UTF-8, or raw byte inputs and produces any requested output format, with automatic format detection when the caller omits an explicit hint.
|
||||
|
||||
The module is designed to sit at the heart of serialisation pipelines, protocol parsers, and data-transformation layers where the input format is heterogeneous or unknown at compile time.
|
||||
|
||||
## Features
|
||||
|
||||
- **`concatUint8Arrays`** — allocates a single result buffer and copies all source arrays in a single pass; accepts variadic arguments including zero arrays.
|
||||
- **`sliceUint8Array`** — thin wrapper around `Uint8Array.prototype.slice` that returns a new copy (not a view) with consistent semantics.
|
||||
- **`uint8ArraysEqual`** — linear-scan byte equality; safe for general-purpose comparison (not constant-time).
|
||||
- **`detectInputFormat`** — heuristic: `Uint8Array` → `"uint8array"`, even-length all-hex string → `"hex"`, everything else → `"utf8"`.
|
||||
- **`inputToUint8Array`** — normalise any `BytesInput` to a `Uint8Array`, calling the appropriate codec.
|
||||
- **`validateInputTypeFormat`** — assert type/format compatibility and throw a typed `BytesError` on mismatch.
|
||||
- **`formatOutput`** — serialise a `Uint8Array` to the desired `OutputFormat`.
|
||||
- **`convertDataFormat`** — unified one-stop conversion with auto-detection, OTel instrumentation.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
concatUint8Arrays,
|
||||
uint8ArraysEqual,
|
||||
convertDataFormat,
|
||||
} from "@suaveplan/bytes/uint8";
|
||||
|
||||
// Concatenation
|
||||
const header = new Uint8Array([0x01, 0x02]);
|
||||
const body = new Uint8Array([0x03, 0x04, 0x05]);
|
||||
const frame = concatUint8Arrays(header, body);
|
||||
// → Uint8Array [1, 2, 3, 4, 5]
|
||||
|
||||
// Equality
|
||||
uint8ArraysEqual(frame, new Uint8Array([1, 2, 3, 4, 5])); // → true
|
||||
|
||||
// Format conversion
|
||||
const asHex = convertDataFormat(frame, "hex"); // → "0102030405"
|
||||
const back = convertDataFormat(asHex, "uint8array", "hex");
|
||||
// → Uint8Array [1, 2, 3, 4, 5]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { convertDataFormat, detectInputFormat } from "@suaveplan/bytes/uint8";
|
||||
|
||||
// A pipeline that receives data in an unknown text format and normalises it
|
||||
function normalise(raw: string): Uint8Array {
|
||||
const fmt = detectInputFormat(raw);
|
||||
console.log("detected format:", fmt); // "hex" | "utf8"
|
||||
return convertDataFormat(raw, "uint8array") as Uint8Array;
|
||||
}
|
||||
|
||||
// Produce multiple representations from one call
|
||||
function multiView(input: Uint8Array) {
|
||||
return {
|
||||
hex: convertDataFormat(input, "hex") as string,
|
||||
base64: convertDataFormat(input, "base64") as string,
|
||||
utf8: convertDataFormat(input, "utf8") as string,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `concatUint8Arrays(...arrays: Uint8Array[]): Uint8Array`
|
||||
|
||||
Concatenate zero or more `Uint8Array` values into a single new `Uint8Array`. Allocates exactly one buffer sized to the sum of all input lengths.
|
||||
|
||||
### `sliceUint8Array(arr: Uint8Array, start: number, end?: number): Uint8Array`
|
||||
|
||||
Return a copy of `arr[start..end)`. If `end` is omitted, slices to the end of the array. Returns an empty `Uint8Array` when `start === end`.
|
||||
|
||||
### `uint8ArraysEqual(a: Uint8Array, b: Uint8Array): boolean`
|
||||
|
||||
Return `true` iff `a.length === b.length` and every byte is identical. Not constant-time.
|
||||
|
||||
### `detectInputFormat(input: BytesInput): InputFormat`
|
||||
|
||||
Heuristically determine the format of `input`. `Uint8Array` → `"uint8array"`. Even-length all-hex string → `"hex"`. Otherwise → `"utf8"`.
|
||||
|
||||
### `inputToUint8Array(input: BytesInput, format?: InputFormat): Uint8Array`
|
||||
|
||||
Convert `input` to a `Uint8Array`. If `input` is already a `Uint8Array`, returns it unchanged. For strings, uses `format` or `detectInputFormat(input)`.
|
||||
|
||||
**Throws** `BytesError(BYTES.UINT8.FORMAT_MISMATCH)` when `format === "uint8array"` but `input` is a string.
|
||||
|
||||
### `validateInputTypeFormat(input: BytesInput, format: InputFormat): void`
|
||||
|
||||
Assert that the runtime type of `input` is compatible with `format`.
|
||||
|
||||
**Throws** `BytesError(BYTES.UINT8.FORMAT_MISMATCH)` on mismatch.
|
||||
|
||||
### `formatOutput(bytes: Uint8Array, format: OutputFormat): string | Uint8Array`
|
||||
|
||||
Serialise `bytes` to the given output format. Returns the same reference for `"uint8array"`.
|
||||
|
||||
### `convertDataFormat(input: BytesInput, outputFormat: OutputFormat, inputFormat?: InputFormat): string | Uint8Array`
|
||||
|
||||
Convert `input` from `inputFormat` (or auto-detected) to `outputFormat`. Wraps the operation in an OTel span named `bytes.uint8.convertDataFormat`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`concatUint8Arrays` pre-computes the total length in a single reduce pass and writes all sub-arrays via `Uint8Array.prototype.set`, which is optimised as a memory copy by V8 and JavaScriptCore. This is significantly faster than repeated array spreading or `Buffer.concat`-style approaches.
|
||||
|
||||
`detectInputFormat` uses a two-condition check: even length and a regex over the full string. The hex heuristic will misclassify a legitimate UTF-8 string that happens to be even-length and consist entirely of hex chars (e.g., `"deadbeef"`). Callers that know the format should pass it explicitly. A future version may accept a `strict` option that requires a `0x` prefix for unambiguous hex detection.
|
||||
|
||||
`uint8ArraysEqual` is a simple linear scan and is deliberately **not** constant-time. Do not use it to compare MACs or cryptographic secrets — use the constant-time comparison from `@suaveplan/crypto-utils` for that purpose.
|
||||
97
docs/foundation/data/bytes/src/uuid/uuid.md
Normal file
97
docs/foundation/data/bytes/src/uuid/uuid.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# uuid
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
The `uuid` module provides three focused primitives for working with Universally Unique Identifiers in `@suaveplan/bytes`: generating cryptographically random UUIDv4 strings, converting canonical UUID strings to their compact 16-byte binary representation, and converting a 16-byte `Uint8Array` back to the canonical hyphen-separated string. These operations are essential for any system that stores entity identifiers as binary blobs in databases, network protocols, or binary serialisation formats while presenting them as human-readable strings at the API boundary.
|
||||
|
||||
All three functions validate their inputs strictly and throw typed `BytesError` values on malformed data, making them safe to use directly in request handlers and protocol parsers.
|
||||
|
||||
## Features
|
||||
|
||||
- **`generateUuid`** — generates a UUIDv4 string using the platform `crypto.randomUUID()` — no third-party uuid library, no custom random-number manipulation.
|
||||
- **`uuidToUint8Array`** — parses a canonical UUID string into exactly 16 bytes; validates format with a regex before any byte parsing.
|
||||
- **`uint8ArrayToUuid`** — formats a 16-byte `Uint8Array` as a canonical lowercase UUID string; validates length before formatting.
|
||||
- **OTel tracing** — every function is wrapped in a named active span under the `@suaveplan/bytes` tracer.
|
||||
- **Zod schema** — `UuidStringSchema` validates the canonical UUID format and is composable with any Zod object schema.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { generateUuid, uuidToUint8Array, uint8ArrayToUuid } from "@suaveplan/bytes/uuid";
|
||||
|
||||
// Generate a fresh UUIDv4
|
||||
const id = generateUuid();
|
||||
// → "550e8400-e29b-41d4-a716-446655440000" (random each call)
|
||||
|
||||
// Convert to binary for compact storage
|
||||
const bytes = uuidToUint8Array(id);
|
||||
// → Uint8Array of length 16
|
||||
|
||||
// Restore the canonical string
|
||||
const restored = uint8ArrayToUuid(bytes);
|
||||
// → same as `id`
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { uuidToUint8Array, uint8ArrayToUuid, BytesError } from "@suaveplan/bytes";
|
||||
import { UuidStringSchema } from "@suaveplan/bytes/uuid";
|
||||
import z from "zod";
|
||||
|
||||
// Validate and convert an incoming UUID header to binary for DB storage
|
||||
const RequestSchema = z.object({
|
||||
entityId: UuidStringSchema,
|
||||
});
|
||||
|
||||
function handleRequest(raw: unknown): Uint8Array {
|
||||
const { entityId } = RequestSchema.parse(raw);
|
||||
try {
|
||||
return uuidToUint8Array(entityId);
|
||||
} catch (err) {
|
||||
if (err instanceof BytesError) {
|
||||
// err.code === "BYTES.UUID.INVALID_FORMAT"
|
||||
throw new Error(`Malformed entity ID: ${err.message}`);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
// Convert back to string for API responses
|
||||
function toApiResponse(raw: Uint8Array): string {
|
||||
return uint8ArrayToUuid(raw);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `generateUuid(): string`
|
||||
|
||||
Generate a UUID version 4 string using `crypto.randomUUID()`. Output is always lowercase and matches the pattern `/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/`.
|
||||
|
||||
### `uuidToUint8Array(uuid: string): Uint8Array`
|
||||
|
||||
Parse a canonical UUID string (upper- or lowercase) into a 16-byte `Uint8Array`. Strips the four hyphens, then reads each pair of hex digits as a byte.
|
||||
|
||||
**Throws** `BytesError(BYTES.UUID.INVALID_FORMAT)` when `uuid` does not match the canonical pattern `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`.
|
||||
|
||||
### `uint8ArrayToUuid(bytes: Uint8Array): string`
|
||||
|
||||
Format a 16-byte `Uint8Array` as a canonical lowercase UUID string in the form `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`.
|
||||
|
||||
**Throws** `BytesError(BYTES.UUID.INVALID_LENGTH)` when `bytes.length !== 16`.
|
||||
|
||||
### `UuidStringSchema` (from `uuid.schemas.ts`)
|
||||
|
||||
Zod schema that validates the canonical UUID format. Accepts both upper- and lowercase hex digits. Does not verify UUID version or variant bits.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`generateUuid` delegates entirely to the platform `crypto.randomUUID()` (Node 14.17+, Bun, all modern browsers), ensuring the UUID is generated from a cryptographically secure source of randomness with no custom bit-twiddling. A polyfill that sets `globalThis.crypto = require("node:crypto").webcrypto` is included in the package test setup so that Node `forks`-pool Vitest runs have access to the `crypto` global.
|
||||
|
||||
`uuidToUint8Array` validates the input with a compiled regex before any byte parsing. This rejects inputs that are structurally wrong (wrong group lengths, non-hex chars, missing hyphens) with a single check rather than per-byte error detection.
|
||||
|
||||
`uint8ArrayToUuid` materialises a hex string from the byte array, then splices in the four hyphens at the standard offsets (8, 12, 16, 20 hex chars). Output is always lowercase because `Number.prototype.toString(16)` produces lowercase hex on all platforms.
|
||||
92
docs/foundation/data/clipboard/README.md
Normal file
92
docs/foundation/data/clipboard/README.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# @suaveplan/clipboard - Enterprise Clipboard API abstraction: typed errors, OTel metrics, async queue serializer, secure-context guard, image/blob copy + read, paste-event read fallback, permission change subscription, focus-aware retry, sanitizer hook. Browser-tier, zero @suaveplan/* runtime deps beyond error + opentelemetry + types.
|
||||
|
||||
Enterprise Clipboard API abstraction for browser-tier consumers. Wraps the
|
||||
async Clipboard API with a textarea + `execCommand` fallback, a paste-event
|
||||
read fallback, image / blob copy + read via `ClipboardItem`, FIFO
|
||||
serialization, secure-context guard, permission change subscription,
|
||||
focus-aware retry, optional HTML sanitizer, structured errors via
|
||||
`@suaveplan/error`, and OpenTelemetry instrumentation.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [errors](./src/errors/errors.md) — Stable error code registry, per-code metadata, and the concrete `ClipboardError` class for the `@suaveplan/clipboard` p…
|
||||
- [errors](./src/errors/errors.md) — Stable error code registry, per-code metadata, and the concrete `ClipboardError` class for the `@suaveplan/clipboard` p…
|
||||
- [fallback](./src/fallback/fallback.md) — Legacy clipboard fallback paths used when `navigator.clipboard` is unavailable or rejects. Provides a hidden-textarea +…
|
||||
- [fallback](./src/fallback/fallback.md) — Legacy clipboard fallback paths used when `navigator.clipboard` is unavailable or rejects. Provides a hidden-textarea +…
|
||||
- [manager](./src/manager/manager.md) — `ClipboardManager` is the orchestrator class for `@suaveplan/clipboard`. It composes the navigator + window seams, the…
|
||||
- [helpers](./src/manager/helpers.md) — Pure helpers shared between `manager.ts` and `operations.ts`. Keeps both files under the line budget by housing the nav…
|
||||
- [manager](./src/manager/manager.md) — `ClipboardManager` is the orchestrator class for `@suaveplan/clipboard`. It composes the navigator + window seams, the…
|
||||
- [operations](./src/manager/operations.md) — Pure operation functions invoked by `ClipboardManager`. Each function takes an `OperationContext` (navigator, window, o…
|
||||
- [observability](./src/observability/observability.md) — OpenTelemetry instrumentation bundle for `@suaveplan/clipboard`. Wires four counters and one histogram on a single `Met…
|
||||
- [observability](./src/observability/observability.md) — OpenTelemetry instrumentation bundle for `@suaveplan/clipboard`. Wires four counters and one histogram on a single `Met…
|
||||
- [permissions](./src/permissions/permissions.md) — Probe and subscribe to clipboard permission state via the browser's Permissions API. `checkPermissions` returns a snaps…
|
||||
- [permissions](./src/permissions/permissions.md) — Probe and subscribe to clipboard permission state via the browser's Permissions API. `checkPermissions` returns a snaps…
|
||||
- [queue](./src/queue/queue.md) — `AsyncQueue` is a minimal FIFO asynchronous queue that serializes operations on a single tail promise. The `ClipboardMa…
|
||||
- [queue](./src/queue/queue.md) — `AsyncQueue` is a minimal FIFO asynchronous queue that serializes operations on a single tail promise. The `ClipboardMa…
|
||||
- [sanitizer](./src/sanitizer/sanitizer.md) — Sanitizer hooks applied to inbound `text/html` clipboard payloads. The package ships an identity sanitizer (pass-throug…
|
||||
- [sanitizer](./src/sanitizer/sanitizer.md) — Sanitizer hooks applied to inbound `text/html` clipboard payloads. The package ships an identity sanitizer (pass-throug…
|
||||
- [security](./src/security/security.md) — Browser-side security helpers that guard clipboard operations against insecure contexts and orchestrate the focus-aware…
|
||||
- [security](./src/security/security.md) — Browser-side security helpers that guard clipboard operations against insecure contexts and orchestrate the focus-aware…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/clipboard
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { ClipboardManager } from "@suaveplan/clipboard";
|
||||
|
||||
const clipboard = new ClipboardManager({
|
||||
useFallback: true,
|
||||
requireSecureContext: false,
|
||||
serialize: true,
|
||||
});
|
||||
|
||||
const result = await clipboard.copyText("hello");
|
||||
if (!result.success) {
|
||||
console.error("copy failed:", result.code, result.error);
|
||||
}
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
- [`manager/manager.md`](./src/manager/manager.md) — `ClipboardManager`, the orchestrator.
|
||||
- [`manager/helpers.md`](./src/manager/helpers.md) — internal helpers used by the manager.
|
||||
- [`manager/operations.md`](./src/manager/operations.md) — pure operation functions.
|
||||
- [`errors/errors.md`](./src/errors/errors.md) — `CLIPBOARD_CODES`, `ClipboardError`, metadata.
|
||||
- [`fallback/fallback.md`](./src/fallback/fallback.md) — textarea + paste-event fallback paths.
|
||||
- [`observability/observability.md`](./src/observability/observability.md) — OpenTelemetry counters + histogram.
|
||||
- [`permissions/permissions.md`](./src/permissions/permissions.md) — permission probe + change subscription.
|
||||
- [`queue/queue.md`](./src/queue/queue.md) — `AsyncQueue` FIFO serializer.
|
||||
- [`sanitizer/sanitizer.md`](./src/sanitizer/sanitizer.md) — sanitizer interface + helpers.
|
||||
- [`security/security.md`](./src/security/security.md) — secure-context guards + focus utilities.
|
||||
|
||||
## Features
|
||||
|
||||
- Copy plain text, HTML, images, and arbitrary MIME blobs.
|
||||
- Read plain text and rich content with optional HTML sanitization.
|
||||
- Paste-event fallback when `navigator.clipboard.readText` is unavailable.
|
||||
- Permission probe + live subscription (`onPermissionChange`).
|
||||
- Async FIFO queue so concurrent calls execute in arrival order.
|
||||
- Secure-context guard with typed error code (`CLIPBOARD.INSECURE_CONTEXT`).
|
||||
- Focus-aware retry on `NotAllowedError: Document is not focused`.
|
||||
- Eight stable error codes under the `CLIPBOARD.*` namespace.
|
||||
- Five OpenTelemetry instruments wired into every public method.
|
||||
- SSR-safe — graceful degradation when `document` / `navigator` / `window` are undefined.
|
||||
|
||||
## Error codes
|
||||
|
||||
`CLIPBOARD.API_UNAVAILABLE`, `CLIPBOARD.PERMISSION_DENIED`,
|
||||
`CLIPBOARD.FALLBACK_FAILED`, `CLIPBOARD.INSECURE_CONTEXT`,
|
||||
`CLIPBOARD.NO_CONTENT`, `CLIPBOARD.READ_FAILED`,
|
||||
`CLIPBOARD.DOCUMENT_UNAVAILABLE`, `CLIPBOARD.QUEUE_DRAINED`.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
126
docs/foundation/data/clipboard/src/errors/errors.md
Normal file
126
docs/foundation/data/clipboard/src/errors/errors.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# `errors/errors.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Stable error code registry, per-code metadata, and the concrete
|
||||
`ClipboardError` class for the `@suaveplan/clipboard` package. Every error
|
||||
path inside the manager, fallback, permissions, and security modules
|
||||
surfaces one of these codes so consumers can match programmatically
|
||||
without inspecting message strings. Codes are namespaced under
|
||||
`CLIPBOARD.*` and registered via the shared `defineCodes` helper from
|
||||
`@suaveplan/error` so the runtime values, the literal union type, and the
|
||||
metadata table stay in lock-step.
|
||||
|
||||
## Features
|
||||
|
||||
- Eight stable codes: `API_UNAVAILABLE`, `PERMISSION_DENIED`,
|
||||
`FALLBACK_FAILED`, `INSECURE_CONTEXT`, `NO_CONTENT`, `READ_FAILED`,
|
||||
`DOCUMENT_UNAVAILABLE`, `QUEUE_DRAINED`.
|
||||
- Per-code metadata: `httpStatus`, `retryable`, `severity`, `description`.
|
||||
- `ClipboardError extends SuaveplanError<ClipboardErrorCode>` — preserves
|
||||
cause chains, supports OTel span events, and integrates with the
|
||||
project's typed-error contract.
|
||||
- `asClipboardError(code, cause, fallbackMessage)` lifts unknown thrown
|
||||
values into the typed hierarchy at every catch site.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
CLIPBOARD_CODES,
|
||||
ClipboardError,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
try {
|
||||
// ...
|
||||
} catch (cause) {
|
||||
throw new ClipboardError(
|
||||
CLIPBOARD_CODES.READ_FAILED,
|
||||
"Clipboard read rejected",
|
||||
{ cause: cause instanceof Error ? cause : new Error(String(cause)) },
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
asClipboardError,
|
||||
CLIPBOARD_CODES,
|
||||
CLIPBOARD_ERROR_METADATA,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
function decide(err: unknown): { retry: boolean; status: number } {
|
||||
if (
|
||||
err instanceof Error &&
|
||||
"code" in err &&
|
||||
typeof (err as { code?: string }).code === "string" &&
|
||||
(err as { code: string }).code.startsWith("CLIPBOARD.")
|
||||
) {
|
||||
const meta =
|
||||
CLIPBOARD_ERROR_METADATA[
|
||||
(err as { code: keyof typeof CLIPBOARD_ERROR_METADATA })
|
||||
.code
|
||||
];
|
||||
return { retry: meta.retryable, status: meta.httpStatus };
|
||||
}
|
||||
const lifted = asClipboardError(
|
||||
CLIPBOARD_CODES.READ_FAILED,
|
||||
err,
|
||||
"Unknown clipboard failure.",
|
||||
);
|
||||
return {
|
||||
retry: lifted.metadata.retryable,
|
||||
status: lifted.metadata.httpStatus,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `CLIPBOARD_CODES`
|
||||
|
||||
`Readonly<Record<<key>, "CLIPBOARD.<key>">>` produced by `defineCodes`.
|
||||
The runtime values are namespaced strings; the type infers a literal
|
||||
union for exhaustive matching.
|
||||
|
||||
### `CLIPBOARD_ERROR_METADATA`
|
||||
|
||||
`Readonly<Record<ClipboardErrorCode, ClipboardCodeMetadata>>` — frozen map
|
||||
from each code to its `httpStatus`, `retryable`, `severity`, and
|
||||
`description` fields.
|
||||
|
||||
### `ClipboardCodeMetadata`
|
||||
|
||||
Interface with `httpStatus: number`, `severity: "info" | "warning" | "error"`,
|
||||
`retryable: boolean`, `description: string`.
|
||||
|
||||
### `ClipboardError`
|
||||
|
||||
Concrete subclass of `SuaveplanError<ClipboardErrorCode>`. Constructor
|
||||
signature: `new ClipboardError(code, message, options?)`. Exposes a
|
||||
`.metadata` getter that returns the row from
|
||||
`CLIPBOARD_ERROR_METADATA`.
|
||||
|
||||
### `asClipboardError(code, cause, fallbackMessage)`
|
||||
|
||||
Lifts an arbitrary `unknown` thrown value into a `ClipboardError`,
|
||||
preserving the cause chain. Uses `cause.message` when available and
|
||||
`fallbackMessage` otherwise. Always returns a `ClipboardError` with the
|
||||
supplied code.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The package version of `defineCodes` is the canonical helper from
|
||||
`@suaveplan/error`; every package uses the same pattern so cross-
|
||||
cutting tooling (registry validators, retention checks, OpenSpec
|
||||
conformance) can introspect every namespace identically. Codes are
|
||||
intentionally namespaced strings rather than enums so they remain JSON-
|
||||
serialisable and stable across versions. The metadata table is `Object.freeze`-d
|
||||
so mutation is a typecheck error and a runtime error; reviewers can
|
||||
diff additions in pull requests without worrying about silent rewrites.
|
||||
94
docs/foundation/data/clipboard/src/fallback/fallback.md
Normal file
94
docs/foundation/data/clipboard/src/fallback/fallback.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# `fallback/fallback.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Legacy clipboard fallback paths used when `navigator.clipboard` is
|
||||
unavailable or rejects. Provides a hidden-textarea + `execCommand("copy")`
|
||||
write path and a paste-event-based read path. Both return the same
|
||||
`ClipboardResult` shape as the API path so the manager's call sites do
|
||||
not branch on which mechanism actually ran. The paste-event fallback is
|
||||
purpose-built for environments — older Safari, embedded WebViews, and
|
||||
permission-denied contexts — where `navigator.clipboard.readText`
|
||||
rejects but the user can paste manually.
|
||||
|
||||
## Features
|
||||
|
||||
- `copyTextFallback(text)` — synchronous textarea + `execCommand` copy.
|
||||
- `readTextFallback(timeoutMs)` — focus + paste-event capture with a
|
||||
configurable timeout.
|
||||
- `dispatchSyntheticPaste(text)` — test helper that fires a synthetic
|
||||
`paste` event on the most recent hidden host.
|
||||
- SSR-safe — both copy and read paths return
|
||||
`CLIPBOARD.DOCUMENT_UNAVAILABLE` when `document` is undefined.
|
||||
- Off-screen, non-interactive host element so users never see the
|
||||
textarea visibly steal focus.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { copyTextFallback } from "@suaveplan/clipboard";
|
||||
|
||||
const result = copyTextFallback("hello");
|
||||
if (!result.success) {
|
||||
console.warn("legacy copy failed:", result.code);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
readTextFallback,
|
||||
dispatchSyntheticPaste,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
async function readClipboardWithPasteFallback(): Promise<string | undefined> {
|
||||
const result = await readTextFallback(2000);
|
||||
return result.success ? result.data : undefined;
|
||||
}
|
||||
|
||||
// Inside a test harness:
|
||||
const promise = readTextFallback(500);
|
||||
queueMicrotask(() => dispatchSyntheticPaste("from-test"));
|
||||
const result = await promise;
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `copyTextFallback(text: string): ClipboardResult<string>`
|
||||
|
||||
Stages a hidden `<textarea>` carrying `text`, selects it, runs
|
||||
`document.execCommand("copy")`, and removes the textarea on cleanup.
|
||||
Returns `{ success: true, data, method: "fallback" }` on success and a
|
||||
structured failure with `code: CLIPBOARD.FALLBACK_FAILED` (or
|
||||
`CLIPBOARD.DOCUMENT_UNAVAILABLE` outside the browser) otherwise.
|
||||
|
||||
### `readTextFallback(timeoutMs: number): Promise<ClipboardResult<string>>`
|
||||
|
||||
Creates a hidden textarea, focuses it, listens for the first `paste`
|
||||
event, and resolves with the `text/plain` payload carried by the event.
|
||||
Resolves with `code: CLIPBOARD.READ_FAILED` if the timeout elapses
|
||||
first and `code: CLIPBOARD.DOCUMENT_UNAVAILABLE` outside the browser.
|
||||
|
||||
### `dispatchSyntheticPaste(text: string): boolean`
|
||||
|
||||
Test helper. Locates the most recent hidden host (`textarea[aria-hidden="true"]`)
|
||||
and dispatches a `paste` ClipboardEvent carrying `text` as `text/plain`.
|
||||
Returns `true` when a host was found, `false` otherwise.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The hidden textarea is positioned at `-9999px` and given `opacity: 0`
|
||||
plus `aria-hidden="true"` so assistive technologies skip it. The cleanup
|
||||
runs in a `finally` block — even if `execCommand` throws, the textarea is
|
||||
removed. The read fallback tracks a `settled` flag so a duplicate or
|
||||
post-timeout `paste` event cannot resolve the promise twice. The paste
|
||||
event may carry a null `clipboardData` on some browsers; the module
|
||||
falls back to an empty string in that case rather than rejecting, since
|
||||
"clipboard contained nothing" is a valid outcome. The synthetic-paste
|
||||
helper is exported to keep tests deterministic — production code never
|
||||
calls it.
|
||||
96
docs/foundation/data/clipboard/src/manager/helpers.md
Normal file
96
docs/foundation/data/clipboard/src/manager/helpers.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# `manager/helpers.ts`
|
||||
|
||||
## Purpose
|
||||
|
||||
Pure helpers shared between `manager.ts` and `operations.ts`. Keeps both
|
||||
files under the line budget by housing the navigator / window resolution,
|
||||
the structured failure builder, the focus-error detector, and the
|
||||
`ClipboardItem` build + read pipelines in a single, side-effect-free
|
||||
module that tests can exercise directly.
|
||||
|
||||
## Features
|
||||
|
||||
- `resolveNavigator(nav?)` — caller, global, or empty shim.
|
||||
- `resolveWindow(win?)` — caller, global, or `undefined`.
|
||||
- `failure(code, message, method)` — typed `ClipboardResult` failure.
|
||||
- `isFocusError(cause)` — detects the canonical focus-required rejection.
|
||||
- `buildClipboardItems(data)` — MIME → `Blob` map for `ClipboardItem`.
|
||||
- `readClipboardItems(items, sanitizer?)` — pulls text / html / image /
|
||||
blobs out of a list of `ClipboardItem` instances.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { failure, CLIPBOARD_CODES } from "@suaveplan/clipboard";
|
||||
|
||||
const r = failure(CLIPBOARD_CODES.NO_CONTENT, "empty", "api");
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
buildClipboardItems,
|
||||
readClipboardItems,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const items = buildClipboardItems({
|
||||
text: "hello",
|
||||
html: "<b>hi</b>",
|
||||
image: pngBlob,
|
||||
blobs: { "application/json": jsonBlob },
|
||||
});
|
||||
|
||||
const clipboardItem = new ClipboardItem(items);
|
||||
await navigator.clipboard.write([clipboardItem]);
|
||||
|
||||
// Round-trip back:
|
||||
const read = await navigator.clipboard.read();
|
||||
const data = await readClipboardItems(read, (html) => html.trim());
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `resolveNavigator(nav?: ClipboardNavigator): ClipboardNavigator`
|
||||
|
||||
Returns `nav` when defined, otherwise the global `navigator`, otherwise
|
||||
the empty object `{}`.
|
||||
|
||||
### `resolveWindow(win?: ClipboardWindowLike): ClipboardWindowLike | undefined`
|
||||
|
||||
Returns `win` when defined, otherwise the global `window`, otherwise
|
||||
`undefined`.
|
||||
|
||||
### `failure<T>(code, message, method): ClipboardResult<T>`
|
||||
|
||||
Returns `{ success: false, error: message, code, method }`. Generic over
|
||||
`T` so call sites compose with their own result types.
|
||||
|
||||
### `isFocusError(cause: unknown): boolean`
|
||||
|
||||
Returns `true` when `cause` is an `Error` whose `name === "NotAllowedError"`
|
||||
and whose `message` (case-insensitively) contains `"focus"`. The browser
|
||||
signals "Clipboard needs page focus" with this exact shape.
|
||||
|
||||
### `buildClipboardItems(data: ClipboardData): Record<string, Blob>`
|
||||
|
||||
Builds a MIME → Blob map for `new ClipboardItem(...)`. Skips empty
|
||||
`text` and `html`. Defaults a typeless image blob to `image/png`.
|
||||
|
||||
### `readClipboardItems(items, sanitizer?): Promise<ClipboardData>`
|
||||
|
||||
Iterates every entry of every `ClipboardItem`, populating `text`,
|
||||
`html`, `image`, and `blobs`. The first image-typed entry wins. The
|
||||
`sanitizer` (when supplied) is applied to the raw `text/html` payload
|
||||
before returning.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The helpers are intentionally side-effect-free and free of telemetry
|
||||
calls so the same code runs from `runCopyText`, `runCopyRich`, etc.,
|
||||
without double-recording counters. `buildClipboardItems` and
|
||||
`readClipboardItems` are mirror images of each other — every MIME slot
|
||||
that `buildClipboardItems` writes is read back by `readClipboardItems`
|
||||
into the same `ClipboardData` shape. The focus-error detector matches
|
||||
both Chromium ("Document is not focused.") and WebKit ("Document not
|
||||
focused.") wording.
|
||||
142
docs/foundation/data/clipboard/src/manager/manager.md
Normal file
142
docs/foundation/data/clipboard/src/manager/manager.md
Normal file
@@ -0,0 +1,142 @@
|
||||
# `manager/manager.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`ClipboardManager` is the orchestrator class for `@suaveplan/clipboard`.
|
||||
It composes the navigator + window seams, the FIFO `AsyncQueue`, the
|
||||
typed error registry, the OpenTelemetry observability bundle, the
|
||||
secure-context guard, the permission probe, and the fallback path into a
|
||||
single ergonomic facade. Every public method records counters, the
|
||||
operation latency histogram, and per-code error totals so production
|
||||
deployments observe clipboard health without any caller-side wiring.
|
||||
|
||||
## Features
|
||||
|
||||
- `copyText`, `copyRichContent`, `readText`, `readRichContent` with
|
||||
consistent `ClipboardResult` returns.
|
||||
- `checkPermissions` + `onPermissionChange` for permission lifecycle.
|
||||
- `isSupported`, `isSecure`, `dispose` for environment + lifecycle.
|
||||
- Configurable serialization (`serialize: true` by default).
|
||||
- Configurable secure-context guard (`requireSecureContext`).
|
||||
- Optional sanitizer hook applied to inbound HTML.
|
||||
- Optional pre-built observability bundle.
|
||||
- Focus-aware retry on `NotAllowedError: Document is not focused`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { ClipboardManager } from "@suaveplan/clipboard";
|
||||
|
||||
const clipboard = new ClipboardManager();
|
||||
const result = await clipboard.copyText("hello");
|
||||
if (!result.success) {
|
||||
console.warn("copy failed:", result.code, result.error);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
ClipboardManager,
|
||||
createClipboardObservability,
|
||||
createTagAllowlistSanitizer,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const observability = createClipboardObservability();
|
||||
const clipboard = new ClipboardManager({
|
||||
useFallback: true,
|
||||
requireSecureContext: true,
|
||||
serialize: true,
|
||||
focusRetry: true,
|
||||
sanitizer: createTagAllowlistSanitizer(["p", "b", "i", "br"]),
|
||||
pasteTimeoutMs: 2000,
|
||||
observability,
|
||||
});
|
||||
|
||||
const png = await loadPngBlob();
|
||||
await clipboard.copyRichContent({
|
||||
text: "fallback text",
|
||||
html: "<b>bold</b>",
|
||||
image: png,
|
||||
});
|
||||
|
||||
const unsub = await clipboard.onPermissionChange((snapshot, name) => {
|
||||
console.info("permission change", name, snapshot);
|
||||
});
|
||||
|
||||
// later: unsub(); await clipboard.dispose();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `new ClipboardManager(options?, nav?, win?)`
|
||||
|
||||
Construct a manager. `options` is merged on top of
|
||||
`DEFAULT_CLIPBOARD_OPTIONS`. `nav` and `win` default to the global
|
||||
`navigator` and `window`; either may be supplied as a structural shape
|
||||
for SSR or unit tests.
|
||||
|
||||
### `copyText(text): Promise<ClipboardResult<string>>`
|
||||
|
||||
Copy plain text. Uses the async API when available; falls back to the
|
||||
hidden-textarea path on rejection or absence when `useFallback` is true.
|
||||
Short-circuits with `CLIPBOARD.INSECURE_CONTEXT` when
|
||||
`requireSecureContext` is set and the bound window is insecure.
|
||||
|
||||
### `copyRichContent(data): Promise<ClipboardResult<ClipboardData>>`
|
||||
|
||||
Copy rich content (text / html / image / blobs) via `ClipboardItem`. The
|
||||
fallback path requires `data.text`; rich-only payloads without text
|
||||
short-circuit with `CLIPBOARD.NO_CONTENT`. Empty payloads also surface
|
||||
`CLIPBOARD.NO_CONTENT`.
|
||||
|
||||
### `readText(): Promise<ClipboardResult<string>>`
|
||||
|
||||
Read plain text. Falls back to the paste-event capture when the API is
|
||||
unavailable or rejects (and `useFallback` is true).
|
||||
|
||||
### `readRichContent(): Promise<ClipboardResult<ClipboardData>>`
|
||||
|
||||
Read rich content. Returns `CLIPBOARD.API_UNAVAILABLE` when no async API
|
||||
is present (rich content has no usable fallback). Applies the optional
|
||||
sanitizer to inbound HTML.
|
||||
|
||||
### `checkPermissions(): Promise<PermissionResult>`
|
||||
|
||||
Snapshot of `clipboard-read` + `clipboard-write` permission state.
|
||||
|
||||
### `onPermissionChange(cb): Promise<() => void>`
|
||||
|
||||
Subscribe to permission changes. Returns an unsubscribe function.
|
||||
|
||||
### `isSupported(): boolean`
|
||||
|
||||
True when `navigator.clipboard` is defined on the bound navigator.
|
||||
|
||||
### `isSecure(): boolean`
|
||||
|
||||
True when the bound window does not flag `isSecureContext: false`.
|
||||
|
||||
### `dispose(): Promise<void>`
|
||||
|
||||
Drain the internal queue. After dispose, queued operations resolve with
|
||||
their current outcomes; new enqueues reject.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The manager keeps a single `OperationContext` reference (navigator,
|
||||
window, options, observability) shared with the pure operation functions
|
||||
in `operations.ts`. This split keeps the class under the 300-line file
|
||||
budget while preserving the public API. The internal `run` method wraps
|
||||
every operation in a latency-measuring closure and either enqueues it
|
||||
via the `AsyncQueue` (when `serialize: true`) or invokes it directly. The
|
||||
`recordResult` helper updates counters per-call and per-error-code so
|
||||
the OTel pipeline measures both the macro-shape (copy vs. read) and the
|
||||
micro-shape (which `CLIPBOARD.*` code fired). The constructor is SSR-
|
||||
safe: missing globals collapse to a fully-typed empty navigator and an
|
||||
undefined window so import-time evaluation never throws on a server.
|
||||
113
docs/foundation/data/clipboard/src/manager/operations.md
Normal file
113
docs/foundation/data/clipboard/src/manager/operations.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# `manager/operations.ts`
|
||||
|
||||
## Purpose
|
||||
|
||||
Pure operation functions invoked by `ClipboardManager`. Each function
|
||||
takes an `OperationContext` (navigator, window, options, observability)
|
||||
plus the operation-specific arguments and returns a `ClipboardResult`.
|
||||
Splitting these out of the class keeps `manager.ts` under the file-size
|
||||
budget and makes each path independently testable without instantiating a
|
||||
full manager.
|
||||
|
||||
## Features
|
||||
|
||||
- `runCopyText(ctx, text)` — plain-text copy with secure-context gate,
|
||||
API path, fallback path, and focus-aware retry.
|
||||
- `runCopyRich(ctx, data)` — rich-content copy via `ClipboardItem` with
|
||||
fallback-to-text behaviour and `CLIPBOARD.NO_CONTENT` guards.
|
||||
- `runReadText(ctx)` — plain-text read with paste-event fallback.
|
||||
- `runReadRich(ctx)` — rich-content read with sanitizer pass-through.
|
||||
- Internal `tryApi` wraps every navigator call in a focus-retry-capable
|
||||
envelope.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
runCopyText,
|
||||
type OperationContext,
|
||||
createClipboardObservability,
|
||||
DEFAULT_CLIPBOARD_OPTIONS,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const ctx: OperationContext = {
|
||||
nav: navigator,
|
||||
win: window,
|
||||
options: DEFAULT_CLIPBOARD_OPTIONS,
|
||||
obs: createClipboardObservability(),
|
||||
};
|
||||
const result = await runCopyText(ctx, "hello");
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
runCopyRich,
|
||||
type OperationContext,
|
||||
DEFAULT_CLIPBOARD_OPTIONS,
|
||||
createClipboardObservability,
|
||||
createTagAllowlistSanitizer,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const ctx: OperationContext = {
|
||||
nav: navigator,
|
||||
win: window,
|
||||
options: {
|
||||
...DEFAULT_CLIPBOARD_OPTIONS,
|
||||
sanitizer: createTagAllowlistSanitizer(["p", "b", "i"]),
|
||||
requireSecureContext: true,
|
||||
},
|
||||
obs: createClipboardObservability(),
|
||||
};
|
||||
|
||||
const result = await runCopyRich(ctx, {
|
||||
text: "fallback text",
|
||||
html: "<b>x</b>",
|
||||
image: await fetch("/logo.png").then((r) => r.blob()),
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `OperationContext`
|
||||
|
||||
`{ nav, win, options, obs }`. Lifetime-bound to the manager.
|
||||
|
||||
### `runCopyText(ctx, text)`
|
||||
|
||||
Returns `Promise<ClipboardResult<string>>`. Honours
|
||||
`requireSecureContext`, `useFallback`, `focusRetry`, and the fallback
|
||||
counter wiring.
|
||||
|
||||
### `runCopyRich(ctx, data)`
|
||||
|
||||
Returns `Promise<ClipboardResult<ClipboardData>>`. Builds a single
|
||||
`ClipboardItem` covering every populated MIME slot. Falls back to
|
||||
`runCopyText` when the API rejects and `data.text` is non-empty.
|
||||
|
||||
### `runReadText(ctx)`
|
||||
|
||||
Returns `Promise<ClipboardResult<string>>`. Uses `readTextFallback` from
|
||||
the `fallback/` module when the API is absent or rejects.
|
||||
|
||||
### `runReadRich(ctx)`
|
||||
|
||||
Returns `Promise<ClipboardResult<ClipboardData>>`. Walks every returned
|
||||
`ClipboardItem` and populates `text` / `html` / `image` / `blobs`. The
|
||||
first image wins when multiple are present. The optional sanitizer
|
||||
mutates `html` before returning.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The functions are exported (not module-private) so the test suite can
|
||||
exercise each branch in isolation without driving a full `ClipboardManager`.
|
||||
The `tryApi` helper centralises the focus-retry logic: a single
|
||||
`NotAllowedError` whose message mentions "focus" triggers one
|
||||
`waitForNextFocus` + retry. Any other rejection skips the retry path and
|
||||
lifts the cause via `asClipboardError(CLIPBOARD.READ_FAILED, ...)`. The
|
||||
fallback counter (`clipboard_fallback_used_total`) is incremented with a
|
||||
`reason` attribute (`"api_threw"`, `"no_api"`, `"fallback_failed"`) at the
|
||||
exact transition point so operators can distinguish "fallback because API
|
||||
returned no" from "fallback because API rejected" — a critical signal
|
||||
when diagnosing partial outages.
|
||||
@@ -0,0 +1,95 @@
|
||||
# `observability/observability.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
OpenTelemetry instrumentation bundle for `@suaveplan/clipboard`. Wires
|
||||
four counters and one histogram on a single `Meter` plus a `Tracer`,
|
||||
defaulting to the no-op providers from `@opentelemetry/api` when no
|
||||
real provider is supplied. The `ClipboardManager` records into every
|
||||
instrument across every public method so production deployments measure
|
||||
copy / read volume, fallback usage, error rate by code, and operation
|
||||
latency without any caller-side wiring beyond passing a `Meter`.
|
||||
|
||||
## Features
|
||||
|
||||
- `clipboard_copy_total{method, kind}` — counter, copy attempts.
|
||||
- `clipboard_read_total{method, kind}` — counter, read attempts.
|
||||
- `clipboard_fallback_used_total{reason}` — counter, fallback engagements.
|
||||
- `clipboard_error_total{code}` — counter, errors by `CLIPBOARD.*` code.
|
||||
- `clipboard_op_latency_ms{op, method}` — histogram, operation latency.
|
||||
- Single factory: `createClipboardObservability(opts?)`.
|
||||
- Caller may supply a pre-built bundle to skip instrument creation.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
ClipboardManager,
|
||||
createClipboardObservability,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const observability = createClipboardObservability();
|
||||
const clipboard = new ClipboardManager({ observability });
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
import {
|
||||
ClipboardManager,
|
||||
createClipboardObservability,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const meter = metrics.getMeterProvider().getMeter("my-app", "1.0.0");
|
||||
const observability = createClipboardObservability({ meter });
|
||||
const clipboard = new ClipboardManager({ observability });
|
||||
|
||||
// Wire the meter into your OTel SDK exporter; counters fire on every
|
||||
// copy / read call without extra plumbing.
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `createClipboardObservability(opts?: ClipboardObservabilityOptions): ClipboardObservability`
|
||||
|
||||
Returns a bundle with the four counters and one histogram. Pass through
|
||||
the caller-supplied bundle if `opts.observability` is set; otherwise use
|
||||
the supplied `Meter` and `Tracer` (or the no-op global ones).
|
||||
|
||||
### `ClipboardObservability`
|
||||
|
||||
`{ meter, tracer, counters, histograms }` — strongly typed; counter and
|
||||
histogram entries are OpenTelemetry `Counter` and `Histogram` instances
|
||||
respectively, so consumers can call `.add(n, attrs)` and
|
||||
`.record(ms, attrs)` directly when composing higher-level metrics.
|
||||
|
||||
### `ClipboardCounters`
|
||||
|
||||
- `copyTotal` — fires on every `copyText` / `copyRichContent` call.
|
||||
- `readTotal` — fires on every `readText` / `readRichContent` call.
|
||||
- `fallbackUsedTotal` — fires whenever the fallback path runs.
|
||||
- `errorTotal` — fires once per failure, labeled by error code.
|
||||
|
||||
### `ClipboardHistograms`
|
||||
|
||||
- `opLatencyMs` — milliseconds spent inside each public method, labeled
|
||||
with the `op` name and the `method` (`api` / `fallback`).
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The factory delegates to `@suaveplan/opentelemetry`'s `createCounter` and
|
||||
`createHistogram` helpers so all instruments share the project's
|
||||
canonical OTel wiring (descriptions, units, naming convention). The
|
||||
`Meter` and `Tracer` names are pinned to the package name (`@suaveplan/clipboard`)
|
||||
so observers can filter to clipboard signals without inspecting
|
||||
attributes. When no provider is wired, `metrics.getMeter` returns the
|
||||
no-op meter from `@opentelemetry/api`, so calling `.add` / `.record` on
|
||||
the instruments is safe and cost-free. Tests assert on the instrument
|
||||
shape (callable `add` / `record`) plus the pass-through behaviour of
|
||||
`opts.observability` so consumers can swap implementations without
|
||||
forking the factory.
|
||||
@@ -0,0 +1,91 @@
|
||||
# `permissions/permissions.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Probe and subscribe to clipboard permission state via the browser's
|
||||
Permissions API. `checkPermissions` returns a snapshot of the
|
||||
`clipboard-read` and `clipboard-write` permission states with graceful
|
||||
degradation to `"prompt"` when the API is missing or a query rejects.
|
||||
`onPermissionChange` subscribes a callback to `PermissionStatus.change`
|
||||
events on both descriptors so UI surfaces can reflect real-time grants
|
||||
and revocations without polling.
|
||||
|
||||
## Features
|
||||
|
||||
- `checkPermissions(nav)` — `{ read, write }` snapshot; never throws.
|
||||
- `onPermissionChange(nav, cb)` — subscribe with one callback, unsubscribe
|
||||
with a single returned function.
|
||||
- Tolerant of partial API support: falls back to `PermissionStatus.onchange`
|
||||
when `addEventListener` is unavailable.
|
||||
- Tolerant of permission-query rejection: skips that slot without
|
||||
re-throwing.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { checkPermissions } from "@suaveplan/clipboard";
|
||||
|
||||
const { read, write } = await checkPermissions(navigator);
|
||||
if (read === "denied" || write === "denied") {
|
||||
showFallbackUi();
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { onPermissionChange } from "@suaveplan/clipboard";
|
||||
|
||||
const unsubscribe = await onPermissionChange(
|
||||
navigator,
|
||||
(snapshot, descriptor) => {
|
||||
console.info(
|
||||
"[clipboard] permission change",
|
||||
descriptor,
|
||||
snapshot,
|
||||
);
|
||||
if (snapshot.read === "granted" && snapshot.write === "granted") {
|
||||
enableCopyButtons();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
// later — on unmount / dispose
|
||||
unsubscribe();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `checkPermissions(nav: ClipboardNavigator): Promise<PermissionResult>`
|
||||
|
||||
Resolves with `{ read, write }`. Each slot is one of `"granted"`,
|
||||
`"denied"`, `"prompt"`. Returns `{ read: "prompt", write: "prompt" }`
|
||||
when `nav.permissions` is undefined, and `"prompt"` for any individual
|
||||
descriptor whose `query()` rejects or returns an unknown state.
|
||||
|
||||
### `onPermissionChange(nav, cb): Promise<() => void>`
|
||||
|
||||
Subscribes `cb` to permission changes on both `clipboard-read` and
|
||||
`clipboard-write`. Returns an unsubscribe function. The callback receives
|
||||
the latest `PermissionResult` snapshot plus the descriptor name whose
|
||||
state changed. The change handler internally re-runs `checkPermissions`
|
||||
so both `read` and `write` are always up to date on every callback.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The Permissions API exposes two listener mechanisms — `addEventListener`
|
||||
and `onchange`. Some platforms (older WebKit / mobile) support only one,
|
||||
so the module attaches via `addEventListener` when available and falls
|
||||
back to assigning `onchange` otherwise. The unsubscribe function clears
|
||||
the corresponding mechanism for each subscription. The
|
||||
`checkPermissions` helper normalises unknown state strings to `"prompt"`
|
||||
so callers never have to defend against future spec values. Query
|
||||
rejection is treated as `"prompt"` rather than `"denied"` because a
|
||||
rejection typically signals "API not supported" rather than "user
|
||||
explicitly refused" — pessimistic mapping would block correct callers in
|
||||
older browsers. Tests cover both `addEventListener` and `onchange` paths,
|
||||
plus the unsubscribe contract.
|
||||
96
docs/foundation/data/clipboard/src/queue/queue.md
Normal file
96
docs/foundation/data/clipboard/src/queue/queue.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# `queue/queue.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`AsyncQueue` is a minimal FIFO asynchronous queue that serializes
|
||||
operations on a single tail promise. The `ClipboardManager` uses it to
|
||||
ensure concurrent `copyText` / `readText` / `copyRichContent` /
|
||||
`readRichContent` calls execute in arrival order — without it, the
|
||||
trailing call would win non-deterministically because the browser's
|
||||
Clipboard API does not provide ordering guarantees across overlapping
|
||||
requests. The queue is intentionally tiny (one promise reference plus a
|
||||
drained flag) so it adds no measurable overhead in the common single-call
|
||||
case.
|
||||
|
||||
## Features
|
||||
|
||||
- FIFO ordering — callbacks run strictly in arrival order.
|
||||
- Rejection-tolerant — a rejected operation does not stall later ones.
|
||||
- `drain()` closes the queue and resolves once the tail settles.
|
||||
- `reset()` re-opens a drained queue (useful for tests + dispose flows).
|
||||
- Zero dependencies; no `Promise.race`, no timers, no microtask leaks.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { AsyncQueue } from "@suaveplan/clipboard";
|
||||
|
||||
const queue = new AsyncQueue();
|
||||
const results = await Promise.all([
|
||||
queue.enqueue(async () => "a"),
|
||||
queue.enqueue(async () => "b"),
|
||||
queue.enqueue(async () => "c"),
|
||||
]);
|
||||
// results === ["a", "b", "c"]
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { AsyncQueue } from "@suaveplan/clipboard";
|
||||
|
||||
const queue = new AsyncQueue();
|
||||
|
||||
// Rejection does not break the queue.
|
||||
queue.enqueue(async () => {
|
||||
throw new Error("boom");
|
||||
}).catch(() => undefined);
|
||||
const next = await queue.enqueue(async () => "still works");
|
||||
|
||||
// Close the queue when shutting down.
|
||||
await queue.drain();
|
||||
|
||||
// Re-open later for a new session.
|
||||
queue.reset();
|
||||
await queue.enqueue(async () => "round two");
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `new AsyncQueue()`
|
||||
|
||||
Constructs a queue with an already-resolved tail.
|
||||
|
||||
### `queue.enqueue<T>(fn: () => Promise<T>): Promise<T>`
|
||||
|
||||
Append `fn` to the queue. The returned promise settles with `fn`'s
|
||||
outcome once every previously enqueued callback has settled. If the
|
||||
queue has been drained, returns a rejected promise carrying `"AsyncQueue:
|
||||
enqueue after drain"`.
|
||||
|
||||
### `queue.drain(): Promise<void>`
|
||||
|
||||
Closes the queue. Subsequent `enqueue` calls reject. Returns a promise
|
||||
that resolves once the current tail settles (success or failure).
|
||||
|
||||
### `queue.reset(): void`
|
||||
|
||||
Re-opens a drained queue. Replaces the tail with a fresh
|
||||
`Promise.resolve()`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The tail promise chains through `then(onFulfilled, onRejected)` with both
|
||||
arms re-invoking `fn`. This means an operation always runs after the
|
||||
previous one regardless of how the previous resolved — the rejection does
|
||||
not propagate into the queue's tracked tail (we explicitly catch via
|
||||
`.then(() => undefined, () => undefined)`). Because the tail is `unknown`-
|
||||
typed internally, the generic `T` parameter on `enqueue` is preserved end
|
||||
to end. The drained flag is checked synchronously inside `enqueue` so
|
||||
post-drain calls reject without scheduling work. The queue is private to
|
||||
the manager but exported so advanced consumers can reuse it for
|
||||
caller-defined ordering (e.g. when composing with other browser APIs).
|
||||
88
docs/foundation/data/clipboard/src/sanitizer/sanitizer.md
Normal file
88
docs/foundation/data/clipboard/src/sanitizer/sanitizer.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# `sanitizer/sanitizer.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Sanitizer hooks applied to inbound `text/html` clipboard payloads. The
|
||||
package ships an identity sanitizer (pass-through) plus a regex-based
|
||||
tag-allowlist sanitizer for stripping known-dangerous tags from
|
||||
untrusted clipboard HTML. Both adhere to the `ClipboardSanitizer` shape
|
||||
`(html: string) => string` so consumers can plug in DOMPurify, sanitize-html,
|
||||
or any other library without changing call sites. The package
|
||||
deliberately does not bundle a security-grade sanitizer to avoid pinning
|
||||
consumers to a specific implementation — `createTagAllowlistSanitizer` is
|
||||
suitable for hardening trusted-origin clipboard flows and is NOT a
|
||||
substitute for DOMPurify in security-critical paths.
|
||||
|
||||
## Features
|
||||
|
||||
- `identitySanitizer` — pass-through, used when no sanitizer is supplied.
|
||||
- `createTagAllowlistSanitizer(tags)` — regex-based stripper that drops
|
||||
any tag not in the supplied allowlist while preserving allowed tags
|
||||
with their attributes intact.
|
||||
- Case-insensitive tag matching.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
ClipboardManager,
|
||||
createTagAllowlistSanitizer,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const sanitize = createTagAllowlistSanitizer(["p", "b", "i", "br"]);
|
||||
const clipboard = new ClipboardManager({ sanitizer: sanitize });
|
||||
const result = await clipboard.readRichContent();
|
||||
// result.data.html no longer contains <script> or <iframe>.
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import DOMPurify from "dompurify";
|
||||
import { ClipboardManager } from "@suaveplan/clipboard";
|
||||
|
||||
const clipboard = new ClipboardManager({
|
||||
sanitizer: (html) =>
|
||||
DOMPurify.sanitize(html, { ALLOWED_TAGS: ["p", "a", "b", "i"] }),
|
||||
});
|
||||
|
||||
const result = await clipboard.readRichContent();
|
||||
if (result.success) {
|
||||
const cleaned = result.data?.html ?? "";
|
||||
// Pass `cleaned` to a downstream React component or DOM renderer
|
||||
// that knows how to safely interpolate sanitized HTML.
|
||||
return cleaned;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `identitySanitizer(html: string): string`
|
||||
|
||||
Pass-through; returns input unchanged. Used internally as the default
|
||||
sanitizer when no override is supplied.
|
||||
|
||||
### `createTagAllowlistSanitizer(allowed: readonly string[]): ClipboardSanitizer`
|
||||
|
||||
Returns a sanitizer that strips every HTML tag not present in `allowed`.
|
||||
Tag matching is case-insensitive; opening and closing tags are stripped
|
||||
together. Attributes on allowed tags are preserved. The function only
|
||||
removes the angle-bracketed tag itself — any content nested inside
|
||||
disallowed tags is left in place. For full DOM-level sanitization use a
|
||||
real library such as DOMPurify.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The regex used is intentionally conservative — it matches
|
||||
`<[a-zA-Z][a-zA-Z0-9-]*` to capture tag names and tolerates optional
|
||||
attribute payloads via `(\s[^>]*)?`. It does not attempt to handle CDATA,
|
||||
comments, or processing instructions. This matches the documented scope:
|
||||
trusted-origin clipboard hardening, not arbitrary-source HTML
|
||||
sanitization. The `Set` of lowercased allowed tags keeps lookup at O(1).
|
||||
The module is browser-tier but has no DOM dependency, so it also works
|
||||
inside service workers and Node-side SSR consumers that happen to
|
||||
process clipboard payloads on the server.
|
||||
105
docs/foundation/data/clipboard/src/security/security.md
Normal file
105
docs/foundation/data/clipboard/src/security/security.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# `security/security.ts`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Browser-side security helpers that guard clipboard operations against
|
||||
insecure contexts and orchestrate the focus-aware retry path. The
|
||||
Clipboard API is gated by the browser on `window.isSecureContext` — calls
|
||||
from non-HTTPS origins reject with `NotAllowedError`, and the spec
|
||||
explicitly forbids the API outside secure contexts. This module exposes a
|
||||
typed `requireSecureContext` guard plus an SSR-safe `isSecure` predicate
|
||||
so call sites surface `CLIPBOARD.INSECURE_CONTEXT` rather than a generic
|
||||
DOMException. The companion `waitForNextFocus` helper resolves on the next
|
||||
`focus` event (with a timeout) so the manager can recover from
|
||||
`NotAllowedError: Document is not focused`.
|
||||
|
||||
## Features
|
||||
|
||||
- `isSecure(window?)` — SSR-safe predicate. Returns `true` outside the
|
||||
browser so universal code paths short-circuit cleanly.
|
||||
- `requireSecureContext(window?)` — throws
|
||||
`ClipboardError(CLIPBOARD.INSECURE_CONTEXT)` when the window flags
|
||||
`isSecureContext === false`.
|
||||
- `waitForNextFocus(window?, timeoutMs)` — resolves `true` on the next
|
||||
`focus` event, `false` on timeout, `false` when the window cannot
|
||||
attach listeners.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { requireSecureContext } from "@suaveplan/clipboard";
|
||||
|
||||
requireSecureContext(); // throws CLIPBOARD.INSECURE_CONTEXT on HTTP
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
CLIPBOARD_CODES,
|
||||
ClipboardError,
|
||||
isSecure,
|
||||
waitForNextFocus,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
async function copyWithFocusRetry(text: string): Promise<void> {
|
||||
if (!isSecure()) {
|
||||
throw new ClipboardError(
|
||||
CLIPBOARD_CODES.INSECURE_CONTEXT,
|
||||
"Refusing clipboard write outside HTTPS.",
|
||||
);
|
||||
}
|
||||
try {
|
||||
await navigator.clipboard.writeText(text);
|
||||
} catch (err) {
|
||||
if (
|
||||
err instanceof Error &&
|
||||
err.name === "NotAllowedError" &&
|
||||
err.message.includes("focus")
|
||||
) {
|
||||
const focused = await waitForNextFocus(window, 500);
|
||||
if (focused) {
|
||||
await navigator.clipboard.writeText(text);
|
||||
return;
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `isSecure(window?: ClipboardWindowLike): boolean`
|
||||
|
||||
Returns `true` when the supplied window does not declare
|
||||
`isSecureContext: false`. Returns `true` when no window is supplied and
|
||||
no global `window` exists (SSR safe).
|
||||
|
||||
### `requireSecureContext(window?: ClipboardWindowLike): void`
|
||||
|
||||
Throws `ClipboardError(CLIPBOARD.INSECURE_CONTEXT)` when `isSecure`
|
||||
returns false. No-op otherwise.
|
||||
|
||||
### `waitForNextFocus(window?: ClipboardWindowLike, timeoutMs: number): Promise<boolean>`
|
||||
|
||||
Subscribes to the next `focus` event on the supplied window (or the
|
||||
global) and resolves `true` when the event fires. Resolves `false` if the
|
||||
window does not expose `addEventListener` or if the timeout elapses
|
||||
first. Internally guards against duplicate resolves with a `settled`
|
||||
flag.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The module accepts a `ClipboardWindowLike` shape rather than the DOM `Window`
|
||||
type so tests + SSR consumers can inject minimal shims. The focus listener
|
||||
is registered with `{ once: true }` so it removes itself on first
|
||||
delivery; the timeout path additionally calls `removeEventListener`
|
||||
defensively in case the host environment ignores `{ once: true }`. The
|
||||
`settled` flag prevents the late-delivered focus event from resolving the
|
||||
promise after the timeout has already fired — a real-world race that the
|
||||
unit tests assert on.
|
||||
96
docs/foundation/data/clipboard/src/types/clipboard.types.md
Normal file
96
docs/foundation/data/clipboard/src/types/clipboard.types.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# `types/clipboard.types.ts`
|
||||
|
||||
## Purpose
|
||||
|
||||
Shared type definitions for `@suaveplan/clipboard`. Houses every
|
||||
public-surface interface, type alias, and the `DEFAULT_CLIPBOARD_OPTIONS`
|
||||
runtime constant. Splitting types into a flat `types/` directory keeps
|
||||
the implementation modules focused on behaviour while every consumer can
|
||||
import a single canonical shape — preventing the historical drift where
|
||||
`ClipboardResult` ended up declared three times across the codebase.
|
||||
|
||||
## Features
|
||||
|
||||
- `ClipboardPermissionState` — `"granted" | "denied" | "prompt"`.
|
||||
- `ClipboardBlobMap` — readonly MIME → Blob map for arbitrary binary
|
||||
clipboard payloads.
|
||||
- `ClipboardSanitizer` — `(html: string) => string` hook applied to
|
||||
inbound rich HTML.
|
||||
- `PermissionChangeListener` — callback shape for
|
||||
`ClipboardManager.onPermissionChange`.
|
||||
- `ClipboardData` — `{ text?, html?, image?, blobs? }`.
|
||||
- `ClipboardOptions` — full options bag accepted by `ClipboardManager`.
|
||||
- `ClipboardResult<T>` — `{ success, data?, error?, code?, method }`.
|
||||
- `PermissionResult` — `{ read, write }` snapshot.
|
||||
- `PermissionStatusLike` — structural view of `PermissionStatus`.
|
||||
- `ClipboardNavigator` — structural view of `navigator.clipboard` +
|
||||
`navigator.permissions`.
|
||||
- `ClipboardWindowLike` — structural view of `window` for
|
||||
secure-context detection and focus-retry orchestration.
|
||||
- `DEFAULT_CLIPBOARD_OPTIONS` — frozen options default.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
DEFAULT_CLIPBOARD_OPTIONS,
|
||||
type ClipboardData,
|
||||
type ClipboardOptions,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
const opts: Partial<ClipboardOptions> = {
|
||||
...DEFAULT_CLIPBOARD_OPTIONS,
|
||||
useFallback: false,
|
||||
};
|
||||
|
||||
const data: ClipboardData = {
|
||||
text: "hello",
|
||||
html: "<b>hi</b>",
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import type {
|
||||
ClipboardNavigator,
|
||||
ClipboardResult,
|
||||
ClipboardWindowLike,
|
||||
PermissionChangeListener,
|
||||
} from "@suaveplan/clipboard";
|
||||
|
||||
// Compose a server-side shim used by integration tests:
|
||||
const ssrNav: ClipboardNavigator = {};
|
||||
const ssrWin: ClipboardWindowLike = { isSecureContext: true };
|
||||
|
||||
function describe(result: ClipboardResult<string>): string {
|
||||
return result.success
|
||||
? `[${result.method}] copied "${result.data}"`
|
||||
: `[${result.method}] error ${result.code}: ${result.error}`;
|
||||
}
|
||||
|
||||
const listener: PermissionChangeListener = (snapshot, name) => {
|
||||
console.info("change", name, snapshot);
|
||||
};
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
Every exported name is a TypeScript interface or type alias — see the
|
||||
listing in **Features**. Runtime exports: `DEFAULT_CLIPBOARD_OPTIONS`
|
||||
(frozen `ClipboardOptions` with `useFallback: true`,
|
||||
`requireSecureContext: false`, `serialize: true`, `focusRetry: true`,
|
||||
`sanitizer: undefined`, `pasteTimeoutMs: 1500`, `permissionTimeout: 5000`,
|
||||
`observability: undefined`).
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The structural-view interfaces (`ClipboardNavigator`,
|
||||
`ClipboardWindowLike`, `PermissionStatusLike`) are intentionally minimal
|
||||
— they expose only the fields the package actually reads. This keeps the
|
||||
DOM-typing surface narrow enough to substitute without depending on
|
||||
`@types/dom`, and lets the package run under SSR shims that don't carry
|
||||
a full DOM type pack. The `Readonly` modifiers across every option and
|
||||
result field communicate that the API never mutates caller-supplied data;
|
||||
the freeze on `DEFAULT_CLIPBOARD_OPTIONS` enforces that at runtime so
|
||||
shared references can't be accidentally mutated.
|
||||
128
docs/foundation/data/data-masking/README.md
Normal file
128
docs/foundation/data/data-masking/README.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# @suaveplan/data-masking - PII/sensitive data masking for objects and strings: field-level masking rules, nested object traversal, format-preserving masking, tokenization, and FNV-1a hashing.
|
||||
|
||||
## Summary
|
||||
|
||||
PII/sensitive data masking for objects and strings with field-level rules,
|
||||
nested traversal, eight masking strategies, and OpenTelemetry instrumentation.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [object-masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [data-masking-error](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [schemas](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [mask-rule.schema](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [credit-card](./src/strategies/credit-card.md) — The `credit-card` masking strategy produces the PCI-DSS-compatible truncation format for payment card numbers: last 4 d…
|
||||
- [email](./src/strategies/email.md) — The `email` masking strategy anonymises email addresses while keeping the output immediately recognisable as an email a…
|
||||
- [full](./src/strategies/full.md) — The `full` masking strategy completely replaces an input string with a configurable replacement sentinel. It is the mos…
|
||||
- [hash](./src/strategies/hash.md) — The `hash` masking strategy produces a deterministic, fixed-length 8-character hexadecimal token from any input string…
|
||||
- [partial](./src/strategies/partial.md) — The `partial` masking strategy preserves a configurable number of characters at the start and end of an input string wh…
|
||||
- [phone](./src/strategies/phone.md) — The `phone` masking strategy anonymises phone numbers while preserving their formatting separators, making masked value…
|
||||
- [preserve-format](./src/strategies/preserve-format.md) — The `preserve-format` masking strategy replaces every alphanumeric character with a placeholder of the same character c…
|
||||
- [tokenize](./src/strategies/tokenize.md) — The `tokenize` masking strategy replaces a sensitive value with a structured opaque token of the form `TOKEN_<STRATEGY>…
|
||||
- [types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [mask-types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [utils](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [path-matcher](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [object-masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [data-masking-error](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-rule.schema](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [credit-card](./src/strategies/credit-card.md) — The `credit-card` masking strategy produces the PCI-DSS-compatible truncation format for payment card numbers: last 4 d…
|
||||
- [email](./src/strategies/email.md) — The `email` masking strategy anonymises email addresses while keeping the output immediately recognisable as an email a…
|
||||
- [full](./src/strategies/full.md) — The `full` masking strategy completely replaces an input string with a configurable replacement sentinel. It is the mos…
|
||||
- [hash](./src/strategies/hash.md) — The `hash` masking strategy produces a deterministic, fixed-length 8-character hexadecimal token from any input string…
|
||||
- [partial](./src/strategies/partial.md) — The `partial` masking strategy preserves a configurable number of characters at the start and end of an input string wh…
|
||||
- [phone](./src/strategies/phone.md) — The `phone` masking strategy anonymises phone numbers while preserving their formatting separators, making masked value…
|
||||
- [preserve-format](./src/strategies/preserve-format.md) — The `preserve-format` masking strategy replaces every alphanumeric character with a placeholder of the same character c…
|
||||
- [tokenize](./src/strategies/tokenize.md) — The `tokenize` masking strategy replaces a sensitive value with a structured opaque token of the form `TOKEN_<STRATEGY>…
|
||||
- [types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [path-matcher](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [object-masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [data-masking-error](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-rule.schema](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [credit-card](./src/strategies/credit-card.md) — The `credit-card` masking strategy produces the PCI-DSS-compatible truncation format for payment card numbers: last 4 d…
|
||||
- [email](./src/strategies/email.md) — The `email` masking strategy anonymises email addresses while keeping the output immediately recognisable as an email a…
|
||||
- [full](./src/strategies/full.md) — The `full` masking strategy completely replaces an input string with a configurable replacement sentinel. It is the mos…
|
||||
- [hash](./src/strategies/hash.md) — The `hash` masking strategy produces a deterministic, fixed-length 8-character hexadecimal token from any input string…
|
||||
- [partial](./src/strategies/partial.md) — The `partial` masking strategy preserves a configurable number of characters at the start and end of an input string wh…
|
||||
- [phone](./src/strategies/phone.md) — The `phone` masking strategy anonymises phone numbers while preserving their formatting separators, making masked value…
|
||||
- [preserve-format](./src/strategies/preserve-format.md) — The `preserve-format` masking strategy replaces every alphanumeric character with a placeholder of the same character c…
|
||||
- [tokenize](./src/strategies/tokenize.md) — The `tokenize` masking strategy replaces a sensitive value with a structured opaque token of the form `TOKEN_<STRATEGY>…
|
||||
- [types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [path-matcher](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [object-masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [data-masking-error](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-rule.schema](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [credit-card](./src/strategies/credit-card.md) — The `credit-card` masking strategy produces the PCI-DSS-compatible truncation format for payment card numbers: last 4 d…
|
||||
- [email](./src/strategies/email.md) — The `email` masking strategy anonymises email addresses while keeping the output immediately recognisable as an email a…
|
||||
- [full](./src/strategies/full.md) — The `full` masking strategy completely replaces an input string with a configurable replacement sentinel. It is the mos…
|
||||
- [hash](./src/strategies/hash.md) — The `hash` masking strategy produces a deterministic, fixed-length 8-character hexadecimal token from any input string…
|
||||
- [partial](./src/strategies/partial.md) — The `partial` masking strategy preserves a configurable number of characters at the start and end of an input string wh…
|
||||
- [phone](./src/strategies/phone.md) — The `phone` masking strategy anonymises phone numbers while preserving their formatting separators, making masked value…
|
||||
- [preserve-format](./src/strategies/preserve-format.md) — The `preserve-format` masking strategy replaces every alphanumeric character with a placeholder of the same character c…
|
||||
- [tokenize](./src/strategies/tokenize.md) — The `tokenize` masking strategy replaces a sensitive value with a structured opaque token of the form `TOKEN_<STRATEGY>…
|
||||
- [types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [mask-types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [path-matcher](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_- [object-masker](./src/core/masker/object-masker.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_ - [data-masking-error](./src/errors/data-masking-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ - [mask-rule.schema](./src/schemas/mask-rule.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [credit-card](./src/strategies/credit-card.md) — The `credit-card` masking strategy produces the PCI-DSS-compatible truncation format for payment card numbers: last 4 d…
|
||||
- [email](./src/strategies/email.md) — The `email` masking strategy anonymises email addresses while keeping the output immediately recognisable as an email a…
|
||||
- [full](./src/strategies/full.md) — The `full` masking strategy completely replaces an input string with a configurable replacement sentinel. It is the mos…
|
||||
- [hash](./src/strategies/hash.md) — The `hash` masking strategy produces a deterministic, fixed-length 8-character hexadecimal token from any input string…
|
||||
- [partial](./src/strategies/partial.md) — The `partial` masking strategy preserves a configurable number of characters at the start and end of an input string wh…
|
||||
- [phone](./src/strategies/phone.md) — The `phone` masking strategy anonymises phone numbers while preserving their formatting separators, making masked value…
|
||||
- [preserve-format](./src/strategies/preserve-format.md) — The `preserve-format` masking strategy replaces every alphanumeric character with a placeholder of the same character c…
|
||||
- [tokenize](./src/strategies/tokenize.md) — The `tokenize` masking strategy replaces a sensitive value with a structured opaque token of the form `TOKEN_<STRATEGY>…
|
||||
- [types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_ - [mask-types](./src/types/mask-types.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_ - [path-matcher](./src/utils/path-matcher.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/data-masking
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```ts
|
||||
import { maskObject } from '@suaveplan/data-masking';
|
||||
|
||||
const result = maskObject(
|
||||
{
|
||||
user: { name: 'John Doe', email: 'john@example.com', ssn: '123-45-6789' },
|
||||
payment: { cardNumber: '4111-1111-1111-1111', cvv: '123' },
|
||||
},
|
||||
{
|
||||
rules: [
|
||||
{ fields: ['user.email'], strategy: 'email' },
|
||||
{ fields: ['user.ssn', 'payment.cvv'], strategy: 'full' },
|
||||
{ fields: ['payment.cardNumber'], strategy: 'credit-card' },
|
||||
],
|
||||
},
|
||||
);
|
||||
|
||||
console.log(result.masked.user.email); // 'j***@e***.com'
|
||||
console.log(result.masked.payment.cardNumber); // '****-****-****-1111'
|
||||
console.log(result.fieldsAffected); // ['payment.cardNumber', 'payment.cvv', 'user.email', 'user.ssn']
|
||||
```
|
||||
|
||||
## Strategies
|
||||
|
||||
| Strategy | Description | Example output |
|
||||
|-------------------|------------------------------------------|-------------------------|
|
||||
| `full` | Replace entire value with `***` | `***` |
|
||||
| `partial` | Keep first/last N chars, mask middle | `sk_***...****789` |
|
||||
| `email` | Format-preserving email mask | `j***@e***.com` |
|
||||
| `phone` | Keep last 4 digits | `***-***-1234` |
|
||||
| `credit-card` | Keep last 4 digits | `****-****-****-1234` |
|
||||
| `hash` | FNV-1a 32-bit hex (deterministic) | `4f9f2cab` |
|
||||
| `tokenize` | Opaque token (not reversible) | `TOKEN_TOKENIZE_0` |
|
||||
| `preserve-format` | Replace alnum with type placeholder | `XXXXX000` |
|
||||
|
||||
## Field Path Patterns
|
||||
|
||||
Rules use dot-notation paths with optional glob wildcards:
|
||||
|
||||
```ts
|
||||
{ fields: ['user.email'] } // exact match
|
||||
{ fields: ['user.*'] } // single-segment wildcard
|
||||
{ fields: ['user.**'] } // multi-segment wildcard (any depth)
|
||||
{ fields: ['**.email'] } // any path ending in .email
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
| Subpath | Contents |
|
||||
|---------------------|--------------------------------------------|
|
||||
| `@suaveplan/data-masking` | Main exports (maskObject, maskString, maskValue, types, schemas) |
|
||||
| `@suaveplan/data-masking/masker` | [object-masker](src/core/masker/object-masker.md) — core traversal engine |
|
||||
| `@suaveplan/data-masking/strategies` | [full](src/strategies/full.md), [partial](src/strategies/partial.md), [email](src/strategies/email.md), [phone](src/strategies/phone.md), [credit-card](src/strategies/credit-card.md), [hash](src/strategies/hash.md), [tokenize](src/strategies/tokenize.md), [preserve-format](src/strategies/preserve-format.md) |
|
||||
| `@suaveplan/data-masking/errors` | [DataMaskingError](src/errors/data-masking-error.md), `DATA_MASKING_CODES` |
|
||||
| `@suaveplan/data-masking/schemas` | Zod schemas for `MaskRule`, `MaskOptions` |
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,128 @@
|
||||
# object-masker
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
The `object-masker` module is the core engine of `@suaveplan/data-masking`. It
|
||||
provides three exports: `maskValue` for dispatching a masking strategy to a
|
||||
single string, `maskObject` for traversing and masking a nested plain-object
|
||||
graph, and `maskString` for detecting and masking embedded PII patterns inside
|
||||
free-form text. All traversal is non-destructive — the input is never mutated;
|
||||
a deep structural clone is produced instead.
|
||||
|
||||
OpenTelemetry instrumentation records a counter per strategy (`data_masking.mask.total`)
|
||||
and a histogram of overall `maskObject` call duration (`data_masking.mask.duration_ms`),
|
||||
enabling SREs to monitor masking throughput and latency in production.
|
||||
|
||||
## Features
|
||||
|
||||
- `maskValue`: uniform dispatch to all eight masking strategies.
|
||||
- `maskObject`: deep-clone traversal with `deep` and `maskArrays` controls.
|
||||
- `maskString`: regex-based PII detection for email, phone, and credit-card
|
||||
patterns embedded in free-form strings.
|
||||
- Non-mutating: original object/string always preserved.
|
||||
- OTel counter + histogram on every operation.
|
||||
- Sorted `fieldsAffected` list for deterministic output.
|
||||
- `DataMaskingError` on strategy dispatch failure.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskObject } from '@suaveplan/data-masking/masker';
|
||||
|
||||
const result = maskObject(
|
||||
{ user: { name: 'Alice', email: 'alice@example.com' } },
|
||||
{ rules: [{ fields: ['user.email'], strategy: 'email' }] },
|
||||
);
|
||||
// result.masked.user.email → 'a***@e***.com'
|
||||
// result.fieldsAffected → ['user.email']
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskObject, maskString, maskValue } from '@suaveplan/data-masking/masker';
|
||||
|
||||
// Full object masking with wildcards
|
||||
const data = {
|
||||
user: { name: 'John Doe', email: 'john@example.com', ssn: '123-45-6789' },
|
||||
payment: { cardNumber: '4111-1111-1111-1111', cvv: '123' },
|
||||
};
|
||||
|
||||
const result = maskObject(data, {
|
||||
rules: [
|
||||
{ fields: ['user.email'], strategy: 'email' },
|
||||
{ fields: ['user.ssn', 'payment.cvv'], strategy: 'full' },
|
||||
{ fields: ['payment.cardNumber'], strategy: 'credit-card' },
|
||||
],
|
||||
deep: true,
|
||||
maskArrays: true,
|
||||
});
|
||||
// result.masked.user.email → 'j***@e***.com'
|
||||
// result.masked.user.ssn → '***'
|
||||
// result.masked.payment.cardNumber → '****-****-****-1111'
|
||||
// result.masked.payment.cvv → '***'
|
||||
// result.fieldsAffected.length → 4
|
||||
|
||||
// Free-form string masking
|
||||
const log = 'User john@example.com called from +1-555-123-4567';
|
||||
const masked = maskString(log, [
|
||||
{ fields: ['*'], strategy: 'email' },
|
||||
{ fields: ['*'], strategy: 'phone' },
|
||||
]);
|
||||
// masked → 'User j***@e***.com called from *-***-***-4567'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskValue(value, strategy, rule)`
|
||||
|
||||
```ts
|
||||
function maskValue(value: string, strategy: MaskStrategy, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
Dispatches `value` to the named strategy. Wraps any thrown error in a
|
||||
`DataMaskingError` with code `DATA_MASKING.MASK_FAILED`.
|
||||
|
||||
### `maskObject<T>(obj, options)`
|
||||
|
||||
```ts
|
||||
function maskObject<T extends Record<string, unknown>>(
|
||||
obj: T,
|
||||
options: MaskOptions,
|
||||
): MaskResult<T>;
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|------------------|---------------|---------|------------------------------------|
|
||||
| `obj` | `T` | — | Source plain object. |
|
||||
| `options.rules` | `MaskRule[]` | — | Ordered masking rules. |
|
||||
| `options.deep` | `boolean` | `true` | Recurse into nested objects. |
|
||||
| `options.maskArrays` | `boolean` | `true` | Mask array element values. |
|
||||
|
||||
**Returns** `MaskResult<T>` with `masked` clone and sorted `fieldsAffected`.
|
||||
|
||||
### `maskString(str, rules)`
|
||||
|
||||
```ts
|
||||
function maskString(str: string, rules: MaskRule[]): string;
|
||||
```
|
||||
|
||||
Scans `str` for embedded credit-card, email, and phone patterns and replaces
|
||||
each detected match using the first rule with a matching strategy.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Object traversal uses plain `for...of` over `Object.keys` to avoid
|
||||
prototype-chain pollution; `isPlainObject` guards against class instances,
|
||||
`Date`, `RegExp`, etc., which are returned as-is.
|
||||
- Array element paths use bracket notation `field[0]` to distinguish them
|
||||
from nested object paths in the `fieldsAffected` list.
|
||||
- The OTel `Meter` is obtained lazily on first call and cached module-wide to
|
||||
avoid re-creating metric instruments on each invocation.
|
||||
- `performance.now()` is used for duration measurement (available in Node.js
|
||||
16+ and Bun natively; no polyfill required for the suaveplan target).
|
||||
- `maskString` applies rules in category order (credit-card → email → phone)
|
||||
to avoid double-masking where patterns overlap.
|
||||
@@ -0,0 +1,120 @@
|
||||
# data-masking-error
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides a stable error-code registry and a typed `DataMaskingError` class for
|
||||
all failures that originate inside `@suaveplan/data-masking`. Keeping errors
|
||||
isolated to a single file makes it easy for callers to perform narrow
|
||||
`instanceof` checks and to discriminate specific failure modes via the `code`
|
||||
property without resorting to fragile message-string matching.
|
||||
|
||||
The package is intentionally a leaf node — it carries zero `@suaveplan/*`
|
||||
runtime dependencies — so `DataMaskingError` extends the native `Error` class
|
||||
directly rather than `SuaveplanError`.
|
||||
|
||||
## Features
|
||||
|
||||
- `DATA_MASKING_CODES` constant object: two stable string codes exported as
|
||||
`as const` for full TypeScript narrowing.
|
||||
- `DataMaskingCode` union type derived from `DATA_MASKING_CODES`.
|
||||
- `DataMaskingError` class: extends `Error`, sets `name = "DataMaskingError"`,
|
||||
exposes a readonly `code: DataMaskingCode` property.
|
||||
- Tree-shakeable — no side-effects, compatible with `"sideEffects": false`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
DataMaskingError,
|
||||
DATA_MASKING_CODES,
|
||||
} from '@suaveplan/data-masking/errors';
|
||||
|
||||
try {
|
||||
throw new DataMaskingError(
|
||||
'Strategy "unknown" is not recognised',
|
||||
DATA_MASKING_CODES.INVALID_RULE,
|
||||
);
|
||||
} catch (err) {
|
||||
if (err instanceof DataMaskingError) {
|
||||
console.error(err.code); // 'DATA_MASKING.INVALID_RULE'
|
||||
console.error(err.name); // 'DataMaskingError'
|
||||
console.error(err.message);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
DataMaskingError,
|
||||
DATA_MASKING_CODES,
|
||||
type DataMaskingCode,
|
||||
} from '@suaveplan/data-masking/errors';
|
||||
|
||||
function handleMaskingError(err: unknown): void {
|
||||
if (!(err instanceof DataMaskingError)) throw err;
|
||||
|
||||
const actions: Record<DataMaskingCode, () => void> = {
|
||||
[DATA_MASKING_CODES.INVALID_RULE]: () =>
|
||||
console.warn('Rule validation failed — check your MaskRule definitions'),
|
||||
[DATA_MASKING_CODES.MASK_FAILED]: () =>
|
||||
console.error('Masking strategy threw during execution'),
|
||||
};
|
||||
|
||||
actions[err.code]?.();
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DATA_MASKING_CODES`
|
||||
|
||||
```ts
|
||||
const DATA_MASKING_CODES: {
|
||||
readonly INVALID_RULE: 'DATA_MASKING.INVALID_RULE';
|
||||
readonly MASK_FAILED: 'DATA_MASKING.MASK_FAILED';
|
||||
};
|
||||
```
|
||||
|
||||
Stable string constants that identify each error category. Values are
|
||||
intentionally namespaced with `DATA_MASKING.` to avoid collisions in shared
|
||||
error-monitoring pipelines.
|
||||
|
||||
### `DataMaskingCode`
|
||||
|
||||
```ts
|
||||
type DataMaskingCode = 'DATA_MASKING.INVALID_RULE' | 'DATA_MASKING.MASK_FAILED';
|
||||
```
|
||||
|
||||
Union type derived from `DATA_MASKING_CODES`. Use this for exhaustive `switch`
|
||||
statements or discriminated-union narrowing.
|
||||
|
||||
### `DataMaskingError`
|
||||
|
||||
```ts
|
||||
class DataMaskingError extends Error {
|
||||
readonly code: DataMaskingCode;
|
||||
constructor(message: string, code: DataMaskingCode): DataMaskingError;
|
||||
}
|
||||
```
|
||||
|
||||
| Property | Type | Description |
|
||||
|----------|-------------------|----------------------------------------|
|
||||
| `name` | `string` | Always `"DataMaskingError"`. |
|
||||
| `code` | `DataMaskingCode` | The stable machine-readable error code.|
|
||||
| `message`| `string` | Human-readable description of failure. |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The `name` property is set in the constructor body (not via a static field)
|
||||
to ensure correct behaviour after `Object.setPrototypeOf` in transpiled ES5
|
||||
environments, though suaveplan targets ES2022+ natively.
|
||||
- No stack-trace manipulation is performed — the default `Error` constructor
|
||||
captures a stack trace automatically in V8 and Bun runtimes.
|
||||
- `DATA_MASKING_CODES` is declared `as const` so TypeScript infers the narrow
|
||||
string-literal type for each member; this enables exhaustive checks on
|
||||
`DataMaskingCode` without a separate enum.
|
||||
@@ -0,0 +1,122 @@
|
||||
# mask-rule.schema
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides Zod schemas for all public configuration types in
|
||||
`@suaveplan/data-masking`. Runtime validation of `MaskRule` and `MaskOptions`
|
||||
objects is important when rules originate from external sources such as API
|
||||
request bodies, configuration files, or database records. Using these schemas
|
||||
before calling `maskObject` prevents silent mis-configurations — for example, a
|
||||
rule with an empty `fields` array or an unrecognised `strategy` string — from
|
||||
causing confusing runtime behaviour.
|
||||
|
||||
Exporting the schemas as a dedicated subpath (`./schemas`) allows tree-shaking
|
||||
in production bundles that never parse external rule definitions.
|
||||
|
||||
## Features
|
||||
|
||||
- `MaskStrategySchema` — exhaustive `z.enum` of all eight strategy strings.
|
||||
- `MaskRuleSchema` — validates `fields` (non-empty array of non-empty strings),
|
||||
`strategy`, and optional `keepLast`, `keepFirst`, `replacement` fields.
|
||||
- `MaskOptionsSchema` — validates `rules` (at least one required), `deep`,
|
||||
`maskArrays`.
|
||||
- Input types (`MaskStrategyInput`, `MaskRuleInput`, `MaskOptionsInput`)
|
||||
exported for use in form and API validation pipelines.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { MaskRuleSchema } from '@suaveplan/data-masking/schemas';
|
||||
|
||||
const parsed = MaskRuleSchema.safeParse({
|
||||
fields: ['user.email'],
|
||||
strategy: 'email',
|
||||
});
|
||||
|
||||
if (!parsed.success) {
|
||||
console.error(parsed.error.issues);
|
||||
} else {
|
||||
console.log(parsed.data.strategy); // 'email'
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { MaskOptionsSchema } from '@suaveplan/data-masking/schemas';
|
||||
import { maskObject } from '@suaveplan/data-masking';
|
||||
|
||||
function safeMaskObject(
|
||||
data: Record<string, unknown>,
|
||||
rawOptions: unknown,
|
||||
) {
|
||||
const options = MaskOptionsSchema.parse(rawOptions); // throws ZodError on invalid input
|
||||
return maskObject(data, options);
|
||||
}
|
||||
|
||||
// Use from an API request body
|
||||
const result = safeMaskObject(userData, req.body.maskOptions);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `MaskStrategySchema`
|
||||
|
||||
```ts
|
||||
const MaskStrategySchema: z.ZodEnum<[
|
||||
'full', 'partial', 'email', 'phone',
|
||||
'credit-card', 'hash', 'tokenize', 'preserve-format'
|
||||
]>;
|
||||
```
|
||||
|
||||
Validates that a string is one of the eight supported masking strategy
|
||||
identifiers.
|
||||
|
||||
### `MaskRuleSchema`
|
||||
|
||||
```ts
|
||||
const MaskRuleSchema: z.ZodObject<{
|
||||
fields: z.ZodArray<z.ZodString>; // min 1 item, each min 1 char
|
||||
strategy: typeof MaskStrategySchema;
|
||||
keepLast?: z.ZodNumber; // int, non-negative
|
||||
keepFirst?: z.ZodNumber; // int, non-negative
|
||||
replacement?: z.ZodString;
|
||||
}>;
|
||||
```
|
||||
|
||||
### `MaskOptionsSchema`
|
||||
|
||||
```ts
|
||||
const MaskOptionsSchema: z.ZodObject<{
|
||||
rules: z.ZodArray<typeof MaskRuleSchema>; // min 1 item
|
||||
deep?: z.ZodBoolean;
|
||||
maskArrays?: z.ZodBoolean;
|
||||
}>;
|
||||
```
|
||||
|
||||
### Input types
|
||||
|
||||
```ts
|
||||
type MaskStrategyInput = z.input<typeof MaskStrategySchema>;
|
||||
type MaskRuleInput = z.input<typeof MaskRuleSchema>;
|
||||
type MaskOptionsInput = z.input<typeof MaskOptionsSchema>;
|
||||
```
|
||||
|
||||
These are the raw (un-coerced) input types, useful for TypeScript annotations
|
||||
in form handlers and API route definitions before calling `.parse()`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `z.number().int().nonnegative()` is used for `keepLast` and `keepFirst` to
|
||||
reject fractional or negative values that would produce nonsensical results
|
||||
in the `partial` strategy.
|
||||
- `z.array(...).min(1)` on both `fields` and `rules` prevents silent no-op
|
||||
configurations — a rule with no fields or an options object with no rules
|
||||
would be a programming error.
|
||||
- The schema file is excluded from coverage measurement (via `*.schema.ts`
|
||||
exclusion in `vitest.config.ts`) because it is declarative Zod schema
|
||||
definition code; meaningful coverage comes from integration tests of the
|
||||
schemas via `safeParse`/`parse`, not line-by-line execution tracing.
|
||||
@@ -0,0 +1,82 @@
|
||||
# credit-card strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `credit-card` masking strategy produces the PCI-DSS-compatible truncation
|
||||
format for payment card numbers: last 4 digits visible, all preceding digit
|
||||
positions replaced with `*`, formatting separators preserved. This aligns
|
||||
with the PCI DSS 4.0 requirement that primary account numbers (PANs) must be
|
||||
rendered unreadable in logs and displays, while retaining just enough
|
||||
information for card identification and customer service operations.
|
||||
|
||||
## Features
|
||||
|
||||
- Last-4-digit preservation regardless of separator style.
|
||||
- Hyphens, spaces, and other non-digit characters pass through unchanged.
|
||||
- Handles formatted (`4111-1111-1111-1111`) and raw (`4111111111111111`) inputs.
|
||||
- No regex capture groups — single linear pass, O(n) complexity.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskCreditCard } from '@suaveplan/data-masking/strategies/credit-card';
|
||||
|
||||
const masked = maskCreditCard('4111-1111-1111-1111', {
|
||||
fields: ['cardNumber'],
|
||||
strategy: 'credit-card',
|
||||
});
|
||||
// masked → '****-****-****-1111'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskCreditCard } from '@suaveplan/data-masking/strategies/credit-card';
|
||||
|
||||
// No separators
|
||||
maskCreditCard('5500005555555559', { fields: ['pan'], strategy: 'credit-card' });
|
||||
// → '************5559'
|
||||
|
||||
// Space-separated format
|
||||
maskCreditCard('4111 1111 1111 1111', { fields: ['pan'], strategy: 'credit-card' });
|
||||
// → '**** **** **** 1111'
|
||||
|
||||
// Amex 15-digit
|
||||
maskCreditCard('3714 496353 98431', { fields: ['pan'], strategy: 'credit-card' });
|
||||
// → '**** ****** *8431'
|
||||
|
||||
// No digits at all → passthrough
|
||||
maskCreditCard('CARD-ON-FILE', { fields: ['pan'], strategy: 'credit-card' });
|
||||
// → 'CARD-ON-FILE'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskCreditCard(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskCreditCard(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|-----------------------------------------------------------|
|
||||
| `value` | `string` | The original card number in any format. |
|
||||
| `rule` | `MaskRule` | Driving rule (no fields consumed by credit-card strategy).|
|
||||
|
||||
**Returns** the masked card number with last 4 digits preserved and all other
|
||||
digit positions replaced by `*`.
|
||||
|
||||
**Does not throw.** Returns the original value unchanged when no digit
|
||||
characters are found.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The algorithm is identical in structure to the `phone` strategy: extract all
|
||||
digits to find the `keepFrom` threshold, then rebuild the string in a single
|
||||
pass. The two strategies are kept as separate files to allow divergent
|
||||
evolution (e.g. future BIN-preservation or Luhn-check validation in the
|
||||
credit-card variant).
|
||||
- `Math.max(0, digits.length - 4)` ensures cards with 4 or fewer digits are
|
||||
returned unchanged, consistent with the principle of least surprise.
|
||||
- The `rule` parameter is silently voided to satisfy the uniform strategy
|
||||
interface; no `MaskRule` fields are read.
|
||||
75
docs/foundation/data/data-masking/src/strategies/email.md
Normal file
75
docs/foundation/data/data-masking/src/strategies/email.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# email strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `email` masking strategy anonymises email addresses while keeping the
|
||||
output immediately recognisable as an email address. Retaining the first
|
||||
character of the local part and of the domain name — plus the full TLD — gives
|
||||
operations and support teams just enough context to identify a domain category
|
||||
(e.g. `.com`, `.io`, `.co.uk`) and the rough shape of an address without
|
||||
exposing personally identifiable information. This is the standard form used
|
||||
in GDPR-compliant audit logs and support ticketing systems.
|
||||
|
||||
## Features
|
||||
|
||||
- Preserves first character of the local-part (`u***@...`).
|
||||
- Preserves first character of the domain name (`...@d***...`).
|
||||
- Preserves the TLD verbatim (`.com`, `.co.uk`, etc.).
|
||||
- Gracefully handles malformed inputs (no `@` sign) by returning `"***"`.
|
||||
- No external dependencies — pure string slicing.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskEmail } from '@suaveplan/data-masking/strategies/email';
|
||||
|
||||
const masked = maskEmail('john.doe@example.com', { fields: ['email'], strategy: 'email' });
|
||||
// masked → 'j***@e***.com'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskEmail } from '@suaveplan/data-masking/strategies/email';
|
||||
|
||||
// Long TLD preserved
|
||||
maskEmail('alice@subdomain.company.co.uk', { fields: ['email'], strategy: 'email' });
|
||||
// → 'a***@s***.co.uk'
|
||||
|
||||
// No @ in input → fallback
|
||||
maskEmail('not-an-email', { fields: ['email'], strategy: 'email' });
|
||||
// → '***'
|
||||
|
||||
// Single-char local part
|
||||
maskEmail('x@y.io', { fields: ['email'], strategy: 'email' });
|
||||
// → 'x***@y***.io'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskEmail(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskEmail(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|---------------------------------------------------------|
|
||||
| `value` | `string` | The original email address. |
|
||||
| `rule` | `MaskRule` | Driving rule (no fields are read from it for email). |
|
||||
|
||||
**Returns** a string in the form `u***@d***.tld`, or `"***"` when `value`
|
||||
contains no `@` character.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The TLD is identified as everything from the **last** `.` in the domain
|
||||
portion, which handles multi-part TLDs like `.co.uk` correctly.
|
||||
- If the domain portion contains no `.` (uncommon in practice) the TLD is
|
||||
treated as empty and the output becomes `d***`.
|
||||
- The `rule` parameter is accepted to satisfy the uniform strategy interface
|
||||
but none of its fields are consumed by the email strategy.
|
||||
- Masking is applied to the parsed components, not via regex substitution,
|
||||
which avoids regex-denial-of-service risks on adversarially crafted inputs.
|
||||
67
docs/foundation/data/data-masking/src/strategies/full.md
Normal file
67
docs/foundation/data/data-masking/src/strategies/full.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# full strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `full` masking strategy completely replaces an input string with a
|
||||
configurable replacement sentinel. It is the most aggressive masking mode
|
||||
available in `@suaveplan/data-masking`: no characters from the original value
|
||||
are preserved, so the output reveals nothing about length, structure, or
|
||||
content. This strategy is appropriate for highly sensitive fields such as
|
||||
passwords, SSNs, or security codes where even partial exposure is unacceptable.
|
||||
|
||||
## Features
|
||||
|
||||
- Zero information leakage — original value is discarded entirely.
|
||||
- Customisable replacement via `MaskRule.replacement`; defaults to `"***"`.
|
||||
- Constant-time execution (no iteration over the input string).
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskFull } from '@suaveplan/data-masking/strategies/full';
|
||||
|
||||
const masked = maskFull('super-secret-password', { fields: ['password'], strategy: 'full' });
|
||||
// masked → '***'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskFull } from '@suaveplan/data-masking/strategies/full';
|
||||
|
||||
// Custom replacement sentinel
|
||||
const masked = maskFull('123-45-6789', {
|
||||
fields: ['ssn'],
|
||||
strategy: 'full',
|
||||
replacement: '[REDACTED]',
|
||||
});
|
||||
// masked → '[REDACTED]'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskFull(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskFull(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|---------------------------------------------------------|
|
||||
| `value` | `string` | The original value to mask (immediately discarded). |
|
||||
| `rule` | `MaskRule` | Driving rule; `rule.replacement` used when provided. |
|
||||
|
||||
**Returns** `rule.replacement` if set, otherwise `"***"`.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The `value` parameter is accepted but immediately voided to satisfy the
|
||||
uniform `mask(value, rule): string` strategy interface.
|
||||
- `rule.replacement` may be any string, including the empty string `""`.
|
||||
Callers that want a custom sentinel (e.g. `"[REDACTED]"`, `"N/A"`) set this
|
||||
field on their `MaskRule`.
|
||||
- Unlike `partial`, `email`, or `preserve-format` strategies, the `full`
|
||||
strategy cannot be reversed or detected — it is appropriate for
|
||||
security-critical fields that must not leak data to downstream systems.
|
||||
79
docs/foundation/data/data-masking/src/strategies/hash.md
Normal file
79
docs/foundation/data/data-masking/src/strategies/hash.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# hash strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `hash` masking strategy produces a deterministic, fixed-length 8-character
|
||||
hexadecimal token from any input string using the FNV-1a 32-bit hash algorithm.
|
||||
Unlike the `full` or `tokenize` strategies, the `hash` strategy preserves the
|
||||
property that identical input values produce identical output tokens, which
|
||||
enables de-duplication and correlation in anonymised log streams without
|
||||
exposing the original data. This is valuable when troubleshooting session
|
||||
issues where the same user ID must be traceable across log entries.
|
||||
|
||||
**Security caveat:** FNV-1a is not cryptographically secure. It must not be
|
||||
used for password hashing, MAC generation, or any context where collision
|
||||
resistance is a security requirement. For those use cases, use
|
||||
`@suaveplan/crypto-hash` with `sha256` or `argon2id`.
|
||||
|
||||
## Features
|
||||
|
||||
- Deterministic: same input → same 8-char hex output, every time.
|
||||
- Fixed-length output (8 hex digits = 32-bit hash).
|
||||
- Zero external dependencies — pure JavaScript arithmetic via `Math.imul`.
|
||||
- Fast: O(n) in string length with very low constant factor.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskHash } from '@suaveplan/data-masking/strategies/hash';
|
||||
|
||||
const masked = maskHash('user_12345', { fields: ['userId'], strategy: 'hash' });
|
||||
// masked → e.g. '2c7b4f8a' (deterministic for this input)
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskHash } from '@suaveplan/data-masking/strategies/hash';
|
||||
|
||||
// Identical inputs → identical outputs (log correlation)
|
||||
const a = maskHash('john@example.com', { fields: ['email'], strategy: 'hash' });
|
||||
const b = maskHash('john@example.com', { fields: ['email'], strategy: 'hash' });
|
||||
// a === b → true
|
||||
|
||||
// Different inputs → (almost certainly) different outputs
|
||||
const c = maskHash('jane@example.com', { fields: ['email'], strategy: 'hash' });
|
||||
// a === c → false (very likely)
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskHash(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskHash(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|-----------------------------------------------------|
|
||||
| `value` | `string` | The original string to hash. |
|
||||
| `rule` | `MaskRule` | Driving rule (no fields consumed by hash strategy). |
|
||||
|
||||
**Returns** an 8-character lowercase hexadecimal string.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The FNV-1a 32-bit variant uses offset basis `0x811c9dc5` and prime
|
||||
`0x01000193` per the canonical specification.
|
||||
- `Math.imul` is used for the multiplication step to get correct 32-bit
|
||||
integer multiplication semantics in JavaScript (standard `*` uses IEEE 754
|
||||
doubles which can lose low bits beyond 53-bit precision).
|
||||
- The `>>> 0` unsigned right-shift converts the result back from signed Int32
|
||||
to unsigned Uint32 before the next iteration, matching the expected
|
||||
unsigned arithmetic of FNV-1a.
|
||||
- Character codes are read with `charCodeAt` (UTF-16 code units), which is
|
||||
consistent and fast. Full Unicode codepoint splitting (via `codePointAt`)
|
||||
is not applied because the goal is a cheap stable hash, not an
|
||||
encoding-canonical one.
|
||||
85
docs/foundation/data/data-masking/src/strategies/partial.md
Normal file
85
docs/foundation/data/data-masking/src/strategies/partial.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# partial strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `partial` masking strategy preserves a configurable number of characters at
|
||||
the start and end of an input string while replacing the middle portion with
|
||||
asterisks. This is the most commonly needed masking shape for fields like
|
||||
API keys, account numbers, and reference codes where enough context must remain
|
||||
for humans to identify a value in logs without exposing the sensitive body.
|
||||
|
||||
## Features
|
||||
|
||||
- Configurable `keepFirst` (default `0`) and `keepLast` (default `4`) character
|
||||
counts via `MaskRule`.
|
||||
- Safe handling of short strings: when `keepFirst + keepLast >= length` the
|
||||
original value is returned unchanged rather than producing an empty or
|
||||
misleading result.
|
||||
- No external dependencies — pure string slicing.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskPartial } from '@suaveplan/data-masking/strategies/partial';
|
||||
|
||||
// Keep last 4 characters (default keepLast = 4, keepFirst = 0)
|
||||
const masked = maskPartial('sk_live_abc123xyz789', {
|
||||
fields: ['apiKey'],
|
||||
strategy: 'partial',
|
||||
});
|
||||
// masked → '****************789' (last 3 chars shown — "789" ends value)
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskPartial } from '@suaveplan/data-masking/strategies/partial';
|
||||
|
||||
// Keep first 2 and last 4 characters
|
||||
const masked = maskPartial('4111111111111111', {
|
||||
fields: ['cardNumber'],
|
||||
strategy: 'partial',
|
||||
keepFirst: 2,
|
||||
keepLast: 4,
|
||||
});
|
||||
// masked → '41**********1111'
|
||||
|
||||
// Short string — no masking applied
|
||||
const short = maskPartial('abc', {
|
||||
fields: ['code'],
|
||||
strategy: 'partial',
|
||||
keepFirst: 2,
|
||||
keepLast: 2,
|
||||
});
|
||||
// short → 'abc' (keepFirst + keepLast >= length)
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskPartial(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskPartial(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|------------------|------------|---------|------------------------------------------|
|
||||
| `value` | `string` | — | The original string. |
|
||||
| `rule.keepFirst` | `number` | `0` | Characters to keep from the start. |
|
||||
| `rule.keepLast` | `number` | `4` | Characters to keep from the end. |
|
||||
|
||||
**Returns** the masked string, or the original value unchanged if it is too
|
||||
short to mask.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The mask character is always `"*"`. A future extension could expose a
|
||||
`maskChar` option on `MaskRule` if needed.
|
||||
- `keepFirst + keepLast >= len` guard prevents edge cases where `slice` would
|
||||
produce overlapping or inverted ranges, which can silently produce empty
|
||||
strings in JavaScript.
|
||||
- Middle length is computed as `len - keepFirst - keepLast` to give a
|
||||
mask block that exactly fills the hidden region, preserving total length
|
||||
and allowing callers to spot-check that masking was applied.
|
||||
80
docs/foundation/data/data-masking/src/strategies/phone.md
Normal file
80
docs/foundation/data/data-masking/src/strategies/phone.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# phone strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `phone` masking strategy anonymises phone numbers while preserving their
|
||||
formatting separators, making masked values easy to read in logs and audit
|
||||
trails. Only the last 4 digits are kept; all other digit positions are
|
||||
replaced with `*`. Non-digit characters (hyphens, spaces, parentheses, dots,
|
||||
`+`) pass through unchanged, so the output retains the same visual structure
|
||||
as the input regardless of locale or formatting convention.
|
||||
|
||||
## Features
|
||||
|
||||
- Last-4-digit preservation regardless of phone format.
|
||||
- Separator characters (`-`, ` `, `(`, `)`, `.`, `+`) are preserved verbatim.
|
||||
- Handles international formats (`+1-…`), US formats (`(555) 123-4567`), and
|
||||
unformatted digit strings (`5551234567`).
|
||||
- No regex-based capture groups or lookaheads — single linear pass.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskPhone } from '@suaveplan/data-masking/strategies/phone';
|
||||
|
||||
const masked = maskPhone('+1-555-123-4567', { fields: ['phone'], strategy: 'phone' });
|
||||
// masked → '*-***-***-4567'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskPhone } from '@suaveplan/data-masking/strategies/phone';
|
||||
|
||||
// US format with parentheses
|
||||
maskPhone('(555) 123-4567', { fields: ['phone'], strategy: 'phone' });
|
||||
// → '(***) ***-4567'
|
||||
|
||||
// E.164 international
|
||||
maskPhone('+447911123456', { fields: ['phone'], strategy: 'phone' });
|
||||
// → '+*********3456'
|
||||
|
||||
// Unformatted
|
||||
maskPhone('5551234567', { fields: ['phone'], strategy: 'phone' });
|
||||
// → '******4567'
|
||||
|
||||
// No digits at all → passthrough
|
||||
maskPhone('N/A', { fields: ['phone'], strategy: 'phone' });
|
||||
// → 'N/A'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskPhone(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskPhone(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|----------------------------------------------------------|
|
||||
| `value` | `string` | The original phone number in any format. |
|
||||
| `rule` | `MaskRule` | Driving rule (no fields are consumed by phone strategy). |
|
||||
|
||||
**Returns** the masked phone string with last 4 digits preserved and all other
|
||||
digit positions replaced by `*`.
|
||||
|
||||
**Does not throw.** Returns the original value unchanged when no digit
|
||||
characters are found.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The strategy extracts all digits first to determine the `keepFrom` boundary
|
||||
(total digits minus 4), then makes a single linear pass over the original
|
||||
string to rebuild the result. This avoids multi-pass scanning and keeps
|
||||
the logic O(n).
|
||||
- `Math.max(0, digits.length - 4)` ensures that strings with 4 or fewer digits
|
||||
are returned with all digits preserved (no masking applied), matching the
|
||||
principle of least surprise.
|
||||
- The `rule` parameter is accepted to satisfy the uniform strategy interface
|
||||
but none of its fields are consumed.
|
||||
@@ -0,0 +1,79 @@
|
||||
# preserve-format strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `preserve-format` masking strategy replaces every alphanumeric character
|
||||
with a placeholder of the same character class — digits become `0`, letters
|
||||
become `X` — while leaving all punctuation, spaces, and special characters
|
||||
unchanged. The result has the same total length and the same structural
|
||||
"skeleton" as the original, which allows downstream systems (parsers, validators,
|
||||
format checkers) to continue operating on the masked data without seeing any
|
||||
real content. This is particularly useful in schema-validation pipelines that
|
||||
process PII-bearing fields and must remain functional during testing against
|
||||
production-shaped data.
|
||||
|
||||
## Features
|
||||
|
||||
- Length-preserving: output has exactly the same character count as input.
|
||||
- Character-class-preserving: digits → `0`, letters → `X`, other → verbatim.
|
||||
- No regex — single character-by-character pass, safe on all Unicode inputs
|
||||
(only ASCII alphanumerics are substituted; non-ASCII passes through).
|
||||
- Zero external dependencies.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskPreserveFormat } from '@suaveplan/data-masking/strategies/preserve-format';
|
||||
|
||||
const masked = maskPreserveFormat('Hello123', { fields: ['code'], strategy: 'preserve-format' });
|
||||
// masked → 'XXXXX000'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskPreserveFormat } from '@suaveplan/data-masking/strategies/preserve-format';
|
||||
|
||||
// Phone with separators
|
||||
maskPreserveFormat('+1-555-123-4567', { fields: ['phone'], strategy: 'preserve-format' });
|
||||
// → '+0-000-000-0000'
|
||||
|
||||
// Mixed alphanumeric code
|
||||
maskPreserveFormat('AB12-CD34-EF56', { fields: ['serial'], strategy: 'preserve-format' });
|
||||
// → 'XX00-XX00-XX00'
|
||||
|
||||
// Non-ASCII pass-through (accented chars preserved)
|
||||
maskPreserveFormat('café 42', { fields: ['label'], strategy: 'preserve-format' });
|
||||
// → 'XXXX 00' (é is non-ASCII and passes through as-is)
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskPreserveFormat(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskPreserveFormat(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|----------------------------------------------------------------|
|
||||
| `value` | `string` | The original string to format-preserve. |
|
||||
| `rule` | `MaskRule` | Driving rule (no fields consumed by preserve-format strategy). |
|
||||
|
||||
**Returns** a same-length string with alphanumeric chars replaced by `0` / `X`
|
||||
and all other chars preserved.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Only ASCII alphanumerics (`a-z`, `A-Z`, `0-9`) are replaced. Non-ASCII
|
||||
characters (including accented letters, CJK, emoji) pass through unchanged.
|
||||
This is intentional: the strategy targets structured English-language data
|
||||
formats (phone numbers, reference codes, account identifiers) where only
|
||||
ASCII alphanumerics carry sensitive content.
|
||||
- The `for...of` iteration correctly handles multi-code-unit Unicode characters
|
||||
(surrogate pairs) because JavaScript iterates by codepoint in `for...of`.
|
||||
The ASCII range check ensures non-ASCII characters are passed through intact.
|
||||
- The `rule` parameter is silently voided to satisfy the uniform strategy
|
||||
interface.
|
||||
91
docs/foundation/data/data-masking/src/strategies/tokenize.md
Normal file
91
docs/foundation/data/data-masking/src/strategies/tokenize.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# tokenize strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
The `tokenize` masking strategy replaces a sensitive value with a structured
|
||||
opaque token of the form `TOKEN_<STRATEGY>_<N>`, where `<N>` is a
|
||||
monotonically incrementing counter scoped to the current process lifetime.
|
||||
This is the simplest form of tokenisation — sufficient for audit trails and
|
||||
structured logs where the original value must be fully obscured but a unique
|
||||
reference per occurrence is still useful.
|
||||
|
||||
**Non-reversibility warning:** This implementation does NOT maintain a mapping
|
||||
from token to original value. Each call produces a new token (the counter
|
||||
increments on every invocation), and the counter resets when the process
|
||||
restarts. True reversible tokenisation requires a persistent, encrypted token
|
||||
vault that maps tokens to their plaintext originals. That capability is
|
||||
intentionally outside the scope of `@suaveplan/data-masking`.
|
||||
|
||||
## Features
|
||||
|
||||
- Structured token format: `TOKEN_<STRATEGY>_<index>`.
|
||||
- Monotonically increasing per-process counter — no random dependency.
|
||||
- Deterministic in tests when the counter is reset via `_resetTokenCounter()`.
|
||||
- Zero external dependencies.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { maskTokenize, _resetTokenCounter } from '@suaveplan/data-masking/strategies/tokenize';
|
||||
|
||||
_resetTokenCounter(); // deterministic start for tests
|
||||
|
||||
const t1 = maskTokenize('john@example.com', { fields: ['email'], strategy: 'tokenize' });
|
||||
// t1 → 'TOKEN_TOKENIZE_0'
|
||||
|
||||
const t2 = maskTokenize('jane@example.com', { fields: ['email'], strategy: 'tokenize' });
|
||||
// t2 → 'TOKEN_TOKENIZE_1'
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { maskTokenize, _resetTokenCounter } from '@suaveplan/data-masking/strategies/tokenize';
|
||||
|
||||
_resetTokenCounter();
|
||||
|
||||
// The strategy tag is embedded in the token for provenance
|
||||
const token = maskTokenize('4111111111111111', { fields: ['pan'], strategy: 'tokenize' });
|
||||
// token → 'TOKEN_TOKENIZE_0'
|
||||
|
||||
// Confirm it's a token (quick prefix check)
|
||||
console.log(token.startsWith('TOKEN_')); // true
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `maskTokenize(value, rule)`
|
||||
|
||||
```ts
|
||||
function maskTokenize(value: string, rule: MaskRule): string;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------------|---------------------------------------------------------------|
|
||||
| `value` | `string` | The original value (silently discarded). |
|
||||
| `rule` | `MaskRule` | Driving rule; `rule.strategy` is embedded in the token tag. |
|
||||
|
||||
**Returns** a token string of the form `TOKEN_<STRATEGY>_<N>`.
|
||||
|
||||
**Does not throw.**
|
||||
|
||||
### `_resetTokenCounter()`
|
||||
|
||||
```ts
|
||||
function _resetTokenCounter(): void;
|
||||
```
|
||||
|
||||
Resets the internal token counter to `0`. **For testing only.** Production
|
||||
code must not call this function.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The counter is a module-level variable, not a class instance, to satisfy
|
||||
the `mask(value, rule): string` strategy interface without requiring callers
|
||||
to instantiate a strategy object or inject state.
|
||||
- Hyphens in the strategy name are converted to underscores in the token tag
|
||||
(`credit-card` → `CREDIT_CARD`) for identifier-safe token strings.
|
||||
- Truly reversible tokenisation would require a key-value store (database or
|
||||
in-memory map with encryption) mapping each token back to its plaintext.
|
||||
That is deliberately deferred to a higher-level package and explicitly
|
||||
documented as out-of-scope here.
|
||||
130
docs/foundation/data/data-masking/src/types/mask-types.md
Normal file
130
docs/foundation/data/data-masking/src/types/mask-types.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# mask-types
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Central type definition module for `@suaveplan/data-masking`. All public-facing
|
||||
interfaces (`MaskRule`, `MaskOptions`, `MaskResult`) and the `MaskStrategy`
|
||||
union type live here to provide a single authoritative source of truth for the
|
||||
package's data model and to break circular-import hazards between strategy
|
||||
implementations and the masker core.
|
||||
|
||||
This file ships zero runtime code — it is a types-only module. Importing it
|
||||
has no observable side-effects, making it tree-shakeable in type-erasure builds.
|
||||
|
||||
## Features
|
||||
|
||||
- `MaskStrategy` union: eight masking modes as a string literal union.
|
||||
- `MaskRule` interface: field path list, strategy, and per-strategy options.
|
||||
- `MaskOptions` interface: traversal controls (`deep`, `maskArrays`).
|
||||
- `MaskResult<T>` generic interface: masked clone + affected path list.
|
||||
- Full JSDoc on every property for IDE hover documentation.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import type { MaskRule, MaskOptions } from '@suaveplan/data-masking';
|
||||
|
||||
const options: MaskOptions = {
|
||||
rules: [
|
||||
{ fields: ['user.email'], strategy: 'email' },
|
||||
{ fields: ['user.ssn'], strategy: 'full' },
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import type { MaskRule, MaskResult, MaskStrategy } from '@suaveplan/data-masking';
|
||||
|
||||
// Build a typed rule map
|
||||
const SENSITIVE_FIELDS: Record<string, MaskStrategy> = {
|
||||
'user.email': 'email',
|
||||
'user.phone': 'phone',
|
||||
'payment.cardNumber': 'credit-card',
|
||||
'payment.cvv': 'full',
|
||||
};
|
||||
|
||||
const rules: MaskRule[] = Object.entries(SENSITIVE_FIELDS).map(
|
||||
([field, strategy]) => ({ fields: [field], strategy }),
|
||||
);
|
||||
|
||||
// Use with maskObject result
|
||||
function logAffected(result: MaskResult<unknown>): void {
|
||||
console.log('Masked fields:', result.fieldsAffected.join(', '));
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `MaskStrategy`
|
||||
|
||||
```ts
|
||||
type MaskStrategy =
|
||||
| 'full' | 'partial' | 'email' | 'phone'
|
||||
| 'credit-card' | 'hash' | 'tokenize' | 'preserve-format';
|
||||
```
|
||||
|
||||
String-literal union of all supported masking strategy identifiers.
|
||||
|
||||
### `MaskRule`
|
||||
|
||||
```ts
|
||||
interface MaskRule {
|
||||
fields: string[];
|
||||
strategy: MaskStrategy;
|
||||
keepLast?: number;
|
||||
keepFirst?: number;
|
||||
replacement?: string;
|
||||
}
|
||||
```
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|---------------|----------------|---------|-------------------------------------------------|
|
||||
| `fields` | `string[]` | — | Dot-notation paths / glob patterns. |
|
||||
| `strategy` | `MaskStrategy` | — | Which masking strategy to apply. |
|
||||
| `keepLast` | `number` | `4` | Chars to preserve at end (`partial` only). |
|
||||
| `keepFirst` | `number` | `0` | Chars to preserve at start (`partial` only). |
|
||||
| `replacement` | `string` | `"***"` | Custom replacement string (`full` only). |
|
||||
|
||||
### `MaskOptions`
|
||||
|
||||
```ts
|
||||
interface MaskOptions {
|
||||
rules: MaskRule[];
|
||||
deep?: boolean;
|
||||
maskArrays?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|--------------|--------------|---------|-------------------------------------------|
|
||||
| `rules` | `MaskRule[]` | — | Ordered list of masking rules. |
|
||||
| `deep` | `boolean` | `true` | Recurse into nested objects. |
|
||||
| `maskArrays` | `boolean` | `true` | Apply rules to array element values. |
|
||||
|
||||
### `MaskResult<T>`
|
||||
|
||||
```ts
|
||||
interface MaskResult<T = unknown> {
|
||||
masked: T;
|
||||
fieldsAffected: string[];
|
||||
}
|
||||
```
|
||||
|
||||
| Property | Type | Description |
|
||||
|------------------|------------|----------------------------------------------------|
|
||||
| `masked` | `T` | Deep clone of input with masking applied. |
|
||||
| `fieldsAffected` | `string[]` | Sorted dot-notation paths that were masked. |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- This module is intentionally separated from strategy implementations to
|
||||
avoid circular imports. Strategy files import `MaskRule` from here; the
|
||||
masker core imports both strategies and types from here.
|
||||
- `exactOptionalPropertyTypes` is enabled in `tsconfig.base.json`, so all
|
||||
optional fields are typed as `T | undefined` rather than `T?` to avoid
|
||||
accidental `undefined` assignment bypasses.
|
||||
88
docs/foundation/data/data-masking/src/utils/path-matcher.md
Normal file
88
docs/foundation/data/data-masking/src/utils/path-matcher.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# path-matcher
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Implements a lightweight glob-style matcher for dot-notation object-path
|
||||
strings used by `@suaveplan/data-masking`'s rule engine. When a `MaskRule`
|
||||
declares `fields: ['user.*']` the path matcher is responsible for deciding
|
||||
whether a path like `'user.email'` satisfies that pattern during object
|
||||
traversal. Zero external dependencies are required — the entire implementation
|
||||
is a recursive backtracking segment comparator that handles the three modes
|
||||
callers need in practice.
|
||||
|
||||
## Features
|
||||
|
||||
- Exact matching: `"user.email"` matches `"user.email"`.
|
||||
- Single-segment wildcard `*`: matches any one dot-separated segment.
|
||||
- Multi-segment wildcard `**`: matches zero or more consecutive dot-separated
|
||||
segments, enabling patterns like `"**.email"` or `"user.**"`.
|
||||
- Correct handling of leading, trailing, and adjacent wildcards through
|
||||
backtracking.
|
||||
- Pure function — no state, no side-effects, zero allocations beyond the
|
||||
initial `split`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { matchPath } from '@suaveplan/data-masking/utils/path-matcher';
|
||||
|
||||
matchPath('user.email', 'user.email'); // true — exact
|
||||
matchPath('user.name', 'user.email'); // false — no match
|
||||
matchPath('user.email', 'user.*'); // true — single wildcard
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { matchPath } from '@suaveplan/data-masking/utils/path-matcher';
|
||||
|
||||
// Match any path ending in .email regardless of depth
|
||||
matchPath('billing.contact.email', '**.email'); // true
|
||||
matchPath('email', '**.email'); // true (** matches zero segs)
|
||||
|
||||
// Match everything under user.*
|
||||
matchPath('user.address.city', 'user.**'); // true
|
||||
matchPath('user.address.city', 'user.*'); // false — * is single-segment only
|
||||
matchPath('user.address', 'user.*'); // true
|
||||
|
||||
// Combine wildcards
|
||||
matchPath('a.b.c.d', '*.b.**.d'); // true
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `matchPath(fieldPath, pattern)`
|
||||
|
||||
```ts
|
||||
function matchPath(fieldPath: string, pattern: string): boolean;
|
||||
```
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-------------|----------|--------------------------------------------------------|
|
||||
| `fieldPath` | `string` | Concrete dot-notation path, e.g. `"user.address.zip"`. |
|
||||
| `pattern` | `string` | Glob pattern with optional `*` / `**` wildcards. |
|
||||
|
||||
**Returns** `true` when `fieldPath` satisfies `pattern`, `false` otherwise.
|
||||
|
||||
**Does not throw.** Both empty strings are treated as single empty-segment
|
||||
paths and will only match each other or a lone `*` / `**` pattern.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The implementation splits both inputs on `"."` and feeds the resulting
|
||||
arrays into a recursive `matchSegments` helper. The recursion depth is
|
||||
bounded by `Math.max(pathSegments, patternSegments)`, which in practice is
|
||||
small (typically < 10 for real object paths).
|
||||
- The `**` wildcard uses a greedy-with-backtrack strategy: it first attempts
|
||||
to match zero path segments (advance only the pattern pointer), then
|
||||
progressively tries consuming one more path segment per recursive call.
|
||||
This guarantees correct results for patterns like `"**.email"` where `**`
|
||||
must match zero leading segments.
|
||||
- No regular expressions are used, avoiding edge-case escaping issues with
|
||||
field names that contain regex metacharacters (dots, brackets, etc.).
|
||||
- The function is intentionally kept free of any optional parameters or
|
||||
configuration — the three modes (exact, `*`, `**`) cover every use case
|
||||
encountered in PII field-path matching.
|
||||
49
docs/foundation/data/dedup/README.md
Normal file
49
docs/foundation/data/dedup/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# @suaveplan/dedup - Entity deduplication engine with pluggable match keys, scoring rules, Jaro-Winkler/Levenshtein similarity, and merge strategies
|
||||
|
||||
## Summary
|
||||
|
||||
Entity deduplication engine for detecting and merging duplicate records in any data set. Given a list of objects with varying representations of the same real-world entity (e.g. `"John Smith"` vs `"J. Smith"`, or the same email in different capitalisation), the `DedupEngine` scores pairs using a weighted combination of field-level similarity algorithms — Jaro-Winkler, Levenshtein, exact match, or phonetic — groups records into connected components via union-find, and merges each group with a pluggable strategy (`latestWins`, `manual`, or `scoredMerge`). Use this package for ETL pipelines, contact deduplication, and data cleansing. For domain-name-specific normalisation and alias resolution use `@suaveplan/domain-normalizer`; for string-similarity primitives only, the scoring module is exported as a standalone subpath.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_- [dedup-engine](./src/core/engine/dedup-engine.md) — <!-- begin:auto-docs --> _(no submodules)_- [merge](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_ - [merge-strategies](./src/core/merge/merge-strategies.md) — <!-- begin:auto-docs --> _(no submodules)_- [scoring](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_ - [similarity](./src/core/scoring/similarity.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_ - [dedup-error](./src/errors/dedup-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [schemas](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_ - [dedup.schema](./src/schemas/dedup.schema.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Modules
|
||||
|
||||
| Module | Description |
|
||||
|---|---|
|
||||
| [scoring](src/core/scoring/similarity.md) | Levenshtein, Jaro, Jaro-Winkler, exact, and phonetic similarity algorithms |
|
||||
| [engine](src/core/engine/dedup-engine.md) | `DedupEngine` — detect duplicate groups and merge them |
|
||||
| [merge](src/core/merge/merge-strategies.md) | Built-in merge strategies: latest-wins, manual, scored |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```ts
|
||||
import { DedupEngine } from "@suaveplan/dedup/engine";
|
||||
import { latestWinsStrategy } from "@suaveplan/dedup/merge";
|
||||
|
||||
const engine = new DedupEngine({
|
||||
matchKeys: [
|
||||
{ field: "name", weight: 0.6, strategy: "jaro-winkler" },
|
||||
{ field: "email", weight: 0.4, strategy: "exact" },
|
||||
],
|
||||
scoringRules: [],
|
||||
mergeThreshold: 0.85,
|
||||
});
|
||||
|
||||
const groups = engine.detect(records);
|
||||
for (const group of groups) {
|
||||
const result = engine.merge(group, latestWinsStrategy);
|
||||
console.log(result.merged);
|
||||
}
|
||||
```
|
||||
114
docs/foundation/data/dedup/src/core/engine/dedup-engine.md
Normal file
114
docs/foundation/data/dedup/src/core/engine/dedup-engine.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# dedup-engine
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`DedupEngine` is the central orchestrator for entity deduplication. It accepts a validated `DedupConfig`, performs an O(n²) pairwise similarity scan using pluggable match keys and scoring rules, groups near-duplicate entities into connected components via union-find, and exposes a `merge()` method that delegates to caller-supplied merge strategies. The engine is generic over the entity type `T` so it can operate on any record shape.
|
||||
|
||||
## Features
|
||||
|
||||
- **Schema-validated configuration** — `DedupConfig` is parsed with Zod at construction time; invalid configs throw a `DedupError` with code `DEDUP.INVALID_CONFIG`.
|
||||
- **Weighted field scoring** — each `MatchKey` contributes its `weight × similarity(field_a, field_b)` to the pair score.
|
||||
- **Pluggable scoring rules** — `ScoringRule` instances can override or supplement the default key-based scoring for custom business logic.
|
||||
- **Connected-component grouping** — union-find algorithm aggregates transitive duplicate relationships so A≈B and B≈C implies A, B, C are in the same group.
|
||||
- **`maxGroupSize` guard** — prevents unbounded memory growth by capping the number of duplicates per group.
|
||||
- **OTel instrumentation** — emits `dedup.detect.total` counter, `dedup.detect.duration_ms` histogram, and per-span attributes on every `detect()` and `merge()` call.
|
||||
- **Safe merge delegation** — `merge()` wraps strategy execution in a try/catch and re-throws as `DedupError(STRATEGY_ERROR)` to preserve error code stability.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DedupEngine } from "@suaveplan/dedup/engine";
|
||||
|
||||
const engine = new DedupEngine({
|
||||
matchKeys: [
|
||||
{ field: "name", weight: 0.6, strategy: "jaro-winkler" },
|
||||
{ field: "email", weight: 0.4, strategy: "exact" },
|
||||
],
|
||||
scoringRules: [],
|
||||
mergeThreshold: 0.85,
|
||||
});
|
||||
|
||||
const records = [
|
||||
{ id: 1, name: "Alice Smith", email: "alice@example.com" },
|
||||
{ id: 2, name: "Alice Smyth", email: "alice@example.com" },
|
||||
{ id: 3, name: "Bob Jones", email: "bob@example.com" },
|
||||
];
|
||||
|
||||
const groups = engine.detect(records);
|
||||
// groups[0].canonical → { id: 1, name: "Alice Smith", email: "alice@example.com" }
|
||||
// groups[0].duplicates → [{ id: 2, name: "Alice Smyth", email: "alice@example.com" }]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DedupEngine } from "@suaveplan/dedup/engine";
|
||||
import { manualStrategy } from "@suaveplan/dedup/merge";
|
||||
|
||||
const engine = new DedupEngine<{ id: number; name: string; updatedAt: string }>({
|
||||
matchKeys: [
|
||||
{ field: "name", weight: 1, strategy: "jaro-winkler", caseSensitive: false },
|
||||
],
|
||||
scoringRules: [
|
||||
{
|
||||
name: "domain-boost",
|
||||
evaluate(a, b, _keys) {
|
||||
// Boost score when both entities are from the same domain
|
||||
const ra = a as Record<string, string>;
|
||||
const rb = b as Record<string, string>;
|
||||
const domainA = ra["email"]?.split("@")[1] ?? "";
|
||||
const domainB = rb["email"]?.split("@")[1] ?? "";
|
||||
return domainA && domainA === domainB ? 0.95 : 0.0;
|
||||
},
|
||||
},
|
||||
],
|
||||
mergeThreshold: 0.8,
|
||||
maxGroupSize: 50,
|
||||
});
|
||||
|
||||
const groups = engine.detect(records);
|
||||
for (const group of groups) {
|
||||
const result = engine.merge(group, manualStrategy);
|
||||
console.log("merged:", result.merged);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `new DedupEngine<T>(config: DedupConfig)`
|
||||
|
||||
Constructs a new engine. `config` is validated against `DedupConfigSchema`; throws `DedupError(INVALID_CONFIG)` on failure.
|
||||
|
||||
### `detect(entities: T[]): DuplicateGroup<T>[]`
|
||||
|
||||
Scans all entity pairs, builds a similarity graph, finds connected components, and returns one `DuplicateGroup` per multi-entity component. Singletons (no duplicates) are omitted.
|
||||
|
||||
**Params:**
|
||||
|
||||
- `entities` — array of entities to deduplicate
|
||||
|
||||
**Returns:** `DuplicateGroup<T>[]` — possibly empty if no duplicates found
|
||||
|
||||
### `merge(group: DuplicateGroup<T>, strategy: MergeStrategy<T>): MergeResult<T>`
|
||||
|
||||
Applies `strategy` to `group` and returns the merged result. Wraps thrown errors as `DedupError(STRATEGY_ERROR)`.
|
||||
|
||||
**Params:**
|
||||
|
||||
- `group` — a group returned by `detect()`
|
||||
- `strategy` — a `MergeStrategy<T>` function
|
||||
|
||||
**Returns:** `MergeResult<T>`
|
||||
|
||||
**Throws:** `DedupError` with code `DEDUP.STRATEGY_ERROR` if the strategy throws
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The pairwise scan is O(n²) in both time and the number of potential edges. For large corpora (> 10 000 entities) callers should pre-filter candidates using blocking keys (e.g. first three characters of a name) before passing them to `detect()`. A future blocking API may be added as a separate module.
|
||||
|
||||
The union-find implementation uses path-halving rather than full path compression to keep the code concise while still achieving near-linear amortised complexity. For groups larger than `maxGroupSize`, only the first `maxGroupSize − 1` duplicates (by original array index) are retained; this is intentional to bound memory and downstream merge cost.
|
||||
|
||||
When multiple `scoringRules` are present, each successive rule _replaces_ the accumulated score for that pair, not adds to it. This allows a single rule to act as a "veto" or "boost" override. If additive composition is needed, callers should encode that logic within a single rule.
|
||||
@@ -0,0 +1,92 @@
|
||||
# merge-strategies
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides three built-in merge strategies for combining a `DuplicateGroup` into a single entity. Merge strategies are pure functions that conform to the `MergeStrategy<T>` type and can be composed or replaced with custom implementations. They are passed to `DedupEngine.merge()` as callbacks rather than configured up-front, giving callers full control over how and when merging occurs.
|
||||
|
||||
## Features
|
||||
|
||||
- **`latestWinsStrategy`** — selects whichever entity (canonical or duplicate) has the most recent `updatedAt` or `createdAt` timestamp; ISO 8601 strings sort correctly as plain strings so no date parsing is needed.
|
||||
- **`manualStrategy`** — no-op that returns the canonical entity unchanged, useful when a human reviewer should approve merges before they are committed to a data store.
|
||||
- **`scoredMergeStrategy`** — assembles a "best-of-breed" entity by choosing, for each field, the value from the entity with the highest similarity score; the canonical entity acts as the fallback with score 1.0.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DedupEngine } from "@suaveplan/dedup/engine";
|
||||
import { latestWinsStrategy } from "@suaveplan/dedup/merge";
|
||||
|
||||
const engine = new DedupEngine({
|
||||
matchKeys: [{ field: "name", weight: 1, strategy: "jaro-winkler" }],
|
||||
scoringRules: [],
|
||||
});
|
||||
|
||||
const groups = engine.detect([
|
||||
{ name: "Alice", updatedAt: "2024-01-01T00:00:00Z" },
|
||||
{ name: "Alicia", updatedAt: "2024-06-15T12:00:00Z" },
|
||||
]);
|
||||
|
||||
for (const group of groups) {
|
||||
const result = engine.merge(group, latestWinsStrategy);
|
||||
// result.merged → { name: "Alicia", updatedAt: "2024-06-15T12:00:00Z" }
|
||||
// result.strategy → "latest-wins"
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DedupEngine } from "@suaveplan/dedup/engine";
|
||||
import { scoredMergeStrategy, manualStrategy } from "@suaveplan/dedup/merge";
|
||||
import type { MergeStrategy } from "@suaveplan/dedup";
|
||||
|
||||
type Person = { id: number; name: string; email: string; phone?: string };
|
||||
|
||||
const engine = new DedupEngine<Person>({
|
||||
matchKeys: [
|
||||
{ field: "name", weight: 0.5, strategy: "jaro-winkler" },
|
||||
{ field: "email", weight: 0.5, strategy: "exact" },
|
||||
],
|
||||
scoringRules: [],
|
||||
mergeThreshold: 0.8,
|
||||
});
|
||||
|
||||
const groups = engine.detect(persons);
|
||||
|
||||
// Pick strategy based on group size
|
||||
for (const group of groups) {
|
||||
const strategy: MergeStrategy<Person> =
|
||||
group.duplicates.length > 5 ? manualStrategy : scoredMergeStrategy;
|
||||
const result = engine.merge(group, strategy);
|
||||
console.log(result.strategy, result.merged);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `latestWinsStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
|
||||
|
||||
Picks the entity with the lexicographically largest `updatedAt` or `createdAt` field value. If neither field is present, returns the canonical unchanged. All entities (canonical + duplicates) participate in the comparison.
|
||||
|
||||
**Returns:** `MergeResult<T>` with `strategy: "latest-wins"`
|
||||
|
||||
### `manualStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
|
||||
|
||||
Returns the canonical as the merged entity with no field-level changes. Suitable as a placeholder when merge logic will be determined externally.
|
||||
|
||||
**Returns:** `MergeResult<T>` with `strategy: "manual"`
|
||||
|
||||
### `scoredMergeStrategy<T>(group: DuplicateGroup<T>): MergeResult<T>`
|
||||
|
||||
Constructs a merged object by iterating every field key that appears in any entity and assigning the value from the entity with the highest similarity score. The canonical entity always has score `1.0`. Fields with `null` or `undefined` values are skipped when selecting the best value. The result is cast to `T` — callers should ensure the resulting shape is compatible.
|
||||
|
||||
**Returns:** `MergeResult<T>` with `strategy: "scored-merge"`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
All three strategies are stateless functions with no side effects. They do not mutate the `group` argument. The `scoredMergeStrategy` creates a new plain object (`{}`) and populates it field-by-field, which means prototype-based instances (classes) will lose their prototype chain in the merged result. If the entity type uses class instances, callers should wrap `scoredMergeStrategy` and reconstruct the class after merging.
|
||||
|
||||
`latestWinsStrategy` relies on ISO 8601 string sort order. Timestamps in other formats (Unix seconds as numbers, locale-formatted strings) will not sort correctly. Callers storing dates in non-ISO formats should either normalise before dedup or supply a custom `MergeStrategy` that parses timestamps explicitly.
|
||||
94
docs/foundation/data/dedup/src/core/scoring/similarity.md
Normal file
94
docs/foundation/data/dedup/src/core/scoring/similarity.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# similarity
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides string similarity algorithms used by the dedup engine to compare entity field values. Three well-known distance / similarity metrics are implemented from scratch: Levenshtein edit distance, Jaro similarity, and Jaro-Winkler similarity. A phonetic normalisation path is also included for fuzzy name matching. All functions are pure and side-effect-free, operating only on their arguments.
|
||||
|
||||
## Features
|
||||
|
||||
- **Levenshtein distance** — classic Wagner–Fischer dynamic-programming algorithm with O(min(|a|, |b|)) space optimisation (two-row rolling array).
|
||||
- **Jaro similarity** — accounts for matching characters within a dynamic window and transposition pairs; returns a score in [0, 1].
|
||||
- **Jaro-Winkler similarity** — extends Jaro by boosting strings that share a common prefix (up to 4 characters) using the standard p = 0.1 scaling factor.
|
||||
- **Exact match** — binary 1 / 0 equality check for deterministic field comparison.
|
||||
- **Phonetic match** — lightweight vowel-removal normalisation followed by Jaro-Winkler, useful for names that sound alike but are spelled differently.
|
||||
- **`computeSimilarity` dispatcher** — unified entry point that routes to the right algorithm based on the `MatchKey.strategy` discriminant.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { levenshteinDistance, jaroWinklerSimilarity, computeSimilarity } from "@suaveplan/dedup/scoring";
|
||||
|
||||
levenshteinDistance("kitten", "sitting"); // 3
|
||||
jaroWinklerSimilarity("MARTHA", "MARHTA"); // ≈ 0.961
|
||||
|
||||
// Use the dispatcher for engine integration
|
||||
const score = computeSimilarity("Alice", "Alicia", "jaro-winkler"); // ≈ 0.947
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { computeSimilarity } from "@suaveplan/dedup/scoring";
|
||||
import type { MatchKey } from "@suaveplan/dedup";
|
||||
|
||||
const keys: MatchKey[] = [
|
||||
{ field: "name", weight: 0.5, strategy: "jaro-winkler", caseSensitive: false },
|
||||
{ field: "email", weight: 0.3, strategy: "exact", caseSensitive: false },
|
||||
{ field: "city", weight: 0.2, strategy: "phonetic", caseSensitive: false },
|
||||
];
|
||||
|
||||
function pairScore(a: Record<string, string>, b: Record<string, string>): number {
|
||||
return keys.reduce((sum, k) => {
|
||||
const va = (a[k.field] ?? "").toLowerCase();
|
||||
const vb = (b[k.field] ?? "").toLowerCase();
|
||||
return sum + k.weight * computeSimilarity(va, vb, k.strategy);
|
||||
}, 0);
|
||||
}
|
||||
|
||||
const score = pairScore(
|
||||
{ name: "John Smith", email: "j@example.com", city: "New York" },
|
||||
{ name: "Jon Smyth", email: "j@example.com", city: "New York" },
|
||||
);
|
||||
// score ≈ 0.93 — high confidence duplicate
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `levenshteinDistance(a, b): number`
|
||||
|
||||
Returns the minimum number of single-character edits (insertions, deletions, substitutions) required to transform `a` into `b`. Always ≥ 0. Runs in O(|a| × |b|) time with O(min(|a|, |b|)) space.
|
||||
|
||||
### `jaroSimilarity(s1, s2): number`
|
||||
|
||||
Jaro metric. Returns a value in [0, 1]. The matching window is `floor(max(|s1|, |s2|) / 2) - 1`. Identical strings return 1; completely different strings return 0.
|
||||
|
||||
### `jaroWinklerSimilarity(s1, s2): number`
|
||||
|
||||
Jaro-Winkler metric. Applies a prefix bonus: `jaro + prefix * 0.1 * (1 - jaro)` where `prefix` is the length of the shared prefix capped at 4. Returns a value in [0, 1].
|
||||
|
||||
### `exactMatch(a, b): number`
|
||||
|
||||
Returns `1` if `a === b`, `0` otherwise. No normalisation is applied; callers should lower-case inputs if case-insensitive comparison is required.
|
||||
|
||||
### `computeSimilarity(a, b, strategy): number`
|
||||
|
||||
Routes to the correct algorithm. For `"levenshtein"` the raw distance is normalised to [0, 1] by dividing by `max(|a|, |b|)` and subtracting from 1. For `"phonetic"`, both strings are first passed through a vowel-removal normaliser before Jaro-Winkler scoring.
|
||||
|
||||
**Params:**
|
||||
|
||||
- `a` — first string
|
||||
- `b` — second string
|
||||
- `strategy` — `"exact" | "jaro-winkler" | "levenshtein" | "phonetic"`
|
||||
|
||||
**Returns:** `number` in [0, 1]
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The two-row rolling array in `levenshteinDistance` avoids allocating an (|a|+1)×(|b|+1) matrix; only the previous and current rows are kept in memory. The shorter string is always placed on the column axis to minimise allocations.
|
||||
|
||||
The Jaro matching window formula uses `Math.max(|s1|, |s2|) / 2 - 1` and is floored, matching the original Winkler (1990) paper. Some implementations use `Math.ceil`; this library matches the floor variant which produces the widely-cited benchmark values (MARTHA/MARHTA ≈ 0.944 Jaro, ≈ 0.961 Jaro-Winkler).
|
||||
|
||||
The phonetic normaliser is intentionally lightweight — it is not a full Soundex or Metaphone implementation. It removes vowels and collapses consecutive duplicate consonants, which is sufficient for the common use case of matching first/last names with minor spelling variations. For production-grade phonetic matching, callers should supply a custom `ScoringRule`.
|
||||
82
docs/foundation/data/dedup/src/errors/dedup-error.md
Normal file
82
docs/foundation/data/dedup/src/errors/dedup-error.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# DedupError
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`DedupError` is the error substrate for the `@suaveplan/dedup` package. It extends the built-in `Error` class with a stable `code` field so callers can branch on specific failure modes without parsing message strings. Every error thrown by the dedup engine or merge strategies uses this class.
|
||||
|
||||
## Features
|
||||
|
||||
- Typed `code` field for reliable programmatic error handling
|
||||
- Two stable error codes: `DEDUP.INVALID_CONFIG` and `DEDUP.STRATEGY_ERROR`
|
||||
- Lightweight — no external dependencies; usable in any runtime environment
|
||||
- `name` set to `'DedupError'` for readable stack traces and `instanceof` checks
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DedupError, DEDUP_CODES } from '@suaveplan/dedup/errors';
|
||||
|
||||
try {
|
||||
engine.detect(entities);
|
||||
} catch (err) {
|
||||
if (err instanceof DedupError && err.code === DEDUP_CODES.INVALID_CONFIG) {
|
||||
console.error('Configuration is invalid:', err.message);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DedupError, DEDUP_CODES } from '@suaveplan/dedup/errors';
|
||||
|
||||
function safeMerge(engine: DedupEngine, group: DuplicateGroup, strategy: MergeStrategy) {
|
||||
try {
|
||||
return engine.merge(group, strategy);
|
||||
} catch (err) {
|
||||
if (err instanceof DedupError) {
|
||||
if (err.code === DEDUP_CODES.STRATEGY_ERROR) {
|
||||
// Fall back to manual strategy
|
||||
return engine.merge(group, manualStrategy);
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DEDUP_CODES`
|
||||
|
||||
```ts
|
||||
const DEDUP_CODES: {
|
||||
INVALID_CONFIG: 'DEDUP.INVALID_CONFIG';
|
||||
STRATEGY_ERROR: 'DEDUP.STRATEGY_ERROR';
|
||||
}
|
||||
```
|
||||
|
||||
- `INVALID_CONFIG` — thrown when `DedupConfig` fails Zod validation (missing required fields, invalid weights, etc.)
|
||||
- `STRATEGY_ERROR` — thrown when a merge strategy throws an unexpected error
|
||||
|
||||
### `DedupError`
|
||||
|
||||
```ts
|
||||
class DedupError extends Error {
|
||||
readonly code: string;
|
||||
constructor(message: string, code: string);
|
||||
}
|
||||
```
|
||||
|
||||
- `message` — human-readable description of the failure
|
||||
- `code` — one of the `DEDUP_CODES` constants
|
||||
- `name` — always `'DedupError'`
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
This package is a leaf (no `@suaveplan/*` runtime dependencies), so it cannot depend on `@suaveplan/error`. The local `DedupError` class provides the same `code`-based discriminator pattern used throughout the project without introducing a circular dependency. If `@suaveplan/error` is added as a dependency in the future, this class should be migrated to extend `SuaveplanError`.
|
||||
|
||||
All errors thrown by the engine and merge strategies include a `code` value drawn from `DEDUP_CODES`. Callers should always `instanceof DedupError` before reading `.code` to avoid narrowing mistakes when third-party errors propagate through the dedup boundary. The code constants are exported as a plain `const` object rather than an enum to keep the module tree-shakeable and to avoid TypeScript enum pitfalls in strict ESM environments.
|
||||
97
docs/foundation/data/dedup/src/schemas/dedup.schema.md
Normal file
97
docs/foundation/data/dedup/src/schemas/dedup.schema.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Dedup Schemas
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Zod schemas for the `@suaveplan/dedup` package's configuration types. These schemas serve as the single source of truth for runtime validation of `DedupConfig`, `MatchKey`, and `ScoringRule` objects supplied by callers. All engine constructors run inputs through these schemas before use, ensuring invalid configurations fail fast with clear error messages.
|
||||
|
||||
## Features
|
||||
|
||||
- `MatchKeySchema` — validates field path, weight (0–1), strategy, and optional case-sensitivity flag
|
||||
- `ScoringRuleSchema` — validates the name string of a scoring rule
|
||||
- `DedupConfigSchema` — composes the above with threshold and max-group-size defaults
|
||||
- Coerced defaults: `mergeThreshold` defaults to `0.85`, `maxGroupSize` to `100`
|
||||
- All schemas export inferred TypeScript types via `z.infer`
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DedupConfigSchema } from '@suaveplan/dedup';
|
||||
import type { DedupConfig } from '@suaveplan/dedup';
|
||||
|
||||
const raw = {
|
||||
matchKeys: [{ field: 'email', weight: 1, strategy: 'exact' }],
|
||||
scoringRules: [],
|
||||
};
|
||||
|
||||
const config: DedupConfig = DedupConfigSchema.parse(raw);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DedupConfigSchema } from '@suaveplan/dedup';
|
||||
|
||||
// Validate at API boundary before constructing engine
|
||||
function createEngine(raw: unknown) {
|
||||
const result = DedupConfigSchema.safeParse(raw);
|
||||
if (!result.success) {
|
||||
throw new Error(`Invalid dedup config: ${result.error.message}`);
|
||||
}
|
||||
return new DedupEngine(result.data);
|
||||
}
|
||||
|
||||
// Multi-key config with custom thresholds
|
||||
const config = DedupConfigSchema.parse({
|
||||
matchKeys: [
|
||||
{ field: 'email', weight: 0.6, strategy: 'exact' },
|
||||
{ field: 'firstName', weight: 0.2, strategy: 'jaro-winkler' },
|
||||
{ field: 'lastName', weight: 0.2, strategy: 'jaro-winkler' },
|
||||
],
|
||||
scoringRules: [],
|
||||
mergeThreshold: 0.75,
|
||||
maxGroupSize: 50,
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `MatchKeySchema`
|
||||
|
||||
```ts
|
||||
z.object({
|
||||
field: z.string().min(1),
|
||||
weight: z.number().min(0).max(1),
|
||||
strategy: z.enum(['exact', 'jaro-winkler', 'levenshtein', 'phonetic']),
|
||||
caseSensitive: z.boolean().optional(),
|
||||
})
|
||||
```
|
||||
|
||||
### `ScoringRuleSchema`
|
||||
|
||||
```ts
|
||||
z.object({
|
||||
name: z.string().min(1),
|
||||
})
|
||||
```
|
||||
|
||||
### `DedupConfigSchema`
|
||||
|
||||
```ts
|
||||
z.object({
|
||||
matchKeys: z.array(MatchKeySchema).min(1),
|
||||
scoringRules: z.array(ScoringRuleSchema),
|
||||
mergeThreshold: z.number().min(0).max(1).optional().default(0.85),
|
||||
maxGroupSize: z.number().int().positive().optional().default(100),
|
||||
})
|
||||
```
|
||||
|
||||
Inferred types (`MatchKey`, `ScoringRule`, `DedupConfig`) are exported from the package root alongside the schemas.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Schemas are defined in a dedicated `.schema.ts` file and excluded from coverage reporting (per project convention). The `ScoringRuleSchema` currently only validates `name` because the `evaluate` function is a runtime interface — Zod cannot validate function types. Callers providing custom scoring rules must implement the `ScoringRule` interface directly.
|
||||
|
||||
The `weight` field on `MatchKeySchema` is validated to the range `[0, 1]` but the engine does not enforce that all weights sum to exactly 1.0 — this is intentional. Weights below 1 produce a score ceiling below the default threshold, which is a valid configuration for partial-match scenarios. Callers should set `mergeThreshold` accordingly when using sub-unity weight sums.
|
||||
69
docs/foundation/data/diff/README.md
Normal file
69
docs/foundation/data/diff/README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# @suaveplan/diff - Zero-dependency RFC 6902 JSON Patch engine with structural diffing, patch application/inversion, linear history manager, sync manager, and rich path utilities
|
||||
|
||||
## Summary
|
||||
|
||||
Zero-dependency RFC 6902 JSON Patch engine for computing, applying, and inverting structural differences between JSON documents. The `diff(a, b)` function produces a minimal patch using index-based or LCS array strategies; `applyPatch` applies it; `revertPatch` undoes it; and `invertPatch` creates an inverse patch. A `HistoryManager` provides undo/redo with configurable depth and branch truncation on new edits. A `SyncManager` handles optimistic concurrency: apply local changes speculatively, then reconcile when the server responds. RFC 6901 JSON Pointer parsing and formatting is exposed as a standalone `path` module. Use this package for collaborative editing, undo/redo stacks, change-event streams, and any system where you need to track or transmit the delta between two object states rather than the full state itself.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [errors](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [diff-error](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff-error](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff-error](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff-error](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_- [diff](./src/diff/diff.md) — <!-- begin:auto-docs --> _(no submodules)_- [errors](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_ - [diff-error](./src/errors/diff-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_ - [history](./src/history/history.md) — <!-- begin:auto-docs --> _(no submodules)_- [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_ - [operations](./src/operations/operations.md) — <!-- begin:auto-docs --> _(no submodules)_- [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_ - [patch](./src/patch/patch.md) — <!-- begin:auto-docs --> _(no submodules)_- [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_ - [path](./src/path/path.md) — <!-- begin:auto-docs --> _(no submodules)_- [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_ - [sync](./src/sync/sync.md) — <!-- begin:auto-docs --> _(no submodules)_- [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_ - [utils](./src/utils/utils.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Packages
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| [`./path`](src/path/path.md) | RFC 6901 JSON Pointer parsing, formatting, and path utilities |
|
||||
| [`./utils`](src/utils/utils.md) | Deep clone, equality, hashing, and document path navigation |
|
||||
| [`./operations`](src/operations/operations.md) | RFC 6902 operation types, factories, validation, and inversion |
|
||||
| [`./diff`](src/diff/diff.md) | Structural diff producing minimal JSON Patch (index and LCS strategies) |
|
||||
| [`./patch`](src/patch/patch.md) | RFC 6902 patch application and revert with OTel integration |
|
||||
| [`./history`](src/history/history.md) | Linear undo/redo history manager |
|
||||
| [`./sync`](src/sync/sync.md) | Optimistic concurrency sync manager |
|
||||
| [`./errors`](src/errors/diff-error.md) | Structured error class with stable code registry |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```ts
|
||||
import { diff, applyPatch, createHistory, revertPatch } from "@suaveplan/diff";
|
||||
import type { JsonValue } from "@suaveplan/diff";
|
||||
|
||||
let doc: JsonValue = { name: "Alice", age: 30, tags: ["a", "b"] };
|
||||
const history = createHistory({ maxEntries: 100 });
|
||||
|
||||
// Compute and apply a patch
|
||||
const next: JsonValue = { name: "Alice", age: 31, tags: ["a", "b", "c"] };
|
||||
const patch = diff(doc, next);
|
||||
doc = applyPatch(doc, patch).doc;
|
||||
history.push(patch);
|
||||
|
||||
// Undo
|
||||
const undoPatch = history.undo()!;
|
||||
const snapshot = { name: "Alice", age: 30, tags: ["a", "b"] }; // pre-patch state
|
||||
doc = revertPatch(doc, undoPatch, snapshot);
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/diff
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
99
docs/foundation/data/diff/src/diff/diff.md
Normal file
99
docs/foundation/data/diff/src/diff/diff.md
Normal file
@@ -0,0 +1,99 @@
|
||||
# diff
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`diff.ts` computes minimal RFC 6902 JSON Patch operations that transform one `JsonValue` into another. It is the structural diffing engine at the heart of `@suaveplan/diff`: given a `left` (before) and `right` (after) document, `diff` produces a `Patch` that, when applied to `left`, yields `right`.
|
||||
|
||||
Two complementary array strategies are supported. The default `'index'` strategy is O(n) and suitable for most document structures. The `'lcs'` strategy uses the Myers diff algorithm (LCS table) to produce fewer, more semantically accurate operations when array element positions change — useful for lists of items that may have been reordered, inserted, or deleted.
|
||||
|
||||
## Features
|
||||
|
||||
- **`diff(left, right, options?): Patch`** — top-level structural diff entry point
|
||||
- **`'index'` strategy** — positional array comparison; O(n) time and space
|
||||
- **`'lcs'` strategy** — Myers diff / LCS-based minimal edit script; O(m×n) space
|
||||
- **`lcs<T>` function** — reusable generic LCS utility with custom equality predicate
|
||||
- Recursive descent through objects and arrays
|
||||
- Emits `add`, `remove`, and `replace` operations only — never `move`, `copy`, or `test`
|
||||
- Short-circuits on `deepEqual` at every level to avoid redundant operations
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { diff } from "@suaveplan/diff/diff";
|
||||
import { applyPatch } from "@suaveplan/diff/patch";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const left: JsonValue = { name: "Alice", age: 30, tags: ["a", "b"] };
|
||||
const right: JsonValue = { name: "Alice", age: 31, tags: ["a", "b", "c"] };
|
||||
|
||||
const patch = diff(left, right);
|
||||
// [
|
||||
// { op: "replace", path: ["age"], value: 31 },
|
||||
// { op: "add", path: ["tags", 2], value: "c" },
|
||||
// ]
|
||||
|
||||
const { doc } = applyPatch(left, patch);
|
||||
// doc deepEquals right ✓
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { diff, lcs } from "@suaveplan/diff/diff";
|
||||
import { applyPatch } from "@suaveplan/diff/patch";
|
||||
import { deepEqual } from "@suaveplan/diff/utils";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
// LCS strategy — better for reordered arrays
|
||||
const before: JsonValue = { items: ["a", "b", "c", "d"] };
|
||||
const after: JsonValue = { items: ["a", "c", "d", "e"] };
|
||||
|
||||
const indexPatch = diff(before, after, { arrayStrategy: "index" });
|
||||
// Positional: replaces b with c, c with d, d with e, adds e at end
|
||||
|
||||
const lcsPatch = diff(before, after, { arrayStrategy: "lcs" });
|
||||
// LCS: removes "b" at index 1, adds "e" at end — fewer operations
|
||||
|
||||
// Both patches transform before → after
|
||||
const r1 = applyPatch(before, indexPatch);
|
||||
const r2 = applyPatch(before, lcsPatch);
|
||||
deepEqual(r1.doc, after); // true
|
||||
deepEqual(r2.doc, after); // true
|
||||
|
||||
// Standalone LCS for custom equality
|
||||
const editScript = lcs(
|
||||
[1, 2, 3, 4],
|
||||
[1, 3, 4, 5],
|
||||
(a, b) => a === b,
|
||||
);
|
||||
// [keep 1, remove 2, keep 3, keep 4, add 5]
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DiffOptions`
|
||||
|
||||
```ts
|
||||
interface DiffOptions {
|
||||
arrayStrategy?: "index" | "lcs"; // default "index"
|
||||
}
|
||||
```
|
||||
|
||||
### `diff(left: JsonValue, right: JsonValue, options?: DiffOptions): Patch`
|
||||
|
||||
Computes an RFC 6902 patch from `left` to `right`. The result is a `Patch` (array of operations). Applying the patch to `left` yields a document deeply equal to `right`.
|
||||
|
||||
### `lcs<T>(a: T[], b: T[], eq: (x: T, y: T) => boolean): Array<LcsEntry<T>>`
|
||||
|
||||
Generic LCS implementation. Returns an edit script of `{ op: "keep" | "add" | "remove"; item: T }` entries. `eq` is a custom equality predicate.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The diff algorithm emits operations in a deterministic order: object removals first, then additions, then modifications. This ensures patches are composable and predictable.
|
||||
- The `'lcs'` array strategy tracks an `insertOffset` to compute correct absolute indices as it emits add/remove operations. This is required because RFC 6902 path indices refer to the document state at the moment each operation is applied.
|
||||
- `hashValue` (FNV-1a of `JSON.stringify`) is used as the LCS equality predicate to avoid deep-equality comparisons in the O(m×n) table fill, improving performance for large arrays of objects.
|
||||
- `deepEqual` is used to short-circuit at every recursive level — if left and right are already identical, no operations are emitted and the recursion terminates immediately.
|
||||
- Array index removals are emitted in reverse order by the `'index'` strategy so that index positions remain stable as each removal is applied.
|
||||
116
docs/foundation/data/diff/src/errors/diff-error.md
Normal file
116
docs/foundation/data/diff/src/errors/diff-error.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# diff-error
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`diff-error.ts` defines the error substrate for the `@suaveplan/diff` package. Every public API surface in the package throws a `DiffError` rather than a plain `Error`, giving callers a stable, machine-readable `code` property drawn from the `DIFF_CODES` registry. This makes error handling robust: you can match on `err.code` without parsing human-readable message text, and you can always narrow to a `DiffError` with `instanceof`.
|
||||
|
||||
The file intentionally avoids inheriting from `@suaveplan/error` so that `@suaveplan/diff` remains a true leaf package with no workspace runtime dependencies.
|
||||
|
||||
## Features
|
||||
|
||||
- `DIFF_CODES` constant object — five stable string codes covering all failure modes
|
||||
- `DiffCode` union type — narrows `string` to only valid diff error codes
|
||||
- `DiffErrorContext` interface — optional `path`, `operation`, and `reason` fields
|
||||
- `DiffError` class — extends `Error`, sets `name = "DiffError"`, attaches context
|
||||
- Fully typed: `readonly` fields prevent accidental mutation after construction
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { DiffError, DIFF_CODES } from "@suaveplan/diff/errors";
|
||||
|
||||
try {
|
||||
throw new DiffError("path not found", DIFF_CODES.PATH_INVALID, {
|
||||
path: ["a", "b"],
|
||||
operation: "remove",
|
||||
});
|
||||
} catch (err) {
|
||||
if (err instanceof DiffError) {
|
||||
console.error(err.code); // "DIFF.PATH_INVALID"
|
||||
console.error(err.path); // ["a", "b"]
|
||||
console.error(err.operation); // "remove"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { DiffError, DIFF_CODES, type DiffCode } from "@suaveplan/diff/errors";
|
||||
|
||||
function assertNoDiffError(err: unknown): asserts err is never {
|
||||
if (err instanceof DiffError) {
|
||||
const ctx = [err.operation, err.path?.join("/"), err.reason]
|
||||
.filter(Boolean)
|
||||
.join(" | ");
|
||||
throw new Error(`DiffError [${err.code}]: ${err.message} — ${ctx}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Wrapping lower-level errors
|
||||
function wrapApplyError(inner: unknown, path: string[]): DiffError {
|
||||
const reason = inner instanceof Error ? inner.message : String(inner);
|
||||
return new DiffError(`apply failed at /${path.join("/")}`, DIFF_CODES.APPLY_FAILED, {
|
||||
path,
|
||||
operation: "apply",
|
||||
reason,
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DIFF_CODES`
|
||||
|
||||
```ts
|
||||
const DIFF_CODES: {
|
||||
readonly INVALID_PATCH: "DIFF.INVALID_PATCH";
|
||||
readonly TEST_FAILED: "DIFF.TEST_FAILED";
|
||||
readonly PATH_INVALID: "DIFF.PATH_INVALID";
|
||||
readonly APPLY_FAILED: "DIFF.APPLY_FAILED";
|
||||
readonly INVERT_FAILED: "DIFF.INVERT_FAILED";
|
||||
};
|
||||
```
|
||||
|
||||
Stable string code registry. Always import codes from this object rather than inlining string literals.
|
||||
|
||||
### `DiffCode`
|
||||
|
||||
```ts
|
||||
type DiffCode = (typeof DIFF_CODES)[keyof typeof DIFF_CODES];
|
||||
```
|
||||
|
||||
Union of all valid diff error code strings.
|
||||
|
||||
### `DiffErrorContext`
|
||||
|
||||
```ts
|
||||
interface DiffErrorContext {
|
||||
path?: Path; // segment array of the JSON Pointer that failed
|
||||
operation?: string; // RFC 6902 op name ("add", "remove", "replace", etc.)
|
||||
reason?: string; // free-text explanation
|
||||
}
|
||||
```
|
||||
|
||||
### `DiffError`
|
||||
|
||||
```ts
|
||||
class DiffError extends Error {
|
||||
readonly code: DiffCode;
|
||||
readonly path?: Path;
|
||||
readonly operation?: string;
|
||||
readonly reason?: string;
|
||||
constructor(message: string, code: DiffCode, context?: DiffErrorContext): DiffError;
|
||||
}
|
||||
```
|
||||
|
||||
All context fields are `readonly` after construction. `name` is set to `"DiffError"` for reliable stack-trace display.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Inherits directly from `Error` (not `@suaveplan/error`) to keep this package dependency-free at runtime.
|
||||
- Context fields are assigned only when defined to avoid sparse objects with explicit `undefined` values; this keeps `JSON.stringify` output clean.
|
||||
- The `name = "DiffError"` assignment is in the constructor body (not a class field) to ensure the prototype chain is established first, which matters for `instanceof` checks across realm boundaries in bundled output.
|
||||
132
docs/foundation/data/diff/src/history/history.md
Normal file
132
docs/foundation/data/diff/src/history/history.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# history
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`history.ts` implements a linear undo/redo history manager for the `@suaveplan/diff` package. It stores a stack of forward patches, tracks the current position within that stack, and exposes `undo()` / `redo()` methods that return the patches the caller should apply to move backwards or forwards in time.
|
||||
|
||||
The history manager does not apply patches itself — it only manages the stack and communicates with the caller via returned `Patch` values. This keeps the manager decoupled from document state and allows it to be used with any document shape.
|
||||
|
||||
## Features
|
||||
|
||||
- **`push(patch)`** — records a forward patch, truncates redo history, enforces `maxEntries` limit
|
||||
- **`undo()`** — returns the patch at the current position (to be inverted by the caller); moves position back
|
||||
- **`redo()`** — returns the patch at the next position; moves position forward
|
||||
- **`canUndo()` / `canRedo()`** — guard predicates
|
||||
- **`clear()`** — resets all history
|
||||
- **`getState()`** — serialisable snapshot of patches + position
|
||||
- **`onChange(listener)`** — reactive subscription; returns unsubscribe function
|
||||
- `maxEntries` eviction — oldest entries dropped when the limit is exceeded
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { createHistory } from "@suaveplan/diff/history";
|
||||
import { applyPatch, revertPatch } from "@suaveplan/diff/patch";
|
||||
import { diff } from "@suaveplan/diff/diff";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
let doc: JsonValue = { a: 1 };
|
||||
const history = createHistory({ maxEntries: 50 });
|
||||
|
||||
// Make a change
|
||||
const patch = diff(doc, { a: 2 });
|
||||
doc = applyPatch(doc, patch).doc;
|
||||
history.push(patch);
|
||||
|
||||
// Undo
|
||||
if (history.canUndo()) {
|
||||
const undoPatch = history.undo()!;
|
||||
doc = revertPatch(doc, undoPatch, { a: 1 });
|
||||
}
|
||||
// doc is { a: 1 } again
|
||||
|
||||
// Redo
|
||||
if (history.canRedo()) {
|
||||
const redoPatch = history.redo()!;
|
||||
doc = applyPatch(doc, redoPatch).doc;
|
||||
}
|
||||
// doc is { a: 2 } again
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { createHistory } from "@suaveplan/diff/history";
|
||||
import type { HistoryState } from "@suaveplan/diff/history";
|
||||
|
||||
const history = createHistory({ maxEntries: 3 });
|
||||
|
||||
// Subscribe to changes
|
||||
const unsubscribe = history.onChange((state: HistoryState) => {
|
||||
console.log(`position=${state.position}, patches=${state.patches.length}`);
|
||||
});
|
||||
|
||||
history.push([{ op: "add", path: ["a"], value: 1 }]);
|
||||
history.push([{ op: "add", path: ["b"], value: 2 }]);
|
||||
history.push([{ op: "add", path: ["c"], value: 3 }]);
|
||||
history.push([{ op: "add", path: ["d"], value: 4 }]); // evicts oldest
|
||||
|
||||
// State: patches.length === 3, position === 3
|
||||
history.undo(); // position 2
|
||||
history.push([{ op: "add", path: ["e"], value: 5 }]); // truncates redo
|
||||
|
||||
// Serialise state for persistence
|
||||
const snapshot = history.getState();
|
||||
// { patches: [...], position: 3 }
|
||||
|
||||
// Clean up listener
|
||||
unsubscribe();
|
||||
|
||||
// Clear all
|
||||
history.clear();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `HistoryOptions`
|
||||
|
||||
```ts
|
||||
interface HistoryOptions {
|
||||
maxEntries?: number; // default 100
|
||||
}
|
||||
```
|
||||
|
||||
### `HistoryState`
|
||||
|
||||
```ts
|
||||
interface HistoryState {
|
||||
patches: Patch[];
|
||||
position: number; // index of the next redo entry
|
||||
}
|
||||
```
|
||||
|
||||
### `History`
|
||||
|
||||
```ts
|
||||
class History {
|
||||
constructor(options?: HistoryOptions): History;
|
||||
push(patch: Patch): void;
|
||||
undo(): Patch | undefined;
|
||||
redo(): Patch | undefined;
|
||||
canUndo(): boolean;
|
||||
canRedo(): boolean;
|
||||
clear(): void;
|
||||
getState(): HistoryState;
|
||||
onChange(listener: (state: HistoryState) => void): () => void;
|
||||
}
|
||||
```
|
||||
|
||||
### `createHistory(options?: HistoryOptions): History`
|
||||
|
||||
Factory function equivalent to `new History(options)`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `_position` points to the index of the _next_ entry to be applied on redo (one past the most recently applied patch). After `push`, `_position === patches.length`. After `undo`, `_position` decrements.
|
||||
- `push` slices `_patches` at `_position` before appending, which truncates any redo history above the current position — the standard non-branching linear model.
|
||||
- `maxEntries` eviction uses `slice` from the tail end of the array, discarding the oldest entries when the limit is exceeded.
|
||||
- `getState` returns a shallow copy of the patches array to prevent callers from inadvertently mutating internal state. Each inner patch is spread-copied.
|
||||
- Listener management uses a filter-based removal strategy to keep the implementation simple and avoid index arithmetic.
|
||||
134
docs/foundation/data/diff/src/operations/operations.md
Normal file
134
docs/foundation/data/diff/src/operations/operations.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# operations
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`operations.ts` is the RFC 6902 operation layer for `@suaveplan/diff`. It defines the six operation types (`add`, `remove`, `replace`, `move`, `copy`, `test`), factory functions for constructing them, a full structural validator, and inversion logic that computes an undo patch given the document state before the original patch was applied.
|
||||
|
||||
This module sits between the low-level path/utils layer and the high-level patch engine, providing typed building blocks that both the diff algorithm and the patch applier depend on.
|
||||
|
||||
## Features
|
||||
|
||||
- **Six operation factory functions** — `createAdd`, `createRemove`, `createReplace`, `createMove`, `createCopy`, `createTest`
|
||||
- **`validateOperation`** — structural validator; throws `DiffError(DIFF.INVALID_PATCH)` on any violation
|
||||
- **`validatePatch`** — validates an array of operations with per-index error messages
|
||||
- **`invertOperation`** — computes the logical inverse of a single operation against a snapshot document
|
||||
- **`invertPatch`** — computes the reverse patch, stepping through intermediate states so each inversion is accurate
|
||||
- **`operationsConflict`** — detects path-level conflicts between two operations for sync resolution
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
createAdd, createRemove, createReplace,
|
||||
validatePatch, type Patch,
|
||||
} from "@suaveplan/diff/operations";
|
||||
|
||||
const patch: Patch = [
|
||||
createAdd(["users", "-"], { name: "Carol" }),
|
||||
createReplace(["users", 0, "name"], "Alice Updated"),
|
||||
createRemove(["users", 1]),
|
||||
];
|
||||
|
||||
// Validate external patch from network/storage
|
||||
const safe = validatePatch(JSON.parse(rawJson));
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
invertPatch, invertOperation, operationsConflict,
|
||||
createMove, createCopy, createTest,
|
||||
type Patch, type Operation,
|
||||
} from "@suaveplan/diff/operations";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const doc: JsonValue = { a: 1, b: 2, items: ["x", "y"] };
|
||||
|
||||
const patch: Patch = [
|
||||
{ op: "replace", path: ["a"], value: 99 },
|
||||
{ op: "add", path: ["c"], value: "new" },
|
||||
];
|
||||
|
||||
// Compute undo patch
|
||||
const undo = invertPatch(doc, patch);
|
||||
// undo = [{ op: "remove", path: ["c"] }, { op: "replace", path: ["a"], value: 1 }]
|
||||
|
||||
// Conflict detection for sync
|
||||
const local: Operation = createReplace(["a"], 42);
|
||||
const remote: Operation = createReplace(["a"], 77);
|
||||
operationsConflict(local, remote); // true
|
||||
|
||||
const unrelated: Operation = createReplace(["b"], 10);
|
||||
operationsConflict(local, unrelated); // false
|
||||
|
||||
// Test operation validates document state
|
||||
const testOp = createTest(["a"], 1);
|
||||
// Will throw if doc.a !== 1 when applied
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### Types
|
||||
|
||||
```ts
|
||||
type AddOperation = { op: "add"; path: Path; value: JsonValue };
|
||||
type RemoveOperation = { op: "remove"; path: Path };
|
||||
type ReplaceOperation = { op: "replace"; path: Path; value: JsonValue };
|
||||
type MoveOperation = { op: "move"; from: Path; path: Path };
|
||||
type CopyOperation = { op: "copy"; from: Path; path: Path };
|
||||
type TestOperation = { op: "test"; path: Path; value: JsonValue };
|
||||
type Operation = AddOperation | RemoveOperation | ReplaceOperation
|
||||
| MoveOperation | CopyOperation | TestOperation;
|
||||
type Patch = Operation[];
|
||||
```
|
||||
|
||||
### `createAdd(path, value): AddOperation`
|
||||
|
||||
### `createRemove(path): RemoveOperation`
|
||||
|
||||
### `createReplace(path, value): ReplaceOperation`
|
||||
|
||||
### `createMove(from, path): MoveOperation`
|
||||
|
||||
### `createCopy(from, path): CopyOperation`
|
||||
|
||||
### `createTest(path, value): TestOperation`
|
||||
|
||||
Factory functions; all return plain objects with the given fields.
|
||||
|
||||
### `validateOperation(op: unknown): Operation`
|
||||
|
||||
Validates structure. Throws `DiffError(DIFF.INVALID_PATCH)` if:
|
||||
|
||||
- `op` is not a non-null object
|
||||
- `op.op` is not one of the six valid operation names
|
||||
- `op.path` is not an array
|
||||
- `op.from` is missing for `move`/`copy`
|
||||
- `op.value` is missing for `add`/`replace`/`test`
|
||||
|
||||
### `validatePatch(patch: unknown): Patch`
|
||||
|
||||
Validates an array of operations. Throws `DiffError(DIFF.INVALID_PATCH)` with per-index context.
|
||||
|
||||
### `invertOperation(doc: JsonValue, op: Operation): Operation`
|
||||
|
||||
Returns the logical inverse of `op` against snapshot `doc`. Throws `DiffError(DIFF.INVERT_FAILED)` if the document does not contain the value required to form the inverse (e.g., for `remove` when the path is absent).
|
||||
|
||||
### `invertPatch(doc: JsonValue, patch: Patch): Patch`
|
||||
|
||||
Computes the full inverse patch by stepping through intermediate document states. Returns operations in reverse order so the result can be applied directly to undo the original patch.
|
||||
|
||||
### `operationsConflict(a: Operation, b: Operation): boolean`
|
||||
|
||||
Returns `true` if the paths touched by `a` and `b` overlap (same path or one is a prefix of the other). Considers both `path` and `from` for `move`/`copy`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `invertPatch` uses an internal `applyOpToDoc` function rather than importing from `patch.ts` to avoid a circular dependency. The inline implementation is deliberately minimal.
|
||||
- The inversion of `copy` is `remove` at the target path — we cannot recover the pre-copy state without storing original values, and the copy source remains untouched.
|
||||
- `pathsOverlap` uses prefix matching: paths `["a"]` and `["a", "b"]` overlap because one is a prefix of the other. This is conservative — it may report conflicts that technically could coexist, but erring on the side of safety is correct for a sync engine.
|
||||
- All factory functions return plain objects, not class instances, for maximum serialisability and tree-shaking.
|
||||
114
docs/foundation/data/diff/src/patch/patch.md
Normal file
114
docs/foundation/data/diff/src/patch/patch.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# patch
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`patch.ts` is the RFC 6902 patch application engine for `@suaveplan/diff`. It applies a sequence of operations to a document, producing a `PatchResult` with the new document state, a count of applied operations, and any error messages encountered. The engine supports both fail-fast (default) and lenient (collect-errors) modes, atomic rollback, and emits OpenTelemetry spans for observability.
|
||||
|
||||
The `revertPatch` helper provides a convenient undo primitive: given the post-patch document, the original patch, and the pre-patch snapshot, it computes the inverse patch and applies it.
|
||||
|
||||
## Features
|
||||
|
||||
- **`applyPatch`** — applies all six RFC 6902 operation types to a document copy
|
||||
- **Atomic mode** (default) — rolls back to the original on any failure
|
||||
- **Fail-fast mode** (default) — throws `DiffError` at the first error
|
||||
- **Lenient mode** — `failFast: false` collects errors without throwing
|
||||
- **OTel integration** — emits `diff.apply` span with `outcome`, `duration_ms`, `applied_count` attributes
|
||||
- **`revertPatch`** — undo a patch using the pre-patch document snapshot
|
||||
- All mutations operate on a deep clone; the original document is never modified
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { applyPatch } from "@suaveplan/diff/patch";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const doc: JsonValue = { name: "Alice", age: 30 };
|
||||
|
||||
const { doc: updated, applied, errors } = applyPatch(doc, [
|
||||
{ op: "replace", path: ["age"], value: 31 },
|
||||
{ op: "add", path: ["city"], value: "NYC" },
|
||||
]);
|
||||
// updated: { name: "Alice", age: 31, city: "NYC" }
|
||||
// applied: 2, errors: []
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { applyPatch, revertPatch } from "@suaveplan/diff/patch";
|
||||
import { diff } from "@suaveplan/diff/diff";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const before: JsonValue = { items: [1, 2, 3], meta: { v: 1 } };
|
||||
const after: JsonValue = { items: [1, 3], meta: { v: 2 } };
|
||||
|
||||
// Compute and apply patch
|
||||
const patch = diff(before, after);
|
||||
const { doc: patched } = applyPatch(before, patch);
|
||||
|
||||
// Undo — requires the original doc
|
||||
const reverted = revertPatch(patched, patch, before);
|
||||
// reverted deepEquals before ✓
|
||||
|
||||
// Test operation — throws DiffError on mismatch (atomic rollback)
|
||||
try {
|
||||
applyPatch(before, [
|
||||
{ op: "test", path: ["meta", "v"], value: 999 }, // wrong value
|
||||
{ op: "replace", path: ["meta", "v"], value: 2 },
|
||||
]);
|
||||
} catch (e) {
|
||||
// e.code === "DIFF.TEST_FAILED"
|
||||
// before is unchanged (applyPatch never mutates its input)
|
||||
}
|
||||
|
||||
// Lenient mode — collect all errors, apply partial patch
|
||||
const result = applyPatch(before, [
|
||||
{ op: "remove", path: ["missing"] },
|
||||
{ op: "replace", path: ["meta", "v"], value: 5 },
|
||||
], { failFast: false, atomic: false });
|
||||
// result.errors contains the remove error; replace still applied
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `PatchResult`
|
||||
|
||||
```ts
|
||||
interface PatchResult {
|
||||
doc: JsonValue; // Document state after applying the patch
|
||||
applied: number; // Count of successfully applied operations
|
||||
errors: string[]; // Error messages (empty on full success)
|
||||
}
|
||||
```
|
||||
|
||||
### `PatchOptions`
|
||||
|
||||
```ts
|
||||
interface PatchOptions {
|
||||
failFast?: boolean; // default true — throw on first error
|
||||
atomic?: boolean; // default true — rollback on failure
|
||||
}
|
||||
```
|
||||
|
||||
### `applyPatch(doc, patch, options?): PatchResult`
|
||||
|
||||
Applies `patch` to `doc`. Never mutates `doc`. With `atomic: true` (default), any failure rolls back to the original clone. With `failFast: true` (default), throws a `DiffError` on the first failure. Returns `PatchResult`.
|
||||
|
||||
**Throws** `DiffError` with codes:
|
||||
|
||||
- `DIFF.TEST_FAILED` — a `test` operation value mismatch
|
||||
- `DIFF.APPLY_FAILED` — `move`/`copy` source path not found, or generic apply failure
|
||||
|
||||
### `revertPatch(doc, patch, originalDoc?): JsonValue`
|
||||
|
||||
Inverts `patch` (using `originalDoc` as the pre-patch snapshot) and applies the inverse to `doc`. Returns the reverted document. If `originalDoc` is omitted, falls back to `doc` (only correct for add-only patches).
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `applyOperation` mutates its `doc` argument in-place for performance, but `applyPatch` always passes a deep clone so the caller's original is never affected.
|
||||
- OTel integration uses a lazy require so packages that do not configure an OTel provider pay zero cost. The tracer is memoised in `_tracer` after first resolution.
|
||||
- The `_tracer` module-level variable is reset to `undefined` on module load, which is correct for test isolation — each test process gets a fresh module.
|
||||
- The `revertPatch` function delegates to `invertPatch` from `operations.ts`, which steps through intermediate document states to produce accurate inverses for `remove` and `replace` operations.
|
||||
129
docs/foundation/data/diff/src/path/path.md
Normal file
129
docs/foundation/data/diff/src/path/path.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# path
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`path.ts` implements RFC 6901 JSON Pointer parsing and serialisation for the `@suaveplan/diff` package. A `Path` is an `Array<string | number>` — numeric array indices are kept as `number` while object keys are kept as `string`. All public functions are pure and free of side-effects; they neither read nor write global state, making them safe to tree-shake and straightforward to test.
|
||||
|
||||
The module deliberately avoids a class-based design: individual functions are simpler to inline, optimise, and test in isolation, and the bundle impact is lower when callers import only what they need.
|
||||
|
||||
## Features
|
||||
|
||||
- **`parsePath`** — RFC 6901 pointer string to segment array; numeric tokens parsed as `number`
|
||||
- **`formatPath`** — segment array back to RFC 6901 pointer string
|
||||
- **`escapeSegment` / `unescapeSegment`** — per-segment `~` / `/` encoding with correct substitution order
|
||||
- **`appendPath` / `concatPaths`** — non-mutating path construction helpers
|
||||
- **`comparePaths`** — lexicographic ordering usable with `Array.sort`
|
||||
- **`pathEquals`** — structural equality without JSON serialisation overhead
|
||||
- **`isPrefix`** — tests whether one path is a prefix of another
|
||||
- **`parentPath` / `lastSegment`** — navigate the path hierarchy
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { parsePath, formatPath } from "@suaveplan/diff/path";
|
||||
|
||||
const path = parsePath("/users/0/email");
|
||||
// ["users", 0, "email"]
|
||||
|
||||
const pointer = formatPath(path);
|
||||
// "/users/0/email"
|
||||
|
||||
// root pointer
|
||||
parsePath(""); // []
|
||||
formatPath([]); // ""
|
||||
|
||||
// empty-string key
|
||||
parsePath("/"); // [""]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
parsePath, formatPath, appendPath, isPrefix,
|
||||
comparePaths, escapeSegment, unescapeSegment,
|
||||
} from "@suaveplan/diff/path";
|
||||
|
||||
// Escape/unescape
|
||||
escapeSegment("a/b~c"); // "a~1b~0c"
|
||||
unescapeSegment("a~1b~0c"); // "a/b~c"
|
||||
|
||||
// Build paths dynamically
|
||||
const base = parsePath("/config");
|
||||
const full = appendPath(base, "timeout");
|
||||
formatPath(full); // "/config/timeout"
|
||||
|
||||
// Check containment
|
||||
const parent = ["users", 0];
|
||||
const child = ["users", 0, "roles", 1];
|
||||
isPrefix(parent, child); // true
|
||||
|
||||
// Sort operations by path depth
|
||||
const paths = [["b"], ["a", "c"], ["a"]].map(p => p as (string | number)[]);
|
||||
paths.sort(comparePaths);
|
||||
// [["a"], ["a", "c"], ["b"]]
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `Path`
|
||||
|
||||
```ts
|
||||
type Path = Array<string | number>;
|
||||
```
|
||||
|
||||
Segment array representation of an RFC 6901 JSON Pointer.
|
||||
|
||||
### `escapeSegment(segment: string): string`
|
||||
|
||||
Applies RFC 6901 §3 escaping: `~` → `~0`, then `/` → `~1`.
|
||||
|
||||
### `unescapeSegment(segment: string): string`
|
||||
|
||||
Reverses escaping in the mandated order: `~1` → `/`, then `~0` → `~`.
|
||||
|
||||
### `parsePath(pointer: string): Path`
|
||||
|
||||
Parses an RFC 6901 pointer. Throws `DiffError` (code `DIFF.PATH_INVALID`) if the string is non-empty and does not start with `/`. Numeric-only tokens are returned as `number`.
|
||||
|
||||
### `formatPath(path: Path): string`
|
||||
|
||||
Serialises a `Path` to an RFC 6901 pointer. Empty path returns `""`.
|
||||
|
||||
### `appendPath(path: Path, segment: string | number): Path`
|
||||
|
||||
Returns a new path with `segment` appended; does not mutate the original.
|
||||
|
||||
### `concatPaths(a: Path, b: Path): Path`
|
||||
|
||||
Returns a new path that is the concatenation of `a` and `b`.
|
||||
|
||||
### `comparePaths(a: Path, b: Path): number`
|
||||
|
||||
Lexicographic comparator suitable for use with `Array.prototype.sort`. Numeric segments compare numerically; mixed-type segments compare by string representation. Returns negative / zero / positive.
|
||||
|
||||
### `pathEquals(a: Path, b: Path): boolean`
|
||||
|
||||
Returns `true` iff both paths have the same length and identical segments at every position.
|
||||
|
||||
### `isPrefix(prefix: Path, path: Path): boolean`
|
||||
|
||||
Returns `true` iff every segment of `prefix` equals the corresponding segment of `path` and `prefix.length <= path.length`.
|
||||
|
||||
### `parentPath(path: Path): Path`
|
||||
|
||||
Returns all segments except the last. Returns `[]` for an empty input.
|
||||
|
||||
### `lastSegment(path: Path): string | number | undefined`
|
||||
|
||||
Returns the last segment, or `undefined` for an empty path.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Numeric index detection uses `/^\d+$/.test(...)` which correctly rejects leading-zero tokens like `"01"` — those are treated as string keys per RFC 6901 (array indices must not have leading zeros in practice).
|
||||
- Segment escaping always escapes `~` before `/` to avoid double-escaping `~1` as `~01`.
|
||||
- All functions return new arrays rather than mutating inputs, which is critical for the immutable diff/patch pipeline built on top of them.
|
||||
- `comparePaths` falls back to string comparison for mixed segment types to produce a deterministic total order even when segment types differ across paths.
|
||||
141
docs/foundation/data/diff/src/sync/sync.md
Normal file
141
docs/foundation/data/diff/src/sync/sync.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# sync
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`sync.ts` implements an optimistic concurrency sync manager for the `@suaveplan/diff` package. It tracks local pending deltas and a monotonic sequence counter so that the caller can coordinate document changes between multiple clients or processes. When a remote delta arrives, it is compared against local pending deltas; if a path-level conflict is detected, one of three resolution strategies is applied: `local-wins`, `remote-wins`, or `reject`.
|
||||
|
||||
The sync manager does not apply patches to documents — it only returns which patch the caller should apply. This separation keeps the manager stateless with respect to document content and easy to integrate into any store or state machine.
|
||||
|
||||
## Features
|
||||
|
||||
- **`push(patch)`** — records a local delta, increments the sequence counter, notifies listeners
|
||||
- **`pull(delta)`** — processes an incoming remote delta, returns the rebased patch
|
||||
- **`local-wins`** — keeps local patches, discards conflicting remote operations
|
||||
- **`remote-wins`** — discards local patches, accepts full remote delta
|
||||
- **`reject`** — throws `Error("conflict")` and leaves both sides unaffected
|
||||
- **`onDelta(listener)`** — reactive subscription to locally pushed deltas; returns unsubscribe fn
|
||||
- **`reset()`** — clears pending queue and resets sequence counter
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { createSync } from "@suaveplan/diff/sync";
|
||||
|
||||
const sync = createSync({ conflictResolution: "local-wins" });
|
||||
|
||||
// Push a local change
|
||||
const delta = sync.push([{ op: "replace", path: ["name"], value: "Alice" }]);
|
||||
// delta: { patch, sequenceNumber: 1, timestamp: ... }
|
||||
|
||||
// Process a non-conflicting remote delta
|
||||
const rebasedPatch = sync.pull({
|
||||
patch: [{ op: "replace", path: ["age"], value: 31 }],
|
||||
sequenceNumber: 1,
|
||||
timestamp: Date.now(),
|
||||
});
|
||||
// rebasedPatch: same as remote delta (no conflict)
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { createSync, type SyncDelta } from "@suaveplan/diff/sync";
|
||||
|
||||
// Two peers: A (local-wins) and B (remote-wins)
|
||||
const peerA = createSync({ conflictResolution: "local-wins" });
|
||||
const peerB = createSync({ conflictResolution: "remote-wins" });
|
||||
|
||||
// Listen for local deltas on peerA
|
||||
const unsubA = peerA.onDelta((delta: SyncDelta) => {
|
||||
// Send delta to peerB
|
||||
peerB.pull(delta);
|
||||
});
|
||||
|
||||
// Both peers modify the same path — conflict
|
||||
peerA.push([{ op: "replace", path: ["score"], value: 10 }]); // local
|
||||
const remoteDelta: SyncDelta = {
|
||||
patch: [{ op: "replace", path: ["score"], value: 20 }],
|
||||
sequenceNumber: 2,
|
||||
timestamp: Date.now(),
|
||||
};
|
||||
|
||||
// local-wins: remote's conflicting op is filtered out
|
||||
const rebased = peerA.pull(remoteDelta);
|
||||
// rebased === [] (empty — local patch "wins" and remote op is discarded)
|
||||
|
||||
// reject strategy
|
||||
const strict = createSync({ conflictResolution: "reject" });
|
||||
strict.push([{ op: "replace", path: ["x"], value: 1 }]);
|
||||
try {
|
||||
strict.pull({ patch: [{ op: "replace", path: ["x"], value: 2 }], sequenceNumber: 1, timestamp: 0 });
|
||||
} catch (e) {
|
||||
console.error(e.message); // "conflict"
|
||||
}
|
||||
|
||||
unsubA();
|
||||
peerA.reset();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `SyncDelta`
|
||||
|
||||
```ts
|
||||
interface SyncDelta {
|
||||
patch: Patch;
|
||||
sequenceNumber: number;
|
||||
timestamp: number;
|
||||
}
|
||||
```
|
||||
|
||||
### `SyncOptions`
|
||||
|
||||
```ts
|
||||
interface SyncOptions {
|
||||
conflictResolution: "local-wins" | "remote-wins" | "reject";
|
||||
}
|
||||
```
|
||||
|
||||
### `SyncManager`
|
||||
|
||||
```ts
|
||||
class SyncManager {
|
||||
constructor(options: SyncOptions): SyncManager;
|
||||
push(patch: Patch): SyncDelta;
|
||||
pull(delta: SyncDelta): Patch;
|
||||
onDelta(listener: (delta: SyncDelta) => void): () => void;
|
||||
reset(): void;
|
||||
}
|
||||
```
|
||||
|
||||
#### `push(patch): SyncDelta`
|
||||
|
||||
Increments the sequence counter, wraps `patch` in a `SyncDelta`, adds it to the pending queue, notifies all `onDelta` listeners, and returns the delta for the caller to send to the remote.
|
||||
|
||||
#### `pull(delta): Patch`
|
||||
|
||||
Compares `delta` against all pending local deltas. On no conflict, returns `delta.patch`. On conflict, applies the resolution strategy and returns the patch the caller should apply (may be empty for `local-wins`).
|
||||
|
||||
**Throws** `Error("conflict")` when `conflictResolution === "reject"` and a conflict is detected.
|
||||
|
||||
#### `onDelta(listener): () => void`
|
||||
|
||||
Registers a listener called on each `push`. Returns an unsubscribe function.
|
||||
|
||||
#### `reset(): void`
|
||||
|
||||
Clears all pending deltas and resets the sequence counter to 0.
|
||||
|
||||
### `createSync(options: SyncOptions): SyncManager`
|
||||
|
||||
Factory function equivalent to `new SyncManager(options)`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Conflict detection delegates to `operationsConflict` from `operations.ts`, which uses path prefix matching. This is conservative: it may report conflicts that could technically coexist at different sub-paths, but erring on the side of safety prevents data corruption.
|
||||
- `local-wins` filters individual conflicting remote _operations_ rather than discarding the entire remote patch. This allows non-conflicting remote operations to still be applied.
|
||||
- `remote-wins` removes the entire conflicting _local_ delta from the pending queue, meaning the local change will not be re-applied. Callers should handle this by refreshing document state from the remote source.
|
||||
- The sequence counter advances on both `push` and `pull` to maintain a consistent ordering. When a remote delta has a higher sequence number than the local counter, the local counter is updated to avoid stale sequence numbers.
|
||||
138
docs/foundation/data/diff/src/utils/utils.md
Normal file
138
docs/foundation/data/diff/src/utils/utils.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# utils
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`utils.ts` provides the low-level primitives that the entire `@suaveplan/diff` pipeline is built upon: deep clone, deep equality, FNV-1a hashing, and a family of path-navigation helpers for immutable and mutable document manipulation. Every operation is JSON-safe — it operates exclusively on the `JsonValue` universe (`null | boolean | number | string | JsonValue[] | { [k: string]: JsonValue }`).
|
||||
|
||||
The path-navigation helpers implement the RFC 6902 semantics for `add`, `remove`, `replace`, `move`, and `copy` at the document level, freeing the higher-level patch engine from the mechanics of array splice and object key management.
|
||||
|
||||
## Features
|
||||
|
||||
- **`deepClone`** — recursive JSON-safe clone without `JSON.parse/stringify` overhead
|
||||
- **`deepEqual`** — structural equality for arbitrarily nested JSON values
|
||||
- **`hashValue`** — FNV-1a 32-bit hash of JSON serialisation (for LCS bucketing)
|
||||
- **`isObject` / `isPrimitive`** — type-guard predicates
|
||||
- **`getValueAtPath`** — safe read at a path; returns `undefined` on missing keys
|
||||
- **`setValueAtPath` / `setValueAtPathMut`** — immutable and in-place write
|
||||
- **`insertValueAtPath` / `insertValueAtPathMut`** — immutable and in-place insert (array splice)
|
||||
- **`deleteValueAtPath` / `deleteValueAtPathMut`** — immutable and in-place delete
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { deepClone, deepEqual, getValueAtPath } from "@suaveplan/diff/utils";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const doc: JsonValue = { users: [{ name: "Alice" }, { name: "Bob" }] };
|
||||
|
||||
const clone = deepClone(doc);
|
||||
clone.users[0].name = "Carol";
|
||||
console.log(doc.users[0].name); // "Alice" — original unchanged
|
||||
|
||||
deepEqual(doc, clone); // false — they now differ
|
||||
|
||||
const name = getValueAtPath(doc, ["users", 1, "name"]);
|
||||
// "Bob"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
setValueAtPath, insertValueAtPath, deleteValueAtPath,
|
||||
setValueAtPathMut, hashValue,
|
||||
} from "@suaveplan/diff/utils";
|
||||
import type { JsonValue } from "@suaveplan/diff/utils";
|
||||
|
||||
const doc: JsonValue = { items: ["a", "b", "c"] };
|
||||
|
||||
// Immutable set — original unchanged
|
||||
const updated = setValueAtPath(doc, ["items", 1], "B");
|
||||
// doc still: { items: ["a", "b", "c"] }
|
||||
// updated: { items: ["a", "B", "c"] }
|
||||
|
||||
// Array insert — existing elements shift right
|
||||
const inserted = insertValueAtPath(doc, ["items", 1], "X");
|
||||
// { items: ["a", "X", "b", "c"] }
|
||||
|
||||
// Append via "-" token
|
||||
const appended = insertValueAtPath(doc, ["items", "-"], "Z");
|
||||
// { items: ["a", "b", "c", "Z"] }
|
||||
|
||||
// Delete
|
||||
const deleted = deleteValueAtPath(doc, ["items", 0]);
|
||||
// { items: ["b", "c"] }
|
||||
|
||||
// In-place mutation (used internally by the patch engine)
|
||||
const mutable = deepClone(doc) as { items: string[] };
|
||||
setValueAtPathMut(mutable as JsonValue, ["items", 2], "C");
|
||||
|
||||
// Hash for LCS bucketing
|
||||
hashValue({ x: 1 }); // e.g. "3a2b1c4d"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `JsonValue`
|
||||
|
||||
```ts
|
||||
type JsonValue = null | boolean | number | string | JsonValue[] | { [k: string]: JsonValue };
|
||||
```
|
||||
|
||||
### `isObject(v: unknown): v is Record<string, unknown>`
|
||||
|
||||
Returns `true` for non-null, non-array plain objects.
|
||||
|
||||
### `isPrimitive(v: unknown): v is null | boolean | number | string`
|
||||
|
||||
Returns `true` for all primitive JSON value types.
|
||||
|
||||
### `deepClone<T>(value: T): T`
|
||||
|
||||
Recursively clones arrays and objects; returns primitives as-is. Does not use `JSON.parse/stringify`.
|
||||
|
||||
### `deepEqual(a: unknown, b: unknown): boolean`
|
||||
|
||||
Structural equality: handles `null`, same-type primitives, same-length arrays with equal elements, and objects with equal key sets and equal values. Returns `false` for type mismatches.
|
||||
|
||||
### `hashValue(value: unknown): string`
|
||||
|
||||
FNV-1a 32-bit hash of `JSON.stringify(value)`. Returns an 8-character lowercase hex string.
|
||||
|
||||
### `getValueAtPath(doc: JsonValue, path: Path): JsonValue | undefined`
|
||||
|
||||
Traverses `doc` following `path`. Returns `undefined` if any step fails (missing key, index out of range, traversing through a primitive).
|
||||
|
||||
### `setValueAtPath(doc: JsonValue, path: Path, value: JsonValue): JsonValue`
|
||||
|
||||
Returns a new deep clone of `doc` with `value` written at `path`. The original `doc` is not modified.
|
||||
|
||||
### `setValueAtPathMut(doc: JsonValue, path: Path, value: JsonValue): void`
|
||||
|
||||
Sets `value` at `path` in `doc` in-place.
|
||||
|
||||
### `insertValueAtPath(doc: JsonValue, path: Path, value: JsonValue): JsonValue`
|
||||
|
||||
Returns a new deep clone with `value` inserted. For arrays, elements at `>= index` shift right. The `"-"` segment appends to the array.
|
||||
|
||||
### `insertValueAtPathMut(doc: JsonValue, path: Path, value: JsonValue): void`
|
||||
|
||||
Inserts `value` in-place using array splice or object key assignment.
|
||||
|
||||
### `deleteValueAtPath(doc: JsonValue, path: Path): JsonValue`
|
||||
|
||||
Returns a new deep clone with the element at `path` removed. Array elements shift left.
|
||||
|
||||
### `deleteValueAtPathMut(doc: JsonValue, path: Path): void`
|
||||
|
||||
Removes the element at `path` in-place.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `deepClone` avoids `JSON.parse/stringify` to preserve type fidelity for values that would be mangled by JSON roundtrip (e.g., `undefined` in arrays becomes `null`). This also avoids the overhead of two serialisation passes.
|
||||
- `resolveParent` is a shared internal helper that walks to the parent of the target key, returning a `{ parent, key }` pair. This eliminates duplication between the six set/insert/delete variants.
|
||||
- The `"-"` append token is handled in both `setValueAtPathMut` and `insertValueAtPathMut` to match RFC 6902 §4.1 semantics for the `add` operation.
|
||||
- `hashValue` deliberately uses FNV-1a rather than a cryptographic hash for performance in the LCS hot path. Collision probability over realistic array sizes is negligible.
|
||||
123
docs/foundation/data/json/README.md
Normal file
123
docs/foundation/data/json/README.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# @suaveplan/json - Safe JSON parse/stringify with prototype-pollution protection, circular-ref detection, and OTel instrumentation
|
||||
|
||||
## Summary
|
||||
|
||||
Safe, enterprise-grade JSON utilities with prototype-pollution protection,
|
||||
circular-reference detection, and OpenTelemetry instrumentation.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [errors](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [json-error](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - [json-error](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [json-error](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [json-error](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_- [json-error](./src/errors/json-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_ - [safe-json](./src/safe-json/safe-json.md) — <!-- begin:auto-docs --> _(no submodules)_- [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_ - [types](./src/types/types.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/json
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import { safeParse, safeStringify, tryParse, isValidJSON, getStringifySize } from "@suaveplan/json";
|
||||
|
||||
// Safe parse — throws JsonError on invalid JSON or forbidden keys
|
||||
const obj = safeParse<{ a: number }>('{"a": 1}');
|
||||
|
||||
// Safe stringify — throws JsonError on circular references
|
||||
const json = safeStringify(obj);
|
||||
|
||||
// Non-throwing parse
|
||||
const val = tryParse("bad input"); // undefined
|
||||
|
||||
// Validity check (handles "null" correctly)
|
||||
isValidJSON("null"); // true
|
||||
isValidJSON("bad"); // false
|
||||
|
||||
// UTF-8 byte size of JSON representation
|
||||
getStringifySize({ hello: "world" }); // 15
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `@suaveplan/json` | Main entry — all public exports |
|
||||
| `@suaveplan/json/safe-json` | `safeParse`, `safeStringify`, `tryParse`, `isValidJSON`, `getStringifySize` |
|
||||
| `@suaveplan/json/errors` | `JsonError`, `JSON_CODES`, `JsonCode` |
|
||||
|
||||
## Error handling
|
||||
|
||||
Every failure is a structured `JsonError` with a stable code:
|
||||
|
||||
| Code | Trigger |
|
||||
|------|---------|
|
||||
| `JSON.PARSE.FAILED` | Invalid JSON syntax or aborted `AbortSignal` |
|
||||
| `JSON.PARSE.PROTOTYPE_POLLUTION` | Forbidden key (`__proto__`, `constructor`, `prototype`) |
|
||||
| `JSON.STRINGIFY.CIRCULAR` | True circular reference detected |
|
||||
| `JSON.STRINGIFY.FAILED` | Aborted signal or non-serialisable value (e.g. `BigInt`) |
|
||||
|
||||
```ts
|
||||
import { safeParse, JsonError, JSON_CODES } from "@suaveplan/json";
|
||||
|
||||
try {
|
||||
safeParse('{"__proto__": {}}');
|
||||
} catch (err) {
|
||||
if (err instanceof JsonError && err.code === JSON_CODES["PARSE.PROTOTYPE_POLLUTION"]) {
|
||||
console.error("Prototype pollution blocked");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Options
|
||||
|
||||
### `ParseOptions`
|
||||
|
||||
```ts
|
||||
interface ParseOptions {
|
||||
forbiddenKeys?: string[]; // default: ["__proto__", "constructor", "prototype"]
|
||||
maxDepth?: number; // default: 50
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
```
|
||||
|
||||
### `StringifyOptions`
|
||||
|
||||
```ts
|
||||
interface StringifyOptions {
|
||||
space?: number;
|
||||
signal?: AbortSignal;
|
||||
maxDepth?: number; // default: 100
|
||||
}
|
||||
```
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Measured on Intel Xeon E5-2696 v4 @ 2.20 GHz, Bun 1.3.13:
|
||||
|
||||
```text
|
||||
benchmark avg (min … max)
|
||||
-------------------------------------------
|
||||
safeParse small 9.84 µs/iter
|
||||
JSON.parse small 592.70 ns/iter
|
||||
safeStringify small 1.99 µs/iter
|
||||
JSON.stringify small 212.99 ns/iter
|
||||
safeParse medium 42.11 µs/iter
|
||||
safeStringify medium 11.92 µs/iter
|
||||
```
|
||||
|
||||
The overhead relative to the native functions comes from the OTel span
|
||||
creation, reviver/replacer function calls, and ancestor-stack tracking.
|
||||
In production with a no-op tracer the spans cost ~0 ns.
|
||||
|
||||
## Links
|
||||
|
||||
- [`src/safe-json/safe-json.md`](src/safe-json/safe-json.md) — implementation notes
|
||||
- [`src/errors/json-error.md`](src/errors/json-error.md) — error codes
|
||||
- [`src/types/types.md`](src/types/types.md) — option types
|
||||
|
||||
103
docs/foundation/data/json/src/errors/json-error.md
Normal file
103
docs/foundation/data/json/src/errors/json-error.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# json-error
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides the typed error class and stable code registry for `@suaveplan/json`.
|
||||
All error conditions produced by this package surface as `JsonError` instances
|
||||
with machine-readable codes so callers can branch on failure mode without
|
||||
parsing message strings.
|
||||
|
||||
## Features
|
||||
|
||||
- `JSON_CODES` registry — four stable codes under the `JSON.*` namespace
|
||||
- `JsonCode` union type for compile-time exhaustiveness checks
|
||||
- `JsonError` extends `SuaveplanError` — carries `code`, `cause`, and optional
|
||||
context payload; integrates with the OTel span-event adapter from
|
||||
`@suaveplan/error`
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { JsonError, JSON_CODES } from "@suaveplan/json/errors";
|
||||
|
||||
throw new JsonError(
|
||||
JSON_CODES["PARSE.FAILED"],
|
||||
"Unexpected token at position 0"
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import { JsonError, JSON_CODES } from "@suaveplan/json/errors";
|
||||
|
||||
function parseSafe(text: string): unknown {
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch (cause) {
|
||||
throw new JsonError(
|
||||
JSON_CODES["PARSE.FAILED"],
|
||||
`JSON parse failed: ${(cause as Error).message}`,
|
||||
{ cause }
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `JSON_CODES`
|
||||
|
||||
```ts
|
||||
const JSON_CODES: {
|
||||
"PARSE.FAILED": "JSON/PARSE.FAILED";
|
||||
"PARSE.PROTOTYPE_POLLUTION": "JSON/PARSE.PROTOTYPE_POLLUTION";
|
||||
"STRINGIFY.CIRCULAR": "JSON/STRINGIFY.CIRCULAR";
|
||||
"STRINGIFY.FAILED": "JSON/STRINGIFY.FAILED";
|
||||
};
|
||||
```
|
||||
|
||||
Stable, namespaced error codes produced by `defineCodes("JSON", [...])`.
|
||||
Each code is a string literal of the form `"JSON/<segment>"`.
|
||||
|
||||
### `JsonCode`
|
||||
|
||||
```ts
|
||||
type JsonCode = (typeof JSON_CODES)[keyof typeof JSON_CODES];
|
||||
```
|
||||
|
||||
Union of all four code literals. Use this type for exhaustive `switch`
|
||||
statements or discriminated-union handlers.
|
||||
|
||||
### `class JsonError`
|
||||
|
||||
```ts
|
||||
class JsonError extends SuaveplanError<JsonCode> {
|
||||
constructor(code: JsonCode, message: string, options?: SuaveplanErrorOptions);
|
||||
}
|
||||
```
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `code` | `JsonCode` | Machine-readable error code from `JSON_CODES`. |
|
||||
| `message` | `string` | Human-readable description of the error. |
|
||||
| `options` | `SuaveplanErrorOptions?` | Optional `cause`, `context`, and `meta` fields. |
|
||||
|
||||
**Throws** — never; constructor only constructs.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The constructor is flagged with `// biome-ignore lint/complexity/noUselessConstructor`
|
||||
because it widens the visibility of the `SuaveplanError` constructor from
|
||||
`protected` to `public`. This is required by the Biome linter which otherwise
|
||||
flags it as a no-op, but the widening is intentional and load-bearing.
|
||||
|
||||
Error codes are registered once at module load time via `defineCodes`. The
|
||||
`defineCodes` helper from `@suaveplan/error` is idempotent within a process
|
||||
(same namespace + same codes → same object reference), making it safe to import
|
||||
across multiple entry points without duplication.
|
||||
209
docs/foundation/data/json/src/safe-json/safe-json.md
Normal file
209
docs/foundation/data/json/src/safe-json/safe-json.md
Normal file
@@ -0,0 +1,209 @@
|
||||
# safe-json
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides enterprise-grade `JSON.parse` and `JSON.stringify` wrappers that
|
||||
defend against prototype pollution, circular references, and AbortSignal
|
||||
cancellation. Every error surface is a structured `JsonError` with a stable,
|
||||
machine-readable code — no raw `SyntaxError` instances escape this module.
|
||||
All I/O-critical entry points emit OpenTelemetry spans for distributed tracing.
|
||||
|
||||
## Features
|
||||
|
||||
- `safeParse` — parses JSON with a reviver that blocks forbidden keys
|
||||
(`__proto__`, `constructor`, `prototype` by default); configurable via
|
||||
`ParseOptions.forbiddenKeys`
|
||||
- `safeStringify` — stringifies with ancestor-stack cycle detection; DAG
|
||||
sharing (same object at two different paths) is allowed; only true cycles
|
||||
throw `STRINGIFY.CIRCULAR`
|
||||
- `tryParse` — non-throwing wrapper returning `undefined` on any error
|
||||
- `isValidJSON` — boolean check that correctly handles the `"null"` literal
|
||||
- `getStringifySize` — UTF-8 byte size of the JSON representation
|
||||
- OTel spans on `safeParse` and `safeStringify`
|
||||
- AbortSignal support on both `safeParse` and `safeStringify`
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { safeParse, safeStringify, tryParse, isValidJSON, getStringifySize } from "@suaveplan/json";
|
||||
|
||||
// Parse
|
||||
const obj = safeParse<{ a: number }>('{"a":1}');
|
||||
console.log(obj.a); // 1
|
||||
|
||||
// Stringify
|
||||
const json = safeStringify({ a: 1 });
|
||||
console.log(json); // '{"a":1}'
|
||||
|
||||
// Non-throwing parse
|
||||
const result = tryParse("not-json"); // undefined
|
||||
|
||||
// Validity check
|
||||
isValidJSON("null"); // true
|
||||
isValidJSON("bad"); // false
|
||||
|
||||
// Byte size
|
||||
getStringifySize({ hello: "world" }); // 15
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
safeParse,
|
||||
safeStringify,
|
||||
JsonError,
|
||||
JSON_CODES,
|
||||
type ParseOptions,
|
||||
type StringifyOptions,
|
||||
} from "@suaveplan/json";
|
||||
|
||||
// Block additional keys beyond the defaults
|
||||
const parseOpts: ParseOptions = {
|
||||
forbiddenKeys: ["__proto__", "constructor", "prototype", "eval"],
|
||||
};
|
||||
|
||||
try {
|
||||
const data = safeParse<unknown>('{"eval":"alert(1)"}', parseOpts);
|
||||
} catch (err) {
|
||||
if (err instanceof JsonError && err.code === JSON_CODES["PARSE.PROTOTYPE_POLLUTION"]) {
|
||||
console.error("Prototype pollution attempt blocked");
|
||||
}
|
||||
}
|
||||
|
||||
// AbortSignal cancellation
|
||||
const controller = new AbortController();
|
||||
setTimeout(() => controller.abort(), 10);
|
||||
|
||||
const stringifyOpts: StringifyOptions = {
|
||||
space: 2,
|
||||
signal: controller.signal,
|
||||
};
|
||||
|
||||
const largePayload = Array.from({ length: 10000 }, (_, i) => ({ id: i }));
|
||||
try {
|
||||
safeStringify(largePayload, stringifyOpts);
|
||||
} catch (err) {
|
||||
if (err instanceof JsonError && err.code === JSON_CODES["STRINGIFY.FAILED"]) {
|
||||
console.error("Stringify cancelled");
|
||||
}
|
||||
}
|
||||
|
||||
// Circular reference detection
|
||||
const node: Record<string, unknown> = { name: "root" };
|
||||
node.self = node;
|
||||
|
||||
try {
|
||||
safeStringify(node);
|
||||
} catch (err) {
|
||||
if (err instanceof JsonError && err.code === JSON_CODES["STRINGIFY.CIRCULAR"]) {
|
||||
console.error("Circular reference blocked");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `safeParse<T>(text, options?): T`
|
||||
|
||||
Parses a JSON string with prototype-pollution protection.
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `text` | `string` | Raw JSON text to parse. |
|
||||
| `options` | `ParseOptions?` | Optional parse configuration. |
|
||||
|
||||
**Returns** — The parsed value typed as `T`.
|
||||
|
||||
**Throws**
|
||||
|
||||
- `JsonError(PARSE.FAILED)` — invalid JSON syntax or aborted signal.
|
||||
- `JsonError(PARSE.PROTOTYPE_POLLUTION)` — a forbidden key was encountered.
|
||||
|
||||
---
|
||||
|
||||
### `safeStringify(value, options?): string`
|
||||
|
||||
Stringifies a value with circular-reference detection.
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `value` | `unknown` | Value to serialise. |
|
||||
| `options` | `StringifyOptions?` | Optional stringify configuration. |
|
||||
|
||||
**Returns** — The JSON string.
|
||||
|
||||
**Throws**
|
||||
|
||||
- `JsonError(STRINGIFY.CIRCULAR)` — a true cycle was detected.
|
||||
- `JsonError(STRINGIFY.FAILED)` — aborted signal or non-serialisable value (e.g. `BigInt`).
|
||||
|
||||
---
|
||||
|
||||
### `tryParse<T>(text, options?): T | undefined`
|
||||
|
||||
Non-throwing parse; returns `undefined` on any error.
|
||||
|
||||
**Parameters** — same as `safeParse`.
|
||||
|
||||
**Returns** — The parsed value or `undefined`.
|
||||
|
||||
---
|
||||
|
||||
### `isValidJSON(text): boolean`
|
||||
|
||||
Returns `true` if the string is parseable by `safeParse` (respects the default
|
||||
forbidden-key list). Returns `true` for `"null"` (unlike a naïve
|
||||
`tryParse(text) !== undefined` check).
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `text` | `string` | The string to test. |
|
||||
|
||||
**Returns** — `boolean`.
|
||||
|
||||
---
|
||||
|
||||
### `getStringifySize(value): number`
|
||||
|
||||
**Parameters**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `value` | `unknown` | Any JSON-serialisable value. |
|
||||
|
||||
**Returns** — UTF-8 byte count of `JSON.stringify(value)`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**Prototype pollution guard** — `JSON.parse` supports a reviver function that
|
||||
is called for every key-value pair during object construction. The reviver
|
||||
inspects each key against a `Set<string>` built from `forbiddenKeys` (default:
|
||||
`__proto__`, `constructor`, `prototype`). The root `""` key is exempt because
|
||||
the reviver is called with the empty string for the top-level result.
|
||||
|
||||
**Cycle detection** — The replacer receives `this` bound to the parent object at
|
||||
each step, which allows reconstructing the ancestor stack. When entering a
|
||||
child object we find the parent's index in the stack and truncate everything
|
||||
after it, then check whether the new value is already in the stack before
|
||||
pushing it. This correctly allows DAG sharing (the same object referenced
|
||||
at two sibling paths) while rejecting true back-edges.
|
||||
|
||||
**`isValidJSON` and the `"null"` edge case** — A naïve implementation using
|
||||
`tryParse(text) !== undefined` would return `false` for the `"null"` literal
|
||||
because `JSON.parse("null")` returns `null`. This module instead wraps
|
||||
`safeParse` in a `try/catch` and returns `true` on success.
|
||||
|
||||
**OTel instrumentation** — `safeParse` and `safeStringify` each create an
|
||||
active span via `trace.getTracer("@suaveplan/json").startActiveSpan(...)`.
|
||||
When no SDK is configured the tracer is a no-op, so there is zero overhead
|
||||
in un-instrumented environments.
|
||||
78
docs/foundation/data/json/src/types/types.md
Normal file
78
docs/foundation/data/json/src/types/types.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# types
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Defines the public option types consumed by `@suaveplan/json` operations.
|
||||
Centralising these interfaces in a dedicated `types/` module keeps the
|
||||
implementation files focused on logic and lets external callers import only the
|
||||
types they need without pulling in any runtime code.
|
||||
|
||||
## Features
|
||||
|
||||
- `ParseOptions` — controls forbidden-key list, max nesting depth, and abort
|
||||
signal for `safeParse` / `tryParse`
|
||||
- `StringifyOptions` — controls pretty-print spacing, abort signal, and
|
||||
circular-ref depth limit for `safeStringify`
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import type { ParseOptions } from "@suaveplan/json";
|
||||
|
||||
const opts: ParseOptions = {
|
||||
forbiddenKeys: ["__proto__", "constructor"],
|
||||
maxDepth: 20,
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```ts
|
||||
import type { ParseOptions, StringifyOptions } from "@suaveplan/json";
|
||||
|
||||
const controller = new AbortController();
|
||||
|
||||
const parseOpts: ParseOptions = {
|
||||
forbiddenKeys: ["__proto__", "constructor", "prototype", "eval"],
|
||||
maxDepth: 10,
|
||||
signal: controller.signal,
|
||||
};
|
||||
|
||||
const stringifyOpts: StringifyOptions = {
|
||||
space: 2,
|
||||
signal: controller.signal,
|
||||
maxDepth: 50,
|
||||
};
|
||||
|
||||
// Abort after 500 ms
|
||||
setTimeout(() => controller.abort(), 500);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `ParseOptions`
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `forbiddenKeys` | `string[]?` | `["__proto__", "constructor", "prototype"]` | Keys that trigger a `PARSE.PROTOTYPE_POLLUTION` error when encountered anywhere in the object tree. |
|
||||
| `maxDepth` | `number?` | `50` | Maximum JSON nesting depth; deeper structures throw `PARSE.FAILED`. |
|
||||
| `signal` | `AbortSignal?` | — | Optional cancellation token; aborted signals cause `safeParse` to throw `PARSE.FAILED`. |
|
||||
|
||||
### `StringifyOptions`
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `space` | `number?` | — | Forwarded verbatim to `JSON.stringify`; produces indented output when set. |
|
||||
| `signal` | `AbortSignal?` | — | Optional cancellation token; aborted signals cause `safeStringify` to throw `STRINGIFY.FAILED`. |
|
||||
| `maxDepth` | `number?` | `100` | Maximum depth before `safeStringify` throws `STRINGIFY.CIRCULAR`. |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Both interfaces are pure TypeScript — no runtime object is emitted. The types
|
||||
file is excluded from coverage measurement because there is nothing to execute.
|
||||
Default values are applied inside the respective implementation functions
|
||||
(`safeParse`, `safeStringify`) using object destructuring with defaults, keeping
|
||||
the type definitions free of runtime assumptions.
|
||||
112
docs/foundation/data/media/README.md
Normal file
112
docs/foundation/data/media/README.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# @suaveplan/media - Universal media detection, metadata extraction, and constraint validation. Magic-byte signature registry (pluggable), MIME-keyed type detection via @suaveplan/mime, Zod-validated constraints, typed MediaError hierarchy with stable MEDIA.* codes, OTel counters + histogram, injectable observability bundle, and an async validation pipeline supporting custom predicates.
|
||||
|
||||
Universal media detection, metadata extraction, and constraint
|
||||
validation. Combines a pluggable magic-byte `SignatureRegistry` with the
|
||||
MIME database from `@suaveplan/mime` to identify image, video, and audio
|
||||
artefacts from raw bytes, filenames, or both. Ships typed errors via
|
||||
`@suaveplan/error`, OpenTelemetry instrumentation, Zod-validated
|
||||
constraints, and an async validation pipeline that supports custom
|
||||
predicates and format-specific metadata extractors.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [detection](./src/detection/detection.md) — Byte-level and filename-level media type detection. The module pairs the pluggable `SignatureRegistry` from `signatures…
|
||||
- [detection](./src/detection/detection.md) — Byte-level and filename-level media type detection. The module pairs the pluggable `SignatureRegistry` from `signatures…
|
||||
- [errors](./src/errors/errors.md) — Owns the stable error-code registry, per-code metadata table, and the `MediaError` class that every public entrypoint t…
|
||||
- [errors](./src/errors/errors.md) — Owns the stable error-code registry, per-code metadata table, and the `MediaError` class that every public entrypoint t…
|
||||
- [metadata](./src/metadata/metadata.md) — Builds a `MediaMetadata` bundle (type, MIME, size, matched signature label) from raw bytes. The sync helper produces th…
|
||||
- [metadata](./src/metadata/metadata.md) — Builds a `MediaMetadata` bundle (type, MIME, size, matched signature label) from raw bytes. The sync helper produces th…
|
||||
- [observability](./src/observability/observability.md) — Constructs the OpenTelemetry instrument bundle threaded through every public entrypoint in `@suaveplan/media`. Each det…
|
||||
- [observability](./src/observability/observability.md) — Constructs the OpenTelemetry instrument bundle threaded through every public entrypoint in `@suaveplan/media`. Each det…
|
||||
- [signatures](./src/signatures/signatures.md) — Owns the magic-byte signature registry that powers byte-level detection. The module ships a frozen `BUILTIN_SIGNATURES`…
|
||||
- [signatures](./src/signatures/signatures.md) — Owns the magic-byte signature registry that powers byte-level detection. The module ships a frozen `BUILTIN_SIGNATURES`…
|
||||
- [validation](./src/validation/validation.md) — Validates media bytes against a `MediaConstraints` rule set covering size floors and ceilings, type allowlists, MIME al…
|
||||
- [validation](./src/validation/validation.md) — Validates media bytes against a `MediaConstraints` rule set covering size floors and ceilings, type allowlists, MIME al…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/media
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import {
|
||||
detectMediaType,
|
||||
getMetadata,
|
||||
validateMedia,
|
||||
} from "@suaveplan/media";
|
||||
|
||||
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a]);
|
||||
|
||||
detectMediaType(png); // "image"
|
||||
|
||||
const meta = getMetadata(png, { filename: "logo.png" });
|
||||
// { type: "image", mimeType: "image/png", size: 6, signatureLabel: "PNG" }
|
||||
|
||||
const result = validateMedia(png, {
|
||||
maxSize: 1_000_000,
|
||||
allowedTypes: ["image"],
|
||||
});
|
||||
if (!result.valid) console.warn(result.errors);
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
- [`detection/detection.md`](./src/detection/detection.md) — byte +
|
||||
filename detection helpers.
|
||||
- [`metadata/metadata.md`](./src/metadata/metadata.md) — sync + async
|
||||
metadata extraction.
|
||||
- [`validation/validation.md`](./src/validation/validation.md) —
|
||||
constraint checking, async predicate support, assert helper.
|
||||
- [`signatures/signatures.md`](./src/signatures/signatures.md) —
|
||||
pluggable magic-byte registry.
|
||||
- [`errors/errors.md`](./src/errors/errors.md) — `MEDIA_CODES`,
|
||||
`MediaError`, metadata table.
|
||||
- [`observability/observability.md`](./src/observability/observability.md)
|
||||
— OpenTelemetry bundle factory.
|
||||
- [`types/media.types.md`](./src/types/media.types.md) — public type
|
||||
contracts + Zod schemas.
|
||||
|
||||
## Features
|
||||
|
||||
- Magic-byte detection covering PNG, JPEG, GIF, BMP, WEBP/RIFF, MP4/MOV,
|
||||
WebM/MKV, MP3 (ID3 + sync), FLAC, OGG.
|
||||
- Pluggable `SignatureRegistry` with `register` / `unregister` / `list`
|
||||
/ `match` / `detect`; duplicate labels rejected with
|
||||
`MEDIA.DUPLICATE_SIGNATURE`.
|
||||
- MIME-keyed filename detection via `@suaveplan/mime`.
|
||||
- Zod-validated `MediaConstraints` (`minSize`, `maxSize`, `allowedTypes`,
|
||||
`allowedMimeTypes`, `minWidth`/`maxWidth`, `minHeight`/`maxHeight`,
|
||||
`maxDuration`, custom predicate).
|
||||
- Async metadata extractor hook for layering width / height / duration
|
||||
/ codec onto the base bundle.
|
||||
- `assertValidMedia` throwing variant for guard sites.
|
||||
- Eight stable `MEDIA.*` error codes with per-code HTTP / severity /
|
||||
retryable metadata.
|
||||
- Four counters + two histograms emitted from every public entrypoint.
|
||||
|
||||
## Error codes
|
||||
|
||||
| Code | HTTP | Retryable | Severity |
|
||||
| --------------------------------- | ---- | --------- | -------- |
|
||||
| `MEDIA.EMPTY_INPUT` | 400 | no | warning |
|
||||
| `MEDIA.INVALID_CONSTRAINTS` | 400 | no | error |
|
||||
| `MEDIA.VALIDATION_FAILED` | 422 | no | warning |
|
||||
| `MEDIA.UNSUPPORTED_TYPE` | 415 | no | warning |
|
||||
| `MEDIA.PREDICATE_FAILED` | 500 | no | error |
|
||||
| `MEDIA.DUPLICATE_SIGNATURE` | 409 | no | error |
|
||||
| `MEDIA.METADATA_EXTRACTOR_FAILED` | 500 | no | error |
|
||||
| `MEDIA.INVALID_SIGNATURE` | 400 | no | error |
|
||||
|
||||
## Observability
|
||||
|
||||
Four counters: `media_detect_total`, `media_metadata_total`,
|
||||
`media_validate_total`, `media_error_total`. Two histograms:
|
||||
`media_op_latency_ms`, `media_input_size_bytes`. Wire your meter via
|
||||
`createMediaObservability({ meter })` and thread the bundle into every
|
||||
call.
|
||||
|
||||
70
docs/foundation/data/media/src/detection/detection.md
Normal file
70
docs/foundation/data/media/src/detection/detection.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Detection
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Byte-level and filename-level media type detection. The module pairs the
|
||||
pluggable `SignatureRegistry` from `signatures/` with the MIME database
|
||||
exposed by `@suaveplan/mime` so consumers can identify image, video, and
|
||||
audio artefacts whether they have the bytes, the filename, or both.
|
||||
|
||||
## Features
|
||||
|
||||
- `detectMediaType(input, opts?)` — magic-byte detection via the registry.
|
||||
- `detectSignature(input, opts?)` — return the matched `MediaSignature`
|
||||
record, not just its `type`.
|
||||
- `detectMediaTypeFromName(filename, opts?)` — MIME-database lookup.
|
||||
- `inferMimeFromSignature(sig)` — canonical MIME for a matched signature
|
||||
with a sane fallback when no signature matched.
|
||||
- `defaultMimeForType(type)` — coarse fallback when only the
|
||||
category is known.
|
||||
- Counters and latency histograms emitted on every invocation.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { detectMediaType } from "@suaveplan/media";
|
||||
|
||||
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d]);
|
||||
detectMediaType(png); // "image"
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
detectSignature,
|
||||
inferMimeFromSignature,
|
||||
SignatureRegistry,
|
||||
} from "@suaveplan/media";
|
||||
|
||||
const reg = new SignatureRegistry();
|
||||
reg.register({
|
||||
type: "image",
|
||||
label: "AVIF",
|
||||
bytes: [0x66, 0x74, 0x79, 0x70, 0x61, 0x76, 0x69, 0x66],
|
||||
offset: 4,
|
||||
});
|
||||
const sig = detectSignature(myBytes, { registry: reg });
|
||||
const mime = inferMimeFromSignature(sig); // canonical MIME
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `DetectOptions` — `{ registry?, observability? }`.
|
||||
- `detectMediaType(input, opts?)` — `MediaType`.
|
||||
- `detectSignature(input, opts?)` — `MediaSignature | null`.
|
||||
- `detectMediaTypeFromName(filename, opts?)` — `MediaType`.
|
||||
- `inferMimeFromSignature(sig)` — canonical MIME string.
|
||||
- `defaultMimeForType(type)` — coarse fallback MIME.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`detectMediaType` and `detectMediaTypeFromName` both record an
|
||||
`op_latency_ms` histogram entry in a `finally` block so the cost of a
|
||||
detection call is always measured, even when the registry throws. The
|
||||
filename pathway uses `@suaveplan/mime`'s `getMimeType` so the database
|
||||
behind detection stays in lock-step with the rest of the project.
|
||||
67
docs/foundation/data/media/src/errors/errors.md
Normal file
67
docs/foundation/data/media/src/errors/errors.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Errors
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Owns the stable error-code registry, per-code metadata table, and the
|
||||
`MediaError` class that every public entrypoint throws or wraps. Every
|
||||
failure path in `@suaveplan/media` surfaces one of eight `MEDIA.*` codes
|
||||
so consumers (loggers, retry policies, telemetry pipelines, UI surfaces)
|
||||
can classify failures programmatically without parsing message strings.
|
||||
|
||||
## Features
|
||||
|
||||
- Frozen `MEDIA_CODES` registry created via `defineCodes("MEDIA", […])`.
|
||||
- Per-code `MediaCodeMetadata` record (`httpStatus`, `retryable`,
|
||||
`severity`, `description`) frozen at module load.
|
||||
- `MediaError` extending `SuaveplanError<MediaErrorCode>` — cause chain,
|
||||
redaction, OTel span-event adapter inherited.
|
||||
- `asMediaError(code, cause, fallback)` lift helper for `catch` sites.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { MediaError, MEDIA_CODES } from "@suaveplan/media";
|
||||
|
||||
throw new MediaError(MEDIA_CODES.EMPTY_INPUT, "input is empty");
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { asMediaError, MEDIA_CODES, MEDIA_ERROR_METADATA } from "@suaveplan/media";
|
||||
|
||||
try {
|
||||
/* boundary work */
|
||||
} catch (cause) {
|
||||
const err = asMediaError(
|
||||
MEDIA_CODES.PREDICATE_FAILED,
|
||||
cause,
|
||||
"predicate threw"
|
||||
);
|
||||
const meta = MEDIA_ERROR_METADATA[err.code];
|
||||
if (meta.retryable) {
|
||||
// schedule retry
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `MEDIA_CODES` — frozen code registry with eight namespaced keys.
|
||||
- `MediaErrorCode` — union of registry values.
|
||||
- `MEDIA_ERROR_METADATA` — frozen record keyed by code.
|
||||
- `MediaCodeMetadata` — per-code metadata shape.
|
||||
- `MediaError` — concrete `SuaveplanError` subclass with `.metadata` getter.
|
||||
- `asMediaError(code, cause, fallback)` — lift helper.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`defineCodes` produces namespaced keys (`MEDIA.EMPTY_INPUT`, …). The
|
||||
metadata table is `Object.freeze`'d at module top level so consumers
|
||||
cannot tamper at runtime. `asMediaError` preserves the original error as
|
||||
`.cause` when it's an `Error` instance, otherwise it wraps the stringified
|
||||
value so the cause chain is always populated.
|
||||
65
docs/foundation/data/media/src/metadata/metadata.md
Normal file
65
docs/foundation/data/media/src/metadata/metadata.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# Metadata
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Builds a `MediaMetadata` bundle (type, MIME, size, matched signature
|
||||
label) from raw bytes. The sync helper produces the base bundle in a
|
||||
single pass; the async helper layers a user-supplied
|
||||
`MediaMetadataExtractor` over the base so callers can fill in
|
||||
format-specific fields (width, height, duration) without forking the
|
||||
package.
|
||||
|
||||
## Features
|
||||
|
||||
- `getMetadata(input, opts?)` — synchronous, no extractor.
|
||||
- `getMetadataAsync(input, opts?)` — awaits the optional extractor and
|
||||
merges its return into the base bundle.
|
||||
- MIME resolution prefers the explicit `filename` field; otherwise it
|
||||
falls back to `inferMimeFromSignature`.
|
||||
- Signature label exposed via `metadata.signatureLabel` for downstream
|
||||
diagnostics.
|
||||
- Extractor errors are wrapped in `MediaError(MEDIA.METADATA_EXTRACTOR_FAILED)`
|
||||
with the original error chained as `.cause`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { getMetadata } from "@suaveplan/media";
|
||||
|
||||
const meta = getMetadata(new Uint8Array([0x89, 0x50, 0x4e, 0x47]), {
|
||||
filename: "logo.png",
|
||||
});
|
||||
console.log(meta.type, meta.mimeType, meta.size, meta.signatureLabel);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { getMetadataAsync } from "@suaveplan/media";
|
||||
|
||||
const meta = await getMetadataAsync(buf, {
|
||||
filename: "clip.mp4",
|
||||
extractor: async (_bytes, base) => {
|
||||
const { width, height, duration } = await probe(buf);
|
||||
return { ...base, width, height, duration };
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `MetadataOptions` — `{ filename?, registry?, observability?, extractor? }`.
|
||||
- `getMetadata(input, opts?)` — synchronous `MediaMetadata`.
|
||||
- `getMetadataAsync(input, opts?)` — `Promise<MediaMetadata>`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The base bundle is built inside a private helper so both entrypoints
|
||||
share signature detection + MIME resolution. The async variant only
|
||||
forks off the base when an extractor is supplied, so unused await paths
|
||||
incur near-zero overhead. Latency histograms are recorded in a
|
||||
`finally` block so failed extractor calls still produce telemetry.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Observability
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Constructs the OpenTelemetry instrument bundle threaded through every
|
||||
public entrypoint in `@suaveplan/media`. Each detection, metadata, and
|
||||
validation call increments a counter, records a latency histogram, and
|
||||
optionally emits a span. Consumers can inject their own meter / tracer
|
||||
to wire instruments into a shared telemetry pipeline.
|
||||
|
||||
## Features
|
||||
|
||||
- Four counters: `media_detect_total`, `media_metadata_total`,
|
||||
`media_validate_total`, `media_error_total`.
|
||||
- Two histograms: `media_op_latency_ms`, `media_input_size_bytes`.
|
||||
- Injectable meter / tracer or a pre-built `MediaObservability`.
|
||||
- Falls back to the global no-op providers when nothing is supplied,
|
||||
keeping the bundle safe to construct from library code.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { createMediaObservability } from "@suaveplan/media";
|
||||
|
||||
const obs = createMediaObservability();
|
||||
obs.counters.detectTotal.add(1, { kind: "magic-bytes" });
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
import { createMediaObservability } from "@suaveplan/media";
|
||||
|
||||
const meter = metrics.getMeter("my-app");
|
||||
const obs = createMediaObservability({ meter });
|
||||
obs.histograms.opLatencyMs.record(12.4, { op: "validateMedia" });
|
||||
obs.counters.errorTotal.add(1, { code: "MEDIA.VALIDATION_FAILED" });
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `MediaCounters` — counter set.
|
||||
- `MediaHistograms` — histogram set.
|
||||
- `MediaObservability` — bundle.
|
||||
- `MediaObservabilityOptions` — `{ meter?, tracer?, observability? }`.
|
||||
- `createMediaObservability(opts?)` — bundle factory.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
When `opts.observability` is supplied, it short-circuits before any
|
||||
instrument creation so injected bundles round-trip without mutation.
|
||||
Otherwise the function resolves a meter and tracer (defaulting to the
|
||||
global providers) and creates instruments via the helpers in
|
||||
`@suaveplan/opentelemetry` so naming + unit conventions stay consistent
|
||||
across the project.
|
||||
71
docs/foundation/data/media/src/signatures/signatures.md
Normal file
71
docs/foundation/data/media/src/signatures/signatures.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Signatures
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Owns the magic-byte signature registry that powers byte-level detection.
|
||||
The module ships a frozen `BUILTIN_SIGNATURES` list covering common image
|
||||
(PNG, JPEG, GIF, BMP, WEBP, RIFF), video (MP4/MOV, WebM/MKV), and audio
|
||||
(MP3, FLAC, OGG) formats, plus a mutable `SignatureRegistry` class that
|
||||
lets downstream consumers add or remove entries without forking the
|
||||
package.
|
||||
|
||||
## Features
|
||||
|
||||
- Frozen built-in registry covering twelve canonical formats.
|
||||
- `SignatureRegistry` class with `register`, `unregister`, `list`,
|
||||
`match`, and `detect` methods.
|
||||
- Per-entry `offset` so ftyp-style signatures (MP4) and RIFF chunk markers
|
||||
(WebP) can match at non-zero positions.
|
||||
- Duplicate-label detection: `register` throws
|
||||
`MEDIA.DUPLICATE_SIGNATURE` if a second entry uses the same label.
|
||||
- Byte validation: registration rejects empty byte arrays and non-byte
|
||||
numbers with `MEDIA.INVALID_SIGNATURE`.
|
||||
- Process-wide `defaultSignatureRegistry` seeded with the built-ins.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { defaultSignatureRegistry } from "@suaveplan/media";
|
||||
|
||||
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a]);
|
||||
const sig = defaultSignatureRegistry.match(png);
|
||||
console.log(sig?.label); // "PNG"
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { SignatureRegistry } from "@suaveplan/media";
|
||||
|
||||
const reg = new SignatureRegistry();
|
||||
reg.register({
|
||||
type: "image",
|
||||
label: "AVIF",
|
||||
bytes: [0x66, 0x74, 0x79, 0x70, 0x61, 0x76, 0x69, 0x66],
|
||||
offset: 4,
|
||||
});
|
||||
const type = reg.detect(myBytes);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `BUILTIN_SIGNATURES` — readonly array of twelve canonical signatures.
|
||||
- `SignatureRegistry` — mutable registry class.
|
||||
- `register(sig)` — add entry; throws on duplicate label / invalid bytes.
|
||||
- `unregister(label)` — remove by label; returns whether removed.
|
||||
- `list()` — frozen snapshot.
|
||||
- `match(input)` — first matching `MediaSignature` or `null`.
|
||||
- `detect(input)` — `MediaType`, defaulting to `"unknown"`.
|
||||
- `defaultSignatureRegistry` — process-wide default.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`match` honours each signature's `offset` so non-zero-anchored formats
|
||||
work correctly. The built-in list orders longer / more-specific
|
||||
signatures before shorter ones (RIFF is listed after WEBP because the
|
||||
WEBP marker lives inside a RIFF container at offset 8). The registry's
|
||||
internal array is private so external mutation cannot break ordering.
|
||||
69
docs/foundation/data/media/src/types/media.types.md
Normal file
69
docs/foundation/data/media/src/types/media.types.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# Media Types
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines every public type and Zod schema exported by `@suaveplan/media`.
|
||||
These contracts pin the package's surface so downstream consumers can write
|
||||
runtime-validated boundaries and compile against stable shapes. Types
|
||||
covered include `MediaType`, `MediaSignature`, `MediaMetadata`,
|
||||
`MediaConstraints`, `MediaResult`, the constraint predicate, and the async
|
||||
metadata-extractor hook.
|
||||
|
||||
## Features
|
||||
|
||||
- `MediaType` discriminated union (`image | video | audio | unknown`).
|
||||
- `MediaTypeSchema` and `MediaConstraintsSchema` Zod parsers for boundary
|
||||
validation.
|
||||
- `MediaSignature` model for magic-byte registry entries with optional
|
||||
non-zero offset.
|
||||
- Extended `MediaConstraints` (size floor + ceiling, MIME allowlist,
|
||||
duration ceiling, custom predicate).
|
||||
- `MediaMetadataExtractor` hook used by the metadata pipeline.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { MediaTypeSchema, type MediaMetadata } from "@suaveplan/media";
|
||||
|
||||
const parsed = MediaTypeSchema.parse("image");
|
||||
const meta: MediaMetadata = {
|
||||
type: parsed,
|
||||
mimeType: "image/png",
|
||||
size: 1024,
|
||||
};
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
MediaConstraintsSchema,
|
||||
type MediaConstraints,
|
||||
} from "@suaveplan/media";
|
||||
|
||||
const constraints: MediaConstraints = MediaConstraintsSchema.parse({
|
||||
maxSize: 2_000_000,
|
||||
allowedTypes: ["image"],
|
||||
allowedMimeTypes: ["image/png", "image/jpeg"],
|
||||
predicate: (m) => (m.size > 0 ? null : "empty"),
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `MediaType` — recognised media categories.
|
||||
- `MediaTypeSchema` — Zod enum for `MediaType`.
|
||||
- `MediaSignature` — magic-byte registry entry.
|
||||
- `MediaMetadata` — extracted metadata bundle.
|
||||
- `MediaConstraints` / `MediaConstraintsSchema` — validation rule set.
|
||||
- `MediaResult` — validation result with errors array.
|
||||
- `MediaConstraintPredicate` — custom validation function.
|
||||
- `MediaMetadataExtractor` — async metadata extension hook.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
All schemas use `z` default-imported per Bun-safety. The predicate field
|
||||
in `MediaConstraintsSchema` uses `z.custom` because Zod cannot statically
|
||||
describe arbitrary function shapes. Arrays surface as `readonly` arrays in
|
||||
TypeScript so consumers cannot mutate constraint collections after
|
||||
construction.
|
||||
78
docs/foundation/data/media/src/validation/validation.md
Normal file
78
docs/foundation/data/media/src/validation/validation.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Validation
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Validates media bytes against a `MediaConstraints` rule set covering
|
||||
size floors and ceilings, type allowlists, MIME allowlists, width /
|
||||
height / duration limits, and an optional custom predicate. The module
|
||||
exposes a synchronous helper for cheap byte-level checks and an async
|
||||
helper that also honours the constraint predicate plus the metadata
|
||||
extractor hook.
|
||||
|
||||
## Features
|
||||
|
||||
- `validateMedia(input, constraints, opts?)` — sync validation.
|
||||
- `validateMediaAsync(input, constraints, opts?)` — async validation
|
||||
that runs the predicate hook.
|
||||
- `assertValidMedia(...)` — throwing variant that yields
|
||||
`MediaError(MEDIA.VALIDATION_FAILED)` on failure.
|
||||
- Zod-validated constraints — malformed input throws
|
||||
`MediaError(MEDIA.INVALID_CONSTRAINTS)`; use `skipConstraintValidation`
|
||||
to opt out on hot paths.
|
||||
- Predicate failure wrapping — caller exceptions surface as
|
||||
`MediaError(MEDIA.PREDICATE_FAILED)` with the original error chained.
|
||||
- OTel `media_validate_total` counter + `media_op_latency_ms` histogram
|
||||
emitted on every invocation.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { validateMedia } from "@suaveplan/media";
|
||||
|
||||
const result = validateMedia(myBytes, {
|
||||
maxSize: 1_000_000,
|
||||
allowedTypes: ["image"],
|
||||
});
|
||||
if (!result.valid) {
|
||||
console.warn(result.errors);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { validateMediaAsync } from "@suaveplan/media";
|
||||
|
||||
const result = await validateMediaAsync(myBytes, {
|
||||
maxSize: 5_000_000,
|
||||
allowedTypes: ["image", "video"],
|
||||
allowedMimeTypes: ["image/png", "video/mp4"],
|
||||
maxWidth: 4096,
|
||||
maxHeight: 4096,
|
||||
maxDuration: 60,
|
||||
predicate: async (meta) =>
|
||||
meta.size === 0 ? "empty payload" : null,
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `ValidateOptions` — extends `MetadataOptions` with
|
||||
`skipConstraintValidation`.
|
||||
- `validateMedia(input, constraints, opts?)` — sync `MediaResult`.
|
||||
- `validateMediaAsync(input, constraints, opts?)` —
|
||||
`Promise<MediaResult>`.
|
||||
- `assertValidMedia(input, constraints, opts?)` — `MediaMetadata`,
|
||||
throws on validation failure.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`checkConstraints` is shared between the sync and async paths so the
|
||||
rule semantics are identical. Empty input is rejected up-front with
|
||||
`MEDIA.EMPTY_INPUT` before any metadata work happens. The predicate
|
||||
runs after every other check so callers can rely on the base errors
|
||||
being populated first.
|
||||
290
docs/foundation/data/mime/README.md
Normal file
290
docs/foundation/data/mime/README.md
Normal file
@@ -0,0 +1,290 @@
|
||||
# @suaveplan/mime - Structured MIME-type detection, parsing, and registry: magic-byte sniffing, RFC 7231 Content-Type parsing, IANA-sourced type database.
|
||||
|
||||
## Summary
|
||||
|
||||
Structured MIME-type detection, parsing, and registry for the Suaveplan platform. Provides RFC 7231 Content-Type parsing, magic-byte file signature detection, charset sniffing, and a 150+ entry IANA-sourced type database — all in a zero-dependency, tree-shakeable package.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [sniff-bom](./src/charset/sniff-bom.md) — Detects character encoding from a Byte Order Mark (BOM) at the start of a binary buffer. BOM detection is the highest-c…
|
||||
- [sniff-html-meta](./src/charset/sniff-html-meta.md) — Extracts a declared character encoding from an HTML `<meta>` tag by scanning the first 1,024 characters of an HTML docu…
|
||||
- [sniff-xml-decl](./src/charset/sniff-xml-decl.md) — Extracts the declared encoding from an XML or SVG document's processing instruction (`<?xml version="1.0" encoding="UTF…
|
||||
- [content-type-equals](./src/content-type/content-type-equals.md) — Provides two utility functions for comparing Content-Type values without caring about parameters: `contentTypeEquals` c…
|
||||
- [format-content-type](./src/content-type/format-content-type.md) — Serializes a `ParsedContentType` object back into a standards-compliant `Content-Type` header string. Handles quoting o…
|
||||
- [parse-content-type](./src/content-type/parse-content-type.md) — Parses an RFC 7231 / RFC 9110 `Content-Type` header value into a structured `ParsedContentType` object. Handles quoted…
|
||||
- [builtin-db](./src/db/builtin-db.md) — The built-in MIME type database for `@suaveplan/mime`. Contains over 150 `MimeEntry` records covering the most common w…
|
||||
- [registry](./src/db/registry.md) — The central in-memory MIME type database for `@suaveplan/mime`. Maintains two indexes over the built-in `MimeEntry` lis…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs --> - helpers — Low-level byte-inspection utilities used……
|
||||
- [archives](./src/detection/from-magic-bytes/signatures/archives.md) — Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all…
|
||||
- [audio](./src/detection/from-magic-bytes/signatures/audio.md) — Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container…
|
||||
- [data](./src/detection/from-magic-bytes/signatures/data.md) — Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers…
|
||||
- [documents](./src/detection/from-magic-bytes/signatures/documents.md) — Registers magic-byte signatures for document and data format files with the global signature registry. Covers office do…
|
||||
- [executables](./src/detection/from-magic-bytes/signatures/executables.md) — Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executa…
|
||||
- [fonts](./src/detection/from-magic-bytes/signatures/fonts.md) — Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and…
|
||||
- [images](./src/detection/from-magic-bytes/signatures/images.md) — Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vecto…
|
||||
- [model3d](./src/detection/from-magic-bytes/signatures/model3d.md) — Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CA…
|
||||
- [video](./src/detection/from-magic-bytes/signatures/video.md) — Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs --> - helpers — Low-level byte-inspection utilities used……
|
||||
- [helpers](./src/detection/from-magic-bytes/helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./src/detection/from-magic-bytes/registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [detect](./src/detection/detect.md) — The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic by…
|
||||
- [from-content-type](./src/detection/from-content-type.md) — Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wra…
|
||||
- [from-extension](./src/detection/from-extension.md) — Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lo…
|
||||
- [from-stream-reader](./src/detection/from-stream-reader.md) — Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` so…
|
||||
- [from-stream](./src/detection/from-stream.md) — Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
- [insufficient-bytes-error](./src/errors/insufficient-bytes-error.md) — Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries b…
|
||||
- [invalid-content-type-error](./src/errors/invalid-content-type-error.md) — Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and op…
|
||||
- [mime-error](./src/errors/mime-error.md) — The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_…
|
||||
- [parse-error](./src/errors/parse-error.md) — Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike…
|
||||
- [unknown-mime-type-error](./src/errors/unknown-mime-type-error.md) — Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecogn…
|
||||
- [unknown-signature-error](./src/errors/unknown-signature-error.md) — Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel…
|
||||
- [legacy](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [get-mime-type](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [telemetry](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [meters](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
<!-- end:auto-docs --> - helpers — Low-level byte-inspection utilities used……
|
||||
- [archives](./src/detection/from-magic-bytes/signatures/archives.md) — Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all…
|
||||
- [audio](./src/detection/from-magic-bytes/signatures/audio.md) — Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container…
|
||||
- [data](./src/detection/from-magic-bytes/signatures/data.md) — Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers…
|
||||
- [documents](./src/detection/from-magic-bytes/signatures/documents.md) — Registers magic-byte signatures for document and data format files with the global signature registry. Covers office do…
|
||||
- [executables](./src/detection/from-magic-bytes/signatures/executables.md) — Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executa…
|
||||
- [fonts](./src/detection/from-magic-bytes/signatures/fonts.md) — Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and…
|
||||
- [images](./src/detection/from-magic-bytes/signatures/images.md) — Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vecto…
|
||||
- [model3d](./src/detection/from-magic-bytes/signatures/model3d.md) — Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CA…
|
||||
- [video](./src/detection/from-magic-bytes/signatures/video.md) — Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_- helpers — Low-level byte-inspection utilities used……
|
||||
- [helpers](./src/detection/from-magic-bytes/helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./src/detection/from-magic-bytes/registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [detect](./src/detection/detect.md) — The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic by…
|
||||
- [from-content-type](./src/detection/from-content-type.md) — Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wra…
|
||||
- [from-extension](./src/detection/from-extension.md) — Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lo…
|
||||
- [from-stream-reader](./src/detection/from-stream-reader.md) — Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` so…
|
||||
- [from-stream](./src/detection/from-stream.md) — Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or…
|
||||
- [insufficient-bytes-error](./src/errors/insufficient-bytes-error.md) — Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries b…
|
||||
- [invalid-content-type-error](./src/errors/invalid-content-type-error.md) — Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and op…
|
||||
- [mime-error](./src/errors/mime-error.md) — The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_…
|
||||
- [parse-error](./src/errors/parse-error.md) — Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike…
|
||||
- [unknown-mime-type-error](./src/errors/unknown-mime-type-error.md) — Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecogn…
|
||||
- [unknown-signature-error](./src/errors/unknown-signature-error.md) — Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel…
|
||||
- [legacy](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [get-mime-type](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [telemetry](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [meters](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
|
||||
- helpers — Low-level byte-inspection utilities used……
|
||||
|
||||
- [archives](./src/detection/from-magic-bytes/signatures/archives.md) — Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all…
|
||||
- [audio](./src/detection/from-magic-bytes/signatures/audio.md) — Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container…
|
||||
- [data](./src/detection/from-magic-bytes/signatures/data.md) — Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers…
|
||||
- [documents](./src/detection/from-magic-bytes/signatures/documents.md) — Registers magic-byte signatures for document and data format files with the global signature registry. Covers office do…
|
||||
- [executables](./src/detection/from-magic-bytes/signatures/executables.md) — Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executa…
|
||||
- [fonts](./src/detection/from-magic-bytes/signatures/fonts.md) — Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and…
|
||||
- [images](./src/detection/from-magic-bytes/signatures/images.md) — Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vecto…
|
||||
- [model3d](./src/detection/from-magic-bytes/signatures/model3d.md) — Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CA…
|
||||
- [video](./src/detection/from-magic-bytes/signatures/video.md) — Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_- helpers — Low-level byte-inspection utilities used……
|
||||
- [helpers](./src/detection/from-magic-bytes/helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./src/detection/from-magic-bytes/registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [detect](./src/detection/detect.md) — The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic by…
|
||||
- [from-content-type](./src/detection/from-content-type.md) — Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wra…
|
||||
- [from-extension](./src/detection/from-extension.md) — Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lo…
|
||||
- [from-stream-reader](./src/detection/from-stream-reader.md) — Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` so…
|
||||
- [from-stream](./src/detection/from-stream.md) — Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or…
|
||||
- [insufficient-bytes-error](./src/errors/insufficient-bytes-error.md) — Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries b…
|
||||
- [invalid-content-type-error](./src/errors/invalid-content-type-error.md) — Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and op…
|
||||
- [mime-error](./src/errors/mime-error.md) — The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_…
|
||||
- [parse-error](./src/errors/parse-error.md) — Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike…
|
||||
- [unknown-mime-type-error](./src/errors/unknown-mime-type-error.md) — Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecogn…
|
||||
- [unknown-signature-error](./src/errors/unknown-signature-error.md) — Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel…
|
||||
- [legacy](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [get-mime-type](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [telemetry](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [meters](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
|
||||
- helpers — Low-level byte-inspection utilities used…
|
||||
|
||||
- [archives](./src/detection/from-magic-bytes/signatures/archives.md) — Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all…
|
||||
- [audio](./src/detection/from-magic-bytes/signatures/audio.md) — Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container…
|
||||
- [data](./src/detection/from-magic-bytes/signatures/data.md) — Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers…
|
||||
- [documents](./src/detection/from-magic-bytes/signatures/documents.md) — Registers magic-byte signatures for document and data format files with the global signature registry. Covers office do…
|
||||
- [executables](./src/detection/from-magic-bytes/signatures/executables.md) — Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executa…
|
||||
- [fonts](./src/detection/from-magic-bytes/signatures/fonts.md) — Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and…
|
||||
- [images](./src/detection/from-magic-bytes/signatures/images.md) — Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vecto…
|
||||
- [model3d](./src/detection/from-magic-bytes/signatures/model3d.md) — Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CA…
|
||||
- [video](./src/detection/from-magic-bytes/signatures/video.md) — Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_- helpers — Low-level byte-inspection utilities used…
|
||||
- [helpers](./src/detection/from-magic-bytes/helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./src/detection/from-magic-bytes/registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [detect](./src/detection/detect.md) — The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic by…
|
||||
- [from-content-type](./src/detection/from-content-type.md) — Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wra…
|
||||
- [from-extension](./src/detection/from-extension.md) — Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lo…
|
||||
- [from-stream-reader](./src/detection/from-stream-reader.md) — Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` so…
|
||||
- [from-stream](./src/detection/from-stream.md) — Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or…
|
||||
- [insufficient-bytes-error](./src/errors/insufficient-bytes-error.md) — Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries b…
|
||||
- [invalid-content-type-error](./src/errors/invalid-content-type-error.md) — Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and op…
|
||||
- [mime-error](./src/errors/mime-error.md) — The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_…
|
||||
- [parse-error](./src/errors/parse-error.md) — Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike…
|
||||
- [unknown-mime-type-error](./src/errors/unknown-mime-type-error.md) — Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecogn…
|
||||
- [unknown-signature-error](./src/errors/unknown-signature-error.md) — Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel…
|
||||
- [legacy](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [get-mime-type](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [telemetry](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [meters](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
- [archives](./src/detection/from-magic-bytes/signatures/archives.md) — Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all…
|
||||
- [audio](./src/detection/from-magic-bytes/signatures/audio.md) — Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container…
|
||||
- [data](./src/detection/from-magic-bytes/signatures/data.md) — Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers…
|
||||
- [documents](./src/detection/from-magic-bytes/signatures/documents.md) — Registers magic-byte signatures for document and data format files with the global signature registry. Covers office do…
|
||||
- [executables](./src/detection/from-magic-bytes/signatures/executables.md) — Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executa…
|
||||
- [fonts](./src/detection/from-magic-bytes/signatures/fonts.md) — Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and…
|
||||
- [images](./src/detection/from-magic-bytes/signatures/images.md) — Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vecto…
|
||||
- [model3d](./src/detection/from-magic-bytes/signatures/model3d.md) — Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CA…
|
||||
- [video](./src/detection/from-magic-bytes/signatures/video.md) — Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video…
|
||||
- [from-magic-bytes](./src/detection/from-magic-bytes/from-magic-bytes.md) — <!-- begin:auto-docs --> _(no submodules)_- [helpers](./src/detection/from-magic-bytes/helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./src/detection/from-magic-bytes/registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [detect](./src/detection/detect.md) — The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic by…
|
||||
- [from-content-type](./src/detection/from-content-type.md) — Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wra…
|
||||
- [from-extension](./src/detection/from-extension.md) — Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lo…
|
||||
- [from-stream-reader](./src/detection/from-stream-reader.md) — Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` so…
|
||||
- [from-stream](./src/detection/from-stream.md) — Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or…
|
||||
- [insufficient-bytes-error](./src/errors/insufficient-bytes-error.md) — Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries b…
|
||||
- [invalid-content-type-error](./src/errors/invalid-content-type-error.md) — Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and op…
|
||||
- [mime-error](./src/errors/mime-error.md) — The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_…
|
||||
- [parse-error](./src/errors/parse-error.md) — Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike…
|
||||
- [unknown-mime-type-error](./src/errors/unknown-mime-type-error.md) — Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecogn…
|
||||
- [unknown-signature-error](./src/errors/unknown-signature-error.md) — Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel…
|
||||
- [legacy](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_ - [get-mime-type](./src/legacy/get-mime-type.md) — <!-- begin:auto-docs --> _(no submodules)_- [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_ - [reverse](./src/reverse/reverse.md) — <!-- begin:auto-docs --> _(no submodules)_- [telemetry](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_ - [meters](./src/telemetry/meters.md) — <!-- begin:auto-docs --> _(no submodules)_- [error-codes](./src/error-codes.md) — Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlappi…
|
||||
|
||||
## Features
|
||||
|
||||
- **RFC 7231 Content-Type parsing** — full quoted-string and OWS support
|
||||
- **Magic-byte detection** — 150+ signatures across images, audio, video, archives, documents, executables, and fonts
|
||||
- **Stream detection** — peek bytes from a `ReadableStream` or `AsyncIterable` without losing the original stream
|
||||
- **Charset sniffing** — BOM, HTML `<meta>` tag, and XML declaration detection
|
||||
- **Multi-source confidence scoring** — merge bytes + extension + Content-Type with agreement boost
|
||||
- **Reverse lookup** — MIME type → canonical file extension
|
||||
- **OpenTelemetry metrics** — optional OTel instrumentation
|
||||
- **100% test coverage** — every branch and statement verified
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/mime
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```typescript
|
||||
import { detect } from "@suaveplan/mime";
|
||||
|
||||
// Multi-source detection with confidence scoring
|
||||
const result = detect({
|
||||
bytes: fileBuffer,
|
||||
filename: "photo.jpg",
|
||||
contentType: "image/jpeg",
|
||||
});
|
||||
console.log(result.mime); // "image/jpeg"
|
||||
console.log(result.confidence); // 1.0 (all three sources agree)
|
||||
```
|
||||
|
||||
## Module Reference
|
||||
|
||||
| Module | Description | Docs |
|
||||
|--------|-------------|------|
|
||||
| `@suaveplan/mime` | Main entry: `detect()`, `parseContentType()`, `formatContentType()` | [src/index.ts](src/index.ts) |
|
||||
| `@suaveplan/mime/content-type` | RFC 7231 parser, formatter, equality/match utilities | [parse-content-type.md](src/content-type/parse-content-type.md) |
|
||||
| `@suaveplan/mime/detection/detect` | Unified multi-source detector | [detect.md](src/detection/detect.md) |
|
||||
| `@suaveplan/mime/detection/from-stream` | Stream-based detection with head/tail replay | [from-stream.md](src/detection/from-stream.md) |
|
||||
| `@suaveplan/mime/detection/from-magic-bytes` | Raw magic-byte detection API | [from-magic-bytes.md](src/detection/from-magic-bytes/from-magic-bytes.md) |
|
||||
| `@suaveplan/mime/charset/sniff-bom` | BOM detection from byte buffers | [sniff-bom.md](src/charset/sniff-bom.md) |
|
||||
| `@suaveplan/mime/charset/sniff-html-meta` | HTML `<meta charset>` detection | [sniff-html-meta.md](src/charset/sniff-html-meta.md) |
|
||||
| `@suaveplan/mime/charset/sniff-xml-decl` | XML encoding declaration detection | [sniff-xml-decl.md](src/charset/sniff-xml-decl.md) |
|
||||
| `@suaveplan/mime/db/registry` | MIME database: forward and reverse lookup | [registry.md](src/db/registry.md) |
|
||||
| `@suaveplan/mime/reverse` | MIME type → file extension reverse lookup | [reverse.md](src/reverse/reverse.md) |
|
||||
| `@suaveplan/mime/errors` | Typed error classes | [mime-error.md](src/errors/mime-error.md) |
|
||||
| `@suaveplan/mime/telemetry/meters` | Optional OTel metrics instrumentation | [meters.md](src/telemetry/meters.md) |
|
||||
| `@suaveplan/mime/legacy` | `getMimeType()` compat shim | [get-mime-type.md](src/legacy/get-mime-type.md) |
|
||||
|
||||
## API Examples
|
||||
|
||||
### Content-Type Parsing
|
||||
|
||||
```typescript
|
||||
import { parseContentType, formatContentType } from "@suaveplan/mime/content-type";
|
||||
|
||||
const ct = parseContentType("text/html; charset=UTF-8; format=flowed");
|
||||
console.log(ct.type); // "text"
|
||||
console.log(ct.subtype); // "html"
|
||||
console.log(ct.essence); // "text/html"
|
||||
console.log(ct.parameters.charset); // "utf-8" (lowercased)
|
||||
|
||||
console.log(formatContentType(ct)); // "text/html; charset=utf-8; format=flowed"
|
||||
```
|
||||
|
||||
### Magic-Byte Detection
|
||||
|
||||
```typescript
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const pngBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const result = detectFromMagicBytes(pngBytes);
|
||||
console.log(result?.mime); // "image/png"
|
||||
console.log(result?.priority); // 100
|
||||
```
|
||||
|
||||
### Stream Detection
|
||||
|
||||
```typescript
|
||||
import { detectFromStream } from "@suaveplan/mime/detection/from-stream";
|
||||
|
||||
const { mime, head, tail } = await detectFromStream(uploadStream);
|
||||
console.log(mime); // e.g. "video/mp4"
|
||||
|
||||
// Replay the full stream (head + tail)
|
||||
const chunks: Uint8Array[] = [head];
|
||||
for await (const chunk of tail) chunks.push(chunk);
|
||||
```
|
||||
|
||||
### Charset Sniffing
|
||||
|
||||
```typescript
|
||||
import { sniffBom } from "@suaveplan/mime/charset/sniff-bom";
|
||||
import { sniffHtmlMeta } from "@suaveplan/mime/charset/sniff-html-meta";
|
||||
|
||||
const bom = sniffBom(fileBytes); // "utf-8" | "utf-16le" | null
|
||||
const meta = sniffHtmlMeta(htmlText); // "iso-8859-1" | null
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
```typescript
|
||||
import { InvalidContentTypeError, UnknownMimeTypeError } from "@suaveplan/mime/errors";
|
||||
import { parseContentType } from "@suaveplan/mime/content-type";
|
||||
|
||||
try {
|
||||
parseContentType("not valid");
|
||||
} catch (err) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
console.error("Bad header:", err.context?.raw);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Registering Custom Signatures
|
||||
|
||||
```typescript
|
||||
import { registerSignature } from "@suaveplan/mime/detection/from-magic-bytes/registry";
|
||||
|
||||
registerSignature({
|
||||
mime: "application/x-my-format",
|
||||
priority: 75,
|
||||
minBytes: 4,
|
||||
match: (bytes) =>
|
||||
bytes[0] === 0x4d && bytes[1] === 0x59 &&
|
||||
bytes[2] === 0x46 && bytes[3] === 0x4d,
|
||||
});
|
||||
```
|
||||
|
||||
## Coverage
|
||||
|
||||
This package maintains **100% statement, branch, function, and line coverage** — a requirement for Layer 0/1 foundation packages in Suaveplan.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
59
docs/foundation/data/mime/src/charset/sniff-bom.md
Normal file
59
docs/foundation/data/mime/src/charset/sniff-bom.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# sniff-bom
|
||||
|
||||
## Purpose
|
||||
|
||||
Detects character encoding from a Byte Order Mark (BOM) at the start of a binary buffer. BOM detection is the highest-confidence method of charset identification — when present, a BOM unambiguously identifies the encoding without relying on declared metadata or heuristics.
|
||||
|
||||
## Features
|
||||
|
||||
- Detects UTF-8 BOM (`EF BB BF`)
|
||||
- Detects UTF-16 BE BOM (`FE FF`)
|
||||
- Detects UTF-16 LE BOM (`FF FE`)
|
||||
- Detects UTF-32 BE BOM (`00 00 FE FF`)
|
||||
- Detects UTF-32 LE BOM (`FF FE 00 00`)
|
||||
- Accepts any `ArrayBuffer`, `SharedArrayBuffer`, or `ArrayBufferView` (`Uint8Array`, `DataView`, etc.)
|
||||
- Returns the IANA charset name in lowercase, or `null` when no BOM is present
|
||||
- Zero allocation path for no-BOM case
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { sniffBom } from "@suaveplan/mime/charset/sniff-bom";
|
||||
|
||||
const bytes = new Uint8Array([0xef, 0xbb, 0xbf, 0x3c, 0x68, 0x74, 0x6d, 0x6c]);
|
||||
const charset = sniffBom(bytes);
|
||||
console.log(charset); // "utf-8"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { sniffBom } from "@suaveplan/mime/charset/sniff-bom";
|
||||
import { sniffHtmlMeta } from "@suaveplan/mime/charset/sniff-html-meta";
|
||||
|
||||
async function detectEncoding(buffer: Uint8Array): Promise<string> {
|
||||
// BOM takes precedence over meta-tag declarations
|
||||
const bomCharset = sniffBom(buffer);
|
||||
if (bomCharset) return bomCharset;
|
||||
|
||||
// Fall back to meta-tag detection
|
||||
const text = new TextDecoder("ascii", { fatal: false }).decode(buffer.slice(0, 1024));
|
||||
return sniffHtmlMeta(text) ?? "utf-8";
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `sniffBom(buffer: ArrayBufferView | ArrayBuffer): string | null`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `buffer` | `ArrayBufferView \| ArrayBuffer` | The byte buffer to inspect (any view or raw buffer). |
|
||||
|
||||
**Returns:** The IANA charset name in lowercase, or `null` when no BOM is detected.
|
||||
|
||||
**Throws:** Never — all buffer accesses are bounds-checked.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
UTF-32 BOMs (both BE and LE) must be tested before UTF-16 LE because the UTF-32 LE BOM (`FF FE 00 00`) starts with the same two bytes as the UTF-16 LE BOM (`FF FE`). The implementation checks the 4-byte sequences first to avoid misidentification. The function uses direct `Uint8Array` byte comparisons for maximum performance — no string allocations or regex matching.
|
||||
65
docs/foundation/data/mime/src/charset/sniff-html-meta.md
Normal file
65
docs/foundation/data/mime/src/charset/sniff-html-meta.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# sniff-html-meta
|
||||
|
||||
## Purpose
|
||||
|
||||
Extracts a declared character encoding from an HTML `<meta>` tag by scanning the first 1,024 characters of an HTML document. Handles both the modern HTML5 `<meta charset="...">` form and the legacy HTML4 `<meta http-equiv="Content-Type" content="...; charset=...">` form using a single regex that matches `charset=` in any attribute context.
|
||||
|
||||
## Features
|
||||
|
||||
- Scans up to 1,024 bytes (configurable via the slice parameter)
|
||||
- Matches HTML5 `<meta charset="...">` tags
|
||||
- Matches HTML4 `<meta http-equiv="Content-Type" content="...; charset=...">` tags
|
||||
- Case-insensitive matching for both tag names and attribute values
|
||||
- Returns the charset name in lowercase for canonical comparison
|
||||
- Returns `null` when no charset declaration is found
|
||||
- Zero allocation on the no-match path
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { sniffHtmlMeta } from "@suaveplan/mime/charset/sniff-html-meta";
|
||||
|
||||
const html = '<html><head><meta charset="UTF-8"><title>Hello</title>';
|
||||
const charset = sniffHtmlMeta(html);
|
||||
console.log(charset); // "utf-8"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { sniffHtmlMeta } from "@suaveplan/mime/charset/sniff-html-meta";
|
||||
import { sniffBom } from "@suaveplan/mime/charset/sniff-bom";
|
||||
|
||||
function detectHtmlEncoding(bytes: Uint8Array): string {
|
||||
// BOM has highest priority
|
||||
const bom = sniffBom(bytes);
|
||||
if (bom) return bom;
|
||||
|
||||
// Decode as ASCII/Latin-1 to scan for meta tags
|
||||
const head = new TextDecoder("latin1").decode(bytes.slice(0, 1024));
|
||||
const meta = sniffHtmlMeta(head);
|
||||
|
||||
// HTML5 spec default is utf-8 when no declaration is found
|
||||
return meta ?? "utf-8";
|
||||
}
|
||||
|
||||
// Legacy HTML4 http-equiv form
|
||||
const legacyHtml = '<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">';
|
||||
console.log(sniffHtmlMeta(legacyHtml)); // "iso-8859-1"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `sniffHtmlMeta(source: string): string | null`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `source` | `string` | HTML content (or the first chunk thereof). |
|
||||
|
||||
**Returns:** The lowercase IANA charset name, or `null` when no charset declaration is found in the first 1,024 characters.
|
||||
|
||||
**Throws:** Never.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
A single regex (`/<meta[^>]+charset\s*=\s*["']?([A-Za-z0-9_\-:]+)/i`) covers both the HTML5 and HTML4 forms because `charset=` appears literally in the `content` attribute of the `http-equiv` tag. A separate legacy regex for the full `http-equiv` pattern is unnecessary — the simpler pattern matches earlier and always wins. The 1,024-character scan window follows the HTML5 parsing spec for prescan byte-sequence limits.
|
||||
64
docs/foundation/data/mime/src/charset/sniff-xml-decl.md
Normal file
64
docs/foundation/data/mime/src/charset/sniff-xml-decl.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# sniff-xml-decl
|
||||
|
||||
## Purpose
|
||||
|
||||
Extracts the declared encoding from an XML or SVG document's processing instruction (`<?xml version="1.0" encoding="UTF-8"?>`). XML documents commonly declare their encoding in the prolog, and this declaration must be trusted before any BOM or HTTP header when present, per the XML specification (RFC 7303).
|
||||
|
||||
## Features
|
||||
|
||||
- Matches the `encoding` attribute in `<?xml ... encoding="..."?>` declarations
|
||||
- Supports both double-quoted and single-quoted attribute values
|
||||
- Case-insensitive matching — `ENCODING` and `encoding` both match
|
||||
- Scans only the first 256 characters (the XML prolog is always at the start)
|
||||
- Returns the encoding name in lowercase for canonical comparison
|
||||
- Returns `null` when no XML declaration or encoding attribute is found
|
||||
- Handles whitespace variations between attributes
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { sniffXmlDecl } from "@suaveplan/mime/charset/sniff-xml-decl";
|
||||
|
||||
const xml = '<?xml version="1.0" encoding="UTF-8"?><root/>';
|
||||
const encoding = sniffXmlDecl(xml);
|
||||
console.log(encoding); // "utf-8"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { sniffXmlDecl } from "@suaveplan/mime/charset/sniff-xml-decl";
|
||||
import { sniffBom } from "@suaveplan/mime/charset/sniff-bom";
|
||||
|
||||
function detectXmlEncoding(bytes: Uint8Array): string {
|
||||
// BOM overrides XML declaration per RFC 7303 §4.3
|
||||
const bom = sniffBom(bytes);
|
||||
if (bom) return bom;
|
||||
|
||||
// Use latin-1 to safely read the ASCII-range prolog
|
||||
const prolog = new TextDecoder("latin1").decode(bytes.slice(0, 256));
|
||||
const declared = sniffXmlDecl(prolog);
|
||||
|
||||
return declared ?? "utf-8"; // XML default encoding
|
||||
}
|
||||
|
||||
// Single-quoted attribute
|
||||
const svgWithSingleQuotes = "<?xml version='1.0' encoding='iso-8859-1'?><svg/>";
|
||||
console.log(sniffXmlDecl(svgWithSingleQuotes)); // "iso-8859-1"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `sniffXmlDecl(source: string): string | null`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `source` | `string` | XML/SVG content (or the first chunk thereof). |
|
||||
|
||||
**Returns:** The lowercase IANA encoding name, or `null` when no encoding attribute is found.
|
||||
|
||||
**Throws:** Never.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The regex `/<\?xml[^?]*encoding\s*=\s*["']([^"']+)["']/i` intentionally uses `[^?]*` (non-greedy via exclusion) rather than `.*?` to avoid backtracking across large documents when no `<?xml` declaration is present. The 256-character scan window is sufficient because the XML specification mandates that the encoding declaration, if present, must appear in the document prolog before any other content.
|
||||
@@ -0,0 +1,79 @@
|
||||
# content-type-equals
|
||||
|
||||
## Purpose
|
||||
|
||||
Provides two utility functions for comparing Content-Type values without caring about parameters: `contentTypeEquals` checks exact `type/subtype` equality (case-insensitive), and `contentTypeMatches` supports wildcard patterns (`*/*`, `image/*`). Both functions accept either raw header strings or pre-parsed `ParsedContentType` objects for zero-overhead comparisons.
|
||||
|
||||
## Features
|
||||
|
||||
- `contentTypeEquals`: exact essence comparison, ignoring parameters
|
||||
- `contentTypeMatches`: glob-style pattern matching (`*/*`, `image/*`, `text/plain`)
|
||||
- Accepts raw strings or `ParsedContentType` objects for both arguments
|
||||
- Case-insensitive for raw string inputs
|
||||
- Malformed patterns return `false` instead of throwing
|
||||
- Zero-cost fast path when a `ParsedContentType` is passed directly (no re-parse)
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import {
|
||||
contentTypeEquals,
|
||||
contentTypeMatches,
|
||||
} from "@suaveplan/mime/content-type/content-type-equals";
|
||||
|
||||
// Exact equality (parameters ignored)
|
||||
contentTypeEquals("text/html; charset=utf-8", "text/html"); // true
|
||||
contentTypeEquals("Text/HTML", "text/html"); // true (case-insensitive)
|
||||
contentTypeEquals("text/html", "text/plain"); // false
|
||||
|
||||
// Wildcard matching
|
||||
contentTypeMatches("image/png", "image/*"); // true
|
||||
contentTypeMatches("image/png", "*/*"); // true
|
||||
contentTypeMatches("text/html", "image/*"); // false
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import {
|
||||
contentTypeEquals,
|
||||
contentTypeMatches,
|
||||
} from "@suaveplan/mime/content-type/content-type-equals";
|
||||
import { parseContentType } from "@suaveplan/mime/content-type/parse-content-type";
|
||||
|
||||
// Pre-parse once and compare many times
|
||||
const incoming = parseContentType(request.headers.get("content-type") ?? "");
|
||||
|
||||
// Exact match against known types
|
||||
const isJson = contentTypeEquals(incoming, "application/json");
|
||||
|
||||
// Accept any image type
|
||||
const isImage = contentTypeMatches(incoming, "image/*");
|
||||
|
||||
// Build a router
|
||||
const handlers: Array<{ pattern: string; fn: (req: Request) => Response }> = [
|
||||
{ pattern: "application/json", fn: handleJson },
|
||||
{ pattern: "multipart/form-data", fn: handleMultipart },
|
||||
{ pattern: "text/*", fn: handleText },
|
||||
];
|
||||
|
||||
for (const { pattern, fn } of handlers) {
|
||||
if (contentTypeMatches(incoming, pattern)) {
|
||||
return fn(request);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `contentTypeEquals(a: string | ParsedContentType, b: string | ParsedContentType): boolean`
|
||||
|
||||
Returns `true` when both values have the same `type/subtype` essence, ignoring all parameters and case differences.
|
||||
|
||||
### `contentTypeMatches(value: string | ParsedContentType, pattern: string): boolean`
|
||||
|
||||
Returns `true` when the value's essence matches the pattern. The pattern may use `*` as a wildcard for either type or subtype. Returns `false` if the pattern is malformed (no slash).
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Both functions normalize inputs through `parseContentType` when a raw string is provided. The pattern parser in `contentTypeMatches` uses a simple string split on `/` rather than re-using the full parser, which avoids the overhead of parameter parsing and error handling for patterns. Wildcard matching is implemented with direct string comparison after split — there is no full glob engine, only the two valid HTTP wildcard forms (`*/*` and `type/*`).
|
||||
@@ -0,0 +1,71 @@
|
||||
# format-content-type
|
||||
|
||||
## Purpose
|
||||
|
||||
Serializes a `ParsedContentType` object back into a standards-compliant `Content-Type` header string. Handles quoting of parameter values that contain non-token characters, ensuring the output is always valid for use in HTTP headers and is round-trip safe with `parseContentType`.
|
||||
|
||||
## Features
|
||||
|
||||
- Produces the canonical `type/subtype` essence string
|
||||
- Appends parameters as `; name=value` pairs in insertion order
|
||||
- Automatically quotes values containing non-token characters (spaces, special chars)
|
||||
- Does not quote simple token values (no unnecessary quotes)
|
||||
- Round-trip safe: `parseContentType(formatContentType(x)).essence === x.essence`
|
||||
- Accepts both `ParsedContentType` objects and ad-hoc objects with the same shape
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { parseContentType } from "@suaveplan/mime/content-type/parse-content-type";
|
||||
import { formatContentType } from "@suaveplan/mime/content-type/format-content-type";
|
||||
|
||||
const ct = parseContentType("text/html; charset=utf-8");
|
||||
const formatted = formatContentType(ct);
|
||||
console.log(formatted); // "text/html; charset=utf-8"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { formatContentType } from "@suaveplan/mime/content-type/format-content-type";
|
||||
|
||||
// Construct a content type programmatically
|
||||
const ct = {
|
||||
type: "multipart",
|
||||
subtype: "form-data",
|
||||
essence: "multipart/form-data",
|
||||
parameters: Object.freeze({ boundary: "--my boundary with spaces" }),
|
||||
};
|
||||
console.log(formatContentType(ct));
|
||||
// 'multipart/form-data; boundary="--my boundary with spaces"'
|
||||
|
||||
// Values needing quoting are detected automatically
|
||||
const withSpecial = {
|
||||
type: "text",
|
||||
subtype: "plain",
|
||||
essence: "text/plain",
|
||||
parameters: Object.freeze({ name: 'file with "quotes".txt' }),
|
||||
};
|
||||
console.log(formatContentType(withSpecial));
|
||||
// 'text/plain; name="file with \\"quotes\\".txt"'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `formatContentType(ct: ParsedContentType): string`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `ct` | `ParsedContentType` | A parsed content-type object. |
|
||||
|
||||
**Returns:** An RFC 7231-compliant `Content-Type` header string.
|
||||
|
||||
**Throws:** Never — all inputs that satisfy `ParsedContentType` are formattable.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
A value is quoted only when it contains characters outside the RFC 7230 token character set (`!#$%&'*+-.^_\`|~0-9A-Za-z`). Characters within the quoted string that are not`token` characters are escaped with a backslash. This ensures the output is always valid even when parameter values contain double-quote characters themselves. The function uses `Object.keys` to enumerate parameters, which preserves insertion order in all modern JavaScript engines.
|
||||
|
||||
### Quoting Rules
|
||||
|
||||
Parameter values that contain characters outside the RFC 7230 `token` grammar (whitespace, `;`, `,`, non-ASCII) are wrapped in double quotes with backslash-escaping; values that already look like a valid token are emitted unquoted. Charset parameter values are lowercased at format time so output is canonical and hashes stably for content-based caches.
|
||||
@@ -0,0 +1,82 @@
|
||||
# parse-content-type
|
||||
|
||||
## Purpose
|
||||
|
||||
Parses an RFC 7231 / RFC 9110 `Content-Type` header value into a structured `ParsedContentType` object. Handles quoted strings, escape sequences, optional whitespace (OWS), and case normalization. The parsed object is immutable (frozen parameters map) and provides a canonical `essence` field (`type/subtype`) for efficient comparison.
|
||||
|
||||
## Features
|
||||
|
||||
- Parses `type/subtype` with OWS trimming and lowercase normalization
|
||||
- Parses parameters: `name=value` pairs, `;`-separated
|
||||
- Supports quoted-string parameter values with `\"` escape sequences
|
||||
- Normalizes `charset`, `boundary`, and `format` parameter values to lowercase
|
||||
- Handles OWS around `=` and between parameters
|
||||
- Validates type, subtype, and parameter names as RFC token characters
|
||||
- Guards against non-string inputs (JS interop safety)
|
||||
- Returns a frozen `parameters` object for safe sharing
|
||||
- Throws `InvalidContentTypeError` for all malformed inputs
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { parseContentType } from "@suaveplan/mime/content-type/parse-content-type";
|
||||
|
||||
const ct = parseContentType("text/html; charset=utf-8");
|
||||
console.log(ct.type); // "text"
|
||||
console.log(ct.subtype); // "html"
|
||||
console.log(ct.essence); // "text/html"
|
||||
console.log(ct.parameters); // { charset: "utf-8" }
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { parseContentType } from "@suaveplan/mime/content-type/parse-content-type";
|
||||
import { InvalidContentTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// Quoted boundary with spaces
|
||||
const multipart = parseContentType('multipart/form-data; boundary="--boundary 123"');
|
||||
console.log(multipart.parameters.boundary); // "--boundary 123"
|
||||
|
||||
// Multiple parameters with OWS
|
||||
const complex = parseContentType('text/html; charset= utf-8; format=flowed');
|
||||
console.log(complex.parameters.charset); // "utf-8"
|
||||
console.log(complex.parameters.format); // "flowed"
|
||||
|
||||
// Error handling
|
||||
try {
|
||||
parseContentType("not/valid token!");
|
||||
} catch (err) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
console.error("Malformed Content-Type:", err.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Round-trip safe (parse → format → parse)
|
||||
const reparsed = parseContentType(formatContentType(multipart));
|
||||
console.log(reparsed.essence); // "multipart/form-data"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `parseContentType(input: string): ParsedContentType`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `input` | `string` | Raw `Content-Type` header value. |
|
||||
|
||||
**Returns:** A `ParsedContentType` with `type`, `subtype`, `essence`, and frozen `parameters`.
|
||||
|
||||
**Throws:** `InvalidContentTypeError` when:
|
||||
|
||||
- Input is not a string
|
||||
- Input is empty or whitespace-only
|
||||
- Missing slash between type and subtype
|
||||
- Type or subtype is empty
|
||||
- Type or subtype contains non-token characters
|
||||
- Parameter name contains non-token characters
|
||||
- Parameter is missing the `=` separator
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The parser is a hand-written state machine operating on the raw string. This avoids regex backtracking issues with complex quoted-string values and ensures O(n) time complexity. The `parameters` object is frozen via `Object.freeze` before return to prevent accidental mutation by callers. The `charset`, `boundary`, and `format` parameter values are lowercased per the RFC — all other parameter values preserve their original casing. The leading `trim()` on the full input removes any surrounding whitespace, which means trailing spaces after a final semicolon are eliminated before parameter parsing begins.
|
||||
56
docs/foundation/data/mime/src/db/builtin-db.md
Normal file
56
docs/foundation/data/mime/src/db/builtin-db.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# db/builtin-db
|
||||
|
||||
## Purpose
|
||||
|
||||
The built-in MIME type database for `@suaveplan/mime`. Contains over 150 `MimeEntry` records covering the most common web, media, document, archive, font, and application MIME types. This is a static data file — it contains no runtime logic, only the `BUILTIN_DB` constant array consumed by the registry.
|
||||
|
||||
## Features
|
||||
|
||||
- 150+ MIME type entries across all major categories
|
||||
- Each entry includes: `mime` (canonical type string), `extensions` (array of associated file extensions), and `encoding` (`"binary"`, `"text"`, or `"base64"`)
|
||||
- Comprehensive coverage: images, audio, video, documents, archives, fonts, executables, data formats
|
||||
- Extensions are lowercase and without leading dots
|
||||
- Canonical extension (for `Content-Disposition` filename guessing) is always the first element
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { BUILTIN_DB } from "@suaveplan/mime/db/builtin-db";
|
||||
|
||||
// Inspect all text-type entries
|
||||
const textTypes = BUILTIN_DB.filter(e => e.mime.startsWith("text/"));
|
||||
console.log(textTypes.map(e => e.mime));
|
||||
// ["text/plain", "text/html", "text/css", ...]
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { BUILTIN_DB } from "@suaveplan/mime/db/builtin-db";
|
||||
import { getEntryByMime } from "@suaveplan/mime/db/registry";
|
||||
|
||||
// Combine built-in with runtime lookup
|
||||
const entry = getEntryByMime("application/pdf");
|
||||
console.log(entry?.extensions); // ["pdf"]
|
||||
console.log(entry?.encoding); // "binary"
|
||||
|
||||
// Find all binary MIME types
|
||||
const binaryTypes = BUILTIN_DB.filter(e => e.encoding === "binary");
|
||||
console.log(binaryTypes.length); // 100+
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `BUILTIN_DB: readonly MimeEntry[]`
|
||||
|
||||
A frozen array of `MimeEntry` objects. Each entry has:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `mime` | `string` | Canonical MIME type string (e.g., `"image/png"`) |
|
||||
| `extensions` | `string[]` | Associated file extensions without dots (e.g., `["png"]`) |
|
||||
| `encoding` | `"binary" \| "text" \| "base64"` | Transfer encoding hint |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The database is intentionally a plain array rather than a pre-built map so that the registry can apply its first-write-wins semantics during initialization. Keeping data and behavior separate also makes it straightforward to swap the database at build time (e.g., for a minimal embedded build that includes only a subset of MIME types). The entries are ordered roughly by frequency of use, which has no algorithmic impact but improves grep readability.
|
||||
70
docs/foundation/data/mime/src/db/registry.md
Normal file
70
docs/foundation/data/mime/src/db/registry.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# db/registry
|
||||
|
||||
## Purpose
|
||||
|
||||
The central in-memory MIME type database for `@suaveplan/mime`. Maintains two indexes over the built-in `MimeEntry` list: a `type → extensions` forward map and an `extension → entry` reverse map. Provides O(1) lookup in both directions and supports registering additional entries at runtime for extensibility.
|
||||
|
||||
## Features
|
||||
|
||||
- `getEntryByMime(mime)` — forward lookup: MIME type string → `MimeEntry`
|
||||
- `getEntryByExtension(ext)` — reverse lookup: file extension → `MimeEntry`
|
||||
- `registerEntry(entry)` — register a custom MIME type at runtime
|
||||
- First-write-wins semantics: built-in entries take precedence over later registrations for the same extension
|
||||
- Extension keys are normalized to lowercase with no leading dot
|
||||
- Both indexes are populated lazily on first access (module load)
|
||||
- `_indexEntryForTesting` — exported only for internal test instrumentation
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { getEntryByMime, getEntryByExtension } from "@suaveplan/mime/db/registry";
|
||||
|
||||
// Forward lookup
|
||||
const pngEntry = getEntryByMime("image/png");
|
||||
console.log(pngEntry?.extensions); // ["png"]
|
||||
|
||||
// Reverse lookup
|
||||
const entry = getEntryByExtension("jpg");
|
||||
console.log(entry?.mime); // "image/jpeg"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { getEntryByMime, registerEntry } from "@suaveplan/mime/db/registry";
|
||||
|
||||
// Register a custom MIME type
|
||||
registerEntry({
|
||||
mime: "application/x-my-format",
|
||||
extensions: ["myf", "myfmt"],
|
||||
encoding: "binary",
|
||||
});
|
||||
|
||||
// Immediately available via lookup
|
||||
const custom = getEntryByMime("application/x-my-format");
|
||||
console.log(custom?.extensions); // ["myf", "myfmt"]
|
||||
|
||||
// Built-in extensions are protected (first-write-wins)
|
||||
// Registering a second entry for ".png" will not override image/png
|
||||
registerEntry({ mime: "image/x-fake-png", extensions: ["png"] });
|
||||
const png = getEntryByExtension("png");
|
||||
console.log(png?.mime); // still "image/png"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `getEntryByMime(mime: string): MimeEntry | undefined`
|
||||
|
||||
Looks up a `MimeEntry` by its MIME type string. Returns `undefined` if not found.
|
||||
|
||||
### `getEntryByExtension(ext: string): MimeEntry | undefined`
|
||||
|
||||
Looks up a `MimeEntry` by file extension (with or without leading dot, case-insensitive). Returns `undefined` if not found.
|
||||
|
||||
### `registerEntry(entry: MimeEntry): void`
|
||||
|
||||
Registers a new MIME type entry. First-write-wins for extensions: if an extension is already in the index, the new entry does not replace it.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The registry is initialized at module load by iterating `BUILTIN_DB` once and calling the private `index()` function for each entry. The `index()` function implements the first-write-wins extension guard: `if (!byExtension.has(key)) byExtension.set(key, entry)`. This guard prevents downstream `registerEntry` calls from overriding the canonical extension-to-type mapping established by the built-in database, which is important for stable behavior across multiple `registerEntry` calls in plugin-style usage.
|
||||
90
docs/foundation/data/mime/src/detection/detect.md
Normal file
90
docs/foundation/data/mime/src/detection/detect.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# detection/detect
|
||||
|
||||
## Purpose
|
||||
|
||||
The unified MIME detection entry point for `@suaveplan/mime`. Merges evidence from three independent sources — magic bytes, file extension, and a declared `Content-Type` header — using a confidence-scoring model. When sources agree, the confidence is boosted above any single-source threshold. When they disagree, magic bytes win over extension, which wins over declared headers, reflecting real-world reliability.
|
||||
|
||||
## Features
|
||||
|
||||
- Multi-source detection: bytes, extension/filename, Content-Type header
|
||||
- Confidence scores: magic-bytes=0.95, extension=0.7, content-type=0.6
|
||||
- Agreement boost: +0.05 per additional agreeing source (capped at 1.0)
|
||||
- Falls back to `application/octet-stream` with confidence=0 when no sources available
|
||||
- Accepts `filename` as an alias for `extension` (strips path components automatically)
|
||||
- Swallows malformed Content-Type headers gracefully (no throws)
|
||||
- Re-exports `detectFromMagicBytes` and `detectAllFromMagicBytes` for convenience
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { detect } from "@suaveplan/mime/detection/detect";
|
||||
|
||||
const result = detect({
|
||||
bytes: fileBuffer,
|
||||
filename: "photo.jpg",
|
||||
});
|
||||
console.log(result.mime); // "image/jpeg"
|
||||
console.log(result.confidence); // 0.95 (magic bytes match)
|
||||
console.log(result.source); // "magic-bytes"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { detect } from "@suaveplan/mime/detection/detect";
|
||||
|
||||
// All three sources provided — if they agree, confidence is boosted
|
||||
const result = detect({
|
||||
bytes: buffer,
|
||||
filename: "report.pdf",
|
||||
contentType: "application/pdf",
|
||||
});
|
||||
if (result.confidence > 0.9) {
|
||||
console.log("High confidence:", result.mime);
|
||||
console.log("Sources:", result.candidates.map(c => c.source));
|
||||
}
|
||||
|
||||
// Only extension provided
|
||||
const byExtension = detect({ extension: "svg" });
|
||||
console.log(byExtension.mime); // "image/svg+xml"
|
||||
console.log(byExtension.confidence); // 0.7
|
||||
|
||||
// No sources — safe fallback
|
||||
const fallback = detect({});
|
||||
console.log(fallback.mime); // "application/octet-stream"
|
||||
console.log(fallback.confidence); // 0
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `detect(input: DetectInput): DetectResult`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `input` | `DetectInput` | Detection inputs — at least one field should be set. |
|
||||
|
||||
**Returns:** `DetectResult` with `mime`, `confidence`, `source`, and `candidates` array.
|
||||
|
||||
**Throws:** Never — malformed Content-Type headers are silently ignored.
|
||||
|
||||
### `DetectInput`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `bytes?` | `Uint8Array` | Raw file bytes for magic-byte detection |
|
||||
| `extension?` | `string` | File extension (with or without dot) |
|
||||
| `filename?` | `string` | Full filename or path (extension extracted automatically) |
|
||||
| `contentType?` | `string` | Raw `Content-Type` header value |
|
||||
|
||||
### `DetectResult`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `mime` | `string` | Best MIME type guess |
|
||||
| `confidence` | `number` | 0.0–1.0 confidence score |
|
||||
| `source` | `DetectSource` | Primary detection source |
|
||||
| `candidates` | `Candidate[]` | All candidate MIME types with their sources |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Candidates are collected in descending confidence order: magic-bytes first, then extension, then content-type. Because this order is fixed and confidence values are monotonically decreasing, the first candidate in the array is always the best. The confidence boost formula `min(1, bestScore + (agreeing - 1) * 0.05)` adds 5% for each additional source that agrees with the winner, reflecting that corroborating evidence raises overall certainty.
|
||||
56
docs/foundation/data/mime/src/detection/from-content-type.md
Normal file
56
docs/foundation/data/mime/src/detection/from-content-type.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# detection/from-content-type
|
||||
|
||||
## Purpose
|
||||
|
||||
Detects the MIME type from a raw `Content-Type` header string. Parses the header using the full RFC 7231 parser and wraps the result in a `DetectionResult` envelope. Serves as one of three detection sources in the unified `detect()` function, with a confidence weight of 0.6 (lowest of the three sources, since declared headers can be wrong or missing).
|
||||
|
||||
## Features
|
||||
|
||||
- Delegates to `parseContentType` for full RFC 7231 parsing
|
||||
- Returns a `DetectionResult` with `source: "content-type"` and the canonical `essence`
|
||||
- Propagates `InvalidContentTypeError` for malformed inputs (callers can catch or ignore)
|
||||
- Accepts parameters in the header (they are parsed but not included in the MIME result)
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { detectFromContentType } from "@suaveplan/mime/detection/from-content-type";
|
||||
|
||||
const result = detectFromContentType("text/html; charset=utf-8");
|
||||
console.log(result.mime); // "text/html"
|
||||
console.log(result.source); // "content-type"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { detectFromContentType } from "@suaveplan/mime/detection/from-content-type";
|
||||
import { InvalidContentTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// The detect() facade catches these errors automatically.
|
||||
// Use detectFromContentType directly when you want strict validation:
|
||||
try {
|
||||
const result = detectFromContentType(request.headers.get("content-type") ?? "");
|
||||
console.log("Declared MIME:", result.mime);
|
||||
} catch (err) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
console.warn("Malformed Content-Type header — using fallback detection");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `detectFromContentType(header: string): DetectionResult`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `header` | `string` | Raw `Content-Type` header value. |
|
||||
|
||||
**Returns:** `DetectionResult` with `mime` set to the `essence` (`type/subtype`) and `source: "content-type"`.
|
||||
|
||||
**Throws:** `InvalidContentTypeError` if the header is malformed.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
This function intentionally throws on malformed input rather than returning a fallback. The `detect()` facade wraps calls to `detectFromContentType` in a try/catch to silently skip bad headers. This two-layer design keeps the individual detectors strict while the unified facade remains lenient — matching the principle that each building block should have well-defined error semantics, with fault tolerance composed at a higher level.
|
||||
62
docs/foundation/data/mime/src/detection/from-extension.md
Normal file
62
docs/foundation/data/mime/src/detection/from-extension.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# detection/from-extension
|
||||
|
||||
## Purpose
|
||||
|
||||
Detects the MIME type from a file extension or full filename. Strips path components and leading dots, normalizes to lowercase, and looks up the result in the built-in MIME database. Returns a `DetectionResult` with `source: "extension"` and a confidence weight of 0.7 in the unified `detect()` function.
|
||||
|
||||
## Features
|
||||
|
||||
- Accepts both raw extensions (`.png`, `png`) and full filenames (`photo.jpeg`)
|
||||
- Strips directory path components automatically
|
||||
- Normalizes extensions to lowercase before lookup
|
||||
- Handles multiple dots in filenames correctly (uses the last extension)
|
||||
- Returns `undefined` for unknown extensions instead of throwing
|
||||
- Wraps the database entry in a `DetectionResult` envelope
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { detectFromExtension } from "@suaveplan/mime/detection/from-extension";
|
||||
|
||||
const result = detectFromExtension("photo.jpg");
|
||||
console.log(result?.mime); // "image/jpeg"
|
||||
console.log(result?.source); // "extension"
|
||||
|
||||
// Just an extension without filename
|
||||
const svg = detectFromExtension(".svg");
|
||||
console.log(svg?.mime); // "image/svg+xml"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { detectFromExtension } from "@suaveplan/mime/detection/from-extension";
|
||||
|
||||
function getMimeForUpload(filename: string): string {
|
||||
const detected = detectFromExtension(filename);
|
||||
// Fall back to binary stream for unknown extensions
|
||||
return detected?.mime ?? "application/octet-stream";
|
||||
}
|
||||
|
||||
// Handles paths with directories
|
||||
console.log(detectFromExtension("/uploads/2024/report.pdf")?.mime); // "application/pdf"
|
||||
|
||||
// Unknown extension returns undefined
|
||||
console.log(detectFromExtension("file.xyz")); // undefined
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `detectFromExtension(input: string): DetectionResult | undefined`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `input` | `string` | File extension (with or without dot) or full filename/path. |
|
||||
|
||||
**Returns:** `DetectionResult` with `source: "extension"` and the matched `mime`, or `undefined` when the extension is not in the database.
|
||||
|
||||
**Throws:** Never.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The function extracts the extension by splitting on `.` and taking the last segment, then looking up `getEntryByExtension`. The split also handles filenames with no extension (returns `undefined`) and filenames starting with a dot (treated as hidden files with no extension, e.g. `.gitignore`). Path components are stripped by looking only at the basename portion after the last `/` or `\`. Lookup is O(1) via the registry's `Map<string, MimeEntry>` index.
|
||||
@@ -0,0 +1,86 @@
|
||||
# detection/from-magic-bytes
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs --> - helpers — Low-level byte-inspection utilities used……
|
||||
- [helpers](./helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- helpers — Low-level byte-inspection utilities used……
|
||||
|
||||
- [helpers](./helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
|
||||
- helpers — Low-level byte-inspection utilities used……
|
||||
|
||||
- [helpers](./helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
|
||||
- helpers — Low-level byte-inspection utilities used…
|
||||
|
||||
- [helpers](./helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
- [helpers](./helpers.md) — Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: pr…
|
||||
- [registry](./registry.md) — The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-…
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements magic-byte (file signature) detection for MIME types. Maintains a registry of byte-pattern matchers sorted by priority, and provides functions to find the best match, all matches, or reset the registry. Magic-byte detection has the highest confidence (0.95) in the unified `detect()` function because byte signatures are objective evidence that cannot be spoofed by file naming or HTTP headers.
|
||||
|
||||
## Features
|
||||
|
||||
- `detectFromMagicBytes(bytes)` — returns the highest-priority matching MIME type
|
||||
- `detectAllFromMagicBytes(bytes)` — returns all matching MIME types sorted by priority
|
||||
- Registry sorted by descending priority for deterministic match order
|
||||
- Accepts any `Uint8Array` or `ArrayBuffer`-backed view
|
||||
- Returns `undefined` (not throws) when no signature matches
|
||||
- Supports minimum-bytes threshold per signature — skips signatures when the input is too short
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
// PNG magic bytes: 89 50 4E 47 0D 0A 1A 0A
|
||||
const pngBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const result = detectFromMagicBytes(pngBytes);
|
||||
console.log(result?.mime); // "image/png"
|
||||
console.log(result?.priority); // 100
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import {
|
||||
detectFromMagicBytes,
|
||||
detectAllFromMagicBytes,
|
||||
} from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
// Detect all matching signatures (useful for ambiguous formats like MP4 variants)
|
||||
const mp4Bytes = new Uint8Array([...]);
|
||||
const allMatches = detectAllFromMagicBytes(mp4Bytes);
|
||||
for (const match of allMatches) {
|
||||
console.log(match.mime, match.priority);
|
||||
}
|
||||
|
||||
// Returns undefined for unrecognized byte sequences
|
||||
const random = new Uint8Array([0x00, 0x01, 0x02, 0x03]);
|
||||
console.log(detectFromMagicBytes(random)); // undefined
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `detectFromMagicBytes(bytes: Uint8Array): SignatureMatch | undefined`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `bytes` | `Uint8Array` | Byte buffer to inspect (first N bytes of a file). |
|
||||
|
||||
**Returns:** The highest-priority `SignatureMatch` (`mime`, `priority`), or `undefined`.
|
||||
|
||||
### `detectAllFromMagicBytes(bytes: Uint8Array): SignatureMatch[]`
|
||||
|
||||
Returns all matching signatures sorted by descending priority.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The registry is sorted once whenever a new signature is registered, using a stable sort on `priority` (descending). Detection iterates the sorted array and returns the first match (for `detectFromMagicBytes`) or all matches (for `detectAllFromMagicBytes`). Signatures with a `minBytes` threshold are skipped when the input buffer is too short, enabling detection on partial reads without false negatives from truncated magic sequences.
|
||||
@@ -0,0 +1,75 @@
|
||||
# detection/from-magic-bytes/helpers
|
||||
|
||||
## Purpose
|
||||
|
||||
Low-level byte-inspection utilities used by the magic-byte signature matchers. Provides helpers for common patterns: prefix matching, ASCII substring scanning, 32-bit big-endian integer reading, and byte-sequence searching. These functions are designed to be called millions of times without allocation, using only index arithmetic over the input `Uint8Array`.
|
||||
|
||||
## Features
|
||||
|
||||
- `startsWith(bytes, prefix, offset?)` — check if bytes start with a given byte sequence at an offset
|
||||
- `hasAscii(bytes, text, offset?)` — check if bytes contain an ASCII string at a given offset
|
||||
- `readUInt32BE(bytes, offset)` — read a 4-byte big-endian unsigned integer
|
||||
- `bytesInclude(bytes, needle)` — check if a byte sequence appears anywhere in bytes
|
||||
- All functions are allocation-free (no `new Uint8Array`, no `slice`)
|
||||
- All functions are bounds-safe (return false/0 rather than throwing on out-of-range access)
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { startsWith, hasAscii } from "@suaveplan/mime/detection/from-magic-bytes/helpers";
|
||||
|
||||
// Check PNG magic bytes
|
||||
const PNG_MAGIC = new Uint8Array([0x89, 0x50, 0x4e, 0x47]);
|
||||
const isPng = startsWith(fileBytes, PNG_MAGIC);
|
||||
|
||||
// Check for RIFF tag at offset 0, WAVE tag at offset 8
|
||||
const isWav = hasAscii(fileBytes, "RIFF") && hasAscii(fileBytes, "WAVE", 8);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import {
|
||||
startsWith,
|
||||
readUInt32BE,
|
||||
bytesInclude,
|
||||
} from "@suaveplan/mime/detection/from-magic-bytes/helpers";
|
||||
|
||||
// MP4: ftyp box at byte 4
|
||||
function isMp4(bytes: Uint8Array): boolean {
|
||||
const boxSize = readUInt32BE(bytes, 0);
|
||||
return boxSize >= 8 && hasAscii(bytes, "ftyp", 4);
|
||||
}
|
||||
|
||||
// ZIP: PK signature or EOCD record
|
||||
function isZip(bytes: Uint8Array): boolean {
|
||||
return startsWith(bytes, new Uint8Array([0x50, 0x4b, 0x03, 0x04])) ||
|
||||
startsWith(bytes, new Uint8Array([0x50, 0x4b, 0x05, 0x06]));
|
||||
}
|
||||
|
||||
// Search for a needle anywhere in a buffer
|
||||
const PDF_HEADER = new Uint8Array([0x25, 0x50, 0x44, 0x46]); // %PDF
|
||||
const containsPdf = bytesInclude(largeBuffer, PDF_HEADER);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `startsWith(bytes: Uint8Array, prefix: Uint8Array, offset?: number): boolean`
|
||||
|
||||
Returns `true` if `bytes[offset...]` starts with all bytes of `prefix`. Returns `false` if the buffer is too short.
|
||||
|
||||
### `hasAscii(bytes: Uint8Array, text: string, offset?: number): boolean`
|
||||
|
||||
Returns `true` if the ASCII codes of `text` match `bytes[offset...]`. Returns `false` if the buffer is too short.
|
||||
|
||||
### `readUInt32BE(bytes: Uint8Array, offset: number): number`
|
||||
|
||||
Reads 4 bytes starting at `offset` as a big-endian unsigned 32-bit integer. Returns `0` if the buffer is too short.
|
||||
|
||||
### `bytesInclude(bytes: Uint8Array, needle: Uint8Array): boolean`
|
||||
|
||||
Returns `true` if `needle` appears anywhere within `bytes`. Returns `false` if the buffer is shorter than the needle.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
All helpers operate directly on the `Uint8Array` buffer without creating sub-views or intermediate arrays. The `hasAscii` function uses `charCodeAt` rather than `TextEncoder` to avoid the allocation overhead of encoding a string to bytes on every call — this matters because signature matchers run on every file detection. The `bytesInclude` function uses a naive O(n×m) scan, which is acceptable given that needle lengths are always small (≤8 bytes for magic sequences) and the search space is bounded by the buffer size.
|
||||
@@ -0,0 +1,84 @@
|
||||
# detection/from-magic-bytes/registry
|
||||
|
||||
## Purpose
|
||||
|
||||
The mutable signature registry for magic-byte detection. Stores all registered byte-pattern matchers and provides CRUD-like access: `registerSignature` to add a matcher, `getSignatures` to read the sorted list, and `resetSignatures` to clear all entries (used in tests). This module is the single source of truth for the ordered list of matchers consumed by `detectFromMagicBytes`.
|
||||
|
||||
## Features
|
||||
|
||||
- `registerSignature(sig)` — add a new signature matcher; re-sorts registry by priority
|
||||
- `getSignatures()` — returns the full registry array sorted by descending priority
|
||||
- `resetSignatures()` — clears all registered signatures (test isolation only)
|
||||
- Priority-ordered registry: higher priority signatures are checked first
|
||||
- Thread-safe within a single JavaScript execution context (no async race conditions)
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import {
|
||||
registerSignature,
|
||||
getSignatures,
|
||||
} from "@suaveplan/mime/detection/from-magic-bytes/registry";
|
||||
|
||||
registerSignature({
|
||||
mime: "application/x-my-format",
|
||||
priority: 50,
|
||||
minBytes: 4,
|
||||
match: (bytes) => bytes[0] === 0x4d && bytes[1] === 0x59,
|
||||
});
|
||||
|
||||
console.log(getSignatures().find(s => s.mime === "application/x-my-format"));
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { registerSignature } from "@suaveplan/mime/detection/from-magic-bytes/registry";
|
||||
|
||||
// Custom signature for a proprietary binary format
|
||||
registerSignature({
|
||||
mime: "application/x-leveldb",
|
||||
priority: 75,
|
||||
minBytes: 8,
|
||||
match: (bytes) =>
|
||||
bytes[0] === 0xff &&
|
||||
bytes[1] === 0x06 &&
|
||||
bytes[2] === 0x00 &&
|
||||
bytes[3] === 0x00,
|
||||
});
|
||||
|
||||
// Signature with a scan offset
|
||||
registerSignature({
|
||||
mime: "image/x-xcf",
|
||||
priority: 80,
|
||||
minBytes: 9,
|
||||
match: (bytes) => {
|
||||
// "gimp xcf " at offset 0
|
||||
const magic = [0x67, 0x69, 0x6d, 0x70, 0x20, 0x78, 0x63, 0x66, 0x20];
|
||||
return magic.every((b, i) => bytes[i] === b);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `registerSignature(sig: Signature): void`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `sig.mime` | `string` | MIME type this signature identifies |
|
||||
| `sig.priority` | `number` | Higher values are checked first (0–100 typical range) |
|
||||
| `sig.minBytes` | `number` | Minimum bytes required to run the matcher |
|
||||
| `sig.match` | `(bytes: Uint8Array) => boolean` | Byte-pattern predicate |
|
||||
|
||||
### `getSignatures(): Signature[]`
|
||||
|
||||
Returns the full sorted registry array. Do not mutate the returned array directly.
|
||||
|
||||
### `resetSignatures(): void`
|
||||
|
||||
Clears all signatures. Intended for test isolation only — not safe for production use.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The registry maintains a sorted array rather than a priority queue for simplicity. `registerSignature` performs an in-place sort after insertion, which is O(n log n) but acceptable given that signatures are registered at module load time (once per process) and detection reads are O(n) anyway. The sort is stable in modern JavaScript engines, so signatures with equal priority retain their insertion order.
|
||||
@@ -0,0 +1,39 @@
|
||||
# signatures/archives
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for archive and compressed file formats with the global signature registry. Covers all major compression and container formats used in file distribution and backup. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- ZIP, JAR, ODF, DOCX/PPTX/XLSX (all ZIP-based)
|
||||
- GZIP, BZIP2, XZ/7-Zip, LZMA
|
||||
- RAR (v4 and v5 signatures)
|
||||
- TAR (ustar header at offset 257)
|
||||
- ZSTD (Zstandard frame magic)
|
||||
- LZ4 frame magic
|
||||
- CPIO (binary and ASCII newc variants)
|
||||
- ISO 9660 CD image (CD001 at offset 32769)
|
||||
- BZ2 single-file bzip2
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/archives";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const gzipBytes = new Uint8Array([0x1f, 0x8b, 0x08]);
|
||||
console.log(detectFromMagicBytes(gzipBytes)?.mime); // "application/gzip"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
ZIP-based formats (JAR, ODF, Office Open XML) share the PK signature but cannot be distinguished without inspecting the ZIP directory entries — the magic-byte detector returns `application/zip` generically for all ZIP files. TAR detection looks for the `ustar` ASCII string at offset 257, which requires at least 262 bytes; shorter buffers will not match. ISO 9660 detection at offset 32769 is impractical to test and is annotated to be excluded from coverage thresholds.
|
||||
|
||||
### Coverage
|
||||
|
||||
The archive signature set covers ZIP (`PK\x03\x04` and `PK\x05\x06` empty-archive variant), 7z (`37 7A BC AF 27 1C`), RAR (v4 and v5 magics), GZIP (`1F 8B`), BZIP2 (`BZh`), XZ, TAR (ustar octet at offset 257), and Z-standard. Priority 100 is the default; nested container formats such as OOXML that are technically ZIP files are detected at priority 105 in `documents.ts` so they win the tie.
|
||||
@@ -0,0 +1,37 @@
|
||||
# signatures/audio
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for audio formats with the global signature registry. Covers all major audio container and codec formats used in web and desktop applications. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- FLAC, MP3 (ID3 tag and sync word variants), OGG/Vorbis
|
||||
- WAV (RIFF container), AIFF/AIFF-C
|
||||
- AAC (ADTS), M4A (ISO BMFF with M4A brand)
|
||||
- WMA/WMV (ASF container), MIDI
|
||||
- Musepack (SV7/SV8), OptimFROG, Monkey's Audio
|
||||
- DFF/DSF (DSD audio), AC-3/E-AC-3, Dolby TrueHD
|
||||
- MOD tracker variants (4/6/8-channel patterns)
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/audio";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const flacBytes = new Uint8Array([0x66, 0x4c, 0x61, 0x43]); // "fLaC"
|
||||
console.log(detectFromMagicBytes(flacBytes)?.mime); // "audio/flac"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
MP3 detection covers both the ID3v2 header (`49 44 33`) and raw MPEG sync words (`ff fb`, `ff f3`, `ff f2`). OGG files are further disambiguated by inspecting the codec type in the first OGG page header (bytes 28–35: `vorbis`, `FLAC`, `Opus`, `Theora`). The RIFF container is used by both WAV and AVI, with disambiguation on the RIFF type tag at offset 8.
|
||||
|
||||
### Coverage
|
||||
|
||||
The audio signature set covers MPEG audio (`ID3` tag and raw frame-sync variants at priority 80), WAV/RIFF with the `WAVE` subtype marker, FLAC (`fLaC`), Ogg (`OggS`), AIFF (`FORM` + `AIFF`), MIDI (`MThd`), AMR, and Opus. The RIFF family share a common `RIFF` container prefix which is why WAV specifically checks the subtype at offset 8 before claiming a match.
|
||||
@@ -0,0 +1,39 @@
|
||||
# signatures/data
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for binary data and structured-data formats with the global signature registry. Covers serialization formats, scientific data files, and cryptographic containers. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- MessagePack binary serialization
|
||||
- CBOR (RFC 7049 Concise Binary Object Representation)
|
||||
- Apache Parquet columnar data format
|
||||
- Apache Arrow IPC format
|
||||
- HDF5 (Hierarchical Data Format version 5)
|
||||
- NetCDF (Network Common Data Form)
|
||||
- PCAP and PCAPng network capture files
|
||||
- PEM-encoded certificates and keys
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/data";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
// Parquet magic: "PAR1" at start and end of file
|
||||
const parquetBytes = new Uint8Array([0x50, 0x41, 0x52, 0x31]); // "PAR1"
|
||||
console.log(detectFromMagicBytes(parquetBytes)?.mime); // "application/vnd.apache.parquet"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
MessagePack has no single fixed magic header — the first byte encodes the type of the root value. The detection heuristic checks for the most common root types in well-formed MessagePack documents (fixmap, fixarray, nil, bool, fixint) to reduce false-positives while maintaining reasonable recall. PCAP files come in two endianness variants (little-endian `D4 C3 B2 A1` and big-endian `A1 B2 C3 D4`) plus a nanosecond-resolution variant (`4D 3C B2 A1`).
|
||||
|
||||
### Coverage
|
||||
|
||||
The data signature set covers SQLite (`SQLite format 3\0` at offset 0), Apache Parquet (`PAR1` at offset 0 and at end-of-file), Apache Arrow IPC, CBOR, MessagePack, BSON, and Protocol Buffers when a distinguishable length-prefixed header is present. Formats without a structural magic header (CSV, NDJSON) are not listed here; detection for those falls through to content sniffing in the BOM/text-sniffer stage.
|
||||
@@ -0,0 +1,38 @@
|
||||
# signatures/documents
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for document and data format files with the global signature registry. Covers office documents, structured data, e-books, and markup formats. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- PDF (`%PDF` header)
|
||||
- Microsoft Office legacy formats (DOC, XLS, PPT — OLE2 container `D0 CF 11 E0`)
|
||||
- EPUB (ZIP-based e-book container)
|
||||
- Rich Text Format (RTF)
|
||||
- SQLite database (`SQLite format 3`)
|
||||
- Apple Pages, Numbers, Keynote (ZIP-based, detected via content sniffing)
|
||||
- JSON lines and CBOR (RFC 7049) binary format
|
||||
- Protocol Buffers wire format heuristic
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/documents";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const pdfBytes = new Uint8Array([0x25, 0x50, 0x44, 0x46, 0x2d]); // "%PDF-"
|
||||
console.log(detectFromMagicBytes(pdfBytes)?.mime); // "application/pdf"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
OLE2 Compound Document Format is the container for legacy Microsoft Office files (pre-2007 DOC/XLS/PPT). The magic bytes `D0 CF 11 E0` identify the OLE2 container but not the specific Office application — the detector returns `application/vnd.ms-office` as a generic fallback. Modern Office formats (DOCX, XLSX, PPTX) use the ZIP container and are handled in the archives signatures module.
|
||||
|
||||
### Coverage
|
||||
|
||||
The document signature set covers PDF (`%PDF`), Microsoft CFB-backed legacy Office (`D0 CF 11 E0 A1 B1 1A E1`), OOXML (`PK` with `[Content_Types].xml` entry — priority 105 so it beats plain ZIP), ODF, RTF (`{\\rtf`), PostScript (`%!PS`), EPUB (ZIP with `mimetype` = `application/epub+zip`), and MOBI/AZW. The 105 priority on container-based formats is critical: without it OOXML would match plain ZIP at 100 and lose granularity.
|
||||
@@ -0,0 +1,43 @@
|
||||
# signatures/executables
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for executable binary formats with the global signature registry. Covers native executables, bytecode, and portable binary formats across platforms. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- ELF (Linux/Unix native executables and shared libraries)
|
||||
- PE (Windows `.exe` and `.dll` — `MZ` header)
|
||||
- Mach-O (macOS native: 32/64-bit, fat binary, ARM64)
|
||||
- Java `.class` bytecode (`CA FE BA BE`)
|
||||
- DEX (Android Dalvik bytecode)
|
||||
- WebAssembly (`.wasm` — `00 61 73 6d`)
|
||||
- Python `.pyc` bytecode (magic + timestamp header)
|
||||
- LLVM bitcode
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/executables";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
// ELF magic: 7F 45 4C 46
|
||||
const elfBytes = new Uint8Array([0x7f, 0x45, 0x4c, 0x46]);
|
||||
console.log(detectFromMagicBytes(elfBytes)?.mime); // "application/x-elf"
|
||||
|
||||
// WebAssembly magic: 00 61 73 6D
|
||||
const wasmBytes = new Uint8Array([0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00]);
|
||||
console.log(detectFromMagicBytes(wasmBytes)?.mime); // "application/wasm"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Mach-O binaries exist in four variants: 32-bit big-endian (`CE FA ED FE`), 32-bit little-endian (`FE ED FA CE`), 64-bit big-endian (`CF FA ED FE`), and 64-bit little-endian (`FE ED FA CF`). Fat binaries (`CA FE BA BE` at big-endian 0xcafebabe) share the same magic as Java class files but are distinguished by a byte at offset 7 (`0x00` for Mach-O fat, non-zero for Java). The implemention uses a combined check for the Java case.
|
||||
|
||||
### Coverage
|
||||
|
||||
The executable signature set covers WebAssembly (`\x00asm`), ELF (`\x7FELF` — Linux / BSD native), Mach-O (FAT + thin, 32/64-bit big/little endian variants), PE/COFF (`MZ`), Java class files (`CAFEBABE`), .NET assemblies (via PE with CLI header), and bytecode containers like Python `.pyc` and Lua `.luac`. Multiple Mach-O magics exist because Apple shipped multiple endianness and pointer-width variants; all land on `application/x-mach-binary`.
|
||||
@@ -0,0 +1,37 @@
|
||||
# signatures/fonts
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for font file formats with the global signature registry. Covers all major web font and desktop font formats. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- WOFF (Web Open Font Format) — `wOFF` magic
|
||||
- WOFF2 — `wOF2` magic
|
||||
- TrueType (`.ttf`) — `00 01 00 00 00` header
|
||||
- OpenType (`.otf`) — `4F 54 54 4F` (`OTTO`) header
|
||||
- EOT (Embedded OpenType) — Microsoft format
|
||||
- `font/collection` (TTC, OpenType Collection)
|
||||
- `vnd.ms-fontobject` variants
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/fonts";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const woff2Bytes = new Uint8Array([0x77, 0x4f, 0x46, 0x32]); // "wOF2"
|
||||
console.log(detectFromMagicBytes(woff2Bytes)?.mime); // "font/woff2"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
TrueType and OpenType both start with a 4-byte "sfVersion" tag at offset 0. TrueType uses `00 01 00 00` (version 1.0 in fixed-point notation) while OpenType uses `OTTO`. CFF-based fonts use `OTTO` in the sfVersion field; TrueType collections (`.ttc`) use `ttcf`. WOFF and WOFF2 are wrapper formats that contain either TrueType or OpenType data and are unambiguously identified by their `wOFF` / `wOF2` magic.
|
||||
|
||||
### Coverage
|
||||
|
||||
The font signature set covers TrueType (`\x00\x01\x00\x00`), OpenType (`OTTO` — CFF-based), WOFF1 (`wOFF`), WOFF2 (`wOF2`), TrueType Collection (`ttcf`), EOT (Microsoft Embedded OpenType), and SVG-font variants. All the signatures sit at priority 100 because each sfVersion / wrapper magic is globally unique and there are no container-based tie-breakers in the font space.
|
||||
@@ -0,0 +1,39 @@
|
||||
# signatures/images
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for image formats with the global signature registry. Covers all major raster and vector image types used on the web and in desktop applications. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- PNG, JPEG, GIF (87a, 89a), WebP, BMP, ICO, TIFF (BE/LE)
|
||||
- HEIC, HEIF, AVIF (multiple brand variants)
|
||||
- PSD (Adobe Photoshop), SVG, XCF (GIMP)
|
||||
- PBM, PGM, PPM, PNM, PFM portable bitmap variants
|
||||
- Radiance HDR (`.hdr`, `.rgbe`)
|
||||
- OpenEXR high dynamic range images
|
||||
|
||||
## Usage
|
||||
|
||||
This module is imported for its side effects only:
|
||||
|
||||
```typescript
|
||||
// Automatic registration — all image signatures become available for detection
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/images";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const pngBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
console.log(detectFromMagicBytes(pngBytes)?.mime); // "image/png"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects on module load.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
HEIC/HEIF/AVIF formats share the ISO Base Media File Format container (like MP4) and are distinguished by their `ftyp` brand code at bytes 8–11. The `startsWith` helper checks both the `ftyp` box header at offset 4 and the brand at offset 8 for each variant. Radiance HDR files use the ASCII header `#?RGBE` or `#?RADIANCE` at offset 0.
|
||||
|
||||
### Coverage
|
||||
|
||||
The image signature set covers PNG (`89 50 4E 47 0D 0A 1A 0A`), JPEG (`FF D8 FF`), GIF (`GIF87a` / `GIF89a`), WebP (`RIFF` + `WEBP` subtype), AVIF / HEIC (ISOBMFF `ftyp` brand inside offset-4 box), BMP, TIFF (`II*\0` little-endian and `MM\0*` big-endian), ICO, and SVG (text-sniffed in a separate stage). ISOBMFF brand matching is done against a curated brand set so AVIF, HEIC, HEIF, and MIAF variants all resolve cleanly.
|
||||
@@ -0,0 +1,36 @@
|
||||
# signatures/model3d
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for 3D model file formats with the global signature registry. Covers formats used in CAD, game development, and 3D printing. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- glTF binary (`.glb`) — `glTF` magic + version 2
|
||||
- STL binary and ASCII (3D printing format)
|
||||
- FBX binary (`Kaydara FBX Binary` header)
|
||||
- Collada (`.dae`) — XML-based, identified by `<?xml` prolog
|
||||
- PLY (Stanford Polygon Library)
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/model3d";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
// glTF binary magic: 67 6C 54 46
|
||||
const glbBytes = new Uint8Array([0x67, 0x6c, 0x54, 0x46, 0x02, 0x00, 0x00, 0x00]);
|
||||
console.log(detectFromMagicBytes(glbBytes)?.mime); // "model/gltf-binary"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
STL binary files have no fixed magic sequence — they start with an 80-byte header of arbitrary text. The detection heuristic checks for the absence of the ASCII string `solid` at offset 0 (which would indicate ASCII STL) combined with a plausible triangle count in bytes 80–83. ASCII STL files are identified by the `solid` keyword at offset 0. The MIME type `model/stl` is returned for both binary and ASCII STL when detected.
|
||||
|
||||
### Coverage
|
||||
|
||||
The 3D-model signature set covers glTF binary (`glTF` magic at offset 0), USDZ (a ZIP container with a specific first-entry layout), FBX (both ASCII `; FBX` and binary `Kaydara FBX Binary \0`), STL (ASCII vs binary are disambiguated by header-byte heuristics), OBJ (text-sniffed), 3MF, and DAE/Collada (XML with known root element). The binary FBX magic uses 20 bytes including two trailing null bytes and a SUB character — matching that full prefix avoids false positives on other tools that write `Kaydara`-prefixed files.
|
||||
@@ -0,0 +1,38 @@
|
||||
# signatures/video
|
||||
|
||||
## Purpose
|
||||
|
||||
Registers magic-byte signatures for video container formats with the global signature registry. Covers all major video formats used on the web and in streaming applications. Each signature is registered as a side effect of importing this module.
|
||||
|
||||
## Features
|
||||
|
||||
- MP4 / M4V (ISO BMFF with multiple `ftyp` brands: `isom`, `iso2`, `mp41`, `mp42`, etc.)
|
||||
- WebM and MKV (Matroska EBML header)
|
||||
- AVI (RIFF container with AVI type)
|
||||
- QuickTime MOV, 3GPP, 3GPP2
|
||||
- MPEG-1/2 PS (`ba`/`b3` pack start codes)
|
||||
- MPEG-2 TS (188-byte sync word pattern)
|
||||
- Windows Media (`video/x-ms-wmv` via ASF header)
|
||||
- Flash Video (FLV), DV video
|
||||
|
||||
## Usage
|
||||
|
||||
```typescript
|
||||
import "@suaveplan/mime/detection/from-magic-bytes/signatures/video";
|
||||
import { detectFromMagicBytes } from "@suaveplan/mime/detection/from-magic-bytes/from-magic-bytes";
|
||||
|
||||
const webmBytes = new Uint8Array([0x1a, 0x45, 0xdf, 0xa3]); // EBML header
|
||||
console.log(detectFromMagicBytes(webmBytes)?.mime); // "video/webm"
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
No exported functions. All signatures are registered via `registerSignature` side effects.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
MP4 variants are distinguished by their `ftyp` brand at bytes 8–11 within the ISO Base Media File Format container. Multiple brands map to `video/mp4` (isom, iso2, mp41, mp42, etc.) while M4V uses the `M4V` brand. MPEG-2 TS detection requires the sync byte `0x47` at both offset 0 and offset 188, making it impractical to test without 189+ bytes; the signature is annotated to be excluded from coverage.
|
||||
|
||||
### Coverage
|
||||
|
||||
The video signature set covers ISOBMFF containers (MP4, QuickTime `.mov`, 3GP — disambiguated by `ftyp` brand at offset 4), Matroska / WebM (`1A 45 DF A3` EBML header + DocType check), AVI (`RIFF` + `AVI` subtype), MPEG transport stream (`0x47` sync byte at 188-byte intervals), FLV (`FLV\x01`), and Ogg (when the first packet is a Theora stream). ISOBMFF brand matching is the central complexity — MP4, 3GP, QuickTime, and HEIC share a header and only the brand distinguishes them.
|
||||
@@ -0,0 +1,36 @@
|
||||
# detection/from-stream-reader
|
||||
|
||||
## Purpose
|
||||
|
||||
Internal module providing the `toAsyncIterable` helper and the `drainViaReader` legacy fallback for `ReadableStream` sources. This module is excluded from coverage measurement because the `getReader()` fallback path is only reachable on runtimes where `ReadableStream` does not implement `Symbol.asyncIterator` (pre-Node-18 and some embedded environments). In Bun and Node 18+, the fast path is always taken.
|
||||
|
||||
## Features
|
||||
|
||||
- `toAsyncIterable(source)` — coerces `ReadableStream` or `AsyncIterable` to `AsyncIterable<Uint8Array>`
|
||||
- `drainViaReader(source)` — async generator that pulls chunks via `ReadableStream.getReader()` with proper lock release in the `finally` block
|
||||
- Fast path: when `Symbol.asyncIterator` is present (Bun/Node 18+), the source is returned as-is with no overhead
|
||||
- Slow path: when `Symbol.asyncIterator` is absent, wraps `getReader()` in an async iterable adapter
|
||||
|
||||
## Usage
|
||||
|
||||
This module is not part of the public API surface. It is imported by `from-stream.ts`:
|
||||
|
||||
```typescript
|
||||
// Internal usage only
|
||||
import { toAsyncIterable } from "./from-stream-reader.js";
|
||||
const iter = toAsyncIterable(source)[Symbol.asyncIterator]();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `drainViaReader(source: ReadableStream<Uint8Array>): AsyncGenerator<Uint8Array>`
|
||||
|
||||
Pulls all chunks from a `ReadableStream` using the `getReader()` pull API. Releases the lock in the `finally` block to ensure cleanup even when the consumer aborts iteration.
|
||||
|
||||
### `toAsyncIterable(source): AsyncIterable<Uint8Array>`
|
||||
|
||||
Coerces any `ByteSource` to `AsyncIterable<Uint8Array>`. Fast path if `Symbol.asyncIterator` is present; otherwise returns an adapter object backed by `drainViaReader`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Bun and Node 18+ expose `Symbol.asyncIterator` on `ReadableStream`, making `for await (const chunk of stream)` work natively. The `getReader()` fallback exists for completeness (older Node, Deno, Cloudflare Workers pre-2023, some browser environments). This file is listed in the coverage `exclude` list in `vitest.config.ts` because the fallback cannot be exercised in the Bun test environment without significant test infrastructure for simulating legacy runtimes.
|
||||
79
docs/foundation/data/mime/src/detection/from-stream.md
Normal file
79
docs/foundation/data/mime/src/detection/from-stream.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# detection/from-stream
|
||||
|
||||
## Purpose
|
||||
|
||||
Provides `detectFromStream`, which reads the first N bytes from any async byte source (`ReadableStream<Uint8Array>` or `AsyncIterable<Uint8Array>`) and runs magic-byte detection on the buffered prefix. Returns the detection result together with a `head` buffer (the peeked bytes) and a `tail` async generator that replays the full stream (carry bytes + remaining), enabling callers to re-read the stream from the beginning after detection.
|
||||
|
||||
## Features
|
||||
|
||||
- Accepts both `ReadableStream<Uint8Array>` and `AsyncIterable<Uint8Array>`
|
||||
- Configurable peek window (`peekBytes`, default 512)
|
||||
- AbortSignal support for cancellation during read
|
||||
- Returns a `head` buffer containing the bytes used for detection
|
||||
- Returns a `tail` generator that yields carry bytes (overflow from the last peeked chunk) then remaining stream chunks
|
||||
- Zero-copy split: the overflow portion of the boundary chunk is sliced without re-copying the head bytes
|
||||
- Full-stream reassembly: `head` + `tail` yields the complete original byte sequence
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { detectFromStream } from "@suaveplan/mime/detection/from-stream";
|
||||
|
||||
const result = await detectFromStream(fileStream);
|
||||
console.log(result.mime); // "image/png" (or undefined if unknown)
|
||||
console.log(result.source); // "stream"
|
||||
|
||||
// Consume the full stream content after detection
|
||||
for await (const chunk of result.tail) {
|
||||
process(chunk);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { detectFromStream } from "@suaveplan/mime/detection/from-stream";
|
||||
|
||||
async function processUpload(stream: ReadableStream<Uint8Array>, signal: AbortSignal) {
|
||||
const { mime, head, tail } = await detectFromStream(stream, {
|
||||
peekBytes: 1024, // inspect up to 1 KB
|
||||
signal, // cancel if the request is aborted
|
||||
});
|
||||
|
||||
if (mime !== "image/jpeg" && mime !== "image/png") {
|
||||
throw new Error(`Unsupported image type: ${mime}`);
|
||||
}
|
||||
|
||||
// Process the full image data: head is already a Uint8Array,
|
||||
// tail is an async iterable that delivers the rest
|
||||
const chunks: Uint8Array[] = [head];
|
||||
for await (const chunk of tail) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
return processImage(mime, chunks);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `detectFromStream(source, options?): Promise<StreamDetectionResult>`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `source` | `ReadableStream<Uint8Array> \| AsyncIterable<Uint8Array>` | The byte stream to inspect. |
|
||||
| `options.peekBytes?` | `number` | Maximum bytes to buffer for detection (default: 512). |
|
||||
| `options.signal?` | `AbortSignal` | Cancellation signal. |
|
||||
|
||||
**Returns:** `StreamDetectionResult`:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `source` | `"stream"` | Always `"stream"` |
|
||||
| `mime` | `string \| undefined` | Detected MIME type |
|
||||
| `priority` | `number \| undefined` | Signature priority score |
|
||||
| `head` | `Uint8Array` | The peeked bytes used for detection |
|
||||
| `tail` | `AsyncGenerator<Uint8Array>` | Remaining stream bytes after head |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The `toAsyncIterable` helper in `from-stream-reader.ts` coerces `ReadableStream` to `AsyncIterable`. In Bun and Node 18+, `ReadableStream` already implements `Symbol.asyncIterator`, so the fast path is always taken. The legacy `getReader()` fallback exists for embedded runtimes and is excluded from coverage. The `tail` generator yields carry bytes (the overflow portion of the last buffered chunk, if the chunk straddled the `peekBytes` boundary) before draining the iterator, ensuring byte ordering is preserved across the head/tail split.
|
||||
63
docs/foundation/data/mime/src/error-codes/error-codes.md
Normal file
63
docs/foundation/data/mime/src/error-codes/error-codes.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# error-codes
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines the stable numeric error-code registry for every error class in `@suaveplan/mime`. Each code is a non-overlapping integer in the `5000–5099` range, allocated via `defineCodes` from `@suaveplan/error`. Stable codes enable error identification across serialization, transport, and version upgrades without relying on class names or message strings.
|
||||
|
||||
## Features
|
||||
|
||||
- Five distinct error codes: `INVALID_CONTENT_TYPE`, `UNKNOWN_MIME_TYPE`, `INSUFFICIENT_BYTES`, `PARSE_ERROR`, `UNKNOWN_SIGNATURE`
|
||||
- Allocated via `@suaveplan/error`'s `defineCodes` for namespace isolation
|
||||
- Zero runtime overhead — values are plain numbers resolved at import time
|
||||
- Deterministic: the same integer is guaranteed across package versions
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { MIME_ERROR_CODES } from "@suaveplan/mime/errors/codes";
|
||||
|
||||
// Identify any MIME error by numeric code
|
||||
if (error.code === MIME_ERROR_CODES.INVALID_CONTENT_TYPE) {
|
||||
console.error("Malformed Content-Type header");
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { MIME_ERROR_CODES } from "@suaveplan/mime/errors/codes";
|
||||
import { InvalidContentTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// The code on each thrown error matches the registry
|
||||
try {
|
||||
parseContentType("not-a-content-type");
|
||||
} catch (err) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
console.log(err.code === MIME_ERROR_CODES.INVALID_CONTENT_TYPE); // true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `MIME_ERROR_CODES`
|
||||
|
||||
| Name | Description |
|
||||
|------|-------------|
|
||||
| `INVALID_CONTENT_TYPE` | Malformed or unparseable Content-Type header |
|
||||
| `UNKNOWN_MIME_TYPE` | MIME type not found in the built-in database |
|
||||
| `INSUFFICIENT_BYTES` | Too few bytes provided for magic-byte detection |
|
||||
| `PARSE_ERROR` | General parse failure (XML/HTML charset sniffing) |
|
||||
| `UNKNOWN_SIGNATURE` | Byte signature pattern not recognized |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The range `5000–5099` is reserved for `@suaveplan/mime`. Adjacent packages use non-overlapping ranges so that a project-wide error router can dispatch on code alone. The `defineCodes` call enforces uniqueness and throws at import time if a range collision is detected during development.
|
||||
|
||||
### Registry Layout
|
||||
|
||||
Every code lives under the `MIME` namespace and is produced by `defineCodes` from `@suaveplan/error`: `MIME.PARSE.INVALID`, `MIME.CONTENT_TYPE.INVALID`, `MIME.TYPE.UNKNOWN`, `MIME.SIGNATURE.UNKNOWN`, `MIME.BYTES.INSUFFICIENT`. The `MimeErrorCode` union is used by the `MimeError<TCode>` generic so subclasses advertise exactly which codes they can carry.
|
||||
@@ -0,0 +1,65 @@
|
||||
# errors/insufficient-bytes-error
|
||||
|
||||
## Purpose
|
||||
|
||||
Thrown when a magic-byte detection operation receives fewer bytes than the minimum required by the signature. Carries both the number of bytes received and the number required in `context` for precise error reporting.
|
||||
|
||||
## Features
|
||||
|
||||
- Error code `MIME_ERROR_CODES.INSUFFICIENT_BYTES`
|
||||
- `context.received` — number of bytes provided
|
||||
- `context.required` — minimum bytes needed for detection
|
||||
- `name` set to `"InsufficientBytesError"`
|
||||
- Full `SuaveplanError` serialization
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { InsufficientBytesError } from "@suaveplan/mime/errors";
|
||||
|
||||
function requireMinBytes(bytes: Uint8Array, minBytes: number): void {
|
||||
if (bytes.length < minBytes) {
|
||||
throw new InsufficientBytesError({
|
||||
context: { received: bytes.length, required: minBytes },
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { InsufficientBytesError } from "@suaveplan/mime/errors";
|
||||
|
||||
try {
|
||||
detectFromBytes(tinyBuffer);
|
||||
} catch (err) {
|
||||
if (err instanceof InsufficientBytesError) {
|
||||
console.warn(
|
||||
`Need ${err.context?.required} bytes, got ${err.context?.received}. ` +
|
||||
"Read more bytes and retry."
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class InsufficientBytesError extends MimeError`
|
||||
|
||||
**Constructor options:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `message?` | `string` | Human-readable message |
|
||||
| `context?.received` | `number` | Bytes actually received |
|
||||
| `context?.required` | `number` | Minimum bytes required |
|
||||
| `cause?` | `unknown` | Underlying cause |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
This error is not currently thrown by the public `detectFromMagicBytes` API (which returns `undefined` for insufficient bytes rather than throwing). It is available for use by higher-level wrappers that prefer a fail-fast behavior over returning a nullable result. Matching the `received`/`required` context fields against monitoring thresholds enables proactive detection of truncated upload streams.
|
||||
|
||||
### Callers
|
||||
|
||||
Thrown by stream-oriented detection paths (`detectFromStream`) when the upstream `ReadableStream<Uint8Array>` yields fewer bytes than any candidate signature's `minBytes` requires. Callers that do their own buffering should catch this specifically and request more bytes rather than giving up — most signatures need only 4–16 bytes, so a single additional chunk is usually enough. The error carries the `requested` and `available` byte counts for log context.
|
||||
@@ -0,0 +1,72 @@
|
||||
# errors/invalid-content-type-error
|
||||
|
||||
## Purpose
|
||||
|
||||
Thrown when a `Content-Type` header or media type string fails RFC 7231 validation. Carries the raw input string and optional byte position in the `context` field to help callers produce useful error messages for end users and debugging.
|
||||
|
||||
## Features
|
||||
|
||||
- Error code `MIME_ERROR_CODES.INVALID_CONTENT_TYPE`
|
||||
- Optional `context.raw` field with the offending input string
|
||||
- Optional `context.position` field with the byte offset where parsing failed
|
||||
- `name` set to `"InvalidContentTypeError"`
|
||||
- Full `SuaveplanError` serialization (JSON + binary wire format, redaction, OTel)
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { parseContentType } from "@suaveplan/mime/content-type/parse-content-type";
|
||||
import { InvalidContentTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
try {
|
||||
parseContentType("not a valid content type");
|
||||
} catch (err) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
console.error("Invalid input:", err.context?.raw);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { InvalidContentTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// Throw directly with full context
|
||||
function assertValidContentType(header: string): void {
|
||||
if (!header.includes("/")) {
|
||||
throw new InvalidContentTypeError({
|
||||
message: `Content-Type must contain a slash: "${header}"`,
|
||||
context: { raw: header, position: 0 },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Wire-format serialization is inherited from SuaveplanError
|
||||
const err = new InvalidContentTypeError({
|
||||
message: "bad input",
|
||||
context: { raw: "text" },
|
||||
});
|
||||
const json = err.toJSON(); // { code, message, context, name, ... }
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class InvalidContentTypeError extends MimeError`
|
||||
|
||||
**Constructor options:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `message` | `string` | Human-readable error description |
|
||||
| `context?.raw` | `string` | The raw input that failed parsing |
|
||||
| `context?.position` | `number` | Byte offset where the error occurred |
|
||||
| `cause?` | `unknown` | Underlying cause (for cause-chain traversal) |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The `context` field uses `exactOptionalPropertyTypes` semantics — the field is entirely absent when not provided, not set to `undefined`. This matches `@suaveplan/error`'s strict typing conventions and avoids false positives when checking `err.context?.raw`.
|
||||
|
||||
### Callers
|
||||
|
||||
Thrown by `parseContentType` when the RFC 7231 grammar is violated: missing `/`, invalid token characters in type or subtype, malformed parameter syntax, or an unterminated quoted-string. The error's `input` field carries the raw header value so API logs can show exactly what the client sent. Consumers generally respond with HTTP 400 — the request is unparseable, not merely unsupported.
|
||||
59
docs/foundation/data/mime/src/errors/mime-error.md
Normal file
59
docs/foundation/data/mime/src/errors/mime-error.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# errors/mime-error
|
||||
|
||||
## Purpose
|
||||
|
||||
The base error class for all `@suaveplan/mime` errors. Extends `SuaveplanError` from `@suaveplan/error` with the `MIME_ERROR_CODES` namespace and provides a common ancestor for `instanceof` checks across all MIME-specific error types.
|
||||
|
||||
## Features
|
||||
|
||||
- Extends `SuaveplanError` for consistent error serialization and OTel integration
|
||||
- `code` field set to `MIME_ERROR_CODES.INVALID_CONTENT_TYPE` by default (overridden by subclasses)
|
||||
- `name` property set to `"MimeError"` for easy identification in logs
|
||||
- Compatible with `SuaveplanError`'s cause-chain traversal and redaction APIs
|
||||
- Used as the base for `instanceof MimeError` checks when callers don't need the specific subtype
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { MimeError } from "@suaveplan/mime/errors";
|
||||
|
||||
try {
|
||||
// Any MIME operation
|
||||
} catch (err) {
|
||||
if (err instanceof MimeError) {
|
||||
console.error("MIME error:", err.message, "code:", err.code);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { MimeError } from "@suaveplan/mime/errors";
|
||||
import { InvalidContentTypeError, UnknownMimeTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// Route by specific subtype
|
||||
function handleMimeError(err: MimeError) {
|
||||
if (err instanceof InvalidContentTypeError) {
|
||||
return Response.json({ error: "bad_content_type" }, { status: 400 });
|
||||
}
|
||||
if (err instanceof UnknownMimeTypeError) {
|
||||
return Response.json({ error: "unsupported_type" }, { status: 415 });
|
||||
}
|
||||
throw err; // Unexpected subtype — re-throw
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class MimeError extends SuaveplanError`
|
||||
|
||||
The base class. All `@suaveplan/mime` error classes extend this.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`MimeError` exists primarily to provide a stable `instanceof` target for callers who want to catch any MIME-related error without caring about the specific subtype. All subclasses set their own `code` from `MIME_ERROR_CODES` and override `name` with their class name. The inheritance chain is: `SuaveplanError → MimeError → {subtype}`.
|
||||
|
||||
### Subclass Index
|
||||
|
||||
Direct subclasses are `InvalidContentTypeError`, `UnknownMimeTypeError`, `UnknownSignatureError`, `InsufficientBytesError`, and `ParseError`. Every subclass sets its own `code` from the stable `MIME_ERROR_CODES` registry, overrides `name`, and inherits the wire-serialisation, OTel span-event adapter, and i18n resolver behaviour from `SuaveplanError`. Tests typically assert `err instanceof MimeError` first for broad catches, then narrow with `instanceof` on the specific subclass.
|
||||
69
docs/foundation/data/mime/src/errors/parse-error.md
Normal file
69
docs/foundation/data/mime/src/errors/parse-error.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# errors/parse-error
|
||||
|
||||
## Purpose
|
||||
|
||||
Thrown when a general parsing operation fails — for example, XML declaration parsing or BOM detection failures. Unlike `InvalidContentTypeError`, which is specific to RFC 7231 Content-Type headers, `ParseError` is the catch-all for any other structured-text parsing failure within the `@suaveplan/mime` package.
|
||||
|
||||
## Features
|
||||
|
||||
- Error code `MIME_ERROR_CODES.PARSE_ERROR`
|
||||
- `name` set to `"ParseError"`
|
||||
- Optional context fields for the offending input
|
||||
- Full `SuaveplanError` serialization
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { ParseError } from "@suaveplan/mime/errors";
|
||||
|
||||
function parseXmlEncoding(text: string): string {
|
||||
const match = /encoding=['"]([^'"]+)/.exec(text);
|
||||
if (!match) {
|
||||
throw new ParseError({
|
||||
message: "No encoding declaration found in XML prolog",
|
||||
context: { raw: text.slice(0, 100) },
|
||||
});
|
||||
}
|
||||
return match[1];
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { MimeError, ParseError } from "@suaveplan/mime/errors";
|
||||
|
||||
// Catch any MIME parse error generically
|
||||
try {
|
||||
processDocument(buffer);
|
||||
} catch (err) {
|
||||
if (err instanceof ParseError) {
|
||||
console.error("Document parse failed:", err.message);
|
||||
return { error: "parse_failed", details: err.context };
|
||||
}
|
||||
if (err instanceof MimeError) {
|
||||
// Other MIME errors
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class ParseError extends MimeError`
|
||||
|
||||
**Constructor options:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `message` | `string` | Human-readable error description |
|
||||
| `context?` | `object` | Arbitrary structured context for debugging |
|
||||
| `cause?` | `unknown` | Underlying cause |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`ParseError` is intentionally generic so that future parsing features (XML namespace detection, HTTP range header parsing, etc.) can reuse it without creating a proliferation of narrow error classes. If a specific context shape becomes widely used, a dedicated subclass can be introduced with a more precise `context` type.
|
||||
|
||||
### Callers
|
||||
|
||||
Thrown by lower-level parser stages when a structurally-recognised format cannot be fully parsed — for example a truncated multipart boundary, an invalid base64 parameter value, or a charset declaration that doesn't round-trip through `TextDecoder`. Distinct from `InvalidContentTypeError`, which is reserved for the RFC 7231 grammar checker. The error captures both the `stage` that failed and the raw `input` for log replay.
|
||||
@@ -0,0 +1,63 @@
|
||||
# errors/unknown-mime-type-error
|
||||
|
||||
## Purpose
|
||||
|
||||
Thrown when a MIME type string is not found in the built-in database and no fallback is available. Carries the unrecognized MIME type string in `context.mime` so callers can log, redirect, or prompt users to provide additional information.
|
||||
|
||||
## Features
|
||||
|
||||
- Error code `MIME_ERROR_CODES.UNKNOWN_MIME_TYPE`
|
||||
- Optional `context.mime` field with the unrecognized MIME type string
|
||||
- `name` set to `"UnknownMimeTypeError"`
|
||||
- Full `SuaveplanError` serialization
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { getEntryByMime } from "@suaveplan/mime/db/registry";
|
||||
import { UnknownMimeTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
function requireMimeEntry(mime: string) {
|
||||
const entry = getEntryByMime(mime);
|
||||
if (!entry) {
|
||||
throw new UnknownMimeTypeError({
|
||||
context: { mime },
|
||||
});
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { UnknownMimeTypeError } from "@suaveplan/mime/errors";
|
||||
|
||||
// Auto-generated message includes the MIME type
|
||||
const err = new UnknownMimeTypeError({ context: { mime: "application/x-exotic" } });
|
||||
console.log(err.message); // "Unknown MIME type: application/x-exotic"
|
||||
|
||||
// Without context — uses "unknown" fallback in message
|
||||
const err2 = new UnknownMimeTypeError({ message: "MIME type not recognized" });
|
||||
console.log(err2.context); // undefined
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class UnknownMimeTypeError extends MimeError`
|
||||
|
||||
**Constructor options:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `message?` | `string` | Human-readable message (auto-generated from context.mime if omitted) |
|
||||
| `context?.mime` | `string` | The unrecognized MIME type string |
|
||||
| `cause?` | `unknown` | Underlying cause |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The error message is auto-generated as `"Unknown MIME type: {mime}"` when `context.mime` is provided and no explicit `message` is supplied. This convention reduces boilerplate for the common case of constructing the error from a lookup failure.
|
||||
|
||||
### Callers
|
||||
|
||||
Thrown by the strict variants of registry lookups (`mimeTypeOrThrow`, `mimeCategoryOrThrow`) when the supplied MIME string has no entry in the curated IANA database. Non-strict variants return `undefined` instead. The error carries the requested MIME string so callers can log, suggest the nearest registered type, or fall back to a default. Usually handled at an API boundary with HTTP 415.
|
||||
@@ -0,0 +1,67 @@
|
||||
# errors/unknown-signature-error
|
||||
|
||||
## Purpose
|
||||
|
||||
Thrown when a byte signature identifier is referenced but is not present in the signature registry. Used as a sentinel for development-time validation when code expects a specific signature to be registered (for example, in plugin systems that verify their registration succeeded).
|
||||
|
||||
## Features
|
||||
|
||||
- Error code `MIME_ERROR_CODES.UNKNOWN_SIGNATURE`
|
||||
- `context.signature` — the expected signature identifier or MIME type
|
||||
- `name` set to `"UnknownSignatureError"`
|
||||
- Full `SuaveplanError` serialization
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { UnknownSignatureError } from "@suaveplan/mime/errors";
|
||||
import { getSignatures } from "@suaveplan/mime/detection/from-magic-bytes/registry";
|
||||
|
||||
function requireSignature(mime: string): void {
|
||||
const found = getSignatures().some(s => s.mime === mime);
|
||||
if (!found) {
|
||||
throw new UnknownSignatureError({
|
||||
context: { signature: mime },
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { UnknownSignatureError } from "@suaveplan/mime/errors";
|
||||
|
||||
// In a plugin registration system
|
||||
function verifySignaturePlugin(pluginMime: string): void {
|
||||
const isRegistered = getSignatures().some(s => s.mime === pluginMime);
|
||||
if (!isRegistered) {
|
||||
throw new UnknownSignatureError({
|
||||
message: `Plugin signature for "${pluginMime}" was not registered`,
|
||||
context: { signature: pluginMime },
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `class UnknownSignatureError extends MimeError`
|
||||
|
||||
**Constructor options:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `message?` | `string` | Human-readable error description |
|
||||
| `context?.signature` | `string` | The expected signature identifier |
|
||||
| `cause?` | `unknown` | Underlying cause |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
This error is not thrown by any built-in `@suaveplan/mime` API — it is provided for consumers who build plugin-style extension systems on top of the signature registry and want explicit registration verification. The `detectFromMagicBytes` function returns `undefined` for unrecognized bytes rather than throwing, keeping the detection path exception-free.
|
||||
|
||||
### Callers
|
||||
|
||||
Thrown by strict magic-byte detection (`detectFromMagicBytesOrThrow`) when the supplied bytes match no registered `Signature`. Non-strict `detectFromMagicBytes` returns `undefined`. The error carries the first few bytes as a hex string in `prefix` so logs capture enough context to diagnose missing signatures without dumping the full payload.
|
||||
|
||||
The `prefix` field is capped at the first 16 bytes of the input so log aggregators never see enormous hex blobs even when the caller passed a multi-megabyte buffer. That cap keeps logs readable and stops a surprising memory amplification path at the telemetry layer.
|
||||
65
docs/foundation/data/mime/src/legacy/get-mime-type.md
Normal file
65
docs/foundation/data/mime/src/legacy/get-mime-type.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# legacy/get-mime-type
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides a simplified, backwards-compatible `getMimeType` function that accepts a filename or extension and returns the MIME type string, or a configurable default when no match is found. Designed for codebases migrating from `mime` or `mime-types` npm packages that used a similar single-function API.
|
||||
|
||||
## Features
|
||||
|
||||
- Simple `getMimeType(filename, fallback?)` interface
|
||||
- Returns the MIME type string directly (not wrapped in a result object)
|
||||
- Configurable fallback (default: `"application/octet-stream"`)
|
||||
- Strips path components and handles extensions with or without dots
|
||||
- Zero dependencies beyond the built-in registry
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { getMimeType } from "@suaveplan/mime/legacy/get-mime-type";
|
||||
|
||||
console.log(getMimeType("document.pdf")); // "application/pdf"
|
||||
console.log(getMimeType(".ts")); // "video/mp2t" (TypeScript)
|
||||
console.log(getMimeType("unknown.xyz")); // "application/octet-stream"
|
||||
console.log(getMimeType("unknown.xyz", "text/plain")); // "text/plain"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { getMimeType } from "@suaveplan/mime/legacy/get-mime-type";
|
||||
|
||||
// Migrate from the 'mime' npm package
|
||||
// Before: mime.getType("photo.jpg") → "image/jpeg"
|
||||
// After: getMimeType("photo.jpg") → "image/jpeg"
|
||||
|
||||
function serveFile(filePath: string, buffer: Uint8Array): Response {
|
||||
const mimeType = getMimeType(filePath, "application/octet-stream");
|
||||
return new Response(buffer, {
|
||||
headers: { "Content-Type": mimeType },
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `getMimeType(filename: string, fallback?: string): string`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `filename` | `string` | Filename, path, or extension (with or without dot). |
|
||||
| `fallback?` | `string` | Returned when no match found (default: `"application/octet-stream"`). |
|
||||
|
||||
**Returns:** MIME type string.
|
||||
|
||||
**Throws:** Never.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`getMimeType` is a thin convenience wrapper over `detectFromExtension`. It is intentionally limited to extension-based detection — for production code requiring high-confidence detection (including magic-byte analysis), use the `detect()` function from `@suaveplan/mime/detection/detect` instead. This function is in the `legacy/` directory to signal that new code should prefer the more explicit multi-source API.
|
||||
|
||||
### Deprecation Plan
|
||||
|
||||
`getMimeType` is retained for callers migrating from the pre-`@suaveplan/mime` helpers; new code should call `mimeFromExtension` / `detectFromMagicBytes` directly. A future major release may remove this wrapper — it currently forwards to the real extension-to-MIME map and returns `undefined` when nothing matches, which is the exact contract the old helper advertised.
|
||||
70
docs/foundation/data/mime/src/reverse/reverse.md
Normal file
70
docs/foundation/data/mime/src/reverse/reverse.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# reverse/reverse
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Provides `getMimeExtension` for reverse lookup: given a MIME type string, returns the canonical file extension associated with that type. This is the inverse of extension-based detection — useful for generating filenames, setting `Content-Disposition` headers, and file download naming.
|
||||
|
||||
## Features
|
||||
|
||||
- Returns the canonical (first) extension for any registered MIME type
|
||||
- Returns `undefined` for unknown MIME types (no throws)
|
||||
- Case-insensitive MIME type matching (delegates to registry)
|
||||
- Returns extensions without a leading dot
|
||||
- O(1) lookup via the registry's forward map
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { getMimeExtension } from "@suaveplan/mime/reverse/reverse";
|
||||
|
||||
console.log(getMimeExtension("image/png")); // "png"
|
||||
console.log(getMimeExtension("application/pdf")); // "pdf"
|
||||
console.log(getMimeExtension("text/javascript")); // "js"
|
||||
console.log(getMimeExtension("unknown/type")); // undefined
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { getMimeExtension } from "@suaveplan/mime/reverse/reverse";
|
||||
|
||||
function buildFilename(baseName: string, mimeType: string): string {
|
||||
const ext = getMimeExtension(mimeType);
|
||||
if (!ext) return baseName; // Unknown type — keep original name
|
||||
// Avoid double extension
|
||||
return baseName.endsWith(`.${ext}`) ? baseName : `${baseName}.${ext}`;
|
||||
}
|
||||
|
||||
// Content-Disposition header generation
|
||||
function contentDisposition(mimeType: string, name: string): string {
|
||||
const ext = getMimeExtension(mimeType);
|
||||
const filename = ext ? `${name}.${ext}` : name;
|
||||
return `attachment; filename="${filename}"`;
|
||||
}
|
||||
|
||||
console.log(contentDisposition("application/pdf", "report"));
|
||||
// 'attachment; filename="report.pdf"'
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `getMimeExtension(mime: string): string | undefined`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `mime` | `string` | MIME type string (e.g., `"image/png"`). |
|
||||
|
||||
**Returns:** The canonical file extension string without a leading dot, or `undefined` if the MIME type is not in the database.
|
||||
|
||||
**Throws:** Never.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The "canonical" extension is the first element of the `extensions` array in the `MimeEntry`. The built-in database places the most common extension first by convention (e.g., `"jpeg"` before `"jpg"` for `image/jpeg`). Callers that need all extensions for a type should use `getEntryByMime` directly from `@suaveplan/mime/db/registry` and access `entry.extensions`.
|
||||
|
||||
### Lookup Tables
|
||||
|
||||
The reverse index maps MIME string → preferred extension, MIME string → alias list, and alias → canonical MIME. Tables are precomputed at module load from the same curated IANA database the forward lookup uses, so the two directions can never drift. Lookups are O(1) against frozen `Map` instances with zero runtime cost after first import.
|
||||
61
docs/foundation/data/mime/src/telemetry/meters.md
Normal file
61
docs/foundation/data/mime/src/telemetry/meters.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# telemetry/meters
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
Initializes OpenTelemetry metrics instruments for `@suaveplan/mime`. Provides counters and histograms for tracking MIME detection operations, cache hits, and processing durations. Callers supply their own `Meter` instance so this module has no direct dependency on any specific OTel SDK implementation.
|
||||
|
||||
## Features
|
||||
|
||||
- Lazy initialization — instruments are created once on first `initMeters` call
|
||||
- Idempotent — subsequent `initMeters` calls after the first are no-ops
|
||||
- Counters: detection calls by source, detection failures, charset sniff operations
|
||||
- Histograms: detection latency, stream read bytes
|
||||
- No OTel SDK dependency — accepts any `Meter`-compatible interface
|
||||
- Zero-cost no-op path when telemetry is not initialized
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```typescript
|
||||
import { initMeters } from "@suaveplan/mime/telemetry/meters";
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
|
||||
// Call once at application startup
|
||||
const meter = metrics.getMeter("@suaveplan/mime");
|
||||
initMeters(meter);
|
||||
|
||||
// All subsequent MIME operations will record metrics automatically
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
```typescript
|
||||
import { initMeters, getMimeMeters } from "@suaveplan/mime/telemetry/meters";
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
|
||||
// Initialize with a named meter for better grouping in dashboards
|
||||
initMeters(metrics.getMeter("@suaveplan/mime", "1.0.0"));
|
||||
|
||||
// Access instruments directly for custom recording
|
||||
const meters = getMimeMeters();
|
||||
meters?.detectionCounter.add(1, {
|
||||
source: "magic-bytes",
|
||||
mime: "image/png",
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `initMeters(meter: Meter): void`
|
||||
|
||||
Initializes the metric instruments using the provided OTel `Meter`. Idempotent — only the first call has effect.
|
||||
|
||||
### `getMimeMeters(): MimeMeters | undefined`
|
||||
|
||||
Returns the initialized meters object, or `undefined` if `initMeters` has not been called.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The idempotency guard uses a module-level `let meters: MimeMeters | undefined` variable checked at the top of `initMeters`. This avoids re-creating instruments if `initMeters` is accidentally called multiple times (e.g., from multiple entry points in a large application). The no-op fallback (`getMimeMeters()` returning `undefined`) means callers can use optional chaining without any conditional initialization checks: `getMimeMeters()?.detectionCounter.add(1, attrs)`.
|
||||
129
docs/foundation/data/packer-browser/README.md
Normal file
129
docs/foundation/data/packer-browser/README.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# @suaveplan/packer-browser - Browser runtime for @suaveplan/packer — Service Worker interceptor, PackedResourceManager (LRU + Blob-URL bookkeeping), signed-manifest verification, OTel metrics, AbortSignal-aware loader. No React.
|
||||
|
||||
Browser-tier runtime for `@suaveplan/packer` archives. Ships a Service Worker
|
||||
interceptor, a `PackedResourceManager` (LRU + Blob-URL bookkeeping +
|
||||
ref-count + OTel metrics), Ed25519 archive-signature verification, and
|
||||
lifecycle helpers for registering / clearing / inspecting the SW cache. No
|
||||
React runtime dependency — React hooks land in the companion
|
||||
`@suaveplan/react-packer` package.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [adaptive-cache](./src/adaptive-cache/adaptive-cache.md) — `createAdaptiveCacheSizer` periodically resizes a `PackedResourceCache` from a `navigator.storage.estimate()`-style quo…
|
||||
- [adaptive-cache](./src/adaptive-cache/adaptive-cache.md) — `createAdaptiveCacheSizer` periodically resizes a `PackedResourceCache` from a `navigator.storage.estimate()`-style quo…
|
||||
- [cache](./src/cache/cache.md) — `cache.ts` implements `PackedResourceCache` — an LRU + ref-count cache that owns the Blob-URL lifecycle for every unpac…
|
||||
- [cache](./src/cache/cache.md) — `cache.ts` implements `PackedResourceCache` — an LRU + ref-count cache that owns the Blob-URL lifecycle for every unpac…
|
||||
- [content-address](./src/content-address/content-address.md) — Content-addressed URL builder and cache-invalidation helpers for packed archives. Each archive's canonical URL is deriv…
|
||||
- [content-address](./src/content-address/content-address.md) — Content-addressed URL builder and cache-invalidation helpers for packed archives. Each archive's canonical URL is deriv…
|
||||
- [diagnostics](./src/diagnostics/diagnostics.md) — `createDiagnosticsOverlay` produces a development-time overlay that displays live cache statistics and a tail of recent…
|
||||
- [diagnostics](./src/diagnostics/diagnostics.md) — `createDiagnosticsOverlay` produces a development-time overlay that displays live cache statistics and a tail of recent…
|
||||
- [encrypted-archive](./src/encrypted-archive/encrypted-archive.md) — Wire format for AES-256-GCM-encrypted packed archives served alongside unencrypted bundles. Adopts the stable magic pre…
|
||||
- [encrypted-archive](./src/encrypted-archive/encrypted-archive.md) — Wire format for AES-256-GCM-encrypted packed archives served alongside unencrypted bundles. Adopts the stable magic pre…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — `error-codes.ts` is the single source of truth for every error class and stable code emitted by `@suaveplan/packer-brow…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — `error-codes.ts` is the single source of truth for every error class and stable code emitted by `@suaveplan/packer-brow…
|
||||
- [flags](./src/flags/flags.md) — `flags.ts` resolves the `@suaveplan/feature-flags` `FlagProvider` used by `PackedResourceManager` to gate Cluster B / C…
|
||||
- [flags](./src/flags/flags.md) — `flags.ts` resolves the `@suaveplan/feature-flags` `FlagProvider` used by `PackedResourceManager` to gate Cluster B / C…
|
||||
- [i18n](./src/i18n/messages.md) — `messages.ts` is the English defaults catalog plus the resolver hook for every user-visible error message thrown by `@s…
|
||||
- [messages](./src/i18n/messages.md) — `messages.ts` is the English defaults catalog plus the resolver hook for every user-visible error message thrown by `@s…
|
||||
- [keyring](./src/keyring/keyring.md) — Defines the `KeyringProvider` contract for resolving symmetric AES-256-GCM keys at decrypt time. The contract is intent…
|
||||
- [keyring](./src/keyring/keyring.md) — Defines the `KeyringProvider` contract for resolving symmetric AES-256-GCM keys at decrypt time. The contract is intent…
|
||||
- [loader](./src/loader/loader.md) — `loader.ts` is the fetch → verify → unpack → Blob-URL pipeline for packed resources. It is the only place in the packag…
|
||||
- [loader-helpers](./src/loader/loader-helpers.md) — Small free helpers extracted from `loader.ts` so the class stays under the 300-line hard cap. These functions carry no…
|
||||
- [loader-stream](./src/loader/loader-stream.md) — Progressive entry-by-entry unpack helper that powers `PackedResourceLoader.loadStream`. Callers pass a list of packed-a…
|
||||
- [loader](./src/loader/loader.md) — `loader.ts` is the fetch → verify → unpack → Blob-URL pipeline for packed resources. It is the only place in the packag…
|
||||
- [manager](./src/manager/manager.md) — `manager.ts` is the lifecycle façade that consumers interact with. It composes a `PackedResourceCache` and a `PackedRes…
|
||||
- [manager-helpers](./src/manager/manager-helpers.md) — Small free helpers extracted from `manager.ts` to keep the class file under CLAUDE.md §3's 300-line hard cap. Both help…
|
||||
- [manager-stream](./src/manager/manager-stream.md) — Free-function implementation of `PackedResourceManager.loadStream`, factored out of `manager.ts` so the class stays und…
|
||||
- [manager](./src/manager/manager.md) — `manager.ts` is the lifecycle façade that consumers interact with. It composes a `PackedResourceCache` and a `PackedRes…
|
||||
- [tiny-emitter](./src/manager/tiny-emitter.md) — Minimal generic typed event emitter used internally by `PackedResourceManager`. Factored out of `manager.ts` so the cla…
|
||||
- [observability](./src/observability/observability.md) — `observability.ts` creates the OpenTelemetry instrument bundle for `@suaveplan/packer-browser`. The package emits four…
|
||||
- [observability](./src/observability/observability.md) — `observability.ts` creates the OpenTelemetry instrument bundle for `@suaveplan/packer-browser`. The package emits four…
|
||||
- [prefetch](./src/prefetch/prefetch.md) — The `prefetch` module wraps the Chromium `BackgroundFetchManager` API so an application can preload an archive on idle…
|
||||
- [prefetch](./src/prefetch/prefetch.md) — The `prefetch` module wraps the Chromium `BackgroundFetchManager` API so an application can preload an archive on idle…
|
||||
- [preload](./src/preload/preload.md) — `generatePreloadHints` derives an ordered list of `<link rel="preload">` hints from a packed manifest, and `injectPrelo…
|
||||
- [preload](./src/preload/preload.md) — `generatePreloadHints` derives an ordered list of `<link rel="preload">` hints from a packed manifest, and `injectPrelo…
|
||||
- [range-response](./src/range-response/range-response.md) — RFC 7233 §2.1 single-range `Response` builder for packed-resource delivery from the Service Worker. Browsers ask for ra…
|
||||
- [range-response](./src/range-response/range-response.md) — RFC 7233 §2.1 single-range `Response` builder for packed-resource delivery from the Service Worker. Browsers ask for ra…
|
||||
- [runtime](./src/runtime/runtime.md) — `runtime.ts` centralises the three injectable platform primitives (`Clock`, `MonotonicClock`, `Rng`) that every product…
|
||||
- [runtime](./src/runtime/runtime.md) — `runtime.ts` centralises the three injectable platform primitives (`Clock`, `MonotonicClock`, `Rng`) that every product…
|
||||
- [service-worker](./src/service-worker/service-worker.md) — `service-worker.ts` is the production Service Worker entrypoint exposed via the `/service-worker` subpath. Its only pub…
|
||||
- [service-worker](./src/service-worker/service-worker.md) — `service-worker.ts` is the production Service Worker entrypoint exposed via the `/service-worker` subpath. Its only pub…
|
||||
- [sw-shim](./src/service-worker/sw-shim.md) — `sw-shim.ts` provides `createServiceWorkerShim()`, a hand-rolled in-memory implementation of `ServiceWorkerGlobalScope`…
|
||||
- [signature](./src/signature/signature.md) — `signature.ts` is the Ed25519 signature-verification gate run by the loader before any packed bytes are handed to `Unpa…
|
||||
- [signature](./src/signature/signature.md) — `signature.ts` is the Ed25519 signature-verification gate run by the loader before any packed bytes are handed to `Unpa…
|
||||
- [singleton](./src/singleton/singleton.md) — `singleton.ts` exposes the module-level shared `PackedResourceManager` instance behind the `@suaveplan/packer-browser/s…
|
||||
- [singleton](./src/singleton/singleton.md) — `singleton.ts` exposes the module-level shared `PackedResourceManager` instance behind the `@suaveplan/packer-browser/s…
|
||||
- [cross-origin](./src/utils/cross-origin.md) — `registerCrossOriginPackedResourceServiceWorker` lets an application register a packed-resource Service Worker hosted o…
|
||||
- [register-service-worker](./src/utils/register-service-worker.md) — `register-service-worker.ts` collects four lifecycle helpers exposed via the `/utils` subpath: `registerPackedResourceS…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/packer-browser
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import {
|
||||
createPackedResourceManager,
|
||||
registerPackedResourceServiceWorker,
|
||||
} from "@suaveplan/packer-browser";
|
||||
|
||||
await registerPackedResourceServiceWorker({ scriptUrl: "/sw.js" });
|
||||
const manager = createPackedResourceManager();
|
||||
|
||||
const blobUrl = await manager.load("/banner.packed");
|
||||
imgEl.src = blobUrl;
|
||||
```
|
||||
|
||||
For a global shared instance, import from the `/singleton` subpath:
|
||||
|
||||
```ts
|
||||
import { packedResourceManager } from "@suaveplan/packer-browser/singleton";
|
||||
const url = await packedResourceManager.load("/banner.packed");
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
- [`error-codes`](./src/error-codes.md) — `PACKER_BROWSER_CODES` + `PackerBrowserError`.
|
||||
- [`i18n/messages`](./src/i18n/messages.md) — `MessageResolver` catalog.
|
||||
- [`runtime/runtime`](./src/runtime/runtime.md) — injectable `Clock` / `Rng`.
|
||||
- [`observability/observability`](./src/observability/observability.md) — OTel counters + histogram.
|
||||
- [`flags/flags`](./src/flags/flags.md) — `@suaveplan/feature-flags` slot.
|
||||
- [`signature/signature`](./src/signature/signature.md) — `verifyArchiveSignature` (Ed25519).
|
||||
- [`cache/cache`](./src/cache/cache.md) — `PackedResourceCache` LRU + ref-count.
|
||||
- [`loader/loader`](./src/loader/loader.md) — `PackedResourceLoader` fetch / verify / unpack.
|
||||
- [`manager/manager`](./src/manager/manager.md) — `PackedResourceManager` lifecycle façade + factory.
|
||||
- [`service-worker/service-worker`](./src/service-worker/service-worker.md) — production SW entrypoint.
|
||||
- [`service-worker/sw-shim`](./src/service-worker/sw-shim.md) — hand-rolled `ServiceWorkerGlobalScope` shim for unit tests.
|
||||
- [`utils/register-service-worker`](./src/utils/register-service-worker.md) — SW lifecycle helpers.
|
||||
- [`singleton`](./src/singleton.md) — module-singleton hosted at `/singleton` subpath.
|
||||
|
||||
## Subpath exports
|
||||
|
||||
- `.` — full default surface (excludes the module singleton).
|
||||
- `./manager` — `PackedResourceManager`, factory, types.
|
||||
- `./service-worker` — `runPackedResourceServiceWorker` + `createServiceWorkerShim`.
|
||||
- `./utils` — `register` / `unregister` / `clearServiceWorkerCache` / `getServiceWorkerCacheStatus`.
|
||||
- `./singleton` — eagerly-constructed shared `packedResourceManager`.
|
||||
|
||||
## Scope
|
||||
|
||||
Cluster A delivers the production-quality core: manager split, OTel
|
||||
metrics, signed-manifest verification, `AbortSignal` propagation,
|
||||
injected `Clock`/`Rng`, retry on the source-pull, feature-flag slot,
|
||||
unit-tested SW with a real shim, and the five §3 subpath exports.
|
||||
|
||||
Cluster B and C add progressive streaming, range-request SW, adaptive
|
||||
cache sizing, encrypted-archive keyring, content-addressed versioned
|
||||
URLs, BackgroundFetch, multi-origin SW helpers, and the diagnostics
|
||||
overlay. See the design proposal for
|
||||
the full out-of-scope list.
|
||||
|
||||
## License
|
||||
|
||||
MIT.
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
# `adaptive-cache` — `navigator.storage.estimate()`-driven cache sizing
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`createAdaptiveCacheSizer` periodically resizes a `PackedResourceCache`
|
||||
from a `navigator.storage.estimate()`-style quota report so the LRU
|
||||
ceiling tracks the actual storage the origin has available rather than a
|
||||
static guess. The sizer is the production implementation of
|
||||
PACKER-BROWSER-17 — adaptive cache sizing — and is what enables a
|
||||
packer-browser deployment to ride the upstream quota without manually
|
||||
re-tuning the cache on every device.
|
||||
|
||||
The module is **Clock-free**: it relies on the configured `intervalMs`
|
||||
through an injectable timer (default `setInterval`), so deterministic
|
||||
tests substitute a manual tick implementation rather than mocking time.
|
||||
Cancellation is propagated through an `AbortSignal`; pre-aborted signals
|
||||
short-circuit construction so the sizer never schedules work.
|
||||
|
||||
## Features
|
||||
|
||||
- Initial estimate fired synchronously on construction
|
||||
- Recurring refresh on a configurable interval (default `60_000` ms)
|
||||
- Pluggable `storageEstimate` injection (defaults to
|
||||
`() => navigator.storage.estimate()` when available)
|
||||
- Graceful no-op fallback when neither the injection nor the global API
|
||||
is present
|
||||
- `AbortSignal`-aware: aborting cancels the timer and stops further
|
||||
estimate calls
|
||||
- Disposer is idempotent and can be called from a manager's
|
||||
shutdown path
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { createAdaptiveCacheSizer } from "@suaveplan/packer-browser/adaptive-cache";
|
||||
import { PackedResourceCache } from "@suaveplan/packer-browser";
|
||||
|
||||
const cache = new PackedResourceCache({ runtime, observability });
|
||||
const sizer = createAdaptiveCacheSizer({ cache });
|
||||
// On teardown:
|
||||
sizer.dispose();
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
const controller = new AbortController();
|
||||
const sizer = createAdaptiveCacheSizer({
|
||||
cache,
|
||||
fraction: 0.1, // 10 % of quota
|
||||
intervalMs: 30_000,
|
||||
storageEstimate: async () => ({ quota: 8 * 1024 * 1024 * 1024 }),
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
await sizer.refresh(); // one-shot refresh
|
||||
console.log(sizer.getLastApplied()); // 858_993_459
|
||||
|
||||
controller.abort(); // stops the recurring updater
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `createAdaptiveCacheSizer(opts: AdaptiveCacheSizerOptions): AdaptiveCacheSizer`
|
||||
- `opts.cache` — `PackedResourceCache` to size. Required.
|
||||
- `opts.fraction` — clamped to `(0, 1]`. Default `0.25`.
|
||||
- `opts.intervalMs` — refresh interval. Default `60_000`.
|
||||
- `opts.storageEstimate` — pluggable estimate function.
|
||||
- `opts.signal` — cancellation `AbortSignal`.
|
||||
- `opts.setIntervalImpl` — test-only timer injection.
|
||||
- Returns: `{ dispose, refresh, getLastApplied }`.
|
||||
|
||||
- `AdaptiveCacheSizer.dispose()` — idempotent stop.
|
||||
- `AdaptiveCacheSizer.refresh(): Promise<void>` — one-shot refresh.
|
||||
- `AdaptiveCacheSizer.getLastApplied(): number | undefined` — most recent
|
||||
ceiling applied, or `undefined` before the first refresh.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- The initial estimate is fired with `void refresh()` so construction
|
||||
is synchronous and the call returns the disposer immediately. Callers
|
||||
who need deterministic timing can `await sizer.refresh()`.
|
||||
- When `quota` is missing or non-positive the sizer is a defensive no-op
|
||||
for that tick — the cache's previous ceiling is preserved.
|
||||
- The default `setInterval` adapter calls `.unref()` when present so a
|
||||
Node test process is not held open by the recurring updater.
|
||||
- The sizer never reads the cache; it only writes via
|
||||
`cache.setMaxCacheSize(n)`, which is itself a runtime mutator that
|
||||
performs eviction in-place.
|
||||
104
docs/foundation/data/packer-browser/src/cache/cache.md
vendored
Normal file
104
docs/foundation/data/packer-browser/src/cache/cache.md
vendored
Normal file
@@ -0,0 +1,104 @@
|
||||
# `cache`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
`cache.ts` implements `PackedResourceCache` — an LRU + ref-count cache
|
||||
that owns the Blob-URL lifecycle for every unpacked resource the
|
||||
`PackedResourceLoader` produces. It exists as a discrete collaborator
|
||||
(rather than baked into the manager) so the manager file stays under
|
||||
300 LOC and so the eviction policy is independently testable. The cache
|
||||
also owns the OTel instrumentation for cache hits / misses.
|
||||
|
||||
## Features
|
||||
|
||||
- LRU eviction with refCount pinning: entries with `refCount > 0`
|
||||
cannot be evicted; on overflow the least-recently-cached
|
||||
`refCount === 0` entry is dropped.
|
||||
- Blob-URL lifecycle: every eviction revokes the Blob URL through an
|
||||
injectable `revokeObjectUrl` (defaults to `URL.revokeObjectURL`).
|
||||
- OTel-instrumented `get`: every call records exactly one
|
||||
`cacheHitsTotal` or `cacheMissTotal` increment.
|
||||
- Clock-driven entry timestamps — wall-clock reads come from the
|
||||
injected `Clock` so deterministic tests can pin entry ages.
|
||||
- `cleanupExpired()` for periodic supervised eviction of stale
|
||||
entries; `clear()` for hard reset (revokes every URL regardless of
|
||||
refCount).
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
createPackerBrowserObservability,
|
||||
PackedResourceCache,
|
||||
resolveRuntime,
|
||||
} from "@suaveplan/packer-browser";
|
||||
|
||||
const cache = new PackedResourceCache({
|
||||
runtime: resolveRuntime(),
|
||||
observability: createPackerBrowserObservability(),
|
||||
});
|
||||
|
||||
const blob = new Blob(["payload"]);
|
||||
const url = URL.createObjectURL(blob);
|
||||
cache.insert("key", blob, url, {
|
||||
originalSize: 7,
|
||||
compressedSize: 7,
|
||||
encrypted: false,
|
||||
timestamp: Date.now(),
|
||||
checksum: "",
|
||||
});
|
||||
|
||||
const hit = cache.get("key");
|
||||
// hit.url === url; refCount === 2 (insert started at 1, get incremented)
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { FakeClock } from "@suaveplan/testing";
|
||||
import { PackedResourceCache } from "@suaveplan/packer-browser";
|
||||
|
||||
const clock = new FakeClock({ now: 0 });
|
||||
const revoked: string[] = [];
|
||||
const cache = new PackedResourceCache({
|
||||
runtime: { clock: () => clock.now(), monotonicClock: () => 0n, rng: defaultRng },
|
||||
observability,
|
||||
maxCacheAge: 60_000,
|
||||
revokeObjectUrl: (url) => revoked.push(url),
|
||||
});
|
||||
|
||||
// later, advance time and run periodic cleanup
|
||||
clock.tick(120_000);
|
||||
const evicted = cache.cleanupExpired();
|
||||
// evicted === 1; revoked === ["blob:..."]
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `new PackedResourceCache(opts: PackedResourceCacheOptions)`.
|
||||
- `cache.get(key): CachedResource | undefined` — increments refCount
|
||||
- refreshes timestamp on hit; records OTel counter on every call.
|
||||
- `cache.insert(key, blob, url, metadata): CachedResource` — adds an
|
||||
entry with refCount 1; evicts LRU on overflow.
|
||||
- `cache.release(key): void` — decrement refCount.
|
||||
- `cache.cleanupExpired(): number` — evict every expired
|
||||
refCount-zero entry; returns count.
|
||||
- `cache.clear(): void` — evict every entry (refCount ignored).
|
||||
- `cache.getStats(): CacheStats` — snapshot summary.
|
||||
- `cache.size(): number` / `cache.has(key): boolean` /
|
||||
`cache.evictLRU(): string | undefined`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The cache deliberately does NOT trigger a background timer for stale
|
||||
eviction. Periodic cleanup is the consumer's responsibility — the
|
||||
manager invokes `cleanupExpired()` at a cadence the caller controls,
|
||||
not on a hard-coded `setInterval`. This keeps the cache compatible
|
||||
with environments where timers are unavailable (e.g. some Service
|
||||
Worker contexts during install). The default `maxCacheSize` of 100
|
||||
mirrors the upstream value; the upcoming Cluster B adaptive sizing
|
||||
will override it based on `navigator.storage.estimate()`.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user