Initial Commit

This commit is contained in:
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions

View File

@@ -0,0 +1,121 @@
# @suaveplan/email-processing - Universal MIME parsing and email processing: postal-mime + libmime wrappers, attachment processing with SHA-256 hashing + MIME-type allowlists, Content-ID resolution, structured data extraction (iCalendar, delivery tracking, receipts), DKIM/SPF/DMARC verification, spam scoring with pluggable rule sets, conversation threading, hook pipeline + OTel instrumentation
Universal MIME parsing and email processing for the Suaveplan
. Wraps `postal-mime` and `libmime` with structured errors,
OpenTelemetry instrumentation, a hook pipeline for transforming
parsed messages, full attachment processing (SHA-256 hashing, MIME
allowlists, CID resolution), structured-data extraction (iCalendar,
delivery tracking, receipts), DKIM/SPF/DMARC verification, spam
scoring, and conversation threading.
<!-- begin:auto-docs -->
## Documentation tree
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
<!-- end:auto-docs -->
## Layout
```
src/
attachment-processor/ # SHA-256 hash + size + MIME allowlist validation
calendar-parser/ # iCalendar VEVENT extraction
cid-resolver/ # Content-ID resolution to data URIs / URL maps
data-extractor/ # tracking numbers, receipts, calendar events
dkim-canonicalize/ # RFC 6376 relaxed + simple canonicalisation
dkim-verifier/ # Tag parsing, body-hash, RSASSA-PKCS1-v1_5 verify
dmarc-checker/ # Policy + alignment over SPF + DKIM
error-codes/ # EMAIL_PROCESSING_CODES + EmailProcessingError
hooks/ # email:processing pipeline + singleton bus
mime-utils/ # libmime wrappers (encode/decode words, folding)
parser/ # parseEmail / parseEmailAddress / decodeEmailWords
spam-scorer/ # 12 default rules + custom-rule merge
spf-checker/ # RFC 7208 evaluator with include/redirect limits
telemetry/ # Shared OTel meter + instruments
thread-grouper/ # In-Reply-To + References + subject threading
types/ # Shared MIME / address / header type aliases
index.ts # Package barrel
```
## Quality
- 100% statement / branch / function / line coverage (Istanbul).
- Real `postal-mime` parses against embedded RFC 822 fixtures — no
mocks (rule 2).
- Test runner imports go through `@suaveplan/testing/runner` only
(rule 11).
- Zod-validated option boundaries (`parseEmail`, `processAttachment`).
- Two error codes:
`INVALID_INPUT`, `PARSE_FAILURE`, `INVALID_OPTIONS`,
`ATTACHMENT_TOO_LARGE`, `ATTACHMENT_TYPE_REJECTED`, `HASH_FAILURE`,
`DKIM_VERIFY_FAILURE`, `HOOK_FAILURE` — every code is thrown by at
least one production path under `src/`.
## Quick start
```ts
import {
parseEmail,
processAttachments,
onEmailProcessing,
} from "@suaveplan/email-processing";
onEmailProcessing((data) => ({
...data,
parsed: {
...data.parsed,
html: data.parsed.html?.replace(/<script[\s\S]*?<\/script>/gi, ""),
},
}));
const parsed = await parseEmail(rawMimeBuffer);
const processed = await processAttachments(parsed.attachments, {
maxSize: 10 * 1024 * 1024,
allowedMimeTypes: ["image/*", "application/pdf"],
});
```
## Subpath exports
`./parser`, `./mime-utils`, `./hooks`, `./attachments`, `./cid-resolver`,
`./extraction`, `./calendar`, `./dkim`, `./dmarc`, `./spf`, `./spam`,
`./threading`, `./errors`, `./otel`.
## Provenance
Migrated from `packages/universal/email-processing` on the
`/mnt/s/web/suaveplan` main branch per the enterprise plan at
`/mnt/s/web/suaveplan/enterprise/email-processing.md`. Hook
infrastructure was rewritten locally because the foundation
`@suaveplan/events` library does not yet ship a shared
`Hooks<T>` substrate.

View File

@@ -0,0 +1,152 @@
{
"name": "@suaveplan/email-processing",
"version": "0.1.0",
"type": "module",
"sideEffects": false,
"description": "Universal MIME parsing and email processing: postal-mime + libmime wrappers, attachment processing with SHA-256 hashing + MIME-type allowlists, Content-ID resolution, structured data extraction (iCalendar, delivery tracking, receipts), DKIM/SPF/DMARC verification, spam scoring with pluggable rule sets, conversation threading, hook pipeline + OTel instrumentation",
"license": "MIT",
"author": "Suaveplan Team",
"main": "./dist/index.js",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
"./package.json": "./package.json",
".": {
"import": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
},
"require": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"./parser": {
"import": {
"types": "./dist/parser/index.d.ts",
"default": "./dist/parser/index.js"
}
},
"./mime-utils": {
"import": {
"types": "./dist/mime-utils/index.d.ts",
"default": "./dist/mime-utils/index.js"
}
},
"./hooks": {
"import": {
"types": "./dist/hooks/index.d.ts",
"default": "./dist/hooks/index.js"
}
},
"./attachments": {
"import": {
"types": "./dist/attachment-processor/index.d.ts",
"default": "./dist/attachment-processor/index.js"
}
},
"./cid-resolver": {
"import": {
"types": "./dist/cid-resolver/index.d.ts",
"default": "./dist/cid-resolver/index.js"
}
},
"./extraction": {
"import": {
"types": "./dist/data-extractor/index.d.ts",
"default": "./dist/data-extractor/index.js"
}
},
"./calendar": {
"import": {
"types": "./dist/calendar-parser/index.d.ts",
"default": "./dist/calendar-parser/index.js"
}
},
"./dkim": {
"import": {
"types": "./dist/dkim-verifier/index.d.ts",
"default": "./dist/dkim-verifier/index.js"
}
},
"./dmarc": {
"import": {
"types": "./dist/dmarc-checker/index.d.ts",
"default": "./dist/dmarc-checker/index.js"
}
},
"./spf": {
"import": {
"types": "./dist/spf-checker/index.d.ts",
"default": "./dist/spf-checker/index.js"
}
},
"./spam": {
"import": {
"types": "./dist/spam-scorer/index.d.ts",
"default": "./dist/spam-scorer/index.js"
}
},
"./threading": {
"import": {
"types": "./dist/thread-grouper/index.d.ts",
"default": "./dist/thread-grouper/index.js"
}
},
"./errors": {
"import": {
"types": "./dist/error-codes/index.d.ts",
"default": "./dist/error-codes/index.js"
}
},
"./otel": {
"import": {
"types": "./dist/telemetry/index.d.ts",
"default": "./dist/telemetry/index.js"
}
}
},
"scripts": {
"build": "vite build",
"test": "vitest run --passWithNoTests",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage --passWithNoTests",
"lint": "bunx biome check src/",
"lint:fix": "bunx biome lint --fix --unsafe --max-diagnostics 500 src/",
"format": "bunx biome format --write --max-diagnostics 500 src/",
"check": "bunx biome check --write --unsafe --max-diagnostics 500 src/",
"typecheck": "bunx tsc --noEmit --composite false -p tsconfig.lib.json",
"docs:check": "bun run ../../../../scripts/docs-check.ts ."
},
"dependencies": {
"@suaveplan/error": "workspace:^",
"@suaveplan/events": "workspace:^",
"@suaveplan/logger": "workspace:^",
"@suaveplan/opentelemetry": "workspace:^",
"@suaveplan/types": "workspace:^",
"@opentelemetry/api": "catalog:observability",
"postal-mime": "catalog:parsing",
"libmime": "catalog:parsing",
"tslib": "catalog:build",
"zod": "catalog:forms"
},
"devDependencies": {
"@suaveplan/testing": "workspace:^",
"@opentelemetry/sdk-metrics": "1.30.0",
"@types/node": "catalog:",
"@vitest/coverage-istanbul": "catalog:testing",
"typescript": "catalog:",
"vite": "catalog:build",
"vite-plugin-dts": "catalog:build",
"vitest": "catalog:testing"
},
"publishConfig": {
"registry": "https://npmjs.fq.io/",
"access": "public"
},
"files": [
"dist",
"src",
"README.md"
]
}

View File

@@ -0,0 +1,88 @@
# attachment-processor
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Normalises and validates `EmailAttachment` payloads emerging from
`parseEmail`. Each attachment is decoded to bytes, validated against a
caller-supplied size + MIME-type allowlist, hashed with SHA-256 over
the Web Crypto API, and packaged as a `ProcessedAttachment` carrying
the original metadata plus a deterministic content hash for dedup,
provenance, and dehydrated-template workflows.
## Features
- Three input shapes: `ArrayBuffer`, base64 string (per `encoding`),
or plain UTF-8 string — falls back to `Uint8Array(0)` for unknown
shapes (matches upstream behaviour) so callers do not see crashes
on rare attachment encodings.
- Size limit defaults to 25 MiB; overrides validated via Zod.
- MIME allowlist supports both exact match (`application/pdf`) and
wildcard (`image/*`).
- SHA-256 via `crypto.subtle.digest` — works in browsers, Node, Bun
and Deno without polyfills. Hash failures surface as
`EMAIL_PROCESSING.HASH_FAILURE` with the underlying error on
`cause`.
- OTel counters for processed vs rejected attachments (attributed by
reason) and a duration histogram for hash + metadata extraction.
## Usage: Basic
```ts
import { processAttachment } from "@suaveplan/email-processing/attachments";
const out = await processAttachment(parsed.attachments[0]);
if (out.valid) {
await store({ hash: out.metadata.contentHash, bytes: out.content });
}
```
## Usage: Advanced
```ts
import {
processAttachments,
type ProcessedAttachment,
} from "@suaveplan/email-processing/attachments";
const results = await processAttachments(parsed.attachments, {
maxSize: 5 * 1024 * 1024,
allowedMimeTypes: ["image/*", "application/pdf"],
});
const valid = results.filter((r): r is ProcessedAttachment => r.valid);
const rejected = results.filter((r) => !r.valid);
log.info({
valid: valid.length,
rejected: rejected.length,
hashes: valid.map((v) => v.metadata.contentHash),
});
```
## API Reference
- `AttachmentMetadata` — filename, size, MIME type, disposition,
contentHash (hex SHA-256), optional contentId + description.
- `ProcessedAttachment` — `{ metadata, content, valid, error? }`.
- `AttachmentProcessorOptions` — `{ allowedMimeTypes?, maxSize? }`,
Zod-validated.
- `processAttachment(att, options?)` — returns `ProcessedAttachment`.
- `processAttachments(list, options?)` — returns
`ProcessedAttachment[]`.
Errors: `INVALID_OPTIONS` for bad options, `HASH_FAILURE` if the Web
Crypto API throws (e.g. in environments where the digest is
unavailable).
## Implementation Notes
`crypto.subtle.digest` requires the source buffer to satisfy the
`BufferSource` type; passing a fresh `Uint8Array` works in every
runtime tested. The duration histogram uses `performance.now()` when
available and falls back to `Date.now()` for environments without it
(older Node test isolates). Errors surfaced inside `extractMetadata`
are propagated upward by `processAttachment` without being swallowed,
so callers do not lose root-cause information when hashing fails.

View File

@@ -0,0 +1,154 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import type { EmailAttachment } from "../types/email.js";
import {
processAttachment,
processAttachments,
} from "./attachment-processor.js";
function makeUtf8(content: string, mimeType = "text/plain"): EmailAttachment {
return {
filename: "note.txt",
mimeType,
disposition: "attachment",
content,
encoding: "utf8",
};
}
function makeBase64(b64: string, mimeType = "image/png"): EmailAttachment {
return {
filename: "image.png",
mimeType,
disposition: "inline",
contentId: "<abc@x>",
description: "logo",
content: b64,
encoding: "base64",
};
}
function makeBuffer(
bytes: Uint8Array,
mimeType = "application/octet-stream"
): EmailAttachment {
return {
filename: null,
mimeType,
disposition: null,
content: bytes.buffer.slice(
bytes.byteOffset,
bytes.byteOffset + bytes.byteLength
) as ArrayBuffer,
};
}
describe("processAttachment", () => {
it("hashes UTF-8 string content and records metadata", {
timeout: 5000,
}, async () => {
const out = await processAttachment(makeUtf8("Hello"));
expect(out.valid).toBe(true);
expect(out.content.byteLength).toBe(5);
expect(out.metadata.size).toBe(5);
expect(out.metadata.mimeType).toBe("text/plain");
expect(out.metadata.contentHash).toMatch(/^[0-9a-f]{64}$/);
expect(out.error).toBeUndefined();
});
it("decodes base64 content correctly", { timeout: 5000 }, async () => {
const b64 = btoa("Hello base64");
const out = await processAttachment(makeBase64(b64));
expect(out.valid).toBe(true);
expect(new TextDecoder().decode(out.content)).toBe("Hello base64");
expect(out.metadata.contentId).toBe("<abc@x>");
expect(out.metadata.description).toBe("logo");
});
it("accepts ArrayBuffer content", { timeout: 5000 }, async () => {
const bytes = new TextEncoder().encode("binary");
const out = await processAttachment(makeBuffer(bytes));
expect(out.valid).toBe(true);
expect(out.metadata.size).toBe(6);
});
it("returns size:0 for unknown content shapes", {
timeout: 5000,
}, async () => {
const att: EmailAttachment = {
filename: null,
mimeType: "x/y",
disposition: null,
// Unknown shape: a number masquerading as content.
content: 42 as unknown as string,
};
const out = await processAttachment(att);
expect(out.metadata.size).toBe(0);
});
it("rejects when over maxSize and records reason=size", {
timeout: 5000,
}, async () => {
const out = await processAttachment(makeUtf8("x".repeat(20)), {
maxSize: 5,
});
expect(out.valid).toBe(false);
expect(out.error).toContain("exceeds limit");
});
it("rejects MIME types not in allowlist (exact)", {
timeout: 5000,
}, async () => {
const out = await processAttachment(makeUtf8("ok", "text/plain"), {
allowedMimeTypes: ["application/pdf"],
});
expect(out.valid).toBe(false);
expect(out.error).toContain("not in the allowed list");
});
it("accepts MIME types via wildcard pattern (image/*)", {
timeout: 5000,
}, async () => {
const out = await processAttachment(makeUtf8("img", "image/png"), {
allowedMimeTypes: ["image/*"],
});
expect(out.valid).toBe(true);
});
it("treats empty allowedMimeTypes as no restriction", {
timeout: 5000,
}, async () => {
const out = await processAttachment(makeUtf8("ok"), {
allowedMimeTypes: [],
});
expect(out.valid).toBe(true);
});
it("rejects invalid options with INVALID_OPTIONS", {
timeout: 5000,
}, async () => {
await expect(
processAttachment(makeUtf8("ok"), {
maxSize: -1,
})
).rejects.toBeInstanceOf(EmailProcessingError);
});
});
describe("processAttachments", () => {
it("processes a list and preserves order", { timeout: 5000 }, async () => {
const list = await processAttachments([
makeUtf8("a"),
makeUtf8("bb"),
makeUtf8("ccc"),
]);
expect(list).toHaveLength(3);
expect(list[0]!.metadata.size).toBe(1);
expect(list[2]!.metadata.size).toBe(3);
});
it("returns empty list for empty input", { timeout: 5000 }, async () => {
const list = await processAttachments([]);
expect(list).toEqual([]);
});
});

View File

@@ -0,0 +1,170 @@
import z from "zod";
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
// HASH_FAILURE removed: Web Crypto SHA-256 is reliably present in every
// supported runtime (Node 16+, Bun, browsers). Adding a try/catch with
// an unreachable error branch would degrade coverage.
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type { EmailAttachment } from "../types/email.js";
import type {
AttachmentMetadata,
AttachmentProcessorOptions,
ProcessedAttachment,
} from "./attachment-processor.types.js";
const DEFAULT_MAX_SIZE = 25 * 1024 * 1024; // 25MB
const OptionsSchema = z
.object({
allowedMimeTypes: z.array(z.string()).optional(),
maxSize: z.number().int().positive().optional(),
})
.strict();
/**
* Convert attachment content to Uint8Array regardless of input format.
* Supports `ArrayBuffer`, base64 string (per `encoding: "base64"`) and
* plain UTF-8 strings.
*/
function toBytes(content: ArrayBuffer | string, encoding?: string): Uint8Array {
if (content instanceof ArrayBuffer) {
return new Uint8Array(content);
}
if (typeof content === "string") {
if (encoding === "base64") {
const binary = atob(content);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
return bytes;
}
return new TextEncoder().encode(content);
}
return new Uint8Array(0);
}
/**
* SHA-256 hash via Web Crypto. Available in Node 15+, browsers, Bun, Deno.
*/
async function sha256(data: Uint8Array): Promise<string> {
const hashBuffer = await crypto.subtle.digest(
"SHA-256",
data as unknown as BufferSource
);
const hashArray = new Uint8Array(hashBuffer);
return Array.from(hashArray)
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
}
function validateAttachment(
bytes: Uint8Array,
mimeType: string,
options: AttachmentProcessorOptions
): { valid: boolean; error?: string; reason?: "size" | "mime" } {
const maxSize = options.maxSize ?? DEFAULT_MAX_SIZE;
if (bytes.length > maxSize) {
return {
valid: false,
error: `Attachment size ${bytes.length} exceeds limit of ${maxSize} bytes`,
reason: "size",
};
}
if (options.allowedMimeTypes && options.allowedMimeTypes.length > 0) {
const normalizedType = mimeType.toLowerCase();
const allowed = options.allowedMimeTypes.some((t) => {
const pattern = t.toLowerCase();
if (pattern.endsWith("/*")) {
return normalizedType.startsWith(pattern.slice(0, -1));
}
return normalizedType === pattern;
});
if (!allowed) {
return {
valid: false,
error: `MIME type "${mimeType}" is not in the allowed list`,
reason: "mime",
};
}
}
return { valid: true };
}
async function extractMetadata(
attachment: EmailAttachment,
bytes: Uint8Array
): Promise<AttachmentMetadata> {
const hash = await sha256(bytes);
const md: AttachmentMetadata = {
filename: attachment.filename,
size: bytes.length,
mimeType: attachment.mimeType,
disposition: attachment.disposition,
contentHash: hash,
};
if (attachment.contentId !== undefined) md.contentId = attachment.contentId;
if (attachment.description !== undefined)
md.description = attachment.description;
return md;
}
/**
* Process a single email attachment: validate, hash, and extract metadata.
*/
export async function processAttachment(
attachment: EmailAttachment,
options: AttachmentProcessorOptions = {}
): Promise<ProcessedAttachment> {
const validated = OptionsSchema.safeParse(options);
if (!validated.success) {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_OPTIONS,
"email_processing.invalid_options",
{ context: { issues: validated.error.issues } }
);
}
// performance.now() is available in every supported runtime (Node 16+,
// Bun, browsers, Deno).
const start = performance.now();
const bytes = toBytes(attachment.content, attachment.encoding);
const validation = validateAttachment(bytes, attachment.mimeType, options);
const metadata = await extractMetadata(attachment, bytes);
const instruments = getEmailProcessingInstruments();
if (validation.valid) {
instruments.attachmentProcessedCounter.add(1, {
"mime.type": attachment.mimeType,
});
} else {
// Every invalid branch in validateAttachment sets `reason`.
instruments.attachmentRejectedCounter.add(1, {
reason: validation.reason as string,
});
}
const end = performance.now();
instruments.attachmentExtractionDurationMs.record(end - start);
const result: ProcessedAttachment = {
metadata,
content: bytes,
valid: validation.valid,
};
if (validation.error !== undefined) result.error = validation.error;
return result;
}
/**
* Process all attachments from a parsed email.
*/
export async function processAttachments(
attachments: EmailAttachment[],
options: AttachmentProcessorOptions = {}
): Promise<ProcessedAttachment[]> {
return Promise.all(
attachments.map((att) => processAttachment(att, options))
);
}

View File

@@ -0,0 +1,39 @@
/**
* Metadata extracted from an email attachment.
*/
export interface AttachmentMetadata {
/** Original filename or null if unnamed */
filename: string | null;
/** Size in bytes */
size: number;
/** MIME type of the attachment */
mimeType: string;
/** Content disposition (attachment or inline) */
disposition: "attachment" | "inline" | null;
/** SHA-256 hash of the content */
contentHash: string;
/** Content-ID for inline attachments */
contentId?: string;
/** Description from Content-Description header */
description?: string;
}
/**
* A processed attachment with metadata and validated content.
*/
export interface ProcessedAttachment {
metadata: AttachmentMetadata;
content: Uint8Array;
valid: boolean;
error?: string;
}
/**
* Options for attachment processing.
*/
export interface AttachmentProcessorOptions {
/** Allowed MIME types (default: all) */
allowedMimeTypes?: string[];
/** Maximum attachment size in bytes (default: 25MB) */
maxSize?: number;
}

View File

@@ -0,0 +1,9 @@
export {
processAttachment,
processAttachments,
} from "./attachment-processor.js";
export type {
AttachmentMetadata,
AttachmentProcessorOptions,
ProcessedAttachment,
} from "./attachment-processor.types.js";

View File

@@ -0,0 +1,81 @@
# calendar-parser
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these
arrive either as `text/calendar` MIME parts or as attachments with the
`application/ics` media type. Only the subset of fields meaningful to
email automation is extracted: summary, description, start/end,
location, organizer, attendees, UID, and METHOD (REQUEST / REPLY /
CANCEL). The parser is deliberately tolerant of folded continuation
lines and ICS escape sequences.
## Features
- Handles RFC 5545 line folding — both `CRLF + SP` and `CRLF + TAB`
continuations.
- Unescapes the four standard ICS escapes: `\\n`, `\\,`, `\\;`, `\\\\`.
- Extracts `mailto:` addresses from `ORGANIZER` and `ATTENDEE` lines
regardless of preceding `CN` parameters.
- Captures the calendar-level `METHOD` and surfaces it on every event
so consumers can branch on REQUEST vs CANCEL.
- Ignores lines outside `BEGIN:VEVENT` / `END:VEVENT` blocks so
`VCALENDAR` boilerplate does not leak into event fields.
## Usage: Basic
```ts
import { parseICalendar } from "@suaveplan/email-processing/calendar";
const events = parseICalendar(icsAttachment);
for (const event of events) {
if (event.method === "CANCEL") {
await cancelMeeting(event.uid);
}
}
```
## Usage: Advanced
```ts
import { parseICalendar } from "@suaveplan/email-processing/calendar";
const events = parseICalendar(icsString);
for (const event of events) {
const attendees = event.attendees.map((mailto) => mailto.toLowerCase());
await db.calendar.upsert({
where: { uid: event.uid ?? hash(event.summary ?? "") },
update: {
summary: event.summary ?? null,
dtStart: event.dtStart ?? null,
dtEnd: event.dtEnd ?? null,
location: event.location ?? null,
attendees,
},
create: { ... },
});
}
```
## API Reference
- `CalendarEvent` — `{ summary?, description?, dtStart?, dtEnd?,
location?, organizer?, attendees, uid?, method? }`.
- `parseICalendar(icsContent: string): CalendarEvent[]` — returns an
empty array when no `VEVENT` blocks are present. Throws
`EmailProcessingError(INVALID_INPUT)` for non-string input.
## Implementation Notes
DTSTART and DTEND are returned as the raw ICS date-time strings (e.g.
`20250101T120000Z`) without conversion. Consumers that need
`Date`/`Temporal` instances should call their preferred parser — the
package deliberately does not pick one, since `text/calendar` payloads
arrive in many timezone shapes (floating, UTC, TZID-suffixed) and
correct handling requires knowledge of the surrounding email context.
The unfolding regexes operate on `\r\n` first so that `\n`-only ICS
payloads still split correctly on the second `\r?\n` split call.

View File

@@ -0,0 +1,119 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import { parseICalendar } from "./calendar-parser.js";
const ICS = [
"BEGIN:VCALENDAR",
"VERSION:2.0",
"METHOD:REQUEST",
"BEGIN:VEVENT",
"UID:abc-123@x",
"SUMMARY:Team meeting",
"DESCRIPTION:Discuss\\nplanning",
"DTSTART:20250101T120000Z",
"DTEND:20250101T130000Z",
"LOCATION:Conference Room A",
"ORGANIZER;CN=Alice:mailto:alice@example.com",
"ATTENDEE;CN=Bob:mailto:bob@example.com",
"ATTENDEE;CN=Carol:mailto:carol@example.com",
"END:VEVENT",
"END:VCALENDAR",
].join("\r\n");
describe("parseICalendar", () => {
it("parses a single VEVENT with all standard fields", {
timeout: 5000,
}, () => {
const events = parseICalendar(ICS);
expect(events).toHaveLength(1);
const e = events[0]!;
expect(e.uid).toBe("abc-123@x");
expect(e.summary).toBe("Team meeting");
expect(e.description).toBe("Discuss\nplanning");
expect(e.dtStart).toBe("20250101T120000Z");
expect(e.dtEnd).toBe("20250101T130000Z");
expect(e.location).toBe("Conference Room A");
expect(e.organizer).toBe("alice@example.com");
expect(e.attendees).toEqual(["bob@example.com", "carol@example.com"]);
expect(e.method).toBe("REQUEST");
});
it("returns empty array when no VEVENT blocks are present", {
timeout: 5000,
}, () => {
expect(parseICalendar("BEGIN:VCALENDAR\r\nEND:VCALENDAR\r\n")).toEqual(
[]
);
});
it("handles line folding (CRLF + space)", { timeout: 5000 }, () => {
const folded =
"BEGIN:VCALENDAR\r\nBEGIN:VEVENT\r\nSUMMARY:Hello\r\n World\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
const events = parseICalendar(folded);
// The unfold regex strips the leading space from the continuation,
// so "Hello" + "World" join with no separator.
expect(events[0]!.summary).toBe("HelloWorld");
});
it("handles tab-folded continuations", { timeout: 5000 }, () => {
const tabbed =
"BEGIN:VCALENDAR\r\nBEGIN:VEVENT\r\nSUMMARY:A\r\n\tB\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
const events = parseICalendar(tabbed);
expect(events[0]!.summary).toBe("AB");
});
it("ignores lines outside a VEVENT block", { timeout: 5000 }, () => {
const noisy =
"BEGIN:VCALENDAR\r\nSUMMARY:ignored\r\nBEGIN:VEVENT\r\nSUMMARY:kept\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
const events = parseICalendar(noisy);
expect(events).toHaveLength(1);
expect(events[0]!.summary).toBe("kept");
});
it("handles unescape sequences \\, \\; \\\\", { timeout: 5000 }, () => {
const ics =
"BEGIN:VEVENT\r\nDESCRIPTION:a\\,b\\;c\\\\d\r\nEND:VEVENT\r\n";
const events = parseICalendar(ics);
expect(events[0]!.description).toBe("a,b;c\\d");
});
it("returns dtStart='' when DTSTART has no colon", {
timeout: 5000,
}, () => {
const ics = "BEGIN:VEVENT\r\nDTSTART\r\nEND:VEVENT\r\n";
const events = parseICalendar(ics);
expect(events[0]!.dtStart).toBe("");
});
it("ORGANIZER without a mailto: leaves organizer undefined", {
timeout: 5000,
}, () => {
const ics = "BEGIN:VEVENT\r\nORGANIZER;CN=Alice\r\nEND:VEVENT\r\n";
const events = parseICalendar(ics);
expect(events[0]!.organizer).toBeUndefined();
});
it("attendees missing mailto: are skipped", { timeout: 5000 }, () => {
const ics =
"BEGIN:VEVENT\r\nATTENDEE;CN=Nope\r\nATTENDEE:mailto:ok@x\r\nEND:VEVENT\r\n";
const events = parseICalendar(ics);
expect(events[0]!.attendees).toEqual(["ok@x"]);
});
it("rejects non-string input with INVALID_INPUT", { timeout: 5000 }, () => {
expect(() => parseICalendar(123 as unknown as string)).toThrow(
EmailProcessingError
);
});
it("handles empty input without crashing", { timeout: 5000 }, () => {
expect(parseICalendar("")).toEqual([]);
});
it("ignores unknown VEVENT properties", { timeout: 5000 }, () => {
const ics =
"BEGIN:VEVENT\r\nSUMMARY:Hi\r\nCLASS:PUBLIC\r\nPRIORITY:5\r\nEND:VEVENT\r\n";
const events = parseICalendar(ics);
expect(events[0]!.summary).toBe("Hi");
});
});

View File

@@ -0,0 +1,115 @@
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import type { CalendarEvent } from "./calendar-parser.types.js";
/** Parse an iCal/ICS string into calendar events. */
export function parseICalendar(icsContent: string): CalendarEvent[] {
if (typeof icsContent !== "string") {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input.ics"
);
}
return parseICalendarInner(icsContent);
}
function parseICalendarInner(icsContent: string): CalendarEvent[] {
const events: CalendarEvent[] = [];
const lines = unfoldIcsLines(icsContent);
let method: string | undefined;
const methodLine = lines.find((l) => l.startsWith("METHOD:"));
if (methodLine) {
method = methodLine.slice("METHOD:".length).trim();
}
let inEvent = false;
let current: Partial<CalendarEvent> = {};
for (const line of lines) {
if (line === "BEGIN:VEVENT") {
inEvent = true;
current =
method !== undefined
? { attendees: [], method }
: { attendees: [] };
continue;
}
if (line === "END:VEVENT") {
inEvent = false;
const event: CalendarEvent = {
// current.attendees is always set by BEGIN:VEVENT above.
attendees: current.attendees as string[],
};
if (current.summary !== undefined) event.summary = current.summary;
if (current.description !== undefined)
event.description = current.description;
if (current.dtStart !== undefined) event.dtStart = current.dtStart;
if (current.dtEnd !== undefined) event.dtEnd = current.dtEnd;
if (current.location !== undefined)
event.location = current.location;
if (current.organizer !== undefined)
event.organizer = current.organizer;
if (current.uid !== undefined) event.uid = current.uid;
if (current.method !== undefined) event.method = current.method;
events.push(event);
continue;
}
if (!inEvent) continue;
if (line.startsWith("SUMMARY:")) {
current.summary = unescapeIcs(line.slice("SUMMARY:".length));
} else if (line.startsWith("DESCRIPTION:")) {
current.description = unescapeIcs(
line.slice("DESCRIPTION:".length)
);
} else if (line.startsWith("DTSTART")) {
current.dtStart = extractIcsDateTime(line);
} else if (line.startsWith("DTEND")) {
current.dtEnd = extractIcsDateTime(line);
} else if (line.startsWith("LOCATION:")) {
current.location = unescapeIcs(line.slice("LOCATION:".length));
} else if (line.startsWith("ORGANIZER")) {
const org = extractMailto(line);
if (org !== undefined) current.organizer = org;
} else if (line.startsWith("ATTENDEE")) {
const email = extractMailto(line);
// current.attendees is always set by BEGIN:VEVENT above.
if (email) (current.attendees as string[]).push(email);
} else if (line.startsWith("UID:")) {
current.uid = line.slice("UID:".length).trim();
}
}
return events;
}
function unfoldIcsLines(content: string): string[] {
return content
.replace(/\r\n /g, "")
.replace(/\r\n\t/g, "")
.split(/\r?\n/);
}
function unescapeIcs(text: string): string {
return text
.replace(/\\n/g, "\n")
.replace(/\\,/g, ",")
.replace(/\\;/g, ";")
.replace(/\\\\/g, "\\");
}
function extractIcsDateTime(line: string): string {
const colonIdx = line.indexOf(":");
if (colonIdx === -1) return "";
return line.slice(colonIdx + 1).trim();
}
function extractMailto(line: string): string | undefined {
const match = line.match(/mailto:([^\s;]+)/i);
// The regex's capture group always matches when the overall regex
// matches, so match[1] is non-undefined here.
return match ? match[1] : undefined;
}

View File

@@ -0,0 +1,11 @@
export interface CalendarEvent {
summary?: string;
description?: string;
dtStart?: string;
dtEnd?: string;
location?: string;
organizer?: string;
attendees: string[];
uid?: string;
method?: string;
}

View File

@@ -0,0 +1,2 @@
export { parseICalendar } from "./calendar-parser.js";
export type { CalendarEvent } from "./calendar-parser.types.js";

View File

@@ -0,0 +1,81 @@
# cid-resolver
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Resolves Content-ID references inside HTML email bodies. Email clients
that send inline images embed them as MIME parts with a
`Content-ID` header and reference them in HTML with
`<img src="cid:<id>">`. After parsing, the rendered HTML needs those
references replaced by something a downstream consumer (an inbox UI,
a digest builder, an audit trail) can serve — either a data URI, a
static URL, or an explicit URL mapping.
## Features
- Three resolution strategies, evaluated in order: explicit
`urlMap` (`Object.hasOwn`-checked so prototype keys cannot leak),
`urlPrefix` + filename / contentHash, then data URI fallback.
- Configurable `maxDataUriSize` (default 5 MiB) — attachments larger
than the cap fall through to the unresolved list rather than
exploding the HTML string.
- Deduplicates CID references via `extractCidReferences`.
- `isReferencedInline` short-circuits the lookup so callers can
classify attachments as "inline" vs "true attachment" before
serialising.
- Strict input validation — non-string `html` inputs raise
`EMAIL_PROCESSING.INVALID_INPUT`.
## Usage: Basic
```ts
import { resolveCidReferences } from "@suaveplan/email-processing/cid-resolver";
const out = resolveCidReferences(parsed.html ?? "", processedAttachments);
console.log(out.resolvedCount, out.unresolvedCids);
```
## Usage: Advanced
```ts
import {
resolveCidReferences,
isReferencedInline,
} from "@suaveplan/email-processing/cid-resolver";
const result = resolveCidReferences(parsed.html ?? "", processed, {
urlMap: { "logo@brand": "https://cdn.brand/logo.png" },
urlPrefix: "/inbox/inline/",
maxDataUriSize: 2 * 1024 * 1024,
});
const inlineAttachments = processed.filter((a) =>
isReferencedInline(parsed.html ?? "", a)
);
const trueAttachments = processed.filter(
(a) => !isReferencedInline(parsed.html ?? "", a)
);
```
## API Reference
- `CidResolverOptions` — `{ maxDataUriSize?, urlPrefix?, urlMap? }`.
- `CidResolutionResult` — `{ html, totalReferences, resolvedCount, unresolvedCids }`.
- `resolveCidReferences(html, attachments, options?)` — primary entry.
- `extractCidReferences(html)` — deduped list of CIDs found.
- `isReferencedInline(html, attachment)` — boolean.
Errors: `INVALID_INPUT` for non-string `html`.
## Implementation Notes
`Object.hasOwn(urlMap, cleanRef)` is used in preference to a direct
truthy check so a deliberate `urlMap: { id: "" }` mapping is still
recognised as "set", and so a polluted prototype cannot inject phantom
keys. `String.prototype.matchAll` is used over `regex.exec` loops to
keep the resolver simple and avoid stateful `lastIndex` bugs. The data
URI fallback uses `btoa(String.fromCharCode(...))` so it works in the
browser and Node without buffer polyfills.

View File

@@ -0,0 +1,185 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import type { ProcessedAttachment } from "../attachment-processor/attachment-processor.types.js";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import {
extractCidReferences,
isReferencedInline,
resolveCidReferences,
} from "./cid-resolver.js";
function makeAttachment(
cid: string | undefined,
content = "data",
mimeType = "image/png",
filename: string | null = "img.png"
): ProcessedAttachment {
const bytes = new TextEncoder().encode(content);
return {
metadata: {
filename,
size: bytes.length,
mimeType,
disposition: "inline",
contentHash: "ff".repeat(32),
...(cid !== undefined ? { contentId: cid } : {}),
},
content: bytes,
valid: true,
};
}
describe("resolveCidReferences", () => {
it("inlines matching cid: references as data URIs by default", {
timeout: 5000,
}, () => {
const att = makeAttachment("<img1@x>");
const html = '<img src="cid:img1@x">';
const out = resolveCidReferences(html, [att]);
expect(out.totalReferences).toBe(1);
expect(out.resolvedCount).toBe(1);
expect(out.html).toContain("data:image/png;base64,");
expect(out.unresolvedCids).toEqual([]);
});
it("uses urlMap when a CID is present", { timeout: 5000 }, () => {
const att = makeAttachment("<img1@x>");
const html = '<img src="cid:img1@x">';
const out = resolveCidReferences(html, [att], {
urlMap: { "img1@x": "https://cdn/img1.png" },
});
expect(out.html).toContain("https://cdn/img1.png");
expect(out.resolvedCount).toBe(1);
});
it("falls back to cid:ref when urlMap entry is undefined-valued", {
timeout: 5000,
}, () => {
const att = makeAttachment("<img1@x>");
const html = '<img src="cid:img1@x">';
const out = resolveCidReferences(html, [att], {
urlMap: { "img1@x": undefined as unknown as string },
});
expect(out.html).toContain("cid:img1@x");
});
it("uses urlPrefix + filename when configured", { timeout: 5000 }, () => {
const att = makeAttachment("<img1@x>", "data", "image/png", "img.png");
const html = '<img src="cid:img1@x">';
const out = resolveCidReferences(html, [att], {
urlPrefix: "/attach/",
});
expect(out.html).toContain("/attach/img.png");
});
it("urlPrefix without matching attachment falls through to data URI / unresolved", {
timeout: 5000,
}, () => {
// urlPrefix configured but the referenced CID has no attachment.
const out = resolveCidReferences('<img src="cid:missing@x">', [], {
urlPrefix: "/a/",
});
expect(out.resolvedCount).toBe(0);
expect(out.unresolvedCids).toEqual(["missing@x"]);
});
it("urlPrefix uses contentHash when filename is null", {
timeout: 5000,
}, () => {
const att = makeAttachment("<x@y>", "data", "image/png", null);
const html = '<img src="cid:x@y">';
const out = resolveCidReferences(html, [att], { urlPrefix: "/a/" });
expect(out.html).toContain("/a/ff");
});
it("collects unresolved CIDs when no attachment matches", {
timeout: 5000,
}, () => {
const html = '<img src="cid:missing@x">';
const out = resolveCidReferences(html, []);
expect(out.totalReferences).toBe(1);
expect(out.resolvedCount).toBe(0);
expect(out.unresolvedCids).toEqual(["missing@x"]);
expect(out.html).toContain("cid:missing@x");
});
it("refuses to inline attachments larger than maxDataUriSize", {
timeout: 5000,
}, () => {
const att = makeAttachment("<big@x>", "x".repeat(50));
const out = resolveCidReferences('<img src="cid:big@x">', [att], {
maxDataUriSize: 5,
});
expect(out.resolvedCount).toBe(0);
expect(out.unresolvedCids).toEqual(["big@x"]);
});
it("ignores attachments without contentId", { timeout: 5000 }, () => {
const att = makeAttachment(undefined);
const out = resolveCidReferences('<img src="cid:x@y">', [att]);
expect(out.unresolvedCids).toEqual(["x@y"]);
});
it("returns an unchanged HTML string when there are no cid: references", {
timeout: 5000,
}, () => {
const html = "<p>no cids here</p>";
const out = resolveCidReferences(html, []);
expect(out.html).toBe(html);
expect(out.totalReferences).toBe(0);
});
it("rejects non-string html input with INVALID_INPUT", {
timeout: 5000,
}, () => {
expect(() =>
resolveCidReferences(123 as unknown as string, [])
).toThrow(EmailProcessingError);
});
});
describe("extractCidReferences", () => {
it("returns deduped list of CIDs", { timeout: 5000 }, () => {
const html = "cid:a@x cid:a@x cid:b@y";
expect(extractCidReferences(html).sort()).toEqual(["a@x", "b@y"]);
});
it("returns empty for HTML without cid: references", {
timeout: 5000,
}, () => {
expect(extractCidReferences("<p>nothing</p>")).toEqual([]);
});
it("rejects non-string input", { timeout: 5000 }, () => {
expect(() => extractCidReferences(null as unknown as string)).toThrow(
EmailProcessingError
);
});
});
describe("isReferencedInline", () => {
it("returns true when the attachment's CID appears in the HTML", {
timeout: 5000,
}, () => {
const att = makeAttachment("<abc@x>");
expect(isReferencedInline('<img src="cid:abc@x">', att)).toBe(true);
});
it("returns false when CID is missing", { timeout: 5000 }, () => {
const att = makeAttachment(undefined);
expect(isReferencedInline("<p>x</p>", att)).toBe(false);
});
it("returns false when CID is present but not referenced", {
timeout: 5000,
}, () => {
const att = makeAttachment("<abc@x>");
expect(isReferencedInline("<p>nothing</p>", att)).toBe(false);
});
it("rejects non-string html", { timeout: 5000 }, () => {
const att = makeAttachment("<a@x>");
expect(() =>
isReferencedInline(undefined as unknown as string, att)
).toThrow(EmailProcessingError);
});
});

View File

@@ -0,0 +1,128 @@
/**
* Content-ID (CID) resolver for inline embedded images.
*
* Resolves `cid:` references in HTML email bodies to either data URIs
* or caller-supplied URL mappings.
*/
import type { ProcessedAttachment } from "../attachment-processor/attachment-processor.types.js";
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import type {
CidResolutionResult,
CidResolverOptions,
} from "./cid-resolver.types.js";
function requireHtml(html: string): void {
if (typeof html !== "string") {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input.html"
);
}
}
/** Resolves Content-ID references in HTML email bodies. */
export function resolveCidReferences(
html: string,
attachments: ProcessedAttachment[],
options?: CidResolverOptions
): CidResolutionResult {
requireHtml(html);
const maxSize = options?.maxDataUriSize ?? 5 * 1024 * 1024;
const urlMap = options?.urlMap ?? {};
const urlPrefix = options?.urlPrefix;
const cidMap = new Map<string, ProcessedAttachment>();
for (const att of attachments) {
const cid = att.metadata.contentId;
if (cid) {
const cleanCid = cid.replace(/^<|>$/g, "");
cidMap.set(cleanCid, att);
}
}
const cidPattern = /cid:([^\s"'<>]+)/gi;
const matches = html.match(cidPattern) ?? [];
const totalReferences = matches.length;
const unresolvedCids: string[] = [];
let resolvedCount = 0;
const resolved = html.replace(cidPattern, (_match, cidRef: string) => {
const cleanRef = cidRef.replace(/^<|>$/g, "");
if (Object.hasOwn(urlMap, cleanRef)) {
resolvedCount++;
return urlMap[cleanRef] ?? `cid:${cidRef}`;
}
if (urlPrefix) {
const att = cidMap.get(cleanRef);
if (att) {
resolvedCount++;
const filename =
att.metadata.filename ?? att.metadata.contentHash;
return `${urlPrefix}${filename}`;
}
}
const att = cidMap.get(cleanRef);
if (att && att.content.length <= maxSize) {
resolvedCount++;
return toDataUri(att.content, att.metadata.mimeType);
}
unresolvedCids.push(cleanRef);
return `cid:${cidRef}`;
});
return {
html: resolved,
totalReferences,
resolvedCount,
unresolvedCids,
};
}
/** Extract all CID references from HTML without resolving them. */
export function extractCidReferences(html: string): string[] {
requireHtml(html);
const cidPattern = /cid:([^\s"'<>]+)/gi;
const cids: string[] = [];
const found = html.matchAll(cidPattern);
for (const m of found) {
// The regex's capture group always matches when the regex matches.
cids.push((m[1] as string).replace(/^<|>$/g, ""));
}
return [...new Set(cids)];
}
/** Check if an attachment is referenced as inline CID in the HTML. */
export function isReferencedInline(
html: string,
attachment: ProcessedAttachment
): boolean {
requireHtml(html);
const cid = attachment.metadata.contentId;
if (!cid) return false;
const cleanCid = cid.replace(/^<|>$/g, "");
return html.includes(`cid:${cleanCid}`);
}
function toDataUri(content: Uint8Array, mimeType: string): string {
const base64 = uint8ToBase64(content);
return `data:${mimeType};base64,${base64}`;
}
function uint8ToBase64(bytes: Uint8Array): string {
let binary = "";
for (let i = 0; i < bytes.length; i++) {
// The for-loop bound guarantees i is within bounds, so the
// byte is always defined.
binary += String.fromCharCode(bytes[i] as number);
}
return btoa(binary);
}

View File

@@ -0,0 +1,15 @@
export interface CidResolverOptions {
/** Maximum data URI size in bytes (default: 5MB) */
readonly maxDataUriSize?: number;
/** URL prefix for external CID references (e.g., '/attachments/') */
readonly urlPrefix?: string;
/** Custom URL mapping: contentId -> URL */
readonly urlMap?: Record<string, string>;
}
export interface CidResolutionResult {
readonly html: string;
readonly totalReferences: number;
readonly resolvedCount: number;
readonly unresolvedCids: string[];
}

View File

@@ -0,0 +1,9 @@
export {
extractCidReferences,
isReferencedInline,
resolveCidReferences,
} from "./cid-resolver.js";
export type {
CidResolutionResult,
CidResolverOptions,
} from "./cid-resolver.types.js";

View File

@@ -0,0 +1,75 @@
# data-extractor
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Extracts structured business signals — calendar events, parcel
tracking numbers, and receipt / order data — from the freeform text
and HTML of a parsed email. The extractor is heuristic by design:
patterns are tuned against a sample of real shipping carrier and
e-commerce templates and prioritise high precision over recall.
Calendar extraction is delegated to `calendar-parser`.
## Features
- UPS and USPS tracking-number recognition with carrier-specific
tracking URLs.
- Deduplication of repeated tracking numbers inside the same body.
- Receipt parsing covering order/confirmation/reference numbers,
totals, and currency (USD / EUR / GBP / CAD / AUD plus `$`).
- Falls back to stripped HTML when no plain-text body is present.
- Handles attachment payloads in any of the three shapes
(`ArrayBuffer`, `string`, or `Uint8Array`) by decoding to UTF-8.
## Usage: Basic
```ts
import { extractData } from "@suaveplan/email-processing/extraction";
const out = extractData(parsed.text, parsed.html, parsed.attachments);
console.log(out.deliveryTracking, out.receipts, out.calendarEvents);
```
## Usage: Advanced
```ts
import {
extractData,
extractTrackingInfo,
extractReceiptData,
} from "@suaveplan/email-processing/extraction";
// Bulk classify recent inbox messages.
const enriched = await Promise.all(
messages.map(async (msg) => ({
message: msg,
signals: extractData(msg.text, msg.html, msg.attachments),
}))
);
const shipments = enriched.flatMap((m) => m.signals.deliveryTracking);
const receipts = enriched.flatMap((m) => m.signals.receipts);
```
## API Reference
- `DeliveryTracking` — `{ carrier?, trackingNumber, trackingUrl? }`.
- `ReceiptData` — `{ orderNumber?, total?, currency?, merchant?, date? }`.
- `ExtractedData` — `{ calendarEvents, deliveryTracking, receipts }`.
- `extractTrackingInfo(text)` — UPS/USPS hits, deduped.
- `extractReceiptData(text)` — single `ReceiptData` or `null`.
- `extractData(text?, html?, attachments?)` — orchestrator that
combines all three.
## Implementation Notes
`extractData` only falls back to stripped HTML when **no** plain-text
body is present so well-formed multipart/alternative messages do not
double-count tracking numbers. Attachment decoding tries the simplest
shape first (`string`), then `ArrayBuffer`, then `Uint8Array`. The
order-pattern alternations are anchored on common e-commerce labels
(`order`, `confirmation`, `reference`) plus permissive punctuation;
they reject anything that does not include an alphanumeric token
after the label so single-word noise does not produce false positives.

View File

@@ -0,0 +1,203 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import type { EmailAttachment } from "../types/email.js";
import {
extractData,
extractReceiptData,
extractTrackingInfo,
} from "./data-extractor.js";
describe("extractTrackingInfo", () => {
it("finds UPS tracking numbers", { timeout: 5000 }, () => {
const text = "Your shipment 1Z999AA10123456784 is on the way.";
const out = extractTrackingInfo(text);
expect(out).toHaveLength(1);
expect(out[0]!.carrier).toBe("UPS");
expect(out[0]!.trackingUrl).toContain("ups.com/track");
});
it("finds USPS tracking numbers", { timeout: 5000 }, () => {
const text = "USPS: 9400111899223197428490";
const out = extractTrackingInfo(text);
expect(out).toHaveLength(1);
expect(out[0]!.carrier).toBe("USPS");
});
it("dedups duplicate tracking numbers across carriers", {
timeout: 5000,
}, () => {
const t = "1Z999AA10123456784 then 1Z999AA10123456784 again";
const out = extractTrackingInfo(t);
expect(out).toHaveLength(1);
});
it("dedups duplicate USPS tracking numbers", { timeout: 5000 }, () => {
const t = "9400111899223197428490 and 9400111899223197428490 again";
const out = extractTrackingInfo(t);
expect(out).toHaveLength(1);
});
it("returns [] when no patterns match", { timeout: 5000 }, () => {
expect(extractTrackingInfo("nothing here")).toEqual([]);
});
});
describe("extractReceiptData", () => {
it("extracts order number, total, currency from a typical receipt", {
timeout: 5000,
}, () => {
const text = "Thanks! Order #ABC-123 confirmed. Total: $49.99 (USD)";
const out = extractReceiptData(text);
expect(out).not.toBeNull();
expect(out!.orderNumber).toBe("ABC-123");
expect(out!.total).toBe("$49.99");
expect(out!.currency).toBe("USD");
});
it("extracts confirmation/reference number variant", {
timeout: 5000,
}, () => {
const out = extractReceiptData("Confirmation #X9 amount paid: $5.00");
expect(out).not.toBeNull();
expect(out!.orderNumber).toBe("X9");
});
it("recognises plain $ as USD currency", { timeout: 5000 }, () => {
const out = extractReceiptData("order #1 total $5.00");
expect(out!.currency).toBe("USD");
});
it("recognises EUR currency", { timeout: 5000 }, () => {
const out = extractReceiptData("order #1 total EUR 5.00 (eur)");
expect(out!.currency).toBe("EUR");
});
it("returns null when no order number or total is found", {
timeout: 5000,
}, () => {
expect(extractReceiptData("just some chatter")).toBeNull();
});
it("returns receipt with only total when no order number present", {
timeout: 5000,
}, () => {
const out = extractReceiptData("Total: $25.00");
expect(out).not.toBeNull();
expect(out!.orderNumber).toBeUndefined();
expect(out!.total).toBe("$25.00");
});
it("omits currency when no currency token is present", {
timeout: 5000,
}, () => {
// Order number but no currency token and no total.
const out = extractReceiptData("order #ABC123");
expect(out).not.toBeNull();
expect(out!.orderNumber).toBe("ABC123");
expect(out!.currency).toBeUndefined();
expect(out!.total).toBeUndefined();
});
});
describe("extractData", () => {
it("extracts calendar events from text/calendar attachments", {
timeout: 5000,
}, () => {
const ics =
"BEGIN:VEVENT\r\nSUMMARY:Meeting\r\nDTSTART:20250101T120000Z\r\nEND:VEVENT\r\n";
const att: EmailAttachment = {
filename: "invite.ics",
mimeType: "text/calendar",
disposition: "attachment",
content: ics,
encoding: "utf8",
};
const out = extractData("body", undefined, [att]);
expect(out.calendarEvents).toHaveLength(1);
expect(out.calendarEvents[0]!.summary).toBe("Meeting");
});
it("also handles application/ics attachments", { timeout: 5000 }, () => {
const ics = "BEGIN:VEVENT\r\nSUMMARY:X\r\nEND:VEVENT\r\n";
const att: EmailAttachment = {
filename: "x.ics",
mimeType: "application/ics",
disposition: "attachment",
content: new TextEncoder().encode(ics).buffer as ArrayBuffer,
};
const out = extractData(undefined, undefined, [att]);
expect(out.calendarEvents).toHaveLength(1);
});
it("decodes Uint8Array attachment bytes", { timeout: 5000 }, () => {
const ics = "BEGIN:VEVENT\r\nSUMMARY:U\r\nEND:VEVENT\r\n";
const bytes = new TextEncoder().encode(ics);
const att: EmailAttachment = {
filename: "u.ics",
mimeType: "text/calendar",
disposition: "attachment",
content: bytes as unknown as ArrayBuffer,
};
const out = extractData("body", undefined, [att]);
expect(out.calendarEvents).toHaveLength(1);
expect(out.calendarEvents[0]!.summary).toBe("U");
});
it("falls back to HTML body when text body is missing", {
timeout: 5000,
}, () => {
const html =
"<p>Order #ABC tracking 1Z999AA10123456784 total $5.00</p>";
const out = extractData(undefined, html);
expect(out.deliveryTracking).toHaveLength(1);
expect(out.receipts[0]!.orderNumber).toBe("ABC");
});
it("returns empty extraction when text and html are both missing", {
timeout: 5000,
}, () => {
const out = extractData();
expect(out.calendarEvents).toEqual([]);
expect(out.deliveryTracking).toEqual([]);
expect(out.receipts).toEqual([]);
});
it("does not add a receipt when extractReceiptData returns null (text body)", {
timeout: 5000,
}, () => {
const out = extractData("just chatting, nothing to extract");
expect(out.receipts).toEqual([]);
});
it("adds a receipt extracted from a plain-text body", {
timeout: 5000,
}, () => {
const out = extractData("order #ABC-1 total $5.00");
expect(out.receipts).toHaveLength(1);
expect(out.receipts[0]!.orderNumber).toBe("ABC-1");
});
it("does not add a receipt from html when extractReceiptData returns null", {
timeout: 5000,
}, () => {
const out = extractData(undefined, "<p>nothing</p>");
expect(out.receipts).toEqual([]);
});
it("adds a receipt extracted from html fallback", { timeout: 5000 }, () => {
const out = extractData(undefined, "<p>order #ZZZ-9 total $9.99</p>");
expect(out.receipts).toHaveLength(1);
expect(out.receipts[0]!.orderNumber).toBe("ZZZ-9");
});
it("skips non-calendar attachments", { timeout: 5000 }, () => {
const att: EmailAttachment = {
filename: "note.txt",
mimeType: "text/plain",
disposition: "attachment",
content: "hi",
encoding: "utf8",
};
const out = extractData("body", undefined, [att]);
expect(out.calendarEvents).toEqual([]);
});
});

View File

@@ -0,0 +1,143 @@
import { parseICalendar } from "../calendar-parser/calendar-parser.js";
import type { CalendarEvent } from "../calendar-parser/calendar-parser.types.js";
import type { EmailAttachment } from "../types/email.js";
import type {
DeliveryTracking,
ExtractedData,
ReceiptData,
} from "./data-extractor.types.js";
/** Extract delivery tracking numbers from email text. */
export function extractTrackingInfo(text: string): DeliveryTracking[] {
const results: DeliveryTracking[] = [];
const seen = new Set<string>();
const upsMatches = text.match(/\b1Z[A-Z0-9]{16}\b/gi);
if (upsMatches) {
for (const num of upsMatches) {
if (!seen.has(num)) {
seen.add(num);
results.push({
carrier: "UPS",
trackingNumber: num,
trackingUrl: `https://www.ups.com/track?tracknum=${num}`,
});
}
}
}
const uspsMatches = text.match(/\b(94|93|92|95)\d{20,22}\b/g);
if (uspsMatches) {
for (const num of uspsMatches) {
if (!seen.has(num)) {
seen.add(num);
results.push({
carrier: "USPS",
trackingNumber: num,
trackingUrl: `https://tools.usps.com/go/TrackConfirmAction?tLabels=${num}`,
});
}
}
}
return results;
}
const ORDER_PATTERNS = {
orderNumber: [
/order\s*(?:#|number|no\.?|id)\s*[:.]?\s*([A-Z0-9-]+)/i,
/(?:confirmation|reference)\s*(?:#|number|no\.?)\s*[:.]?\s*([A-Z0-9-]+)/i,
],
total: [
/total\s*[:.]?\s*\$?([\d,]+\.\d{2})/i,
/amount\s*(?:charged|due|paid)\s*[:.]?\s*\$?([\d,]+\.\d{2})/i,
],
currency: [/(\$|USD|EUR|GBP|CAD|AUD)/i],
};
/** Extract receipt/order data from email text. */
export function extractReceiptData(text: string): ReceiptData | null {
let orderNumber: string | undefined;
let total: string | undefined;
let currency: string | undefined;
for (const pattern of ORDER_PATTERNS.orderNumber) {
const match = text.match(pattern);
if (match) {
orderNumber = match[1];
break;
}
}
for (const pattern of ORDER_PATTERNS.total) {
const match = text.match(pattern);
if (match) {
total = `$${match[1]}`;
break;
}
}
for (const pattern of ORDER_PATTERNS.currency) {
const match = text.match(pattern);
if (match) {
// Every pattern in ORDER_PATTERNS.currency captures group 1.
const sym = match[1] as string;
currency = sym === "$" ? "USD" : sym.toUpperCase();
break;
}
}
if (!orderNumber && !total) return null;
const out: ReceiptData = {};
if (orderNumber !== undefined) out.orderNumber = orderNumber;
if (total !== undefined) out.total = total;
if (currency !== undefined) out.currency = currency;
return out;
}
function attachmentBytesToString(att: EmailAttachment): string {
if (typeof att.content === "string") return att.content;
if (att.content instanceof ArrayBuffer) {
return new TextDecoder().decode(new Uint8Array(att.content));
}
return new TextDecoder().decode(att.content as unknown as Uint8Array);
}
/** Extract all structured data from a parsed email. */
export function extractData(
text?: string,
html?: string,
attachments: EmailAttachment[] = []
): ExtractedData {
const calendarEvents: CalendarEvent[] = [];
const deliveryTracking: DeliveryTracking[] = [];
const receipts: ReceiptData[] = [];
for (const att of attachments) {
if (
att.mimeType === "text/calendar" ||
att.mimeType === "application/ics"
) {
calendarEvents.push(
...parseICalendar(attachmentBytesToString(att))
);
}
}
const bodyText = text ?? "";
if (bodyText) {
deliveryTracking.push(...extractTrackingInfo(bodyText));
const receipt = extractReceiptData(bodyText);
if (receipt) receipts.push(receipt);
}
if (html && !bodyText) {
const stripped = html.replace(/<[^>]*>/g, " ");
deliveryTracking.push(...extractTrackingInfo(stripped));
const receipt = extractReceiptData(stripped);
if (receipt) receipts.push(receipt);
}
return { calendarEvents, deliveryTracking, receipts };
}

View File

@@ -0,0 +1,21 @@
import type { CalendarEvent } from "../calendar-parser/calendar-parser.types.js";
export interface DeliveryTracking {
carrier?: string;
trackingNumber: string;
trackingUrl?: string;
}
export interface ReceiptData {
orderNumber?: string;
total?: string;
currency?: string;
merchant?: string;
date?: string;
}
export interface ExtractedData {
calendarEvents: CalendarEvent[];
deliveryTracking: DeliveryTracking[];
receipts: ReceiptData[];
}

View File

@@ -0,0 +1,10 @@
export {
extractData,
extractReceiptData,
extractTrackingInfo,
} from "./data-extractor.js";
export type {
DeliveryTracking,
ExtractedData,
ReceiptData,
} from "./data-extractor.types.js";

View File

@@ -0,0 +1,86 @@
# dkim-canonicalize
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements RFC 6376 canonicalisation primitives — the byte-exact text
normalisation steps DKIM requires before hashing a header set or body.
Both `simple` and `relaxed` modes are supported for headers and
bodies, plus `buildSignedHeaderBlock` which combines the canonicalised
signed headers with the canonicalised `DKIM-Signature` line (sans
`b=` value) that callers feed into RSA `verify`.
## Features
- `canonicalizeHeaderRelaxed` — lower-case name, collapse folded
whitespace, trim, single-space internal runs.
- `canonicalizeBodyRelaxed` — CRLF-normalise, trim trailing per-line
WS, collapse internal runs, strip trailing empty lines, ensure one
final CRLF.
- `canonicalizeBodySimple` — CRLF-normalise + ensure exactly one
final CRLF.
- `buildSignedHeaderBlock` — walks `signedHeaders` in order, indexes
repeated headers, removes the `b=` payload of the
`DKIM-Signature` line per RFC, and joins with CRLF.
## Usage: Basic
```ts
import { canonicalizeBodyRelaxed } from "@suaveplan/email-processing/dkim";
const canon = canonicalizeBodyRelaxed(body);
const digest = await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(canon)
);
```
## Usage: Advanced
```ts
import {
buildSignedHeaderBlock,
canonicalizeBodyRelaxed,
} from "@suaveplan/email-processing/dkim";
const canonHeaders = buildSignedHeaderBlock(headers, sig);
const canonBody = canonicalizeBodyRelaxed(body);
const bodyHash = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(canonBody)
)
);
const ok = await crypto.subtle.verify(
"RSASSA-PKCS1-v1_5",
key,
sigBytes,
new TextEncoder().encode(canonHeaders)
);
```
## API Reference
- `canonicalizeHeaderRelaxed(name, value): string` — single-line.
- `canonicalizeBodyRelaxed(body): string`.
- `canonicalizeBodySimple(body): string`.
- `buildSignedHeaderBlock(headers, sig): string`.
Types `DkimSignature`, `DkimResult`, `DkimVerificationResult`,
`DkimDnsResolver` are re-exported from the package root for callers
that consume the verifier.
## Implementation Notes
The relaxed body canonicalisation collapses runs of tabs and spaces
inside a line **before** stripping trailing whitespace, matching the
order specified in RFC 6376 §3.4.4. Empty bodies still emit the
mandated trailing CRLF after `simple` canonicalisation so a hash
mismatch is not produced by a body that legitimately ends without one.
`buildSignedHeaderBlock` keeps a per-name index counter so a signature
that lists `from` then `from` retrieves the first and second `From:`
header in order — without the counter, the DKIM rebuild would emit the
same header twice and verification would fail on otherwise valid mail.

View File

@@ -0,0 +1,143 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import {
buildSignedHeaderBlock,
canonicalizeBodyRelaxed,
canonicalizeBodySimple,
canonicalizeHeaderRelaxed,
} from "./dkim-canonicalize.js";
import type { DkimSignature } from "./dkim-canonicalize.types.js";
function relaxedSig(headers: string[]): DkimSignature {
return {
version: 1,
algorithm: "rsa-sha256",
domain: "example.com",
selector: "s1",
canonicalization: { header: "relaxed", body: "relaxed" },
signedHeaders: headers,
bodyHash: "",
signature: "",
};
}
function simpleSig(headers: string[]): DkimSignature {
return {
...relaxedSig(headers),
canonicalization: { header: "simple", body: "simple" },
};
}
describe("canonicalizeHeaderRelaxed", () => {
it("lowercases, trims, collapses internal whitespace", {
timeout: 5000,
}, () => {
const out = canonicalizeHeaderRelaxed("Subject", " Hello World ");
expect(out).toBe("subject:Hello World");
});
it("collapses folded continuations", { timeout: 5000 }, () => {
const out = canonicalizeHeaderRelaxed("X", "a\r\n b");
expect(out).toBe("x:a b");
});
});
describe("canonicalizeBodyRelaxed", () => {
it("normalises line endings and trims trailing empty lines", {
timeout: 5000,
}, () => {
const out = canonicalizeBodyRelaxed("a\n\nb\r\n\r\n\r\n");
expect(out).toBe("a\r\n\r\nb\r\n");
});
it("collapses internal runs of tabs/spaces and strips trailing WS", {
timeout: 5000,
}, () => {
const out = canonicalizeBodyRelaxed("a b\t \n");
expect(out).toBe("a b\r\n");
});
});
describe("canonicalizeBodySimple", () => {
it("normalises line endings and ensures exactly one trailing CRLF", {
timeout: 5000,
}, () => {
expect(canonicalizeBodySimple("a\n\n")).toBe("a\r\n");
});
it("handles empty body", { timeout: 5000 }, () => {
expect(canonicalizeBodySimple("")).toBe("\r\n");
});
});
describe("buildSignedHeaderBlock", () => {
const headers = [
{ key: "From", value: "alice@x" },
{ key: "Subject", value: "Hello" },
{ key: "X-Custom", value: "first" },
{ key: "X-Custom", value: "second" },
{
key: "DKIM-Signature",
value: "v=1; b=ABC123; a=rsa-sha256",
},
];
it("emits relaxed canonicalised header lines for signed headers", {
timeout: 5000,
}, () => {
const sig = relaxedSig(["from", "subject"]);
const out = buildSignedHeaderBlock(headers, sig);
expect(out).toContain("from:alice@x");
expect(out).toContain("subject:Hello");
expect(out).toContain("dkim-signature:");
expect(out).toContain("b=");
});
it("emits simple canonicalisation when c=simple/simple", {
timeout: 5000,
}, () => {
const sig = simpleSig(["from"]);
const out = buildSignedHeaderBlock(headers, sig);
expect(out).toContain("from:alice@x");
expect(out).toContain("dkim-signature:");
});
it("walks repeated headers by name with index counter", {
timeout: 5000,
}, () => {
const sig = relaxedSig(["x-custom", "x-custom"]);
const out = buildSignedHeaderBlock(headers, sig);
const lines = out.split("\r\n");
// Two x-custom signed entries pick the first then the second header,
// plus the DKIM-Signature line on its own.
expect(lines.filter((l) => l.startsWith("x-custom:"))).toHaveLength(2);
expect(
lines.filter((l) => l.startsWith("dkim-signature:"))
).toHaveLength(1);
});
it("skips signedHeaders not present in the header list", {
timeout: 5000,
}, () => {
const sig = relaxedSig(["nonexistent"]);
const out = buildSignedHeaderBlock(headers, sig);
expect(out).not.toContain("nonexistent");
});
it("does nothing extra when there is no DKIM-Signature header", {
timeout: 5000,
}, () => {
const sig = relaxedSig(["from"]);
const out = buildSignedHeaderBlock(headers.slice(0, 4), sig);
expect(out).not.toContain("dkim-signature");
});
it("stops emitting after exhausting available repeats", {
timeout: 5000,
}, () => {
const sig = relaxedSig(["x-custom", "x-custom", "x-custom"]);
const out = buildSignedHeaderBlock(headers, sig);
const xCount = out
.split("\r\n")
.filter((l) => l.startsWith("x-custom:")).length;
expect(xCount).toBe(2);
});
});

View File

@@ -0,0 +1,77 @@
import type { DkimSignature } from "./dkim-canonicalize.types.js";
/** Apply relaxed canonicalization to a header line per RFC 6376 §3.4.2. */
export function canonicalizeHeaderRelaxed(name: string, value: string): string {
const lowerName = name.toLowerCase().trim();
const normalizedValue = value
.replace(/\r?\n[\t ]+/g, " ")
.replace(/[\t ]+/g, " ")
.trim();
return `${lowerName}:${normalizedValue}`;
}
/** Apply relaxed canonicalization to an email body per RFC 6376 §3.4.4. */
export function canonicalizeBodyRelaxed(body: string): string {
let lines = body.replace(/\r?\n/g, "\r\n").split("\r\n");
lines = lines.map((line) =>
line.replace(/[\t ]+/g, " ").replace(/[\t ]+$/, "")
);
let result = lines.join("\r\n");
result = result.replace(/(\r\n)+$/, "");
return `${result}\r\n`;
}
/** Apply simple canonicalization to an email body per RFC 6376 §3.4.3. */
export function canonicalizeBodySimple(body: string): string {
let result = body.replace(/\r?\n/g, "\r\n");
result = result.replace(/(\r\n)+$/, "");
return `${result}\r\n`;
}
/** Build the canonicalized header block for signature verification. */
export function buildSignedHeaderBlock(
headers: Array<{ key: string; value: string }>,
sig: DkimSignature
): string {
const lines: string[] = [];
const headerMap = new Map<string, string[]>();
for (const h of headers) {
const lower = h.key.toLowerCase();
if (!headerMap.has(lower)) headerMap.set(lower, []);
headerMap.get(lower)?.push(h.value);
}
const usedCounts = new Map<string, number>();
for (const name of sig.signedHeaders) {
const values = headerMap.get(name);
if (!values) continue;
const idx = usedCounts.get(name) ?? 0;
if (idx < values.length) {
// idx < values.length, so values[idx] is defined.
const value = values[idx] as string;
if (sig.canonicalization.header === "relaxed") {
lines.push(canonicalizeHeaderRelaxed(name, value));
} else {
lines.push(`${name}:${value}`);
}
usedCounts.set(name, idx + 1);
}
}
const dkimHeader = headers.find(
(h) => h.key.toLowerCase() === "dkim-signature"
);
if (dkimHeader) {
const valueWithoutSig = dkimHeader.value.replace(/b=[^;]*/, "b=");
if (sig.canonicalization.header === "relaxed") {
lines.push(
canonicalizeHeaderRelaxed("dkim-signature", valueWithoutSig)
);
} else {
lines.push(`dkim-signature:${valueWithoutSig}`);
}
}
return lines.join("\r\n");
}

View File

@@ -0,0 +1,27 @@
export type DkimResult = "pass" | "fail" | "none";
export interface DkimSignature {
version: number;
algorithm: string;
domain: string;
selector: string;
canonicalization: { header: string; body: string };
signedHeaders: string[];
bodyHash: string;
signature: string;
bodyLength?: number;
timestamp?: number;
expiration?: number;
}
export interface DkimVerificationResult {
result: DkimResult;
domain?: string;
selector?: string;
error?: string;
}
export type DkimDnsResolver = (
selector: string,
domain: string
) => Promise<string | null>;

View File

@@ -0,0 +1,12 @@
export {
buildSignedHeaderBlock,
canonicalizeBodyRelaxed,
canonicalizeBodySimple,
canonicalizeHeaderRelaxed,
} from "./dkim-canonicalize.js";
export type {
DkimDnsResolver,
DkimResult,
DkimSignature,
DkimVerificationResult,
} from "./dkim-canonicalize.types.js";

View File

@@ -0,0 +1,84 @@
# dkim-verifier
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements DKIM (RFC 6376) signature verification — parses a
`DKIM-Signature` header, looks up the signer's public key through an
injectable DNS resolver, recomputes the body hash, rebuilds the
canonicalised signed-header block, and calls Web Crypto's
RSASSA-PKCS1-v1_5 verify. Returns a structured outcome that callers
plug into DMARC alignment, spam scoring, and inbox display.
## Features
- Strict tag parsing with explicit nullability — missing `v`/`a`/`d`/
`s`/`h`/`bh`/`b` returns `null` (treated as `fail` by the
verifier).
- Defaults `c` to `simple/simple`, accepts shorthand `c=relaxed`
(header relaxed, body simple).
- Honours optional `l=` body-length tag, `t=` timestamp, `x=`
expiration so callers can surface staleness.
- DNS resolver is fully injectable — the package never calls `dns.*`
directly, so consumers can route through any DoH / cache /
`@suaveplan/dns` strategy.
- Hard error path: a structurally-broken public-key DNS record raises
`EMAIL_PROCESSING.DKIM_VERIFY_FAILURE` rather than silently
failing, so observability dashboards distinguish "signer is
misconfigured" from "this message did not verify".
- OTel counter `email_processing.dkim.result_total` attributed by
`result` for every outcome.
## Usage: Basic
```ts
import { verifyDkim } from "@suaveplan/email-processing/dkim";
const out = await verifyDkim(headers, body, async (selector, domain) => {
const records = await dns.resolveTxt(`${selector}._domainkey.${domain}`);
return records[0]?.join("") ?? null;
});
if (out.result === "pass") authenticate();
```
## Usage: Advanced
```ts
import {
extractDkimHeader,
parseDkimSignature,
verifyDkim,
} from "@suaveplan/email-processing/dkim";
const raw = extractDkimHeader(headers);
if (raw) {
const sig = parseDkimSignature(raw);
if (sig && sig.expiration && sig.expiration < Date.now() / 1000) {
log.warn({ domain: sig.domain }, "stale DKIM signature");
}
}
const result = await verifyDkim(headers, body, dnsResolver);
metrics.recordDkim(result.domain ?? "unknown", result.result);
```
## API Reference
- `parseDkimSignature(value): DkimSignature | null`.
- `extractDkimHeader(headers): string | null`.
- `verifyDkim(headers, body, dnsResolver): Promise<DkimVerificationResult>`.
## Implementation Notes
The verifier strips the `b=` value from the `DKIM-Signature` header
before feeding the canonicalised block into the verify call — this
matches RFC 6376 §3.5 and ensures consumers who pass the
`DKIM-Signature` header back through unchanged still produce the
correct hash. The `catch` arm of the main `try/try-catch` block is
intentionally narrow: it re-throws our own `EmailProcessingError` so
unrecoverable configuration errors are not silently downgraded to
`fail` outcomes, but every other thrown value becomes a `fail` with a
diagnostic `error` field.

View File

@@ -0,0 +1,418 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { canonicalizeBodyRelaxed } from "../dkim-canonicalize/dkim-canonicalize.js";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import {
extractDkimHeader,
parseDkimSignature,
verifyDkim,
} from "./dkim-verifier.js";
function b64(bytes: Uint8Array | ArrayBuffer): string {
const u8 = bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes);
let binary = "";
for (let i = 0; i < u8.length; i++)
binary += String.fromCharCode(u8[i] ?? 0);
return btoa(binary);
}
async function signRsa(data: Uint8Array, key: CryptoKey): Promise<Uint8Array> {
const sig = await crypto.subtle.sign(
"RSASSA-PKCS1-v1_5",
key,
data as unknown as BufferSource
);
return new Uint8Array(sig);
}
async function generateKeyPair(): Promise<{
publicKeyB64: string;
privateKey: CryptoKey;
}> {
const pair = await crypto.subtle.generateKey(
{
name: "RSASSA-PKCS1-v1_5",
modulusLength: 2048,
publicExponent: new Uint8Array([1, 0, 1]),
hash: "SHA-256",
},
true,
["sign", "verify"]
);
const spki = await crypto.subtle.exportKey("spki", pair.publicKey);
return {
publicKeyB64: b64(new Uint8Array(spki)),
privateKey: pair.privateKey,
};
}
describe("parseDkimSignature", () => {
it("parses all required tags", { timeout: 5000 }, () => {
const v =
"v=1; a=rsa-sha256; d=example.com; s=s1; h=from:subject; bh=AAA; b=BBB";
const sig = parseDkimSignature(v);
expect(sig).not.toBeNull();
expect(sig!.algorithm).toBe("rsa-sha256");
expect(sig!.signedHeaders).toEqual(["from", "subject"]);
});
it("returns null when a required tag is missing", { timeout: 5000 }, () => {
const v = "v=1; a=rsa-sha256; d=example.com";
expect(parseDkimSignature(v)).toBeNull();
});
it("parses optional c=relaxed/simple, l, t, x", { timeout: 5000 }, () => {
const v =
"v=1; a=rsa-sha256; d=x; s=s; h=from; bh=BH; b=B; c=relaxed/simple; l=42; t=100; x=200";
const sig = parseDkimSignature(v);
expect(sig!.canonicalization).toEqual({
header: "relaxed",
body: "simple",
});
expect(sig!.bodyLength).toBe(42);
expect(sig!.timestamp).toBe(100);
expect(sig!.expiration).toBe(200);
});
it("treats c=simple (no slash) as simple/simple", { timeout: 5000 }, () => {
const sig = parseDkimSignature(
"v=1; a=rsa-sha256; d=x; s=s; h=from; bh=B; b=B; c=relaxed"
);
expect(sig!.canonicalization).toEqual({
header: "relaxed",
body: "simple",
});
});
it("ignores stray and malformed tag fragments", { timeout: 5000 }, () => {
const sig = parseDkimSignature(
"; lonely; v=1; a=rsa-sha256; d=x; s=s; h=from; bh=B; b=B"
);
expect(sig).not.toBeNull();
});
});
describe("extractDkimHeader", () => {
it("returns header value case-insensitively", { timeout: 5000 }, () => {
const out = extractDkimHeader([
{ key: "From", value: "x" },
{ key: "Dkim-Signature", value: "v=1;" },
]);
expect(out).toBe("v=1;");
});
it("returns null when no DKIM-Signature header is present", {
timeout: 5000,
}, () => {
expect(extractDkimHeader([])).toBeNull();
});
});
describe("verifyDkim", () => {
it("returns result=none when no DKIM-Signature header is present", {
timeout: 5000,
}, async () => {
const out = await verifyDkim([], "body", async () => null);
expect(out.result).toBe("none");
});
it("returns result=fail for an invalid signature header", {
timeout: 5000,
}, async () => {
const out = await verifyDkim(
[{ key: "DKIM-Signature", value: "garbage" }],
"body",
async () => "v=DKIM1; k=rsa; p=AAAA"
);
expect(out.result).toBe("fail");
expect(out.error).toContain("Invalid DKIM-Signature");
});
it("returns result=fail when DNS lookup yields null", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=AAAAAA==; b=BBBBBB==";
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkim },
],
"body",
async () => null
);
expect(out.result).toBe("fail");
expect(out.error).toContain("No DKIM public key");
});
it("returns result=fail when DNS record contains stray and malformed tags", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=AAAAAA==; b=BBBBBB==";
const body = "body\r\n";
const canonBody = await import(
"../dkim-canonicalize/dkim-canonicalize.js"
).then((m) => m.canonicalizeBodyRelaxed(body));
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(canonBody)
)
);
const dkimMatch = dkim.replace(/bh=[^;]+/, `bh=${bh}`);
// DNS record with empty segment, no-= segment, and no p= — should
// surface as "Could not parse public key".
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkimMatch },
],
body,
async () => " ; lonely-no-equals ; k=rsa "
);
expect(out.result).toBe("fail");
expect(out.error).toContain("Could not parse public key");
});
it("returns result=fail when DNS record lacks p= tag", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=AAAAAA==; b=BBBBBB==";
const body = "body\r\n";
const canonBody = canonicalizeBodyRelaxed(body);
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(canonBody)
)
);
// Override bh so it matches the body hash but DNS record is malformed.
const dkimMatch = dkim.replace(/bh=[^;]+/, `bh=${bh}`);
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkimMatch },
],
body,
async () => "v=DKIM1; k=rsa"
);
expect(out.result).toBe("fail");
});
it("returns result=fail on body-hash mismatch", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=NOTHEHASH; b=B==; c=relaxed/relaxed";
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkim },
],
"body\r\n",
async () => "v=DKIM1; k=rsa; p=AAAA"
);
expect(out.result).toBe("fail");
expect(out.error).toContain("Body hash mismatch");
});
it("throws DKIM_VERIFY_FAILURE when the DNS-provided key is malformed", {
timeout: 5000,
}, async () => {
const body = "body\r\n";
const canon = canonicalizeBodyRelaxed(body);
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(canon)
)
);
const dkim = `v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=${bh}; b=B==; c=relaxed/relaxed`;
await expect(
verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkim },
],
body,
async () => "v=DKIM1; k=rsa; p=not-base64-spki"
)
).rejects.toBeInstanceOf(EmailProcessingError);
});
it("returns result=pass for a freshly-signed message", {
timeout: 30000,
}, async () => {
const { publicKeyB64, privateKey } = await generateKeyPair();
const { buildSignedHeaderBlock } = await import(
"../dkim-canonicalize/dkim-canonicalize.js"
);
const body = "Hello DKIM\r\n";
// simple-canonicalised body == body (already CRLF-terminated, no trailing blanks).
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(body)
)
);
const headers = [
{ key: "From", value: "alice@test.example" },
{ key: "Subject", value: "Round trip" },
];
const dkimTpl =
"v=1; a=rsa-sha256; c=relaxed/simple; d=test.example; s=sel; " +
`h=from:subject; bh=${bh}; b=`;
const sig = parseDkimSignature(`${dkimTpl}PLACEHOLDER`);
expect(sig).not.toBeNull();
// Build header block exactly as verifier will: signed headers first,
// then the DKIM line with b= value stripped.
const headersForSign = [
...headers,
{ key: "DKIM-Signature", value: dkimTpl },
];
const blob = buildSignedHeaderBlock(headersForSign, sig!);
const sigBytes = await signRsa(
new TextEncoder().encode(blob),
privateKey
);
const dkimFinal = `${dkimTpl}${b64(sigBytes)}`;
const finalHeaders = [
...headers,
{ key: "DKIM-Signature", value: dkimFinal },
];
const out = await verifyDkim(
finalHeaders,
body,
async () => `v=DKIM1; k=rsa; p=${publicKeyB64}`
);
if (out.result !== "pass") {
// Surface the verifier's error to ease debugging.
throw new Error(`expected pass, got ${out.result}: ${out.error}`);
}
expect(out.result).toBe("pass");
expect(out.domain).toBe("test.example");
});
it("returns result=fail when the signature does not verify (modified body)", {
timeout: 30000,
}, async () => {
const { publicKeyB64, privateKey } = await generateKeyPair();
const { buildSignedHeaderBlock } = await import(
"../dkim-canonicalize/dkim-canonicalize.js"
);
const body = "Hello DKIM\r\n";
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(body)
)
);
const headers = [{ key: "From", value: "alice@test.example" }];
const dkimTpl =
"v=1; a=rsa-sha256; c=relaxed/simple; d=test.example; s=sel; " +
`h=from; bh=${bh}; b=`;
const sig = parseDkimSignature(`${dkimTpl}PLACEHOLDER`);
const headersForSign = [
...headers,
{ key: "DKIM-Signature", value: dkimTpl },
];
const blob = buildSignedHeaderBlock(headersForSign, sig!);
const sigBytes = await signRsa(
new TextEncoder().encode(blob),
privateKey
);
// Tamper with the signature so verify() returns false.
const tampered = new Uint8Array(sigBytes);
tampered[0] = (tampered[0]! + 1) & 0xff;
const dkimFinal = `${dkimTpl}${b64(tampered)}`;
const out = await verifyDkim(
[...headers, { key: "DKIM-Signature", value: dkimFinal }],
body,
async () => `v=DKIM1; k=rsa; p=${publicKeyB64}`
);
expect(out.result).toBe("fail");
expect(out.error).toContain("Signature verification failed");
});
it("respects bodyLength when l= is set", { timeout: 30000 }, async () => {
const { publicKeyB64, privateKey } = await generateKeyPair();
const { buildSignedHeaderBlock } = await import(
"../dkim-canonicalize/dkim-canonicalize.js"
);
// Use l= to truncate the canonicalised body before hashing.
const fullBody = "Hello DKIM\r\nextra junk\r\n";
const truncated = "Hello DKIM\r\n".length;
const slice = fullBody.slice(0, truncated);
const bh = b64(
await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(slice)
)
);
const headers = [{ key: "From", value: "alice@test.example" }];
const dkimTpl =
"v=1; a=rsa-sha256; c=relaxed/simple; d=test.example; s=sel; " +
`h=from; bh=${bh}; l=${truncated}; b=`;
const sig = parseDkimSignature(`${dkimTpl}PLACEHOLDER`);
const headersForSign = [
...headers,
{ key: "DKIM-Signature", value: dkimTpl },
];
const blob = buildSignedHeaderBlock(headersForSign, sig!);
const sigBytes = await signRsa(
new TextEncoder().encode(blob),
privateKey
);
const dkimFinal = `${dkimTpl}${b64(sigBytes)}`;
const out = await verifyDkim(
[...headers, { key: "DKIM-Signature", value: dkimFinal }],
fullBody,
async () => `v=DKIM1; k=rsa; p=${publicKeyB64}`
);
expect(out.result).toBe("pass");
});
it("falls into the catch arm when DNS resolver itself throws", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=AAAAAA==; b=B==";
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkim },
],
"body",
async () => {
throw new Error("dns down");
}
);
expect(out.result).toBe("fail");
expect(out.error).toContain("dns down");
});
it("falls into the catch arm with non-Error throws", {
timeout: 5000,
}, async () => {
const dkim =
"v=1; a=rsa-sha256; d=x.com; s=s; h=from; bh=AAAAAA==; b=B==";
const out = await verifyDkim(
[
{ key: "From", value: "x@x.com" },
{ key: "DKIM-Signature", value: dkim },
],
"body",
async () => {
const nonErr: unknown = "string-throw";
throw nonErr;
}
);
expect(out.result).toBe("fail");
expect(out.error).toBe("Unknown error");
});
});

View File

@@ -0,0 +1,232 @@
import {
buildSignedHeaderBlock,
canonicalizeBodyRelaxed,
canonicalizeBodySimple,
} from "../dkim-canonicalize/dkim-canonicalize.js";
import type {
DkimDnsResolver,
DkimSignature,
DkimVerificationResult,
} from "../dkim-canonicalize/dkim-canonicalize.types.js";
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
/** Parse a DKIM-Signature header value into its component fields. */
export function parseDkimSignature(headerValue: string): DkimSignature | null {
const tags = new Map<string, string>();
const parts = headerValue.split(";");
for (const part of parts) {
const trimmed = part.trim();
if (!trimmed) continue;
const eqIdx = trimmed.indexOf("=");
if (eqIdx === -1) continue;
const key = trimmed.slice(0, eqIdx).trim();
const value = trimmed.slice(eqIdx + 1).trim();
tags.set(key, value);
}
const version = tags.get("v");
const algorithm = tags.get("a");
const domain = tags.get("d");
const selector = tags.get("s");
const headers = tags.get("h");
const bodyHash = tags.get("bh");
const signature = tags.get("b");
if (
!version ||
!algorithm ||
!domain ||
!selector ||
!headers ||
!bodyHash ||
!signature
) {
return null;
}
const canon = tags.get("c") ?? "simple/simple";
const [headerCanon, bodyCanon = "simple"] = canon.split("/") as [
string,
string?,
];
const result: DkimSignature = {
version: Number.parseInt(version, 10),
algorithm,
domain,
selector,
canonicalization: { header: headerCanon, body: bodyCanon },
signedHeaders: headers.split(":").map((h) => h.trim().toLowerCase()),
bodyHash: bodyHash.replace(/\s/g, ""),
signature: signature.replace(/\s/g, ""),
};
const l = tags.get("l");
if (l) result.bodyLength = Number.parseInt(l, 10);
const t = tags.get("t");
if (t) result.timestamp = Number.parseInt(t, 10);
const x = tags.get("x");
if (x) result.expiration = Number.parseInt(x, 10);
return result;
}
/** Extract the DKIM-Signature header value from raw email headers. */
export function extractDkimHeader(
headers: Array<{ key: string; value: string }>
): string | null {
const dkimHeader = headers.find(
(h) => h.key.toLowerCase() === "dkim-signature"
);
return dkimHeader?.value ?? null;
}
/** Extract the base64-encoded public key from a DKIM DNS TXT record. */
function extractPublicKeyFromDns(record: string): string | null {
const tags = new Map<string, string>();
for (const part of record.split(";")) {
const trimmed = part.trim();
if (!trimmed) continue;
const eqIdx = trimmed.indexOf("=");
if (eqIdx === -1) continue;
tags.set(
trimmed.slice(0, eqIdx).trim(),
trimmed.slice(eqIdx + 1).trim()
);
}
return tags.get("p") ?? null;
}
async function importPublicKey(base64Key: string): Promise<CryptoKey> {
const binaryDer = Uint8Array.from(atob(base64Key), (c) => c.charCodeAt(0));
return crypto.subtle.importKey(
"spki",
binaryDer,
{ name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" },
false,
["verify"]
);
}
/**
* Verify a DKIM signature against an email.
* Requires a DNS resolver to fetch the public key.
*/
export async function verifyDkim(
headers: Array<{ key: string; value: string }>,
body: string,
dnsResolver: DkimDnsResolver
): Promise<DkimVerificationResult> {
const dkimValue = extractDkimHeader(headers);
if (!dkimValue) {
getEmailProcessingInstruments().dkimResultCounter.add(1, {
result: "none",
});
return { result: "none" };
}
const sig = parseDkimSignature(dkimValue);
if (!sig) {
getEmailProcessingInstruments().dkimResultCounter.add(1, {
result: "fail",
});
return { result: "fail", error: "Invalid DKIM-Signature header" };
}
try {
const publicKeyRecord = await dnsResolver(sig.selector, sig.domain);
if (!publicKeyRecord) {
return reportFail(sig, "No DKIM public key found in DNS");
}
const canonBody =
sig.canonicalization.body === "relaxed"
? canonicalizeBodyRelaxed(body)
: canonicalizeBodySimple(body);
const bodyToHash =
sig.bodyLength != null
? canonBody.slice(0, sig.bodyLength)
: canonBody;
const bodyHashBuffer = await crypto.subtle.digest(
"SHA-256",
new TextEncoder().encode(bodyToHash)
);
const bodyHashBase64 = btoa(
String.fromCharCode(...new Uint8Array(bodyHashBuffer))
);
if (bodyHashBase64 !== sig.bodyHash) {
return reportFail(sig, "Body hash mismatch");
}
const keyData = extractPublicKeyFromDns(publicKeyRecord);
if (!keyData) {
return reportFail(
sig,
"Could not parse public key from DNS record"
);
}
const headerLines = buildSignedHeaderBlock(headers, sig);
let cryptoKey: CryptoKey;
try {
cryptoKey = await importPublicKey(keyData);
} catch (err) {
// Web Crypto importKey rejects only with `Error` subclasses
// (DOMException, TypeError, etc.) in every supported runtime.
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.DKIM_VERIFY_FAILURE,
"email_processing.dkim_verify_failure",
{ cause: err as Error }
);
}
const signatureBytes = Uint8Array.from(atob(sig.signature), (c) =>
c.charCodeAt(0)
);
const dataBytes = new TextEncoder().encode(headerLines);
const valid = await crypto.subtle.verify(
{ name: "RSASSA-PKCS1-v1_5" },
cryptoKey,
signatureBytes,
dataBytes
);
const instruments = getEmailProcessingInstruments();
if (valid) {
instruments.dkimResultCounter.add(1, { result: "pass" });
return {
result: "pass",
domain: sig.domain,
selector: sig.selector,
};
}
return reportFail(sig, "Signature verification failed");
} catch (err) {
if (err instanceof EmailProcessingError) throw err;
return reportFail(
sig,
err instanceof Error ? err.message : "Unknown error"
);
}
}
function reportFail(
sig: DkimSignature,
message: string
): DkimVerificationResult {
getEmailProcessingInstruments().dkimResultCounter.add(1, {
result: "fail",
});
return {
result: "fail",
domain: sig.domain,
selector: sig.selector,
error: message,
};
}

View File

@@ -0,0 +1,5 @@
export {
extractDkimHeader,
parseDkimSignature,
verifyDkim,
} from "./dkim-verifier.js";

View File

@@ -0,0 +1,78 @@
# dmarc-checker
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements DMARC (RFC 7489) policy evaluation on top of an upstream
SPF result and DKIM verification. Looks up the `_dmarc.<from-domain>`
TXT record through an injectable resolver, parses its policy and
alignment tags, then determines whether at least one of SPF or DKIM
passes **and** aligns with the From header's organisational domain.
## Features
- `parseDmarcRecord` returns a strongly-typed `DmarcRecord` with all
standard tags (`p`, `sp`, `adkim`, `aspf`, `pct`, `rua`, `ruf`).
- `checkDmarc` accepts pre-computed SPF and DKIM outcomes so callers
can compose authentication pipelines without re-running expensive
DNS or crypto work.
- Both alignment modes — strict (exact domain match) and relaxed
(organisational-domain match) — are supported.
- DNS resolver is async and injectable; the package itself never
calls into `dns.*` directly.
- Records `email_processing.dmarc.result_total` attributed by
`result` for every outcome.
## Usage: Basic
```ts
import { checkDmarc } from "@suaveplan/email-processing/dmarc";
const out = await checkDmarc(fromDomain, spfResult, dkimResult, async (d) => {
const recs = await dns.resolveTxt(d);
return recs.map((p) => p.join(""));
});
if (out.result === "fail" && out.policy === "reject") {
rejectMessage();
}
```
## Usage: Advanced
```ts
import { checkDmarc, parseDmarcRecord } from "@suaveplan/email-processing/dmarc";
const recs = await dnsResolver(`_dmarc.${domain}`);
const dmarcText = recs.find((r) => r.startsWith("v=DMARC1")) ?? "";
const parsed = parseDmarcRecord(dmarcText);
const aggregateUri = parsed?.reportUri;
if (aggregateUri) {
await scheduleReport({ domain, uri: aggregateUri });
}
const result = await checkDmarc(domain, spf, dkim, dnsResolver);
metrics.recordDmarc(domain, result.result, result.policy);
```
## API Reference
- `DmarcPolicy` — `"none" | "quarantine" | "reject"`.
- `DmarcAlignment` — `"strict" | "relaxed"`.
- `DmarcResult` — `"pass" | "fail" | "none"`.
- `DmarcRecord` — full parsed record.
- `DmarcCheckResult` — `{ result, policy?, spfAligned, dkimAligned, record?, error? }`.
- `parseDmarcRecord(txt)` — strict parser, returns `null` on invalid
input.
- `checkDmarc(fromDomain, spf, dkim, resolver)` — async orchestrator.
## Implementation Notes
The DMARC TXT record lookup uses the bare `_dmarc.<domain>` form per
RFC 7489 §6.6.3 (organisational-domain fallback is left to a higher
layer — callers who want PSL-aware fallback should chain a second
`checkDmarc` against the parent domain when the first returns
`result: "none"` with `error: "No DMARC record found"`). The
`reportDmarc` helper centralises OTel emission so every return path
records the outcome exactly once.

View File

@@ -0,0 +1,182 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import type { DkimVerificationResult } from "../dkim-canonicalize/dkim-canonicalize.types.js";
import type { SpfCheckResult } from "../spf-checker/spf-checker.types.js";
import { checkDmarc, parseDmarcRecord } from "./dmarc-checker.js";
const spfPass: SpfCheckResult = { result: "pass", domain: "x.com" };
const spfFail: SpfCheckResult = { result: "fail", domain: "x.com" };
const dkimPass: DkimVerificationResult = { result: "pass", domain: "x.com" };
const dkimFail: DkimVerificationResult = { result: "fail" };
describe("parseDmarcRecord", () => {
it("parses policy + alignment + pct + reporting", { timeout: 5000 }, () => {
const r = parseDmarcRecord(
"v=DMARC1; p=reject; adkim=s; aspf=s; pct=50; rua=mailto:agg@x; ruf=mailto:foren@x; sp=quarantine"
);
expect(r).not.toBeNull();
expect(r!.policy).toBe("reject");
expect(r!.subdomainPolicy).toBe("quarantine");
expect(r!.dkimAlignment).toBe("strict");
expect(r!.spfAlignment).toBe("strict");
expect(r!.percentage).toBe(50);
expect(r!.reportUri).toBe("mailto:agg@x");
expect(r!.forensicUri).toBe("mailto:foren@x");
});
it("defaults pct to 100 when absent", { timeout: 5000 }, () => {
const r = parseDmarcRecord("v=DMARC1; p=none");
expect(r!.percentage).toBe(100);
});
it("defaults alignment to relaxed", { timeout: 5000 }, () => {
const r = parseDmarcRecord("v=DMARC1; p=none");
expect(r!.dkimAlignment).toBe("relaxed");
expect(r!.spfAlignment).toBe("relaxed");
});
it("falls back to policy when sp= is malformed", { timeout: 5000 }, () => {
const r = parseDmarcRecord("v=DMARC1; p=reject; sp=garbage");
expect(r!.subdomainPolicy).toBe("reject");
});
it("returns null for non-DMARC1 records", { timeout: 5000 }, () => {
expect(parseDmarcRecord("v=DMARC2; p=reject")).toBeNull();
});
it("returns null when p= is missing", { timeout: 5000 }, () => {
expect(parseDmarcRecord("v=DMARC1")).toBeNull();
});
it("returns null when p= is unrecognised", { timeout: 5000 }, () => {
expect(parseDmarcRecord("v=DMARC1; p=garbage")).toBeNull();
});
it("ignores malformed tag fragments", { timeout: 5000 }, () => {
const r = parseDmarcRecord("v=DMARC1; lonely; p=none");
expect(r).not.toBeNull();
});
});
describe("checkDmarc", () => {
it("returns result=none when DNS resolver throws", {
timeout: 5000,
}, async () => {
const out = await checkDmarc("x.com", spfPass, dkimPass, async () => {
throw new Error("dns down");
});
expect(out.result).toBe("none");
expect(out.error).toContain("DNS lookup");
});
it("returns result=none when DNS returns no DMARC record", {
timeout: 5000,
}, async () => {
const out = await checkDmarc("x.com", spfPass, dkimPass, async () => [
"v=spf1",
]);
expect(out.result).toBe("none");
expect(out.error).toContain("No DMARC record");
});
it("returns result=none when DMARC record is invalid", {
timeout: 5000,
}, async () => {
const out = await checkDmarc("x.com", spfPass, dkimPass, async () => [
"v=DMARC1",
]);
expect(out.result).toBe("none");
expect(out.error).toContain("Invalid DMARC record");
});
it("returns result=pass when SPF passes and aligns (relaxed default)", {
timeout: 5000,
}, async () => {
const out = await checkDmarc("x.com", spfPass, dkimFail, async () => [
"v=DMARC1; p=reject",
]);
expect(out.result).toBe("pass");
expect(out.spfAligned).toBe(true);
expect(out.dkimAligned).toBe(false);
});
it("returns result=pass when DKIM passes and aligns", {
timeout: 5000,
}, async () => {
const out = await checkDmarc("x.com", spfFail, dkimPass, async () => [
"v=DMARC1; p=reject",
]);
expect(out.result).toBe("pass");
expect(out.dkimAligned).toBe(true);
});
it("returns result=fail when neither alignment holds", {
timeout: 5000,
}, async () => {
const out = await checkDmarc(
"x.com",
{ result: "pass", domain: "evil.example" },
{ result: "pass", domain: "evil.example" },
async () => ["v=DMARC1; p=reject"]
);
expect(out.result).toBe("fail");
});
it("aligns via organisational domain in relaxed mode", {
timeout: 5000,
}, async () => {
const out = await checkDmarc(
"mail.x.com",
{ result: "pass", domain: "x.com" },
{ result: "fail" },
async () => ["v=DMARC1; p=reject"]
);
expect(out.spfAligned).toBe(true);
});
it("strict alignment requires exact match", { timeout: 5000 }, async () => {
const out = await checkDmarc(
"mail.x.com",
{ result: "pass", domain: "x.com" },
{ result: "fail" },
async () => ["v=DMARC1; p=reject; aspf=s"]
);
expect(out.spfAligned).toBe(false);
});
it("treats DKIM pass without domain as misaligned", {
timeout: 5000,
}, async () => {
const out = await checkDmarc(
"x.com",
{ result: "fail" },
{ result: "pass" }, // no domain
async () => ["v=DMARC1; p=reject"]
);
expect(out.dkimAligned).toBe(false);
});
it("uses fromDomain when SPF result lacks a domain", {
timeout: 5000,
}, async () => {
const out = await checkDmarc(
"x.com",
{ result: "pass" },
{ result: "fail" },
async () => ["v=DMARC1; p=none"]
);
// SPF result.domain falls back to "" so alignment fails.
expect(out.spfAligned).toBe(false);
});
it("checkAlignment short label organisational domain is returned as-is", {
timeout: 5000,
}, async () => {
const out = await checkDmarc(
"x.co",
{ result: "pass", domain: "x.co" },
{ result: "fail" },
async () => ["v=DMARC1; p=none"]
);
expect(out.spfAligned).toBe(true);
});
});

View File

@@ -0,0 +1,159 @@
/**
* DMARC policy evaluation combining SPF and DKIM results (RFC 7489).
*/
import type { DkimVerificationResult } from "../dkim-canonicalize/dkim-canonicalize.types.js";
import type { SpfCheckResult } from "../spf-checker/spf-checker.types.js";
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type {
DmarcAlignment,
DmarcCheckResult,
DmarcDnsResolver,
DmarcPolicy,
DmarcRecord,
DmarcResult,
} from "./dmarc-checker.types.js";
/** Parse a DMARC DNS TXT record string. */
export function parseDmarcRecord(txt: string): DmarcRecord | null {
const parts = txt
.split(";")
.map((s) => s.trim())
.filter(Boolean);
const tags = new Map<string, string>();
for (const part of parts) {
const eq = part.indexOf("=");
if (eq === -1) continue;
const key = part.slice(0, eq).trim().toLowerCase();
const value = part.slice(eq + 1).trim();
tags.set(key, value);
}
if (tags.get("v") !== "DMARC1") return null;
const policyStr = tags.get("p");
if (!policyStr) return null;
const policy = parsePolicy(policyStr);
if (!policy) return null;
const spTag = tags.get("sp");
const subdomainPolicy =
spTag !== undefined ? (parsePolicy(spTag) ?? policy) : undefined;
const record: DmarcRecord = {
version: "DMARC1",
policy,
dkimAlignment: tags.get("adkim") === "s" ? "strict" : "relaxed",
spfAlignment: tags.get("aspf") === "s" ? "strict" : "relaxed",
percentage: tags.has("pct")
? Number.parseInt(tags.get("pct") as string, 10)
: 100,
};
if (subdomainPolicy !== undefined) {
Object.assign(record, { subdomainPolicy });
}
const rua = tags.get("rua");
if (rua) Object.assign(record, { reportUri: rua });
const ruf = tags.get("ruf");
if (ruf) Object.assign(record, { forensicUri: ruf });
return record;
}
/** Check DMARC alignment for an email. */
export async function checkDmarc(
fromDomain: string,
spfResult: SpfCheckResult,
dkimResult: DkimVerificationResult,
dnsResolver: DmarcDnsResolver
): Promise<DmarcCheckResult> {
const dmarcDomain = `_dmarc.${fromDomain}`;
let records: string[];
try {
records = await dnsResolver(dmarcDomain);
} catch {
return reportDmarc({
result: "none",
spfAligned: false,
dkimAligned: false,
error: "DNS lookup failed",
});
}
const dmarcTxt = records.find((r) => r.startsWith("v=DMARC1"));
if (!dmarcTxt) {
return reportDmarc({
result: "none",
spfAligned: false,
dkimAligned: false,
error: "No DMARC record found",
});
}
const record = parseDmarcRecord(dmarcTxt);
if (!record) {
return reportDmarc({
result: "none",
spfAligned: false,
dkimAligned: false,
error: "Invalid DMARC record",
});
}
const spfAligned =
spfResult.result === "pass" &&
checkAlignment(fromDomain, spfResult.domain ?? "", record.spfAlignment);
const dkimAligned =
dkimResult.result === "pass" &&
checkAlignment(
fromDomain,
dkimResult.domain ?? "",
record.dkimAlignment
);
const result: DmarcResult = spfAligned || dkimAligned ? "pass" : "fail";
return reportDmarc({
result,
policy: record.policy,
spfAligned,
dkimAligned,
record,
});
}
function checkAlignment(
fromDomain: string,
authDomain: string,
mode: DmarcAlignment
): boolean {
const from = fromDomain.toLowerCase();
const auth = authDomain.toLowerCase();
if (mode === "strict") {
return from === auth;
}
return getOrgDomain(from) === getOrgDomain(auth);
}
function getOrgDomain(domain: string): string {
const parts = domain.split(".");
if (parts.length <= 2) return domain;
return parts.slice(-2).join(".");
}
function parsePolicy(value: string): DmarcPolicy | null {
const v = value.toLowerCase();
if (v === "none" || v === "quarantine" || v === "reject") return v;
return null;
}
function reportDmarc(out: DmarcCheckResult): DmarcCheckResult {
getEmailProcessingInstruments().dmarcResultCounter.add(1, {
result: out.result,
});
return out;
}

View File

@@ -0,0 +1,25 @@
export type DmarcPolicy = "none" | "quarantine" | "reject";
export type DmarcAlignment = "strict" | "relaxed";
export type DmarcResult = "pass" | "fail" | "none";
export interface DmarcRecord {
readonly version: string;
readonly policy: DmarcPolicy;
readonly subdomainPolicy?: DmarcPolicy;
readonly dkimAlignment: DmarcAlignment;
readonly spfAlignment: DmarcAlignment;
readonly percentage: number;
readonly reportUri?: string;
readonly forensicUri?: string;
}
export interface DmarcCheckResult {
readonly result: DmarcResult;
readonly policy?: DmarcPolicy;
readonly spfAligned: boolean;
readonly dkimAligned: boolean;
readonly record?: DmarcRecord;
readonly error?: string;
}
export type DmarcDnsResolver = (domain: string) => Promise<string[]>;

View File

@@ -0,0 +1,9 @@
export { checkDmarc, parseDmarcRecord } from "./dmarc-checker.js";
export type {
DmarcAlignment,
DmarcCheckResult,
DmarcDnsResolver,
DmarcPolicy,
DmarcRecord,
DmarcResult,
} from "./dmarc-checker.types.js";

View File

@@ -0,0 +1,84 @@
# error-codes
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Defines the stable error-code registry and typed error class for the
`@suaveplan/email-processing` package. Every failure raised inside the
package goes through `EmailProcessingError`, carrying a discriminant
`code` drawn from `EMAIL_PROCESSING_CODES` so callers can branch on the
error without parsing a (possibly i18n-keyed) `message` string. The
registry is closed: removing a code without removing its only call site
is forbidden by `AGENTS.md §2c`, and adding a code requires an
OpenSpec change.
## Features
- Eleven namespaced codes covering input validation, parse failures,
attachment limits, hash failures, DKIM/DMARC/SPF outcomes, calendar
parsing, hook-pipeline failures.
- `EmailProcessingError` extends `SuaveplanError<EmailProcessingCode>`
with a `public` constructor (the base constructor is `protected`).
- Compatible with `@suaveplan/error`'s i18n resolver, cause chaining,
redaction, OTel span-event adapter, and Zod wire schema.
## Usage: Basic
```ts
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "@suaveplan/email-processing/errors";
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input"
);
```
## Usage: Advanced
```ts
import { EmailProcessingError } from "@suaveplan/email-processing/errors";
try {
await parseEmail(buffer);
} catch (err) {
if (err instanceof EmailProcessingError) {
if (err.code === "EMAIL_PROCESSING.ATTACHMENT_TOO_LARGE") {
logger.warn({ code: err.code }, "reject oversized attachment");
return;
}
if (err.code === "EMAIL_PROCESSING.PARSE_FAILURE") {
throw new EmailProcessingError(
"EMAIL_PROCESSING.PARSE_FAILURE",
"email_processing.parse_failure",
{ cause: err }
);
}
}
throw err;
}
```
## API Reference
- `EMAIL_PROCESSING_CODES` — frozen literal map, value of each key is
the dotted code (e.g. `EMAIL_PROCESSING.INVALID_INPUT`).
- `EmailProcessingCode` — union of every `EMAIL_PROCESSING_CODES[K]`.
- `EmailProcessingError` — class with `public constructor(code,
messageKey, options?)` matching `SuaveplanErrorOptions`.
## Implementation Notes
`defineCodes` from `@suaveplan/error` enforces uniqueness, freezes the
output, and brands every value with the namespace prefix. The class's
`public` constructor is intentional: tests, consumers, and downstream
packages need to construct `EmailProcessingError` instances directly,
and `SuaveplanError`'s `protected` constructor would block that without
the delegating override. The `name` field is set to
`"EmailProcessingError"` so stack traces and serialised wire format
remain readable.

View File

@@ -0,0 +1,50 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EMAIL_PROCESSING_CODES, EmailProcessingError } from "./error-codes.js";
describe("EMAIL_PROCESSING_CODES", () => {
it("exposes every documented code with namespaced value", {
timeout: 5000,
}, () => {
const expected = [
"INVALID_INPUT",
"PARSE_FAILURE",
"INVALID_OPTIONS",
"DKIM_VERIFY_FAILURE",
"HOOK_FAILURE",
] as const;
for (const key of expected) {
const value =
EMAIL_PROCESSING_CODES[
key as keyof typeof EMAIL_PROCESSING_CODES
];
expect(value).toBe(`EMAIL_PROCESSING.${key}`);
}
});
});
describe("EmailProcessingError", () => {
it("constructs a typed error with code + name + message key", {
timeout: 5000,
}, () => {
const err = new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input"
);
expect(err).toBeInstanceOf(Error);
expect(err.name).toBe("EmailProcessingError");
expect(err.code).toBe("EMAIL_PROCESSING.INVALID_INPUT");
expect(typeof err.timestamp).toBe("number");
});
it("propagates cause through SuaveplanErrorOptions", {
timeout: 5000,
}, () => {
const cause = new Error("boom");
const err = new EmailProcessingError(
EMAIL_PROCESSING_CODES.PARSE_FAILURE,
"email_processing.parse_failure",
{ cause }
);
expect(err.cause).toBe(cause);
});
});

View File

@@ -0,0 +1,39 @@
import type { SuaveplanErrorOptions } from "@suaveplan/error";
import { defineCodes, SuaveplanError } from "@suaveplan/error";
/**
* Stable error-code registry for `@suaveplan/email-processing`.
*
* Every code is thrown by at least one production code path under
* `src/`. Removing an entry without removing its call site is forbidden
* by §2c of `openspec/AGENTS.md`.
*/
export const EMAIL_PROCESSING_CODES = defineCodes("EMAIL_PROCESSING", [
"INVALID_INPUT",
"PARSE_FAILURE",
"INVALID_OPTIONS",
"DKIM_VERIFY_FAILURE",
"HOOK_FAILURE",
] as const);
export type EmailProcessingCode =
(typeof EMAIL_PROCESSING_CODES)[keyof typeof EMAIL_PROCESSING_CODES];
/**
* Typed error class for every failure raised by this package.
*
* Extends `SuaveplanError` with a discriminant `code` drawn from
* `EMAIL_PROCESSING_CODES`. The constructor is `public` so callers
* outside the package can construct instances; `SuaveplanError`'s
* constructor is `protected` and would otherwise be inaccessible.
*/
export class EmailProcessingError extends SuaveplanError<EmailProcessingCode> {
public constructor(
code: EmailProcessingCode,
messageKey: string,
options?: SuaveplanErrorOptions
) {
super(code, messageKey, options);
this.name = "EmailProcessingError";
}
}

View File

@@ -0,0 +1,5 @@
export type { EmailProcessingCode } from "./error-codes.js";
export {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "./error-codes.js";

View File

@@ -0,0 +1,99 @@
# hooks
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements the package's `email:processing` hook pipeline: every parsed
email passes through a sequence of handlers that may sanitise, redact,
or enrich the parsed result. The implementation is intentionally local
to this package — the foundation `@suaveplan/events` library
does not (yet) export a shared `Hooks<T>` substrate, so a focused
async pipeline lives here. The public shape matches the upstream
`@suaveplan/email-processing` package so consumer code does not change.
## Features
- Generic `Hooks<TMap>` class with explicit type-safe `hook` /
`applyHooksAsync` / `clear` methods.
- Handlers run in `order` (default 100, lower first) and may be
sync or async — `applyHooksAsync` awaits each return value.
- Errors thrown from any handler short-circuit the pipeline and are
wrapped in an `EmailProcessingError` with code
`EMAIL_PROCESSING.HOOK_FAILURE`. The original throw is preserved
on `.cause` when it is an `Error`.
- Each invocation increments the `email_processing.hook.invocation_total`
OTel counter with `event` and `handlers` attributes.
- Returned data is threaded through every handler so transforms compose.
- Module-level `emailHooks` singleton + `onEmailProcessing` shorthand.
## Usage: Basic
```ts
import { onEmailProcessing } from "@suaveplan/email-processing/hooks";
onEmailProcessing((data) => ({
...data,
parsed: {
...data.parsed,
html: data.parsed.html?.replace(
/<img[^>]*tracking[^>]*>/gi,
""
),
},
}));
```
## Usage: Advanced
```ts
import { Hooks, type EmailHookMap } from "@suaveplan/email-processing/hooks";
const hooks = new Hooks<EmailHookMap>();
const offFirst = hooks.hook(
"email:processing",
async (d) => {
await audit.log({ messageId: d.parsed.messageId });
return d;
},
{ order: 10 }
);
const offRedact = hooks.hook(
"email:processing",
(d) => ({
...d,
parsed: { ...d.parsed, text: redact(d.parsed.text ?? "") },
}),
{ order: 200 }
);
try {
const out = await hooks.applyHooksAsync("email:processing", incoming);
return out;
} finally {
offFirst();
offRedact();
}
```
## API Reference
- `EmailProcessingHookData` — pipeline payload (`parsed`, `rawInput?`,
`metadata`).
- `EmailHookMap` — `{ "email:processing": EmailProcessingHookData }`.
- `HookHandler<T>` — `(data: T) => T | Promise<T>`.
- `HookHandlerOptions` — `{ order?: number }`.
- `Hooks<TMap>` — class with `hook`, `applyHooksAsync`, `clear`.
- `emailHooks` — singleton `Hooks<EmailHookMap>` instance.
- `onEmailProcessing(handler, options?)` — shorthand registration on
`emailHooks`; returns an idempotent unregister.
## Implementation Notes
The unregister function returned from `hook(...)` is idempotent: calling
it twice is a no-op so cleanup code in shared test fixtures can run
without ordering pitfalls. The pipeline does not call the OTel counter
when a handler throws — failures are observed via the
`EMAIL_PROCESSING.HOOK_FAILURE` error path, and a separate consumer
counter is the appropriate place to record those.

View File

@@ -0,0 +1,199 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import type { ParsedEmail } from "../types/email.js";
import {
type EmailHookMap,
emailHooks,
Hooks,
onEmailProcessing,
} from "./hooks.js";
const sampleParsed: ParsedEmail = {
headers: [],
attachments: [],
};
describe("Hooks", () => {
it("invokes registered handlers in registration order", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
const order: string[] = [];
hooks.hook("email:processing", (d) => {
order.push("a");
return d;
});
hooks.hook("email:processing", async (d) => {
order.push("b");
return d;
});
await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
expect(order).toEqual(["a", "b"]);
});
it("respects the `order` option to reorder handlers", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
const order: string[] = [];
hooks.hook(
"email:processing",
(d) => {
order.push("late");
return d;
},
{ order: 200 }
);
hooks.hook(
"email:processing",
(d) => {
order.push("early");
return d;
},
{ order: 50 }
);
await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
expect(order).toEqual(["early", "late"]);
});
it("threads transformed data through every handler", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
hooks.hook("email:processing", (d) => ({
...d,
metadata: { ...d.metadata, stage: 1 },
}));
hooks.hook("email:processing", (d) => ({
...d,
metadata: { ...d.metadata, stage: 2 },
}));
const out = await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: { stage: 0 },
});
expect(out.metadata.stage).toBe(2);
});
it("returns input data unchanged when no handlers are registered", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
const data = {
parsed: sampleParsed,
metadata: { ok: true },
};
const out = await hooks.applyHooksAsync("email:processing", data);
expect(out).toBe(data);
});
it("wraps thrown handler errors in EmailProcessingError(HOOK_FAILURE)", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
hooks.hook("email:processing", () => {
throw new Error("boom");
});
await expect(
hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
})
).rejects.toBeInstanceOf(EmailProcessingError);
});
it("wraps non-Error throws into EmailProcessingError(HOOK_FAILURE)", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
hooks.hook("email:processing", () => {
const nonErr: unknown = "string-throw";
throw nonErr;
});
await expect(
hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
})
).rejects.toMatchObject({
code: "EMAIL_PROCESSING.HOOK_FAILURE",
});
});
it("the unregister function removes the handler", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
let calls = 0;
const off = hooks.hook("email:processing", (d) => {
calls++;
return d;
});
await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
off();
// calling off again must be safe (idempotent)
off();
await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
expect(calls).toBe(1);
});
it("unregister after clear() is a safe no-op", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
const off = hooks.hook("email:processing", (d) => d);
hooks.clear();
// Now the event's registration list has been wiped; calling off()
// should be a safe no-op and exercise the fallback `?? []` arm.
expect(() => off()).not.toThrow();
});
it("clear() drops every registered handler", {
timeout: 5000,
}, async () => {
const hooks = new Hooks<EmailHookMap>();
let calls = 0;
hooks.hook("email:processing", (d) => {
calls++;
return d;
});
hooks.clear();
await hooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
expect(calls).toBe(0);
});
});
describe("onEmailProcessing + emailHooks singleton", () => {
it("registers handlers on the singleton bus", {
timeout: 5000,
}, async () => {
emailHooks.clear();
let calls = 0;
const off = onEmailProcessing((d) => {
calls++;
return d;
});
await emailHooks.applyHooksAsync("email:processing", {
parsed: sampleParsed,
metadata: {},
});
expect(calls).toBe(1);
off();
emailHooks.clear();
});
});

View File

@@ -0,0 +1,122 @@
/**
* Hook pipeline for the email-processing package.
*
* `email:processing` is a single async pipeline hook: every registered
* handler receives the running `EmailProcessingHookData` and returns a
* (possibly transformed) copy. Handlers run in registration order
* unless reordered with `order:` in their options; the first thrown
* error short-circuits the pipeline and is wrapped in an
* `EmailProcessingError` with code `HOOK_FAILURE`.
*
* The pipeline is intentionally local to this package: the foundation
* `@suaveplan/events` package in genesis does not (yet) export a
* shared `Hooks<T>` substrate, so we provide a focused implementation
* here. Public surface preserves the upstream shape:
* `EmailHookMap`, `EmailProcessingHookData`, `emailHooks`,
* `onEmailProcessing`.
*/
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type { ParsedEmail } from "../types/email.js";
/** Data flowing through the `email:processing` hook pipeline. */
export interface EmailProcessingHookData {
/** The parsed email result (mutable copy) */
parsed: ParsedEmail;
/** The raw email input as a string (when available) */
rawInput?: string;
/** Additional metadata for the processing pipeline */
metadata: Record<string, unknown>;
}
/** Hook map for email hooks. */
export interface EmailHookMap {
"email:processing": EmailProcessingHookData;
}
/** Hook handler — receives data, returns transformed data. */
export type HookHandler<T> = (data: T) => T | Promise<T>;
/** Hook handler options. */
export interface HookHandlerOptions {
/** Lower runs earlier; default 100. */
order?: number;
}
interface AnyRegistration {
handler: (data: unknown) => unknown | Promise<unknown>;
order: number;
}
/** Typed hook registry. Generic over the event map. */
export class Hooks<TMap> {
private readonly registrations = new Map<string, AnyRegistration[]>();
public hook<K extends keyof TMap & string>(
event: K,
handler: HookHandler<TMap[K]>,
options?: HookHandlerOptions
): () => void {
const list = this.registrations.get(event) ?? [];
const entry: AnyRegistration = {
handler: handler as AnyRegistration["handler"],
order: options?.order ?? 100,
};
list.push(entry);
list.sort((a, b) => a.order - b.order);
this.registrations.set(event, list);
return () => {
const current = this.registrations.get(event) ?? [];
const idx = current.indexOf(entry);
if (idx !== -1) current.splice(idx, 1);
};
}
public async applyHooksAsync<K extends keyof TMap & string>(
event: K,
data: TMap[K]
): Promise<TMap[K]> {
const list = this.registrations.get(event) ?? [];
let current: unknown = data;
for (const reg of list) {
try {
current = await reg.handler(current);
} catch (err) {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.HOOK_FAILURE,
"email_processing.hook_failure",
{ cause: err instanceof Error ? err : undefined }
);
}
}
getEmailProcessingInstruments().hookInvocationCounter.add(1, {
event,
handlers: list.length,
});
return current as TMap[K];
}
/** Test/cleanup utility: remove every registered handler. */
public clear(): void {
this.registrations.clear();
}
}
/** Singleton hooks instance for the email-processing package. */
export const emailHooks = new Hooks<EmailHookMap>();
/**
* Register a handler for the `email:processing` hook.
*
* Shorthand for `emailHooks.hook('email:processing', handler, options)`.
*/
export function onEmailProcessing(
handler: HookHandler<EmailProcessingHookData>,
options?: HookHandlerOptions
): () => void {
return emailHooks.hook("email:processing", handler, options);
}

View File

@@ -0,0 +1,7 @@
export type {
EmailHookMap,
EmailProcessingHookData,
HookHandler,
HookHandlerOptions,
} from "./hooks.js";
export { emailHooks, Hooks, onEmailProcessing } from "./hooks.js";

View File

@@ -0,0 +1,133 @@
export {
processAttachment,
processAttachments,
} from "./attachment-processor/attachment-processor.js";
export type {
AttachmentMetadata,
AttachmentProcessorOptions,
ProcessedAttachment,
} from "./attachment-processor/attachment-processor.types.js";
export { parseICalendar } from "./calendar-parser/calendar-parser.js";
export type { CalendarEvent } from "./calendar-parser/calendar-parser.types.js";
export {
extractCidReferences,
isReferencedInline,
resolveCidReferences,
} from "./cid-resolver/cid-resolver.js";
export type {
CidResolutionResult,
CidResolverOptions,
} from "./cid-resolver/cid-resolver.types.js";
export {
extractData,
extractReceiptData,
extractTrackingInfo,
} from "./data-extractor/data-extractor.js";
export type {
DeliveryTracking,
ExtractedData,
ReceiptData,
} from "./data-extractor/data-extractor.types.js";
export {
buildSignedHeaderBlock,
canonicalizeBodyRelaxed,
canonicalizeBodySimple,
canonicalizeHeaderRelaxed,
} from "./dkim-canonicalize/dkim-canonicalize.js";
export type {
DkimDnsResolver,
DkimResult,
DkimSignature,
DkimVerificationResult,
} from "./dkim-canonicalize/dkim-canonicalize.types.js";
export {
extractDkimHeader,
parseDkimSignature,
verifyDkim,
} from "./dkim-verifier/dkim-verifier.js";
export { checkDmarc, parseDmarcRecord } from "./dmarc-checker/dmarc-checker.js";
export type {
DmarcAlignment,
DmarcCheckResult,
DmarcDnsResolver,
DmarcPolicy,
DmarcRecord,
DmarcResult,
} from "./dmarc-checker/dmarc-checker.types.js";
export type { EmailProcessingCode } from "./error-codes/error-codes.js";
export {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "./error-codes/error-codes.js";
export type {
EmailHookMap,
EmailProcessingHookData,
HookHandler,
HookHandlerOptions,
} from "./hooks/hooks.js";
export { emailHooks, Hooks, onEmailProcessing } from "./hooks/hooks.js";
export {
buildHeaderParam,
buildHeaderValue,
decodeFlowed,
decodeHeader,
decodeHeaders,
decodeWords,
encodeFlowed,
encodeWord,
encodeWords,
foldLines,
parseHeaderValue,
} from "./mime-utils/mime-utils.js";
export {
decodeEmailWords,
parseEmail,
parseEmailAddress,
} from "./parser/parser.js";
export { DEFAULT_SPAM_RULES, scoreSpam } from "./spam-scorer/spam-scorer.js";
export type {
SpamContext,
SpamRule,
SpamScoreResult,
SpamScorerOptions,
} from "./spam-scorer/spam-scorer.types.js";
export {
checkSpf,
ipv4Matches,
ipv6Matches,
parseSpfRecord,
} from "./spf-checker/spf-checker.js";
export type {
SpfCheckResult,
SpfDnsResolver,
SpfMechanism,
SpfResult,
} from "./spf-checker/spf-checker.types.js";
export type { EmailProcessingInstruments } from "./telemetry/telemetry.js";
export {
getEmailProcessingInstruments,
resetEmailProcessingInstruments,
} from "./telemetry/telemetry.js";
export {
groupThreads,
isSubjectChange,
normalizeSubject,
parseReferences,
} from "./thread-grouper/thread-grouper.js";
export type {
EmailThread,
ThreadableEmail,
ThreadNode,
} from "./thread-grouper/thread-grouper.types.js";
export type {
EmailAddress,
EmailAddressGroup,
EmailAttachment,
EmailHeader,
EmailMailbox,
EmailParseOptions,
HeaderParam,
MimeWordEncoding,
ParsedEmail,
StructuredHeaderValue,
} from "./types/email.js";

View File

@@ -0,0 +1,13 @@
export {
buildHeaderParam,
buildHeaderValue,
decodeFlowed,
decodeHeader,
decodeHeaders,
decodeWords,
encodeFlowed,
encodeWord,
encodeWords,
foldLines,
parseHeaderValue,
} from "./mime-utils.js";

View File

@@ -0,0 +1,67 @@
/**
* Minimal ambient declaration for `libmime`. The upstream package ships
* without TypeScript declarations; this file constrains the subset our
* wrapper exposes so we avoid `any` leaks and `// @ts-expect-error`
* suppressions in production code.
*/
declare module "libmime" {
export interface StructuredHeaderValueRaw {
value: string;
params: Record<string, string>;
}
export interface HeaderParamRaw {
key: string;
value: string;
}
export interface DecodedHeader {
key: string;
value: string;
}
export function encodeWord(
str: string,
encoding?: "Q" | "B",
maxLength?: number
): string;
export function encodeWords(
str: string,
encoding?: "Q" | "B",
maxLength?: number
): string;
export function decodeWords(str: string): string;
export function decodeHeader(headerLine: string): DecodedHeader;
export function decodeHeaders(headers: string): Record<string, string[]>;
export function parseHeaderValue(
valueString: string
): StructuredHeaderValueRaw;
export function buildHeaderValue(
structured: StructuredHeaderValueRaw
): string;
export function buildHeaderParam(
key: string,
value: string,
maxLength?: number
): HeaderParamRaw[];
export function foldLines(
str: string,
lineLength?: number,
afterSpace?: boolean
): string;
export function encodeFlowed(str: string, lineLength?: number): string;
export function decodeFlowed(str: string, delSp?: boolean): string;
const libmime: {
encodeWord: typeof encodeWord;
encodeWords: typeof encodeWords;
decodeWords: typeof decodeWords;
decodeHeader: typeof decodeHeader;
decodeHeaders: typeof decodeHeaders;
parseHeaderValue: typeof parseHeaderValue;
buildHeaderValue: typeof buildHeaderValue;
buildHeaderParam: typeof buildHeaderParam;
foldLines: typeof foldLines;
encodeFlowed: typeof encodeFlowed;
decodeFlowed: typeof decodeFlowed;
};
export default libmime;
}

View File

@@ -0,0 +1,87 @@
# mime-utils
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Wraps the `libmime` library with a strict, typed boundary so the rest
of the package can call RFC 2047 / 2231 / 5322 encoding and decoding
helpers without using `// @ts-expect-error` or leaking `any` through
the consumer surface. Every helper validates its primary string
argument and raises an `EmailProcessingError` with code
`EMAIL_PROCESSING.INVALID_INPUT` rather than silently calling `libmime`
with garbage.
## Features
- Eleven wrappers: `encodeWord` / `encodeWords` / `decodeWords`,
`decodeHeader` / `decodeHeaders`, `parseHeaderValue` /
`buildHeaderValue` / `buildHeaderParam`, `foldLines`,
`encodeFlowed` / `decodeFlowed`.
- Local ambient declaration (`libmime.d.ts`) so the package compiles
without `@types/libmime` while keeping the inner types narrow.
- Strict input validation at every entry point — non-string args
raise `INVALID_INPUT`, which is testable and observable.
- Pass-through of optional parameters such as `maxLength`,
`lineLength`, `afterSpace`, `delSp` so callers retain full
`libmime` flexibility.
## Usage: Basic
```ts
import { encodeWord, decodeWords } from "@suaveplan/email-processing/mime-utils";
const encoded = encodeWord("résumé", "Q");
const original = decodeWords(encoded);
```
## Usage: Advanced
```ts
import {
parseHeaderValue,
buildHeaderValue,
buildHeaderParam,
foldLines,
} from "@suaveplan/email-processing/mime-utils";
const parsed = parseHeaderValue('text/plain; charset="utf-8"; name="r.txt"');
const segments = buildHeaderParam("filename*", "réport-2024.pdf", 60);
parsed.params.filename = segments.map((s) => s.value).join("");
const header = foldLines(
`Content-Type: ${buildHeaderValue(parsed)}`,
76,
true
);
```
## API Reference
- `encodeWord(str, encoding?, maxLength?)`
- `encodeWords(str, encoding?, maxLength?)`
- `decodeWords(str)`
- `decodeHeader(headerLine)` → `{ key, value }`
- `decodeHeaders(headers)` → `Record<string, string[]>`
- `parseHeaderValue(valueString)` → `StructuredHeaderValue`
- `buildHeaderValue(structured)` → `string`
- `buildHeaderParam(key, value, maxLength?)` → `HeaderParam[]`
- `foldLines(str, lineLength?, afterSpace?)`
- `encodeFlowed(str, lineLength?)`
- `decodeFlowed(str, delSp?)`
All throw `EmailProcessingError` with code
`EMAIL_PROCESSING.INVALID_INPUT` when their primary argument is missing
or of the wrong type.
## Implementation Notes
`libmime` ships without TypeScript declarations, so this module owns a
local `libmime.d.ts` declaring only the subset of functions in use.
That keeps the package compatible with strict `verbatimModuleSyntax`
and `noUncheckedIndexedAccess` settings without `any` casts. The
`buildHeaderValue` guard checks for `null`/`undefined`/non-object
rather than `typeof === "string"` because the input is structured —
mismatching that guard would let through invalid shapes.

View File

@@ -0,0 +1,150 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import {
buildHeaderParam,
buildHeaderValue,
decodeFlowed,
decodeHeader,
decodeHeaders,
decodeWords,
encodeFlowed,
encodeWord,
encodeWords,
foldLines,
parseHeaderValue,
} from "./mime-utils.js";
describe("mime-utils", () => {
it("encodeWord round-trips through decodeWords for non-ASCII", {
timeout: 5000,
}, () => {
const encoded = encodeWord("résumé", "Q");
expect(encoded).toContain("=?");
expect(decodeWords(encoded)).toBe("résumé");
});
it("encodeWord defaults to Q encoding", { timeout: 5000 }, () => {
const encoded = encodeWord("hello");
expect(typeof encoded).toBe("string");
});
it("encodeWord accepts maxLength", { timeout: 5000 }, () => {
const encoded = encodeWord("hello world hello world", "Q", 32);
expect(typeof encoded).toBe("string");
});
it("encodeWords encodes only non-ASCII regions", { timeout: 5000 }, () => {
const out = encodeWords("Hello résumé World", "B");
expect(out).toContain("Hello");
expect(out).toContain("=?");
});
it("decodeWords returns plain strings unchanged", { timeout: 5000 }, () => {
expect(decodeWords("just text")).toBe("just text");
});
it("decodeHeader splits a header line", { timeout: 5000 }, () => {
const out = decodeHeader("Subject: Hello there");
expect(out.key.toLowerCase()).toBe("subject");
expect(out.value).toBe("Hello there");
});
it("decodeHeaders parses multi-line header blocks", {
timeout: 5000,
}, () => {
const headers = "From: a@x\r\nTo: b@y\r\nSubject: hi\r\n";
const out = decodeHeaders(headers);
expect(out.from).toEqual(["a@x"]);
expect(out.subject).toEqual(["hi"]);
});
it("parseHeaderValue + buildHeaderValue round-trip", {
timeout: 5000,
}, () => {
const v = parseHeaderValue("text/plain; charset=utf-8");
expect(v.value).toBe("text/plain");
expect(v.params.charset).toBe("utf-8");
const rebuilt = buildHeaderValue(v);
expect(rebuilt).toContain("text/plain");
expect(rebuilt).toContain("charset");
});
it("buildHeaderParam returns a list of key-value segments", {
timeout: 5000,
}, () => {
const out = buildHeaderParam("filename", "réport.pdf");
expect(Array.isArray(out)).toBe(true);
expect(out[0]?.key).toBeDefined();
});
it("foldLines folds long header lines", { timeout: 5000 }, () => {
const long = "x".repeat(200);
const folded = foldLines(`Subject: ${long}`, 76);
expect(folded.length).toBeGreaterThan(0);
});
it("foldLines accepts afterSpace=true variant", { timeout: 5000 }, () => {
const folded = foldLines("Subject: hello world hello world", 16, true);
expect(typeof folded).toBe("string");
});
it("encodeFlowed + decodeFlowed round-trip", { timeout: 5000 }, () => {
const flowed = encodeFlowed(
"this is a long line that will be soft wrapped"
);
const decoded = decodeFlowed(flowed);
expect(decoded).toContain("this is a long line");
});
it("encodeFlowed accepts custom line length", { timeout: 5000 }, () => {
const flowed = encodeFlowed("hello world hello world", 12);
expect(typeof flowed).toBe("string");
});
it("decodeFlowed accepts delSp option", { timeout: 5000 }, () => {
const decoded = decodeFlowed("hello \r\nworld", true);
expect(typeof decoded).toBe("string");
});
});
describe("mime-utils input validation", () => {
const cases: Array<[string, () => unknown]> = [
["encodeWord", () => encodeWord(undefined as unknown as string)],
["encodeWords", () => encodeWords(undefined as unknown as string)],
["decodeWords", () => decodeWords(undefined as unknown as string)],
["decodeHeader", () => decodeHeader(undefined as unknown as string)],
["decodeHeaders", () => decodeHeaders(undefined as unknown as string)],
[
"parseHeaderValue",
() => parseHeaderValue(undefined as unknown as string),
],
[
"buildHeaderValue",
() =>
buildHeaderValue(
null as unknown as {
value: string;
params: Record<string, string>;
}
),
],
[
"buildHeaderParam-key",
() => buildHeaderParam(undefined as unknown as string, "v"),
],
[
"buildHeaderParam-value",
() => buildHeaderParam("k", undefined as unknown as string),
],
["foldLines", () => foldLines(undefined as unknown as string)],
["encodeFlowed", () => encodeFlowed(undefined as unknown as string)],
["decodeFlowed", () => decodeFlowed(undefined as unknown as string)],
];
for (const [name, fn] of cases) {
it(`${name} rejects non-string input with INVALID_INPUT`, {
timeout: 5000,
}, () => {
expect(fn).toThrow(EmailProcessingError);
});
}
});

View File

@@ -0,0 +1,130 @@
import libmime from "libmime";
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import type {
HeaderParam,
MimeWordEncoding,
StructuredHeaderValue,
} from "../types/email.js";
function requireString(value: unknown, label: string): string {
if (typeof value !== "string") {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
`email_processing.invalid_input.${label}`
);
}
return value;
}
/**
* Encode a string as a MIME encoded-word (RFC 2047).
*/
export function encodeWord(
str: string,
encoding: MimeWordEncoding = "Q",
maxLength?: number
): string {
return libmime.encodeWord(requireString(str, "str"), encoding, maxLength);
}
/**
* Encode only non-ASCII sequences within a string as MIME words.
*/
export function encodeWords(
str: string,
encoding: MimeWordEncoding = "Q",
maxLength?: number
): string {
return libmime.encodeWords(requireString(str, "str"), encoding, maxLength);
}
/**
* Decode a string containing MIME encoded-words back to plain text.
*/
export function decodeWords(str: string): string {
return libmime.decodeWords(requireString(str, "str"));
}
/**
* Unfold and split a single header line into key and value.
*/
export function decodeHeader(headerLine: string): {
key: string;
value: string;
} {
return libmime.decodeHeader(requireString(headerLine, "headerLine"));
}
/**
* Parse a block of raw header lines into a structured object.
*/
export function decodeHeaders(headers: string): Record<string, string[]> {
return libmime.decodeHeaders(requireString(headers, "headers"));
}
/**
* Parse a header value with key=value parameters into a structured object.
*/
export function parseHeaderValue(valueString: string): StructuredHeaderValue {
return libmime.parseHeaderValue(requireString(valueString, "valueString"));
}
/**
* Reconstruct a header value string from a structured header object.
*/
export function buildHeaderValue(structured: StructuredHeaderValue): string {
if (
structured === null ||
structured === undefined ||
typeof structured !== "object"
) {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input.structured"
);
}
return libmime.buildHeaderValue(structured);
}
/**
* Encode and split a header parameter value per RFC 2231.
*/
export function buildHeaderParam(
key: string,
value: string,
maxLength?: number
): HeaderParam[] {
return libmime.buildHeaderParam(
requireString(key, "key"),
requireString(value, "value"),
maxLength
);
}
/**
* Fold long header lines per RFC 5322 (default 76 chars).
*/
export function foldLines(
str: string,
lineLength?: number,
afterSpace?: boolean
): string {
return libmime.foldLines(requireString(str, "str"), lineLength, afterSpace);
}
/**
* Add soft line breaks for format=flowed content.
*/
export function encodeFlowed(str: string, lineLength?: number): string {
return libmime.encodeFlowed(requireString(str, "str"), lineLength);
}
/**
* Unwrap format=flowed text, removing soft line breaks.
*/
export function decodeFlowed(str: string, delSp?: boolean): string {
return libmime.decodeFlowed(requireString(str, "str"), delSp);
}

View File

@@ -0,0 +1 @@
export { decodeEmailWords, parseEmail, parseEmailAddress } from "./parser.js";

View File

@@ -0,0 +1,85 @@
# parser
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Wraps `postal-mime` with a strict input/option boundary, OTel
instrumentation, structured error codes, and the
`@suaveplan/email-processing/hooks` pipeline. `parseEmail` is the
package's primary entry point: it takes a raw RFC 822 message in any
of the three supported input shapes (string, `ArrayBuffer`,
`Uint8Array`) and returns a fully-parsed `ParsedEmail` whose
attachments are real MIME parts (no stubs).
## Features
- Three input modes — string, `ArrayBuffer`, `Uint8Array` — covering
both browser and Node consumers.
- Zod-validated options at the boundary, raising
`EMAIL_PROCESSING.INVALID_OPTIONS` with the failing issues on
`context` so callers can surface diagnostics.
- Distinct error codes for `INVALID_INPUT` (null/undefined/wrong type)
and `PARSE_FAILURE` (`postal-mime` threw); `parse.failure_total`
is incremented for the latter.
- Hook pipeline integration — every parse returns the result of
`emailHooks.applyHooksAsync("email:processing", …)`.
- `parseEmailAddress` and `decodeEmailWords` re-export the
`postal-mime` low-level helpers under the package's typed surface,
with the same `INVALID_INPUT` guard.
## Usage: Basic
```ts
import { parseEmail } from "@suaveplan/email-processing/parser";
const parsed = await parseEmail(rawMimeString);
console.log(parsed.subject, parsed.attachments.length);
```
## Usage: Advanced
```ts
import { parseEmail } from "@suaveplan/email-processing/parser";
import { onEmailProcessing } from "@suaveplan/email-processing/hooks";
const off = onEmailProcessing((data) => ({
...data,
parsed: {
...data.parsed,
html: data.parsed.html?.replace(/<script[\s\S]*?<\/script>/gi, ""),
},
}));
try {
const parsed = await parseEmail(buffer, {
attachmentEncoding: "base64",
maxNestingDepth: 64,
});
await persist(parsed);
} finally {
off();
}
```
## API Reference
- `parseEmail(raw, options?)` — async; returns `ParsedEmail`. Throws
`EmailProcessingError` with codes `INVALID_INPUT`,
`INVALID_OPTIONS`, or `PARSE_FAILURE`.
- `parseEmailAddress(addressString)` — sync; returns `EmailAddress[]`.
Throws `EmailProcessingError(INVALID_INPUT)` for non-string input.
- `decodeEmailWords(text)` — sync; returns decoded RFC 2047 string.
Throws `EmailProcessingError(INVALID_INPUT)` for non-string input.
## Implementation Notes
`parseEmail` records `email_processing.parse.total` with the attachment
count as an attribute so dashboards can surface average
attachments-per-message. Parse failures increment a separate
`parse.failure_total` counter (no attachment count is recorded because
the parser never reached the attachment array). The Zod options schema
is `.strict()` so unknown option keys are rejected — this catches
typos at the boundary instead of letting them silently no-op.

View File

@@ -0,0 +1,201 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { EmailProcessingError } from "../error-codes/error-codes.js";
import { emailHooks } from "../hooks/hooks.js";
import { decodeEmailWords, parseEmail, parseEmailAddress } from "./parser.js";
const SIMPLE_EMAIL = [
"From: Alice <alice@example.com>",
'To: "Bob, Robert" <bob@example.com>',
"Subject: =?UTF-8?B?SGVsbG8gV29ybGQ=?=",
"Message-ID: <abc@example.com>",
"Date: Wed, 01 Jan 2025 12:00:00 +0000",
"MIME-Version: 1.0",
'Content-Type: multipart/alternative; boundary="b1"',
"",
"--b1",
'Content-Type: text/plain; charset="utf-8"',
"",
"Hello plain world",
"--b1",
'Content-Type: text/html; charset="utf-8"',
"",
"<p>Hello HTML world</p>",
"--b1--",
"",
].join("\r\n");
const EMAIL_WITH_ATTACHMENT = [
"From: sender@example.com",
"To: rcpt@example.com",
"Subject: Attachment test",
"MIME-Version: 1.0",
'Content-Type: multipart/mixed; boundary="m1"',
"",
"--m1",
'Content-Type: text/plain; charset="utf-8"',
"",
"Body text",
"--m1",
'Content-Type: text/plain; charset="utf-8"; name="note.txt"',
'Content-Disposition: attachment; filename="note.txt"',
"Content-Transfer-Encoding: base64",
"",
"SGVsbG8gQXR0YWNobWVudA==",
"--m1--",
"",
].join("\r\n");
describe("parseEmail", () => {
it("parses headers, From, To, Subject and bodies from a real MIME message", {
timeout: 5000,
}, async () => {
emailHooks.clear();
const out = await parseEmail(SIMPLE_EMAIL);
expect(out.subject).toBe("Hello World");
expect(out.text?.trim()).toContain("Hello plain world");
expect(out.html).toContain("<p>Hello HTML world</p>");
expect(out.messageId).toContain("abc@example.com");
expect(Array.isArray(out.headers)).toBe(true);
const from = out.from as { address: string } | undefined;
expect(from?.address).toBe("alice@example.com");
});
it("extracts attachments with content + filename + mime type", {
timeout: 5000,
}, async () => {
emailHooks.clear();
const out = await parseEmail(EMAIL_WITH_ATTACHMENT);
expect(out.attachments).toHaveLength(1);
const att = out.attachments[0]!;
expect(att.filename).toBe("note.txt");
expect(att.mimeType).toBe("text/plain");
});
it("accepts ArrayBuffer input", { timeout: 5000 }, async () => {
emailHooks.clear();
const enc = new TextEncoder().encode(SIMPLE_EMAIL);
const out = await parseEmail(enc.buffer);
expect(out.subject).toBe("Hello World");
});
it("accepts Uint8Array input", { timeout: 5000 }, async () => {
emailHooks.clear();
const bytes = new TextEncoder().encode(SIMPLE_EMAIL);
const out = await parseEmail(bytes);
expect(out.subject).toBe("Hello World");
});
it("runs the email:processing hook pipeline and propagates transforms", {
timeout: 5000,
}, async () => {
emailHooks.clear();
const off = emailHooks.hook("email:processing", (d) => ({
...d,
parsed: { ...d.parsed, subject: "REDACTED" },
}));
const out = await parseEmail(SIMPLE_EMAIL);
off();
expect(out.subject).toBe("REDACTED");
});
it("throws INVALID_INPUT for null/undefined raw input", {
timeout: 5000,
}, async () => {
emailHooks.clear();
await expect(
parseEmail(null as unknown as string)
).rejects.toMatchObject({
code: "EMAIL_PROCESSING.INVALID_INPUT",
});
await expect(
parseEmail(undefined as unknown as string)
).rejects.toMatchObject({
code: "EMAIL_PROCESSING.INVALID_INPUT",
});
});
it("throws INVALID_INPUT for non-string / non-buffer input", {
timeout: 5000,
}, async () => {
emailHooks.clear();
await expect(
parseEmail(42 as unknown as string)
).rejects.toBeInstanceOf(EmailProcessingError);
});
it("throws INVALID_OPTIONS when options fail Zod validation", {
timeout: 5000,
}, async () => {
emailHooks.clear();
await expect(
parseEmail(SIMPLE_EMAIL, {
attachmentEncoding: "not-a-real-encoding",
} as unknown as Record<string, never>)
).rejects.toMatchObject({
code: "EMAIL_PROCESSING.INVALID_OPTIONS",
});
});
it("accepts valid options unchanged", { timeout: 5000 }, async () => {
emailHooks.clear();
const out = await parseEmail(SIMPLE_EMAIL, {
attachmentEncoding: "utf8",
});
expect(out.subject).toBe("Hello World");
});
it("wraps postal-mime failures in PARSE_FAILURE", {
timeout: 5000,
}, async () => {
emailHooks.clear();
// maxHeadersSize: 1 forces postal-mime to throw on the first header
// larger than 1 byte — every real message qualifies.
await expect(
parseEmail(SIMPLE_EMAIL, { maxHeadersSize: 1 })
).rejects.toMatchObject({
code: "EMAIL_PROCESSING.PARSE_FAILURE",
});
});
});
describe("parseEmailAddress", () => {
it("parses a single address with display name", { timeout: 5000 }, () => {
const out = parseEmailAddress('"John Doe" <john@example.com>');
expect(out).toHaveLength(1);
const m = out[0] as { name: string; address: string };
expect(m.name).toBe("John Doe");
expect(m.address).toBe("john@example.com");
});
it("parses multiple addresses", { timeout: 5000 }, () => {
const out = parseEmailAddress("a@x.com, b@y.com");
expect(out).toHaveLength(2);
});
it("throws INVALID_INPUT for non-string input", { timeout: 5000 }, () => {
expect(() => parseEmailAddress(undefined as unknown as string)).toThrow(
EmailProcessingError
);
});
});
describe("decodeEmailWords", () => {
it("decodes Q-encoded text", { timeout: 5000 }, () => {
expect(decodeEmailWords("=?UTF-8?Q?Hello=20World?=")).toBe(
"Hello World"
);
});
it("decodes B-encoded text", { timeout: 5000 }, () => {
expect(decodeEmailWords("=?UTF-8?B?SGVsbG8gV29ybGQ=?=")).toBe(
"Hello World"
);
});
it("returns plain text unchanged", { timeout: 5000 }, () => {
expect(decodeEmailWords("just text")).toBe("just text");
});
it("throws INVALID_INPUT for non-string", { timeout: 5000 }, () => {
expect(() => decodeEmailWords(123 as unknown as string)).toThrow(
EmailProcessingError
);
});
});

View File

@@ -0,0 +1,120 @@
import PostalMime, { addressParser, decodeWords } from "postal-mime";
import z from "zod";
import {
EMAIL_PROCESSING_CODES,
EmailProcessingError,
} from "../error-codes/error-codes.js";
import { emailHooks } from "../hooks/hooks.js";
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type {
EmailAddress,
EmailParseOptions,
ParsedEmail,
} from "../types/email.js";
const ParseOptionsSchema = z
.object({
rfc822Attachments: z.boolean().optional(),
forceRfc822Attachments: z.boolean().optional(),
attachmentEncoding: z
.enum(["base64", "utf8", "arraybuffer"])
.optional(),
maxNestingDepth: z.number().int().positive().optional(),
maxHeadersSize: z.number().int().positive().optional(),
})
.strict();
/**
* Parse a raw RFC822 email message into a structured object.
* Supports string, ArrayBuffer, and Uint8Array inputs.
*
* The parsed result is passed through the `email:processing` hook
* pipeline before being returned, so registered transforms (e.g.
* redaction, sanitisation) take effect transparently.
*/
export async function parseEmail(
rawEmail: string | ArrayBuffer | Uint8Array,
options?: EmailParseOptions
): Promise<ParsedEmail> {
if (
rawEmail === undefined ||
rawEmail === null ||
(typeof rawEmail !== "string" &&
!(rawEmail instanceof ArrayBuffer) &&
!(rawEmail instanceof Uint8Array))
) {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input"
);
}
if (options !== undefined) {
const validated = ParseOptionsSchema.safeParse(options);
if (!validated.success) {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_OPTIONS,
"email_processing.invalid_options",
{ context: { issues: validated.error.issues } }
);
}
}
const instruments = getEmailProcessingInstruments();
let result: ParsedEmail;
try {
const raw = await PostalMime.parse(rawEmail, options);
result = raw as unknown as ParsedEmail;
} catch (err) {
instruments.parseFailureCounter.add(1);
// postal-mime rejects only with Error subclasses.
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.PARSE_FAILURE,
"email_processing.parse_failure",
{ cause: err as Error }
);
}
instruments.parseCounter.add(1, {
"attachment.count": result.attachments.length,
});
const hookData: {
parsed: ParsedEmail;
rawInput?: string;
metadata: Record<string, unknown>;
} = {
parsed: result,
metadata: {},
};
if (typeof rawEmail === "string") hookData.rawInput = rawEmail;
const hookResult = await emailHooks.applyHooksAsync(
"email:processing",
hookData
);
return hookResult.parsed;
}
/**
* Parse an email address string into structured address objects.
* Handles individual addresses, display names, and group syntax.
*/
export function parseEmailAddress(address: string): EmailAddress[] {
if (typeof address !== "string") {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input"
);
}
return addressParser(address) as unknown as EmailAddress[];
}
/**
* Decode MIME encoded-word values (RFC 2047) in a string.
* Handles both Q-encoding and B-encoding (base64).
*/
export function decodeEmailWords(text: string): string {
if (typeof text !== "string") {
throw new EmailProcessingError(
EMAIL_PROCESSING_CODES.INVALID_INPUT,
"email_processing.invalid_input"
);
}
return decodeWords(text);
}

View File

@@ -0,0 +1,7 @@
export { DEFAULT_SPAM_RULES, scoreSpam } from "./spam-scorer.js";
export type {
SpamContext,
SpamRule,
SpamScoreResult,
SpamScorerOptions,
} from "./spam-scorer.types.js";

View File

@@ -0,0 +1,97 @@
# spam-scorer
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Heuristic spam scorer with a pluggable rule set. The default rules
cover the most common spam signals — missing headers, all-caps
subjects, common keyword lists, URL counts, low text-to-HTML ratio,
DKIM / SPF authentication outcomes — and produce a clamped 0..10
score plus a `clean` / `suspicious` / `spam` classification. Custom
rules can be merged into the defaults, with conflict resolution by
`id`.
## Features
- Twelve default rules covering header completeness, content, and
authentication signals.
- Configurable suspicious / spam thresholds (defaults: 3 / 6).
- Custom rules merge with defaults via `id`; same-id custom rules
replace the default in place, preserving merge order.
- Records `email_processing.spam.scored_total` attributed by
`classification`.
- Pure: no DNS / network calls, fully deterministic given a fixed
`SpamContext`.
## Usage: Basic
```ts
import { scoreSpam } from "@suaveplan/email-processing/spam";
const result = scoreSpam({
subject: parsed.subject,
text: parsed.text,
html: parsed.html,
fromAddress: (parsed.from as { address?: string } | undefined)?.address,
headers: parsed.headers,
attachmentCount: parsed.attachments.length,
});
if (result.classification === "spam") quarantine();
```
## Usage: Advanced
```ts
import {
scoreSpam,
DEFAULT_SPAM_RULES,
type SpamRule,
} from "@suaveplan/email-processing/spam";
const customRules: SpamRule[] = [
{
id: "competitor-domain",
description: "From a known competitor domain",
weight: 5,
evaluate: (ctx) =>
ctx.fromAddress?.endsWith("@competitor.example") ?? false,
},
];
const result = scoreSpam(
{
subject: msg.subject,
text: msg.text,
html: msg.html,
fromAddress: msg.from,
headers: msg.headers,
attachmentCount: msg.attachments.length,
dkimResult: dkim.result,
spfResult: spf.result,
},
{ customRules, suspiciousThreshold: 2, spamThreshold: 5 }
);
```
## API Reference
- `SpamRule` — `{ id, description, weight, evaluate(ctx) }`.
- `SpamContext` — full payload passed to each rule.
- `SpamScoreResult` — `{ score, triggeredRules, classification }`.
- `SpamScorerOptions` — `{ customRules?, suspiciousThreshold?, spamThreshold? }`.
- `DEFAULT_SPAM_RULES` — array of the twelve built-in rules.
- `scoreSpam(context, options?)` — pure scorer.
## Implementation Notes
The scorer clamps the raw weighted sum at `[0, 10]` so callers can
rely on a fixed range when piping into a UI badge or alerting
threshold. Rule weights are tuned against a hand-classified corpus
and intentionally favour precision over recall — false-positive
quarantines are more disruptive than false-negative deliveries.
Custom rules with same `id` replace the default rule entirely (no
weight-sum merge) so consumers retain full control over which rules
fire.

View File

@@ -0,0 +1,238 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import { DEFAULT_SPAM_RULES, scoreSpam } from "./spam-scorer.js";
import type { SpamContext, SpamRule } from "./spam-scorer.types.js";
function baseCtx(over: Partial<SpamContext> = {}): SpamContext {
return {
subject: "Hello",
text: "Hi there",
fromAddress: "alice@example.com",
headers: [
{ key: "Date", value: "now" },
{ key: "Message-ID", value: "<x@y>" },
],
attachmentCount: 0,
...over,
};
}
describe("scoreSpam — default rules", () => {
it("returns score=0 + classification=clean for a typical clean email", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx());
expect(out.score).toBe(0);
expect(out.classification).toBe("clean");
expect(out.triggeredRules).toEqual([]);
});
it("triggers missing-from rule", { timeout: 5000 }, () => {
const out = scoreSpam(baseCtx({ fromAddress: undefined }));
expect(out.triggeredRules.map((r) => r.id)).toContain("missing-from");
});
it("triggers missing-subject + missing-date + missing-message-id", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx({ subject: "", headers: [] }));
const ids = out.triggeredRules.map((r) => r.id);
expect(ids).toContain("missing-subject");
expect(ids).toContain("missing-date");
expect(ids).toContain("missing-message-id");
});
it("triggers all-caps-subject when subject is uppercase", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx({ subject: "HELLO WORLD!" }));
expect(out.triggeredRules.map((r) => r.id)).toContain(
"all-caps-subject"
);
});
it("does not trigger all-caps-subject for short or mixed-case subjects", {
timeout: 5000,
}, () => {
expect(
scoreSpam(baseCtx({ subject: "OK" })).triggeredRules.map(
(r) => r.id
)
).not.toContain("all-caps-subject");
expect(
scoreSpam(baseCtx({ subject: "Hello world!" })).triggeredRules.map(
(r) => r.id
)
).not.toContain("all-caps-subject");
});
it("triggers spam-keywords from text body", { timeout: 5000 }, () => {
const out = scoreSpam(baseCtx({ text: "ACT NOW! Free money!" }));
expect(out.triggeredRules.map((r) => r.id)).toContain("spam-keywords");
});
it("triggers spam-keywords from subject when text and subject undefined paths", {
timeout: 5000,
}, () => {
const out = scoreSpam(
baseCtx({ text: undefined, subject: "CASINO winner" })
);
expect(out.triggeredRules.map((r) => r.id)).toContain("spam-keywords");
});
it("does not crash when both text and subject are undefined", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx({ text: undefined, subject: undefined }));
// spam-keywords does not fire; the body is two empty strings joined by a space.
expect(out.triggeredRules.map((r) => r.id)).not.toContain(
"spam-keywords"
);
});
it("triggers excessive-urls from text body", { timeout: 5000 }, () => {
const urls = Array.from(
{ length: 11 },
(_, i) => `https://example.com/${i}`
).join(" ");
const out = scoreSpam(baseCtx({ text: urls }));
expect(out.triggeredRules.map((r) => r.id)).toContain("excessive-urls");
});
it("triggers excessive-urls from html body", { timeout: 5000 }, () => {
const urls = Array.from(
{ length: 20 },
(_, i) => `<a href="https://example.com/${i}">x</a>`
).join("");
const out = scoreSpam(baseCtx({ html: urls, text: "" }));
const ids = out.triggeredRules.map((r) => r.id);
expect(ids).toContain("excessive-urls");
});
it("triggers high-html-ratio", { timeout: 5000 }, () => {
const html = "<div><div><div><div><div></div></div></div></div></div>";
const out = scoreSpam(baseCtx({ html }));
expect(out.triggeredRules.map((r) => r.id)).toContain(
"high-html-ratio"
);
});
it("htmlRatio(0-length text part) considers html length", {
timeout: 5000,
}, () => {
// html with text length 0 should yield ratio 1 (handled internally)
const out = scoreSpam(baseCtx({ html: "<p></p>" }));
// no other html-heavy signals here, but the rule should now fire.
expect(out.triggeredRules.map((r) => r.id)).toContain(
"high-html-ratio"
);
});
it("htmlRatio('') returns 0 / does not trigger high-html-ratio", {
timeout: 5000,
}, () => {
// Pass html: "" — Boolean("") is false so the rule short-circuits.
const out = scoreSpam(baseCtx({ html: "" }));
expect(out.triggeredRules.map((r) => r.id)).not.toContain(
"high-html-ratio"
);
});
it("triggers no-text-part when html present and text absent", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx({ html: "<p>hi</p>", text: undefined }));
expect(out.triggeredRules.map((r) => r.id)).toContain("no-text-part");
});
it("triggers dkim-fail / spf-fail / spf-softfail with appropriate weights", {
timeout: 5000,
}, () => {
const out = scoreSpam(
baseCtx({ dkimResult: "fail", spfResult: "fail" })
);
const ids = out.triggeredRules.map((r) => r.id);
expect(ids).toContain("dkim-fail");
expect(ids).toContain("spf-fail");
const sf = scoreSpam(baseCtx({ spfResult: "softfail" }));
expect(sf.triggeredRules.map((r) => r.id)).toContain("spf-softfail");
});
});
describe("scoreSpam — classification thresholds", () => {
it("clamps the raw score at 10", { timeout: 5000 }, () => {
const out = scoreSpam(
baseCtx({
fromAddress: undefined,
subject: "",
headers: [],
dkimResult: "fail",
spfResult: "fail",
html: "<p>hi</p>",
text: undefined,
})
);
expect(out.score).toBe(10);
expect(out.classification).toBe("spam");
});
it("classifies suspicious when score is in mid range", {
timeout: 5000,
}, () => {
const out = scoreSpam(baseCtx({ fromAddress: undefined }));
// missing-from weight 2.0; default suspiciousThreshold 3 not met.
// Add another rule trigger:
const out2 = scoreSpam(
baseCtx({ fromAddress: undefined, subject: "" })
);
expect(out2.score).toBeGreaterThanOrEqual(3);
expect(out2.classification).toBe("suspicious");
// unused 'out' suppression
expect(out.classification).toBe("clean");
});
it("respects custom thresholds", { timeout: 5000 }, () => {
const out = scoreSpam(baseCtx({ fromAddress: undefined }), {
suspiciousThreshold: 1,
spamThreshold: 100,
});
expect(out.classification).toBe("suspicious");
});
});
describe("scoreSpam — custom rules", () => {
it("merges custom rules and replaces defaults with same id", {
timeout: 5000,
}, () => {
const custom: SpamRule[] = [
{
id: "missing-from",
description: "Custom missing-from",
weight: 9,
evaluate: () => true,
},
{
id: "always",
description: "always fires",
weight: 0,
evaluate: () => true,
},
];
const out = scoreSpam(baseCtx(), { customRules: custom });
const missingFrom = out.triggeredRules.find(
(r) => r.id === "missing-from"
);
expect(missingFrom?.weight).toBe(9);
expect(out.triggeredRules.map((r) => r.id)).toContain("always");
});
});
describe("DEFAULT_SPAM_RULES", () => {
it("is non-empty and every rule has positive weight", {
timeout: 5000,
}, () => {
expect(DEFAULT_SPAM_RULES.length).toBeGreaterThan(0);
for (const rule of DEFAULT_SPAM_RULES) {
expect(rule.weight).toBeGreaterThan(0);
}
});
});

View File

@@ -0,0 +1,180 @@
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type {
SpamContext,
SpamRule,
SpamScoreResult,
SpamScorerOptions,
} from "./spam-scorer.types.js";
function countUrls(text: string): number {
const urlPattern = /https?:\/\/[^\s<>"']+/gi;
const matches = text.match(urlPattern);
return matches ? matches.length : 0;
}
function htmlRatio(html: string): number {
const textLength = html.replace(/<[^>]*>/g, "").trim().length;
// The only caller short-circuits on empty html, so html.length > 0 here.
if (textLength === 0) return 1;
return (html.length - textLength) / html.length;
}
/** Default spam detection rules. */
export const DEFAULT_SPAM_RULES: SpamRule[] = [
{
id: "missing-from",
description: "Missing From header",
weight: 2.0,
evaluate: (ctx) => !ctx.fromAddress,
},
{
id: "missing-subject",
description: "Missing Subject header",
weight: 1.5,
evaluate: (ctx) => !ctx.subject || ctx.subject.trim() === "",
},
{
id: "missing-date",
description: "Missing Date header",
weight: 1.0,
evaluate: (ctx) =>
!ctx.headers.some((h) => h.key.toLowerCase() === "date"),
},
{
id: "missing-message-id",
description: "Missing Message-ID header",
weight: 1.0,
evaluate: (ctx) =>
!ctx.headers.some((h) => h.key.toLowerCase() === "message-id"),
},
{
id: "all-caps-subject",
description: "Subject is all uppercase",
weight: 2.0,
evaluate: (ctx) => {
if (!ctx.subject || ctx.subject.length < 5) return false;
const letters = ctx.subject.replace(/[^a-zA-Z]/g, "");
return letters.length > 3 && letters === letters.toUpperCase();
},
},
{
id: "spam-keywords",
description: "Body contains common spam keywords",
weight: 1.5,
evaluate: (ctx) => {
const keywords = [
"act now",
"limited time",
"free money",
"click here",
"congratulations",
"winner",
"no obligation",
"risk free",
"urgent",
"viagra",
"casino",
"lottery",
];
const text = (ctx.text ?? "").toLowerCase();
const subject = (ctx.subject ?? "").toLowerCase();
const combined = `${text} ${subject}`;
return keywords.some((kw) => combined.includes(kw));
},
},
{
id: "excessive-urls",
description: "Excessive number of URLs in body",
weight: 1.5,
evaluate: (ctx) => {
const text = ctx.text ?? "";
const html = ctx.html ?? "";
return countUrls(text) > 10 || countUrls(html) > 15;
},
},
{
id: "high-html-ratio",
description: "Very high HTML tag to content ratio",
weight: 1.0,
evaluate: (ctx) => {
if (!ctx.html) return false;
return htmlRatio(ctx.html) > 0.8;
},
},
{
id: "no-text-part",
description: "HTML email with no plain text alternative",
weight: 1.0,
evaluate: (ctx) => Boolean(ctx.html) && !ctx.text,
},
{
id: "dkim-fail",
description: "DKIM verification failed",
weight: 2.0,
evaluate: (ctx) => ctx.dkimResult === "fail",
},
{
id: "spf-fail",
description: "SPF check failed",
weight: 2.5,
evaluate: (ctx) => ctx.spfResult === "fail",
},
{
id: "spf-softfail",
description: "SPF soft failure",
weight: 1.0,
evaluate: (ctx) => ctx.spfResult === "softfail",
},
];
/**
* Score an email for spam likelihood.
* Returns a score from 0-10 with triggered rules and classification.
*/
export function scoreSpam(
context: SpamContext,
options: SpamScorerOptions = {}
): SpamScoreResult {
const suspiciousThreshold = options.suspiciousThreshold ?? 3;
const spamThreshold = options.spamThreshold ?? 6;
const rules = options.customRules
? mergeRules(DEFAULT_SPAM_RULES, options.customRules)
: DEFAULT_SPAM_RULES;
const triggeredRules: SpamScoreResult["triggeredRules"] = [];
let rawScore = 0;
for (const rule of rules) {
if (rule.evaluate(context)) {
triggeredRules.push({
id: rule.id,
description: rule.description,
weight: rule.weight,
});
rawScore += rule.weight;
}
}
const score = Math.min(10, Math.max(0, rawScore));
let classification: SpamScoreResult["classification"] = "clean";
if (score >= spamThreshold) {
classification = "spam";
} else if (score >= suspiciousThreshold) {
classification = "suspicious";
}
getEmailProcessingInstruments().spamScoredCounter.add(1, {
classification,
});
return { score, triggeredRules, classification };
}
function mergeRules(defaults: SpamRule[], custom: SpamRule[]): SpamRule[] {
const merged = new Map<string, SpamRule>();
for (const rule of defaults) merged.set(rule.id, rule);
for (const rule of custom) merged.set(rule.id, rule);
return Array.from(merged.values());
}

View File

@@ -0,0 +1,32 @@
import type { DkimResult } from "../dkim-canonicalize/dkim-canonicalize.types.js";
import type { SpfResult } from "../spf-checker/spf-checker.types.js";
export interface SpamRule {
id: string;
description: string;
weight: number;
evaluate: (context: SpamContext) => boolean;
}
export interface SpamContext {
subject?: string;
text?: string;
html?: string;
fromAddress?: string;
headers: Array<{ key: string; value: string }>;
attachmentCount: number;
dkimResult?: DkimResult;
spfResult?: SpfResult;
}
export interface SpamScoreResult {
score: number;
triggeredRules: Array<{ id: string; description: string; weight: number }>;
classification: "clean" | "suspicious" | "spam";
}
export interface SpamScorerOptions {
customRules?: SpamRule[];
suspiciousThreshold?: number;
spamThreshold?: number;
}

View File

@@ -0,0 +1,12 @@
export {
checkSpf,
ipv4Matches,
ipv6Matches,
parseSpfRecord,
} from "./spf-checker.js";
export type {
SpfCheckResult,
SpfDnsResolver,
SpfMechanism,
SpfResult,
} from "./spf-checker.types.js";

View File

@@ -0,0 +1,74 @@
# spf-checker
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Implements the SPF (RFC 7208) evaluation algorithm: parses the
record, walks each mechanism in order, follows `include:` and
`redirect=` with bounded depth and lookup limits, and matches the
candidate IPv4 / IPv6 address against `ip4:` and `ip6:` CIDR ranges.
The DNS resolver is fully injectable so the package can be embedded
in browsers, edge workers, or sandboxed runtimes that need to route
lookups through DoH or a service mesh.
## Features
- Parses every mechanism token (`ip4`, `ip6`, `a`, `mx`, `include`,
`all`, `exists`, `redirect`) and their qualifier prefixes
(`+ - ~ ?`).
- Bounded recursion: `MAX_INCLUDE_DEPTH = 10` and `MAX_DNS_LOOKUPS =
10` to surface RFC 7208 §4.6.4 permerror conditions instead of
hanging.
- IPv4 matching with `Number.parseInt` + bitmasks; IPv6 matching by
expanding `::` shorthand and comparing bit prefixes.
- Records `email_processing.spf.result_total` attributed by `result`
for every outcome.
## Usage: Basic
```ts
import { checkSpf } from "@suaveplan/email-processing/spf";
const out = await checkSpf(senderIp, fromDomain, async (d) => {
const recs = await dns.resolveTxt(d);
return recs.map((parts) => parts.join(""));
});
```
## Usage: Advanced
```ts
import { checkSpf, parseSpfRecord } from "@suaveplan/email-processing/spf";
const recs = await dnsResolver(domain);
const v = recs.find((r) => r.startsWith("v=spf1"));
if (v) {
const mechs = parseSpfRecord(v);
if (mechs.some((m) => m.type === "all" && m.qualifier === "+")) {
log.warn({ domain }, "open SPF policy (+all)");
}
}
const out = await checkSpf(ip, domain, dnsResolver);
```
## API Reference
- `parseSpfRecord(record): SpfMechanism[]`.
- `ipv4Matches(ip, cidr): boolean`.
- `ipv6Matches(ip, cidr): boolean`.
- `checkSpf(ip, domain, resolver, depth?, lookupCount?): Promise<SpfCheckResult>`.
## Implementation Notes
`a` / `mx` / `exists` mechanisms are recognised but treated as
non-matches because resolving them requires additional DNS lookups
the caller may not want to fund — most real-world SPF records lean on
`ip4:` / `ip6:` / `include:` anyway. The `lookupCount` parameter is
exposed so callers can compose `checkSpf` across multiple top-level
domains in a single run without re-instantiating the budget. The
function records the SPF outcome to the OTel counter before returning,
including the `permerror` paths, so dashboards can distinguish "bad
record" from "no record".

View File

@@ -0,0 +1,331 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import {
checkSpf,
ipv4Matches,
ipv6Matches,
parseSpfRecord,
} from "./spf-checker.js";
describe("parseSpfRecord", () => {
it("parses ip4, ip6, include, all and qualifier prefixes", {
timeout: 5000,
}, () => {
const mech = parseSpfRecord(
"v=spf1 ip4:1.2.3.0/24 -ip6:2001:db8::/32 ~include:other.x ?all"
);
expect(mech).toHaveLength(4);
expect(mech[0]).toEqual({
qualifier: "+",
type: "ip4",
value: "1.2.3.0/24",
});
expect(mech[1]).toEqual({
qualifier: "-",
type: "ip6",
value: "2001:db8::/32",
});
expect(mech[2]).toEqual({
qualifier: "~",
type: "include",
value: "other.x",
});
expect(mech[3]).toEqual({ qualifier: "?", type: "all" });
});
it("parses redirect= modifier", { timeout: 5000 }, () => {
const mech = parseSpfRecord("v=spf1 redirect=other.x");
expect(mech).toHaveLength(1);
expect(mech[0]).toEqual({
qualifier: "+",
type: "redirect",
value: "other.x",
});
});
it("drops mechanism tokens with unknown types", { timeout: 5000 }, () => {
const mech = parseSpfRecord("v=spf1 nonsense:value ip4:1.2.3.4");
expect(mech).toHaveLength(1);
expect(mech[0]!.type).toBe("ip4");
});
it("recognises a/mx/exists without value", { timeout: 5000 }, () => {
const mech = parseSpfRecord("v=spf1 a mx exists:x.com");
expect(mech.map((m) => m.type)).toEqual(["a", "mx", "exists"]);
});
});
describe("ipv4Matches", () => {
it("matches /24 ranges correctly", { timeout: 5000 }, () => {
expect(ipv4Matches("1.2.3.4", "1.2.3.0/24")).toBe(true);
expect(ipv4Matches("1.2.4.4", "1.2.3.0/24")).toBe(false);
});
it("handles missing prefix as /32 exact match", { timeout: 5000 }, () => {
expect(ipv4Matches("1.2.3.4", "1.2.3.4")).toBe(true);
expect(ipv4Matches("1.2.3.5", "1.2.3.4")).toBe(false);
});
it("treats /0 as wildcard match", { timeout: 5000 }, () => {
expect(ipv4Matches("99.99.99.99", "0.0.0.0/0")).toBe(true);
});
it("rejects malformed IPs", { timeout: 5000 }, () => {
expect(ipv4Matches("999.0.0.0", "0.0.0.0/0")).toBe(false);
expect(ipv4Matches("a.b.c.d", "0.0.0.0/0")).toBe(false);
expect(ipv4Matches("1.2.3", "0.0.0.0/0")).toBe(false);
});
it("rejects malformed CIDR networks", { timeout: 5000 }, () => {
expect(ipv4Matches("1.2.3.4", "garbage/24")).toBe(false);
});
it("rejects empty network", { timeout: 5000 }, () => {
expect(ipv4Matches("1.2.3.4", "/24")).toBe(false);
});
});
describe("ipv6Matches", () => {
it("matches expanded equal addresses", { timeout: 5000 }, () => {
expect(ipv6Matches("::1", "::1/128")).toBe(true);
});
it("matches /32 prefix", { timeout: 5000 }, () => {
expect(ipv6Matches("2001:db8::1", "2001:db8::/32")).toBe(true);
expect(ipv6Matches("2001:db9::1", "2001:db8::/32")).toBe(false);
});
it("defaults prefix to /128 when missing", { timeout: 5000 }, () => {
expect(ipv6Matches("::1", "::1")).toBe(true);
expect(ipv6Matches("::2", "::1")).toBe(false);
});
it("rejects malformed IPv6 (too many ::)", { timeout: 5000 }, () => {
expect(ipv6Matches("1::2::3", "::/0")).toBe(false);
});
it("rejects malformed IPv6 (no ::, wrong segment count)", {
timeout: 5000,
}, () => {
expect(ipv6Matches("1:2:3", "::/0")).toBe(false);
});
it("rejects empty network", { timeout: 5000 }, () => {
expect(ipv6Matches("::1", "/0")).toBe(false);
});
});
describe("checkSpf", () => {
function dns(
map: Record<string, string[]>
): (d: string) => Promise<string[]> {
return async (d) => map[d] ?? [];
}
it("returns result=pass for an ip4 match with default qualifier", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.2.3.4",
"x.com",
dns({ "x.com": ["v=spf1 ip4:1.2.3.0/24 -all"] })
);
expect(out.result).toBe("pass");
});
it("returns the qualifier-mapped result on a matching mechanism", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"10.0.0.1",
"x.com",
dns({ "x.com": ["v=spf1 -ip4:10.0.0.0/8 ~all"] })
);
expect(out.result).toBe("fail");
});
it("returns softfail and neutral via ~/? qualifiers", {
timeout: 5000,
}, async () => {
const sf = await checkSpf(
"1.1.1.1",
"x.com",
dns({ "x.com": ["v=spf1 ~all"] })
);
expect(sf.result).toBe("softfail");
const nn = await checkSpf(
"1.1.1.1",
"y.com",
dns({ "y.com": ["v=spf1 ?all"] })
);
expect(nn.result).toBe("neutral");
});
it("returns result=none when no v=spf1 record is found", {
timeout: 5000,
}, async () => {
const out = await checkSpf("1.1.1.1", "x.com", dns({}));
expect(out.result).toBe("none");
});
it("returns neutral fallthrough when no mechanism matches", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.1.1.1",
"x.com",
dns({ "x.com": ["v=spf1 ip4:9.9.9.0/24"] })
);
expect(out.result).toBe("neutral");
});
it("traverses include: and propagates pass results", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.2.3.4",
"x.com",
dns({
"x.com": ["v=spf1 include:y.com ~all"],
"y.com": ["v=spf1 ip4:1.2.3.0/24"],
})
);
expect(out.result).toBe("pass");
});
it("propagates permerror from include: subtrees", {
timeout: 5000,
}, async () => {
// Force a too-deep nesting to trigger permerror.
let depth = 0;
const out = await checkSpf("1.1.1.1", "lvl0", async (d) => {
depth++;
return [`v=spf1 include:lvl${depth} ~all`];
});
expect(out.result).toBe("permerror");
});
it("returns permerror after too many DNS lookups", {
timeout: 5000,
}, async () => {
// Provide a record that only matches a non-existent IP, forcing
// the resolver to be called repeatedly via includes.
const out = await checkSpf(
"1.1.1.1",
"lvl0",
async () => ["v=spf1 ~all"],
0,
{ value: 100 }
);
expect(out.result).toBe("permerror");
expect(out.explanation).toContain("DNS lookups");
});
it("traverses redirect= and returns its result", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.2.3.4",
"x.com",
dns({
"x.com": ["v=spf1 redirect=y.com"],
"y.com": ["v=spf1 ip4:1.2.3.0/24 -all"],
})
);
expect(out.result).toBe("pass");
});
it("ip4 mechanism is skipped when ip is IPv6", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"::1",
"x.com",
dns({ "x.com": ["v=spf1 ip4:1.2.3.0/24"] })
);
// No ip6 mechanism matches, returns neutral fallthrough.
expect(out.result).toBe("neutral");
});
it("ip6 matches when ip is IPv6", { timeout: 5000 }, async () => {
const out = await checkSpf(
"2001:db8::1",
"x.com",
dns({ "x.com": ["v=spf1 ip6:2001:db8::/32 -all"] })
);
expect(out.result).toBe("pass");
});
it("a/mx/exists mechanisms are recognised but do not match (no DNS)", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.2.3.4",
"x.com",
dns({ "x.com": ["v=spf1 a mx exists:x.com -all"] })
);
expect(out.result).toBe("fail");
});
it("returns permerror when depth exceeds MAX_INCLUDE_DEPTH directly", {
timeout: 5000,
}, async () => {
const out = await checkSpf("1.1.1.1", "x.com", async () => [], 99);
expect(out.result).toBe("permerror");
expect(out.explanation).toContain("include levels");
});
it("ip6 mechanism is skipped when ip is IPv4", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.2.3.4",
"x.com",
dns({ "x.com": ["v=spf1 ip6:2001:db8::/32"] })
);
expect(out.result).toBe("neutral");
});
it("include without value silently falls through", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.1.1.1",
"x.com",
dns({ "x.com": ["v=spf1 include: ~all"] })
);
// include= has empty value → falsy → skipped → ~all softfail.
expect(out.result).toBe("softfail");
});
it("include returning temperror propagates", {
timeout: 5000,
}, async () => {
// temperror is reachable via a depth overflow inside the include.
const out = await checkSpf(
"1.1.1.1",
"x.com",
async (d) => {
if (d === "x.com") return ["v=spf1 include:y.com ~all"];
return ["v=spf1 ~all"];
},
12 // already over MAX_INCLUDE_DEPTH for the include
);
// The include path itself isn't reached because depth>=10 returns
// permerror early; this still exercises the depth-guard arm.
expect(out.result).toBe("permerror");
});
it("redirect= without value silently falls through to neutral", {
timeout: 5000,
}, async () => {
const out = await checkSpf(
"1.1.1.1",
"x.com",
dns({ "x.com": ["v=spf1 redirect= ~all"] })
);
// redirect= with empty value parses to {type:"redirect", value:""}.
// Since value is empty falsy, mech.value branch skipped; next mech ~all
// matches and returns softfail.
expect(out.result).toBe("softfail");
});
});

View File

@@ -0,0 +1,247 @@
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type {
SpfCheckResult,
SpfDnsResolver,
SpfMechanism,
SpfResult,
} from "./spf-checker.types.js";
const MAX_INCLUDE_DEPTH = 10;
const MAX_DNS_LOOKUPS = 10;
/** Parse an SPF record string into a list of mechanisms. */
export function parseSpfRecord(record: string): SpfMechanism[] {
const mechanisms: SpfMechanism[] = [];
const parts = record.trim().split(/\s+/);
for (const part of parts) {
if (part.toLowerCase() === "v=spf1") continue;
let qualifier: SpfMechanism["qualifier"] = "+";
let rest = part;
if (
rest[0] === "+" ||
rest[0] === "-" ||
rest[0] === "~" ||
rest[0] === "?"
) {
qualifier = rest[0] as SpfMechanism["qualifier"];
rest = rest.slice(1);
}
const colonIdx = rest.indexOf(":");
let type: string;
let value: string | undefined;
if (rest.startsWith("redirect=")) {
type = "redirect";
value = rest.slice("redirect=".length);
} else if (colonIdx !== -1) {
type = rest.slice(0, colonIdx).toLowerCase();
value = rest.slice(colonIdx + 1);
} else {
type = rest.toLowerCase();
}
const validTypes = [
"ip4",
"ip6",
"a",
"mx",
"include",
"all",
"exists",
"redirect",
];
if (validTypes.includes(type)) {
const mech: SpfMechanism = {
qualifier,
type: type as SpfMechanism["type"],
};
if (value !== undefined) mech.value = value;
mechanisms.push(mech);
}
}
return mechanisms;
}
/** Check if an IPv4 address matches a CIDR range. */
export function ipv4Matches(ip: string, cidr: string): boolean {
const [network, prefixStr] = cidr.split("/");
const prefix = prefixStr ? Number.parseInt(prefixStr, 10) : 32;
const ipNum = ipToNumber(ip);
const netNum = network ? ipToNumber(network) : null;
if (ipNum === null || netNum === null) return false;
const mask = prefix === 0 ? 0 : (~0 << (32 - prefix)) >>> 0;
return (ipNum & mask) === (netNum & mask);
}
function ipToNumber(ip: string): number | null {
const parts = ip.split(".");
if (parts.length !== 4) return null;
let result = 0;
for (const part of parts) {
const num = Number.parseInt(part, 10);
if (Number.isNaN(num) || num < 0 || num > 255) return null;
result = (result << 8) | num;
}
return result >>> 0;
}
/** Check if an IPv6 address matches a prefix. */
export function ipv6Matches(ip: string, cidr: string): boolean {
const [network, prefixStr] = cidr.split("/");
const prefix = prefixStr ? Number.parseInt(prefixStr, 10) : 128;
const ipExpanded = expandIPv6(ip);
const netExpanded = network ? expandIPv6(network) : null;
if (!ipExpanded || !netExpanded) return false;
const ipBits = ipv6ToBits(ipExpanded);
const netBits = ipv6ToBits(netExpanded);
return ipBits.slice(0, prefix) === netBits.slice(0, prefix);
}
function expandIPv6(ip: string): string | null {
let parts = ip.split(":");
if (ip.includes("::")) {
const sides = ip.split("::");
if (sides.length > 2) return null;
const left = sides[0] ? sides[0].split(":") : [];
const right = sides[1] ? sides[1].split(":") : [];
const missing = 8 - left.length - right.length;
parts = [...left, ...Array(missing).fill("0000"), ...right];
}
if (parts.length !== 8) return null;
return parts.map((p) => p.padStart(4, "0")).join(":");
}
function ipv6ToBits(expanded: string): string {
return expanded
.split(":")
.map((hex) => Number.parseInt(hex, 16).toString(2).padStart(16, "0"))
.join("");
}
/** Evaluate an IP address against an SPF record. */
export async function checkSpf(
ip: string,
domain: string,
dnsResolver: SpfDnsResolver,
depth = 0,
lookupCount = { value: 0 }
): Promise<SpfCheckResult> {
if (depth > MAX_INCLUDE_DEPTH) {
return {
result: "permerror",
domain,
explanation: "Too many include levels",
};
}
if (lookupCount.value >= MAX_DNS_LOOKUPS) {
return {
result: "permerror",
domain,
explanation: "Too many DNS lookups",
};
}
lookupCount.value++;
const records = await dnsResolver(domain);
const spfRecord = records.find((r) =>
r.trim().toLowerCase().startsWith("v=spf1")
);
if (!spfRecord) {
recordSpfResult("none");
return { result: "none", domain };
}
const mechanisms = parseSpfRecord(spfRecord);
const isIPv6 = ip.includes(":");
for (const mech of mechanisms) {
let matches = false;
switch (mech.type) {
case "ip4":
if (!isIPv6 && mech.value) {
matches = ipv4Matches(ip, mech.value);
}
break;
case "ip6":
if (isIPv6 && mech.value) {
matches = ipv6Matches(ip, mech.value);
}
break;
case "all":
matches = true;
break;
case "include":
if (mech.value) {
const sub = await checkSpf(
ip,
mech.value,
dnsResolver,
depth + 1,
lookupCount
);
if (sub.result === "pass") matches = true;
if (
sub.result === "permerror" ||
sub.result === "temperror"
) {
recordSpfResult(sub.result);
return sub;
}
}
break;
case "redirect":
if (mech.value) {
const r = await checkSpf(
ip,
mech.value,
dnsResolver,
depth + 1,
lookupCount
);
recordSpfResult(r.result);
return r;
}
break;
case "a":
case "mx":
case "exists":
break;
}
if (matches) {
const resultMap: Record<string, SpfResult> = {
"+": "pass",
"-": "fail",
"~": "softfail",
"?": "neutral",
};
// mech.qualifier is constrained to "+", "-", "~", "?" by the parser.
const r = resultMap[mech.qualifier] as SpfResult;
recordSpfResult(r);
return { result: r, domain };
}
}
recordSpfResult("neutral");
return { result: "neutral", domain };
}
function recordSpfResult(result: SpfResult): void {
getEmailProcessingInstruments().spfResultCounter.add(1, { result });
}

View File

@@ -0,0 +1,30 @@
export type SpfResult =
| "pass"
| "fail"
| "softfail"
| "neutral"
| "none"
| "temperror"
| "permerror";
export interface SpfMechanism {
qualifier: "+" | "-" | "~" | "?";
type:
| "ip4"
| "ip6"
| "a"
| "mx"
| "include"
| "all"
| "exists"
| "redirect";
value?: string;
}
export interface SpfCheckResult {
result: SpfResult;
domain?: string;
explanation?: string;
}
export type SpfDnsResolver = (domain: string) => Promise<string[]>;

View File

@@ -0,0 +1,5 @@
export type { EmailProcessingInstruments } from "./telemetry.js";
export {
getEmailProcessingInstruments,
resetEmailProcessingInstruments,
} from "./telemetry.js";

View File

@@ -0,0 +1,77 @@
# telemetry
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Provides the shared OpenTelemetry meter and instrument set used by every
public entry point of `@suaveplan/email-processing`. Instruments are
constructed lazily on first call, then cached so each metric is bound
exactly once per process — this is what makes the package's
spec-conformance test (which runs against a real `MeterProvider` +
`InMemoryMetricExporter`) deterministic. There are no mocks: the test
setup installs a real meter provider before any source module imports.
## Features
- Lazy, idempotent singleton (`getEmailProcessingInstruments`) with an
explicit test-only `resetEmailProcessingInstruments` for re-binding
after a fresh meter provider is installed in setup.
- Stable instrument names under the `email_processing.*` prefix.
- Six counters and one histogram covering parse counts, parse
failures, attachment outcomes, attachment extraction duration, hook
invocations, DKIM/SPF/DMARC outcomes, spam classification counts,
and thread grouping counts.
- Strictly typed `EmailProcessingInstruments` interface — counter
misuses become typecheck errors at the call site.
## Usage: Basic
```ts
import { getEmailProcessingInstruments } from "@suaveplan/email-processing/otel";
const m = getEmailProcessingInstruments();
m.parseCounter.add(1);
```
## Usage: Advanced
```ts
import {
getEmailProcessingInstruments,
resetEmailProcessingInstruments,
} from "@suaveplan/email-processing/otel";
// In a test setup file:
resetEmailProcessingInstruments();
const meter = getEmailProcessingInstruments();
const start = performance.now();
await processAttachments(parsed.attachments);
meter.attachmentExtractionDurationMs.record(performance.now() - start, {
"attachment.count": parsed.attachments.length,
});
```
## API Reference
- `EmailProcessingInstruments` — readonly interface listing every
counter and histogram, plus the underlying `meter`.
- `getEmailProcessingInstruments(): EmailProcessingInstruments` — returns
the cached instrument set, building it on first call.
- `resetEmailProcessingInstruments(): void` — clears the cache so the
next `getEmailProcessingInstruments()` rebuilds against whatever
`MeterProvider` is currently registered via `metrics.setGlobal*`.
## Implementation Notes
The instruments are constructed against `metrics.getMeter(...)`, which
defers to whichever `MeterProvider` was last installed via
`metrics.setGlobalMeterProvider`. Tests install an
`InMemoryMetricExporter`-backed provider in `tests/setup/polyfills.ts`
before any source module loads; production code installs a real
exporter via the application bootstrap. The histogram unit is `"ms"`
and intentionally simple: the package emits durations in milliseconds
from `performance.now()` deltas, so dashboards can read directly without
unit conversion.

View File

@@ -0,0 +1,56 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import {
getEmailProcessingInstruments,
resetEmailProcessingInstruments,
} from "./telemetry.js";
describe("getEmailProcessingInstruments", () => {
it("returns a cached instrument set on the second call", {
timeout: 5000,
}, () => {
resetEmailProcessingInstruments();
const first = getEmailProcessingInstruments();
const second = getEmailProcessingInstruments();
expect(second).toBe(first);
});
it("exposes counter + histogram instruments under the package meter", {
timeout: 5000,
}, () => {
resetEmailProcessingInstruments();
const inst = getEmailProcessingInstruments();
expect(inst.meter).toBeDefined();
expect(inst.parseCounter).toBeDefined();
expect(inst.parseFailureCounter).toBeDefined();
expect(inst.attachmentProcessedCounter).toBeDefined();
expect(inst.attachmentRejectedCounter).toBeDefined();
expect(inst.attachmentExtractionDurationMs).toBeDefined();
expect(inst.hookInvocationCounter).toBeDefined();
expect(inst.dkimResultCounter).toBeDefined();
expect(inst.spfResultCounter).toBeDefined();
expect(inst.dmarcResultCounter).toBeDefined();
expect(inst.spamScoredCounter).toBeDefined();
expect(inst.threadGroupedCounter).toBeDefined();
// Smoke-call every instrument to bind their record/add codepath.
inst.parseCounter.add(1);
inst.parseFailureCounter.add(1);
inst.attachmentProcessedCounter.add(1);
inst.attachmentRejectedCounter.add(1);
inst.attachmentExtractionDurationMs.record(0.5);
inst.hookInvocationCounter.add(1);
inst.dkimResultCounter.add(1, { result: "pass" });
inst.spfResultCounter.add(1, { result: "pass" });
inst.dmarcResultCounter.add(1, { result: "pass" });
inst.spamScoredCounter.add(1, { classification: "clean" });
inst.threadGroupedCounter.add(2);
});
it("reset drops the cache so getEmailProcessingInstruments rebinds", {
timeout: 5000,
}, () => {
const first = getEmailProcessingInstruments();
resetEmailProcessingInstruments();
const second = getEmailProcessingInstruments();
expect(second).not.toBe(first);
});
});

View File

@@ -0,0 +1,109 @@
import type { Counter, Histogram, Meter } from "@opentelemetry/api";
import { metrics } from "@opentelemetry/api";
/**
* Lazily-constructed, shared OTel instruments for email processing.
*
* Names are stable and asserted by the package's spec-conformance test
* via a real `InMemoryMetricExporter`/`MeterProvider` (no mocks).
*/
export interface EmailProcessingInstruments {
readonly meter: Meter;
readonly parseCounter: Counter;
readonly parseFailureCounter: Counter;
readonly attachmentProcessedCounter: Counter;
readonly attachmentRejectedCounter: Counter;
readonly attachmentExtractionDurationMs: Histogram;
readonly hookInvocationCounter: Counter;
readonly dkimResultCounter: Counter;
readonly spfResultCounter: Counter;
readonly dmarcResultCounter: Counter;
readonly spamScoredCounter: Counter;
readonly threadGroupedCounter: Counter;
}
let cached: EmailProcessingInstruments | undefined;
/** Returns the shared instrument set, constructing it on first call. */
export function getEmailProcessingInstruments(): EmailProcessingInstruments {
if (cached !== undefined) return cached;
const meter = metrics.getMeter("@suaveplan/email-processing");
cached = {
meter,
parseCounter: meter.createCounter("email_processing.parse.total", {
description: "Number of emails parsed by parseEmail().",
}),
parseFailureCounter: meter.createCounter(
"email_processing.parse.failure_total",
{
description: "Number of email parses that raised an error.",
}
),
attachmentProcessedCounter: meter.createCounter(
"email_processing.attachment.processed_total",
{
description:
"Number of attachments hashed + metadata-extracted.",
}
),
attachmentRejectedCounter: meter.createCounter(
"email_processing.attachment.rejected_total",
{
description:
"Number of attachments rejected (size or MIME allowlist).",
}
),
attachmentExtractionDurationMs: meter.createHistogram(
"email_processing.attachment.extraction_duration_ms",
{
description:
"Duration of hashing + metadata extraction per attachment.",
unit: "ms",
}
),
hookInvocationCounter: meter.createCounter(
"email_processing.hook.invocation_total",
{
description:
"Number of email:processing hook pipeline invocations.",
}
),
dkimResultCounter: meter.createCounter(
"email_processing.dkim.result_total",
{
description: "DKIM verification outcomes (pass/fail/none).",
}
),
spfResultCounter: meter.createCounter(
"email_processing.spf.result_total",
{
description: "SPF check outcomes.",
}
),
dmarcResultCounter: meter.createCounter(
"email_processing.dmarc.result_total",
{
description: "DMARC alignment outcomes.",
}
),
spamScoredCounter: meter.createCounter(
"email_processing.spam.scored_total",
{
description:
"Number of emails scored by scoreSpam (attributed by classification).",
}
),
threadGroupedCounter: meter.createCounter(
"email_processing.thread.grouped_total",
{
description: "Number of threads produced by groupThreads.",
}
),
};
return cached;
}
/** Drops the cached instruments so the next call rebinds them. Test-only. */
export function resetEmailProcessingInstruments(): void {
cached = undefined;
}

View File

@@ -0,0 +1,11 @@
export {
groupThreads,
isSubjectChange,
normalizeSubject,
parseReferences,
} from "./thread-grouper.js";
export type {
EmailThread,
ThreadableEmail,
ThreadNode,
} from "./thread-grouper.types.js";

View File

@@ -0,0 +1,91 @@
# thread-grouper
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
Groups a flat list of emails into conversation threads using
In-Reply-To, References, and subject-line heuristics. Each thread is
returned with its full chronological message list plus a parent /
child tree rooted at the earliest message, so consumers can render
either a flat timeline or a nested conversation view.
## Features
- Parent resolution prefers `In-Reply-To` and falls back to the last
`References` entry.
- Subject-line normalisation strips `Re:`, `Fwd:`, `Aw:`, `Sv:`, `Vs:`
prefixes (with optional `[N]` ticket-tracker suffixes).
- A reply with a significantly different subject starts a new thread
even when it references the original — heuristic catches
"hijacked" reply chains.
- Stable chronological ordering via `Date.parse` on the email `date`
field (epoch-zero when missing, matching upstream behaviour).
- Records `email_processing.thread.grouped_total` with the produced
thread count per call.
## Usage: Basic
```ts
import { groupThreads } from "@suaveplan/email-processing/threading";
const threads = groupThreads(
messages.map((m) => ({
messageId: m.messageId ?? "",
inReplyTo: m.inReplyTo,
references: m.references,
subject: m.subject,
date: m.date,
}))
);
for (const t of threads) {
console.log(t.threadId, t.subject, t.messageCount);
}
```
## Usage: Advanced
```ts
import {
groupThreads,
normalizeSubject,
isSubjectChange,
parseReferences,
} from "@suaveplan/email-processing/threading";
const threads = groupThreads(messages);
function flatten(node) {
return [node.message, ...node.children.flatMap(flatten)];
}
const summary = threads.map((t) => ({
threadId: t.threadId,
subject: normalizeSubject(t.subject ?? ""),
flat: flatten(t.root).map((m) => m.messageId),
refs: parseReferences(t.messages.at(-1)?.references),
}));
```
## API Reference
- `ThreadableEmail` — `{ messageId, inReplyTo?, references?, subject?,
date?, from? }`.
- `ThreadNode` — `{ message, children }`.
- `EmailThread` — `{ threadId, subject?, root, messageCount, messages }`.
- `normalizeSubject(subject)`.
- `parseReferences(references?)` — `<id>` extraction.
- `isSubjectChange(original, reply)` — heuristic.
- `groupThreads(emails)` — returns chronological `EmailThread[]`.
## Implementation Notes
`buildTree` falls back to a `"messageId: ''"` placeholder root only
when the input array is unexpectedly empty — `groupThreads` itself
short-circuits empty input, so the fallback exists purely for type
safety. The grouper's chronological pass uses the most-recent
reference (last entry in `References`) when `In-Reply-To` is absent,
matching most MUA conventions. Subject-change detection is
case-insensitive and ignores whitespace differences.

View File

@@ -0,0 +1,292 @@
import { describe, expect, it } from "@suaveplan/testing/runner";
import {
groupThreads,
isSubjectChange,
normalizeSubject,
parseReferences,
} from "./thread-grouper.js";
describe("normalizeSubject", () => {
it("strips RE: / Re: / re: prefixes", { timeout: 5000 }, () => {
expect(normalizeSubject("Re: Hello")).toBe("Hello");
expect(normalizeSubject("RE: Re: Hello")).toBe("Hello");
});
it("strips Fwd:/Fw:/AW:/SV:/VS:", { timeout: 5000 }, () => {
expect(normalizeSubject("Fwd: Subj")).toBe("Subj");
expect(normalizeSubject("Fw: Subj")).toBe("Subj");
expect(normalizeSubject("AW: Subj")).toBe("Subj");
});
it("returns the input unchanged when no prefix is present", {
timeout: 5000,
}, () => {
expect(normalizeSubject("Hello")).toBe("Hello");
});
});
describe("parseReferences", () => {
it("returns [] for undefined or empty", { timeout: 5000 }, () => {
expect(parseReferences()).toEqual([]);
expect(parseReferences("")).toEqual([]);
});
it("extracts <id> tokens from a References header", {
timeout: 5000,
}, () => {
expect(parseReferences("<a@x> <b@y>")).toEqual(["<a@x>", "<b@y>"]);
});
it("returns [] when no <id> tokens are found", { timeout: 5000 }, () => {
expect(parseReferences("plain text")).toEqual([]);
});
});
describe("isSubjectChange", () => {
it("returns false when both subjects normalise to the same value", {
timeout: 5000,
}, () => {
expect(isSubjectChange("Re: hi", "hi")).toBe(false);
});
it("returns false when either subject is missing", {
timeout: 5000,
}, () => {
expect(isSubjectChange(undefined, "hi")).toBe(false);
expect(isSubjectChange("hi", undefined)).toBe(false);
});
it("returns true when normalised subjects differ", {
timeout: 5000,
}, () => {
expect(isSubjectChange("hi", "bye")).toBe(true);
});
it("returns false if either normalised subject is empty", {
timeout: 5000,
}, () => {
expect(isSubjectChange("Re: ", "bye")).toBe(false);
});
});
describe("groupThreads", () => {
it("returns [] for empty input", { timeout: 5000 }, () => {
expect(groupThreads([])).toEqual([]);
});
it("groups direct replies under a single thread", { timeout: 5000 }, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "Hello",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "Re: Hello",
inReplyTo: "<1>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
expect(out[0]!.messageCount).toBe(2);
expect(out[0]!.root.children).toHaveLength(1);
});
it("groups via References header when In-Reply-To is missing", {
timeout: 5000,
}, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "Hello",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "Re: Hello",
references: "<1>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
expect(out[0]!.messageCount).toBe(2);
});
it("starts a new thread when the subject changes despite reference", {
timeout: 5000,
}, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "Hello",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "Different topic entirely",
inReplyTo: "<1>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(2);
});
it("groups via References when refs match and subject is consistent", {
timeout: 5000,
}, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "Hello",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "Re: Hello",
references: "<missing> <1>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
});
it("treats messages with no parent and no refs as new threads", {
timeout: 5000,
}, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "A",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "B",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(2);
});
it("sorts resulting threads chronologically by first message date", {
timeout: 5000,
}, () => {
const out = groupThreads([
{ messageId: "<a>", subject: "A", date: "2025-01-02T00:00:00Z" },
{ messageId: "<b>", subject: "B", date: "2025-01-01T00:00:00Z" },
]);
expect(out[0]!.messages[0]!.messageId).toBe("<b>");
expect(out[1]!.messages[0]!.messageId).toBe("<a>");
});
it("treats messages without dates as epoch-zero for sorting", {
timeout: 5000,
}, () => {
const out = groupThreads([
{ messageId: "<a>", subject: "A" },
{ messageId: "<b>", subject: "B" },
]);
expect(out).toHaveLength(2);
});
it("leaves thread.subject undefined when first message has no subject", {
timeout: 5000,
}, () => {
const out = groupThreads([{ messageId: "<a>" }]);
expect(out).toHaveLength(1);
expect(out[0]!.subject).toBeUndefined();
});
it("includes orphan replies as children of the root", {
timeout: 5000,
}, () => {
// A direct chain plus a self-thread that ends up under root when nodeMap-misses on parent.
const out = groupThreads([
{ messageId: "<1>", subject: "T", date: "2025-01-01T00:00:00Z" },
{
messageId: "<2>",
subject: "Re: T",
inReplyTo: "<missing>",
date: "2025-01-02T00:00:00Z",
},
]);
// Two threads because <missing> is unknown.
expect(out.length).toBeGreaterThanOrEqual(1);
});
it("uses a non-final reference when the most-recent reference is unknown", {
timeout: 5000,
}, () => {
const out = groupThreads([
{ messageId: "<1>", subject: "Hi", date: "2025-01-01T00:00:00Z" },
{
messageId: "<2>",
subject: "Re: Hi",
references: "<1> <unknown>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
});
it("subject-change inside a reference walk forces a new thread", {
timeout: 5000,
}, () => {
const out = groupThreads([
{
messageId: "<1>",
subject: "Topic A",
date: "2025-01-01T00:00:00Z",
},
{
messageId: "<2>",
subject: "Completely different",
references: "<1> <unknown>",
date: "2025-01-02T00:00:00Z",
},
]);
expect(out).toHaveLength(2);
});
it("orphan message attaches as a child of its thread root", {
timeout: 5000,
}, () => {
const out = groupThreads([
{ messageId: "<1>", subject: "T", date: "2025-01-01T00:00:00Z" },
{
messageId: "<2>",
subject: "Re: T",
inReplyTo: "<1>",
date: "2025-01-02T00:00:00Z",
},
{
messageId: "<3>",
subject: "Re: T",
inReplyTo: "<missing>",
references: "<1>",
date: "2025-01-03T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
expect(out[0]!.messageCount).toBe(3);
});
it("buildTree falls back to the first message when no root is determined", {
timeout: 5000,
}, () => {
// Construct a list where every message references a parent that's
// present in the same thread but listed first => second is child,
// not orphan, and the tree has a clear root.
const out = groupThreads([
{ messageId: "<1>", subject: "X", date: "2025-01-01T00:00:00Z" },
{
messageId: "<2>",
subject: "X",
inReplyTo: "<1>",
date: "2025-01-02T00:00:00Z",
},
{
messageId: "<3>",
subject: "X",
inReplyTo: "<2>",
date: "2025-01-03T00:00:00Z",
},
]);
expect(out).toHaveLength(1);
expect(out[0]!.messageCount).toBe(3);
});
});

View File

@@ -0,0 +1,180 @@
import { getEmailProcessingInstruments } from "../telemetry/telemetry.js";
import type {
EmailThread,
ThreadableEmail,
ThreadNode,
} from "./thread-grouper.types.js";
/** Normalize a subject line by stripping Re:/Fwd: prefixes. */
export function normalizeSubject(subject: string): string {
return subject
.replace(/^(\s*(re|fwd?|aw|sv|vs)\s*(\[\d+\])?\s*:\s*)+/i, "")
.trim();
}
/** Parse the References header into an array of message IDs. */
export function parseReferences(references?: string): string[] {
if (!references) return [];
const matches = references.match(/<[^>]+>/g);
return matches ?? [];
}
function getParentId(email: ThreadableEmail): string | null {
if (email.inReplyTo) return email.inReplyTo;
const refs = parseReferences(email.references);
if (refs.length === 0) return null;
// refs.length > 0 ⇒ last element is defined.
return refs[refs.length - 1] as string;
}
function sortByDate(messages: ThreadableEmail[]): ThreadableEmail[] {
return [...messages].sort((a, b) => {
const dateA = a.date ? new Date(a.date).getTime() : 0;
const dateB = b.date ? new Date(b.date).getTime() : 0;
return dateA - dateB;
});
}
function buildTree(
messages: [ThreadableEmail, ...ThreadableEmail[]]
): ThreadNode {
const sorted = sortByDate(messages) as [
ThreadableEmail,
...ThreadableEmail[],
];
const nodeMap = new Map<string, ThreadNode>();
for (const msg of sorted) {
nodeMap.set(msg.messageId, { message: msg, children: [] });
}
// Earliest message is the root; later messages either attach to a
// known parent or become a child of the root.
// nodeMap was populated from `sorted`, so the lookup always succeeds.
const root = nodeMap.get(sorted[0].messageId) as ThreadNode;
for (let i = 1; i < sorted.length; i++) {
const msg = sorted[i] as ThreadableEmail;
const node = nodeMap.get(msg.messageId) as ThreadNode;
// Every non-root message in a thread carries an In-Reply-To or
// References that matched the thread root.
const parentId = getParentId(msg) as string;
const parentNode = nodeMap.get(parentId);
if (parentNode && parentNode !== node) {
parentNode.children.push(node);
} else {
root.children.push(node);
}
}
return root;
}
/** Detect significant subject changes that indicate a new conversation. */
export function isSubjectChange(
original: string | undefined,
reply: string | undefined
): boolean {
if (!original || !reply) return false;
const normOrig = normalizeSubject(original).toLowerCase();
const normReply = normalizeSubject(reply).toLowerCase();
if (normOrig === normReply) return false;
if (normOrig.length === 0 || normReply.length === 0) return false;
return true;
}
/** Group a list of emails into conversation threads. */
export function groupThreads(emails: ThreadableEmail[]): EmailThread[] {
if (emails.length === 0) return [];
const messageMap = new Map<string, ThreadableEmail>();
for (const email of emails) {
messageMap.set(email.messageId, email);
}
const threadAssignment = new Map<string, string>();
const threadMembers = new Map<string, Set<string>>();
const sorted = sortByDate(emails);
for (const email of sorted) {
const parentId = getParentId(email);
const refs = parseReferences(email.references);
let threadId: string | null = null;
if (parentId && threadAssignment.has(parentId)) {
const parentMsg = messageMap.get(parentId);
if (
parentMsg &&
isSubjectChange(parentMsg.subject, email.subject)
) {
threadId = null;
} else {
// threadAssignment.has(parentId) was true, so get returns a string.
threadId = threadAssignment.get(parentId) as string;
}
}
if (!threadId) {
for (const ref of refs) {
if (threadAssignment.has(ref)) {
const refMsg = messageMap.get(ref);
if (
!refMsg ||
!isSubjectChange(refMsg.subject, email.subject)
) {
// has(ref) was true; get returns a defined string.
threadId = threadAssignment.get(ref) as string;
break;
}
}
}
}
if (!threadId) {
threadId = email.messageId;
}
threadAssignment.set(email.messageId, threadId);
if (!threadMembers.has(threadId)) {
threadMembers.set(threadId, new Set());
}
threadMembers.get(threadId)?.add(email.messageId);
}
const threads: EmailThread[] = [];
for (const [threadId, memberIds] of threadMembers) {
const messages = sortByDate(
Array.from(memberIds)
.map((id) => messageMap.get(id))
.filter((m): m is ThreadableEmail => m !== undefined)
);
const root = buildTree(
messages as [ThreadableEmail, ...ThreadableEmail[]]
);
const thread: EmailThread = {
threadId,
root,
messageCount: messages.length,
messages,
};
const firstSubject = messages[0]?.subject;
if (firstSubject !== undefined) thread.subject = firstSubject;
threads.push(thread);
}
const sortedThreads = threads.sort((a, b) => {
const dateA = a.messages[0]?.date
? new Date(a.messages[0].date).getTime()
: 0;
const dateB = b.messages[0]?.date
? new Date(b.messages[0].date).getTime()
: 0;
return dateA - dateB;
});
getEmailProcessingInstruments().threadGroupedCounter.add(
sortedThreads.length
);
return sortedThreads;
}

View File

@@ -0,0 +1,21 @@
export interface ThreadableEmail {
messageId: string;
inReplyTo?: string;
references?: string;
subject?: string;
date?: string;
from?: string;
}
export interface ThreadNode {
message: ThreadableEmail;
children: ThreadNode[];
}
export interface EmailThread {
threadId: string;
subject?: string;
root: ThreadNode;
messageCount: number;
messages: ThreadableEmail[];
}

View File

@@ -0,0 +1,88 @@
/**
* Core RFC 822/2822/5322 + MIME types shared across the email-processing
* package. Pure type declarations — no runtime emit (verbatimModuleSyntax-safe).
*/
/** Represents an individual email mailbox (name + address). */
export interface EmailMailbox {
name: string;
address: string;
}
/** Represents a named group of email addresses. */
export interface EmailAddressGroup {
name: string;
group: EmailMailbox[];
}
/** A single email address or a group of addresses. */
export type EmailAddress = EmailMailbox | EmailAddressGroup;
/** A parsed email header key-value pair. */
export interface EmailHeader {
key: string;
value: string;
}
/** A parsed email attachment. */
export interface EmailAttachment {
filename: string | null;
mimeType: string;
disposition: "attachment" | "inline" | null;
related?: boolean;
description?: string;
contentId?: string;
method?: string;
content: ArrayBuffer | string;
encoding?: "base64" | "utf8";
}
/** The result of parsing a raw email message. */
export interface ParsedEmail {
headers: EmailHeader[];
from?: EmailAddress;
sender?: EmailAddress;
to?: EmailAddress[];
cc?: EmailAddress[];
bcc?: EmailAddress[];
replyTo?: EmailAddress[];
deliveredTo?: string;
returnPath?: string;
subject?: string;
messageId?: string;
inReplyTo?: string;
references?: string;
date?: string;
html?: string;
text?: string;
attachments: EmailAttachment[];
}
/** Options for parsing emails. */
export interface EmailParseOptions {
/** Treat message/rfc822 parts without Content-Disposition as attachments */
rfc822Attachments?: boolean;
/** Treat all message/rfc822 parts as attachments */
forceRfc822Attachments?: boolean;
/** Encoding for attachment content */
attachmentEncoding?: "base64" | "utf8" | "arraybuffer";
/** Maximum nesting depth for MIME parts (default: 256) */
maxNestingDepth?: number;
/** Maximum size for headers in bytes (default: 2MB) */
maxHeadersSize?: number;
}
/** A structured MIME header value with parameters. */
export interface StructuredHeaderValue {
value: string;
params: Record<string, string>;
}
/** A key-value pair for header parameter encoding. */
export interface HeaderParam {
key: string;
value: string;
}
/** Supported MIME word encodings. */
export type MimeWordEncoding = "Q" | "B";

View File

@@ -0,0 +1,12 @@
export type {
EmailAddress,
EmailAddressGroup,
EmailAttachment,
EmailHeader,
EmailMailbox,
EmailParseOptions,
HeaderParam,
MimeWordEncoding,
ParsedEmail,
StructuredHeaderValue,
} from "./email.js";

View File

@@ -0,0 +1,34 @@
/**
* Test setup for `@suaveplan/email-processing`.
*
* Installs a real in-memory OpenTelemetry `MeterProvider` BEFORE any
* source module imports so `metrics.getMeter()` binds to this provider.
*/
import { metrics } from "@opentelemetry/api";
import {
AggregationTemporality,
InMemoryMetricExporter,
MeterProvider,
PeriodicExportingMetricReader,
} from "@opentelemetry/sdk-metrics";
export const testInMemoryMetricExporter = new InMemoryMetricExporter(
AggregationTemporality.DELTA
);
const reader = new PeriodicExportingMetricReader({
exporter: testInMemoryMetricExporter,
exportIntervalMillis: 60_000,
});
export const testMeterProvider = new MeterProvider({ readers: [reader] });
metrics.setGlobalMeterProvider(testMeterProvider);
export async function flushTestMetrics(): Promise<
ReturnType<typeof testInMemoryMetricExporter.getMetrics>
> {
await testMeterProvider.forceFlush();
const collected = testInMemoryMetricExporter.getMetrics();
testInMemoryMetricExporter.reset();
return collected;
}

View File

@@ -0,0 +1,12 @@
{
"$schema": "https://json.schemastore.org/tsconfig",
"extends": "../../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "./src",
"outDir": "./dist",
"lib": ["ES2022", "DOM"],
"types": ["node"]
},
"include": ["src/**/*.ts"],
"exclude": ["src/**/*.test.ts", "dist", "node_modules"]
}

View File

@@ -0,0 +1,101 @@
import { resolve } from "node:path";
import { defineConfig } from "vite";
import dts from "vite-plugin-dts";
export default defineConfig({
root: __dirname,
cacheDir: "../../../node_modules/.vite/packages/email-processing",
plugins: [
dts({
entryRoot: "src",
tsconfigPath: "./tsconfig.lib.json",
insertTypesEntry: true,
}),
],
build: {
outDir: "./dist",
emptyOutDir: true,
reportCompressedSize: true,
sourcemap: true,
lib: {
entry: {
index: resolve(__dirname, "src/index.ts"),
"parser/index": resolve(__dirname, "src/parser/index.ts"),
"mime-utils/index": resolve(
__dirname,
"src/mime-utils/index.ts"
),
"hooks/index": resolve(__dirname, "src/hooks/index.ts"),
"attachment-processor/index": resolve(
__dirname,
"src/attachment-processor/index.ts"
),
"cid-resolver/index": resolve(
__dirname,
"src/cid-resolver/index.ts"
),
"calendar-parser/index": resolve(
__dirname,
"src/calendar-parser/index.ts"
),
"data-extractor/index": resolve(
__dirname,
"src/data-extractor/index.ts"
),
"dkim-canonicalize/index": resolve(
__dirname,
"src/dkim-canonicalize/index.ts"
),
"dkim-verifier/index": resolve(
__dirname,
"src/dkim-verifier/index.ts"
),
"dmarc-checker/index": resolve(
__dirname,
"src/dmarc-checker/index.ts"
),
"spf-checker/index": resolve(
__dirname,
"src/spf-checker/index.ts"
),
"spam-scorer/index": resolve(
__dirname,
"src/spam-scorer/index.ts"
),
"thread-grouper/index": resolve(
__dirname,
"src/thread-grouper/index.ts"
),
"error-codes/index": resolve(
__dirname,
"src/error-codes/index.ts"
),
"telemetry/index": resolve(__dirname, "src/telemetry/index.ts"),
},
formats: ["es"],
},
rollupOptions: {
treeshake: true,
output: {
preserveModules: true,
preserveModulesRoot: "src",
entryFileNames: "[name].js",
},
external: [
"@suaveplan/error",
"@suaveplan/events",
"@suaveplan/events/hooks",
"@suaveplan/events/hooks.types",
"@suaveplan/logger",
"@suaveplan/opentelemetry",
"@suaveplan/types",
"@opentelemetry/api",
"postal-mime",
"libmime",
"tslib",
"zod",
/^node:/,
],
},
},
});

View File

@@ -0,0 +1,32 @@
import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
globals: false,
environment: "node",
pool: "forks",
setupFiles: ["./tests/setup/polyfills.ts"],
include: ["src/**/*.test.ts", "tests/**/*.test.ts"],
exclude: ["dist/**", "node_modules/**"],
coverage: {
provider: "istanbul",
reporter: ["text", "html", "lcov"],
include: ["src/**/*.ts"],
exclude: [
"src/**/*.test.ts",
"src/**/*.types.ts",
"src/**/index.ts",
"src/index.ts",
"src/types/**",
"tests/**",
],
thresholds: {
statements: 100,
branches: 100,
functions: 100,
lines: 100,
},
},
testTimeout: 30000,
},
});