Move docs/ package mirror into document-navigator plugin
Relocate the ~6k-file docs tree from the repo root into plugins/document-navigator/docs/ so the plugin is self-contained and portable. docs/superpowers/ (specs and plans) stays at the repo root. Moved via cp+rm rather than git mv because directory rename() fails with EPERM on this WSL /mnt DrvFs mount; git rename-detection pairs the identical-content files at commit time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,121 @@
|
||||
# @suaveplan/email-processing - Universal MIME parsing and email processing: postal-mime + libmime wrappers, attachment processing with SHA-256 hashing + MIME-type allowlists, Content-ID resolution, structured data extraction (iCalendar, delivery tracking, receipts), DKIM/SPF/DMARC verification, spam scoring with pluggable rule sets, conversation threading, hook pipeline + OTel instrumentation
|
||||
|
||||
Universal MIME parsing and email processing for the Suaveplan
|
||||
. Wraps `postal-mime` and `libmime` with structured errors,
|
||||
OpenTelemetry instrumentation, a hook pipeline for transforming
|
||||
parsed messages, full attachment processing (SHA-256 hashing, MIME
|
||||
allowlists, CID resolution), structured-data extraction (iCalendar,
|
||||
delivery tracking, receipts), DKIM/SPF/DMARC verification, spam
|
||||
scoring, and conversation threading.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
|
||||
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
|
||||
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
|
||||
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
|
||||
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
|
||||
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
|
||||
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
|
||||
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
|
||||
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
|
||||
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
|
||||
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
|
||||
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
|
||||
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
|
||||
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
|
||||
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
|
||||
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
|
||||
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
|
||||
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
|
||||
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
|
||||
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
|
||||
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
|
||||
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
|
||||
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
|
||||
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
|
||||
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
|
||||
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
|
||||
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
|
||||
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
attachment-processor/ # SHA-256 hash + size + MIME allowlist validation
|
||||
calendar-parser/ # iCalendar VEVENT extraction
|
||||
cid-resolver/ # Content-ID resolution to data URIs / URL maps
|
||||
data-extractor/ # tracking numbers, receipts, calendar events
|
||||
dkim-canonicalize/ # RFC 6376 relaxed + simple canonicalisation
|
||||
dkim-verifier/ # Tag parsing, body-hash, RSASSA-PKCS1-v1_5 verify
|
||||
dmarc-checker/ # Policy + alignment over SPF + DKIM
|
||||
error-codes/ # EMAIL_PROCESSING_CODES + EmailProcessingError
|
||||
hooks/ # email:processing pipeline + singleton bus
|
||||
mime-utils/ # libmime wrappers (encode/decode words, folding)
|
||||
parser/ # parseEmail / parseEmailAddress / decodeEmailWords
|
||||
spam-scorer/ # 12 default rules + custom-rule merge
|
||||
spf-checker/ # RFC 7208 evaluator with include/redirect limits
|
||||
telemetry/ # Shared OTel meter + instruments
|
||||
thread-grouper/ # In-Reply-To + References + subject threading
|
||||
types/ # Shared MIME / address / header type aliases
|
||||
index.ts # Package barrel
|
||||
```
|
||||
|
||||
## Quality
|
||||
|
||||
- 100% statement / branch / function / line coverage (Istanbul).
|
||||
- Real `postal-mime` parses against embedded RFC 822 fixtures — no
|
||||
mocks (rule 2).
|
||||
- Test runner imports go through `@suaveplan/testing/runner` only
|
||||
(rule 11).
|
||||
- Zod-validated option boundaries (`parseEmail`, `processAttachment`).
|
||||
- Two error codes:
|
||||
`INVALID_INPUT`, `PARSE_FAILURE`, `INVALID_OPTIONS`,
|
||||
`ATTACHMENT_TOO_LARGE`, `ATTACHMENT_TYPE_REJECTED`, `HASH_FAILURE`,
|
||||
`DKIM_VERIFY_FAILURE`, `HOOK_FAILURE` — every code is thrown by at
|
||||
least one production path under `src/`.
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import {
|
||||
parseEmail,
|
||||
processAttachments,
|
||||
onEmailProcessing,
|
||||
} from "@suaveplan/email-processing";
|
||||
|
||||
onEmailProcessing((data) => ({
|
||||
...data,
|
||||
parsed: {
|
||||
...data.parsed,
|
||||
html: data.parsed.html?.replace(/<script[\s\S]*?<\/script>/gi, ""),
|
||||
},
|
||||
}));
|
||||
|
||||
const parsed = await parseEmail(rawMimeBuffer);
|
||||
const processed = await processAttachments(parsed.attachments, {
|
||||
maxSize: 10 * 1024 * 1024,
|
||||
allowedMimeTypes: ["image/*", "application/pdf"],
|
||||
});
|
||||
```
|
||||
|
||||
## Subpath exports
|
||||
|
||||
`./parser`, `./mime-utils`, `./hooks`, `./attachments`, `./cid-resolver`,
|
||||
`./extraction`, `./calendar`, `./dkim`, `./dmarc`, `./spf`, `./spam`,
|
||||
`./threading`, `./errors`, `./otel`.
|
||||
|
||||
## Provenance
|
||||
|
||||
Migrated from `packages/universal/email-processing` on the
|
||||
`/mnt/s/web/suaveplan` main branch per the enterprise plan at
|
||||
`/mnt/s/web/suaveplan/enterprise/email-processing.md`. Hook
|
||||
infrastructure was rewritten locally because the foundation
|
||||
`@suaveplan/events` library does not yet ship a shared
|
||||
`Hooks<T>` substrate.
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
# attachment-processor
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Normalises and validates `EmailAttachment` payloads emerging from
|
||||
`parseEmail`. Each attachment is decoded to bytes, validated against a
|
||||
caller-supplied size + MIME-type allowlist, hashed with SHA-256 over
|
||||
the Web Crypto API, and packaged as a `ProcessedAttachment` carrying
|
||||
the original metadata plus a deterministic content hash for dedup,
|
||||
provenance, and dehydrated-template workflows.
|
||||
|
||||
## Features
|
||||
|
||||
- Three input shapes: `ArrayBuffer`, base64 string (per `encoding`),
|
||||
or plain UTF-8 string — falls back to `Uint8Array(0)` for unknown
|
||||
shapes (matches upstream behaviour) so callers do not see crashes
|
||||
on rare attachment encodings.
|
||||
- Size limit defaults to 25 MiB; overrides validated via Zod.
|
||||
- MIME allowlist supports both exact match (`application/pdf`) and
|
||||
wildcard (`image/*`).
|
||||
- SHA-256 via `crypto.subtle.digest` — works in browsers, Node, Bun
|
||||
and Deno without polyfills. Hash failures surface as
|
||||
`EMAIL_PROCESSING.HASH_FAILURE` with the underlying error on
|
||||
`cause`.
|
||||
- OTel counters for processed vs rejected attachments (attributed by
|
||||
reason) and a duration histogram for hash + metadata extraction.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { processAttachment } from "@suaveplan/email-processing/attachments";
|
||||
|
||||
const out = await processAttachment(parsed.attachments[0]);
|
||||
if (out.valid) {
|
||||
await store({ hash: out.metadata.contentHash, bytes: out.content });
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
processAttachments,
|
||||
type ProcessedAttachment,
|
||||
} from "@suaveplan/email-processing/attachments";
|
||||
|
||||
const results = await processAttachments(parsed.attachments, {
|
||||
maxSize: 5 * 1024 * 1024,
|
||||
allowedMimeTypes: ["image/*", "application/pdf"],
|
||||
});
|
||||
|
||||
const valid = results.filter((r): r is ProcessedAttachment => r.valid);
|
||||
const rejected = results.filter((r) => !r.valid);
|
||||
log.info({
|
||||
valid: valid.length,
|
||||
rejected: rejected.length,
|
||||
hashes: valid.map((v) => v.metadata.contentHash),
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `AttachmentMetadata` — filename, size, MIME type, disposition,
|
||||
contentHash (hex SHA-256), optional contentId + description.
|
||||
- `ProcessedAttachment` — `{ metadata, content, valid, error? }`.
|
||||
- `AttachmentProcessorOptions` — `{ allowedMimeTypes?, maxSize? }`,
|
||||
Zod-validated.
|
||||
- `processAttachment(att, options?)` — returns `ProcessedAttachment`.
|
||||
- `processAttachments(list, options?)` — returns
|
||||
`ProcessedAttachment[]`.
|
||||
|
||||
Errors: `INVALID_OPTIONS` for bad options, `HASH_FAILURE` if the Web
|
||||
Crypto API throws (e.g. in environments where the digest is
|
||||
unavailable).
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`crypto.subtle.digest` requires the source buffer to satisfy the
|
||||
`BufferSource` type; passing a fresh `Uint8Array` works in every
|
||||
runtime tested. The duration histogram uses `performance.now()` when
|
||||
available and falls back to `Date.now()` for environments without it
|
||||
(older Node test isolates). Errors surfaced inside `extractMetadata`
|
||||
are propagated upward by `processAttachment` without being swallowed,
|
||||
so callers do not lose root-cause information when hashing fails.
|
||||
@@ -0,0 +1,81 @@
|
||||
# calendar-parser
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these
|
||||
arrive either as `text/calendar` MIME parts or as attachments with the
|
||||
`application/ics` media type. Only the subset of fields meaningful to
|
||||
email automation is extracted: summary, description, start/end,
|
||||
location, organizer, attendees, UID, and METHOD (REQUEST / REPLY /
|
||||
CANCEL). The parser is deliberately tolerant of folded continuation
|
||||
lines and ICS escape sequences.
|
||||
|
||||
## Features
|
||||
|
||||
- Handles RFC 5545 line folding — both `CRLF + SP` and `CRLF + TAB`
|
||||
continuations.
|
||||
- Unescapes the four standard ICS escapes: `\\n`, `\\,`, `\\;`, `\\\\`.
|
||||
- Extracts `mailto:` addresses from `ORGANIZER` and `ATTENDEE` lines
|
||||
regardless of preceding `CN` parameters.
|
||||
- Captures the calendar-level `METHOD` and surfaces it on every event
|
||||
so consumers can branch on REQUEST vs CANCEL.
|
||||
- Ignores lines outside `BEGIN:VEVENT` / `END:VEVENT` blocks so
|
||||
`VCALENDAR` boilerplate does not leak into event fields.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { parseICalendar } from "@suaveplan/email-processing/calendar";
|
||||
|
||||
const events = parseICalendar(icsAttachment);
|
||||
for (const event of events) {
|
||||
if (event.method === "CANCEL") {
|
||||
await cancelMeeting(event.uid);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { parseICalendar } from "@suaveplan/email-processing/calendar";
|
||||
|
||||
const events = parseICalendar(icsString);
|
||||
for (const event of events) {
|
||||
const attendees = event.attendees.map((mailto) => mailto.toLowerCase());
|
||||
await db.calendar.upsert({
|
||||
where: { uid: event.uid ?? hash(event.summary ?? "") },
|
||||
update: {
|
||||
summary: event.summary ?? null,
|
||||
dtStart: event.dtStart ?? null,
|
||||
dtEnd: event.dtEnd ?? null,
|
||||
location: event.location ?? null,
|
||||
attendees,
|
||||
},
|
||||
create: { ... },
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `CalendarEvent` — `{ summary?, description?, dtStart?, dtEnd?,
|
||||
location?, organizer?, attendees, uid?, method? }`.
|
||||
- `parseICalendar(icsContent: string): CalendarEvent[]` — returns an
|
||||
empty array when no `VEVENT` blocks are present. Throws
|
||||
`EmailProcessingError(INVALID_INPUT)` for non-string input.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
DTSTART and DTEND are returned as the raw ICS date-time strings (e.g.
|
||||
`20250101T120000Z`) without conversion. Consumers that need
|
||||
`Date`/`Temporal` instances should call their preferred parser — the
|
||||
package deliberately does not pick one, since `text/calendar` payloads
|
||||
arrive in many timezone shapes (floating, UTC, TZID-suffixed) and
|
||||
correct handling requires knowledge of the surrounding email context.
|
||||
The unfolding regexes operate on `\r\n` first so that `\n`-only ICS
|
||||
payloads still split correctly on the second `\r?\n` split call.
|
||||
@@ -0,0 +1,81 @@
|
||||
# cid-resolver
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Resolves Content-ID references inside HTML email bodies. Email clients
|
||||
that send inline images embed them as MIME parts with a
|
||||
`Content-ID` header and reference them in HTML with
|
||||
`<img src="cid:<id>">`. After parsing, the rendered HTML needs those
|
||||
references replaced by something a downstream consumer (an inbox UI,
|
||||
a digest builder, an audit trail) can serve — either a data URI, a
|
||||
static URL, or an explicit URL mapping.
|
||||
|
||||
## Features
|
||||
|
||||
- Three resolution strategies, evaluated in order: explicit
|
||||
`urlMap` (`Object.hasOwn`-checked so prototype keys cannot leak),
|
||||
`urlPrefix` + filename / contentHash, then data URI fallback.
|
||||
- Configurable `maxDataUriSize` (default 5 MiB) — attachments larger
|
||||
than the cap fall through to the unresolved list rather than
|
||||
exploding the HTML string.
|
||||
- Deduplicates CID references via `extractCidReferences`.
|
||||
- `isReferencedInline` short-circuits the lookup so callers can
|
||||
classify attachments as "inline" vs "true attachment" before
|
||||
serialising.
|
||||
- Strict input validation — non-string `html` inputs raise
|
||||
`EMAIL_PROCESSING.INVALID_INPUT`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { resolveCidReferences } from "@suaveplan/email-processing/cid-resolver";
|
||||
|
||||
const out = resolveCidReferences(parsed.html ?? "", processedAttachments);
|
||||
console.log(out.resolvedCount, out.unresolvedCids);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
resolveCidReferences,
|
||||
isReferencedInline,
|
||||
} from "@suaveplan/email-processing/cid-resolver";
|
||||
|
||||
const result = resolveCidReferences(parsed.html ?? "", processed, {
|
||||
urlMap: { "logo@brand": "https://cdn.brand/logo.png" },
|
||||
urlPrefix: "/inbox/inline/",
|
||||
maxDataUriSize: 2 * 1024 * 1024,
|
||||
});
|
||||
|
||||
const inlineAttachments = processed.filter((a) =>
|
||||
isReferencedInline(parsed.html ?? "", a)
|
||||
);
|
||||
const trueAttachments = processed.filter(
|
||||
(a) => !isReferencedInline(parsed.html ?? "", a)
|
||||
);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `CidResolverOptions` — `{ maxDataUriSize?, urlPrefix?, urlMap? }`.
|
||||
- `CidResolutionResult` — `{ html, totalReferences, resolvedCount, unresolvedCids }`.
|
||||
- `resolveCidReferences(html, attachments, options?)` — primary entry.
|
||||
- `extractCidReferences(html)` — deduped list of CIDs found.
|
||||
- `isReferencedInline(html, attachment)` — boolean.
|
||||
|
||||
Errors: `INVALID_INPUT` for non-string `html`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`Object.hasOwn(urlMap, cleanRef)` is used in preference to a direct
|
||||
truthy check so a deliberate `urlMap: { id: "" }` mapping is still
|
||||
recognised as "set", and so a polluted prototype cannot inject phantom
|
||||
keys. `String.prototype.matchAll` is used over `regex.exec` loops to
|
||||
keep the resolver simple and avoid stateful `lastIndex` bugs. The data
|
||||
URI fallback uses `btoa(String.fromCharCode(...))` so it works in the
|
||||
browser and Node without buffer polyfills.
|
||||
@@ -0,0 +1,75 @@
|
||||
# data-extractor
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Extracts structured business signals — calendar events, parcel
|
||||
tracking numbers, and receipt / order data — from the freeform text
|
||||
and HTML of a parsed email. The extractor is heuristic by design:
|
||||
patterns are tuned against a sample of real shipping carrier and
|
||||
e-commerce templates and prioritise high precision over recall.
|
||||
Calendar extraction is delegated to `calendar-parser`.
|
||||
|
||||
## Features
|
||||
|
||||
- UPS and USPS tracking-number recognition with carrier-specific
|
||||
tracking URLs.
|
||||
- Deduplication of repeated tracking numbers inside the same body.
|
||||
- Receipt parsing covering order/confirmation/reference numbers,
|
||||
totals, and currency (USD / EUR / GBP / CAD / AUD plus `$`).
|
||||
- Falls back to stripped HTML when no plain-text body is present.
|
||||
- Handles attachment payloads in any of the three shapes
|
||||
(`ArrayBuffer`, `string`, or `Uint8Array`) by decoding to UTF-8.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { extractData } from "@suaveplan/email-processing/extraction";
|
||||
|
||||
const out = extractData(parsed.text, parsed.html, parsed.attachments);
|
||||
console.log(out.deliveryTracking, out.receipts, out.calendarEvents);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
extractData,
|
||||
extractTrackingInfo,
|
||||
extractReceiptData,
|
||||
} from "@suaveplan/email-processing/extraction";
|
||||
|
||||
// Bulk classify recent inbox messages.
|
||||
const enriched = await Promise.all(
|
||||
messages.map(async (msg) => ({
|
||||
message: msg,
|
||||
signals: extractData(msg.text, msg.html, msg.attachments),
|
||||
}))
|
||||
);
|
||||
const shipments = enriched.flatMap((m) => m.signals.deliveryTracking);
|
||||
const receipts = enriched.flatMap((m) => m.signals.receipts);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `DeliveryTracking` — `{ carrier?, trackingNumber, trackingUrl? }`.
|
||||
- `ReceiptData` — `{ orderNumber?, total?, currency?, merchant?, date? }`.
|
||||
- `ExtractedData` — `{ calendarEvents, deliveryTracking, receipts }`.
|
||||
- `extractTrackingInfo(text)` — UPS/USPS hits, deduped.
|
||||
- `extractReceiptData(text)` — single `ReceiptData` or `null`.
|
||||
- `extractData(text?, html?, attachments?)` — orchestrator that
|
||||
combines all three.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`extractData` only falls back to stripped HTML when **no** plain-text
|
||||
body is present so well-formed multipart/alternative messages do not
|
||||
double-count tracking numbers. Attachment decoding tries the simplest
|
||||
shape first (`string`), then `ArrayBuffer`, then `Uint8Array`. The
|
||||
order-pattern alternations are anchored on common e-commerce labels
|
||||
(`order`, `confirmation`, `reference`) plus permissive punctuation;
|
||||
they reject anything that does not include an alphanumeric token
|
||||
after the label so single-word noise does not produce false positives.
|
||||
@@ -0,0 +1,86 @@
|
||||
# dkim-canonicalize
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements RFC 6376 canonicalisation primitives — the byte-exact text
|
||||
normalisation steps DKIM requires before hashing a header set or body.
|
||||
Both `simple` and `relaxed` modes are supported for headers and
|
||||
bodies, plus `buildSignedHeaderBlock` which combines the canonicalised
|
||||
signed headers with the canonicalised `DKIM-Signature` line (sans
|
||||
`b=` value) that callers feed into RSA `verify`.
|
||||
|
||||
## Features
|
||||
|
||||
- `canonicalizeHeaderRelaxed` — lower-case name, collapse folded
|
||||
whitespace, trim, single-space internal runs.
|
||||
- `canonicalizeBodyRelaxed` — CRLF-normalise, trim trailing per-line
|
||||
WS, collapse internal runs, strip trailing empty lines, ensure one
|
||||
final CRLF.
|
||||
- `canonicalizeBodySimple` — CRLF-normalise + ensure exactly one
|
||||
final CRLF.
|
||||
- `buildSignedHeaderBlock` — walks `signedHeaders` in order, indexes
|
||||
repeated headers, removes the `b=` payload of the
|
||||
`DKIM-Signature` line per RFC, and joins with CRLF.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { canonicalizeBodyRelaxed } from "@suaveplan/email-processing/dkim";
|
||||
|
||||
const canon = canonicalizeBodyRelaxed(body);
|
||||
const digest = await crypto.subtle.digest(
|
||||
"SHA-256",
|
||||
new TextEncoder().encode(canon)
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
buildSignedHeaderBlock,
|
||||
canonicalizeBodyRelaxed,
|
||||
} from "@suaveplan/email-processing/dkim";
|
||||
|
||||
const canonHeaders = buildSignedHeaderBlock(headers, sig);
|
||||
const canonBody = canonicalizeBodyRelaxed(body);
|
||||
const bodyHash = b64(
|
||||
await crypto.subtle.digest(
|
||||
"SHA-256",
|
||||
new TextEncoder().encode(canonBody)
|
||||
)
|
||||
);
|
||||
const ok = await crypto.subtle.verify(
|
||||
"RSASSA-PKCS1-v1_5",
|
||||
key,
|
||||
sigBytes,
|
||||
new TextEncoder().encode(canonHeaders)
|
||||
);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `canonicalizeHeaderRelaxed(name, value): string` — single-line.
|
||||
- `canonicalizeBodyRelaxed(body): string`.
|
||||
- `canonicalizeBodySimple(body): string`.
|
||||
- `buildSignedHeaderBlock(headers, sig): string`.
|
||||
|
||||
Types `DkimSignature`, `DkimResult`, `DkimVerificationResult`,
|
||||
`DkimDnsResolver` are re-exported from the package root for callers
|
||||
that consume the verifier.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The relaxed body canonicalisation collapses runs of tabs and spaces
|
||||
inside a line **before** stripping trailing whitespace, matching the
|
||||
order specified in RFC 6376 §3.4.4. Empty bodies still emit the
|
||||
mandated trailing CRLF after `simple` canonicalisation so a hash
|
||||
mismatch is not produced by a body that legitimately ends without one.
|
||||
`buildSignedHeaderBlock` keeps a per-name index counter so a signature
|
||||
that lists `from` then `from` retrieves the first and second `From:`
|
||||
header in order — without the counter, the DKIM rebuild would emit the
|
||||
same header twice and verification would fail on otherwise valid mail.
|
||||
@@ -0,0 +1,84 @@
|
||||
# dkim-verifier
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements DKIM (RFC 6376) signature verification — parses a
|
||||
`DKIM-Signature` header, looks up the signer's public key through an
|
||||
injectable DNS resolver, recomputes the body hash, rebuilds the
|
||||
canonicalised signed-header block, and calls Web Crypto's
|
||||
RSASSA-PKCS1-v1_5 verify. Returns a structured outcome that callers
|
||||
plug into DMARC alignment, spam scoring, and inbox display.
|
||||
|
||||
## Features
|
||||
|
||||
- Strict tag parsing with explicit nullability — missing `v`/`a`/`d`/
|
||||
`s`/`h`/`bh`/`b` returns `null` (treated as `fail` by the
|
||||
verifier).
|
||||
- Defaults `c` to `simple/simple`, accepts shorthand `c=relaxed`
|
||||
(header relaxed, body simple).
|
||||
- Honours optional `l=` body-length tag, `t=` timestamp, `x=`
|
||||
expiration so callers can surface staleness.
|
||||
- DNS resolver is fully injectable — the package never calls `dns.*`
|
||||
directly, so consumers can route through any DoH / cache /
|
||||
`@suaveplan/dns` strategy.
|
||||
- Hard error path: a structurally-broken public-key DNS record raises
|
||||
`EMAIL_PROCESSING.DKIM_VERIFY_FAILURE` rather than silently
|
||||
failing, so observability dashboards distinguish "signer is
|
||||
misconfigured" from "this message did not verify".
|
||||
- OTel counter `email_processing.dkim.result_total` attributed by
|
||||
`result` for every outcome.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { verifyDkim } from "@suaveplan/email-processing/dkim";
|
||||
|
||||
const out = await verifyDkim(headers, body, async (selector, domain) => {
|
||||
const records = await dns.resolveTxt(`${selector}._domainkey.${domain}`);
|
||||
return records[0]?.join("") ?? null;
|
||||
});
|
||||
if (out.result === "pass") authenticate();
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
extractDkimHeader,
|
||||
parseDkimSignature,
|
||||
verifyDkim,
|
||||
} from "@suaveplan/email-processing/dkim";
|
||||
|
||||
const raw = extractDkimHeader(headers);
|
||||
if (raw) {
|
||||
const sig = parseDkimSignature(raw);
|
||||
if (sig && sig.expiration && sig.expiration < Date.now() / 1000) {
|
||||
log.warn({ domain: sig.domain }, "stale DKIM signature");
|
||||
}
|
||||
}
|
||||
|
||||
const result = await verifyDkim(headers, body, dnsResolver);
|
||||
metrics.recordDkim(result.domain ?? "unknown", result.result);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `parseDkimSignature(value): DkimSignature | null`.
|
||||
- `extractDkimHeader(headers): string | null`.
|
||||
- `verifyDkim(headers, body, dnsResolver): Promise<DkimVerificationResult>`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The verifier strips the `b=` value from the `DKIM-Signature` header
|
||||
before feeding the canonicalised block into the verify call — this
|
||||
matches RFC 6376 §3.5 and ensures consumers who pass the
|
||||
`DKIM-Signature` header back through unchanged still produce the
|
||||
correct hash. The `catch` arm of the main `try/try-catch` block is
|
||||
intentionally narrow: it re-throws our own `EmailProcessingError` so
|
||||
unrecoverable configuration errors are not silently downgraded to
|
||||
`fail` outcomes, but every other thrown value becomes a `fail` with a
|
||||
diagnostic `error` field.
|
||||
@@ -0,0 +1,78 @@
|
||||
# dmarc-checker
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements DMARC (RFC 7489) policy evaluation on top of an upstream
|
||||
SPF result and DKIM verification. Looks up the `_dmarc.<from-domain>`
|
||||
TXT record through an injectable resolver, parses its policy and
|
||||
alignment tags, then determines whether at least one of SPF or DKIM
|
||||
passes **and** aligns with the From header's organisational domain.
|
||||
|
||||
## Features
|
||||
|
||||
- `parseDmarcRecord` returns a strongly-typed `DmarcRecord` with all
|
||||
standard tags (`p`, `sp`, `adkim`, `aspf`, `pct`, `rua`, `ruf`).
|
||||
- `checkDmarc` accepts pre-computed SPF and DKIM outcomes so callers
|
||||
can compose authentication pipelines without re-running expensive
|
||||
DNS or crypto work.
|
||||
- Both alignment modes — strict (exact domain match) and relaxed
|
||||
(organisational-domain match) — are supported.
|
||||
- DNS resolver is async and injectable; the package itself never
|
||||
calls into `dns.*` directly.
|
||||
- Records `email_processing.dmarc.result_total` attributed by
|
||||
`result` for every outcome.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { checkDmarc } from "@suaveplan/email-processing/dmarc";
|
||||
|
||||
const out = await checkDmarc(fromDomain, spfResult, dkimResult, async (d) => {
|
||||
const recs = await dns.resolveTxt(d);
|
||||
return recs.map((p) => p.join(""));
|
||||
});
|
||||
if (out.result === "fail" && out.policy === "reject") {
|
||||
rejectMessage();
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { checkDmarc, parseDmarcRecord } from "@suaveplan/email-processing/dmarc";
|
||||
|
||||
const recs = await dnsResolver(`_dmarc.${domain}`);
|
||||
const dmarcText = recs.find((r) => r.startsWith("v=DMARC1")) ?? "";
|
||||
const parsed = parseDmarcRecord(dmarcText);
|
||||
const aggregateUri = parsed?.reportUri;
|
||||
if (aggregateUri) {
|
||||
await scheduleReport({ domain, uri: aggregateUri });
|
||||
}
|
||||
const result = await checkDmarc(domain, spf, dkim, dnsResolver);
|
||||
metrics.recordDmarc(domain, result.result, result.policy);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `DmarcPolicy` — `"none" | "quarantine" | "reject"`.
|
||||
- `DmarcAlignment` — `"strict" | "relaxed"`.
|
||||
- `DmarcResult` — `"pass" | "fail" | "none"`.
|
||||
- `DmarcRecord` — full parsed record.
|
||||
- `DmarcCheckResult` — `{ result, policy?, spfAligned, dkimAligned, record?, error? }`.
|
||||
- `parseDmarcRecord(txt)` — strict parser, returns `null` on invalid
|
||||
input.
|
||||
- `checkDmarc(fromDomain, spf, dkim, resolver)` — async orchestrator.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The DMARC TXT record lookup uses the bare `_dmarc.<domain>` form per
|
||||
RFC 7489 §6.6.3 (organisational-domain fallback is left to a higher
|
||||
layer — callers who want PSL-aware fallback should chain a second
|
||||
`checkDmarc` against the parent domain when the first returns
|
||||
`result: "none"` with `error: "No DMARC record found"`). The
|
||||
`reportDmarc` helper centralises OTel emission so every return path
|
||||
records the outcome exactly once.
|
||||
@@ -0,0 +1,84 @@
|
||||
# error-codes
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines the stable error-code registry and typed error class for the
|
||||
`@suaveplan/email-processing` package. Every failure raised inside the
|
||||
package goes through `EmailProcessingError`, carrying a discriminant
|
||||
`code` drawn from `EMAIL_PROCESSING_CODES` so callers can branch on the
|
||||
error without parsing a (possibly i18n-keyed) `message` string. The
|
||||
registry is closed: removing a code without removing its only call site
|
||||
is forbidden by `AGENTS.md §2c`, and adding a code requires an
|
||||
OpenSpec change.
|
||||
|
||||
## Features
|
||||
|
||||
- Eleven namespaced codes covering input validation, parse failures,
|
||||
attachment limits, hash failures, DKIM/DMARC/SPF outcomes, calendar
|
||||
parsing, hook-pipeline failures.
|
||||
- `EmailProcessingError` extends `SuaveplanError<EmailProcessingCode>`
|
||||
with a `public` constructor (the base constructor is `protected`).
|
||||
- Compatible with `@suaveplan/error`'s i18n resolver, cause chaining,
|
||||
redaction, OTel span-event adapter, and Zod wire schema.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import {
|
||||
EMAIL_PROCESSING_CODES,
|
||||
EmailProcessingError,
|
||||
} from "@suaveplan/email-processing/errors";
|
||||
|
||||
throw new EmailProcessingError(
|
||||
EMAIL_PROCESSING_CODES.INVALID_INPUT,
|
||||
"email_processing.invalid_input"
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { EmailProcessingError } from "@suaveplan/email-processing/errors";
|
||||
|
||||
try {
|
||||
await parseEmail(buffer);
|
||||
} catch (err) {
|
||||
if (err instanceof EmailProcessingError) {
|
||||
if (err.code === "EMAIL_PROCESSING.ATTACHMENT_TOO_LARGE") {
|
||||
logger.warn({ code: err.code }, "reject oversized attachment");
|
||||
return;
|
||||
}
|
||||
if (err.code === "EMAIL_PROCESSING.PARSE_FAILURE") {
|
||||
throw new EmailProcessingError(
|
||||
"EMAIL_PROCESSING.PARSE_FAILURE",
|
||||
"email_processing.parse_failure",
|
||||
{ cause: err }
|
||||
);
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `EMAIL_PROCESSING_CODES` — frozen literal map, value of each key is
|
||||
the dotted code (e.g. `EMAIL_PROCESSING.INVALID_INPUT`).
|
||||
- `EmailProcessingCode` — union of every `EMAIL_PROCESSING_CODES[K]`.
|
||||
- `EmailProcessingError` — class with `public constructor(code,
|
||||
messageKey, options?)` matching `SuaveplanErrorOptions`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`defineCodes` from `@suaveplan/error` enforces uniqueness, freezes the
|
||||
output, and brands every value with the namespace prefix. The class's
|
||||
`public` constructor is intentional: tests, consumers, and downstream
|
||||
packages need to construct `EmailProcessingError` instances directly,
|
||||
and `SuaveplanError`'s `protected` constructor would block that without
|
||||
the delegating override. The `name` field is set to
|
||||
`"EmailProcessingError"` so stack traces and serialised wire format
|
||||
remain readable.
|
||||
@@ -0,0 +1,99 @@
|
||||
# hooks
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements the package's `email:processing` hook pipeline: every parsed
|
||||
email passes through a sequence of handlers that may sanitise, redact,
|
||||
or enrich the parsed result. The implementation is intentionally local
|
||||
to this package — the foundation `@suaveplan/events` library
|
||||
does not (yet) export a shared `Hooks<T>` substrate, so a focused
|
||||
async pipeline lives here. The public shape matches the upstream
|
||||
`@suaveplan/email-processing` package so consumer code does not change.
|
||||
|
||||
## Features
|
||||
|
||||
- Generic `Hooks<TMap>` class with explicit type-safe `hook` /
|
||||
`applyHooksAsync` / `clear` methods.
|
||||
- Handlers run in `order` (default 100, lower first) and may be
|
||||
sync or async — `applyHooksAsync` awaits each return value.
|
||||
- Errors thrown from any handler short-circuit the pipeline and are
|
||||
wrapped in an `EmailProcessingError` with code
|
||||
`EMAIL_PROCESSING.HOOK_FAILURE`. The original throw is preserved
|
||||
on `.cause` when it is an `Error`.
|
||||
- Each invocation increments the `email_processing.hook.invocation_total`
|
||||
OTel counter with `event` and `handlers` attributes.
|
||||
- Returned data is threaded through every handler so transforms compose.
|
||||
- Module-level `emailHooks` singleton + `onEmailProcessing` shorthand.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { onEmailProcessing } from "@suaveplan/email-processing/hooks";
|
||||
|
||||
onEmailProcessing((data) => ({
|
||||
...data,
|
||||
parsed: {
|
||||
...data.parsed,
|
||||
html: data.parsed.html?.replace(
|
||||
/<img[^>]*tracking[^>]*>/gi,
|
||||
""
|
||||
),
|
||||
},
|
||||
}));
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { Hooks, type EmailHookMap } from "@suaveplan/email-processing/hooks";
|
||||
|
||||
const hooks = new Hooks<EmailHookMap>();
|
||||
const offFirst = hooks.hook(
|
||||
"email:processing",
|
||||
async (d) => {
|
||||
await audit.log({ messageId: d.parsed.messageId });
|
||||
return d;
|
||||
},
|
||||
{ order: 10 }
|
||||
);
|
||||
const offRedact = hooks.hook(
|
||||
"email:processing",
|
||||
(d) => ({
|
||||
...d,
|
||||
parsed: { ...d.parsed, text: redact(d.parsed.text ?? "") },
|
||||
}),
|
||||
{ order: 200 }
|
||||
);
|
||||
try {
|
||||
const out = await hooks.applyHooksAsync("email:processing", incoming);
|
||||
return out;
|
||||
} finally {
|
||||
offFirst();
|
||||
offRedact();
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `EmailProcessingHookData` — pipeline payload (`parsed`, `rawInput?`,
|
||||
`metadata`).
|
||||
- `EmailHookMap` — `{ "email:processing": EmailProcessingHookData }`.
|
||||
- `HookHandler<T>` — `(data: T) => T | Promise<T>`.
|
||||
- `HookHandlerOptions` — `{ order?: number }`.
|
||||
- `Hooks<TMap>` — class with `hook`, `applyHooksAsync`, `clear`.
|
||||
- `emailHooks` — singleton `Hooks<EmailHookMap>` instance.
|
||||
- `onEmailProcessing(handler, options?)` — shorthand registration on
|
||||
`emailHooks`; returns an idempotent unregister.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The unregister function returned from `hook(...)` is idempotent: calling
|
||||
it twice is a no-op so cleanup code in shared test fixtures can run
|
||||
without ordering pitfalls. The pipeline does not call the OTel counter
|
||||
when a handler throws — failures are observed via the
|
||||
`EMAIL_PROCESSING.HOOK_FAILURE` error path, and a separate consumer
|
||||
counter is the appropriate place to record those.
|
||||
@@ -0,0 +1,87 @@
|
||||
# mime-utils
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Wraps the `libmime` library with a strict, typed boundary so the rest
|
||||
of the package can call RFC 2047 / 2231 / 5322 encoding and decoding
|
||||
helpers without using `// @ts-expect-error` or leaking `any` through
|
||||
the consumer surface. Every helper validates its primary string
|
||||
argument and raises an `EmailProcessingError` with code
|
||||
`EMAIL_PROCESSING.INVALID_INPUT` rather than silently calling `libmime`
|
||||
with garbage.
|
||||
|
||||
## Features
|
||||
|
||||
- Eleven wrappers: `encodeWord` / `encodeWords` / `decodeWords`,
|
||||
`decodeHeader` / `decodeHeaders`, `parseHeaderValue` /
|
||||
`buildHeaderValue` / `buildHeaderParam`, `foldLines`,
|
||||
`encodeFlowed` / `decodeFlowed`.
|
||||
- Local ambient declaration (`libmime.d.ts`) so the package compiles
|
||||
without `@types/libmime` while keeping the inner types narrow.
|
||||
- Strict input validation at every entry point — non-string args
|
||||
raise `INVALID_INPUT`, which is testable and observable.
|
||||
- Pass-through of optional parameters such as `maxLength`,
|
||||
`lineLength`, `afterSpace`, `delSp` so callers retain full
|
||||
`libmime` flexibility.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { encodeWord, decodeWords } from "@suaveplan/email-processing/mime-utils";
|
||||
|
||||
const encoded = encodeWord("résumé", "Q");
|
||||
const original = decodeWords(encoded);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
parseHeaderValue,
|
||||
buildHeaderValue,
|
||||
buildHeaderParam,
|
||||
foldLines,
|
||||
} from "@suaveplan/email-processing/mime-utils";
|
||||
|
||||
const parsed = parseHeaderValue('text/plain; charset="utf-8"; name="r.txt"');
|
||||
const segments = buildHeaderParam("filename*", "réport-2024.pdf", 60);
|
||||
parsed.params.filename = segments.map((s) => s.value).join("");
|
||||
|
||||
const header = foldLines(
|
||||
`Content-Type: ${buildHeaderValue(parsed)}`,
|
||||
76,
|
||||
true
|
||||
);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `encodeWord(str, encoding?, maxLength?)`
|
||||
- `encodeWords(str, encoding?, maxLength?)`
|
||||
- `decodeWords(str)`
|
||||
- `decodeHeader(headerLine)` → `{ key, value }`
|
||||
- `decodeHeaders(headers)` → `Record<string, string[]>`
|
||||
- `parseHeaderValue(valueString)` → `StructuredHeaderValue`
|
||||
- `buildHeaderValue(structured)` → `string`
|
||||
- `buildHeaderParam(key, value, maxLength?)` → `HeaderParam[]`
|
||||
- `foldLines(str, lineLength?, afterSpace?)`
|
||||
- `encodeFlowed(str, lineLength?)`
|
||||
- `decodeFlowed(str, delSp?)`
|
||||
|
||||
All throw `EmailProcessingError` with code
|
||||
`EMAIL_PROCESSING.INVALID_INPUT` when their primary argument is missing
|
||||
or of the wrong type.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`libmime` ships without TypeScript declarations, so this module owns a
|
||||
local `libmime.d.ts` declaring only the subset of functions in use.
|
||||
That keeps the package compatible with strict `verbatimModuleSyntax`
|
||||
and `noUncheckedIndexedAccess` settings without `any` casts. The
|
||||
`buildHeaderValue` guard checks for `null`/`undefined`/non-object
|
||||
rather than `typeof === "string"` because the input is structured —
|
||||
mismatching that guard would let through invalid shapes.
|
||||
@@ -0,0 +1,85 @@
|
||||
# parser
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Wraps `postal-mime` with a strict input/option boundary, OTel
|
||||
instrumentation, structured error codes, and the
|
||||
`@suaveplan/email-processing/hooks` pipeline. `parseEmail` is the
|
||||
package's primary entry point: it takes a raw RFC 822 message in any
|
||||
of the three supported input shapes (string, `ArrayBuffer`,
|
||||
`Uint8Array`) and returns a fully-parsed `ParsedEmail` whose
|
||||
attachments are real MIME parts (no stubs).
|
||||
|
||||
## Features
|
||||
|
||||
- Three input modes — string, `ArrayBuffer`, `Uint8Array` — covering
|
||||
both browser and Node consumers.
|
||||
- Zod-validated options at the boundary, raising
|
||||
`EMAIL_PROCESSING.INVALID_OPTIONS` with the failing issues on
|
||||
`context` so callers can surface diagnostics.
|
||||
- Distinct error codes for `INVALID_INPUT` (null/undefined/wrong type)
|
||||
and `PARSE_FAILURE` (`postal-mime` threw); `parse.failure_total`
|
||||
is incremented for the latter.
|
||||
- Hook pipeline integration — every parse returns the result of
|
||||
`emailHooks.applyHooksAsync("email:processing", …)`.
|
||||
- `parseEmailAddress` and `decodeEmailWords` re-export the
|
||||
`postal-mime` low-level helpers under the package's typed surface,
|
||||
with the same `INVALID_INPUT` guard.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { parseEmail } from "@suaveplan/email-processing/parser";
|
||||
|
||||
const parsed = await parseEmail(rawMimeString);
|
||||
console.log(parsed.subject, parsed.attachments.length);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { parseEmail } from "@suaveplan/email-processing/parser";
|
||||
import { onEmailProcessing } from "@suaveplan/email-processing/hooks";
|
||||
|
||||
const off = onEmailProcessing((data) => ({
|
||||
...data,
|
||||
parsed: {
|
||||
...data.parsed,
|
||||
html: data.parsed.html?.replace(/<script[\s\S]*?<\/script>/gi, ""),
|
||||
},
|
||||
}));
|
||||
|
||||
try {
|
||||
const parsed = await parseEmail(buffer, {
|
||||
attachmentEncoding: "base64",
|
||||
maxNestingDepth: 64,
|
||||
});
|
||||
await persist(parsed);
|
||||
} finally {
|
||||
off();
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `parseEmail(raw, options?)` — async; returns `ParsedEmail`. Throws
|
||||
`EmailProcessingError` with codes `INVALID_INPUT`,
|
||||
`INVALID_OPTIONS`, or `PARSE_FAILURE`.
|
||||
- `parseEmailAddress(addressString)` — sync; returns `EmailAddress[]`.
|
||||
Throws `EmailProcessingError(INVALID_INPUT)` for non-string input.
|
||||
- `decodeEmailWords(text)` — sync; returns decoded RFC 2047 string.
|
||||
Throws `EmailProcessingError(INVALID_INPUT)` for non-string input.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`parseEmail` records `email_processing.parse.total` with the attachment
|
||||
count as an attribute so dashboards can surface average
|
||||
attachments-per-message. Parse failures increment a separate
|
||||
`parse.failure_total` counter (no attachment count is recorded because
|
||||
the parser never reached the attachment array). The Zod options schema
|
||||
is `.strict()` so unknown option keys are rejected — this catches
|
||||
typos at the boundary instead of letting them silently no-op.
|
||||
@@ -0,0 +1,97 @@
|
||||
# spam-scorer
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Heuristic spam scorer with a pluggable rule set. The default rules
|
||||
cover the most common spam signals — missing headers, all-caps
|
||||
subjects, common keyword lists, URL counts, low text-to-HTML ratio,
|
||||
DKIM / SPF authentication outcomes — and produce a clamped 0..10
|
||||
score plus a `clean` / `suspicious` / `spam` classification. Custom
|
||||
rules can be merged into the defaults, with conflict resolution by
|
||||
`id`.
|
||||
|
||||
## Features
|
||||
|
||||
- Twelve default rules covering header completeness, content, and
|
||||
authentication signals.
|
||||
- Configurable suspicious / spam thresholds (defaults: 3 / 6).
|
||||
- Custom rules merge with defaults via `id`; same-id custom rules
|
||||
replace the default in place, preserving merge order.
|
||||
- Records `email_processing.spam.scored_total` attributed by
|
||||
`classification`.
|
||||
- Pure: no DNS / network calls, fully deterministic given a fixed
|
||||
`SpamContext`.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { scoreSpam } from "@suaveplan/email-processing/spam";
|
||||
|
||||
const result = scoreSpam({
|
||||
subject: parsed.subject,
|
||||
text: parsed.text,
|
||||
html: parsed.html,
|
||||
fromAddress: (parsed.from as { address?: string } | undefined)?.address,
|
||||
headers: parsed.headers,
|
||||
attachmentCount: parsed.attachments.length,
|
||||
});
|
||||
if (result.classification === "spam") quarantine();
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
scoreSpam,
|
||||
DEFAULT_SPAM_RULES,
|
||||
type SpamRule,
|
||||
} from "@suaveplan/email-processing/spam";
|
||||
|
||||
const customRules: SpamRule[] = [
|
||||
{
|
||||
id: "competitor-domain",
|
||||
description: "From a known competitor domain",
|
||||
weight: 5,
|
||||
evaluate: (ctx) =>
|
||||
ctx.fromAddress?.endsWith("@competitor.example") ?? false,
|
||||
},
|
||||
];
|
||||
|
||||
const result = scoreSpam(
|
||||
{
|
||||
subject: msg.subject,
|
||||
text: msg.text,
|
||||
html: msg.html,
|
||||
fromAddress: msg.from,
|
||||
headers: msg.headers,
|
||||
attachmentCount: msg.attachments.length,
|
||||
dkimResult: dkim.result,
|
||||
spfResult: spf.result,
|
||||
},
|
||||
{ customRules, suspiciousThreshold: 2, spamThreshold: 5 }
|
||||
);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `SpamRule` — `{ id, description, weight, evaluate(ctx) }`.
|
||||
- `SpamContext` — full payload passed to each rule.
|
||||
- `SpamScoreResult` — `{ score, triggeredRules, classification }`.
|
||||
- `SpamScorerOptions` — `{ customRules?, suspiciousThreshold?, spamThreshold? }`.
|
||||
- `DEFAULT_SPAM_RULES` — array of the twelve built-in rules.
|
||||
- `scoreSpam(context, options?)` — pure scorer.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The scorer clamps the raw weighted sum at `[0, 10]` so callers can
|
||||
rely on a fixed range when piping into a UI badge or alerting
|
||||
threshold. Rule weights are tuned against a hand-classified corpus
|
||||
and intentionally favour precision over recall — false-positive
|
||||
quarantines are more disruptive than false-negative deliveries.
|
||||
Custom rules with same `id` replace the default rule entirely (no
|
||||
weight-sum merge) so consumers retain full control over which rules
|
||||
fire.
|
||||
@@ -0,0 +1,74 @@
|
||||
# spf-checker
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Implements the SPF (RFC 7208) evaluation algorithm: parses the
|
||||
record, walks each mechanism in order, follows `include:` and
|
||||
`redirect=` with bounded depth and lookup limits, and matches the
|
||||
candidate IPv4 / IPv6 address against `ip4:` and `ip6:` CIDR ranges.
|
||||
The DNS resolver is fully injectable so the package can be embedded
|
||||
in browsers, edge workers, or sandboxed runtimes that need to route
|
||||
lookups through DoH or a service mesh.
|
||||
|
||||
## Features
|
||||
|
||||
- Parses every mechanism token (`ip4`, `ip6`, `a`, `mx`, `include`,
|
||||
`all`, `exists`, `redirect`) and their qualifier prefixes
|
||||
(`+ - ~ ?`).
|
||||
- Bounded recursion: `MAX_INCLUDE_DEPTH = 10` and `MAX_DNS_LOOKUPS =
|
||||
10` to surface RFC 7208 §4.6.4 permerror conditions instead of
|
||||
hanging.
|
||||
- IPv4 matching with `Number.parseInt` + bitmasks; IPv6 matching by
|
||||
expanding `::` shorthand and comparing bit prefixes.
|
||||
- Records `email_processing.spf.result_total` attributed by `result`
|
||||
for every outcome.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { checkSpf } from "@suaveplan/email-processing/spf";
|
||||
|
||||
const out = await checkSpf(senderIp, fromDomain, async (d) => {
|
||||
const recs = await dns.resolveTxt(d);
|
||||
return recs.map((parts) => parts.join(""));
|
||||
});
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import { checkSpf, parseSpfRecord } from "@suaveplan/email-processing/spf";
|
||||
|
||||
const recs = await dnsResolver(domain);
|
||||
const v = recs.find((r) => r.startsWith("v=spf1"));
|
||||
if (v) {
|
||||
const mechs = parseSpfRecord(v);
|
||||
if (mechs.some((m) => m.type === "all" && m.qualifier === "+")) {
|
||||
log.warn({ domain }, "open SPF policy (+all)");
|
||||
}
|
||||
}
|
||||
const out = await checkSpf(ip, domain, dnsResolver);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `parseSpfRecord(record): SpfMechanism[]`.
|
||||
- `ipv4Matches(ip, cidr): boolean`.
|
||||
- `ipv6Matches(ip, cidr): boolean`.
|
||||
- `checkSpf(ip, domain, resolver, depth?, lookupCount?): Promise<SpfCheckResult>`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`a` / `mx` / `exists` mechanisms are recognised but treated as
|
||||
non-matches because resolving them requires additional DNS lookups
|
||||
the caller may not want to fund — most real-world SPF records lean on
|
||||
`ip4:` / `ip6:` / `include:` anyway. The `lookupCount` parameter is
|
||||
exposed so callers can compose `checkSpf` across multiple top-level
|
||||
domains in a single run without re-instantiating the budget. The
|
||||
function records the SPF outcome to the OTel counter before returning,
|
||||
including the `permerror` paths, so dashboards can distinguish "bad
|
||||
record" from "no record".
|
||||
@@ -0,0 +1,77 @@
|
||||
# telemetry
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Provides the shared OpenTelemetry meter and instrument set used by every
|
||||
public entry point of `@suaveplan/email-processing`. Instruments are
|
||||
constructed lazily on first call, then cached so each metric is bound
|
||||
exactly once per process — this is what makes the package's
|
||||
spec-conformance test (which runs against a real `MeterProvider` +
|
||||
`InMemoryMetricExporter`) deterministic. There are no mocks: the test
|
||||
setup installs a real meter provider before any source module imports.
|
||||
|
||||
## Features
|
||||
|
||||
- Lazy, idempotent singleton (`getEmailProcessingInstruments`) with an
|
||||
explicit test-only `resetEmailProcessingInstruments` for re-binding
|
||||
after a fresh meter provider is installed in setup.
|
||||
- Stable instrument names under the `email_processing.*` prefix.
|
||||
- Six counters and one histogram covering parse counts, parse
|
||||
failures, attachment outcomes, attachment extraction duration, hook
|
||||
invocations, DKIM/SPF/DMARC outcomes, spam classification counts,
|
||||
and thread grouping counts.
|
||||
- Strictly typed `EmailProcessingInstruments` interface — counter
|
||||
misuses become typecheck errors at the call site.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { getEmailProcessingInstruments } from "@suaveplan/email-processing/otel";
|
||||
|
||||
const m = getEmailProcessingInstruments();
|
||||
m.parseCounter.add(1);
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
getEmailProcessingInstruments,
|
||||
resetEmailProcessingInstruments,
|
||||
} from "@suaveplan/email-processing/otel";
|
||||
|
||||
// In a test setup file:
|
||||
resetEmailProcessingInstruments();
|
||||
const meter = getEmailProcessingInstruments();
|
||||
const start = performance.now();
|
||||
await processAttachments(parsed.attachments);
|
||||
meter.attachmentExtractionDurationMs.record(performance.now() - start, {
|
||||
"attachment.count": parsed.attachments.length,
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `EmailProcessingInstruments` — readonly interface listing every
|
||||
counter and histogram, plus the underlying `meter`.
|
||||
- `getEmailProcessingInstruments(): EmailProcessingInstruments` — returns
|
||||
the cached instrument set, building it on first call.
|
||||
- `resetEmailProcessingInstruments(): void` — clears the cache so the
|
||||
next `getEmailProcessingInstruments()` rebuilds against whatever
|
||||
`MeterProvider` is currently registered via `metrics.setGlobal*`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The instruments are constructed against `metrics.getMeter(...)`, which
|
||||
defers to whichever `MeterProvider` was last installed via
|
||||
`metrics.setGlobalMeterProvider`. Tests install an
|
||||
`InMemoryMetricExporter`-backed provider in `tests/setup/polyfills.ts`
|
||||
before any source module loads; production code installs a real
|
||||
exporter via the application bootstrap. The histogram unit is `"ms"`
|
||||
and intentionally simple: the package emits durations in milliseconds
|
||||
from `performance.now()` deltas, so dashboards can read directly without
|
||||
unit conversion.
|
||||
@@ -0,0 +1,91 @@
|
||||
# thread-grouper
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Purpose
|
||||
|
||||
Groups a flat list of emails into conversation threads using
|
||||
In-Reply-To, References, and subject-line heuristics. Each thread is
|
||||
returned with its full chronological message list plus a parent /
|
||||
child tree rooted at the earliest message, so consumers can render
|
||||
either a flat timeline or a nested conversation view.
|
||||
|
||||
## Features
|
||||
|
||||
- Parent resolution prefers `In-Reply-To` and falls back to the last
|
||||
`References` entry.
|
||||
- Subject-line normalisation strips `Re:`, `Fwd:`, `Aw:`, `Sv:`, `Vs:`
|
||||
prefixes (with optional `[N]` ticket-tracker suffixes).
|
||||
- A reply with a significantly different subject starts a new thread
|
||||
even when it references the original — heuristic catches
|
||||
"hijacked" reply chains.
|
||||
- Stable chronological ordering via `Date.parse` on the email `date`
|
||||
field (epoch-zero when missing, matching upstream behaviour).
|
||||
- Records `email_processing.thread.grouped_total` with the produced
|
||||
thread count per call.
|
||||
|
||||
## Usage: Basic
|
||||
|
||||
```ts
|
||||
import { groupThreads } from "@suaveplan/email-processing/threading";
|
||||
|
||||
const threads = groupThreads(
|
||||
messages.map((m) => ({
|
||||
messageId: m.messageId ?? "",
|
||||
inReplyTo: m.inReplyTo,
|
||||
references: m.references,
|
||||
subject: m.subject,
|
||||
date: m.date,
|
||||
}))
|
||||
);
|
||||
for (const t of threads) {
|
||||
console.log(t.threadId, t.subject, t.messageCount);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced
|
||||
|
||||
```ts
|
||||
import {
|
||||
groupThreads,
|
||||
normalizeSubject,
|
||||
isSubjectChange,
|
||||
parseReferences,
|
||||
} from "@suaveplan/email-processing/threading";
|
||||
|
||||
const threads = groupThreads(messages);
|
||||
|
||||
function flatten(node) {
|
||||
return [node.message, ...node.children.flatMap(flatten)];
|
||||
}
|
||||
|
||||
const summary = threads.map((t) => ({
|
||||
threadId: t.threadId,
|
||||
subject: normalizeSubject(t.subject ?? ""),
|
||||
flat: flatten(t.root).map((m) => m.messageId),
|
||||
refs: parseReferences(t.messages.at(-1)?.references),
|
||||
}));
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `ThreadableEmail` — `{ messageId, inReplyTo?, references?, subject?,
|
||||
date?, from? }`.
|
||||
- `ThreadNode` — `{ message, children }`.
|
||||
- `EmailThread` — `{ threadId, subject?, root, messageCount, messages }`.
|
||||
- `normalizeSubject(subject)`.
|
||||
- `parseReferences(references?)` — `<id>` extraction.
|
||||
- `isSubjectChange(original, reply)` — heuristic.
|
||||
- `groupThreads(emails)` — returns chronological `EmailThread[]`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`buildTree` falls back to a `"messageId: ''"` placeholder root only
|
||||
when the input array is unexpectedly empty — `groupThreads` itself
|
||||
short-circuits empty input, so the fallback exists purely for type
|
||||
safety. The grouper's chronological pass uses the most-recent
|
||||
reference (last entry in `References`) when `In-Reply-To` is absent,
|
||||
matching most MUA conventions. Subject-change detection is
|
||||
case-insensitive and ignores whitespace differences.
|
||||
Reference in New Issue
Block a user