first commit
This commit is contained in:
121
docs/communication/emails/email-processing/README.md
Normal file
121
docs/communication/emails/email-processing/README.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# @suaveplan/email-processing - Universal MIME parsing and email processing: postal-mime + libmime wrappers, attachment processing with SHA-256 hashing + MIME-type allowlists, Content-ID resolution, structured data extraction (iCalendar, delivery tracking, receipts), DKIM/SPF/DMARC verification, spam scoring with pluggable rule sets, conversation threading, hook pipeline + OTel instrumentation
|
||||
|
||||
Universal MIME parsing and email processing for the Suaveplan
|
||||
. Wraps `postal-mime` and `libmime` with structured errors,
|
||||
OpenTelemetry instrumentation, a hook pipeline for transforming
|
||||
parsed messages, full attachment processing (SHA-256 hashing, MIME
|
||||
allowlists, CID resolution), structured-data extraction (iCalendar,
|
||||
delivery tracking, receipts), DKIM/SPF/DMARC verification, spam
|
||||
scoring, and conversation threading.
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
|
||||
- [attachment-processor](./src/attachment-processor/attachment-processor.md) — Normalises and validates `EmailAttachment` payloads emerging from `parseEmail`. Each attachment is decoded to bytes, va…
|
||||
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
|
||||
- [calendar-parser](./src/calendar-parser/calendar-parser.md) — Parses iCalendar (RFC 5545) / ICS payloads embedded in emails — these arrive either as `text/calendar` MIME parts or as…
|
||||
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
|
||||
- [cid-resolver](./src/cid-resolver/cid-resolver.md) — Resolves Content-ID references inside HTML email bodies. Email clients that send inline images embed them as MIME parts…
|
||||
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
|
||||
- [data-extractor](./src/data-extractor/data-extractor.md) — Extracts structured business signals — calendar events, parcel tracking numbers, and receipt / order data — from the fr…
|
||||
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
|
||||
- [dkim-canonicalize](./src/dkim-canonicalize/dkim-canonicalize.md) — Implements RFC 6376 canonicalisation primitives — the byte-exact text normalisation steps DKIM requires before hashing…
|
||||
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
|
||||
- [dkim-verifier](./src/dkim-verifier/dkim-verifier.md) — Implements DKIM (RFC 6376) signature verification — parses a `DKIM-Signature` header, looks up the signer's public key…
|
||||
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
|
||||
- [dmarc-checker](./src/dmarc-checker/dmarc-checker.md) — Implements DMARC (RFC 7489) policy evaluation on top of an upstream SPF result and DKIM verification. Looks up the `_dm…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
|
||||
- [error-codes](./src/error-codes/error-codes.md) — Defines the stable error-code registry and typed error class for the `@suaveplan/email-processing` package. Every failu…
|
||||
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
|
||||
- [hooks](./src/hooks/hooks.md) — Implements the package's `email:processing` hook pipeline: every parsed email passes through a sequence of handlers tha…
|
||||
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
|
||||
- [mime-utils](./src/mime-utils/mime-utils.md) — Wraps the `libmime` library with a strict, typed boundary so the rest of the package can call RFC 2047 / 2231 / 5322 en…
|
||||
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
|
||||
- [parser](./src/parser/parser.md) — Wraps `postal-mime` with a strict input/option boundary, OTel instrumentation, structured error codes, and the `@suavep…
|
||||
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
|
||||
- [spam-scorer](./src/spam-scorer/spam-scorer.md) — Heuristic spam scorer with a pluggable rule set. The default rules cover the most common spam signals — missing headers…
|
||||
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
|
||||
- [spf-checker](./src/spf-checker/spf-checker.md) — Implements the SPF (RFC 7208) evaluation algorithm: parses the record, walks each mechanism in order, follows `include:…
|
||||
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
|
||||
- [telemetry](./src/telemetry/telemetry.md) — Provides the shared OpenTelemetry meter and instrument set used by every public entry point of `@suaveplan/email-proces…
|
||||
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
|
||||
- [thread-grouper](./src/thread-grouper/thread-grouper.md) — Groups a flat list of emails into conversation threads using In-Reply-To, References, and subject-line heuristics. Each…
|
||||
<!-- end:auto-docs -->
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
attachment-processor/ # SHA-256 hash + size + MIME allowlist validation
|
||||
calendar-parser/ # iCalendar VEVENT extraction
|
||||
cid-resolver/ # Content-ID resolution to data URIs / URL maps
|
||||
data-extractor/ # tracking numbers, receipts, calendar events
|
||||
dkim-canonicalize/ # RFC 6376 relaxed + simple canonicalisation
|
||||
dkim-verifier/ # Tag parsing, body-hash, RSASSA-PKCS1-v1_5 verify
|
||||
dmarc-checker/ # Policy + alignment over SPF + DKIM
|
||||
error-codes/ # EMAIL_PROCESSING_CODES + EmailProcessingError
|
||||
hooks/ # email:processing pipeline + singleton bus
|
||||
mime-utils/ # libmime wrappers (encode/decode words, folding)
|
||||
parser/ # parseEmail / parseEmailAddress / decodeEmailWords
|
||||
spam-scorer/ # 12 default rules + custom-rule merge
|
||||
spf-checker/ # RFC 7208 evaluator with include/redirect limits
|
||||
telemetry/ # Shared OTel meter + instruments
|
||||
thread-grouper/ # In-Reply-To + References + subject threading
|
||||
types/ # Shared MIME / address / header type aliases
|
||||
index.ts # Package barrel
|
||||
```
|
||||
|
||||
## Quality
|
||||
|
||||
- 100% statement / branch / function / line coverage (Istanbul).
|
||||
- Real `postal-mime` parses against embedded RFC 822 fixtures — no
|
||||
mocks (rule 2).
|
||||
- Test runner imports go through `@suaveplan/testing/runner` only
|
||||
(rule 11).
|
||||
- Zod-validated option boundaries (`parseEmail`, `processAttachment`).
|
||||
- Two error codes:
|
||||
`INVALID_INPUT`, `PARSE_FAILURE`, `INVALID_OPTIONS`,
|
||||
`ATTACHMENT_TOO_LARGE`, `ATTACHMENT_TYPE_REJECTED`, `HASH_FAILURE`,
|
||||
`DKIM_VERIFY_FAILURE`, `HOOK_FAILURE` — every code is thrown by at
|
||||
least one production path under `src/`.
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import {
|
||||
parseEmail,
|
||||
processAttachments,
|
||||
onEmailProcessing,
|
||||
} from "@suaveplan/email-processing";
|
||||
|
||||
onEmailProcessing((data) => ({
|
||||
...data,
|
||||
parsed: {
|
||||
...data.parsed,
|
||||
html: data.parsed.html?.replace(/<script[\s\S]*?<\/script>/gi, ""),
|
||||
},
|
||||
}));
|
||||
|
||||
const parsed = await parseEmail(rawMimeBuffer);
|
||||
const processed = await processAttachments(parsed.attachments, {
|
||||
maxSize: 10 * 1024 * 1024,
|
||||
allowedMimeTypes: ["image/*", "application/pdf"],
|
||||
});
|
||||
```
|
||||
|
||||
## Subpath exports
|
||||
|
||||
`./parser`, `./mime-utils`, `./hooks`, `./attachments`, `./cid-resolver`,
|
||||
`./extraction`, `./calendar`, `./dkim`, `./dmarc`, `./spf`, `./spam`,
|
||||
`./threading`, `./errors`, `./otel`.
|
||||
|
||||
## Provenance
|
||||
|
||||
Migrated from `packages/universal/email-processing` on the
|
||||
`/mnt/s/web/suaveplan` main branch per the enterprise plan at
|
||||
`/mnt/s/web/suaveplan/enterprise/email-processing.md`. Hook
|
||||
infrastructure was rewritten locally because the foundation
|
||||
`@suaveplan/events` library does not yet ship a shared
|
||||
`Hooks<T>` substrate.
|
||||
|
||||
Reference in New Issue
Block a user