Initial Commit

packages/devtools/tooling/profiler/src/comparison/comparison.md

@@ -0,0 +1,57 @@
+# comparison
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+
+## Purpose
+
+`ProfileComparator` diffs two sets of profiling data — either raw `ProfileEntry[]` arrays or pre-aggregated `ProfileReport` objects — and classifies each per-label delta as a regression, an improvement, or unchanged. The classification threshold and the wall-clock used for the report's `generatedAt` field are both injectable, so test runs are deterministic and CI gates can configure their own sensitivity.
+
+## Features
+
+- `compareReports(baseline, current)` operates at category granularity (uses `report.byCategory` means).
+- `compareEntries(baseline, current)` operates at label granularity (computes per-label averages internally).
+- `regressionThreshold` controls how many percent of change is required to classify a delta as a regression or improvement. Default 5.
+- Sorts: regressions descending by `percentChange`, improvements ascending.
+- Zero-baseline edge case handled symmetrically — a label that appears only in `current` is classified by treating baseline as 0 and reporting 100% change.
+
+## Usage: Basic
+
+```ts
+import { ProfileComparator, ProfileReporter } from "@suaveplan/profiler";
+
+const reporter = new ProfileReporter();
+const baseline = reporter.generateReport(oldEntries);
+const current = reporter.generateReport(newEntries);
+const diff = new ProfileComparator().compareReports(baseline, current);
+console.log("regressions:", diff.regressions.length);
+```
+
+## Usage: Advanced
+
+```ts
+import { ProfileComparator } from "@suaveplan/profiler";
+
+const comparator = new ProfileComparator({
+    regressionThreshold: 2, // tighter than default 5%
+    clock: () => Date.parse("2025-01-01T00:00:00Z"),
+});
+
+const diff = comparator.compareEntries(baseline.getEntries(), pr.getEntries());
+if (diff.regressions.length > 0) {
+    process.exitCode = 1;
+    console.error("perf regression gate triggered:", diff.regressions);
+}
+```
+
+## API Reference
+
+- `class ProfileComparator` — `new ProfileComparator(options?: ProfileComparatorOptions)`.
+- `interface ProfileComparatorOptions { regressionThreshold?: number; clock?: Clock }`.
+- Methods: `compareReports(baseline, current)`, `compareEntries(baseline, current)`.
+- Errors: `ProfilerError` with `PROFILER.INVALID_OPTIONS` on negative threshold.
+
+## Implementation Notes
+
+`compareEntries` ignores entries with `duration === undefined` so partially-completed profiles can be compared safely. Per-label averages are computed from raw durations rather than from any pre-aggregated stats so the same comparator can mix labels with different occurrence counts in baseline vs current. The `unchanged` bucket exists explicitly so consumers wanting a full per-label audit can read it without re-grouping. The `generatedAt` timestamp is produced from `new Date(clock()).toISOString()` so the default and the injected clock paths produce identically-formatted strings.