comparison

(no submodules)

Purpose

ProfileComparator diffs two sets of profiling data — either raw ProfileEntry[] arrays or pre-aggregated ProfileReport objects — and classifies each per-label delta as a regression, an improvement, or unchanged. The classification threshold and the wall-clock used for the report's generatedAt field are both injectable, so test runs are deterministic and CI gates can configure their own sensitivity.

Features

compareReports(baseline, current) operates at category granularity (uses report.byCategory means).
compareEntries(baseline, current) operates at label granularity (computes per-label averages internally).
regressionThreshold controls how many percent of change is required to classify a delta as a regression or improvement. Default 5.
Sorts: regressions descending by percentChange, improvements ascending.
Zero-baseline edge case handled symmetrically — a label that appears only in current is classified by treating baseline as 0 and reporting 100% change.

Usage: Basic

import { ProfileComparator, ProfileReporter } from "@suaveplan/profiler";

const reporter = new ProfileReporter();
const baseline = reporter.generateReport(oldEntries);
const current = reporter.generateReport(newEntries);
const diff = new ProfileComparator().compareReports(baseline, current);
console.log("regressions:", diff.regressions.length);

Usage: Advanced

import { ProfileComparator } from "@suaveplan/profiler";

const comparator = new ProfileComparator({
    regressionThreshold: 2, // tighter than default 5%
    clock: () => Date.parse("2025-01-01T00:00:00Z"),
});

const diff = comparator.compareEntries(baseline.getEntries(), pr.getEntries());
if (diff.regressions.length > 0) {
    process.exitCode = 1;
    console.error("perf regression gate triggered:", diff.regressions);
}

API Reference

class ProfileComparator — new ProfileComparator(options?: ProfileComparatorOptions).
interface ProfileComparatorOptions { regressionThreshold?: number; clock?: Clock }.
Methods: compareReports(baseline, current), compareEntries(baseline, current).
Errors: ProfilerError with PROFILER.INVALID_OPTIONS on negative threshold.

Implementation Notes

compareEntries ignores entries with duration === undefined so partially-completed profiles can be compared safely. Per-label averages are computed from raw durations rather than from any pre-aggregated stats so the same comparator can mix labels with different occurrence counts in baseline vs current. The unchanged bucket exists explicitly so consumers wanting a full per-label audit can read it without re-grouping. The generatedAt timestamp is produced from new Date(clock()).toISOString() so the default and the injected clock paths produce identically-formatted strings.

2.9 KiB Raw Blame History