Initial Commit

packages/testkit/testing/ab-testing/src/results/results.md

@@ -0,0 +1,118 @@
+# results.ts — Experiment Results Computation
+
+<!-- begin:auto-docs -->
+_(no submodules)_
+<!-- end:auto-docs -->
+## Purpose
+
+Provides the `computeResults` function, which aggregates raw assignment and
+conversion data into structured per-variant statistics. This module is a pure
+function with no side effects, making it straightforward to test exhaustively
+and to call from any context — the manager uses it on demand, but it can also
+be called directly from analytics pipelines or reporting jobs.
+
+The function produces assignment counts, conversion counts, conversion rates,
+and per-metric aggregates (count, sum, average) for every variant that received
+at least one in-experiment assignment.
+
+## Features
+
+- Groups assignments by `variantId`, excluding out-of-experiment users
+- Counts unique conversions per user (one conversion per user, regardless of
+  how many metric events they fire)
+- Aggregates named metric events into `count`, `sum`, and `avg` per metric
+  per variant
+- Computes `conversionRate = conversions / assignments` (0 when no assignments)
+- Returns `totalAssignments` and `totalConversions` rolled up across all
+  variants for dashboard summary views
+- Fully deterministic — given the same inputs, produces byte-identical output
+
+## Usage: Basic Example
+
+```ts
+import { computeResults } from '@suaveplan/ab-testing/results';
+import type { Assignment, MetricEvent } from '@suaveplan/ab-testing';
+
+const assignments: Assignment[] = [
+  { experimentId: 'exp-1', userId: 'u1', variantId: 'ctrl', isControl: true,  timestamp: 1, inExperiment: true },
+  { experimentId: 'exp-1', userId: 'u2', variantId: 'treat', isControl: false, timestamp: 2, inExperiment: true },
+];
+const conversions = new Map<string, MetricEvent[]>([
+  ['u1', [{ name: 'checkout_started', value: 1 }]],
+]);
+
+const results = computeResults('exp-1', assignments, conversions);
+console.log(results.totalAssignments); // 2
+console.log(results.totalConversions); // 1
+console.log(results.variants[0]?.conversionRate); // 1.0 (ctrl: 1/1)
+console.log(results.variants[1]?.conversionRate); // 0.0 (treat: 0/1)
+```
+
+## Usage: Advanced Example
+
+```ts
+import { computeResults } from '@suaveplan/ab-testing/results';
+import type { Assignment, MetricEvent } from '@suaveplan/ab-testing';
+
+// Multiple metric events per user (only first triggers conversion flag)
+const assignments: Assignment[] = [
+  { experimentId: 'exp-2', userId: 'u1', variantId: 'v1', isControl: true,  timestamp: 1, inExperiment: true },
+  { experimentId: 'exp-2', userId: 'u2', variantId: 'v1', isControl: true,  timestamp: 2, inExperiment: true },
+  { experimentId: 'exp-2', userId: 'u3', variantId: 'v2', isControl: false, timestamp: 3, inExperiment: true },
+  // excluded user — not counted
+  { experimentId: 'exp-2', userId: 'u4', variantId: '',   isControl: false, timestamp: 4, inExperiment: false },
+];
+const conversions = new Map<string, MetricEvent[]>([
+  ['u1', [
+    { name: 'revenue', value: 49.99 },
+    { name: 'revenue', value: 19.99 },
+  ]],
+  ['u3', [{ name: 'revenue', value: 99.00 }]],
+]);
+
+const results = computeResults('exp-2', assignments, conversions);
+// v1: 2 assignments, 1 conversion (u1), revenue avg = (49.99 + 19.99) / 2
+// v2: 1 assignment,  1 conversion (u3), revenue avg = 99.00
+```
+
+## API Reference
+
+### `computeResults(experimentId, assignments, conversions): ExperimentResults`
+
+- **`experimentId`** — `string` — stable experiment identifier copied to result
+- **`assignments`** — `Assignment[]` — all assignments for the experiment
+- **`conversions`** — `Map<string, MetricEvent[]>` — keyed by `userId`
+- **Returns** — `ExperimentResults` with `variants`, `totalAssignments`,
+  `totalConversions`
+
+#### `ExperimentResults`
+
+| Field | Type | Description |
+|---|---|---|
+| `experimentId` | `string` | Echoed from input |
+| `variants` | `VariantResults[]` | Per-variant aggregates |
+| `totalAssignments` | `number` | Sum across all variants |
+| `totalConversions` | `number` | Sum across all variants |
+
+#### `VariantResults`
+
+| Field | Type | Description |
+|---|---|---|
+| `variantId` | `string` | Variant identifier |
+| `assignments` | `number` | In-experiment assignment count |
+| `conversions` | `number` | Unique-user conversion count |
+| `conversionRate` | `number` | `conversions / assignments` |
+| `metrics` | `Array<{name, count, sum, avg}>` | Per-metric aggregates |
+
+## Implementation Notes
+
+Conversion counting follows the convention that a user can convert at most
+once per experiment — the first metric event for each user triggers the
+conversion flag, but all subsequent events for that user are still accumulated
+into the metric aggregates. This avoids double-counting in funnel analyses
+while preserving the full metric history for revenue and value calculations.
+
+Out-of-experiment assignments (`inExperiment: false`) are excluded entirely
+from all counts. They are recorded so that the manager can return a stable
+assignment for a user across multiple calls, but they should not pollute the
+statistical results.