suaveplan/openspec/archive/2026-04-24-enterprise-ab-testing/proposal.md

# OpenSpec Change: enterprise-ab-testing

- **Change ID**: enterprise-ab-testing
- **Package**: `@suaveplan/ab-testing`
- **Category**: universal/workflow
- **Target path**: `universal/workflow/ab-testing`
- **Status**: PROPOSED
- **Date**: 2026-04-24
- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/ab-testing.md`

---

## Summary

Deliver `@suaveplan/ab-testing` on the genesis branch from scratch at enterprise quality. The package provides an A/B testing engine with deterministic assignment, audience targeting, experiment lifecycle management, metric tracking, and results analysis. Genesis adds structured errors via `@suaveplan/error`, Zod schemas for all experiment types, OTel instrumentation on assignment and conversion events, injectable `Clock` and `Rng` for deterministic bucketing, integration with `@suaveplan/feature-flags` as an optional provider, event-bus emission for assignment/conversion events, and property-based tests for bucketing distribution invariants.

---

## Current-state audit (genesis)

The package does not exist on genesis yet — this is a fresh build.

---

## Available workspace deps on genesis

| Package | Role |
|---|---|
| `@suaveplan/error` (`workspace:^`) | Base class for `AbTestingError`; stable error codes |
| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `Rng`, `EventBusLike` |
| `@suaveplan/events` (`workspace:^`) | Event bus for assignment and conversion events |
| `@suaveplan/logger` (`workspace:^`) | Structured logging with OTel trace correlation |
| `@suaveplan/feature-flags` (`workspace:^`) | Optional feature-flag provider for experiment gating |
| `zod` (`catalog:forms`) | Zod schemas for all public types |
| `@opentelemetry/api` (`catalog:observability`) | Spans + counters on assignment/conversion |
| `tslib` (`catalog:build`) | Runtime helpers |

Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).

---

## Public API surface

### Core

```ts
function createExperimentManager(options?: ExperimentManagerOptions): ExperimentManager

interface ExperimentManager {
  defineExperiment(experiment: ExperimentDefinition): void
  getExperiment(id: string): ExperimentDefinition | null
  listExperiments(): ReadonlyArray<ExperimentDefinition>
  assign(experimentId: string, userId: string, attributes?: UserAttributes): Assignment
  track(experimentId: string, userId: string, metric: MetricEvent): void
  getResults(experimentId: string): ExperimentResults
  use(provider: ExperimentProvider): this
}
```

### Experiment definition

```ts
interface ExperimentDefinition {
  id: string
  name: string
  status: 'draft' | 'running' | 'paused' | 'concluded'
  variants: ReadonlyArray<Variant>
  targeting?: TargetingRule[]
  startDate?: string
  endDate?: string
  sampleRate?: number  // 0–1, default 1.0
  primaryMetric?: string
  secondaryMetrics?: string[]
}

interface Variant {
  id: string
  name: string
  weight: number  // relative weight, will be normalised
  isControl?: boolean
}

interface TargetingRule {
  attribute: string
  operator: 'eq' | 'neq' | 'in' | 'notIn' | 'gt' | 'lt' | 'gte' | 'lte'
  value: unknown
}
```

### Assignment

```ts
interface Assignment {
  readonly experimentId: string
  readonly userId: string
  readonly variantId: string
  readonly isControl: boolean
  readonly timestamp: number
  readonly inExperiment: boolean  // false if excluded by targeting or sample rate
}
```

### Metrics

```ts
interface MetricEvent {
  name: string
  value?: number
  timestamp?: number
  properties?: Record<string, unknown>
}

interface ExperimentResults {
  readonly experimentId: string
  readonly variants: ReadonlyArray<VariantResults>
  readonly totalAssignments: number
  readonly totalConversions: number
}

interface VariantResults {
  readonly variantId: string
  readonly assignments: number
  readonly conversions: number
  readonly conversionRate: number
  readonly metrics: ReadonlyArray<{ name: string; count: number; sum: number; avg: number }>
}
```

### Provider interface

```ts
interface ExperimentProvider {
  readonly name: string
  getAssignment?(experimentId: string, userId: string): string | null
  trackAssignment?(assignment: Assignment): void
  trackConversion?(experimentId: string, userId: string, metric: MetricEvent): void
}

type UserAttributes = Record<string, string | number | boolean>

interface ExperimentManagerOptions {
  logger?: LoggerLike
  clock?: Clock
  rng?: Rng
  eventBus?: EventBusLike
  hashSeed?: number
}
```

### Bucketing utilities

```ts
function bucketUser(userId: string, experimentId: string, seed?: number): number  // 0–1
function normaliseWeights(variants: Variant[]): Array<Variant & { normalizedWeight: number }>
function selectVariant(variants: Variant[], bucket: number): Variant
```

### Error codes

```ts
type AbTestingErrorCode =
  | 'AB_EXPERIMENT_NOT_FOUND'
  | 'AB_EXPERIMENT_INACTIVE'
  | 'AB_INVALID_DEFINITION'
  | 'AB_VARIANT_NOT_FOUND'
  | 'AB_METRIC_INVALID'

function registerAbTestingErrorCodes(): void
```

---

## Genesis additions

1. **Structured `AbTestingError`** extending `SuaveplanError` with stable `AB.*` codes
2. **Zod schemas** for `ExperimentDefinition`, `Variant`, `TargetingRule`, `MetricEvent`
3. **Deterministic FNV-1a bucketing** — injectable `Rng` for testing; FNV-1a hash by default for consistency with `@suaveplan/feature-flags`
4. **Targeting rule evaluation** — 8 operators, evaluated before assignment
5. **Provider plugin interface** — `use()` for external integrations
6. **`AbortSignal`** available on async provider calls
7. **OTel instrumentation** — span `ab.assign`, counter `ab.assignments_total`, counter `ab.conversions_total`, attributes include `experiment_id`, `variant_id`
8. **Property-based tests** — bucketing distribution, targeting rule evaluation, weight normalisation

---

## File layout

```text
universal/workflow/ab-testing/
  package.json
  tsconfig.json
  vitest.config.ts
  src/
    index.ts
    error-codes.ts
    constants/
      index.ts
    types/
      common.ts
      common.md
    core/
      experiment-manager/
        experiment-manager.ts
        experiment-manager.md
        experiment-manager.types.ts
        experiment-manager.test.ts
        index.ts
    bucketing/
      bucketing.ts
      bucketing.md
      bucketing.test.ts
      index.ts
    targeting/
      targeting.ts
      targeting.md
      targeting.test.ts
      index.ts
    results/
      results.ts
      results.md
      results.test.ts
      index.ts
    providers/
      provider.ts
      provider.md
      provider.test.ts
      index.ts
    __benchmarks__/
      ab-testing.bench.ts
  tests/
    integration.test.ts
    property.test.ts
```

---

## Impact

**Downstream packages**: Any application layer that needs experiment assignment. No existing genesis package currently depends on it.

**Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/logger`, `@suaveplan/feature-flags` must be published.