# OpenSpec Change: enterprise-ab-testing - **Change ID**: enterprise-ab-testing - **Package**: `@suaveplan/ab-testing` - **Category**: universal/workflow - **Target path**: `universal/workflow/ab-testing` - **Status**: PROPOSED - **Date**: 2026-04-24 - **Reference spec**: `/mnt/s/web/suaveplan/enterprise/ab-testing.md` --- ## Summary Deliver `@suaveplan/ab-testing` on the genesis branch from scratch at enterprise quality. The package provides an A/B testing engine with deterministic assignment, audience targeting, experiment lifecycle management, metric tracking, and results analysis. Genesis adds structured errors via `@suaveplan/error`, Zod schemas for all experiment types, OTel instrumentation on assignment and conversion events, injectable `Clock` and `Rng` for deterministic bucketing, integration with `@suaveplan/feature-flags` as an optional provider, event-bus emission for assignment/conversion events, and property-based tests for bucketing distribution invariants. --- ## Current-state audit (genesis) The package does not exist on genesis yet — this is a fresh build. --- ## Available workspace deps on genesis | Package | Role | |---|---| | `@suaveplan/error` (`workspace:^`) | Base class for `AbTestingError`; stable error codes | | `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `Rng`, `EventBusLike` | | `@suaveplan/events` (`workspace:^`) | Event bus for assignment and conversion events | | `@suaveplan/logger` (`workspace:^`) | Structured logging with OTel trace correlation | | `@suaveplan/feature-flags` (`workspace:^`) | Optional feature-flag provider for experiment gating | | `zod` (`catalog:forms`) | Zod schemas for all public types | | `@opentelemetry/api` (`catalog:observability`) | Spans + counters on assignment/conversion | | `tslib` (`catalog:build`) | Runtime helpers | Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`). --- ## Public API surface ### Core ```ts function createExperimentManager(options?: ExperimentManagerOptions): ExperimentManager interface ExperimentManager { defineExperiment(experiment: ExperimentDefinition): void getExperiment(id: string): ExperimentDefinition | null listExperiments(): ReadonlyArray assign(experimentId: string, userId: string, attributes?: UserAttributes): Assignment track(experimentId: string, userId: string, metric: MetricEvent): void getResults(experimentId: string): ExperimentResults use(provider: ExperimentProvider): this } ``` ### Experiment definition ```ts interface ExperimentDefinition { id: string name: string status: 'draft' | 'running' | 'paused' | 'concluded' variants: ReadonlyArray targeting?: TargetingRule[] startDate?: string endDate?: string sampleRate?: number // 0–1, default 1.0 primaryMetric?: string secondaryMetrics?: string[] } interface Variant { id: string name: string weight: number // relative weight, will be normalised isControl?: boolean } interface TargetingRule { attribute: string operator: 'eq' | 'neq' | 'in' | 'notIn' | 'gt' | 'lt' | 'gte' | 'lte' value: unknown } ``` ### Assignment ```ts interface Assignment { readonly experimentId: string readonly userId: string readonly variantId: string readonly isControl: boolean readonly timestamp: number readonly inExperiment: boolean // false if excluded by targeting or sample rate } ``` ### Metrics ```ts interface MetricEvent { name: string value?: number timestamp?: number properties?: Record } interface ExperimentResults { readonly experimentId: string readonly variants: ReadonlyArray readonly totalAssignments: number readonly totalConversions: number } interface VariantResults { readonly variantId: string readonly assignments: number readonly conversions: number readonly conversionRate: number readonly metrics: ReadonlyArray<{ name: string; count: number; sum: number; avg: number }> } ``` ### Provider interface ```ts interface ExperimentProvider { readonly name: string getAssignment?(experimentId: string, userId: string): string | null trackAssignment?(assignment: Assignment): void trackConversion?(experimentId: string, userId: string, metric: MetricEvent): void } type UserAttributes = Record interface ExperimentManagerOptions { logger?: LoggerLike clock?: Clock rng?: Rng eventBus?: EventBusLike hashSeed?: number } ``` ### Bucketing utilities ```ts function bucketUser(userId: string, experimentId: string, seed?: number): number // 0–1 function normaliseWeights(variants: Variant[]): Array function selectVariant(variants: Variant[], bucket: number): Variant ``` ### Error codes ```ts type AbTestingErrorCode = | 'AB_EXPERIMENT_NOT_FOUND' | 'AB_EXPERIMENT_INACTIVE' | 'AB_INVALID_DEFINITION' | 'AB_VARIANT_NOT_FOUND' | 'AB_METRIC_INVALID' function registerAbTestingErrorCodes(): void ``` --- ## Genesis additions 1. **Structured `AbTestingError`** extending `SuaveplanError` with stable `AB.*` codes 2. **Zod schemas** for `ExperimentDefinition`, `Variant`, `TargetingRule`, `MetricEvent` 3. **Deterministic FNV-1a bucketing** — injectable `Rng` for testing; FNV-1a hash by default for consistency with `@suaveplan/feature-flags` 4. **Targeting rule evaluation** — 8 operators, evaluated before assignment 5. **Provider plugin interface** — `use()` for external integrations 6. **`AbortSignal`** available on async provider calls 7. **OTel instrumentation** — span `ab.assign`, counter `ab.assignments_total`, counter `ab.conversions_total`, attributes include `experiment_id`, `variant_id` 8. **Property-based tests** — bucketing distribution, targeting rule evaluation, weight normalisation --- ## File layout ```text universal/workflow/ab-testing/ package.json tsconfig.json vitest.config.ts src/ index.ts error-codes.ts constants/ index.ts types/ common.ts common.md core/ experiment-manager/ experiment-manager.ts experiment-manager.md experiment-manager.types.ts experiment-manager.test.ts index.ts bucketing/ bucketing.ts bucketing.md bucketing.test.ts index.ts targeting/ targeting.ts targeting.md targeting.test.ts index.ts results/ results.ts results.md results.test.ts index.ts providers/ provider.ts provider.md provider.test.ts index.ts __benchmarks__/ ab-testing.bench.ts tests/ integration.test.ts property.test.ts ``` --- ## Impact **Downstream packages**: Any application layer that needs experiment assignment. No existing genesis package currently depends on it. **Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/logger`, `@suaveplan/feature-flags` must be published.