Initial Commit

openspec/archive/2026-04-24-enterprise-ab-testing/proposal.md

@@ -0,0 +1,241 @@
+# OpenSpec Change: enterprise-ab-testing
+
+- **Change ID**: enterprise-ab-testing
+- **Package**: `@suaveplan/ab-testing`
+- **Category**: universal/workflow
+- **Target path**: `universal/workflow/ab-testing`
+- **Status**: PROPOSED
+- **Date**: 2026-04-24
+- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/ab-testing.md`
+
+---
+
+## Summary
+
+Deliver `@suaveplan/ab-testing` on the genesis branch from scratch at enterprise quality. The package provides an A/B testing engine with deterministic assignment, audience targeting, experiment lifecycle management, metric tracking, and results analysis. Genesis adds structured errors via `@suaveplan/error`, Zod schemas for all experiment types, OTel instrumentation on assignment and conversion events, injectable `Clock` and `Rng` for deterministic bucketing, integration with `@suaveplan/feature-flags` as an optional provider, event-bus emission for assignment/conversion events, and property-based tests for bucketing distribution invariants.
+
+---
+
+## Current-state audit (genesis)
+
+The package does not exist on genesis yet — this is a fresh build.
+
+---
+
+## Available workspace deps on genesis
+
+| Package | Role |
+|---|---|
+| `@suaveplan/error` (`workspace:^`) | Base class for `AbTestingError`; stable error codes |
+| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `Rng`, `EventBusLike` |
+| `@suaveplan/events` (`workspace:^`) | Event bus for assignment and conversion events |
+| `@suaveplan/logger` (`workspace:^`) | Structured logging with OTel trace correlation |
+| `@suaveplan/feature-flags` (`workspace:^`) | Optional feature-flag provider for experiment gating |
+| `zod` (`catalog:forms`) | Zod schemas for all public types |
+| `@opentelemetry/api` (`catalog:observability`) | Spans + counters on assignment/conversion |
+| `tslib` (`catalog:build`) | Runtime helpers |
+
+Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
+
+---
+
+## Public API surface
+
+### Core
+
+```ts
+function createExperimentManager(options?: ExperimentManagerOptions): ExperimentManager
+
+interface ExperimentManager {
+  defineExperiment(experiment: ExperimentDefinition): void
+  getExperiment(id: string): ExperimentDefinition | null
+  listExperiments(): ReadonlyArray<ExperimentDefinition>
+  assign(experimentId: string, userId: string, attributes?: UserAttributes): Assignment
+  track(experimentId: string, userId: string, metric: MetricEvent): void
+  getResults(experimentId: string): ExperimentResults
+  use(provider: ExperimentProvider): this
+}
+```
+
+### Experiment definition
+
+```ts
+interface ExperimentDefinition {
+  id: string
+  name: string
+  status: 'draft' | 'running' | 'paused' | 'concluded'
+  variants: ReadonlyArray<Variant>
+  targeting?: TargetingRule[]
+  startDate?: string
+  endDate?: string
+  sampleRate?: number  // 0–1, default 1.0
+  primaryMetric?: string
+  secondaryMetrics?: string[]
+}
+
+interface Variant {
+  id: string
+  name: string
+  weight: number  // relative weight, will be normalised
+  isControl?: boolean
+}
+
+interface TargetingRule {
+  attribute: string
+  operator: 'eq' | 'neq' | 'in' | 'notIn' | 'gt' | 'lt' | 'gte' | 'lte'
+  value: unknown
+}
+```
+
+### Assignment
+
+```ts
+interface Assignment {
+  readonly experimentId: string
+  readonly userId: string
+  readonly variantId: string
+  readonly isControl: boolean
+  readonly timestamp: number
+  readonly inExperiment: boolean  // false if excluded by targeting or sample rate
+}
+```
+
+### Metrics
+
+```ts
+interface MetricEvent {
+  name: string
+  value?: number
+  timestamp?: number
+  properties?: Record<string, unknown>
+}
+
+interface ExperimentResults {
+  readonly experimentId: string
+  readonly variants: ReadonlyArray<VariantResults>
+  readonly totalAssignments: number
+  readonly totalConversions: number
+}
+
+interface VariantResults {
+  readonly variantId: string
+  readonly assignments: number
+  readonly conversions: number
+  readonly conversionRate: number
+  readonly metrics: ReadonlyArray<{ name: string; count: number; sum: number; avg: number }>
+}
+```
+
+### Provider interface
+
+```ts
+interface ExperimentProvider {
+  readonly name: string
+  getAssignment?(experimentId: string, userId: string): string | null
+  trackAssignment?(assignment: Assignment): void
+  trackConversion?(experimentId: string, userId: string, metric: MetricEvent): void
+}
+
+type UserAttributes = Record<string, string | number | boolean>
+
+interface ExperimentManagerOptions {
+  logger?: LoggerLike
+  clock?: Clock
+  rng?: Rng
+  eventBus?: EventBusLike
+  hashSeed?: number
+}
+```
+
+### Bucketing utilities
+
+```ts
+function bucketUser(userId: string, experimentId: string, seed?: number): number  // 0–1
+function normaliseWeights(variants: Variant[]): Array<Variant & { normalizedWeight: number }>
+function selectVariant(variants: Variant[], bucket: number): Variant
+```
+
+### Error codes
+
+```ts
+type AbTestingErrorCode =
+  | 'AB_EXPERIMENT_NOT_FOUND'
+  | 'AB_EXPERIMENT_INACTIVE'
+  | 'AB_INVALID_DEFINITION'
+  | 'AB_VARIANT_NOT_FOUND'
+  | 'AB_METRIC_INVALID'
+
+function registerAbTestingErrorCodes(): void
+```
+
+---
+
+## Genesis additions
+
+1. **Structured `AbTestingError`** extending `SuaveplanError` with stable `AB.*` codes
+2. **Zod schemas** for `ExperimentDefinition`, `Variant`, `TargetingRule`, `MetricEvent`
+3. **Deterministic FNV-1a bucketing** — injectable `Rng` for testing; FNV-1a hash by default for consistency with `@suaveplan/feature-flags`
+4. **Targeting rule evaluation** — 8 operators, evaluated before assignment
+5. **Provider plugin interface** — `use()` for external integrations
+6. **`AbortSignal`** available on async provider calls
+7. **OTel instrumentation** — span `ab.assign`, counter `ab.assignments_total`, counter `ab.conversions_total`, attributes include `experiment_id`, `variant_id`
+8. **Property-based tests** — bucketing distribution, targeting rule evaluation, weight normalisation
+
+---
+
+## File layout
+
+```text
+universal/workflow/ab-testing/
+  package.json
+  tsconfig.json
+  vitest.config.ts
+  src/
+    index.ts
+    error-codes.ts
+    constants/
+      index.ts
+    types/
+      common.ts
+      common.md
+    core/
+      experiment-manager/
+        experiment-manager.ts
+        experiment-manager.md
+        experiment-manager.types.ts
+        experiment-manager.test.ts
+        index.ts
+    bucketing/
+      bucketing.ts
+      bucketing.md
+      bucketing.test.ts
+      index.ts
+    targeting/
+      targeting.ts
+      targeting.md
+      targeting.test.ts
+      index.ts
+    results/
+      results.ts
+      results.md
+      results.test.ts
+      index.ts
+    providers/
+      provider.ts
+      provider.md
+      provider.test.ts
+      index.ts
+    __benchmarks__/
+      ab-testing.bench.ts
+  tests/
+    integration.test.ts
+    property.test.ts
+```
+
+---
+
+## Impact
+
+**Downstream packages**: Any application layer that needs experiment assignment. No existing genesis package currently depends on it.
+
+**Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/logger`, `@suaveplan/feature-flags` must be published.