suaveplan/openspec/archive/2026-04-24-enterprise-ab-testing/tasks.md

# Tasks: enterprise-ab-testing

**Change ID**: enterprise-ab-testing
**Package**: `@suaveplan/ab-testing`
**Target path**: `universal/workflow/ab-testing`
**Prerequisite packages on Verdaccio**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events`, `@suaveplan/logger`, `@suaveplan/feature-flags`

---

## 1. Scaffold

- [x] Create `universal/workflow/ab-testing/` with `src/`, `tests/`, `src/__benchmarks__/`, and feature subdirectories per the layout in `proposal.md`.
- [x] Write `package.json` at version `0.1.0`:
  - `"type": "module"`, `"sideEffects": false`
  - `exports` map: `"."` barrel + one subpath per feature module
  - `dependencies`: `@suaveplan/error workspace:^`, `@suaveplan/types workspace:^`, `@suaveplan/events workspace:^`, `@suaveplan/logger workspace:^`, `@suaveplan/feature-flags workspace:^`, `zod catalog:forms`, `@opentelemetry/api catalog:observability`, `tslib catalog:build`
  - `devDependencies`: `vitest catalog:testing`, `@vitest/coverage-istanbul catalog:testing`, `fast-check catalog:testing`, `mitata catalog:testing`, `@suaveplan/testing workspace:^`
- [x] Write `tsconfig.json` extending `@suaveplan/typescript-config/library.json`.
- [x] Write `vitest.config.ts` using `createTestConfig()`.
- [x] Add `lint`, `typecheck`, `test`, `test:coverage`, `build`, `docs:check`, `bench` scripts.

---

## 2. Types and error codes

- [x] Define all public types in `src/types/common.ts`: `ExperimentDefinition`, `Variant`, `TargetingRule`, `Assignment`, `MetricEvent`, `ExperimentResults`, `VariantResults`, `ExperimentManagerOptions`, `ExperimentProvider`, `UserAttributes`.
- [x] Write `src/types/common.md` (≥200 words, simple + advanced examples).
- [x] Define error codes in `src/error-codes.ts`: `AbTestingErrorCode` union, metadata, `registerAbTestingErrorCodes()`.
- [x] Write Zod schemas in `src/types/schemas.ts` + `src/types/schemas.md`.

---

## 3. Bucketing (`src/bucketing/`)

- [x] Implement `bucketUser(userId, experimentId, seed?)`:
  - FNV-1a hash of `${experimentId}:${userId}` (with optional seed XOR'd in).
  - Normalises to `[0, 1)` float.
- [x] Implement `normaliseWeights(variants)` — normalises `weight` values to sum to 1.
- [x] Implement `selectVariant(variants, bucket)` — binary search over cumulative normalised weights.
- [x] Write `bucketing.test.ts` (≤500 lines): determinism for same userId+experimentId, distribution spread across 10k users, weight edge cases (all equal, single variant, zero-weight skipped).
- [x] Write `bucketing.md` (≥200 words).

---

## 4. Targeting (`src/targeting/`)

- [x] Implement `evaluateTargeting(rules: TargetingRule[], attributes: UserAttributes): boolean`:
  - All rules must pass (implicit AND).
  - Operators: `eq`, `neq`, `in`, `notIn`, `gt`, `lt`, `gte`, `lte`.
  - Returns `true` when user matches all rules (or no rules defined).
- [x] Write `targeting.test.ts`: each operator, multi-rule AND, no rules = true, missing attribute = false.
- [x] Write `targeting.md` (≥200 words).

---

## 5. Results (`src/results/`)

- [x] Implement `computeResults(experimentId, assignments, conversions): ExperimentResults`:
  - Group assignments and conversions by variant.
  - Compute `conversionRate` = conversions / assignments.
  - Aggregate metrics (count, sum, avg) per metric name per variant.
- [x] Write `results.test.ts`: known data produces expected conversion rate, empty data, single variant.
- [x] Write `results.md` (≥200 words).

---

## 6. Experiment manager (`src/core/experiment-manager/`)

- [x] Implement `createExperimentManager(options?)`:
  - Stores experiments in `Map<string, ExperimentDefinition>`.
  - Stores assignments and metric events in `Map` keyed by `${experimentId}:${userId}`.
- [x] Implement `manager.defineExperiment(def)` — validates with Zod; throws `AbTestingError(AB_INVALID_DEFINITION)` on schema error.
- [x] Implement `manager.assign(experimentId, userId, attributes?)`:
  - Check experiment exists and is `running`; throw `AB_EXPERIMENT_INACTIVE` otherwise.
  - Evaluate targeting rules; set `inExperiment: false` if excluded.
  - Apply `sampleRate` check using `rng.next()`.
  - Use `bucketUser` to select variant.
  - Store assignment, emit OTel span `ab.assign`, counter `ab.assignments_total`.
  - Emit event-bus event `ab:assigned`.
  - Call any registered provider `trackAssignment`.
- [x] Implement `manager.track(experimentId, userId, metric)` — appends metric event; counter `ab.conversions_total`.
- [x] Implement `manager.getResults(experimentId)` — delegates to `computeResults`.
- [x] Implement `manager.use(provider)` — registers provider.
- [x] Write `experiment-manager.test.ts` (≤500 lines): full assignment flow, targeting exclusion, sample rate, metric tracking, results, provider called.
- [x] Write `experiment-manager.md` (≥200 words).

---

## 7. Provider interface (`src/providers/`)

- [x] Define `ExperimentProvider` interface.
- [x] Write `provider.test.ts`: provider registered, `trackAssignment` called on assign.
- [x] Write `provider.md`.

---

## 8. Property-based tests (`tests/property.test.ts`)

- [x] Bucketing determinism: same `userId` + `experimentId` always produces same bucket value.
- [x] Weight distribution: across 10k users, each variant's assignment count is within 10% of its weight proportion.
- [x] Targeting AND semantics: if any rule fails, result is `false`; all rules passing → `true`.

---

## 9. Benchmarks (`src/__benchmarks__/ab-testing.bench.ts`)

- [x] `assign_hot`: 10k `assign` calls for the same experiment (cache warm).
- [x] `bucket_hot`: 10k `bucketUser` calls.

---

## 10. Docs

- [x] All per-file `.md` files per layout in `proposal.md`.
- [x] `universal/workflow/ab-testing/README.md`.

---

## 11. Verify and publish

- [x] `bunx turbo lint --filter=@suaveplan/ab-testing` — zero errors.
- [x] `bunx turbo typecheck --filter=@suaveplan/ab-testing` — zero errors.
- [x] `bunx turbo test --filter=@suaveplan/ab-testing` — all tests pass.
- [x] `bunx turbo test:coverage --filter=@suaveplan/ab-testing` — 100% on all touched modules.
- [x] `bun run docs:check --filter=@suaveplan/ab-testing` — zero errors.
- [x] `bun scripts/publish.ts --package @suaveplan/ab-testing`.
- [x] Update `CLAUDE.md` package listing.
- [x] Archive: move `openspec/changes/enterprise-ab-testing/` to `openspec/archive/2026-04-24-enterprise-ab-testing/`.