suaveplan/openspec/archive/2026-04-24-enterprise-ab-testing/tasks.md

Tasks: enterprise-ab-testing

Change ID: enterprise-ab-testing Package: @suaveplan/ab-testing Target path: universal/workflow/ab-testing Prerequisite packages on Verdaccio: @suaveplan/error, @suaveplan/types, @suaveplan/events, @suaveplan/logger, @suaveplan/feature-flags

---

1. Scaffold

• Create universal/workflow/ab-testing/ with src/, tests/, src/__benchmarks__/, and feature subdirectories per the layout in proposal.md.
• Write package.json at version 0.1.0:
  • "type": "module", "sideEffects": false
  • exports map: "." barrel + one subpath per feature module
  • dependencies: @suaveplan/error workspace:^, @suaveplan/types workspace:^, @suaveplan/events workspace:^, @suaveplan/logger workspace:^, @suaveplan/feature-flags workspace:^, zod catalog:forms, @opentelemetry/api catalog:observability, tslib catalog:build
  • devDependencies: vitest catalog:testing, @vitest/coverage-istanbul catalog:testing, fast-check catalog:testing, mitata catalog:testing, @suaveplan/testing workspace:^
• Write tsconfig.json extending @suaveplan/typescript-config/library.json.
• Write vitest.config.ts using createTestConfig().
• Add lint, typecheck, test, test:coverage, build, docs:check, bench scripts.

---

2. Types and error codes

• Define all public types in src/types/common.ts: ExperimentDefinition, Variant, TargetingRule, Assignment, MetricEvent, ExperimentResults, VariantResults, ExperimentManagerOptions, ExperimentProvider, UserAttributes.
• Write src/types/common.md (≥200 words, simple + advanced examples).
• Define error codes in src/error-codes.ts: AbTestingErrorCode union, metadata, registerAbTestingErrorCodes().
• Write Zod schemas in src/types/schemas.ts + src/types/schemas.md.

---

3. Bucketing (src/bucketing/)

• Implement bucketUser(userId, experimentId, seed?):
  • FNV-1a hash of ${experimentId}:${userId} (with optional seed XOR'd in).
  • Normalises to [0, 1) float.
• Implement normaliseWeights(variants) — normalises weight values to sum to 1.
• Implement selectVariant(variants, bucket) — binary search over cumulative normalised weights.
• Write bucketing.test.ts (≤500 lines): determinism for same userId+experimentId, distribution spread across 10k users, weight edge cases (all equal, single variant, zero-weight skipped).
• Write bucketing.md (≥200 words).

---

4. Targeting (src/targeting/)

• Implement evaluateTargeting(rules: TargetingRule[], attributes: UserAttributes): boolean:
  • All rules must pass (implicit AND).
  • Operators: eq, neq, in, notIn, gt, lt, gte, lte.
  • Returns true when user matches all rules (or no rules defined).
• Write targeting.test.ts: each operator, multi-rule AND, no rules = true, missing attribute = false.
• Write targeting.md (≥200 words).

---

5. Results (src/results/)

• Implement computeResults(experimentId, assignments, conversions): ExperimentResults:
  • Group assignments and conversions by variant.
  • Compute conversionRate = conversions / assignments.
  • Aggregate metrics (count, sum, avg) per metric name per variant.
• Write results.test.ts: known data produces expected conversion rate, empty data, single variant.
• Write results.md (≥200 words).

---

6. Experiment manager (src/core/experiment-manager/)

• Implement createExperimentManager(options?):
  • Stores experiments in Map<string, ExperimentDefinition>.
  • Stores assignments and metric events in Map keyed by ${experimentId}:${userId}.
• Implement manager.defineExperiment(def) — validates with Zod; throws AbTestingError(AB_INVALID_DEFINITION) on schema error.
• Implement manager.assign(experimentId, userId, attributes?):
  • Check experiment exists and is running; throw AB_EXPERIMENT_INACTIVE otherwise.
  • Evaluate targeting rules; set inExperiment: false if excluded.
  • Apply sampleRate check using rng.next().
  • Use bucketUser to select variant.
  • Store assignment, emit OTel span ab.assign, counter ab.assignments_total.
  • Emit event-bus event ab:assigned.
  • Call any registered provider trackAssignment.
• Implement manager.track(experimentId, userId, metric) — appends metric event; counter ab.conversions_total.
• Implement manager.getResults(experimentId) — delegates to computeResults.
• Implement manager.use(provider) — registers provider.
• Write experiment-manager.test.ts (≤500 lines): full assignment flow, targeting exclusion, sample rate, metric tracking, results, provider called.
• Write experiment-manager.md (≥200 words).

---

7. Provider interface (src/providers/)

• Define ExperimentProvider interface.
• Write provider.test.ts: provider registered, trackAssignment called on assign.
• Write provider.md.

---

8. Property-based tests (tests/property.test.ts)

• Bucketing determinism: same userId + experimentId always produces same bucket value.
• Weight distribution: across 10k users, each variant's assignment count is within 10% of its weight proportion.
• Targeting AND semantics: if any rule fails, result is false; all rules passing → true.

---

9. Benchmarks (src/__benchmarks__/ab-testing.bench.ts)

• assign_hot: 10k assign calls for the same experiment (cache warm).
• bucket_hot: 10k bucketUser calls.

---

10. Docs

• All per-file .md files per layout in proposal.md.
• universal/workflow/ab-testing/README.md.

---

11. Verify and publish

• bunx turbo lint --filter=@suaveplan/ab-testing — zero errors.
• bunx turbo typecheck --filter=@suaveplan/ab-testing — zero errors.
• bunx turbo test --filter=@suaveplan/ab-testing — all tests pass.
• bunx turbo test:coverage --filter=@suaveplan/ab-testing — 100% on all touched modules.
• bun run docs:check --filter=@suaveplan/ab-testing — zero errors.
• bun scripts/publish.ts --package @suaveplan/ab-testing.
• Update CLAUDE.md package listing.
• Archive: move openspec/changes/enterprise-ab-testing/ to openspec/archive/2026-04-24-enterprise-ab-testing/.