Initial Commit

openspec/archive/2026-04-24-enterprise-queue/proposal.md

@@ -0,0 +1,267 @@
+# OpenSpec Change: enterprise-queue
+
+- **Change ID**: enterprise-queue
+- **Package**: `@suaveplan/queue`
+- **Category**: universal
+- **Target path**: `universal/streams/queue`
+- **Status**: PROPOSED
+- **Date**: 2026-04-24
+- **Reference spec**: `/mnt/s/web/suaveplan/enterprise/queue.md`
+
+---
+
+## Summary
+
+Deliver `@suaveplan/queue` on the genesis branch from scratch at enterprise quality. The package provides queue management primitives for task/job processing: a typed `Queue` interface, in-memory FIFO/priority implementations, `QueueWorker` for concurrent job processing with bounded concurrency, retry/dead-letter policies, `AbortSignal` cancellation, event emission for job lifecycle, OTel instrumentation, and Zod-validated configuration. This is the single-process queue substrate; `@suaveplan/queue-distribution` builds the distributed layer on top.
+
+---
+
+## Current-state audit (genesis)
+
+The package does not exist on genesis yet — fresh build. No prior source, no test results, no coverage data.
+
+---
+
+## Available workspace deps on genesis
+
+| Package | Role |
+|---|---|
+| `@suaveplan/error` (`workspace:^`) | Base class for `QueueError`; `defineCodes` for `QUEUE.*` codes |
+| `@suaveplan/types` (`workspace:^`) | `LoggerLike`, `Clock`, `EventBusLike`, `IdGenerator` injection interfaces |
+| `@suaveplan/events` (`workspace:^`) | Job lifecycle event emission (`job:enqueued`, `job:started`, `job:completed`, `job:failed`, `job:dead`) |
+| `zod` (`catalog:forms`) | Zod schemas for all public option and job types |
+| `@opentelemetry/api` (`catalog:observability`) | Spans, counters, histograms on enqueue, process, and retry operations |
+| `tslib` (`catalog:build`) | Runtime helpers |
+
+Dev deps: `vitest`, `@vitest/coverage-istanbul`, `fast-check`, `mitata` (all `catalog:testing`).
+
+---
+
+## Public API surface
+
+### Queue interface
+
+```ts
+interface Queue<T> {
+  readonly name: string
+  readonly size: number
+  enqueue(job: Job<T>): Promise<string>          // returns jobId
+  dequeue(): Promise<Job<T> | undefined>
+  peek(): Promise<Job<T> | undefined>
+  get(jobId: string): Promise<Job<T> | undefined>
+  ack(jobId: string): Promise<void>
+  nack(jobId: string, reason?: string): Promise<void>
+  remove(jobId: string): Promise<boolean>
+  clear(): Promise<void>
+  drain(): Promise<void>
+}
+
+interface Job<T> {
+  id: string
+  payload: T
+  priority?: number           // higher = processed first; default: 0
+  scheduledAt?: Date
+  maxRetries?: number
+  attemptCount: number
+  createdAt: Date
+  state: JobState
+}
+
+type JobState = "pending" | "processing" | "completed" | "failed" | "dead"
+```
+
+### Queue implementations
+
+```ts
+// In-memory FIFO queue
+function createMemoryQueue<T>(options?: MemoryQueueOptions<T>): Queue<T>
+
+// In-memory priority queue (max-heap on job.priority)
+function createPriorityQueue<T>(options?: PriorityQueueOptions<T>): Queue<T>
+
+// Delayed job queue (jobs are not dequeued before scheduledAt)
+function createDelayedQueue<T>(options?: DelayedQueueOptions<T>): Queue<T>
+
+interface MemoryQueueOptions<T> {
+  name?: string
+  maxSize?: number
+  clock?: Clock
+  idGenerator?: IdGenerator
+  bus?: EventBusLike
+  logger?: LoggerLike
+}
+
+type PriorityQueueOptions<T> = MemoryQueueOptions<T>
+type DelayedQueueOptions<T> = MemoryQueueOptions<T> & { pollIntervalMs?: number }
+```
+
+### QueueWorker
+
+```ts
+function createQueueWorker<T>(options: QueueWorkerOptions<T>): QueueWorker<T>
+
+interface QueueWorker<T> {
+  start(signal?: AbortSignal): Promise<void>
+  stop(): Promise<void>
+  readonly stats: WorkerStats
+}
+
+interface QueueWorkerOptions<T> {
+  queue: Queue<T>
+  handler: JobHandler<T>
+  concurrency?: number            // default: 1
+  retryPolicy?: RetryPolicy
+  deadLetterQueue?: Queue<T>
+  logger?: LoggerLike
+  clock?: Clock
+}
+
+type JobHandler<T> = (job: Job<T>, signal: AbortSignal) => Promise<void>
+
+interface RetryPolicy {
+  maxRetries: number
+  backoff: "constant" | "linear" | "exponential"
+  initialDelayMs: number
+  maxDelayMs?: number
+}
+
+interface WorkerStats {
+  processed: number
+  succeeded: number
+  failed: number
+  dead: number
+  retried: number
+  avgProcessingMs: number
+}
+```
+
+### Job lifecycle events (when bus provided)
+
+```ts
+"queue:job:enqueued"    // { queueName, jobId, priority }
+"queue:job:started"     // { queueName, jobId, attempt }
+"queue:job:completed"   // { queueName, jobId, durationMs }
+"queue:job:failed"      // { queueName, jobId, error, attempt, willRetry }
+"queue:job:dead"        // { queueName, jobId, reason }
+```
+
+### Error handling
+
+```ts
+class QueueError extends SuaveplanError<QueueErrorCode>
+
+type QueueErrorCode =
+  | "QUEUE_FULL"
+  | "QUEUE_JOB_NOT_FOUND"
+  | "QUEUE_WORKER_ABORTED"
+  | "QUEUE_HANDLER_FAILED"
+  | "QUEUE_INVALID_STATE"
+  | "QUEUE_DEAD_LETTER_FAILED"
+
+function registerQueueErrorCodes(): void
+```
+
+### Zod schemas
+
+```ts
+const JobSchema: ZodSchema
+const RetryPolicySchema: ZodSchema
+const MemoryQueueOptionsSchema: ZodSchema
+const QueueWorkerOptionsSchema: ZodSchema
+```
+
+---
+
+## New additions for genesis
+
+### 1. Delayed queue with injectable clock
+
+`createDelayedQueue` uses the injectable `Clock` for `scheduledAt` comparisons, enabling deterministic tests without real timers.
+
+### 2. Dead letter queue
+
+When a job exceeds `maxRetries`, it is moved to the `deadLetterQueue` (if configured) with the failure reason and full retry history attached. The dead letter queue is itself a `Queue<T>`, allowing inspection and manual replay.
+
+### 3. OTel instrumentation
+
+- `queue.enqueue` — span `queue.enqueue`, attributes: `queue.name`, `queue.priority`
+- `worker.process` — span `queue.worker.process`, attributes: `queue.name`, `queue.job_id`, `queue.attempt`
+- Counters: `queue.jobs_enqueued_total`, `queue.jobs_completed_total`, `queue.jobs_failed_total`, `queue.jobs_dead_total`
+- Histogram: `queue.processing_duration_ms` (attribute: `queue.name`)
+- Gauge: `queue.depth` (attribute: `queue.name`)
+
+### 4. Property-based tests (fast-check)
+
+- FIFO invariant: dequeue order matches enqueue order for equal-priority jobs
+- Priority invariant: highest-priority job is always dequeued first
+- Retry count bound: job.attemptCount never exceeds maxRetries + 1 before moving to dead letter
+- Worker concurrency: at most `concurrency` jobs are in `processing` state at any time
+
+### 5. Benchmark suite (`src/__benchmarks__/`)
+
+- Enqueue + dequeue 100k items (memory queue)
+- Worker with concurrency=10 processing 1k instant jobs
+
+---
+
+## File layout
+
+```text
+universal/streams/queue/
+  package.json
+  tsconfig.json
+  vitest.config.ts
+  src/
+    index.ts
+    error-codes.ts
+    constants/
+      index.ts
+    types/
+      common.ts
+      common.md
+    queues/
+      memory-queue/
+        memory-queue.ts
+        memory-queue.md
+        memory-queue.types.ts
+        memory-queue.test.ts
+        index.ts
+      priority-queue/
+        priority-queue.ts
+        priority-queue.md
+        priority-queue.types.ts
+        priority-queue.test.ts
+        index.ts
+      delayed-queue/
+        delayed-queue.ts
+        delayed-queue.md
+        delayed-queue.types.ts
+        delayed-queue.test.ts
+        index.ts
+    worker/
+      queue-worker/
+        queue-worker.ts
+        queue-worker.md
+        queue-worker.types.ts
+        queue-worker.test.ts
+        index.ts
+    utils/
+      retry-policy.ts
+      retry-policy.md
+      retry-policy.test.ts
+    __benchmarks__/
+      queue.bench.ts
+  tests/
+    integration.test.ts
+    property.test.ts
+```
+
+---
+
+## Impact
+
+**Downstream packages**: `@suaveplan/queue-distribution` extends this package with distributed worker pools. Any background-job, batch-processing, or task-scheduling feature in the monorepo uses this as the foundation.
+
+**Workspace dep prerequisites**: `@suaveplan/error`, `@suaveplan/types`, `@suaveplan/events` must be published to Verdaccio.
+
+**Breaking changes**: none (new package).