skills/docs/foundation/flow/queue/src/worker/queue-worker.md

queue-worker

(no submodules)

Purpose

QueueWorker<T> drains a QueueLike<T> queue by calling a user-supplied handler for each job. It manages concurrency (up to N simultaneous jobs), implements exponential-backoff retry, forwards exhausted jobs to an optional dead-letter queue, and honours AbortSignal for graceful shutdown.

The worker is intentionally decoupled from any specific queue implementation — it accepts any object that satisfies QueueLike<T>, allowing MemoryQueue, PriorityQueue, DelayedQueue, or a custom adapter to be swapped in.

Features

• Configurable concurrency: processes up to N jobs simultaneously
• Exponential-backoff retry with configurable maxAttempts, backoffMs, and backoffMultiplier
• Dead-letter queue forwarding when all retries are exhausted
• AbortSignal cancellation: throws QueueWorkerAbortedError on abort
• WorkerStats snapshot (processed, succeeded, failed, dead, retried)
• runWorker() convenience wrapper for fire-and-forget usage

Usage: Basic Example

import { createMemoryQueue } from "@suaveplan/queue/queue/memory-queue";
import { QueueWorker } from "@suaveplan/queue/worker/queue-worker";

const queue = createMemoryQueue<string>();
await queue.enqueue("hello");
await queue.enqueue("world");

const worker = new QueueWorker(queue, async (job) => {
  console.log("Processing:", job.payload);
});
await worker.run();
console.log(worker.stats); // { processed: 2, succeeded: 2, failed: 0, dead: 0, retried: 0 }

Usage: Advanced Example

import { createMemoryQueue } from "@suaveplan/queue/queue/memory-queue";
import { runWorker } from "@suaveplan/queue/worker/queue-worker";

const mainQueue = createMemoryQueue<{ taskId: string }>();
const deadLetterQueue = createMemoryQueue<unknown>();

await mainQueue.enqueue({ taskId: "a" });
await mainQueue.enqueue({ taskId: "b" });

let callCount = 0;
const stats = await runWorker(
  mainQueue,
  async (job) => {
    callCount++;
    if (callCount <= 4) throw new Error("transient failure"); // triggers retries
    // succeeds on later attempts
  },
  {
    concurrency: 2,
    retryPolicy: { maxAttempts: 2, backoffMs: 10, backoffMultiplier: 2 },
    deadLetterQueue,
  }
);

console.log(stats.dead); // jobs that exhausted all retries
console.log(deadLetterQueue.size()); // dead-letter queue depth

API Reference

QueueWorker<T>

new QueueWorker(queue, handler, options?)

• queue: QueueLike<T> — source queue
• handler: (job: Job<T>) => Promise<void> — async job processor; throw to fail the job
• options.concurrency — default 1
• options.retryPolicy — default { maxAttempts: 3, backoffMs: 1000, backoffMultiplier: 2 }
• options.deadLetterQueue? — receives exhausted jobs
• options.signal? — AbortSignal for cancellation

worker.run(): Promise<void>

Drains the queue. Resolves when empty. Throws QueueWorkerAbortedError on abort.

worker.stats: WorkerStats

Snapshot: { processed, succeeded, failed, dead, retried }.

worker.running: boolean

true while run() has not resolved.

runWorker<T>(queue, handler, options?): Promise<WorkerStats>

Convenience factory: creates a worker, runs it, and returns stats.

Implementation Notes

The drain loop maintains a Set<Promise<void>> of in-flight job promises. When slots.size reaches concurrency, it awaits Promise.race(slots) to free up a slot before dequeuing the next job. This achieves true concurrent processing without spawning unbound promises. Dead-letter forwarding errors wrap the original error as QueueDeadLetterFailedError so callers can distinguish delivery failures from job failures.