first commit

This commit is contained in:
2026-05-29 08:20:56 +01:00
commit 949299f58d
6368 changed files with 527176 additions and 0 deletions

View File

@@ -0,0 +1,100 @@
# queue-worker
<!-- begin:auto-docs -->
_(no submodules)_
<!-- end:auto-docs -->
## Purpose
`QueueWorker<T>` drains a `QueueLike<T>` queue by calling a user-supplied handler for each job. It manages concurrency (up to N simultaneous jobs), implements exponential-backoff retry, forwards exhausted jobs to an optional dead-letter queue, and honours `AbortSignal` for graceful shutdown.
The worker is intentionally decoupled from any specific queue implementation — it accepts any object that satisfies `QueueLike<T>`, allowing `MemoryQueue`, `PriorityQueue`, `DelayedQueue`, or a custom adapter to be swapped in.
## Features
- Configurable concurrency: processes up to N jobs simultaneously
- Exponential-backoff retry with configurable `maxAttempts`, `backoffMs`, and `backoffMultiplier`
- Dead-letter queue forwarding when all retries are exhausted
- `AbortSignal` cancellation: throws `QueueWorkerAbortedError` on abort
- `WorkerStats` snapshot (processed, succeeded, failed, dead, retried)
- `runWorker()` convenience wrapper for fire-and-forget usage
## Usage: Basic Example
```ts
import { createMemoryQueue } from "@suaveplan/queue/queue/memory-queue";
import { QueueWorker } from "@suaveplan/queue/worker/queue-worker";
const queue = createMemoryQueue<string>();
await queue.enqueue("hello");
await queue.enqueue("world");
const worker = new QueueWorker(queue, async (job) => {
console.log("Processing:", job.payload);
});
await worker.run();
console.log(worker.stats); // { processed: 2, succeeded: 2, failed: 0, dead: 0, retried: 0 }
```
## Usage: Advanced Example
```ts
import { createMemoryQueue } from "@suaveplan/queue/queue/memory-queue";
import { runWorker } from "@suaveplan/queue/worker/queue-worker";
const mainQueue = createMemoryQueue<{ taskId: string }>();
const deadLetterQueue = createMemoryQueue<unknown>();
await mainQueue.enqueue({ taskId: "a" });
await mainQueue.enqueue({ taskId: "b" });
let callCount = 0;
const stats = await runWorker(
mainQueue,
async (job) => {
callCount++;
if (callCount <= 4) throw new Error("transient failure"); // triggers retries
// succeeds on later attempts
},
{
concurrency: 2,
retryPolicy: { maxAttempts: 2, backoffMs: 10, backoffMultiplier: 2 },
deadLetterQueue,
}
);
console.log(stats.dead); // jobs that exhausted all retries
console.log(deadLetterQueue.size()); // dead-letter queue depth
```
## API Reference
### `QueueWorker<T>`
#### `new QueueWorker(queue, handler, options?)`
- `queue: QueueLike<T>` — source queue
- `handler: (job: Job<T>) => Promise<void>` — async job processor; throw to fail the job
- `options.concurrency` — default `1`
- `options.retryPolicy` — default `{ maxAttempts: 3, backoffMs: 1000, backoffMultiplier: 2 }`
- `options.deadLetterQueue?` — receives exhausted jobs
- `options.signal?` — `AbortSignal` for cancellation
#### `worker.run(): Promise<void>`
Drains the queue. Resolves when empty. Throws `QueueWorkerAbortedError` on abort.
#### `worker.stats: WorkerStats`
Snapshot: `{ processed, succeeded, failed, dead, retried }`.
#### `worker.running: boolean`
`true` while `run()` has not resolved.
### `runWorker<T>(queue, handler, options?): Promise<WorkerStats>`
Convenience factory: creates a worker, runs it, and returns stats.
## Implementation Notes
The drain loop maintains a `Set<Promise<void>>` of in-flight job promises. When `slots.size` reaches `concurrency`, it awaits `Promise.race(slots)` to free up a slot before dequeuing the next job. This achieves true concurrent processing without spawning unbound promises. Dead-letter forwarding errors wrap the original error as `QueueDeadLetterFailedError` so callers can distinguish delivery failures from job failures.