Move docs/ package mirror into document-navigator plugin
Relocate the ~6k-file docs tree from the repo root into plugins/document-navigator/docs/ so the plugin is self-contained and portable. docs/superpowers/ (specs and plans) stays at the repo root. Moved via cp+rm rather than git mv because directory rename() fails with EPERM on this WSL /mnt DrvFs mount; git rename-detection pairs the identical-content files at commit time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,129 @@
|
||||
# @suaveplan/queue-distribution - Worker-pool distribution layer: load-balancing strategies (round-robin / least-busy / weighted), worker pool, manager with retry + health checks, in-process LocalWorker. Transport-agnostic — HTTP and worker-thread plugins ship as separate packages.
|
||||
|
||||
## Summary
|
||||
|
||||
Worker-pool distribution layer: load-balancing strategies, worker pool management, manager with retry and health checks, in-process `LocalWorker`. Transport-agnostic — HTTP and worker-thread transports ship as separate plugin packages (`@suaveplan/queue-distribution-http`, `@suaveplan/queue-distribution-worker-thread`).
|
||||
|
||||
The package is decoupled from `@suaveplan/queue`'s `Job` model: distribution operates on a minimal `DistributedTask<I, O>` contract so it can serve any consumer that produces typed work units (queue, batch, RPC, agent dispatch).
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
## Documentation tree
|
||||
|
||||
- [errors](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [queue-distribution-error](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [internal](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [noop-logger](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [manager-dispatch](./src/manager/manager-dispatch.md) — `dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrat…
|
||||
- [manager-supervisor](./src/manager/manager-supervisor.md) — The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart o…
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [observability](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [instruments](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->- abstract-metrics — `abstract-metrics.ts` carries the…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [abstract-metrics](./src/worker/abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->
|
||||
- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_ <!-- end:auto-docs -->- abstract-metrics — `abstract-metrics.ts` carries the…
|
||||
<!-- end:auto-docs --> - [queue-distribution-error](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [internal](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [noop-logger](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager-dispatch](./src/manager/manager-dispatch.md) — `dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrat…
|
||||
- [manager-supervisor](./src/manager/manager-supervisor.md) — The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart o…
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [observability](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [instruments](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [abstract-metrics](./src/worker/abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- [queue-distribution-error](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [internal](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [noop-logger](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager-dispatch](./src/manager/manager-dispatch.md) — `dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrat…
|
||||
- [manager-supervisor](./src/manager/manager-supervisor.md) — The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart o…
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [observability](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [instruments](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [abstract-metrics](./src/worker/abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract…
|
||||
- [queue-distribution-error](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [internal](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [noop-logger](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager-dispatch](./src/manager/manager-dispatch.md) — `dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrat…
|
||||
- [manager-supervisor](./src/manager/manager-supervisor.md) — The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart o…
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [observability](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [instruments](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract-metrics.ts` carries th…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [abstract-metrics](./src/worker/abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract-metrics.ts` carries th…
|
||||
- [queue-distribution-error](./src/errors/queue-distribution-error.md) — <!-- begin:auto-docs --> _(no submodules)_- [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_ - [health-checker](./src/health-checker/health-checker.md) — <!-- begin:auto-docs --> _(no submodules)_- [internal](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_ - [noop-logger](./src/internal/noop-logger.md) — <!-- begin:auto-docs --> _(no submodules)_- [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_ - [base](./src/load-balancing/base/base.md) — <!-- begin:auto-docs --> _(no submodules)_- [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_ - [least-busy](./src/load-balancing/least-busy/least-busy.md) — <!-- begin:auto-docs --> _(no submodules)_- [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_ - [round-robin](./src/load-balancing/round-robin/round-robin.md) — <!-- begin:auto-docs --> _(no submodules)_- [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_ - [weighted](./src/load-balancing/weighted/weighted.md) — <!-- begin:auto-docs --> _(no submodules)_- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_ - [manager-dispatch](./src/manager/manager-dispatch.md) — `dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrat…
|
||||
- [manager-supervisor](./src/manager/manager-supervisor.md) — The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart o…
|
||||
- [manager](./src/manager/manager.md) — <!-- begin:auto-docs --> _(no submodules)_- [observability](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_ - [instruments](./src/observability/instruments.md) — <!-- begin:auto-docs --> _(no submodules)_- [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_ - [pool](./src/pool/pool.md) — <!-- begin:auto-docs --> _(no submodules)_- [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_ - [task](./src/task/task.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_ - [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_ - [abstract-metrics](./src/worker/abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./src/worker/abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_ - [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ - [local](./src/worker/local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ - [worker](./src/worker/worker.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @suaveplan/queue-distribution
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
```ts
|
||||
import {
|
||||
LocalWorker,
|
||||
WorkerManager,
|
||||
WorkerPool,
|
||||
} from "@suaveplan/queue-distribution";
|
||||
|
||||
const pool = new WorkerPool();
|
||||
pool.addWorker(new LocalWorker({}, { id: "a", capacity: 4 }));
|
||||
pool.addWorker(new LocalWorker({}, { id: "b", capacity: 4 }));
|
||||
await pool.startAllWorkers();
|
||||
|
||||
const manager = new WorkerManager(pool, {
|
||||
autoHealthCheck: false,
|
||||
maxTaskRetries: 2,
|
||||
});
|
||||
|
||||
const task = {
|
||||
id: "double",
|
||||
execute: async (n: number) => n * 2,
|
||||
getStatus: () => "PENDING" as const,
|
||||
setStatus: () => {},
|
||||
getMetadata: () => ({ createdAt: new Date(), attempts: 0 }),
|
||||
};
|
||||
|
||||
console.log(await manager.assignTask(task, 21)); // 42
|
||||
```
|
||||
|
||||
## Modules
|
||||
|
||||
- [Task contract](./src/task/task.md) — `DistributedTask<I, O>`, `TaskStatus`, `TaskMetadata`.
|
||||
- [Worker contract](./src/worker/worker.md) — `Worker`, `WorkerStatus`, `WorkerHealth`, `WorkerInfo` + Zod schemas.
|
||||
- [AbstractWorker](./src/worker/abstract/abstract.md) — base class with lifecycle FSM, metrics, OTel emission.
|
||||
- [`AbstractWorker` metrics helpers](./src/worker/abstract/abstract-metrics.md) — `WorkerMetricsState` accumulator.
|
||||
- [LocalWorker](./src/worker/local/local.md) — in-process executor.
|
||||
- [Strategy base](./src/load-balancing/base/base.md) — `LoadBalancingStrategy`, `AbstractLoadBalancingStrategy`.
|
||||
- [RoundRobinStrategy](./src/load-balancing/round-robin/round-robin.md).
|
||||
- [LeastBusyStrategy](./src/load-balancing/least-busy/least-busy.md) — optional EWMA performance bias.
|
||||
- [WeightedStrategy](./src/load-balancing/weighted/weighted.md) — injectable `Rng`.
|
||||
- [PeriodicHealthChecker](./src/health-checker/health-checker.md) — thresholds + cache + timeout.
|
||||
- [WorkerPool](./src/pool/pool.md) — CRUD + criteria-based search.
|
||||
- [WorkerManager](./src/manager/manager.md) — public dispatch surface.
|
||||
- [WorkerManager dispatch](./src/manager/manager-dispatch.md) — bounded retry loop with @suaveplan/retry.
|
||||
- [WorkerManager supervisor](./src/manager/manager-supervisor.md) — health-loop and restart logic.
|
||||
- [Errors](./src/errors/queue-distribution-error.md) — `QueueDistributionError` + 12 stable codes.
|
||||
- [Observability](./src/observability/instruments.md) — every counter, histogram, span the package emits.
|
||||
- [Internal: NOOP_LOGGER](./src/internal/noop-logger.md).
|
||||
|
||||
## Design highlights
|
||||
|
||||
- 100% test coverage (statements + branches + functions + lines) under `provider: "istanbul"`.
|
||||
- Injectable `Clock`, `IdGenerator`, `Rng`, `LoggerLike` everywhere — no `Math.random` / `Date.now` / `uuid` at module scope.
|
||||
- `AbortSignal` threaded through every async public method.
|
||||
- Retry routed through `@suaveplan/retry`; bounded loop, no recursion.
|
||||
- Zod schemas at every public boundary (`WorkerHealthSchema`, `WorkerInfoSchema`, `TaskMetadataSchema`, …).
|
||||
- Structured `SuaveplanError` subclasses with a frozen stable-code registry.
|
||||
- Tree-shakeable via 15 per-feature subpath exports.
|
||||
|
||||
## Status
|
||||
|
||||
This package is part of the rebuild — version `0.1.0` while it stabilises. Graduation to `1.0.0` follows the criteria in [`CLAUDE.md`](../../../CLAUDE.md).
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# `QueueDistributionError` registry
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`QueueDistributionError` is the base class for every structured error thrown by `@suaveplan/queue-distribution`. Each error subclass carries a stable code from the registry produced by `defineCodes("QUEUE_DIST", [...])`, so consumers can match on `error.code` programmatically without string-comparing the message. The registry is exhaustive: every code is thrown by at least one production code path, and consumers can rely on the closed union.
|
||||
|
||||
## Features
|
||||
|
||||
- 12 stable codes — `WORKER_NOT_AVAILABLE`, `WORKER_DUPLICATE`, `WORKER_BUSY_STOP`, `WORKER_NOT_FOUND`, `WORKER_RESTART_EXHAUSTED`, `NO_AVAILABLE_WORKERS`, `STRATEGY_NO_WORKER_SELECTED`, `TASK_RETRIES_EXHAUSTED`, `HEALTH_CHECK_TIMEOUT`, `HEALTH_CHECK_FAILED`, `INVALID_OPTIONS`, `TASK_CANCELLED`.
|
||||
- One concrete subclass per code with a typed constructor signature.
|
||||
- Optional `cause` is narrowed to `Error` instances only (non-Error causes are dropped to avoid pollution of the cause chain).
|
||||
- Compatible with `SuaveplanError`'s OTel span-event adapter and i18n message resolver.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
QueueDistributionError,
|
||||
QUEUE_DIST_CODES,
|
||||
TaskRetriesExhaustedError,
|
||||
} from "@suaveplan/queue-distribution/errors/queue-distribution-error";
|
||||
|
||||
try {
|
||||
await manager.assignTask(task, input);
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof QueueDistributionError) {
|
||||
if (error.code === QUEUE_DIST_CODES.TASK_RETRIES_EXHAUSTED) {
|
||||
// Route to dead-letter handling.
|
||||
}
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Combine the discriminated codes with `@suaveplan/error`'s cause-chain traversal to extract the originating worker error.
|
||||
|
||||
```ts
|
||||
import {
|
||||
QueueDistributionError,
|
||||
QUEUE_DIST_CODES,
|
||||
} from "@suaveplan/queue-distribution/errors/queue-distribution-error";
|
||||
|
||||
function rootCauseOf(error: QueueDistributionError): unknown {
|
||||
let current: unknown = error;
|
||||
while (current instanceof Error && current.cause !== undefined) {
|
||||
current = current.cause;
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
if (
|
||||
error instanceof QueueDistributionError &&
|
||||
error.code === QUEUE_DIST_CODES.TASK_RETRIES_EXHAUSTED
|
||||
) {
|
||||
console.error("root cause:", rootCauseOf(error));
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `const QUEUE_DIST_CODES` — map produced by `defineCodes("QUEUE_DIST", [...])`.
|
||||
- `type QueueDistributionErrorCode` — union of every code in the registry.
|
||||
- `class QueueDistributionError extends SuaveplanError<QueueDistributionErrorCode>`.
|
||||
- `class WorkerNotAvailableError({ workerId, status, cause? })`.
|
||||
- `class WorkerDuplicateError({ workerId })`.
|
||||
- `class WorkerBusyStopError({ workerId, activeTasks })`.
|
||||
- `class WorkerNotFoundError({ workerId })`.
|
||||
- `class WorkerRestartExhaustedError({ workerId, attempts })`.
|
||||
- `class NoAvailableWorkersError()`.
|
||||
- `class StrategyNoWorkerSelectedError({ strategy })`.
|
||||
- `class TaskRetriesExhaustedError({ taskId, attempts, cause? })`.
|
||||
- `class HealthCheckTimeoutError({ workerId, timeoutMs })`.
|
||||
- `class HealthCheckFailedError({ workerId, cause? })`.
|
||||
- `class InvalidOptionsError({ field, reason, cause? })`.
|
||||
- `class TaskCancelledError({ taskId })`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`asErrorCause` narrows the optional `cause` constructor argument to `Error | undefined`. Non-Error values (strings, plain objects, numbers) are silently dropped because `SuaveplanError`'s cause chain expects an `Error`-shaped chain. This keeps `error.cause.message` reliable for consumers who walk the chain. The registry is built with `as const` so every code is a string-literal subtype of `QueueDistributionErrorCode`, letting TypeScript exhaustiveness-check `switch` statements on `error.code`.
|
||||
@@ -0,0 +1,75 @@
|
||||
# `PeriodicHealthChecker`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`PeriodicHealthChecker` races each worker's `getHealth()` against a configurable timeout, tracks consecutive failure and success counts, flips workers between `ONLINE`/`BUSY` and `UNHEALTHY` only after sustained badness or recovery, and optionally caches recent results so back-to-back probes don't hammer the underlying transport. It implements the `WorkerHealthChecker` interface and is the default health checker that `WorkerManager` constructs when no other is supplied.
|
||||
|
||||
## Features
|
||||
|
||||
- Promise.race between `worker.getHealth()` and a `setTimeout`-driven timeout (default 5 s).
|
||||
- `unhealthyThreshold` (default 3) — consecutive failures required to flip to `UNHEALTHY`.
|
||||
- `healthyThreshold` (default 2) — consecutive successes required to recover.
|
||||
- Optional caching (`cacheResults`, `cacheTimeMs`; both default-on at 5 s) — backed by the injected `Clock`.
|
||||
- OTel `queue_dist.health.check` span + `queue_dist.health_check.duration_ms` histogram.
|
||||
- `getHealthCheckCounts(id)`, `resetHealthCheckCounts(id)`, `clearCache()` for inspection / reset.
|
||||
- Recovery transitions to `BUSY` when `activeTasks > 0`, else `ONLINE`.
|
||||
- Injectable `Clock` and `LoggerLike`; defaults are `Date.now` and `NOOP_LOGGER`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { PeriodicHealthChecker } from "@suaveplan/queue-distribution/health-checker/health-checker";
|
||||
|
||||
const checker = new PeriodicHealthChecker({
|
||||
unhealthyThreshold: 3,
|
||||
healthyThreshold: 2,
|
||||
});
|
||||
const health = await checker.checkHealth(worker);
|
||||
if (!health.isHealthy) {
|
||||
console.warn(`Worker ${worker.id} unhealthy`);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Combine with a `WorkerPool` to fan-out checks and react to status transitions.
|
||||
|
||||
```ts
|
||||
import { PeriodicHealthChecker } from "@suaveplan/queue-distribution/health-checker/health-checker";
|
||||
import { WorkerStatus } from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
const checker = new PeriodicHealthChecker(
|
||||
{ unhealthyThreshold: 2, healthyThreshold: 1, cacheResults: false },
|
||||
{ logger: console as never },
|
||||
);
|
||||
|
||||
async function probeAll(workers: readonly Worker[]): Promise<void> {
|
||||
const ids = await Promise.all(
|
||||
workers.map(async (worker) => {
|
||||
const isHealthy = await checker.isHealthy(worker);
|
||||
return { id: worker.id, status: worker.status, isHealthy };
|
||||
}),
|
||||
);
|
||||
for (const row of ids) {
|
||||
if (row.status === WorkerStatus.UNHEALTHY) {
|
||||
console.error(`Worker ${row.id} is unhealthy`);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface WorkerHealthChecker` — `checkHealth(worker, signal?)`, `isHealthy(worker, signal?)`.
|
||||
- `interface HealthCheckerOptions` — `checkIntervalMs?`, `unhealthyThreshold?`, `healthyThreshold?`, `healthCheckTimeoutMs?`, `cacheResults?`, `cacheTimeMs?`.
|
||||
- `interface HealthCheckerDeps` — `clock?`, `logger?`.
|
||||
- `class PeriodicHealthChecker implements WorkerHealthChecker`.
|
||||
- `getCheckInterval(): number`, `checkHealth(worker, signal?): Promise<WorkerHealth>`, `isHealthy(worker, signal?): Promise<boolean>`.
|
||||
- `getHealthCheckCounts(workerId): { failures, successes } | null`, `resetHealthCheckCounts(workerId): void`, `clearCache(): void`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The class flips status both ways without holding a lock; concurrent calls into `checkHealth` against the same worker can theoretically race the threshold check. In practice that means a brief window where status oscillates — acceptable because both `BUSY` and `ONLINE` are eligible for dispatch and the threshold acts as a debouncer over a longer window. The cache is keyed by `worker.id` so removing and re-adding a worker with the same id continues from the old cache; call `clearCache()` after `pool.removeWorker` if isolation matters. The Promise.race uses `setTimeout` rather than the injected `Clock` so the timeout fires under real time even when consumers mock the clock — vitest's `vi.useFakeTimers()` does work because it patches `setTimeout` globally.
|
||||
@@ -0,0 +1,50 @@
|
||||
# `NOOP_LOGGER`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`NOOP_LOGGER` is a shared `LoggerLike` implementation whose four methods (`debug` / `info` / `warn` / `error`) all return `undefined`. Components inside `@suaveplan/queue-distribution` use it as the default when no logger is injected, so they can call `this.logger.warn(...)` unconditionally without a per-call `?.` check. Sharing one constant across `WorkerPool`, `WorkerManager`, `PeriodicHealthChecker`, `AbstractWorker`, and the supervisor avoids per-file duplication of dead-code stubs that would otherwise show up as uncovered branches in coverage reports.
|
||||
|
||||
## Features
|
||||
|
||||
- Conforms to the `LoggerLike` structural contract from `@suaveplan/types`.
|
||||
- All four methods return `undefined` synchronously.
|
||||
- Lazy by construction: no allocation per call, just an immediate return.
|
||||
- Reused across every component that accepts an optional `LoggerLike`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { NOOP_LOGGER } from "@suaveplan/queue-distribution/internal/noop-logger";
|
||||
|
||||
const logger = process.env.DEBUG === "1" ? console : NOOP_LOGGER;
|
||||
logger.info("starting up");
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Use the noop logger when wiring a test fixture that doesn't care about log output. Tests that DO want to inspect logs can substitute a simple array-pusher:
|
||||
|
||||
```ts
|
||||
import type { LoggerLike } from "@suaveplan/types";
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
|
||||
const lines: string[] = [];
|
||||
const probe: LoggerLike = {
|
||||
debug: (m) => lines.push(`debug: ${m}`),
|
||||
info: (m) => lines.push(`info: ${m}`),
|
||||
warn: () => {},
|
||||
error: () => {},
|
||||
};
|
||||
const pool = new WorkerPool({ logger: probe });
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `const NOOP_LOGGER: LoggerLike` — frozen-in-shape no-op implementation.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Centralising the noop methods here means coverage tools see one set of empty-body functions rather than one per component, and a single test covers all of them (`internal/noop-logger.test.ts`). Components import `NOOP_LOGGER` rather than declaring their own local constant to keep this contract honest. The methods are written as `(): void => {}` rather than the shorter `noop` shorthand so TypeScript's structural typing matches the `LoggerLike` signature exactly.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Load-balancing strategy base
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`LoadBalancingStrategy` is the structural interface every load-balancing strategy implements. `AbstractLoadBalancingStrategy` provides the `filterAvailableWorkers` helper that every concrete strategy uses to narrow the candidate pool to workers that are actually able to accept a task. Splitting the filter into the abstract base means each strategy file stays focused on its scoring policy rather than reimplementing eligibility logic.
|
||||
|
||||
## Features
|
||||
|
||||
- `selectWorker(workers, task?)` — pick one worker or `null` if none are eligible.
|
||||
- Optional `notifyWorkerSelected(worker, task)` callback for strategies that maintain per-worker counters.
|
||||
- Optional `notifyTaskCompleted(worker, task, success, durationMs)` for strategies that learn from outcomes (e.g. EWMA in `LeastBusyStrategy`).
|
||||
- `getName(): string` — short identifier used in logs and OTel attributes.
|
||||
- `filterAvailableWorkers(workers)` — keeps `status` in `{ONLINE, BUSY}` AND `activeTasks < capacity` AND `canAcceptTask()` returns `true`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
AbstractLoadBalancingStrategy,
|
||||
type LoadBalancingStrategy,
|
||||
} from "@suaveplan/queue-distribution/load-balancing/base/base";
|
||||
import type { Worker } from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
class FirstAvailableStrategy extends AbstractLoadBalancingStrategy {
|
||||
selectWorker(workers: readonly Worker[]): Worker | null {
|
||||
return this.filterAvailableWorkers(workers)[0] ?? null;
|
||||
}
|
||||
getName(): string {
|
||||
return "first-available";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Strategies are pluggable — implement the interface directly when you need something more elaborate than what the shipped strategies cover.
|
||||
|
||||
```ts
|
||||
import type { LoadBalancingStrategy } from "@suaveplan/queue-distribution/load-balancing/base/base";
|
||||
import type { Worker } from "@suaveplan/queue-distribution/worker/worker";
|
||||
import type { DistributedTask } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
class AffinityStrategy implements LoadBalancingStrategy {
|
||||
readonly #lastWorkerByTaskId = new Map<string, string>();
|
||||
selectWorker(workers: readonly Worker[], task?: DistributedTask): Worker | null {
|
||||
if (task !== undefined) {
|
||||
const sticky = this.#lastWorkerByTaskId.get(task.id);
|
||||
const hit = workers.find((worker) => worker.id === sticky && worker.canAcceptTask());
|
||||
if (hit !== undefined) {
|
||||
return hit;
|
||||
}
|
||||
}
|
||||
return workers.find((worker) => worker.canAcceptTask()) ?? null;
|
||||
}
|
||||
notifyWorkerSelected(worker: Worker, task: DistributedTask): void {
|
||||
this.#lastWorkerByTaskId.set(task.id, worker.id);
|
||||
}
|
||||
getName(): string {
|
||||
return "affinity";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface LoadBalancingStrategy` — fields above.
|
||||
- `abstract class AbstractLoadBalancingStrategy implements LoadBalancingStrategy` — `filterAvailableWorkers(workers)` returns `Worker[]`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`filterAvailableWorkers` enforces three predicates rather than just one because workers can be `BUSY` but still capacity-available (running another task while having spare slots). The `canAcceptTask()` call is the worker's own override point — `RemoteWorker`, for example, returns `false` while reconnecting even if the pool's view of `status` and `activeTasks` would otherwise permit assignment. Strategies should not bypass this helper.
|
||||
@@ -0,0 +1,57 @@
|
||||
# `LeastBusyStrategy`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`LeastBusyStrategy` selects the eligible worker with the fewest in-flight tasks. When `usePerformanceMetrics` is enabled it also tracks a rolling-average execution time per worker (success only) and uses it to break ties — preferring faster workers on equal load. The strategy is the right default for heterogeneous pools where worker performance varies but capacity counts are the primary distribution signal.
|
||||
|
||||
## Features
|
||||
|
||||
- Tie-break order: lower `activeTasks` → (optional) lower `avgExecutionTime` → higher `capacity` → higher `weight`.
|
||||
- `notifyTaskCompleted(worker, task, success, durationMs)` updates the per-worker EWMA cumulative average; failures are ignored so a flaky worker can't bias selection downward.
|
||||
- `getWorkerPerformance(id)` returns the stored record or `null`.
|
||||
- `resetPerformanceMetrics()` clears the registry.
|
||||
- `setUsePerformanceMetrics(enabled)` toggles the flag at runtime; `isUsingPerformanceMetrics()` reports state.
|
||||
- `getName()` returns `"least-busy"`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { LeastBusyStrategy } from "@suaveplan/queue-distribution/load-balancing/least-busy/least-busy";
|
||||
|
||||
const strategy = new LeastBusyStrategy();
|
||||
// Pool of workers with mixed activeTasks; the one with the fewest wins.
|
||||
const selected = strategy.selectWorker(pool.getAvailableWorkers());
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Switch on performance bias when the pool contains a mix of fast and slow nodes (e.g. cross-region workers). The strategy will gradually steer load away from slower nodes.
|
||||
|
||||
```ts
|
||||
import { LeastBusyStrategy } from "@suaveplan/queue-distribution/load-balancing/least-busy/least-busy";
|
||||
|
||||
const strategy = new LeastBusyStrategy({ usePerformanceMetrics: true });
|
||||
// As tasks complete, the worker manager feeds outcomes back.
|
||||
strategy.notifyTaskCompleted(workerA, task, true, 120);
|
||||
strategy.notifyTaskCompleted(workerB, task, true, 380);
|
||||
strategy.selectWorker([workerA, workerB]); // prefers A when activeTasks tied
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class LeastBusyStrategy extends AbstractLoadBalancingStrategy`.
|
||||
- `constructor(options?: { usePerformanceMetrics?: boolean })`.
|
||||
- `selectWorker(workers, task?): Worker | null`.
|
||||
- `notifyTaskCompleted(worker, task, success, durationMs): void`.
|
||||
- `getWorkerPerformance(workerId): LeastBusyPerformanceRecord | null` — `{ totalExecutionTime, taskCount, avgExecutionTime }`.
|
||||
- `resetPerformanceMetrics(): void`.
|
||||
- `setUsePerformanceMetrics(enabled: boolean): void`.
|
||||
- `isUsingPerformanceMetrics(): boolean`.
|
||||
- `getName(): string` — returns `"least-busy"`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Tie-break by `capacity` then `weight` (when no performance data is available) prefers larger, more powerful workers — so a freshly added high-capacity worker absorbs additional load even if every worker currently has zero in-flight tasks. The performance map is per-worker keyed by id and persists across pool changes; removing and re-adding a worker with the same id continues the rolling average. If that behaviour is undesirable, call `resetPerformanceMetrics()` after `pool.removeWorker`.
|
||||
@@ -0,0 +1,63 @@
|
||||
# `RoundRobinStrategy`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`RoundRobinStrategy` selects available workers in rotation order. The pointer is internal to the instance — two strategies sharing a pool advance independently — and clamps when the pool shrinks below the current index so iteration stays in bounds. Round-robin is the right default for homogeneous worker pools where every worker has similar capacity, weight, and latency characteristics.
|
||||
|
||||
## Features
|
||||
|
||||
- Cyclic order across eligible workers (filtered via `AbstractLoadBalancingStrategy.filterAvailableWorkers`).
|
||||
- `reset()` zeroes the pointer for deterministic test setups.
|
||||
- Pool-shrink-safe: if the pointer index is now beyond the available-worker list, it clamps to zero.
|
||||
- `getName()` returns `"round-robin"`.
|
||||
- No external dependencies (no `Clock`, no `Rng`, no `IdGenerator`).
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { RoundRobinStrategy } from "@suaveplan/queue-distribution/load-balancing/round-robin/round-robin";
|
||||
import { LocalWorker } from "@suaveplan/queue-distribution/worker/local/local";
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
|
||||
const pool = new WorkerPool();
|
||||
pool.addWorker(new LocalWorker({}, { id: "a" }));
|
||||
pool.addWorker(new LocalWorker({}, { id: "b" }));
|
||||
const strategy = new RoundRobinStrategy();
|
||||
strategy.selectWorker(pool.getAvailableWorkers()); // worker "a"
|
||||
strategy.selectWorker(pool.getAvailableWorkers()); // worker "b"
|
||||
strategy.selectWorker(pool.getAvailableWorkers()); // worker "a"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Combine `RoundRobinStrategy` with a `WorkerManager` to drive task dispatch.
|
||||
|
||||
```ts
|
||||
import { WorkerManager } from "@suaveplan/queue-distribution/manager/manager";
|
||||
import { RoundRobinStrategy } from "@suaveplan/queue-distribution/load-balancing/round-robin/round-robin";
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
|
||||
const pool = new WorkerPool();
|
||||
// ... add workers ...
|
||||
const manager = new WorkerManager(pool, {
|
||||
balancer: new RoundRobinStrategy(),
|
||||
autoHealthCheck: false,
|
||||
maxTaskRetries: 2,
|
||||
});
|
||||
|
||||
await manager.assignTask(task, input);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class RoundRobinStrategy extends AbstractLoadBalancingStrategy`.
|
||||
- `selectWorker(workers, task?): Worker | null` — return next eligible worker or `null`.
|
||||
- `reset(): void` — set pointer to zero.
|
||||
- `getName(): string` — returns `"round-robin"`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The strategy stores its pointer with a `#`-prefixed private field so subclasses cannot tamper with it. The clamp-on-shrink behaviour is non-obvious but matters: if a worker is removed from the pool mid-dispatch, the next selection might otherwise reach into a non-existent index. The clamp keeps iteration bounded without needing a synchronisation primitive between pool-shrink events and strategy state.
|
||||
@@ -0,0 +1,63 @@
|
||||
# `WeightedStrategy`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`WeightedStrategy` selects workers with probability proportional to their `weight`. Randomness is sourced from a caller-supplied `Rng` (from `@suaveplan/types`) — the strategy never calls `Math.random()` at module scope, so tests can replay deterministic sequences and seeded production runs are reproducible. When the pool's total weight is zero or negative, the strategy degrades to uniform random so dispatch can still make forward progress.
|
||||
|
||||
## Features
|
||||
|
||||
- Proportional probability — a worker with `weight: 6` is selected six times as often as one with `weight: 1`.
|
||||
- Uniform fallback when `totalWeight <= 0`.
|
||||
- Injectable `Rng`; deterministic under fixed seeds.
|
||||
- Zero-allocation hot path (single linear scan through filtered workers).
|
||||
- `getName()` returns `"weighted"`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { WeightedStrategy } from "@suaveplan/queue-distribution/load-balancing/weighted/weighted";
|
||||
|
||||
const rng = {
|
||||
nextUint32: () => Math.floor(Math.random() * 0xffffffff),
|
||||
nextFloat: () => Math.random(),
|
||||
};
|
||||
const strategy = new WeightedStrategy({ rng });
|
||||
const selected = strategy.selectWorker(pool.getAvailableWorkers());
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
For deterministic tests, drive the strategy from a seeded sequence so each call returns a predictable worker.
|
||||
|
||||
```ts
|
||||
import type { Rng } from "@suaveplan/types";
|
||||
import { WeightedStrategy } from "@suaveplan/queue-distribution/load-balancing/weighted/weighted";
|
||||
|
||||
function fromSequence(values: readonly number[]): Rng {
|
||||
let i = 0;
|
||||
return {
|
||||
nextUint32: () => values[i++ % values.length] * 0xffffffff,
|
||||
nextFloat: () => values[i++ % values.length],
|
||||
};
|
||||
}
|
||||
|
||||
// totalWeight = 10, cumulative = [1, 4, 10]
|
||||
// nextFloat 0.05 -> target 0.5 -> w1; 0.35 -> 3.5 -> w2; 0.95 -> 9.5 -> w3
|
||||
const strategy = new WeightedStrategy({
|
||||
rng: fromSequence([0.05, 0.35, 0.95]),
|
||||
});
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class WeightedStrategy extends AbstractLoadBalancingStrategy`.
|
||||
- `constructor(deps: { rng: Rng })`.
|
||||
- `selectWorker(workers, task?): Worker | null`.
|
||||
- `getName(): string` — returns `"weighted"`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The weighted selection uses inclusive cumulative bucketing — `target <= cumulative` — so the very-edge case where `target === totalWeight` (only possible if `nextFloat()` returns exactly `1.0`, which is technically allowed by the half-open `[0, 1)` contract for some Rng implementations) still falls inside the last worker's bucket. The trailing `return available[length - 1] as Worker` is a safety net for floating-point precision drift on very long pools where cumulative sums could drift slightly below the target.
|
||||
@@ -0,0 +1,67 @@
|
||||
# `dispatchWithRetry`
|
||||
|
||||
## Purpose
|
||||
|
||||
`dispatchWithRetry` is the core retry loop of `WorkerManager`. It picks a worker via the configured `LoadBalancingStrategy`, executes the task, and retries up to `maxTaskRetries` on failure, sleeping `taskRetryDelayMs` between attempts via `@suaveplan/retry`'s `sleep`. The loop is bounded (no recursive self-call) and honours `AbortSignal` mid-flight — interrupting the sleep, surfacing `TaskCancelledError` thrown from the worker, and short-circuiting cleanly. Implementation is factored out of `manager.ts` so each file stays under the 300-LOC budget.
|
||||
|
||||
## Features
|
||||
|
||||
- Re-selects a worker on every iteration so a single bad worker can't capture every retry.
|
||||
- Emits OTel `queue_dist.manager.assign` span with `task_id` and `worker_id` attributes; histogram + counter per attempt.
|
||||
- Bounded loop — no recursion stack growth on long retry counts.
|
||||
- `TaskCancelledError` thrown by the worker propagates immediately without further retries.
|
||||
- Dead-letter via `config.onFailedTask(task)` callback when retries exhaust.
|
||||
- Per-attempt retry count tracked via `config.trackRetryCount(taskId, attempts)`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
`dispatchWithRetry` is internal to `manager.ts`; consumers use `WorkerManager.assignTask`. Direct usage looks like:
|
||||
|
||||
```ts
|
||||
import { dispatchWithRetry } from "@suaveplan/queue-distribution/manager/manager-dispatch";
|
||||
|
||||
const result = await dispatchWithRetry(
|
||||
task,
|
||||
input,
|
||||
{
|
||||
pool,
|
||||
balancer,
|
||||
maxTaskRetries: 3,
|
||||
taskRetryDelayMs: 250,
|
||||
clock: Date.now,
|
||||
onFailedTask: (t) => deadLetter.add(t),
|
||||
trackRetryCount: (id, n) => retries.set(id, n),
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
The dispatcher consults `pickWorker` and `runOnce` (helpers in this file) to apply the strategy and run a single attempt with full OTel + balancer-callback emission. To extend behaviour without forking, swap the `LoadBalancingStrategy` or wrap the `onFailedTask` callback.
|
||||
|
||||
```ts
|
||||
import { dispatchWithRetry } from "@suaveplan/queue-distribution/manager/manager-dispatch";
|
||||
|
||||
await dispatchWithRetry(task, input, {
|
||||
pool,
|
||||
balancer: {
|
||||
selectWorker: (workers) => workers[0] ?? null,
|
||||
notifyTaskCompleted: (worker, t, success, durationMs) => metrics.record(success, durationMs),
|
||||
getName: () => "first",
|
||||
},
|
||||
maxTaskRetries: 5,
|
||||
taskRetryDelayMs: 500,
|
||||
clock: () => Date.now(),
|
||||
onFailedTask: (task) => alertingClient.fire(task.id),
|
||||
trackRetryCount: () => undefined,
|
||||
}, ctrl.signal);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface DispatchConfig` — `pool`, `balancer`, `maxTaskRetries`, `taskRetryDelayMs`, `clock`, `onFailedTask`, `trackRetryCount`.
|
||||
- `function dispatchWithRetry<I, O>(task, input, config, signal?): Promise<O>` — throws `NoAvailableWorkersError`, `StrategyNoWorkerSelectedError`, `TaskCancelledError`, or `TaskRetriesExhaustedError`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The sleep between retries is routed through `@suaveplan/retry`'s `sleep(ms, signal)` so aborting the supplied signal cleanly interrupts the wait and rethrows as `TaskCancelledError`. The reason the abort handler uses `.catch(() => throw new TaskCancelledError(...))` rather than checking the rejection type is that `sleep` only ever rejects via `AbortSignal` — so any rejection is by definition a cancellation. This keeps the catch arm single-branch and 100%-coverable, and avoids a dead-code arm for non-existent non-abort rejections.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Manager supervisor helpers
|
||||
|
||||
## Purpose
|
||||
|
||||
The supervisor module owns the worker-health side of `WorkerManager`: periodic health checks across the pool, restart of `UNHEALTHY` workers up to a configurable limit, and structured failure handling. It is split into pure functions taking a `SupervisorConfig` so each unit is testable in isolation and so `manager.ts` stays under the 300-LOC budget.
|
||||
|
||||
## Features
|
||||
|
||||
- `checkAllWorkersHealth(config, signal?)` — fan-out over every worker, returns map of id → `WorkerHealth`, logs a warning when online count falls below `minOnlineWorkers`.
|
||||
- `handleWorkerFailure(worker, error, config)` — probes health after a failure; flips to `UNHEALTHY` and optionally triggers a restart.
|
||||
- `restartIfNeeded(worker, config)` — bounded restart attempt with stop+start, post-restart health check, and OTel emission.
|
||||
- `WORKER_RESTART_EXHAUSTED` emitted on the OTel counter when the limit is reached.
|
||||
- Per-worker restart counter (`config.restartCounts`) lets the orchestrator share the budget across runs.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
These helpers are internal to `WorkerManager`. A consumer wanting raw access can compose them directly:
|
||||
|
||||
```ts
|
||||
import { checkAllWorkersHealth } from "@suaveplan/queue-distribution/manager/manager-supervisor";
|
||||
|
||||
const results = await checkAllWorkersHealth({
|
||||
pool,
|
||||
healthChecker,
|
||||
autoRestartUnhealthy: true,
|
||||
maxWorkerRestarts: 3,
|
||||
minOnlineWorkers: 1,
|
||||
clock: Date.now,
|
||||
logger: console as never,
|
||||
restartCounts: new Map(),
|
||||
});
|
||||
for (const [id, health] of results) {
|
||||
console.log(`${id}: ${health.isHealthy ? "OK" : "DOWN"}`);
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Drive the supervisor manually for a control-plane component that wants to coordinate restarts itself rather than letting `WorkerManager` schedule them.
|
||||
|
||||
```ts
|
||||
import {
|
||||
handleWorkerFailure,
|
||||
restartIfNeeded,
|
||||
} from "@suaveplan/queue-distribution/manager/manager-supervisor";
|
||||
|
||||
async function escalate(worker, err): Promise<void> {
|
||||
await handleWorkerFailure(worker, err, supervisorConfig);
|
||||
if (worker.status === "UNHEALTHY") {
|
||||
await restartIfNeeded(worker, supervisorConfig);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface SupervisorConfig` — `pool`, `healthChecker`, `autoRestartUnhealthy`, `maxWorkerRestarts`, `minOnlineWorkers`, `clock`, `logger`, `restartCounts`.
|
||||
- `async function checkAllWorkersHealth(config, signal?): Promise<Map<string, WorkerHealth>>`.
|
||||
- `async function handleWorkerFailure(worker, error, config): Promise<void>`.
|
||||
- `async function restartIfNeeded(worker, config): Promise<void>`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`restartIfNeeded` swallows pre-restart `stop` errors and post-restart health-check failures by logging rather than throwing — the goal is "best-effort recovery", not "block the supervisor loop on every flake". Counter resetting on successful restart is intentional: a transient failure shouldn't permanently consume the worker's restart budget. The `WORKER_RESTART_EXHAUSTED` log is emitted with the structured error message but not thrown, again so a single saturated worker doesn't poison the whole supervisor scan.
|
||||
@@ -0,0 +1,91 @@
|
||||
# `WorkerManager`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`WorkerManager` is the public entry point for distributing tasks across a `WorkerPool`. It owns task assignment (load-balancing, retry, dead-letter), supervision (periodic health checks, auto-restart of unhealthy workers), and option lifecycle (configurable at construction and at runtime via `updateOptions`). Implementation details — the retry-with-backoff loop and the supervisor — live in `manager-dispatch.ts` and `manager-supervisor.ts` so this file stays focused on the surface contract.
|
||||
|
||||
## Features
|
||||
|
||||
- `assignTask<I, O>(task, input, signal?): Promise<O>` — pick a worker, execute, retry up to `maxTaskRetries`, record dead-letter on exhaustion.
|
||||
- `assignTasks<I, O>(tasks[], inputs[], signal?)` — fan-out wrapper over `assignTask`.
|
||||
- Auto health-check loop driven by `setInterval` over `healthCheckIntervalMs`; toggleable at runtime.
|
||||
- Auto-restart of `UNHEALTHY` workers up to `maxWorkerRestarts`.
|
||||
- `failedTasks` Set and per-task retry counter for observability.
|
||||
- Injectable `LoadBalancingStrategy` (default `RoundRobinStrategy`), `WorkerHealthChecker` (default `PeriodicHealthChecker`), `Clock`, `LoggerLike`.
|
||||
- AbortSignal threaded through every public async method.
|
||||
- Throws `InvalidOptionsError` for negative numeric options at construction and on `updateOptions`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { WorkerManager } from "@suaveplan/queue-distribution/manager/manager";
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
import { LocalWorker } from "@suaveplan/queue-distribution/worker/local/local";
|
||||
|
||||
const pool = new WorkerPool();
|
||||
pool.addWorker(new LocalWorker({}, { id: "a" }));
|
||||
const manager = new WorkerManager(pool, {
|
||||
autoHealthCheck: false,
|
||||
maxTaskRetries: 2,
|
||||
});
|
||||
|
||||
const result = await manager.assignTask(task, input);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Stand up a production-shaped manager with a least-busy strategy, performance bias, a custom health checker, and a deterministic clock for tests.
|
||||
|
||||
```ts
|
||||
import { WorkerManager } from "@suaveplan/queue-distribution/manager/manager";
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
import { LeastBusyStrategy } from "@suaveplan/queue-distribution/load-balancing/least-busy/least-busy";
|
||||
import { PeriodicHealthChecker } from "@suaveplan/queue-distribution/health-checker/health-checker";
|
||||
|
||||
const clock = (): number => Date.now();
|
||||
const pool = new WorkerPool();
|
||||
// ... add workers ...
|
||||
|
||||
const manager = new WorkerManager(
|
||||
pool,
|
||||
{
|
||||
balancer: new LeastBusyStrategy({ usePerformanceMetrics: true }),
|
||||
healthChecker: new PeriodicHealthChecker({ unhealthyThreshold: 3 }),
|
||||
autoHealthCheck: true,
|
||||
healthCheckIntervalMs: 10_000,
|
||||
maxTaskRetries: 5,
|
||||
taskRetryDelayMs: 250,
|
||||
autoRestartUnhealthy: true,
|
||||
maxWorkerRestarts: 5,
|
||||
minOnlineWorkers: 2,
|
||||
},
|
||||
{ clock, logger: console as never },
|
||||
);
|
||||
|
||||
const ctrl = new AbortController();
|
||||
setTimeout(() => ctrl.abort(), 30_000); // hard ceiling
|
||||
await manager.assignTasks(tasks, inputs, ctrl.signal);
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface WorkerManagerOptions` — `healthCheckIntervalMs?`, `autoHealthCheck?`, `taskRetryDelayMs?`, `maxTaskRetries?`, `autoRestartUnhealthy?`, `maxWorkerRestarts?`, `minOnlineWorkers?`.
|
||||
- `interface WorkerManagerDeps` — `clock?`, `logger?`.
|
||||
- `class WorkerManager`.
|
||||
- `constructor(pool, options?: { balancer?, healthChecker?, ...WorkerManagerOptions }, deps?: WorkerManagerDeps)`.
|
||||
- `assignTask<I, O>(task, input, signal?): Promise<O>`.
|
||||
- `assignTasks<I, O>(tasks[], inputs[], signal?): Promise<O[]>`.
|
||||
- `startHealthChecks(): void`, `stopHealthChecks(): void`.
|
||||
- `checkAllWorkersHealth(signal?): Promise<Map<string, WorkerHealth>>`.
|
||||
- `handleWorkerFailure(worker, error): Promise<void>`.
|
||||
- `getWorkerPool(): WorkerPool`, `getLoadBalancer(): LoadBalancingStrategy`, `setLoadBalancer(b): void`, `getHealthChecker(): WorkerHealthChecker`.
|
||||
- `getFailedTasks(): ReadonlySet<DistributedTask>`, `clearFailedTasks(): void`.
|
||||
- `getTaskRetries(taskId): number`.
|
||||
- `getOptions(): Required<WorkerManagerOptions>`, `updateOptions(partial): void`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`updateOptions` performs validate-then-apply: a negative `maxTaskRetries` is rejected before any mutation, so the manager remains in a consistent state across failed updates. Toggling `autoHealthCheck` mid-run actually starts or stops the interval — passing the same value as the current state is a no-op, which lets callers idempotently bounce settings without disrupting an active loop. The dispatch and supervisor configs are recreated on each call to `assignTask` / `checkAllWorkersHealth` so changes via `setLoadBalancer` are picked up immediately without rebuilding the manager.
|
||||
@@ -0,0 +1,62 @@
|
||||
# OTel instruments
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`observability/instruments.ts` centralises every OpenTelemetry counter, histogram, up-down counter, and tracer the package emits. All instruments are registered under the meter name `"@suaveplan/queue-distribution"` so consumers configuring an SDK can route them with a single rule. Tracer / meter handles are cached behind a `resetInstrumentCache()` escape hatch so tests can swap providers between assertions.
|
||||
|
||||
## Features
|
||||
|
||||
- Counters: `queue_dist.task.assigned_total`, `queue_dist.task.completed_total{outcome}`, `queue_dist.task.retried_total`, `queue_dist.worker.restart_total{outcome}`, `queue_dist.strategy.no_worker_total`, `queue_dist.worker.execute_total{outcome}`.
|
||||
- Histograms: `queue_dist.task.duration_ms`, `queue_dist.health_check.duration_ms`, `queue_dist.worker.restart.duration_ms`, `queue_dist.worker.executeTask.duration_ms{outcome}`.
|
||||
- Up/down counter: `queue_dist.workers.active`.
|
||||
- Tracers: `queue_dist.manager.assign`, `queue_dist.worker.executeTask`, `queue_dist.health.check`, `queue_dist.worker.restart` (all share the package-level tracer).
|
||||
- `resetInstrumentCache()` for tests that install a new provider mid-suite.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
The instruments are called from the production code path automatically. A consumer adding a `MeterProvider` and `TracerProvider` to their app sees them immediately:
|
||||
|
||||
```ts
|
||||
import { metrics, trace } from "@opentelemetry/api";
|
||||
import { MeterProvider } from "@opentelemetry/sdk-metrics";
|
||||
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
|
||||
|
||||
metrics.setGlobalMeterProvider(new MeterProvider({/* …readers… */}));
|
||||
trace.setGlobalTracerProvider(new NodeTracerProvider({/* …exporters… */}));
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
In tests, install an in-memory `MeterProvider`, run the code under test, then assert directly against the recorded metrics. Reset the cache between cases so the new provider is picked up.
|
||||
|
||||
```ts
|
||||
import { resetInstrumentCache } from "@suaveplan/queue-distribution/observability/instruments";
|
||||
import { metrics } from "@opentelemetry/api";
|
||||
import {
|
||||
InMemoryMetricExporter,
|
||||
MeterProvider,
|
||||
PeriodicExportingMetricReader,
|
||||
AggregationTemporality,
|
||||
} from "@opentelemetry/sdk-metrics";
|
||||
|
||||
const exporter = new InMemoryMetricExporter(AggregationTemporality.DELTA);
|
||||
const reader = new PeriodicExportingMetricReader({ exporter, exportIntervalMillis: 100 });
|
||||
metrics.setGlobalMeterProvider(new MeterProvider({ readers: [reader] }));
|
||||
resetInstrumentCache();
|
||||
// Run the system under test, then await reader.forceFlush() and inspect exporter._metrics.
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `function resetInstrumentCache(): void`.
|
||||
- Counters: `recordTaskAssigned`, `recordTaskCompleted("success"|"failure")`, `recordTaskRetried`, `recordWorkerRestart("success"|"failure"|"exhausted")`, `recordStrategyNoWorker`, `recordExecuteTaskOutcome("success"|"failure")`.
|
||||
- Histograms: `recordTaskDuration(ms)`, `recordHealthCheckDuration(ms)`, `recordWorkerRestartDuration(ms)`, `recordExecuteTaskDuration(ms, "success"|"failure")`.
|
||||
- Up/down counter: `recordWorkerActiveDelta(delta)`.
|
||||
- Tracers: `getManagerAssignTracer`, `getWorkerExecuteTaskTracer`, `getHealthCheckTracer`, `getWorkerRestartTracer`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The instruments are created lazily — the first call to `getMeter()` resolves the global meter and caches the handle, so the per-call overhead in the hot path is a single map lookup. All instruments use a single meter (not one per file) because the OTel API resolves names case-sensitively and a typo in a file-local meter name would create a parallel meter that consumers wouldn't see in their dashboards. `resetInstrumentCache()` exists exclusively for tests; production code never needs to call it because the underlying global provider is set once at startup.
|
||||
@@ -0,0 +1,76 @@
|
||||
# `WorkerPool`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`WorkerPool` is an in-memory registry of `Worker` instances. It owns identity bookkeeping — `addWorker`, `removeWorker`, `getWorkerById`, search via `findWorkers(criteria)` — plus bulk lifecycle (`startAllWorkers`, `stopAllWorkers`). It deliberately does not own task assignment or health checking; those live in `WorkerManager` and `PeriodicHealthChecker` respectively, so the pool stays a focused container.
|
||||
|
||||
## Features
|
||||
|
||||
- Duplicate-id rejection via `WorkerDuplicateError` (`QUEUE_DIST.WORKER_DUPLICATE`).
|
||||
- `requireWorker(id)` throws `WorkerNotFoundError` for callers that prefer an exception over a `null` return.
|
||||
- `findWorkers(criteria)` predicate matrix: `status`, `minCapacity` / `maxCapacity`, `minWeight` / `maxWeight`, `type`, and a custom `matcher` function.
|
||||
- Bulk lifecycle: `startAllWorkers(signal?)`, `stopAllWorkers(force?, signal?)`.
|
||||
- `getAvailableWorkers()` shortcut returning only workers whose `canAcceptTask()` is `true`.
|
||||
- Logs every add/remove/clear at `debug` and bulk lifecycle at `info` via the injected `LoggerLike`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
import { LocalWorker } from "@suaveplan/queue-distribution/worker/local/local";
|
||||
|
||||
const pool = new WorkerPool();
|
||||
pool.addWorker(new LocalWorker({}, { id: "a", capacity: 2 }));
|
||||
pool.addWorker(new LocalWorker({}, { id: "b", capacity: 4 }));
|
||||
|
||||
await pool.startAllWorkers();
|
||||
const available = pool.getAvailableWorkers();
|
||||
console.log(`${available.length} worker(s) ready`);
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Use `findWorkers` with the custom matcher to apply policy beyond the built-in predicates (e.g. drain workers whose health is stale).
|
||||
|
||||
```ts
|
||||
import { WorkerPool } from "@suaveplan/queue-distribution/pool/pool";
|
||||
import { WorkerStatus } from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
const stale = pool.findWorkers({
|
||||
status: WorkerStatus.ONLINE,
|
||||
minCapacity: 4,
|
||||
type: "local",
|
||||
matcher: (worker) => {
|
||||
const lastCheck = worker.getInfo().lastHealthCheck?.getTime() ?? 0;
|
||||
return Date.now() - lastCheck > 60_000;
|
||||
},
|
||||
});
|
||||
|
||||
for (const worker of stale) {
|
||||
await worker.stop(true);
|
||||
pool.removeWorker(worker.id);
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class WorkerPool`.
|
||||
- `constructor(deps?: { logger?: LoggerLike })`.
|
||||
- `addWorker(worker): void` — throws `WorkerDuplicateError`.
|
||||
- `removeWorker(id): boolean` — `true` on hit.
|
||||
- `requireWorker(id): Worker` — throws `WorkerNotFoundError` on miss.
|
||||
- `getWorkerById(id): Worker | null`.
|
||||
- `getAllWorkers(): Worker[]`, `size(): number`.
|
||||
- `getWorkersByStatus(status): Worker[]`, `getAvailableWorkers(): Worker[]`.
|
||||
- `getWorkersInfo(): WorkerInfo[]`.
|
||||
- `findWorkers(criteria: WorkerSearchCriteria): Worker[]`.
|
||||
- `async startAllWorkers(signal?): Promise<void>`, `async stopAllWorkers(force?, signal?): Promise<void>`.
|
||||
- `clear(): void`.
|
||||
- `interface WorkerSearchCriteria` — `status?`, `minCapacity?`, `maxCapacity?`, `minWeight?`, `maxWeight?`, `type?`, `matcher?`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`getAllWorkers()` returns a fresh array each call rather than exposing the internal `Map` — callers can therefore mutate the returned array without affecting pool state. `findWorkers` short-circuits on the first failing predicate so callers can stack many criteria without paying their full cost when an earlier one rules a worker out. The pool intentionally takes no `AbortSignal` on `clear()` — clearing the registry is a synchronous bookkeeping operation, distinct from `stopAllWorkers` which is the right API for graceful drain.
|
||||
@@ -0,0 +1,80 @@
|
||||
# `DistributedTask` contract
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`DistributedTask<I, O>` is the unit of work the rest of `@suaveplan/queue-distribution` executes. It is deliberately decoupled from `@suaveplan/queue`'s `Job` / state-machine model so the distribution layer can serve queue, batch, RPC, or agent-dispatch consumers via thin adapters. The contract is generic over an input `I` and an output `O` so a caller's static types flow end-to-end through `Worker.executeTask`.
|
||||
|
||||
## Features
|
||||
|
||||
- `id: string` — caller-assigned identity used in logs, metrics, retry tracking.
|
||||
- `execute(input, context): Promise<O>` — the work itself. `context` is a caller-defined `ExecutionContext`.
|
||||
- `TaskStatus` enum (`PENDING | RUNNING | COMPLETED | FAILED | CANCELLED | TIMED_OUT`) + `TaskStatusSchema` for Zod validation.
|
||||
- `TaskMetadata` shape (`createdAt`, optional `startedAt` / `completedAt`, `attempts`, optional `tags`, indexed extras) + `TaskMetadataSchema`.
|
||||
- `setStatus` / `getStatus` / `getMetadata` for lifecycle observation by the worker.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
type DistributedTask,
|
||||
TaskStatus,
|
||||
type TaskStatusValue,
|
||||
} from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
function makeTask(): DistributedTask<number, string> {
|
||||
let status: TaskStatusValue = TaskStatus.PENDING;
|
||||
return {
|
||||
id: "double-then-stringify",
|
||||
execute: async (n) => String(n * 2),
|
||||
getStatus: () => status,
|
||||
setStatus: (s) => {
|
||||
status = s;
|
||||
},
|
||||
getMetadata: () => ({ createdAt: new Date(), attempts: 0 }),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
`DistributedTask` is the seam where consumers bridge their own work model. For a `@suaveplan/queue` consumer, wrap a `Job` so each retry advances `attempts` and propagates failure into the queue's state machine. For RPC, encode the remote-call envelope in `execute`.
|
||||
|
||||
```ts
|
||||
import type { DistributedTask } from "@suaveplan/queue-distribution/task/task";
|
||||
import { TaskMetadataSchema } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
class RpcTask<I, O> implements DistributedTask<I, O> {
|
||||
constructor(readonly id: string, private fetcher: (i: I) => Promise<O>) {}
|
||||
#status = "PENDING" as const;
|
||||
#metadata = { createdAt: new Date(), attempts: 0 };
|
||||
async execute(input: I): Promise<O> {
|
||||
this.#metadata.attempts += 1;
|
||||
return this.fetcher(input);
|
||||
}
|
||||
getStatus = () => this.#status;
|
||||
setStatus = (s: typeof this.#status): void => {
|
||||
this.#status = s;
|
||||
};
|
||||
getMetadata = () => {
|
||||
const parsed = TaskMetadataSchema.parse(this.#metadata);
|
||||
return parsed;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface DistributedTask<I = unknown, O = unknown>` — fields above.
|
||||
- `interface ExecutionContext` — opaque marker; callers define the concrete shape.
|
||||
- `const TaskStatus` — frozen literal map.
|
||||
- `type TaskStatusValue` — union of `TaskStatus` values.
|
||||
- `const TaskStatusSchema: z.ZodType<TaskStatusValue>` — Zod enum.
|
||||
- `interface TaskMetadata` — `createdAt: Date`, `startedAt?: Date`, `completedAt?: Date`, `attempts: number`, `tags?: readonly string[]`, indexed extras.
|
||||
- `const TaskMetadataSchema: z.ZodType<TaskMetadata>` — Zod object with `passthrough` for extras.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`TaskMetadataSchema` uses `passthrough` because the indexed-extras signature on `TaskMetadata` is intentional: callers can carry arbitrary diagnostic context (queue name, trace id, retry deadline) without ratcheting the schema each time. The schema enforces only the typed fields. `ExecutionContext` is an empty interface — a marker rather than a contract — so callers can pass any object (request scope, tenant id, AbortSignal, dependency container). The distribution layer never inspects the context.
|
||||
@@ -0,0 +1,76 @@
|
||||
# `WorkerMetricsState` helpers
|
||||
|
||||
## Purpose
|
||||
|
||||
`abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/failure counts and average response time. Splitting them out of `abstract.ts` keeps each file under the 300-LOC budget, makes the bookkeeping unit-testable in isolation, and lets a subclass reuse the same helpers if it overrides `getHealth` with additional fields layered on top of the base payload.
|
||||
|
||||
## Features
|
||||
|
||||
- `createMetricsState()` — fresh `WorkerMetricsState` (`tasksSucceeded`, `tasksFailed`, `totalResponseTime`, `totalTasksCompleted`).
|
||||
- `recordSuccess(state, responseTimeMs)` — increment successes, accumulate response time.
|
||||
- `recordFailure(state)` — increment failures only (response time is undefined for failures).
|
||||
- `buildDefaultHealth(state, { activeTasks, isHealthy })` — synthesise the default `WorkerHealth` payload using the accumulator's totals.
|
||||
- CPU / memory in the synthesised payload scale linearly with `activeTasks` and cap at 100 — purely synthetic, intended as a placeholder until subclasses inject real measurements.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
createMetricsState,
|
||||
recordSuccess,
|
||||
recordFailure,
|
||||
buildDefaultHealth,
|
||||
} from "@suaveplan/queue-distribution/worker/abstract/abstract-metrics";
|
||||
|
||||
const state = createMetricsState();
|
||||
recordSuccess(state, 120);
|
||||
recordSuccess(state, 80);
|
||||
recordFailure(state);
|
||||
|
||||
const health = buildDefaultHealth(state, { activeTasks: 1, isHealthy: true });
|
||||
// health.metrics.responseTime === 100
|
||||
// health.metrics.tasksSucceeded === 2
|
||||
// health.metrics.tasksFailed === 1
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
A subclass can compose its own metrics on top of the default payload — for example, a remote worker that wants to pass through CPU / memory from the upstream server while keeping the synthesised counters.
|
||||
|
||||
```ts
|
||||
import { AbstractWorker } from "@suaveplan/queue-distribution/worker/abstract/abstract";
|
||||
import { buildDefaultHealth } from "@suaveplan/queue-distribution/worker/abstract/abstract-metrics";
|
||||
import type { WorkerHealth } from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
class RemoteShim extends AbstractWorker {
|
||||
override async getHealth(): Promise<WorkerHealth> {
|
||||
const base = buildDefaultHealth(
|
||||
(this as unknown as { metrics: ReturnType<typeof createMetricsState> }).metrics,
|
||||
{ activeTasks: this.activeTasks, isHealthy: this.status !== "OFFLINE" },
|
||||
);
|
||||
const upstream = await fetchUpstreamHealth();
|
||||
return {
|
||||
isHealthy: base.isHealthy && upstream.ok,
|
||||
metrics: { ...base.metrics, cpu: upstream.cpu, memory: upstream.memory },
|
||||
};
|
||||
}
|
||||
protected async doExecuteTask<I, O>(t, i): Promise<O> {
|
||||
return undefined as never;
|
||||
}
|
||||
protected getType(): string {
|
||||
return "shim";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface WorkerMetricsState` — `tasksSucceeded`, `tasksFailed`, `totalResponseTime`, `totalTasksCompleted` (all numbers).
|
||||
- `function createMetricsState(): WorkerMetricsState`.
|
||||
- `function recordSuccess(state, responseTimeMs): void`.
|
||||
- `function recordFailure(state): void`.
|
||||
- `function buildDefaultHealth(state, { activeTasks, isHealthy }): WorkerHealth`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The CPU / memory placeholders (`activeTasks * 25`, `activeTasks * 15`, capped at 100) are explicitly synthetic — the upstream library returned the same shape and downstream consumers may have come to expect a non-zero CPU value when the worker is doing anything. Subclasses that have access to real OS-level metrics should override `getHealth()` end-to-end rather than calling `buildDefaultHealth` first; the helper exists so subclasses can layer on top without losing the success/failure counters.
|
||||
@@ -0,0 +1,95 @@
|
||||
# `AbstractWorker`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`AbstractWorker` is the base class every concrete worker (in-process, remote, worker-thread) extends. It owns lifecycle bookkeeping (`start` / `stop`), metric accumulation (success / failure counts, response-time totals), OTel span emission per task, status transitions between `ONLINE` and `BUSY`, and an injectable `Clock` / `IdGenerator` / `LoggerLike` so subclasses focus exclusively on transport-specific behaviour. Splitting the bookkeeping out of every transport means `LocalWorker`, `RemoteWorker`, and `WorkerThreadWorker` need only override `doExecuteTask`, `doStart`, and `doStop`.
|
||||
|
||||
## Features
|
||||
|
||||
- Idempotent `start()` (no-op when not `OFFLINE`).
|
||||
- `stop(force)` rejects when `activeTasks > 0 && !force` with `WORKER_BUSY_STOP`; force-stop drains active count.
|
||||
- `executeTask` increments `activeTasks`, transitions status to `BUSY` when going 0→1, records duration via injected `Clock`, emits OTel histograms/counters/spans, decrements on completion, restores `ONLINE` when count returns to zero.
|
||||
- `markUnhealthy(reason)` flips status to `UNHEALTHY` and warns through the logger.
|
||||
- `getHealth()` synthesises a default payload from accumulated metrics; subclasses override for real probes.
|
||||
- `getInfo()` builds a structural `WorkerInfo` snapshot for pool introspection.
|
||||
- Injectable `Clock`, `IdGenerator`, `LoggerLike`; defaults are `Date.now`, `crypto.randomUUID`, and `NOOP_LOGGER`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { AbstractWorker } from "@suaveplan/queue-distribution/worker/abstract/abstract";
|
||||
import type { DistributedTask } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
class HelloWorker extends AbstractWorker {
|
||||
constructor() {
|
||||
super({ capacity: 4, weight: 1 });
|
||||
}
|
||||
protected async doExecuteTask<I, O>(
|
||||
task: DistributedTask<I, O>,
|
||||
input: I,
|
||||
): Promise<O> {
|
||||
return task.execute(input, {});
|
||||
}
|
||||
protected getType(): string {
|
||||
return "hello";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Inject a deterministic `Clock` and `IdGenerator` for property-based tests. Override `doStart` / `doStop` for any transport that needs connection setup or graceful teardown.
|
||||
|
||||
```ts
|
||||
import { AbstractWorker } from "@suaveplan/queue-distribution/worker/abstract/abstract";
|
||||
import type { DistributedTask } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
class ConnectedWorker extends AbstractWorker {
|
||||
constructor(private readonly endpoint: string) {
|
||||
super(
|
||||
{ capacity: 2, weight: 1 },
|
||||
{
|
||||
clock: () => 1_700_000_000_000,
|
||||
idGenerator: () => `worker-${endpoint}`,
|
||||
logger: console as never,
|
||||
},
|
||||
);
|
||||
}
|
||||
protected async doStart(): Promise<void> {
|
||||
// open connection here
|
||||
}
|
||||
protected async doStop(force: boolean): Promise<void> {
|
||||
// tear connection down here; honour `force`
|
||||
}
|
||||
protected async doExecuteTask<I, O>(
|
||||
task: DistributedTask<I, O>,
|
||||
input: I,
|
||||
): Promise<O> {
|
||||
return task.execute(input, { endpoint: this.endpoint });
|
||||
}
|
||||
protected getType(): string {
|
||||
return "connected";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `abstract class AbstractWorker implements Worker`.
|
||||
- `constructor(options?: AbstractWorkerOptions, deps?: AbstractWorkerDeps)`.
|
||||
- `AbstractWorkerOptions = { id?, capacity?, weight? }`.
|
||||
- `AbstractWorkerDeps = { clock?, idGenerator?, logger? }`.
|
||||
- `executeTask<I, O>(task, input, signal?): Promise<O>`.
|
||||
- `getHealth(signal?): Promise<WorkerHealth>` — synthesised default; subclasses may override.
|
||||
- `start(signal?): Promise<void>`, `stop(force?, signal?): Promise<void>`.
|
||||
- `canAcceptTask(): boolean`, `getInfo(): WorkerInfo`.
|
||||
- `markUnhealthy(reason): void`.
|
||||
- Abstract: `protected doExecuteTask<I, O>(task, input): Promise<O>`, `protected getType(): string`.
|
||||
- Hookable: `protected doStart(signal?)`, `protected doStop(force, signal?)`, `protected getCapabilities()`, `protected getName()`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The hot path inside `#runWithBookkeeping` always restores `activeTasks` via `finally`, even on synchronous throws inside `doExecuteTask`, so the worker can never leak a phantom active-task slot. The OTel emission uses a single tracer reference cached in `observability/instruments.ts` so emitter setup doesn't allocate per task. Defaulting `idGenerator` to `crypto.randomUUID` removes the `uuid` runtime dependency the upstream library carried; consumers needing a different format inject their own generator.
|
||||
@@ -0,0 +1,79 @@
|
||||
# `LocalWorker`
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
## Purpose
|
||||
|
||||
`LocalWorker<Ctx>` is an in-process executor — it runs `DistributedTask.execute` directly in the calling JavaScript event loop, with no IO indirection. Use it when the work can run on the same node as the dispatcher: tiny CPU jobs, fan-out across a thread, or local-only test setups. The constructor is parameterised by an `ExecutionContext` of type `Ctx`, which the worker passes verbatim into every `task.execute(input, ctx)` call.
|
||||
|
||||
## Features
|
||||
|
||||
- Inherits lifecycle, metrics, OTel emission, and FSM from `AbstractWorker`.
|
||||
- Generic over caller-defined `ExecutionContext`.
|
||||
- `getCapabilities()` advertises `{ hasFileSystemAccess: true, isRemote: false, contextType }` so strategies can route work accordingly.
|
||||
- `getType()` returns `"local"`.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import { LocalWorker } from "@suaveplan/queue-distribution/worker/local/local";
|
||||
import type { DistributedTask, ExecutionContext } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
class EmptyContext implements ExecutionContext {}
|
||||
|
||||
const worker = new LocalWorker(new EmptyContext(), { capacity: 4 });
|
||||
await worker.start();
|
||||
|
||||
const task: DistributedTask<number, string> = {
|
||||
id: "format-number",
|
||||
execute: async (n) => `n=${n}`,
|
||||
getStatus: () => "PENDING",
|
||||
setStatus: () => {},
|
||||
getMetadata: () => ({ createdAt: new Date(), attempts: 0 }),
|
||||
};
|
||||
|
||||
const result = await worker.executeTask(task, 7); // "n=7"
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Thread your dependency container through the `ExecutionContext` so every task gets access to the same logger, tracer, and DB client.
|
||||
|
||||
```ts
|
||||
import { LocalWorker } from "@suaveplan/queue-distribution/worker/local/local";
|
||||
import type { DistributedTask, ExecutionContext } from "@suaveplan/queue-distribution/task/task";
|
||||
|
||||
interface AppContext extends ExecutionContext {
|
||||
logger: { info: (m: string) => void };
|
||||
db: { query: (sql: string) => Promise<unknown> };
|
||||
}
|
||||
|
||||
const ctx: AppContext = {
|
||||
logger: { info: console.log },
|
||||
db: { query: async () => undefined },
|
||||
};
|
||||
|
||||
const worker = new LocalWorker<AppContext>(ctx, { capacity: 8 });
|
||||
await worker.start();
|
||||
|
||||
const task: DistributedTask<string, unknown> = {
|
||||
id: "lookup",
|
||||
execute: async (sql, c) => (c as AppContext).db.query(sql),
|
||||
getStatus: () => "PENDING",
|
||||
setStatus: () => {},
|
||||
getMetadata: () => ({ createdAt: new Date(), attempts: 0 }),
|
||||
};
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `class LocalWorker<Ctx extends ExecutionContext = ExecutionContext> extends AbstractWorker`.
|
||||
- `constructor(context: Ctx, options?: AbstractWorkerOptions, deps?: AbstractWorkerDeps)`.
|
||||
- `protected doExecuteTask<I, O>(task, input): Promise<O>` — invokes `task.execute(input, this.context)`.
|
||||
- `protected getType(): string` — returns `"local"`.
|
||||
- `protected getCapabilities(): Readonly<Record<string, unknown>>` — `{ hasFileSystemAccess: true, isRemote: false, contextType: <ctor name> }`.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
`getCapabilities` reflects the context constructor name (`Object` when the context is `Object.create(null)`) so logs and dashboards can distinguish similarly-typed workers running with different contexts (e.g. tenant-specific). The `hasFileSystemAccess: true` flag is a best-effort signal — strategies that route IO-heavy work to file-system-aware workers can filter on it. Remote / worker-thread plugins explicitly set `isRemote: true` or `isThreaded: true` so the negation is meaningful in dashboards.
|
||||
@@ -0,0 +1,82 @@
|
||||
# `Worker` contract
|
||||
|
||||
<!-- begin:auto-docs -->
|
||||
_(no submodules)_
|
||||
<!-- end:auto-docs -->
|
||||
- [abstract-metrics](./abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ - [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract-metrics.ts` carries the…
|
||||
- [abstract-metrics](./abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./worker.md) — <!-- begin:auto-docs --> _(no submodules)_- [abstract-metrics](./abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract…
|
||||
- [abstract-metrics](./abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./worker.md) — <!-- begin:auto-docs --> _(no submodules)_- abstract-metrics — `abstract-metrics.ts` carries th…
|
||||
- [abstract-metrics](./abstract/abstract-metrics.md) — `abstract-metrics.ts` carries the pure functions and the small accumulator used by `AbstractWorker` to track success/fa…
|
||||
- [abstract](./abstract/abstract.md) — <!-- begin:auto-docs --> _(no submodules)_- [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_ - [local](./local/local.md) — <!-- begin:auto-docs --> _(no submodules)_- [worker](./worker.md) — <!-- begin:auto-docs --> _(no submodules)_
|
||||
## Purpose
|
||||
|
||||
`Worker` is the structural interface every distribution-tier executor implements. It describes lifecycle (`start`, `stop`), task execution (`executeTask`), health probing (`getHealth`, `canAcceptTask`), and introspection (`getInfo`). Concrete workers (`LocalWorker`, `RemoteWorker` from the HTTP plugin, `WorkerThreadWorker` from the worker-thread plugin) extend `AbstractWorker`, but downstream code should program against `Worker` so transport choices stay swappable.
|
||||
|
||||
## Features
|
||||
|
||||
- `id`, `capacity`, `weight`, `status: WorkerStatusValue`, `activeTasks` fields kept in sync by the worker base class.
|
||||
- `executeTask<I, O>(task, input, signal?): Promise<O>` — preserves caller's `<I, O>` generics end-to-end.
|
||||
- `getHealth(signal?): Promise<WorkerHealth>` — pluggable health probe; the abstract base returns synthesised metrics, subclasses override with real measurements.
|
||||
- `start(signal?)`, `stop(force?, signal?)` — idempotent lifecycle.
|
||||
- `canAcceptTask(): boolean` — used by load-balancing strategies to filter the candidate pool.
|
||||
- `getInfo(): WorkerInfo` — snapshot of identity, type, capacity, weight, status, active count, optional `startedAt` / `lastHealthCheck` / `lastHealthStatus` / `capabilities`.
|
||||
- `WorkerStatusSchema`, `WorkerHealthSchema`, `WorkerInfoSchema` — Zod schemas at every public boundary.
|
||||
|
||||
## Usage: Basic Example
|
||||
|
||||
```ts
|
||||
import {
|
||||
type Worker,
|
||||
WorkerStatus,
|
||||
WorkerHealthSchema,
|
||||
} from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
function printWorker(worker: Worker): void {
|
||||
const info = worker.getInfo();
|
||||
console.log(`${info.type}/${info.id} — ${info.status}, ${info.activeTasks}/${info.capacity}`);
|
||||
}
|
||||
|
||||
async function probeHealth(worker: Worker): Promise<boolean> {
|
||||
const health = await worker.getHealth();
|
||||
WorkerHealthSchema.parse(health); // validate at the boundary
|
||||
return health.isHealthy && worker.status !== WorkerStatus.UNHEALTHY;
|
||||
}
|
||||
```
|
||||
|
||||
## Usage: Advanced Example
|
||||
|
||||
Workers can carry arbitrary capabilities — exposed via `getInfo().capabilities` — that strategies and supervisors inspect to make routing decisions.
|
||||
|
||||
```ts
|
||||
import {
|
||||
type Worker,
|
||||
WorkerInfoSchema,
|
||||
} from "@suaveplan/queue-distribution/worker/worker";
|
||||
|
||||
function findGpuWorkers(workers: readonly Worker[]): Worker[] {
|
||||
return workers.filter((worker) => {
|
||||
const info = WorkerInfoSchema.parse(worker.getInfo());
|
||||
return info.capabilities?.gpu === true && info.status === "ONLINE";
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
- `interface Worker` — fields and methods above.
|
||||
- `const WorkerStatus` — frozen literal map (`ONLINE` / `BUSY` / `OFFLINE` / `UNHEALTHY`).
|
||||
- `type WorkerStatusValue` — union of the four states.
|
||||
- `const WorkerStatusSchema: z.ZodEnum` — accepts only the four states.
|
||||
- `interface WorkerHealth` — `isHealthy: boolean` + `WorkerHealthMetrics`.
|
||||
- `interface WorkerHealthMetrics` — `cpu`, `memory`, `tasksFailed`, `tasksSucceeded`, `responseTime`, indexed extras.
|
||||
- `const WorkerHealthSchema: z.ZodType<WorkerHealth>`.
|
||||
- `interface WorkerInfo` — identity + capacity + weight + status + active count + optional timestamps + optional health snapshot + optional capabilities map.
|
||||
- `const WorkerInfoSchema: z.ZodType<WorkerInfo>` — `capacity` must be a positive integer; `weight` non-negative; `activeTasks` non-negative integer.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
The interface mutates the worker's `status` and `activeTasks` fields directly rather than threading them through accessors because the `AbstractWorker` base maintains them as the source of truth, and load-balancing strategies access them in hot paths where allocator pressure matters. `getInfo()` returns a fresh structural snapshot every call — callers must not mutate it. Schemas exist so transport-specific subclasses (e.g. a deserialised remote worker payload) can validate the shape they construct before publishing it to the pool.
|
||||
Reference in New Issue
Block a user