Files
skills/docs/foundation/flow/queue-distribution-http/src/worker/remote/remote.md
2026-05-29 08:20:56 +01:00

4.2 KiB

RemoteWorker

(no submodules)

Purpose

RemoteWorker extends AbstractWorker and delegates execution to a remote endpoint via an injected WorkerCommunicator. It manages the connect / reconnect lifecycle (with @suaveplan/retry-style exponential backoff routed through sleep), runs a periodic health-check loop, and reacts to communication failures by pinging and either reporting the worker still-reachable or marking it UNHEALTHY. The connect loop is bounded — no recursion — and every retry attempt is captured in the OTel queue_dist_http.connect.attempts_total counter.

Features

  • start() calls #connect (exponential backoff between attempts) then starts a setInterval health-check loop.
  • stop(force, signal?) halts the loop, disconnects the communicator, and flips #isConnected to false.
  • doExecuteTask refuses when not connected (HTTP.CONNECT_FAILED).
  • Communication-error recovery: ping with a 5-second timeout; reachable → log + propagate; unreachable → markUnhealthy("Unreachable") + #isConnected = false.
  • getHealth() proxies to the remote and refreshes lastHealthCheck / lastHealthStatus; on failure returns a fallback unhealthy payload.
  • Configurable healthCheckIntervalMs, maxConnectionAttempts, initialBackoffMs, maxBackoffMs.
  • getCapabilities() advertises { isRemote: true, protocol, endpoint } so strategies can route work.

Usage: Basic Example

import { HttpWorkerCommunicator } from "@suaveplan/queue-distribution-http";
import { RemoteWorker } from "@suaveplan/queue-distribution-http/worker/remote/remote";

const communicator = new HttpWorkerCommunicator();
const worker = new RemoteWorker("https://worker.example", communicator, {
    healthCheckIntervalMs: 15_000,
    maxConnectionAttempts: 5,
});

await worker.start();
const result = await worker.executeTask(task, input);
await worker.stop();

Usage: Advanced Example

For deterministic property-tests, inject a custom communicator and watch connect retries with a fake clock.

import { RemoteWorker } from "@suaveplan/queue-distribution-http/worker/remote/remote";
import type { WorkerCommunicator } from "@suaveplan/queue-distribution-http";

let connectAttempts = 0;
const flakyCommunicator: WorkerCommunicator = {
    sendTask: async (_e, _t, input) => input as never,
    connect: async () => {
        connectAttempts += 1;
        if (connectAttempts < 3) throw new Error("refused");
    },
    checkStatus: async () => "ONLINE",
    checkHealth: async () => ({
        isHealthy: true,
        metrics: { cpu: 0, memory: 0, tasksFailed: 0, tasksSucceeded: 0, responseTime: 0 },
    }),
    disconnect: async () => undefined,
    ping: async () => 5,
    getProtocol: () => "http",
};

const worker = new RemoteWorker("https://worker", flakyCommunicator, {
    maxConnectionAttempts: 5,
    initialBackoffMs: 0,
    healthCheckIntervalMs: 60_000,
});
await worker.start();
console.log(connectAttempts); // 3

API Reference

  • class RemoteWorker extends AbstractWorker.
  • constructor(endpoint, communicator, options?: RemoteWorkerOptions, deps?: AbstractWorkerDeps).
  • interface RemoteWorkerOptions extends AbstractWorkerOptions — healthCheckIntervalMs?, maxConnectionAttempts?, initialBackoffMs?, maxBackoffMs?.
  • getEndpoint(): string, getCommunicator(): WorkerCommunicator, isConnected(): boolean.
  • getHealth(signal?): Promise<WorkerHealth> — overrides the abstract default.
  • Protected: doExecuteTask, doStart, doStop, getType(), getCapabilities().

Implementation Notes

The connect loop iterates up to maxConnectionAttempts times, doubling the delay each iteration (capped at maxBackoffMs). On every retry, the loop also re-checks the remote status — so a worker that responds OK on the third attempt connects without further delay. Communication errors during sendTask are NOT silently swallowed: the worker rethrows so the calling WorkerManager can retry against a different worker via its load-balancing strategy. The ping-based reachability probe runs in parallel with the rethrow so the worker's status is up-to-date by the time the next strategy invocation observes the pool.