Initial Commit

docs/superpowers/plans/2026-04-24-docker-infrastructure.md

@@ -0,0 +1,542 @@
+# Implementation Plan: Docker Compose Infrastructure
+
+**Goal:** Create the `docker/` workspace at the repo root with three Docker Compose files, a Caddyfile, `.env.example`, a `package.json`, and a `README.md`. This is pure Docker/YAML infrastructure — no TypeScript.
+
+**Architecture:** Three compose stacks share the same six services (SurrealDB, Redis, PostgreSQL, Mailpit, Typesense, Garage S3) but serve different purposes: `dev` runs persistent volumes for local development; `test` runs ephemeral containers for CI integration tests; `testbed` extends the services with the showcase application and a Caddy reverse proxy for staging.
+
+**Tech Stack:** Docker Compose v2, Caddy 2
+
+> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan.
+
+---
+
+## Chunk 1: Package setup + dev stack
+
+- [ ] **Create the `docker/` directory at repo root**
+
+  Path: `/mnt/s/web/genesis/docker/`
+
+- [ ] **Create `docker/package.json`**
+
+  Exact content:
+
+  ```json
+  {
+    "name": "@suaveplan/docker",
+    "private": true,
+    "scripts": {
+      "dev:up":       "docker compose -f docker-compose.dev.yml up -d",
+      "dev:down":     "docker compose -f docker-compose.dev.yml down",
+      "test:up":      "docker compose -f docker-compose.test.yml up -d",
+      "test:down":    "docker compose -f docker-compose.test.yml down",
+      "testbed:up":   "docker compose -f docker-compose.testbed.yml up --build -d",
+      "testbed:down": "docker compose -f docker-compose.testbed.yml down"
+    }
+  }
+  ```
+
+- [ ] **Create `docker/docker-compose.dev.yml`**
+
+  Full content — all six services with named volumes and healthchecks on SurrealDB and PostgreSQL:
+
+  ```yaml
+  services:
+    surrealdb:
+      image: surrealdb/surrealdb:v3.0.4
+      command: start --auth --user root --pass root
+      ports:
+        - "8001:8000"
+      volumes:
+        - surrealdb-data:/data
+      healthcheck:
+        test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+        interval: 5s
+        timeout: 3s
+        retries: 5
+
+    redis:
+      image: redis:7-alpine
+      ports:
+        - "8002:6379"
+      volumes:
+        - redis-data:/data
+
+    postgres:
+      image: postgres:16-alpine
+      environment:
+        POSTGRES_USER: postgres
+        POSTGRES_PASSWORD: postgres
+        POSTGRES_DB: postgres
+      ports:
+        - "8003:5432"
+      volumes:
+        - postgres-data:/var/lib/postgresql/data
+      healthcheck:
+        test: ["CMD", "pg_isready", "-U", "postgres"]
+        interval: 5s
+        timeout: 3s
+        retries: 5
+
+    mailpit:
+      image: axllent/mailpit:latest
+      ports:
+        - "8004:1025"
+        - "8005:8025"
+
+    typesense:
+      image: typesense/typesense:27.1
+      environment:
+        TYPESENSE_DATA_DIR: /data
+        TYPESENSE_API_KEY: test
+      ports:
+        - "8006:8108"
+      volumes:
+        - typesense-data:/data
+
+    garage-s3:
+      image: ghcr.io/deuxfleurs/garage:v2.0.0
+      ports:
+        - "8900:3900"
+        - "8902:3902"
+      volumes:
+        - garage-data:/data
+      environment:
+        GARAGE_ALLOW_WORLD_READABLE_CONFIG: "true"
+
+  volumes:
+    surrealdb-data:
+    redis-data:
+    postgres-data:
+    typesense-data:
+    garage-data:
+  ```
+
+- [ ] **Create `docker/.env.example`**
+
+  Exact content:
+
+  ```bash
+  # Set these to use your already-running local services instead of spinning containers.
+  # A fixture only enters passthrough mode when ALL its required vars are set and non-empty.
+
+  SURREALDB_URL=ws://localhost:8001
+  SURREALDB_USERNAME=root
+  SURREALDB_PASSWORD=root
+
+  REDIS_URL=redis://localhost:8002
+
+  POSTGRES_URL=postgresql://postgres:postgres@localhost:8003/postgres
+
+  MAILPIT_SMTP_URL=smtp://localhost:8004
+  MAILPIT_API_URL=http://localhost:8005
+
+  TYPESENSE_URL=http://localhost:8006
+  TYPESENSE_API_KEY=test
+
+  GARAGE_S3_ENDPOINT=http://localhost:8900
+  GARAGE_S3_ADMIN_ENDPOINT=http://localhost:8902
+  GARAGE_S3_ACCESS_KEY_ID=
+  GARAGE_S3_SECRET_ACCESS_KEY=
+  ```
+
+- [ ] **Verify dev compose YAML is valid**
+
+  ```bash
+  docker compose -f /mnt/s/web/genesis/docker/docker-compose.dev.yml config
+  ```
+
+  Expected: compose prints the resolved YAML with no errors.
+
+- [ ] **Smoke-test the dev stack starts cleanly**
+
+  ```bash
+  cd /mnt/s/web/genesis/docker && \
+    docker compose -f docker-compose.dev.yml up -d && \
+    docker compose -f docker-compose.dev.yml ps
+  ```
+
+  Expected: all six services listed with `Status` = `running` (or `healthy` once healthchecks pass). Tear down immediately after verification:
+
+  ```bash
+  docker compose -f /mnt/s/web/genesis/docker/docker-compose.dev.yml down
+  ```
+
+---
+
+## Chunk 2: Test stack + testbed stack + docs
+
+- [ ] **Create `docker/docker-compose.test.yml`**
+
+  Identical structure to `dev.yml` with two mechanical differences:
+  1. Port range substituted to `7004–7011`.
+  2. All `volumes:` keys removed from every service definition.
+  3. The top-level `volumes:` section removed entirely.
+
+  Full content:
+
+  ```yaml
+  services:
+    surrealdb:
+      image: surrealdb/surrealdb:v3.0.4
+      command: start --auth --user root --pass root
+      ports:
+        - "7004:8000"
+      healthcheck:
+        test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+        interval: 5s
+        timeout: 3s
+        retries: 5
+
+    redis:
+      image: redis:7-alpine
+      ports:
+        - "7005:6379"
+
+    postgres:
+      image: postgres:16-alpine
+      environment:
+        POSTGRES_USER: postgres
+        POSTGRES_PASSWORD: postgres
+        POSTGRES_DB: postgres
+      ports:
+        - "7006:5432"
+      healthcheck:
+        test: ["CMD", "pg_isready", "-U", "postgres"]
+        interval: 5s
+        timeout: 3s
+        retries: 5
+
+    mailpit:
+      image: axllent/mailpit:latest
+      ports:
+        - "7007:1025"
+        - "7008:8025"
+
+    typesense:
+      image: typesense/typesense:27.1
+      environment:
+        TYPESENSE_DATA_DIR: /data
+        TYPESENSE_API_KEY: test
+      ports:
+        - "7009:8108"
+
+    garage-s3:
+      image: ghcr.io/deuxfleurs/garage:v2.0.0
+      ports:
+        - "7010:3900"
+        - "7011:3902"
+      environment:
+        GARAGE_ALLOW_WORLD_READABLE_CONFIG: "true"
+  ```
+
+- [ ] **Create `docker/docker-compose.testbed.yml`**
+
+  Full content — adds `testbed-net` internal network, the testbed app container (built from `../apps/testbed/Dockerfile`), and Caddy on ports 3200/3201:
+
+  ```yaml
+  networks:
+    testbed-net:
+
+  services:
+    surrealdb:
+      image: surrealdb/surrealdb:v3.0.4
+      command: start --auth --user root --pass root
+      networks:
+        - testbed-net
+
+    redis:
+      image: redis:7-alpine
+      networks:
+        - testbed-net
+
+    postgres:
+      image: postgres:16-alpine
+      environment:
+        POSTGRES_USER: postgres
+        POSTGRES_PASSWORD: postgres
+        POSTGRES_DB: postgres
+      networks:
+        - testbed-net
+
+    mailpit:
+      image: axllent/mailpit:latest
+      networks:
+        - testbed-net
+
+    typesense:
+      image: typesense/typesense:27.1
+      environment:
+        TYPESENSE_DATA_DIR: /data
+        TYPESENSE_API_KEY: test
+      networks:
+        - testbed-net
+
+    garage-s3:
+      image: ghcr.io/deuxfleurs/garage:v2.0.0
+      environment:
+        GARAGE_ALLOW_WORLD_READABLE_CONFIG: "true"
+      networks:
+        - testbed-net
+
+    testbed:
+      build:
+        context: ../apps/testbed
+        dockerfile: Dockerfile
+      environment:
+        SURREALDB_URL: ws://surrealdb:8000
+        GARAGE_S3_ENDPOINT: http://garage-s3:3900
+        GARAGE_S3_ADMIN_ENDPOINT: http://garage-s3:3902
+        REDIS_URL: redis://redis:6379
+        POSTGRES_URL: postgresql://postgres:postgres@postgres:5432/postgres
+        TYPESENSE_URL: http://typesense:8108
+        TYPESENSE_API_KEY: test
+        MAILPIT_SMTP_URL: smtp://mailpit:1025
+        MAILPIT_API_URL: http://mailpit:8025
+      networks:
+        - testbed-net
+      depends_on:
+        - surrealdb
+        - redis
+        - postgres
+        - garage-s3
+        - typesense
+        - mailpit
+
+    caddy:
+      image: caddy:2-alpine
+      ports:
+        - "3200:80"
+        - "3201:443"
+      volumes:
+        - ./Caddyfile:/etc/caddy/Caddyfile
+      networks:
+        - testbed-net
+  ```
+
+- [ ] **Create `docker/Caddyfile`**
+
+  Exact content:
+
+  ```caddyfile
+  :80 {
+    reverse_proxy testbed:3000
+  }
+  ```
+
+- [ ] **Create `docker/README.md`**
+
+  Full content:
+
+  ````markdown
+  # @suaveplan/docker
+
+  Docker Compose infrastructure for the genesis monorepo. Three stacks cover local development, CI integration testing, and full staging with the testbed showcase app.
+
+  ## Stacks
+
+  | File | Purpose | Ports |
+  |---|---|---|
+  | `docker-compose.dev.yml` | Persistent dev services | 8001–8006, 8900, 8902 |
+  | `docker-compose.test.yml` | Ephemeral CI services (no volumes) | 7004–7011 |
+  | `docker-compose.testbed.yml` | Full staging with testbed app + Caddy | 3200–3201 (+ internal) |
+
+  ## Starting the dev stack
+
+  ```bash
+  bun run --filter=@suaveplan/docker dev:up
+  ```
+
+  Stops it:
+
+  ```bash
+  bun run --filter=@suaveplan/docker dev:down
+  ```
+
+  ## Port table
+
+  ### Dev stack (`docker-compose.dev.yml`) — persistent volumes
+
+  | Host port | Container port | Service |
+  |---|---|---|
+  | 8001 | 8000 | SurrealDB WebSocket + HTTP |
+  | 8002 | 6379 | Redis |
+  | 8003 | 5432 | PostgreSQL |
+  | 8004 | 1025 | Mailpit SMTP |
+  | 8005 | 8025 | Mailpit Web UI |
+  | 8006 | 8108 | Typesense |
+  | 8900 | 3900 | Garage S3 API |
+  | 8902 | 3902 | Garage S3 Admin |
+
+  ### Test stack (`docker-compose.test.yml`) — ephemeral, no volumes
+
+  | Host port | Container port | Service |
+  |---|---|---|
+  | 7004 | 8000 | SurrealDB |
+  | 7005 | 6379 | Redis |
+  | 7006 | 5432 | PostgreSQL |
+  | 7007 | 1025 | Mailpit SMTP |
+  | 7008 | 8025 | Mailpit Web UI |
+  | 7009 | 8108 | Typesense |
+  | 7010 | 3900 | Garage S3 API |
+  | 7011 | 3902 | Garage S3 Admin |
+
+  ### Testbed staging (`docker-compose.testbed.yml`)
+
+  | Host port | Service |
+  |---|---|
+  | 3200 | apps/testbed HTTP (via Caddy) |
+  | 3201 | apps/testbed HTTPS (via Caddy) |
+
+  All other services in the testbed stack communicate on the internal `testbed-net` Docker network by service name — no host ports are exposed for those services.
+
+  ## Verifying services are healthy
+
+  After `dev:up`, check status:
+
+  ```bash
+  docker compose -f docker/docker-compose.dev.yml ps
+  ```
+
+  SurrealDB and PostgreSQL have healthchecks. Wait for their `Status` column to show `healthy`:
+
+  ```bash
+  # SurrealDB
+  curl -f http://localhost:8001/health
+
+  # PostgreSQL
+  docker compose -f docker/docker-compose.dev.yml exec postgres pg_isready -U postgres
+
+  # Mailpit Web UI
+  curl -f http://localhost:8005/api/v1/messages
+
+  # Typesense
+  curl -f http://localhost:8006/health
+
+  # Garage S3 Admin
+  curl -f http://localhost:8902/health
+  ```
+
+  ## Env-bridge pattern (testcontainers integration)
+
+  When `@suaveplan/testing-integration` fixture packages start a container, they first check for environment variable overrides. If **all** required vars for a fixture are set and non-empty, the fixture enters passthrough mode: it constructs config directly from the env vars, skips container startup, and skips the healthcheck (the service is assumed healthy by virtue of the env vars being explicitly set).
+
+  Copy `.env.example` to `.env` at the repo root and fill in the values pointing at the dev stack:
+
+  ```bash
+  cp docker/.env.example .env
+  ```
+
+  The default values in `.env.example` are pre-wired to the dev stack ports. Once the dev stack is running and `.env` is populated, integration test runs will reuse your live containers rather than spinning new ones — saving 30–60 seconds per test suite.
+
+  A partial set of vars for a fixture (e.g. `GARAGE_S3_ENDPOINT` set but `GARAGE_S3_ACCESS_KEY_ID` missing) causes the fixture to fall through to container startup. All required vars must be set for passthrough mode to activate.
+
+  ### Env-bridge variable reference
+
+  | Fixture | Required vars |
+  |---|---|
+  | `surrealdb` | `SURREALDB_URL`, `SURREALDB_USERNAME`, `SURREALDB_PASSWORD` |
+  | `redis` | `REDIS_URL` |
+  | `postgres` | `POSTGRES_URL` |
+  | `mailpit` | `MAILPIT_SMTP_URL`, `MAILPIT_API_URL` |
+  | `typesense` | `TYPESENSE_URL`, `TYPESENSE_API_KEY` |
+  | `garage-s3` | `GARAGE_S3_ENDPOINT`, `GARAGE_S3_ADMIN_ENDPOINT`, `GARAGE_S3_ACCESS_KEY_ID`, `GARAGE_S3_SECRET_ACCESS_KEY` |
+
+  Note: Garage S3 requires `ACCESS_KEY_ID` and `SECRET_ACCESS_KEY` which are generated when the container first initialises. The dev stack container creates these on first run; retrieve them by running `garage key info test-key` inside the container, then persist them in `.env`.
+
+  ## Running the testbed staging environment
+
+  The testbed stack builds `apps/testbed/` from its `Dockerfile` and serves it behind Caddy on port 3200.
+
+  Prerequisites: `apps/testbed/Dockerfile` must exist (created as part of sub-project 3).
+
+  Start:
+
+  ```bash
+  bun run --filter=@suaveplan/docker testbed:up
+  ```
+
+  Open `http://localhost:3200` in a browser. All six backing services are available to the testbed app by service name on the internal `testbed-net` Docker network.
+
+  Stop:
+
+  ```bash
+  bun run --filter=@suaveplan/docker testbed:down
+  ```
+
+  ## Scripts
+
+  | Script | Command |
+  |---|---|
+  | `dev:up` | `docker compose -f docker-compose.dev.yml up -d` |
+  | `dev:down` | `docker compose -f docker-compose.dev.yml down` |
+  | `test:up` | `docker compose -f docker-compose.test.yml up -d` |
+  | `test:down` | `docker compose -f docker-compose.test.yml down` |
+  | `testbed:up` | `docker compose -f docker-compose.testbed.yml up --build -d` |
+  | `testbed:down` | `docker compose -f docker-compose.testbed.yml down` |
+  ````
+
+- [ ] **Verify test compose YAML is valid**
+
+  ```bash
+  docker compose -f /mnt/s/web/genesis/docker/docker-compose.test.yml config
+  ```
+
+  Expected: compose prints resolved YAML with no errors.
+
+- [ ] **Verify testbed compose YAML is valid**
+
+  ```bash
+  docker compose -f /mnt/s/web/genesis/docker/docker-compose.testbed.yml config
+  ```
+
+  Expected: compose prints resolved YAML with no errors. (The `testbed` service build context will warn that `../apps/testbed` does not exist yet — this is expected at this stage and is not a failure. The YAML itself must be structurally valid.)
+
+- [ ] **Add `@suaveplan/docker` to Turborepo workspace**
+
+  Open `/mnt/s/web/genesis/turbo.json` (or `turbo.json` at repo root). Confirm `docker/` is picked up by the workspace glob. Bun workspace glob is set in `/mnt/s/web/genesis/package.json` under `"workspaces"`. Add `"docker"` to the array if not already covered by a wildcard.
+
+  Check the root `package.json` workspaces field:
+
+  ```bash
+  grep -A 10 '"workspaces"' /mnt/s/web/genesis/package.json
+  ```
+
+  If the wildcard already covers `docker/` (e.g. `"*"` or `["apps/*", "universal/**", "docker"]`), no change needed. If it does not, add `"docker"` to the array.
+
+- [ ] **Final verification: bun install resolves the workspace**
+
+  ```bash
+  cd /mnt/s/web/genesis && bun install
+  ```
+
+  Then confirm the package is linked:
+
+  ```bash
+  ls /mnt/s/web/genesis/node_modules/@suaveplan/docker 2>/dev/null || echo "symlink not found"
+  ```
+
+  Expected: symlink present pointing to `../../docker`.
+
+---
+
+## Summary of files created
+
+| File | Description |
+|---|---|
+| `docker/package.json` | Workspace package descriptor with six compose scripts |
+| `docker/docker-compose.dev.yml` | Dev stack — persistent volumes, ports 8001–8006, 8900, 8902 |
+| `docker/docker-compose.test.yml` | Test stack — ephemeral, ports 7004–7011 |
+| `docker/docker-compose.testbed.yml` | Staging stack — testbed app + Caddy, ports 3200–3201 |
+| `docker/Caddyfile` | Caddy reverse proxy config for testbed staging |
+| `docker/.env.example` | Env-bridge variable reference with dev stack defaults |
+| `docker/README.md` | Full documentation: ports, commands, env-bridge, testbed |
+
+## Notes for implementers
+
+- The Typesense image tag is `27.1` (not `27.1.0` — this matches the exact Docker Hub tag).
+- Garage S3 container port for S3 API is `3900`; admin API is `3902`. The dev stack maps these to host ports `8900` and `8902` respectively.
+- The testbed compose file does not expose service ports to the host — all inter-service communication goes through `testbed-net`. Only Caddy (3200/3201) is host-accessible.
+- The `GARAGE_ALLOW_WORLD_READABLE_CONFIG: "true"` environment variable is required for Garage to start without a mounted config file.
+- SurrealDB's `healthcheck` targets `http://localhost:8000/health` (HTTP, same port as WebSocket). This is correct — SurrealDB v3 exposes an HTTP health endpoint on the same port.
+- Mailpit does not have a healthcheck defined in the compose files — it starts reliably without one. Integration fixture healthchecks poll `<apiUrl>/api/v1/messages` directly.
+- The `TYPESENSE_DATA_DIR` env var is required for Typesense to know where to store data. In the test stack (no volumes), this dir lives in the ephemeral container filesystem and is discarded on `down`.