# Docker

Deploy SteelEngine with Docker Compose

For Azure DevOps pipeline authentication, follow [Entra workload identities](/self-hosting/entra-workload-identities). The Compose stack forwards `WORKLOAD_AUTH_ENABLED` and `WORKLOAD_AUTH_PROFILE` from `.env` to web and the worker. Configure organization identities and grants in the application after installation trust is enabled.

## Quick Start

The production Compose file is `docker/selfhost/compose.yml`. It pulls prebuilt images from `ghcr.io/steelengineai` and connects to a Temporal server that you run separately. Set up [Temporal](/self-hosting/temporal) first. Then write the `.env` from [1. Configure Environment](#1-configure-environment) and start the stack.

```bash
# Clone, sign in, configure, start
git clone https://github.com/steelengineai/steelengine.git && cd steelengine
docker login ghcr.io
# Write .env (see Production Setup below)
docker compose -f docker/selfhost/compose.yml up -d
```

Open [http://localhost:3000](http://localhost:3000)

`docker compose up` fails immediately when `TEMPORAL_WORKER_BROKER_SECRET` or `TEMPORAL_WORKER_APP_URL` is missing. The `steelengine-worker` container restarts until `TEMPORAL_ADDRESS`, `TEMPORAL_NAMESPACE`, `TEMPORAL_AUTH_ISSUER`, `TEMPORAL_AUTH_JWKS_URL`, and `TEMPORAL_BUILD_ID` are set. There is no working bare `up -d`.

## Production Setup

### 1. Configure Environment

```bash
# Generate secrets
cat > .env << EOF
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
TEMPORAL_WORKER_BROKER_SECRET=$(openssl rand -hex 32)
POSTGRES_PASSWORD=$(openssl rand -hex 16)
NEXT_PUBLIC_APP_URL=https://steelengine.yourdomain.com
BETTER_AUTH_URL=https://steelengine.yourdomain.com
NEXT_PUBLIC_SOCKET_URL=https://steelengine.yourdomain.com
TEMPORAL_WORKER_APP_URL=https://steelengine.yourdomain.com
TEMPORAL_ADDRESS=temporal.yourdomain.com:7233
TEMPORAL_NAMESPACE=steelengine-production
TEMPORAL_SERVER_NAME=temporal.yourdomain.com
TEMPORAL_AUTH_ISSUER=https://steelengine.yourdomain.com
TEMPORAL_AUTH_JWKS_URL=https://steelengine.yourdomain.com/api/auth/jwks
TEMPORAL_BUILD_ID=<image digest or git sha>
EOF
```

The Compose file builds `DATABASE_URL` from `POSTGRES_USER`, `POSTGRES_PASSWORD`, and `POSTGRES_DB`. The defaults are `postgres`, `postgres`, and `steelengine`. It always points at the bundled `db` service, so a `DATABASE_URL` in `.env` has no effect. To use an external database, edit the Compose file or deploy with [Helm](/self-hosting/kubernetes#external-database).

`TEMPORAL_WORKER_APP_URL` must be an HTTPS URL reachable from the Worker. The
Worker authenticates to that web endpoint with the normal internal-service
token plus a separate, purpose-bound proof signed by
`TEMPORAL_WORKER_BROKER_SECRET`. Use different random values for the two
secrets. Only web and Worker receive the broker secret. Production Compose does
not accept a static `TEMPORAL_API_KEY` for the Worker.

The `TEMPORAL_*` settings describe the server you set up on the [Temporal](/self-hosting/temporal) page:

* `TEMPORAL_ADDRESS` is `host:port` with no scheme. `TEMPORAL_TLS_ENABLED` defaults to `true`. `TEMPORAL_SERVER_NAME` is the TLS name on the server certificate.
* `TEMPORAL_AUTH_ISSUER` is the issuer the app writes into every Temporal token. The app defaults it to `http://localhost:3000` and the Worker has no default. Set it to your app URL and give the Temporal server the same value.
* `TEMPORAL_AUTH_JWKS_URL` is the app's JWKS endpoint, `<app URL>/api/auth/jwks`. The Worker and the Temporal server both verify tokens against it, so both must reach it.
* `TEMPORAL_BUILD_ID` identifies the Worker build to Temporal. Change it whenever you pull a new `worker` image.

### 2. Start Services

Sign in to the registry with the credentials from your subscription, then start the stack.

```bash
docker login ghcr.io
docker compose -f docker/selfhost/compose.yml up -d
```

### 3. Set Up SSL

**Caddy (Recommended)**

Caddy automatically handles SSL certificates.

```bash
# Install Caddy
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install caddy
```

Create `/etc/caddy/Caddyfile`:

```
steelengine.yourdomain.com {
    reverse_proxy localhost:3000

    handle /socket.io/* {
        reverse_proxy localhost:3002
    }
}
```

```bash
sudo systemctl restart caddy
```

**Nginx + Certbot**

```bash
# Install
sudo apt install nginx certbot python3-certbot-nginx -y

# Create /etc/nginx/sites-available/steelengine
server {
    listen 80;
    server_name steelengine.yourdomain.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /socket.io/ {
        proxy_pass http://127.0.0.1:3002;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

# Enable and get certificate
sudo ln -s /etc/nginx/sites-available/steelengine /etc/nginx/sites-enabled/
sudo certbot --nginx -d steelengine.yourdomain.com
```

## Temporal Worker

The `steelengine-worker` service runs `ghcr.io/steelengineai/worker` with a 4 GB memory limit. It executes workflows, schedules, knowledge connector syncs, webhook ingestion, notifications, document processing, and platform jobs. It polls eight Temporal Task Queues over one connection. It publishes no ports. Its health server listens on port 3003 inside the container at `/health/live` and `/health/ready`. It also serves the backlog of its Task Queues at `/metrics/task-queue-backlog`, which the Helm chart's autoscaler reads.

Scheduled **Sub Agent** tasks also execute their complete assistant loop and tool calls in this container. The app and Worker images include the shared `assistant-runtime`, `platform-tools`, and `server-email` packages. When building custom images, include the Worker's workspace dependencies as well as the web app's dependencies.

Give the Worker access to the model providers, integration services, and storage used by your tasks. Workspace and organization BYOK keys are resolved at run time; provider keys supplied through environment variables must also be available to the Worker. The Worker still contacts the app for scoped credentials when an assistant starts a Temporal workflow. See [Scheduled Sub Agents](/self-hosting/temporal#scheduled-sub-agents).

```bash
# Confirm the Worker is polling
docker compose -f docker/selfhost/compose.yml exec steelengine-worker curl -fsS http://127.0.0.1:3003/health/ready
```

The Worker reads the same `ENCRYPTION_KEY`, `API_ENCRYPTION_KEY`, `INTERNAL_API_SECRET`, and `EGRESS_ALLOWLIST` as the app. Keep those values identical. These variables set how many Activities each queue runs at once (defaults in parentheses):

| Variable                                                                                 | Queue                            |
| ---------------------------------------------------------------------------------------- | -------------------------------- |
| `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` (20), `TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS` (20) | Workflows                        |
| `TEMPORAL_KNOWLEDGE_CONNECTOR_MAX_CONCURRENT_ACTIVITIES` (5)                             | Knowledge connector syncs        |
| `TEMPORAL_WEBHOOK_INGESTION_MAX_CONCURRENT_ACTIVITIES` (30)                              | Webhook ingestion                |
| `TEMPORAL_NOTIFICATION_MAX_CONCURRENT_ACTIVITIES` (10)                                   | Notifications                    |
| `TEMPORAL_DOCUMENT_PROCESSING_MAX_CONCURRENT_ACTIVITIES` (20)                            | Document processing              |
| `TEMPORAL_COPY_BACKFILL_MAX_CONCURRENT_ACTIVITIES` (5)                                   | Copy backfill                    |
| `TEMPORAL_SCHEDULED_TASKS_MAX_CONCURRENT_ACTIVITIES` (20)                                | Scheduled tasks                  |
| `TEMPORAL_STEELENGINE_PLATFORM_MAX_CONCURRENT_ACTIVITIES` (10)                           | Source control and platform jobs |

Set `TEMPORAL_ACTIVITY_MEMORY_TARGET` to a fraction greater than `0` and less than `1`, such as `0.85`, to limit new Activity reservations by memory as well. At or above that fraction of the container's memory limit, each queue pauses new slot reservations above a minimum of one until usage drops. Already-reserved slots and running Activities are retained. Work that has not been picked up stays on the Temporal queue, where another Worker replica can take it. This needs a readable container memory limit. Without one, or when unset, only the counts above apply.

Each queue Worker also keeps up to `TEMPORAL_MAX_ACTIVITY_TASK_POLLERS` Activity Task polls and `TEMPORAL_MAX_WORKFLOW_TASK_POLLERS` Workflow Task polls open on the Temporal server. The defaults suit one or a few Worker replicas. Lower them before running many replicas, because the server limits the polls a namespace may hold. See [Temporal → Scaling Workers](/self-hosting/temporal#scaling-workers).

The Worker's database pool defaults to `DATABASE_POOL_MAX=25` with a 60,000 ms statement timeout. The app uses 20 and 30,000 ms. The Worker needs more connections because it runs Activities for every queue at once. Set `OTEL_EXPORTER_OTLP_ENDPOINT` to export traces and metrics over OTLP. `HYPERDX_API_KEY` and `OTEL_EXPORTER_OTLP_HEADERS` carry the collector credentials.

## Private Network Configuration

Set deployment policies in the Compose `.env` file, then recreate the app and Worker containers:

```dotenv
EGRESS_ALLOWLIST=https://erp.corp.internal:8443=10.0.42.0/24
ALLOWED_MCP_DOMAINS=mcp.corp.internal
# Optional: when API and webhook callers use a different host from the app.
PUBLIC_API_URL=https://api.example.com
```

The production Compose file passes both policies to the app and Worker. `EGRESS_ALLOWLIST` matches a workflow HTTP destination's scheme, host, and port; the optional CIDR pin restricts its resolved address. `ALLOWED_MCP_DOMAINS` separately permits MCP hostnames, including approved private destinations. Neither setting changes network routing or firewalls. See [Workflow Egress Allowlist](/self-hosting/environment-variables#workflow-egress-allowlist) and [Allowed Server Domains](/mcp#allowed-server-domains) for the policy limits.

`PUBLIC_API_URL` defaults to `NEXT_PUBLIC_APP_URL`. Compose gives the app the same effective value as `NEXT_PUBLIC_API_URL` and passes `PUBLIC_API_URL` to the Worker. Keep `TEMPORAL_WORKER_APP_URL` as the separate HTTPS address the Worker uses to contact the app, and keep the Temporal issuer and JWKS settings consistent with your Temporal server.

```bash
docker compose -f docker/selfhost/compose.yml up -d --force-recreate steelengine steelengine-worker
```

Verify an approved and an unapproved private destination through both editor and deployed execution, and inspect the egress audit log. Test MCP connection and tool execution using the final endpoint URL; redirects are rejected. Incoming webhooks still need a reachable public endpoint.

## Ollama

Point the stack at an Ollama server the containers can reach. `OLLAMA_URL` defaults to `http://localhost:11434`. Inside the container that address is the container itself, so you must override it.

`docker/selfhost/compose.ollama.yml` bundles an Ollama container with the app, realtime, migrations, and PostgreSQL. It has no `steelengine-worker` service and builds every image from source. A stack started from it cannot run workflows. Use the production Compose file and point it at Ollama instead.

```bash
# macOS/Windows
OLLAMA_URL=http://host.docker.internal:11434 docker compose -f docker/selfhost/compose.yml up -d

# Linux - use your host IP
OLLAMA_URL=http://192.168.1.100:11434 docker compose -f docker/selfhost/compose.yml up -d
```

Inside Docker, `localhost` refers to the container, not your host. Use `host.docker.internal` or your host's IP. The production Compose file does not define `host.docker.internal`. On Linux Docker Engine, use the host IP.

## Laya Decision

The [Laya Decision](/blocks/laya) block needs the Laya decision server, which you run next to the stack. Until the app and the Temporal Worker both have `LAYA_URL`, the block stays hidden from the toolbar, search, and Mothership, and Laya does not appear in the model controls.

`docker/selfhost/compose.laya.yml` adds the server and wires both services to it:

```bash
export LAYA_API_KEY=$(openssl rand -hex 24)
docker compose -f docker/selfhost/compose.yml -f docker/selfhost/compose.laya.yml up -d
```

The server is CPU only. Reserve about 1 vCPU and 4 GB of RAM; it loads two models (English and multilingual, about 3 GB) at startup, which takes a minute or two. The image (`ghcr.io/steelengineai/laya`, tagged by laya version and model commit, override with `LAYA_IMAGE_TAG`) contains the models and makes no network calls, so the server needs no outbound access. Keep `LAYA_API_KEY` identical on all three services.

Once it runs, organization admins can still turn Laya off under **Organization Settings → Models**, and workspaces under **Settings → BYOK → Workspace model overrides**, like any model.

## Workflow Checkpoint Upgrades

Checkpoint state V4 reduces repeated workflow payload storage. In releases containing V4 support, migration `0288` prepares the database but leaves the write version unchanged. An operator activates V4 once per database after upgrading all application readers and workers, including any older draining processes.

Use `docs/runbooks/workflow-state-performance.md` from your compatible release checkout or request it with your dedicated-installation release package. The [repository runbook](https://github.com/steelengineai/steelengine/blob/develop/docs/runbooks/workflow-state-performance.md#enable-v4-after-compatible-code-is-running) includes source and Compose commands, database selection, verification, and an operator completion record. Use your installation's existing Compose project, environment, overrides, and compatible image tags.

After activation, rollback builds must still support V4; lowering the gate is not a supported rollback. Existing V2/V3 execution chains retain their version, and historical checkpoints are not converted. The always-on cloning improvements do not require activation.

## Commands

```bash
# View logs
docker compose -f docker/selfhost/compose.yml logs -f steelengine
docker compose -f docker/selfhost/compose.yml logs -f steelengine-worker

# Stop
docker compose -f docker/selfhost/compose.yml down

# Update
docker compose -f docker/selfhost/compose.yml pull && docker compose -f docker/selfhost/compose.yml up -d

# Re-run migrations by hand
docker compose -f docker/selfhost/compose.yml run --rm migrations

# Backup database
docker compose -f docker/selfhost/compose.yml exec db pg_dump -U postgres steelengine > backup.sql
```

## Common Questions

**What containers are started by docker/selfhost/compose.yml?**

Five services are started: steelengine (main app on port 3000, 8 GB memory limit), steelengine-worker (Temporal Worker, 4 GB memory limit, internal only, health on port 3003), realtime (WebSocket server on port 3002, 1 GB memory limit), db (PostgreSQL 17 with pgvector on port 5432), and migrations (runs once to apply database schema changes, then exits). The Temporal server is not included. The TEMPORAL_* variables in .env point the app and the Worker at the one you run.

**How do I configure SSL for production?**

You can use either Caddy (recommended, handles certificates automatically) or Nginx with Certbot. Both need to reverse-proxy port 3000 for the main app and port 3002 for WebSocket connections at the /socket.io/ path.

**Why can I not connect to Ollama running on my host from inside Docker?**

Inside a Docker container, localhost refers to the container itself, not your host machine. On macOS and Windows, use http://host.docker.internal:11434. On Linux, use your host machine's actual IP address (e.g., http://192.168.1.100:11434). The production Compose file does not define host.docker.internal.

**Why does docker compose up exit before starting anything?**

docker/selfhost/compose.yml marks TEMPORAL_WORKER_BROKER_SECRET and TEMPORAL_WORKER_APP_URL as required. Compose stops with 'must be at least 32 bytes' or 'must be an HTTPS URL' when either is unset. Add both to .env. The broker secret must differ from INTERNAL_API_SECRET. The URL must be the HTTPS address of the app as seen from the Worker.

**How do I update SteelEngine to the latest version?**

Run docker compose -f docker/selfhost/compose.yml pull to fetch the latest images, then docker compose -f docker/selfhost/compose.yml up -d to restart with the new versions. The migrations container will automatically apply any new database schema changes on startup. Set a new TEMPORAL_BUILD_ID whenever the worker image changes.

**How do I back up and restore the database?**

Back up with: docker compose -f docker/selfhost/compose.yml exec db pg_dump -U postgres steelengine > backup.sql. Restore with: docker compose -f docker/selfhost/compose.yml exec -T db psql -U postgres steelengine < backup.sql. The database data is persisted in a Docker volume named postgres_data.

**Can I customize the PostgreSQL credentials?**

Yes. docker/selfhost/compose.yml uses environment variable defaults: POSTGRES_USER (default: postgres), POSTGRES_PASSWORD (default: postgres), POSTGRES_DB (default: steelengine), and POSTGRES_PORT (default: 5432). Set these in your .env file to override them. The Compose file assembles DATABASE_URL for every service from these values.
