feat(cloud): harden nginx-proxy + sql foundation; HTTP-01 interim cert plan

Wire up the three foundation stacks (nginx-proxy, sql, portainer) in cloud/compose.yml and add real configs for the first two. nginx-proxy - Base nginx.conf with http + stream contexts, modern TLS profile, client_max_body_size baseline for gitea LFS / mlflow artifacts. - Vhosts under conf.d/: grafana, gitea, keycloak, nodered, mlflow, jupyter, portainer (HTTPS upstream), rabbitmq, jenkins. WebSocket upgrade headers where needed (grafana live, node-red editor, jupyterhub kernels, jenkins agents). - conf.d/00-default.conf serves /.well-known/acme-challenge/ on :80 and 301-redirects everything else. - stream.d/mqtt.conf terminates MQTT-TLS at 8883, proxies to rabbitmq:1883 internally. - All vhosts reference /etc/letsencrypt/live/infra/* — a stable path via certbot --cert-name infra, so the wildcard migration changes nothing in the vhost files. - README documents: HTTP-01 SAN interim during Versio period → DNS-01 wildcard via certbot-dns-transip after migration; bootstrap procedure (self-signed fallback → real cert issuance → reload). sql - config/init.d/01-databases.sh provisions gitea/keycloak/mlflow databases + roles on first start. Idempotent only via fresh data volume — change the script after first run requires manual psql or a volume wipe. - compose env extended with GITEA_DB_PASSWORD, KEYCLOAK_DB_PASSWORD, MLFLOW_DB_PASSWORD. cloud - include: now wires nginx-proxy + sql + portainer. Other stacks stay commented for future rounds. - .env.example adds KEYCLOAK_DB_PASSWORD and sensible defaults (LETSENCRYPT_EMAIL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME, GITEA_ROOT_URL, POSTFIX_FROM_DOMAIN all pointing at wbd-rd.nl). - Operator note inline: bring portainer's standalone instance down before deploying via cloud compose; comment its ports: block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:43:35 +02:00
parent 67b37b9b2a
commit 5d95f8bfcc
19 changed files with 426 additions and 41 deletions
--- a/cloud/.env.example
+++ b/cloud/.env.example
@@ -4,18 +4,21 @@
 TZ=Europe/Amsterdam

 # Domain / TLS
-PRIMARY_DOMAIN=
-LETSENCRYPT_EMAIL=
+PRIMARY_DOMAIN=wbd-rd.nl
+LETSENCRYPT_EMAIL=r.de.ren@brabantsedelta.nl
+# Production CA: https://acme-v02.api.letsencrypt.org/directory
+# Staging CA (testing): https://acme-staging-v02.api.letsencrypt.org/directory
 ACME_CA_URI=https://acme-v02.api.letsencrypt.org/directory

 # WireGuard server
 WG_SERVER_PORT=51820
 WG_SERVER_PUBLIC_HOST=

-# Keycloak (admin bootstrap)
+# Keycloak (admin bootstrap + DB)
 KEYCLOAK_ADMIN=admin
 KEYCLOAK_ADMIN_PASSWORD=
-KEYCLOAK_HOSTNAME=
+KEYCLOAK_HOSTNAME=keycloak.wbd-rd.nl
+KEYCLOAK_DB_PASSWORD=

 # InfluxDB
 INFLUX_ADMIN_USER=admin
@@ -27,7 +30,7 @@ INFLUX_BUCKET=telemetry
 # Grafana
 GRAFANA_ADMIN_USER=admin
 GRAFANA_ADMIN_PASSWORD=
-GRAFANA_ROOT_URL=
+GRAFANA_ROOT_URL=https://grafana.wbd-rd.nl

 # SQL (postgres — single point of config)
 SQL_DB=config
@@ -41,10 +44,10 @@ RABBITMQ_VHOST=/

 # Postfix
 POSTFIX_RELAYHOST=
-POSTFIX_FROM_DOMAIN=
+POSTFIX_FROM_DOMAIN=wbd-rd.nl

 # Gitea (HTTPS-only; uses sql backend)
-GITEA_ROOT_URL=
+GITEA_ROOT_URL=https://gitea.wbd-rd.nl
 GITEA_DB_HOST=sql:5432
 GITEA_DB_NAME=gitea
 GITEA_DB_USER=gitea
--- a/cloud/compose.yml
+++ b/cloud/compose.yml
@@ -6,13 +6,14 @@ name: cloud

 # Uncomment includes as each stack is hardened beyond stub.
 include:
-  # Core ingress + identity
-  # - ../stacks/nginx-proxy/compose.yml
+  # Foundation (round 3) — ingress, auth backing store, ops console
+  - ../stacks/nginx-proxy/compose.yml
+  - ../stacks/sql/compose.yml
+  - ../stacks/portainer/compose.yml
+  # Core identity + VPN
  # - ../stacks/wireguard-server/compose.yml
  # - ../stacks/keycloak/compose.yml
-  # - ../stacks/portainer/compose.yml
  # Data
-  # - ../stacks/sql/compose.yml
  # - ../stacks/influxdb/compose.yml
  # Apps
  # - ../stacks/node-red/compose.yml
@@ -28,6 +29,13 @@ include:
  # FROST (when deployed)
  # - ../stacks/mosquitto/compose.yml

+# NOTE on portainer transition:
+#   The portainer stack publishes 9443+8000 for standalone first-run use.
+#   When bringing it up through this cloud compose, take the standalone
+#   instance down first (`cd stacks/portainer && docker compose down`) and
+#   comment out the `ports:` block in stacks/portainer/compose.yml so
+#   nginx-proxy is the only ingress. Access then via https://portainer.wbd-rd.nl/.
+
 networks:
  edge:
    name: cloud-edge
@@ -38,7 +46,7 @@ networks:
  data:
    name: cloud-data
    driver: bridge
-    internal: true      # databases — no internet egress
+    internal: true       # databases — no internet egress
  mgmt:
    name: cloud-mgmt
    driver: bridge
--- a/stacks/nginx-proxy/README.md
+++ b/stacks/nginx-proxy/README.md
@@ -1,47 +1,100 @@
 # nginx-proxy

-The single web ingress for cloud + edge. Reverse-proxies HTTPS UIs and stream-proxies MQTT-TLS to RabbitMQ. TLS certificates managed by a certbot sidecar (Let's Encrypt, HTTP-01 webroot challenge).
+The single web ingress for cloud + edge. Reverse-proxies HTTPS UIs and stream-proxies MQTT-TLS to RabbitMQ. TLS certs managed by a certbot sidecar (Let's Encrypt).

- **Image**: stock `nginx:1.27-alpine` (we don't use `nginxproxy/nginx-proxy` because we need the `stream {}` context for MQTT-TLS, which that image doesn't expose cleanly)
- **Sidecar**: `certbot/certbot:latest` — renews every 12h, shared `nginx-certs` + `nginx-acme-challenge` volumes
+- **Image**: stock `nginx:1.27-alpine` (we don't use `nginxproxy/nginx-proxy` because we need the `stream {}` context for MQTT-TLS)
+- **Sidecar**: `certbot/certbot:latest` — renews every 12h via HTTP-01 webroot challenges
 - **Networks**: `edge` (the only port-publisher) + `app` (talks to upstream services)
 - **Host ports**: `tcp/80`, `tcp/443`, `tcp/8883`

+## Cert strategy
+
+**Interim (Versio DNS)**: HTTP-01 SAN cert covering all subdomains, issued via `--webroot`. Requires:
+- Public DNS A records for each subdomain pointing at the cloud host
+- `tcp/80` reachable from the internet
+
+**After TransIP migration**: switch to DNS-01 wildcard (`*.wbd-rd.nl`). Swap the `certbot/certbot` image for a build that includes `certbot-dns-transip` and reissue with `--cert-name infra` so the cert path stays stable — **no vhost config changes needed**.
+
 ## Config layout

 ```
 config/
-├── nginx.conf            # base config — must include `stream {}` directive
-├── conf.d/               # HTTP vhosts (one per upstream UI)
-│   ├── grafana.conf
-│   ├── node-red.conf
-│   ├── gitea.conf
-│   └── ...
+├── nginx.conf                    # base — http + stream contexts
+├── conf.d/
+│   ├── 00-default.conf           # port 80: ACME challenge + HTTPS redirect
+│   ├── grafana.conf              # grafana.wbd-rd.nl
+│   ├── gitea.conf                # gitea.wbd-rd.nl
+│   ├── keycloak.conf             # keycloak.wbd-rd.nl
+│   ├── nodered.conf              # nodered.wbd-rd.nl
+│   ├── mlflow.conf               # mlflow.wbd-rd.nl
+│   ├── jupyter.conf              # jupyter.wbd-rd.nl
+│   ├── portainer.conf            # portainer.wbd-rd.nl (HTTPS upstream)
+│   ├── rabbitmq.conf             # rabbitmq.wbd-rd.nl (mgmt UI)
+│   └── jenkins.conf              # jenkins.wbd-rd.nl
 └── stream.d/
-    └── mqtt.conf         # MQTT-TLS stream block, SNI route to rabbitmq:1883
+    └── mqtt.conf                 # mqtt.wbd-rd.nl:8883 → rabbitmq:1883
 ```

 Volumes:
- `nginx-certs` — Let's Encrypt cert chains (`/etc/letsencrypt`), read-only mounted into nginx, writable from certbot
- `nginx-acme-challenge` — webroot for HTTP-01 challenges (`/var/www/certbot`)
+- `nginx-certs` — Let's Encrypt cert chains at `/etc/letsencrypt/`; read-only into nginx, writable from certbot
+- `nginx-acme-challenge` — webroot for HTTP-01 challenges at `/var/www/certbot/`

-## Initial cert issuance
+All vhosts reference `/etc/letsencrypt/live/infra/fullchain.pem` and `privkey.pem` — a stable path independent of the issuance method.

-1. Start with HTTP-only nginx config (serving `/.well-known/acme-challenge/`).
-2. Issue:
-   ```bash
-   docker compose run --rm certbot certonly \
-     --webroot -w /var/www/certbot \
-     --email "$LETSENCRYPT_EMAIL" --agree-tos --no-eff-email \
-     -d gitea.example.com -d grafana.example.com -d nodered.example.com
-   ```
-3. Drop HTTPS vhost configs into `config/conf.d/` and reload nginx.
+## First-run bootstrap

-The sidecar then renews automatically.
+The HTTPS server blocks won't load without a cert at `/etc/letsencrypt/live/infra/`. Bootstrap procedure (one-time):
+
+```bash
+cd stacks/nginx-proxy
+
+# 1. Self-signed fallback so nginx starts and serves /.well-known/acme-challenge/
+docker compose run --rm --entrypoint=/bin/sh nginx -c \
+  "mkdir -p /etc/letsencrypt/live/infra && \
+   openssl req -x509 -nodes -days 1 -newkey rsa:2048 \
+     -keyout /etc/letsencrypt/live/infra/privkey.pem \
+     -out   /etc/letsencrypt/live/infra/fullchain.pem \
+     -subj  '/CN=bootstrap-infra'"
+
+# 2. Start nginx (HTTPS blocks load with the dummy cert)
+docker compose up -d nginx
+
+# 3. Issue the real cert via HTTP-01
+docker compose run --rm certbot certonly \
+  --webroot -w /var/www/certbot \
+  --email "$LETSENCRYPT_EMAIL" --agree-tos --no-eff-email \
+  --cert-name infra \
+  -d grafana.wbd-rd.nl  -d gitea.wbd-rd.nl     -d keycloak.wbd-rd.nl \
+  -d nodered.wbd-rd.nl  -d mlflow.wbd-rd.nl    -d jupyter.wbd-rd.nl \
+  -d portainer.wbd-rd.nl -d rabbitmq.wbd-rd.nl -d jenkins.wbd-rd.nl \
+  -d mqtt.wbd-rd.nl
+
+# 4. Reload nginx to pick up the real cert
+docker compose exec nginx nginx -s reload
+```
+
+The certbot sidecar then renews every 12h automatically.
+
+## DNS prereqs (HTTP-01)
+
+Before bootstrap, ensure A records exist in Versio for:
+
+```
+grafana.wbd-rd.nl     A  <cloud-public-ip>
+gitea.wbd-rd.nl       A  <cloud-public-ip>
+keycloak.wbd-rd.nl    A  <cloud-public-ip>
+nodered.wbd-rd.nl     A  <cloud-public-ip>
+mlflow.wbd-rd.nl      A  <cloud-public-ip>
+jupyter.wbd-rd.nl     A  <cloud-public-ip>
+portainer.wbd-rd.nl   A  <cloud-public-ip>
+rabbitmq.wbd-rd.nl    A  <cloud-public-ip>
+jenkins.wbd-rd.nl     A  <cloud-public-ip>
+mqtt.wbd-rd.nl        A  <cloud-public-ip>
+```

 ## TODO

- Write base `config/nginx.conf` (`http` + `stream` contexts)
- Per-upstream vhost templates with OIDC `auth_request` to Keycloak
- Decide internal PKI vs Let's Encrypt for cloud-internal hostnames not reachable from the public internet
- Edge-side variant: bind to plant-LAN IP only, internal CA for plant.local hostnames
+- Wildcard cert via `certbot-dns-transip` (post TransIP migration)
+- OIDC `auth_request` to Keycloak in front of services without native OIDC (mlflow, portainer-CE)
+- Edge-side variant: bind to plant-LAN IP, internal CA for `*.local` hostnames
+- HSTS + security headers (`add_header Strict-Transport-Security ...`)
--- a/stacks/nginx-proxy/config/conf.d/00-default.conf
+++ b/stacks/nginx-proxy/config/conf.d/00-default.conf
@@ -0,0 +1,15 @@
+# Port 80 — ACME HTTP-01 challenges + force HTTPS redirect for everything else.
+
+server {
+    listen 80 default_server;
+    server_name _;
+
+    location /.well-known/acme-challenge/ {
+        root /var/www/certbot;
+        default_type "text/plain";
+    }
+
+    location / {
+        return 301 https://$host$request_uri;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/gitea.conf
+++ b/stacks/nginx-proxy/config/conf.d/gitea.conf
@@ -0,0 +1,19 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name gitea.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    # Allow large LFS uploads + raw blob downloads
+    client_max_body_size 1G;
+
+    location / {
+        proxy_pass http://gitea:3000;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/grafana.conf
+++ b/stacks/nginx-proxy/config/conf.d/grafana.conf
@@ -0,0 +1,21 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name grafana.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://grafana:3000;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        # Grafana Live + alerting use WebSockets
+        proxy_http_version 1.1;
+        proxy_set_header   Upgrade    $http_upgrade;
+        proxy_set_header   Connection "upgrade";
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/jenkins.conf
+++ b/stacks/nginx-proxy/config/conf.d/jenkins.conf
@@ -0,0 +1,23 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name jenkins.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://jenkins:8080;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        proxy_redirect http:// https://;
+
+        # Jenkins agent + plugin upgrades use WebSockets
+        proxy_http_version 1.1;
+        proxy_set_header   Upgrade    $http_upgrade;
+        proxy_set_header   Connection "upgrade";
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/jupyter.conf
+++ b/stacks/nginx-proxy/config/conf.d/jupyter.conf
@@ -0,0 +1,24 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name jupyter.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://jupyterhub:8000;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        # JupyterHub uses WebSockets for terminals + notebook kernels
+        proxy_http_version 1.1;
+        proxy_set_header   Upgrade    $http_upgrade;
+        proxy_set_header   Connection "upgrade";
+
+        # Long-running notebooks
+        proxy_read_timeout 24h;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/keycloak.conf
+++ b/stacks/nginx-proxy/config/conf.d/keycloak.conf
@@ -0,0 +1,17 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name keycloak.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://keycloak:8080;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_set_header X-Forwarded-Host  $host;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/mlflow.conf
+++ b/stacks/nginx-proxy/config/conf.d/mlflow.conf
@@ -0,0 +1,22 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name mlflow.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    # Large model artifact uploads
+    client_max_body_size 5G;
+
+    location / {
+        proxy_pass http://mlflow:5000;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        proxy_read_timeout 10m;
+        proxy_send_timeout 10m;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/nodered.conf
+++ b/stacks/nginx-proxy/config/conf.d/nodered.conf
@@ -0,0 +1,23 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name nodered.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://node-red:1880;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        # Node-RED editor + deploy API use WebSockets
+        proxy_http_version 1.1;
+        proxy_set_header   Upgrade    $http_upgrade;
+        proxy_set_header   Connection "upgrade";
+
+        proxy_read_timeout 1h;
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/portainer.conf
+++ b/stacks/nginx-proxy/config/conf.d/portainer.conf
@@ -0,0 +1,23 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name portainer.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        # Portainer CE 2.x exposes HTTPS only (self-signed cert inside the container)
+        proxy_pass https://portainer:9443;
+        proxy_ssl_verify off;
+
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        proxy_http_version 1.1;
+        proxy_set_header   Upgrade    $http_upgrade;
+        proxy_set_header   Connection "upgrade";
+    }
+}
--- a/stacks/nginx-proxy/config/conf.d/rabbitmq.conf
+++ b/stacks/nginx-proxy/config/conf.d/rabbitmq.conf
@@ -0,0 +1,16 @@
+server {
+    listen 443 ssl;
+    http2 on;
+    server_name rabbitmq.wbd-rd.nl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+
+    location / {
+        proxy_pass http://rabbitmq:15672;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}
--- a/stacks/nginx-proxy/config/nginx.conf
+++ b/stacks/nginx-proxy/config/nginx.conf
@@ -0,0 +1,41 @@
+# nginx.conf — base config (http + stream contexts)
+
+user  nginx;
+worker_processes  auto;
+
+error_log  /var/log/nginx/error.log notice;
+pid        /var/run/nginx.pid;
+
+events {
+    worker_connections  1024;
+}
+
+http {
+    include       /etc/nginx/mime.types;
+    default_type  application/octet-stream;
+
+    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
+                    '$status $body_bytes_sent "$http_referer" '
+                    '"$http_user_agent" "$http_x_forwarded_for"';
+    access_log /var/log/nginx/access.log main;
+
+    sendfile        on;
+    tcp_nopush      on;
+    tcp_nodelay     on;
+    keepalive_timeout  65;
+    server_tokens   off;
+    client_max_body_size 200M;
+
+    ssl_protocols TLSv1.2 TLSv1.3;
+    ssl_ciphers HIGH:!aNULL:!MD5;
+    ssl_prefer_server_ciphers off;
+    ssl_session_cache shared:SSL:10m;
+    ssl_session_timeout 1d;
+    ssl_session_tickets off;
+
+    include /etc/nginx/conf.d/*.conf;
+}
+
+stream {
+    include /etc/nginx/stream.d/*.conf;
+}
--- a/stacks/nginx-proxy/config/stream.d/mqtt.conf
+++ b/stacks/nginx-proxy/config/stream.d/mqtt.conf
@@ -0,0 +1,16 @@
+# MQTT-TLS reverse proxy.
+# Public:   mqtt.wbd-rd.nl:8883 (TLS, terminated here)
+# Upstream: rabbitmq:1883 (plaintext on internal `app` network)
+
+server {
+    listen 8883 ssl;
+
+    ssl_certificate     /etc/letsencrypt/live/infra/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;
+    ssl_protocols TLSv1.2 TLSv1.3;
+    ssl_ciphers HIGH:!aNULL:!MD5;
+
+    proxy_pass rabbitmq:1883;
+    proxy_timeout 10m;
+    proxy_connect_timeout 5s;
+}
--- a/stacks/sql/.env.example
+++ b/stacks/sql/.env.example
@@ -1,3 +1,9 @@
+# sql — postgres superuser
 SQL_DB=config
 SQL_USER=config
 SQL_PASSWORD=
+
+# Per-app passwords (consumed by config/init.d/01-databases.sh)
+GITEA_DB_PASSWORD=
+KEYCLOAK_DB_PASSWORD=
+MLFLOW_DB_PASSWORD=
--- a/stacks/sql/README.md
+++ b/stacks/sql/README.md
@@ -5,5 +5,32 @@ Central configuration database — the "single point of config" backing Keycloak
 - **Engine**: postgres 16-alpine
 - **Network**: `data` only (no internet egress)
 - **Volume**: `sql-data` (PGDATA)
- **Init scripts**: `config/init.d/*.sql` runs on first start — provisions per-app databases/roles (gitea, keycloak, mlflow, …)
- **TODO**: backup strategy (pg_dump cron sidecar vs streaming replica)
+- **Init scripts**: `config/init.d/*.sh` runs once on first container start
+
+## Per-app databases
+
+On first start, `config/init.d/01-databases.sh` provisions:
+
+| Database | Owner role | Used by |
+|---|---|---|
+| `gitea` | `gitea` | gitea stack |
+| `keycloak` | `keycloak` | keycloak stack |
+| `mlflow` | `mlflow` | mlflow stack |
+
+Passwords come from env vars (`GITEA_DB_PASSWORD`, `KEYCLOAK_DB_PASSWORD`, `MLFLOW_DB_PASSWORD`) which must be set in the cloud `.env` *before* first start.
+
+**Important**: init scripts only run when `sql-data` is empty. Changing the script after first start has no effect until the volume is wiped. To add a new app DB later, connect with `psql` and create it manually, then update this script for fresh deploys.
+
+## Reset / re-init
+
+```bash
+docker compose down
+docker volume rm cloud_sql-data       # ⚠ destroys all data
+docker compose up -d
+```
+
+## TODO
+
+- Backup strategy (pg_dump cron sidecar vs streaming replica vs WAL archiving to MinIO)
+- Per-app least-privilege grants (currently each role owns its DB only — fine for now)
+- Monitoring (postgres_exporter for Prometheus when observability stack lands)
--- a/stacks/sql/compose.yml
+++ b/stacks/sql/compose.yml
@@ -10,6 +10,10 @@ services:
      POSTGRES_DB: ${SQL_DB}
      POSTGRES_USER: ${SQL_USER}
      POSTGRES_PASSWORD: ${SQL_PASSWORD}
+      # Per-app passwords (read by config/init.d/01-databases.sh on first start)
+      GITEA_DB_PASSWORD: ${GITEA_DB_PASSWORD}
+      KEYCLOAK_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD}
+      MLFLOW_DB_PASSWORD: ${MLFLOW_DB_PASSWORD}
      TZ: ${TZ:-Europe/Amsterdam}
    volumes:
      - sql-data:/var/lib/postgresql/data
--- a/stacks/sql/config/init.d/01-databases.sh
+++ b/stacks/sql/config/init.d/01-databases.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+# 01-databases.sh — provision per-app databases and roles on first start.
+#
+# Runs ONLY when the data directory is empty (first container start).
+# Modifying this file later has no effect unless you wipe sql-data first.
+#
+# Env vars used here come from the sql service's `environment:` block;
+# values are sourced from cloud/.env (or the standalone stack's .env).
+
+set -eu
+
+psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
+    -- gitea
+    CREATE ROLE gitea LOGIN PASSWORD '${GITEA_DB_PASSWORD}';
+    CREATE DATABASE gitea OWNER gitea;
+
+    -- keycloak
+    CREATE ROLE keycloak LOGIN PASSWORD '${KEYCLOAK_DB_PASSWORD}';
+    CREATE DATABASE keycloak OWNER keycloak;
+
+    -- mlflow
+    CREATE ROLE mlflow LOGIN PASSWORD '${MLFLOW_DB_PASSWORD}';
+    CREATE DATABASE mlflow OWNER mlflow;
+EOSQL