Files
infra/stacks/sql/compose.yml

33 lines
883 B
YAML
Raw Normal View History

feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks. Locked decisions - sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning - nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO). Chose stock over nginxproxy/nginx-proxy because stream{} is required for MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883. - gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published. MQTT split - Remove stacks/mqtt placeholder. - Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin), used at both cloud and edge. External MQTT clients reach cloud broker via nginx stream-proxy on 8883. - Add stacks/mosquitto — reserved for the FROST (SensorThings) stack only. Cloud-only. Internal to its own stack; no external ingress. ML / notebooks (cloud-only) - stacks/mlflow — experiment tracking + model registry. Postgres backend on sql stack; local volume for artifacts (S3/MinIO is a TODO). - stacks/jupyterhub — multi-user notebook server. DockerSpawner via mounted docker.sock; users spawn into cloud-app network so they can reach mlflow, influxdb (via grafana), rabbitmq. Sites - sites/gemaal1 — first edge deployment scaffold. Site-local override template for binding nginx to PLANT_LAN_IP. Docs - README + docs/architecture.md updated: stacks table now lists 15 stacks, ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy section locked, MQTT-split section added, Gitea HTTPS-only noted. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
# sql — central config DB (postgres, cloud only)
# Networks: data (no internet egress)
services:
sql:
image: postgres:16-alpine
restart: unless-stopped
networks: [data]
environment:
POSTGRES_DB: ${SQL_DB}
POSTGRES_USER: ${SQL_USER}
POSTGRES_PASSWORD: ${SQL_PASSWORD}
feat(cloud): harden nginx-proxy + sql foundation; HTTP-01 interim cert plan Wire up the three foundation stacks (nginx-proxy, sql, portainer) in cloud/compose.yml and add real configs for the first two. nginx-proxy - Base nginx.conf with http + stream contexts, modern TLS profile, client_max_body_size baseline for gitea LFS / mlflow artifacts. - Vhosts under conf.d/: grafana, gitea, keycloak, nodered, mlflow, jupyter, portainer (HTTPS upstream), rabbitmq, jenkins. WebSocket upgrade headers where needed (grafana live, node-red editor, jupyterhub kernels, jenkins agents). - conf.d/00-default.conf serves /.well-known/acme-challenge/ on :80 and 301-redirects everything else. - stream.d/mqtt.conf terminates MQTT-TLS at 8883, proxies to rabbitmq:1883 internally. - All vhosts reference /etc/letsencrypt/live/infra/* — a stable path via certbot --cert-name infra, so the wildcard migration changes nothing in the vhost files. - README documents: HTTP-01 SAN interim during Versio period → DNS-01 wildcard via certbot-dns-transip after migration; bootstrap procedure (self-signed fallback → real cert issuance → reload). sql - config/init.d/01-databases.sh provisions gitea/keycloak/mlflow databases + roles on first start. Idempotent only via fresh data volume — change the script after first run requires manual psql or a volume wipe. - compose env extended with GITEA_DB_PASSWORD, KEYCLOAK_DB_PASSWORD, MLFLOW_DB_PASSWORD. cloud - include: now wires nginx-proxy + sql + portainer. Other stacks stay commented for future rounds. - .env.example adds KEYCLOAK_DB_PASSWORD and sensible defaults (LETSENCRYPT_EMAIL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME, GITEA_ROOT_URL, POSTFIX_FROM_DOMAIN all pointing at wbd-rd.nl). - Operator note inline: bring portainer's standalone instance down before deploying via cloud compose; comment its ports: block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:43:35 +02:00
# Per-app passwords (read by config/init.d/01-databases.sh on first start)
GITEA_DB_PASSWORD: ${GITEA_DB_PASSWORD}
KEYCLOAK_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD}
MLFLOW_DB_PASSWORD: ${MLFLOW_DB_PASSWORD}
TZ: ${TZ:-Europe/Amsterdam}
volumes:
- sql-data:/var/lib/postgresql/data
feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks. Locked decisions - sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning - nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO). Chose stock over nginxproxy/nginx-proxy because stream{} is required for MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883. - gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published. MQTT split - Remove stacks/mqtt placeholder. - Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin), used at both cloud and edge. External MQTT clients reach cloud broker via nginx stream-proxy on 8883. - Add stacks/mosquitto — reserved for the FROST (SensorThings) stack only. Cloud-only. Internal to its own stack; no external ingress. ML / notebooks (cloud-only) - stacks/mlflow — experiment tracking + model registry. Postgres backend on sql stack; local volume for artifacts (S3/MinIO is a TODO). - stacks/jupyterhub — multi-user notebook server. DockerSpawner via mounted docker.sock; users spawn into cloud-app network so they can reach mlflow, influxdb (via grafana), rabbitmq. Sites - sites/gemaal1 — first edge deployment scaffold. Site-local override template for binding nginx to PLANT_LAN_IP. Docs - README + docs/architecture.md updated: stacks table now lists 15 stacks, ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy section locked, MQTT-split section added, Gitea HTTPS-only noted. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
- ./config/init.d:/docker-entrypoint-initdb.d:ro
feat(cloud): single-shot deploy.sh + FROST stack + healthchecks Stage 5 — make the cloud composition spin up in one command and add the SensorThings (FROST) stack as a fully segregated tenant. cloud/deploy.sh — idempotent, 7-step bring-up: preflight → validate → up + wait → cert state → issue/renew → service status → endpoint smoke test. Reissues LE cert only when current issuer no longer matches ACME_CA_URI. Move-aside-then- restore-on-failure so the bootstrap cert survives a failed certbot. stacks/frost — new stack, segregated from shared sql/rabbitmq: - dedicated postgis container (frost-db) - dedicated internal mosquitto bus (frost-mosquitto) - frost-http + frost-mqtt on a private frost-internal network, joined to cloud-app only for nginx ingress at frost.wbd-rd.nl - shared mosquitto stack deleted; rabbitmq remains the only public MQTT broker (mqtt.wbd-rd.nl:8883 via stream proxy) stacks/sql — pg_isready healthcheck so keycloak/gitea/mlflow can gate on service_healthy via cloud-level depends_on overrides. stacks/nginx-proxy: - nginx-init service generates a self-signed bootstrap cert on fresh deploy so nginx starts before certbot has issued a real one - frost.wbd-rd.nl vhost (/FROST-Server → frost-http:8080, /mqtt → frost-mqtt:9876 WebSocket) stacks/mlflow — custom Dockerfile (upstream + psycopg2-binary) so the official image can speak to the shared sql backend. stacks/jupyterhub — DummyAuthenticator stub gated by JUPYTERHUB_ADMIN_PASSWORD; TODO comments point at OIDC + DockerSpawner. stacks/rabbitmq — config/{enabled_plugins,rabbitmq.conf} stubs (management + mqtt plugins, MQTT auth required). stacks/portainer — ports unpublished; nginx now the only ingress. stacks/node-red — pin to 4.1 (the floating "4" tag does not exist). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:37:58 +02:00
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${SQL_USER} -d ${SQL_DB}"]
interval: 10s
timeout: 5s
retries: 10
start_period: 30s
networks:
data:
volumes:
sql-data: