scaffold: hub-and-spoke layout, 4-network topology, 13 stack stubs
Initial structure for R&D infrastructure:
- stacks/ — 13 reusable, runnable stack stubs (kebab-case)
cloud-and-edge: node-red, influxdb, grafana, keycloak, portainer,
nginx-proxy, mqtt, postfix
cloud-only: wireguard-server, gitea, jenkins, sql (postgres stub)
edge-only: wireguard-client
- cloud/ — single central hub composition with 4 networks
(edge, app, data internal, mgmt) and include: stubs
- sites/ — per-plant edge folders (template README only for now)
- docs/architecture.md — hub-and-spoke + ingress + segmentation rationale
Network model: only nginx-proxy (80/443/8883) and wireguard-server
(51820/udp) publish ports on the cloud host. Edge nginx publishes
80/443 on plant-LAN interface only. MQTT cloud-side via nginx stream
proxy; MQTT edge-side internal-only; Postfix outbound-only.
OT layer (OPCUA, PLCs) is out of scope for this repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:37:59 +02:00
|
|
|
# Cloud / Central layer composition.
|
2026-05-21 16:37:58 +02:00
|
|
|
# Pulls in every stack that runs on the central hub and adds cross-stack
|
|
|
|
|
# dependencies (the per-stack composes stay standalone-runnable).
|
|
|
|
|
#
|
|
|
|
|
# Fresh-deploy procedure (see ../docs/architecture.md for the long version):
|
|
|
|
|
# 1. cp .env.example .env && fill secrets
|
|
|
|
|
# 2. Set DNS A records for the 10 short subdomains + vpn.wbd-rd.nl
|
|
|
|
|
# 3. docker compose up -d
|
|
|
|
|
# - nginx-init creates a self-signed bootstrap cert
|
|
|
|
|
# - sql comes up, init.d/01-databases.sh provisions per-app DBs
|
|
|
|
|
# - keycloak / gitea / mlflow wait on sql healthcheck before starting
|
|
|
|
|
# 4. ./deploy.sh — single command. Brings everything up, runs first-time cert
|
|
|
|
|
# issuance via certbot HTTP-01 (SAN over all *.wbd-rd.nl), reloads nginx,
|
|
|
|
|
# smoke-tests every vhost. Idempotent; safe to rerun.
|
|
|
|
|
# 5. Flip ACME_CA_URI from staging → prod in .env, ./deploy.sh again.
|
scaffold: hub-and-spoke layout, 4-network topology, 13 stack stubs
Initial structure for R&D infrastructure:
- stacks/ — 13 reusable, runnable stack stubs (kebab-case)
cloud-and-edge: node-red, influxdb, grafana, keycloak, portainer,
nginx-proxy, mqtt, postfix
cloud-only: wireguard-server, gitea, jenkins, sql (postgres stub)
edge-only: wireguard-client
- cloud/ — single central hub composition with 4 networks
(edge, app, data internal, mgmt) and include: stubs
- sites/ — per-plant edge folders (template README only for now)
- docs/architecture.md — hub-and-spoke + ingress + segmentation rationale
Network model: only nginx-proxy (80/443/8883) and wireguard-server
(51820/udp) publish ports on the cloud host. Edge nginx publishes
80/443 on plant-LAN interface only. MQTT cloud-side via nginx stream
proxy; MQTT edge-side internal-only; Postfix outbound-only.
OT layer (OPCUA, PLCs) is out of scope for this repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:37:59 +02:00
|
|
|
|
|
|
|
|
name: cloud
|
|
|
|
|
|
|
|
|
|
include:
|
2026-05-21 16:37:58 +02:00
|
|
|
# Foundation — ingress, DB, ops console
|
feat(cloud): harden nginx-proxy + sql foundation; HTTP-01 interim cert plan
Wire up the three foundation stacks (nginx-proxy, sql, portainer) in
cloud/compose.yml and add real configs for the first two.
nginx-proxy
- Base nginx.conf with http + stream contexts, modern TLS profile,
client_max_body_size baseline for gitea LFS / mlflow artifacts.
- Vhosts under conf.d/: grafana, gitea, keycloak, nodered, mlflow,
jupyter, portainer (HTTPS upstream), rabbitmq, jenkins. WebSocket
upgrade headers where needed (grafana live, node-red editor,
jupyterhub kernels, jenkins agents).
- conf.d/00-default.conf serves /.well-known/acme-challenge/ on :80
and 301-redirects everything else.
- stream.d/mqtt.conf terminates MQTT-TLS at 8883, proxies to
rabbitmq:1883 internally.
- All vhosts reference /etc/letsencrypt/live/infra/* — a stable path
via certbot --cert-name infra, so the wildcard migration changes
nothing in the vhost files.
- README documents: HTTP-01 SAN interim during Versio period →
DNS-01 wildcard via certbot-dns-transip after migration; bootstrap
procedure (self-signed fallback → real cert issuance → reload).
sql
- config/init.d/01-databases.sh provisions gitea/keycloak/mlflow
databases + roles on first start. Idempotent only via fresh
data volume — change the script after first run requires
manual psql or a volume wipe.
- compose env extended with GITEA_DB_PASSWORD, KEYCLOAK_DB_PASSWORD,
MLFLOW_DB_PASSWORD.
cloud
- include: now wires nginx-proxy + sql + portainer. Other stacks
stay commented for future rounds.
- .env.example adds KEYCLOAK_DB_PASSWORD and sensible defaults
(LETSENCRYPT_EMAIL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME,
GITEA_ROOT_URL, POSTFIX_FROM_DOMAIN all pointing at wbd-rd.nl).
- Operator note inline: bring portainer's standalone instance down
before deploying via cloud compose; comment its ports: block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:43:35 +02:00
|
|
|
- ../stacks/nginx-proxy/compose.yml
|
|
|
|
|
- ../stacks/sql/compose.yml
|
|
|
|
|
- ../stacks/portainer/compose.yml
|
2026-05-21 16:37:58 +02:00
|
|
|
# Identity + VPN
|
2026-05-21 13:54:57 +02:00
|
|
|
- ../stacks/keycloak/compose.yml
|
2026-05-21 18:34:37 +00:00
|
|
|
- ../stacks/oauth2-proxy/compose.yml
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/wireguard-server/compose.yml
|
feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site
Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks.
Locked decisions
- sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning
- nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO).
Chose stock over nginxproxy/nginx-proxy because stream{} is required for
MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883.
- gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published.
MQTT split
- Remove stacks/mqtt placeholder.
- Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin),
used at both cloud and edge. External MQTT clients reach cloud broker
via nginx stream-proxy on 8883.
- Add stacks/mosquitto — reserved for the FROST (SensorThings) stack
only. Cloud-only. Internal to its own stack; no external ingress.
ML / notebooks (cloud-only)
- stacks/mlflow — experiment tracking + model registry. Postgres backend
on sql stack; local volume for artifacts (S3/MinIO is a TODO).
- stacks/jupyterhub — multi-user notebook server. DockerSpawner via
mounted docker.sock; users spawn into cloud-app network so they can
reach mlflow, influxdb (via grafana), rabbitmq.
Sites
- sites/gemaal1 — first edge deployment scaffold. Site-local override
template for binding nginx to PLANT_LAN_IP.
Docs
- README + docs/architecture.md updated: stacks table now lists 15 stacks,
ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy
section locked, MQTT-split section added, Gitea HTTPS-only noted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
|
|
|
# Data
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/influxdb/compose.yml
|
feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site
Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks.
Locked decisions
- sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning
- nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO).
Chose stock over nginxproxy/nginx-proxy because stream{} is required for
MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883.
- gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published.
MQTT split
- Remove stacks/mqtt placeholder.
- Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin),
used at both cloud and edge. External MQTT clients reach cloud broker
via nginx stream-proxy on 8883.
- Add stacks/mosquitto — reserved for the FROST (SensorThings) stack
only. Cloud-only. Internal to its own stack; no external ingress.
ML / notebooks (cloud-only)
- stacks/mlflow — experiment tracking + model registry. Postgres backend
on sql stack; local volume for artifacts (S3/MinIO is a TODO).
- stacks/jupyterhub — multi-user notebook server. DockerSpawner via
mounted docker.sock; users spawn into cloud-app network so they can
reach mlflow, influxdb (via grafana), rabbitmq.
Sites
- sites/gemaal1 — first edge deployment scaffold. Site-local override
template for binding nginx to PLANT_LAN_IP.
Docs
- README + docs/architecture.md updated: stacks table now lists 15 stacks,
ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy
section locked, MQTT-split section added, Gitea HTTPS-only noted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
|
|
|
# Apps
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/node-red/compose.yml
|
|
|
|
|
- ../stacks/grafana/compose.yml
|
2026-05-21 14:10:54 +02:00
|
|
|
- ../stacks/gitea/compose.yml
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/jenkins/compose.yml
|
feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site
Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks.
Locked decisions
- sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning
- nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO).
Chose stock over nginxproxy/nginx-proxy because stream{} is required for
MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883.
- gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published.
MQTT split
- Remove stacks/mqtt placeholder.
- Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin),
used at both cloud and edge. External MQTT clients reach cloud broker
via nginx stream-proxy on 8883.
- Add stacks/mosquitto — reserved for the FROST (SensorThings) stack
only. Cloud-only. Internal to its own stack; no external ingress.
ML / notebooks (cloud-only)
- stacks/mlflow — experiment tracking + model registry. Postgres backend
on sql stack; local volume for artifacts (S3/MinIO is a TODO).
- stacks/jupyterhub — multi-user notebook server. DockerSpawner via
mounted docker.sock; users spawn into cloud-app network so they can
reach mlflow, influxdb (via grafana), rabbitmq.
Sites
- sites/gemaal1 — first edge deployment scaffold. Site-local override
template for binding nginx to PLANT_LAN_IP.
Docs
- README + docs/architecture.md updated: stacks table now lists 15 stacks,
ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy
section locked, MQTT-split section added, Gitea HTTPS-only noted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
|
|
|
# Messaging + mail
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/rabbitmq/compose.yml
|
|
|
|
|
- ../stacks/postfix/compose.yml
|
feat: SQL=postgres, nginx+certbot, MQTT split, ML stacks, gitea HTTPS-only, gemaal1 site
Round-2 changes locking in scaffold-phase decisions and adding ML/notebook stacks.
Locked decisions
- sql: postgres 16-alpine (was TBD); init.d/ mount for per-app DB provisioning
- nginx-proxy: stock nginx + certbot sidecar (was nginx:alpine TODO).
Chose stock over nginxproxy/nginx-proxy because stream{} is required for
MQTT-TLS reverse-proxy on tcp/8883 to rabbitmq:1883.
- gitea: HTTPS-only (DISABLE_SSH=true). No SSH port published.
MQTT split
- Remove stacks/mqtt placeholder.
- Add stacks/rabbitmq — general-purpose broker (AMQP + MQTT plugin),
used at both cloud and edge. External MQTT clients reach cloud broker
via nginx stream-proxy on 8883.
- Add stacks/mosquitto — reserved for the FROST (SensorThings) stack
only. Cloud-only. Internal to its own stack; no external ingress.
ML / notebooks (cloud-only)
- stacks/mlflow — experiment tracking + model registry. Postgres backend
on sql stack; local volume for artifacts (S3/MinIO is a TODO).
- stacks/jupyterhub — multi-user notebook server. DockerSpawner via
mounted docker.sock; users spawn into cloud-app network so they can
reach mlflow, influxdb (via grafana), rabbitmq.
Sites
- sites/gemaal1 — first edge deployment scaffold. Site-local override
template for binding nginx to PLANT_LAN_IP.
Docs
- README + docs/architecture.md updated: stacks table now lists 15 stacks,
ingress + attachment tables reflect mlflow/jupyterhub, TLS strategy
section locked, MQTT-split section added, Gitea HTTPS-only noted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:22:46 +02:00
|
|
|
# ML / notebooks
|
2026-05-21 16:37:58 +02:00
|
|
|
- ../stacks/mlflow/compose.yml
|
|
|
|
|
- ../stacks/jupyterhub/compose.yml
|
|
|
|
|
# SensorThings
|
|
|
|
|
- ../stacks/frost/compose.yml
|
scaffold: hub-and-spoke layout, 4-network topology, 13 stack stubs
Initial structure for R&D infrastructure:
- stacks/ — 13 reusable, runnable stack stubs (kebab-case)
cloud-and-edge: node-red, influxdb, grafana, keycloak, portainer,
nginx-proxy, mqtt, postfix
cloud-only: wireguard-server, gitea, jenkins, sql (postgres stub)
edge-only: wireguard-client
- cloud/ — single central hub composition with 4 networks
(edge, app, data internal, mgmt) and include: stubs
- sites/ — per-plant edge folders (template README only for now)
- docs/architecture.md — hub-and-spoke + ingress + segmentation rationale
Network model: only nginx-proxy (80/443/8883) and wireguard-server
(51820/udp) publish ports on the cloud host. Edge nginx publishes
80/443 on plant-LAN interface only. MQTT cloud-side via nginx stream
proxy; MQTT edge-side internal-only; Postfix outbound-only.
OT layer (OPCUA, PLCs) is out of scope for this repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:37:59 +02:00
|
|
|
|
2026-05-21 16:37:58 +02:00
|
|
|
# Cross-stack dependencies. Declared at the cloud level so each stack's
|
|
|
|
|
# own compose.yml stays standalone-runnable (no required peers).
|
|
|
|
|
services:
|
|
|
|
|
keycloak:
|
|
|
|
|
depends_on:
|
|
|
|
|
sql:
|
|
|
|
|
condition: service_healthy
|
|
|
|
|
gitea:
|
|
|
|
|
depends_on:
|
|
|
|
|
sql:
|
|
|
|
|
condition: service_healthy
|
|
|
|
|
mlflow:
|
|
|
|
|
depends_on:
|
|
|
|
|
sql:
|
|
|
|
|
condition: service_healthy
|
feat(cloud): harden nginx-proxy + sql foundation; HTTP-01 interim cert plan
Wire up the three foundation stacks (nginx-proxy, sql, portainer) in
cloud/compose.yml and add real configs for the first two.
nginx-proxy
- Base nginx.conf with http + stream contexts, modern TLS profile,
client_max_body_size baseline for gitea LFS / mlflow artifacts.
- Vhosts under conf.d/: grafana, gitea, keycloak, nodered, mlflow,
jupyter, portainer (HTTPS upstream), rabbitmq, jenkins. WebSocket
upgrade headers where needed (grafana live, node-red editor,
jupyterhub kernels, jenkins agents).
- conf.d/00-default.conf serves /.well-known/acme-challenge/ on :80
and 301-redirects everything else.
- stream.d/mqtt.conf terminates MQTT-TLS at 8883, proxies to
rabbitmq:1883 internally.
- All vhosts reference /etc/letsencrypt/live/infra/* — a stable path
via certbot --cert-name infra, so the wildcard migration changes
nothing in the vhost files.
- README documents: HTTP-01 SAN interim during Versio period →
DNS-01 wildcard via certbot-dns-transip after migration; bootstrap
procedure (self-signed fallback → real cert issuance → reload).
sql
- config/init.d/01-databases.sh provisions gitea/keycloak/mlflow
databases + roles on first start. Idempotent only via fresh
data volume — change the script after first run requires
manual psql or a volume wipe.
- compose env extended with GITEA_DB_PASSWORD, KEYCLOAK_DB_PASSWORD,
MLFLOW_DB_PASSWORD.
cloud
- include: now wires nginx-proxy + sql + portainer. Other stacks
stay commented for future rounds.
- .env.example adds KEYCLOAK_DB_PASSWORD and sensible defaults
(LETSENCRYPT_EMAIL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME,
GITEA_ROOT_URL, POSTFIX_FROM_DOMAIN all pointing at wbd-rd.nl).
- Operator note inline: bring portainer's standalone instance down
before deploying via cloud compose; comment its ports: block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:43:35 +02:00
|
|
|
|
scaffold: hub-and-spoke layout, 4-network topology, 13 stack stubs
Initial structure for R&D infrastructure:
- stacks/ — 13 reusable, runnable stack stubs (kebab-case)
cloud-and-edge: node-red, influxdb, grafana, keycloak, portainer,
nginx-proxy, mqtt, postfix
cloud-only: wireguard-server, gitea, jenkins, sql (postgres stub)
edge-only: wireguard-client
- cloud/ — single central hub composition with 4 networks
(edge, app, data internal, mgmt) and include: stubs
- sites/ — per-plant edge folders (template README only for now)
- docs/architecture.md — hub-and-spoke + ingress + segmentation rationale
Network model: only nginx-proxy (80/443/8883) and wireguard-server
(51820/udp) publish ports on the cloud host. Edge nginx publishes
80/443 on plant-LAN interface only. MQTT cloud-side via nginx stream
proxy; MQTT edge-side internal-only; Postfix outbound-only.
OT layer (OPCUA, PLCs) is out of scope for this repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:37:59 +02:00
|
|
|
networks:
|
|
|
|
|
edge:
|
|
|
|
|
name: cloud-edge
|
|
|
|
|
driver: bridge
|
|
|
|
|
app:
|
|
|
|
|
name: cloud-app
|
|
|
|
|
driver: bridge
|
|
|
|
|
data:
|
|
|
|
|
name: cloud-data
|
|
|
|
|
driver: bridge
|
feat(cloud): harden nginx-proxy + sql foundation; HTTP-01 interim cert plan
Wire up the three foundation stacks (nginx-proxy, sql, portainer) in
cloud/compose.yml and add real configs for the first two.
nginx-proxy
- Base nginx.conf with http + stream contexts, modern TLS profile,
client_max_body_size baseline for gitea LFS / mlflow artifacts.
- Vhosts under conf.d/: grafana, gitea, keycloak, nodered, mlflow,
jupyter, portainer (HTTPS upstream), rabbitmq, jenkins. WebSocket
upgrade headers where needed (grafana live, node-red editor,
jupyterhub kernels, jenkins agents).
- conf.d/00-default.conf serves /.well-known/acme-challenge/ on :80
and 301-redirects everything else.
- stream.d/mqtt.conf terminates MQTT-TLS at 8883, proxies to
rabbitmq:1883 internally.
- All vhosts reference /etc/letsencrypt/live/infra/* — a stable path
via certbot --cert-name infra, so the wildcard migration changes
nothing in the vhost files.
- README documents: HTTP-01 SAN interim during Versio period →
DNS-01 wildcard via certbot-dns-transip after migration; bootstrap
procedure (self-signed fallback → real cert issuance → reload).
sql
- config/init.d/01-databases.sh provisions gitea/keycloak/mlflow
databases + roles on first start. Idempotent only via fresh
data volume — change the script after first run requires
manual psql or a volume wipe.
- compose env extended with GITEA_DB_PASSWORD, KEYCLOAK_DB_PASSWORD,
MLFLOW_DB_PASSWORD.
cloud
- include: now wires nginx-proxy + sql + portainer. Other stacks
stay commented for future rounds.
- .env.example adds KEYCLOAK_DB_PASSWORD and sensible defaults
(LETSENCRYPT_EMAIL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME,
GITEA_ROOT_URL, POSTFIX_FROM_DOMAIN all pointing at wbd-rd.nl).
- Operator note inline: bring portainer's standalone instance down
before deploying via cloud compose; comment its ports: block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:43:35 +02:00
|
|
|
internal: true # databases — no internet egress
|
scaffold: hub-and-spoke layout, 4-network topology, 13 stack stubs
Initial structure for R&D infrastructure:
- stacks/ — 13 reusable, runnable stack stubs (kebab-case)
cloud-and-edge: node-red, influxdb, grafana, keycloak, portainer,
nginx-proxy, mqtt, postfix
cloud-only: wireguard-server, gitea, jenkins, sql (postgres stub)
edge-only: wireguard-client
- cloud/ — single central hub composition with 4 networks
(edge, app, data internal, mgmt) and include: stubs
- sites/ — per-plant edge folders (template README only for now)
- docs/architecture.md — hub-and-spoke + ingress + segmentation rationale
Network model: only nginx-proxy (80/443/8883) and wireguard-server
(51820/udp) publish ports on the cloud host. Edge nginx publishes
80/443 on plant-LAN interface only. MQTT cloud-side via nginx stream
proxy; MQTT edge-side internal-only; Postfix outbound-only.
OT layer (OPCUA, PLCs) is out of scope for this repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:37:59 +02:00
|
|
|
mgmt:
|
|
|
|
|
name: cloud-mgmt
|
|
|
|
|
driver: bridge
|
