feat(cloud): single-shot deploy.sh + FROST stack + healthchecks
Stage 5 — make the cloud composition spin up in one command and add
the SensorThings (FROST) stack as a fully segregated tenant.
cloud/deploy.sh — idempotent, 7-step bring-up:
preflight → validate → up + wait → cert state → issue/renew →
service status → endpoint smoke test. Reissues LE cert only when
current issuer no longer matches ACME_CA_URI. Move-aside-then-
restore-on-failure so the bootstrap cert survives a failed certbot.
stacks/frost — new stack, segregated from shared sql/rabbitmq:
- dedicated postgis container (frost-db)
- dedicated internal mosquitto bus (frost-mosquitto)
- frost-http + frost-mqtt on a private frost-internal network,
joined to cloud-app only for nginx ingress at frost.wbd-rd.nl
- shared mosquitto stack deleted; rabbitmq remains the only public
MQTT broker (mqtt.wbd-rd.nl:8883 via stream proxy)
stacks/sql — pg_isready healthcheck so keycloak/gitea/mlflow can gate
on service_healthy via cloud-level depends_on overrides.
stacks/nginx-proxy:
- nginx-init service generates a self-signed bootstrap cert on
fresh deploy so nginx starts before certbot has issued a real one
- frost.wbd-rd.nl vhost (/FROST-Server → frost-http:8080,
/mqtt → frost-mqtt:9876 WebSocket)
stacks/mlflow — custom Dockerfile (upstream + psycopg2-binary) so the
official image can speak to the shared sql backend.
stacks/jupyterhub — DummyAuthenticator stub gated by
JUPYTERHUB_ADMIN_PASSWORD; TODO comments point at OIDC + DockerSpawner.
stacks/rabbitmq — config/{enabled_plugins,rabbitmq.conf} stubs
(management + mqtt plugins, MQTT auth required).
stacks/portainer — ports unpublished; nginx now the only ingress.
stacks/node-red — pin to 4.1 (the floating "4" tag does not exist).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -22,7 +22,7 @@ R&D infrastructure for Waterschap Brabantse Delta. Hub-and-spoke topology:
|
||||
│ rabbitmq, postfix, portainer │
|
||||
│ sql (postgres, single config) │
|
||||
│ mlflow, jupyterhub │
|
||||
│ mosquitto (FROST stack only) │
|
||||
│ frost (SensorThings API) │
|
||||
└───────────────┬────────────────────┘
|
||||
│ WireGuard tunnels
|
||||
┌───────┼────────┬───────────┐
|
||||
@@ -56,13 +56,14 @@ Each layer uses **four internal Docker networks**:
|
||||
```
|
||||
edge : nginx, wireguard-server
|
||||
app : nginx, rabbitmq, postfix, node-red, grafana,
|
||||
jenkins, gitea, keycloak, mlflow, jupyterhub
|
||||
jenkins, gitea, keycloak, mlflow, jupyterhub,
|
||||
portainer, frost-http, frost-mqtt
|
||||
data : influxdb, sql, grafana, mlflow
|
||||
mgmt : portainer, keycloak, wireguard-server, jupyterhub
|
||||
frost-internal (private to frost stack) :
|
||||
frost-db (postgis), frost-mosquitto, frost-http, frost-mqtt
|
||||
```
|
||||
|
||||
(`mosquitto` joins `app` only when the FROST stack is deployed.)
|
||||
|
||||
### Edge attachments
|
||||
|
||||
```
|
||||
@@ -117,12 +118,13 @@ For cloud-internal hostnames not reachable via Let's Encrypt HTTP-01, the longer
|
||||
|
||||
Postfix is **outbound-only**. It initiates SMTP to internet MX servers but accepts no inbound. Zero ingress, no published port, no listener facing internet. Just needs egress (every container has it via host NAT).
|
||||
|
||||
### MQTT — two brokers
|
||||
### MQTT — RabbitMQ for public traffic, dedicated mosquitto inside FROST
|
||||
|
||||
- **RabbitMQ** is the **general-purpose** broker. Runs at both cloud and edge. MQTT plugin enabled. Cloud-side reachable externally via nginx stream proxy on `tcp/8883`. Edge-side fully internal.
|
||||
- **Mosquitto** is reserved for the **FROST (SensorThings API) stack** only — cloud-only. Internal to its own stack — no external ingress unless FROST publishers need to push from outside (in which case use a separate stream block on a different port).
|
||||
- **RabbitMQ** is the **only public MQTT broker**. SCADA / IoT / edge clients connect to `mqtt.wbd-rd.nl:8883` (TLS, via nginx `stream {}` block proxying to `rabbitmq:1883`). Authentication uses the standard RABBITMQ_USER/PASS.
|
||||
- **frost-mosquitto** lives **inside the frost stack** on the private `frost-internal` docker network — it is purely the message bus between `frost-http` and `frost-mqtt`. It is not reachable from anywhere outside the frost stack.
|
||||
- SensorThings-protocol MQTT (the FROST native MQTT API) is exposed to clients via `frost-mqtt`'s WebSocket port, proxied as `https://frost.wbd-rd.nl/mqtt`.
|
||||
|
||||
If FROST needs cross-broker forwarding, add a RabbitMQ `shovel` plugin pointing at `mosquitto`. Not wired up by default.
|
||||
If FROST consumers also need to see SCADA traffic on RabbitMQ, add a RabbitMQ `shovel` plugin pointing into the frost stack. Not wired up by default.
|
||||
|
||||
### Gitea — HTTPS only
|
||||
|
||||
@@ -137,7 +139,7 @@ WireGuard is connectionless UDP with crypto-routed packets. Proxying through ngi
|
||||
The repo defines **15 stacks** under `stacks/`:
|
||||
|
||||
- **Cloud + edge**: `nginx-proxy`, `node-red`, `influxdb`, `grafana`, `keycloak`, `portainer`, `rabbitmq`, `postfix`
|
||||
- **Cloud-only**: `wireguard-server`, `gitea` (HTTPS), `jenkins`, `sql` (postgres), `mlflow`, `jupyterhub`, `mosquitto` (FROST)
|
||||
- **Cloud-only**: `wireguard-server`, `gitea` (HTTPS), `jenkins`, `sql` (postgres), `mlflow`, `jupyterhub`, `frost` (SensorThings, dedicated postgis + internal bus)
|
||||
- **Edge-only**: `wireguard-client`
|
||||
|
||||
## Sites
|
||||
@@ -164,6 +166,7 @@ Tracked here so we don't forget. Each lands when we harden the relevant stack.
|
||||
- **MinIO / artifact store** — MLflow uses local volume for now; switch to S3-compatible MinIO sidecar when artifacts grow.
|
||||
- **JupyterHub auth** — target Keycloak OIDC via `oauthenticator.generic.GenericOAuthenticator`.
|
||||
- **WG client routing** — split-tunnel vs full; per-peer `AllowedIPs` policy.
|
||||
- **FROST auth** — currently `BasicAuthProvider` against the USERS table in `frost-db`; swap to Keycloak OIDC via the FROST OIDC plugin when SSO is rolled out.
|
||||
- **MQTT cross-broker shovel** — only if FROST consumers must see RabbitMQ traffic or vice versa.
|
||||
- **Internal PKI** — for cloud-internal hostnames not eligible for Let's Encrypt HTTP-01.
|
||||
- **Backup strategy** — for `sql` (postgres), `influxdb`, `gitea-data`, `jenkins-home`, `mlflow-artifacts`.
|
||||
|
||||
Reference in New Issue
Block a user