RnD/infra
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(cloud): short function-based subdomains + harden keycloak with postgres

Browse Source 
Subdomain rename (Versio side keeps original tool-named hostnames)
- nginx vhosts updated:
    grafana   -> dash.wbd-rd.nl
    gitea     -> git.wbd-rd.nl
    keycloak  -> auth.wbd-rd.nl
    node-red  -> flow.wbd-rd.nl
    mlflow    -> ml.wbd-rd.nl
    jupyter   -> hub.wbd-rd.nl
    portainer -> ops.wbd-rd.nl
    rabbitmq  -> mq.wbd-rd.nl
    jenkins   -> ci.wbd-rd.nl
    mqtt      -> mqtt.wbd-rd.nl   (no Versio conflict assumed)
- nginx-proxy README: bootstrap cert -d list + DNS A-record prereqs updated
- cloud/.env.example: GITEA_ROOT_URL, GRAFANA_ROOT_URL, KEYCLOAK_HOSTNAME

Function-based names are tool-agnostic (a Grafana -> Kibana swap leaves
dash.wbd-rd.nl meaningful) and avoid one-off "*2" suffixes.

Keycloak hardening
- Switch backend from bundled file storage to postgres (keycloak DB
  already provisioned by sql/config/init.d/01-databases.sh).
- KC_HOSTNAME=auth.wbd-rd.nl, KC_PROXY_HEADERS=xforwarded for nginx
  reverse-proxy posture; KC_HTTP_ENABLED=true since nginx terminates TLS.
- Added KC_HOSTNAME_STRICT, KC_HEALTH_ENABLED, KC_METRICS_ENABLED.
- Service joins app + mgmt + data networks (data needed for postgres).
- Mounted config/realms/ for realm-as-code (kc.sh import) — TODO to
  populate once realm + clients are designed.
- README documents the recommended realm structure (wbd realm, one
  client per app with redirect URIs) and the oauth2-proxy approach
  for apps without native OIDC (mlflow, portainer-CE).

cloud
- Uncomment keycloak include in cloud/compose.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
znetsixe
2026-05-21 13:54:57 +02:00
parent 5d95f8bfcc
commit af354a4b9e
15 changed files with 112 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
cloud/.env.example
View File

@@ -17,7 +17,7 @@ WG_SERVER_PUBLIC_HOST=
# Keycloak (admin bootstrap + DB)
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=
KEYCLOAK_HOSTNAME=keycloak.wbd-rd.nl
KEYCLOAK_HOSTNAME=auth.wbd-rd.nl
KEYCLOAK_DB_PASSWORD=
# InfluxDB
@@ -30,7 +30,7 @@ INFLUX_BUCKET=telemetry
# Grafana
GRAFANA_ADMIN_USER=admin
GRAFANA_ADMIN_PASSWORD=
GRAFANA_ROOT_URL=https://grafana.wbd-rd.nl
GRAFANA_ROOT_URL=https://dash.wbd-rd.nl
# SQL (postgres — single point of config)
SQL_DB=config
@@ -47,7 +47,7 @@ POSTFIX_RELAYHOST=
POSTFIX_FROM_DOMAIN=wbd-rd.nl
# Gitea (HTTPS-only; uses sql backend)
GITEA_ROOT_URL=https://gitea.wbd-rd.nl
GITEA_ROOT_URL=https://git.wbd-rd.nl
GITEA_DB_HOST=sql:5432
GITEA_DB_NAME=gitea
GITEA_DB_USER=gitea

2
cloud/compose.yml
View File

@@ -12,7 +12,7 @@ include:
- ../stacks/portainer/compose.yml
# Core identity + VPN
# - ../stacks/wireguard-server/compose.yml
# - ../stacks/keycloak/compose.yml
- ../stacks/keycloak/compose.yml
# Data
# - ../stacks/influxdb/compose.yml
# Apps

8
stacks/keycloak/.env.example
View File

@@ -1,3 +1,9 @@
# Master admin (first-start bootstrap only — change password after first login)
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=
KEYCLOAK_HOSTNAME=
# Public hostname (must match nginx-proxy vhost server_name)
KEYCLOAK_HOSTNAME=auth.wbd-rd.nl
# Postgres backend (DB + role provisioned by sql stack)
KEYCLOAK_DB_PASSWORD=

63
stacks/keycloak/README.md
View File

@@ -1,7 +1,62 @@
# keycloak
Identity provider for SSO across grafana, node-red, gitea, jenkins, portainer.
Identity provider for SSO across all R&D services. **Cloud-only** for now (edges get their own realms once we cover the edge layer).
- **Networks**: `app` (OIDC endpoints for relying apps) + `mgmt` (admin console)
- **Storage**: stub uses bundled file storage; move to `sql` stack before production
- **TODO**: realm + client provisioning (kc.sh or terraform-keycloak), session/token lifetimes, theme branding
- **Public hostname**: `auth.wbd-rd.nl` (reverse-proxied via nginx-proxy at HTTPS → backend HTTP 8080)
- **Networks**: `app` (OIDC endpoints for relying-party apps) + `mgmt` (admin console) + `data` (postgres backend)
- **Backend**: postgres `keycloak` database in `sql` stack — provisioned automatically by `sql/config/init.d/01-databases.sh` on first start
- **Volume**: `keycloak-data` (themes, providers, realm exports)
## First-run bootstrap
`KC_BOOTSTRAP_ADMIN_USERNAME` + `KC_BOOTSTRAP_ADMIN_PASSWORD` create the master-realm admin on first start. **Change the password immediately after first login** via the admin console.
After deployment:
```bash
# 1. Bring it up (sql must be running first)
cd /mnt/d/gitea/RnD/infra/cloud
docker compose up -d sql # if not already up
docker compose up -d keycloak
# 2. Watch logs until you see "Keycloak <version> on JVM started"
docker compose logs -f keycloak
# 3. Browse https://auth.wbd-rd.nl/ → admin console
# (until cert is bootstrapped, https://<cloud-host>:9443 portainer can show logs)
```
## Realm + clients (TODO — design before deploy day 2)
**Recommended structure**: one realm `wbd` containing all R&D apps as separate clients.
| Client ID | App | Redirect URI | Flow |
|---|---|---|---|
| grafana | Grafana | `https://dash.wbd-rd.nl/login/generic_oauth` | code |
| gitea | Gitea | `https://git.wbd-rd.nl/user/oauth2/keycloak/callback` | code |
| node-red | Node-RED | `https://flow.wbd-rd.nl/auth/strategy/callback/` | code |
| jenkins | Jenkins | `https://ci.wbd-rd.nl/securityRealm/finishLogin` | code |
| jupyterhub | JupyterHub | `https://hub.wbd-rd.nl/hub/oauth_callback` | code |
| mlflow | MLflow (via oauth2-proxy) | `https://ml.wbd-rd.nl/oauth2/callback` | code |
| portainer-ce | Portainer (via oauth2-proxy) | `https://ops.wbd-rd.nl/oauth2/callback` | code |
Apps **without native OIDC** (mlflow, portainer-CE) sit behind an `oauth2-proxy` sidecar that nginx `auth_request`s to. That's a TODO stack we'll add when we wire up mlflow / portainer SSO.
## Realm-as-code
Drop exported realm JSON into `config/realms/`. On first start, Keycloak imports anything in `/opt/keycloak/data/import/` if `KC_IMPORT_REALM_DIR` is set or if you run `kc.sh import` manually. Recommended workflow:
1. Configure the realm by hand once in the UI
2. `docker compose exec keycloak /opt/keycloak/bin/kc.sh export --dir /opt/keycloak/data/import --realm wbd`
3. Commit the exported file under `stacks/keycloak/config/realms/`
4. Subsequent fresh deploys auto-import
## TODO
- Realm bootstrap script (provision `wbd` realm + clients above)
- Theme: WBD branding (logo, colors)
- User federation (LDAP from corporate AD, if applicable)
- 2FA policy
- Session / token lifetimes per client
- oauth2-proxy stack for apps without native OIDC
- Realm export → `config/realms/wbd.json` committed

24
stacks/keycloak/compose.yml
View File

@@ -1,25 +1,39 @@
# keycloak — identity / SSO
# Networks: app (apps reach the realm endpoints) + mgmt (admin console)
# Hostname: auth.wbd-rd.nl (reverse-proxied via nginx-proxy on port 8080)
# Networks: app (relying-party endpoints) + mgmt (admin console traffic) + data (postgres backend)
services:
keycloak:
image: quay.io/keycloak/keycloak:26.0
restart: unless-stopped
command: ["start", "--optimized"]
networks: [app, mgmt]
command: ["start"]
networks: [app, mgmt, data]
environment:
# Master admin bootstrap (first start only — change password after first login)
KC_BOOTSTRAP_ADMIN_USERNAME: ${KEYCLOAK_ADMIN}
KC_BOOTSTRAP_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
KC_HOSTNAME: ${KEYCLOAK_HOSTNAME:-}
# Reverse-proxy posture
KC_HOSTNAME: ${KEYCLOAK_HOSTNAME}
KC_HOSTNAME_STRICT: "true"
KC_PROXY_HEADERS: xforwarded
KC_HTTP_ENABLED: "true"
# TODO: external DB (KC_DB=postgres) once sql stack lands
# Postgres backend (DB + role provisioned by sql/config/init.d/01-databases.sh)
KC_DB: postgres
KC_DB_URL: jdbc:postgresql://sql:5432/keycloak
KC_DB_USERNAME: keycloak
KC_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD}
# Misc
KC_HEALTH_ENABLED: "true"
KC_METRICS_ENABLED: "true"
TZ: ${TZ:-Europe/Amsterdam}
volumes:
- keycloak-data:/opt/keycloak/data
- ./config/realms:/opt/keycloak/data/import:ro
networks:
app:
mgmt:
data:
volumes:
keycloak-data:

28
stacks/nginx-proxy/README.md
View File

@@ -64,9 +64,9 @@ docker compose run --rm certbot certonly \
--webroot -w /var/www/certbot \
--email "$LETSENCRYPT_EMAIL" --agree-tos --no-eff-email \
--cert-name infra \
-d grafana.wbd-rd.nl -d gitea.wbd-rd.nl -d keycloak.wbd-rd.nl \
-d nodered.wbd-rd.nl -d mlflow.wbd-rd.nl -d jupyter.wbd-rd.nl \
-d portainer.wbd-rd.nl -d rabbitmq.wbd-rd.nl -d jenkins.wbd-rd.nl \
-d git.wbd-rd.nl -d auth.wbd-rd.nl -d dash.wbd-rd.nl \
-d flow.wbd-rd.nl -d ml.wbd-rd.nl -d hub.wbd-rd.nl \
-d ops.wbd-rd.nl -d mq.wbd-rd.nl -d ci.wbd-rd.nl \
-d mqtt.wbd-rd.nl
# 4. Reload nginx to pick up the real cert
@@ -77,19 +77,19 @@ The certbot sidecar then renews every 12h automatically.
## DNS prereqs (HTTP-01)
Before bootstrap, ensure A records exist in Versio for:
Before bootstrap, ensure A records exist in Versio for the 10 new short subdomains (the canonical tool-named ones — `gitea.wbd-rd.nl`, `grafana.wbd-rd.nl`, etc. — stay pointed at the existing Versio stack during the transition):
```
grafana.wbd-rd.nl A <cloud-public-ip>
gitea.wbd-rd.nl A <cloud-public-ip>
keycloak.wbd-rd.nl A <cloud-public-ip>
nodered.wbd-rd.nl A <cloud-public-ip>
mlflow.wbd-rd.nl A <cloud-public-ip>
jupyter.wbd-rd.nl A <cloud-public-ip>
portainer.wbd-rd.nl A <cloud-public-ip>
rabbitmq.wbd-rd.nl A <cloud-public-ip>
jenkins.wbd-rd.nl A <cloud-public-ip>
mqtt.wbd-rd.nl A <cloud-public-ip>
git.wbd-rd.nl A <cloud-public-ip> # gitea (new)
auth.wbd-rd.nl A <cloud-public-ip> # keycloak
dash.wbd-rd.nl A <cloud-public-ip> # grafana (new)
flow.wbd-rd.nl A <cloud-public-ip> # node-red (new)
ml.wbd-rd.nl A <cloud-public-ip> # mlflow
hub.wbd-rd.nl A <cloud-public-ip> # jupyterhub
ops.wbd-rd.nl A <cloud-public-ip> # portainer
mq.wbd-rd.nl A <cloud-public-ip> # rabbitmq mgmt UI
ci.wbd-rd.nl A <cloud-public-ip> # jenkins
mqtt.wbd-rd.nl A <cloud-public-ip> # MQTT-TLS broker
```
## TODO

2
stacks/nginx-proxy/config/conf.d/gitea.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name gitea.wbd-rd.nl;
server_name git.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/grafana.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name grafana.wbd-rd.nl;
server_name dash.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/jenkins.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name jenkins.wbd-rd.nl;
server_name ci.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/jupyter.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name jupyter.wbd-rd.nl;
server_name hub.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/keycloak.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name keycloak.wbd-rd.nl;
server_name auth.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/mlflow.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name mlflow.wbd-rd.nl;
server_name ml.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/nodered.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name nodered.wbd-rd.nl;
server_name flow.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/portainer.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name portainer.wbd-rd.nl;
server_name ops.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;

2
stacks/nginx-proxy/config/conf.d/rabbitmq.conf
View File

@@ -1,7 +1,7 @@
server {
listen 443 ssl;
http2 on;
server_name rabbitmq.wbd-rd.nl;
server_name mq.wbd-rd.nl;
ssl_certificate /etc/letsencrypt/live/infra/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/infra/privkey.pem;