EVOLV/wiki/Telemetry.md

# Telemetry

![code-ref](https://img.shields.io/badge/code--ref-9ab9f6b-blue)
![source](https://img.shields.io/badge/source-CONTRACTS.md_%C2%A710-orange)

> [!NOTE]
> Every node sends on three output ports: Port 0 (process data), Port 1 (InfluxDB line protocol), Port 2 (registration / control plumbing). All output is formatted by `outputUtils.formatMsg` with delta compression: only changed fields are sent each tick. InfluxDB cardinality discipline: tags = identity (low cardinality), fields = numbers. Source of truth: `.claude/refactor/CONTRACTS.md` §10.

---

## Three-port model

```mermaid
flowchart LR
    sc["specificClass — 'output-changed' or tick()"]
    ou["outputUtils.formatMsg — delta-compress"]
    p0[("Port 0 — process")]
    p1[("Port 1 — InfluxDB line")]
    p2[("Port 2 — register / control")]
    dl["Downstream Node-RED — dashboards, functions"]
    influx[("InfluxDB")]
    parent["Parent EVOLV node"]

    sc -- getOutput() --> ou
    ou --> p0 --> dl
    ou --> p1 --> influx
    sc -. child.register .-> p2 --> parent

    class sc tier3
    class ou tier2
    class p0 p0c
    class p1 p1c
    class p2 p2c
    class dl,parent dn
    class influx ext

    classDef tier3 fill:#50a8d9,color:#000
    classDef tier2 fill:#86bbdd,color:#000
    classDef p0c fill:#0c99d9,color:#fff
    classDef p1c fill:#50a8d9,color:#000
    classDef p2c fill:#a9daee,color:#000
    classDef dn fill:#dddddd,color:#000
    classDef ext fill:#fff2cc,color:#000
```

### Port summary

| Port | Carries | Format | Configured by | Trigger |
|:---|:---|:---|:---|:---|
| 0 (process) | Snapshot of changed measurement / state keys | JSON delta object | `outputUtils.formatMsg(..., 'process')` via `config.output.process` | `'output-changed'` on the emitter, or each tick |
| 1 (telemetry) | Numeric fields only — time-series | InfluxDB line protocol | `outputUtils.formatMsg(..., 'influxdb')` via `config.output.dbase` | Same trigger as Port 0 |
| 2 (register) | `child.register` plus internal control msgs | Plain object | `BaseNodeAdapter` init | Once 100ms after init; on demand |

---

## Port 0 — Process data (delta-compressed)

Purpose: feed downstream Node-RED logic such as dashboards, alarms, control function nodes.

Shape: `msg.payload` contains only keys that changed since the last tick. Consumers must cache and merge.

> [!IMPORTANT]
> Why delta compression: at 1 Hz with 50+ fields per node, full snapshots flood downstream nodes and dashboards. A typical delta carries 0–5 fields per tick.

Example output, one rotatingMachine tick:

```json
{
  "topic": "rotatingMachine#pump-A",
  "payload": {
    "flow.predicted.downstream.default": 12.4,
    "predictionConfidence": 0.87
  }
}
```

Most other fields (state, pressure, mode, ...) didn't change this tick and are omitted.

Trigger: `outputUtils.checkForChanges()` compares the current `getOutput()` against the previous snapshot. No change means no send.

---

## Port 1 — InfluxDB line protocol

Purpose: time-series storage for trending, regulatory reporting, ML training.

Shape: `msg.payload` is a string (or array of strings) in InfluxDB line protocol:

```
<measurement>,<tag-set> <field-set> <timestamp-ns>
```

Example, rotatingMachine telemetry:

```
rotatingMachine,id=pump-A,softwareType=rotatingMachine flow_predicted_downstream=12.4,power_measured_atequipment=18.2 1714752000000000000
```

### Conventions

| Element | Rule |
|:---|:---|
| Measurement (table name) | The node's `softwareType`, lowercase |
| Tag set | Low-cardinality identity: `id`, `softwareType`, location-style tags. Never raw measurement values |
| Field set | Numeric values only. Keys flatten `<type>_<variant>_<position>` (underscore, not dot &mdash; InfluxDB constraint) |
| Timestamp | Nanoseconds. Set by `outputUtils` from the node's clock |

See [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) for cardinality discipline.

---

## Port 2 &mdash; Registration / control

Purpose: upward `child.register` at startup; later, internal control msgs.

Shape:

```json
{
  "topic": "child.register",
  "payload": {
    "ref": "<node reference>",
    "softwareType": "machine",
    "config": { }
  }
}
```

Trigger: the nodeClass adapter emits `child.register` on init if a parent is configured. Parent's `commandRegistry` dispatches into `ChildRouter.onRegister(...)`.

The legacy alias `registerChild` is still accepted; it logs a one-time deprecation warning on first use.

---

## The output composition pipeline

```mermaid
sequenceDiagram
    autonumber
    participant tick as Tick (1 Hz) or event source
    participant sc as specificClass
    participant mc as MeasurementContainer
    participant ou as outputUtils
    participant ports as Ports 0 / 1

    tick->>sc: tick() OR emit('output-changed')
    sc->>sc: concern modules update mc + state
    sc->>ou: getOutput() snapshot
    ou->>ou: diff vs last snapshot
    alt no change
        ou-->>sc: skip
    else change
        ou->>ports: Port 0 &mdash; JSON delta
        ou->>ports: Port 1 &mdash; line protocol
    end
```

> [!CAUTION]
> Never write directly to `node.send` from specificClass. Go through `outputUtils.formatMsg`. Direct sends bypass delta compression and flood downstream nodes.

---

## InfluxDB schema layout (recommended)

| InfluxDB element | Maps to |
|:---|:---|
| Database / bucket | One per plant, or per environment: `evolv_dev`, `evolv_prod` |
| Measurement (table) | Node `softwareType` |
| Tags | `id`, `softwareType`, `area`, `processCell`, `unit` (for hierarchical drill-down) |
| Fields | Numeric series &mdash; every numeric key from `getOutput()`, flattened with `_` |
| Retention &mdash; hot | 7 days at 1 s |
| Retention &mdash; cold | 1 year at 1 min downsample |

> [!WARNING]
> Cardinality discipline. Keep tag sets stable. Do not put `state` (string) as a tag &mdash; emit it as a field with a code (`state_code=2`). High-cardinality tags fragment InfluxDB's index and degrade query performance dramatically.

See [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) for full guidance.

---

## FlowFuse dashboard wiring

Port 0 is the natural source for FlowFuse `ui-chart` widgets &mdash; the delta-compressed JSON maps cleanly to `msg.topic` (series label) plus `msg.payload` (y-value).

Layout rule (from `.claude/rules/node-red-flow-layout.md` §4):

- One chart per metric type (one for flow, one for power, one for level).
- A trend-feeder function splits Port-0 deltas into per-series outputs.
- Each output wires to one chart. The chart's `category: "topic"` and `categoryType: "msg"` plot one series per unique `msg.topic`.

```mermaid
flowchart LR
    p0[("Port 0")]
    split["trend-feeder &mdash; function (N outputs)"]
    chart1["ui-chart: flow"]
    chart2["ui-chart: power"]

    p0 --> split
    split --> chart1
    split --> chart2

    class p0 p0c
    class split tier2
    class chart1,chart2 neutral

    classDef p0c fill:#0c99d9,color:#fff
    classDef tier2 fill:#86bbdd,color:#000
    classDef neutral fill:#dddddd
```

See [FlowFuse ui-chart manual](Manual-NodeRED-Flowfuse-Ui-Chart-Manual) for the required chart properties.

---

## Grafana dashboard provisioning

`dashboardAPI` consumes registrations and emits Grafana dashboard JSON via HTTP.

```mermaid
flowchart LR
    evolv["EVOLV node (any softwareType)"]
    dash[dashboardAPI]
    grafana[("Grafana HTTP API &mdash; POST /api/dashboards/db")]

    evolv -- child.register --> dash
    dash -- composed JSON --> grafana

    class evolv tier3
    class dash util
    class grafana ext

    classDef tier3 fill:#50a8d9,color:#000
    classDef util fill:#dddddd
    classDef ext fill:#fff2cc,color:#000
```

dashboardAPI looks up a template per softwareType (in `nodes/dashboardAPI/src/config/templates/`), substitutes the node's id + tags, and POSTs an upsert. Bearer-token auth is supported via `config.grafanaConnector.bearerToken`.

---

## Debug recipes

| Symptom | First thing to check |
|:---|:---|
| InfluxDB rows missing for a node | Port 1 wired to an `influxdb out` node? Tap Port 1 with a debug node to verify line-protocol output |
| Dashboard widgets stuck on `n/a` | Port 0 reaching the trend-feeder? Many widgets need `msg.topic` set for series labelling |
| `child.register` not arriving | Tap Port 2 with debug. Confirm parent's `commandRegistry` accepts `child.register` (or `registerChild` alias) |
| Too many InfluxDB writes | Likely a tick-driven debug node bypassed the delta filter. Confirm `outputUtils.checkForChanges()` is firing |
| Grafana dashboard not created on boot | Inspect dashboardAPI's HTTP response. Check bearer token + base URL in its config |
| High cardinality alarm in InfluxDB | A string value is being written as a tag (probably `state` or similar). Move it to a field |

> [!CAUTION]
> Never ship `enableLog: 'debug'` in a demo. Fills the container log within seconds and obscures real errors. Use only for live debugging.

---

## Related pages

| Page | Why |
|:---|:---|
| [Architecture](Architecture) | Output port wiring in the three-tier model |
| [Topic Conventions](Topic-Conventions) | What topics map to what fields |
| [Topology Patterns](Topology-Patterns) | Typical telemetry flows |
| [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) | Cardinality discipline |
| [FlowFuse ui-chart manual](Manual-NodeRED-Flowfuse-Ui-Chart-Manual) | Required chart properties |