EVOLV/docs/DEVELOPER_GUIDE.md

# EVOLV Developer Guide: Creating a New Node

This guide walks through creating a new EVOLV node from scratch, following the project's three-layer architecture pattern.

## Prerequisites

- **Node.js** (v18+)
- **Node-RED** installed globally or as a dev dependency
- Clone the repo and run `npm install` in the root (no build step required)
- The `generalFunctions` submodule must be initialized (`git submodule update --init`)

## Architecture Overview

Every EVOLV node follows a **three-layer pattern**:

| Layer | File | Responsibility |
|-------|------|---------------|
| 1 - Wrapper | `<name>.js` | Registers the node type with Node-RED, sets up HTTP endpoints for menus/config |
| 2 - Node Adapter | `src/nodeClass.js` | Bridges Node-RED with domain logic: config loading, tick loop, input routing, lifecycle |
| 3 - Domain Logic | `src/specificClass.js` | Pure business logic with no Node-RED dependencies |

Plus a UI definition: `<name>.html` for the Node-RED editor.

## Step-by-Step: Creating a New Node

We will create a hypothetical `flowMeter` node as an example.

### Step 1: Create Directory Structure

```
nodes/flowMeter/
  flowMeter.js          # Layer 1 - wrapper
  flowMeter.html        # UI definition
  src/
    nodeClass.js        # Layer 2 - node adapter
    specificClass.js    # Layer 3 - domain logic
  test/
    specificClass.test.js
```

### Step 2: Write the Wrapper (`flowMeter.js`)

The wrapper registers the node type with Node-RED and exposes HTTP endpoints for dynamic menus and config data.

```js
const nameOfNode = 'flowMeter';
const nodeClass = require('./src/nodeClass.js');
const { MenuManager, configManager } = require('generalFunctions');

module.exports = function(RED) {
  // Register the node type
  RED.nodes.registerType(nameOfNode, function(config) {
    RED.nodes.createNode(this, config);
    this.nodeClass = new nodeClass(config, RED, this, nameOfNode);
  });

  // Menu endpoint (dynamic dropdowns in the editor UI)
  const menuMgr = new MenuManager();
  RED.httpAdmin.get(`/${nameOfNode}/menu.js`, (req, res) => {
    try {
      const script = menuMgr.createEndpoint(nameOfNode, ['asset', 'logger', 'position']);
      res.type('application/javascript').send(script);
    } catch (err) {
      res.status(500).send(`// Error generating menu: ${err.message}`);
    }
  });

  // Config data endpoint (exposes JSON config to the editor)
  const cfgMgr = new configManager();
  RED.httpAdmin.get(`/${nameOfNode}/configData.js`, (req, res) => {
    try {
      const script = cfgMgr.createEndpoint(nameOfNode);
      res.type('application/javascript').send(script);
    } catch (err) {
      res.status(500).send(`// Error generating configData: ${err.message}`);
    }
  });
};
```

Key points:
- `nameOfNode` must match the file name, the `registerType` name, and the HTML `data-template-name`.
- Menu categories (`['asset', 'logger', 'position']`) control which shared UI sections appear.
- The config endpoint is optional if you do not need dynamic config in the editor.

### Step 3: Write `nodeClass.js` (Node Adapter)

This class bridges Node-RED's API with your domain logic.

```js
const { outputUtils, configManager } = require('generalFunctions');
const Specific = require('./specificClass');

class nodeClass {
  constructor(uiConfig, RED, nodeInstance, nameOfNode) {
    this.node = nodeInstance;
    this.RED = RED;
    this.name = nameOfNode;

    this._loadConfig(uiConfig);
    this._setupSpecificClass();
    this._bindEvents();
    this._registerChild();
    this._startTickLoop();
    this._attachInputHandler();
    this._attachCloseHandler();
  }

  _loadConfig(uiConfig) {
    const cfgMgr = new configManager();
    this.defaultConfig = cfgMgr.getConfig(this.name);

    // buildConfig merges base sections (general, asset, functionality)
    // with node-specific domain config from the UI
    this.config = cfgMgr.buildConfig(this.name, uiConfig, this.node.id, {
      // Add domain-specific config sections here:
      flowSettings: {
        maxFlow: uiConfig.maxFlow,
        pipeSize: uiConfig.pipeSize,
      },
    });

    this._output = new outputUtils();
  }

  _setupSpecificClass() {
    this.source = new Specific(this.config);
    this.node.source = this.source;
  }

  _bindEvents() {
    // Subscribe to domain events for Node-RED status display
    this.source.emitter.on('flowUpdate', (val) => {
      this.node.status({ fill: 'green', shape: 'dot', text: `${val} ${this.config.general.unit}` });
    });
  }

  _registerChild() {
    // Delayed to avoid Node-RED startup race conditions
    setTimeout(() => {
      this.node.send([
        null,
        null,
        {
          topic: 'registerChild',
          payload: this.node.id,
          positionVsParent: this.config?.functionality?.positionVsParent || 'atEquipment',
          distance: this.config?.functionality?.distance || null,
        },
      ]);
    }, 100);
  }

  _startTickLoop() {
    setTimeout(() => {
      this._tickInterval = setInterval(() => this._tick(), 1000);
    }, 1000);
  }

  _tick() {
    this.source.tick();
    const raw = this.source.getOutput();
    const processMsg = this._output.formatMsg(raw, this.config, 'process');
    const influxMsg  = this._output.formatMsg(raw, this.config, 'influxdb');
    this.node.send([processMsg, influxMsg]);
  }

  _attachInputHandler() {
    this.node.on('input', (msg, send, done) => {
      switch (msg.topic) {
        case 'measurement':
          if (typeof msg.payload === 'number') {
            this.source.inputValue = parseFloat(msg.payload);
          }
          break;
        // Add more input topics as needed
      }
      done();
    });
  }

  _attachCloseHandler() {
    this.node.on('close', (done) => {
      clearInterval(this._tickInterval);
      done();
    });
  }
}

module.exports = nodeClass;
```

Essential methods every `nodeClass` must implement:
- `_loadConfig()` -- merges default JSON config with UI config via `configManager.buildConfig()`
- `_setupSpecificClass()` -- instantiates the domain class
- `_registerChild()` -- sends a `registerChild` message on output port 2 (parent)
- `_startTickLoop()` -- drives periodic output at 1-second intervals
- `_tick()` -- calls `source.getOutput()` and formats via `outputUtils.formatMsg()`
- `_attachInputHandler()` -- routes incoming `msg.topic` to domain methods
- `_attachCloseHandler()` -- clears timers on node removal

### Step 4: Write `specificClass.js` (Domain Logic)

This is pure JavaScript with no Node-RED dependencies.

```js
const EventEmitter = require('events');
const { logger, configUtils, configManager, MeasurementContainer } = require('generalFunctions');

class FlowMeter {
  constructor(config = {}) {
    this.emitter = new EventEmitter();
    this.configManager = new configManager();
    this.defaultConfig = this.configManager.getConfig('flowMeter');
    this.configUtils = new configUtils(this.defaultConfig);
    this.config = this.configUtils.initConfig(config);

    this.logger = new logger(
      this.config.general.logging.enabled,
      this.config.general.logging.logLevel,
      this.config.general.name
    );

    // MeasurementContainer stores typed/positioned values
    this.measurements = new MeasurementContainer({
      autoConvert: true,
      windowSize: this.config.smoothing?.smoothWindow || 10,
    });
    this.measurements.setChildId(this.config.general.id);
    this.measurements.setChildName(this.config.general.name);

    // Domain state
    this.currentFlow = 0;
  }

  tick() {
    // Called every 1 second by nodeClass._tick()
    this.calculateFlow();
  }

  calculateFlow() {
    // Your domain logic here
    const flow = this.currentFlow;

    // Store in MeasurementContainer using the chainable API:
    //   .type(measType).variant(variant).position(pos).value(val, timestamp, unit)
    this.measurements
      .type(this.config.asset.type)
      .variant('measured')
      .position(this.config.functionality.positionVsParent)
      .value(flow, Date.now(), this.config.asset.unit);

    this.emitter.emit('flowUpdate', flow);
  }

  getOutput() {
    return {
      flow: this.currentFlow,
    };
  }
}

module.exports = FlowMeter;
```

Key patterns:
- Always create an `emitter` (EventEmitter) -- parents subscribe to child events through it.
- Use `MeasurementContainer` for storing measurements. The chainable API follows the pattern: `measurements.type(t).variant(v).position(p).value(val, timestamp, unit)`.
- Expose `tick()` and `getOutput()` for the node adapter to call.
- Use `logger` instead of `console.log`.

### Step 5: Write the HTML (UI Definition)

```html
<script src="/flowMeter/menu.js"></script>
<script src="/flowMeter/configData.js"></script>

<script>
  RED.nodes.registerType("flowMeter", {
    category: "EVOLV",
    color: "#a9daee",          // S88 Control Module color
    defaults: {
      name: { value: "flowMeter" },
      maxFlow: { value: 100, required: true },
      pipeSize: { value: 0.1, required: true },
      // Standard fields (asset, logger, position)
      uuid: { value: "" },
      supplier: { value: "" },
      category: { value: "" },
      assetType: { value: "" },
      model: { value: "" },
      unit: { value: "" },
      enableLog: { value: false },
      logLevel: { value: "error" },
      positionVsParent: { value: "" },
      positionIcon: { value: "" },
    },
    inputs: 1,
    outputs: 3,
    outputLabels: ["process", "dbase", "parent"],
    icon: "font-awesome/fa-tachometer",
    label: function() {
      return this.name || "flowMeter";
    },
    oneditprepare: function() {
      // Wait for shared menu system to initialize
      const waitForMenuData = () => {
        if (window.EVOLV?.nodes?.flowMeter?.initEditor) {
          window.EVOLV.nodes.flowMeter.initEditor(this);
        } else {
          setTimeout(waitForMenuData, 50);
        }
      };
      waitForMenuData();
    },
    oneditsave: function() {
      if (window.EVOLV?.nodes?.flowMeter?.assetMenu?.saveEditor) {
        window.EVOLV.nodes.flowMeter.assetMenu.saveEditor(this);
      }
      if (window.EVOLV?.nodes?.flowMeter?.loggerMenu?.saveEditor) {
        window.EVOLV.nodes.flowMeter.loggerMenu.saveEditor(this);
      }
      if (window.EVOLV?.nodes?.flowMeter?.positionMenu?.saveEditor) {
        window.EVOLV.nodes.flowMeter.positionMenu.saveEditor(this);
      }
    },
  });
</script>

<script type="text/html" data-template-name="flowMeter">
  <div class="form-row">
    <label for="node-input-maxFlow"><i class="fa fa-arrows-v"></i> Max Flow</label>
    <input type="number" id="node-input-maxFlow" placeholder="100" />
  </div>
  <div class="form-row">
    <label for="node-input-pipeSize"><i class="fa fa-circle-o"></i> Pipe Size (m)</label>
    <input type="number" id="node-input-pipeSize" placeholder="0.1" step="0.01" />
  </div>
  <!-- Shared UI sections injected by MenuManager -->
  <div id="asset-fields-placeholder"></div>
  <div id="logger-fields-placeholder"></div>
  <div id="position-fields-placeholder"></div>
</script>

<script type="text/html" data-help-name="flowMeter">
  <p><b>Flow Meter Node</b>: Measures and processes flow data.</p>
</script>
```

**S88 color scheme** (pick based on your node's hierarchy level):

| S88 Level | Color | Text Color |
|-----------|-------|-----------|
| Area | `#0f52a5` | white |
| Process Cell | `#0c99d9` | white |
| Unit | `#50a8d9` | black |
| Equipment | `#86bbdd` | black |
| Control Module | `#a9daee` | black |

All nodes must have **3 outputs**: `[process, dbase, parent]`.

### Step 6: Create Config JSON Schema

Create `nodes/generalFunctions/src/configs/flowMeter.json`. This defines defaults and validation rules for every config property. The `configManager` reads this file by node name.

```json
{
  "general": {
    "name": {
      "default": "FlowMeter",
      "rules": { "type": "string", "description": "Human-readable name." }
    },
    "id": {
      "default": null,
      "rules": { "type": "string", "nullable": true }
    },
    "unit": {
      "default": "m3/h",
      "rules": { "type": "string" }
    },
    "logging": {
      "logLevel": {
        "default": "info",
        "rules": {
          "type": "enum",
          "values": [
            { "value": "debug" }, { "value": "info" },
            { "value": "warn" }, { "value": "error" }
          ]
        }
      },
      "enabled": { "default": true, "rules": { "type": "boolean" } }
    }
  },
  "functionality": {
    "softwareType": { "default": "flowMeter", "rules": { "type": "string" } },
    "role": { "default": "Sensor", "rules": { "type": "string" } },
    "positionVsParent": {
      "default": "atEquipment",
      "rules": {
        "type": "enum",
        "values": [
          { "value": "atEquipment" }, { "value": "upstream" }, { "value": "downstream" }
        ]
      }
    }
  },
  "asset": {
    "supplier": { "default": "Unknown", "rules": { "type": "string" } },
    "category": { "default": "sensor", "rules": { "type": "string" } },
    "type": { "default": "flow", "rules": { "type": "string" } },
    "model": { "default": "Unknown", "rules": { "type": "string" } },
    "unit": { "default": "m3/h", "rules": { "type": "string" } }
  }
}
```

Each property has a `default` value and a `rules` object specifying the type (`string`, `number`, `boolean`, `enum`, `object`), optional constraints (`min`, `max`, `nullable`), and a description.

### Step 7: Register with `package.json`

Add your node to the root `package.json` under `node-red.nodes`:

```json
{
  "node-red": {
    "nodes": {
      "flowMeter": "nodes/flowMeter/flowMeter.js"
    }
  }
}
```

Restart Node-RED to pick up the new node.

### Step 8: Add Tests

Create `nodes/flowMeter/test/specificClass.test.js`. Tests target Layer 3 (domain logic) directly, without Node-RED.

```js
const FlowMeter = require('../src/specificClass');

function makeConfig(overrides = {}) {
  const base = {
    general: { name: 'TestFlow', id: 'test-1', logging: { enabled: false, logLevel: 'error' } },
    functionality: { softwareType: 'flowMeter', role: 'sensor', positionVsParent: 'atEquipment' },
    asset: { category: 'sensor', type: 'flow', model: 'test', supplier: 'Test', unit: 'm3/h' },
  };
  for (const key of Object.keys(overrides)) {
    base[key] = typeof overrides[key] === 'object' ? { ...base[key], ...overrides[key] } : overrides[key];
  }
  return base;
}

describe('FlowMeter specificClass', () => {
  it('should create an instance', () => {
    const fm = new FlowMeter(makeConfig());
    expect(fm).toBeDefined();
  });

  it('should return output with expected keys', () => {
    const fm = new FlowMeter(makeConfig());
    const out = fm.getOutput();
    expect(out).toHaveProperty('flow');
  });

  it('tick() should not throw', () => {
    const fm = new FlowMeter(makeConfig());
    expect(() => fm.tick()).not.toThrow();
  });
});
```

Run tests with: `npm test` (uses Jest). The project also supports `node:test` for basic smoke tests.

**Test organization conventions** (based on existing nodes):
- `test/specificClass.test.js` -- unit tests for domain logic
- `test/basic/*.test.js` -- structural/smoke tests (module loads, exports exist)
- `test/edge/*.test.js` -- edge case and boundary tests
- `test/integration/*.test.js` -- multi-component integration tests

## Key APIs Reference

### MeasurementContainer

Chainable storage for typed, positioned measurements. Used by every domain class.

```js
const { MeasurementContainer } = require('generalFunctions');
const mc = new MeasurementContainer({ autoConvert: true, windowSize: 10 });
mc.setChildId('node-id');
mc.setChildName('PT-001');

// Store a value
mc.type('pressure').variant('measured').position('upstream').value(3.5, Date.now(), 'bar');

// Parents subscribe to events via mc.emitter
mc.emitter.on('pressure.measured.upstream', (data) => { /* { value, unit, ... } */ });
```

The event name follows the pattern: `{type}.{variant}.{position}`.

### configManager.buildConfig()

Merges the JSON config schema defaults with UI-provided values. Called in `nodeClass._loadConfig()`.

```js
const { configManager } = require('generalFunctions');
const cfgMgr = new configManager();
const defaults = cfgMgr.getConfig('myNode');             // loads myNode.json
const config = cfgMgr.buildConfig('myNode', uiConfig, nodeId, domainOverrides);
```

### POSITIONS

Canonical position constants. Use these instead of hardcoded strings.

```js
const { POSITIONS } = require('generalFunctions');
// POSITIONS.UPSTREAM      = 'upstream'
// POSITIONS.DOWNSTREAM    = 'downstream'
// POSITIONS.AT_EQUIPMENT  = 'atEquipment'
// POSITIONS.DELTA         = 'delta'
```

### outputUtils.formatMsg()

Formats raw output data into either `process` or `influxdb` messages. Only sends changed fields.

```js
const { outputUtils } = require('generalFunctions');
const out = new outputUtils();
const processMsg = out.formatMsg(rawData, config, 'process');
const influxMsg  = out.formatMsg(rawData, config, 'influxdb');
node.send([processMsg, influxMsg]);
```

### childRegistrationUtils

Manages parent-child node relationships. Parents use this to accept child registrations.

```js
const { childRegistrationUtils } = require('generalFunctions');
const regUtils = new childRegistrationUtils(this); // 'this' is the parent specificClass
// Called when a child's registerChild message arrives:
regUtils.registerChild(childSource, positionVsParent, distance);
```

The parent's `registerChild()` method subscribes to the child's `measurements.emitter` events for data propagation.

## Common Patterns

### Parent-Child Registration

1. Child sends `{ topic: 'registerChild', payload: nodeId, positionVsParent }` on output port 2.
2. Parent's `nodeClass._attachInputHandler()` catches `msg.topic === 'registerChild'`.
3. Parent calls `childRegistrationUtils.registerChild(child, position)`.
4. Parent subscribes to child's `measurements.emitter` events (e.g., `'flow.measured.downstream'`).
5. When the child updates a measurement, the parent's listener fires and updates its own state.

### Tick Loop

Every node runs a 1-second tick loop that:
1. Calls `source.tick()` to advance domain logic.
2. Calls `source.getOutput()` for current state.
3. Formats into `process` and `influxdb` messages via `outputUtils.formatMsg()`.
4. Sends on ports 0 (process) and 1 (dbase).

The tick loop starts with a 1-second delay to allow child registration to complete.

### Three-Output Format

All nodes send on three ports: `node.send([processMsg, influxMsg, parentMsg])`.

| Port | Purpose | When |
|------|---------|------|
| 0 | Process data for downstream nodes | Every tick (if changed) |
| 1 | InfluxDB line protocol for persistence | Every tick (if changed) |
| 2 | Parent registration/control messages | On startup; on parent commands |

### Event-Driven Communication

Nodes communicate via `EventEmitter`, not Node-RED wires:
- `measurements.emitter` fires `{type}.{variant}.{position}` events.
- Parents listen to children's emitters after registration.
- The `emitter` on the specificClass itself is used for internal state changes (e.g., updating Node-RED node status display).

## Checklist

Before submitting a new node, verify:

- [ ] Three-layer structure: wrapper, nodeClass, specificClass
- [ ] Config JSON in `generalFunctions/src/configs/<name>.json`
- [ ] Registered in root `package.json` under `node-red.nodes`
- [ ] HTML registers under category `'EVOLV'` with correct S88 color
- [ ] Three outputs: `[process, dbase, parent]`
- [ ] Uses `logger` (not `console.log`)
- [ ] Uses `MeasurementContainer` for measurement storage
- [ ] Uses `outputUtils.formatMsg()` for output formatting
- [ ] Tick loop cleans up in `_attachCloseHandler()`
- [ ] Tests exist for specificClass domain logic
- [ ] Node-specific UI fields plus shared placeholders (asset, logger, position)