prototype: flows:started + diff distinguishes dashboardAPI subtree #32

New Issue

vps1_gitea_admin · 2026-05-26T15:13:01Z

vps1_gitea_admin commented

2026-05-26 15:13:01 +00:00

Type: spike / prototype

Depends on: none

Estimate: S (½ day)

Slice — layers touched

Node-RED runtime → dashboardAPI adapter → throwaway test harness.

Context

PRD Open Question O-1 (prd). The flows:started runtime event is undocumented but readable in Node-RED source; the payload's {type, diff} shape needs verifying with real deploys before F-1 can be implemented.

Scope

Spin up a stub Node-RED node that subscribes to RED.events.on('flows:started') and logs type and diff.added/changed/removed.
Run three deploy types against a flow with two tabs (one containing the stub, one unrelated): full deploy, nodes-only deploy on the other tab, modified-flows deploy that touches a sibling of the stub.
Document which fields in diff cleanly identify "this subtree changed" vs. "an unrelated flow changed."
Code lives in .prototypes/ and is gitignored. Findings written back as a comment on this issue.

Out of scope

Productionizing the subscription — that's the dependent issue.
Handling nodes-started (deprecated per research).

Acceptance criteria

Output of all 3 deploy-type tests recorded in this issue.
Decision recorded: "the de-dupe predicate is diff.<field>.includes(<our nodeId>) || diff.<field>.includes(<any child id>)" — or, if the diff doesn't carry enough info, the fallback predicate.
If fallback is needed, name it (e.g. "always regen; rely on idempotency").

Notes

Node-RED source: packages/.../runtime/lib/flows/index.js (~lines 483–640). Run via /prototype skill.

**Type:** spike / prototype **Depends on:** none **Estimate:** S (½ day) ## Slice — layers touched Node-RED runtime → dashboardAPI adapter → throwaway test harness. ## Context PRD Open Question O-1 ([prd](../src/branch/development/docs/prd/dashboardapi-graph-aware-grafana-generator.md)). The `flows:started` runtime event is undocumented but readable in Node-RED source; the payload's `{type, diff}` shape needs verifying with real deploys before F-1 can be implemented. ## Scope - Spin up a stub Node-RED node that subscribes to `RED.events.on('flows:started')` and logs `type` and `diff.added/changed/removed`. - Run three deploy types against a flow with two tabs (one containing the stub, one unrelated): full deploy, nodes-only deploy on the *other* tab, modified-flows deploy that touches a sibling of the stub. - Document which fields in `diff` cleanly identify "this subtree changed" vs. "an unrelated flow changed." - Code lives in `.prototypes/` and is gitignored. Findings written back as a comment on this issue. ## Out of scope - Productionizing the subscription — that's the dependent issue. - Handling `nodes-started` (deprecated per research). ## Acceptance criteria - [ ] Output of all 3 deploy-type tests recorded in this issue. - [ ] Decision recorded: "the de-dupe predicate is `diff.<field>.includes(<our nodeId>) || diff.<field>.includes(<any child id>)`" — or, if the diff doesn't carry enough info, the fallback predicate. - [ ] If fallback is needed, name it (e.g. "always regen; rely on idempotency"). ## Notes Node-RED source: `packages/.../runtime/lib/flows/index.js` (~lines 483–640). Run via `/prototype` skill.

vps1_gitea_admin referenced this issue

2026-05-26 15:13:01 +00:00

Skip regen when diff shows no subtree change #36

vps1_gitea_admin commented

2026-05-26 15:22:27 +00:00

Spike findings: confirmed — use `flows:started` + `diff` as designed

Verdict: confirmed.

What I did

Installed a throwaway custom Node-RED node (.prototypes/spike-flows-started/, gitignored) into the running evolv-nodered container that subscribes to flows:starting/started/stopping/stopped and appends each event to /data/spike-events.log with the full diff content. Then drove 5 deploys via the admin API (POST /flows with Node-RED-API-Version: v2 and Node-RED-Deployment-Type: full|nodes|flows) and inspected the resulting payloads.

diff payload shape

payload.diff is always present (on deploys; empty on cold startup) with exactly these keys:

{added, changed, removed, rewired, linked, flowChanged}

All six are arrays of node ids (tab nodes included). On startup-from-cold the whole diff object is empty (no keys).

Per-deploy-type evidence

Two test tabs (Spike A, Spike B) each with a debug node. Ran 5 deploys:

Step	Action	type	`diff.added`	`diff.changed`
1	Add Spike A + Spike B tabs + debug nodes	full	`[tab_a, tab_b, dbg_a, dbg_b]`	`[tab_a, tab_b]`
2	Rename `dbg_b` only	nodes	`[]`	`[dbg_b, tab_b]`
3	Rename `dbg_b` again	flows	`[]`	`[dbg_b, tab_b]`
4	Rename `dbg_a` (different tab)	flows	`[]`	`[dbg_a, tab_a]`
5	Deploy with no logical change	full	`[]`	`[]`

removed, rewired, linked, flowChanged were [] in every step of this test.

Key observations

Event fires on every deploy regardless of type. Type is exposed via payload.type ∈ {"full","nodes","flows"}.
The diff is per-deploy-type-independent — same edit produces the same diff on Step 2 (nodes) and Step 3 (flows). The deploy type controls what Node-RED restarts, not what diff reports.
Editing one node N includes both N's id AND its tab's id in diff.changed. Tabs land in the diff when any of their nodes change. So tab-id-based predicates over-trigger if dashboardAPI shares a tab with unrelated work.
No-op deploys produce empty diff arrays — a legitimate outcome: "no-diff" for the F-1 log line.
linked and flowChanged were never populated in these tests — likely fire for cross-tab link nodes or whole-flow imports. Not load-bearing for the predicate.

Predicate for "did my subtree change"

Use the dashboardAPI's node id + the ids of all registered children (not tab ids — avoid the tab false-positive):

function subtreeChanged(diff, dashboardApiId, registeredChildIds) {
  if (!diff) return true; // cold start / unknown — regen
  const mine = new Set([dashboardApiId, ...registeredChildIds]);
  for (const field of ['added', 'changed', 'removed', 'rewired']) {
    const arr = diff[field] || [];
    if (arr.some(id => mine.has(id))) return true;
  }
  return false;
}

Known edge case (for #36 to handle)

When a brand-new child is wired to a registered parent (e.g. a new measurement connected to an existing pumpingStation already registered to a dashboardAPI), the new node lands in diff.added but is not yet in ChildRegistrationUtils.registeredChildren at the moment flows:started fires — registration happens on child node init, which is concurrent with the event.

Two acceptable mitigations:

(a) Also include grandchildren ids (children-of-children one level up) in the predicate set, computed from the previous-deploy snapshot.
(b) Accept a one-deploy race: skip regen on this deploy; next deploy will see the new child registered and regen will fire. For R&D-only this is fine.

Recommendation for issue #36

Implement the predicate above; pick mitigation (b) initially and revisit if it manifests as user-visible lag.

Prototype location (do not import)

.prototypes/spike-flows-started/

## Spike findings: confirmed — use `flows:started` + `diff` as designed **Verdict:** confirmed. ### What I did Installed a throwaway custom Node-RED node (`.prototypes/spike-flows-started/`, gitignored) into the running `evolv-nodered` container that subscribes to `flows:starting/started/stopping/stopped` and appends each event to `/data/spike-events.log` with the full `diff` content. Then drove 5 deploys via the admin API (`POST /flows` with `Node-RED-API-Version: v2` and `Node-RED-Deployment-Type: full|nodes|flows`) and inspected the resulting payloads. ### diff payload shape `payload.diff` is always present (on deploys; empty on cold startup) with exactly these keys: ``` {added, changed, removed, rewired, linked, flowChanged} ``` All six are arrays of **node ids** (tab nodes included). On startup-from-cold the whole `diff` object is empty (no keys). ### Per-deploy-type evidence Two test tabs (Spike A, Spike B) each with a `debug` node. Ran 5 deploys: | Step | Action | type | `diff.added` | `diff.changed` | |---|---|---|---|---| | 1 | Add Spike A + Spike B tabs + debug nodes | full | `[tab_a, tab_b, dbg_a, dbg_b]` | `[tab_a, tab_b]` | | 2 | Rename `dbg_b` only | nodes | `[]` | `[dbg_b, tab_b]` | | 3 | Rename `dbg_b` again | flows | `[]` | `[dbg_b, tab_b]` | | 4 | Rename `dbg_a` (different tab) | flows | `[]` | `[dbg_a, tab_a]` | | 5 | Deploy with no logical change | full | `[]` | `[]` | `removed`, `rewired`, `linked`, `flowChanged` were `[]` in every step of this test. ### Key observations 1. **Event fires on every deploy** regardless of type. Type is exposed via `payload.type ∈ {"full","nodes","flows"}`. 2. **The diff is per-deploy-type-independent** — same edit produces the same `diff` on Step 2 (`nodes`) and Step 3 (`flows`). The deploy type controls what Node-RED *restarts*, not what `diff` reports. 3. **Editing one node N includes both N's id AND its tab's id in `diff.changed`.** Tabs land in the diff when any of their nodes change. So tab-id-based predicates over-trigger if dashboardAPI shares a tab with unrelated work. 4. **No-op deploys produce empty `diff` arrays** — a legitimate `outcome: "no-diff"` for the F-1 log line. 5. **`linked` and `flowChanged` were never populated in these tests** — likely fire for cross-tab `link` nodes or whole-flow imports. Not load-bearing for the predicate. ### Predicate for "did my subtree change" Use the dashboardAPI's node id + the ids of all registered children (not tab ids — avoid the tab false-positive): ```js function subtreeChanged(diff, dashboardApiId, registeredChildIds) { if (!diff) return true; // cold start / unknown — regen const mine = new Set([dashboardApiId, ...registeredChildIds]); for (const field of ['added', 'changed', 'removed', 'rewired']) { const arr = diff[field] || []; if (arr.some(id => mine.has(id))) return true; } return false; } ``` ### Known edge case (for #36 to handle) When a brand-new child is wired to a registered parent (e.g. a new `measurement` connected to an existing `pumpingStation` already registered to a dashboardAPI), the new node lands in `diff.added` but is **not yet** in `ChildRegistrationUtils.registeredChildren` at the moment `flows:started` fires — registration happens on child node *init*, which is concurrent with the event. Two acceptable mitigations: - **(a)** Also include grandchildren ids (children-of-children one level up) in the predicate set, computed from the previous-deploy snapshot. - **(b)** Accept a one-deploy race: skip regen on this deploy; next deploy will see the new child registered and regen will fire. For R&D-only this is fine. ### Recommendation for issue #36 Implement the predicate above; pick mitigation (b) initially and revisit if it manifests as user-visible lag. ### Prototype location (do not import) `.prototypes/spike-flows-started/`

vps1_gitea_admin referenced this issue

2026-05-26 15:54:25 +00:00

Walking skeleton: measurement → dashboardAPI → Grafana panel on deploy #34

vps1_gitea_admin referenced this issue

2026-05-26 15:57:45 +00:00

Skip regen when diff shows no subtree change #36

Sign in to join this conversation.