tools: add output-manifest-verify; extend flow-lint with fan-out checks
- tools/output-manifest-verify/ — enforces .claude/rules/output-coverage.md
§3: every node ships test/_output-manifest.md and every declared key
is referenced by at least one test file. First run shows only
machineGroupControl has the manifest (16 keys covered); all other nodes
warn. --strict escalates "missing manifest" to an error for CI gating.
- flow-lint gains two rules from the same output-coverage rule:
* FN_OUTPUT_WIRES_MISMATCH — function declares outputs=N but wires has
M arrays (causes silent dropped or duplicate emissions).
* FN_PAYLOAD_NULL_LITERAL — function source contains `payload: null`
literal (the η-null ui-chart crash pattern from 2026-05-14).
First run found 1 instance in mgc/02-Dashboard.json.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
47
tools/output-manifest-verify/README.md
Normal file
47
tools/output-manifest-verify/README.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# @evolv/output-manifest-verify
|
||||
|
||||
Enforce `.claude/rules/output-coverage.md` §3:
|
||||
|
||||
1. Every node ships `test/_output-manifest.md`.
|
||||
2. Every key declared in the manifest is referenced by at least one test
|
||||
file under `test/**/*.test.js`.
|
||||
|
||||
Designed to replace the manual verification checklist in the rule. The
|
||||
rule itself can become a thin pointer to this tool once every node is in
|
||||
compliance.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# scan every node with a CONTRACT.md
|
||||
node tools/output-manifest-verify/bin/output-manifest-verify.js
|
||||
|
||||
# one node
|
||||
node tools/output-manifest-verify/bin/output-manifest-verify.js nodes/machineGroupControl
|
||||
|
||||
# fail (exit 1) when a manifest is missing, not just warn
|
||||
node tools/output-manifest-verify/bin/output-manifest-verify.js --strict
|
||||
|
||||
# CI / JSON output (one line per node)
|
||||
node tools/output-manifest-verify/bin/output-manifest-verify.js --json
|
||||
```
|
||||
|
||||
## Severity
|
||||
|
||||
| Condition | Severity |
|
||||
|---|---|
|
||||
| `test/_output-manifest.md` missing | `warn` (escalates to `error` under `--strict`) |
|
||||
| Manifest has no `##` sections / no tables | `error` |
|
||||
| Declared key not referenced by any test file | `warn` |
|
||||
| Manifest declares keys but `test/**/*.test.js` directory empty | `error` |
|
||||
|
||||
## What it does NOT check (yet)
|
||||
|
||||
- Whether each key has both a **populated** and **degraded** test (the
|
||||
rule's §3 promise). Reaching that level of confidence requires
|
||||
instrumenting the actual node runtime; tracked as a follow-up.
|
||||
- The Port 1 (InfluxDB) cardinality discipline — see
|
||||
`.claude/rules/telemetry.md`.
|
||||
|
||||
Run after touching `_output-manifest.md`, `test/**/output-*.test.js`,
|
||||
or any Port 0/1/2 emission key.
|
||||
Reference in New Issue
Block a user