Node coverage for flows

[lvenier]

Dear team,

I'm trying to figure out how to have a "node coverage" for my flows.
Lest says I have a flow with 20 nodes. I am building functional tests to test/validate my flows.

I would love to know how many nodes or path are being tested with my test plan.

What is you opinion ?

[TotallyInformation]

One way might be to read in the flows.json file, you can then collate the nodes and compare against the nodes your tests touch. You could, of course, walk through all the nodes via Node-RED's RED object if you are doing the processing from within the same flow. But the other way means you can do it outside of Node-RED if you wanted to.

@gregorius might be a good person to wade in on this request as well since he has done a fair bit of creative work analysing flows.

[lvenier]

Thanks @TotallyInformation

To illustrate a bit more let's imagine I have the following flow :

I would love to have somehow the following report if I have implemented one test :

[jbudd]

How would you assess coverage of a switch node with multiple outputs?
A change node having multiple rules, possibly affecting multiple targets, but only one output?

It would be cute if recently traversed wires show in a different colour, but that has been discussed before and seems impossible or undesirable.

[gregorius]

@TotallyInformation

@gregorius might be a good person to wade in on this request as well since he has done a fair bit of creative work analysing flows.

@TotallyInformation the irony of it all - just today I created a stats button in the introspection node package:

(Nodes shown are mindmap nodes, further down in that list are the "classic" nodes.)

It's of course not exactly what you want but you can also check out the code base (as a flow, function is called collectNodeStats) if you want to extend it.

Edit: This solution uses the Node-RED client API (i.e. stats are the browser version of the flow) not the deployed - server side - flow. The solution does not use the flows.json file.

[gregorius]

Btw the introspection package also has a the sink & seeker nodes that will trace all paths between two nodes. Taking your flow, that looks a little like this:

That's probably a piece of the puzzle to work out which paths there actually are to test. The problem is the relationship between test cases and possible paths in a flow.

Again that works in the client only not on the node-red server.

[TotallyInformation]

@lvenier

To illustrate a bit more let's imagine I have the following flow

Not quite the way I thought you might be doing things. And that flow probably wouldn't ever let you test both responses - the first response change to complete would trigger the http-out, the switch should have 2 outputs, 1 for each response type. As drawn, you wouldn't need two change nodes anyway, only 1.

@jbudd

How would you assess coverage of a switch node with multiple outputs?

You would need to have something to count the number of node outputs and a way to detect which leg of the flow you went down. As the number of multi-output nodes increased in your flow, the test summary would have to get ever more complex of course and your tests would require a way to trigger a specific path through your flow.

[lvenier]

For sure I would have 2 output for the switch but I was demonstrating the objective : making sure my tests will trigger all the nodes (not all the combination of nodes. That would be an other complexity).