Expose Index of Flow -> Node (and be updatable)

Core Development Feature Requests
1

Ok...

On occasion I see questions around the order of a Flow/Node being instantiated.

And to be honest, It's a very valid query: Some Nodes may have a dependency.

So...

What if, the index (and I guess the order) in which a Node/Flow is instantiated is revealed (at the minimum)

And if possible, being able to change it, to alter the loading order?

I cant help thinking this will address some Use Cases that pop up every so often (I cant find one currently, its just a subconscious thought, that seems to have surfaced)

it seems this can add (or reveal answers at least) to some questions around it all

(I would like to at this point not get into Config Nodes)

2

I've never come across a loading order issue I don't believe. So struggling to understand the need here.

And you can't ignore config nodes if thinking about order as I think they may get run first?

3

They do.

But I suppose one example would be.

"I need this flow to start first, before this one"

But then, I see where you are going...
Develop your flows to react accordingly.

I get it... but it feels something like this could add some control of execution order?

4

This is more like my post for a sequential NR.

5

One thing I just uncovered :sweat_smile: ...

I can re-order flows... does that state the order in which they are loaded?
that would be a large portion of my thoughts if so....

if that is correct, it will be a nice thing to (reveal) the Node order in that flow?

A lot of this is aloud thinking of course, but will be great to at least see what Node is started first.

.....

Realising the Flows should be based on event, again I get it.. but could add some nice meaningful meta - especially in complex setups

6

if I remember correctly, the order follows the one of the flow.json array but I'm not sure it's reliable to think in that way. I cannot remember what happens with multiple flows.

7

And that is the reason why I think it would be a nice... 'thing' to show?
and at a push alter its order of execution?

Yes.... one should develop the flow to address expectation (and I do - and most should)

But a very crude example awaits :grinning_face::

 - A node (that doesn't output anything) - Adds a value to the context (global , flow whatever..)
 - A Users function node wants to use that value.
 - The user cant understand why their function node fails (value is undefined)

If the order of execution is displayed, that user will know that their function node started before the other node

I hope this makes sence

Waiting for the 'Use a delay Node' :grin:

I think this is more about showing the user what nodes are started first, than actually having control over it

8

Each flow is started in the order they appear in the editor. Reordering the tabs will change the order things are started. That doesn't guarantee all the flows have run anything, but does give you a fair chance.

1 Like
9

If its ever possible - I think it will be nice metadata to show the Node order (within the canvas)
If node order also denotes the Node loading order of course.

10

I thought that you were talking about how the nodes start up rather than the flows.

OK, well the first tab is executed first but after that, I think things generally happen in the order that they were added. Inject, of course, allows some control over startup order. But I get what you mean.

1 Like
11

Yup.

I think it can actually lend some debugging assistance, if some indication of when each node is.... Likely? going to be started.

For the flows them self, its obvious (mostly).. the ordering in which they appear in the UI, but in each flow, it doesn't extend this "hint" to the nodes.

12

As it’s based on the order of the flows file, I’m sure bit of jsonata could print out the list of node names in order from the flows file…

1 Like
13

An amusing diversion into the inner workings of the flow.json file.

Editor tabs are flows.
On each flow are zero or more sequences.
A sequence starts with a node that is wired out-of but not wired into (but in practice only 'event trigger' nodes).
Nodes will be executed in the order of the wiring (simplistic viewpoint).

Inside the flows.json file, components are ordered as:

tabs > subflows(dec) > groups > junctions > configs > nodes

A bit of JSONata can indeed analyse and print out the:

  • list of tabs (flows) in order [same as in the editor]
  • the configuration nodes (these don't have 'wires' property)
  • list of groups and subflows
  • generate all the different types of (wired) nodes in use, in alphabetical order, with a list of each tab on which they appear and the count of these nodes on that tab
  • a list of all sequences, by tab, showing the node type, id, and name

With this information, it is then easy to select the tab and the sequence in the tab (flow-sequence entry point) and then walk the node path.

I have tested this JSONata on my flows.json and I have checked that the path walk does ignore loops so it should avoid going around for ever. Naturally, with a very large flow file the JSONata may not cope too well with the array sizes in use, but you are welcome to try.

Ignoring the tabs, groups, and configs, all other nodes are 'wired' and I have added a 'file-order-index' so that I can see where the node sits inside flow.json.

example flow for analysis
example flow for analysis943Ă—631 79.2 KB

The above block of code has three sequence start points, but taking the 'Every Hour' inject node as the main entry point for analysis gives:

[
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->P1 East< (557) [4]->API FCS< (558) [5]->OK?< (561) [6]->API Left< (613) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->P1 East< (557) [4]->API FCS< (558) [5]->OK?< (561) [6]->Dates / Array / Plane< (614) [7]->by plane< (565) [8]->Solar FC Power East< (569) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->P1 East< (557) [4]->API FCS< (558) [5]->OK?< (561) [6]->Dates / Array / Plane< (614) [7]->by plane< (565) [8]->Solar FC Power West< (570) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->P1 East< (557) [4]->API FCS< (558) [5]->debug 329< (623) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->P2 West< (560) [5]->API FCS< (558) [6]->OK?< (561) [7]->API Left< (613) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->P2 West< (560) [5]->API FCS< (558) [6]->OK?< (561) [7]->Dates / Array / Plane< (614) [8]->by plane< (565) [9]->Solar FC Power East< (569) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->P2 West< (560) [5]->API FCS< (558) [6]->OK?< (561) [7]->Dates / Array / Plane< (614) [8]->by plane< (565) [9]->Solar FC Power West< (570) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->P2 West< (560) [5]->API FCS< (558) [6]->debug 329< (623) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->10s< (562) [5]->History Times< (615) [6]->History< (622) [7]->Parse< (616) [8]->Actual energy last hour< (617) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->10s< (562) [5]->History Times< (615) [6]->History< (622) [7]->Parse< (616) [8]->Update SFC (act & fc)< (618) [9]->link out 1< (619) [10]->link in 1< (620) [11]->5s< (568) [12]->< (11) [13]->Analyse Forecast< (567) [14]->FC Estimate Today< (572) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->10s< (562) [5]->History Times< (615) [6]->History< (622) [7]->Parse< (616) [8]->Update SFC (act & fc)< (618) [9]->link out 1< (619) [10]->link in 1< (620) [11]->5s< (568) [12]->< (11) [13]->Analyse Forecast< (567) [14]->FC Estimate Tomorrow< (586) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->10s< (562) [5]->History Times< (615) [6]->History< (622) [7]->Parse< (616) [8]->Update SFC (act & fc)< (618) [9]->link out 1< (619) [10]->link in 1< (620) [11]->5s< (568) [12]->< (11) [13]->HA Graph Array< (566) [14]->FC Solar Table< (571) ",
   "[0]->Every Hour< (612) [1]->2 min past< (611) [2]->Site< (556) [3]->10s< (559) [4]->10s< (562) [5]->History Times< (615) [6]->History< (622) [7]->Parse< (616) [8]->Update SFC (act & fc)< (618) [9]->link out 1< (619) [10]->link in 1< (620) [11]->5s< (568) [12]->< (11) [13]->Chart Array< (564) [14]->Forecast< (544) "
]

Which is correct for all possible routing paths given all possible outcomes. [n] is the path step depth, and (x) is the all-wired-nodes in-file index number.

Conclusions:

Flows (the tabs) are ordered by the editor, so moving a tab and deploying re-saves the flow.json with the new order. Hence all important sequences can go in the first tab, and will load first.

All junctions come first, then all wired nodes are saved in the order that they were added to the editor. Hence, if you write a new sequence in order, it will be saved in that order, but if you write it 'backwards' it will be saved in reverse order. Adding a new sequence to a tab will place that sequence order after all other existing sequences on the tab, however it is easy to delete-recreate any other sequence trigger node, thus placing that existing node back at the end of the file (much like 'send to the back' in something like MS PowerPoint). Hence, bringing any one sequence in a flow to the front of the flow-sequence order requires a delete-add action on all other sequence start nodes in that tab.

Sequences must primarily execute in the order of the wiring.

sequence execution order theory
sequence execution order theory1719Ă—521 58.9 KB

Taking a simple sequence 'Theory' -> A -> B & C -> D - Debug, after A the message is cloned and one goes to B to flow through to D, and the other goes to C, also to flow through to D.

The route-paths show:

[
   "[0]->Theory< (366) [1]->A< (348) [2]->B< (349) [3]->Delay-B< (353) [4]->D< (350) [5]->debug 33< (351) ",
   "[0]->Theory< (366) [1]->A< (348) [2]->C< (352) [3]->Delay-C< (354) [4]->D< (350) [5]->debug 33< (351) ",
   "[0]->Theory< (366) [1]->A< (348) [2]->link out 1< (370) [3]->link in 1< (371) [4]->D< (350) [5]->debug 33< (351) "
]

Ignoring the links (just for testing), there are only two paths. However I understand that node execution is carried out 'round robin' to avoid, in this case, one message going through to the end, followed by the other message.

Hence, all possible node execution sequences are (ignoring the delay nodes):

  • Theory > A > B > C > D1 > D2 > debug
  • Theory > A > B > C > D2 > D1 > debug
  • Theory > A > B > D1 > C > D2 > debug
  • Theory > A > C > B > D1 > D2 > debug
  • Theory > A > C > B > D2 > D1 > debug
  • Theory > A > C > D2 > B > D1 > debug

Although B is (349) in the file, and the first wire connection out of A, I assume that C (352) could be executed before B. When the message is then placed back into the execution queue, it would suggest that if B goes first, C goes before D1 then finally D2. In practice I have only found that message A-B-D always arrives before A-C-D, but these nodes are so simple that the entire flow takes less than 1 ms to execute, and of course the Debug node only pushes the outcome when it is called, so the order of a debug message only indicates the order of the debug node being used, not the order of in-flight node execusion.

And finally, deleting the wiring between A and B/C then re-wiring backwards saves the wire order in reverse, and the flow paths reverse

["[0]->Theory< (366) [1]->A< (348) [2]->C< (352) [3]->Delay-C< (354) [4]->D< (350) [5]->debug 33< (351) ",
 "[0]->Theory< (366) [1]->A< (348) [2]->B< (349) [3]->Delay-B< (353) [4]->D< (350) [5]->debug 33< (351) "]

as does the order of execution.

I think it would be indeed possible (but not necessarily desirable or sensible) to select one particular tab(flow), and 'priority' node, and from this auto-edit the flows file to move the enclosing tab forwards, re-order the tab sequence start nodes to promote the sequence, and possibly re-order the node wiring within the sequence.

2 Likes
14

Great investigation @biscuit

Not much going on at work today hu!? :rofl:

And welcome to the forums of course! :smiley:

15

I have been three days at this, but then I am retired and it has been on my 'do' list to poke around in the flows.json file. I have also proved to myself that I can write a tree-walk recursive function in JSONata, which is another tick.

In the grand-plan is the idea to write an auto-generator for Node-RED that creates flows from parameters.

3 Likes
16

Well welcome to the community, and great to have you on board

1 Like