Hello
I have come up with a contraption to split flows into separate files so that they could be kept in different repositories. That works quite well and isn't related to the main Node-RED functionality.
Yet I notice quite often that after deploy a lot of flows are changed — 'group' nodes have width and height values added or removed, some nodes of the flow are reordered.
Is this done by the instrument responsible for splitting into separate flow files? I wouldn't imagine that as from its source code I see that it doesn't do much processing. Is this something that Node-RED does under the hood and just feeds those changes to this "storage plugin"?
I've forked that plugin to add two changes:
don't bother writing w/h values for 'group' nodes to output files,
sort all direct child nodes of a 'tab' or 'subflow' node by id.
This way such automatic changes are not propagated into git. So far everything seems to work fine and Node-RED doesn't seem to care about w/h missing or node order changing.
I would like to understand, though, whether I am breaking something, making a problem for myself in the future.
As you note, discarding w/h have no affect (they are simply recalculated). And, lets not forget, w/h are for visual purposes only. Where this might be a (very slight) issue is if you intend on using the online flow viewer built into node-red flows site.
As for re-ordering, that is part of the way nodes are add/removed/recreated etc. There might be an edge case where the order of a node has a side-effect (i.e. the nodes in your running flows may be ordered 1,4,3,2 but your backup is 1,2,3,4) This should NOT really be an issue but if a (node or flow) bug were present then the order of nodes in the flow might make a difference. A basic example would be if the msg in your running flow is forked to 3 nodes and the 3rd node mutates the msg object (or context) then your flow might work as expected, however, the backup might make the node that mutates executes first and the other nodes get incorrect data. It is tenuous but I think possible.
EDIT: It might be the order of wires that matter in the contrived example above. Do they also change?
w/h are for visual purposes only. Where this might be a (very slight) issue is if you intend on using the online flow viewer built into node-red flows site.
Not an issue for me. Even if something did glitch then it would be purely a UI "wrong border around elements" kind of problem, right? Nothing remotely close to data loss or incorrect flow logic.
There might be an edge case where the order of a node has a side-effect
If I understand correctly then this is something like a classic race condition case, right? If all of my logic is linear (flow progresses from one node to another using connections and at most switches) then there would be no case of this issue, right?
In a couple of cases when I have the need to fetch several unrelated things I use split/join nodes, so again the order shouldn't matter, right?
EDIT: It might be the order of wires that matter in the contrived example above. Do they also change?
No, I have not noticed any change in wire order. Please see screenshots for an example:
It really looks like two nodes were just swapped around and git tried to do its best matching what remained and what was changed.
Am I right to assume that the most important part here are the node wires (and their order), which contain representation of actual business logic (how the flow proceeds)? And if the wires are not affected, then the node order isn't critical at all, I guess.
P.S. How do you manage to reply in such a short time? Companies that I pay money for their products sometimes take days or weeks to provide a non-answer and here you are with a helpful explanation within an hour. Thank you very much!
That's passion driven action - whereas at other "companies" it might just be paycheck driven action.
One of the things that make Node-RED positively special is the people being its community!
I am not going to disagree with that, this is true. I was also wondering about Steve himself. He isn't just a member of the community, he's contributing code to Node-RED (to put it simply) yet he's also somehow providing timely and helpful explanations.
Are you saying that sometimes, even though you have made no changes to the flow, that git shows differences, or is it that sometimes you see unrelated changes, with nodes moving within the file?
This is a hard question to answer.
I do a lot of changes, but I have a lot of flows. For Node-RED itself they all exist within one active project.
Now I think that the problem was (until my fix ignoring w/h changes and child node order) that if I make a change in one tab and deploy those changes then sometimes other tabs would get dirtied (group w/h changed, for example).
Under the hood, the full flow is transmitted (regardless of the deploy type). There is logic in the core that determines what has changed, what the deploy type was and it figures out what will be re-generated. Additionally, there are sometimes changes in a node and the properties it has compared to when the flow file was first written (for example an upgraded node package), that can kinda do an "in-place upgrade" and thus affect the resulting flow file. This usually isn't an issue but as you witness, some things can change.
disclaimer: it has been some time since I stepped this logic - it is v big and v complex, so there may be some slight inaccuracies, but it good enough to provide the gist of what happens
Just out of curiosity...
Tab or subflow child nodes are stored in a JSON array, but I suspect that they aren't strongly ordered from the point of view of Node-RED (before they are fed into the JSON representation), are they?
Order of wires and, most importantly, the ids of outgoing wires are obviously very important. But the order in which items were placed on the canvas isn't that critical to keep, is it? Or what else could the child node order represent?
That could explain why the resulting order mutates without any other visible changes to the nodes themselves.
And similar to what I said earlier about side-effects.
We might need the author (Nick) to pop in and stretch his memory for the finer detail (or you could try your hand at reading the source - but it is a complex, async, nested promises, melon-twisting maze of magic)
You might be able to tease out the necessary info with some flows designed to fall into the traps of concurrency and "by reference" side-effect bugs?
That could be a curious exercise, but would it be able to introduce any bugs in such a flow?
Http in node → function → switch → function → http response node.
Here all the steps are clearly defined by wires and no matter what order all of those nodes are listed in the JSON representation there is just one track the msg can possibly take, no?
In general, the order of nodes in the flow definition should not matter.
The main thing it influences is the order the runtime creates the nodes. The main area where the runtime does extra work is ensuring configuration nodes are started first to ensure they exist before any flow nodes are started that explicitly depend on them.
And again you've fixed a bug right after it has started bothering me
ensuring configuration nodes are started first
Are you talking about global, standard Node-RED nodes or also things like node-red-contrib-config?
Now I'm wondering whether my fix is slightly breaking something. After I deploy these flows runtime should be able to find all of the configs in the flow and hopefully there the order for the items in the incoming JSON doesn't matter any more...
Config nodes are things like the mqtt-broker configuration that can be shared by multiple mqtt nodes - the ones that get listed in the Configuration Nodes sidebar.
I've found a subflow with a node and its config exported into the same file (not globally). In the resulting JSON the node (of type postgresql) precedes its own config (of type postgreSQLConfig) and the runtime is fine with that. I guess it's resilient enough to overcome any node order trickery.
So to sum it up:
width and height for a group node a not critical and have already been fixed,
child node order for flows could be important when loading config nodes, but so far seem to be working without issues.
Thank you for helping me understand the situation!
