As per Nicks original blog post, there will be a setting to allow you to switch back to the old/current behaviour.
As Colin points out today a row of simple function nodes (a,b,c,d) with debugs attached to each will report in the order d,c,b,a which also seems illogical to most people.
The current 1.0 beta 3 has this feature enabled so please try it and see.
3 Likes
Speaking as a non-programmer... I've found the solutions to handle synchronous flow problems more difficult to implement, than understanding the asynchronous flow concept.
FWIW - asynchronous gets my vote.
2 Likes
I looked through everything on my flows and I think, by luck of using function nodes more often than composing nodes together, Im not effected too much. The only place I see that might be interesting is the node-red-dashboard chart nodes that take multiple sensor readings as input, and persist to file. Im curious how multiple sensors which all publish VERY close together going through this are going to work with the persistence. At the very least, if there's an issue around it it'll showcase an obvious race condition issue in a nice pretty graph form with missing inputs 
Luckily that's not a major issue for me though if there is an issue there, as its just informational. When 1.0 goes live Ill try it and see if it causes what I think its going to.
The beta is available now. It would be great if you could try it before we ship given you have so many concerns.
My concern is more about if my existing, today very well functioning, flows starts to behave strange
Due to my inexperience and lacking knowledge in the beginning, I might have used too many function nodes, context variables and not properly considered "things" when hacking javascript. I'm pretty sure, if I would redo it, I would do it differently. But as I said, what I have is working just excellent & super stable. It would be a pity if I have to do a massive maintenance of the whole stuff, we are maybe taking about many months of difficult work
Coming back to what I asked 21 days ago; how do I find & fix those parts? The runtimeSyncDelivery "lifeline" will help me to keep the thing running but if I really would like to update the flows correctly, how? There must be some ways to check an existing flow and identify where things would break? A separate inconsistency tool...
And at first start-up with an old flow; "Your flow is not ready for async mode, automatically activating runtimeSyncDelivery..." put into NR log
Just the fact that you are providing a "rescue flag" like runtimeSyncDelivery indicates to me that you are not fully sure either
Au contraire . the reason for the new setting is because we ARE fully sure. There will be some flows that are relying on the existing behaviour and this setting will enable them to continue. However we believe that to position Node-RED for the future capabilities we have in mind, making this change now to a more consistent behaviour between nodes is the right thing to do. (And it fits with the philosophy of a major semantic version number change.)
I have no idea how a reliable flow analysis tool could work. Even in a simple flow, renaming / copying a few variables could easily break any static analysis. And running live would be fraught with timing and logging issues (as any extra probes would themselves probably be async).
3 Likes
You ARE fully sure....some of our flows will not work as is??? If so, for those the setting needs to be set. If one really wan't to figure out what is not compatible, long nights ahead eventually?
EDIT: in my case I use NR for automation and it could be difficult to understand why certain commands suddenly fail, especially those that might be executed only a few times a day or week and relies on statuses of various kind kept in context
I must say I fully agree with the developers on these coming changes.
Yes, some flows that use context might get broken, but that could happen anyway even with the current version.
So, speaking for v1.0:
- It is new major version. So now is the right time for that!
- Up to now, it has never been a documented behavior, now we get consistency with that
- It is the right thing regarding performance and latency, utilizing the underlying semantics of NodeJS much more efficiently.
Don't see it as a bad thing, but an opportunity to rethink some of your flows. Use context the way it should be used. Maybe even remove its usage where unnecessary.
And generally speaking, that is not a problem with Node-RED, it is an issue to be aware of with ANY programming language. If you use global variables or side-channels for data, you have to be aware of the consequences.
4 Likes
Have I missed the part where updating is mandatory?
2 Likes
I can see where it would cause some grief with newcomers from our field. Especially if they have only used AB Micrologix, SLC 500, and PLC5. Both platforms update their IO tables in a synchronous fashion. The Compact/Contrologix however updates its IO asynchronously. If they are used to it and have used both sync and async PLCs then they will know the change of mindset needed when going from sync to async.
Most people visit here from the U.K. and people there predominantly use Siemens. I have never used it but I am sure it updates itâs IO async as well.
Still, I donât think most will know that it is async when they fire node-red up for the first time. We generally donât read the instructions. lol
One suggestion I would make is to have the wires change to a certain color if they are supporting an async operation/branch. Giving it some visual difference will at least show the new guy that something different is happening in one section of their flow.
I personally wouldnât place a handicap in node-red to support sync only.
1 Like
Here are a couple of simple flows
[{"id":"1198adc.340e752","type":"inject","z":"3658255b.2041aa","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":120,"y":280,"wires":[["698209e3.e609a8"]]},{"id":"698209e3.e609a8","type":"function","z":"3658255b.2041aa","name":"add A","func":"msg.payload = {};\nmsg.payload.ts = Date.now();\nmsg.payload.A = \"added A\";\nreturn msg;","outputs":1,"noerr":0,"x":270,"y":280,"wires":[["574bfc80.494a24","4aa610ad.af89b"]]},{"id":"574bfc80.494a24","type":"function","z":"3658255b.2041aa","name":"add B","func":"msg.payload.ts = Date.now();\nmsg.payload.B = \"added B\";\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":280,"wires":[["529d20d9.56f74","4aa610ad.af89b"]]},{"id":"529d20d9.56f74","type":"function","z":"3658255b.2041aa","name":"add C","func":"msg.payload.ts = Date.now();\nmsg.payload.C = \"added C\";\nreturn msg;","outputs":1,"noerr":0,"x":550,"y":280,"wires":[["4069ecb0.613f64","4aa610ad.af89b"]]},{"id":"4aa610ad.af89b","type":"file","z":"3658255b.2041aa","name":"MyLog","filename":"/tmp/mylog.txt","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":770,"y":360,"wires":[[]]},{"id":"4069ecb0.613f64","type":"function","z":"3658255b.2041aa","name":"add D","func":"msg.payload.ts = Date.now();\nmsg.payload.D = \"added D\";\nreturn msg;","outputs":1,"noerr":0,"x":710,"y":280,"wires":[["4aa610ad.af89b"]]},{"id":"1aec21bf.93232e","type":"inject","z":"3658255b.2041aa","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":120,"y":440,"wires":[["7531d1f7.316ac"]]},{"id":"7531d1f7.316ac","type":"function","z":"3658255b.2041aa","name":"add A","func":"\nmsg.payload = {};\nmsg.payload.ts = Date.now();\nmsg.payload.A = \"added A\";\nreturn msg;","outputs":1,"noerr":0,"x":270,"y":440,"wires":[["2a229a8c.d497c6","f9669b4c.f298c8"]]},{"id":"2a229a8c.d497c6","type":"function","z":"3658255b.2041aa","name":"add B","func":"msg.payload.ts = Date.now();\nmsg.payload.B = \"added B\";\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":440,"wires":[["64470f6e.a2875","f9669b4c.f298c8"]]},{"id":"f9669b4c.f298c8","type":"file","z":"3658255b.2041aa","name":"MyLog2","filename":"/tmp/mylog2.txt","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":760,"y":520,"wires":[[]]},{"id":"64470f6e.a2875","type":"function","z":"3658255b.2041aa","name":"async C","func":"msg.payload.ts = Date.now();\nmsg.payload.C = \"added C\";\nsetTimeout(function() { node.send(msg) }, 500 );\nreturn null;","outputs":1,"noerr":0,"x":560,"y":440,"wires":[["f9669b4c.f298c8","1f860906.302147"]]},{"id":"1f860906.302147","type":"function","z":"3658255b.2041aa","name":"add D","func":"msg.payload.ts = Date.now();\nmsg.payload.D = \"added D\";\nreturn msg;","outputs":1,"noerr":0,"x":710,"y":440,"wires":[["f9669b4c.f298c8"]]}]
both add things to the msg payload as they go along and update a timestamp and send the result to the log file... the top flow is "synchronous" as-in each msg is passed directly to the next node.
The bottom has an async action added to node C (a 500 mS delay) - but as noted it could be an http or database lookup, a jsonata expression, almost anything hidden in a node.
Most users would expect the top flow to create a nice log as it "goes along" the flow... but what you get today is
{"ts":1567864864625,"A":"added A","B":"added B","C":"added C","D":"added D"}
{"ts":1567864864625,"A":"added A","B":"added B","C":"added C"}
{"ts":1567864864625,"A":"added A","B":"added B"}
{"ts":1567864864624,"A":"added A"}
so totally non-intuitive I would say....
and the second gives you
{"ts":1567864865615,"A":"added A","B":"added B"}
{"ts":1567864865615,"A":"added A"}
{"ts":1567864865615,"A":"added A","B":"added B","C":"added C"}
{"ts":1567864866118,"A":"added A","B":"added B","C":"added C","D":"added D"}
so again - err - a bit mad.. (IMHO) - unless you like re-sorting files later on.
With "async" on you get this for the first
{"ts":1567864880160,"A":"added A"}
{"ts":1567864880161,"A":"added A","B":"added B"}
{"ts":1567864880162,"A":"added A","B":"added B","C":"added C"}
{"ts":1567864880162,"A":"added A","B":"added B","C":"added C","D":"added D"}
and this for the second
{"ts":1567864881217,"A":"added A"}
{"ts":1567864881217,"A":"added A","B":"added B"}
{"ts":1567864881217,"A":"added A","B":"added B","C":"added C"}
{"ts":1567864881722,"A":"added A","B":"added B","C":"added C","D":"added D"}
which we think is much more consistent and what users would "expect" to happen.
But anyway - as we have said - it's there in the beta-3 now for you to try. We think most people won't even notice the difference in day to day running, but yes it IS a difference so it may trip up some flows.
5 Likes
Totally agree. This thread has certainly been useful to highlight the possible issues for certain edge cases and I do think these are more likely to catch us "old timers" out more than newcomers!
Lets not fall too far into the trap of assuming the worst.
v1 is a MAJOR milestone and so far with Node-RED really drastic changes have largely been avoided which is an amazing testament to the development and design.
I know that we will all pull together and help anyone with issues that arise from this change & hopefully we will all learn and improve from the experience.
So far as I can tell from my limited understanding of this thread, I've not seen anyone raise any showstoppers which is great.
2 Likes
LOL, I'll say it one more time: I get the change, and I can live with it. I think its going to be counter intuitive and cause issues with existing flows and the intuitive ways people not familiar with async workflows will try to build stuff (I'm thinking of some of the stuff I've seen on the homeautomation and nodered subreddits)... and I STILL think this is a good idea.
My point is not to roll this back, its that the repercussions from this need to be handled, some with good documentation, and some with maybe some new constructs to allow for control for those of us understanding a bit better. Specifically:
- Good documentation on this, especially the repercussions on delivery orders, and the impact on persistence mechanisms (like a note on the pages for global / flow variables, etc.)
- The more I consider this, the more I keep thinking a set of lock/unlock nodes to allow single entrance to a flow would solve pretty much any use case I can think of that would be hurt. Im not sure how easy it would be to explain those nodes usages to someone who doesn't understand the implication of a lock, but wrapping sensitive flow pieces with those would solve the possible race conditions without really having to reconsider anything here.
Edit: Meant this to be a main thread reply, not specific, sorry, the threading model of these forums always confuses me.
Yes, I agree,
One thing though, now when I have started to look into my own flows, that would help a lot to make an old flow "compatible" I think is to find where a specific context data is used in a flow. I mean, it is created/born in one node and then used in others (some will just read, others will update). In newer NR releases you have the menu choice "Context Data". It will present all your context data nicely. You can copy, refresh and delete. But what if you could do "used by"? This would help a lot during the re-work of the flow
As example below, in my flow I have a number of context data variables of various kind that I might consider getting rid of or re-design in some way. For instance, the "currentStatuses". It would be great if I could find where it used in an easier way. Obviously using a text editor and searching the flow can do it but it would really help having this feature in the flow editor. I think...
2 Likes
But have you TRIED your flow with the new beta? Does it actually break ?
Walter, short notice because this is way too much off-topic. Your idea would be useful, but don't think it can be implemented because you don't know which nodes (or function nodes) use which data. If you have doubts, please open a separate discussion.
Bart
2 Likes
Yes, of course
Not yet, no, since the flow is running my main automation system for our house and has several hw interfaces connected, I did not dare to touch it before I understand the changes better 
2 Likes
Then the only logical conclusion would be to build a second house for beta testing. 
8 Likes
That would be something 
1 Like
