Making flows asynchronous by default

But - instead of building a new house - you could launch a Node-RED instance with a copy of your current flow file in a test environment.

Then fake some of the inputs with inject nodes and see if everything works as expected.

Sorry, just for my understanding,,,hopefully topic related,,,,

Let's assume we have a global shared object. In a function node, I create a copy of this object, make a change to any of the property values and then finally update the object again

Like:

var ob = global.get('test');
//
//...some code doing a lot of time consuming stuff
//
ob['whatever'] = 'I am lost';
global.set('test', ob);

Is it like this:

  • in v 0.x another node has to wait for the first node to finish it's execution ???
  • in v 1.0, another node could eventually modify another or the same property values in the same object and then update it. When the first node finally finishes, it will overwrite with "old obsolete values" ???

If so, what would be the recommended way to update a shared object? How should this be handled correctly if I need such, i.e. to keep track of current statuses? Would it lead to thoughts about locking mechanisms, like you have in Python for multithreading, locking objects with acquire/release? Or writing queues...

Maybe I have misunderstood the thing completely...no need to worry

Sure, I know, but,,,well my system is rather complex and very difficult to simulate with all events coming in and out all the time. I guess I will have to go through each and every part of it and figure out if a change is needed or not

If your function node's code is purely synchronous, then you are safe.

Btw: var ob = global.get('test'); does not create a copy, you just assign a reference of the object to a local variable. Your object's properties will be updated even without storing it to the context again.

Is this time consuming stuff executing asynchronous? Thatā€™s what would make the difference

1 Like

Oh, that is good news, learning something every day, thank you. So this means it is unnecessary to do global.set('test', ob); after I change a property value, it will be changed in the object right away anyway "automatically", that's good. Yes, did some testing, you are right, means if I have a reference of a global object and another function/node is updating the same or another property value it will be reflected in my reference as well

You mean if the code would start a timer or so?

Asynchronous of any kind. Things with node.send in a loop, API calls that are internally handled asynchronously, things like that...

1 Like

Yes, you are modifying the same object in memory.

Unless you want to replace the object itself, there is no need to perform a .set()

However... just came to my mind:

That is true for the memory-based context store. I do not know for certain if this assumption holds for the file-based store as well (or any other custom implementation).

Well the way the example was "coded" the final set was outside of any callback so would still be synchronous...

If the set was inside a callback then even today you could be having fun with other things also changing the value, nothing to do with the the change we are making.

Hence why we think the cases where things will break are fairly few - personally in my flows and testing I have yet to see one - but that doesn't mean it's impossible. Hence the heads up / request for people to test / feedback actual examples etc...

OK, I go for it,,,,I do the update to my system now

The find dialogue does some of this for you of course:

While I don't know for sure from this what is updating and what is reading, I can at least see all references.

One of the advantages to running Node-RED not as a global installation but rather as a local one is that this kind of testing becomes trivial.

  • Install another copy of Node-RED (see my alternate installer)
  • Choose a time when the house is fairly quiet
  • Copy your live userDir contents to the new installation
  • Stop the live version, start the test version
  • Run some tests
  • Stop the test version, start the live version

You might need to think about some of the data you are writing external to Node-RED but by and large, that process should have minimal impacts.

Similarly, when moving to a new major version of Node-RED, I simply make a complete copy of the master folder which is where Node-RED itself is installed, the userDir folder is within this. Then I can install the new version over the top and if anything goes wrong I can simply restore everything very easily.

2 Likes

Excellent, just what I wished, thank you!!! So it was possible...this helps a lot

That's why I never use global installs.

My common practice is to separate both runtime install and workspace.
I simply install Node-RED into a random directory via npm install.
Then I launch it with the -u parameter to specify the workspace.

1 Like

Yes, that's why I put together alternate-node-red-installer :smiley_cat:

You install it globally and then you can use it to create the master folder, install Node-RED and create the userDir folder and the two correct package.json files with useful scripts already defined.

Then just cd into the new master folder and run npm start and you are good to go.

It is also cross-platform so it works as well on Windows as it does on a Pi.

1 Like

Updated my home automation system to this latest beta. The update process went very smoothly, just one node type I had to add afterwards (feedparse)

Everything started fine but shortly after I realized I had some stuff not working. It was my "solution" for keeping track of system statuses. Took some while until I found the reason why I was punished...

I had the following typical code in some function nodes:

if (msg.method === "turnon"){
    msg.payload = "carport turnon";
    node.send(msg);
    msg.payload = "facade entrance turnon";
    node.send(msg);
    msg.payload = "facade washroom turnon";
    node.send(msg);
    msg.payload = "facade garden turnon";
    node.send(msg);
}
if (msg.method === "turnoff"){
    msg.payload = "carport turnoff";
    node.send(msg);
    msg.payload = "facade entrance turnoff";
    node.send(msg);
    msg.payload = "facade washroom turnoff";
    node.send(msg);
    msg.payload = "facade garden turnoff";
    node.send(msg);
}

This did work fine in the old version but not in the new. I modified the code like this and now everything looks ok. I assume this is a typical example of the difference, when asynchronous the msg.payload is updated faster and BEFORE all node.send commands have had enough time to be executed. Somehow.

if (msg.method === "turnon"){
    node.send({ payload:"facade washroom turnon" });
    node.send({ payload:"carport turnon" });
    node.send({ payload:"facade entrance turnon" });
    node.send({ payload:"facade garden turnon" });
}
if (msg.method === "turnoff"){
    node.send({ payload:"facade washroom turnoff" });
    node.send({ payload:"carport turnoff" });
    node.send({ payload:"facade entrance turnoff" });
    node.send({ payload:"facade garden turnoff" });
}

Please forgive if this question is at a basic level, but...

Will this also impact upon sequential javascript commands executed within a function node, or is limited to the 'node to node' msg flow order of execution?

No it will not

1 Like

This hightlights a common problem with JavaScript.

Because JS reuses variables, you can easily get caught out with async operations. That's why there is already a utility function in RED that lets you create a new object rather than reusing an existing one.

So as per Nick & Dave's explanations, the second example is actually more consistent as you are forcing new objects to be created.

Note that, depending on use cases, you might want to also pass any source topic to the output as well for consistency.