here is my challange:
I have a key/value object with around 30-50 k/v pairs. The values get updated "serially" one by one.
My straight forward way is e.g.
var myglobalvar = global.get('myObject');
myglobalvar.prop2update = 'newValue';
global.set('myObject',myglobalvar);
In this way it would rewrite the whole global object. But I think it could happen that 2 or 3 update flows could overwrite each other if they run unfortunately at same time. So it would be a great thing if there is a way to do only incremental updates on a subproperty but not the whole object.
All values are fetched from an arduino microcontroller which is connected to my heater controller and get updated ongoing all few seconds or minutes - depending on how granular I need this specific value.
normally there is one procedure inside a flow which is responsible to hold the data up to date. So there should be no overlapping with other instances of that procedure.
But I'm not sure if its really the best way to fetch the whole global object, edit only one sub prop and rewrite the complete object.
Maybe I'm completely wrong but this way feels not efficient to me
Keep in mind, I must update that glob obj many thousands times a day.
I think the answer depends on what type of context store you are using.
For the default in-memory store, I don't believe it makes a difference because you are receiving a REFERENCE to the data. Whether you use the full variable or a sub-property won't make any difference to memory or performance as far as I know.
For the default file store, the whole variable is read into memory and the whole variable written to file periodically (if there is an update) - 30s by default I think? So again, I doubt there is any difference.
If you are using other store types though, it may be different and likely depends on whether the whole variable is kept in-memory all the time. Which I suspect they all will.
I do a similar thing where I gather information from mobile robots, it involves checking a DB for current jobs and last command sent, use HTTP requests to receive the current status, including position, battery voltage and current. I do this for several mobile robots, every few seconds depending on the activity they're doing (every 20 seconds if they're offline or non-responsive, every 10 if they're idle, every 3 if they're navigating, and every second if they are docking into a machine).
At the beginning of each loop, I read the last status object for each robot from global, and attach it into the msg, into a property that won't be overwritten by any of my subsequent nodes unless I want to (in my case, msg.agv) . Then I gather the information from different sources and update that object.
After that, I have my finite state machine transition logic, I make decisions about what's the next state, manage the jobs, produce alerts if any are required, and so on.
To close it up, I compare the previous status in the global context with my current msg.agv object, and file a log line in the DB if any critical parameters are changed or there is a significant update. After that, I upload the whole object.
I do this in several parallel threads for the mobile robots, and use a similar but simplified structure for other machines that do not require logic but only display parameters and throw alerts.
Yes I think our requirements are quite similar although mine one isn't as complex.
I have one Job for each update sample rate I need.
e.g.
1 Job for 10min
1 Job for 1min
1 Job for 10sec
...
All jobs manipulate the same object but different properties of that object.
If 1st job finishes simultaneous to 2nd job and/or 3rd job and every job writes the whole object back to global store, every time the last one will "win".
I can build in a random sleep before job startup and hope that no collision happens. But hoping is not a good solution
I also could use an external SQL-DB instead of variables. But in this case my main node-red instance for home automation (which is running on a raspi) depends on an external resource and will fail if my NAS (which provides SQL-Instance) is down for any reason. I'm trying to avoid such dependencies as far as possible.
I had a similar issue: the mobile robots get information about their status from the fleet manager, the jobs from a DB, and I have my own feedback from the logic. Everything must be inside a single object.
What you can do is a similar approach to what I mentioned, but you can sort the variables in smaller objects depending on their origin. That way, you can just overwrite that part of the object, and the rest is untouched. But you still can read the whole object if you want.
In my case as you can see in the previous picture, I have a single object for all the machines in production, separated by vendor/role. As you can see, I have 6 AGVs. Each one is an object, and it's overwritten and read separately. Inside each AGV, I have a sub-object for the configuration parameters (they're set once and never modified again), an AIC (all the data related to the operational logic), the plan (that knows if the AGV has a plan ready, and if it has been commited by the logic or not yet), an incubed sub-object (the feedback from the fleet manager), and a sub-object for errors, warnings, and information messages that will be commited to the log.
When you have a flow that handles part of the info, you can overwrite the whole object, or just one of these sub-objects.
By arranging things into sub-objects, you can overwrite only the part of the information handled by a branch (only the configuration, or the logic values, or the feedback, or whatever). This allows to have parallel branches handling separate parts of the object and updating only those.
At any given time, you can read the whole object and get the most updated version of all the parameters.
The point, I think, was a question as to whether it is better to do that or to get a reference once, update directly and then do a single set. As I said, I don't believe there is anything to be gained by using the more complex process.
