Hi, I´m worikng with the global context to stora an array of devices. For a device´s incoming message I use the global.get and filter on the whole devicelist array to find the specific object (device). I´m updating some values on the device and then in order to save the object (device) in the global context array, I use the global.set on the whole array. I don´t think this way is really optimised for performance, and would like to get som e ideas of how to just update one object in an array in the global context. How would you do it?
Why don't you think this is optimised? In fact this is likely the fastest possible method. When you call global.set you are only storing the reference (not the whole array).
See this recent post
Well... The array contains about 1500 objects, which all is quite large. When I call global.get I assume I get a reference to all objects in the array. I use the devices = global.get ... .findIndex((obj => obj.DeviceId == deviceId)); in order to find the object in the array I´m looking for. Then I do some updates on the object: devices[objIndex].LatestFcntUp = latestFcntUp;
Then I go: global.set("Devices",devices); And I figure I save the whole array with all objects in the global context. The thing is that I do this several times every second and I´m worried that this is not optimal for performance. I just want to update and save one object at the time that is. But how do I ensure that with global.set() ?
ok, so again, global.set only stores the reference so its not writing 1500 objects (unless you have persistent storage then it will write the whole shebang to file when it flushes to file every 30s)
If you insist on looking for an alternative then you can use global.set("myobject.subobject",value) BUT as Nick said in the post I linked to "If I had to guess, I'd say getting the whole object, then addressing the sub property directly then writing the whole object back would be mildly more efficient as you can let native JavaScript do the property lookup in the object, rather than have our code parse your property expression and dig into the object"
If you REALLY want an improvement - use a lookup object instead of searching an array.
e.g.
var devices = global.get("Devices") || {};
var device = devices[deviceId]; //Lookup a key instead of searching array
if(device) {
device.LatestFcntUp = latestFcntUp;
} else {
devices[deviceId] = {LatestFcntUp : latestFcntUp};
}
global.set("Devices", devices )
HOWEVER that said, remember, premature optimisation is the root of all evil - are you actually seeing an issue?
No, when you call global.get you a reference to one object, the array. The array then contains the references to the contained objects. When you use global.set, referencing the whole array, then again it just stores the reference to that array.
If, however, you are using persistent context, for example to the file system, then it occasionally flushes the whole structure out to disc. The maximum rate that that will happen at can be set as required.
I ran tests a few months back on an array with 10000 elements, It is faster to call the complete global object ie.
global.get("object")["element"] is faster then global.get("object.element") The bigger the object the slower it was.
So i would think global.set("object.element", value) would be less eficient.
First, many thanks for great input. I don´t insist an alternative if I´m doing it ok! Just wanna optimize as best as I can. About the optimizaton: "use a lookup object instead of searching an array". How do I do that with objects with peroperties like DeviceId, DevEui etc. I don´t see how this works: var device = devices[deviceId]; //Lookup a key instead of searching array.
Furthermore, I have enabled the "Persist contexts to disk" for my instance, this is because I can´t risk to loose the global context array. What are the proos and cons here. I guess that having it in memory will be much faster, or is it just the writing to disk every 30 sec that makes it slower? I´m using the global context as a cache at this point, to avoid roundtrips to the DB. Maybe some cache service DB or something would be a better way to do it (e.g. Redis)? Suggestions?
Essentially, this is the same as object assignment.
consider this...
var myObject = {};
myObject.a_new_property = 123
is the same as
var myObject = {};
myObject["a_new_property"] = 123
is the same as
var myObject = {};
var propName = "a_new_property";
myObject[propName] = 123
so you can also access the sub property like this...
var the_value = myObject["a_new_property"]; //123
or like this...
var device = devices[deviceId]; //no array search here - just a lookup
where devices is an object with devices key'd by name
where deviceId is a string with the name of the device that was assigned
Here is a demo...
https://repl.it/join/glljpnpb-stephenmclaughl
... this is the final line of the repl.it in case you dont open it...
Look up is 52.17971960357747 times faster than searching an array of 1500 (depending on the element you ask for) (note this changes for each execution but usually it is ~50x faster)
1 Like
It isn't any slower, the context you use is in memory exactly as normal, you just have the extra overhead of writing it to disk at whatever rate you specify, and the actual write will take place in a different process (I think) as the OS will cache it.
If you really do find that you need to optimise things, you are probably looking at using a database engine rather than a Node-RED variable.
As for having to search through arrays all the time, personally I always err to using Objects rather than arrays given that it is easy to get an array from an Object but not the other way around. With an object, you can simply do myObj.key or myObj[varThatHoldsKey], no searching/filtering/looping required.
If you need the array from the object then Object.values(myObj) does the trick.
1 Like
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
