Hello,
so I have this task - after about 8 years of running NR used for aggregating data we finally decided to migrate from v1.0.6 to latest. Ofcourse the system have to run 24/7 without any longer shutdowns and through the years there is huge collection of different services, protocols, etc. So I have installed a new server on which I slowly transfer every functionality. But now I found out that some of function nodes use their context storage as cumulative history (for example there are functions that calculate total produced energy from power).
So my question: is there any easy and straightforward solution to transfer this context data between servers when they are constantly updated?
I found the context files in .nodered folder, so I can copy them. But there will be certain time window when the incoming data will be lost. I was thinking about some context transfer via tcp comm right in nodered, but that would work only for flow and global context, not the node context, unless I rewrite all the functions to restore the data in flow context.
Probably it will be okay if few minutes of data are lost, but it would be better to minimize this time as much as possible.
If you can stand a short downtime. One possibility would be to set up the new server and set up a ready-to-go script to copy the context files. Then use a Proxy server change - again, scripting would be the quickest approach.
Start the new Node-RED
Close the old Node-RED
Run the copy script
Switch the Proxy from the old to new Node-RED (might need to reload the proxy config or restart it.)
That should generally only take a few seconds to fully complete. I doubt though that even that would absolutely guarantee no data loss at all.
There might be other things that could be done depending on how your incoming data feeds are connected. An obvious improvement would be to ensure everything is coming in via MQTT - then you could potentially cut out the proxy switch so you would only need steps 2 and 3. really.
One other alternative might be to switch to a timeseries DB such as InfluxDB. That way, while you might have to live with some missing stats for a short while, once you have finalised you re-engineered flows so that new data is going straight into the DB, you could then process the previous data into the DB using a separate flow.
Thank you, the scripting would really be helpful.
There are few other problems, as you mentioned the data feeds - incoming data are supplied via Modbus TCP so I probably have to close the comm on old server and open on new. Simultaneous access seems not working, or I have bad configuration of network/firewall access from new server, have to check that. The outgoing data are fed into some sort of DB via mqtt and the mqtt node is only one, common for every flow of this kind, connected by links. So I have to manage it that there are not data sent from both servers simultaneously, but also I have to transfer all flows at once because of that single mqtt node. My intention was to transfer it flow by flow for easier compatibility control, but that seems not to be possible.
Good point. But as I wrote, the final data are actually stored in external DB connected via MQTT. The context is here used for calculating cumulative value. Otherwise I would have to read out the last value from db, add new value and store it back. Or is there some way to write data to DB cumulatively?
Both Node-RED and the DB are backed up.
