I read a post a few days ago by @Trying_to_learn about losing his node-RED flow, which had taken him no doubt many hours to construct, and as his backup was on the same machine - he lost that too.
So... I've spent a few days looking at off-site storage, and stumbled upon node-red-contrib-dsm
which was written by @cflurin complete with an example flow to perform automated backups to dropbox.
I played about with the configuration, and made some changes to the way that the backups are stored, basically to stop dropbox from filling up, yet conduct a nightly backup and be able to retrieve any one of those backups for 30 days (30 backups). The older ones are automatically weeded by dropbox, so it's 'fit & forget'.
Instead of the backups being named as a timestamp, they are named as the day of the week - Monday_nodered.zip
, Tuesday_nodered.zip
etc.
Looking at the dropbox web dashboard, you will only see 7 files (the current 7 days), but if you go to 'Version History', you can see & download the previous versions of that day, so for example every Monday's backup for the past 30 days.
Anyway, here's the flow, it uses node-red-contrib-dsm
, node-red-node-dropbox
and a couple of inject nodes.
Also you need to ensure that zip
is installed, try zip --help
and if help isn't displayed, then install it as sudo apt-get install zip unzip
.
The flow should work out of the box, once you've configured the dropbox node (leave the Filename & Local Filename boxes empty).
PS The inject node is set to run the flow 3.15am every night.
NOTE: See edit 17/4/21 below for +v1.3.0 version
[{"id":"7868f6a6.feb728","type":"dropbox out","z":"4487e413.bb781c","dropbox":"","filename":"","localFilename":"","name":"","x":430,"y":780,"wires":[]},{"id":"72f00992.3036f8","type":"inject","z":"4487e413.bb781c","name":"backup","props":[{"p":"payload","v":"","vt":"date"},{"p":"topic","v":"start","vt":"string"}],"repeat":"","crontab":"15 03 * * *","once":false,"onceDelay":0.1,"topic":"start","payload":"","payloadType":"date","x":132,"y":760,"wires":[["b0bc8692.a8e948"]]},{"id":"8b602104.6656a","type":"inject","z":"4487e413.bb781c","name":"reset","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reset","payload":"","payloadType":"date","x":140,"y":800,"wires":[["b0bc8692.a8e948"]]},{"id":"b0bc8692.a8e948","type":"dsm","z":"4487e413.bb781c","name":"backup","sm_config":"{\n \"currentState\": \"step1\",\n \"states\": {\n \"step1\": {\n \"start\": \"step2\",\n \"reset\": \"step1\"\n },\n \"step2\": {\n \"zip\": \"step3\",\n \"reset\": \"step1\"\n },\n \"step3\": {\n \"upload\": \"step1\",\n \"reset\": \"step1\"\n }\n },\n \"data\": {\n },\n \"methods\": {\n \"init\": [\n \"sm.udir = RED.settings.userDir + '/';\",\n \"sm.exec = require('child_process').exec;\",\n \"sm.hostname = require('os').hostname();\",\n \"sm.unlink = require('fs').unlink;\"\n ],\n \"start\": [\n \"/* delete old zip file */\",\n \"var zipfile = sm.udir+'node-red.zip';\",\n \n \"sm.fill = 'grey';\",\n \"sm.text = 'deleting';\",\n \"output = false;\",\n\n \"sm.unlink(zipfile, function (err) {\",\n \" if (err) {\",\n \" node.warn('no file '+zipfile);\",\n \" }\",\n \" resume('zip', msg);\",\n \"});\"\n ],\n \"zip\": [\n \"var pre = ' ' + sm.udir;\",\n \"var cmd = 'zip -r';\",\n \"cmd += pre + 'node-red.zip';\",\n \"cmd += pre + 'flows*.json';\",\n \"cmd += pre + '.config*.json';\",\n \"cmd += pre + 'settings.js';\",\n \"cmd += pre + 'package.json';\",\n \"cmd += pre + 'package-lock.json';\",\n \"cmd += pre + 'lib/*';\",\n\n \"sm.fill = 'grey';\",\n \"sm.text = 'zipping';\",\n \"output = false;\",\n \n \"sm.exec(cmd, function(error, stdout, stderr) {\",\n \" if (error) {\",\n \" node.warn(error);\",\n \" } else {\",\n \" resume('upload', msg);\",\n \" }\",\n \"});\"\n ],\n \"upload\": [\n \"sm.ts = new Date();\",\n \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\n \"sm.day = sm.days[sm.ts.getDay()];\",\n \"msg.filename = sm.day + '_nodered.zip';\",\n \"msg.localFilename = sm.udir+'node-red.zip';\",\n \"sm.fill = 'green';\",\n \"sm.text = sm.day;\"\n ],\n \"reset\": [\n \"sm.fill = 'grey';\",\n \"sm.text = 'reset';\",\n \"output = false;\"\n ],\n \"status\": {\n \"fill\": {\n \"get\": \"sm.fill\"\n },\n \"shape\": \"dot\",\n \"text\": {\n \"get\": \"sm.text\"\n }\n }\n }\n}","x":285,"y":780,"wires":[["7868f6a6.feb728"]]}]
EDIT 20/10/2020 - Flow amended to ensure that all of the config files are backed up following the changes to node-RED in v1.2.0
EDIT 17/4/2021 - Flow updated below to include the additional package.json
contained within the externalModules
folder, which was introduced in v1.3.0
[{"id":"7868f6a6.feb728","type":"dropbox out","z":"b3b413d1.05b1b","dropbox":"","filename":"","localFilename":"","name":"","x":470,"y":1605,"wires":[]},{"id":"72f00992.3036f8","type":"inject","z":"b3b413d1.05b1b","name":"backup","props":[{"p":"payload","v":"","vt":"date"},{"p":"topic","v":"start","vt":"string"}],"repeat":"","crontab":"15 03 * * *","once":false,"onceDelay":0.1,"topic":"start","payload":"","payloadType":"date","x":172,"y":1585,"wires":[["b0bc8692.a8e948"]]},{"id":"8b602104.6656a","type":"inject","z":"b3b413d1.05b1b","name":"reset","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reset","payload":"","payloadType":"date","x":180,"y":1625,"wires":[["b0bc8692.a8e948"]]},{"id":"b0bc8692.a8e948","type":"dsm","z":"b3b413d1.05b1b","name":"backup","sm_config":"{\n \"currentState\": \"step1\",\n \"states\": {\n \"step1\": {\n \"start\": \"step2\",\n \"reset\": \"step1\"\n },\n \"step2\": {\n \"zip\": \"step3\",\n \"reset\": \"step1\"\n },\n \"step3\": {\n \"upload\": \"step1\",\n \"reset\": \"step1\"\n }\n },\n \"data\": {\n },\n \"methods\": {\n \"init\": [\n \"sm.udir = RED.settings.userDir + '/';\",\n \"sm.exec = require('child_process').exec;\",\n \"sm.hostname = require('os').hostname();\",\n \"sm.unlink = require('fs').unlink;\"\n ],\n \"start\": [\n \"/* delete old zip file */\",\n \"var zipfile = sm.udir+'node-red.zip';\",\n \n \"sm.fill = 'grey';\",\n \"sm.text = 'deleting';\",\n \"output = false;\",\n\n \"sm.unlink(zipfile, function (err) {\",\n \" if (err) {\",\n \" node.warn('no file '+zipfile);\",\n \" }\",\n \" resume('zip', msg);\",\n \"});\"\n ],\n \"zip\": [\n \"var pre = ' ' + sm.udir;\",\n \"var cmd = 'zip -r';\",\n \"cmd += pre + 'node-red.zip';\",\n \"cmd += pre + 'flows*.json';\",\n \"cmd += pre + '.config*.json';\",\n \"cmd += pre + 'settings.js';\",\n \"cmd += pre + 'package.json';\",\n \"cmd += pre + 'package-lock.json';\",\n \"cmd += pre + 'externalModules/package.json';\",\n \"cmd += pre + 'externalModules/package-lock.json';\",\n \"cmd += pre + 'lib/*';\",\n\n \"sm.fill = 'grey';\",\n \"sm.text = 'zipping';\",\n \"output = false;\",\n \n \"sm.exec(cmd, function(error, stdout, stderr) {\",\n \" if (error) {\",\n \" node.warn(error);\",\n \" } else {\",\n \" resume('upload', msg);\",\n \" }\",\n \"});\"\n ],\n \"upload\": [\n \"sm.ts = new Date();\",\n \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\n \"sm.day = sm.days[sm.ts.getDay()];\",\n \"msg.filename = sm.day + '_nodered.zip';\",\n \"msg.localFilename = sm.udir+'node-red.zip';\",\n \"sm.fill = 'green';\",\n \"sm.text = sm.day;\"\n ],\n \"reset\": [\n \"sm.fill = 'grey';\",\n \"sm.text = 'reset';\",\n \"output = false;\"\n ],\n \"status\": {\n \"fill\": {\n \"get\": \"sm.fill\"\n },\n \"shape\": \"dot\",\n \"text\": {\n \"get\": \"sm.text\"\n }\n }\n }\n}","x":325,"y":1605,"wires":[["7868f6a6.feb728"]]}]
EDIT 21/7/2021 - Flow updated below to accommodate changes made in node-RED v2.0.
The externalModules
is now depreciated, and the file .config.modules.json
has been added to the backup set.
[{"id":"7868f6a6.feb728","type":"dropbox out","z":"b3b413d1.05b1b","dropbox":"","filename":"","localFilename":"","name":"","x":440,"y":1380,"wires":[]},{"id":"72f00992.3036f8","type":"inject","z":"b3b413d1.05b1b","name":"backup","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"15 03 * * *","once":false,"onceDelay":0.1,"topic":"start","payloadType":"date","x":137,"y":1360,"wires":[["b0bc8692.a8e948"]]},{"id":"8b602104.6656a","type":"inject","z":"b3b413d1.05b1b","name":"reset","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reset","payload":"","payloadType":"date","x":145,"y":1400,"wires":[["b0bc8692.a8e948"]]},{"id":"b0bc8692.a8e948","type":"dsm","z":"b3b413d1.05b1b","name":"backup","sm_config":"{\n \"currentState\": \"step1\",\n \"states\": {\n \"step1\": {\n \"start\": \"step2\",\n \"reset\": \"step1\"\n },\n \"step2\": {\n \"zip\": \"step3\",\n \"reset\": \"step1\"\n },\n \"step3\": {\n \"upload\": \"step1\",\n \"reset\": \"step1\"\n }\n },\n \"data\": {\n },\n \"methods\": {\n \"init\": [\n \"sm.udir = RED.settings.userDir + '/';\",\n \"sm.exec = require('child_process').exec;\",\n \"sm.hostname = require('os').hostname();\",\n \"sm.unlink = require('fs').unlink;\"\n ],\n \"start\": [\n \"/* delete old zip file */\",\n \"var zipfile = sm.udir+'node-red.zip';\",\n \n \"sm.fill = 'grey';\",\n \"sm.text = 'deleting';\",\n \"output = false;\",\n\n \"sm.unlink(zipfile, function (err) {\",\n \" if (err) {\",\n \" node.warn('no file '+zipfile);\",\n \" }\",\n \" resume('zip', msg);\",\n \"});\"\n ],\n \"zip\": [\n \"var pre = ' ' + sm.udir;\",\n \"var cmd = 'zip -r';\",\n \"cmd += pre + 'node-red.zip';\",\n \"cmd += pre + 'flows*.json';\",\n \"cmd += pre + '.config*.json';\",\n \"cmd += pre + 'settings.js';\",\n \"cmd += pre + 'package.json';\",\n \"cmd += pre + 'package-lock.json';\",\n \"cmd += pre + '.config.modules.json';\",\n \"cmd += pre + 'lib/*';\",\n\n \"sm.fill = 'grey';\",\n \"sm.text = 'zipping';\",\n \"output = false;\",\n \n \"sm.exec(cmd, function(error, stdout, stderr) {\",\n \" if (error) {\",\n \" node.warn(error);\",\n \" } else {\",\n \" resume('upload', msg);\",\n \" }\",\n \"});\"\n ],\n \"upload\": [\n \"sm.ts = new Date();\",\n \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\n \"sm.day = sm.days[sm.ts.getDay()];\",\n \"msg.filename = sm.day + '_nodered.zip';\",\n \"msg.localFilename = sm.udir+'node-red.zip';\",\n \"sm.fill = 'green';\",\n \"sm.text = sm.day;\"\n ],\n \"reset\": [\n \"sm.fill = 'grey';\",\n \"sm.text = 'reset';\",\n \"output = false;\"\n ],\n \"status\": {\n \"fill\": {\n \"get\": \"sm.fill\"\n },\n \"shape\": \"dot\",\n \"text\": {\n \"get\": \"sm.text\"\n }\n }\n }\n}","x":290,"y":1380,"wires":[["7868f6a6.feb728"]]}]