Backup and restore Node-red

General
1

Can anyone confirm / improve on these backup and restore instructions?

Linux

Backup

Assuming that everything for your Node-red installation (flows, file based context, static content, any external scripts run from exec, etc) resides within ~/.node-red you can make a backup file nodered.tar with this command

cd 
sudo systemctl stop nodered
tar cvf nodered.tar --exclude=node_modules .node-red
sudo systemctl start nodered

Restore

With the backup file in the home directory

cd
sudo systemctl stop nodered
tar xvf nodered.tar
cd .node-red
npm install         # restores the node_modules folder which was excluded from the backup
sudo systemctl start nodered

Windows

Who knows?
2

cannot comment on Windows, but as for Linux what works fine are scripts by @TotallyInformation

1 Like
3

Thanks for showing me that @KarolisL, it never occurred to me to look for backup info in an installer!

Mind you, I see no restore script there...

4

:rofl:

Well that's because you really don't need one. If you need to restore, either rename your live instance and COPY the appropriate folder back to the live folder name. Or, you can restore any part(s) simply by copying and overwriting stuff.

If you really want a script, look in the backup script and simply reverse the rsync commands. :grinning:

5

I use a variation of a previously published backup to Dropbox to backup to a local directory. (Uses a dsm node) (I also backup to Dropbox)

This was designed to run on a Debian instance in a VM, hence the use of a function node to create the Date (creating it in Debian was beyond me as the Pi version didn't work)

The only remaining issue for me is how to exclude the node_modules folder in uibuilder - without specifying all the folders I DO want.

[{"id":"216f880395c9bdcf","type":"group","z":"8428164b1645b4b8","name":"Backup Node-RED Flows to Local Drive","style":{"label":true,"fill":"#e3f3d3","label-position":"n","color":"#000000"},"nodes":["8aaff02f6aa94d51","7ac4cb95d54817ad","52950de559414189","b5233c43e1cea7f1","e2dbf7db77d0c7ad","76dd9fe208e111cc","3903b94699774ca5","8d51bf8c3e6d0ccc"],"x":1434,"y":1019,"w":912,"h":154},{"id":"8aaff02f6aa94d51","type":"inject","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"reset","props":[{"p":"payload","v":"","vt":"date"},{"p":"topic","v":"reset","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reset","payload":"","payloadType":"date","x":1530,"y":1120,"wires":[["52950de559414189"]]},{"id":"7ac4cb95d54817ad","type":"inject","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"Start Backup","props":[{"p":"payload"},{"p":"backup","v":"start","vt":"str"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"15 03 * * *","once":false,"onceDelay":0.1,"topic":"Backup","payload":"IoTDevelopment","payloadType":"str","x":1560,"y":1060,"wires":[["e2dbf7db77d0c7ad"]]},{"id":"52950de559414189","type":"dsm","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"Zip Node-RED \\n Directory","sm_config":"{\n    \"triggerInput\": \"backup\",\n    \"stateOutput\": \"postState\",\n    \"currentState\": \"step1\",\n    \"states\": {\n        \"step1\": {\n            \"start\": \"step2\",\n            \"reset\": \"step1\"\n        },\n        \"step2\": {\n            \"zip\": \"step3\",\n            \"reset\": \"step1\"\n        },\n        \"step3\": {\n            \"upload\": \"step1\",\n            \"reset\": \"step1\"\n        }\n    },\n    \"data\": {\n    },\n    \"methods\": {\n        \"init\": [\n            \"sm.udir = RED.settings.userDir + '/';\",\n            \"sm.exec = require('child_process').exec;\",\n            \"sm.hostname = require('os').hostname();\",\n            \"sm.unlink = require('fs').unlink;\"\n        ],\n        \"start\": [\n            \"/* delete old backup file */\",\n            \"var zipfile = sm.udir + 'node-red.zip';\",\n            \n            \"sm.fill = 'grey';\",\n            \"sm.text = 'deleting';\",\n            \"output = false;\",\n\n            \"sm.unlink(zipfile, function (err) {\",\n            \"   if (err) {\",\n            \"       node.warn('no file '+zipfile);\",\n            \"   }\",\n            \"   resume('zip', msg);\",\n            \"});\"\n        ],\n        \"zip\": [\n            \"let pre = ' ' + sm.udir;\",\n            \"let cmd = 'zip -r';\",\n            \"cmd += pre + 'node-red.zip';\",\n            \"cmd += pre + 'flows*.json';\",\n            \"cmd += pre + '.config*.json';\",\n            \"cmd += pre + 'settings.js';\",\n            \"cmd += pre + 'package.json';\",\n            \"cmd += pre + 'package-lock.json';\",\n            \"cmd += pre + '.config.modules.json';\",\n            \"cmd += pre + 'lib/*';\",\n            \"cmd += pre + 'context/*';\",\n            \"cmd += pre + 'cronplusdata/*';\",\n            \"cmd += pre + 'uibuilder/*';\",\n\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'zipping';\",\n            \"output = false;\",\n            \n            \"sm.exec(cmd, function(error, stdout, stderr) {\",\n            \"   if (error) {\",\n            \"       node.warn(error);\",\n            \"   } else {\",\n            \"       resume('upload', msg);\",\n            \"   }\",\n            \"});\"\n        ],\n        \"upload\": [\n            \"let localFilename = sm.udir + 'node-red.zip';\",\n            \"let targetDirectory = '/home/pi/documents/data/';\",\n            \"msg.payload = localFilename + ' ' + targetDirectory + msg.payload;\",\n\n            \"sm.fill = 'green';\",\n            \"sm.text = sm.day;\"\n        ],\n        \"reset\": [\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'reset';\",\n            \"output = false;\"\n        ],\n        \"status\": {\n            \"fill\": {\n                \"get\": \"sm.fill\"\n            },\n            \"shape\": \"dot\",\n            \"text\": {\n                \"get\": \"sm.text\"\n            }\n        }\n    }\n}","x":1900,"y":1120,"wires":[["76dd9fe208e111cc","8d51bf8c3e6d0ccc"]],"info":"            \"sm.ts = msg.payload.ts;\",\r\n            \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\r\n            \"sm.day = msg.payload.day;\",            \r\n            \"msg.filename = msg.payload + '_' + sm.day + '_nodered.zip';\",\r\n\r\n            "},{"id":"b5233c43e1cea7f1","type":"debug","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"Copy Output","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2230,"y":1120,"wires":[]},{"id":"e2dbf7db77d0c7ad","type":"function","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"Create Filename","func":"/** \n * Input\n *  mgs.payload     -   Name of system to be backed up\n */\n\nconst currentDate = new Date()\nconst weekDays = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday']\nconst currentDay = weekDays[currentDate.getDay()]\n\nmsg.payload = msg.payload + '_' + currentDay + '_nodered.zip'\n\nreturn msg","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1740,"y":1060,"wires":[["52950de559414189"]]},{"id":"76dd9fe208e111cc","type":"exec","z":"8428164b1645b4b8","g":"216f880395c9bdcf","command":"cp ","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"Copy File","x":2080,"y":1120,"wires":[["b5233c43e1cea7f1"],[],[]]},{"id":"3903b94699774ca5","type":"debug","z":"8428164b1645b4b8","g":"216f880395c9bdcf","name":"DSM Output","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2230,"y":1060,"wires":[]},{"id":"8d51bf8c3e6d0ccc","type":"junction","z":"8428164b1645b4b8","g":"216f880395c9bdcf","x":2020,"y":1060,"wires":[["3903b94699774ca5"]]}]
6

Not checked your flow. But if you are using node.js 16+, you can use the experimental cp or one of its variants - have a look at the docs for node's fs module. Or, if you want a slightly easier life, try out the fs-extra package - Node-RED uses this itself. Those functions have include and exclude filters.

I have been using fs-extra in UIBUILDER and it is good. But I'm trying to reduce the number of dependencies at the moment so it will eventually go.

7

I assume you mean exclude the node_modules in the DSM node config??
I've previously asked the same, but not yet found an answer, and like you, I have to list each file or folder that I DO need which is a pain.

If you do find a solution please let me know, and I'll update my original topic - node-RED Backup Flow

8

The backup flows linked above use zip -r, while my minimal command at the top uses tar -c.

With tar you can use the --exclude=node_modules parameter. thus archiving the entire .node-red directory.
And with zip it seems you have to list all the required files. The version I have grabs these files:

 flows*.json
 .config*.json
 settings.js
 package.json
 package-lock.json
 .config.modules.json
 lib/*
 context/*
 cronplusdata/*
 uibuilder/*
  • Are there advantages to using zip over tar (or other archive command)?
  • Does zip preserve the ownership of files? I'm thinking of the current advice to chown root:root settings.js for security.
  • If a user chose to call their flow file eg myflowfile.json, it would not be covered by this default list. Similarly any static content directory will not be caught. Yet a backup flow could determine these by parsing settings.js.
  • Is there anything in settings.js to indicate that uibuilder or cronplusdata directories exist?
  • What other Node-red add-ons would one hope an automatic backup program would know about?
9

No, not really. As long as you gzip the tar file.

For UIBUILDER, there would be a uibuilder section in settings.js - but only if you wanted any of the custom settings - you put it there, it isn't automatic.

The package.json file in the userDir folder tells you what is installed. As it is JSON, it is pretty easily parsed. But there is, in any case, no harm in trying to backup the extra folders, it shouldn't fail if they aren't there.

If you wanted to separate out the uibuilder backup, you could always move it to a different folder. The custom settings in settings.js lets you do that.

Oh, and remember that uibuilder may introduce more node_modules folders. One set under the uibuilder root folder but each uibuilder node instance has the opportunity to also install its own modules. So you should generally be excluding ALL node_module folders.

10

Only that I understand zip and have never used tar (other than as directed in a how-to-do-it). I can check the zipped files in Windows easily. Looking at tar as used in Windows it seems a bit limited.

11

Apparently you can exclude files with zip

-x "node_modules*"

Will be giving it a go.

Edit. Tried it and it worked. Probably still a bit long winded. Replace the zip section in the dsm node

        "zip": [
            "let pre = ' ' + sm.udir;",
            "let cmd = 'zip -r';",
            "cmd += pre + 'node-red.zip';",
            "cmd += pre + '*';",
            "cmd += pre + '.config*.json';",
            "cmd += ' -x ' + '\"**/node_modules/*\"';",
            "cmd += ' -x' + pre + '\"*.backup\"';",
            "cmd += ' -x' + pre + '\"*.zip\"';",
            "msg.command = cmd;",

            "sm.fill = 'grey';",
            "sm.text = 'zipping';",
            "output = false;",

            "sm.exec(cmd, function(error, stdout, stderr) {",
            "   if (error) {",
            "       node.warn(error);",
            "   } else {",
            "       resume('upload', msg);",
            "   }",
            "});"
        ],
1 Like
12

It's possible I broke something trying your solution Jeff.

The paths in the zip file start from home (not /home)

pi@zerotwogreen:~ $ unzip -v node-red.zip | head -5
Archive:  node-red.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Stored        0   0% 2024-04-02 12:55 00000000  home/pi/.node-red/context/
       0  Stored        0   0% 2024-04-02 12:55 00000000  home/pi/.node-red/context/78c9efc5cc8c54ee/

So if I unzip the file in /home/pi it creates the wrong structure /home/pi/home/pi/.node-red
Is that what you get or have I broken it?

I think absolute pathnames would be wrong, then you couldn't restore to a different directory eg ~/oldnodered. Not sure of the best approach.

Has anyone used this backup approach on Windows? Does it need Windows 10+? WSL?

13

Nope, that is how it would work for me. I have never noticed because I just copy the zip file to a Windows directory and then go to the .node-red directory using normal Windows file manager and copy the files. I have never run unzip.

A good point on the restoring using unzip so I am going to investigate further

14

My flow to make a compressed backup of Node-red on Linux.

If you only inject a payload, it will backup ~/.node-red to ~/nodered.tar.gz but you can override these with msg.nodereddirectory and msg.backupfilename.

Everything in the directory except for node_modules* will be included in the archive

image
image1002Ă—190 22.5 KB

[{"id":"7f20ad1a9174e987","type":"tab","label":"Backup Node-red","disabled":false,"info":"","env":[]},{"id":"f854b7672172886b","type":"inject","z":"7f20ad1a9174e987","name":"Node-red location & backup file name","props":[{"p":"payload"},{"p":"nodereddirectory","v":".node-red","vt":"str"},{"p":"backupfilename","v":"$moment().format(\"YYYYMMDD.HHmm\") & \"nr.tar.gz\"","vt":"jsonata"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"go","payloadType":"str","x":190,"y":80,"wires":[["40ec46521782eb90"]]},{"id":"40ec46521782eb90","type":"template","z":"7f20ad1a9174e987","name":"Backup script","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"# To override defaults pass these as msg properties\nNODEREDDIR={{{nodereddirectory}}}\nBACKUPTO={{{backupfilename}}}\n\n# Default location\nif [ -z $NODEREDDIR ]\nthen\nNODEREDDIR=.node-red\nfi\n\n# Default backup file\nif [ -z $BACKUPTO ]\nthen\n   BACKUPTO=nodered.tar.gz\nfi\n\n# Don't overwrite backupfile\n#if [ -s \"$BACKUPTO\" ]\n#then\n#  echo \"Error: $BACKUPTO already exists\" >&2\n#  exit 1\n#fi\n\necho \"$BACKUPTO: $(tar -cvzf $BACKUPTO --exclude=\"node_modules*\" $NODEREDDIR | wc -l) files\"","output":"str","x":260,"y":140,"wires":[["cf19faaabdfb2dbe"]]},{"id":"cf19faaabdfb2dbe","type":"exec","z":"7f20ad1a9174e987","command":"","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":430,"y":140,"wires":[["fb47ca1393ddc5aa"],["3c4d7bf52fe7f7f6"],[]]},{"id":"fb47ca1393ddc5aa","type":"debug","z":"7f20ad1a9174e987","name":"Count files saved","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":610,"y":100,"wires":[]},{"id":"3c4d7bf52fe7f7f6","type":"debug","z":"7f20ad1a9174e987","name":"Errors","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":570,"y":160,"wires":[]}]

Listing backup contents

tar -tvf nodered.tar.gz

Restoring from the backup

If you backed up the default .node-red directory to the default file, the backup will have relative pathnames eg .node-red/settings.js To restore, put the backup file in the home directory ~ and run 
tar -xf nodered.tar.gz
cd .node-red
npm install

Note that if your settings file is owned by root for security, you need to run

sudo tar -xf nodered.tar.gz