node-RED Backup Flow

Just came here following the backup dicussion in another thread and decided to try it out on my win10 machine

It didn't work because win10 has no command line tool called zip

But it does have one called tar so I just did a simple mode of the dsm node and it works fine :slight_smile:
image

here is the win10 alternative dsm node only

[{"id":"20a4ec54.6c6454","type":"dsm","z":"fa1b22c7.a8823","name":"backup","sm_config":"{\n    \"currentState\": \"step1\",\n    \"states\": {\n        \"step1\": {\n            \"start\": \"step2\",\n            \"reset\": \"step1\"\n        },\n        \"step2\": {\n            \"zip\": \"step3\",\n            \"reset\": \"step1\"\n        },\n        \"step3\": {\n            \"upload\": \"step1\",\n            \"reset\": \"step1\"\n        }\n    },\n    \"data\": {\n    },\n    \"methods\": {\n        \"init\": [\n            \"sm.udir = RED.settings.userDir + '/';\",\n            \"sm.exec = require('child_process').exec;\",\n            \"sm.hostname = require('os').hostname();\",\n            \"sm.unlink = require('fs').unlink;\"\n        ],\n        \"start\": [\n            \"/* delete old zip file */\",\n            \"var zipfile = sm.udir+'node-red.zip';\",\n            \n            \"sm.fill = 'grey';\",\n            \"sm.text = 'deleting';\",\n            \"output = false;\",\n\n            \"sm.unlink(zipfile, function (err) {\",\n            \"   if (err) {\",\n            \"       node.warn('no file '+zipfile);\",\n            \"   }\",\n            \"   resume('zip', msg);\",\n            \"});\"\n        ],\n        \"zip\": [\n            \"var pre = ' ' + sm.udir;\",\n            \"var cmd = 'tar.exe -a -c -f';\",\n            \"cmd += pre + 'node-red.zip';\",\n            \"cmd += pre + 'flows*.json';\",\n            \"cmd += pre + '.config.json';\",\n            \"cmd += pre + '.sessions.json';\",\n            \"cmd += pre + 'settings.js';\",\n            \"cmd += pre + 'package.json';\",\n            \"cmd += pre + 'package-lock.json';\",\n            \"cmd += pre + 'lib/*';\",\n\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'zipping';\",\n            \"output = false;\",\n            \n            \"sm.exec(cmd, function(error, stdout, stderr) {\",\n            \"   if (error) {\",\n            \"       node.warn(error);\",\n            \"   } else {\",\n            \"       resume('upload', msg);\",\n            \"   }\",\n            \"});\"\n        ],\n        \"upload\": [\n            \"sm.ts = new Date();\",\n            \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\n            \"sm.day = sm.days[sm.ts.getDay()];\",\n            \"msg.filename = sm.day + '_nodered.zip';\",\n            \"msg.localFilename = sm.udir+'node-red.zip';\",\n            \"sm.fill = 'green';\",\n            \"sm.text = sm.day;\"\n        ],\n        \"reset\": [\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'reset';\",\n            \"output = false;\"\n        ],\n        \"status\": {\n            \"fill\": {\n                \"get\": \"sm.fill\"\n            },\n            \"shape\": \"dot\",\n            \"text\": {\n                \"get\": \"sm.text\"\n            }\n        }\n    }\n}","x":355,"y":160,"wires":[["44702648.1f2418"]]}]
2 Likes

Just a note, the user may need to install zip. All my installs have unzip but none had zip

Hi @Buckskin, I've moved your post here because that issue was raised earlier today by @cymplecy, and also other users trying this solution will read it.

Cheers, thanks for the flow. Works well.

I have no clue about the DSM node so I have changed the save filename (to distinguish WHICH Pi I am backing up) in a Change node between it and the Dropbox node. Is there any way of specifying something in the Inject node to do the same thing?

Probably something that the DSM node author @cflurin may be able to answer.
I'm not sure that the msg.payload is accessible from within the node itself.

I've just tried adding "sm.id = msg.payload;", but it didn't seem to like it!

The flow did its first auto backup early this morning :slight_smile:
image

I then managed to modify it to prefix the file name with the msg.payload from the inject node
image

image

Modified DSM node only

[{"id":"20a4ec54.6c6454","type":"dsm","z":"fa1b22c7.a8823","name":"backup","sm_config":"{\n    \"currentState\": \"step1\",\n    \"states\": {\n        \"step1\": {\n            \"start\": \"step2\",\n            \"reset\": \"step1\"\n        },\n        \"step2\": {\n            \"zip\": \"step3\",\n            \"reset\": \"step1\"\n        },\n        \"step3\": {\n            \"upload\": \"step1\",\n            \"reset\": \"step1\"\n        }\n    },\n    \"data\": {\n    },\n    \"methods\": {\n        \"init\": [\n            \"sm.udir = RED.settings.userDir + '/';\",\n            \"sm.exec = require('child_process').exec;\",\n            \"sm.hostname = require('os').hostname();\",\n            \"sm.unlink = require('fs').unlink;\"\n        ],\n        \"start\": [\n            \"/* delete old zip file */\",\n            \"var zipfile = sm.udir+'node-red.zip';\",\n            \n            \"sm.fill = 'grey';\",\n            \"sm.text = 'deleting';\",\n            \"output = false;\",\n\n            \"sm.unlink(zipfile, function (err) {\",\n            \"   if (err) {\",\n            \"       node.warn('no file '+zipfile);\",\n            \"   }\",\n            \"   resume('zip', msg);\",\n            \"});\"\n        ],\n        \"zip\": [\n            \"var pre = ' ' + sm.udir;\",\n            \"var cmd = 'tar -a -c -f';\",\n            \"cmd += pre + 'node-red.zip';\",\n            \"cmd += pre + 'flows*.json';\",\n            \"cmd += pre + '.config.json';\",\n            \"cmd += pre + '.sessions.json';\",\n            \"cmd += pre + 'settings.js';\",\n            \"cmd += pre + 'package.json';\",\n            \"cmd += pre + 'package-lock.json';\",\n            \"cmd += pre + 'lib/*';\",\n\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'zipping';\",\n            \"output = false;\",\n            \n            \"sm.exec(cmd, function(error, stdout, stderr) {\",\n            \"   if (error) {\",\n            \"       node.warn(error);\",\n            \"   } else {\",\n            \"       resume('upload', msg);\",\n            \"   }\",\n            \"});\"\n        ],\n        \"upload\": [\n            \"sm.ts = new Date();\",\n            \"sm.days = ['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'];\",\n            \"sm.day = sm.days[sm.ts.getDay()];\",\n            \"msg.filename = msg.payload + '_' + sm.day + '_nodered.zip';\",\n            \"msg.localFilename = sm.udir + 'node-red.zip';\",\n            \"sm.fill = 'green';\",\n            \"sm.text = sm.day;\"\n        ],\n        \"reset\": [\n            \"sm.fill = 'grey';\",\n            \"sm.text = 'reset';\",\n            \"output = false;\"\n        ],\n        \"status\": {\n            \"fill\": {\n                \"get\": \"sm.fill\"\n            },\n            \"shape\": \"dot\",\n            \"text\": {\n                \"get\": \"sm.text\"\n            }\n        }\n    }\n}","x":355,"y":160,"wires":[["44702648.1f2418"]]}]
2 Likes

I think that I must have been overthinking it last night :shushing_face:

1 Like

You probably added it in the init method.
This method is called once on deploy/start, as the input isn't triggered the msg object is undefined. All other methods can access the msg object (payload, topic etc.).

1 Like

JFI I tried my modified DSM node out on a Pi but it kept erroring out

The tar command didn't like the fact that there was no .sessions.json file on my Pi

It worked fine once I deleted it from the list of backup files

Is it important to backup .sessions.json? what is that file for?

I did give some thought to removing that, because as far as I'm aware, it's a file that keeps users logged in, with the assumption that if deleted, all users would again need to login, and the file would be recreated (I'm happy to be corrected @dceejay ).
If so, maybe restoring the .sessions.json could create more problems than not?

But... it was in @cflurin's original DSM configuration, and this post, which persuaded me to keep it in.
Maybe it should go??

As far as I know the .session.json can be omitted.
I thought it would not be wrong to backup .session.json.
At restore you can decide to use it or not.

What is the consensus re .sessions.json, does it stay, or shall I remove it from the example flow config that I posted in post #1?

ie. Is it necessary for .sessions.json to form part of a node-RED backup archive.

No responses... so it has now been removed from the example flow above.

Missed your original question. The sessions file contains the active login tokens for the editor. No need to back it up.

2 Likes

Since node-RED v1.2.0, the config.json file has been split up into seperate files, so to capture those files for the backup, please change this line in the node-red-contrib-dsm node;

"cmd += pre + '.config.json';",

to

"cmd += pre + '.config*.json';",

...which includes the wildcard character * to ensure that all of the conf files are included.

3 Likes

Flow updated (see post #1 above) to include the additional package.json & package-lock.json'contained within the externalModules folder, which was introduced in v1.3.0.

The changes in the DSM node are the inclusion of;

"cmd += pre + 'externalModules/package.json';",
"cmd += pre + 'externalModules/package-lock.json';",

Flow updated (see post #1 above) to accommodate changes made in node-RED v2.0.0.
The package.json & package-lock.json' contained within the externalModules folder, which was introduced in node-RED v1.3.0 has now been discontinued, and a .config.modules.json file added to manage the external modules instead.

The changes in the DSM node therefore are the removal of lines;

"cmd += pre + 'externalModules/package.json';",
"cmd += pre + 'externalModules/package-lock.json';",

and the addition of;

"cmd += pre + '.config.modules.json';",
1 Like

Paul, I'm using projects, are you covering all files required also in this case?

Hi Walter, I haven't added projects, but it shouldn't be too difficult to add.
I don't use projects, but I'm assuming that the default node-RED structure would be;

.node-red
         |__ projects
                     |__ my_first_project
                     |__ my_second_project

...and the projects subfolders will include the respective project's flow files, etc.

Is that correct?

If so, you would need to add this at line 52 in the dsm node

"cmd += pre + 'projects/*';",

so it looks like this

projects

Adding that line will add all sub-folders & files within the projects folder to the zip backup file.

Hope that helps!