Is it in any way possible to use Dropbox in one of the following ways?
I need a tool to be able to buffer files locally, and upload them to Dropbox - and this works fine with node-red-node-dropbox, but if my internet connection goes down, my file(s) are not uploaded.
Any help is appreciated. 
but if my internet connection goes down, my file(s) are not uploaded
.
And how do you think any other node/flow, will be able to access Dropbox if your internet is down?
I expect the internet connectivity issue to be temporary, so that the upload will continue again, when the connection is back up. When testing this, it works fine again, when the connection is reestablished, but only for the new files.
Ok so why don't you keep a revolving list of files that enter the folder. When they are uploaded to dropbox and you remove them froom the older, remove them from the list. If you loose the internet, you will still be updating the list and when the internet reconnects, you can have some code to read the list and upload the files that need to go.
If you make the folder a subfolder of your dropbox folder then all you need to do is to put the files there.
That could be a way to manage it, thanks. Unfortunately, I don't have a lot of experience with scripting (or whatever it takes). Any pointers?
So if the files are not put in the top level Dropbox folder, it is possible to transfer an entire folder (from local storage to Dropbox)?
All files, folders and subfolders in the Dropbox folder are automatically synced. So any time you add, delete or change a file one of those folders that change will be synced. You don't need to use a Dropbox node, just put the files in a subfolder and they will be synced.
I am not sure that we are talking about the same thing here. I am using a Linux system and Node-RED is the only means by which it connects to Dropbox. And as far as I can see, the Dropbox API does not work the same way it does on a PC. I can use it to upload and download individual files as per request, but not sync or upload an entire folder.
That is what you are currently doing, but you asked how to sync complete folders. There is a Dropbox daemon for Linux that allows it to work in Linux in the same way that it does on Windows (I presume, I don't use Windows so am guessing a bit here). Install that and you will get what you want.
I'm doing something similar, see https://gist.github.com/Paul-Reed/c29cfa755fed666805956eb6da42a0d0
Sorry for the late reply.
I have attempted to install Dropbox in another way, but my Linux (pre-compiled Yocto) does not seem to support a daemon. The dropbox-v2-api package installed with npm does not automatically sync folders, as far as I can see. I might be wrong, or there could be some way to make this work, but right now it seems to me that I need to compile Dropbox into a custom Yocto to get it to work.
Definitely interesting. I can see that your flow incorporates some SQL bits and is made for a Linux distro with sudo etc. (and running on an RPi). In my scenario, I am not using any SQL and am running Yocto on a slow IOT unit. Could this somehow work anyway?
Yes, just adapt the flow to back up whatever you want
In my case I wanted to back up the mysql database, as well as others folders.
Once setup, it runs reliably and transparently in the background, and I've used the resulting backups several times to restore data.
Great, thanks. I have attempted to do so, but am getting the following error message:
node: Read filenamesfunction : (error)
"TypeError: Cannot read property 'readdirSync' of undefined".
The Read filenames node looks like this:
"var path = ('/home/root/logfiler/');
var fs = (global.get('fse'));
fs.readdirSync(path).forEach(file => {
node.send({localFilename: "home/root/logfiler/"+(file)});
});
return null;"
and I have rebooted the system in case fs-extra requires this. Does it? Or is there some other potential issue that needs to be addressed regarding fs-extra? I was unsure whether there should be a semicolon after "functionGlobalContext: {
fse:require('fs-extra')" or not (in settings.js), so I tried with and without (and with a comma), but either way I get the error.
Also, I edited the Generate commands node as follows:
var emonpath = 'home/root/logfiler'; //location of emoncms datafile directory
...
var path = '/home/root/.temp_nodered/';
...
var m4={payload: "/home/root/logfiler"};
...
return [ [ m1, m2, m3, m4 ] ];
I apologize for my shortcomings, but I am quite new to this whole Node-RED environment. I hope that I am just missing a tiny thing that solves everything. 
There was no need to change this node, as it should be looking for the temporary directory /home/root/.temp_nodered/ that was created at line 6 of the 'Generate commands' node.
If you can restore the 'Read filenames' node to my original settings and re-try. Or go through the flow and replace other /home/root/.temp_nodered/ url's with another url of your choice.
It should be;
functionGlobalContext: {
fse:require('fs-extra')
},
I changed the node because I don't have a folder named pi in my home directory, but I have changed it back to your settings now. I have also made sure the settings file looks as described above. I changed the folder name in line 6 in the "Generate commands" node back as well. And I have now even created a /home/pi/.temp_nodered folder manually.
Nevertheless, I get the same error.
I did a bit of cleaning up, to make things a little easier to read. Can you spot something that is obviously wrong?
Here is the complete flow (except for the Dropbox node):
[
{
"id": "b9e243ba.d1081",
"type": "tab",
"label": "Flow 2",
"disabled": false,
"info": ""
},
{
"id": "6273b7f8.ff81e",
"type": "comment",
"z": "b9e243ba.d1081",
"name": "Nightly Backups",
"info": "",
"x": 220,
"y": 100,
"wires": []
},
{
"id": "7d3acada.026084",
"type": "inject",
"z": "b9e243ba.d1081",
"name": "Start backup",
"topic": "",
"payload": "",
"payloadType": "date",
"repeat": "",
"crontab": "00 03 * * *",
"once": false,
"x": 227,
"y": 150,
"wires": [
[
"aaa9bd2a.8cdbf8"
]
]
},
{
"id": "1e7979a0.4c4ad6",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Triggered queue",
"func": "// if queue doesn't exist, create it\ncontext.queue = context.queue || [];\ncontext.busy = context.busy || false;\n\n// if the msg is a trigger one release next message\nif (msg.hasOwnProperty("trigger")) {\n if (context.queue.length > 0) {\n var m = context.queue.shift();\n return {payload:m};\n }\n else {\n context.busy = false;\n // node.send({payload:"result"});\n var msg2 = { payload:"OK" };\n }\n}\nelse {\n if (context.busy) {\n // if busy add to queue\n context.queue.push(msg.payload);\n }\n else {\n // otherwise we are empty so just pass through and set busy flag\n context.busy = true;\n return msg;\n }\n}\n\nreturn [null,msg2];",
"outputs": "2",
"noerr": 0,
"x": 400,
"y": 240,
"wires": [
[
"5f65c136.b3456"
],
[
"17c228d2.0f6de7"
]
]
},
{
"id": "3467a1aa.1b8016",
"type": "function",
"z": "b9e243ba.d1081",
"name": "set trigger",
"func": "// handle the return from the exec in here \n// if all is good then set msg.trigger property to exist\nmsg.trigger = 1;\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 680,
"y": 200,
"wires": [
[
"1e7979a0.4c4ad6"
]
]
},
{
"id": "5f65c136.b3456",
"type": "exec",
"z": "b9e243ba.d1081",
"command": "",
"addpay": true,
"append": "",
"useSpawn": "false",
"timer": "",
"oldrc": false,
"name": "Execute commands",
"x": 750,
"y": 140,
"wires": [
[
"3467a1aa.1b8016"
],
[],
[
"17d51249.48eea6"
]
]
},
{
"id": "17c228d2.0f6de7",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Read filenames",
"func": "var path = ('/home/pi/.temp_nodered');\nvar fs = (global.get('fse'));\n\nfs.readdirSync(path).forEach(file => {\n node.send({localFilename: "/home/pi/.temp_nodered/"+(file)});\n});\nreturn null;",
"outputs": 1,
"noerr": 0,
"x": 400,
"y": 340,
"wires": [
[]
]
},
{
"id": "aaa9bd2a.8cdbf8",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Generate commands",
"func": "//--Edit these to your own installation\n//Don't change these!\nvar path = '/home/pi/.temp_nodered';\nvar tmp_path = '/tmp/mysql_backup';\n\n//Create directory structure\nvar m1={payload: "rm -rf "+[path]};\nvar m2={payload: "mkdir "+[path]};\nvar m3={payload: "mkdir "+[tmp_path]};\n\n// ************************************************************************* //\n// *** The below commands will archive the entire node-red user directory,\n// *** emoncms data directories & mysql. To add further commands, use the\n// *** same format, and add the corresponding msg id to the 'return' statement.\n// ************************************************************************* //\n//---NODE-RED---//\n //Create ~/.node-red archive\n var m4={payload: "/home/root/logfiler"};\n\n\n// Output the commands for execution\nreturn [ [ m1, m2, m3, m4, ] ];",
"outputs": "1",
"noerr": 0,
"x": 431,
"y": 150,
"wires": [
[
"1e7979a0.4c4ad6"
]
]
},
{
"id": "17d51249.48eea6",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Report any errors",
"func": "var code = (msg.payload.code);\nif ([code] != "0") {\nmsg.payload = (msg.payload.message);\nmsg.topic = "Raspberry backup process failed";\nreturn msg;\n}\nelse {\nreturn null;\n}",
"outputs": 1,
"noerr": 0,
"x": 390,
"y": 480,
"wires": [
[]
]
},
{
"id": "c33bf6c3.2bc5e",
"type": "comment",
"z": "b9e243ba.d1081",
"name": "README (select & view in info panel!)",
"info": "REQUIREMENTS \nRequires the library fs-extra installing & enabling\nglobally in settngs.js \n(used in Read Filenames function node)\n\nTO INSTALL FS-EXTRA \n\ncd ./node-red \nnpm install --save fs-extra\n\nTO MAKE FS-EXTRA GLOBAL \nAdd fse:require('fs-extra') to the\nfunctionGlobalContext section in NR settings.js file.\n\nCONFIGURATION \nEdit the Generate commands function node with \nyour own; \n- Emoncms data directory path\n- Emoncms MYSQL username (default = emoncms)\n- Emoncms MYSQL password \n\nSetup the dropbox node, (details in the node info) \nAlso edit the Start backup inject node with your preferred time to run. \nIf you want to be alerted about archive problems or failures, you can use the Report any errors node output to dispatch the message via Pushover, email, twitter or whatever.",
"x": 476,
"y": 99,
"wires": []
}
]
Could you please edit your last post above and add the required formatting to your code, as detailed in this post - How to share code or flow json so that I am able to import it OK (it won't currently import).
If you update it, I'll take a look this evening.
Thanks for helping out.
[
{
"id": "b9e243ba.d1081",
"type": "tab",
"label": "Flow 2",
"disabled": false,
"info": ""
},
{
"id": "6273b7f8.ff81e",
"type": "comment",
"z": "b9e243ba.d1081",
"name": "Nightly Backups",
"info": "",
"x": 220,
"y": 100,
"wires": []
},
{
"id": "7d3acada.026084",
"type": "inject",
"z": "b9e243ba.d1081",
"name": "Start backup",
"topic": "",
"payload": "",
"payloadType": "date",
"repeat": "",
"crontab": "00 03 * * *",
"once": false,
"x": 227,
"y": 150,
"wires": [
[
"aaa9bd2a.8cdbf8"
]
]
},
{
"id": "1e7979a0.4c4ad6",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Triggered queue",
"func": "// if queue doesn't exist, create it\ncontext.queue = context.queue || [];\ncontext.busy = context.busy || false;\n\n// if the msg is a trigger one release next message\nif (msg.hasOwnProperty(\"trigger\")) {\n if (context.queue.length > 0) {\n var m = context.queue.shift();\n return {payload:m};\n }\n else {\n context.busy = false;\n // node.send({payload:\"result\"});\n var msg2 = { payload:\"OK\" };\n }\n}\nelse {\n if (context.busy) {\n // if busy add to queue\n context.queue.push(msg.payload);\n }\n else {\n // otherwise we are empty so just pass through and set busy flag\n context.busy = true;\n return msg;\n }\n}\n\nreturn [null,msg2];",
"outputs": "2",
"noerr": 0,
"x": 400,
"y": 240,
"wires": [
[
"5f65c136.b3456"
],
[
"17c228d2.0f6de7"
]
]
},
{
"id": "3467a1aa.1b8016",
"type": "function",
"z": "b9e243ba.d1081",
"name": "set trigger",
"func": "// handle the return from the exec in here \n// if all is good then set msg.trigger property to exist\nmsg.trigger = 1;\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 680,
"y": 200,
"wires": [
[
"1e7979a0.4c4ad6"
]
]
},
{
"id": "5f65c136.b3456",
"type": "exec",
"z": "b9e243ba.d1081",
"command": "",
"addpay": true,
"append": "",
"useSpawn": "false",
"timer": "",
"oldrc": false,
"name": "Execute commands",
"x": 750,
"y": 140,
"wires": [
[
"3467a1aa.1b8016"
],
[],
[
"17d51249.48eea6"
]
]
},
{
"id": "17c228d2.0f6de7",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Read filenames",
"func": "var path = ('/home/pi/.temp_nodered');\nvar fs = (global.get('fse'));\n\nfs.readdirSync(path).forEach(file => {\n node.send({localFilename: \"/home/pi/.temp_nodered/\"+(file)});\n});\nreturn null;",
"outputs": 1,
"noerr": 0,
"x": 400,
"y": 340,
"wires": [
[]
]
},
{
"id": "aaa9bd2a.8cdbf8",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Generate commands",
"func": "//--Edit these to your own installation\n//Don't change these!\nvar path = '/home/pi/.temp_nodered';\nvar tmp_path = '/tmp/mysql_backup';\n\n//Create directory structure\nvar m1={payload: \"rm -rf \"+[path]};\nvar m2={payload: \"mkdir \"+[path]};\nvar m3={payload: \"mkdir \"+[tmp_path]};\n\n// ************************************************************************* //\n// *** The below commands will archive the entire node-red user directory,\n// *** emoncms data directories & mysql. To add further commands, use the\n// *** same format, and add the corresponding msg id to the 'return' statement.\n// ************************************************************************* //\n//---NODE-RED---//\n //Create ~/.node-red archive\n var m4={payload: \"/home/root/logfiler\"};\n\n\n// Output the commands for execution\nreturn [ [ m1, m2, m3, m4, ] ];",
"outputs": "1",
"noerr": 0,
"x": 431,
"y": 150,
"wires": [
[
"1e7979a0.4c4ad6"
]
]
},
{
"id": "17d51249.48eea6",
"type": "function",
"z": "b9e243ba.d1081",
"name": "Report any errors",
"func": "var code = (msg.payload.code);\nif ([code] != \"0\") {\nmsg.payload = (msg.payload.message);\nmsg.topic = \"Raspberry backup process failed\";\nreturn msg;\n}\nelse {\nreturn null;\n}",
"outputs": 1,
"noerr": 0,
"x": 390,
"y": 480,
"wires": [
[]
]
},
{
"id": "c33bf6c3.2bc5e",
"type": "comment",
"z": "b9e243ba.d1081",
"name": "README (select & view in info panel!)",
"info": "**REQUIREMENTS** \nRequires the library fs-extra installing & enabling\nglobally in settngs.js \n(used in `Read Filenames` function node)\n\nTO INSTALL FS-EXTRA \n```\ncd ./node-red \nnpm install --save fs-extra\n```\nTO MAKE FS-EXTRA GLOBAL \nAdd `fse:require('fs-extra')` to the\n`functionGlobalContext` section in NR settings.js file.\n\n**CONFIGURATION** \nEdit the `Generate commands` function node with \nyour own; \n- Emoncms data directory path\n- Emoncms MYSQL username (default = emoncms)\n- Emoncms MYSQL password \n\nSetup the `dropbox` node, (details in the node info) \nAlso edit the `Start backup` inject node with your preferred time to run. \nIf you want to be alerted about archive problems or failures, you can use the `Report any errors` node output to dispatch the message via Pushover, email, twitter or whatever.",
"x": 476,
"y": 99,
"wires": []
}
]
Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
The OpenJS Foundation | Terms of Use | Privacy Policy | OpenJS Foundation Bylaws | Trademark Policy | Trademark List | Cookie Policy