Upon pressing the start button, I'm sending measurements via MQTT. When I press the stop button, I want to have the option to either append a CSV file with the data series or discard the data in case of faulty measurements. The start button initiates processes, while the stop button halts them and sends the data series to the output. Each data series should be stored in a separate CSV file. How can I achieve this? Could you please provide guidance on the Node-RED flow for implementing this functionality?
Of course there are lots of ways to do this. Personally I would shy away from storing the data within Node-red for fear of running out of memory. Instead I would write the data to file and later on choose to save or delete it:
Use a context variable to hold a filename. Value set by the start button and deleted by stop button.
If the variable exists, write incoming data to the file.
A second context variable holds the most recent filename. Value set by the stop button and deleted by the save button.
"Save" just deletes this variable.
"Delete" deletes the file and the variable.
A possible flow, not tested. I used Jsonata to generate a filename based on the time since the world began in 1970.
You might dislike having a file not yet approved in the same directory as saved data. Easy enough to save it in a temp directory and move it with the Save button.
[{"id":"b588fd8afb458d0f","type":"ui_button","z":"a3c3807e16ca19c8","name":"start","group":"f149abdd741ca58e","order":13,"width":0,"height":0,"passthru":false,"label":"button","tooltip":"","color":"","bgcolor":"","className":"","icon":"","payload":"open","payloadType":"str","topic":"control","topicType":"str","x":70,"y":60,"wires":[["42023d421fe24b39"]]},{"id":"03b27043117f0b95","type":"mqtt in","z":"a3c3807e16ca19c8","name":"stram of input data","topic":"foo/bar","qos":"0","datatype":"auto-detect","broker":"bf2923ac3d80c3c0","nl":false,"rap":true,"rh":0,"inputs":0,"x":110,"y":320,"wires":[["455ca0c685a855e6"]]},{"id":"baf83d31e0a71544","type":"ui_button","z":"a3c3807e16ca19c8","name":"stop","group":"f149abdd741ca58e","order":13,"width":0,"height":0,"passthru":false,"label":"button","tooltip":"","color":"","bgcolor":"","className":"","icon":"","payload":"close","payloadType":"str","topic":"control","topicType":"str","x":70,"y":100,"wires":[["7b62e4d8414552d1"]]},{"id":"7b62e4d8414552d1","type":"change","z":"a3c3807e16ca19c8","name":"set flow.lastfile, delete flow.csvfile","rules":[{"t":"set","p":"lastfile","pt":"flow","to":"csvfile","tot":"flow"},{"t":"delete","p":"csvfile","pt":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":300,"y":100,"wires":[[]]},{"id":"42023d421fe24b39","type":"change","z":"a3c3807e16ca19c8","name":"","rules":[{"t":"set","p":"csvfile","pt":"flow","to":"\"/home/pi/data/\" & $millis() & \".csv\"","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":240,"y":60,"wires":[[]]},{"id":"455ca0c685a855e6","type":"switch","z":"a3c3807e16ca19c8","name":"flow.csvfile is not null?","property":"csvfile","propertyType":"flow","rules":[{"t":"nnull"}],"checkall":"true","repair":false,"outputs":1,"x":320,"y":320,"wires":[["ed0ebdbc736f42dd"]]},{"id":"ed0ebdbc736f42dd","type":"change","z":"a3c3807e16ca19c8","name":"","rules":[{"t":"set","p":"filename","pt":"msg","to":"csvfile","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":530,"y":320,"wires":[["8ab015f930dfe52d"]]},{"id":"8ab015f930dfe52d","type":"file","z":"a3c3807e16ca19c8","name":"append to file","filename":"","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":720,"y":320,"wires":[[]]},{"id":"2c4eda8d005c9540","type":"ui_button","z":"a3c3807e16ca19c8","name":"save","group":"f149abdd741ca58e","order":13,"width":0,"height":0,"passthru":false,"label":"button","tooltip":"","color":"","bgcolor":"","className":"","icon":"","payload":"open","payloadType":"str","topic":"control","topicType":"str","x":70,"y":180,"wires":[["763ccbd17a7e5c49"]]},{"id":"745c490b897cecfa","type":"ui_button","z":"a3c3807e16ca19c8","name":"delete","group":"f149abdd741ca58e","order":13,"width":0,"height":0,"passthru":false,"label":"button","tooltip":"","color":"","bgcolor":"","className":"","icon":"","payload":"close","payloadType":"str","topic":"control","topicType":"str","x":70,"y":240,"wires":[["23e260282502db65"]]},{"id":"23e260282502db65","type":"file","z":"a3c3807e16ca19c8","name":"delete file just written","filename":"lastfile","filenameType":"msg","appendNewline":true,"createDir":false,"overwriteFile":"delete","encoding":"none","x":260,"y":240,"wires":[["763ccbd17a7e5c49"]]},{"id":"763ccbd17a7e5c49","type":"change","z":"a3c3807e16ca19c8","name":"","rules":[{"t":"delete","p":"lastfile","pt":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":250,"y":180,"wires":[[]]},{"id":"4a8dc385bcb80a0c","type":"comment","z":"a3c3807e16ca19c8","name":"Start writing data to eg /home/pi/data/1693560823480.csv","info":"","x":550,"y":60,"wires":[]},{"id":"f4d0a90c27279970","type":"comment","z":"a3c3807e16ca19c8","name":"Stop writing file, save filename","info":"","x":580,"y":100,"wires":[]},{"id":"368dec13227c03f6","type":"comment","z":"a3c3807e16ca19c8","name":"delete the saved filename - file is now \"permanent\"","info":"","x":570,"y":180,"wires":[]},{"id":"719360f9fea45916","type":"comment","z":"a3c3807e16ca19c8","name":"don't like this data - delete file","info":"","x":520,"y":240,"wires":[]},{"id":"f149abdd741ca58e","type":"ui_group","name":"Demo","tab":"1caa8458.b17814","order":1,"disp":true,"width":"12","collapse":false,"className":"test"},{"id":"bf2923ac3d80c3c0","type":"mqtt-broker","name":"","broker":"192.168.1.11","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closeRetain":"false","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":""},{"id":"1caa8458.b17814","type":"ui_tab","name":"Demo","icon":"dashboard","order":3,"disabled":false,"hidden":false}]
ps that's supposed to be a stream of data, not a stram. 
I understand that I might need to change my approach, and my question is as follows: How can I save files to an SD card in Node-RED with the filename being the current date? Additionally, is it possible to create a separate interface, like a dashboard, for working with these saved files? Specifically, I'd like to be able to view the files on the dashboard and have the option to delete specific files. How can I achieve this functionality?
You can use JSONata and $moment() in the file write node path input to append/create a date to the file name.
1 Like
Is it possible to create a window in the dashboard for working with these files, primarily for deletion?
Many ways to possible do this.
You could use the exex node to read the directory and then create a drop down that lists the files. You could then select the file and show a notification to confirm delete. Then reread the directory to refresh the drop down.
e.g.
[{"id":"34616e52c802c16f","type":"ui_ui_control","z":"b9860b4b9de8c8da","name":"","events":"all","x":420,"y":60,"wires":[["c3682381e6552931"]]},{"id":"c3682381e6552931","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"name","propertyType":"msg","rules":[{"t":"eq","v":"Home","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":550,"y":60,"wires":[["901eb24a764b0511"]]},{"id":"901eb24a764b0511","type":"exec","z":"b9860b4b9de8c8da","command":"ls .node-red/static/files","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":730,"y":60,"wires":[["d3356343d785da61"],[],[]]},{"id":"1cbfdc3f03ec2a91","type":"file","z":"b9860b4b9de8c8da","name":"","filename":"topic","filenameType":"msg","appendNewline":true,"createDir":false,"overwriteFile":"delete","encoding":"none","x":1050,"y":160,"wires":[["2706439caf889a08","901eb24a764b0511"]]},{"id":"973484de1628b7ea","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"OK","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":930,"y":160,"wires":[["1cbfdc3f03ec2a91"],["901eb24a764b0511"]]},{"id":"d3356343d785da61","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"options","pt":"msg","to":"[$split($$.payload, \"\\n\")[$ != \"\"]]","tot":"jsonata"},{"t":"set","p":"payload","pt":"msg","to":".node-red/static/files/","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"path","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":940,"y":20,"wires":[["a741327ae4d4610e","1a724cf5683b053f"]]},{"id":"2706439caf889a08","type":"debug","z":"b9860b4b9de8c8da","name":"debug 328","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":950,"y":220,"wires":[]},{"id":"f29677a93df3ac9c","type":"ui_toast","z":"b9860b4b9de8c8da","position":"dialog","displayTime":"3","highlight":"","sendall":false,"outputs":1,"ok":"OK","cancel":"Cancel","raw":false,"className":"","topic":"","name":"","x":790,"y":160,"wires":[["973484de1628b7ea"]]},{"id":"a741327ae4d4610e","type":"ui_dropdown","z":"b9860b4b9de8c8da","name":"","label":"","tooltip":"","place":"click to delete file","group":"8b5cde76.edd58","order":15,"width":0,"height":0,"passthru":true,"multiple":false,"options":[{"label":"","value":"","type":"str"}],"payload":"","topic":"file","topicType":"str","className":"","x":960,"y":80,"wires":[["1a724cf5683b053f"]]},{"id":"1a724cf5683b053f","type":"join","z":"b9860b4b9de8c8da","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":true,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":1110,"y":60,"wires":[["51b7b43642ca03f8"]]},{"id":"829252eea2554b28","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"$$.payload.path & $$.payload.file","tot":"jsonata"},{"t":"set","p":"payload","pt":"msg","to":"confirm deletion","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":600,"y":160,"wires":[["f29677a93df3ac9c"]]},{"id":"51b7b43642ca03f8","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"file","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":450,"y":160,"wires":[["829252eea2554b28"]]},{"id":"8b5cde76.edd58","type":"ui_group","name":"default","tab":"8f03e639.85956","order":1,"disp":false,"width":"12","collapse":false},{"id":"8f03e639.85956","type":"ui_tab","name":"Home","icon":"dashboard","order":3,"disabled":false,"hidden":false}]
You will need to edit the path in the exec node and the following change node.
The path where I have the files is in such a directory /csv_files/. How should I correctly enter it into the exec node?
is that absolute path or relative?
Where is it in relation to .node-red folder?
The files are located on the SD card of the Raspberry Pi.
This is the path I provided when saving in the 'write' node.
It looks like an absolute path.
I am not a pi user. I think you would have to mount the sd card, if ls /csv_files/ is not absolute path.
or this may work instead of exec node
[edit]
Try cd / | ls /csv_files/
1 Like
Now the files are displayed in the dashboard
In order to allow the selected file to be downloaded to the machine it is currently running on, what would I need to add to the flow?
What do you mean by the machine it is running on? The files are already on the machine that node-red is running on.
I mean the device on which I run the dashboard, in my case a stationary computer.
I would like to extend the functionality with the ability to copy the selected file to a desktop computer.
To what extent is it even possible?
So you mean you want to download it to the machine running the browser. That is a regularly asked question here, if you search the forum for
download file
I am sure you will get numerous suggestions.
I thought I could do it in a similar way, simply by adding a 'download' button to the pop-up window to download the file to the computer using the exec node. Is that possible?
How do you propose to do it using an exec node? The exec node runs commands on the node red machine. What command are you going to use to send it to the machine running the browser? It is possible if the file system of the machine running the browser is accessible from the node-red server, via some form of file sharing protocol, or ssh, but otherwise I don't think it is possible.
You can set up a http static folder or create a http endpoint.
Then set the notification node to accept html/javascript. then add a <a href> html tag with the http static url or endpoint url using msg.payload.file.
e.g. endpoint
[{"id":"03dcc8920c0258d0","type":"ui_ui_control","z":"b9860b4b9de8c8da","name":"","events":"all","x":600,"y":60,"wires":[["166d64eb3d3a96c5"]]},{"id":"166d64eb3d3a96c5","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"name","propertyType":"msg","rules":[{"t":"eq","v":"Home","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":730,"y":60,"wires":[["7bebb81c3027c923"]]},{"id":"7bebb81c3027c923","type":"exec","z":"b9860b4b9de8c8da","command":"ls .node-red/static/files","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":910,"y":60,"wires":[["e0bd7347702a1ee2"],[],[]]},{"id":"af3d9f0f66391b42","type":"file","z":"b9860b4b9de8c8da","name":"","filename":"topic","filenameType":"msg","appendNewline":true,"createDir":false,"overwriteFile":"delete","encoding":"none","x":1230,"y":160,"wires":[["a0f4ec2e59441f0e","7bebb81c3027c923"]]},{"id":"fa000add34d9da3d","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"OK","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1110,"y":160,"wires":[["af3d9f0f66391b42"],["7bebb81c3027c923"]]},{"id":"e0bd7347702a1ee2","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"options","pt":"msg","to":"[$split($$.payload, \"\\n\")[$ != \"\"]]","tot":"jsonata"},{"t":"set","p":"payload","pt":"msg","to":".node-red/static/files/","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"path","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1120,"y":20,"wires":[["adf4ee957f2bb63a","43036f988b2f6607"]]},{"id":"a0f4ec2e59441f0e","type":"debug","z":"b9860b4b9de8c8da","name":"debug 328","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1130,"y":220,"wires":[]},{"id":"a04d990df724a403","type":"ui_toast","z":"b9860b4b9de8c8da","position":"dialog","displayTime":"3","highlight":"","sendall":false,"outputs":1,"ok":"OK","cancel":"Cancel","raw":true,"className":"","topic":"","name":"","x":970,"y":160,"wires":[["fa000add34d9da3d"]]},{"id":"adf4ee957f2bb63a","type":"ui_dropdown","z":"b9860b4b9de8c8da","name":"","label":"","tooltip":"","place":"click to delete file","group":"8b5cde76.edd58","order":15,"width":0,"height":0,"passthru":true,"multiple":false,"options":[{"label":"","value":"","type":"str"}],"payload":"","topic":"file","topicType":"str","className":"","x":1140,"y":80,"wires":[["43036f988b2f6607"]]},{"id":"43036f988b2f6607","type":"join","z":"b9860b4b9de8c8da","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":true,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":1290,"y":60,"wires":[["70f36d0d28f26a61"]]},{"id":"364723a1cf2234d5","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"$$.payload.path & $$.payload.file","tot":"jsonata"},{"t":"set","p":"payload","pt":"msg","to":"'confirm deletion<br><br><a href=\"http://192.168.1.10:1880/download/?file=' & $$.payload.file & '\" target=\"_blank\"> Download ' & $$.payload.file & '</a>'","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":780,"y":160,"wires":[["a04d990df724a403"]]},{"id":"70f36d0d28f26a61","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"file","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":630,"y":160,"wires":[["364723a1cf2234d5"]]},{"id":"b5b418eb1329f202","type":"http in","z":"b9860b4b9de8c8da","name":"","url":"/download","method":"get","upload":false,"swaggerDoc":"","x":520,"y":320,"wires":[["eff168b963f64c25"]]},{"id":"eff168b963f64c25","type":"file in","z":"b9860b4b9de8c8da","name":"","filename":"\"/data/data/com.termux/files/home/.node-red/static/files/\" & $$.payload.file","filenameType":"jsonata","format":"","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":680,"y":360,"wires":[["653d1659c9a4fff2","d44a43c3f9ff9fdd"]]},{"id":"d44a43c3f9ff9fdd","type":"function","z":"b9860b4b9de8c8da","name":"function 21","func":"msg.headers = {\n \"Content-Type\": \"text/plain\",\n \"Content-Transfer-Encoding\": \"binary\",\n \"Content-Disposition\": \"attachment; filename=\" + msg.filename,\n \"Content-Length\": msg.payload.count\n} \nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":850,"y":360,"wires":[["d815c952f5a121b8"]]},{"id":"653d1659c9a4fff2","type":"debug","z":"b9860b4b9de8c8da","name":"debug 328","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":830,"y":320,"wires":[]},{"id":"d815c952f5a121b8","type":"http response","z":"b9860b4b9de8c8da","name":"","statusCode":"","headers":{},"x":1010,"y":340,"wires":[]},{"id":"8b5cde76.edd58","type":"ui_group","name":"default","tab":"8f03e639.85956","order":1,"disp":false,"width":"12","collapse":false},{"id":"8f03e639.85956","type":"ui_tab","name":"Home","icon":"dashboard","order":3,"disabled":false,"hidden":false}]
[edit] I forgot to mention that you will need to edit the a href url in the change node prior to the notification node and add your node-red IP.
In the read node, in the first quote you refer to what path?
