Hi all,
I'm new to Node-RED and this community; I'm trying to develop my first project and I need some help: in my project I collect some buffers of data in csv files via MQTT; after that, I expose with Node RED a web page where these files are downloadable. For serving the files I used the example found in the documentation (https://cookbook.nodered.org/http/serve-a-local-file) but I found that the buffering provided by the File In node is very slow if the file is not small (some problems starting from 20MB and beyond), it take up to one minute to pass the buffer to the Http Response node. I tried to figure out how to speed up the process, and I found a partial solution avoiding to use the File In node and insteed serving directly the file via the method res.download() in a function node, provided by the express.js library on which the http in node is built upon; in this way the file got served in no time, even if is failry big. The only "drawback" is the warn "Deprecated call to msg.res.download" in the Node RED panel. There are more elegant ways to serve a file with this grade of efficienty without "breaking" the Node RED logic? There are any drawback to use directly the res.download() method from espress.js (memory/resource related in the long term maybe?)? I forgot to say that my Node RED instance is deployed on a RPi 4 B with 4GB, but I think is not relevant in this case. Thank you in advance for your responses and you time, have a nice day!
If you build your CSV file in a local directory where node-red can serve it as a static file, then you would need NO flow logic at all. You will need to uncomment a couple lines in your settings.js file (if you have not already done so, of course) -- here is an example:
// By default, the Node-RED UI is available at http://localhost:1880/
// The following property can be used to specify a different root path.
// If set to false, this is disabled.
httpAdminRoot: '/admin/',
// Some nodes, such as HTTP In, can be used to listen for incoming http requests.
// By default, these are served relative to '/'. The following property
// can be used to specifiy a different root path. If set to false, this is
// disabled.
httpNodeRoot: '/red/',
// The following property can be used in place of 'httpAdminRoot' and 'httpNodeRoot',
// to apply the same root to both parts.
//httpRoot: '/',
// When httpAdminRoot is used to move the UI to a different root path, the
// following property can be used to identify a directory of static content
// that should be served at http://localhost:1880/.
httpStatic: '/opt/node-red/public/',
With these settings, the startup logs show the urls where the editor, the dashboard, and local files can be found:
30 Sep 11:21:48 - [info] Settings file : /opt/node-red/.node-red/settings.js
30 Sep 11:21:48 - [info] HTTP Static : /opt/node-red/public
30 Sep 11:21:48 - [info] Context store : 'default' [module=memory]
30 Sep 11:21:48 - [info] User directory : /opt/node-red/.node-red
30 Sep 11:21:48 - [warn] Projects disabled : editorTheme.projects.enabled=false
30 Sep 11:21:48 - [info] Flows file : /opt/node-red/.node-red/flows_zpaper.json
30 Sep 11:21:48 - [info] Server now running at http://127.0.0.1:1880/admin/
The relevant line shows HTTP Static : /opt/node-red/public meaning that any files found within that directory automatically can be retrieved from that base url. Say your flow is constantly updating that csv file in /opt/node-red/public/files/mydata.csv -- then the url to retrieve the file contents would be http://localhost:1880/public/files/mydata.csv (your hostname/port may vary).
Hope this helps -- good luck with your project!
Hi all and thank you for you answers.
Shrickus, what I forget to tell is that I'm actually serving a bunch of files, actually up to hundreds, all selectable from a dashboard served via HTTP, so I'm good with masking these files through some HTTP endpoints with "Http In" node, but your post gave me some informations very useful for my project I didn't know, so thank you very much!
Steve-Mcl, that flow is what I was doing in OP, and is not very suitable because I found is not a good idea to buffer a large file in memory and serve it all to HTTP response node.
Anyway, in these days I found some confirmations to my doubts: the best way to serve a file as download is using the res.sendDownload() method exposed by Express.js and available in any message originated from HTTP In nodes. In this way, you don't have to do a double copy of the data (buffering and then serving to the HTTP Response methods) and this result to be much more efficient (I guess that express.js sendDownload methods take advantage of chunking both in the retrieval and the sending phases, but is just a supposition). The warn message I mentioned in the OP is due to the fact that HTTP In node wraps the res object, but it's not something to really care about.
Thank you again for your time and your answers, have a good day!
