Serving a big archive (.zip or.tar) file built on the fly

Hi,

I'm running Node-RED on a Raspberry Pi and I'm looking for a way to easily export (over HTTP) a bunch of files from the local file system.

I managed to build a simple dashboard where you can just click a button and it will spawn a process (using the exec node) with a purposedly-built 'tar' command line which will prepare a tar file on a temporary directory on the local file system.
When the process completes, this is notified on the dashboard where you can now click on a link letting you download the resulting file.

The file is actually served by using a pair of http in and http response nodes and following @Abraxas's suggestion of taking advantage of the underlying implementation being based on Express.js, see thread Efficient way to download file.

This approach works just fine, except it has two disadvantages, especially when the resulting files are pretty big:

  1. There is no way to tell the user how long the whole "prepare" operation is going to take. It might be minutes, for that matter.
  2. The resulting (temporary) file might take up just as much space on the drive as its contents

So I was wondering if there would be any easy way to serve a .zip o .tar file on the fly, so that the user might start downloading it while it's e.g. being output by the tar command on its standard output.
If I'm not mistaken, that's what cgi scripts would be doing back in the day.

Any ideas on how to achieve that somehow within Node-RED?

I'd rather not use an external HTTP server as it would require an extra TCP port for serving, while I really like keeping everything through port 1880 as it makes your life easier if you're using port forwarding in any way.

Thanks!