How to unzip .gz file

[Colin]

Does anybody know how I can unzip a .gz file other than by using an exec node calling gunzip? That does work but I want it to work on Windows as well as Linux and I presume that solution won't work on Windows.

I have found pako which does seem to have the ability to do that, but I can't work out exactly how. The gz wrapper includes the file names somehow and I don't know how to handle that. I can't find any relevant examples.

[bakman2]

On windows 10 you should be able to use

gzip -d file.gz

But is it a .gz or a .tar.gz ?

[knolleary]

There's a node for that... https://flows.nodered.org/node/node-red-contrib-gzip

Not used it for ages, but I do know I've used it successfully in the past.

[Colin]

@bakman2

But is it a .gz or a .tar.gz ?

It is a tar.gz but I don't want to untar it, just unzip it.

[Colin]

@knolleary

node-red-contrib-gzip (node) - Node-RED

I had tried that but I can't get it to work. It works ok with a text file but with a .tar.gz file unzipping to a .tar file ends up with a file shorter than it should be. The tar wrapper seems ok but the contents won't extract.
It says in the info tab for the node "If the input is a compressed buffer it tries to decompress to a utf8 string" which I suspect is the problem, I need it to decompress to binary I think.
It is likely a matter of specifying the right options to pako.

[Colin]

Hmm, this is very odd. The output from the gzip node is exactly the right length (61952) which suggests the unzip is happening correctly. I am then feeding that into a file out node configured as below, but the resulting file is only 59559 bytes long, which suggests that the file is not all being written. I saw the comment in the help for the file node that said it would close the file after every write if I supplied the filename in the message but that didn't make any difference.
I can't believe that there is such a basic bug in the file node, but if I send it a buffer and the file is shorter than the buffer then what on earth is going on?

[Colin]

Well I appear to have got to the bottom of this. There were two issues.

Firstly was the fact that the File out node was not correctly writing the buffer. The solution to that was to set the Encoding to Default. Why that is the case I don't know. I don't think the node should even be calling the fs.write method that takes an encoding option when it is passed a Buffer.

Secondly was that fact that the gzip node appears to expect that it should generate a string, so it doesn't work with binary files (for me at least). The fix is in node-modules/node-red-contrib-gzip/lib/gzip.js to change the unzipping code from

           var out = pako.inflate(msg.payload);
           msg.payload = new txtd.TextDecoder('utf-8').decode(out);
           node.send(msg);

to

           var out = pako.inflate(msg.payload);
           msg.payload = new Buffer.from(out);
           node.send(msg);

then all works as it should. Ideally I think the node should have a binary checkbox. However, in fact I want to do this in a created node rather than a flow so now that I know how to drive pako I believe I am good to go

[dceejay]

but yes - would be good to alert the owner of the gzip node. it could at least do an isUTF test before forcing the conversion... or have an option as you suggest.

[Colin]

I have submitted an Issue, but I suspect it is not much supported.

Does not correctly unzip .tar.gz files (and probably other binary files)

[Colin]

@dceejay have you any comment on why sending a Buffer to a File Out node set to binary encoding doesn't just write the buffer out?

[dceejay]

not offhand