Writing an image using file node and a binary buffer

General
1

Hi,
my flow is using the file node as output, to dump an image i am receiving by socket, so i am working with binary data and Buffer.
the node file is configured in overwrite mode, and binary encoding, without appending any \n character,
so i would expect that the length of my msg.payload should be the same as my file.
actually in msg.payload the binary buffer of the image is stored, in the node exactly before the file node i write the length of the payload:

node.warn("going to Dump image bytes: " + msg.payload.length)

it prints:

"going to Dump image bytes: 1573302"

but the file node saves on the file system: 1560204 bytes.

Why? and the image is corrupted obliviously.
thanks in advance

Luca

2

Hi.

I have a similar problem
Recieving data from socket and everything from byte 11 in file is not the same as what comes from stream.

3

Hi and welcome

What system are you running on - what version of Node-RED etc etc

Is the file your trying to write large?

4

here is the buffer:
[66,90,104,57,49,65,89,38,83,89,151,206,141,167,0,0,134,159,128,80,14,127,240,40,0,4,0,190,255,222,106,48,0,184,131,85,63,9,164,201,234,9,144,26,0,25,61,65,128,25,52,208,100,48,67,76,70,140,13,79,68,10,122,153,61,38,67,79,77,77,0,208,19,1,160,2,65,26,200,6,81,210,112,166,89,4,51,61,150,175,96,31,75,6,251,162,68,172,69,54,168,59,6,1,229,177,128,186,252,2,176,171,149,200,30,224,168,131,20,171,67,133,149,130,189,53,98,201,68,147,38,62,4,81,38,234,17,182,214,48,57,208,79,141,36,82,254,25,175,115,210,169,227,68,13,122,103,157,102,153,4,250,95,3,89,224,141,93,228,196,158,197,172,250,89,233,75,184,144,214,88,66,12,215,106,53,188,94,166,90,56,73,204,70,147,60,94,172,66,73,168,241,217,92,45,107,130,171,161,96,42,136,167,236,191,139,185,34,156,40,72,75,231,70,211,128]

Node red 1.0.3
Linux Debian 10
Node.js 6.13.2

The output from Filewriter says the same as the buffer read from the network stream
I strip tjhe first 4 bytes from the stream as the are the lenght of the buffer, the rest is supposed to be a bz2 file

[{"id":"db7cca2.dffc038","type":"debug","z":"958f84ab.d1a518","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":886,"y":44,"wires":[]},{"id":"c3aba263.b16cc","type":"file","z":"958f84ab.d1a518","name":"Create celox2.bz2","filename":"c:\\test\\celox2.bz2","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"binary","x":637,"y":45,"wires":[["db7cca2.dffc038"]]},{"id":"863f9aa8.183f48","type":"function","z":"958f84ab.d1a518","name":"","func":"msg.payload = msg.payload.slice(4, msg.payload.length);\nreturn msg;","outputs":1,"noerr":0,"x":440,"y":47,"wires":[["c3aba263.b16cc"]]},{"id":"8b550368.55051","type":"tcp-client","z":"958f84ab.d1a518","action":"Listen","actionType":"str","host":"","hostType":"str","port":"payload","portType":"msg","datamode":"stream","datatype":"buffer","newline":"","xmlAttrkey":"$","xmlCharkey":"_","xmlStrip":false,"xmlArray":false,"xmlNormalizeTags":false,"xmlNormalize":false,"write":"","writeType":"str","topic":"test","name":"","debug":"all","x":254.5,"y":48,"wires":[["863f9aa8.183f48"]]},{"id":"64387411.66150c","type":"inject","z":"958f84ab.d1a518","name":"","topic":"","payload":"9999","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":96.5,"y":49,"wires":[["8b550368.55051"]]}]
5

1st thing I've noticed is the filename is a windows format but you said you are running on Linux

6

I used your buffer and directly injected it into your function node and then compared its output with the actual file and I can see it seems to deviate after the 6th byte - not got a clue at moment as to what is going on!

image
image854×580 206 KB

Even cutting the function node out and only sending 16 bytes - its deviating after the 53 59 bytes

image
image1348×287 97.7 KB 

[{"id":"b3adce26.3c2e","type":"debug","z":"3517604c.8ef54","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":440,"wires":[]},{"id":"218eb06c.6c894","type":"file","z":"3517604c.8ef54","name":"Create celox2.bz2","filename":"/home/pi/celox2.bz2","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":490,"y":440,"wires":[["b3adce26.3c2e"]]},{"id":"663fd446.f51e5c","type":"inject","z":"3517604c.8ef54","name":"","topic":"","payload":"[66,90,104,57,49,65,89,38,83,89,151,206,141,167,0,0]","payloadType":"bin","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":310,"y":440,"wires":[["9fd55654.5b72d8","218eb06c.6c894"]]},{"id":"9fd55654.5b72d8","type":"debug","z":"3517604c.8ef54","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":470,"y":500,"wires":[]}]
7

Switching encoding from binary to default gets rid of the issue on the 16 byte test sequence

image

image
image837×185 63.6 KB

8

Typical!
I thought that i'v tried that!
Works perfectly!

9

But it shouldn't do it so it needs further investigation

10

the mainsystem is on linux.
Forgot that i moved to a windowssystem to try it there also.