Byte hickups within a super, super, super basic flow

[mhoppe]

Hello nice people! Thank you for reading.

I have a very basic flow:

TCP request: sends a request to a
TCP In, that attaches a buffer (file) to msg.payload and replies with
TCP Reply to send the file to the requester.

The size of the reply buffer is the same as the file that has been sent, but the buffer is not the same anymore. I attached the flow and below is the link to the files.. Any idea what I am doing wrong? Thank you so much!

BEFORE

AFTER

The files for download.

send_webp.json (4.8 KB)

[mhoppe]

Noone no idea, not?

[E1cid]

This works joining the buffers and not closing the connection.

[{"id":"b066e8e0bc83cf63","type":"join","z":"d1395164b4eec73e","name":"","mode":"custom","build":"buffer","property":"payload","propertyType":"msg","key":"topic","joiner":"","joinerType":"bin","useparts":false,"accumulate":false,"timeout":"1","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":530,"y":7480,"wires":[["13266d6552ea583a"]]},{"id":"396ce2af69d65350","type":"tcp request","z":"d1395164b4eec73e","name":"REQUEST 44000","server":"127.0.0.1","port":"44000","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":590,"y":7620,"wires":[["7ff0ab45743c3c44","b066e8e0bc83cf63"]]},{"id":"13266d6552ea583a","type":"image","z":"d1395164b4eec73e","name":"","width":160,"data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":true,"outputs":1,"x":700,"y":7480,"wires":[["c79365b6e51eab14"]]},{"id":"60373028f0a39933","type":"inject","z":"d1395164b4eec73e","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":400,"y":7620,"wires":[["396ce2af69d65350"]]},{"id":"7ff0ab45743c3c44","type":"debug","z":"d1395164b4eec73e","name":"debug COMPLETE","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":830,"y":7680,"wires":[]},{"id":"c79365b6e51eab14","type":"file","z":"d1395164b4eec73e","name":"","filename":"/storage/emulated/0/Download/r.webp","filenameType":"str","appendNewline":false,"createDir":false,"overwriteFile":"true","encoding":"none","x":980,"y":7480,"wires":[[]]}]

The contrib node is node-red-contrib-image-output (node) - Node-RED

[Steve-Mcl]

TCP data is chunked based on the MTU. Using a joint node can help here. However, ideally you would have control characters in the data to help you decipher the beginning and end of an actual file instead of depending on timing (if multiple transmissions are made in quick succession you could end up joining data from different files). This is entirely the reason I suggested not to use raw TCP in your other thread.

[mhoppe]

Wow! Thank you so much! Also the Image preview node is very useful!

Now that I know what the "bug" was (it was sitting in front of the screen), I have to fix that in PHP too, because the TCP-request is initially coming from there.

I will create sort of a header, so that the first 4 byte will describe the length of the following buffer, then make it receiver read that size.

Like so in node-red:

const imageBuffer = msg.payload;
const imageSize = imageBuffer.length; // Get the length of the image buffer
const headerBuffer = Buffer.alloc(4); // Create a new buffer for the header (4 bytes for the length) + image data
headerBuffer.writeUInt32BE(imageSize, 0); // Write the length in big-endian format
const combinedBuffer = Buffer.concat([headerBuffer, imageBuffer]); // Concatenate the header and image buffers
msg.payload = combinedBuffer; // Set the combined buffer as the payload
return msg; // Send the message

And then in PHP:

$header = socket_read($socket, 4);
$imageSize = unpack("N", $header)[1]; // "N" for big-endian unsigned 32-bit integer
$imageData = socket_read($socket, $imageSize); // Now read the image data based on the size
file_put_contents("received_image.webp", $imageData);

Do you think that this is a good way, or do you maybe have a better suggestion?

Thank you so much! Thank you!