Tcp msg is reassigned to the next data before debug output

[fullmooooon]

After my node-red connects to the tcp server, if there is a slight load in the function that gets the data.
It seems that debug sometimes outputs the next set of buffer data. If I reuse msg objects.

Testing function code

// msg = { payload: [] }
msg.payload = []
; (() => {
    let i = 0
    while (i < 100000000) i++
})()
return msg;

If I change line 649 of "node-red\ node_modules\ @ node-red\ nodes\ core\ network\ 31-tcpin.js"
Or use msg = {payload: []}
Then the output is always correct.

My Example flow cannot reproduce the problem for most people because it requires node-red to be in an unknown special error state. This state will disappear in a few days.

This special state may be related to the fact that I recently connected to the same tcp server using a python exe program.
Then I close the program, and I don't know if it handles tcp connections properly when it is closed.
I can't get rid of this state when I restart my computer. It might be useful to wait a few days.Because my node-red returned to normal yesterday, until I opened that python program again today.

Example flow

[{"id":"fa98aed08e74fe56","type":"function","z":"8616f41adb0d8dd9","name":"Testing function","func":"// msg = { payload: [] }\nmsg.payload = []\n; (() => {\n    let i = 0\n    while (i < 100000000) i++\n})()\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":920,"y":2620,"wires":[["e2db179b7d78ea59"]]},{"id":"7e6b012ff3c07eec","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"10.168.1.115","port":"6789","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":760,"y":2540,"wires":[["fa98aed08e74fe56"]]},{"id":"ad2ecd82c8c72223","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\nmsg.payload = Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":570,"y":2620,"wires":[["7e6b012ff3c07eec"]]},{"id":"97f6ebfb53da899d","type":"function","z":"8616f41adb0d8dd9","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": \"register_client\",\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:16',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":390,"y":2540,"wires":[["ad2ecd82c8c72223"]]},{"id":"0859d213ea46f128","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":210,"y":2600,"wires":[["97f6ebfb53da899d"]]},{"id":"e2db179b7d78ea59","type":"debug","z":"8616f41adb0d8dd9","name":"debug 358","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1140,"y":2540,"wires":[]}]

[fullmooooon]

https://t.co/YKQxsJBVuj
Uploaded a video.
The running result of this code is very confusing.
It seems that something is confusingly updating the TCP msg object.
My actual running code is affected by this situation, and it is almost (sometimes successful) unable to get a large tcp data continuously.

In addition, the data received in the same millisecond will be executed in the wrong order, which seems to have something to do with this problem.
I currently have a series of abnormal behaviors with "tcp request".
The example I gave is probably the simplest way to show the root of the problem.

I confirmed in "Wireshark" that the order of TCP data is fine.

[Steve-Mcl]

Ok, I think I know what is going on now.

I believe, due to the nature of "pass by reference" that the msg.payload is being updated, by reference, by earlier nodes WHILE you are in the hard while loop.

To test this theory, please do the following and try again.

Inside "Add length buffer header", instead of setting msg.payload create and return a new msg. e.g:

const m2 = { 
    payload: Buffer.from(.......)
}
return m2

---

NOTE: I know that the while loop is there to demonstrate the issue but I suspect this is just an async issue. For example, the code you normally have in place of the simulated workload may be causing the same problem if it is hogging the event queue. In short, you should try to avoid running hard loops (use the features provided like the split and join nodes)

[fullmooooon]

Thank you for your help.
In my actual code, only the buffer is operated by "concat" and "subarray", which is less time-consuming and occasionally causes this error.
Because according to communication rules, I have to constantly accept data until I parse JSON one by one.
My real code is actually followed by a "split".
It sometimes causes the next buffer data to be splited with "\n" before next buffer data entering the "function"!
As a result, I lost one byte of data and was unable to perform JSON parsing.
Sometimes it is missing a byte, and sometimes it is a complete package in the wrong order.
I can solve the problem of losing a byte by creating a new msg object. Even if I avoid referencing any suspicious objects. The phenomenon of incorrect TCP data order will still occur.

Disorder only occurs when multiple sets of data are obtained in the same millisecond.

There was a ten-hour period before today, my code didn't issue any errors, and I'm not sure what caused node-red to enter an irrecoverable error state.

When I return to the company tomorrow, I will test your code.

[fullmooooon]

hi
Do you mean to modify it like this?

It seems to have nothing to do with the nodes before "tcp request".
Because it will only be executed once.

[{"id":"fa98aed08e74fe56","type":"function","z":"8616f41adb0d8dd9","name":"Testing function","func":"// msg = { payload: [] }\nmsg.payload = []\n; (() => {\n    let i = 0\n    while (i < 100000000) i++\n})()\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":960,"y":2620,"wires":[["e2db179b7d78ea59"]]},{"id":"7e6b012ff3c07eec","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"10.168.1.115","port":"6789","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":760,"y":2540,"wires":[["fa98aed08e74fe56"]]},{"id":"ad2ecd82c8c72223","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\nmsg.payload = Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":570,"y":2420,"wires":[["7e6b012ff3c07eec"]]},{"id":"97f6ebfb53da899d","type":"function","z":"8616f41adb0d8dd9","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": \"register_client\",\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:16',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":390,"y":2540,"wires":[["39636e5ece70c6c8"]]},{"id":"0859d213ea46f128","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":210,"y":2600,"wires":[["97f6ebfb53da899d"]]},{"id":"e2db179b7d78ea59","type":"debug","z":"8616f41adb0d8dd9","name":"debug 358","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1140,"y":2540,"wires":[]},{"id":"39636e5ece70c6c8","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\n\nconst m2 = {\n    payload: Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\n}\nreturn m2","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":550,"y":2660,"wires":[["7e6b012ff3c07eec"]]}]

[Steve-Mcl]

@fullmooooon

Do you mean to modify it like this?

Yes

@fullmooooon

Because it will only be executed onc

No, it will be executed every time the Node receives a message.

But I think you mean it will be a new message arriving from upstream so it will be a different object anyway (and you'd be correct)

---

Thinking about it again with a fresh mind, I now think the TCP request node is receiving the larger packets in chunks and the message is being updated (by reference) while in the middle of your loop.

The solution to that is to clone the msg to a local variable in the function after the TCP node to avoid by reference updates affecting it. By that I mean, the TCP request may still have a reference to the original msg while data chunks are recurved.

Try this instead (in the function after the TCP node):

const m = RED.util.cloneMessage(msg)
// do work
return m

[fullmooooon]

@Steve-Mcl Yes，now no buffer
But the problem of disordered execution of my TCP has not been solved.
I dare to judge that the data that reaches "the function after the TCP node" in the same millisecond is still executed in the wrong order.After doing "cloneMessage" in my real code.

[{"id":"fa98aed08e74fe56","type":"function","z":"8616f41adb0d8dd9","name":"Testing function","func":"const m = RED.util.cloneMessage(msg)\nm.payload = []\n; (() => {\n    let i = 0\n    while (i < 100000000) i++\n})()\nreturn m;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":960,"y":2620,"wires":[["e2db179b7d78ea59"]]},{"id":"7e6b012ff3c07eec","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"10.168.1.115","port":"6789","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":780,"y":2540,"wires":[["fa98aed08e74fe56"]]},{"id":"97f6ebfb53da899d","type":"function","z":"8616f41adb0d8dd9","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": \"register_client\",\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:16',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":390,"y":2540,"wires":[["39636e5ece70c6c8"]]},{"id":"0859d213ea46f128","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":210,"y":2600,"wires":[["97f6ebfb53da899d"]]},{"id":"e2db179b7d78ea59","type":"debug","z":"8616f41adb0d8dd9","name":"debug 358","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1140,"y":2540,"wires":[]},{"id":"39636e5ece70c6c8","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\n\nconst m2 = {\n    payload: Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\n}\nreturn m2","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":550,"y":2620,"wires":[["7e6b012ff3c07eec"]]}]

[Steve-Mcl]

@fullmooooon

@Steve-Mcl Yes，now no buffer

Good, that is solved.

@fullmooooon

But the problem of disordered execution of my TCP has not been solved.
I dare to judge that the data that reaches "the function after the TCP node" in the same millisecond is still executed in the wrong order.After doing "cloneMessage" in my real code

Hi, I cant quite understand your statement above.

I suspect your issue now is dealing with broken/part responses.

You will need to "collect" (store) the responses from the TCP request node until you know what the last part is. Perhaps you can already clearly identify the "last part" of the stream? If you can, then use that to release this "collected" transmission once it is fully re-assembled.

Or perhaps I am missing your point?

---

Can I ask - where is the data coming from? Do you have control of the application sending this TCP data? Could you for example, add something to the data that clearly signifies the end for the stream?

Alternatively, you could greatly simplify this by using something like MQTT - is that an option? Could your application sending the data be modified to utilise a known protocol like MQTT?

[fullmooooon]

@Steve-Mcl For normal node-red, "Tcp msg is reassigned to the next data before debug output" will not appear even if the msg object is reused.
An uncertain opportunity caused my node-red to enter an irrecoverable error state.
This problem occurs when msg objects are reused in this state, and tcp data received in the same millisecond will arrive at "the function after the TCP node" in the wrong order.

So I think these two phenomena are related, and instead of using a new msg object or "cloneMessage", eradicating asynchronous errors from the source code will allow my code to output normally.

I'm communicating with a robot server.
I can't change it.
There is an Android program and a python program to communicate with it normally, and my task now is to replace them with node-red.
I agreed with it to add "length buffer header" before every json message we communicated.
Tcp connection establishment will not stop. I can sometimes parse multiple json from one data acquisition.
If I can accept tcp data correctly and continuously, I can parse it into a json.
Now there is something asynchronous error with my "tcp request" related code, and it has many manifestations.
Covering one of them doesn't really solve the problem.

By the way, can you reproduce this asynchronous problem on your node-red?

[Steve-Mcl]

@fullmooooon

By the way, can you reproduce this asynchronous problem on your node-red?

Unfortunately not.

If you could create an "all in one" flow that demonstrates the issue for me I will certainly look at it and fix the TCP Request node if it turns out to be a bug.

By "all in one" flow, i mean:

• add a flow that simulates your robot
  • tcp-in → Function that simulates and returns realistic data → tcp-out
• add realistic version of your production flow/code - the part where you see this issue
• some annotations as to what you expect vs what you end up with.

If you can provide that, I will dig deeper into the weeds and try to either find you a workaround AND/OR fix the source/raise a fix PR.

[fullmooooon]

I will try to do it, maybe once again the mistake will only appear on my computer.
Or maybe my computer "forget about the TCP server" and everything will go back to normal.

Thank you for your kind help.

[fullmooooon]

OK, I seem to be making some progress.
But I'm still not sure if this will work on your computer.
I'm sorry, I can't write a better example.
Change any node, and then redeploy the code can help you stop executing my code immediately
The error seems to have occurred when multiple tcp data streams were created
I'm not sure if this is expected behavior, but I want to ask if it's possible that tcp connections created by something other than node-red have also been captured.

“https://t.co/N3P19NiV32” / X (twitter.com)

[{"id":"e770d1a87a75d75a","type":"tcp in","z":"8616f41adb0d8dd9","name":"","server":"server","host":"","port":"11451","datamode":"stream","datatype":"buffer","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":680,"y":5180,"wires":[["0ae87cea6bd17e46"]]},{"id":"213fc0bc686abead","type":"tcp out","z":"8616f41adb0d8dd9","name":"","host":"","port":"","beserver":"reply","base64":false,"end":false,"tls":"","x":1050,"y":5180,"wires":[]},{"id":"5de0adffb377b635","type":"function","z":"8616f41adb0d8dd9","name":"Testing function","func":"msg.payload = []\n    ; (() => {\n        let i = 0\n        while (i < 100000000) i++\n    })()\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":1080,"y":5360,"wires":[["7d782d8388e38375"]]},{"id":"17c40231229c2fd2","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"127.0.0.1","port":"11451","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":870,"y":5280,"wires":[["5de0adffb377b635"]]},{"id":"4a3383e84f2ac2f1","type":"function","z":"8616f41adb0d8dd9","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": \"register_client\",\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:16',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":5300,"wires":[["068626661e01ea38"]]},{"id":"7d782d8388e38375","type":"debug","z":"8616f41adb0d8dd9","name":"debug 365","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1260,"y":5280,"wires":[]},{"id":"068626661e01ea38","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\nmsg.payload = Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\nreturn msg","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":690,"y":5360,"wires":[["17c40231229c2fd2"]]},{"id":"be012550c9a22fb5","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":350,"y":5240,"wires":[["4a3383e84f2ac2f1"]]},{"id":"dd2349240851b3c4","type":"function","z":"8616f41adb0d8dd9","name":"function 72","func":"\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1310,"y":5160,"wires":[["0ae87cea6bd17e46"]]},{"id":"ec532563e6157b52","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":670,"y":5460,"wires":[["2e8111af9d679cce"]]},{"id":"2e8111af9d679cce","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"127.0.0.1","port":"11451","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":870,"y":5460,"wires":[[]]},{"id":"0ae87cea6bd17e46","type":"function","z":"8616f41adb0d8dd9","name":"function 73","func":"msg.payload = {\n    \"occupied_color_thresh\": 50,\n    \"create_time\": \"2023/03/07 16:55:02\",\n    \"resolution\": 0.05000000074505806,\n    \"name\": \"4F\",\n    \"width\": 1188,\n    \"no_information_color\": 127,\n    \"md5\": \"a04d4590e0004f239c3d58b42b33a271\",\n    \"free_color_thresh\": 220,\n    \"update_time\": \"2023/03/07 16:56:12\",\n    \"uuid\": \"2023_02_07_16_36_19-ab4d2605-2135-4140-a6e0-879d3a0333f5\",\n    \"height\": 628,\n    \"version\": \"1.0.0\"\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":870,"y":4980,"wires":[["213fc0bc686abead","acbe1ae00b772871"]]},{"id":"acbe1ae00b772871","type":"delay","z":"8616f41adb0d8dd9","name":"","pauseType":"delay","timeout":"0.2","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1100,"y":4980,"wires":[["dd2349240851b3c4"]]}]

[marcus-j-davies]

I'm not going to pretend this is an answer to this discussion.

But here is subflow that I put together, that fires up a TCP Client inside of a Function node
But it may help to diagnose where the fault is maybe? as it will not use the builtin TCP nodes. I use the net module directly.

[{"id":"1c48198f5312c5be","type":"subflow","name":"TCP Client","info":"","category":"","in":[{"x":50,"y":70,"wires":[{"id":"f367588fb2ddf1d2"}]}],"out":[{"x":440,"y":225,"wires":[{"id":"18cda588030ee587","port":1}]}],"env":[{"name":"IP","type":"str","value":""},{"name":"PORT","type":"num","value":""}],"meta":{},"color":"#3FADB5","icon":"font-awesome/fa-sitemap","status":{"x":440,"y":170,"wires":[{"id":"18cda588030ee587","port":0}]}},{"id":"18cda588030ee587","type":"function","z":"1c48198f5312c5be","name":"Connect","func":"const Client = new net.Socket();\nClient.connect(env.get('PORT'), env.get('IP'), function() {\n\tnode.send([{payload:\"Connected to server!\"},undefined])\n\tflow.set('Client',Client)\n\n});\n\nClient.on('data', function(data) {\n\tnode.send([undefined,{payload:data}])\n});","outputs":2,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"net","module":"net"}],"x":295,"y":190,"wires":[[],[]]},{"id":"756cf26e0613298c","type":"inject","z":"1c48198f5312c5be","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":125,"y":190,"wires":[["18cda588030ee587"]]},{"id":"f367588fb2ddf1d2","type":"function","z":"1c48198f5312c5be","name":"Send","func":"flow.get('Client').write(msg.payload)","outputs":0,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":285,"y":75,"wires":[]},{"id":"81978563d81f2eb8","type":"tab","label":"Flow 2","disabled":false,"info":"","env":[]},{"id":"aa93bbc05a9fdd80","type":"debug","z":"81978563d81f2eb8","name":"debug 3","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":705,"y":140,"wires":[]},{"id":"45d1ad52e7010ac0","type":"inject","z":"81978563d81f2eb8","name":"Hello, Wolrd Buffer","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[72,101,108,108,111,44,32,87,111,114,108,100]","payloadType":"bin","x":240,"y":140,"wires":[["a191d407285a22cb"]]},{"id":"a191d407285a22cb","type":"subflow:1c48198f5312c5be","z":"81978563d81f2eb8","name":"TCP CLient","env":[{"name":"IP","value":"127.0.0.1","type":"str"},{"name":"PORT","value":"4050","type":"num"}],"x":485,"y":140,"wires":[["aa93bbc05a9fdd80"]]},{"id":"8ea97c3691e14a89","type":"tcp in","z":"81978563d81f2eb8","name":"","server":"server","host":"","port":"4050","datamode":"stream","datatype":"buffer","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":215,"y":215,"wires":[["f3a9f84b0c879034"]]},{"id":"f3a9f84b0c879034","type":"function","z":"81978563d81f2eb8","name":"Echo","func":"\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":465,"y":215,"wires":[["ca9ba53aada77196"]]},{"id":"ca9ba53aada77196","type":"tcp out","z":"81978563d81f2eb8","name":"","host":"","port":"","beserver":"reply","base64":false,"end":false,"tls":"","x":700,"y":210,"wires":[]}]

[Steve-Mcl]

Question 1:

Are there multiple robots sending data to the SAME IP:PORT (to one Node-RED)?

Question 2 :

Are you using the TCP request to "Start" a "subscription" of some kind?

I.e. Are you sending some kind of "start sending your data" command to the robot(s)

Question 3:

Assuming there is no gateway, how do you "Start" multiple robots "talking"?

---

I think you might be using the wrong nodes.

The TCP Request node is typically for sending a REQUEST and expecting a single RESPONSE.

It is likely you need to use the TCP OUT (for sending your REQUEST command) and the TCP-in for listening. This is designed for (and will permit) multiple connections.

Note how the TCP-in tells you which port and IP the message originated from?
As I only have 1 computer available to me right now, I had to simulate this using different port numbers but the theory is the same. You will be able to use the msg IP and PORT to keep your streams of data isolated.

e.g:

[{"id":"e770d1a87a75d75a","type":"tcp in","z":"e9971c95786fd2be","name":"Robot 1 (:11451)","server":"server","host":"","port":"11451","datamode":"stream","datatype":"buffer","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":1200,"y":80,"wires":[["bd729694ead6ac74"]]},{"id":"5de0adffb377b635","type":"function","z":"e9971c95786fd2be","name":"Testing function","func":"msg.payload = []\n    ; (() => {\n        let i = 0\n        while (i < 100000000) i++\n    })()\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":1680,"y":520,"wires":[["7d782d8388e38375"]]},{"id":"4a3383e84f2ac2f1","type":"function","z":"e9971c95786fd2be","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": msg.payload,\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:02',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1430,"y":300,"wires":[["068626661e01ea38"]]},{"id":"7d782d8388e38375","type":"debug","z":"e9971c95786fd2be","name":"debug 365","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1910,"y":520,"wires":[]},{"id":"068626661e01ea38","type":"function","z":"e9971c95786fd2be","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\nmsg.payload = Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\nreturn msg","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1650,"y":300,"wires":[["b4e1590c27127a22"]]},{"id":"0ae87cea6bd17e46","type":"function","z":"e9971c95786fd2be","name":"fake data","func":"\nif (Buffer.isBuffer(msg.payload) && msg.payload.length > 4) {\n    /** @type {Buffer} */\n    const buf = msg.payload\n    const sanitised = buf.slice(4) // remove header\n    const payload = JSON.parse(sanitised)\n\n    if (payload.message_type === 'register_client') {\n        context.set('subscribed', true)\n    } else if (payload.message_type === 'deregister_client') {\n        context.set('subscribed', false)\n    }\n}\n\nconst subscribed = context.get('subscribed') === true\n\nif (!subscribed) {\n    return null // halt flow\n}\n\nmsg.payload = {\n    \"occupied_color_thresh\": 50,\n    \"create_time\": \"2023/03/07 16:55:02\",\n    \"resolution\": 0.05000000074505806,\n    \"name\": \"4F\",\n    \"width\": 1188,\n    \"no_information_color\": 127,\n    \"md5\": \"a04d4590e0004f239c3d58b42b33a271\",\n    \"free_color_thresh\": 220,\n    \"update_time\": \"2023/03/07 16:56:12\",\n    \"uuid\": \"2023_02_07_16_36_19-ab4d2605-2135-4140-a6e0-879d3a0333f5\",\n    \"height\": 628,\n    \"version\": \"1.0.0\"\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1400,"y":200,"wires":[["3d6b2d4dd2cb4687","acbe1ae00b772871"]]},{"id":"acbe1ae00b772871","type":"delay","z":"e9971c95786fd2be","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1600,"y":200,"wires":[["0ae87cea6bd17e46"]]},{"id":"b4e1590c27127a22","type":"tcp out","z":"e9971c95786fd2be","name":"","host":"127.0.0.1","port":"11451","beserver":"client","base64":false,"end":false,"tls":"","x":1870,"y":300,"wires":[]},{"id":"3d6b2d4dd2cb4687","type":"tcp out","z":"e9971c95786fd2be","name":"","host":"127.0.0.1","port":"11400","beserver":"client","base64":false,"end":true,"tls":"","x":1630,"y":140,"wires":[]},{"id":"6df17ef935d04e70","type":"tcp out","z":"e9971c95786fd2be","name":"","host":"127.0.0.1","port":"11452","beserver":"client","base64":false,"end":false,"tls":"","x":1870,"y":400,"wires":[]},{"id":"7151a0663278bd6f","type":"tcp in","z":"e9971c95786fd2be","name":"Robot 2 (:11452)","server":"server","host":"","port":"11452","datamode":"stream","datatype":"buffer","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":1200,"y":200,"wires":[["0ae87cea6bd17e46"]]},{"id":"93a4c1057939636a","type":"tcp in","z":"e9971c95786fd2be","name":"Listener (:11400)","server":"server","host":"","port":"11400","datamode":"stream","datatype":"buffer","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":1420,"y":520,"wires":[["115951d3fe1d2e15"]]},{"id":"bd5b3255b0968ce0","type":"function","z":"e9971c95786fd2be","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": msg.payload,\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:02',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1430,"y":400,"wires":[["f408e93a924c5042"]]},{"id":"f408e93a924c5042","type":"function","z":"e9971c95786fd2be","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\nmsg.payload = Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\nreturn msg","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1650,"y":400,"wires":[["6df17ef935d04e70"]]},{"id":"115951d3fe1d2e15","type":"debug","z":"e9971c95786fd2be","name":"FULL MSG","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1560,"y":580,"wires":[]},{"id":"bd729694ead6ac74","type":"function","z":"e9971c95786fd2be","name":"fake data","func":"\nif (Buffer.isBuffer(msg.payload) && msg.payload.length > 4) {\n    /** @type {Buffer} */ \n    const buf = msg.payload\n    const sanitised = buf.slice(4) // remove header\n    const payload = JSON.parse(sanitised)\n\n    if (payload.message_type === 'register_client') {\n        context.set('subscribed', true)\n    } else if (payload.message_type === 'deregister_client') {\n        context.set('subscribed', false)\n    }\n}\n\nconst subscribed = context.get('subscribed') === true\n\nif (!subscribed) {\n    return null // halt flow\n}\n\nmsg.payload = {\n    \"occupied_color_thresh\": 40,\n    \"create_time\": \"2023/03/07 16:55:02\",\n    \"resolution\": 0.04000000074505806,\n    \"name\": \"44\",\n    \"width\": 1144,\n    \"no_information_color\": 104,\n    \"md5\": \"a04d4590e0004f239c3d58b42b33a271\",\n    \"free_color_thresh\": 220,\n    \"update_time\": \"2023/03/07 16:56:12\",\n    \"uuid\": \"2023_02_07_16_36_19-ab4d2605-2135-4140-a6e0-879d3a0333f5\",\n    \"height\": 644,\n    \"version\": \"1.0.4\"\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1400,"y":80,"wires":[["0728efffc32ad281","e506212cafdc89d0"]]},{"id":"e506212cafdc89d0","type":"delay","z":"e9971c95786fd2be","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1600,"y":80,"wires":[["bd729694ead6ac74"]]},{"id":"0728efffc32ad281","type":"tcp out","z":"e9971c95786fd2be","name":"","host":"127.0.0.1","port":"11400","beserver":"client","base64":false,"end":true,"tls":"","x":1630,"y":20,"wires":[]},{"id":"4d895830606f27da","type":"comment","z":"e9971c95786fd2be","name":"Fake robot 1","info":"","x":1190,"y":40,"wires":[]},{"id":"cce629cb4f534de3","type":"comment","z":"e9971c95786fd2be","name":"Fake robot 2","info":"","x":1190,"y":160,"wires":[]},{"id":"680cf1306e91dda0","type":"inject","z":"e9971c95786fd2be","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"register_client","payloadType":"str","x":1210,"y":400,"wires":[["bd5b3255b0968ce0"]]},{"id":"4f0d6dc29c7050fd","type":"inject","z":"e9971c95786fd2be","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"deregister_client","payloadType":"str","x":1220,"y":440,"wires":[["bd5b3255b0968ce0"]]},{"id":"e55b993e7c0952cb","type":"inject","z":"e9971c95786fd2be","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"register_client","payloadType":"str","x":1210,"y":300,"wires":[["4a3383e84f2ac2f1"]]},{"id":"9b4be3671275a06d","type":"inject","z":"e9971c95786fd2be","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"deregister_client","payloadType":"str","x":1220,"y":340,"wires":[["4a3383e84f2ac2f1"]]}]

[fullmooooon]

The robot server has only one port on one ip, and the robot has only one.
There is a wifi router on the robot, and my laptop will connect it.
The robot will use udp to broadcast "Hey, you can connect me" to the local area network, and then after I tcp connect to it, it will send data non-stop (the data that is reported regularly and the result of my request).
If I want to control it, I'll send a piece of data.
Question 3 I don't quite understand.
In the future, my program will run directly on the robot system and communicate with the local area network.

I think I'm preconceived.
"TCP-OUT" and "TCP-in" I didn't succeed in the test at first.
"tcp request" sometimes works well and I use it all the time.
There are indeed a lot of "node" connections at the left end of my current code "tcp request", some of which start at the right end of it.
Usage restrictions are not specified in the "help" of "tcp request".
I will try to use "TCP-OUT" and "TCP-in" tomorrow.

[Steve-Mcl]

Hi, sorry, it is unclear to me how this all fits together. You didn't really answer my Q1 or Q2 (which then makes Q3 not relevant)

If you provide a drawing that shows Node-REDs placement and how it is intended to be used from all sides e.g. its relationship to 1 (or multiple?) robot(s), its network connectivity, any external interfacing etc.

Regards.

EDIT:

I will add though, based on this:

@fullmooooon

I tcp connect to it, it will send data non-stop

Suggests TCP-out and TCP-In are the correct choice (not the TCP Request)

[fullmooooon]

Are there multiple robots sending data to the SAME IP:PORT (to one Node-RED)?

only one robot from one port on one ip

Are you using the TCP request to "Start" a "subscription" of some kind?

I send a tcp data saying, "Hey, robot, go to A1." it will tell me later that "I know" is interspersed with periodic data.
Specifically, it will only be subscribed once.

Node-red will be used in conjunction with Electron, and I'm still doing early work.
The left end of "tcp request" will be triggered many times, and I suspect that this behavior is illegal.

There is only one active "TCP request" in my actual flow.

[Steve-Mcl]

@fullmooooon

interspersed with periodic data.

You need to use the "TCP-In" for unsolicited data (the periodic data)

And you should use "TCP-out" for sending the commands to end point.

If you do manage to put together a drawing and can provide the protocol detail i may be able to help further - but as I only have "part of the overall picture" I can not help more at this point.

Give it a go, see if you get any further, pop back if you get stuck.

Good luck.

[fullmooooon]

@Steve-Mcl Please forget my wild guess above, now I have solved all the problems.
I understand what happened now.

At first, my data processing was not closely followed by the right end of "tcp request".And the reference is not cut off.
It causes my program to completely lose its ability to work after an asynchronous object reference error occurs.
So I added a piece of code to restore processing power after an error. If the result of the "4-bit length header buffer" readUInt32LE is greater than 10000, I abandon it completely.
Then, of course, there will be several data parsing errors.
And then get back to normal.

We have actually solved the fundamental problem, and we need to create a new msg object in the first function that follows "tcp request".At this point, it can be performed in the expected order.
I misestimated the possible length of a json.
The server sometimes sends me files, which, of course, can be more than 10000 bytes.
Today I've deleted this temporary code because it's supposed to be the wrong length of buffer, and I don't have asynchronous errors now.

"tcp in" and "tcp out" do not work.
I need to first send a registration request for the first connection, and then keep the connection.
I can't explain it clearly. Anyway, it doesn't work. The connection will be disconnected later.
Now "tcp request" is working very well.
Just pay attention to some "details" that will not be written in the help and documentation.

Hey, thank you, too @marcus-j-davies .

@Steve-Mcl

If you could create an "all in one" flow that demonstrates the issue for me I will certainly look at it and fix the TCP Request node if it turns out to be a bug.

Maybe we need a little data backlog. The local service is too fast.

The data is changed between "Testing function" and "debug"。

[{"id":"fa98aed08e74fe56","type":"function","z":"8616f41adb0d8dd9","name":"Testing function","func":"msg.payload = []\n; (() => {\n    let i = 0\n    while (i < 100000000) i++\n})()\nnode.error(msg.payload);\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"fs","module":"fs"}],"x":840,"y":1180,"wires":[["e2db179b7d78ea59"]]},{"id":"7e6b012ff3c07eec","type":"tcp request","z":"8616f41adb0d8dd9","name":"","server":"10.168.1.115","port":"6789","out":"sit","ret":"buffer","splitc":" ","newline":"","trim":false,"tls":"","x":620,"y":1280,"wires":[["fa98aed08e74fe56"]]},{"id":"97f6ebfb53da899d","type":"function","z":"8616f41adb0d8dd9","name":"Construction data","func":"let jsonBody = {\n    \"message_type\": \"register_client\",\n    \"client_type\": 3,\n    \"mac_address\": '00:e0:67:2f:cd:16',\n}\n\nmsg.payload = jsonBody\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":230,"y":1280,"wires":[["39636e5ece70c6c8"]]},{"id":"0859d213ea46f128","type":"inject","z":"8616f41adb0d8dd9","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":1360,"wires":[["97f6ebfb53da899d"]]},{"id":"e2db179b7d78ea59","type":"debug","z":"8616f41adb0d8dd9","name":"debug 358","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":980,"y":1280,"wires":[]},{"id":"39636e5ece70c6c8","type":"function","z":"8616f41adb0d8dd9","name":"Add length buffer header","func":"let jsonBody = msg.payload\nlet headBuffer = Buffer.alloc(4)\nheadBuffer.writeUint32LE(JSON.stringify(jsonBody).length)\n\nconst m2 = {\n    payload: Buffer.from([...headBuffer, ...Buffer.from(JSON.stringify(jsonBody), 'utf-8')])\n}\nreturn m2","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":430,"y":1340,"wires":[["7e6b012ff3c07eec"]]}]

[Steve-Mcl]

@fullmooooon

and we need to create a new msg object in the first function that follows "tcp request"

I did suggest this some time ago

@Steve-Mcl

The solution to that is to clone the msg to a local variable in the function after the TCP node to avoid by reference updates affecting it. By that I mean, the TCP request may still have a reference to the original msg while data chunks are recurved.

Try this instead (in the function after the TCP node):

const m = RED.util.cloneMessage(msg)
// do work
return m

I still maintain that the TCP-In/Out nodes are the correct nodes to use. But as you have a working solution, I will leave you with that. If I fully understood all devices involved and had a schematic I may be able to advise but that is moot at this point.

Glad you are up and running.