TCP not working for 30 min since new IP is given to 3G network interface

General
1

Hi everyone,

we have developed a TCP client on a Gateway which runs Node.Red 0.11. This device receives LoRa messages from a LoRa In node and forwards these to our own network service through a TCP Out which connects to a IP:Port.

Meanwhile, a TCP In node connects to the same IP:Port and forwards incoming messages to the LoRa nodes through a LoRa out node.

In our service we see two TCP sockets connected from our gateway IP address and two different ports. From one of them we receive the messages and through the other one we can send our responses.

This is working fine except when the 3G interface (ppp0) is given a new IP once every 24h. We check this using an Exec node with ifconfig ppp0 command and store this info in a log file. At this time we don't receive any more messages in our service for approx. the next 30 minutes. Meanwhile there is no error ouput in the debug tab and the TCP nodes remain "connected". After this period, new socket is established from the new IP and messages are passing through again. We have tried "close connection after each message is send" but there is no difference.

Moreover, since the first IP change (after the first 24h) the TCP In socket is not created from the new 3G IP, therefore leaving the gateway unreachable. TCP In node remains though "connected".

We have checked the sockets with lsof -np and it turns out that there is a TCP socket connected to our service from the oldest ip marked as "ESTABLISHED" which must be the one belonging to the TCP In node (which doesn't work). The other socket connected to our service has the updated IP except during that half an hour after the IP change. During those minutes the origin IP doesn't match the one of the ppp0 interface and the socket is marked as FIN_WAIT1.

Is there some way to reset the TCP connections? Is it a known issue of this node.red version?

Thanks in advance for the help.

2

This is the exact sort of situation that MQTT will solve in a heartbeat.
Use MQTT to send the data to a broker and have the client subscribe to it. Put the broker either on premise or in the cloud where its host name is fixed.
No more managing TCP sockets, timeouts or IP changes.
It should take less time to setup a broker and tweak your flows than it will to get all your existing issues resolved.

3

Thanks @thebaldgeek, MQTT is probably worth considering, we'll take a look. However, since our rx service is already in an intermediate development phase we would like to solve this problem with the TCP sockets. Any idea?

4

The simplest (to me) would be to have an external (to Node-RED) watchdog script, that monitored its IP and if it see it change to stop and restart Node-RED.

1 Like
5

Restarting node-red service seemed like the last alternative but this is what we finally did until we find another solution or it solves by updating the system.

Thanks @dceejay for the suggestion, we finally implemented a node-red restart from inside node-red for simplicity. It worked, and now we don't have a 30-minute lapse with no outgoing socket connection but only a total disconnection for the time node-red takes to restart (around 2-4 minutes). It also solved the problem with our lost incoming socket that had an old IP.

This is the flow, in case it's useful to anybody:

RestartNodeRedWithIPChange
RestartNodeRedWithIPChange908Ă—134 13.4 KB

[{"id":"e695f428.1e5e08","type":"inject","name":"Get IP","topic":"ip","payload":"eth0","payloadType":"none","repeat":"30","crontab":"","once":false,"x":155,"y":628,"z":"d36858bd.a88d38","wires":[["39010e50.9b6242"]]},{"id":"39010e50.9b6242","type":"exec","command":"/sbin/ifconfig ppp0 | grep 'inet addr:' | cut -d: -f2| cut -d' ' -f1","addpay":false,"append":"","useSpawn":"","name":"Check IP ppp0","x":308,"y":628,"z":"d36858bd.a88d38","wires":[["acdd3259.38a01"],[],[]]},{"id":"acdd3259.38a01","type":"switch","name":"Integrity check","property":"payload","rules":[{"t":"regex","v":"\\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\b"},{"t":"else"}],"checkall":"true","outputs":2,"x":477,"y":614.5,"z":"d36858bd.a88d38","wires":[["ba5b706a.e224c"],[]]},{"id":"ba5b706a.e224c","type":"function","name":"Compare IP","func":"context.lastip = context.lastip || 'initial';\nvar currentip = msg.payload;\n\nif (context.lastip == 'initial') {\ncontext.lastip = currentip;\n}\nelse if (context.lastip != currentip) {\nmsg.payload = \"My current IP is \"+currentip+\" at \"+ Date();\ncontext.lastip = currentip;\nreturn msg;\n}","outputs":1,"noerr":0,"x":641,"y":607.5,"z":"d36858bd.a88d38","wires":[["3eb06151.f4998e","8856eeec.d96f9","ae78da2.c33ae28","5adaee06.c2476"]]},{"id":"5adaee06.c2476","type":"delay","name":"Delay 5s","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":728,"y":693,"z":"d36858bd.a88d38","wires":[["4fcfb7a6.a06ef8"]]},{"id":"4fcfb7a6.a06ef8","type":"exec","command":"/etc/init.d/node-red restart","addpay":false,"append":"","useSpawn":"","name":"Restart node-red","x":894,"y":693,"z":"d36858bd.a88d38","wires":[[],[],[]]}]
6

Are you saying it takes 2-4 mins for node-red to start up?

If so, what platform is it running on?

I've seen node-red take 3min on a Windows box when it's behind a proxy. I don't have answer to hand but have posted it before. Now my windows based node-red instances start up in about 8 secs.

7

This thread about if behind proxy and long start time Node-RED v0.19.4 3 minute delay during startup on windows

1 Like
8

Ah yes - there it is (>> this << is the link to the solution i was talking about on that very thread)

9

I've checked and it takes approx. 2 min 10 sec since I restart it till it connects the first socket, but as I said it's a LoRa Gateway with most likely an ARM9. I don't know how much should it take to restart in such a device, does anyone have a similar platform and has measured the node-red boot time?

10

Can I assume this device does not have internet?

It not, then it may still help speed up node-red start time.

The issue is (something like) nodejs attempts to check for updates and stalls.

Try setting an env var
NO_UPDATE_NOTIFIER to 1 before launching node-red.

It may be just a windows issue but worth a go.

11

This device does have internet, it's a LoRa gateway which forwards data from/to a server through a TCP socket. The problem was that the 3G internet connection IP gets refreshed once every 24 hours.

I've tried though to apply the patch you mention, but there is no node-red.cmd in the whole system, only an npm.cmd with nothing related to NO_UPDATE_NOTIFIER. :frowning:

12

If its a linux box there wont be one :slight_smile:

As a test, just set env var NO_UPDATE_NOTIFIER to 1 before launching node-red e.g.

export NO_UPDATE_NOTIFIER=1
node-red

Though i do realise a closed box like this may not permit tinkering.

TBH, the node-red version is ancient on that device (latest on their updates page is ~ V0.15.x) - personally, I'd buy a raspberry pi for node-red operations & leave that device doing what its doing (but thats me)

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.