Apologies for the backticks... I'm still a bit dumb here!!
Like this?: Testing ... Just learnt something - Thanks!!!!
Would Disabling them be sufficient, or, would disabling the flow that they are on also suffice?
Tia
E
And this:
` ` `
some code - but without the spaces between the ticks of course
` ` `
It should be enough. Though to be safe, you could export the single node then delete it. You can reimport later, just use a sensible naming convention for the saved files.
For nodes that use configuration nodes though, don't forget about those.
Are you seeing thousands of lines altogether?
If there are thousands then there might be indications as to what is using them all up.
Feel free to PM the file to me if you want me to have a look but don't want to make it public.
The problem with disabling, for example MQTT is that it will presumably dramatically change what happens as there will be no data, so whatever is the problem may no longer happen, even if it is not directly related to the MQTT comms.
Seeing Lots.... (Not understanding much, but seeing lots nevertheless!!)
Not worried about sending here, nothing incriminating that I can see in it!
Colin.txt (979.8 KB)
Here you go!!
Ed
Yes, I know. But it should show which node is causing the issue. A case of trial and error since there appear to be no other clues at this time.
Ouch! You have a LOT running there. MySQL, Apache, REDIS, Pure-FTP, some kind of Python server, some kind of PHP server. But you are also running a Linux desktop and a remote VNC session. That is a lot of load even for a 4GB Pi4.
But the node entries are the most interesting. The many TCP sockets are clearly the problem. That doesn't appear to be related to MQTT. So something else is creating the sockets.
Can you please share with us, all of the node types you have installed and all the ones that you have instances of in use? You can process the flows file to get that.
Hmm, and don't forget that if you are using a MySQL node, that is creating connections as well - in fact, that would be my guess as to what is causing the issues.
Before I go B to the W and start disabling things... Does this look "generally ok"?
And looking at this:
Particularly the number behind that little mosquito.... It might take me a while!!
Ta
E
It is.... but.... in defence, I have kept strict tabs on CPU/Mem/Swap use.... CPU around 20% avg, mem around 30% avg, swap 0% normally... or very low... The Pi is "perky" to say the least and more than quick on average!
Would be glad to! Could you give me a pointer as to what is needed to process the flows file?
Regds
E
Did you see my comment about MySQL as well?
What is "chronos"?
Yes, you certainly have your work cut out I'm afraid. But I don't see any way around it.
By the way, you will get a better picture from lsof if you do lsof | grep "^node" as that shows you all of the connections that Node-RED is doing and will show some more detail. Alternatively, use sudo lsof -c node to see the connections opened by the node process or sudo lsof -p 14996 (there is a lot more you can do: lsof(8) - Linux manual page (man7.org).
I think that, if we want to dig deeper on these connections, we need to use netstat next.
I did, Logging for Emoncms if I remember correctly... (Cleared all data about a week back, maybe a bit longer, no difference to stability per se... The data files went back 3 or more years, 4gb, now a couple of weeks old and sitting at 130Mb)
Pretty much just a "Time Based Filter":
Just sends me a reminder based on the time...
E
Hopefully you only have the one mqtt connection set up like that - otherwise if you have multiple with the same clientID (InvPI) then they will knock each other off the broker - which would then lead to a lot of hanging tcp sockets. If in doubt don't set the clientID and let the system randomly assign one.
1 Like
Sorry, I keep updating my comments - see my update on using netstat.
Yep, its only the one client.... It was set as no clientID originally (earlier today) and I "tied it down" to this setting, in case it was a problem... No difference though...
Tnx
E
No problem at all!!.... I am just uber grateful for all the help!!
Just point me in the direction to go!!
Tia
Ed
First of all, make sure you've checked out Dave's last post and corrected that if it applies. If it does, you can ignore everything else probably.
Other wise, lets try starting with sudo netstat -ap - likely to produce a LOT of data so feed it to a file. See if you can spot the same socket connections. You may be able to match the I-Node number in this output to the inode column in your previous file output from lsof. For example "28721882". I don't have any open sock entries on any of my systems so I can't test that for you.
Ok...Waaaaaaay past being out of my depth here.... Sorry...
I hope I have done this right... I dumped them to two files....
Tif.txt (193.4 KB) Tif2.txt (360.0 KB)
Had a look into them ..... and uhhhh..... I'm the monkey with the Macbeth book....
E
And here is a list of nodes in use in all flows at the moment(According to Pallette):
Nodes in use (According to Pallette):
node-red
change
comment
debug
delay
exec
function
inject
json
link in
link out
mqtt in
mqtt out
mqtt-broker
split
join
status
switch
trigger
node-red-contrib-advanced-ping
node-red-contrib-bigtimer
node-red-contrib-chronos
chronos-config
chronos-filter
node-red-contrib-cpu
node-red-contrib-isonline
node-red-contrib-mqtt-json
node-red-contrib-moment
moment
node-red-contrib-os
node-red-contrib-schedex
node-red-contrib-string
node-red-contrib-telegrambot
node-red-contrib-toggle
node-red-contrib-ui-led
node-red-contrib-ui-multistate-switch
node-red-dashboard
ui_base
ui_button
ui_dropdown
ui_gauge
ui_group
ui_numeric
ui_slider
ui_spacer
ui_switch
ui_tab
ui_text
ui_text_input
ui_toast
node-red-node-emoncms
node-red-node-openweathermap
node-red-node-pi-gpio
node-red-node-ping
node-red-node-rbe
node-red-node-smooth
In response to the nodes in use.....
E
I see you are using an ancient, effectively undocumented, contrib node for MQTT -
node-red-contrib-mqtt-json. I would ditch that for a start. You can get the same effect with the standard MQTT node by selecting the output as parsed JSON in the node config. There has been a lot of work on MQTT in the last three years.
Uninstall that node after you have moved everything over.
1 Like
Thanx Colin... Will start doing it now.... Hold thumbs!! (About 130 nodes to replace...Maybe a few more, maybe a few less...)
