Node red flows seem to be slowing down

There are two aspects to this question. The first is whether the flow is operating as it should do, the second is whether it is too much for the processor.
That is why I asked if there is more MQTT data than you expected. I would not be happy just adding the RBE node in and accepting that that improves the problem without understanding what is causing the problem in the first place. So in your situation I would take the RBE out again, run it, and look at the mqtt data and try to determine whether there is more data that there should be. If you let it get to the point where it appears to be clogging up and then disable the debug node that will stop the output so you can then look at it, see the topics that are being published to, and how often, and see if there is something not right.
In fact, probably easier to analyse, would be to check the mqtt data in the console rather than in node red. So take out the MQTT In node subscribing to # and instead, when you get to the point that it seems to be clogging up, then in a terminal run
mosquitto_sub -t "#" -v | perl -pe 'print scalar(localtime()), " ";'
That will show all the MQTT activity on the local mosquitto, with timestamps (thats what the perl on the end does). You can then Ctrl-C out of that when it has run a bit and analyse it at your leisure.

As to the second problem, whether you are overloading the processor, with a Pi 4 and mosquitto I would expect to be able to do thousands, of MQTT writes/second before encountering a problem, assuming there is not much else going on. So for your system, unless there is a lot more activity than there should be (which the test above will determine), that should not be an issue. So do the tests above first and see where that takes us.

Hi Colin,

Another new trick for my toolbag.... not only am i not very experienced with node-red I also am a newbie to linux.

Removed the rbe nodes and

Ran the script and watched the activity and it seems that we are averaging about 4 to 5 messages per second. From your last reply that would seem to be ok.

I started noticing the issue with the latency after adding that Comparatore function that I got from the forum when I was stalled trying to figure out how to compare two temp sensors for my solar pump for the spa.

I know I don't know that that is the issue and while I only suspect it because I don't totally understand it all and that I did not do it myself.

the function does what it is supposed to do but could it be blocking in some other way than a mqtt loop?

It is doing the same thing still when I try to turn on any of my outputs without the rbe

Thanks again for taking your time to help me with this..... the forum out to have a way to send a virtual beer or drink or coffee to allow members to send a beer or coffee or something to those that help them..... I keep the forum members in beer probably myself.

Troy

It would be worth seeing if something is overloading the processor. To do that, in a terminal, run top and leave that running. See if any process is using up the processor. A pi 4 should easily cope with what you are trying to do.

Hi Colin,
Done that and I am watching it .... what am I looking for?

Colin,
The highest percentage is 1% among the users and a total of .2us .2sy 0.1ni 99.3 id 0 hi 0si ost....
not exactly sure of what this all means.
Troy

By default it should be sorting the processes in the table with the highest CPU at the top. When it starts slowing down see how much CPU is being used by what tasks. A Pi 4 has 4 cores I think the total CPU available is actually 200%. Not sure why. But in your application I wouldn't expect anything much above 10%.

[Edit] The processes are the list down the right hand side.

So then I should be fine with the highest use around 2% and the rest .3 and a bunch of 0%?

Absolutely. You should not see anything slow down unless you are getting readings around 80. You said earlier that when it slows down it can take several seconds for an event to be actioned. Is it still true that in those several seconds gap that the readings are still negligible?

Hi Colin,
Well it is like it is being blocked..... last time I tested it without the rbe node after the function comparatore in the flow it took like 45 seconds or more for any of the switch nodes to take effect.... also the last time it almost seemed to be random..... like the item would turn off with out me doing anything to the switch node.

The question is, though, what happens to the processor usage in that time?

Hi Colin,
I know I dont know what I am doing with node red, linux and the like,
But I am a reasonable logical person and have done a whole lot of troubleshooting on many things over the years.... too many hobies.

So my idea of logic says it is not mqtt being looped,
and it is not the raspberry pi being overwhelmed.

so I am not sure but that seems to point to a node-red problem and with my lack of experiance it would seem like I may have a mistake in my flow???
Troy

so I should run those diagnostics you show above when the RBE node is removed to see what may be happening to the mqtt and to the Pi???

Troy

Hi Colin,
Just did that and the numbers seem to show a little more cpu use but still under 2%.
and the latency is again a problem.....

now it takes about 2 mins to get the action to happen

I think you are saying that during those two minutes the processor usage is still low. If so then add some debug nodes to the flow so you follow the button press (or whatever it is) through the flow and see where it is getting stuck. So for example add a debug on the output of the dashboard button, one on the output of the next node along (or a few nodes along if there are lots, and so on. Give all the debug nodes names so it is easier to follow. Then you should be able to see where along the line it gets stuck. By shrinking the windows a bit you should be able to get the dashboard and the editor on the screen at once.

Hi Colin,
OK will do.... any way to make the debug side screen stop so I can read it better?

OK Colin,
Node red and the debugs at my output tell me that it is seeing the message with the output being as I would expect and instantly. However the client is not making the relays do as they should.

in fact it has been several minutes and nothing from the client. ???

What is the client?

Hi Colinm,
The client is a wemos d1 mini esp8266

Troy

So have you decided that it is a problem with D1?