How many messages are "flowing" in my NR?

#1

Hi! I see that the performance of the web page to control my NR are directly connected to the number of messages flowing, for example, as I use "loops" on some functions to introduce delay, I can commit sometimes some mistakes, and as result I've a loop of messages that slowing down the page (not really loading the Raspberry cpu instead). SO how can I avice if I did any kind of these mistakes and I'm having an uncontrolled amout of messages running? Is in NR a general "counter" of all the messages produced in a second?
I also have an unstabilty after abotu 10 days, NR keep working with I2C for example, but all the TCP of raspberry is gone, I can't do debug. I would be sure it is not caused from these infinite "loops".
Thank you for help!

0 Likes

#2

Can you give an example of how exactly you are doing that?

0 Likes

#3

This is a BAD IDEA. A loop like this will probably cause blocking!

*Node.js is a single threaded language which in background uses multiple threads to execute asynchronous code. Node . js is non-blocking which means that all functions ( callbacks ) are delegated to the event loop and they are ( or can be ) executed by different threads

Why not use a delay node?

0 Likes

#4

Thank you for answer, Yes I use delays and I use them exactly for this reason..
I need to produce multiple outputs from a function, and not all together! Sometime I need fix delay, sometime I need the function pass the delay value.
In these cases usually I output a compatible message as the input, puls I add an index. I defile this index 0 as the first external message doesn't contain it, and I add one to every time I need to output it again, till the number that will not produce any match on function internal if, so no return are execute, the loop will die.
For some reasons I can fail this check and the message may loop till the next deploy.
ANy chances to see if there are an uncontrollable numbers of messages? maybe log them all? maybe a simple counter?
Thank you

Example, check the incoming messages, if it's a "on" then it will switch on light of Corridoio, if it's off it will activate the Tuttospento (all lights off, which is an input of a remote Arduino that will switch off all the lights it manages), this output tuttospento should be turned off after some time.
:

[{"id":"99034f3d.9acd18","type":"delay","z":"fef64de3.66c588","name":"","pauseType":"delay","timeout":"10","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":1330,"y":1480,"wires":[["b044349b.8df628","c20a1a0c.a7844","ca983400.132a58"]]},{"id":"b044349b.8df628","type":"delay","z":"fef64de3.66c588","name":"","pauseType":"delay","timeout":"200","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":1050,"y":1560,"wires":[["7eb8d8ee.022b38"]]},{"id":"7eb8d8ee.022b38","type":"function","z":"fef64de3.66c588","name":"Gestisci TuttoSP Soppalco","func":"var fermati = msg.stopp || 0\nif (fermati===1) {msg={topic:\"TuttoSPSopp\", payload:\"On\", stopp:2};msg2={}; return [msg,msg2];} else {\nif ((fermati===0) & (msg.on===true) & (msg.comando==\"TuttoSP\")) {msg={topic:\"Corridoio\", on:true, stopp:2}; msg2={reset:1}; return [msg,msg2];}\nif ((fermati===0) & (msg.on===false) &(msg.comando==\"TuttoSP\")) {msg={topic:\"TuttoSPSopp\", payload:\"Off\", stopp:1}; msg2={reset:1}; return [msg,msg2];}\n}","outputs":2,"noerr":0,"x":1060,"y":1460,"wires":[["99034f3d.9acd18"],["b044349b.8df628"]]}]
0 Likes

#5

Another more clear example, here I activate a sequence that ill turn on or off all the lights:

[{"id":"f876ce1a.69e488","type":"delay","z":"fef64de3.66c588","name":"","pauseType":"delay","timeout":"220","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":870,"y":1300,"wires":[["97b87d45.156188","aaa099ca.3f89d8"]]},{"id":"97b87d45.156188","type":"function","z":"fef64de3.66c588","name":"Spegne tutto ciclico","func":"ciclospegnel= msg.ciclospegneluci || 0;\nciclospegnel++;\nSpento3mem=global.get('Spento3mem')|| false;\nif ((msg.topic==\"Spento3\") & (Spento3mem===true)) {msg= {on:true};Spento3mem=false;global.set('Spento3mem',Spento3mem);}\nif ((msg.topic==\"Spento3\") &  (Spento3mem===false)) {msg= {on:false};Spento3mem=true;global.set('Spento3mem',Spento3mem);}\nif ((ciclospegnel===1) & (msg.on===true)) {msg= {topic:\"Salotto\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===2) & (msg.on===true)) {msg= {topic:\"Cucina\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===3) & (msg.on===true)) {msg= {topic:\"Caminetto\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===4) & (msg.on===true)) {msg= {topic:\"Lampadario\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===5) & (msg.on===true)) {msg= {topic:\"Piantane\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===6) & (msg.on===true)) {msg= {topic:\"Pavimento\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===7) & (msg.on===true)) {msg= {topic:\"Finestre\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===8) & (msg.on===true)) {msg= {topic:\"Camera\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===9) & (msg.on===true)) {msg= {topic:\"Quadro\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===10) & (msg.on===true)) {msg= {topic:\"Soppalco\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===11) & (msg.on===true)) {msg= {topic:\"Cameretta\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===12) & (msg.on===true)) {msg= {topic:\"Specchio\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===13) & (msg.on===true)) {msg= {topic:\"Bagno\", on:true, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===14) & (msg.on===true)) {msg= {comando:\"TuttoSP\", on:true, ciclospegneluci:ciclospegnel};return msg;}\n\nif ((ciclospegnel===1) & (msg.on===false)) {msg= {topic:\"Salotto\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===2) & (msg.on===false)) {msg= {topic:\"Cucina\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===3) & (msg.on===false)) {msg= {topic:\"Caminetto\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===4) & (msg.on===false)) {msg= {topic:\"Lampadario\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===5) & (msg.on===false)) {msg= {topic:\"Piantane\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===6) & (msg.on===false)) {msg= {topic:\"Pavimento\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===7) & (msg.on===false)) {msg= {topic:\"Finestre\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===8) & (msg.on===false)) {msg= {topic:\"Camera\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===9) & (msg.on===false)) {msg= {topic:\"Quadro\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===10) & (msg.on===false)) {msg= {topic:\"Soppalco\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===11) & (msg.on===false)) {msg= {topic:\"Cameretta\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===12) & (msg.on===false)) {msg= {topic:\"Specchio\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===13) & (msg.on===false)) {msg= {topic:\"Bagno\", on:false, ciclospegneluci:ciclospegnel};return msg;}\nif ((ciclospegnel===14) & (msg.on===false)) {msg= {comando:\"TuttoSP\", on:false, ciclospegneluci:ciclospegnel};return msg;}","outputs":1,"noerr":0,"x":590,"y":1300,"wires":[["f876ce1a.69e488","7eb8d8ee.022b38","8fde5902.ee963"]]}]
0 Likes

#6

Unfortunately your flows are not importable. Please see this for how to share flows here. You can go back and edit your previous post.

0 Likes

#7

Do you really want to delay messages? Wouldn't it be better to rate limit them instead - if all you want to do is to ensure that you are not overloading the output?

When using the delay node to rate limit, you can see how many outstanding messages are in the q.

[{"id":"2381fa66.51b736","type":"delay","z":"63281c77.40a064","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":410,"y":160,"wires":[["e2ed9d77.d2ea4"]]},{"id":"7ab6a064.30508","type":"inject","z":"63281c77.40a064","name":"","topic":"","payload":"","payloadType":"date","repeat":"0.2","crontab":"","once":false,"onceDelay":0.1,"x":210,"y":160,"wires":[["2381fa66.51b736"]]},{"id":"e2ed9d77.d2ea4","type":"debug","z":"63281c77.40a064","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":580,"y":160,"wires":[]}]

If you are seeing the queue get too large, you should consider throwing away intermediate messages if you can, otherwise you need to revisit your inputs to try to slow them down. Without knowing more about the input(s) to your flow, it is not possible to give more specific advise.

You also need to think about how you are loading your Pi. If, as I suspect, you are running a desktop on the Pi and using the browser on that desktop as well as using the Pi to run Node-RED and who know's what other services, is that realistic? A Pi has a really limited amount of RAM and gets very slow very quickly if it has to use storage to offload RAM - this is known as paging and since the Pi only uses slow SD-cards for storage, paging is very slow indeed.

Turn off the Pi's desktop and use a laptop or other device to load the Node-RED admin ui, dashboard, etc. Then you can look at what is causing any remaining slowness. Is it on the Pi or in the browser?

0 Likes

#8

thank you guys for support, I formatted it, I didn't know, I'll not do the mistake again.
On the PI side CPU level is perfect, very low form 1 to 4% constanly, but from web I'm having some slowness yes.. and usually it is when there are too many messages incoming, i suppose.
NR is running on PI3, laptop I use for web-ui. PI only running NR, ntp update service, log2ram. I've never used this PI with HDMI cable (so all UIs, etc.)

Any chance to have a real log on SSH of all the messages of the flow? Or a counter somewhere on the web ui?
Thank you!

0 Likes

#9

Currently, you need to insert a debug node in the right place and set it to dump to console only which avoids updating the debug panel. You could also add a counter node and display the count only on the debug node:

[{"id":"2381fa66.51b736","type":"delay","z":"63281c77.40a064","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":350,"y":160,"wires":[["e2ed9d77.d2ea4","7950c233.9fbb4c"]]},{"id":"7ab6a064.30508","type":"inject","z":"63281c77.40a064","name":"","topic":"","payload":"","payloadType":"date","repeat":"0.2","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":160,"wires":[["2381fa66.51b736","8bf6028c.b0343"]]},{"id":"e2ed9d77.d2ea4","type":"debug","z":"63281c77.40a064","name":"","active":true,"tosidebar":false,"console":true,"tostatus":false,"complete":"payload","targetType":"msg","x":520,"y":160,"wires":[]},{"id":"8bf6028c.b0343","type":"counter","z":"63281c77.40a064","name":"","init":"0","step":"1","lower":null,"upper":null,"mode":"increment","outputs":2,"x":340,"y":100,"wires":[["e1c2c984.5dfb98"],[]]},{"id":"e1c2c984.5dfb98","type":"debug","z":"63281c77.40a064","name":"","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":510,"y":100,"wires":[]},{"id":"7950c233.9fbb4c","type":"counter","z":"63281c77.40a064","name":"","init":"0","step":"1","lower":null,"upper":null,"mode":"increment","outputs":2,"x":500,"y":200,"wires":[["212271b5.d49cde"],[]]},{"id":"212271b5.d49cde","type":"debug","z":"63281c77.40a064","name":"","active":true,"tosidebar":false,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":650,"y":200,"wires":[]}]
0 Likes

#10

If you enable the metrics option in the logging configuration you'll get a log message for every messaging passing into and out of each node. Just be sure that's what you really want to do.

0 Likes

#11

Had to bold that. I've seen unexpected log inflation do some really terrible things. We're talking over $100k in lost revenue per day bad things.

0 Likes

#12

In production systems, logs must be actively managed to limit size!

Also, production systems should always have independent monitoring of basic platform/device metrics such as CPU, memory, temperature and free storage on each filing system.

0 Likes

#13

As I understtod I have to be careful with that.
I'll disable the periodic inject node and configure it only ad default inject manually, I'll inject a command to every node that has this "loop" and see the results with metrics.
To log on consolle I'll add this as the example in logging section:

    // Custom logger
    myCustomLogger: {
        level: 'debug',
        metrics: true,
        handler: function(settings) {
            return function(msg) {
                console.log(msg.timestamp, msg.event);

So I hope to have the logs on SSH

0 Likes