Understanding ratelimit per topic

I am confused by the delay / rate limit node, not sure it can do what I want.

Here is what I wish to:

  • sending message with different topics (say topic1, topic2).
  • rate limit for each topic (say 1 msg per minute).

just like if I had separated path for each topic with a simple rate limit.

if I send 10xtopic1 in a minute, only first message will go through, next are dropped.
if I send 10xtopic2 in a minute, only first message will go through, next are dropped.
if I send a mix of several message of topic1 & topic2 in a minute, only first message of each topic will go through, other are dropped.

I set the rate limit node as this:

Action:

  • Rate limit
  • For each msg topic
    Rate : 1msg per 1 minute
    uncheck "allow ..."
    Drop intermediate messages
    Send each topic in turn

It seems to more or less work (as far as I can see), but some message are quite delayed.
it may take a few seconds, or even a minute in some cases to go through the rate limit node.
This is not critical, but it make me wonder if I catch correctly how this node is working.

Did I missed something?

When in handling multiple topics mode - at deploy time a global timer is started that "ticks" at the required rate (eg once per minute) - so the first message can be delayed up to one minute from start... Also then when it ticks it can either send "each message in turn" (ie only one message from the next topic in the list of known topics) - or all at once - i.e. the latest message from all topics.

So if you had 5 active topics - in the first mode (you selected above) you may only get topic A once every 5 minutes - ie A, (delay), B, {delay), C, (delay), D, (delay), E, (delay), A ...

in the second mode (all at once) you would get A,B,C,D,E, (delay), A,B,C,D,E, (delay)...

2 Likes

Thank you for the feedback!

I think I might better use the "send all topics" option, but even so, it might not be exactly what I want.

In this use case, I want a rate limit to avoid spamming myself.

For example:

  • I ping some servers to monitor their availability every 20 seconds.
  • If the ping is not OK (no response), I send a Telegram message: "Server x is not responding to ping."
  • But I don't want to get a message every 20 seconds, so I use a rate limit of (for example): 5 messages per 12 hours. This way, if a server is down, it will send a burst of 5 alerts over a few minutes and then stay quiet for almost 12 hours.

This was working fine (one rate limit per server, no topic sorting)

To simplify my Node-RED flows a bit, instead of using a full chain of ping/processing/Telegram message for each device I want to monitor, I thought I could use some common processing and a rate limit per topic to handle this at the end.

But if I have to wait up to 12 hours to get the alert, it's a bit annoying. :smiling_face_with_tear:

Do you need the 24 hour reminder? Would a message each time that it goes from good to bad be ok? Perhaps a message each time it changes from bad to good would also be useful.

1 Like

Why not use the filter node per topic, that way you get a message only when the state changes. i.e server on line/server off line

2 Likes

well, currently, it just send alert message, no recovery messages, no there nothing to filter :slight_smile:
I like to have a few alerte message when problem happens, but also some reminder next days, without spamming myself too much.

I use telegram to get the alert message, it's a trade-off between :

  • not missing a message in the flow: I have other messages and commands in the same telegram chat, so I could miss a single alert message.
  • but also not warning too often. Sometime I know something is wrong, but don't have time to repair for some days.

But your posts are good ideas, I can try different things :slight_smile:
I also have solution to make it work as I want (one data rate limit per messages) : the point was mostly to understand better how the rate limit works...pushing my knowledge bundaries a bit more and play around with nodered :smiley: