Limit message number per Hour

General
1

I want to limit the messages between two nodes to 30 messages per hour. I know about rate limit of the delay node but the problem is that instead of passing the first 30 messages as soon as they arrive, the delay node will either drop it or queue it until the time of 1 hour divided by 30 comes.

I want a system which can pass on the first 30 messages it receives in a hour without any delay and then drop rest to the messages for that hour. I need it to limit the number api requests being made to a server or else it will over load server.

2

Have a look at the node-red-contrib-counter node.
Think this should work;
Set the upper limit in the counter node to 30.
Check in a split node if the `countUpperLimitReached' is reached, then route your message to nothing (output 2).
One's a hour send a msg.reset to the counter node to reset and starts counting again.

@cymplecy, nice, same idea at the same time :wink:

2 Likes
3

I think you could do this by adding a counter before a simple switch and just routing the outputs depending on the count

And then add a repeating inject node to reset the counter every hour

Not tried but something like this

image

Sent at same time as @edje11 idea

1 Like
4

I would use standard nodes and save a count and time to context, limiting the count with a switch node. After 1 hour reset the time and count
e.g.

[{"id":"18bb0729.610489","type":"inject","z":"b779de97.b1b46","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":60,"wires":[["5581dee7.a1e68"]]},{"id":"5581dee7.a1e68","type":"change","z":"b779de97.b1b46","name":"limit time 3600000 ms","rules":[{"t":"set","p":"m_count","pt":"flow","to":"$flowContext(\"m_count.time\") < ($millis() - 3600000) or \t$exists($flowContext(\"m_count.count\")) = false ?\t{\t\"count\": 1, \t\"time\": $millis()\t} :\t{\t\"count\": $flowContext(\"m_count.count\") + 1,\t\"time\": $flowContext(\"m_count.time\")\t}\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":360,"y":60,"wires":[["63fe332b.659f2c"]]},{"id":"63fe332b.659f2c","type":"switch","z":"b779de97.b1b46","name":"switch limit 30","property":"m_count.count","propertyType":"flow","rules":[{"t":"lte","v":"30","vt":"num"}],"checkall":"false","repair":false,"outputs":1,"x":610,"y":60,"wires":[["56af3108.1d2cf"]]},{"id":"56af3108.1d2cf","type":"debug","z":"b779de97.b1b46","name":"","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"$flowContext(\"m_count.count\") & \" -- \" & $flowContext(\"m_count.time\")","statusType":"jsonata","x":810,"y":60,"wires":[]}]

This way the timer resets its self when a message comes in over an hour later.

1 Like
5

If I understand you, this is exactly what the delay node can do:

image
image1002Ă—850 34.6 KB

6

That seems to only send the most recently received mesage every 2 minutes, not the first 30 messages in the hour, as soon as they arrive.

7

I have the feeling your problem is then before this node. Can you share the flow? Or investigate with a debug node before and after the delay node.l

8

The help text for the delay node in rate limit mode says
"When configured to rate limit messages, their delivery is spread across the configured time period."
That means that if it is configured for 30 per hour, and to queue messages, and 30 come in right at the start then it will not send out all 30 immediately (as the OP requires), but will send them one at a time every 2 minutes. Additionally any more that arrive in that hour will be queued for sending later, whereas it is required that additional ones are dropped. If the node is configured to Drop Intermediate messages, then when all thirty arrive at the start then 29 of them will be discarded.

9

Is there any simple single node that can do this? I think this is a very basic need.

10

It is a delicate line knowing how to do things.

From an outside point of view, I would go with what has been suggested.

Get/install the count node, add a switch node after it that only passes messages who's msg.count < 30.

Every hour a signal is sent to the count node to reset it and it starts again at 0.

If only for proof of concept and to get it working.

You can - later on - get tricky with context stuff and integrate it all into a function node.
But that needs a good knowledge of java script. If you are new, that is a big step.

Again: If only for proof of concept, try what has been suggested and see how it all fits together.

For the sake of it here is a basic flow doing just that.

[{"id":"585dc602eed1b4cd","type":"inject","z":"8bb4de19.f72c88","name":"Your incomming messages","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":1210,"y":3840,"wires":[["d3bf8075a6187089"]]},{"id":"d3bf8075a6187089","type":"counter","z":"8bb4de19.f72c88","name":"","init":"0","step":"1","lower":null,"upper":null,"mode":"increment","outputs":"1","x":1430,"y":3840,"wires":[["676a2446e572a3b8"]]},{"id":"676a2446e572a3b8","type":"switch","z":"8bb4de19.f72c88","name":"Pass the first 30","property":"count","propertyType":"msg","rules":[{"t":"lte","v":"30","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":1610,"y":3840,"wires":[["c4ebf57e24ea1a90"]]},{"id":"faf0538fdebc0a69","type":"inject","z":"8bb4de19.f72c88","name":"Hourly signal","props":[{"p":"reset","v":"true","vt":"bool"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":1180,"y":3890,"wires":[["d3bf8075a6187089"]]},{"id":"c4ebf57e24ea1a90","type":"debug","z":"8bb4de19.f72c88","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1600,"y":3900,"wires":[]}]
1 Like
11

No. If there were then someone would have suggested it by now.

12

Well...... There is, but it isn't that simple.

It is a function node and you will have to write the code in it.

So before you get to that, you need to make it happen with discreet nodes and make sure it does what you want.

(This is just like deja vous all over again)

13

I already have a function for the same purpose but its a bit complicated and when i want to change the codes, its difficult, Thats why I am looking for a easier and elegant route.

14

Well, as @Colin said.... you have been shown how to do it in an easy way.
And the example I posted.

15

The problem is that in a single flow, I have 12 spaces where I need this function, which would be making the flow complicated with that solution. Anyways thanks for the help.

16

Ok, the answer is still NO.

You just make a subflow from it.

Then all you see is one node in the flow.

Study SUBFLOWS and you may be happy.

Believe me: you will kick yourself postponing learning about them.

I was in your position a LONG time ago and wasn't bothered to learn about them.

One day I did and I haven't looked back.

Quick "how to...."

Get the nodes that do what you want. Note: Only 1 input allowed.
Drag select them and make them into a sub-flow.

Done.

Now just put that node in and things will magically happen.

17

Why not set it in a subflow then you can have configurable inputs to, as in this example set to 5 messages in 10 seconds

[{"id":"43d574be.4067ac","type":"subflow","name":"limit time count","info":"","category":"","in":[{"x":160,"y":140,"wires":[{"id":"737efd96.8c5ce4"}]}],"out":[{"x":460,"y":160,"wires":[{"id":"737efd96.8c5ce4","port":0}]}],"env":[{"name":"time","type":"num","value":"3600000","ui":{"type":"input","opts":{"types":["num"]}}},{"name":"limit","type":"num","value":"10","ui":{"label":{"en-US":"limit"},"type":"input","opts":{"types":["num"]}}},{"name":"name","type":"str","value":"limit1","ui":{"label":{"en-US":"name"},"type":"input","opts":{"types":["str"]}}}],"color":"#DDAA99"},{"id":"737efd96.8c5ce4","type":"function","z":"43d574be.4067ac","name":"","func":"const time = env.get(\"time\");\nconst limit = env.get(\"limit\");\nconst name = env.get(\"name\");\nconst millis = new Date().valueOf()\nlet stored = flow.get(`limit_count.${name}`) || {\"time\": 0, \"count\": 0};\n\nif(stored.time  <  millis - time){\n    stored = {\"time\": millis, \"count\": 1};\n}else{\n    stored = {\"time\": stored.time, \"count\": stored.count+1};\n}\nflow.set(`limit_count.${name}`, stored)\n\nif (stored.count > limit) msg = null;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":310,"y":140,"wires":[[]]},{"id":"65cb04ca.eaf084","type":"inject","z":"c791cbc0.84f648","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":20,"wires":[["eca4e433.85c7d"]]},{"id":"eca4e433.85c7d","type":"subflow:43d574be.4067ac","z":"c791cbc0.84f648","name":"","env":[{"name":"time","value":"10000","type":"num"},{"name":"limit","value":"5","type":"num"},{"name":"name","value":"limit2","type":"str"}],"x":400,"y":20,"wires":[["893d956f.72bd28"]]},{"id":"893d956f.72bd28","type":"debug","z":"c791cbc0.84f648","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":610,"y":120,"wires":[]},{"id":"a1e68c60.aa8788","type":"subflow:43d574be.4067ac","z":"c791cbc0.84f648","name":"","env":[{"name":"time","value":"5000","type":"num"},{"name":"limit","value":"2","type":"num"}],"x":390,"y":80,"wires":[["893d956f.72bd28"]]},{"id":"2a0a9c4f.8cfb6c","type":"inject","z":"c791cbc0.84f648","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":80,"wires":[["a1e68c60.aa8788"]]}]

[edit] I see Trying_to_learn suggested the same.
[edit] updated to store in seperate properties, so the individual use of subflows will write to there own name property. If you use the same name it will count the messages from both flows as one count.

18

We can look at adding an option for this to the Delay node. I'll put something on the backlog for it.

1 Like
19

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.