Rate limiting based on exec node completion

General
1

In my NR -> Alexa project, I want to add rate limiting type behaviour so that messages are queued and only processed if the previous one has actually been spoken out.
image
i.e only want messages passed into pico2wave if last one has been spoken by the exec node

The exec node returns {code :0} when speaking is finished so that seems a good candidate to check but before I re-invent the wheel and bodge up something involving flow variables, is there a nice elegant way of achieving this?

As always, preferred method are built-in nodes but willing to use a nice contrib one - NO JS :slight_smile:

2

The latest version of the delay node will accept a msg.flush=1 property to release the next message instantly - so you can set the rate to some long timeout (ie a failsafe value) and then trigger an earlier release when you want.

2 Likes
3

What do you want to do with the next message(s) if they arrive too early?

4

queue them in some manner

5

That's nice to know :slight_smile:

6

OK - I've come up with this

image
image757Ă—205 76.2 KB

Complete node monitors exec node output and sends a msg.flush=1 into rate limiter.

I needed the switch on the output so that the flush message itself (which has empty payload) doesn't get fed on.

7

[off-main-topic]

The complete node outputs whatever node its monitoring sends to it

Now, I'm just wanting to use it as a trigger to just send a message with msg.flush set and nothing else

But I couldn't come up with a generic way of deleting all existing msg properties and then just adding my own

Is there a simple core node method of doing this?

I simply delete msg.rc in this case but I'd prefer to not worry about what the complete node outputs so I can use technique generically in future

8

Have a look at this flow. It queues messages until it is told they have been actioned or (optionally) there was an error, in which case it retries after a specified time. You may not need the retry on error feature, but you don't need to use that. The example is sending emails to a server, but I think the principle is the same.
https://flows.nodered.org/flow/05e6d61f14ef6af763ec4cfd1049ab61

9

( and likewise the latest version of delay node also accepts msg.toFront which adds a message to the front of the queue - so in case of an error - you can push that message to the front to retry before continuing the rest of the queue... (and if combined with flush 1 - then it will retry immediately...))

10

I found it very difficult to eliminate all possible race conditions trying to do this with discrete nodes, which is why I eventually put it all in one function node (which is not complex) in a subflow.
If anyone wants to propose an alternative that copes with race conditions then I will gladly have a look at it.

11

How have you configured the Rate node?

12

Bog standard default

Currently added a delay node after complete to get a gap between spoken phrases but its not a perfect solution as sometimes the 15 sec default delay kicks in and sometimes I get 2 messages going through on top of each other and at other times I get a backlog!

image
image955Ă—137 36.4 KB

I'll look at it again tomorrow :slight_smile:

13

Have you had a look at my flow?

14

I've modified it to use exec node to test it and it basically works but not tried it out for real

Will give it a go in the morning :slight_smile:

15

Make sure you catch all possible routes out, if there is, for example, an error condition that means that the OK message never gets fed back then it will hang.

16

I would probably just use a wired connection from the exec node 3rd output back to the 2sec delay as there should always be a return code whether the command passes or fails

1 Like
17

I think I don't need to worry about error for this application - the exec node code output should always emit something

This mod (with the simple feedback loop as per Dave) of your flow works fine :slight_smile: (I always thought it would BTW but I'm trying to avoid function nodes)

image

@dceejay I think the simple delay node approach is not quite good enough for this so I'm going to look into using a@drmibell q-gate node and see how that approach pans out

18

I’d like to understand why not.

19

So would I :slight_smile:

The basic issue is that I'm trying to bend the rate limiter into being a form of turnstile and I haven't come up with the right logic to do that

One of the issues I think, is the fact that sending a msg.flush=1 is added to the queue and not just treated as a special control only message.

Now I'm currently filtering that out on the output of the delay node but they can easily end up "clogging up" the queue. They eventually get released every 15 seconds but while they are in there - they stop real incoming messages from being dealt with

JFI I've tried playing around with the all singing/all dancing q-gate node and even with that, I've not managed to come up with a usable flow.

I'm coming around to thinking that this does require something like Colin's state machine approach

20

The "problem" is that yes the msg.flush is added to the properties of whatever msg you have - if that is the ONLY property then the delay node will not send it on... so you need to delete the existing msg.payload and maybe msg.topic and any other properties using your change node. (Or use a function node to create a "clean" msg with only msg.flush=1 .

eg

[{"id":"31721b1737c1207d","type":"delay","z":"257090c402849288","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"20","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":430,"y":520,"wires":[["055e2529428f571a"]]},{"id":"055e2529428f571a","type":"exec","z":"257090c402849288","command":"sleep","addpay":"","append":"5","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":630,"y":520,"wires":[["dde23da0b5477d33"],[],["d5e88b9341d02686"]]},{"id":"dde23da0b5477d33","type":"debug","z":"257090c402849288","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":810,"y":500,"wires":[]},{"id":"d5e88b9341d02686","type":"change","z":"257090c402849288","name":"","rules":[{"t":"set","p":"flush","pt":"msg","to":"1","tot":"num"},{"t":"delete","p":"payload","pt":"msg"},{"t":"delete","p":"topic","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":620,"wires":[["31721b1737c1207d","481eafc6e4f48447"]]},{"id":"481eafc6e4f48447","type":"debug","z":"257090c402849288","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":670,"y":620,"wires":[]},{"id":"7e0141d6ac754791","type":"inject","z":"257090c402849288","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":180,"y":520,"wires":[["31721b1737c1207d"]]}]