Help with `delay` node set to `rate limit`

Ok, so new problem.

I have a delay node set to rate limit.

Is there a way that if it has messages queued up (queueing up?) and the first one completes, can I send the node a signal to send the next message?
As in: not peek at it, but actually step to the next message.

Thanks.

I recommend node-red-contrib-queue-gate (node) - Node-RED when doing this sort of thing

I’ll throw my hat in this one :slightly_smiling_face:

May help you also, may not - but just in case

@cymplecy
Thanks!

Forgot I had that.

Not to add more nodes, I'll use it.

Silly me for forgetting I had it.

Does sending a message with no payload, and msg.flush = 1 not release the next message from the delay node?

Sending in msg.flush set to a number will release that many right away. So msg.flush=1 will release the next message

That's a point of contention.

FLUSH.....

So that sends the next message or PEEKS at it?

I seem to remember the latter.

That causes all sorts of trouble.

OK, I was wrong.

Thanks. That maybe nicer to apply.

Here is my standard flow, using a delay node, for handling messages sequentially. Open the comment node for full details of how to use it.

If you want to include a retry feature to retry messages that fail for some reason (for example sending email messages where the send may fail) then you can use @colinl/node-red-guaranteed-delivery.

[{"id":"c8c6291649a49d8f","type":"group","z":"c558f696974f8e89","name":"Sequential Operation","style":{"label":true},"nodes":["b6630ded2db7d680","ed63ee4225312b40","a82c03c3d34f683c","2128a855234c1016","7c6253e5d34769ac","b23cea1074943d4d","d4d479e614e82a49","3a9faf0a95b4a9bb","7eb760e019b512dc","e35f37deeae94860"],"x":34,"y":79,"w":942,"h":282},{"id":"b6630ded2db7d680","type":"inject","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":180,"y":240,"wires":[["ed63ee4225312b40"]]},{"id":"ed63ee4225312b40","type":"delay","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"Queue","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":350,"y":240,"wires":[["d4d479e614e82a49","7eb760e019b512dc"]]},{"id":"a82c03c3d34f683c","type":"delay","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"Some more stuff to do","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":840,"y":240,"wires":[["7c6253e5d34769ac","b23cea1074943d4d"]]},{"id":"2128a855234c1016","type":"link in","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"link in 1","links":["7c6253e5d34769ac"],"x":135,"y":320,"wires":[["3a9faf0a95b4a9bb"]]},{"id":"7c6253e5d34769ac","type":"link out","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"link out 1","mode":"link","links":["2128a855234c1016"],"x":935,"y":320,"wires":[]},{"id":"b23cea1074943d4d","type":"debug","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"OUT","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":710,"y":180,"wires":[]},{"id":"d4d479e614e82a49","type":"debug","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"IN","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":490,"y":180,"wires":[]},{"id":"3a9faf0a95b4a9bb","type":"function","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"Flush","func":"return {flush: 1}","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":230,"y":320,"wires":[["ed63ee4225312b40"]]},{"id":"7eb760e019b512dc","type":"function","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"Some functions to be performed","func":"\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":590,"y":240,"wires":[["a82c03c3d34f683c"]]},{"id":"e35f37deeae94860","type":"comment","z":"c558f696974f8e89","g":"c8c6291649a49d8f","name":"Set the queue timeout to larger than you ever expect the process to take","info":"This is a simple flow which allows a sequence of nodes to be \nprotected so that only one message is allowed in at a time. \nIt uses a Delay node in Rate Limit mode to queue them, but \nreleases them, using the Flush mechanism, as soon as the \nprevious one is complete. Set the timeout in the delay node to \na value greater than the maximum time you expect it ever to take. \nIf for some reason the flow locks up (a message fails to indicate \ncompletion) then the next message will be released after that time.\n\nMake sure that you trap any errors and feed back to the Flush \nnode when you have handled the error. Also make sure only one \nmessage is fed back for each one in, even in the case of errors.","x":310,"y":120,"wires":[]}]

Thanks.

Just completed something just like that.

You may already have this built-in but there should probably be some kind of limiter on the total number of messages queued? In theory, it would be possible to crash Node-RED without any limiter I think?

Yes - setting nodeMessageBufferMaxLength can set a max number of messages to store - default is "infinite" ie no limit. But you can set it if you are worried... of course what to set it to will depend on size of messages being held, rate of arrival, processing, and physical space etc....

The same is true of a simple Delay node in rate limit mode, and queuing intermediate messages. If you send it messages at a rate greater than it can cope with then it will eventually crash node-red.

So as it has been mentioned here a couple of times in the latter posts.....

Where/how do I set a maximum queue size?

You can't set it specifically in the delay node. You can set it for all nodes in settings.js

    /** The maximum number of messages nodes will buffer internally as part of their
     * operation. This applies across a range of nodes that operate on message sequences.
     * defaults to no limit. A value of 0 also means no limit is applied.
     */
    //nodeMessageBufferMaxLength: 0,

If you want to specifically set it in a particular flow with a queue then use @colinl/node-red-guaranteed-delivery which does have a setting for that.

Actually I am not sure which nodes this applies to, whether it is just a set of core nodes or all nodes. Also, I don't know what they do with additional messages when the count is exceeded. The guaranteed delivery node drops the oldest messages and queues new messages once the count is exceeded.

It applies to the switch, delay, sort, batch and split nodes - ie the core ones that can potentially buffer up a load of messages. When exceeded it throws a "too many messages" error and drops any stored messages.