Wait for payload to finish flow before passing another payload

The node node-red-contrib-semaphore conditions the dependency semaphore. This would make a node-RED instance dependent on another dependency. This increases the potential for errors, or the risk of problems with the dependency itself.

Thus, there is one source of error with the node node-red-contrib-semaphore and one with the library semaphore.

Nodes without dependencies reduce the error source to the node itself.

I always try to keep the dependencies as small as possible.

Understood.

@colin- err no... the delay node (in rate limit node) - can use msg.flush:1 to release the next message at any time... so the "delay" time is the safety net / timeout if no feedback is received... If the process completes successfully it just needs a change node to send back that message to release the next one - so it will run as fast as they are processed. - and if the process fails for some reason then the timeout will indeed cut in and release the next one anyway. (Whereupon you would hope it all speeds up again...)

not sure why ? the first message to arrive is "just processed" right away... no need for a delay... only subsequent messages are backed up - either waiting to be released (on completion of the first message) or the timeout (if it fails)

I can't get that to work as I expected. The flush does release the next message, but then the message after that is released as if the flush had not been sent, rather than restarting the delay from the flush time, if that makes sense. Also the flush adds itself into the queue, which is not desirable.

To see this, click the inject four times and after 3 seconds click the flush and note the times at which the messages are sent.

[{"id":"f6ba326c51aa4d42","type":"delay","z":"bdd7be38.d3b55","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"5","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":420,"y":2300,"wires":[["7b1947d634ba9556"]]},{"id":"3687bb40d026d527","type":"inject","z":"bdd7be38.d3b55","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":2300,"wires":[["f6ba326c51aa4d42"]]},{"id":"f5a6178b5cdad459","type":"inject","z":"bdd7be38.d3b55","name":"Flush","props":[{"p":"flush","v":"1","vt":"num"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"flush","payloadType":"str","x":110,"y":2460,"wires":[["f6ba326c51aa4d42"]]},{"id":"7b1947d634ba9556","type":"debug","z":"bdd7be38.d3b55","name":"debug 6","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":600,"y":2300,"wires":[]}]

err you have added a payload as well as the flush so yes that does get added to the queue...


Remove that so you only have the msg.flush if you just want the flush.

Yes, that does prevent the flush message being added to the queue (that is not documented as far as I can see). It doesn't fix the timing problem. With a 1 message per 5 seconds config, if the inject node is clicked four times at time 0, then the flush is clicked at time 3 the messages are released at time 0, 3, 5, 10, 15, instead of 0, 3, 8, 13, 18. Perhaps that is a bug.

if the timeout is set much longer than the expected max time (ie only as an emergency backstop) then I suspect it will be of minimal consequence as the timer is cleared whenever the queue does completely empty. But yes also reseting the timer could be an improvement (rather than a bug :slight_smile:

OK, yes it should work in that case. However there does appear to be another issue. If you set the delay node to one message every minute and click the inject node twice then the first gets sent immediately, as it should, and the second gets queued. Click the flush inject to indicate that the first has been actioned, and the second message gets sent, also as it should. However, if you then click the first inject again that message gets passed on immediately, instead of being queued waiting for timeout or flush.

that is correct - working as designed - as I mentioned above the timer gets cleared when the queue is empty.

So in fact it can't be used in place of semaphore nodes, using the flush to release the next message, even with the timeout just being used as an emergency backstop, as in the case I described the third message gets released while the second is still being handled.

err no - there is no spurious inject in his case - the flush is only performed by the completion of the previous message - please refer to my original example.

What am I doing wrong here then? The Debug output should always show a message from the IN node, then a message from the OUT node 5 seconds later.
Click the timestamp node and one sees IN then OUT as expected.
Click the timestamp node twice and one sees IN, OUT, IN, OUT, also as expected.
However, if one clicks the timestamp node twice, waits till the first OUT is seen, then clicks it again then the sequence in the debug, which should be IN, OUT, IN, OUT, IN, OUT is instead, IN, OUT, IN, IN, OUT, OUT.

[{"id":"b6630ded2db7d680","type":"inject","z":"bdd7be38.d3b55","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":2620,"wires":[["ed63ee4225312b40"]]},{"id":"ed63ee4225312b40","type":"delay","z":"bdd7be38.d3b55","name":"Queue","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":310,"y":2620,"wires":[["a82c03c3d34f683c","d4d479e614e82a49"]]},{"id":"a82c03c3d34f683c","type":"delay","z":"bdd7be38.d3b55","name":"Process taking 5 seconds","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":530,"y":2620,"wires":[["7c6253e5d34769ac","b23cea1074943d4d"]]},{"id":"2128a855234c1016","type":"link in","z":"bdd7be38.d3b55","name":"link in 1","links":["7c6253e5d34769ac"],"x":95,"y":2700,"wires":[["1db3d896b21fee1b"]]},{"id":"7c6253e5d34769ac","type":"link out","z":"bdd7be38.d3b55","name":"link out 1","mode":"link","links":["2128a855234c1016"],"x":665,"y":2700,"wires":[]},{"id":"1db3d896b21fee1b","type":"change","z":"bdd7be38.d3b55","name":"Flush","rules":[{"t":"set","p":"flush","pt":"msg","to":"1","tot":"num"},{"t":"delete","p":"payload","pt":"msg"},{"t":"delete","p":"topic","pt":"msg"},{"t":"delete","p":"_event","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":190,"y":2700,"wires":[["ed63ee4225312b40"]]},{"id":"b23cea1074943d4d","type":"debug","z":"bdd7be38.d3b55","name":"OUT","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":670,"y":2540,"wires":[]},{"id":"d4d479e614e82a49","type":"debug","z":"bdd7be38.d3b55","name":"IN","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":470,"y":2540,"wires":[]}]

@dceejay
I did't know the function with the flush yet. Good to know that this function exists. That makes my idea with the release function in the msg object unnecessary. Thanks for this hint.

Using the flush method automatically creates a path back to the delay node. This is equivalent to the loop nodes pointed out. The advantage of the loop nodes, however, is that they are specifically designed for loop processing and should in all conditions ensure that a message is fully processed before the next one is processed. I do not see this protection with the Delay Node at the moment.

If the node is changed as it was discussed with @Colin, the delay node could completely replace the loop nodes. This would give Node-RED a solid standard for loops, which does not need any other dependencies and is maintained for a long time.

Recommendations for such standard tasks should be listed in the official Node-RED cookbook. This way, users can fall back on official implementation recommendations and do not always have to try to develop a solution themselves. In addition, this implementation would be stable and extensively tested.

The range of functions of Node-RED and its nodes is now so large that normal users can no longer know all the functions and their possibilities.

Perhaps @knolleary and his core team can add appropriate standard procedures to the cookbook. I would be very happy about this.

There isn't a core team. Node-RED is an open community of contributors. If anyone wants to contribute something to the cookbook, then they are most welcome. Sadly it has never really attracted much interest from the community to help improve - despite the value it would bring to everyone.

I am aware that Node-RED is an open community of contributors. That's why I like Node-RED very much. But from the entry "Contributing to Node-RED" of the Node-RED homepage committers are named. These are for me members with extended rights, which I consider as core team. A contributor can make a maximum of one proposal in this case. The committers have to take it over.

Furthermore I assume that the committers have the best insight into Node-RED and its functionality. So these users should have the best insight about the functionality of the standard nodes and their intended use. This could lead to recommendations on how Node-RED should be officially used to achieve the best result.

The inclusion of use cases from the community would complement these official recommendations. But I realize that suggestions for the cookbook, which are discussed about the community and released by the committers, can also be considered as official recommendations :wink:

PR raised here - Fix delay rate limit last timing when empty by dceejay · Pull Request #3709 · node-red/node-red · GitHub

1 Like

Maybe a better place for demos would be the built in examples...

1 Like

Good thought - will do if I get 5 mins this pm.

2 Likes