Design pattern for restarting a message loop (which should contain only one message at the time) with multiple dynamic delays

[rko]

We are struggling with this for quite a while now. I found a cool way here by @dceejay (see first example flow below). However, because we are using link in and link out nodes, the messages arrive in wrong order at the delay node and the loop does not restart. And I am not sure about the switch node either (if it’s deterministic). But without link nodes it worked well.

[{"id":"13bbb7508b3bb572","type":"switch","z":"3dfe0c63f0cf19d7","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"reset","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":440,"y":300,"wires":[["b987a4a6e324e26a"],["f5fc3d878dd4c32f"]]},{"id":"d49fa8ce1f5bca4f","type":"function","z":"3dfe0c63f0cf19d7","name":"reset","func":"msg.topic = \"should come after reset\";\nreturn [[{reset: true, topic: \"reset\"},msg]];","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":320,"y":300,"wires":[["13bbb7508b3bb572"]]},{"id":"49fdbec74d480310","type":"inject","z":"3dfe0c63f0cf19d7","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":200,"y":300,"wires":[["d49fa8ce1f5bca4f"]]},{"id":"b987a4a6e324e26a","type":"link out","z":"3dfe0c63f0cf19d7","name":"reset timer","links":["c2038469c0ba3306"],"x":555,"y":250,"wires":[]},{"id":"c2038469c0ba3306","type":"link in","z":"3dfe0c63f0cf19d7","name":"reset timer","links":["b987a4a6e324e26a"],"x":675,"y":250,"wires":[["987e20e6877a3177","c7af6cf407200e12"]]},{"id":"987e20e6877a3177","type":"delay","z":"3dfe0c63f0cf19d7","name":"","pauseType":"delayv","timeout":"2.5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":870,"y":350,"wires":[["12910a1494edce46"]]},{"id":"f5fc3d878dd4c32f","type":"change","z":"3dfe0c63f0cf19d7","name":"","rules":[{"t":"set","p":"delay","pt":"msg","to":"2500","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":610,"y":350,"wires":[["987e20e6877a3177","c7af6cf407200e12"]]},{"id":"c7af6cf407200e12","type":"debug","z":"3dfe0c63f0cf19d7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":860,"y":250,"wires":[]},{"id":"12910a1494edce46","type":"function","z":"3dfe0c63f0cf19d7","name":"do something","func":"return msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1050,"y":350,"wires":[["987e20e6877a3177"]]}]

Our current solution (see second example flow below): We just use an extra delay node to make sure, the reset command arrives first (before the restart message). I am sure there's a more elegant and shorter way. The reset procedure gets quite complicated because we do have three delay nodes in our loop (with dynamic delay time).

[{"id":"d98df5cb5546faf3","type":"switch","z":"3dfe0c63f0cf19d7","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"reset","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":360,"y":640,"wires":[["f263ba15d8efa8bb"],["122585f51bb77468"]]},{"id":"2c208884d3b86217","type":"function","z":"3dfe0c63f0cf19d7","name":"reset","func":"msg.topic = \"should come after reset\";\nreturn [[{payload: \"blub\", topic: \"reset\"},msg]];","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":240,"y":640,"wires":[["d98df5cb5546faf3"]]},{"id":"b1b58aecb8d9920a","type":"inject","z":"3dfe0c63f0cf19d7","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":120,"y":640,"wires":[["2c208884d3b86217"]]},{"id":"f263ba15d8efa8bb","type":"link out","z":"3dfe0c63f0cf19d7","name":"reset timer","links":["9daedb40f8e87c45"],"x":465,"y":590,"wires":[]},{"id":"9daedb40f8e87c45","type":"link in","z":"3dfe0c63f0cf19d7","name":"reset timer","links":["f263ba15d8efa8bb"],"x":615,"y":590,"wires":[["1688e141bdbbc996"]]},{"id":"5a60a576632f60e8","type":"delay","z":"3dfe0c63f0cf19d7","name":"","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":990,"y":640,"wires":[["607f32fae5f53c3f","d43a8864b16c4803"]]},{"id":"1688e141bdbbc996","type":"change","z":"3dfe0c63f0cf19d7","name":"","rules":[{"t":"set","p":"reset","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":750,"y":590,"wires":[["5a60a576632f60e8","607f32fae5f53c3f"]]},{"id":"122585f51bb77468","type":"delay","z":"3dfe0c63f0cf19d7","name":"delay restart (15)","pauseType":"delay","timeout":"15","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":540,"y":690,"wires":[["95eb9c2c66e8b992"]]},{"id":"95eb9c2c66e8b992","type":"change","z":"3dfe0c63f0cf19d7","name":"","rules":[{"t":"set","p":"delay","pt":"msg","to":"2500","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":750,"y":690,"wires":[["5a60a576632f60e8"]]},{"id":"607f32fae5f53c3f","type":"debug","z":"3dfe0c63f0cf19d7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1160,"y":590,"wires":[]},{"id":"d43a8864b16c4803","type":"function","z":"3dfe0c63f0cf19d7","name":"do something","func":"return msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1190,"y":640,"wires":[["5a60a576632f60e8"]]}]

We even got rid of the loop and used a node called node-red-contrib-controltimer. However, that node does not allow dynamic delays (yet). So back to square one.

We would be happy if someone could guide us in the right direction. Feel free to ask questions if something is unclear.

[Steve-Mcl]

What is the goal?

Can't you simply use an inject set to repeat? Or cron-plus node with multiple schedules?

I mostly advise against loops (you can easily tie up the nodejs thread and cause a crash or other issues)

[rko]

Thanks Steve. We are using the mentioned loop as a measuring/heating cycle in an industrial application. The main Interval is typically about 10 seconds, and the shortest delay is about 25 ms.

Is it possible to have dynamic interval time in inject nodes?

I am okay if it’s not a loop. But the delays must be dynamic (set by user through the dashboard) and need to be reset after changes.

I will have a look at the cron-plus node, thanks for that.

[Colin]

That doesn't really explain what you are trying to do, not to me at least.

[rko]

Sorry, it seems that I have difficulties to explain what we are trying to do. I understand that the explanation in the initial post is rather abstract but I do believe that the use case doesn't matter. Maybe you can look into the isolated problem without knowing much about the real application anyway?

[Colin]

No, I don't understand what you are trying to do. You talk about a variable delay, what is it that you are delaying? Or do you mean that you are trying to trigger a flow with a variable interval between triggers? Also you say you need to reset after changes, what does that mean? What is it that you are resetting?

[rko]

Variable delay: it's a delay node which uses msg.delay as the delay time.
The reset/restart of a delay node is needed because the user does not want to wait for the next cycle.
Because the flow is not deterministic it's not stable and sometimes, a restart fails or there is more than one message in the loop (which we don't want).

Did you have a look at the sample flows I posted?

[Colin]

That is talking mostly about the implementation, not the requirement, I am trying to understand the requirement.

1. There is an action to be performed repeatedly, with a delay between repetitions.
2. The delay is variable, controlled by input via the dashboard. The delay time starts when the previous action is completed.
3. There must be the ability to restart the action immediately, without waiting for any currently running action or the following delay to complete.
4. Does it matter if the operator asks for a restart while the previous action is ongoing? If so what is to be done about it?
5. Is it guaranteed that a running action will always complete?

[rko]

Thanks @Colin. Please find my answers below.

@Colin

That is talking mostly about the implementation, not the requirement, I am trying to understand the requirement.

1. There is an action to be performed repeatedly, with a delay between repetitions. Yes, there are multiple actions (or sub steps) with delays defined by the user

2. The delay is variable, controlled by input via the dashboard. The delay time starts when the previous action is completed. Correct.

3. There must be the ability to restart the action immediately, without waiting for any currently running action or the following delay to complete. Yes (action = loop), we're resetting all delays (sub steps) because at the moment of reset we do not know where the message is.

4. Does it matter if the operator asks for a restart while the previous action is ongoing? If so what is to be done about it? It does not matter when a reset is triggered by the user.

5. Is it guaranteed that a running action will always complete? Not sure if I understand this question completely. But it is not relevant whether a sub step is being aborted or not (see answer four).

[rko]

Please see a screenshot of the mentioned loop.

The marked link out nodes feed the reset messages into the loop. There are currently two messages trapped in the loop (there should only be one message in the loop at a time). This happens occasionally after a re-deploy. Initially, I did not want to share this loop because it won't work without the connected hardware (if somebody wants to test). That’s why I created the example flows in the initial post. Hopefully the screenshot helps to understand what I am trying to achieve.

Again, my goal is to come up with a set of nodes which allow a robust reset and restart of the loop.

[PdeJ]

I hope this approach might be useful to you.

[{"id":"56172419.a3b51c","type":"delay","z":"533d10a2.042f3","name":"","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":600,"y":1400,"wires":[["7baf9165.62f6d"]]},{"id":"54eb0858.d75938","type":"function","z":"533d10a2.042f3","name":"do something","func":"\nmsg.payload=msg.payload+1;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":940,"y":1400,"wires":[["56172419.a3b51c","2e36bbaf.787074","974fe026.15ef2"]]},{"id":"2e36bbaf.787074","type":"debug","z":"533d10a2.042f3","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":770,"y":1340,"wires":[]},{"id":"90ba11d.518d8f","type":"change","z":"533d10a2.042f3","name":"","rules":[{"t":"set","p":"reset","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":1300,"wires":[["56172419.a3b51c"]]},{"id":"65738e84.bc021","type":"change","z":"533d10a2.042f3","name":"","rules":[{"t":"set","p":"delay","pt":"msg","to":"interval","tot":"flow"},{"t":"set","p":"payload","pt":"msg","to":"d","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":240,"y":1300,"wires":[["fd676b9c.d905e8","56172419.a3b51c"]]},{"id":"465b6808.0514a8","type":"status","z":"533d10a2.042f3","name":"status of variable delay node","scope":["56172419.a3b51c"],"x":220,"y":1220,"wires":[["f1cec912.f8e938"]]},{"id":"f1cec912.f8e938","type":"switch","z":"533d10a2.042f3","name":"","property":"status.text","propertyType":"msg","rules":[{"t":"eq","v":"reset","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":210,"y":1260,"wires":[["65738e84.bc021"]]},{"id":"fd676b9c.d905e8","type":"debug","z":"533d10a2.042f3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":230,"y":1340,"wires":[]},{"id":"7baf9165.62f6d","type":"change","z":"533d10a2.042f3","name":"","rules":[{"t":"set","p":"d","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":780,"y":1400,"wires":[["54eb0858.d75938"]]},{"id":"fb755e8b.022bc","type":"inject","z":"533d10a2.042f3","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"0","payloadType":"num","x":390,"y":1420,"wires":[["56172419.a3b51c"]]},{"id":"e3c9c405.2691c8","type":"ui_numeric","z":"533d10a2.042f3","name":"Interval","label":"Refresh Interval","tooltip":"","group":"83f69684.32da88","order":1,"width":0,"height":0,"wrap":false,"passthru":true,"topic":"topic","topicType":"msg","format":"{{value}}","min":"1","max":10,"step":1,"x":540,"y":1220,"wires":[["2593cb7a.8b5ce4"]]},{"id":"2593cb7a.8b5ce4","type":"change","z":"533d10a2.042f3","name":"","rules":[{"t":"set","p":"interval","pt":"flow","to":"1000*payload","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":1260,"wires":[["90ba11d.518d8f"]]},{"id":"974fe026.15ef2","type":"ui_text","z":"533d10a2.042f3","group":"83f69684.32da88","order":1,"width":0,"height":0,"name":"","label":"I'm counting","format":"{{msg.payload}}","layout":"row-spread","x":970,"y":1320,"wires":[]},{"id":"83f69684.32da88","type":"ui_group","name":"Default","tab":"940e1897.3ad938","order":1,"disp":true,"width":"6","collapse":false},{"id":"940e1897.3ad938","type":"ui_tab","name":"Test","icon":"dashboard","disabled":false,"hidden":false}]

The example should start automatically on Deploying. When the delay interval is changed in the dashboard Numeric node the flow.interval is updated and a msg.reset is sent to the Delay node. The resulting change in status of the node is detected using the Status node and the updated flow,interval is then sent as the msg.delay.

[rko]

To use the status node is exactly the approach I wanted to try next, after posting a feature request for the delay node over here.

Thank you for the example as well. It does not properly work with my Node-RED version because the status messages are different. But I will definitely play with this approach and report back.

[PdeJ]

Guess you need update.

[rko]

Hmm, I am running 2.0.5

[PdeJ]

I'm using Node Red 1.2.6. I'll try it on another RPiI running Node Red 2.0.3

[PdeJ]

Oh Dear doesn't work on v2.0.3......very strange.

[rko]

It's no big deal. Your flow helps anyway!

That's a reset (ring):

[rko]

This one should work. The question remains: Is it a robust flow. I will do some intense tests and will give feedback.

[{"id":"56172419.a3b51c","type":"delay","z":"0bb4e6d7fce94c00","name":"","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":880,"y":1900,"wires":[["7baf9165.62f6d"]]},{"id":"54eb0858.d75938","type":"function","z":"0bb4e6d7fce94c00","name":"do something","func":"\nmsg.payload=msg.payload+1;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1220,"y":1900,"wires":[["56172419.a3b51c","2e36bbaf.787074","974fe026.15ef2"]]},{"id":"2e36bbaf.787074","type":"debug","z":"0bb4e6d7fce94c00","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1050,"y":1840,"wires":[]},{"id":"90ba11d.518d8f","type":"change","z":"0bb4e6d7fce94c00","name":"","rules":[{"t":"set","p":"reset","pt":"msg","to":"true","tot":"bool"},{"t":"set","p":"topic","pt":"msg","to":"reset delay","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":1800,"wires":[["dacb7bd972830604","56172419.a3b51c"]]},{"id":"65738e84.bc021","type":"change","z":"0bb4e6d7fce94c00","name":"","rules":[{"t":"set","p":"delay","pt":"msg","to":"interval","tot":"flow"},{"t":"set","p":"payload","pt":"msg","to":"d","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":1820,"wires":[["fd676b9c.d905e8","56172419.a3b51c"]]},{"id":"465b6808.0514a8","type":"status","z":"0bb4e6d7fce94c00","name":"status of variable delay node","scope":["56172419.a3b51c"],"x":420,"y":1660,"wires":[["882af437c4d61531"]]},{"id":"fd676b9c.d905e8","type":"debug","z":"0bb4e6d7fce94c00","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":650,"y":1720,"wires":[]},{"id":"7baf9165.62f6d","type":"change","z":"0bb4e6d7fce94c00","name":"","rules":[{"t":"set","p":"d","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1060,"y":1900,"wires":[["54eb0858.d75938"]]},{"id":"fb755e8b.022bc","type":"inject","z":"0bb4e6d7fce94c00","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"0","payloadType":"num","x":670,"y":1920,"wires":[["56172419.a3b51c"]]},{"id":"e3c9c405.2691c8","type":"ui_numeric","z":"0bb4e6d7fce94c00","name":"Interval","label":"Refresh Interval","tooltip":"","group":"83f69684.32da88","order":1,"width":0,"height":0,"wrap":false,"passthru":true,"topic":"topic","topicType":"msg","format":"{{value}}","min":"1","max":10,"step":1,"x":800,"y":1680,"wires":[["2593cb7a.8b5ce4"]]},{"id":"2593cb7a.8b5ce4","type":"change","z":"0bb4e6d7fce94c00","name":"","rules":[{"t":"set","p":"interval","pt":"flow","to":"1000*payload","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":1760,"wires":[["90ba11d.518d8f"]]},{"id":"974fe026.15ef2","type":"ui_text","z":"0bb4e6d7fce94c00","group":"83f69684.32da88","order":1,"width":0,"height":0,"name":"","label":"I'm counting","format":"{{msg.payload}}","layout":"row-spread","x":1290,"y":1780,"wires":[]},{"id":"882af437c4d61531","type":"function","z":"0bb4e6d7fce94c00","name":"","func":"\n// is it a reset?\nif(msg.status.fill == \"blue\" &&\n   msg.status.shape == \"ring\" &&\n   msg.status.text == \"0\"){\n\n    return msg;\n\n}\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":440,"y":1740,"wires":[["65738e84.bc021"]]},{"id":"dacb7bd972830604","type":"debug","z":"0bb4e6d7fce94c00","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1090,"y":1740,"wires":[]},{"id":"83f69684.32da88","type":"ui_group","name":"Default","tab":"940e1897.3ad938","order":1,"disp":true,"width":"6","collapse":false},{"id":"940e1897.3ad938","type":"ui_tab","name":"Test","icon":"dashboard","disabled":false,"hidden":false}]

[rko]

Coming back to this. Unfortunately this is not robust either. If the msg object is not "in" the delay node during reset, it's a fail. I think I have to "fire" multiple resets to make sure that the message is "dead"