Join/Wait Node: Timeout Feature request

[dxdc]

One major limitation of the join node is the lack of a timeout. For example, two paths should be received within 10 seconds. I feel this is important enough that it should be a core feature. It has enormous potential, especially in the home automation space -- e.g, turn light on if 1) motion detected, and 2) door closed within 10 seconds.

So, I created this plugin, which is working well:

dxdc/node-red-contrib-join-wait

The following features are available:

• Timeout can be given in ms, s, min, etc.
• Paths received in "any order" or "exact order"
• Unexpected path received can trigger error (optional)
• Regex support for path names
• Support for msg.complete property and paths which immediately expire any stored entries
• Support for various payload merging options
• Regex is supported for path names
• Many properties can be set at runtime (via msg properties)

I have other ideas for expanding it, but interested to know what the community thinks about making this either an extension of the existing 'join' node or a separate 'wait' type node.

[bakman2]

If you use the join node in conjunction with the batch node, you can specify the timeframe.
If you use join node by itself in manual mode, you can specify the timeframe.

turn light on if 1) motion detected, and 2) door closed within 10 seconds

This can be done with (a/multiple) trigger node(s).

[dxdc]

@bakman2 can you provide an example flow to support your idea?

I'm not sure it would work in this case. Here's why:

• Suppose we are handling multiple events: door open/closed, motion detected, etc. How would the batch node know which sequence we're interested in? It needs to be 10 seconds starting from the first event that triggers our sequence.

• Suppose this case:

a) 0 sec: motion received
b) 9 sec: motion received
c) 11 sec: door close event

The node has to be smart enough to recognize that the b/c events are 2 seconds apart and act accordingly. This means that the time needs to be continually recalculated as events expire out of the timeout window.

[ristomatti]

This post should've probably fit better on the "share my nodes" category but thanks for sharing! Your node might come in handy at some point.

[bakman2]

How would the batch node know which sequence we're interested in?

In your example: it wouldn't.

I was responding to:

One major limitation of the join node is the lack of a timeout.

Which is not correct.

[dxdc]

@bakman2

One major limitation of the join node is the lack of a timeout.

The join node doesn't have a timeout, and I feel these kind of hacky workarounds (like batch) don't address the core issue I'm trying to raise. Chopping up a series of messages by a specific time window using batch is fine, and has its uses, but doesn't address the timeout issue with a join node if messages from path_1, path_2 should both be received within n seconds.

[bakman2]

and I feel these kind of hacky workarounds (like batch ) don't address the core issue I'm trying to raise.

a) they are not "hacky" - this is the way they work, they provide elements to the msg that are used as input for the join node (so does the split node).
b) the join node does have a timeout option when you put it in manual mode and so does the batch node.

The trigger node can do this. If door is closed within 10 seconds, cancel the trigger.

(come to think of it, a single trigger would do it too).

There are multiple ways to Rome, in your case you might be better off with your own node, but it is not "undoable".

[dxdc]

The trigger node can do this. If door is closed within 10 seconds, cancel the trigger.

I'm not sure I'm getting your idea, can you share the source for the flow you proposed?

b) the join node does have a timeout option when you put it in manual mode and so does the batch node.

Point taken, but these are fairly useless in their design for the purpose I illustrated, as they are a timeout from the time any message is received. Having the ability to dictate which paths should be allowed, which paths should expire, and having the ability to recalculate the time window as messages pass through the expiry window are -- I think -- incredibly helpful features for any join node user.

There are multiple ways to Rome, in your case you might be better off with your own node, but it is not "undoable".

Certainly, and I hear your point on that. But, I'm not convinced that my idea is actually doable using built-in nodes, which is why I'd like to review your flow. If it is, I haven't found how to do it yet. The hardest part to deal with are the kinds of cases I described earlier where multiple messages are received, i.e. (0:00 door opened, 0:02 door closed, 0:07 door opened, 0:09 door closed, 0:11 motion detected). Any type of handling that relies on when the "first message was received" isn't helpful in this case.

[bakman2]

I have edited my post, because it did not take the door into consideration.

So to capture your issue:

• door open/close event
• PIR event
• 10s timer.

PIR and door from different sources.
I will have a play.

[dxdc]

@bakman2

So to capture your issue:

• door open/close event
• PIR event
• 10s timer.

PIR and door from different sources.
I will have a play.

Thanks @bakman2, yes, exactly. Because these are independent sources, multiple events from each can be fired but we are only interested in a specific sequence being fired (door close, PIR) in a given time interval. For bonus points, it should be in that specific order without a door open event in between.

I think it's quite hard to do actually, which is why I crafted the node I mentioned, which handles it easily. At least in the home automation space, this kind of logic is extremely helpful and I found no way to do it with core or any contrib nodes -- except this one node-red-contrib-wait-paths (node) - Node-RED -- which was extremely buggy.

[bakman2]

Would this cover it ?

[{"id":"40a7e6e8.75b928","type":"inject","z":"a8260bce.a28be8","name":"","topic":"","payload":"open","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":122,"y":384,"wires":[["e177207d.70fe98"]]},{"id":"3f85770b.4cb678","type":"inject","z":"a8260bce.a28be8","name":"","topic":"","payload":"closed","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":122,"y":432,"wires":[["e177207d.70fe98"]]},{"id":"8c51a823.3cbb58","type":"debug","z":"a8260bce.a28be8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":528,"wires":[]},{"id":"851c13ae.8e9a08","type":"trigger","z":"a8260bce.a28be8","op1":"","op2":"light on","op1type":"nul","op2type":"str","duration":"10","extend":false,"units":"s","reset":"reset","bytopic":"all","name":"","x":478,"y":552,"wires":[["8c51a823.3cbb58","9e667440.917b38"]]},{"id":"e177207d.70fe98","type":"change","z":"a8260bce.a28be8","name":"door: state","rules":[{"t":"set","p":"door","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":286,"y":408,"wires":[["3674d2d7.4cfdbe"]]},{"id":"3674d2d7.4cfdbe","type":"switch","z":"a8260bce.a28be8","name":"if open","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"open","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":434,"y":408,"wires":[["799e4cd0.6fd684"]]},{"id":"799e4cd0.6fd684","type":"change","z":"a8260bce.a28be8","name":"reset","rules":[{"t":"set","p":"payload","pt":"msg","to":"reset","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":554,"y":408,"wires":[["851c13ae.8e9a08"]]},{"id":"ddc58661.985cd","type":"inject","z":"a8260bce.a28be8","name":"","topic":"","payload":"PIR","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":122,"y":552,"wires":[["b770d5c3.92ae58"]]},{"id":"b770d5c3.92ae58","type":"switch","z":"a8260bce.a28be8","name":"door closed?","property":"door","propertyType":"flow","rules":[{"t":"eq","v":"closed","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":286,"y":552,"wires":[["851c13ae.8e9a08"]]},{"id":"9e667440.917b38","type":"change","z":"a8260bce.a28be8","name":"","rules":[{"t":"delete","p":"door","pt":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":680,"y":576,"wires":[[]]},{"id":"e21d7045.b39898","type":"comment","z":"a8260bce.a28be8","name":"delete flow var to ignore PIR","info":"","x":720,"y":624,"wires":[]}]

[Colin]

For that sort of problem I would use a state machine. node-red-contrib-dsm for example.

[dxdc]

@bakman2 clever idea

Maybe I'm misreading your idea, but this approach won't work. I'll outline a few of the issues. As I mentioned, I think this is fairly difficult to achieve!

Using a variable like flow.door fails in the following case:

1. Close door. Now, flow.door is set to closed.
2. Wait 20 seconds (longer than the prescribed timeout). flow.door has no knowledge that time has passed.
3. Send a PIR event. Now, there's an additional 10s delay from the trigger node (that shouldn't be there) before the flow finishes. Also, the flow shouldn't have finished since both events didn't fire within 10s.

The 10s trigger waits too long in some cases, and fails in others

1. Send a PIR event
2. Send a close event
3. Send an open event (soon thereafter)

The flow will not finish. It should have finished immediately after 1&2.

For that sort of problem I would use a state machine. node-red-contrib-dsm for example.

Sure, but using a variable like that doesn't address the above issue either.

[bakman2]

I tried to match the flow with this statement:

@dxdc

but we are only interested in a specific sequence being fired ( door close , PIR ) in a given time interval

[dxdc]

@bakman2

I tried to match the flow with this statement:

Understood, but your flow doesn't address these issues:

1. flow.door does not expire once it is set to closed. It only expires if the door is open. So, if a door is closed... an unlimited amount of time can occur before the PIR event, and then it will still fire the event.

2. If door close, PIR are fired, the flow waits an additional 10s. During this time, if a door open event is sent, the entire flow expires.

[Colin]

Which just goes to show the importance of getting the requirement right in the first place. Perhaps you can define exactly what the requirement is?

[dxdc]

Basic requirement:

• Turn light on if 1) motion detected, and 2) door closed within 10 seconds.

Note that "motion detected" and "door closed" are from independent sources and can fire multiple times in a row.

• The flow should complete immediately once the conditions are satisfied

Advanced requirement:

• Require that "motion" event precedes "door close" event (or vice versa for this example)
• If "door open" event is fired, reset all tracking. For example, if we are looking for (door close, motion), then (door close, door open, motion) would not trigger the light on event.

[Colin]

How about this? It uses node-red-contrib-simple-gate to allow or stop the door close event through to switch the light on.

[{"id":"a0a4582c.c1a8","type":"inject","z":"bdd7be38.d3b55","name":"Motion event","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":103,"y":1831,"wires":[["fa620c0a.33c2b"]]},{"id":"5921f58c.b4f52c","type":"inject","z":"bdd7be38.d3b55","name":"Door open event","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":115,"y":1918,"wires":[["e71aeb24.9ec5c8"]]},{"id":"ce7ce70b.0381c8","type":"inject","z":"bdd7be38.d3b55","name":"Door close event","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":114,"y":2005,"wires":[["9640704d.fb5aa8"]]},{"id":"fa620c0a.33c2b","type":"trigger","z":"bdd7be38.d3b55","op1":"open","op2":"close","op1type":"str","op2type":"str","duration":"10","extend":true,"units":"s","reset":"","bytopic":"all","name":"","x":520,"y":1832,"wires":[["18683277.e9b3d6"]]},{"id":"a35dc80d.18eb78","type":"debug","z":"bdd7be38.d3b55","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":702,"y":2001,"wires":[]},{"id":"e71aeb24.9ec5c8","type":"change","z":"bdd7be38.d3b55","name":"Reset trigger, close gate","rules":[{"t":"set","p":"reset","pt":"msg","to":"true","tot":"bool"},{"t":"set","p":"payload","pt":"msg","to":"close","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":332,"y":1917,"wires":[["fa620c0a.33c2b","18683277.e9b3d6"]]},{"id":"a84ecc10.cf2bf","type":"gate","z":"bdd7be38.d3b55","name":"","controlTopic":"control","defaultState":"closed","openCmd":"open","closeCmd":"close","toggleCmd":"toggle","defaultCmd":"default","persist":false,"x":526,"y":2002,"wires":[["a35dc80d.18eb78"]]},{"id":"18683277.e9b3d6","type":"change","z":"bdd7be38.d3b55","name":"Set control topic","rules":[{"t":"set","p":"topic","pt":"msg","to":"control","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":683,"y":1914,"wires":[["a84ecc10.cf2bf"]]},{"id":"9640704d.fb5aa8","type":"change","z":"bdd7be38.d3b55","name":"Light on","rules":[{"t":"set","p":"payload","pt":"msg","to":"light on","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":322,"y":2004,"wires":[["a84ecc10.cf2bf"]]}]

[dxdc]

I haven't played with node-red-contrib-simple-gate, but I really like this approach @Colin! I made one small edit to your flow - essentially to close the gate immediately after a successful run. This is needed to avoid multiple door close events from firing the light on again.

I'm not sure if there would be any race conditions here or not with the resets and such (at really fast intervals) but it does seem like a cool approach.

This becomes a bit mind stretching if more items are added (e.g., 4 sequential events), but in general, do you agree that this kind of functionality should be baked into the core set of nodes? I feel this is a common situation that is challenging to do.

[bakman2]

This becomes a bit mind stretching if more items are added (e.g., 4 sequential events),

Then indeed it will become more complex to manage, I am not sure that it is so "common".

Your original approach (node) was to combine messages based on timeframes and that part does not appear necessary, albeit very useful. Keeping track of a "state" is the key i think, if there were additional events were involved, using XOR's (like the boolean-logic nodes) could perhaps work in some way.

What i like about Colin's flow is that you can "read" what the flow is doing.