[New Node] node-red-contrib-events: Alternative to link nodes

(and does some more things)

Ah right - yes that was the one (ok so there are already 2 :slight_smile:

Hmm.

Maybe not so much. They look similar on the surface but they certainly they aren't under the skin.

The topic-link nodes use a pair of internal objects. sub-link requires a config node.

Both require configuration and use MQTT-style wildcards and neither make use of the native event system at all.

The approach I've suggested requires zero configuration and makes use of the native events process and so should be as efficient as it is possible to be. The code is also a lot simpler, though it will have to get a bit more complex to handle the fact that I can't use Node-RED's own event service (I will likely use a singleton class so as to avoid the need for a config node).

So I'm still thinking there may be room for another node. Shoot me down if you think that I'm wrong. I would add references to the other options as well of course.

what kind of configuration does "topic-link" require that "events" doesn't?

Regarding efficiency: If a topic is specified on both in and out nodes, then it is as efficient as possible, since during runtime no search over nodes is necessary.
In case the topic is specified via the message, then it has to loop over all out nodes, which the event emitter class has to do as well. Could still be that the event emitter class is a bit more efficient in this case, but one would has to test.
Using event emitters does not allow wildcards, which was the reason for this implementation.

Apologies, I should have realised that you were the author!

Ah, see what you mean. My bad. I hadn't installed it and I was just reading the code. You are right, no config required.

The topic-link node uses 2 sets and a map to keep track of things I think?

The event node uses the node.js event handler which only requires a string (topic) and an event handler function and all that function does is a send. So no need to keep track of data at all. And no need to loop over anything. The close processing is similarly very simple and takes into account having any number of node instances of any topic.

However, of course, I have not included any wild-card handling and a new day with fresh coffee suggests that maybe that isn't as easy as I thought.

Indeed, I should have remembered, having looked at this recently, that the EventEmitter2 module is what is needed for wildcards and other nice advanced event handling. And it claims to be even faster than the native node.js handler.

The downside of EventEmitter2 is that it only supports '*' as a wildcard, not sure if it would be possible to manipulate that to support MQTT-style wildcards.

The advantage is that it would be fairly easy to add support for subscribing to multiple topics as an array.

No worries!

Yes, those sets and maps are to realize wildcards both in- and out-bound with the advantage of the linking between the nodes happening on deploy (if the topic is set on the node).
Without wildcards, one could get rid of the loop during runtime for the case where one wants to set the topic through the message.

I have pushed a new version of contrib-events, you might want to have a look. It uses the EventEmitter2 node instead of the RED.events and so requires no additional coding to support * wildcards. I've set it to recognise / as a namespace separator just like MQTT. It uses a shared module between the two separate node modules.

2 Likes

There's also node-red-contrib-pubsub (node) - Node-RED which seems to achive the same end result?

Annoying to have to install it to look at the code.

These nodes have no real docs. They manually do what the native event system does for you. They manually convert msg objects to a string and back. The stringify process isn't wrapped in a try/catch which may be a little fragile.

And finally, they don't support wildcard subscriptions.

Currently, @cinhcet's node-red-contrib-topic-link nodes are the closest to mine.

As such, I don't think I will publish these with the node-red tag (assuming I publish them at all) to save any further confusion even though I much prefer my own version here :slight_smile: I will undoubtedly continue to use mine and they will remain on GitHub with some enhancements.

If you want to use my version, please feel free to do so from GitHub. Or contact me direct if you think they should be published. I've added references to the other nodes listed in this thread - many thanks for those people who went digging :mage: - in the readme.

4 Likes

Why don't you collaborate? Labor shared is halved.

In the end, I decided to publish these nodes because their execution is sufficiently different from the alternatives that are available.

I've also added a 3rd node called event-return. This enables sub-flows and loops to be easily created. You add the node to the end of an event-in triggered flow and it will return the flow back to the event-out node that triggered it.

Should be used (as should all of these nodes) with caution since the apparent routing of msg's can get somewhat confusing - which is why Node-RED tries (sensibly) to stick with nodes linked by wires - these are much easier to follow. However, an event-based approach can occasionally massively simplify your flows.

To help with any debugging, the event-out node adds a msg._eventOriginator property which is the node ID of the originating event-out node. The event-return node adds a msg._eventReturner property which is the node ID of the originating event-return node. Simply paste the values from these properties into Node-RED's search to find the matching node.

The nodes require virtually no configuration, they use the msg.topic to define the event names, both glob and MQTT style wildcards are supported.

The nodes use a new external module that implements a sharable event handler. This means that any node.js package can require the module, all requires in a single node.js app will get the same event handler.

I'll likely be using this shared event handler module in uibuilder as well in the future to facilitate integration between uibuilder-compatible nodes.

The event names are deliberately quite long and complex in order to help ensure that no name clashes will occur. All of the events in the events nodes for example start with node-red-contrib-events/....

That's all curiously familiar to the design I shared yesterday for the new call link node and the link return mode for the link out node. Even to the style of the message property used to track the caller.

Amazing! Great minds obviously think alike - or maybe just shamelessly borrow from each other :thinking:

That genuinly is a coincidence because I didn't really understand all of the design note.

Honestly, I just found the idea of a way to create a sub-flow interesting and wondered how easy it would be to achieve with node.js event handlers. Turned out to be way easier than I thought.

Still, as previously discussed, the overall approach to the event nodes is rather different to the link nodes. However, you may spot that I've added some warnings to the documentation about not going mad with these things and the benefits of sticking with nodes and wires.

Julian, I am sure this was never the intended usage (since I had to jump through hoops to stash the topic) but I'll raise it anyhow. If I try to do nested "subroutine"-alike operations, a weird thing happens - the first event-out node continuously spits out the same unmodified message

[{"id":"445f49b44309142d","type":"event-in","z":"af952aeaa20f4f97","name":"","topic":"red/test","x":1630,"y":1860,"wires":[["09fafe2e65457d0a"]]},{"id":"aa103474b84d5a58","type":"event-out","z":"af952aeaa20f4f97","name":"","topic":"","passthrough":"return","outputs":1,"x":1880,"y":1760,"wires":[["3c32a6b94faee583"]]},{"id":"2cbb3cd788dff965","type":"event-return","z":"af952aeaa20f4f97","name":"","topic":"","passthrough":false,"outputs":0,"x":2350,"y":1860,"wires":[]},{"id":"7dc71f19ca5fcdd7","type":"inject","z":"af952aeaa20f4f97","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"red/test","payloadType":"date","x":1660,"y":1760,"wires":[["aa103474b84d5a58"]]},{"id":"3c32a6b94faee583","type":"debug","z":"af952aeaa20f4f97","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2080,"y":1760,"wires":[]},{"id":"acf119f388e431ec","type":"event-in","z":"af952aeaa20f4f97","name":"","topic":"blue/test","x":1860,"y":2020,"wires":[["93dbe802654942a4","fa0fcadc9a7342fa"]]},{"id":"3a96f909199808d6","type":"event-return","z":"af952aeaa20f4f97","name":"","topic":"","passthrough":false,"outputs":0,"x":2230,"y":2020,"wires":[]},{"id":"93dbe802654942a4","type":"change","z":"af952aeaa20f4f97","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"(\t    $minimum := 1;\t    $maximum := 10;\t    $round(($random() * ($maximum-$minimum)) + $minimum, 0)\t)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":2040,"y":2020,"wires":[["3a96f909199808d6","b573d1dcece1fdd3"]]},{"id":"09fafe2e65457d0a","type":"change","z":"af952aeaa20f4f97","name":"","rules":[{"t":"set","p":"topicOrig","pt":"msg","to":"topic","tot":"msg"},{"t":"set","p":"topic","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1800,"y":1860,"wires":[["74f80e84b2cb1a4d","d16d39fab75667fd"]]},{"id":"7fed6d4ded4432e6","type":"change","z":"af952aeaa20f4f97","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"topicOrig","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":2150,"y":1860,"wires":[["2cbb3cd788dff965"]]},{"id":"74f80e84b2cb1a4d","type":"debug","z":"af952aeaa20f4f97","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1970,"y":1920,"wires":[]},{"id":"fa0fcadc9a7342fa","type":"debug","z":"af952aeaa20f4f97","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1970,"y":2060,"wires":[]},{"id":"b573d1dcece1fdd3","type":"debug","z":"af952aeaa20f4f97","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2230,"y":2060,"wires":[]},{"id":"aa5329c75fc8c224","type":"debug","z":"af952aeaa20f4f97","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2150,"y":1920,"wires":[]},{"id":"d16d39fab75667fd","type":"event-out","z":"af952aeaa20f4f97","name":"","topic":"blue/test","passthrough":"return","outputs":1,"x":1980,"y":1860,"wires":[["7fed6d4ded4432e6","aa5329c75fc8c224"]]}]

Perhaps you might want to handle this - or discourage users from attempting it?

Also, what was the thought process behind permitting msg.topic to override the configured topic?
For example, the MQTT Out nodes state " The topic used can be configured in the node or, if left blank, can be set by msg.topic" whereas your new node states "Optionally, the MQTT topic to use. Takes preference over the topic defined in settings"

Indeed. If the user sets it, they usually want it set for a reason, so should be honoured by default if possible.

There's always one! :grinning: - I'll have a look.

I probably need to think through the nested thing a bit more. For now, just don't do it :grinning:

For the return, the thinking was that you might want to override the topic as you return so that the remaining flow uses something different. Of course, after a nights sleep, I realise that you can do that anyway because the return node doesn't use the topic in the msg, it uses a hidden event name based on the event-out node's ID.

Another thing I will change.

UPDATE: So, post work, I worked out the better way to do this and that supports nested calls :slight_smile:

Back to using msg._eventReturner along with msg._eventOriginator but now BOTH are ARRAYs.

This means that you can track both the out nodes and have them nested, and the return nodes which will return back to the start and your example flow works correctly :mage:

Any through message isn't impacted by the amended msg's either so those flows also work as expected.

v1.1.3 on its way.


So it turns out that lazy programming is not always the best approach. :thinking:

I added a second property msg._eventReturner rather than changing msg._eventOriginator to the returner node's ID because good old async issues meant that the originator ID was then changed downstream as well which I didn't want.

By not being so lazy, I can do tiEvents.emit(eventName, { ...msg, ...{_eventOriginator: this.id} } ) to merge the new id into the emitted msg and that immediately fixes the loop.

However, it means that you never get any output on that original event.out because the return is hijacked by the second event.out. I'm not sure whether that is the best approach or not?

The alternative would be to prevent you from having an event.out attached to an event.in but that seems like a rather drastic block for something that is likely to be a fairly rare occurance.

What do people think?

If you have the time and inclination Steve, I've pushed the new version to GitHub but not yet published to npm so you could test it to see if it meets your expectations :grinning:

There is even a new example flow based on your example.

Hi Julian, looking good - works for me.

One small niggle from me - I wonder if there is something that could be done to avoid having to stash and restore topics? :thinking: hmmm, its a doozy.