"mqtt in" node 'Best Practice'

I've looked around and not spotted information about best practice approaches regarding the use of the mqtt in node.

There are two obvious approaches;

(a) In each flow where a particular message is needed, create a mqtt in subscribe node. Hence subscribing many times to the same data in the same Node-Red configuration.
image

(b) Subscribe to each message only once and send the output to a link out node. Re-use the message by consuming with as many link in nodes as needed.

image

I was wondering which might be less resource intense.
Is there any 'best practice' guidance in this area?

Premature optimisation is the root of all evil :wink:


IMO, option B is better.

The difference in processing time will be virtually identical but maintenance will be easier. Consider this...

With option A, if you ever needed to change the topic, you'd have to do this 3 (or more) times. With option B you would only need to do this once.

With option A, If you ever needed to temporarily disable this topic, you'd have to do that in 3 (or more) places. With option B you would only need to do this once.

With option B, you have only 1 source of truth. That is usually the better option.

1 Like

If you haven't read it yet, you might want to read the MQTT Essentials - All Core Concepts explained.

While it won't answer your initial question, it will give you more information about how to use MQTT. I like to sbscribe to the high level topic and then have a switch node examine the topic and send the messages where they should go.

1 Like

I tend to use option (b) because, as Steve pointed out - maintenance is much easier.
Here's an example of some recent Room-Temperature-Display modules I've made as PCBs.

This is a simple dashboard showing three of the modules in the Study, Lounge and Garage.

Here's part of the main flow.
Screen Shot 01-09-22 at 08.27 AM 001

This is the format of the MQTT payload - notice the part called 'node_number'
Screen Shot 01-09-22 at 08.27 AM

This is what one of the sub-flows looks like.
Screen Shot 01-09-22 at 08.28 AM

The top function node filters out data for the lounge. e.g. node_number == 32.
Screen Shot 01-09-22 at 08.44 AM

I go for option A as I try to keep flows as independent as possible. I find lots of links between flows confusing. Also I try to keep functional independence for flows, so one flow handles one aspect of the system, with minimum coupling with other flows.
So for example, I might have an Inputs flow, which collects data from sensors and outputs values to MQTT. The MQTT topics will be independent of sensor type. So for example if I measure a room temperature with a Zigbee device the value will come into the Inputs flow with a Zigbee topic, but I will send it out with a meaningful topic such as home/bedroom/temperature. If the sensor type were to change I only have to change this flow and it does not affect the rest of the system.
Then I may have a dashboard flow that wants to show that temperature, so it will pick up home/bedroom/temperature and display it.
In addition there will be a Control flow that picks up the temperature and drives the heating as appropriate.

2 Likes

One important word of caution with option A: some broker implementations (not mosquitto) will send a separate message for each subscription for a topic. That in turn will cause each of those messages to be sent to each matching node in your flow. So you end up with multiple copies of each message being sent by each node.

Mosquitto thankfully doesn't so that, but it can catch people out when they try a different broker.

2 Likes

Just to put the cat among the pigeons - there is a fly in the optimisation ointment (I do love to mix metaphors!). Doing it that way forces node.js to create 3 clones of the incoming message. For large, complex messages, this is actually relatively slow.

Of course, you wouldn't notice in most cases but if you were driving your platform hard or dealing with many messages per second, this could easily become a bottleneck.

One way to get the best of both worlds is to use groups. You can define an environment variable in a group and use that in your mqtt-in node. Meaning that you could then create 3 groups, possibly all using the same function node via link-call's.

If you have common functions, the link-call feature is great.

1 Like

You'll end up with three copies of the message in either approach. It either gets cloned in the internals of the mqtt node, or it gets cloned where the flow branches.

1 Like

To me it just seemed the easiest and quickest way to do it.
I'll willingly take and accept advice if there is a more efficient way to code it.

To chime in, I use the option of a node for every topic. Like @Colin I feel it's easier to maintain and far less confusing. I have 24 topics/ nodes on one pi and each tab in the editor is a specific function. So if something goes wrong or trying to troubleshoot everything is right there. You can argue having a node or two on another tab is no big deal but I tried that way for a while and personally found it annoying.

3 Likes

So is it fair to say that it doesn't make any difference in terms of performance / efficiency ?

I would say it isn't worth losing sleep over. The choice will be entirely based on personal preference weighing up the pros/cons of maintaining a single entry point vs many. Performance/efficiency will not be a concern.

2 Likes

Good to know - I have a mix of the two based on what works best in each case.

Like @zenofmud I use a single MQTT in node and a switch to send the content along its merry way.

1 Like