MQTT Dynamic broker - multiple brokers at the same time?

General
1

I want to connect to multiple brokers at the same time - can I use a single mqtt in+out node for this with dynamic subscriptions ? If yes, how do I publish to a specific broker without disconnecting from other brokers ?

Some context - I am using a 3d printer that has its own broker.

If I have more than one and want to build a custom web interface to monitor them like status/progress/temperatures etc, can I build this fully dynamic so that the webinterface can add/remove printers and produce an status overview with all printers together ? ie. not disconnect from broker, connect to "next" broker, publish, disconnect etc.

In other words, I want to build a single flow that does not require modifications and everything can be managed from a single custom web interface.

2

First off there are two separate nodes, one for subscribing to a topic and one for publishing a topic.

Two, a single mqtt-in or mqtt-out node can only be connected to a single broker at a time but they can be connected to different brokers qt the same time. So mqtt-in could be connected to broker1 while the mqtt-out is connected to broker2.

If you can control the topics you could have one mqtt-in by using the just part of the topic and for sending msgs to the printers and you could use one mqtt-out and dynamically change the configuration dynamically.

If each printer uses a unique topic, one idea would be to create a flow that has several mqtt-in nodes (one for each printers mqtt broker) and pass the data directly to a single broker that you use in your monitoring flow.

And you could create a similar flow for the mqtt-out nodes with a switch node to direct the msgs to the right broker

Screenshot 2024-01-27 at 3.44.41 AM

Others might have some other ideas.

3

Hmm i am not following your suggestion.

I am using dynamic subscriptions in the mqtt-in/out node and I do not want to modify the flow to add more printers, but connect/subscribe/publish dynamically, while remaining connected to all brokers at the same time.

4

An MQTT node can only be connected to one broker at a time.

5

While the connection and subscriptions are dynamic, the quantity of brokers is not. You would need to setup (for example) 10 connections and 10 MQTT-In (set for dynamic)

Of course, you would set all 10 broker configs to "NOT" connect automatically and only send a connect action message (complete with broker IP/Host etc) to the MQTT node of choice as and when required.

something like:

image
image964Ă—644 75.3 KB

[{"id":"3129c0416afae8c3","type":"inject","z":"dd881c01761698db","name":"Dashboard connect button","props":[{"p":"payload.ip","v":"192.168.2.12","vt":"str"},{"p":"payload.port","v":"1883","vt":"num"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"2","x":170,"y":480,"wires":[["7f7bd70107dcad9e"]]},{"id":"8635e76fec16d33b","type":"link out","z":"dd881c01761698db","name":"To Pool =>","mode":"link","links":["300f86b5ae1c83ef"],"x":830,"y":480,"wires":[],"l":true},{"id":"7f7bd70107dcad9e","type":"change","z":"dd881c01761698db","name":"gather dashboard conn setup values","rules":[{"t":"set","p":"topic","pt":"msg","to":"dashboard.selected","tot":"global"},{"t":"set","p":"payload.ip","pt":"msg","to":"dashboard.ip","tot":"global"},{"t":"set","p":"payload.port","pt":"msg","to":"dashboard.port","tot":"global"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":480,"wires":[["8635e76fec16d33b"]]},{"id":"dd92f20b87562daa","type":"link in","z":"dd881c01761698db","name":"=> MQTT Data","links":["faf63715eaaa7371"],"x":440,"y":1040,"wires":[["b23f7380c1df877a"]],"l":true},{"id":"b23f7380c1df877a","type":"debug","z":"dd881c01761698db","name":"Printers Data to DB / MQTT / whatever","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":750,"y":1040,"wires":[]},{"id":"97febdd0f58294bc","type":"group","z":"dd881c01761698db","name":"MQTT Pool","style":{"label":true},"nodes":["0e700877a5aafc01","0e700877a5aafc02","0e700877a5aafc03","0e700877a5aafc04","0e700877a5aafc05","0e700877a5aafc06","0e700877a5aafc07","0e700877a5aafc08","0e700877a5aafc09","0e700877a5aafc10","faf63715eaaa7371","300f86b5ae1c83ef","4f251f8c550ea898","cf0a3b870662a48a","04ae804953c84f20"],"x":34,"y":519,"w":872,"h":442},{"id":"0e700877a5aafc01","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer01","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cfb01","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":560,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc02","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer02","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf302","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":600,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc03","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer03","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf303","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":640,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc04","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer04","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf304","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":680,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc05","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer05","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf305","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":720,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc06","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer06","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf306","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":760,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc07","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer07","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf307","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":800,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc08","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer08","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf308","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":840,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc09","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer09","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf309","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":880,"wires":[["faf63715eaaa7371"]]},{"id":"0e700877a5aafc10","type":"mqtt in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"printer10","topic":"","qos":"2","datatype":"auto-detect","broker":"bf9fddeea38cf310","nl":false,"rap":true,"rh":0,"inputs":1,"x":510,"y":920,"wires":[["faf63715eaaa7371"]]},{"id":"faf63715eaaa7371","type":"link out","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"MQTT Subs Data =>","mode":"link","links":["dd92f20b87562daa"],"x":780,"y":720,"wires":[],"l":true},{"id":"300f86b5ae1c83ef","type":"link in","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"To MQTT Pool =>","links":["8635e76fec16d33b"],"x":140,"y":720,"wires":[["cf0a3b870662a48a"]],"l":true},{"id":"4f251f8c550ea898","type":"junction","z":"dd881c01761698db","g":"97febdd0f58294bc","x":400,"y":720,"wires":[[]]},{"id":"cf0a3b870662a48a","type":"switch","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"num"},{"t":"eq","v":"2","vt":"num"},{"t":"eq","v":"3","vt":"num"},{"t":"eq","v":"4","vt":"num"},{"t":"eq","v":"5","vt":"num"},{"t":"eq","v":"6","vt":"num"},{"t":"eq","v":"7","vt":"num"},{"t":"eq","v":"8","vt":"num"},{"t":"eq","v":"9","vt":"num"},{"t":"eq","v":"10","vt":"num"},{"t":"else"}],"checkall":"true","repair":false,"outputs":11,"x":290,"y":720,"wires":[["0e700877a5aafc01"],["0e700877a5aafc02"],["0e700877a5aafc03"],["0e700877a5aafc04"],["0e700877a5aafc05"],["0e700877a5aafc06"],["0e700877a5aafc07"],["0e700877a5aafc08"],["0e700877a5aafc09"],["0e700877a5aafc10"],["04ae804953c84f20"]]},{"id":"04ae804953c84f20","type":"function","z":"dd881c01761698db","g":"97febdd0f58294bc","name":"out of bounds error","func":"node.error(\"Broker Selection out of bounds\", msg);\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":240,"y":880,"wires":[[]]},{"id":"bf9fddeea38cfb01","type":"mqtt-broker","name":"printer01","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf302","type":"mqtt-broker","name":"printer02","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf303","type":"mqtt-broker","name":"printer03","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf304","type":"mqtt-broker","name":"printer04","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf305","type":"mqtt-broker","name":"printer05","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf306","type":"mqtt-broker","name":"printer06","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf307","type":"mqtt-broker","name":"printer07","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf308","type":"mqtt-broker","name":"printer08","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf309","type":"mqtt-broker","name":"printer09","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}},{"id":"bf9fddeea38cf310","type":"mqtt-broker","name":"printer10","broker":"localhost","port":"1883","clientid":"","autoConnect":false,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":"","credentials":{"user":"","password":""}}]
6

Are you certain that you cannot configure the printers to use an external broker?

7

I am certain.

While the connection and subscriptions are dynamic, the quantity of brokers is not.

Looking at dynamic subscriptions - I can connect to a broker, publish, receive data and disconnect and repeat for the next one, but that would introduce a lot of noise on the wire and have to do it often to keep track of the current status.

8

I think you will have to so something like Steve has suggesteed.

What are the printers?

9

I think you will have to so something like Steve has suggesteed.

That is a pre-defined setup, which i wanted to avoid :wink: let's say a print farm with 100 printers would require 100 predefined flow-setups.

What are the printers?

bambulab printer

10

You won't have to pre-define the IP addresses, just the max number of printers there may be.

Another alternative would be to use the MQTT Bridge feature, http://www.steves-internet-guide.com/mosquitto-bridge-configuration/.

Using this you would setup your central MQTT broker to automatically suck in the data from the printers` brokers. That would mean configuring a config file for your broker with all the printers in it. If necessary you could generate that automatically in node-red. Alternatively you can do it with one config file per printer. If you are using Mosquitto then it looks in /etc/mosquitto/conf.d and any .conf files there are read so you could have one file per printer which you place there when you configure the printer.

Or perhaps the brokers in the printers supports setting up a bridge from them to you central broker.

11

Which is similar to integrating it into a (sub)flow, which is easier, but not what i was aiming for :')
thanks for all the input.

12

Just to say that if you need to talk to many brokers, doing it directly in Node-RED may not be the best approach.

Also consider using some brokers as bridges. Then you can use topics to specify the target broker and let the bridge forward the messages.

13

No, not if I understand what you mean by that. The user would not need to use the flow editor, you could do it through your custom web interface.

14

Let's assume that there is no central broker, only the existing brokers on the printers - which cannot be modified.

15

In that case it is back to something like Steve's suggestion. Have mqtt nodes for as many printers as there may be and configure them via your web i/f.

16

It doesn't take much to set up a broker though. Can't you run one on the Node-RED server?

17

I could, but it just doesnt make the flow very portable for other users.

18

then there is a Broker, that can be deployed with a flow in NR, see: node-red-contrib-aedes (node) - Node-RED

19

node-red-contrib-aedes is fairly restricted in capability, at least it was last time I looked. Does it supports Bridging?

20

...didn't look or test, but according to the Feature list GitHub - moscajs/aedes: Barebone MQTT broker that can run on any stream server, the node way aedes does support incoming bridge connections and bridging between its own kind.

Is a bridge really required in this use case?