MQTT clients list and additional details

General
1

Node-red and MQTT newbie here:
Scenario:

  1. Several ESP32/Raspberry Pi devices clients connected to a MQTT broker on the local LAN, e.g (CONNECT packet) and have a response (CONNACK)
  2. Node-Red server

What I am trying to achieve:

  1. Obtain a list of Clients in Node-RED (include details: Client ID, current subscriptions if any)
  2. Send a command from Node-Red to selected clients (as above) to subscribe to a topic

Unicorn goal.
Is to have a file on the client (String) which has details on the client (connected to the MQTT broker)
This would be manually entered when the client came online, the Node-RED could query this file and get information (file format would be text-based, however, have not yet formalised the actual structure at this time)

2

I don't believe that brokers expose the connected client ID's, only a count of how many are connected.

image
image399Ă—470 11.6 KB

Similarly, the subscriptions are also not exposed, only a count.

That is certainly true for Mosquitto anyway.

To get round that, you need to get each client to send an initial connect topic message:

image
image425Ă—410 10.3 KB

Along with other info such as the connection timestamp. You use a Last Will & Testament (LWT) setting on the connection to mark the device offline and the initial connection mesage to mark it online. You can add any other info you like to other topics as shown in the example.

3

The MQTT protocol has no way to provide the information you want about connected clients.

Some brokers may have an out of band management interface, normally a HTTP API that might be able to supply it, but it would be specific to the broker (e.g. emqx) and not related to the MQTT protocol at all.

And there is no way to send a message to a specific MQTT client without either

  1. Publishing to a single topic all clients are subscribed to and including a client identifier in either the message body or if using MQTTv5 a header property and having the clients filter messages intended for them
  2. Have ever client listen on their own specific topic e.g. command/<client-id>

As for retrieving a file from a specific client, again you will need to write this into the client code and have them listen for commands as described above and respond on a topic (this could be in the command request, or a dedicated response topic for each client) with the content of the file

4

Appreciated and thanks.
I am trying to determine which path is possible to reach an outcome. This helps immensely.

where is a the best source for command/ you refer to

5

Appreciated

any good info on this approach I can refer to?

6

Can't think of any off the top of my head. one of us should probably really do a write up on it.

When your IoT device powers up, it should connect to your broker and send the initial online message. The connection should define the LWT message, this actually is triggered on the broker itself if the client device stops talking to the broker. The initial msg and the LWT should use the same topic, just a different payload.

You may want to check out HiveMQ's good documentation on MQTT basics: MQTT Essentials - All Core Concepts Explained

1 Like
7

thanks,

learning this and it hard to know where to start - I best learn via a project goal and work my way backwards - hence the can it, how to and best approach questions

1 Like
8

What firmware do you use on your ESPs?

I use Tasmota, which automatically sends useful messages including birth and lwt.

For example, one device has the mqtt topic set to tasmota/wemos/door

When I tell it to reboot I get the following MQTT messages. It doesn't include the client ID but it does identify the ESP by hostname and IP address:

Edit: the MQTT client ID for this ESP is DVES_C07971. It's possible to infer this from the fallback topic in INFO1 (as long as I don't mess with the defaults)

{topic: "tele/tasmota/wemos/door/LWT", 
payload: "Offline"}

{topic: "tele/tasmota/wemos/door/LWT",
 payload: "Online"}

{topic: "tele/tasmota/wemos/door/INFO1", 
payload: {"Info1":{
   "Module":"Generic", 
   "Version":"12.5.0(sensors)",
   "FallbackTopic":"cmnd/DVES_C07971_fb/",
   "GroupTopic":"cmnd/tasmotas/"}}
}

{topic: "tele/tasmota/wemos/door/INFO2", 
payload:{"Info2":{
   "WebServerMode":"Admin",
   "Hostname":"tasmota-wemos-door-6513",
   "IPAddress":"192.168.1.74"}}
}

{topic: "tele/tasmota/wemos/door/INFO"3, 
payload:{"Info3":{
   "RestartReason":"Software/System restart",
   "BootCount":211}}
}

If you use Arduino firmware of course you get to program all of this by hand :face_with_raised_eyebrow:

I don't do so, but I could use the GroupTopic to send a single command which all my Tasmota devices would receive, requesting them to reply with (for example) their uptime.

9

There isn't one, you need to implement it yourself

10

beside the last will, i use a standardized topic structure for sending the current timestamp (like client/garden/node-red/ping)

In the node-red Dashboard, i use a table for getting a quick overview:

grafik
grafik1664Ă—281 12.8 KB

I also have an ansible playbook which i roll out on every device for getting more infos. Its a simple script which send data over mqtt:

grafik
grafik2121Ă—427 20.6 KB

11

this is heading where I wanted to go - The ip4 Address will be somewhat dynamic, so i was heading name resolution route,
I was hoping to pull; the MAC address to concatenate and UID

12

I am not clear on your precise requirement but with Tasmota you can:

  • Get a response from each one connected to the network including it's hostname, IP, MAC, client ID and the MQTT topics it is subscribed to *.
  • For those which have sensors connected, retrieve the sensor values.
  • For those with relays, toggle the power.

* Tasmotas subscribe to :

  • A topic which you can modify from Node-red (though I find it hard to see why you would, perhaps to setup a new device which only knows about the wifi)
  • A fallback topic which you can't directly modify
  • A group topic which can be used to address devices with similar functions.
13

Personally, I never got on with the complexity (as I saw it) of Tasmota's topic structures. Probably because I am OCD and a control freak! :smile:

I prefer the flexibility of ESPHome.

But the point remains. The trick with MQTT is to get a workable topic structure. I typically de-normalise my topics.

That's to say that I have hardware based structures such as ESP/<device-id>/... where ESP/<device-id> shows online or offline and the sub-topics contain lots of detail.

Then I will have location based structures such as LOCATION/Living Room/Temperature and so on.

I also typically have data-type structures such as ENVIRONMENT/Temperature/<location>.

That way it is easy to subscribe, using wildcards, to whatever data you need for different uses.

2 Likes