Strange Happening with MQTT & zigbee2mqtt Nodes

General
1

I have recently started using zigbee sensors (very small so SHMBO likes that they can be hidden). I already have a variety of 433Mhz sensors chatting to Node-Red though rtl_433 & MQTT. There seem to be a variety of zigbee2mqtt nodes so I chose node-red-contrib-zigbee2mqtt (the bridge node allows editing the friendly name and groups without manually editing the configuration.yaml file)

However, once I added the zigbee2mqtt in node the MQTT node displayed a status of 'not connected', even though it was, and was sending data. If I add another MQTT node it does not display its status (even if added to another flow on the same Node-Red instance). The MQTT out node DOES display status, as do any other in nodes using the same MQTT broker on other Node-Red instances.

Although, as the data is still being provided by the MQTT node, this is not an end of the world event I wondered it anyone had any thoughts as to why this might be happening.

Apologies if this has already been raised but I couldn't find anything when I searched.

2

What exactly do the nodes do? Does it provide an mqtt broker? If not then what is the server node? I am wondering whether you have a conflict between brokers.

What are you using for your broker and have you looked at its logs?

3

The broker is Mosquitto and seems to be OK when I look at the output with MQTT Explorer and I cannot see anything that leaps out in the log. The MQTT node does collect the zigbee data because the Zigbee2MQTT.io is broadcasting to the server. The zigbee nodes are also connected to the Mosquitto server
image
As far as I can tell the nodes are doing the same job as the MQTT node, on steroids. The MQTT node just receives the basic zigbee payload (temperature, humidity etc.) the zigbee2mqtt node receives that plus device data,

image
image216×615 25.7 KB
etc.
I am running zigbee2mqtt.io as a daemon on a Pi as per the documentation on GitHub with a basic configuration.yaml image .
I am assuming that I have configured stuff correctly because it is all working, but you know what assuming does.

4

You haven't told us what the mosquito logs show. If some of the nodes are failing to connect then you should see that.

5

The logs look fine. As I said the nodes are connecting and passing data into Node-red. The only issue is the incorrect / lack of status message. Here is a snapshot of the Mosquitto log

1593206451: New client connected from 192.168.1.36 as DVES_FBFEA4 (c1, k30, u'jeff').
1593206452: New connection from 192.168.1.32 on port 1883.
1593206452: New connection from 192.168.1.31 on port 1883.
1593206452: New client connected from 192.168.1.31 as DVES_7556B9 (c1, k30, u'jeff').
1593206452: New client connected from 192.168.1.32 as DVES_C847DA (c1, k30, u'jeff').
1593206453: New connection from 192.168.1.30 on port 1883.
1593206453: New client connected from 192.168.1.30 as DVES_E15D05 (c1, k30, u'jeff').
1593206459: New connection from 192.168.1.33 on port 1883.
1593206459: New client connected from 192.168.1.33 as DVES_B0441A (c1, k30, u'jeff').
1593206857: Client DVES_FBFEA4 has exceeded timeout, disconnecting.
1593206857: Socket error on client DVES_FBFEA4, disconnecting.
1593206858: Client DVES_C847DA has exceeded timeout, disconnecting.
1593206858: Socket error on client DVES_C847DA, disconnecting.
1593206859: Client DVES_E15D05 has exceeded timeout, disconnecting.
1593206859: Socket error on client DVES_E15D05, disconnecting.
1593206876: Socket error on client DVES_7556B9, disconnecting.
1593206876: New connection from 192.168.1.31 on port 1883.
1593206876: New client connected from 192.168.1.31 as DVES_7556B9 (c1, k30, u'jeff').
1593206876: New connection from 192.168.1.32 on port 1883.
1593206876: New client connected from 192.168.1.32 as DVES_C847DA (c1, k30, u'jeff').
1593206877: New connection from 192.168.1.30 on port 1883.
1593206877: New client connected from 192.168.1.30 as DVES_E15D05 (c1, k30, u'jeff').
1593206888: New connection from 192.168.1.36 on port 1883.
1593206888: New client connected from 192.168.1.36 as DVES_FBFEA4 (c1, k30, u'jeff').
1593207132: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1593208671: New connection from 192.168.1.26 on port 1883.
1593208671: New client connected from 192.168.1.26 as mqttjs_32372e49 (c1, k60, u'jeff').
1593208708: Client mqttjs_32372e49 disconnected.
1593208771: New connection from 192.168.1.26 on port 1883.
1593208771: New client connected from 192.168.1.26 as mqttjs_26e84a71 (c1, k60, u'jeff').
1593208816: Client mqttjs_26e84a71 disconnected.
1593208848: New connection from 192.168.1.26 on port 1883.
1593208848: New client connected from 192.168.1.26 as mqttjs_36e36110 (c1, k60, u'jeff').
1593208895: Client mqtt_d6a090fc.397b4 disconnected.
1593208933: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1593208972: New connection from 192.168.1.20 on port 1883.
1593208972: New client connected from 192.168.1.20 as mqtt_622cc24b.0e9dac (c1, k60, u'jeff').
1593209081: Client mqtt_754211dc.a2694 disconnected.
1593209083: New connection from 192.168.1.21 on port 1883.
1593209083: New client connected from 192.168.1.21 as mqtt_3f74e1e.3c8bd1e (c1, k60, u'jeff').
1593210734: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1593210999: Client DVES_7556B9 has exceeded timeout, disconnecting.
1593210999: Socket error on client DVES_7556B9, disconnecting.
1593211026: Socket error on client DVES_B0441A, disconnecting.
1593211026: Socket error on client DVES_C847DA, disconnecting.
1593211027: Socket error on client DVES_E15D05, disconnecting.
1593211030: New connection from 192.168.1.30 on port 1883.
1593211030: New client connected from 192.168.1.30 as DVES_E15D05 (c1, k30, u'jeff').
1593211030: New connection from 192.168.1.31 on port 1883.
1593211030: New client connected from 192.168.1.31 as DVES_7556B9 (c1, k30, u'jeff').

As fa as I can tell tis appears normal. 192.168.1.26 is the relevant connection for the zigbee2mqtt client. The DVES clients are the Tasmota devices on rtl_433

6

We had the same question some days ago. The zigbee2mqtt node does not use mqtt to talk to his devices, so you can't see any flow in the mqtt broker!

7

That doesn't look very normal to me, all the devices are continually connecting and disconnecting, including 192.168.1.26. Also notice that there are multiple different client ids of the form mqttjs_nnnn which is odd.

8

I think many MQTT client libs generate an auto client ID if not specified by the DEV/User. Tho I am not 100% certain if that is true when just re-connecting. Perhaps his devices are re-creating MQTT object when they think they have lost connection? Or perhaps the code on the devices is just plain wrong (i.e. always generating a new MQTT connection object every time they want to send data)?

Either way, I agree it doesn't look right.

9

The mqttjs_nnnn is on 192.168.1.26 which is the Pi that the zigbee2MQTT is running on so I am pretty sure that is what that is (i.e they are the zibee2mqtt nodes). The DVES units disconnecting is possibly down to power cuts / outages / reboots. (I have connected them to another instance of Mosquitto I have running and they stay stable. I have reconnected them back to the original instance and again, so far, they are stable). Client mqtt-explorer is MQTT Explorer.

I have been doing some experiments and there are node disconnections and reconnections when a full Deploy is carried out.

1593883268: New client connected from 192.168.1.30 as DVES_E15D05 (c1, k30, u'j$
1593883290: New connection from 192.168.1.31 on port 1883.
1593883290: New client connected from 192.168.1.31 as DVES_7556B9 (c1, k30, u'j$
1593883311: New connection from 192.168.1.32 on port 1883.
1593883311: New client connected from 192.168.1.32 as DVES_C847DA (c1, k30, u'j$
1593883333: New connection from 192.168.1.36 on port 1883.
1593883333: New client connected from 192.168.1.36 as DVES_FBFEA4 (c1, k30, u'j$
1593884488: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1593884946: Client mqtt-explorer-a1a87eac disconnected.
1593885249: Client mqtt_e0f5c38.579854 disconnected.
1593885250: New connection from 192.168.1.21 on port 1883.
1593885250: New connection from 192.168.1.21 on port 1883.
1593885250: Client NodeRed-389a8ae8.055bd6 disconnected.
1593885250: New client connected from 192.168.1.21 as NodeRed-389a8ae8.055bd6 ($
1593885250: New client connected from 192.168.1.21 as mqtt_a40eba6.3b0c148 (c1,$
1593885251: New connection from 192.168.1.21 on port 1883.
1593885251: New client connected from 192.168.1.21 as NodeRed-389a8ae8.055bd6-t$
1593885251: Socket error on client NodeRed-389a8ae8.055bd6-tmp, disconnecting.
1593885372: New connection from 192.168.1.20 on port 1883.

The first reconnections are where I reconnected the Sonoff devices back to this instance of Mosquitto, the rest is reconnecting nodes in Node-Red and then Deploying them.

The status of the MQTT Node was showing correctly for a while until I carried out the Deploy.

The Time field of the Mosquitto log is (I assume) the time running as as a date it is sometime in 1970 so this is no help in tracking an entry.

I would like to thank everyone for their interest because I do not have a clue. Fortunately the lack of a correct status is having no effect on the transmission & reception of MQTT data.

10

I believe those are epoch times in seconds rather than javascript times (which are ms) so they are likely correct.

11

You're not only right, you are right. Thank you :grinning:

12

What do you see if you use a status node linked to the mqtt node?
Can I just check, when you say the MQTT In node is not showing the status correctly, you do mean one of the standard MQTT nodes don't you?

13

Also could you post the startup log when you start node-red in a terminal in case we can see anything unusual there.

14

  1. image
    I connected the output to a Debug node (full msg) and set it to only monitor the MQTT in node. As you can see it thinks the Status is Connected.

  2. Yes, bog standard MQTT In node (Note: the MQTT Out node shows its status with no problem)

15 
**4 Jul 21:14:56 - [info]**
**Welcome to Node-RED**
**===================**
**4 Jul 21:14:56 - [info] Node-RED version: v1.0.4**
**4 Jul 21:14:56 - [info] Node.js  version: v10.21.0**
**4 Jul 21:14:56 - [info] Linux 4.19.118-v7l+ arm LE**
**4 Jul 21:14:58 - [info] Loading palette nodes**
**4 Jul 21:15:03 - [info] Worldmap version 2.3.8**
**4 Jul 21:15:04 - [info] Dashboard version 2.21.0 started at /ui**
**4 Jul 21:15:05 - [info] Settings file  : /home/pi/.node-red/settings.js**
**4 Jul 21:15:05 - [info] HTTP Static    : /home/pi/.node-red/public**
**4 Jul 21:15:05 - [info] Context store  : 'memoryOnly' [module=memory]**
**4 Jul 21:15:05 - [info] Context store  : 'file' [module=localfilesystem]**
**4 Jul 21:15:05 - [info] User directory : /home/pi/.node-red**
**4 Jul 21:15:05 - [warn] Projects disabled : editorTheme.projects.enabled=false**
**4 Jul 21:15:05 - [info] Flows file     : /home/pi/.node-red/flows.json**
**4 Jul 21:15:05 - [info] Server now running at http://127.0.0.1:1880/**
**4 Jul 21:15:05 - [warn]**
**---------------------------------------------------------------------**
**Your flow credentials file is encrypted using a system-generated key.**
**If the system-generated key is lost for any reason, your credentials**
**file will not be recoverable, you will have to delete it and re-enter**
**your credentials.**
**You should set your own key using the 'credentialSecret' option in**
**your settings file. Node-RED will then re-encrypt your credentials**
**file using your chosen key the next time you deploy a change.**
**---------------------------------------------------------------------**
**4 Jul 21:15:05 - [info] Starting flows**
**4 Jul 21:15:06 - [info] Started flows**
**4 Jul 21:15:06 - [info] [sqlitedb:bbd8125b.375d88] opened /media/usbDrive/HomeAutomation.db ok**
**4 Jul 21:15:07 - [info] [zigbee2mqtt-server:389a8ae8.055bd6] MQTT Connected**
**4 Jul 21:15:07 - [info] [zigbee2mqtt-server:389a8ae8.055bd6] Refreshing devices**
**4 Jul 21:15:07 - [info] [mqtt-broker:b1a8e581.0e3f6] Connected to broker: mqtt://192.168.1.21:1883**
**4 Jul 21:15:07 - [info] [zigbee2mqtt-server:389a8ae8.055bd6] MQTT Subscribed to: "zigbee2mqtt/#**
16

Was that while it was displaying disconnected?

17

In case this runs into a dead end, you might want to take a look at Zigbee2MqttAssistant (https://github.com/yllibed/Zigbee2MqttAssistant) for managing the connections and using regular mqtt nodes for the data.

18

Seems to only work with Docker or Home Assistant, neither of which I use. Also, it isn't Node-red :wink:

19

Hi, yes - MQTT Node displaying - Disconnected. Still passing data into flow though.

20

Just thought that I would mention that at the moment the problem seems to have gone away. I updated the Node-Red instance to 1.1 and so far the MQTT Node shows Connected after each 'Deploy'