Issues in persisting messages in MQTT Broker using NodeRed MQTT Nodes

I am not seeing the queueing. I configured a node-red subscriber on PC B subscribed to a broker on PC A with the publisher on PC A. After checking everything was working I pulled the ethernet cable on B and waited till the LWT Offline message was sent by the broker (monitored on A). I then injected some data to the broker on A, then plugged the cable back in again. Nothing was received by B.

It is correct that if the cable is only unplugged for a short time, so that B is disconnected but still Online then the broker does queue the messages, but once B is Offline then the broker does not queue messages for it.

Not tried this out since Nick pointed out my error but have been reading up on it.

Are you using a standard Mosquitto broker on PC A? (I'm assuming yes but just checking)

is your subscription set to QOS 1 or 2

Have you set a fixed Client ID in the Node-RED broker config and also un-ticked clean session?

Simon

Hmm, no I hadn't, and now I am astonished. How long is the broker going to hang onto the messages in case the client reconnects? 10 minutes or 10 years? Is it going to hang onto hundreds, possibly thousands, of messages if it needs to?

It depends how you have configured the broker.

Mosquitto, for example, will queue up 100 messages by default per client and will discard the oldest messages as new ones are queued. It also has an option to queue qos 0 (turned off by default). See the max_queued_messages, max_queued_bytes and queue_qos0_messages options in the docs - mosquitto.conf man page | Eclipse Mosquitto

Well I have been RTFM'd with full justification. Thanks, I don't remember ever noticing that.

Messages are not queued if it is the publisher that is disconnected from the broker, only if it is the subscriber that is disconnected. That doesn't surprise me.

1 Like

yes - queuing at the publisher side is down to the implementation of the application

:slight_smile:

thanks @knolleary, Please guide me on the right approach to test this.

Use two PCs in the way I described in my post. Then you can disconnect the subscribed client by pulling the network cable.

What can we do if it needs to keep the queue on the publisher's side?

Example 1: Node-red mounted on a vehicle that eventually loses connection

Example 2: Sensor data during an electrical blackout (the switch loses power and the network is lost)

How do you recommend taking these scenarios into account?

thx!!

For such requirements I write the data to an sqlite db. Then I have a separate flow that monitors the db and if there is anything then it attempts to send it. If it succeeds then it removes it from the db and does the same for any remaining records. If it can't send it then it leaves them in the db and has another go a bit later.

But in this way shipping is not guaranteed in order. I have a FIFO queue where I write 1 message. In the case that there is connection, it sent the entire queue in order. Thus, if while I am sending a new message arrives (or several) it is put at the end of the queue and sent in the correct order.

Regards!!

The order is guaranteed provided you take them out in the same order you put them in.
[Edit]
The database is a FIFO when used like this. For me it is important that I do not lose messages over a power down or restart which is guaranteed with a database. You could do it with persistent context storage the same way.

If you already do it with a FIFO why are you asking for help?

I wanted to know what was the right way to do it or if there was some better way. I also wanted to say it to help anyone who needs to visit this thread over the coming years

OK, I think a FIFO is certainly the way to do it. Which sort of FIFO doesn't really matter provided it meets your requirements. One implementation detail is not to bother putting in any extra logic to test whether the FIFO is empty when a new message comes along. It is not worth the effort and the logic gets surprisingly complex. Just shove the new in the FIFO and then you can trigger the polling if appropriate which will immediately send it and take it out.

1 Like

Also send each one before removing from the FIFO, and make sure that the recipient can cope with messages being repeated as, if the sender is reset after sending but before removing, then that message will be sent again later.

1 Like

hi
is this unsubscribing issue resloved?
and regarding disabling the flow will ir unsubscribe? or will it just disonnect ?
and when we stop the nodered itself from console will the client gets unsubsribe and then disconnect or just disconnects?
thanks in advance for your time

Please stick to one thread.