Issues in persisting messages in MQTT Broker using NodeRed MQTT Nodes


#1

Hi
I have an use case which requires persisting the MQTT messages in Broker whenever a subscriber goes offline. When the subscriber reconnects all the messages published when it was offline should be delivered.

To achieve this I am using the Node Red MQTT Out and In nodes for publish subscribe. My Node Red Version is 0.17.5 . I am using Mosquitto Broker Version 1.4.9.

For persisting messages,

At the subscriber side:
I have provided valid entries for Topic , Qos(1) ,Server, Port ,Keep Alive(60s),Client Id.Following entries are kept unchecked
1.Use Clean Session 2. Enable secure SSL/TLS Connection.
The option 'Use Legacy MQTT 3.1' support is kept checked.

At the Publisher side:
Similar settings are replicated. Client Id is automatically set to the client id of subscriber .Retain option is not chosen.

I triggered timestamp values at publisher side by keeping subscriber connected to broker in a separate flow .The subscriber was able to receive messages. Then I disabled the flow with subscriber. Did a redeploy. Triggered the time stamp messages again from publisher side. Reenabled the flow with subscriber and redeployed. The subscriber was not able to retrieve the messages published when it was disabled.

I was able to reproduce persistence using Mosquitto Pub and Sub clients with the following command line options but I am not able to achieve the same using Node Red. I could achieve the message retrieval/subscription in Node Red only when the subscriber is online at the time of publishing.

mosquitto_sub -h 127.0.0.1 --disable-clean-session -t demoone -q 1 -i demooneclient

mosquitto_pub -h 127.0.0.1 -t demoone -m "Hello" -q 1.

Please revert.

Thanks

Surya


#2


Got it wrong - ignore
I only use clean-sessions so didn't know about the facility of stored messages when using QOS 1 or 2


#3

As @cymplecy MQTT does not itself support queueing of messages when the client cannot connect to the server. You would have to implement something yourself to accomplish this. You could for example push the messages into an array or sqlite db on the client as they become available and then send them to MQTT with an incrementing id or timestamp when the server becomes available. Then in the receiving client as each one is received send back a handshake saying that one has been received so the sender can drop that one off the list waiting to be sent. Obviously there are all sorts of edge cases to consider when designing this. A bit of googling might reveal that someone has designed something like this already.


#4

FYI - current version of NR is 0.19.5


#5

Sorry Chaps, you're wrong here. MQTT does support queuing up QoS 1 and 2 messages whilst a client is offline.

  1. connect client with Clean Session set to False
  2. subscribe to a topic at qos 1 or 2
  3. disconnect

(messages continue to arrive on the subscribed topic)

  1. Reconnect with the same ID - Clean Session still set to false
  2. subscribe to a topic

You will then receive any qos 1 or 2 messages that were published whilst offline. QoS 0 messages are not queued.


@suryas I can recreate what you describe. Having added some debug in the node's code, I can see the node is receiving the messages as expected, but it happens so quickly on startup, that the later nodes in the flow may not be ready to receive them so they don't get handled. Please can you raise an issue on github.


#6

@suryas actually, I think the issue is a bit more subtle and related to the enable/disable of flows.

In my testing, I was starting/stopping Node-RED and I definitely see the messages arrive when I restart NR (that were published from mos_pub). When I enable/disable the tab with the subscriber on, I don't see the messages arrive. So definitely an issue, just not sure what it is yet...


#7

@suryas I have found the root cause. When an MQTT In node is 'closed' - ie when the flows are stopped - the node is explicitly unsubscribing from the topic. This means no messages will be stored in the broker for it.

After the node was originally written, we have added the ability for the node to know if it is being closed because it has been removed, or closed because the flow is stopping. If the node is removed, then it should still unsubscribe as it won't be coming back. If the node is just being stopped, then it should not unsubscribe - that's a fix we'll apply for the next release.

However, the method you're using to test the scenario will not benefit from the fix. Disabling the flow appears to the runtime as if the node is being removed. In that case the node will still unsubscribe to tidy up the resources it used. That is the right behaviour and not one we'll change.


#8

I am not seeing the queueing. I configured a node-red subscriber on PC B subscribed to a broker on PC A with the publisher on PC A. After checking everything was working I pulled the ethernet cable on B and waited till the LWT Offline message was sent by the broker (monitored on A). I then injected some data to the broker on A, then plugged the cable back in again. Nothing was received by B.

It is correct that if the cable is only unplugged for a short time, so that B is disconnected but still Online then the broker does queue the messages, but once B is Offline then the broker does not queue messages for it.


#9

Not tried this out since Nick pointed out my error but have been reading up on it.

Are you using a standard Mosquitto broker on PC A? (I'm assuming yes but just checking)

is your subscription set to QOS 1 or 2

Have you set a fixed Client ID in the Node-RED broker config and also un-ticked clean session?

Simon


#10

Hmm, no I hadn't, and now I am astonished. How long is the broker going to hang onto the messages in case the client reconnects? 10 minutes or 10 years? Is it going to hang onto hundreds, possibly thousands, of messages if it needs to?


#11

It depends how you have configured the broker.

Mosquitto, for example, will queue up 100 messages by default per client and will discard the oldest messages as new ones are queued. It also has an option to queue qos 0 (turned off by default). See the max_queued_messages, max_queued_bytes and queue_qos0_messages options in the docs - https://mosquitto.org/man/mosquitto-conf-5.html


#12

Well I have been RTFM'd with full justification. Thanks, I don't remember ever noticing that.

Messages are not queued if it is the publisher that is disconnected from the broker, only if it is the subscriber that is disconnected. That doesn't surprise me.


#13

yes - queuing at the publisher side is down to the implementation of the application


#14

:slight_smile:


#15

thanks @knolleary, Please guide me on the right approach to test this.


#16

Use two PCs in the way I described in my post. Then you can disconnect the subscribed client by pulling the network cable.