# Understanding MQTT v5 retain options

**URL:** <https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118>\
**Category:** General\
**Tags:** mqtt\
**Created:** [31 March 2023 15:57 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118 "2023-03-31T15:57:44Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [31 March 2023 15:57 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/1 "2023-03-31T15:57:44Z")

</div>

Hi, I'm trying to fully understand this two option.

- keep retain flag of original publish
- retained message handling  
Can someone please summarize them or share a good guide to follow? Googling I haven't find a clear one, and even making several test it's not that clear to me

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/c/4c214b39a98acf99c80c7d46d22b73cebe13d053.png)  
thanks so mucj

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [31 March 2023 16:28 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/2 "2023-03-31T16:28:09Z")

</div>

Keep retain is as it describes. What you may not be aware of is that in MQTT version 3, the retain flag is cleared before sending the message. This option allows the incoming messages to KEEP the flag.

The send retained messages are also pretty much as described. The 1st option is send retained messages. The 2nd option is send retained for only new subscriptions and the 3rd option is don't send retained messages.  
What may make this clearer is this is about retained messages and therefore about what happens when the client (node-red) connects or subscribes to a topic. Re-read the options with that in mind.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [31 March 2023 18:34 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/3 "2023-03-31T18:34:45Z")

</div>

Thanks @Steve-Mcl

- Sorry but for the flag I don't understand it. If I make a test, no matter if it's checked or not, the received message always contains the property retain = true
- For the retain message handling, at the end these are the instruction for the broker? Or node-red anyway receives the retained messages, but based on this settings are either or not sent to the output?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [31 March 2023 21:07 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/4 "2023-03-31T21:07:41Z")

</div>

I'll see if I can find the demo flows I had when adding MQTT v5 support.

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [31 March 2023 23:02 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/5 "2023-03-31T23:02:11Z")

</div>

> [@Steve-Mcl](#):
>
> in MQTT version 3, the retain flag is cleared before sending the message

Not quite true… it is cleared if the message is passed directly to a subscriber (ie not retained on the broker). This is so the subscriber can tell if the message is live or not. This flag overrides that and just passes it through always.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [1 April 2023 11:58 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/6 "2023-04-01T11:58:13Z")

</div>

Thanks for the reminder Dave. It is amazing (and concerning) how much of the spec I have forgotten since doing the mqtt5 stuff.

@Lupin_III I'll not be up and running until Monday. I'll try to find those demo flows

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [1 April 2023 16:22 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/7 "2023-04-01T16:22:05Z")

</div>

Thanks so much to all.  
@Steve-Mcl It will help a lot. I've tried some more test even for the retained message handling, but I cannot see any difference. When node-red starts, or client disconnects and reconnects, or restarting the flow, it always get the retained messages. So, maybe the only difference could be for dynamic subscription that you can manage once node-red is running. But even in that case, if for whatever reason the client get disconnected and reconnected, it always gets the retained message.  
But maybe I've set the test in the wrong way.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [1 April 2023 16:35 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/8 "2023-04-01T16:35:08Z")

</div>

> [@Lupin\_III](#):
>
> When node-red starts, or client disconnects and reconnects, or restarting the flow, it always get the retained messages

Yes, it always gets the retained messages, it is the retained flag (msg.retained) that is affected. That can be used to identify whether the message is being sent because it is retained, or because a new value has just been published.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [5 April 2023 09:30 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/9 "2023-04-05T09:30:30Z")

</div>

Thanks @Colinn. Nothing, I always get the retained flag (msg.retained) as true. Probably the way I do the test. But conceptually I think I have understood it  
Yes a flow could help for sure to better understand it. Thanks so much

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [5 April 2023 13:05 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/10 "2023-04-05T13:05:36Z")

</div>

There does seem to be an issue here, or I too am misunderstanding it.  
I have an Out node publishing with the Retain flag set, and two In nodes, one with Keep set and one with it not set. As I understand it, on publishing a message then the subscriber should see msg.retain set according to the Keep flag, but it is set in both cases

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/7/97d61ac8b8f6aa93819975a9ba4b7418e6adfa1c.png)

```auto
[{"id":"310782544a09a97e","type":"mqtt out","z":"bdd7be38.d3b55","name":"","topic":"test/testretained","qos":"0","retain":"true","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"adf2dac3.b20fe8","x":520,"y":2120,"wires":[]},{"id":"53ea2ca73a91f2eb","type":"mqtt in","z":"bdd7be38.d3b55","name":"Don't keep retained flag","topic":"test/testretained","qos":"2","datatype":"auto-detect","broker":"adf2dac3.b20fe8","nl":false,"rap":false,"rh":0,"inputs":0,"x":240,"y":2180,"wires":[["9061500bbf32ed93"]]},{"id":"9061500bbf32ed93","type":"debug","z":"bdd7be38.d3b55","name":"debug 99","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":420,"y":2180,"wires":[]},{"id":"bab8ee4c7cda13cf","type":"inject","z":"bdd7be38.d3b55","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":2120,"wires":[["310782544a09a97e"]]},{"id":"1bbd8c1f2edfbeb9","type":"mqtt in","z":"bdd7be38.d3b55","name":"Keep retained flag","topic":"test/testretained","qos":"2","datatype":"auto-detect","broker":"adf2dac3.b20fe8","nl":false,"rap":true,"rh":0,"inputs":0,"x":230,"y":2240,"wires":[["860c3b9007cf85c4"]]},{"id":"860c3b9007cf85c4","type":"debug","z":"bdd7be38.d3b55","name":"debug 100","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":420,"y":2240,"wires":[]},{"id":"adf2dac3.b20fe8","type":"mqtt-broker","name":"localhost V5","broker":"localhost","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"tigger/LWT","birthQos":"0","birthPayload":"Online","birthMsg":{},"closeTopic":"tigger/LWT","closeQos":"0","closePayload":"Offline","closeMsg":{},"willTopic":"tigger.LWT","willQos":"0","willPayload":"Offline","willMsg":{},"userProps":"","sessionExpiry":""}]

```

I have also had it lock into the situation where neither of the received messages have that flag set, but I haven't worked out what triggers that. A node red restart fixed that.

Node red 3.0.2 with nodejs 18

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [5 April 2023 16:44 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/11 "2023-04-05T16:44:00Z")

</div>

Further to the above, I think it is having two In nodes, one keeping and one not that is causing the problem

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [5 April 2023 20:24 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/12 "2023-04-05T20:24:50Z")

</div>

@Lupin_III If you import my flow, but delete one pair of MQTT In and debug nodes, does it work for you? You should see the received Retain flag change as you change the setting in the MQTT In node.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [8 April 2023 18:21 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/13 "2023-04-08T18:21:52Z")

</div>

Thanks @Colin Yes, it works and it is probably why I didn't see any dfference with my test.  
So in order to summirize it, even for the other readers, the "Keep retain flag of original publish" works in this way, as even @dceejay said:

- if it's NOT flagged and the message is received on the fly, then retain: false
- if it's NOT flagged and the message is received on connection, then retain: true (doesn't matter if it was already received before the client got disconnected)
- if it's flagged then always retain: true

Now I'm still having some doubts on the "retained message handling" option. I have tested all possible combination but I always get the message and its retain property as true, a part from the "Do not send" option, whne I don't receive the message at all, as espected.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [8 April 2023 18:49 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/14 "2023-04-08T18:49:02Z")

</div>

This is probably a nicer explanation: [MQTT 5.0 Features | Retain Message | by EMQ Technologies | Medium](https://emqx.medium.com/mqtt-5-0-features-retain-message-104698b070d1)

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [8 April 2023 18:50 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/15 "2023-04-08T18:50:31Z")

</div>

Here is another from Steve's internet site:

> #### Retained Message Control
> 
> Retained messages still work as in 3.1.1 but subscribe options have been added to control what the client receives.:
> 
> 0 -Send retained messages when client subscribes. Same as in MQTT 3.1.1  
> 1 -Send retained messages when client subscribes. if the subscription does not currently exist.  
> 2 – Don’t send retained messages when client subscribes.

Key part being "when client subscribes"

So ordinarily you won't see this as node-red subscribes at point of start up. To see this in action you will need to trigger subscribe/unsubscribe actions dynamically and watch the FULL msg coming from the MQTT in node.

Sorry but I couldn't find my Dev/test flows (was over 1y since I did this work)

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [8 April 2023 20:52 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/16 "2023-04-08T20:52:57Z")

</div>

> [@Steve-Mcl](#):
>
> So ordinarily you won't see this as node-red subscribes at point of start up.

You can see it more easily by configuring the debug node to output to the console, then monitor the log and do a full deploy (or restart flows).

@Steve-Mcl did you try my test with two In nodes with different settings, and confirm that it does not work in this case if the two nodes have the flag configured differently?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [8 April 2023 20:58 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/17 "2023-04-08T20:58:16Z")

</div>

> [@Lupin\_III](#):
>
> Thanks @Colin Yes, it works and it is probably why I didn't see any dfference with my test.

> [@Lupin\_III](#):
>
> Now I'm still having some doubts on the "retained message handling" option. I have tested all possible combination but I always get the message and its retain property as true

Those two sentences appear to be in contradiction with each other.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [8 April 2023 22:10 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/18 "2023-04-08T22:10:42Z")

</div>

> [@Colin](#):
>
> @Steve-Mcl did you try my test with two In nodes with different settings, and confirm that it does not work in this case if the two nodes have the flag configured differently

Not yet. Not had the time Colin.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 April 2023 07:46 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/19 "2023-04-09T07:46:51Z")

</div>

I suspect the two subscriptions, with and without the flag, are merged into one subscription to the broker, instead of being kept separate.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [9 April 2023 08:00 UTC](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118/20 "2023-04-09T08:00:30Z")

</div>

> [@Colin](#):
>
> suspect the two subscriptions, with and without the flag, are merged into one subscription to the broker, instead of being kept separate.

That is a very strong possibility. And the first place I will look.

Would you be kind enough to raise an issue with your demo flow (or a link back) please?

[Next page](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118.md?page=2)