# MQTT5 retained message

**URL:** <https://discourse.nodered.org/t/mqtt5-retained-message/83494>\
**Category:** General\
**Tags:** mqtt\
**Created:** [7 December 2023 11:28 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494 "2023-12-07T11:28:03Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 11:28 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/1 "2023-12-07T11:28:03Z")

</div>

Hi, my question almost close to these topics  
[Understanding MQTT v5 retain options](https://discourse.nodered.org/t/understanding-mqtt-v5-retain-options/77118) and [Ignore retained MQTT messages at restart](https://discourse.nodered.org/t/ignore-retained-mqtt-messages-at-restart/45458)  
but I'm still not able to get what I need without awful workarounds.  
My problem is this:

- MQTT5
- MQTT server, publisher and subscriber on 3 different machines (debians)
- messages QoS 2 and retained flag.  
I need to get and use this messages just once, not more. The problem is that doesn't matter the configuration I use regarding clean start and retained flag handled, at reboot I always get the last message with the retained flag set as true. I cannot simply ignore the first message because the subscriber could be off while the publisher send it, and in this case I have to process the message. I know I could use the timestamp to detect the last message, but I would like to use just the MQTT option, avoiding to add the timestamp to each single message

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 12:15 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/2 "2023-12-07T12:15:04Z")

</div>

If you don't want the retained message, then make the topic not retained. Then you will only get a message at the time it is published.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 12:17 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/3 "2023-12-07T12:17:27Z")

</div>

No, I need the rained messge. As you see, otherwise if the subscriber is off, it wan't get the message pubblished in that time

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 12:20 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/4 "2023-12-07T12:20:38Z")

</div>

I see what you mean now. I can't think of any option other than adding a timestamp (or other unique id) to the message, so you can ignore duplicates at the receiver end.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 12:46 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/5 "2023-12-07T12:46:01Z")

</div>

Yes, and save it in persistent context. By the way at the moment (since this saving must be immediatly in order to avoid to process two times the same message) I use this configuration in the settings,js

```auto
file: {
      module: 'localfilesystem',
      config: {
            cache: false
      },
}

```

and this in a function node:  
`flow.set("mqttID", msg.payload.timestamp), "file", function(err){if(err){node.error(err, msg)}});`

Is that correct or there is a simpler way?

---

<div class="post-metadata">

**Author:** ![jbudd](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jbudd](https://discourse.nodered.org/u/jbudd)\
**Post date:** [7 December 2023 12:59 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/6 "2023-12-07T12:59:53Z")

</div>

> [@Lupin\_III](#):
>
> I need to get and use this messages just once, not more. ...  
> ... because the subscriber could be off while the publisher send it, and in this case I have to process the message.

This sounds like you are hoping the broker will queue messages up until your subscriber comes back online.

My understanding is that it only retains the last message received on any topic, so if your subscriber is down it can still miss messages regardless of the Retain flag from the publisher.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 13:05 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/8 "2023-12-07T13:05:51Z")

</div>

Yes, i't ok missing the intermediate messages in this case

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 13:44 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/9 "2023-12-07T13:44:22Z")

</div>

> [@Lupin\_III](#):
>
> flow.set("mqttID", msg.payload.timestamp), "file", function(err){if(err){node.error(err, msg)}})

Personally I would use node context in the function rather than flow. Always keep the scope of data as local as possible.

Bear in mind, though, that this method is not infallible. Node red writes the context out to file at a configurable interval, or on shutdown. That means that a normal shutdown of node-red will guarantee that the most recent value is saved in the file, however if node-red crashes or there is a power fail then the latest value may not yet have been written to disc.

As a matter of interest, what operation is being performed at the receiving end that must not be repeated?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 14:17 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/10 "2023-12-07T14:17:28Z")

</div>

Also, obviously, before setting it in file storage you need to get it and compare it with the one in the current message, and discard the current message if they are the same.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 16:11 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/11 "2023-12-07T16:11:35Z")

</div>

> [@Colin](#):
>
> Personally I would use node context in the function rather than flow. Always keep the scope of data as local as possible.

Yesm you're rigth

> [@Colin](#):
>
> Bear in mind, though, that this method is not infallible. Node red write .....

I thought the configuration I posted, would have written immediately on file. It's not so?

> [@Colin](#):
>
> As a matter of interest, what operation is being performed at the receiving end that must not be repeated?

it depends on the messages, but for example it could even reboot the system.

> [@Colin](#):
>
> Also, obviously, before setting....

Oh yes, sure, I was more concerned about passing the idea than the all logic. You're right

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 16:20 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/12 "2023-12-07T16:20:58Z")

</div>

> [@Lupin\_III](#):
>
> I thought the configuration I posted, would have written immediately on file. It's not so?

No, as in[the docs](https://nodered.org/docs/user-guide/context):  
" it caches the values in memory and only writes them out to the file-system every 30 seconds."

That is to prevent it spending all its time saving stuff to disc if a context var is updated rapidly. Imagine what would happen if the context were set 1000 times a second if it wrote the file every time.

The time is configurable in settings.js.

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 16:49 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/13 "2023-12-07T16:49:29Z")

</div>

Looking here [Local Filesystem Context Store : Node-RED](https://nodered.org/docs/api/context/store/localfilesystem#options) I understand that flushInterval works only if cache is set to true. Setting it to false, it writes directly on disk.  
If it's not like this, how can I set it to write the changed context after 1 seconds (I means as soon as possible) on disk?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [7 December 2023 16:55 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/14 "2023-12-07T16:55:48Z")

</div>

> [@Lupin\_III](#):
>
> flushInterval works only if cache is set to true. Setting it to false, it writes directly on disk.

Oh you are right, I did not know you could do that. Be careful how often you write to any context vars using that store.

> [@Lupin\_III](#):
>
> how can I set it to write the changed context after 1 seconds

That is on the next line under the cache definition you pointed to 🙂

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [7 December 2023 17:09 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/15 "2023-12-07T17:09:45Z")

</div>

Thanks. So just to be sure I'll recap it:

- this writes immediatly on disk

```auto
file: {
      module: 'localfilesystem',
      config: {
            cache: false
      },
}

```

and in a function node:  
context.set("mqttID", msg.payload.timestamp), "file", function(err){if(err){node.error(err, msg)}});

- While this one writes on disk each second

```auto
file: {
      module: 'localfilesystem',
      config: {
            cache: true,
            flushInterval: '1'
      },
}

```

and in a function node, as above,  
context.set("mqttID", msg.payload.timestamp), "file", function(err){if(err){node.error(err, msg)}});

---

<div class="post-metadata">

**Author:** ![chrisn-au](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/chrisn-au/32/25504_2.png) [@chrisn-au](https://discourse.nodered.org/u/chrisn-au)\
**Post date:** [7 December 2023 19:22 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/16 "2023-12-07T19:22:13Z")

</div>

I had a similar but slightly different requirement. As I already have a redis instance in my environment I used redis to store the message ts - all works well however the standard context storage to disk would be the default soon

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [9 December 2023 10:24 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/17 "2023-12-09T10:24:17Z")

</div>

I didn't know about redis, I'll give it a look even if I think that node-red context is ok for my use, Just I need to understand a bit better how to directly write on disk as soon as possible, since these messages are not so often

---

<div class="post-metadata">

**Author:** ![zenofmud](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/zenofmud/32/316_2.png) [@zenofmud](https://discourse.nodered.org/u/zenofmud)\
**Post date:** [9 December 2023 10:42 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/18 "2023-12-09T10:42:41Z")

</div>

> [@Lupin\_III](#):
>
> Just I need to understand a bit better how to directly write on disk as soon as possible, since these messages are not so often

This might help you: [Local Filesystem Context Store : Node-RED](https://nodered.org/docs/api/context/store/localfilesystem)

---

<div class="post-metadata">

**Author:** ![Lupin\_III](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/lupin_iii/32/67650_2.png) [@Lupin\_III](https://discourse.nodered.org/u/Lupin_III)\
**Post date:** [9 December 2023 11:48 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/19 "2023-12-09T11:48:08Z")

</div>

Yes, I even posted the same link.  
Is this my recap correct? [MQTT5 retained message - #15 by Lupin\_III](https://discourse.nodered.org/t/mqtt5-retained-message/83494/15)

---

<div class="post-metadata">

**Author:** ![zenofmud](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/zenofmud/32/316_2.png) [@zenofmud](https://discourse.nodered.org/u/zenofmud)\
**Post date:** [9 December 2023 11:56 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/20 "2023-12-09T11:56:47Z")

</div>

I think the flushInterval should to be a number and you have it set to a string:

```auto
file: {
      module: 'localfilesystem',
      config: {
            cache: true,
            flushInterval: '1'
      },
}

```

Try changing it to

```auto
file: {
      module: 'localfilesystem',
      config: {
            cache: true,
            flushInterval: 1
      },
}

```

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 December 2023 13:17 UTC](https://discourse.nodered.org/t/mqtt5-retained-message/83494/21 "2023-12-09T13:17:47Z")

</div>

> [@Lupin\_III](#):
>
> So just to be sure I'll recap it:
> 
> - this writes immediatly on disk

> [@Lupin\_III](#):
>
> While this one writes on disk each second

Well you tell us. It is easy enough to check by injecting into a change node setting a global var while watching .node-red/context/global.

[Next page](https://discourse.nodered.org/t/mqtt5-retained-message/83494.md?page=2)