I have trouble testing the persistence of MQTT messages with Node-RED.
I have a Node-RED 2.1.6 server, in which a "mqtt in" node subscribes to a topic.
I would like that if my Node-RED server goes down, I can retrieve the messages from the topic I subscribed to when the server starts again.
For that, I subscribe with the "clean start" option disabled, and QoS2. All is good theoretically I already tested this process outside Node-RED.
However, when I stop the Node-RED server, there is an unsubscription to the topic before the disconnection, so the Mosquitto broker can't keep in memory that it has to keep the messages for the ID RED-DEV client.
Here are the logs from Mosquitto after a Node-RED service stop :
2022-01-18T15:04:29: Received UNSUBSCRIBE from RED-DEV
2022-01-18T15:04:29: Sending UNSUBACK to RED-DEV
2022-01-18T15:04:29: Received DISCONNECT from RED-DEV
2022-01-18T15:04:29: Client RED-DEV disconnected.
In what situation can we then recover the messages sent while Node-RED was not available? Is it possible to override the initial behavior to only disconnect the client and not unsubscribe?
I don't use the "retain" feature because I read that it only retains the last message sent to a topic, and I need to retrieve all messages.
I have tested persistence with other techno (Flutter), when a problem occurs, the client (in same configuration with no clean start and QoS2 of course) is disconnected without unsubscribing first, so the broker knows it has to retain all the messages until it returns.
If you need the history, you don't want MQTT but a database. Many of us use InfluxDB which is a timeseries database well suited for recording ongoing sensor and other events.
MQTT is meant to be lightweight and only meant to retain the last value when using the retain flag. It acts as an event hub letting you have subscriptions to topics and get notified when something changes and the new value.
You can set up persistent storage in Node-RED as well, have a rummage through the docs. But this is also ill-suited to storing historic data. A database is what you need for that.
For MQTT, you can choose whether you have a "clean start" so that when a client reconnects it doesn't get all of the retained and subscribed topics. However, this is not often very useful and not really of any use if you have restarted Node-RED completely.
If you need something more, have a look at the MQTT v5 settings as these give you more options & they may help you get what you want.
However, you still only ever have the last value in MQTT, anything more requires a database.
From your answer, I understand that Mosquitto is the one that offers the 1+ message retention feature, and that it is not available with Node-RED by design.
I will look into database solutions, thank you for your answers!
is this issue resloved?
Im also getting the same issue ,
when im disabling the flow in which my mqtt client subscribe (with persistance and Qos level 2 connection) to a topic.
when the flow is enabled im receiving messages properly no issue here,but when im disabling the flow and then connecting after sometime im not getting the messages that were sent during the disabled time period ,but it should receive right.
i have checked same test situation in MQTTX tool and in that im able to get the messages when i come online again ,but in node-red it is not .
as said @Keylies the node-red is unsubscribing the topic before it is getting disabled. so is there any way to prevent it from unsubscribing?
It may or may not get done in time for node-red V3 (Planned for April) - it depends on achieving other assigned tasks (or if a community member picks it up)
