Similar to this thread, but, I've read it all and tried a lot of the things, afraid I'm still stuck.
(I'd really like to avoid using a database to solve this, I'm a pretty average programmer, my applications are simple and I like the easy nature of NodeRed)
What I'd like to do:
If my NodeRed client goes down temporarily e.g. due to laptop going to sleep or being unplugged. I'd liked it to pick up the messages it missed from the MQTT broker whilst it was offline.
My test application is a simple temperature sensor sending three temperatures to a dashboard with a gauge & a chart. I'd like the chart to keep it's old data if NodeRed goes down.
I have already done the following:
Subscriber (My Windows Laptop)
MQTT in node:
Use Clean Session - set to false (unticked)
Enable secure SSL/TLS Connection.
Set the subscriber QoS to 1
Used a change node to save the data from the chart output to my filestore then an inject node to put it back once on startup. I can see old data but only from times the system was online. The chart just flatlines the gaps inbetween.
Publishing Client Side (Pi Pico):
Use Clean Session - set to false
client.connect(clean_session=False)
Set the publish of the message QoS to 1
client.publish(temperatureIdBytes, temperatureReadingBytes, qos=1)
Perhaps people can give me some ideas of how to debug this problem myself or if its obvious I'm missing something?
Debugging tried:
On HiveHQ I can see the messages coming into the broker with a QoS of 1 via the webclient. So I know they're there.
But when I close my CMD window to fake the NodeRed going down or disconnect the wifi temporarily I still get flat spots on the my chart.
Queuing of QoS 1 and 2 messages
All messages sent with QoS 1 and 2 are queued for offline clients until the client is available again. However, this queuing is only possible if the client has a persistent session.
@ickidebary when you close the CMD, I believe node-red is shutdown more cleanly than you suspect. Try disabling or unplugging network instead.
Are you using the free test broker there? If so then it may not support buffering of data for offline clients. Try it with Mosquitto installed locally.
How, exactly are you running Node-RED. Please explain your method of running.
OS please. Linux, Windows, etc...
Node-RED startup. Startup Script, Scheduled Task,
Where does the data come from? Node-RED itself? or other source?
Please consider running under the system. A Service. This will allow your Node-RED to run even while it is "Asleep", "Locked", or "Logged off".
This should be the proper approach to a system which you just need to have the PC, Laptop turned on and running to provide host operations.
This does not solve the problem of PC shutting down for other reasons, but it does address most of the interruptions you may be faced with.
It does sound like you need another system to cross monitor the states of each. Then do something when the state is deemed "down". You can also apply more sophisticated MQTT systems which can store information on its own. I have no idea what your infrastructure is.
What it looks like you need is a server or cloud based MQTT Broker which can log its values to a database as it recives messages. Then you can log the events which are successful to each client. This gives you the missing messages and then a means you can program to recover them.
There is not an easy breezy solution and it will take knowing the messages which you missed in order to process them.
Your solution won't be easy, There is MQTT Software at reasonable pricing which can do this. You can also work with the company to custom program your solution for you.
bevywise.com is a fantastic resource to start with. Out-of-the-box it will record all of your MQTT Messages to the database. All you then need to do is query the database from your last "known" point to process all messages. This of course would work if your host system is not interrupted. If your information provider is interrupted, there is no information to log.
There are also other companies which offer similar solutions.
As well as a consistent user name (so it knows you are the same "thing" reconnecting), you need to have "clean session" not selected - otherwise by default it will do as it says and start you with a clean session...
