# How is your experience on Node-RED, MQTT and Linux for 24/7 data logger?

**URL:** <https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471>\
**Category:** General\
**Created:** [26 November 2024 09:52 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471 "2024-11-26T09:52:52Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Leo\_3](https://avatars.discourse-cdn.com/v4/letter/l/f1d935/32.png) [@Leo\_3](https://discourse.nodered.org/u/Leo_3)\
**Post date:** [26 November 2024 09:52 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/1 "2024-11-26T09:52:52Z")

</div>

Which MQTT broker (self hosted vs. Mosquitto or others)?

Which Linux? Command line or with Desktop GUI?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 November 2024 10:28 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/2 "2024-11-26T10:28:43Z")

</div>

> [@Leo\_3](#):
>
> Which Linux? Command line or with Desktop GUI?

See [Best Linux Version to Node-Red](https://discourse.nodered.org/t/best-linux-version-to-node-red/93449) for a discussion on this. I am surprised that you did not see that post, presumably you did search the forum before asking the question.  
Whether you want a gui or command line install is up to how you want to use it.

> [@Leo\_3](#):
>
> Which MQTT broker (self hosted vs. Mosquitto or others)?

Mosquitto. Self hosted.

---

<div class="post-metadata">

**Author:** ![Leo\_3](https://avatars.discourse-cdn.com/v4/letter/l/f1d935/32.png) [@Leo\_3](https://discourse.nodered.org/u/Leo_3)\
**Post date:** [26 November 2024 12:17 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/3 "2024-11-26T12:17:34Z")

</div>

Many thanks

---

<div class="post-metadata">

**Author:** ![greengolfer](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/greengolfer/32/28276_2.png) [@greengolfer](https://discourse.nodered.org/u/greengolfer)\
**Post date:** [26 November 2024 12:57 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/4 "2024-11-26T12:57:58Z")

</div>

Which linux has been addressed many times. I won't come back to it.

If you need 24/7 operations, then having a redundant broker is (for me) key.  
Mosquitto is an excellent choice, unfortunately, the free and open source version can't be clustered.  
For clustered broker, EMQX as free and open source is an option.  
RabbitMQ for MQTT only is very complex.  
VerneMQ is a very good choice, but, you'd have to pay for a licence.  
In front of those, you will need a (redundant) load balancer.  
HAProxy is a good option.

My 2 cents...

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 November 2024 13:22 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/5 "2024-11-26T13:22:04Z")

</div>

> [@greengolfer](#):
>
> If you need 24/7 operations, then having a redundant broker is (for me) key.

Can you explain why? What is the benefit?

---

<div class="post-metadata">

**Author:** ![greengolfer](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/greengolfer/32/28276_2.png) [@greengolfer](https://discourse.nodered.org/u/greengolfer)\
**Post date:** [26 November 2024 13:37 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/6 "2024-11-26T13:37:45Z")

</div>

Well, your unique instance of broker may fail (the software, the host were it runs...), it is a SPOF.  
To avoid that, having redundancy active/standby or else is key.  
For MQTT, the clustering is a way to address 24/7 operations.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 November 2024 13:55 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/7 "2024-11-26T13:55:56Z")

</div>

But if your node-red has failed then what is the point of having a standby mqtt broker? So that is also a single point of failure, as is your influxdb service.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [26 November 2024 14:17 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/8 "2024-11-26T14:17:24Z")

</div>

I would agree that if you really want guaranteed uptimes, you need to think about end-to-end resilience.

But as usual, this is totally dependent on the background requirements - which haven't been offered to us as yet.

So we really need to ask for more background info before recommending something.

---

<div class="post-metadata">

**Author:** ![greengolfer](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/greengolfer/32/28276_2.png) [@greengolfer](https://discourse.nodered.org/u/greengolfer)\
**Post date:** [26 November 2024 14:43 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/9 "2024-11-26T14:43:02Z")

</div>

In my app, I have an MQTT cluster (VerneMQ), Redis in cluster, and NodeRed runs in docker and I have two instances running in actve/passive mode.

I agree with Julian, all is very context dependant...

---

<div class="post-metadata">

**Author:** ![Becker](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@Becker](https://discourse.nodered.org/u/Becker)\
**Post date:** [28 November 2024 20:29 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/10 "2024-11-28T20:29:25Z")

</div>

> [@Leo\_3](#):
>
> Which MQTT broker (self hosted vs. Mosquitto or others)?

[FlashMQ](https://www.flashmq.org/download/) - better than Mosquitto

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [29 November 2024 07:06 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/11 "2024-11-29T07:06:36Z")

</div>

Depending on your needs.

But OP's subject is 'data logger' and that is not what MQTT does.

---

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [29 November 2024 08:00 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/12 "2024-11-29T08:00:18Z")

</div>

Ok, I have a story which needed a log to help me _win_ the battle.

My IP would change daily anywhere from twice to eight times.

I rang my _PROVIDER_ and they tried to dismiss my claims.

I sent them my logs. 4 years of data of my IP address and the UPLINK status.  
Logged every 20 seconds for 4 years.

Yes, there were holes in there now and then. You can't really work around power losses.

Alas my _solution_ is to log the data locally.  
Though I do use MQTT to share information between my `3` machines.

I found a lot of problems in code I wrote too - though now I digress. Sorry.

I only have ONE MQTT broker and it worked for my needs.  
Though I do agree that TWO may be better.  
Though if the network goes down, how ever brokers you have: they won't work.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [29 November 2024 09:34 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/13 "2024-11-29T09:34:49Z")

</div>

> [@Becker](#):
>
> better than Mosquitto

I'm afraid that needs some explanation if you want to recommend something other than a tool that has been battle tested by many over a very long period. They don't even include Mosquitto in their benchmark comparison. It also doesn't seem to be open source and only provides Debian Linux versions. It also appears to be just a couple of years old.

Mosquitto is open source, backed by the Eclipse Foundation, Has GitHub releases going back to 2014 and is open source. And has the benefit that a lot of people on the forum know it well.

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [29 November 2024 10:13 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/14 "2024-11-29T10:13:17Z")

</div>

> [@greengolfer](#):
>
> RabbitMQ for MQTT only is very complex.  
> VerneMQ is a very good choice, but, you'd have to pay for a licence.

I was thinking about this recently and thought suddenly of [Kafka](https://kafka.apache.org/). Kafka has the ability to store messages that are sent by a producer. Consumers can rewind and reload old messages or listen to the current messages coming in.

The basic idea is that a producer just does fire&forget and Kafka does the storage of messages and sending out of messages to consumers. This allows the producer not to worry about message delivery or loss - Kafka ensures the consistence and preservation of data.

Data loss is avoided because Kafka keeps a store of the last X days/weeks/months. Any new consumer can begin at the beginning of that store. The Kafka protocol ensures that messages are indexed in order so that rewind is possible.

Question is does MQTT/Mosquitto offer this feature too?

---

<div class="post-metadata">

**Author:** ![Becker](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@Becker](https://discourse.nodered.org/u/Becker)\
**Post date:** [29 November 2024 11:02 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/15 "2024-11-29T11:02:46Z")

</div>

I came across FlashMQ through Victron's OS (Venus), which replaced it some time ago. I've also had a few contacts with the developer, nice man, very helpful.  
I have now switched all brokers at home from mosquitto to flashMQ

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [29 November 2024 11:10 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/16 "2024-11-29T11:10:27Z")

</div>

> [@gregorius](#):
>
> Question is does MQTT/Mosquitto offer this feature too?

MQTT v3 has some basic controls around guaranteed delivery. MQTT v5 expands on that considerably and adds capabilities for message receipts as well. But it is an MQ service and so not expected to record history and you should attach suitable, MQTT enabled data recording if you need that. Kafka has significantly higher requirements and overheads than an MQTT Broker. There are other MQ systems that handle larger message queues and multi-step receipts and other features. But again, these tend to be enterprise scale tools with appropriate complexity and resource requirements.

The purpose of MQTT is to be lightweight and simple. Allowing brokers to run just about anywhere and still maintain high throughput rates.

---

<div class="post-metadata">

**Author:** ![greengolfer](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/greengolfer/32/28276_2.png) [@greengolfer](https://discourse.nodered.org/u/greengolfer)\
**Post date:** [29 November 2024 13:25 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/17 "2024-11-29T13:25:37Z")

</div>

I confirm what Julian wrote. If you use QOS = 2, then it gives some confidence that messages sent will be received. If the client (subscriber) is offline for a while, then with QOS = 2, messages kept will be sent. However, in all broker implementation, there is a limit on the number of messages that are kept "in the fridge" when the client is away. This is why I wrote "some".  
MQTT is a data exchange protocol. Not a database.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [27 February 2025 13:25 UTC](https://discourse.nodered.org/t/how-is-your-experience-on-node-red-mqtt-and-linux-for-24-7-data-logger/93471/18 "2025-02-27T13:25:44Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.