Easiest way to store data and plot graphs and alarms

Hi. I'm working on a project which consists of 3 raspberry pi's collecting data from different points in the field. These data collected to node-red are sent to a fourth raspberry pi by mqtt broker mosquitto (I'm thinking about using linux as the OS installed on the RPIs), which should store all data that are sent at regular short intervals (2 seconds to half an hour, depending on the variable). This fourth RPI not only have to store data from all the time is collected, but it should also design graphs for analyses, set alarms based on that, and front-end.

Based ond that I'm considering about using InfluxDB + Grafana. But from what I've searched over the internet and forum, I saw other options like MySQL, so I'm not sure.

  1. What are your thoughts on this subject?
  2. Do you recommend Node-Red Dashboard to use as professional front-end, so a client can access information from it?

Grafana is for graphics/graphs display. Node-red dashboard is for interaction, while it can show graphics/graphs.

Your project sounds like a clear case for telegraf+influxdb+grafana.

Thanks for the reply! Let's say that I want the client to choose which variable or information he wants to see in a display, and this includes graphs. In this case, can I import grafana graphs into node-red dashboard? Or is there a better front-end application which gathers data and graphs from influxdb/grafana?

Grafana is the dashboard. And it is really easy to setup once you have the data.
It can send alerts on thresholds among many other things.

I have done something similar and started with this Youtube video

This has now progressed to running Influxdb and Grafana on AWS EC2 and it works great

Is Telegraf just used to collect data from a computer such as CPU, Ram, Memory, HD usage, temperatures etc?
What sort of data are you collecting?
I'm pretty new to all this but Node-Red, InfluxDB and Grafana are perfect for it.

I'll give a try to node-red + influxdb + grafana. Thanks!

Can you tell me more about this AWS EC2 and why did you choose to use it instead Node-red?

If you search for Amazon Web Services EC2 there is plenty of information and some tutorials. With the information from the You Tube video I was able to use an instance of a Linux Debian machine on EC2 to run Node Red, Influxdb and Grafana. The Grafana dashboards are then accessible from the internet and the machine can be upgraded as the database gets larger, so I am still using Node Red.
There is a similar offering from IBM Cloud

1 Like

I have the Node-RED + InfluxDB + Grafana working nicely on a Pi, but would like to move to a cloud-hosted provider so I can access my data when I am out. Just so I am clear, should all 3 of these be running on the cloud, and then Node-RED would be collecting its data from the datasource (in this case a temperature sensor with 16 channel) that is on my local network? Or could I keep Node-RED running locally and then configure the InfluxDB node to be on AWS, IBM Cloud, etc?

I would keep Node Red running locally and use the InfluxDB node to send the data to the cloud based service running Influxdb and Grafana

I have it all working nicely now. For me, the most difficult aspect of the migration for local (Pi) to cloud was setting up the Amazon EC2 Debian instance. Once I was able to SSH in, I set up InfluxDB just like it was done on my Pi. I have signed up for a free trial of Grafana Cloud, which then queries InfluxDB that is hosted on EC2.

1 Like

So after about a week, I have noticed a phenomenon whereby Node-RED stops taking readings for 5 to 10 minutes, about every 8 to 12 hours. See the debug screen below from 4:54 PM to 5 PM.

This shows up in Grafana as "blank" and indeed when I check InfluxDB for those times, I see no values. So the problem is definitely Node-RED. I am running Node-RED on a Raspberry Pi 4 and 8 gb of ram. Any idea why Node-RED stops running like this for 5-10 minutes every 8 to 12 hours?

Is it crashing and restarting?. Scan the logs for clues.

If not that then add debug nodes further up the flow and find which node is not doing what it should.