Hi All,
I am looking for some suggestions.
Have any of you set up a Node-red server and stored the incoming data in a time series database?
If so, which db have you found works best and is easy to use?
Thanks in advance.
Yes. Lots of us use InfluxDB. One of its advantages, apart from needing relatively few resources, is its integration with Grafana.
The main downside of InfluxDB is getting your head around the terminology. It may seem at first sight to be similar to SQL but in reality it absolutely isn't. I did post an article in the forum quite a bit ago about some best practices on configuration for IoT.
There is a supporting node-red node though you don't actually need it since feeding data to it is done via a web interface - but it does make things a little easier.
I also use Telegraf from the same vendor which is a lightweight service for grabbing all sorts of useful info such as system and service metrics and feeding them into InfluxDB (and to other outputs such as alerts and MQTT) making it very easy to chart with Grafana.
Here are a few random examples from my Grafana dashboards.
All the data bar this last chart comes via Node-RED or Telegraf. This last one comes from a JSON file that is updated daily by a node-red flow reading API data from npmjs.org.
1 Like
People get stunning dashboard graphs with grafana and grafana & influxdb often appear in the same sentence together, it must be a sweet combination.
Actually I'm rather confused by the concept of a time series database. Is it faster than a well structured and indexed SQL database? Less effort to set up?
But what made me settle on Mariadb rather than influx to store my sensor data was the fact that I learned SQL donkeys' years ago and I find an SQL database more familiar than vaguely sqlish Influx.
Possibly it's just my impression but I think mariadb puts less demands on my raspberry pi than influx did.
For accessing data that is Time driven ( example sensors), Influxdb is way way faster than any sql database. I have years of data stored and if I want to look back at 3/4 years of data and visualise it on Grafana it's almost instantaneous.
1 Like
I wonder how it achieves that, does it automatically index by day, hour, minute etc?
Does it apply statistical jiggery-pokery at insert that makes retrieval and analysis faster?
However it does it, influx and grafana make a most impressive combo.
1 Like
Yes, absolutely correct. The data structures and indexes are heavily optimised.
InfluxDB goes further with its continuous query and automatic trim functions so that you can keep your detailed IoT data - lets say 100 sensors reporting every minute - for a week or a month or whatever - and also auto aggregate the data to hourly with max/min/avg readings for each detail.
I did read up on it when I first started to look for something to replace MongoDB which I'd been using until it completely corrupted a db well before its supposed max db size. Their refusal to support ARM put me off as well (not sure if that has changed now). However, I can't remember the details any more. There are, of course, plenty of other time series dbs on the market but the low resource usage of InfluxDB and their willingness to provide something free that doesn't require a postgrad degree in astro-physics for folks like us is hard to beat.
InfluxDB is free for a lot of use, it doesn't chew up a lot of resources and works blindingly fast if you are sensible about data sizes.
Grafana too is amazingly fast at visualising complex data queries. It too doesn't suck up loads of resources.
I've run Node-RED, Mosquitto, InfluxDB, Telegraf and Grafana even on a single Pi2. Did start to get a few pausing issues if I let the db's grow too big but they went away when configured properly.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
