Write a monthly temperatures File

General
21

And again noting that InfluxDB does all of this for you apart from backup. There, you can simply copy the DB folder I think. Personally, I don't bother because it doesn't really matter to me if I loose my environmental data history. I do have a separate record of how to (re)build the db's.

1 Like
22

I work with large json arrays (200k+) with large objects (60+ properties with nested arrays) within node-red, which are both stored in memory and disk and searching/collecting/summarizing through them, within memory, is a breeze (milliseconds). For some I use alasql so that i can use sql instead of javascript, this is a little slower, but still very acceptable performance.

1 Like
23

Then sneak up on the time-series database stuff. Get your data flowing in and pick the easiest way for you, to store it. CSV might be a good first step.

Individual temperatures (or any other sensor data) is good to see. But if you're storing it, it means you've got plans to come back and do some analysis on that data.

You can absolutely do that analysis in Excel or an abacus. :slight_smile:

As your needs grow, you'll likely find that storing them in a database is useful. If that time comes, then I strongly agree with the others that a Time Series database (like InfluxDB) is the cat's meow for this type of data. They simply outperform most other databases for holding sensor data.

IMHO...

1 Like
24

I am not sure if this is still relevent, but I build this "auto logger" flow a while back. You pass the data in a structure, and it will generate a filename (on a daily, weekly, monthly basis) and keep writing the data to a file and create a CSV file if it does not exist:

How to set it up:

  1. The flow requires node-red-contrib-fs, so install it.
  2. Modify the Set Data node and prepare your data. The payload should be an simple object with a list properies that you want to save to CSV.
  3. Modify the Filename generator node and update the file name pattern in line 11. In the example it will generate a filename per day. You can change that to for example monthly file like: msg.fname = "maplin_"+ yyyy + mm + ".csv"; Also change the path in line 13.
  4. Modify the File lister node as well, and update the path where the files will be saved (same to what you entered in line 13 in the previous step.
  5. Modify the two csv nodes and on the top update all the values you want to include in the CSV. That should match the propery name in the payload.

Pretty basic flow, but works fine.

3 Likes
25

Hi nygma,

Thanks for this info, it may be basic but it sounds like it could do what I want it to.

cheers
Stuart

26

I do use InfluxDB & Grafan for a while (> 4 years). I connected an external USB-Disk to my Orange Pi 3LTS and have the InfluxDB write the data to the DB on the harddisk. Having it write to the SD-Card, would damage the card in short time.For me, the combination of InfluxDB & Grafana is great and I will not use any other combination :slight_smile:

1 Like