Influx DB data collection

General
1

Hi,

I am an electrician and gas installer by trade. I am a complete noob to programing and see that I need to evolve with the times and venturing into the remote monitoring space.

I have been able to so far, with the help of google and youtube set up on Digital ocean Mosquito MQTT, Node Red, Influx DB and Grafana.

I have a device that where I am collecting data over GSM via MQTT topics from 3 x temp probes and 2 x current transformers, as well as some other data of the monitoring device, such as signal strength, tilt switch, power ect...

I have set up some flows in Node Red to set up some email alerts and currently using the built in Node Red dashboard for testing proposes.

I am now at the point where I would like to get the data stored in Influx DB and create dashboard in Grafana linked to the Influx DB database set up for this device.

I understand that I use the Node Red Influx DB node to store the data in the created Influx DB database created for this device eg: "Gas Boiler 1"

This is the point where I am stuck.

What I understand is that I need to use a function node in Node Red to convert the data (payload) collected in Node Red from the MQTT node, so that the data can be stored correctly in the database created for this device in Influx DB.

The device publishes the topics for each sensor every 60 seconds, and other data of the monitoring device which on gets published at state change, via MQTT. I am using an MQTT node for each topic to collect the data and sensor reading (payload).

For example, the topic for the sensors is:
/eontec/model1/temperature/temp1
/eontec/model1/temperature/temp2
/eontec/model1/temperature/temp3
/eontec/model1/currentsensor/ct1
/eontec/model1/currentsensor/ct2

  1. What would be the correct code to enter into the Function Node, to convert the payload (sensor reading) into the correct format to be stored in influx DB in the created database "Gas Boiler 1" for each sensor?

  2. In the Influx DB Out node, in the server settings properties tab, I enter the server details and enter the database eg "Gas Boiler 1" I would like the reading to be saved in. On the DB Out node properties tab, in the measurement field, do I insert which sensor the reading is from, eg: temp 1?

2

Are there going to be multiple boilers? If so then if I were doing it I would put them all into one database and use an influx tag to specify the boiler ("Gas Boiler 1" in this case). Then I would put them all the gas boilers into one measurement, called Boilers maybe, and with a different field name for each sensor, so maybe temperature1, temperature2 etc. Others might suggest organising the data in a different way.

I suggest you start by reading carefully through the node red doc Working with Messages.
I use node-red-contrib-influxdb to write to the database (there are alternatives), so if you decided to use that then have a look at the readme and work out what format you need to get the data into as each sensor value comes in. Once you know that then at least you know what you want to achieve. If you can't work it out then ask again.
Once you start building the flow then don't connect to the database straight away, feed it into a debug node instead and check the format is correct first.

3

InfluxDB is a little tricky to get started with.

My recommendation would be to set up a local test and work through some of the examples from the InfluxDB website. But do it via Node-RED.

That should then help you understand how to structure the data.

Honestly, I always struggle when I come back to my databases, it takes me a while to remember how they are built.

4

I have done something similar and used this video as a starting point, it may help.

5

Thanks, this video gave me the idea. However, the step im stuck on in all videos its "assumed" that you know what to do.

6

Hi Thanks,

Yeh have tried to get understanding from the Influx Db, however Im still lost. Will keep trying.

7

Hi thanks,

Yes, there will be more boilers. However, I would like to have a separate database for each boiler, and have different field names for each sensor for that boiler. I have gone through the documentation before posting. I will go through it again and see if anything makes sense

8

Do you have a particular reason for that? It would, for example, make it difficult to use Grafana to show multiple boilers on one dashboard, I think.
However you do it, assume that the first go will not be what you end up with, as you will almost certainly decide you did it wrong once you start using it in earnest.

9

A simple example of the function node and the configuration of the InfluxDB node. The database name would be Boiler1 and the function node would be an object with a name and msg.payload as the value. The measurement in the database would be Temp1.

Colin has good advice, if you have multiple databases (one for each boiler) you cannot compare data from different boilers on a single dashboard.
This is a simple sample flow to show the function and InfluxDB nodes

[{"id":"5c8e39ab.c5e0d8","type":"influxdb out","z":"c4e4429b.8f961","influxdb":"4c6c1d4d.99d704","name":"","measurement":"Temp1","precision":"","retentionPolicy":"","x":710,"y":100,"wires":[]},{"id":"514e211b.a0fe7","type":"function","z":"c4e4429b.8f961","name":"","func":"msg.payload = {\n    name: \"Temp1\",\n    DegC: msg.payload,\n};\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":100,"wires":[["5c8e39ab.c5e0d8"]]},{"id":"788b3a1e.92dfd4","type":"inject","z":"c4e4429b.8f961","name":"Boiler 1 Temp","topic":"","payload":"56","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":190,"y":80,"wires":[["514e211b.a0fe7"]]},{"id":"4c6c1d4d.99d704","type":"influxdb","z":"","hostname":"127.0.0.1","port":"8086","protocol":"http","database":"Boiler1","name":"","usetls":false,"tls":""}]
10

Hi @mtoko , Thank you for the sample

Each boiler is at a different customer and each boiler would have its own dashboard. At this stage there is no need to see different boilers on the same dashboard, however, the may be the need in the future.....

11

The main reason is each boiler is at a different customer and each boiler would have its own dashboard. At this stage there is no need to see different boilers on the same dashboard, however, the may be the need in the future.....

I will have a look at your first suggestion as a setup.

12

@eontec

Grafana is ideal for this requirement, as you can have multiple organisations each with multiple users, so if you have a single database you can give each organisation and user access to the correct data and still have the option to create dashboards (for your use?) from all boilers.

13

mmmmm, sounds like the suggested way would be the correct way.

Will create a DB called gas boilers. Just need to learn how to get the data correctly into influx DB for each seperate gas boiler device with its specific sensors for each device.

The fun never stops

14

Hi @mtoko,

Have used @Colin setup advise, thanks for that @Colin.

Quick question, thank you for the sample flow, helped me to understand better. I see that it saves the Temp reading (degC) from the sensor as a string in Influx DB. How do I get it to save the Temp reading (degC) to save as a float in Influx DB? Is there something I need to do in the function node in your sample flow?

15

Hi
I have just checked the sample flow and the value from the inject node is a number and then when this is saved to the influxDB it is saved as a float. How do you know that the reading is saved as a string? Remember the DegC tag will be saved as a string

16

If you have once written a field as a String then I think you are stuck with that and cannot change it other than by deleting the measurement and starting again, or changing the field name.
If the value you are getting from wherever it is coming is a string then in a function node you can convert it to a Number using something like
msg.payload = Number(msg.payload)
assuming it is in msg.payload that is.

17

Hi @Colin,

Thanks, that did the trick

18

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.