Beginner-questions: save BMP280-Data to InfluxDB

Hello,
I started yesterday with NodeRed and got stuck a bit. So I hope you can help me further.

I have a BMP280 sensor to read temperature and humidity connected to an ESP8266-Board. This sends my data to Node Red and with MQTT I can also read this values. The script is so that it sends only data, if the value changes more than a threshold-value. Now I want to save my data to an Influx-DB to visualize it later with Grafana.

Unfortunalety I don't get it working:
Here the code on the ESP:
`/*
Project: BME280 Sensor data send over MQTT with a ESP8266 / NodeMCU
Author: Thomas Edlinger for www.edistechlab.com
Date: Created 19.10.2019 / updated 27.10.2019
Version: V1.1
Changelog:
OTA implementation
Placeholder for inTopics

Required libraries (Tools -> manage libraries)

  • PubSubClient by Nick O'Leary V2.7.0
  • Adafruit BME280 Library V2.0.1
  • Adafruit Unified Sensor V1.1.2
    Required Board (Tools -> Board -> Boards Manager...)
  • Board: esp8266 by ESP8266 Community V2.6.3
    Wirering for the BME280 Sensor:
    BME280 NodeMCU
    VCC 3.3V
    GND G
    SCL D1 / GPIO5
    SDA D2 / GPIO4
    */

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
#include <ArduinoOTA.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_Si7021.h"
#include <Adafruit_BMP280.h>

#define wifi_ssid "wifiSSD"
#define wifi_password "wifiPW"

#define mqtt_server "192.168.0.112"
#define mqtt_user "your_username"
#define mqtt_password "your_password"

#define ESPHostname "ESP8266_01"

#define humidity_topic "esp01/humidity"
#define temperature_topic "esp01/temperature"
#define pressure_topic "esp01/pressure"

#define inTopic "esp01/inTopic"
#define outTopic "esp01/outTopic"

Adafruit_BMP280 bme; // I2C
Adafruit_Si7021 sensor = Adafruit_Si7021();

float temp = 0.0;
float hum = 0.0;
float pres = 0.0;
float diff = 0.2;

unsigned long delayTime;

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;

void setup() {
Serial.begin(115200);
setup_wifi();
ArduinoOTA.setHostname(ESPHostname);
// ArduinoOTA.setPassword("admin");
ArduinoOTA.begin();

client.setServer(mqtt_server, 1883);
client.setCallback(callback);
while(!Serial); // time to get serial running
unsigned status;
status = bme.begin(0x76); //I2C address is either 0x76 or 0x77
if (!status) {
Serial.println("Could not find a valid BME280 sensor, check wiring, address, sensor ID!");
while (1);
}
if (!sensor.begin())
{
Serial.println("Did not find Si7021 sensor!");
while (true);
}

delayTime = 1000;

}

void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
ArduinoOTA.handle();
long now = millis();
if (now - lastMsg > 2000) {
lastMsg = now;
getBME280Values();
}
}

void setup_wifi() {
delay(10);
// We start by connecting to a WiFi network
Serial.println();
Serial.print("Connecting to ");
Serial.println(wifi_ssid);

WiFi.begin(wifi_ssid, wifi_password);

while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
if ((char)payload[0] == '1') {
//If the first Value of the payload is 1
} else {
// Room for Code
}
}

bool checkBound(float newValue, float prevValue, float maxDiff) {
return !isnan(newValue) &&
(newValue < prevValue - maxDiff || newValue > prevValue + maxDiff);
}

void getBME280Values() {
float newPres = bme.readPressure() / 100.0F;
float newTemp = bme.readTemperature();
float newHum = sensor.readHumidity();

if (checkBound(newTemp, temp, diff)) {
  temp = newTemp;
  Serial.print("New temperature:");
  Serial.println(String(temp).c_str());
  client.publish(temperature_topic, String(temp).c_str(), true);
}

if (checkBound(newHum, hum, diff)) {
  hum = newHum;
  Serial.print("New Humidity:");
  Serial.println(String(hum).c_str());
  client.publish(humidity_topic, String(hum).c_str(), true);
}

if (checkBound(newPres, pres, diff)) {
  pres = newPres;
  Serial.print("New Pressure:");
  Serial.println(String(pres).c_str());
  client.publish(pressure_topic, String(pres).c_str(), true);
}  

}

void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Create a random client ID
String clientId = "ESP8266-";
clientId += String(random(0xffff), HEX);
// Attempt to connect
if (client.connect(clientId.c_str(), mqtt_user, mqtt_password)) {
Serial.println("connected");
// Once connected, publish an announcement...
client.publish(outTopic, ESPHostname);
// ... and resubscribe
client.subscribe(inTopic);
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}`:

And here some points of my NodeRed-Sketch

Unfortunately I always get only 1 value transmitted and then there comes the error and I can't add the value to the database.

Anyway my approach seems pretty complex to me. Do you see any solution, which works and is simpler than my solution?

Would be great to get some input from you.

Thanks in advance :slight_smile:

I am afraid I find your screenshot virtually unreadable.
I suggest initially forgetting about influx and check that you are able to receive the data into node-red, or perhaps you already have that going, it is not clear.

Personally I would not do the threshold detection in the ESP, I would send the data at an appropriate rate whether it has changed or not. You can do the threshold detection in node-red if you need it. Your charts in grafana will be better if you provide as much data as possible.

Ok, thanks you. Then I will try it this way. I managed to get the data to node-red, but got stuck implementing it into the database.

Actually I wanted to reduce the amount of transmitted data, as I use a raspberry pi and want to reduce the number of write-operations to save my SDcard. So I hope to get a solution to save it on an SSD or Cloud later.

How often are you sampling the sensors and how many sensors altogether are you planning?

For writing to influx is it that you don't know how to organise the data for that node or you think you have it right but it doesn't seem to be working? If you think you have it right feed it into a debug node and show us what you have.

I get the same error as above even when sending all data at once from my ESP8266

      client.publish(temperature_topic, String(temp).c_str(), true);
      client.publish(humidity_topic, String(hum).c_str(), true);
      client.publish(pressure_topic, String(pres).c_str(), true);

Have you tried clicking on the pictures, so that it shows itself larger? Unfortunately I can only upload 2 images here as a new user, which is why I uploaded everything as 1 image.

I just don`t really know how to organize my data, so that it can be written into Influx. I saw other tutorials, where there is one object like

msg.payload = {
    temperature: 22,
    humidity123: 68
}


Unfortunately when I try to aggregate my data like this in the function node,in the debug-node I get 2 independent debug-messages (1 for temp, 1 for humidity). As far as I understood, they need to be in 1 Object, so that the table can read them properly.

At the same time, the data seem to be aggregated in the Influx-node, but it gets only 1 of the 2 data (either temp or humidity) and the other value is missing and cannot be interpreted:

"Error: A 400 Bad Request error occurred: {"error":"unable to parse 'neuedb humidity123=62.92,temperature=undefined': invalid boolean"}↵"

Then, I am not sure, what is meant with invalid boolean

You need to join the messages containing the individual values into one message if you want to put them into influx in one influx Measurement. You can do that using a Join node as described in the node-red cookbook recipe Joining Streams. However, since you have control of the s/w sending the values to MQTT the best way would be to send them all in one go in a JSON string. So that means a string, something like
{"temperature": 22, "humidity": 68, "pressure": 1010}
Then in node red the MQTT node will be able to convert that straight into a js object ready for influx.

You have given it a value temperature=undefined which is obviously not valid. Don't know why it says invalid boolean, but I wouldn't worry about that.

I solved my problem by sending all data in 1 JSON-String to the Node-Red and converting it there from String to JSON. Then I could add it to my database.

Thanks for your help, Colin!

1 Like

I have a working routine very similar to yours. It takes readings from a remote weather station and sends them via an RF radio to a base station where the information is packed into a json string and transmitted via WiFi, using the MQTT transport, to Node-RED.

I discovered the Arduino library called ArduinoJson which makes a lot of the serialisation and de-serialisation tasks very easy to implement. Get version 6 as that has more features and improvements.

1 Like