Mqtt from ESP to Node-red

hi

i am using EspMQTTClient.h on my ESP for sending mqtt to node-red. From my Mqtt nodes i am sending my data to influxdb node.

Every 5min i send around 12mqtt msg from my ESP.

On Node-red side i have set up a debug node to see all my mqtt payloads.

Problem is that sometimes i receive all messages. Sometimes only few and sometimes nothing.

ESP:

// Optional functionalities of EspMQTTClient
client.enableDebuggingMessages(); // Enable debugging messages sent to serial output

ESP Serial Monitor:

21:34:29.540 -> MQTT << [home/weather_station/sensors/tmp-1] 22.13
21:34:29.588 -> MQTT << [home/weather_station/sensors/tmp-2] 22.13
21:34:29.634 -> MQTT << [home/weather_station/sensors/tmp-3] 22.13
21:34:29.822 -> MQTT << [home/weather_station/sensors/moisture-1] 3857
21:34:30.246 -> MQTT << [home/weather_station/sensors/moisture-2] 3840

etc..

ESP serial monitors always writes all 12 msq as sent. But how can i be sure ...

Node-Red Debug:

Every time i should get 12 mqtt msg. But i dont. I seems random. Sometimes there are gaps of 5min to 1h without any msg received.

Dont know where should i check for the problem. Any tips?

I would start by installing MQTTExplorer on my PC and connecting the broker, and checking what that sees. If it sees the same as node-red then the messages are not getting to the broker.

Which MQTT broker are you using? Is it running on the pi?

I am using mosquito and it is running on Pi

Mosquitto is good, the problem will not be with mosquitto.
Try MQTTExplorer. It is a useful tool to have anyway, often one wants to know what topics things are publishing on.

i have installed MQTTExplorer

1st cycle
MQTTExplorer
image

ESP
image

but cant see in 1st cyle MqttExplorer this one:

image
image

etc

2nd cycle(after 5min)
new msg appeared in MQTTExplorer but not all 12 at the same time.
image

3rd cycle
new msg appeared in MQTTExplorer but not all 12 at the same time.
image

All 12 msg should appear in the 1st cycle and not separately

Ok, will focus on ESP coz it seems that it is not publishing or something is wrong there... will try tomorrow

I would guess your sending loop on the ESP is the problem - you have probably not completed filling the messages before you exit the loop and send out the messages

Also there is a limited buffer on the ESP (i assume you are using Ardunio IDE for this ?) are you sure you are not overfilling it ?

Lastly have you looked at the excellent Tasmota code ? even if you do not use their precompiled code you can use the source code as the basis for this - check out their sensors stream

Craig

Alternatively you could try to go with node-red-mcu & my node-red-mcu-plugin. MQTT works like a charm - as @phoddie has shown @ Node-RED Conf 22.

1 Like

My code is something like this:

I also added some time delays 100milis after each client.publish and 500 before deepsleep.

Setup:

simpleTimer_sensors = timer.setInterval(1000, sensorData_callback);

Functions:

void sensorData_callback() {
  if(WiFi.status() == WL_CONNECTED && client.isMqttConnected() && sensorData_checkOnlyOnce && sensorRain_allowSensors){
    timer.disable(simpleTimer_sensors);
    sensorData_checkOnlyOnce = false;
    readLight();
    readLux();
    readSensorIR();
    readDS18B20();
    readMoisture();
    readBatteryVoltage();

    deepSleep();
  }
}

And then each sensor function for example Light:

void readLight() {
  int sensorLight = analogRead(sensorLight_PIN);

  client.publish("vikend/weather_station/sensors/light", String(sensorLight), true);
  delay(DELAY_AFTER_PUBLISH); // ADDED a delay hoping that would help but it doesnt

}
void readLux() {
  sensorLightLux.begin();
  int sensorLux = sensorLightLux.GetLightIntensity();

  client.publish("vikend/weather_station/sensors/lux", String(sensorLux), true);
  delay(DELAY_AFTER_PUBLISH);
}

etc

At the end there is deepsleep call:

void deepSleep() {

 ...some other code

    delay(500); // ADDED a delay hoping that would help but it doesnt
    esp_deep_sleep_start();

}

MQTTExplorer:

Each square represents random cycles(5min) of data that i get. Most of the time i get all data. But sometimes i get only red, blue or green.

After adding delays ... any other suggestion?

How about only publishing on one topic and not deep sleeping at all

Is that reliable?

Doesn't that call the function every second? Does that mean that it restarts it before the previous one has finished?

no i have a flag there and i have also disabled timer

timer.disable(simpleTimer_sensors);
sensorData_checkOnlyOnce = false;

Now i am going to try without deepsleep shorter interval from 5min to 1min...

After disabling deepsleep and after around 30 x 1min cycles i did get everytime my all 9 sensor data without skipping.

(example for last client-publish data voltage-raw; 1min cycles)

So question is why with deepsleep enabled skips randomly?
Should i put some code before it goes to deepsleep? Some sort of cleaning, disabling connection.. not good at this, any suggestion?

My procedure to go to deep sleep in something like this ...

void deepSleep() {
  
    esp_sleep_enable_timer_wakeup(TIME_TO_SLEEP * uS_TO_S_FACTOR);

    delay(500); // Added doesnt help
    esp_deep_sleep_start();
  }   

}

With DeepSleep enabled after 30 x 1min cycles i got 9 skips on voltage-raw sensor which is the last in the series that i publish. 30% loss which is bad.
Other sensors have different skip numbers.

Why not create an object with all the readings and just publish one message?

1 Like

You have to remember that after a deep sleep, you need time to wake, reconnect to WiFi and then reconnect to MQTT. My guess is that you are not getting enough time to do all of that before you try your next sleep.

Honestly, if you don't have much experience in programming microprocessors, it is likely that you would be better off using a pre-built firmware rather than rolling your own.

My personal choice is ESPHome which gives a nice balance between robustness, being able to simply configure all of your sensors, outputs, timings and being able to drop into C++ code should you need something that the project hasn't already covered (though they have virtually all of the common sensors and controls already pre-configured). It also has the advantage that you can break your config up into reusable parts which is great if, like me, you have a lot of different ESP devices of different types.

If you do want to roll your own, I would advise you to get the basics right without dealing with deep sleep (which is a whole subject area in itself) and then go do some research on the best way to use deep sleep with MQTT.

1 Like

I'd try replacing that delay with a client.loop(); command, or at the very least explicitly calling client.loop() before initiating deepsleep.

Pete.

1 Like

i have done this but without any success.

I have created 1 object with multiple sensors readings and only 1 client.publish instead of having 9 seperated sensors readings / 9 client.publish events.

Results are the same. Skipping if using deepsleep. Without deepsleep it works normally

I don’t know if this will help but when you wake from deep sleep your setup code will run. How does that effect your processing?