Function to process incoming msg from MQTT before passing to Influxdb batch out

General
,
1

The overall shape of the project follows the familiar route:
ESP32 collects sensor data --> passes readings to MQTT broker via a JSON document
The Broker sits on a raspberry pi with all applications running in containers under docker:
MQTT Broker --> Node-Red --> InfluxDB V1.8 --> Grafana
I have everything working for a single reading, but when it comes to a batch of readings, the error reported is that the values are 'undefined'
The MQTT IN Node settings are:

  Server: raspberrypi
  Action: subscribe to single topic
  Topic: some_prefix/boiler
  Output: A parsed JSON object

The incoming message is received with the above MQTT In Node. Here's the debug output:

  some_prefix/boiler : msg.payload : Object
  object
    temp_dhw_flow: 71.6
    temp_dhw_return: 46.6
    temp_ch_flow: 62.7
    temp_ch_return: 51.2
    pres_boiler: 1.23
    leak_boiler: 407.9

The message is then passed to a function that strips out the prefix:

   var t=msg.topic;
   t = t.replace('/some_prefix/', '');
   t = t.replace('some_prefix/', '');
   while(t.search("/")!=-1){
     t = t.replace('/', '-');
   }
   msg.topic=t;
   msg.measurement=t;
   return msg;

A debug node connected to the output of this function correctly shows:

  boiler : msg.payload : Object    <-- (note the prefix has been stripped)
  object
    temp_dhw_flow: 71.6
    temp_dhw_return: 46.6
    temp_ch_flow: 62.7
    temp_ch_return: 51.2
    pres_boiler: 1.23
    leak_boiler: 407.9

The message is then passed to a further function to extract the values prior to sending to the connected Influxdb batch out node:

msg.payload = [
    {
        measurement: "boiler",
        timestamp: new Date(),
        fields: {
            tempDHW_Flow: msg.payload.tempDHW_Flow,
            tempDHW_Return: msg.payload.tempDHW_Return,
            tempCH_Flow: msg.payload.tempCH_Flow,
            tempCH_Return: msg.payload.tempCH_Return,
            presBoiler: msg.payload.presBoiler,
            leakBoiler: msg.payload.leakBoiler
         },
        tags:{
            location:"garage",
            /*sensor: "board 1"*/
        }
    }
];
return msg;

I have almost exactly the same construct working for a single reading (tempDHW_Flow) but the above yields the following in a debug node connected to this second function (also presented as an error in a Catch All node):

boiler : msg.payload : array[1]
array[1]
0: object
 measurement: "boiler"
 timestamp: "2024-02-11T15:25:29.589Z"
 fields: object
    tempDHW_Flow: undefined
    tempDHW_Return: undefined
    tempCH_Flow: undefined
    tempCH_Return: undefined
    presBoiler: undefined
    leakBoiler: undefined
 tags: object
    location: "garage"

Consequently no data is passed to the InfluxDB batch Out Node.

I've spent some time trying to research this problem, without success. I have successfully written to InfluxDB previously using the same Batch out Node, so the problem is almost certainly the failure of the function given the form of the input data.

Suggestions welcome.
Thanks in advance,
Ric

2

Is there a question here?

3

temp_dhw_flow :point_up:


tempDHW_Flow :point_up:

These are different.

4

Hi and thanks,

I had piggy-backed some other code where I sent a JSON document to an AsyncWebServer running on the ESP32. The JSON document fed readings to a number of HighCharts graphs. I hadn't noticed I changed the names (to avoid ambiguity elsewhere) and blindly re-used the same JSON document for the MQTT broker. A very simple change to the 2nd function solved this.

If only we could see the obvious...

As a footnote: I've had only a few days exposure to Node-Red and think it's just amazing. Javascript and JSON are all new to me, but I am truly impressed by the power and flexibility of Node-Red.

Thank you again. Appreciated.
Ric

2 Likes