Function help to deal with mqtt retained messages/influxdb "unable to parse"

[877]

When deploying, I'm getting influxdb "unable to parse" errors in debug. Only happens after deploy, and works fine at all other times.

I believe this is due to retained mqtt message not containing the full payload. In this case linkQuality Not sure why, as some devices do retain this.

I'm thinking to stop this, I could amend the function code, so if there is an undefined value, it is ignored and not passed onto influxdb. Or I am open to other suggestions.

Here is the flow:

And here is the function:

//Convert the contact payload to 1 or 0 value
var open = msg.payload.contact === false ? 1 : 0

//Now configure the payload as normal for influxDB
msg.payload = [
    {
        contact_binary: open,
        contact:  msg.payload.contact,
        batteryPercent: msg.payload.battery,
        linkQuality: msg.payload.linkquality,
        batteryVolts: msg.payload.voltage
    },
    {
        topic: "zigbee2mqtt/Xiaomi_4_letterbox",
        make: "aqara",
        location: "hallway",
        connection: "zigbee",
        power: "battery",
        friendlyname: "Letterbox"
    }
];

return msg;

Any advice appreciated.

[Colin]

Are you sure it is that influx node generating the error? If so I don't see how it can be from the message you show in the debug as that doesn't even have some of the data in the error message. Unless it is in the second element of the array that is. Give the node a name to check. Or you can Search for the node id shown above the error.

[877]

Hi @Colin , yes it is definitely that influxdb node generating the error. I didn't expand the 2nd array element to fit it on the page better. It's just tag data which I add with the function node.

Here is an example flow showing the issue:

[{"id":"fac377e1.b80978","type":"function","z":"be5275e8.e06c78","name":"Format front door letterbox Aqara","func":"//Convert the contact payload to 1 or 0 value\nvar open = msg.payload.contact === false ? 1 : 0\n\n\n//Now configure the payload as normal for influxDB\n\n\nmsg.payload = [\n {\n contact_binary: open,\n contact: msg.payload.contact,\n batteryPercent: msg.payload.battery,\n linkQuality: msg.payload.linkquality,\n batteryVolts: msg.payload.voltage\n },\n {\n topic: \"zigbee2mqtt/Xiaomi_4_letterbox\",\n make: \"aqara\",\n location: \"hallway\",\n connection: \"zigbee\",\n power: \"battery\",\n friendlyname: \"Letterbox\"\n }\n];\n\nreturn msg;\n\n//877 info:\n//TAG KEYS are topic/make/location/connection/power\n//TAG VALUES are \"aqara/lounge/zigbee/battery\"\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":850,"y":4210,"wires":[["8e8d224e.117fd"]]},{"id":"d736fb41.90d238","type":"debug","z":"be5275e8.e06c78","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":680,"y":4290,"wires":[]},{"id":"8e8d224e.117fd","type":"debug","z":"be5275e8.e06c78","name":"debug (goes to influxdb node)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1160,"y":4210,"wires":[]},{"id":"d175859e.f18018","type":"inject","z":"be5275e8.e06c78","name":"Complete payload","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"zigbee2mqtt/Xiaomi_4_letterbox","payload":"{\"battery\":97,\"contact\":true,\"linkquality\":39,\"voltage\":2995}","payloadType":"json","x":460,"y":4180,"wires":[["fac377e1.b80978","d736fb41.90d238"]]},{"id":"1cdf3b83.74b9f4","type":"inject","z":"be5275e8.e06c78","name":"Partial payload","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"zigbee2mqtt/Xiaomi_4_letterbox","payload":"{\"battery\":97,\"contact\":true,\"voltage\":2995}","payloadType":"json","x":450,"y":4240,"wires":[["fac377e1.b80978","d736fb41.90d238"]]}]

It does not only happen with this flow, it occurs with any flow where there is an missing payload, which end up as undefined.

I'm sure there must be a way to deal with this in the function..

[Colin]

You can easily test for undefined in a function
if (typeof somevar === "undefined") ...

For future reference, in order to make flows more easily imported, see this post for details of how to share flows here - How to share code or flow json

[877]

Trouble is, it’s not undefined when it enters the function, it’s missing from the payload. Not sure that is treated the same?

[Colin]

Add before the return

if (typeof msg.payload[0].linkQuality === "undefined") delete msg.payload[0].linkQuality

or if you want to put in a default value then do it the same as you have for open, testing for === "undefined " and assigning the default value.

[877]

Ah, that works great thanks!

This can of course be used to filter all the payloads. What would be good a few lines to to check if ANY of the payloads === "undefined" , and delete only that payload.

[Colin]

There are a number of ways of doing it, this is one way

const fields = ["contact", "batteryPercent", "linkQuality"]

for (let i = 0; i<fields.length; i++) {
    if (typeof msg.payload[0][fields[i]] === "undefined") delete msg.payload[0][fields[i]]
}
return msg;

[877]

That is awesome thanks for your help!

[Colin]

Actually a better way to check all of them is

let obj = msg.payload[0]        // this saves typing
for (let field in obj) {
    if (obj.hasOwnProperty(field)) {
        if (typeof obj[field] === "undefined") delete obj[field]
    }
}
return msg;

[877]

That is even better, no need to type in the fields!

Thanks a lot!

[Colin]

You should report this issue against the influx node, it should not behave like that if it gets an undefined field. Issues · mblackstock/node-red-contrib-influxdb · GitHub

I presume you are using the latest version of the influx node. I should have asked that earlier.

[877]

I'm using node-red-contrib-influxdb node version 0.5.4, no update showing in pallette manager.

However, I am using influxdb 1.8.4 (Docker), as I had issues I can't remember right now with higher versions..

[Colin]

0.5.4 is the latest influxdb node so that is ok, influxdb 1.8.4 is the latest 1.8 series so that should be fine too. So you should submit an issue so that it gets looked at.

[877]

No problems, I will raise an issue now