Need help with extracting values in Function node

[LosinIt]

I'm trying to get 4 electrical values from 4 MQTT messages from a Z-wave device to log in Influx. I want the 4 values in one DB record so I'm using a Join node to put them together followed by a Function node to extract the values and format for Influx.

I have several similar functions working as desired so I know how to handle the formatting part but they are getting Zigbee data with everything in a single message.

The only way I could get something out of the join node that I know how to deal with (almost) is with it setup like this

Here is 1 of the 4 messages going into the Join node followed by what comes out of the Join node when all 4 arrive within a half second. Since these Z-wave devices spit out these 4 topics every 10 seconds I don't have a problem with timing (that I know of)

Here is my test flow. I've tried various incantations using the lines you see in the function, not all at once as it is shown. The structure after the "=" in this line
const path = payload.value["zwave/Master_Closet_Light/50/0/value/66561"]
came from copying the path from the debug output.

[{"id":"93fe7cb10253b290","type":"mqtt in","z":"d0f2cf2d.02296","name":"Master_Closet_Light","topic":"zwave/Master_Closet_Light/50/0/value/#","qos":"1","datatype":"json","broker":"a8641058.accf9","nl":false,"rap":false,"inputs":0,"x":230,"y":3190,"wires":[["9e36c487cf67cfaa","9e1da59c.3ea6b8"]]},{"id":"9e1da59c.3ea6b8","type":"debug","z":"d0f2cf2d.02296","name":"Original msg","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":410,"y":3250,"wires":[]},{"id":"9e36c487cf67cfaa","type":"join","z":"d0f2cf2d.02296","name":"","mode":"custom","build":"object","property":"payload.value","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":".5","count":"4","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":400,"y":3190,"wires":[["e0ec91e09dc7bfb2","3dcce2d74ae4ab00"]]},{"id":"3dcce2d74ae4ab00","type":"function","z":"d0f2cf2d.02296","name":"multiple metrics","func":"const device = msg.topic.split('/');\nconst lastIndex = msg.topic.lastIndexOf('/');\nconst path = msg.topic.slice(0, lastIndex + 1);\nconst newpath = eval(\"msg.payload.value.\" + path + '66561');\n\nconst voltage = parseFloat(payload.value[\"zwave/\" + device[1] + \"/50/0/value/66561\"]);\nconst path = payload.value[\"zwave/Master_Closet_Light/50/0/value/66561\"]\nconst voltage = parseFloat(path);\n\nvar payload = newpath;\n\nreturn { payload:payload };\n","outputs":1,"noerr":4,"initialize":"","finalize":"","libs":[],"x":580,"y":3190,"wires":[["fda39dd4170aa710"]]},{"id":"e0ec91e09dc7bfb2","type":"debug","z":"d0f2cf2d.02296","name":"debug 1","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":590,"y":3250,"wires":[]},{"id":"fda39dd4170aa710","type":"debug","z":"d0f2cf2d.02296","name":"debug 2","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":750,"y":3190,"wires":[]},{"id":"a8641058.accf9","type":"mqtt-broker","name":"Mosquitto on server-ha","broker":"192.168.5.5","port":"1883","clientid":"","autoConnect":true,"usetls":false,"compatmode":false,"protocolVersion":"5","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"1","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":""}]

Everything I've tried so far results in these errors

I'm a novice with JS and Function nodes so the various attempts came from web searches that seemed to be close to what I'm trying to accomplish.

All information/advice/pointers are welcome.

Thanks for your time!

[Steve-Mcl]

Use the tools node-red provides

There is a copy path button that saves you all the effort of figuring this out

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path to any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

Ps, since this is a function, you will need to prefix the copied path with msg.

[LosinIt]

@Steve-Mcl

you will need to prefix the copied path with msg.

I can't believe I overlooked that! Most of my lines had the msg. but not the one that now works. I guess I was so focused on learning how to use the Join node that I completely spaced out on the basics.

Thanks much, @Steve-Mcl

P.S. I got the syntax for the path by doing exactly as you said with the debug "path"

[Colin]

@LosinIt

I want the 4 values in one DB record

Why? Send them individually to separate measurements. What advantage do you see in adding the complexity of joining them?

[LosinIt]

@Colin

What advantage do you see in adding the complexity of joining them?

I'm a novice with Influx too but it seems more efficient to pass one message with 4 values than to send them individually.

[Colin]

@LosinIt

it seems more efficient to pass one message with 4 values than to send them individually

How many thousand samples per second are you sending to influx?

Where are the values coming from?

Is it possible that sometimes there may be missing values from one or more of the input channels? How are you coping with that?

[LosinIt]

@Colin

How many thousand samples per second are you sending to influx?

@LosinIt

Since these Z-wave devices spit out these 4 topics every 10 seconds I don't have a problem with timing (that I know of)

@Colin

Where are the values coming from?

@LosinIt

I'm trying to get 4 electrical values from 4 MQTT messages from a Z-wave device to log in Influx.

@Colin

Is it possible that sometimes there may be missing values from one or more of the input channels?

I don't know yet but it won't surprise me if it happens.

@Colin

How are you coping with that?

No idea at this point. I take it you ask because your suggestion to not aggregate will avoid the problem, right?

[Colin]

If they do all come from one device, and normally they will all be there then that is an argument for joining them as you suggest. However, adding them individually gives you the option of using a different update rate for each one. Dependent on what they are then that may be useful.
Also when you get as far as using retention policies and continuous queries then again it may be useful to have them in separate measurements. For example, you might only want to keep one of the measurements long term, and discard the others after a week. That is much simpler to achieve if they are in separate measurements.

For missing values, I see you have specified a 0.5 second timeout in the join node, so if one of them is missing then it will send the message after that time. Provided your following code copes with missing values then influx will populate the db with an empty value for the missing one. That should not be a problem though.

[LosinIt]

Well, shoot. After getting it all proper in the Function I discover that you were right as I'm getting too many bad writes from the Influx node. You win

[Colin]

@LosinIt

bad writes from the Influx node

You should not get bad writes. What is in msg.payload when it complains?

[LosinIt]

Sorry, bad choice of words. The Influx Batch node is complaining about missing data when a message does not have all 4 metrics which results in nothing being entered into the database.

So I'll use 4 Influx Write nodes, one for each metric, which will accomplish what you suggested in the first place.

[Colin]

@LosinIt

The Influx Batch node is complaining about missing data

It should not complain. What is in msg.payload when it complains? I would like to understand the problem.

@LosinIt

So I'll use 4 Influx Write nodes, one for each metric,

Not necessary. Leave the measurement blank in the node and put it in the message, as described in the node help text.

[LosinIt]

@Colin

It should not complain. What is in msg.payload

It happens when the incoming payload is missing one or more of the 4 values. The complaint is not in any payload, it shows up in the debug side-panel.

Here is a payload with only 1 of the 4 values

And here is the error

@Colin

Leave the measurement blank in the node and put it in the message

Yes, good point.

[Colin]

The error is because you have a value of NaN. If you were to do it this way then you should leave the whole field item blank.

On the measurement, if you can derive the measurement name from the topic then you may be able to make the whole thing automatic, so all the messages go through the same nodes to the influx node.

[LosinIt]

@Colin

The error is because you have a value of NaN

Of course.

@Colin

On the measurement, if you can derive the measurement name from the topic then you may be able to make the whole thing automatic, so all the messages go through the same nodes to the influx node.

This is what is working for obtaining the values. There is probably a better way but, hey, it works.

const topicParts = msg.topic.split('/');
const device = topicParts[1]

const powerNum = 66049;
const cumulative_powerNum = 65537;
const currentNum = 66817;
const voltageNum = 66561;

var path = msg.payload.value["zwave/" + device + "/50/0/value/" + powerNum];
const power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + cumulative_powerNum];
const cumulative_power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + currentNum];
const current = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + voltageNum];
const voltage = parseFloat(path);

Then the payload is populated this way currently

var payload = [
    {
        measurement: "power",
        fields: {
            value: power
        },
        tags: {
            deviceName: device,
            unit: "Watts",
            type: "float"
        }
    },
**The other 3 values go here**
];
return { payload: payload };

If I understand what you're saying here then I need to build the payload from a start of

var payload = [

Then add each object whose value: is !NaN and close with

if payload !== "[" // meaning that at least one object was added
then 
   payload = payload + "]";
   return { payload: payload };
else
   return;

Doe that sum it up? Yes I know the syntax is wrong, just consider it pseudo-code.

[LosinIt]

So here's what I came up with, including the "Template literals" I stumbled upon.

Function code

const topicParts = msg.topic.split('/');
const device = topicParts[1]

const powerNum = 66049;
const cumulative_powerNum = 65537;
const currentNum = 66817;
const voltageNum = 66561;

var path = msg.payload.value["zwave/" + device + "/50/0/value/" + powerNum];
const power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + cumulative_powerNum];
const cumulative_power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + currentNum];
const current = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + voltageNum];
const voltage = parseFloat(path);

var payload = "[";

if (!isNaN(power)) {
    payload = payload + `
        {
            measurement: 'power',
            fields: {
                value: ${power}
            },
            tags: {
                deviceName: ${device},
                unit: 'Watts',
                type: 'float'
            }
        },`
}

if (!isNaN(cumulative_power)) {
    payload = payload + `
        {
            measurement: 'cumulative_power',
            fields: {
            value: ${cumulative_power}
            },
            tags: {
                deviceName: ${device},
                unit: 'kWh',
                type: 'float'
            }
        },`
}

if (!isNaN(current)) {
    payload = payload + `
        {
            measurement: 'current',
            fields: {
            value: ${current},
            tags: {
                deviceName: ${device},
                unit: 'Amps',
                type: 'float'
            }
        },`
}

if (!isNaN(voltage)) {
    payload = payload + `
        {
            measurement: 'volts',
            fields: {
                value: ${voltage},
            tags: {
                deviceName: ${device},
                type: 'float'
            }
        }`
}

if (payload == "[") {
    return;
}
else {
    payload = payload + "];"
    return { payload: payload };
}

Which produces a payload like this

Payload

[
        {
            measurement: 'power',
            fields: {
                value: 0
            },
            tags: {
                deviceName: Driveway_Lights,
                unit: 'Watts',
                type: 'float'
            }
        },
        {
            measurement: 'cumulative_power',
            fields: {
            value: 1.599
            },
            tags: {
                deviceName: Driveway_Lights,
                unit: 'kWh',
                type: 'float'
            }
        },
        {
            measurement: 'current',
            fields: {
            value: 0.007,
            tags: {
                deviceName: Driveway_Lights,
                unit: 'Amps',
                type: 'float'
            }
        },
        {
            measurement: 'volts',
            fields: {
                value: 121.461,
            tags: {
                deviceName: Driveway_Lights,
                type: 'float'
            }
        }];

But all is not well because the Influx Batch node produces this error with every payload

So the payload is not right and I can't tell what because my brain is exhausted so I'll take a break from this.

[UnborN]

Hi,

Im not sure what influx accepts .. but you can try filling in the missing value with null if its not a number (isNaN). Ternary operator used for shorter syntax (link)

const topicParts = msg.topic.split('/');
const device = topicParts[1]

const powerNum = 66049;
const cumulative_powerNum = 65537;
const currentNum = 66817;
const voltageNum = 66561;

var power = msg.payload.value["zwave/" + device + "/50/0/value/" + powerNum];
power = !isNaN(power) ? parseFloat(power) : null;

var cumulative_power = msg.payload.value["zwave/" + device + "/50/0/value/" + cumulative_powerNum];
cumulative_power = !isNaN(cumulative_power) ? parseFloat(cumulative_power) : null;

var current = msg.payload.value["zwave/" + device + "/50/0/value/" + currentNum];
current = !isNaN(current) ? parseFloat(current) : null;

var voltage = msg.payload.value["zwave/" + device + "/50/0/value/" + voltageNum];
voltage = !isNaN(voltage) ? parseFloat(voltage) : null;


var payload = [
    {
        measurement: "power",
        fields: {
            value: power
        },
        tags: {
            deviceName: device,
            unit: "Watts",
            type: "float"
        }
    },
// The other 3 values go here**
];

return msg;

ps. what were you trying to do with those template strings ? concatenating strings to create an object (reminded me of my struggles when i started programming )

@LosinIt

because my brain is exhausted so I'll take a break from this

hehe .. no pain no gain

[LosinIt]

Thanks for chiming in.

@UnborN

Im not sure what influx accepts .. but you can try filling in the missing value with null if its not a number (isNaN).

I'm not sure either. In fact, after this jumble I'm not sure of anything!

I'm aware of the Ternary operator construction and I use it after I get things working without it because I can easily misread it.

I made a living by coding with many languages but never JS and I get confused because, IMO, there are too many ways to skin the cat and examples I find are a mix of them. OK, full disclosure: I'm almost 70 and new tricks aren't so easy to learn anymore. I know lots of folks my age whose minds are as sharp as ever. Sadly, I'm not one of them.

@UnborN

ps. what were you trying to do with those template strings ? concatenating strings to create an object

Seems the easy way to convert what I had before into what @Colin suggested (I think) but since it's not working I think the string is the root problem so I'll see what I get when I try to add objects to an array instead. Is that what you suggest?

[UnborN]

@LosinIt

I know lots of folks my age whose minds are as sharp as ever. Sadly, I'm not one of them.

on the contrary sir your approach on solving the coding problem was correct ..
use the same logic but instead of concatenating strings
you can start with an empty array (let influxdata = []) and .push() a JS object to it, if it satisfies the condition (as you had)

Modified Function

const topicParts = msg.topic.split('/');
const device = topicParts[1]

const powerNum = 66049;
const cumulative_powerNum = 65537;
const currentNum = 66817;
const voltageNum = 66561;

var power = msg.payload.value["zwave/" + device + "/50/0/value/" + powerNum];
var cumulative_power = msg.payload.value["zwave/" + device + "/50/0/value/" + cumulative_powerNum];
var current = msg.payload.value["zwave/" + device + "/50/0/value/" + currentNum];
var voltage = msg.payload.value["zwave/" + device + "/50/0/value/" + voltageNum];

let influxdata = [] // initialize influx data

if (!isNaN(power)) {
    influxdata.push({
        measurement: "power",
        fields: {
            value: parseFloat(power)
        },
        tags: {
            deviceName: device,
            unit: "Watts",
            type: "float"
        }
    })
}

if (!isNaN(cumulative_power)) {
    influxdata.push({
        measurement: 'cumulative_power',
        fields: {
            value: parseFloat(cumulative_power)
        },
        tags: {
            deviceName: device,
            unit: 'kWh',
            type: 'float'
        }
    })
}

if (!isNaN(current)) {
    influxdata.push({
        measurement: 'current',
        fields: {
            value: parseFloat(current),
        },
        tags: {
            deviceName: device,
            unit: 'Amps',
            type: 'float'
        }
    })
}

if (!isNaN(voltage)) {
    influxdata.push({
        measurement: 'volts',
        fields: {
            value: parseFloat(voltage),
        },
        tags: {
            deviceName: device,
            type: 'float'
        }
    })
}


msg.payload = influxdata;

return msg;

[LosinIt]

I think I've got it! Your jab about strings clued me in to what I was doing wrong so thank you for that.

This is what I have now and, so far, it's doing exactly what I was trying to accomplish

Code

const topicParts = msg.topic.split('/');
const device = topicParts[1]

const powerNum = 66049;
const cumulative_powerNum = 65537;
const currentNum = 66817;
const voltageNum = 66561;

var path = msg.payload.value["zwave/" + device + "/50/0/value/" + powerNum];
const power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + cumulative_powerNum];
const cumulative_power = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + currentNum];
const current = parseFloat(path);

path = msg.payload.value["zwave/" + device + "/50/0/value/" + voltageNum];
const voltage = parseFloat(path);

var payload = ;
var index = 0;

if (!isNaN(power)) {
payload[index] =
{
measurement: 'power',
fields: {
value: power
},
tags: {
deviceName: device,
unit: 'Watts',
type: 'float'
}
}
index++;
}

if (!isNaN(cumulative_power)) {
payload[index] =
{
measurement: 'cumulative_power',
fields: {
value: cumulative_power
},
tags: {
deviceName: device,
unit: 'kWh',
type: 'float'
}
}
index++;
}

if (!isNaN(current)) {
payload[index] =
{
measurement: 'current',
fields: {
value: current
},
tags: {
deviceName: device,
unit: 'Amps',
type: 'float'
}
}
index++;
}

if (!isNaN(voltage)) {
    payload[index] =
    {
        measurement: 'volts',
        fields: {
            value: voltage
        },
        tags: {
            deviceName: device,
            type: 'float'
            }
    }
index++
}

if (index == 0) {
    return;
}
else {
    return { payload: payload };
}