I use MQTT to receive a string that contains sensor data. That string is formatted like this:
"ID;TIME;VAL1;VAL2;VAL3;VAL4;VAL5" where the capitalized names are strings of varying length.
So far I was able to split my message in parts and, using the switch node, send the data to text labels and graphs.
But now I would like to also send that data, or parts of it, to a telegram channel and / or to influxDB.
I'm having a bit of trouble getting my head around how I should prepare my data in order to send it.
As far as I understand, for e.g. Telegram I need my data to be in a msg.payload with a payload of {"value1": "VAL1", "value2": "VAL2,...}. But how would I do that?
I tried to set global variables but that didn't work.
I'm not really experienced in javascript programming, I do most of my novice programming work in Python or micropython.
I'd be really thankful for any pointers in the right direction
I do have control of the source. It's a ESP32 with sensors in a 3D printed box. I just thought I'd get away with a little software tinkering and maybe learn something in the process.
Here's an example of actual data, I just redacted the IP address information:
ESP32;8.09.2021-08:06:19;22.10C;50.13%;943.33hPa;35.3125;192.1xx.xxx.xx
If I'd change my sent data, I'd have to change it into a JSON-friendly format by using key:value-pairs I guess?
There is a trade off here between the additional overhead of constructing the JSON string on the EPS vs the overhead of parsing the string in Node-RED.
So it very much depends on what you're most comfortable doing as ultimately you have to maintain it.
To answer your question on how to do the parsing in JavaScript, the following should work for you in a Function node.
// Assume msg.payload is "ID;TIME;VAL1;VAL2;VAL3;VAL4;VAL5"
// Split the string into an array of values using ; as the delimiter
var parts = msg.payload.split(";");
// Set msg.payload to a new object ready to populate
msg.payload = {};
// Assume id and time are the first two values
msg.payload.id = parts.shift();
msg.payload.time = parts.shift();
// Now the rest of 'parts' are value1, value2 ... etc.
// Loop over them and add them to msg.payload:
parts.forEach(function(value,index) {
msg.payload["value"+(index+1)] = value;
});
// Return the message
return msg;
For future reference, the MDN site provides a good reference of the JavaScript String object (and everything else in the language...): String - JavaScript | MDN
Yes, if you sent something like
{"id": "ESP32", "timestamp": "8.09.2021-08:06:19", "temperature": 22.1, ....}
then you could tell MQTT In node to parse it as json and it would give you it in a javascript object.
Either way it is usually best not to include the C, hPa, % etc in the value, it just complicates things.
One advantage of using this technique over a function node in node-red is that if you ever change the data by, for example, adding another field, then you don't need to make a change to the decoding in node-red (which involves trying to remember how the function node works).
Okay, I'll make some editing to my sent data also. The units in the data fields are a result of me being lazy, as the library I use on the ESP32 (micropython) returns the sensor data with units.
But stripping these off in uPy will be less of a problem for me than trying to strip them in Javascript
In the JSON string, for any that are numbers, do not put the values in quotes. Then they will be automatically converted to number type rather than string in the javascript object produced.
Ah, I see. Just for understandig better, if I'm using a split function like in the above example, how would I access individual parts of the split?
In python, I'd just use parts[0] for the first part etc, but how would I do that here? In my case here, I know my message length will be always the same, with the location of the data fixed as well. So I could assign a variable to a fixed part and output that.
My goal is to have a message with three value fields for IFTTT, and discard others. Of course my first thought was to use a for loop that begins at index number X and goes to Y, but I don't know how that would work out here too there's a lot to learn for me
Also that ; on every line end is killing me, it's super inconvenient Guess I'm spoiled by Python (and VisualBasic 6, if one could call that spoiled..)
I'd thought it's necessary for that language? At least it's in every example for NodeRED functions I've seen
Okay, that's great to know
Yeah, I'd have guessed that. But when I wrote my sensor platform, I didn't have that usecase in mind and just sent plain strings to be processed by another python script on a target platform. NodeRED was just to be used for some nice gauges
Do you mean in the javascript language, as used in function nodes? That is nothing to do with the format of any data that you choose to send. In fact in modern javascript the semicolon is mostly only necessary if you choose to put multiple statements on one line, which is generally not advisable anyway.
Yes, I thought it was necessary in the function nodes. These are node.js or javascript, if I googled correctly
It's one of the reasons I never got much into C or Arduino, it's a bit too abstract sometimes for me. I find Python to be much more reasonable at times.
I now have understood, or at least I think I have, how to split my message and partially how to handle my split parts. I can also send selected data parts to different outputs of the function node. Where I'm struggling is the construction of a valid msg.payload object from select parts.
I did try to put together a string, but that did not work out as expected.
I've managed to get NodeRED to do what I want by simply declaring the parts I want as part of payload using
msg.payload.value1 = parts[0]
and so on. That's working like a charm. However, I still do have the unit chars in there. Let's say I want to remove those from the string or what it is. In Python, I could do that easily by going
msg.payload.value1 = parts[0][:-1] to remove the last char of the string.
Is something similar possible here?
Of course, I know it'd be easier if I'd have the received data being formatted correctly, but first I don't want to change the code on the sender at the moment and second I'm trying to get to know NodeRED a bit better
Thank you for pointing that out to me. By using a step in between, I'm now able to strip units off of my measurements, so it'll be easier to send them to influxdb.
I now use
var temp = parts[0].slice(0, -1) before declaring the payload value with
msg.payload.value1 = temp
You could do that in one line if you want to msg.payload.value1 = parts[0].slice(0, -1)
If you are sending it to Influx you probably also want to convert it to a number from a string, in which case you can use
Good point, I was already looking up how to convert strings to numbers
It's working now with influxdb, which I handle in a separate function node. Coincidentally, although no changes where made to the IFTTT / Telegram handling function node, I do not receive telegram messages anymore.
It just never ends
edit: I just checked with IFTTT, and even when manually triggering an event in the "test" section of IFTTT, it says "successfully triggered" but still does not send a message. Telegram works fine, MQTT too. It's just the IFTTT part that doesn't work. Strange.
The issue with IFTTT has been resolved. I don't know how but it has been. All of a sudden all messages came through at once. Maybe my NodeRED instance had trouble...
and maybe learn something in the process.
there's a lot to learn for me 
Guess I'm spoiled by Python (and VisualBasic 6, if one could call that spoiled..)
