Send only on changes (for slow modbus poll) - newbie

[PPz]

I poll data from a modbus master. 1 poll with 20 consecutive addresses (for now, will increase that later) every 10 seconds.
My goal is to send them (MQTT) every minute, or even slower (but I have some modbus connection errors but it's not the purpose of this post). Some data can be changed very quickly (1 per minute) and some are very slow (1 per hour).

Look at my flow: is it the right way to control this ? I work as usual in "parcimony mode" because I don't want to send data if they are no changing. Those data are then sent to a Mosquitto MQTT broker and used in Hubitat.

I poll, I extract from the array separately, I create a new topic for each poll (because MQTT) and then I send them in 1 unique function who waits for a change. Then send to my MQTT broker.

*Edit: is it way to simplify this multiple extracting function by a loop ? I think this is the key:

node-red-contrib-loop-processing

Thank you for your answer (and maybe a link)

[Andrei]

@PPz

Look at my flow: is it the right way to control this ?

Sure it is. In the first place, your flow is easy to understand. Secondly, it works as you want. Thumbs up.

You mentioned that you are going to send data to MQTT every minute. In such a case you can consider using the delay node in the rate-limit mode. This way you do the traffic shaping in the output of the flow instead of messing with the modbus part.

Now, if you need to extract much more than 20 addresses then the flow can grow too large too quickly and it seems to me this is the reason you edited the flow asking how to simplify the extracting, right?

One possible way is by using a function node, like the one below. In that flow, you will hard code in the change node an array of topic names. The code in the function node will map the values coming from modbus to the topic names based on array indexes.

[{"id":"cca4f0d8.ff67b","type":"tab","label":"Map array of values to array of topics","disabled":false,"info":""},{"id":"a8a1f8b5.ccfbf8","type":"inject","z":"cca4f0d8.ff67b","name":"","topic":"","payload":"[\"a\",\"b\",\"c\",\"d\",\"e\"]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":190,"y":200,"wires":[["e87907b0.632788"]]},{"id":"e87907b0.632788","type":"function","z":"cca4f0d8.ff67b","name":"Map values to topics","func":"let pay = msg.payload;\nlet arrTopic = global.get(\"topic\");\n\nfunction maptopic(v,i,a)  {\n    return {\"payload\":v, \"topic\":arrTopic[i]};\n}\n\nlet arr = pay.map(maptopic);\n\nnode.warn(arr);\nreturn msg;","outputs":1,"noerr":0,"x":440,"y":200,"wires":[[]]},{"id":"ef8960d3.49ebf","type":"inject","z":"cca4f0d8.ff67b","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":120,"wires":[["d8ee374f.59c068"]]},{"id":"d8ee374f.59c068","type":"change","z":"cca4f0d8.ff67b","name":"","rules":[{"t":"set","p":"topic","pt":"global","to":"[\"SDM-a\",\"SDM-b\",\"SDM-c\",\"SDM-d\",\"SDM-e\",\"SDM-f\",\"SDM-g\",\"SDM-h\",\"SDM-i\",\"SDM-j\"]","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":340,"y":120,"wires":[[]]}]

[PPz]

@Andrei

your flow is easy to understand

Newbie, here

I looked at your function: I'm a little bit lost here but that reading will teach me how it processes.

I saw that you defined an array for the global topic. I think (think...) that it also could be generated in the loop itself (will change my hubitat to look for standardized topics instead of custom ones), like

example:
data flow: [12, 54, 325, 1217, 9, 665]
extracted values from the array: 12,54,325,1217,9,665

=> the topic would be something like modbus[loop.value], harvested from the loop itself

results:

msg.payload[0] with a topic equal to modbus0
msg.payload[1] with a topic equal to modbus1
msg.payload[2] with a topic equal to modbus2
...

@Andrei

you can consider using the delay node in the rate-limit mode

But if I use the "send only on change" function, is it a risk to loose data if in between the minute delay other data are coming to the function ? Other way to ask the question: is there a buffer ? Hmmm, will try by myself.

@Andrei

Now, if you need to extract much more than 20 addresses then the flow can grow too large

Yep. I program plc's and obviously it's easier (faster) to poll registers and slaves on a loop than separately. That's why the idea came in.

But because I'm new with node-red, I practice and learn by creating small functions and then optimizing them, then creating more difficult ones (because I made too many copy-paste in my flow).
Then (...) read the doc. I have 2 computers here open at the same time: 1 with my plc code (harvesting my registers), the second one with node-red and a lot of open tabs...

So, I'll dig a little bit more about this and review (and try to understand) your function. Thanks++

Mike

[Andrei]

@PPz

msg.payload[0] with a topic equal to modbus0
msg.payload[1] with a topic equal to modbus1
msg.payload[2] with a topic equal to modbus2

That would be straightforward.

[{"id":"6563cdec.eddf14","type":"inject","z":"cca4f0d8.ff67b","name":"","topic":"","payload":"[\"a\",\"b\",\"c\",\"d\",\"e\"]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":190,"y":280,"wires":[["a56f815e.6cfa3"]]},{"id":"a56f815e.6cfa3","type":"function","z":"cca4f0d8.ff67b","name":"Map values to topics","func":"let pay = msg.payload;\nlet arrTopic = global.get(\"topic\");\n\nfunction maptopic(v,i,a)  {\n    return {\"payload\":v, \"topic\":\"modbus\"+i};\n}\n\nlet arr = pay.map(maptopic);\n\nnode.warn(arr);\nreturn msg;","outputs":1,"noerr":0,"x":440,"y":280,"wires":[[]]}]

[PPz]

@Andrei

msg.payload[0] with a topic equal to modbus0

Thank you for your answer. English is not my mother-tongue so maybe you was mistaken by my explanation.

When I connect your function to my modbus and my mqtt broker, I obtain 1 message (various) with an array filled with all the values of the poll.
However, due to the way my mqtt works (and hubitat after), I need to split this and obtain in my loop several messages with a separate topic name and the value.

Instead of various = [2100, 1940, 10,40,0,0,0, etc], I need to obtain a serie of messages like this:

modbus1 = 2100
modbus2 = 1940
modbus3 = 10
modbus4= 40
modbus5=0
...

Obviously in this case I splitted the array to assign a value (from the array) and name the topic as the position of the poll (in place of specific topic names I have now). I can change the specific names I have now for modbus[n] because I can change the way my hubitat handle those names.

I think it's possible and you don't need to write a function for me. Some tips will teach me what to understand to create such function. I program plc's (not ladder, kind of basic language) and I'm familiar with IF/THEN/WHILE/... and loop management. Ok, Node-red is pretty new for me and I need to learn the differences, but I'll digest it one day or another...

*Edit: could this contrib (the array-loop) be useful for my purpose ?

node-red-contrib-loop-processing

Mike

[Trying_to_learn]

@PPz

Instead of various = [2100, 1940, 10,40,0,0,0, etc], I need to obtain a serie of messages like this:

modbus1 = 2100
modbus2 = 1940
modbus3 = 10
modbus4= 40
modbus5=0

Hi.

I don't want to just butt in, but here goes.

It may be better sending the messages over MQTT like that.
Pulling the message apart is not that difficult. Use the split node and separate them with a ,

Then the node spits out a sequence of messages similar to what you want.

Also if how you are creating the message in the flow leads to that kind of format, then it may be better with just the extra split node at the receiving end of the message.

(I'm not really that good with Node-Red, and I have just jumped into this a fair way down. So please understand I may not really be helping. But I am pointing out something I see.)

[PPz]

Tagging @Andrei,

Thank you. I checked and asked on my side. It is theorically possible to send 1 message with an array of values (and it's probably faster that way), but if I need to obtain the split values like it was before, I need to add another layer on my Hubitat.
So, I'll stick for the moment by sending more messages with just one topic who has a value instead of a long array.

I need now, obviously, to learn how to split the array created by the loop before sending it to my broker.
Another task on my todo list

Ok, baby steps again because I need to learn a "lot" of functions to fullfil my task.
I start again with a small poll (not looped), add some delay (because I don't need to update frequently those values) and also add a filter to send data to MQTT only if they are changing.
That will limit the (useless) to my Hubitat and MQTT broker/App.
Sounds good ?

For unknow reasons, when I decrease the poll rate directly on the modbus side (like 1 poll every minute instead of 1poll/10 sec), I always obtain some connection errors. Seems strange to me because normally, when you lower the poll, you have less errors. But I did what Andrei suggested by limiting the flow in the output.