Clarifying use of split node

[fmarzocca]

Hi, I need a clarification on use of split node.
I have an incoming message like this one:
*#*1##*#*1##*5*1*##*5*5*##*5*7*##*5*9*##*5*18*#1##*5*18*#2##*5*18*#3##*5*18*#4##*5*11*#5##*5*11*#6##*5*11*#7##*5*11*#8##

and I need to split it into several frames using "##" as a separator. I thought to use a split node splitting on "##", then (at output) add again the "##" to fill the frames back:

msg.payload = msg.payload.concat("##");

In such a way I should have a sequence of frames, from the original single message. Is it correct? Do I need a join node (and why?)

[Colin]

What comes out of the split node and what comes out of the function node adding it back?

[Colin]

It works ok for me

[{"id":"8ccd7975.e2417","type":"inject","z":"84405ff5.25fa6","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"*#*1##*#*1##*5*1*##*5*5*##*5*7*##*5*9*##*5*18*#1##*5*18*#2##*5*18*#3##*5*18*#4##*5*11*#5##*5*11*#6##*5*11*#7##*5*11*#8##","payloadType":"str","x":100,"y":1480,"wires":[["febf1037.257068"]]},{"id":"febf1037.257068","type":"split","z":"84405ff5.25fa6","name":"","splt":"##","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":230,"y":1480,"wires":[["940431f2.fa6de"]]},{"id":"940431f2.fa6de","type":"function","z":"84405ff5.25fa6","name":"","func":"msg.payload = msg.payload.concat(\"##\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":360,"y":1480,"wires":[["4c5b48fb.c32b1"]]},{"id":"4c5b48fb.c32b1","type":"debug","z":"84405ff5.25fa6","name":"true","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":430,"y":1540,"wires":[]}]

[fmarzocca]

Yes, it works. Sorry, my question was not complete. I meant: is this the correct (and most efficient) way to do this?

[Steve-Mcl]

@fmarzocca

is this the correct (and most efficient) way to do this?

Correct?

That is subjective. It is probably a more "node-red" way of doing it.

---

Most efficient way?

no.
It is about 4 times slower than a straight function...

... the top flow is using split node - averages around 22ms
the botttom flow is using a function to get the same results - averages around 6ms

Function code...

msg.payload.split("##").forEach(e => { node.send({ payload: e + "##"}) });

---

I have asked before but never got an answer, what generates that long data string?
If you do, then I would personally suggest you stop working with all this string concatenation and splitting. Using strings like this is possibly one of the least efficient way of working.

[Colin]

Since you are already using a function node you might as well do it all in the one function node.

[fmarzocca]

I cannot avoid them. They are generated by a closed proprietary home automation server (Legrand) and I have no access to edit its frames.
Thank you!

[Steve-Mcl]

(speculation) In which case, depending on how much processing and passing you do, it might be wise to generate objects at the start of your flows and pass those around.

[fmarzocca]

that's not easy... Messages can be like the one I am reporting at the first post, or just singles frames. Messages length is not fixed. The only standard thing is that each frame is closed by "##"...

[Steve-Mcl]

Sure it's easy. Explain to me a few of those frames and I'll show you. (I'll need a bit context too, like what you use the parts for)

[fmarzocca]

Each frame has its own meaning (automation/lights/alarm/contacts/etc).
This is just a short excerpt from the log:

2021-03-23 16:51:32,288 - *#13**#1*02*23*03*2021##
2021-03-23 16:51:33,099 - *#13**#0*16*50*00*001##
2021-03-23 16:51:58,163 - *#1*0113*6*0##
2021-03-23 16:53:33,812 - *#4*200*15*1*0128*0001##
2021-03-23 16:56:57,655 - *#1*0113*6*0##
2021-03-23 16:58:22,615 - *#4*800*15*1*0215*0001##
2021-03-23 17:01:32,378 - *#13**#1*02*23*03*2021##
2021-03-23 17:01:33,176 - *#13**#0*17*00*00*001##
2021-03-23 17:01:57,156 - *#1*0113*6*0##
2021-03-23 17:06:56,594 - *#1*0113*6*0##
2021-03-23 17:09:05,607 - *25*32#1*385##
2021-03-23 17:11:09,525 - *#4*200*15*1*0125*0001##
2021-03-23 17:11:32,903 - *#13**#1*02*23*03*2021##
2021-03-23 17:11:33,972 - *#13**#0*17*10*00*001##
2021-03-23 17:11:56,042 - *#1*0113*6*0##
2021-03-23 17:13:30,610 - *1*34*0113##
2021-03-23 17:13:34,661 - *1*34*0113##
2021-03-23 17:13:37,142 - *25*31#1*311##

[fmarzocca]

Thank you Steve for your kind support!

[fmarzocca]

@Steve-Mcl , what did you use to measure flow performances? I may be interested in other measures.

[Steve-Mcl]

It is a simple subflow i knocked up for times like this

You can grab it from here...

https://flows.nodered.org/flow/63f29373e826a9fd4943d679d43e725f

[fmarzocca]

Cool!

From my tests it seems that substr() or IndexOf() is faster than match(regex). Do you agree?

[Steve-Mcl]

@fmarzocca

it seems that substr() or IndexOf() is faster than match(regex). Do you agree?

That would make sense. regex is (can be) a complex beast. substr and indexOf are simple array slicing (as in an array of chars not a JS array)

however, as i have said before, working with strings in throughout your flow not ideal. You really should parse your strings (at the first opportunity) into usable/sensible objects with string/number/bool props (as appropriate) (for of a few reasons I'll not go into again)

[fmarzocca]

Yes, I'm following your advise, that's why I am testing regex and string slicing in order to transform the initial string frame into an object. Regex is much easier for this, while string slicing involve much more attention and can possibly introduce errors.

As an example, I am transforming the frame (string) "*25*32#1*341##" into this:

With regex, it is a simple matter of this function:

var r=/^\*(\d+)\*(\d+)#(\d)\*3(\d+)##/g
if (!msg.payload.match(r)) return;
var m= msg.payload.split(r);
msg.payload = {who: m[1], what: m[2], param: m[3], where:m[4]};
return msg;

While with indexOf() or substr() it is much more complex.