How to split/join after discarding values

[cverbiest]

I'm a NodeRED newbie trying to figure out how to split an array, select some values and join the selected values back into an array.
My sample works fine if I select all values.
I discard values I don't get a result.

Extra questions :

• Can I split the values directly without the values2payload change

[{"id":"f20aa3b6.3ed58","type":"inject","z":"def5c4c1.d0947","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"values\":[{\"name\":\"1 keep\",\"keep\":\"true\"},{\"name\":\"2 keep\",\"keep\":\"true\"},{\"name\":\"3 drop\",\"keep\":\"false\"},{\"name\":\"4 keep\",\"keep\":\"true\"},{\"name\":\"5 drop\",\"keep\":\"false\"}]}","payloadType":"json","x":90,"y":80,"wires":[["b18d8293.ceaa1"]]},{"id":"27f5d0bb.200d3","type":"split","z":"def5c4c1.d0947","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":430,"y":80,"wires":[["a23e9796.102ba8"]]},{"id":"b18d8293.ceaa1","type":"change","z":"def5c4c1.d0947","name":"values2payload","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.values","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":260,"y":80,"wires":[["27f5d0bb.200d3"]]},{"id":"a23e9796.102ba8","type":"switch","z":"def5c4c1.d0947","name":"","property":"payload.keep","propertyType":"msg","rules":[{"t":"eq","v":"true","vt":"str"},{"t":"eq","v":"false","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":110,"y":160,"wires":[["ef3c5089.bf875"],[]]},{"id":"ef3c5089.bf875","type":"join","z":"def5c4c1.d0947","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":270,"y":160,"wires":[["606018e2.810f"]]},{"id":"606018e2.810f","type":"debug","z":"def5c4c1.d0947","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":430,"y":160,"wires":[]}]

[knolleary]

Hi @cverbiest

at the bottom of the Switch node is a checkbox labelled recreate message sequences - that's the option you want to enable. It will repair the sequence after elements have been removed from it, allowing the Join node to operate on what's left and not sit waiting for the removed elements to arrive.

[Colin]

It is often a good idea to use core nodes rather than dropping down into javascript in a function node, but in case you are interested in an alternative solution, if you put this in a function node it will do it all for you, using the javascript Array filter method to filter out the ones you don't want.

msg.payload = msg.payload.values.filter(element => element.keep === "true")
return msg;

[cverbiest]

Thanks to Colin and knolleary both answers work. Unfortunately I had simplified my real case too much, for more info see TL/DR.
I have about 5 switches with different paths that lead to the join node.

I changed my sample to 2 switches, in reality there are more.

[{"id":"6d4adf3e.d21ed8","type":"inject","z":"9ffb9782.fff388","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"values\":[{\"name\":\"1 on main project\",\"project\":\"main\",\"reviewer\":\"me\"},{\"name\":\"2 on main project\",\"project\":\"main\",\"reviewer\":\"notme\"},{\"name\":\"3 on sideproject but I review\",\"project\":\"sideproject\",\"reviewer\":\"me\"},{\"name\":\"4 on main project\",\"project\":\"main\",\"reviewer\":\"notme\"},{\"name\":\"5 drop\",\"on main project\":\"false\",\"reviewer\":\"notme\"}]}","payloadType":"json","x":90,"y":100,"wires":[["ffee5f2a.49879"]]},{"id":"d962a3d.8cb70e","type":"split","z":"9ffb9782.fff388","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":430,"y":100,"wires":[["d1f48e37.14091"]]},{"id":"ffee5f2a.49879","type":"change","z":"9ffb9782.fff388","name":"values2payload","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.values","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":260,"y":100,"wires":[["d962a3d.8cb70e"]]},{"id":"d1f48e37.14091","type":"switch","z":"9ffb9782.fff388","name":"main project ?","property":"payload.project","propertyType":"msg","rules":[{"t":"eq","v":"main","vt":"str"},{"t":"else"}],"checkall":"false","repair":true,"outputs":2,"x":160,"y":180,"wires":[["8d6748d9.de215"],["9cd456.a9075ba8"]]},{"id":"8d6748d9.de215","type":"join","z":"9ffb9782.fff388","name":"","mode":"auto","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"1","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":410,"y":180,"wires":[["9503410e.602ca8"]]},{"id":"9503410e.602ca8","type":"debug","z":"9ffb9782.fff388","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":590,"y":180,"wires":[]},{"id":"9cd456.a9075ba8","type":"switch","z":"9ffb9782.fff388","name":"reviewer = me ?","property":"payload.reviewer","propertyType":"msg","rules":[{"t":"eq","v":"me","vt":"str"},{"t":"else"}],"checkall":"true","repair":true,"outputs":2,"x":280,"y":260,"wires":[["8d6748d9.de215"],[]]}]

With the sample I expect an array with 4 entries, 3 from project main + 1 where I review.
With recreate message sequence I get only the 3 from project main.
I have not tried a function, but I want to keep the decisions in individual nodes, not one node for all conditions.

I found I could solve it with join mode=manual + a timeout of 0.1 second but if I seet the timeout too high it slows everything down, if I set it too low I miss results so I'd prefer not to use that.

TL/DR
I'm calling a BitBucket server api that returns all current pull requests, I filter those requests on various criteria

• Always keep requests from the main project
• don't look at requests that have merge conflicts
• don't look at requests marked "need work"
• dont look at requests with open task
• ...

I'll test a different approach where I don't drop messages immediately to see if I can get it working that way.

[cverbiest]

I was able to get it working by

1. not dropping messages before the join node

2. Add a logical property "keep" to the messages and use that to filter the joined array.
   

   

[{"id":"6d4adf3e.d21ed8","type":"inject","z":"9ffb9782.fff388","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"values\":[{\"name\":\"1 on main project\",\"project\":\"main\",\"reviewer\":\"me\"},{\"name\":\"2 on main project\",\"project\":\"main\",\"reviewer\":\"notme\"},{\"name\":\"3 on sideproject but I review\",\"project\":\"sideproject\",\"reviewer\":\"me\"},{\"name\":\"4 on main project\",\"project\":\"main\",\"reviewer\":\"notme\"},{\"name\":\"5 drop\",\"on main project\":\"false\",\"reviewer\":\"notme\"}]}","payloadType":"json","x":90,"y":40,"wires":[["ffee5f2a.49879"]]},{"id":"d962a3d.8cb70e","type":"split","z":"9ffb9782.fff388","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":430,"y":40,"wires":[["e914e32a.1bea68"]]},{"id":"ffee5f2a.49879","type":"change","z":"9ffb9782.fff388","name":"values2payload","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.values","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":260,"y":40,"wires":[["d962a3d.8cb70e"]]},{"id":"d1f48e37.14091","type":"switch","z":"9ffb9782.fff388","name":"main project ?","property":"payload.project","propertyType":"msg","rules":[{"t":"eq","v":"main","vt":"str"},{"t":"else"}],"checkall":"false","repair":false,"outputs":2,"x":120,"y":100,"wires":[["8d6748d9.de215"],["9cd456.a9075ba8"]]},{"id":"8d6748d9.de215","type":"join","z":"9ffb9782.fff388","name":"","mode":"auto","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"0.1","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":370,"y":160,"wires":[["e11ef816.b622b8"]]},{"id":"9cd456.a9075ba8","type":"switch","z":"9ffb9782.fff388","name":"reviewer = me ?","property":"payload.reviewer","propertyType":"msg","rules":[{"t":"eq","v":"me","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":120,"y":160,"wires":[["8d6748d9.de215"],["1b121390.d2eb4c"]]},{"id":"e914e32a.1bea68","type":"change","z":"9ffb9782.fff388","name":"keep = true","rules":[{"t":"set","p":"payload.keep","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":40,"wires":[["d1f48e37.14091"]]},{"id":"1b121390.d2eb4c","type":"change","z":"9ffb9782.fff388","name":"keep = false","rules":[{"t":"set","p":"payload.keep","pt":"msg","to":"false","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":130,"y":220,"wires":[["8d6748d9.de215"]]},{"id":"e11ef816.b622b8","type":"function","z":"9ffb9782.fff388","name":"filter keep","func":"msg.payload = msg.payload.filter(element => element.keep == true)\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":520,"y":160,"wires":[["6bc6c221.f13624"]]},{"id":"6bc6c221.f13624","type":"debug","z":"9ffb9782.fff388","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":690,"y":160,"wires":[]}]