Join: make a complete array

Hello people,

I’ve got a question regarding the join and create an array out of x elements. What I want is to create an array out of two other elements. Until now I am able to make incomplete arrays.

I noticed when you set the join function to automatic and set in the msg.parts the proper values. In this case msg.parts.count = 2, msg.parts.id = aa55 and one has msg.parts.index = 0 and the other msg.parts.index = 1
The join will make a an array as soon the amount of msg.payloads that are received by the join met the value of msg.parts.count. When there are two msg’s with msg.parts.index = 1, it will create an array with [null, (2nd message of msg.parts.index = 1] and throws the first msg away.

What I want is that the join will create an array with the latest msg received from both flows. So, only create an array when index = 0 and index = 1 are arrived in the join node.

A small example. Two injections called Top and Bottom with msg.payload “Top” and “Bottom”. These will go to a join via change node that msg.parst.id, .count, and .index. And in the end debug node for msg.payload.
When I press bottom twice ther array [null, “Bottom”] is created.
When I press after that twice top an array [‘Top’] (no null) is created.
When I press Top and then Bottom, I have the perfect array [“Top”, “Bottom”].

I hope someone can help me.

[{"id":"f89328c7.f1a65","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"99c04129.058e1","type":"inject","z":"f89328c7.f1a65","name":"","topic":"test","payload":"Top","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":270,"y":160,"wires":[["74fb3e5a.6df3b"]]},{"id":"188f0069.4763f8","type":"join","z":"f89328c7.f1a65","name":"","mode":"auto","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":730,"y":200,"wires":[["3fd7a07b.ac1fd8"]]},{"id":"3fd7a07b.ac1fd8","type":"debug","z":"f89328c7.f1a65","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":950,"y":200,"wires":[]},{"id":"74fb3e5a.6df3b","type":"change","z":"f89328c7.f1a65","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"0","tot":"num"},{"t":"set","p":"parts.count","pt":"msg","to":"2","tot":"num"},{"t":"set","p":"parts.id","pt":"msg","to":"55aa","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":500,"y":160,"wires":[["188f0069.4763f8"]]},{"id":"e0843b9c.d3484","type":"change","z":"f89328c7.f1a65","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"1","tot":"num"},{"t":"set","p":"parts.count","pt":"msg","to":"2","tot":"num"},{"t":"set","p":"parts.id","pt":"msg","to":"55aa","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":500,"y":240,"wires":[["188f0069.4763f8"]]},{"id":"89d249c6.8c383","type":"inject","z":"f89328c7.f1a65","name":"","topic":"test","payload":"Bottom","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":290,"y":240,"wires":[["e0843b9c.d3484"]]}]

`

What would be the expected output if you press the inject nodes in the order: top + bottom + top ? Do you expect to see only one output or two outputs ?

I created a flow using the RBE node to block repeated msgs but I am not sure if it behaves the way you want. The function node was used only to display the status of the msg.parts.index (to easy the visualization).


[{"id":"868e7889.942fa8","type":"inject","z":"7582da4a.bf55c4","name":"","topic":"test","payload":"Top","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":187.50000762939453,"y":142.50000381469727,"wires":[["d2f68539.a9d1e8"]]},{"id":"d2f68539.a9d1e8","type":"change","z":"7582da4a.bf55c4","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"0","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":377.5000305175781,"y":142.5000171661377,"wires":[["ce9bf697.86fc18"]]},{"id":"426fb8ce.98a738","type":"change","z":"7582da4a.bf55c4","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"1","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":378.7500305175781,"y":241.25003051757812,"wires":[["ce9bf697.86fc18"]]},{"id":"43f67d5f.060f54","type":"inject","z":"7582da4a.bf55c4","name":"","topic":"test","payload":"Bottom","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180.00001525878906,"y":242.50000762939453,"wires":[["426fb8ce.98a738"]]},{"id":"6bd9933e.0eac1c","type":"rbe","z":"7582da4a.bf55c4","name":"","func":"rbe","gap":"","start":"","inout":"in","property":"parts.index","x":692.6250686645508,"y":196.2500171661377,"wires":[["31da1801.c65018"]],"outputLabels":["msg.parts.index"]},{"id":"26ffaf63.b9e8e","type":"debug","z":"7582da4a.bf55c4","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":958.7501258850098,"y":197.50000476837158,"wires":[]},{"id":"31da1801.c65018","type":"join","z":"7582da4a.bf55c4","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":826.3751220703125,"y":195.75003051757812,"wires":[["26ffaf63.b9e8e"]]},{"id":"ce9bf697.86fc18","type":"function","z":"7582da4a.bf55c4","name":"","func":"node.status({fill:\"red\",shape:\"ring\",text:\"index :\" + msg.parts.index});\nreturn msg;","outputs":1,"noerr":0,"x":563.8750648498535,"y":196.2500057220459,"wires":[["6bd9933e.0eac1c"]]}]


Alternatively if you accept msgs having different topics:


[{"id":"81801f21.9d5dd","type":"inject","z":"a4218b38.dfca28","name":"","topic":"test1","payload":"Top","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":221.25,"y":545,"wires":[["75d9e007.e4607"]]},{"id":"fea98989.6f09b8","type":"join","z":"a4218b38.dfca28","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":671.25,"y":585,"wires":[["86e32fdc.0003e"]]},{"id":"7fd8c3ed.dbb65c","type":"debug","z":"a4218b38.dfca28","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1063.7500305175781,"y":585.0000171661377,"wires":[]},{"id":"75d9e007.e4607","type":"change","z":"a4218b38.dfca28","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"0","tot":"num"},{"t":"set","p":"parts.count","pt":"msg","to":"2","tot":"num"},{"t":"set","p":"parts.id","pt":"msg","to":"55aa","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":441.25,"y":545,"wires":[["fea98989.6f09b8"]]},{"id":"3719487f.18b6e8","type":"change","z":"a4218b38.dfca28","name":"","rules":[{"t":"set","p":"parts.index","pt":"msg","to":"1","tot":"num"},{"t":"set","p":"parts.count","pt":"msg","to":"2","tot":"num"},{"t":"set","p":"parts.id","pt":"msg","to":"55aa","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":441.25,"y":625,"wires":[["fea98989.6f09b8"]]},{"id":"92309eaa.42a5d","type":"inject","z":"a4218b38.dfca28","name":"","topic":"test2","payload":"Bottom","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":231.25,"y":625,"wires":[["3719487f.18b6e8"]]},{"id":"86e32fdc.0003e","type":"function","z":"a4218b38.dfca28","name":"Change object to array","func":"var arr = Object.values(msg.payload);\nreturn {payload:arr};","outputs":1,"noerr":0,"x":867.6250686645508,"y":585.0000171661377,"wires":[["7fd8c3ed.dbb65c"]]}]

Are they always arriving as a pair ? or are you trying to create the array from randomly arriving items ? The id is what ties them together (when using a split) - so it can work out if it has parts 1 and 2 of a message id abc and put them in the correct order in the output array.

That is nifty piece of code.
It only works in the sequence Top and then Bottom.

The point is that I think that my problem is that I expect that the join count the different messages. So Top and Bottom (id 0 and 1). But the count is the total amount of messages and has nothing to do with the array length.

The problem came visible to me that I want to combine two sensors with a different time interval to a table in a database. The DB gave error’s that it can’t write null in to a column. It makes sens because my Lux measurement gives every 5 minutes a change, my temperature only when a change appears. So when temperature remains the same, but it gets darker I have a challenge. (It was fixable, with a flow variable, but I would like to see things solved in GUI design :wink: )

I think I should contact the developer of this join if he has plans to make an option to full array. Or start creating my own node for that.

OK, it would be better (imho) to use the join node in manual mode to create a key/value object using msg.topic as the key (and set different topics for each sensor) and set it to send after two inputs and then on any change… that will then produce a message whenever either sensor sends an input, containing the most recent two values in two properties referenced by their topics. eg {temperature: 25, lux:50}

Then construct your query to save to the database using those two properties - for example in the manner recently posted in another thread - MsgPayload into a query on nodered-sqlite (though other methods are perfectly valid)

why don’t you create an array of objects that get’s saved on a flow level variable (or global if it is what you need) and then you push new values on the object array, to avoid the null you could add the previous value of the ones that you have nothing.

That is not going to work at start. When Bottom sends twice a message the object for Top is still null.
I implemented something like this with a check. Sort circular buffer of 1 followed by a check if any element is not null.

Thanks. I have to take a dive into it. It looks interesting.

I am thinking to make a suggestion that the join gets an option to send the result only when every index received a new value or is not null. I don’t get it why join should send the result just by counting the messages and throws away the doubles instead checking that the array/list is full.