Join node multiple problems / questions

Hi,

I'm trying to use the join node without having previously split a message into parts.
I first tried to perform a manual combination to a key/value object, with a msg.uid key, which should be unique.
My condition to finish the join is to have exactly 2 messages.
And this does not work.

I simplified my flows to only use some inject nodes to test the join node.
What I get from these tests is:

  • only the msg.complete property triggers the join, not the count, which is a problem for me
  • if I use a merged object for my result, the count condition works, but only for the first time:
    • the first msg is buffered in the join node
    • the second one trigger the join
    • the third one trigger the join, even if I didn't the 'subsequent message option'

I didn't find any issue on the github repo about these probems.
Should I report a bug?

I use Node-RED v1.0.5, running on a Rpi 3b, with ArchlinuxARM. NodeJS v10.
Even on my WSL (Ubuntu) install, the behavior is the same.

Can you share your inject based example thanks

Hi @dceejay,

Thanks for your answer.
Here is my test flow:

[{"id":"465bf6d5.382638","type":"join","z":"295d70f2.01c4b","name":"join to key/value obj on 2 msgs","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":570,"y":600,"wires":[["16b0329b.f87d25"]]},{"id":"16b0329b.f87d25","type":"debug","z":"295d70f2.01c4b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":770,"y":600,"wires":[]},{"id":"eb31ed8c.becb88","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"1","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":480,"wires":[["465bf6d5.382638"]]},{"id":"5711a091.217af8","type":"join","z":"295d70f2.01c4b","name":"join to merged obj on 2 msgs","mode":"custom","build":"merged","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":740,"y":660,"wires":[["5a4d49cb.07e19"]]},{"id":"5a4d49cb.07e19","type":"debug","z":"295d70f2.01c4b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":950,"y":660,"wires":[]},{"id":"96f344a4.ddd35","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"2","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":520,"wires":[["465bf6d5.382638"]]},{"id":"4f3f57d5.96d96","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"3","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":560,"wires":[["31f6a669.031942"]]},{"id":"e42b4639.237c18","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic2","payload":"4","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":600,"wires":[["465bf6d5.382638"]]},{"id":"640cb8fe.5f5608","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"{\"first_content\" : 1}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":660,"wires":[["5711a091.217af8"]]},{"id":"31f6a669.031942","type":"change","z":"295d70f2.01c4b","name":"","rules":[{"t":"set","p":"complete","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":270,"y":560,"wires":[["465bf6d5.382638"]]},{"id":"898a53c0.94985","type":"change","z":"295d70f2.01c4b","name":"","rules":[{"t":"set","p":"complete","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":410,"y":740,"wires":[["5711a091.217af8"]]},{"id":"3542d6ea.6c0e3a","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"{\"second_content\" : 2}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":700,"wires":[["5711a091.217af8"]]},{"id":"76c89b43.62d2dc","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic2","payload":"{\"fourth_content\" : 1}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":780,"wires":[["5711a091.217af8"]]},{"id":"ac5f5189.ba88f","type":"inject","z":"295d70f2.01c4b","name":"","topic":"topic1","payload":"{\"third_content\" : 3}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":740,"wires":[["898a53c0.94985"]]}]

I actually tested it again this morning. And the join output is triggeres when 2 messages with 2 different topics are received. Is this the way a join node should work?
If yes, how can I join messages based on a unique key?

My use case is to generate a totp code for a device and store the secret key to be able to check the code later.
Workflow is:

  1. Ask for a device registration, with a UUID
  2. Generate a secret key (msg with secret is hold in a join node, using the UUID for correlating messages)
  3. Generate the QRcode
  4. The device flashes the QRcode to store it
  5. The device generates and sends a TOTP to NodeRED, giving its UUID
  6. Join is triggered (2 messages with the same UUID) -> this is my problem
  7. The TOTP is checked by generating a TOTP on the server with the stored secret key
  8. If codes matches, store the secret and uuid in a database

Thanks again for any hint.

If you click the join node once and read the information panel: the join node expects certain input:

To automatically join a sequence of messages, they should all have this property set. The split node generates this property but it can be manually created. It has the following properties:

  • id - an identifier for the group of messages
  • index - the position within the group
  • count - the total number of messages in the group
  • type - the type of message - string/array/object/buffer
  • ch - for a string or buffer, the data used to the split the message as either the string or an array of bytes
  • key - for an object, the key of the property this message was created from
  • len - the length of each message when split using a fixed length value

To get this information, use the batch node. The batch and split node will provide all the information the join node requires. (thus used in conjunction; batch+join or split+join)

Example flow

[{"id":"d955a21.d0779e","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"1","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":156,"y":240,"wires":[["641565e5.dcbdcc"]]},{"id":"ffea8058.3ee34","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"2","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":156,"y":280,"wires":[["641565e5.dcbdcc"]]},{"id":"42dce0eb.76814","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"3","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":156,"y":320,"wires":[["641565e5.dcbdcc"]]},{"id":"74cdebde.6e4ecc","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic2","payload":"4","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":156,"y":360,"wires":[["641565e5.dcbdcc"]]},{"id":"c867fd2e.d40868","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"{\"first_content\" : 1}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":206,"y":432,"wires":[["4c118f71.2a3608"]]},{"id":"12a7b576.f194bb","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"{\"second_content\" : 2}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":216,"y":480,"wires":[["4c118f71.2a3608"]]},{"id":"dad5e33f.3c9598","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic2","payload":"{\"fourth_content\" : 1}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":216,"y":576,"wires":[["4c118f71.2a3608"]]},{"id":"870a4831.4855e8","type":"inject","z":"491d28d8.4552f","name":"","topic":"topic1","payload":"{\"third_content\" : 3}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":206,"y":528,"wires":[["4c118f71.2a3608"]]},{"id":"ac759afd.2b1068","type":"join","z":"491d28d8.4552f","name":"","mode":"custom","build":"array","property":"","propertyType":"full","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":602,"y":288,"wires":[["e5ba804d.2b5ab"]]},{"id":"641565e5.dcbdcc","type":"batch","z":"491d28d8.4552f","name":"","mode":"count","count":"2","overlap":0,"interval":10,"allowEmptySequence":false,"topics":[],"x":482,"y":288,"wires":[["ac759afd.2b1068"]]},{"id":"6fcc788e.cb314","type":"join","z":"491d28d8.4552f","name":"","mode":"custom","build":"array","property":"","propertyType":"full","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":602,"y":480,"wires":[["e5ba804d.2b5ab"]]},{"id":"4c118f71.2a3608","type":"batch","z":"491d28d8.4552f","name":"","mode":"count","count":"2","overlap":0,"interval":10,"allowEmptySequence":false,"topics":[],"x":482,"y":480,"wires":[["6fcc788e.cb314"]]},{"id":"e5ba804d.2b5ab","type":"debug","z":"491d28d8.4552f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":794,"y":360,"wires":[]}]

So yes the node is working as designed... to join/merge things that have n different "keys".

for your scenario I would create my own function...
To start I would just save the key in flow context (via change node) - using the uuid as the key.
Then after qr generation a small function to read the flow context using then relevant key, do whatever combining I needed to do and send on to the database.

Ok, thanks for your explanation @dceejay.
Now I understand why it wasn't working.
I'm used to use EIP patterns at work, and I thought the Node-RED's join node would work like the aggregate pattern: https://camel.apache.org/components/latest/eips/aggregate-eip.html
Here you can see that the correlationExpression is required to aggregate messages together.
I think this would be nice to have such a node in Node-RED, this would avoid storing data in the flow context. I may propose this feature on github. What do you think?

EDIT: I found this library which might do what I want (https://flows.nodered.org/node/node-red-contrib-combine), I'll check it out.

Well the join node does have a reduce mode


which can probably do it... but you need to be up on JSONata skills as well to work it out...
(I'm not :slight_smile: - though the info side bar gives some examples.

This is a good idea, I'll try it a try.
But I'm also a not very skilled in JSONata :thinking: :slightly_smiling_face:
I'll report here if I find a solution (I may change the topic to match my use case more precisely)

Thanks again for your time and your anwers!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.