# Handling split/join node logic when "losing" split msgs is expected

**URL:** <https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362>\
**Category:** General\
**Tags:** split\
**Created:** [21 September 2021 22:44 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362 "2021-09-21T22:44:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![utmostGrandPoobah](https://avatars.discourse-cdn.com/v4/letter/u/43a26b/32.png) [@utmostGrandPoobah](https://discourse.nodered.org/u/utmostGrandPoobah)\
**Post date:** [21 September 2021 22:44 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/1 "2021-09-21T22:44:01Z")

</div>

**Use-case** : I am `split`'ing an array of recordIds to make sequential API requests (YouTube endpoint) and then evaluating the JSON responses, specifically determining if an array has any items in it. If the array is empty, then I "drop" that msg containing the response with the empty array (there's no further action my flow can perform on it

**Problem** : as you can begin to intuit, dropping msgs becomes a problem when attempting to aggregate them together again using the `join` node which in this case is not propagating the msgs as expected because the node is seeing that not all parts for that sequence/runtime are present

**Questions:**

1. I am considering using the `flow.get()` state to effectively track how many items are in the 'macro' array of channels that I am querying, and then subtract from that array for any msgs that are dropped to determine if the `msg.complete` should be assigned to a specific msg - am I overthinking this?
2. I'm aggregating these values so I can iterate thru them and perform batch updates to my datastore. I wonder if I should just scrap the `join` node and shoot them all off one by one but I see this becoming a massive scaling/volume problem over time - are there any other patterns I should be considering?

Edit: I read this post from earlier this year and tried to follow through with @knolleary's suggestion of using a `switch` node with the `preserve sequence` checkbox checked and unchecked, but neither approach had the expected outcome I'm looking for.

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [21 September 2021 23:03 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/2 "2021-09-21T23:03:02Z")

</div>

Is msg.parts making it to the join node from the split node?

---

<div class="post-metadata">

**Author:** ![utmostGrandPoobah](https://avatars.discourse-cdn.com/v4/letter/u/43a26b/32.png) [@utmostGrandPoobah](https://discourse.nodered.org/u/utmostGrandPoobah)\
**Post date:** [21 September 2021 23:39 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/3 "2021-09-21T23:39:54Z")

</div>

yep! But keep in mind, msg.parts is a property on each msg, so each msg's `part` property would have to be evaluated against some kind of 'other' state -- at least that's how I currently understand it

edit: Clarifying my point above: if one or more msgs doesn't make it to the `join` node, then the way I understand it is that there is an incomplete number of `part`s and so all the msgs are "stuck" in the `join` node until I manually send a `msg.complete` (true bool) msg to release them all and propagate to the next node in the flow

---

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [22 September 2021 00:12 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/4 "2021-09-22T00:12:37Z")

</div>

So-o...

I would put a node in there - somewhere, sorry haven't done too much home work on this exact scenario - and create a `msg.something` that holds the total number.

Then sends it to the `join` node setting how many messages are needed.  
(Sorry, this is q **quick** reply.)

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [22 September 2021 06:22 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/5 "2021-09-22T06:22:22Z")

</div>

Yes, but how are you removing the unwanted messages.  
Can we see an example flow?  
If you are using a switch node with "recreate sequence" msg.parts is adjusted for missing parts and the join node then receives the adjusted msg.parts payload.  
e.g.

```auto
[{"id":"d8cf3f78.d1f4c","type":"join","z":"b779de97.b1b46","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":610,"y":3060,"wires":[["71ce21dc.3f43a"]]},{"id":"20bc118e.c906f6","type":"switch","z":"b779de97.b1b46","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"num"}],"checkall":"true","repair":true,"outputs":1,"x":460,"y":3060,"wires":[["d8cf3f78.d1f4c"]]},{"id":"466b9d1c.13d084","type":"split","z":"b779de97.b1b46","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":270,"y":3000,"wires":[["20bc118e.c906f6"]]},{"id":"a312c696.a5b76","type":"inject","z":"b779de97.b1b46","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[1,2,1,1,1,2,1,1,2,1]","payloadType":"json","x":170,"y":3080,"wires":[["466b9d1c.13d084"]]},{"id":"71ce21dc.3f43a","type":"debug","z":"b779de97.b1b46","name":"","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"responseUrl","statusType":"msg","x":620,"y":3300,"wires":[]}]

```

---

<div class="post-metadata">

**Author:** ![utmostGrandPoobah](https://avatars.discourse-cdn.com/v4/letter/u/43a26b/32.png) [@utmostGrandPoobah](https://discourse.nodered.org/u/utmostGrandPoobah)\
**Post date:** [27 September 2021 17:57 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/6 "2021-09-27T17:57:05Z")

</div>

That's interesting, I tried the switch node with the `create sequence` option enabled and it did not impact the outcome, where I still had a bunch of msgs 'stuck' in the `join` node.

I might try this again on a separate mock flow, but I solved the problem by just sending _all_ msgs to the join node, and handling which data I wrote to the msg properties

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [26 November 2021 17:57 UTC](https://discourse.nodered.org/t/handling-split-join-node-logic-when-losing-split-msgs-is-expected/51362/7 "2021-11-26T17:57:47Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.