# Need help analyzing flow

**URL:** <https://discourse.nodered.org/t/need-help-analyzing-flow/28923>\
**Category:** General\
**Created:** [24 June 2020 23:28 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923 "2020-06-24T23:28:38Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [24 June 2020 23:28 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/1 "2020-06-24T23:28:39Z")

</div>

I have 3 Inject nodes will run every hour and 1 Inject node every 30 minutes.  
And all of them will flow into a switch node where I will check if there's an existing inject node running or not using a flow variable. I will let the flow continue from that inject node if flow.switch is empty. If not, it will loop and wait until the flow.switch is empty.

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/5/75aa61bd6a80a33f69b2966f2d5960c54ee032a8.png)

Now my question, is it possible that 2 or 3 injects nodes will simultaneously enter the "ON/OFF?" switch node? If that's possible then I have a problem...

---

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [24 June 2020 23:32 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/2 "2020-06-24T23:32:31Z")

</div>

Well,

As you used `link` nodes (which are good for some things) it makes it difficult to know who is connected to whom.

If all those 3 (4?) run every hour, there is a chance they will all send their message at the same time, because they are all _started_ when the flow is deployed. (Or machine booted)

As there doesn't seem to be any _exotic_ nodes, how about you post the flow and people may be able to get a better understanding of what is happening.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [24 June 2020 23:39 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/3 "2020-06-24T23:39:26Z")

</div>

> [@Trying\_to\_learn](#):
>
> If all those 3 (4?) run every hour, there is a chance they will all send their message at the same time, because they are all _started_ when the flow is deployed. (Or machine booted)

node-RED runs asynchronous, so no two actions run at the same time.

---

<div class="post-metadata">

**Author:** ![Trying\_to\_learn](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/trying_to_learn/32/28400_2.png) [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)\
**Post date:** [24 June 2020 23:42 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/4 "2020-06-24T23:42:15Z")

</div>

Ok, I stand corrected.

But if they are all set to repeat at 1 hour intervals...... The machine boots and......  
Every hour the trigger.

I think the actual flow would be better to understand though.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [24 June 2020 23:56 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/5 "2020-06-24T23:56:49Z")

</div>

I posted a screenshot of the flow..

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [24 June 2020 23:59 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/6 "2020-06-24T23:59:38Z")

</div>

Hi Paul!

Thanks for answering.  
Just to clarify, are you saying that these 3 inject nodes which runs every hour will never reach the "ON/OFF?" switch node at the same time?

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/5/0/50c32b2690c624b88e5857915fc60bc1e61bdd88.png)

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [25 June 2020 03:16 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/7 "2020-06-25T03:16:17Z")

</div>

Correct. Node js runs on a single thread, so although notionally "at the same time" two events will still be microseconds apart.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [25 June 2020 03:34 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/8 "2020-06-25T03:34:09Z")

</div>

Thanks @dceejay for the clarification.  
So that's why does using split node in large array, example with 20k elements, slows down nodered to a crawl for 2-4 minutes until it's done?

---

<div class="post-metadata">

**Author:** ![krambriw](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/krambriw/32/5429_2.png) [@krambriw](https://discourse.nodered.org/u/krambriw)\
**Post date:** [25 June 2020 04:34 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/9 "2020-06-25T04:34:59Z")

</div>

I guess situations like that encouraged the innovations like multi-threading and multi-processing. If you have such workloads that needs parallel processing, you may have to re-think about your solution design. You could split the functionality, let's say keep the major part in NR and put that "time critical" part into a Python script or another external module that supports parallel processing

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [25 June 2020 07:02 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/10 "2020-06-25T07:02:46Z")

</div>

That may or may not be related. It will depend how big large is and also maybe what you are then doing subsequently. Nodejs passes objects by reference, so the large array will be one object in memory, and splitting it has to create n new objects, whereas just passing one object is just moving a pointer.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [26 June 2020 05:42 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/11 "2020-06-26T05:42:24Z")

</div>

"whereas just passing one object is just moving a pointer." - My problem with this is that traversing 20 thousand records takes a lot of time. That's why I used split node to split the array to multiple messages and be processed simultaneously in the flow. This cuts down processing time but increased the load much.

For example: array with 20,000 elements and I used split with a fixed length of 1,000 will create 20 messages. These messages will be processed simultaneously.. But also increases the load.

Is there any other way of doing it though?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 June 2020 08:36 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/12 "2020-06-26T08:36:39Z")

</div>

> [@mikeh](#):
>
> That's why I used split node to split the array to multiple messages and be processed simultaneously in the flow.

They won't be processed simultaneously, they will be processed one after another (assuming that 'processing' doesn't include an asynchronous activity like database, file or internet access). So by splitting it and passing messages about you are increasing the overheads. The quickest way would likely be to do it all in a single function node traversing the array and doing whatever is necessary, but nothing else would get a look in while that was going on. Alternatively, if how long it takes is not critical, but you don't want to slow down other things going on then if you split them as you currently are but then put a rate limit node in the flow (using a Delay node) the processing would take longer but there would be gaps to allow node-red to continue. the slower you set the rate limit the longer it will take but the less the effect on the rest of the system.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [26 June 2020 10:06 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/13 "2020-06-26T10:06:10Z")

</div>

Thanks @Colin for the detailed explanation!  
Yes I'm doing API call (internet access) in the process, depending on a certain condition. So this might be ideal then.  
Using a delay in between loops is a good idea. Would putting a random 100-500ms per loop make a difference? I really don't want to make the process take longer.

I also tested the same number of elements, around 20 thousand records. Using split (with loop) vs without using split (just loop) and the results are quite the same in terms of speed. Kinda weird, because I was expecting the split one would be much slower.

**NO SPLIT:**

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/c/4/c464338f30f20accd8c13671d0094ded5c01f833.png)

**WITH SPLIT**

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/d/7d1af8a993064c82fa88e7c346ef089b3e554a63.png)

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [26 June 2020 10:15 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/14 "2020-06-26T10:15:27Z")

</div>

> [@mikeh](#):
>
> are quite the same in terms of speed.

which mostly shows that it is the API calls that are taking the time. Could be network latencies - or the fact you are DDOSing the API server. A delay between loops would allow some of the queues to drain down - but whether that is a problem or not depends on how much memory you have to play with or how much the server cares about many calls.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 June 2020 10:28 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/15 "2020-06-26T10:28:47Z")

</div>

Add another debug after the Get Data, I imagine that is where most of the delay is, but time spent fetching (assuming that is a different machine or process) shouldn't lock up node-red. If it does then I guess the Get Data process must be on the same machine and that process is locking up the whole machine, so nothing to do with node-red itself.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [27 June 2020 06:58 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/16 "2020-06-27T06:58:53Z")

</div>

@dceejay thanks for the explanation.  
Question about setting delay between loops:  
Would a loop inside a function and using setTimeout have the same effect as using the delay node?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [27 June 2020 09:59 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/17 "2020-06-27T09:59:38Z")

</div>

Yes, basically, but it isn't quite as easy to do as you might think, unless you have done it before.

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [27 June 2020 23:48 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/18 "2020-06-27T23:48:22Z")

</div>

@Colin What do you mean by "isn't quite as easy"? I hope you can explain. 🙂

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [28 June 2020 06:21 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/19 "2020-06-28T06:21:27Z")

</div>

How do you propose to implement it?

---

<div class="post-metadata">

**Author:** ![mikeh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mikeh](https://discourse.nodered.org/u/mikeh)\
**Post date:** [29 June 2020 00:06 UTC](https://discourse.nodered.org/t/need-help-analyzing-flow/28923/20 "2020-06-29T00:06:14Z")

</div>

I'm just curious on what you meant by "it isn't quite as easy to do".  
What I was going to do is do an array loop inside a function and in between loop do a time delay using setTimeout. Would a timeout inside a function (instead of using delay node) allow some of the queue to drain down?

[Next page](https://discourse.nodered.org/t/need-help-analyzing-flow/28923.md?page=2)