# Size of flows.json and performance

**URL:** <https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799>\
**Category:** General\
**Created:** [11 June 2022 18:20 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799 "2022-06-11T18:20:59Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [11 June 2022 18:20 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/1 "2022-06-11T18:20:59Z")

</div>

Hi ,

Is there any performance stats vis-à-vis size of flows.json?  
We have ~3500 nodes in flows .json - spanning multiple flows / subflows.  
What we see is - once we do deploy - even if its deploy modified nodes - it takes significant time (more than few mins) for runtime to be available again.

During this period i can click inject - but nothing gets executed and once runtime is ready - all flows which i clicked - start.

While we are already actively checking where to prune out all not used / non active flows/nodes, wanted to check if anyone else faced such issues and if any suggestions.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![ghayne](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ghayne/32/39_2.png) [@ghayne](https://discourse.nodered.org/u/ghayne)\
**Post date:** [11 June 2022 18:27 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/2 "2022-06-11T18:27:59Z")

</div>

What hardware are you running on?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [11 June 2022 18:51 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/3 "2022-06-11T18:51:46Z")

</div>

> [@SandeepA](#):
>
> We have ~3500 nodes in flows .json - spanning multiple flows / subflows.

and you can multiple the number of nodes in each subflow by the number of instances

e.g.

- Main flow has 20 `inject` nodes and 20 `subnode:my_big_subflow` instances
- `subnode:my_big_subflow`contains 100 nodes  
Total nodes = 20 injects 20 subflow instances + (20 \* 100) = 2040 nodes to instanciate.

However, if you are not using context in subflows or just using them to tidy flows / hide implementation, they can pretty much be replaced with LINK-CALLs

> [@SandeepA](#):
>
> even if its deploy modified nodes - it takes significant time (more than few mins) for runtime to be available again

That doesnt seem right - is that all node types or certain node types? and are you modifying a node inside a subflow or main flow?

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [11 June 2022 19:11 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/4 "2022-06-11T19:11:30Z")

</div>

> [@ghayne](#):
>
> What hardware are you running on?

Let me check this and come back. Will take some time to connect to that piece since its at work.

> [@Steve-Mcl](#):
>
> and you can multiple the number of nodes in each subflow by the number of instances

Agree. Did not think of that. So in actual the number of nodes (in memory / wher ever its stored) are much much more.

> [@Steve-Mcl](#):
>
> However, if you are not using context in subflows or just using them to tidy flows / hide implementation, they can pretty much be replaced with LINK-CALLs

This is what i want to do eventually - esp since Dynamic Call is available with v3. I have significant dynamic calls. (i have raised a note regards this on Beta 3.2 topic).

> [@Steve-Mcl](#):
>
> That doesnt seem right - is that all node types or certain node types? and are you modifying a node inside a subflow or main flow?

I change nodes in both subflows and main flows. But let me do this - when i connect to work - let me run a test - and see if in both scenarios - the runtime takes same time to be available. I will come back.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [11 June 2022 22:16 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/5 "2022-06-11T22:16:24Z")

</div>

I would certainly check the usual suspects such as exceeding your actual memory and therefore getting lots of paging - that will kill any performance. Also check whether it is Node-RED causing that or something else such as MongoDB.

Personally, I would also look to collapse some of the multiple-node flows into single function nodes.

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 07:58 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/6 "2022-06-13T07:58:54Z")

</div>

> [@Steve-Mcl](#):
>
> That doesnt seem right - is that all node types or certain node types? and are you modifying a node inside a subflow or main flow?

I did the test. Created inject + debug on tab / main flow. No subflows changes.

1. Changed only the position of debug flag
2. Clicked Deploy Nodes
3. Deployed finished immediately
4. I clicked inject.
5. Took a good 8 minutes before result was reflected.

This is the usual behavior i am seeing off late.

Is there some trace or something which can help me investigate what the system is doing during those 8 mins ?

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [13 June 2022 08:07 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/7 "2022-06-13T08:07:16Z")

</div>

If you enable trace level logging in the runtime you will get much more detailed information about starting/stopping individual nodes. With so many nodes that will be a lot of information, but it's also the best way to try to see where time is being spent.

---

<div class="post-metadata">

**Author:** ![BartButenaers](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bartbutenaers/32/10476_2.png) [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)\
**Post date:** [13 June 2022 08:19 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/8 "2022-06-13T08:19:35Z")

</div>

> [@SandeepA](#):
>
> Is there some trace or something which can help me investigate what the system is doing during those 8 mins ?

I always do cpu profiling in those cases. See [here](https://github.com/bartbutenaers/node-red-contrib-v8-cpu-profiler/wiki/Do-a-Node-RED-CPU-profiling-analysis-via-Chrome-Developer-Tools) for more info. Be aware that analysing the profiling result is not always easy...

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 08:50 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/9 "2022-06-13T08:50:41Z")

</div>

> [@knolleary](#):
>
> If you enable trace level logging in the runtime you will get much more detailed information about starting/stopping individual nodes. With so many nodes that will be a lot of information, but it's also the best way to try to see where time is being spent.

Thank You Nick.  
Is this the .."logging.{.. console {}}" setting in settings.js ?

> [@BartButenaers](#):
>
> I always do cpu profiling in those cases. See [here](https://github.com/bartbutenaers/node-red-contrib-v8-cpu-profiler/wiki/Do-a-Node-RED-CPU-profiling-analysis-via-Chrome-Developer-Tools) for more info. Be aware that analysing the profiling result is not always easy...

Thank You Bart.  
Let me check this.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [13 June 2022 08:52 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/10 "2022-06-13T08:52:26Z")

</div>

> [@SandeepA](#):
>
> Is this the ..

Yes - you want to set `logging.console.level` to `trace`

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 June 2022 09:14 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/11 "2022-06-13T09:14:09Z")

</div>

> [@SandeepA](#):
>
> I did the test. Created inject + debug on tab / main flow. No subflows changes.
> 
> 1. Changed only the position of debug flag
> 2. Clicked Deploy Nodes
> 3. Deployed finished immediately

I assume "2. Clicked Deploy Nodes" means you were in "Modified Nodes" deploy mode?

Since it deployed immediately but clicking inject took 8 minutes - there may be something else going on. Was the debug set to log to console? If so, do you see the new debug message in console immediately or after 8 minutes?

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 15:53 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/12 "2022-06-13T15:53:09Z")

</div>

> [@knolleary](#):
>
> Yes - you want to set `logging.console.level` to `trace`

Thanks. Will do . Will come back. Work machine ... some of these changes take longer than needed.

> [@Steve-Mcl](#):
>
> I assume "2. Clicked Deploy Nodes" means you were in "Modified Nodes" deploy mode?

Yes. That's the one.

> [@Steve-Mcl](#):
>
> Since it deployed immediately but clicking inject took 8 minutes - there may be something else going on. Was the debug set to log to console? If so, do you see the new debug message in console immediately or after 8 minutes?

That's the next step. Will come back.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 June 2022 15:54 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/13 "2022-06-13T15:54:06Z")

</div>

better still, set the inject to repeat every 2 seconds - then check console after deploy.

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 15:56 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/14 "2022-06-13T15:56:00Z")

</div>

Makes sense. Will do. Thanks Steve.

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 18:38 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/15 "2022-06-13T18:38:49Z")

</div>

Hi All , i did the test.

1. Change the settings.js to trace logging
2. Restarted NR
3. Starting message in console log

13 Jun 21:08:32 - [info] Node-RED version: v2.1.4  
13 Jun 21:08:32 - [info] Node.js version: v16.13.2  
...  
...  
13 Jun 21:08:57 - [trace] [flow:22d00a62550cbf2c] ------------------|--------------|-----------------  
13 Jun 21:08:57 - [trace] [flow:5f762746ada88902] start flow [5f762746ada88902]

\<run time was available at 21:14\>

13 Jun 21:14:55 - [trace] runtime event: {"id":"runtime-state","retain":true}  
13 Jun 21:14:55 - [info] Started flows  
13 Jun 21:14:59 - [trace] comms.open /cfMhkppko/GRP7IpuafhhecY0N9f2+CxOK88rrju0s=

\< I changed the position of a node and deployed.. following appears then\>

13 Jun 21:20:08 - [trace] utils.writeFile - copied C:\ProgramData\node-red\flows.json TO C:\ProgramData\node-red.flows.json.backup  
.....  
.....  
.....  
13 Jun 21:20:08 - [trace] [flow:5f762746ada88902] start flow [5f762746ada88902]  
13 Jun 21:26:05 - [trace] runtime event: {"id":"runtime-state","retain":true}  
13 Jun 21:26:05 - [info] Started modified nodes  
13 Jun 21:26:05 - [trace] runtime event: {"id":"runtime-deploy","payload":{"revision":"7d5d7c28857d69806fe298dd7d8f9e38"},"retain":true}

Runtime became available at 21:26.

My observations :

1. My inject + debug doesnt seem to reflect here. I ran it much after 21:26. And i don't see anything here. Do i need to do something special to log it. (Btw this instance is on 2.1.4).
2. 5f762746ada88902 is an empty Flow 1. Probably the last flow in flows.json to be loaded by system.

Not clear what system does for 6 mins.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 June 2022 18:41 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/16 "2022-06-13T18:41:53Z")

</div>

> [@SandeepA](#):
>
> Not clear what system does for 6 mins

Is your windows server behind a proxy or have no internet?

> [@SandeepA](#):
>
> My inject + debug doesnt seem to reflect here

Is the debug set to output to system console?

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 19:08 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/17 "2022-06-13T19:08:02Z")

</div>

> [@Steve-Mcl](#):
>
> Is your windows server behind a proxy or have no internet?

Yes. Behind a proxy. No internet. Can this be the reason ?

> [@Steve-Mcl](#):
>
> Is the debug set to output to system console?

This worked.  
Now i see

\< changed the position and clicked deploy \>

13 Jun 21:58:14 - [trace] [flow:5f762746ada88902] start flow [5f762746ada88902]

\< runtime became available \>  
13 Jun 22:05:15 - [trace] runtime event: {"id":"runtime-state","retain":true}

13 Jun 22:05:15 - [info] Started modified nodes  
13 Jun 22:05:15 - [trace] runtime event: {"id":"runtime-deploy","payload":{"revision":"167d20e72d4f8206300d0340def8c3f4"},"retain":true}  
13 Jun 22:05:15 - [info] [debug:4fb4329b2df72b1a]

\< The above is from my debug message \>

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 June 2022 19:21 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/18 "2022-06-13T19:21:12Z")

</div>

> [@SandeepA](#):
>
> Yes. Behind a proxy. No internet. Can this be the reason ?

Yes

See here: [Node-RED v0.19.4 3 minute delay during startup on windows - #15 by Steve-Mcl](https://discourse.nodered.org/t/node-red-v0-19-4-3-minute-delay-during-startup-on-windows/3798/15)

---

<div class="post-metadata">

**Author:** ![SandeepA](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/sandeepa/32/63225_2.png) [@SandeepA](https://discourse.nodered.org/u/SandeepA)\
**Post date:** [13 June 2022 19:48 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/19 "2022-06-13T19:48:21Z")

</div>

Will try and update.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [13 June 2022 20:28 UTC](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799/20 "2022-06-13T20:28:45Z")

</div>

Good spot Steve.

I assume that delay only applies if you use the command, not if you start node-red using node.exe directly?

[Next page](https://discourse.nodered.org/t/size-of-flows-json-and-performance/63799.md?page=2)