# Cloning performance - using rfdc instead of clone

**URL:** <https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288>\
**Category:** Core Development\
**Created:** [13 September 2023 20:30 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288 "2023-09-13T20:30:37Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![asr9](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/asr9/32/82841_2.png) [@asr9](https://discourse.nodered.org/u/asr9)\
**Post date:** [13 September 2023 20:30 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/1 "2023-09-13T20:30:38Z")

</div>

Hi all,

So I have a use case where on a single instance I have 18K+ nodes across multiple tabs. To modify a single flow I use the modify single flow API (using /docs/api/admin/methods/put/flow/). With that many nodes, the API was taking ~6 secs to return a success. Profiling lead to the conclusion that the `clone` library was the biggest culprit.

So I did an experiment & replaced the `clone` library with [rfdc](https://github.com/davidmarkclements/rfdc) in the runtime module. This made quite an impact and the API (modify single flow) time reduced to ~2 seconds. My changes can be seen [here on github](https://github.com/anshumanr/node-red/commit/4410ce1486a7210e257869b2da417ffbf65bef22).

My questions:

- is this approach correct? i am no where close to an expert on node or node-red, thus the question.
- and is this something that the community would be interested in, if I were to invest more time to make a proper PR out of it?

Thanks.

PS: as a newbie here, I can put more than 2 links hence the improper API link.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 September 2023 21:22 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/2 "2023-09-13T21:22:28Z")

</div>

Hi @asr9 , welcome to the forum.

> [@asr9](#):
>
> So I have a use case where on a single instance I have 18K+ nodes across multiple tabs

Wow. I mean WOW.

have you not considered breaking them up across multiple instances? I assume all of the nodes+flows are a single related process? Or are you multi purposing one instance?

> [@asr9](#):
>
> and is this something that the community would be interested in

It is certainly interesting.

Do you have any flame charts or profiles that can be summarised/ posted for a look see?

Looking at your changes, it looks to be a drop in replacement - a PR would be fairly straightforward. And if you were to raise a PR, it should be based on/targeted to the Dev branch.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [13 September 2023 21:41 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/3 "2023-09-13T21:41:37Z")

</div>

> [@asr9](#):
>
> 18K+ nodes

That might well be a record!

I can't help but feel that it is time for you to start optimising. 😁

---

<div class="post-metadata">

**Author:** ![kuema](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/kuema/32/6542_2.png) [@kuema](https://discourse.nodered.org/u/kuema)\
**Post date:** [14 September 2023 06:55 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/4 "2023-09-14T06:55:29Z")

</div>

> [@TotallyInformation](#):
>
> That might well be a record!

And certainly even higher in the runtime when using subflows!

We only got about 6-8k nodes in bigger projects. Peanuts compared to that. 🤓

---

<div class="post-metadata">

**Author:** ![asr9](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/asr9/32/82841_2.png) [@asr9](https://discourse.nodered.org/u/asr9)\
**Post date:** [14 September 2023 14:26 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/6 "2023-09-14T14:26:20Z")

</div>

Thank you @Steve-Mcl

> Blockquote  
> Or are you multi purposing one instance?

Yes, I am multi-purposing a single instance.  
I will come back with profiling information. Glad to hear this can be of interest.

> Blockquote  
> I can't help but feel that it is time for you to start optimising. 😁

I hope to get there. After profiling, this (change to rfdc) looked like really low hanging fruit.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [14 September 2023 14:32 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/7 "2023-09-14T14:32:35Z")

</div>

> [@asr9](#):
>
> > Or are you multi purposing one instance?
> 
> Yes, I am multi-purposing a single instance.

ah ha - we did a video just recently that might be of use you [Single instance vs Multiple instances](https://www.youtube.com/watch?v=KOnQnR7yfT0)

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [14 September 2023 14:36 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/8 "2023-09-14T14:36:24Z")

</div>

> [@asr9](#):
>
> > I can't help but feel that it is time for you to start optimising. 😁
> 
> I hope to get there. After profiling, this (change to rfdc) looked like really low hanging fruit.

I think that the thing to remember with a tool such as Node-RED is that it has a LOT of overhead just to run the tool. This is great for prototyping and quick development but it will never be the most efficient. In many cases, this isn't an issue, but with many thousands of nodes, a visual programming tool looses a lot of its benefits. I would honestly firstly look for repeat patterns and rationalise those, then convert obvious processing groups into straight JavaScript code in function nodes. I'd also be looking to see whether some processing could be moved out of Node-RED altogether, perhaps into their own microservices.

---

<div class="post-metadata">

**Author:** ![asr9](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/asr9/32/82841_2.png) [@asr9](https://discourse.nodered.org/u/asr9)\
**Post date:** [14 September 2023 17:58 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/9 "2023-09-14T17:58:31Z")

</div>

So I have some flame graphs here, before and after the changes, this time using the node-red `dev` branch as base. [changes on github](https://github.com/node-red/node-red/compare/dev...anshumanr:node-red:rfdc?expand=1)

Setup:

- IDE VS code
- JS debugger nightly (vscode-js-debug)
- CPU profiling for 30 seconds
- 2 requests, with distinct payloads, are made to modify a given flow (PUT \flow\:id)

First, the flame graph for `dev` branch (`"version": "4.0.0-dev"`)

 ![nr-dev-fg](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/7/e7af50c61f65e47146e2908cd357d75a62bd1314.png)

and after the change to using `rfdc`

 ![nr-rfdc-fg](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/3/93c0fabce5e81451c42a608652e6bb2782c2055b.png)

I have raw CPU profile files in case they are needed.

The changes so far are only in the runtime folder, and the profiling was only for the `PUT \flow\:id` API.

---

<div class="post-metadata">

**Author:** ![asr9](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/asr9/32/82841_2.png) [@asr9](https://discourse.nodered.org/u/asr9)\
**Post date:** [18 September 2023 21:03 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/10 "2023-09-18T21:03:06Z")

</div>

I have opened a PR for the same [https://github.com/node-red/node-red/pull/4352](https://github.com/node-red/node-red/pull/4352)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [17 November 2023 21:03 UTC](https://discourse.nodered.org/t/cloning-performance-using-rfdc-instead-of-clone/81288/11 "2023-11-17T21:03:41Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.