So I have a use case where on a single instance I have 18K+ nodes across multiple tabs. To modify a single flow I use the modify single flow API (using /docs/api/admin/methods/put/flow/). With that many nodes, the API was taking ~6 secs to return a success. Profiling lead to the conclusion that the clone library was the biggest culprit.
So I did an experiment & replaced the clone library with rfdc in the runtime module. This made quite an impact and the API (modify single flow) time reduced to ~2 seconds. My changes can be seen here on github.
My questions:
is this approach correct? i am no where close to an expert on node or node-red, thus the question.
and is this something that the community would be interested in, if I were to invest more time to make a proper PR out of it?
Thanks.
PS: as a newbie here, I can put more than 2 links hence the improper API link.
have you not considered breaking them up across multiple instances? I assume all of the nodes+flows are a single related process? Or are you multi purposing one instance?
It is certainly interesting.
Do you have any flame charts or profiles that can be summarised/ posted for a look see?
Looking at your changes, it looks to be a drop in replacement - a PR would be fairly straightforward. And if you were to raise a PR, it should be based on/targeted to the Dev branch.
I think that the thing to remember with a tool such as Node-RED is that it has a LOT of overhead just to run the tool. This is great for prototyping and quick development but it will never be the most efficient. In many cases, this isn't an issue, but with many thousands of nodes, a visual programming tool looses a lot of its benefits. I would honestly firstly look for repeat patterns and rationalise those, then convert obvious processing groups into straight JavaScript code in function nodes. I'd also be looking to see whether some processing could be moved out of Node-RED altogether, perhaps into their own microservices.
