# Analysis and optimization solutions for the high CPU usage of Node red?

**URL:** <https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052>\
**Category:** General\
**Created:** [17 May 2024 08:29 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052 "2024-05-17T08:29:50Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 08:29 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/1 "2024-05-17T08:29:50Z")

</div>

**If the length of the transmitted msg is large, it will significantly increase the CPU usage** of the system.

What optimization solutions are available when the payload length cannot be reduced?

I just noticed this phenomenon, but I'm not sure why the length of the msg leads to an increase in CPU usage. What is the reason?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 May 2024 08:38 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/2 "2024-05-17T08:38:14Z")

</div>

> [@Yesung](#):
>
> **If the length of the transmitted msg is large, it will significantly increase the CPU usage** of the system.

what do you mean by "transited"? To another node? to an external thing (via MQTT? HTTP? TCP/UDP? PLC?)

> [@Yesung](#):
>
> I just noticed this phenomenon, but I'm not sure why the length of the msg leads to an increase in CPU usage

Internally, messages are passed by reference when passed out to a single wire and should be near instantaneous. If you have multiple wires coming out of a node, then it is cloned for the 2nd, 3rd, nth wire - this was a design decision when V1.0.0 was developed.

Another reason for slowdown increasing with larger payload might be the node receiving data does some heavy processing - for example, the CSV node in earlier versions of Node-RED was notoriously slow. Visibly slow when processing large CSV data.

  

Can you give us an idea of what "large" means? rough size of the message data?

An example flow would be useful too.

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 08:43 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/4 "2024-05-17T08:43:10Z")

</div>

> [@Steve-Mcl](#):
>
> Can you give us an idea of what "large" means? rough size of the message data?
> 
> An example flow would be useful too.

Please wait a moment, I will provide a detailed description

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 08:45 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/5 "2024-05-17T08:45:27Z")

</div>

> [@Steve-Mcl](#):
>
> what do you mean by "transited"?

For example, msg is passed across multiple adjacent **function** nodes,  
It is not sent out through communication nodes like mqtt

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 May 2024 08:51 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/6 "2024-05-17T08:51:00Z")

</div>

> [@Yesung](#):
>
> msg is passed across multiple adjacent **function** nodes

So you have multiple wires coming out of a node going to multiple functions.

What are you doing in these functions - are you looping data?

Also, please answer other questions

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 08:51 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/7 "2024-05-17T08:51:04Z")

</div>

> [@Steve-Mcl](#):
>
> Can you give us an idea of what "large" means? rough size of the message data?
> 
> An example flow would be useful too.

I'm very sorry, it's difficult for me to capture an example of a certain section of the flowchart because it's too large.

I have calculated that in my process, the length of the MSG is 66.0 KB. My process carries such a large MSG, which flows through multiple **function** nodes, **switch** nodes, **filter** nodes, etc.

Finally, it is split through the split node and sent out using mqtt.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 May 2024 08:52 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/8 "2024-05-17T08:52:20Z")

</div>

Can you provide a small sample flow, preferably with a small sample of data (you can provide dummy data via an inject node or template node)

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 08:53 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/9 "2024-05-17T08:53:16Z")

</div>

> [@Steve-Mcl](#):
>
> So you have multiple wires coming out of a node going to multiple functions.

Okay, I'll give it a try

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 09:07 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/10 "2024-05-17T09:07:47Z")

</div>

> [@Steve-Mcl](#):
>
> Can you provide a small sample flow, preferably with a small sample of data (you can provide dummy data via an inject node or template node)

I tried it, but as you can see, my flowchart is very complex and there are three similar flowcharts running simultaneously. Even with the use of pseudo data, it is still difficult to extract an example flowchart,

But I can describe the purpose of my program, which is to collect data from a remote device through the modbus protocol, classify the data (such as physical quantities, alarm quantities, control variables, etc.), and then send it to the cloud platform in batches using mqtt;

In the flowchart, I used quite a few loops because I had to traverse each collected data and classify it into different arrays for storage.

At the same time, there is indeed a situation where multiple wires are led out from one node.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [17 May 2024 09:13 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/12 "2024-05-17T09:13:47Z")

</div>

Unfortunately, that picture is unclear and I cannot read your language (sorry).

Are you doing manual data conversions and data splitting of your modbus data?

Are you doing multiple small modbus reads?

You might benefit from reading this article: [Modernize your legacy industrial data. Part 2. • FlowFuse](https://flowfuse.com/blog/2023/09/modernize-your-legacy-industrial-data-part2/)

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [17 May 2024 11:21 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/13 "2024-05-17T11:21:22Z")

</div>

You have a LOT of nodes that have two wires, one of which is a debug. Try removing a load of them and see if that helps. Normally, such detailed debug nodes are useful when developing a specific section of your flow but should then be removed to avoid the processing that Steve has mentioned where multiple wires coming out of the same output port force Node-RED to make a deep copy of the msg data and this is an expensive process.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [17 May 2024 11:27 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/14 "2024-05-17T11:27:09Z")

</div>

> [@Yesung](#):
>
> I just noticed this phenomenon, but I'm not sure why the length of the msg leads to an increase in CPU usage.

Do not assume that the problem is due to the large message size, it may well be due to what you are doing with the processing of the message in the nodes where you process it.

How many messages per second is your system handling?

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 11:40 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/15 "2024-05-17T11:40:25Z")

</div>

> [@Steve-Mcl](#):
>
> Are you doing manual data conversions and data splitting of your modbus data?
> 
> Are you doing multiple small modbus reads?
> 
> You might benefit from reading this article: [Modernize your legacy industrial data. Part 2. • FlowFuse](https://flowfuse.com/blog/2023/09/modernize-your-legacy-industrial-data-part2/)

No, I read over a hundred bytes at once and slowly parse them in subsequent nodes.  
Thank you very much for your suggestion, it has inspired me a lot

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 11:42 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/16 "2024-05-17T11:42:41Z")

</div>

> [@TotallyInformation](#):
>
> You have a LOT of nodes that have two wires, one of which is a debug. Try removing a load of them and see if that helps. Normally, such detailed debug nodes are useful when developing a specific section of your flow but should then be removed to avoid the processing that Steve has mentioned where multiple wires coming out of the same output port force Node-RED to make a deep copy of the msg data and this is an expensive process.

Your suggestion is really enlightening to me. I am trying to remove unnecessary debugging nodes

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 11:52 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/17 "2024-05-17T11:52:29Z")

</div>

> [@Colin](#):
>
> Do not assume that the problem is due to the large message size, it may well be due to what you are doing with the processing of the message in the nodes where you process it.
> 
> How many messages per second is your system handling?

I trigger an inject every 7 seconds, and each trigger results in 12 rounds of data collection, parsing, and reporting.

The process by which I became skeptical about the size of the msg is as follows:

My original **msg** was even larger, possibly more than twice the size of 66KB. At this time, my CPU usage was almost fixed at 110% (my system has a 4-core processor), and all node were running very slowly, taking almost 40 seconds to report data.

Later, I accidentally **delete** a large portion of unused fields during the transfer process and found that the CPU's duration of maintaining 110% was shortened, with most of the time between 50% and 80%. At the same time, the entire flowchart can be completed within 5 seconds.

---

<div class="post-metadata">

**Author:** ![Yesung](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/yesung/32/87392_2.png) [@Yesung](https://discourse.nodered.org/u/Yesung)\
**Post date:** [17 May 2024 12:03 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/18 "2024-05-17T12:03:05Z")

</div>

> [@Yesung](#):
>
> I trigger an inject every 7 seconds, and each trigger results in 12 rounds of data collection, parsing, and reporting.

During the process of running the flowchart, there were also countless debug nodes and mqtt sending nodes (used for debugging)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [15 August 2024 12:03 UTC](https://discourse.nodered.org/t/analysis-and-optimization-solutions-for-the-high-cpu-usage-of-node-red/88052/19 "2024-08-15T12:03:08Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.