# High CPU usage due to watchdirectory contrib node

**URL:** <https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800>\
**Category:** General\
**Created:** [13 March 2022 16:08 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800 "2022-03-13T16:08:59Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [13 March 2022 16:08 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/1 "2022-03-13T16:08:59Z")

</div>

Just a heads-up in case anyone else has the problem of his/her Node Red using high CPU usage + high LOAD values:

> <https://github.com/fatoldsun00/node-red-contrib-watchdirectory/issues/9>
>
> Hi,
> 
> your plugin is a great tool in our Node Red environment, checking for cha…nges of CSV files in a very broad and deep folder structure.
> 
> The problem though is: Your node obviously checks for changes too often. I don't know by how many times per second, it's causing the CPU of my system to constantly burn at 100% in a docker container. Due to this problem other services are prone to crash and thus renders Node Red unstable for productive use.
> 
> Here're a few flows to see how the CPU + LOAD is before and after:
> !\[image\](https://user-images.githubusercontent.com/4894323/158068006-d20120ca-b4b7-47e1-84e7-20e214dfbf49.png)
> 
> CPU is just bored, has nothing to do. LOAD is also extremely low:
> !\[image\](https://user-images.githubusercontent.com/4894323/158068046-835e75a0-a701-43cb-8142-2479ef1ef2e2.png)
> 
> 
> Now activating just one of your watch-directory node back ON, then waiting a minute to let the statistics normalize:
> !\[image\](https://user-images.githubusercontent.com/4894323/158068109-e78e6953-cf92-4c0c-b8a7-baed1e42c2b1.png)
> 
> The CPU usage spiked massively, and it's not going down anymore! Load time is also nearly at 2.
> 
> If I activate a 2nd 'watch-directory' node back on, CPU will ramp up even higher. LOAD time is now over 3:
> !\[image\](https://user-images.githubusercontent.com/4894323/158068188-20302ccc-cad2-4eb8-88e8-9591141a8dd8.png)
> 
> Would it be possible to implement a parameter, for example?
> Trigger Every: X Seconds / Minutes / Hours

Would be great if there's a way to install a throttle or delay cause my Node Red platform became more and more unstable the broader and deeper the directory became in which the watch-directory nodes are constantly checking changes.

Best regards,  
Marcel

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [13 March 2022 16:31 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/2 "2022-03-13T16:31:25Z")

</div>

What does that node do for you that the core Watch node does not?

---

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [13 March 2022 19:51 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/3 "2022-03-13T19:51:34Z")

</div>

Hey Colin! There a built-in node that does the same!?!? 😮

Update: Works like a charm. And no additional CPU usage!

Thanks for your help! 🍻

---

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [13 March 2022 22:28 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/4 "2022-03-13T22:28:45Z")

</div>

Quick question:

If I change & save a file for which the "watch" node notices a change, the node sends multiple events.  
It looks like the node is so fast, that it generates messages while the file is being written.

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/4/d47e36dd9201a8a7ee524754133b11cc5de6456b.png)

Right now I cannot think of a smart way to filter the last relevant message (from a series of multiple messages) from the watch node. How would you do that?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [13 March 2022 23:14 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/5 "2022-03-13T23:14:33Z")

</div>

Try a trigger node set to release the last message if nothing arrives for a few seconds

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [14 March 2022 09:12 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/6 "2022-03-14T09:12:58Z")

</div>

Watch uses chokidar which has a lot more controls that the watch node doesn't expose. You could always use it directly in a function node if you wanted to. It has options to control the problem you are hitting.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [14 March 2022 12:20 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/7 "2022-03-14T12:20:35Z")

</div>

A trigger node is what I have always used for this, but actually I don't think it fully solves the problem. If one is watching a directory and two, or more, significant sized files get written at the same time then the watch node outputs of stream of messages where the payload consists of one or or the other file names. Since the messages for the two files are interleaved I cannot think of a simple way of converting that stream into a single message for each file.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [14 March 2022 12:23 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/8 "2022-03-14T12:23:33Z")

</div>

But couldnt that be handled per topic (or in this case per msg.filename) ...

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/b/0b4851cd3ad642c0cc58721583102f76d563cd3f.png)

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [14 March 2022 12:25 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/9 "2022-03-14T12:25:09Z")

</div>

Oh yes, I had forgotten the trigger node could do that.

---

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [14 March 2022 18:47 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/10 "2022-03-14T18:47:03Z")

</div>

Thank you all for your great tips! I'll try the trigger node and see how it works with multiple files being changed.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [14 March 2022 19:03 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/11 "2022-03-14T19:03:47Z")

</div>

I have tested it with the settings shown above and it does work as expected, producing one message for each file.

---

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [14 March 2022 21:19 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/12 "2022-03-14T21:19:24Z")

</div>

Not for me. It always generates 2 messages: one where (file) size = 0 Byte and the 2nd one where file size is the byte size of the file after it has been written to, as you can see in this screenshot:

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/e/7/e720ce1e9eb20dcc3d3cb9cdbd39cbc617dbec6c.png)

The solution for me is simple: I let the 2nd output message sent on a 2nd output channel like this:

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/f/2/f2bab31d3a3d894acbaedd2e9d2f50960a1ccac5.png)

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/9/f/9f1a3565a0481245737293cc1c5861ef9a91f00e.png)

By just handling the message on output channel 2 I can now handle my signals without running into the problem of processing multiple duplicates.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [14 March 2022 21:34 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/14 "2022-03-14T21:34:03Z")

</div>

Show us how you had the Trigger configured the first time. Are you sure that you had it as shown? In particular to Send Nothing initially.  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/6/1/618b0cf6b2fa6d29a46c0b3b0bce278bccd35c7c.png)

---

<div class="post-metadata">

**Author:** ![CreativeWarlock](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/creativewarlock/32/102051_2.png) [@CreativeWarlock](https://discourse.nodered.org/u/CreativeWarlock)\
**Post date:** [14 March 2022 21:37 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/15 "2022-03-14T21:37:55Z")

</div>

Sorry, I have missed that important setting in the above post.

Yes, that works! 4 Different files changed at the exact same time:

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/8/4/844294d3df6acbcf85edfe8be6100ae9f64d31cf.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [28 March 2022 21:38 UTC](https://discourse.nodered.org/t/high-cpu-usage-due-to-watchdirectory-contrib-node/59800/16 "2022-03-28T21:38:36Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.