High CPU usage due to watchdirectory contrib node

Just a heads-up in case anyone else has the problem of his/her Node Red using high CPU usage + high LOAD values:

Would be great if there's a way to install a throttle or delay cause my Node Red platform became more and more unstable the broader and deeper the directory became in which the watch-directory nodes are constantly checking changes.

Best regards,
Marcel

What does that node do for you that the core Watch node does not?

Hey Colin! There a built-in node that does the same!?!? :open_mouth:

Update: Works like a charm. And no additional CPU usage!

Thanks for your help! :beers:

Quick question:

If I change & save a file for which the "watch" node notices a change, the node sends multiple events.
It looks like the node is so fast, that it generates messages while the file is being written.

Right now I cannot think of a smart way to filter the last relevant message (from a series of multiple messages) from the watch node. How would you do that?

Try a trigger node set to release the last message if nothing arrives for a few seconds

Watch uses chokidar which has a lot more controls that the watch node doesn't expose. You could always use it directly in a function node if you wanted to. It has options to control the problem you are hitting.

1 Like

A trigger node is what I have always used for this, but actually I don't think it fully solves the problem. If one is watching a directory and two, or more, significant sized files get written at the same time then the watch node outputs of stream of messages where the payload consists of one or or the other file names. Since the messages for the two files are interleaved I cannot think of a simple way of converting that stream into a single message for each file.

1 Like

But couldnt that be handled per topic (or in this case per msg.filename) ...

1 Like

Oh yes, I had forgotten the trigger node could do that.

Thank you all for your great tips! I'll try the trigger node and see how it works with multiple files being changed.

I have tested it with the settings shown above and it does work as expected, producing one message for each file.

1 Like

Not for me. It always generates 2 messages: one where (file) size = 0 Byte and the 2nd one where file size is the byte size of the file after it has been written to, as you can see in this screenshot:

The solution for me is simple: I let the 2nd output message sent on a 2nd output channel like this:

By just handling the message on output channel 2 I can now handle my signals without running into the problem of processing multiple duplicates.

Show us how you had the Trigger configured the first time. Are you sure that you had it as shown? In particular to Send Nothing initially.
image

1 Like

Sorry, I have missed that important setting in the above post.

Yes, that works! 4 Different files changed at the exact same time:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.