[ANNOUNCE]node-red-cleanup-filesystem - request for testing

Hi folks,

I had installed a Reolink video doorbell a few weeks ago, and I was a bit tired of cleaning up the files and folders with recorded video footage.

Of course I could have created a cron job to do this, but where is the fun that way... I want to show everything that is going on on my machine visually in Node-RED.

So I quickly developed @bartbutenaers/node-red-cleanup-filesystem:

Did some testing and looks to work fine. But unfortunately I have still very limited spare time, and I don't want people to loose important documents because me having some bug in my node...

Therefore I would really appreciate if somebody could do some more testing, also for edge cases like when properties are not filled in and so on. The safest way to do such tests is with dry-run option active.

This node is currently not yet available on npm, so it need to be installed within the .node-red folder:

npm install bartbutenaers/node-red-cleanup-filesystem

Thanks!!!
Bart

3 Likes

I built a file structure on a RPI I have and added folders and files and this is what the file structure is:
Screenshot 2024-05-21 at 6.13.35 AM

So I have folders within folders and some of them have files and some are empty.

(1) When I run with this setup:
Screenshot 2024-05-21 at 6.34.06 AM

It successfully shows all the files and nested files:

but no folders are listed

(2) when run with this setup:
Screenshot 2024-05-21 at 6.44.39 AM
It reports all the files in the bart_test folder and all the files in the bart_test/deleteMe folder


I expected it to only show the files in the bart_test/deleteMe folder

(3) Documentation issues (a picture is suppose to be worth a thousand words)


I believe the documentation should use Filename instead of fileNamdPatterns and the same for the other options.

(4) Passing in a parameter:
I tried passing a parameters to the node using msg.baseFolder to send the folder name. That generated an error:

Specify 'payload.baseFolder' (different from root folder) when not specified in the config screen.

using msg.payload.Basefolder seems confusing and is not in the documentation. I'd suggest using msg.Basefolder of msg.cleanup.Basefolder

That's all for now :grinning:

1 Like

@zenofmud
This is most kind of you...
Thanks a lot!! Really appreciated.
Will get to it this evening.

2 Likes

Hi @zenofmud,
thanks a lot for testing!
You guys in the US have other food habbits compared to me, while having a look at the file names of your pictures in your folders :yum:

That should be ok. If you have unselected the "Remove empty subfolders" option, then no folders will be removed. So no folder names in the report, because the report only shows the folders and files that will be removed. I have added that in bit more in detail in the docs now...

Yes you are absolutely right. The folder name filtering was very confusing and incorrect. I hope it is fixed now in the code. I have also added an example in the readme page.

Yes indeed a picture with thousands pink lines is more worth than a thousand words :rofl:
That should now also be fixed in the code.

Yes that is normal because you should inject msg.payload.baseFolder instead of msg.baseFolder. Is that not clear in the docs?

Hopefully you can find somewhere some extra time to test it again. Thanks!!

Very nice @BartButenaers

Now I can replace my old Python scripts that basically does the same but has no gui, so very nice idea and thanks a lot!!

1 Like

I expect that list is just the starter??

1 Like

Haha indeed :rofl:
It is a shame that I have abused Paul's screenshot to start joking, because he has already been so kind to assist me :smiling_face_with_three_hearts:

Just a thought, will this plugin be able to delete files & folders that are not owned by the current user? Why asking is because in my Python scripts I use sudo for those

I assume it can only remove files/folders to which the Node-RED user has access. And otherwise give the Node-RED user access to those resources.

OK, fine I think

I also automatically schedule clean up of video captures from my cameras, they do consume a lot of space, but after 21 days so you have enough time "to act". Here is a fresh example from a week ago, unexpected visitor seemed interested in our cars


:open_mouth:

What is that gadget he is wearing round his neck ?

I have a couple more changes you need to make.

  1. in the HELP for the node I'd suggest for Filename you use the full description from the GitHub description:
    Files with a name matching this regex pattern will be removed. Use `.*` for all files. Multiple regex patterns can be entered `;` separated, for example `.*mp4;.*jpg` to remove only files with extension "mp4" or "jpg". The value from the config screen can be overwritten via `payload.fileNamePatterns` in the input message.

(side note: using payload.fileNamePatterns will not work because the code looks for payload.fileNamePattern note the missing 's'

  1. For Foldername I would use:
    Folders with a (full path folder) name matching this regex pattern will be removed. Use .* for all folders. Multiple regex patterns can be entered following the example shown in Filename. The value from the config screen can be overwritten via payload.folderNamePatterns in the input message.

You will need to change payload.folderNamePatterns in the code or documentation so they match.

File patterns are better suited to glob patterns.

With glob, you can do things more usual like *.jpg

If the node is supporting multiple matches, it makes more sense to use a typedList

1 Like

Can't really tell from the pic. Could just be a bag on a neck strap.

I agree with that. I use the fast-glob module to get good support. It also includes regex matches so you don't need lots of code to support both. There are examples in the uib-file-list node in uibuilder.

I assume some useful tools. Nothing stolen or damaged this time, just the uncomfortable feeling remains (two visits with two days in between)

@krambriw Are any of your cars 'keyless entry'? - just thinking about what is around his neck .. (no need to answer here!)

1 Like

We keep all our car keys in tin cans with lids (old mustard powder tins I think). Greatly reduces the ability to pick up any signals.

1 Like

yeah - that is what I was thinking.