Long before somebody gets close enough to clone a key I figure the dog will get them
Thanks for the feedback!!
I have implemented the changes from @Steve-Mcl to support (by default) glob patterns and to use a list instead of a ; separated string. Which were obvious feature requests...
Now I am going to start finalizing this node, because it took me more time as expected already...
Would be nice if @zenofmud could repeat his tests on this new implementation 
There is one thing I don't like about this node, and that is the name. A name like node-red-contrib-cleanup-filesystem creates too much expectations about being a general purpose filesystem cleanup node. Which is not the case. It simply allows you to cleanup files (and folders) in a subtree below some base folder, if those files have exceeded some retention period (i.e. age). Which is quite convenient to cleanup video footage from my IP camera's...
So I would like to rename it, but not sure what name would be best:
- node-red-cleanup-aged-files
- node-red-file-retention-manager
- ...
Anybody an idea for a good name?
Thanks!!
Bart
1 Like
Scoped would be a sensible start 
node-red-directory-cleanse
node-red-file-purge
node-red-file-eraser
node-red-file-exterminator
node-red-file-remover
node-red-file-cleaner
However, if you suspect additional features like rotation or zip & delete source, file moves, bulk renames etc, something more generic would be sensible.
node-red-file-operations
node-red-fs-toolkit
node-red-filesystem-tools
node-red-file-manager
Any good?
Yeah that is what I meant that this node is not a general purpose filesystem cleanup node. Just need now something that can fit in my free time. So I am going to keep it simple this time, otherwise I will never be able to complete my own home automation...
Have you been talking to chatgpt 
I did but my keywords were perhaps not very good, because I was not really impressed with his creativity this time...
Things node-red-file-purge doesn't really tell WHY the files have becoming obsolete. In this case it is only because the files are too old, or have expired, ... Something in that area. But my creativity in english is a bit too limited to come something for that.
Are you saying I sound like a computer 
I should have been more explicit. I meant @scope/node-name
I think @scope/filesystem-tools is enough to convey filesystem based package and that it provides "tools" for the user.
@scope/fs-toolkit is a bit more cryptic (the fs part may not be obvious for non programmers) but I still like it.
I also like this: @scope/file-operations
I don't like file-manager (or similar) as it conveys something like a windows file manager (i.e. something with a UI)
1 Like
Ah yes. I use scopes indeed for my new nodes. And no "contrib" stuff anymore...
I think that sums it up nicely.
1 Like
@BartButenaers Just tried to install the new version and I get this
paul@PaulsM1 .node-red % npm install bartbutenaers/node-red-cleanup-filesystem
npm notice
npm notice New major version of npm available! 8.19.2 -> 10.8.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.0
npm notice Run npm install -g npm@10.8.0 to update!
npm notice
npm ERR! code EJSONPARSE
npm ERR! path /Users/paul/.npm/_cacache/tmp/git-clone45Nwz9/package.json
npm ERR! JSON.parse Unexpected token ":" (0x3A) in JSON at position 408 while parsing near "... \"glob-to-regexp\": \"^0.4.1\"\n ],\n ..."
npm ERR! JSON.parse Failed to parse JSON data.
npm ERR! JSON.parse Note: package.json must be actual JSON, not just JavaScript.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/paul/.npm/_logs/2024-05-26T13_11_38_920Z-debug-0.log
Thoughts? (this is on my Mac mini M1 using Ventura v13.16.4)
UPDATE: I copied the package.json file into a JSON processor I have and see this:
Yes. Pebcak issue. Apologies!!
BTW I have renamed everything, and node is now called node-red-file-retention-manager
bah, "retention-manager" sounds like something that manages a service or application that does "retention"!
I would never think to search for that when looking for something to do with files!
Ignore me - its fine as is 
In the UK, a Retention Manager usually refers to a managerial position held by someone tasked with retaining customer loyalty.
For example, people wanting to cancel their broadband service usually get put through to the Retention department, which is managed by a ....
Oh what a relieve. That is good news.
I wanted to start the development of a new node, that allowed me to rename other nodes. Then I could rename my nodes e.g. every 5 seconds, without any human intervention 
The word "retention" sounded immediately familiar to me, because we use Kafka at work. And Kafka uses a "retention period" to specify how long the messages need to be remembered on a topic...
4 Likes
Crashed NR:
26 May 12:31:26 - [red] Uncaught Exception:
26 May 12:31:26 - [error] TypeError: Cannot read properties of undefined (reading 'length')
at fileRetentionManager._inputCallback (/Users/paul/.node-red/node_modules/@bartbutenaers/node-red-file-retention-manager/cleanup_fs.js:61:79)
at /usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:214:26
at Object.trigger (/usr/local/lib/node_modules/node-red/node_modules/@node-red/util/lib/hooks.js:166:13)
at Node._emitInput (/usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:206:11)
at Node.emit (/usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:190:25)
at Node.receive (/usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:499:10)
at deliverMessageToDestination (/usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/flows/Flow.js:803:40)
at Immediate.<anonymous> (/usr/local/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/flows/Flow.js:819:21)
at process.processImmediate (node:internal/timers:471:21)
here is the flow:
[{"id":"e8128048cf12e796","type":"inject","z":"f82aec0811679377","name":".PNG","props":[{"p":"payload.fileNamePattern","v":".PNG","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":270,"y":560,"wires":[["40f5d29127ba14e7"]]},{"id":"9e7deb562e878bc6","type":"debug","z":"f82aec0811679377","name":"debug 3437","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":560,"wires":[]},{"id":"40f5d29127ba14e7","type":"file-retention-manager","z":"f82aec0811679377","name":"","baseFolder":"/users/paul/bart_test","patternType":"glob","age":"1","ageUnit":"minutes","removeEmptyFolders":false,"dryRun":false,"report":true,"patterns":[],"x":540,"y":560,"wires":[["9e7deb562e878bc6"]]}]
Note I had not changed the payload.fileNamePattern in the inject node. However after changing the inject node to use payload.patternType it still crashed NR.
1 Like
Should be fixed now on Github.
Nice catch!! But not catched by the code...
Ok, I can't figure out how to use glob and if I change to REGEX and run pointing to a folder with three items, this is what I see:
the error seems to point to line 137 in cleanup_fs.js::
if (fileAge > ageInSeconds && patterns.some(pattern => pattern.test(filePath))) {
Bart, I'm sure you know but glob patterns can be /../../xxx meaning a user could accidentally delete stuff from a directory above the base folder - do you sanity check paths do not resolve outside of base folder?
Retention period is also an Information Governance term related to privacy and things like GDPR. For personal information, you are only permitted to retain things for set periods. So this is a very familiar term.
Yes, this is the first thing to do. Remove all text containing ... Similarly, you may need to prevent absolute references as well (leading /). Also remove everything before the LAST : found. Which prevents the use of networked file systems as well as preventing the use of different drives on Windows.
Not sure that is the right approach.
I would recommend a check prior to any operations that join the paths, resolves the joined paths, extract the base & test the resolved paths contains the base. If not, raise an error. Along those line anyhow.
Should be fixed now on Github. Was indeed not working for most regex expressions.
Well I have to admit that I hadn't used it also not before. Do you want to find all png files like in your screenshot? Just ask chatgpt
The pattern seems to work...
Normally not. The node does a traversel of all folders starting from the base folder down. And then each folder is matched with the pattern. So not only a traversal of the folders specified in the patterns.
By doing it that way I can use the patterns to specify subfolders and filenames or extensions. Just what I need to cleanup the video footage and snapshot images from my camera's. For example a ReoLink doorbell generates a lot of subfolders ({year}/{month}/{day}), which I can now easily clean up.
Nothing more, nothing less. As I mentioned before, it is not a full blown filesystem cleanup node. I didn't design it to process a large amount of files or directories, just something for my own use case that I share with others on this blue planet
Nice to get a confirmation from a native speaking English human being...
1 Like
