Importing the latest file(s) from a file folder into Node-RED

[gettmad]

I am currently working with only one file where I am using the 'Read File' node to access it however the application I am dealing with requires me to automatically update the file every 6 hours. The file will have an alternative name each time and therefore I do not believe I can use the 'Read File' node. Both the CSV File as well as the TXT File will be located in the same directory and I will require both for my flow.

Can anyone provide any insight into this type of issue?

[Colin]

Welcome to the forum.

Could you use a Watch node to watch the folder for new files?

[dceejay]

If you leave the Filename in the node config blank then you can (must) set it with msg.filename property on the input message.

[gettmad]

Thanks! I was considering using this Node however I wasn’t sure of its full capabilities. The documentation felt vague to me. Is it a Boolean flag which turns true upon any changes to the folder?

Appreciate the help in advance.

[gettmad]

I will look into this! Thanks for your reply.

[dceejay]

I can’t recall exactly but just try it with a debug node attached and set to show complete msg.. I think it returns the changed file names and some other useful info. The sidebar help should tell you.

[cymplecy]

The watch node just outputs the name of any file that changes in the watched folder

[gettmad]

Thank you all for the input. Below I will post the solution to the problem in my post.

Solution

As mentioned by various commenters, the 'Watch' node can be used to a specific file directory for any changes. If you would like the functionality mentioned in my post you can import the flow which I will attach at the bottom of this comment.

This flow will monitor a file directory for changes and when any 2 new files are added it will join them into a 'msg.payload' array. For my case I wanted the csv file which was stored in 'msg.payload[0]' to go to the above stream and then .txt file which was stored in the 'msg.payload[1]' to go to the below stream.

This flow is flexible and in the case you need to handle more files you can change the 'After a number of message parts' setting in the 'Join' node to your desired number of files.

Please note that for whichever reason in the 'Switch' node I could not define multiple outputs and direct the 'msg.filename' to their respective output so instead I added a 'Switch' node for each file I was trying to read.

[
    {
        "id": "f99455d03588f4ae",
        "type": "tab",
        "label": "Flow 6",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "5e24854f53bf1b74",
        "type": "watch",
        "z": "f99455d03588f4ae",
        "name": "watch",
        "files": "",
        "recursive": "",
        "x": 130,
        "y": 240,
        "wires": [
            [
                "49d9727cf0c331fc"
            ]
        ]
    },
    {
        "id": "6b3cf284cad2620e",
        "type": "debug",
        "z": "f99455d03588f4ae",
        "name": "debug 34",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 1220,
        "y": 360,
        "wires": []
    },
    {
        "id": "ae753ec17e7ac25e",
        "type": "file in",
        "z": "f99455d03588f4ae",
        "name": "txt",
        "filename": "payload[1]",
        "filenameType": "msg",
        "format": "utf8",
        "chunk": false,
        "sendError": false,
        "encoding": "none",
        "allProps": false,
        "x": 830,
        "y": 360,
        "wires": [
            [
                "6b3cf284cad2620e"
            ]
        ]
    },
    {
        "id": "f1167323c41b2a5e",
        "type": "file in",
        "z": "f99455d03588f4ae",
        "name": "csv",
        "filename": "payload[0]",
        "filenameType": "msg",
        "format": "utf8",
        "chunk": false,
        "sendError": false,
        "encoding": "none",
        "allProps": false,
        "x": 830,
        "y": 180,
        "wires": [
            [
                "6dcb09327f7aa3d0"
            ]
        ]
    },
    {
        "id": "6dcb09327f7aa3d0",
        "type": "debug",
        "z": "f99455d03588f4ae",
        "name": "debug 38",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 1220,
        "y": 180,
        "wires": []
    },
    {
        "id": "03b9bb99a97ebab9",
        "type": "join",
        "z": "f99455d03588f4ae",
        "name": "",
        "mode": "custom",
        "build": "array",
        "property": "payload",
        "propertyType": "msg",
        "key": "topic",
        "joiner": "\\n",
        "joinerType": "str",
        "accumulate": false,
        "timeout": "",
        "count": "2",
        "reduceRight": false,
        "reduceExp": "",
        "reduceInit": "",
        "reduceInitType": "num",
        "reduceFixup": "",
        "x": 450,
        "y": 240,
        "wires": [
            [
                "0631ee86439ddd12",
                "9a1172a406afe598"
            ]
        ]
    },
    {
        "id": "49d9727cf0c331fc",
        "type": "split",
        "z": "f99455d03588f4ae",
        "name": "",
        "splt": "\\n",
        "spltType": "str",
        "arraySplt": "1",
        "arraySpltType": "len",
        "stream": false,
        "addname": "",
        "x": 290,
        "y": 240,
        "wires": [
            [
                "03b9bb99a97ebab9"
            ]
        ]
    },
    {
        "id": "330d60dd6a512ca3",
        "type": "comment",
        "z": "f99455d03588f4ae",
        "name": "Test File Upload Func.",
        "info": "",
        "x": 180,
        "y": 280,
        "wires": []
    },
    {
        "id": "0631ee86439ddd12",
        "type": "switch",
        "z": "f99455d03588f4ae",
        "name": "",
        "property": "filename",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "payload[1]",
                "vt": "msg"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 630,
        "y": 240,
        "wires": [
            [
                "f1167323c41b2a5e"
            ]
        ]
    },
    {
        "id": "5450ed817fab47fa",
        "type": "debug",
        "z": "f99455d03588f4ae",
        "name": "debug 43",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 1220,
        "y": 440,
        "wires": []
    },
    {
        "id": "c1dfb7a8753f4abe",
        "type": "file in",
        "z": "f99455d03588f4ae",
        "name": "txt",
        "filename": "payload[1]",
        "filenameType": "msg",
        "format": "utf8",
        "chunk": false,
        "sendError": false,
        "encoding": "none",
        "allProps": false,
        "x": 830,
        "y": 440,
        "wires": [
            [
                "5450ed817fab47fa"
            ]
        ]
    },
    {
        "id": "9a1172a406afe598",
        "type": "switch",
        "z": "f99455d03588f4ae",
        "name": "",
        "property": "filename",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "payload[1]",
                "vt": "msg"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 630,
        "y": 300,
        "wires": [
            [
                "ae753ec17e7ac25e",
                "c1dfb7a8753f4abe"
            ]
        ]
    }
]