Can not get files via sftp

General
1

Hi,

I'm trying to get all files from a folder on a server via sftp and download them into a specific local folder.
If possible, I'd only like to download those files that start with XYC.

I tried the following flow, but am only getting the buffer of those files. How would I either set the local folder where the files should be stored or use the "write file" function to store all files? (there will always be multiple files, so I can not set a filename in the "write file" function)

[
    {
        "id": "e72b337e9c7f6a48",
        "type": "debug",
        "z": "a69da9d2e0a258af",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 1130,
        "y": 120,
        "wires": []
    },
    {
        "id": "8c9f3d4c732120fe",
        "type": "sftp in",
        "z": "a69da9d2e0a258af",
        "name": "",
        "sftp": "",
        "operation": "get",
        "filename": "",
        "localFilename": "",
        "workdir": "/root/transfer/in/",
        "x": 930,
        "y": 240,
        "wires": [
            [
                "e72b337e9c7f6a48"
            ]
        ]
    },
    {
        "id": "e44b9fe017c1de25",
        "type": "change",
        "z": "a69da9d2e0a258af",
        "name": "",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "\"/sample\" & msg.payload.name",
                "tot": "jsonata"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 760,
        "y": 240,
        "wires": [
            [
                "8c9f3d4c732120fe"
            ]
        ]
    },
    {
        "id": "99ff4c9b2c5eafdb",
        "type": "debug",
        "z": "a69da9d2e0a258af",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload.name",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 730,
        "y": 320,
        "wires": []
    },
    {
        "id": "dfec3f0ea75af213",
        "type": "split",
        "z": "a69da9d2e0a258af",
        "name": "Split message to file names",
        "splt": "\\n",
        "spltType": "str",
        "arraySplt": 1,
        "arraySpltType": "len",
        "stream": false,
        "addname": "",
        "x": 500,
        "y": 260,
        "wires": [
            [
                "99ff4c9b2c5eafdb",
                "e44b9fe017c1de25"
            ]
        ]
    },
    {
        "id": "c8dc3ac050ba705d",
        "type": "sftp in",
        "z": "a69da9d2e0a258af",
        "name": "",
        "sftp": "",
        "operation": "list",
        "filename": "",
        "localFilename": "",
        "workdir": "/sample/",
        "x": 290,
        "y": 260,
        "wires": [
            [
                "dfec3f0ea75af213"
            ]
        ]
    },
    {
        "id": "caa682de3c9ad73f",
        "type": "inject",
        "z": "a69da9d2e0a258af",
        "name": "",
        "props": [
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "x": 110,
        "y": 260,
        "wires": [
            [
                "c8dc3ac050ba705d"
            ]
        ]
    }
]

Thanks!

2

Which sftp node are you using (I'll need to load it in to test it out)

3

Hi,

I'm using node-red-contrib-better-sftp

Thanks

4

Can you attach a debug node to the output (set to "show complete message") & and show us the results from the sftp node? Expand the properies of the debug msg - ideally we would see where the "files" and "file names" are presented in the msg.

5

Right - tried it out and AFACS the problem is the sftp in node is not passing thru the msg.payload.name so that you can then use it :frowning:

Workaround (but might be dodgy/unacceptable to you) is to copy the name to a flow variable and then use it later on in the flow to write the filename

But you'd either have to rate limit slow enough to ensure file has been downloaded from the website (which of course might not work 100% of time)

Or complicate the flow with a gate node that fed with status of what is happening further along the flow

That's all I can come with for now

[edit] tried setting topic to the filename but even that gets filtered out :frowning:

1 Like
6

There are a few sftp nodes so one of them might work better and not do the filtering?

7

Hi,

I'm now using node-red-contrib-sftpco and this one works.

Thanks

2 Likes
8

Good find :slight_smile:
I tested it myself and it worked lovely except for the fact that my remote test folder I'm using contains sub-folders and I can't work out how to ignore them

Apart from trying to detect whether the end of the filename contains .xxxxx :slight_smile:

The better-sftp node does provide a property of type which I was previously using to filter out sub-folders

End result is the get node only gets the 1st file entry and then stops processing on any more requests as the 2nd one is a folder :frowning:

image
image1053Ă—137 9.98 KB

image

The node info on github/npm implies we can filter directory names but I can't actually work out how to do it :frowning:

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.