Using FTP(S) to transfer files to/from a remote FTP server

Some of my IoT students have encountered problems trying to get some of the FTP (Node-RED) contrib nodes working. This may be due to the restrictions on the school network and/or the school's ISP.

As an exercise I tried writing a couple of Python scripts to use the FTP_TLS library to send and receive files between a Raspberry Pi and a remote FTP server. As shown below, the scripts are called from a Node-RED flow using an Exec node.

[{"id":"678f819f147e416a","type":"exec","z":"4eb6989c0ae472d1","command":"","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"main_exec","x":590,"y":220,"wires":[["295ffd186354873d"],["295ffd186354873d"],["295ffd186354873d"]]},{"id":"295ffd186354873d","type":"debug","z":"4eb6989c0ae472d1","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":790,"y":220,"wires":[]},{"id":"1013b9dd79f38f81","type":"function","z":"4eb6989c0ae472d1","name":"READ from remote FTP-server","func":"let local_path      = \"/home/pi/camera_stills/\";\nlet local_filename  = \"latest_dave.jpg\"; \nlet remote_filename = \"dave.jpg\";\n\nlet ftp_server      = \"enter your FTP host server details\";\nlet ftp_user        = \"enter your FTP user details\";\nlet ftp_passwd      = \"enter your FTP password\";\n\nmsg.payload         = \"python /home/pi/python_scripts/read_from_ftp_server.py \"+ local_filename + \" \" + local_path + \" \" + remote_filename + \" \" + ftp_server + \" \" + ftp_user + \" \" + ftp_passwd;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":330,"y":260,"wires":[["678f819f147e416a"]]},{"id":"5594bd8aaa14b632","type":"inject","z":"4eb6989c0ae472d1","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"1","payloadType":"num","x":110,"y":260,"wires":[["1013b9dd79f38f81"]]},{"id":"e9475ff83ddb2478","type":"function","z":"4eb6989c0ae472d1","name":"SEND to remote FTP-server","func":"let local_path      = \"/home/pi/camera_stills/\";\nlet local_filename  = \"garage.jpg\";\nlet remote_filename = \"dave.jpg\";\n\nlet ftp_server      = \"enter your FTP host server details\";\nlet ftp_user        = \"enter your FTP user details\";\nlet ftp_passwd      = \"enter your FTP password\";\n\nmsg.payload         = \"python /home/pi/python_scripts/send_to_ftp_server.py \"+ local_filename + \" \" + local_path + \" \" + remote_filename + \" \" + ftp_server + \" \" + ftp_user + \" \" + ftp_passwd;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":320,"y":200,"wires":[["678f819f147e416a"]]},{"id":"2898d0bdd1189d27","type":"inject","z":"4eb6989c0ae472d1","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"1","payloadType":"num","x":110,"y":200,"wires":[["e9475ff83ddb2478"]]},{"id":"862fa1d96f252595","type":"comment","z":"4eb6989c0ae472d1","name":"Enter your FTP credentials in these two function nodes","info":"","x":240,"y":140,"wires":[]},{"id":"ee0fa8e5d73a67c8","type":"comment","z":"4eb6989c0ae472d1","name":"READ ME - location of Python scripts","info":"SEND to remote FTP server\n/home/pi/python_scripts/send_to_ftp_server.py\n\nREAD from remote FTP server\n/home/pi/python_scripts/read_from_ftp_server.py","x":190,"y":80,"wires":[]}]

The two Python scripts need to be located in the Raspberry Pi that is running Node-RED. Obviously you can change the location to suit your system - but remember to change the settings in the function nodes.

SEND to remote FTP server
/home/pi/python_scripts/send_to_ftp_server.py

READ from remote FTP server
/home/pi/python_scripts/read_from_ftp_server.py

Here's what part of the SEND to remote FTP-server function node looks like.
Screen Shot 05-17-22 at 09.27 AM

As you can see the 'jpg' file is located in the folder... /home/pi/camera_stills/
This is a folder the students used to capture images from a Raspberry Pi camera.

You need to enter the credentials for your ftp_server here.

If you inspect the two Python scripts you'll see a sub-folder is used to store the files on the FTP server. My IoT students decided to do this so they didn't interfere with each others files. You may decide you don't need to do this and could comment out this line.

A few things to note.

  • I'm not fluent in writing Python and will accept there are probably better ways to implement the above.
  • You may want to consider a method to 'hide' your credentials.
  • There is zero error-checking in the Python scripts (especially checking the sequence and content of the various arguments that are passed from Node-RED to the script).

My students and I hope someone finds this useful.

I'm wondering why/how your python works when the contrib ones don't?

Does an error show up in the logs when the SFTP node is installed?

Not to be pedantic here... But what you have implemented is FTPS (the FTP protocol using SSL encryption) and not SFTP, which is a file transfer emulation over SSH.
Despite the similar names it is not the same beast at all :slight_smile:
As a consequence an SFTP node can't connect to a FTP(S) server as these are two different protocols.

1 Like

Thanks - I stand corrected on the terminology/names I used.
I've amended the wording in my write-up so it makes more sense.

If you need to script up FTP then maybe look at the lftp Debian package that handles the interactivity for you.