Trying to get my head around accessing a remote directory for file copying

General
1

I know I have been down this road before, but slightly differently this time.

I have a script to backup node-red stuff locally on the same machine.

As things go on, I am needing to now access a remote machine (probably a NUC, but who knows at this stage) and copy files from the machine to the remote machine.

It will be a USB drive and may not always be there.

I have the share set up on the remote machine.
showmount -e <ip address> shows me the share.

So, how I am stuck how to access that share.

Do I mount the share, or can I copy the files to the remote machine another way?
If I have to mount it, it isn't too much of a problem, but it makes it slightly more complicated, but not impossible.

So for now I am wanting to understand the options of how to copy the files (directories?) between machines.

2

ssh, scp, rsync or mount it.

There is a node for rsync, makes it easier to do in node-red (same goes for the backup btw, you can do that in node-red).

1 Like
3

I'm looking at rsync just now.

That saves on mounting the drive on the local machine.

Tried dryrun but it failed. :frowning: Something about passwords.

I am copying to a shared directory, so I am not getting what's going on there.

Could you help me please?

This is the error:

Permission denied, please try again.Permission denied, please try again.Permission denied (publickey,password).rsync: connection unexpectedly closed (0 bytes received so far) [sender]rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.2]
4

What I do is more of the advanced route, but I have network shares in a SystemD unit file set up for mounting, including setting the permissions right. If needed I can control those through a systemctl command but most I’ve set up to automount once a network connection is present. These run on a pi that is connected over WiFi, so setting up mount points for those in /etc/fstab isn’t the right path as that will run before a network connection is present. So far it’s working good, and I’ve a feeling I should be able to get it working through a flow too, though I’m not there yet (for lack of trying mostly)

5

fstab is parts unknown to me.

I can mount the drive/device/folder but as it is early days......

Also the machine to where they will be sent may not always be online and the folder also may not be there.

So: it is a bit complicated. I don't know how to do all those tricks yet.

6

For remote backups I generally do it the other way round... and have the central "server" just go out to each client and copy the files back to itself at regular intervals using a simple loop and rsync - if one of the clients isn't there for some reason then it just moves on to the next.

2 Likes
7

That's a good idea. Thanks.

But then, the remote directory needs to be shared - yes?

All I know about sharing is that I edit the /etc/exports file and that shows up in the showmount command.

8

rsync can be used for local backup creation, but the r in rsync stands for remote, usage:

rsync options source destination

There is always security in between, rsync has an environment variable available for supplying passwords (RSYNC_PASSWORD).

To make life easier, use ssh key pairs between your devices so that you don't need to enter a password. (ie. generate key on the source device, copy the key to the remote device). Then you can use rsync over ssh.

1 Like
9

Yes, I have keys between the machines. Though I think it is only one way. (Luckily it is the right way)

But there is a "master password" used so I don't have to remember umpteen passwords. Just one.

Will this cause problems?

10

no - you can just scp in (or rsync) and copy the files from the remote directory. nothing needs to be mounted. As long as the remote user can a) login to the client computer and b) has permission to read the files in that directory then it should be fine. So in my case I log as "pi" user and use the required password, (or set up certificates - which to me is easier - but not for this thread).

11

Yeah, I have all that set up.

Now just getting it working in the node.

Alas the errors are not helpful at this stage.

I am adopting your idea that the sever gets the file from the remote machines.
I can SSH to the remote machine/s ok - but I need to enter the master password - to log in.

In the remote machine field.....

This is the path:
192.168.0.93:/home/pi/Backups/NR/2019-09-15.16.59.29/

But do I need it to be
pi@192.168.0.93:/home/pi/Backups/NR/2019-09-15.16.59.29/?

Yeah, I'll be trying but just to disclose all things I am doing.

12

typically an rsync command would look like

rsync -avz ssh pi@192.168.0.100:/home/pi/.node-red mylocalBackupDir1/
13

Ok, but if I am using the node, where does the -avz come in?

14

Um, sorry.
Putting aside the IP number that doesn't work.

However if I try:
rsync -avz pi@192.168.0.93:/home/pi/.node-red /home/me/TEMP/NR/

It works.

What is the need for the ssh part? and the leading / on the local path. Is that needed too?

15

Ok, but if I am using the node , where does the -avz come in?

Linux has a nice command available for you if you don't understand what something does: man (manual)

man rsync and if you don't have it installed you can google for it.

What is the need for the ssh part? and the leading / on the local path. Is that needed too?

You need to copy it to a remote location, you need ssh for that. Yes you need the /, the concept: @host 'mount point' : directory /

16

Yes, but . . . . .

The example is for "cli" use. I get that. But I seem to remember it being pointed out that there is a rsync node for NR which simplifies the use.

I was trying to use the rsync node.

So using what @dceejay posted I tried it from a CLI. Error.

Modifying it to what I posted, it works.

I don't get how @dceejay could make that kind of mistake.
(Not blaming.) But it threw a spanner in the works.

So I can get it to work from the CLI, but then I would use/need the exec node.
As there is a rsync node, why not use it?

But I didn't see where the -avz bit gets written in to the command.

This is what happens:

me@me-desktop:~/TEMP/NR$ rsync -avz ssh pi@192.168.0.93:/home/pi/.node-red home/me/TEMP/NR/
Unexpected remote arg: pi@192.168.0.93:/home/pi/.node-red
rsync error: syntax or usage error (code 1) at main.c(1361) [sender=3.1.2]
me@me-desktop:~/TEMP/NR$

Changing only the source path I get this:

me@me-desktop:~/TEMP/NR$ rsync -avz pi@192.168.0.93:/home/pi/TEMP /home/me/TEMP/NR/
receiving incremental file list

sent 25 bytes  received 115 bytes  93.33 bytes/sec
total size is 661  speedup is 4.72
me@me-desktop:~/TEMP/NR$

So, I don't get why it doesn't work.

Ok, the command I posted (as the second (working) one) doesn't have a / at the end of the remote path.

So:

me@me-desktop:~/TEMP/NR$ rsync -avz pi@192.168.0.93:/home/pi/TEMP/ /home/me/TEMP/NR/
receiving incremental file list
./
FF_Update2.txt
hosts

sent 65 bytes  received 550 bytes  246.00 bytes/sec
total size is 661  speedup is 1.07
me@me-desktop:~/TEMP/NR$
17

But I seem to remember it being pointed out that there is a rsync node for NR which simplifies the use.

Touche. I digress :wink: didn't look at the actual node.

rsync -avz ssh pi@192.168.0.93:/home/pi/.node-red home/me/TEMP/NR/

should be; source - destination

rsync -avz ssh /home/me/TEMP/NR/ pi@192.168.0.93:/home/pi/.node-red

(note the absolute path /home/zz the first / is important.)

The example dceejay shows, is to get a file from a remote and store it locally (is that what you want ?)

18

No problems.

Putting aside the node now - though yes I brought it up:

What is with the ssh in the command?

If I do it that way, it fails. I posted what I used and it works. That I can work out the real command which works worries me. I'm not supposed to be that smart.

And

But this is what was given to me as working:
rsync -avz ssh pi@192.168.0.100:/home/pi/.node-red mylocalBackupDir1/
With no leading / on the local path.

I am not wanting to be rude @dceejay.

Just confused and wanting to get things clear.

19

Sorry. . . . .
That is back to front.

I am on "machine A". I am copying to "machine A" from "machine B".

So shouldn't it be:
rsync -avz pi@192.168.0.93:/home/pi/.node-red/ /home/me/TEMP/NR/
source - destination.

20

I am on "machine A". I am copying to "machine A" from "machine B".

I am confused. If you are on A than you are not copying from B.

let me think :wink: