SSH certificates not working anymore

Upgraded my Docker image to the 0.20.4 image, and suddenly my known_host mount doesn't seem to inject the Git SSH key into the container anymore. Did something change?

Command below:

docker run --restart=always --name nodered --user=1000 -dit -p 1880:1880 -v /srv/node-red:/data -v /home/user/.ssh/known_hosts:/usr/src/node-red/.ssh/known_hosts:ro -e TZ=America/New_York nodered/node-red-docker:latest

Edit: Also my keys ARE under the mounted volume, /srv/node-red/projects/.sshkeys

Hi - the latest image is 0.20.3 not .4.

What version did you upgrade from? Hard to say whether something has changed without knowing across what version range you are asking.

I have to admit I can't quite understand your question. Your docker command is mounting the individual file /home/user/.ssh/known_hosts into the container - not the whole directory of keys. So I'm not sure what you mean by it not injecting the Git SSH key into the container.

I was on the 0.19 branch, but not sure which build version.

The docker mount was necessary to pass in a known_hosts file specifically (if I remember right, otherwise it would reject the host). The keys themselves, as I said in my edit, were under the mounted "projects/.sshkeys" folder as a .pub public key, and matching private key.

I did look at the docs and saw this which I think is new: "If you have adminAuth enabled, the files are scoped to the current logged in user so they do not need to share credentials for the remote."

Given the topic is "known_hosts" not working anymore I assumed that is what you meant.

A bug was fixed yesterday that if you have adminAuth enabled and using projects it wasn't properly picking up the keys for the authenticated user.

0.20.4 will be released on Thursday.

Ah, sorry, I posted before reminding myself of the setup. Yeah once I read the documentation it seemed to be not picking up the certificates themselves. Will wait for that release.

Did this make it into 20.4 / 20.5? Im still not seeing my SSH keys again.

Yes, the fix was in 0.20.4.

Can you provide some more details to help understand what problem you are hitting?

What are the files names of the keys you have under /srv/node-red/projects/.sshkeys?

You've not confirmed whether or not you have adminAuth enabled - if you do, how is that configured? Is it with the built-in auth scheme or a custom one? What username do you use to log in?

Yes I use adminAuth, standard built in auth scheme. I use "admin" to login.

Have two files in /srv/node-red/projects/.sshkeys (which is mounted correctly into the container), owned by the use ID running the docker container:

__default_DiskStation -rw------
__default_DiskStation.pub -rw-r--r--

If you login as the admin user then it won't list the keys for the __default user.

If you rename those files as admin_Diskstation... then the admin user will be allowed to use them

That did it, thank you!

After upgrading to 1.0, I seem to have some oddities around this again. I have verified the key on disk as openssl rsa -in admin_CertName -check.

When I try to push to remote, it spins for about 10 seconds and then pops up the passphrase input for the cert. I input that and click retry, and now nothing else happens for about 10 seconds, and it pops the interface again. I know the pass phrase is correct as provided, and nothing else has changed.

It popped up a box to let me see log for a second which when I click popped a window that had the following:

2019-10-26T21:29:40.823Z Push changes

2019-10-26T21:29:39.389Z git -c credential.helper= push origin HEAD:master --porcelain
2019-10-26T21:29:39.587Z [err] ssh_askpass: exec(/usr/src/node-red/node_modules/@node-red/runtime/lib/storage/localfilesystem/projects/git/node-red-ask-pass.sh): Exec format error
2019-10-26T21:29:39.588Z [err] ssh_askpass: exec(/usr/src/node-red/node_modules/@node-red/runtime/lib/storage/localfilesystem/projects/git/node-red-ask-pass.sh): Exec format error

2019-10-26T21:29:43.807Z [err] Permission denied, please try again.

Note that I mount known_hosts into the container as well:

-v /my/location/.ssh/known_hosts:/usr/src/node-red/.ssh/known_hosts:ro

Also, forcing eval of the cert for authentication using the following works fine:

git clone -c core.sshCommand="ssh -i /path/to/my/pk" ssh://user@1.1.1.1/myRepo.git

Note also that manual pushes work fine. It seems to know that its ahead of master as well somehow, so it apparently can connect for that, but not for anything else.

Just checking in, I saw some points in the 1.0.5 upgrade about "Add better handling of host-key-verify error with projects", this still seems to be an issue.

I think it might be helpful to have the comment regarding known_hosts added to the documentation for the docker-compose example:

I've been running through forums all day trying to figure out clarity on an issue similar to this. I think it's because my ~/.ssh folder gets blown away since it's not a persistent volume.