ssh login fun on Ubuntu VM

Won’t use ed25519

I’ve written about setting up ssh logins many times in the past. It’s something I do all the time. But I came across an interesting situation trying to ssh between two adjacent Ubuntu VMs. Ubuntu 22.04.1 LTS, to be exact. Not exactly the latest release, which makes ssh certificate problems more exciting.

So use ssh-keygen -t ed25519 to create a key pair. The snappily named ed25519 is everyone’s favourite cypher right now, and going between two identical Ubuntu installations compatibility is assured, right? Wrong!

I copied the public key into ~/.ssh/authorized_keys using the PuTTY method (cut/paste between windows) and tried an ssh login. It still asked me for my password.

I set the security on the file, although authorized_keys can just as well be world readable – it contains public keys after all. Still no dice.

Here’s a tip – if ssh isn’t playing nice run it in debug mode. Unlike other software, it’s really simple – just use the -v option. Use it multiple times (-vvv) to get a reasonable level of debug output.

$ ssh -vvv user@otherhost
<snip>
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/user1/.ssh/id_rsa RSA SHA256:oOgKcEHVqRgQqZXh5E2++iJUnacbXHDzLsnSHsngNpw
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug1: Offering public key: /home/user1/.ssh/id_dsa DSA SHA256:7jTWaHnN1cjNAnqD4bOZL9E/3nYMbioPgSimRsgAwuk
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/user1/.ssh/id_ecdsa_sk
debug3: no such identity: /home/user1/.ssh/id_ecdsa_sk: No such file or directory
debug1: Trying private key: /home/user1/.ssh/id_ed25519_sk
debug3: no such identity: /home/user1/.ssh/id_ed25519_sk: No such file or directory
debug1: Trying private key: /home/user1/.ssh/id_xmss
debug3: no such identity: /home/user1/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
user@otherhost's password: ???

What’s going on here then? It’s not even trying the key (id_ed25519) – it’s trying one with the _sk suffix instead. Apparently it’ll use a XMSS certificate, nice! Authentication that works after the quantum apocalypse. Unfortunately ssh-keygen didn’t know about it, so no dice. Okay, if ed25519_sk is preferred to plain old ed25519, let’s make one:

$ ssh-keygen -t ed25519_sk
unknown key type ed25519_sk

Hmm. User-hostile software. With a bit of thought it’d list the ones it does know about when this happens, but you can trick it into telling you by typing “ssh-keygen -t” (or some other command syntax error). This will output a somewhat helpful summary that will include valid options for -t type.

$ ssh-keygen -t
option requires an argument -- t
usage: ssh-keygen [-q] [-a rounds] [-b bits] [-C comment] [-f output_keyfile]
                  [-m format] [-N new_passphrase] [-O option]
                  [-t dsa | ecdsa | ecdsa-sk | ed25519 | ed25519-sk | rsa]
                  [-w provider] [-Z cipher]

It turns out it uses a hyphen instead of an underscore, so try it again.

$ ssh-keygen -t ed25519-sk
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
Key enrollment failed: device not found

Eh? Device not found? What is a -sk key anyway? It sounds like a hardware key of some kind going by the error message, which is clearly not appropriate for a VM. Googling didn’t help, so I asked another member of the OS/2 Drinking club that evening and he thought it was indeed a FIDO-related, or at least some kind of discoverable key requiring a hardware authenticator such as YubiKey.
Meanwhile what to do? Trying to edit /etc/ssh/ssh_config to make it offer a standard ed25519 key defeated me. I have a suspicion that Ubuntu drops them from the list if a FIDO hardware is available, and it clearly thought it was. Adding explicitly for all hosts didn’t work:

Host *
        IdentitiesOnly yes
        IdentityFile ~/.ssh/id_rsa
        IdentityFile ~/.ssh/id_dsa
        IdentityFile ~/.ssh/id_ecdsa
        IdentityFile ~/.ssh/id_ecdsa_sk
        IdentityFile ~/.ssh/id_ed25519
        IdentityFile ~/.ssh/id_ed25519_sk

It may be possible to force it using a user’s config file, but using -i to force an identity failed – it just reverted to the keys it wanted to use.

I gave up fighting with it and switched to old trusty RSA. On some systems (like BSD) you’ll need to add it to /etc/ssh/sshd_config as it’s too 1970’s for some people to abide. But Ubuntu was happy with it out of the box, and it’s still the default keytype for ssh-keygen.

PubkeyAcceptedAlgorithms +ssh-rsa
HostKeyAlgorithms +ssh-rsa

There’s a principle on cryptography that a code is either broken or unbroken, and no one has broken RSA yet in spite of theories that it might be possible (and anyway, it’s too old, grandad!) And in this case, Ubuntu seems to have shot itself in the foot forcing you to use RSA anyway. Or DSA, which is actually broken. Which seems appropriate in the circumstances.