Not in fact any relation to the famous large Greek meal of the same name.

Tuesday 2 May 2023

Three SSH settings which aren’t the default, but which you probably want

Previously on #homelab:

     
Everybody uses OpenSSH to securely log in to remote machines. And they have done for ages. But that’s actually a problem, because OpenSSH has been around for so long that some of the security decisions made earlier in the project’s history no longer match current best practices. Here are a few things which would probably be the default if OpenSSH was starting out today, but which – for sound backward-compatibility reasons – you’ll need to arrange for yourself.

ssh-add -c

One criticism of the ssh-agent system is that when using it, you lose visibility of exactly when you’re signing for things. One way to mitigate this is, when using ssh-add to add local private keys to the agent, adding the -c option. This makes ssh-agent ask for confirmation (on console or in a pop-up dialog) every time a private-key operation is requested; unexpected pop-ups could be a sign that nefarious software is trying to use your key.

This also somewhat mitigates the risk, when using the agent-forwarding (ssh -A) feature, of attacks by a malicious actor who has root on the remote computer.

ssh-add -D” on screen lock

On most reasonable systems, you have to give your local Unix password to unlock the screen once the screensaver has kicked-in. But if you feel that your SSH private key is more valuable than your local password – which you probably do, otherwise you wouldn’t’ve bothered encrypting your SSH private key in the first place – then that’s effectively a privilege-escalation attack: if you’ve left a ssh-agent session running, then knowing only your local password gives an attacker login ability using your SSH private key.

The obvious way to mitigate that, is to empty all saved keys from the ssh-agent session, every time the screen locks. Honestly it’s a bit surprising that KDE and Gnome don’t already have this built-in – but (at least in KDE) there’s a hook that lets you do just that.

In KDE “System Settings”, go to “Notifications” then next to “Applications:” click “Configure...”. Scroll down to “Screen Saver” and click “Configure Events...”. In the resulting pop-up window, choose “Screen locked” then below tick “Run command” and enter:

/usr/bin/ssh-add -D

Alternatively, manually add the following to ~/.config/ksmserver.notify:

[Event/locked]
Action=Execute
Execute=/usr/bin/ssh-add -D
Logfile=
Sound=
TTS=

Under Gnome it’s less straightforward; there’s a script available called lockheed, but as it stands it only listens for unlock events; perhaps it would be possible to modify it to listen for lock events too. (Or perhaps wiping the keys from ssh-agent only on unlock is actually okay.)

ssh-keygen -a 1000

Over the years, OpenSSH has gone through a few design iterations on how to encrypt SSH private keys – in other words, what the passphrase you give to unlock the key actually does. If your private key file (usually ~/.ssh/id_rsa) starts with

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
...

then it is encrypted in a very obsolete way using PKCS#1; such passphrases are vulnerable to brute-forcing, should an attacker get their hands on the file. If, alternatively, it starts with:

-----BEGIN ENCRYPTED PRIVATE KEY-----
MII...

then it’s PKCS#8, which is better but still not great. What you hope to see is the OpenSSH key format, which allows specifying the number of rounds of key-derivation function to use (i.e., how much work an attacker would have to do per guess in order to brute-force the passphrase):

-----BEGIN OPENSSH PRIVATE KEY-----

This particular recommendation slightly undermines the title of the blog post, because the OpenSSH format, which you probably want, is the default when creating new keys nowadays – but only since the OpenSSH 7.8 release of 2018-Aug-24, and OpenSSH does not upgrade from one format to another automatically. (Which is the Right Answer of course, as doing so would break compatibility with older versions if an upgrade ever had to be rolled-back.) This means that if your key was created with an older version of OpenSSH, the encryption used was likely to be, and is likely to still remain, one of the weaker forms.

Fortunately, there’s an OpenSSH facility to update just the encryption of any private key to the newest, most secure format, without altering the actual key:

ssh-keygen -p -f id_rsa -a 1000

This command upgrades the passphrase protection. It asks for the old passphrase (to decrypt the key) and then twice for the new passphrase (to re-encrypt it). If you’re confident that your old private key hasn’t leaked anywhere, you can re-use the same passphrase. (If you aren’t confident of that, you probably need to generate all-new keys anyway.) The -a 1000 sets the number of rounds to 1,000 – up from the default of 16. On this oldish Core-i7 machine, the setting of 1,000 makes checking the passphrase take about ten seconds. (Whether successful or unsuccessful!) This is a slight annoyance for you, but each time you’re waiting those ten seconds you can be thinking about how any attacker trying to brute-force your passphrase will be using up all that CPU on every single wrong guess they make.

Unfortunately, although the number of rounds is stored unencrypted in the key file, there appears to be no straightforward way of reading it out again directly. Following the directions given in a stackoverflow answer, you can use this command to get a hex dump of the base64-decoded encrypted key structure:

cat id_rsa | head -n -1 | tail -n +2 | base64 -d | hexdump -C | head

On a key I had lying around, this produced the bytes shown below. What you’re looking for is the string “bcrypt”, followed by 24 bytes you don’t care about (the KDF-descriptor length 0x0000_0018, the salt length 0x0000_0010, and the salt itself as 16 random bytes) followed by a 32-bit big-endian value which is the number of rounds:

00000000  6f 70 65 6e 73 73 68 2d  6b 65 79 2d 76 31 00 00  |openssh-key-v1..|
00000010  00 00 0a 61 65 73 32 35  36 2d 63 74 72 00 00 00  |...aes256-ctr...|
00000020  06 62 63 72 79 70 74 00  00 00 18 00 00 00 10 1c  |.bcrypt.........|
00000030  d1 ab a0 6b cd 50 a7 8e  01 8c 9a f7 98 32 a6 00  |...k.P.......2..|
00000040  00 00 10 00 00 00 01 00  00 00 33 00 00 00 0b 73  |..........3....s|

In this file it’s 16 (0x0000_0010), the default. Once I’d run the ssh-keygen command on it, it instead appears as 1,000 (0x0000_03e8):

00000000  6f 70 65 6e 73 73 68 2d  6b 65 79 2d 76 31 00 00  |openssh-key-v1..|
00000010  00 00 0a 61 65 73 32 35  36 2d 63 74 72 00 00 00  |...aes256-ctr...|
00000020  06 62 63 72 79 70 74 00  00 00 18 00 00 00 10 cb  |.bcrypt.........|
00000030  d4 4a be 47 b2 26 c9 15  d4 8d 0d d0 36 4c 62 00  |.J.G.&......6Lb.|
00000040  00 03 e8 00 00 00 01 00  00 00 33 00 00 00 0b 73  |..........3....s|

Whenever you need to generate a new SSH key, the ssh-keygen command accepts the -a 1000 option in that case too.

Bonus fourth SSH thing while you’re here

The encfs-agent script lets you set up EncFS encrypted filesystems in such a way that you can use ssh-agent signing operations to unlock (mount) them – with no need to remember or enter a separate passphrase. Use it with ssh-add -c!

Note that the filesystem then remains mounted/decrypted until manually unmounted; ssh-agent is only consulted during the mount operation. The ssh-add -D command does not unmount the filesystem (analogously, it doesn’t close existing SSH sessions either). If you want screen-locking to umount these filesystems, consider the command:

umount -a -t fuse.encfs

That’s a slightly dangerous thing to do, though; it’s not guaranteed to work if any process is holding the filesystem open (by holding a file on it open, or having a current-directory inside it). Even if it does work, running processes might get very confused – perhaps, for instance, by writing important files into the unencrypted mount-point directory outside the EncFS, instead of inside the EncFS where you wanted them.

LATER EDIT: Extra bonus fifth SSH thing

TIL you can put

AddKeysToAgent confirm

in your .ssh/config and it’ll automatically do the equivalent of “ssh-add -c” for any key whose passphrase you supply to an ordinary ssh invocation. Semi-life-changing given the number of times I think my key is in ssh-agent, but it isn’t, and I end up having to enter the passphrase all over again...

About Me

Cambridge, United Kingdom
Waits for audience applause ... not a sossinge.
CC0 To the extent possible under law, the author of this work has waived all copyright and related or neighboring rights to this work.