SSH Hardening: Mitigating Man-in-the-Middle Attacks with Host Key Verification

SSH host key verification makes your client confirm the server's identity before you send credentials. This guide shows how to read fingerprints, verify them out of band, pin them in known_hosts and rotate host keys without locking yourself out.

  • Intermediate
  • Linux VPS
  • 6 min read

By RunRacks Team · Published

SSH host key verification is what stops an attacker on the network path from impersonating your Linux VPS and collecting your password or session. Your client compares the key the server presents against the fingerprint it already trusts, and refuses to continue when they differ. This guide walks through reading host key fingerprints on the server, confirming them through a second channel, pinning them in known_hosts, and rotating keys when you have to.

Before you start

You need:

  • A Linux VPS running Ubuntu 22.04/24.04, Debian 12, AlmaLinux or Rocky 8–9, with root or sudo access.
  • An SSH client on your workstation. OpenSSH ships with Linux, macOS and Windows 10/11, so ssh-keygen and ssh-keyscan are already available.
  • Access to the RunRacks Dashboard browser console. This is your out-of-band channel — it does not travel over the same SSH connection you are verifying.
  • Key-based login already working, so a rejected password prompt never costs you access. See Set up SSH keys if you have not done that yet.

How host keys stop a MITM

Every SSH server holds one or more host key pairs in /etc/ssh/, typically Ed25519, RSA and ECDSA. When a client connects, the server signs part of the handshake with the private host key and sends the public key. The client looks that key up in ~/.ssh/known_hosts.

On the first connection there is nothing to compare against, so OpenSSH asks you to trust the key — this is trust on first use. That prompt is the weak point: if an attacker sits between you and the VPS and you type yes without checking, you have just pinned the attacker's key instead. Verification means comparing the fingerprint against a source the attacker cannot touch.

Step 1 — Read the host key fingerprints on the server

Log in and list the public host keys:

Terminal
ls -l /etc/ssh/ssh_host_*_key.pub

Print the SHA256 fingerprint of each one:

Terminal
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

Expected output looks like this:

Text
256 SHA256:8Qn1vXk3pR7tYb2LmWc0dHs4ZaE9fJg6NhUu1oPqRsA root@vps (ED25519)

Confirm which keys the daemon actually offers:

Terminal
sudo sshd -T | grep -i hostkey

You should see one hostkey line per configured key. Anything you do not recognise deserves a look before you continue.

Step 2 — Verify the fingerprint out of band

Open the RunRacks Dashboard, start the browser console for your VPS and run the same ssh-keygen -lf command there. Compare the SHA256 string character by character with what your local client shows.

If you manage DNS for the domain, publishing SSHFP records lets clients validate keys automatically. Details change between OpenSSH releases, so check the vendor documentation for the current record format and your registrar's interface.

Step 3 — Pin the host key in known_hosts

On your workstation, scan the key and inspect it before trusting it:

Terminal
ssh-keyscan -t ed25519 203.0.113.10 > /tmp/vps-hostkey
ssh-keygen -lf /tmp/vps-hostkey

Compare that fingerprint with the one from Step 2. Only when they match, append it:

Terminal
cat /tmp/vps-hostkey >> ~/.ssh/known_hosts

Check what your client already stores, and remove stale entries:

Terminal
ssh-keygen -F 203.0.113.10
ssh-keygen -R 203.0.113.10

Step 4 — Enforce strict host key checking

Edit ~/.ssh/config and create an entry for the VPS:

Text
Host vps
    HostName 203.0.113.10
    User deploy
    StrictHostKeyChecking yes
    UserKnownHostsFile ~/.ssh/known_hosts
    HashKnownHosts yes

With StrictHostKeyChecking yes, the client refuses to connect when the key is unknown or has changed. That is what you want for anything you manage by hand. For automated jobs that connect to a freshly built server, accept-new pins the key on first contact and still fails on later changes — a reasonable middle ground. Set the same options system-wide in /etc/ssh/ssh_config if you prefer.

Step 5 — Rotate host keys safely

Rotate when a key is exposed, when you rebuild the OS, or on a schedule that matches your policy.

Terminal
sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""
sudo ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
sudo chmod 600 /etc/ssh/ssh_host_*_key
sudo chown root:root /etc/ssh/ssh_host_*_key
sudo systemctl restart ssh

On AlmaLinux and Rocky the unit is sshd:

Terminal
sudo systemctl restart sshd

Then repeat Steps 1 to 3 to publish the new fingerprints to your team and your automation. If you use SELinux, restore the file contexts after replacing keys:

Terminal
sudo restorecon -v /etc/ssh/ssh_host_*

Check it works

Run a verbose connection and read the host key lines:

Terminal
ssh -v -o StrictHostKeyChecking=yes [email protected]

You should see the key type, the fingerprint and a line confirming the match against known_hosts, followed by a normal login. To prove the protection is real, deliberately corrupt the stored entry on a test machine:

Terminal
ssh-keygen -R 203.0.113.10
printf '203.0.113.10 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBogusKeyForTestingOnly\n' >> ~/.ssh/known_hosts
ssh [email protected]

The client must stop with REMOTE HOST IDENTIFICATION HAS CHANGED and refuse to send credentials. Remove the fake line with ssh-keygen -R 203.0.113.10 afterwards.

Troubleshooting

  • `REMOTE HOST IDENTIFICATION HAS CHANGED` — either you rotated the key or someone is intercepting the connection. Verify the new fingerprint through the Dashboard console first, then clear the old entry with ssh-keygen -R 203.0.113.10 and reconnect.
  • `Host key verification failed` — StrictHostKeyChecking yes is set but no entry exists. Add the verified key to known_hosts rather than weakening the setting.
  • `No ED25519 host key is known` — the server does not offer that algorithm. Scan the type it does offer, for example ssh-keyscan -t rsa 203.0.113.10, and check the HostKey lines in /etc/ssh/sshd_config.
  • You were prompted for a password after a warning — the warning was accepted blindly and the connection continued. Treat the credentials you typed as compromised and rotate them.
  • `grep` finds nothing in `known_hosts` — entries are hashed. Use ssh-keygen -F 203.0.113.10 instead of searching the file.
  • `sshd` will not start after key rotation — check ownership and permissions on /etc/ssh/ssh_host_*_key and run restorecon on SELinux systems.

Next steps

Was this guide helpful?

Still stuck?

Ask RunRacks AI for a quick answer, or open a ticket and our team will take a look at your server.

Security

Check the Windows Firewall rule for RDP port 12230

Find and verify the inbound TCP 12230 rule that lets Remote Desktop reach your RunRacks server, recreate it if it was deleted and scope it without locking yourself out.

  • Intermediate
  • Windows VPS · Admin RDP
  • 5 min read

Security

iptables basics: list, allow, drop and save rules

Learn how iptables chains and rule order work, then build a small ruleset that allows SSH and web traffic, drops everything else and can be edited by line number.

  • Intermediate
  • Linux VPS
  • 6 min read