How to Harden SSH on a Linux Server

A fresh Linux VPS with password SSH enabled will show several hundred failed login attempts within hours. That’s not paranoia — it’s the internet. Bots scan every reachable IP continuously, trying common usernames and passwords. Hardening SSH means removing the cheap wins: no password auth, no root login, no default port. This guide does that in order, from highest to lowest impact.

:::note[TL;DR]

Most impactful: disable password auth, require SSH keys
Disable direct root login (PermitRootLogin no)
Change the default port (reduces bot noise, not actual security)
Install fail2ban to automatically ban brute-force IPs
Optional: add 2FA with Google Authenticator or TOTP :::

Prerequisites

Ubuntu 22.04/24.04 or Debian 12 (commands are the same for both)
SSH access to the server
A non-root user with sudo privileges
An SSH key pair already set up — do not disable password auth until you’ve confirmed key auth works

:::warning Never disable password authentication before verifying that your SSH key works. If you lock yourself out, you’ll need console access (your hosting provider’s web console) or a rescue boot to fix it. :::

Step 1: Set up SSH key authentication

If you haven’t already, generate a key pair on your local machine:

# On your local machine (not the server)
ssh-keygen -t ed25519 -C "your@email.com"

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your-server-ip

Or manually append it:

# On the server
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "your-public-key-contents" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Test that key auth works before changing anything else:

# In a new terminal window — keep your existing session open
ssh -i ~/.ssh/id_ed25519 user@your-server-ip

If this connects without asking for a password, you’re ready to proceed.

Step 2: Edit the SSH daemon configuration

All changes go in /etc/ssh/sshd_config. Make a backup first:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
sudo nano /etc/ssh/sshd_config

Find and change these directives (add them if they don’t exist):

# Disable root login
PermitRootLogin no

# Disable password authentication
PasswordAuthentication no
ChallengeResponseAuthentication no

# Disable empty passwords (belt and suspenders)
PermitEmptyPasswords no

# Only allow your specific user (optional but strong)
AllowUsers yourusername

# Limit login attempts per connection
MaxAuthTries 3

# Disconnect idle sessions after 10 minutes
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable unused auth methods
KbdInteractiveAuthentication no
UsePAM yes

# Disable X11 forwarding if you don't need it
X11Forwarding no

On modern Ubuntu, there’s a drop-in directory that may override sshd_config. Check it:

ls /etc/ssh/sshd_config.d/

If there’s a file like 50-cloud-init.conf that contains PasswordAuthentication yes, it overrides your main config. Either edit that file or delete it.

Step 3: Test and reload

Never reload SSH without testing the config first:

sudo sshd -t

If it returns no output, the config is valid. If there’s an error, fix it before reloading.

Reload the SSH service. Reloading (not restarting) keeps existing connections alive:

sudo systemctl reload ssh
# or on some systems:
sudo systemctl reload sshd

In a new terminal window, confirm you can still connect with your key. Keep your existing session open. If the new connection fails, you can use the existing session to revert.

How do you change the default SSH port?

Changing from port 22 doesn’t stop a determined attacker — any scanner worth using tries common alternate ports too. But it eliminates most of the automated bot traffic, which reduces log noise and fail2ban load.

In /etc/ssh/sshd_config:

Port 2222

If your server has a firewall (it should), allow the new port before reloading:

sudo ufw allow 2222/tcp
sudo ufw deny 22/tcp   # Only after confirming the new port works

Test with the explicit port:

ssh -p 2222 user@your-server-ip

Update your ~/.ssh/config on your local machine to avoid typing -p every time:

Host myserver
    HostName your-server-ip
    User yourusername
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Now ssh myserver just works.

How do you install and configure fail2ban?

fail2ban monitors SSH logs and bans IPs that fail authentication too many times. It writes firewall rules (via iptables/nftables) automatically.

sudo apt install fail2ban -y

Create a local config that won’t be overwritten on package updates:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

Find the [sshd] section and configure it:

[sshd]
enabled  = true
port     = 2222          # Your SSH port
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 5             # Ban after 5 failures
bantime  = 1h            # Ban for 1 hour
findtime = 10m           # Count failures within this window

Enable and start fail2ban:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Check that the SSH jail is active:

sudo fail2ban-client status sshd

Useful commands:

# See all currently banned IPs
sudo fail2ban-client status sshd

# Manually ban an IP
sudo fail2ban-client set sshd banip 1.2.3.4

# Unban an IP
sudo fail2ban-client set sshd unbanip 1.2.3.4

How do you add two-factor authentication?

2FA adds a TOTP code requirement on top of your SSH key. Even if your private key is compromised, an attacker still can’t log in without the current one-time code.

Install the PAM module:

sudo apt install libpam-google-authenticator -y

Run the setup as your user (not root):

google-authenticator

Answer yes to:

Time-based tokens
Update the .google_authenticator file
Disallow multiple uses of the same token
Emergency scratch codes (save these)

In /etc/pam.d/sshd, add at the top:

auth required pam_google_authenticator.so

In /etc/ssh/sshd_config:

KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

The AuthenticationMethods publickey,keyboard-interactive line means the user must provide both a key AND the TOTP code. Neither alone is sufficient.

Reload SSH and test in a new window.

What other hardening steps are worth doing?

Disable unused authentication methods:

# In sshd_config
GSSAPIAuthentication no
HostbasedAuthentication no
IgnoreRhosts yes

Use a stronger host key algorithm. Regenerate RSA host keys at 4096 bits if you have old ones:

sudo rm /etc/ssh/ssh_host_rsa_key*
sudo ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
sudo systemctl restart ssh

Restrict SSH to specific IPs using UFW or iptables:

# Only allow SSH from your office IP
sudo ufw allow from 203.0.113.10 to any port 2222
sudo ufw deny 2222

Audit who has access: review ~/.ssh/authorized_keys for every user periodically. Remove keys for team members who have left.

# Check all authorized_keys files on the system
sudo find /home -name authorized_keys -exec cat {} \; -print
sudo cat /root/.ssh/authorized_keys 2>/dev/null

Summary

Keys over passwords is the most important change — do this first
PermitRootLogin no and PasswordAuthentication no in sshd_config cover 90% of the attack surface
Always sudo sshd -t before reloading, and test from a new window before closing the existing session
fail2ban automates IP banning for brute-force attempts
2FA with AuthenticationMethods publickey,keyboard-interactive requires both a key and a TOTP code

FAQ

I locked myself out. What do I do?

If your hosting provider offers a web console (DigitalOcean, Hetzner, AWS EC2 Instance Connect, etc.), connect through that. It bypasses SSH entirely. Once in, fix your sshd_config and reload. If there’s no web console, you’ll need to boot from a rescue image, mount your drive, and fix the config from there. This is why you keep your existing session open while testing.

Should I use RSA or Ed25519 for my key?

Ed25519 for new keys. It’s smaller, faster, and considered stronger than 2048-bit RSA. If you have existing 2048-bit RSA keys still in use, they’re not broken — but new keys should be Ed25519. Avoid DSA (broken) and ECDSA with NIST curves if you’re security-paranoid about curve choice.

Does changing the SSH port actually help?

It eliminates most automated bot traffic, which keeps your logs clean and reduces load on fail2ban. It doesn’t stop a targeted attack — any real scanner probes all ports. Think of it as reducing noise, not adding real security.