SSH Config Hardening: Locking Down Remote Access¶

Default SSH config on most Linux distros is functional but not tight. It allows password authentication, doesn't restrict which users can log in, and leaves a bunch of options at permissive defaults. If your server is exposed to the internet at all — even just on a non-standard port — you will get probed constantly. This article covers the changes that actually matter: key-based auth, disabling root login, restricting access, and a few other settings that close the most common attack vectors without making SSH annoying to use.

The Short Answer¶

# Edit the SSH server config
sudo nano /etc/ssh/sshd_config

# Key changes to make:
PermitRootLogin no
PasswordAuthentication no
AuthenticationMethods publickey
AllowUsers yourusername
Port 22  # or change to non-standard port

# Restart SSH after changes (pick the right one for your distro)
sudo systemctl restart sshd      # most distros
sudo systemctl restart ssh       # Debian/Ubuntu

Background¶

SSH is the front door to your Linux machines. The default config prioritizes compatibility over security — it has to, because it ships on everything from Raspberry Pis to enterprise servers with wildly different use cases. That means it's on you to tighten it up for your specific situation.

The threat model for a homelab or personal server is mostly automated scanners and bots. They probe every IP on port 22, try common username/password combos, and move on. Key-based authentication defeats this entirely — no password, no brute force. The rest of the hardening is defense-in-depth: make it harder to even attempt an attack in the first place.

If your server is behind Tailscale and not directly internet-exposed, some of this is belt-and-suspenders. Do it anyway. Defense in depth means you're not one misconfiguration away from an open door.

Steps¶

1. Generate SSH Keys (If You Haven't Already)¶

Do this on your client machine (the machine you're connecting from), not the server.

# Generate an Ed25519 key (preferred — smaller, faster, more secure than RSA)
ssh-keygen -t ed25519 -C "your-comment-here"

# Or RSA 4096 if you need compatibility with older systems
ssh-keygen -t rsa -b 4096 -C "your-comment-here"

When prompted for a passphrase, use one. It encrypts the private key on disk so that stealing the key file isn't enough to use it.

Your keys will be at: - Private key: ~/.ssh/id_ed25519 — never share this, never copy this to a server - Public key: ~/.ssh/id_ed25519.pub — this goes on the server

2. Copy Your Public Key to the Server¶

# Easiest method — works if password auth is still enabled
ssh-copy-id username@server-ip

# Manual method — if ssh-copy-id isn't available
cat ~/.ssh/id_ed25519.pub | ssh username@server-ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Verify you can log in with the key before disabling password auth:

ssh username@server-ip
# Should log in without asking for a password (may ask for key passphrase)

Do not skip this verification step. Disabling password auth before confirming key auth works will lock you out.

3. Edit sshd_config¶

sudo nano /etc/ssh/sshd_config
# or
sudo vim /etc/ssh/sshd_config

Changes to make:

# Disable root login — no reason to ever SSH in as root
PermitRootLogin no

# Disable password authentication — keys only
PasswordAuthentication no
PermitEmptyPasswords no

# Explicitly set allowed auth methods
AuthenticationMethods publickey

# Restrict which users can SSH in (whitelist approach)
# Only users listed here can authenticate — everyone else is denied
AllowUsers yourusername anotherusername

# Disable unused auth methods
KbdInteractiveAuthentication no
UsePAM yes  # keep this — PAM is needed for other things, just disable the password methods above

# Limit login attempts per connection
MaxAuthTries 3

# Close idle sessions after inactivity (seconds)
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable X11 forwarding if you don't use it
X11Forwarding no

# Disable TCP forwarding if you don't need it
# (comment this out if you use SSH tunnels)
# AllowTcpForwarding no

# Log level — INFO is fine, VERBOSE if you want more detail
LogLevel INFO

4. Optional: Change the Default Port¶

Moving SSH off port 22 doesn't make you secure — anyone doing a port scan will find it. But it does cut the noise in your logs dramatically since most automated scanners only hit 22.

# In sshd_config
Port 2222  # or any port above 1024 you prefer

If you change the port, remember to: - Update your firewall rules to allow the new port - Update your SSH client config (~/.ssh/config) so you don't have to type -p 2222 every time - Update any Tailscale or reverse proxy configs that reference SSH

If your server is only accessible via Tailscale, changing the port is optional — the attack surface is already minimal.

5. Validate the Config Before Restarting¶

sudo sshd -t

This checks sshd_config for syntax errors without actually restarting the service. If it returns nothing, the config is valid. If it returns errors, fix them before proceeding — a bad config can lock you out on restart.

6. Restart SSH¶

# Debian/Ubuntu
sudo systemctl restart ssh

# RHEL/Fedora/CentOS/Alma/Rocky
sudo systemctl restart sshd

# Verify it's running
sudo systemctl status sshd

7. Test From a New Terminal Window¶

Before closing your current SSH session, open a new terminal and test that you can still log in:

ssh username@server-ip
# or if you changed the port:
ssh -p 2222 username@server-ip

If the new session connects, you're good. If it fails, go back to your existing session and troubleshoot — you're still in. This is the one precaution that prevents the "locked myself out" disaster.

8. Set Up Your SSH Client Config¶

On your client machine, create or edit ~/.ssh/config to avoid typing connection details every time:

Host myserver
    HostName 100.x.x.x          # Tailscale IP or MagicDNS hostname
    User yourusername
    Port 22                      # or your custom port
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3

Now you just type ssh myserver instead of the full command.

9. Set Up Fail2ban (Optional but Recommended)¶

If your server has any public exposure, Fail2ban automatically bans IPs that repeatedly fail authentication:

# Debian/Ubuntu
sudo apt install fail2ban

# RHEL/Fedora
sudo dnf install fail2ban

Create a local config (don't edit the default — it gets overwritten on updates):

sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port = ssh          # or your custom port number
maxretry = 3
bantime = 1h
findtime = 10m

sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd  # verify it's watching SSH

Gotchas & Notes¶

Always verify key auth works before disabling passwords. This is worth repeating. Open a second terminal, test the key login, confirm it works, then disable password auth. Skipping this step is how people lock themselves out.
sshd_config vs ssh_config — sshd_config is the server config (controls how your machine accepts connections). ssh_config and ~/.ssh/config are client configs (controls how your machine makes connections). They're different files. You want sshd_config for hardening.
Debian/Ubuntu use ssh as the service name, most others use sshd. systemctl restart ssh vs systemctl restart sshd. Get this wrong and you'll think the restart failed when it actually just couldn't find the service name.
Some distros drop overrides in /etc/ssh/sshd_config.d/. On Ubuntu 22.04+, there's a 50-cloud-init.conf or similar in that directory that can override your sshd_config settings. Check what's in there: ls /etc/ssh/sshd_config.d/. If there's a file setting PasswordAuthentication yes, it will override your change in the main config.
AllowUsers is a whitelist, not a blacklist. If you add AllowUsers to your config, only the users you list can log in. Everyone else is denied, even if they have a valid key. If you have multiple users who need access, list them all: AllowUsers alice bob charlie.
Ed25519 keys are better than RSA for new setups. Shorter, faster, more secure. The only reason to use RSA 4096 is compatibility with very old SSH clients or servers. If you're connecting to modern Linux machines, use Ed25519.
Keep a recovery option. If you're on a cloud VM or a machine you can't physically access, make sure you have a console/out-of-band access method (provider's web console, IPMI, etc.) before tightening SSH. If you're on a homelab machine you can physically reach, less of a concern — but still worth having a bootable USB around.