Securing Linux Servers with SSH Keys: A Step-by-Step Guide for Ubuntu, Debian, and compatible Distros

Boost your Linux server security with SSH key-based authentication. This quick guide covers key generation, setup, and disabling password logins. Works with Ubuntu, Debian, CentOS, Fedora, and Arch Linux. Perfect for sysadmins and developers.

Securing Linux Servers with SSH Keys: A Step-by-Step Guide for Ubuntu, Debian, and compatible Distros
Photo by rc.xyz NFT gallery / Unsplash

🔐 Introduction

SSH (Secure Shell) is a cryptographic protocol that enables secure communication between a client and a server. Utilizing SSH keys instead of passwords enhances security by mitigating risks associated with brute-force attacks and password theft. This guide will walk you through generating SSH keys, copying them to your server, and configuring your system for key-based authentication.


🛠️ Step 1: Generate an SSH Key Pair

On your local machine (the client), open a terminal and run:

 ssh-keygen -t rsa -b 4096
  • -t rsa: Specifies the RSA algorithm.
  • -b 4096: Sets the key length to 4096 bits for enhanced security.

You'll be prompted to specify a file to save the key:

Enter file in which to save the key (/home/your_user/.ssh/id_rsa):

Press Enter to accept the default location or provide a custom path.

Next, you'll be asked to enter a passphrase:

Enter passphrase (empty for no passphrase):

While optional, adding a passphrase is recommended for additional security. After completion, your keys will be saved as:

  • Private key: ~/.ssh/id_rsa
  • Public key: ~/.ssh/id_rsa.pub

📤 Step 2: Copy the Public Key to Your Server

If your local system has ssh-copy-id installed, you can transfer your public key to the server with:

ssh-copy-id username@remote_host

Replace username with your server's username and remote_host with its IP address or hostname. You'll be prompted to enter the user's password. This command appends your public key to the server's ~/.ssh/authorized_keys file.

Option 2: Manually Copying the Public Key

If ssh-copy-id isn't available, manually copy your public key:

  1. Display your public key: cat ~/.ssh/id_rsa.pub
  2. Connect to your server using SSH: ssh username@remote_host
  3. On the server, create the .ssh directory if it doesn't exist:mkdir -p ~/.ssh
  4. Open the authorized_keys file in a text editor: nano ~/.ssh/authorized_keys
  5. Paste your public key into the file, save, and exit.

Set appropriate permissions: chmod 600 ~/.ssh/authorized_keys
chmod 700 ~/.ssh


✅ Step 3: Test SSH Key Authentication

Now, attempt to SSH into your server:

ssh username@remote_host

If everything is set up correctly, you should be logged in without being prompted for a password. If you set a passphrase during key generation, you'll be asked to enter it.


To enhance security, disable password authentication:

  1. Open the SSH daemon configuration file on your server: sudo nano /etc/ssh/sshd_config
  2. Locate or add the following lines:
  3. PasswordAuthentication no
    ChallengeResponseAuthentication no
    UsePAM no
  4. Save and exit the file.
  5. Restart the SSH service: sudo systemctl restart ssh
Note: Ensure you have SSH key-based access configured correctly before disabling password authentication to prevent being locked out.

🧪 Step 5: Verify Configuration

Open a new terminal session and attempt to SSH into your server:

ssh username@remote_host

If successful, your SSH key-based authentication is correctly configured, and password authentication has been disabled.


You've successfully set up SSH key-based authentication on your server, enhancing its security posture. This method is applicable to Ubuntu 22.04 and other Debian-based distributions. Remember to keep your private key secure and consider using a passphrase for added protection.