Linux SSH Keys

Why SSH Keys Matter

SSH keys are one of the most common ways to authenticate securely to Linux servers, Git hosts, and automation systems. Compared to passwords, they are more resistant to guessing attacks and much easier to automate safely.

For DevOps work, SSH keys are everywhere: server access, Git operations, CI deployments, bastion hosts, and remote automation.

How SSH Key Authentication Works

SSH key authentication is based on a public/private key pair.

• the private key stays on your machine and must be protected
• the public key can be shared with servers or services

When you connect, the server checks whether your public key is trusted and then proves that your client holds the matching private key.

The private key should never be copied casually between machines or committed to source control.

Generating a Key with ssh-keygen

The recommended modern key type for most users is ed25519.

ssh-keygen -t ed25519 -C "devops@example.com"

You will normally be asked:

• where to save the key
• whether to set a passphrase

A passphrase protects the private key at rest. If someone copies your key file, the passphrase adds an extra barrier.

Example custom filename:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_work -C "work key"

Tip: Use separate keys for different environments when you need better isolation, such as work, personal, and production access.

Installing the Public Key on a Server

The server stores trusted public keys in:

~/.ssh/authorized_keys

A convenient way to copy your key is:

ssh-copy-id devops@server.example.com

That appends your public key to the remote authorized_keys file.

You can also do it manually by copying the contents of your .pub file.

cat ~/.ssh/id_ed25519.pub

Then paste that public key into ~/.ssh/authorized_keys on the server.

Correct File Permissions

SSH is strict about permissions because weak permissions can expose keys.

Typical secure settings are:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

If permissions are too open, SSH may refuse to use the files.

Note: Permission problems are a very common reason key authentication fails even when the key content itself is correct.

Using ~/.ssh/config

The SSH client config file lets you define shortcuts and per-host settings.

Example:

Host prod-web
    HostName server.example.com
    User devops
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_work

After this, you can connect with:

ssh prod-web

Useful options in ~/.ssh/config include:

• Host for the alias
• HostName for the real server name or IP
• User for the login account
• Port for non-default SSH ports
• IdentityFile for selecting a specific key

This keeps complex SSH commands out of your memory and scripts.

Working with ssh-agent and ssh-add

If your private key has a passphrase, entering it every time can be inconvenient. ssh-agent stores unlocked keys in memory for the session.

Start or use the agent, then add a key:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l

Now SSH can use the unlocked key without prompting for the passphrase on every connection.

Agent Forwarding

Agent forwarding lets a remote host use your local SSH agent to authenticate to another host without copying your private key to the first server.

Example:

ssh -A bastion.example.com

This is useful in jump-host workflows, but it should be enabled carefully. If the intermediate host is compromised, agent forwarding can increase risk.

Only use it where you trust the systems involved.

A Practical Workflow

A typical first-time setup looks like this:

ssh-keygen -t ed25519 -C "devops@example.com"
ssh-copy-id devops@server.example.com
ssh devops@server.example.com

If the login fails, check:

• whether the correct public key is in authorized_keys
• whether ~/.ssh and key files have safe permissions
• whether the correct private key is being used
• whether the server allows key authentication

For noisy debugging, SSH can show details:

ssh -v devops@server.example.com

What You Should Remember

SSH keys work by matching a local private key with a trusted public key on the server. Generate modern keys with ssh-keygen, protect them with a passphrase, use authorized_keys correctly, and simplify repeated connections with ~/.ssh/config and ssh-agent.

Once this workflow feels natural, remote Linux access becomes faster and more secure.

Test Your Understanding

---

Frequently Asked Questions

Should I generate an ed25519 or an RSA SSH key?

ed25519, unless you must talk to something ancient. ssh-keygen -t ed25519 -C "you@machine" produces a short, fast key with security comparable to a 3072-bit RSA key, and it is supported everywhere current. Choose RSA only for legacy systems that do not accept ed25519, and then use at least 4096 bits. Do not use DSA or ECDSA: DSA is disabled in modern OpenSSH, and ECDSA carries curve concerns ed25519 avoids.

How do I fix "Permissions 0644 for id_ed25519 are too open"?

SSH refuses to use a private key that other accounts on the machine can read, and simply stops rather than risking it. chmod 600 ~/.ssh/id_ed25519 fixes the key, chmod 700 ~/.ssh the directory, and chmod 644 is correct for the .pub file, which is public by design. On the server side, ~/.ssh/authorized_keys must be 600 and the user's home directory must not be group-writable, that last one is the cause of most "key is correct but login still fails" cases.

What is the difference between the public and private key?

The private key stays on your machine and proves who you are; the public key is copied to every server you want to reach and can be shared freely. Authentication works by the server sending a challenge that only the matching private key can answer, so the private key never crosses the network, which is why it is safer than a password. ssh-copy-id user@host installs the public key into the server's authorized_keys. If you are ever asked to send a key, it is the .pub one.

Should I put a passphrase on my SSH key?

Yes. Without one, anyone who obtains the file has your access to every server that trusts it, including from a stolen laptop or a backup. A passphrase encrypts the key at rest, and ssh-agent means you type it once per session rather than per connection: ssh-add ~/.ssh/id_ed25519, or on macOS ssh-add --apple-use-keychain to persist across reboots. Keys used by automation are the exception, and those should be separate, narrowly scoped keys rather than your personal one.

What does ~/.ssh/config do and why use it?

It stores per-host settings so you stop retyping them. A block such as Host prod with HostName 10.0.4.12, User deploy, Port 2222 and IdentityFile ~/.ssh/prod_ed25519 turns a long command into ssh prod, and scp, rsync and Git all honour it. It is also where you configure jump hosts with ProxyJump bastion, which is far cleaner than chaining SSH commands, and ServerAliveInterval 60 to stop idle sessions dropping.

Is SSH agent forwarding safe?

Not entirely, and it is worth knowing why. ssh -A lets processes on the remote host use your agent to authenticate onward, which means root on that host, or anyone who compromises it, can use your key while you are connected. They cannot steal the key itself, but they can act as you for the duration. Prefer ProxyJump (ssh -J bastion target), which tunnels the connection without exposing the agent. If you must forward, do it per-host in ~/.ssh/config, never globally.