AWS EC2 Key Pairs

A key pair is how SSH authentication to an EC2 instance traditionally works: AWS holds the public key, you hold the private key, and the public key is written into the instance at first boot.

It works, and for most workloads there is now a better option. This page covers both, because inherited infrastructure rarely gets to choose.

How it actually works

When you launch an instance with a key pair, cloud-init writes the public key into ~/.ssh/authorized_keys for the default user, ec2-user on Amazon Linux, ubuntu on Ubuntu, admin on Debian.

That single sentence explains most key pair behaviour:

• It happens once, at first boot. Changing the key pair on a stopped instance has no effect.
• AWS never has your private key. If you lose it, AWS cannot recover it.
• The key is just a file on the instance, which is why the recovery procedure below works.

Creating one

aws ec2 create-key-pair \
  --key-name acme-prod \
  --key-type ed25519 \
  --query 'KeyMaterial' --output text > ~/.ssh/acme-prod.pem

chmod 400 ~/.ssh/acme-prod.pem

Use ed25519 rather than RSA. It is faster, the keys are shorter, and it is the modern default. RSA remains available for older tooling that needs it.

That chmod 400 is not optional. SSH refuses to use a private key readable by others:

Permissions 0644 for 'acme-prod.pem' are too open.
It is required that your private key files are NOT accessible by others.

The private key is shown once, at creation. There is no way to download it again.

Alternatively, import a key you already have, so AWS never sees a private key at all:

aws ec2 import-key-pair --key-name acme-prod \
  --public-key-material fileb://~/.ssh/id_ed25519.pub

This is the better approach for personal keys.

Connecting

ssh -i ~/.ssh/acme-prod.pem ec2-user@<public-ip>

Getting the right username is a common first stumble:

Beyond the key, SSH needs a security group allowing inbound 22 from your address, a route to the instance, and a public IP if you are outside the VPC.

Prefer Session Manager

The key pair model has real drawbacks: a private key to distribute and protect, port 22 open somewhere, a bastion host for private subnets, and no audit trail of who connected.

SSM Session Manager removes all of it:

aws ssm start-session --target i-0abc123

• No key pairs. Nothing to distribute, lose or rotate.
• No inbound ports. The instance makes an outbound connection to SSM.
• No bastion. Works for instances in private subnets with no public IP.
• IAM-controlled, so access is granted and revoked like any other permission.
• Logged to CloudTrail, and session content can be recorded to S3 or CloudWatch.

The requirements are modest: the SSM agent, preinstalled on current Amazon Linux and Ubuntu AMIs, and an instance profile with AmazonSSMManagedInstanceCore.

For anything new, this should be the default. Launch instances with no key pair at all.

You can even use it as an SSH transport, which keeps your SSH tooling while removing the open port:

# ~/.ssh/config
Host i-* mi-*
  ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters portNumber=%p"

ssh -i ~/.ssh/acme-prod.pem ec2-user@i-0abc123

EC2 Instance Connect

A middle option: push a temporary public key valid for 60 seconds, authorised by IAM.

aws ec2-instance-connect send-ssh-public-key \
  --instance-id i-0abc123 \
  --instance-os-user ec2-user \
  --ssh-public-key file://~/.ssh/id_ed25519.pub

ssh -i ~/.ssh/id_ed25519 ec2-user@<public-ip>

No permanent key on the instance, and access controlled by IAM. It still needs network reachability on port 22, which is why Session Manager is generally the better choice, though the EC2 Instance Connect Endpoint variant removes that requirement too.

Recovering an instance with a lost key

AWS cannot recover a private key, but the key is only a file on a volume, so access is recoverable.

The easy route, if the SSM agent is running: connect with Session Manager and append a new public key to ~/.ssh/authorized_keys. No downtime.

The easy route on a modern AMI: EC2 Instance Connect, if the instance supports it.

The manual route, when neither is available:

1. Stop the instance and detach its root EBS volume.
2. Attach that volume to a second instance you can access, as a secondary device.
3. Mount it, and append your new public key to /home/ec2-user/.ssh/authorized_keys on the mounted filesystem.
4. Unmount, detach, reattach to the original instance as its root device, start it.

Tedious but reliable. An alternative is replacing the instance entirely from an AMI with the new key, which is often faster if the instance is cattle rather than a pet.

Key pairs are regional

A key pair exists in one region. Launching in a second region requires creating or importing it there:

aws ec2 import-key-pair --key-name acme-prod \
  --public-key-material fileb://~/.ssh/id_ed25519.pub \
  --region us-east-1

Importing the same public key into several regions keeps one private key working everywhere, which is why importing is more convenient than generating per region.

Quick reference

---

Key takeaways

• The public key is written at first boot only. Changing the key pair on an existing instance does nothing.
• AWS never holds your private key and cannot recover it.
• chmod 400 the .pem, or SSH refuses to use it.
• Use ed25519, and prefer importing your own public key so AWS never sees a private one.
• Session Manager is the better default: no keys, no open ports, no bastion, IAM-controlled and audited.
• A lost key is recoverable via SSM, Instance Connect, or by mounting the root volume elsewhere.
• Key pairs are regional. Import the same public key into each region you use.

---

Check your understanding

---

Frequently Asked Questions

I lost my EC2 private key. Can AWS recover it?

No. AWS never stores private keys and has no way to retrieve one. Access is still recoverable, though. If the SSM agent is running, connect with aws ssm start-session and append a new public key to ~/.ssh/authorized_keys, which needs no downtime. If not, stop the instance, detach its root volume, attach it to another instance you can reach, mount it and add your key to the file, then reattach. Replacing the instance from an AMI is often faster if it holds no unique state.

How do I change the key pair on an existing EC2 instance?

You cannot change it through the AWS API in any meaningful way. The public key is written into ~/.ssh/authorized_keys by cloud-init on first boot and never touched again, so the key pair association is effectively historical metadata afterwards. To add a new key, edit that file on the instance. Through Session Manager, EC2 Instance Connect, or by mounting the root volume elsewhere. Then remove the old key from the file if you want to revoke it.

Should I use SSH keys or Session Manager?

Session Manager, for anything new. It requires no key pairs, no inbound port 22, and no bastion host; it works for instances in private subnets with no public IP; access is granted through IAM and revoked the same way; and every session is recorded in CloudTrail with optional full session logging. The requirements are just the SSM agent, preinstalled on current AMIs, and an instance profile with AmazonSSMManagedInstanceCore. Launch new instances with no key pair at all.

Why does SSH reject my .pem file with "permissions too open"?

SSH refuses to use a private key that other users on the machine could read, as a safety measure. Run chmod 400 your-key.pem so only your user can read it. On Windows the equivalent is removing inherited permissions and granting access only to your account. This is one of the first errors nearly everyone hits, and the message is at least explicit about the cause.

What username do I use to SSH into an EC2 instance?

It depends on the AMI: ec2-user for Amazon Linux and RHEL, ubuntu for Ubuntu, admin for Debian, bitnami for Bitnami images. Using root almost always fails, since most AMIs disable direct root login and print a message telling you the correct user. If you are unsure, attempt a connection as root and read the error. It usually names the right account.

Are EC2 key pairs regional?

Yes. A key pair created in eu-west-1 does not exist in us-east-1, so launching there requires creating or importing one in that region. Importing the same public key into each region is more convenient than generating separate pairs, since one private key then works everywhere, and importing has the further advantage that AWS never handles a private key at all.