Harden Your SSH Keys: Generate, Store and Rotate Them

An SSH key is the credential most people create once and then forget for years. That is exactly what makes it valuable to an attacker. A copied private key gives silent, password free access to every server that trusts it, and nothing in the login log looks unusual. The steps below take about fifteen minutes and cover generation, storage, and rotation.

Generate, protect, rotate

1. Generate a modern key pair. Run the OpenSSH key generation tool (ssh keygen) and select ed25519 as the key type when it prompts you. Ed25519 keys are short, fast to verify, and avoid the weak randomness problems that have burned some ECDSA implementations. If you still need to reach very old servers, create a separate RSA key for those hosts alone so one legacy key does not unlock everything. Threat defended: a key type that is cheap to brute force or was generated with predictable entropy.
2. Put a strong passphrase on the private key. Use a long, unique passphrase stored in your password manager, not a short one you can type from memory. Threat defended: a stolen laptop, a synced backup, or malware that reads your home directory. Without a passphrase the private key file is the whole credential.
3. Lock down file permissions. Set your key directory to 700 and the private key to 600, leaving the public key readable at 644. OpenSSH will refuse a key that is too open, but other tools will not. Threat defended: another local account, a container, or a compromised service reading your key off disk.
4. Use an agent with a timeout, or a hardware token. If you load keys into an agent, set a lifetime so the unlocked key is dropped after a few hours, and never enable agent forwarding to hosts you do not fully control. A hardware token keeps the private key off the disk entirely. Threat defended: an unlocked key sitting in memory for days, and a compromised jump host borrowing your agent to reach other machines.
5. Verify host keys on first connection. Compare the fingerprint through a second channel before you accept it, and keep strict host key checking on. Threat defended: a man in the middle who terminates your connection and relays it onward while reading everything in between.
6. Keep the public key where it belongs. Only the public half goes into authorized_keys, tickets, or a repository. Never paste a private key into chat, an issue tracker, or a config file, even to move it between your own machines. Threat defended: secret sprawl, where one leaked message or commit exposes access to every host that trusted the key.
7. Rotate on a schedule and after any doubt. Create the new key, add its public half everywhere the old one is trusted, confirm you can log in, then remove the old public key from authorized_keys on every host. Oracle's 2026 certificate guidance notes that lifetimes across the industry are shrinking and machine identities are multiplying, so treat keys as inventory with an owner and an expiry date rather than permanent fixtures.

If you cannot say when a key was created, where its private half lives, and who holds a copy, assume it is compromised and replace it. Rotation is boring, quick, and the only thing that limits the damage of a key you have already lost.