Give every job its own key
What you will learn
- Why copying one key to every machine feels great and ages badly
- The single question that tells you how bad a leak would be
- Why the reason to change this is not leaks at all, it is replacing keys
There is a moment, once you have a few machines talking to each other, when you make one key, copy it everywhere, and everything works. It is a genuinely great feeling. It is also the single decision most likely to turn a small future mistake into a large one.
A key here is a long secret file that proves one machine is allowed to talk to another, rather like a password that is too long to type. Programs use them so they can connect without a person present.
The appeal is obvious. One key means one thing to back up, one thing to remember, and one thing to fix when something cannot connect. Every new machine joins instantly. Nothing about it feels careless, and that is exactly why it is worth thinking about before you have twelve machines holding the same secret.
The problem only shows up on your worst day
A shared key means every job holding it can do everything any of them can do.
So the small script you wrote in a hurry, the one that just moves some files around, is carrying exactly the same authority as the thing that manages your backups. It does not need that authority. It has it because copying was easier than making a new one.
If this exact key leaked today, what could somebody do with it?
With one shared key, the honest answer is everything, everywhere. With one key per job it is one job on one machine, and that is a difference you can sleep on.
The real reason, which is not leaks
A leak is not the only way this bites, and honestly it is not the most likely one. Replacing the key is.
The day you need to replace a shared key, you have to replace it everywhere at the same moment, and anything you miss breaks without warning. That job is unpleasant enough that people put it off, which means the shared key stays in place long past the point where anybody remembers every copy of it.
One key per job turns that from an event into a chore. Replacing one is a two minute task affecting one thing, so you actually do it.
The key that gets replaced regularly is worth more than the one that is theoretically stronger and has not been touched in two years.
A building master key opens every door, so the cleaner, the contractor and the courier all carry the same one.
Nobody thinks that is fine in a building. It is somehow fine on our own machines, right up until it is not.
When I added a new automatic job recently, I gave it its own key rather than handing it the shared one. The whole detour took a couple of minutes. Now if that job is ever compromised, the honest answer to the question above is one account on one machine, and I can replace its key without touching anything else.
Capability is risk. Hand out the smallest amount that lets the job work, so one mistake can never become all of them.
Shown with the keys machines use to log into each other, because that is where most home setups share one. The principle is identical for the keys that programs use to reach online services.
1. Find out what is sharing a key today
What this does: lists which keys are allowed to log into this machine and how many times each appears.
What it tells you: if one key appears everywhere, that is your exposure, written down in front of you.
cat ~/.ssh/authorized_keys | awk '{print $NF}' | sort | uniq -c2. Make a separate key for each job
What this does: creates a new key just for one job.
Name it after the job, not the machine, so you can tell later what it is for.
Why no password on it: because nobody is present to type one. That is exactly why the next step matters so much.
ssh-keygen -t ed25519 -f ~/.ssh/id_backup_job -C "backup-job" -N ""3. Restrict what the key is allowed to do
What this does: pins the key to one source address, switches off extra abilities, and forces it to run one specific command and nothing else.
Where it goes: on the machine being connected to, as a single line.
restrict,from="10.0.0.5",command="/usr/local/bin/backup-receive" ssh-ed25519 AAAA... backup-job4. Give it its own account
What this does: creates a limited user account that cannot log in as a person.
What it buys you: the job can now only touch what that account owns.
sudo useradd -m -s /usr/sbin/nologin backupjob5. Lock down the file permissions
What these do: the first makes the key readable only by you. The second lists any key file that is not.
What to expect: the second should print nothing. Anything it lists is wrong. Run it on every machine, not just the one you set up carefully.
chmod 600 ~/.ssh/id_backup_job
find ~/.ssh -type f ! -perm 600 -ls6. Point the job at its own key, explicitly
What this is: a settings file, not a command.
The important last line: it stops the program offering any other key it happens to find, so the pairing is deliberate rather than accidental.
Host backup-target
HostName 10.0.0.9
User backupjob
IdentityFile ~/.ssh/id_backup_job
IdentitiesOnly yes # do not offer any other key7. Prove the restriction actually holds
What this does: deliberately tries to do something the key should not be allowed to do.
What should happen: it should not give you a command prompt. If it does, your restriction is decorative.
This takes thirty seconds and it is the only reason to believe any of the above.
ssh -i ~/.ssh/id_backup_job [email protected] "whoami"8. Replace keys one at a time
Make the replacement, add it alongside the old one, switch the job over, confirm it still works, and only then remove the old line. Because it belongs to one job, nothing else can break.
9. The same rule applies to online services
Everything here transfers. One key per service, limited to the minimum it needs, named for its job, replaced individually. A read-only key for a job that only reads is the same idea.
Related: where passwords really leak from, and capping what an automatic job can spend.
This page in the original Kyber Cypher voice: Every automation gets its own key