Every automation gets its own key

Field Log // 016 Status Live Difficulty Free Cost A few minutes each

The Story

There is a moment, once you have a few machines talking to each other, when you generate one key, copy it everywhere, and everything works. It is a genuinely great feeling. It is also the single decision most likely to turn a small future mistake into a large one.

The appeal is obvious. One key means one thing to back up, one thing to remember, one thing to fix when something cannot connect. Every new machine joins the club instantly. Nothing about it feels careless, and that is exactly why it is worth thinking about before you have twelve machines holding the same secret.

The problem only shows up on your worst day. A shared key means every automation holding it can do everything any of them can do. So the small script you wrote in a hurry, the one that just moves some files around, is carrying the same authority as the thing that manages your backups. It does not need that authority. It has it because copying was easier than generating.

Ask the blast radius question. If this exact credential leaked today, what could someone do with it? With one shared key the honest answer is everything, everywhere. With one key per job it is one job on one machine, and that is a difference you can sleep on.

A leak is not the only way this bites, and honestly it is not the most likely one. Rotation is. The day you need to replace a shared key, you have to replace it everywhere simultaneously, and anything you miss breaks without warning. That job is unpleasant enough that people put it off, which means the shared key stays in place long past the point where anyone remembers every copy of it.

Per-job keys turn that from an event into a chore. Rotating one is a two-minute task affecting one thing, so you actually do it. The credential that gets rotated regularly is worth more than the one that is theoretically stronger and has not been touched in two years.

The physical version everyone understands: a building master key opens every door, so the cleaner, the contractor and the courier all carry the same one. Nobody thinks that is fine in a building. It is somehow fine on our own machines, right up until it is not.

When I added a new automation recently, I gave it its own dedicated key rather than handing it the fleet one. The whole detour was a couple of minutes: generate, install, restrict it to the one account on the one host it needs. Now if that automation is ever compromised, the honest answer to the blast radius question is one account on one machine, and I can rotate it without touching anything else.

Capability is risk. Hand out the smallest amount of it that lets the job work, so one mistake can never become all of them.

The Build

One key per automation, scoped to the minimum. Shown with SSH keys because that is where most home setups share one, but the principle is identical for API tokens and service accounts.

1. Inventory what is sharing a credential today

Find every place the same public key is authorised. If one key appears everywhere, that is your blast radius, written down.

# which keys can log into this machine, and how many are there?
cat ~/.ssh/authorized_keys | awk '{print $NF}' | sort | uniq -c

2. Generate a dedicated key per job

Name it after the job, not the machine, so you can tell later what it is for. No passphrase for unattended automation, which is precisely why it must be narrowly scoped.

ssh-keygen -t ed25519 -f ~/.ssh/id_backup_job -C "backup-job" -N ""
# one key, one job, obvious name

3. Restrict what the key is allowed to do

An authorised key can carry restrictions. Pin it to the source address, turn off port forwarding and terminal allocation, and where the job is a single command, force exactly that command.

# in authorized_keys on the TARGET machine, one line:
restrict,from="10.0.0.5",command="/usr/local/bin/backup-receive" ssh-ed25519 AAAA... backup-job

4. Give it its own account where it makes sense

A dedicated unprivileged user per automation caps things further. It can only touch what that user owns, and its home directory keeps its own state separate.

sudo useradd -m -s /usr/sbin/nologin backupjob
# the job's blast radius is now one unprivileged user

5. Lock down the file permissions

A private key readable by others is not private. Check this on every machine, not just the one you set up carefully.

chmod 600 ~/.ssh/id_backup_job
find ~/.ssh -type f ! -perm 600 -ls     # anything listed is wrong

6. Point the job at its own key explicitly

Do not let it pick up whatever the default is. Name the key in the config or on the command line so the association is deliberate and visible.

# ~/.ssh/config
Host backup-target
  HostName 10.0.0.9
  User backupjob
  IdentityFile ~/.ssh/id_backup_job
  IdentitiesOnly yes        # do not offer any other key

7. Prove the restriction actually holds

Try to do something the key should not be able to do. If it works, your restriction is decorative. This takes thirty seconds and is the only reason to believe any of the above.

# with a forced-command key, this must NOT give you a shell
ssh -i ~/.ssh/id_backup_job [email protected] "whoami"

8. Rotate one key at a time

Generate the replacement, add it alongside the old one, switch the job over, confirm it still works, then remove the old line. Because it is scoped to one job, nothing else can break.

# add new, verify, then remove the old line. no fleet-wide flag day.
ssh-keygen -t ed25519 -f ~/.ssh/id_backup_job_new -N ""

9. Apply the same rule to API tokens

Everything here transfers. One token per integration, scoped to the minimum permissions, named for its job, rotated individually. A read-only token for a job that only reads is the same idea as a forced-command SSH key.