Key Management in the Cloud – Part 2
| DevOps |
Key Pair Management: Cloud Security
This is the second and final installment of our Key Management in the Cloud blog posts. You can find part one here.
In the first part of this series, we described three concepts of key management:
- A key pair is composed of a public and a private key.
- Public keys are not secrets.
- Selective distribution of public keys conflates authentication and authorization.
Today, let’s examine the flaws of “public key pushing”, and describe a better alternative.
Strike #1: “Public key pushing” allows escalation of privilege.
Once Bob gains root access to a host, he can see all the public keys on that machine. Normally this is not a problem, since public keys are public and not secrets; but here, what happens if Bob finds Alice’s public key on the host (or finds it elsewhere), and starts copying it to other hosts? Bob is using his knowledge of Alice’s public key to escalate her privilege level in the infrastructure. In effect, Bob can grant Alice access to any system that he himself has access to. The “public key pushing” technique is supposed to facilitate centralized management of SSH access, but in fact, the access is managed on each host itself in the
authorized_keys file. The host access list can be changed by anyone with root-level access to the host, and the intended authorization policy cannot be guaranteed to be correct. Even if the management server is constantly re-pushing the public key lists out to every host, the key lists are only eventually consistent; as the server keeps wiping Alice’s key out of
authorized_keys, Bob can keep putting it back, giving Alice plenty of opportunities to login.
Strike 2: Removing a user’s public key removes their ability to be identified.
It’s important to be able to audit the succeeded and failed login attempts to all your hosts. What happens when Alice tries to SSH to a host which hasn’t been “pushed” her public key? The host is unable to verify Alice’s identity. In effect, it looks like a non-Alice person has tried to authenticate as Alice, but that is not what happened at all! Alice is still Alice, but she doesn’t have access. These are two very different things. A failed authentication attempt means “someone you don’t know tried to claim to be someone you know”. A failed authorization attempt means “someone you know tried to access a system she doesn’t have access to”. From a security standpoint, these are quite different; but a “public key pushing” system can’t tell the difference.
Strike 3: Key-pushing scripts are fragile and easy to break.
Key-pushing scripts are typically stored in a configuration management system, along with scripts that serve many other purposes. Ideally, all these scripts function independently and reliably, but in practice, it’s easy to make an “unrelated” change to script A which breaks script B. During the time it takes to get everything working again, the (already weak) capabilities of the key-pushing system are seriously compromised.
At this point it should be clear that public keys are not secrets and shouldn’t be treated as such. So what is the right way to manage access to hosts?
SSH does have a proper mechanism for authorization: once authenticated, a user can be checked against an access list (e.g. via LDAP search). If the user isn’t authorized, they can’t login. The system doesn’t deny their identity, but it does deny their access. The proper outcome is effected (Alice can’t login), and the proper event is recorded (Alice was authenticated, but not authorized).
Now, let’s turn our attention now to the private key. Just as public keys should not be treated as secrets, there is an argument that private keys shouldn’t either. It’s not uncommon to see private keys loaded into secrets storage (version control, configuration management, etc), or even shared by email. What’s the problem with this? In effect, private keys are intended to be so secret that they are not shared with anyone. Sharing a private key is akin to sharing a password; it’s a security and compliance no-no. When you choose a password for a web application, the web app doesn’t actually store your password; it salts and hashes your password in a non-recoverable way, which is sufficient to verify your identity in the future, but it’s not possible to recover the password itself from its salted hash. Similarly, you should never provide a private key to any system or any other person. The public key is specifically designed to be safely sharable. If you need to authenticate yourself to another person or system, they don’t need to know your private key for that; just your public key.