Secrets Management: Sharks, DevOps, and Dark Alleys
December 4, 2014 | DevOps | Kevin O'Brien
If secrets management were (painfully) made into a weekly CW drama, it would be something like a horror film trope: we’d watch our vulnerable protagonists slip from dark servers to ominously lit git repositories, tensely waiting for someone to slip from the shadows and steal them away.
In the admittedly more prosaic real world, this plot unfolds regularly.
When it goes awry — when the mechanisms of access to important systems and data are exposed — it tends to be the media that cries foul, and user data that is outlined on the metaphorical asphalt as systems are breached and careers fall apart. At this point, after so many high-profile breaches over the past 2 years, the story is nearly a trope unto itself.
Why? Is this (mis)management due to a lack of product solutions, good intentions executed not as well, or just a nascent complacency vis a vis, “it won’t happen to me”?
The fact is, security and convenience are often at odds. For modern infrastructure — the kind built by DevOps teams designing around principles of scalability, automation, and elasticity — secrets represent the “how” of access management. API keys, credentials, and certificates are among the more common types of data that broker both user-to-service and service-to-service connectivity, and making them available to code and people is a requirement that often gets balanced poorly against keeping them highly secure.
Make access mechanisms hard to use, and there’s an immediate negative: things break. Distribute secrets widely and there’s a theoretical negative: something bad will happen in the future.
This trade-off plays to well-known blind spot, and “do-ers” — like DevOps people — can be particularly susceptible to it given the sheer volume of work and urgency that they operate under. The weakness is inherent to human nature itself; risk assessment is not something we’re good at. 10 years ago, in an oft-quoted interview, Bruce Schneier quipped, “More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating [it].”
As foundational as the storage, distribution, and auditability of secrets is for protecting important data, many organizations have their secrets moving through channels that are less-than-secure on a daily basis: emails, text messages, chats, Github repos, shared VM images, Docker images, etc..
Problematically, none of these were designed to deal with this kind of data, and there is often no audit trail available for how the information moved into or out of them. This isn’t simply a “good housekeeping” problem — it’s an essential weakness in architecting defensible infrastructure.
Balancing technical elegance and simplicity (which are of obvious importance to DevOps) with robust, secure, well-designed secrets management is not a trivial problem. It’s also not one that most teams are primarily concerned with — unless you’re a security company, keeping API keys safe and audited is likely not why you go to work every day. Conjur is a purpose built permissions and secrets management platform, designed to make both easy.