What is a DevOps Secrets Server?
There was a lot of interest in Square’s announcement of the open sourcing of their “system for managing and distributing secrets” called Keywhiz. The discussion on Hacker News revealed at least two other open source projects – Sneaker and CredStash – that also provided similar types of capabilities by leveraging some of the built in capabilities of the AWS platform.
The emergence of these projects and the strong follow-on interest is validation of the need for a new type of technology called a “DevOps Secrets Server”. In case you missed it, Conjur’s CEO Elizabeth Lawler spoke about this in her talk at RSA. This is part of what she called SecDevOps 2.0 – the formalization of proper security controls for DevOps workflows and the applications they deliver.
So what is a DevOps Secrets Server? Here are some key points:
End to end encryption
It almost goes without saying that secrets must be encrypted from end-to-end. For example:
- The secrets should be encrypted when “at rest” in the secrets server
- Each secret should be encrypted with a unique key, which is itself encrypted by a master key (or set of master keys)
- Cryptography should be profesionally audited, and ideally open-sourced.
- Secrets should be encrypted in transit, using e.g. TLS
- SSL verification must be ON! Never SSL_VERIFY_NONE, not even in development systems.
RBAC for people, machines, and code
The access to the secrets in the server needs to be governed by a robust policy model. This model must support all the identities that are common in the world of DevOps – people, servers, VMs, containers, and web services. DevOps, by nature, is dynamic. Machines, containers, and services are constantly being created and destroyed. This is commonly referred to as treating servers like cattle. Once the policies have been defined and agreed upon, both the DevOps and Security teams can be confident that the workflows are following security best practices like least privilege, defense in depth, and separation of duties.
In globally distributed, dynamic systems where machines are automatically provisoned and destroyed by code, simple authentication mechanisms like OAuth or API Keys work better than traditional IT server management approaches such as client certificates and Kerberos. And along with the identity of each instance comes a role, which gives each machine privileges within an authorization policy. This type of automated “host factory” eliminates the need for manual intervention by information security staff in the DevOps workflow.
The model must also support RBAC for human users. For example, DevOps teams are often loaded with contractors; in many regulated environments contractor access to systems needs to be restricted. This type of model is required to support security best practices as separation of duties and least privilege.
The complete lifecycle of the objects managed by the DevOps Secrets Server must be audited. All access to secrets, whether successful or not, must be audited as well. Without proper auditing and other robust controls like strong encryption and the aforementioned RBAC, the server is just serving up name value pairs. It’s good for a shared data layer but it’s not a secrets server.
Fully programmable with fine granularity
Since many of the actors in the workflow are silicon-based (machines), and not carbon-based (people), the DevOps secret server needs to have a comprehensive set of APIs. The APIs must be protected by the same RBAC model and all API calls must be audited as well. GUIs are important for reporting, alerting, and visualizing the policy model but not as a primary means of interacting with the system. Self service and other business as usual activities can be easily built on the APIs.
Highly available across any cloud, hybrid, and global architecture
Security systems are mission critical and need to be highly available and support zero downtime upgrades. This requires an architecture that is purpose built for the types of infrastructures that DevOps applications leverage or plan to leverage in the future. They need to work with any cloud (public/hybrid) and across clouds. The scaling, fault-tolerance, and failure recovery features should use cloud-native primitives like autoscaling and containerized cluster deployment.
This is essential because the DevOps secrets server is going to be run by the same DevOps team as the rest of the infrastructure. The more integrated the secrets server is with the rest of the environment,the more seamless the experience. Increased integration also means that security can flow with, not against, the tide.
Conjur is a company dedicated to addressing the security challenges of DevOps. Secrets management is one such use case that the Conjur platform addresses. The DevOps world is complicated and constantly changing, therefore, various solutions are emerging. Conjur is already deployed by DevOps leaders like Netflix, Rally Software, and Novartis. Conjur is ready today. Are you?
A note on cryptography in Conjur
All secrets managed by Conjur are encrypted and decrypted by the open-source library slosilo. Slosilo has been professionally reviewed by a 3rd party auditor. The crypto audit report is avaliable on request.