How to Evaluate Your DevOps Secrets Management Program
Editor’s Note: Part 5 of a 5-part series providing practical guidance and insights to security leaders for securing DevOps environments. This series is based on insights from Global 1000 CISOs.
Over the past several months, we have mined the real-world experiences of CISOs from Global 1000 organizations to learn how to meet the challenges of securing DevOps. We’ve explored a number of actionable steps security teams should take to align themselves to DevOps culture and methods while addressing the risks of privileged access. We covered:
- Five key steps for bringing DevOps and security teams into alignment
- How to prioritize DevOps tools and cloud infrastructure
- How to establish enterprise requirements for DevOps
- Four things security can do to keep up with DevOps CI/CD
In our final installment of The CISO View blog series, we’ll outline best practices for evaluating the results of a DevOps secrets management program. These recommendations can help security teams showcase early successes, home in on areas that require additional efforts and ensure continuous program refinement.
Like all enterprise-wide cybersecurity programs, rolling out a secrets management program typically happens through a phased approach of incremental improvements tightly aligned with DevOps culture. For example, to start, teams can identify and prioritize securing a finite set of build pipelines, applications, databases or other resources with access to sensitive data, by ensuring that the privileged credentials to access them are managed by the centralized secrets management system. Types of sensitive data include regulated data (covered by GDPR, HIPAA, SOX or PCI), proprietary financial data or intellectual property. This incremental approach can both help security teams achieve “quick wins” to demonstrate security and efficiency gains to key stakeholders, while also establishing a path for a broader roll-out to extend the reach of the program.
Test the Deployment of Your Centralized Secrets Management System
In testing the efficacy of your secrets management system, there are four key areas to evaluate.
- Developer experience. Developers should be able to use their own machines for testing and then smoothly promote the code to the production environment. It’s important to evaluate if developers can easily find and use well-documented code modules for handling secrets. Additionally, can they test their code for secrets retrieval in their preferred environment?
- Failover. If a call to the credential vault does not reach the master system, it should fail over to a second system. Does your program include a robust failover system for the retrieval of secrets?
- Scalability. For an enterprise-wide deployment, the secrets management system may need to scale across many thousands of application instances and multiple data centers. Is your system able to handle the scale your organization requires today – and down the road?
- Break-glass capability. In circumstances such as networking issues that make secrets inaccessible from the centralized system, it’s important to ensure that remote sites aren’t locked out. Do you have a secure mechanism that can provide temporary access to secrets when the centralized site is not reachable?
Measure Improvements in Risk Reduction and DevOps Practices
It’s also important to establish and track metrics to gain visibility on attack surface coverage and the effectiveness of privileged access. Evaluate the following areas to influence those metrics:
- How many secrets have been secured for tools and applications? (It’s important to have a plan for extending this program to secure more secrets over time?)
- How capable are you of detecting if privileged credentials are being misused?
- According to penetration tests or Red Team exercises, are your secrets becoming increasingly secure against attackers?
Some organizations provide scorecards for each DevOps team to measure how well their practices comply with security requirements. These scorecards could measure:
- If any of your secrets can be found in code repositories on the Internet?
- What percentage of code commits contain a secret, such as a cloud access key?
- How many secrets are there per line of code?
- How often is the secrets vault being used? (If you’re not tracking many transactions after secrets are vaulted, there’s a chance that the DevOps team may be bypassing the process.)
- Are unique credentials being used where “uniqueness” is mandated?
In addition to measuring these improvements, it’s important to help internal and external auditors understand the security controls in place for protecting your organization’s DevOps and cloud environments. They may not be familiar with the newer tools, techniques and automated methods that the security team is implementing. To ensure their compliance assessments are accurate, auditors need to be kept up-to-date.
One of the report contributors said it best: “DevOps is a game changer. People are starting to recognize that you can’t do it without security.”
For organizations embarking on digital transformation initiatives, it has never been more important to align security and risk postures across new tools and technologies. We hope this CISO View series has inspired you to examine your own programs and commit to driving deeper developer collaboration, more effectively assessing risk and prioritizing steps to protect DevOps processes while maintaining developer velocity.