Microsoft’s acquisition of GitHub for $7.5 billion has raised questions about the future of GitHub’s privacy and security within the developer community. Some developers are already moving to other code version control platforms like GitLab. As Microsoft works to assuage these concerns, it’s a good time to review GitHub security concerns beyond Microsoft’s responsibility. Past breaches have proven that a private GitHub repository is not the most secure place to keep sensitive and confidential information like secrets, i.e., passwords, API keys, SSH keys and certificates. These credentials equate to access to databases, cloud instance and privileged system control consoles that contain the crown jewels of an enterprise.
The Uber breach reported in November of 2017 is a high-profile example of secrets management gone wrong. Hackers stole the personal data of 57 million customers and drivers from Uber Technologies Inc. They were able to breach Uber’s private GitHub repository and find code with imbedded secrets. With these secrets, the attackers were able to more laterally from Uber’s development environment to their production Amazon Web Services (AWS) account. This is the same AWS account that the company used to process live company data, so it contained a lot of sensitive client information.
Uber likely has sophisticated DevOps, yet the breach reported in 2017 was not the first time Uber was hacked via secrets in GitHub. In 2014, attackers leveraged mishandled secrets in GitHub to steal the personal data of 50,000 people. In the aftermath of the 2014 breach, the ridesharing company sued GitHub for information about any users who accessed the repo page with the exposed secrets.
Uber is not alone. In the fall of 2017, Kevin Finisterre found private keys for DJIs – a Chinese drone manufacturer — web domains and AWS s3 keys publicly available in GitHub during a bug bounty program. Using this data, an attacker could access drone flight log data and any images users uploaded to DJI, including photos of government IDs, driver’s licenses and passports. Some of the exposed flight log data was associated with government and military domains. Ironically, the self-proclaimed “most lethal bounty hunter in the galaxy” found the keys to the kingdom in a repo called “skypixel_lottery.” Finisterre said that DJI also left some of their AWS s3 buckets open to the public, a common security mistake. This is one way to let an attacker in and weak secrets management allows them to use privileged escalation and credential theft to advance the breach.
Many of the publicly announced breaches are at companies that should know a thing or two about DevOps security. So what’s going wrong? The shift from monolithic applications, static servers and long release cycles to the dynamic, fast-paced world of DevOps has drastically changed how software is developed and deployed. However, in the race to go to market first, many organizations have not properly evaluated how they think about DevOps security.
DevOps developers now have more control over security than ever before. In many organizations, developers can check code into repositories with little to no oversight from security teams. As we saw, this code could contain access keys to production systems, secrets embedded in a file, for example. Sometimes it’s as simple as accidentally checking your own GitHub key into the source code management system. Sometimes developers falsely believe deleting the offending code solves the problem, but the leaked secrets could remain in the repo history. Truffle Hog is an open source tool that searches git repositories for secrets by digging deep into commit histories and branches.
No one should ever put any secrets into GitHub or any code repository, it doesn’t matter if it is a public, private repository or if the secrets are encrypted. Doing so increases your attack surface as anyone who can view the code will have direct access to system or infrastructure credentials that they might not be authorized to have.
According to the Twelve-factor App methodology, this is not only insecure, it is also bad application design because it introduces code dependencies on access tokens that are subject to change. This means an organization’s developers will need to change their code and rebuild their applications every time a secret needs to be rotated. Securing your secrets with a centrally managed secrets provider such as CyberArk Conjur Open Source is a step in the right direction because it manages credential rotation and security for you. However, you can remove API calls to your secrets provider with Summon, an open source command-line tool maintained by CyberArk. This is holistic secrets management solution lets developers get out of the secrets management game while allowing security teams to provide security as a service.
If you are a developer or already have a secrets management solution you might want to check out open source Summon first. Summon allows you to replace any secret or API call to a secrets vault in your code with a simple environment variable. Summon will securely inject secrets into the process at runtime without exposing or leaking the credentials. This is not only more secure, but it also abstracts the application code from the secrets management tool, making it easy to swap out secrets providers.
It’s easy to get started. Check out the CyberArk Conjur Open Source hosted tutorial for a tour of core secrets management concepts, such as storing and fetching secrets, machine authentication and authorization and security policy as code.
If you get stuck or have any feedback on our open source tools, I would love to hear from you on Slack or reach out to me on Twitter @WalshSec.
