Why Continuous Integration And Continuous Delivery Are Not The Same
July 8, 2015 | DevOps | Josh Bregman
Jenkins is a powerful platform for continuous integration. As enterprises embrace high velocity deployments enabled by DevOps, the demands on Jenkins have increased. Its common for Jenkins to be used not just for building artifacts, but to actually deploy the artifacts to production as well. Creating artifacts is not a particularly sensitive operation, but deploying artifacts to production requires rigorous security controls. Jenkins is a tool for collaboration and visibility and therefore not well suited to provide the restricted access required for such sensitive credentials and processes.
Many organizations have attempted to solve this problem in the following ways:
Running multiple Jenkins masters – each with their own set of allowed users thus restricting who can push to production.
Adding Jenkins Plugins – there are some very good third party plugins focused on access control and privileges.
Create a dedicated “Jenkins management” team – due to the complexity of the Jenkins eco-system, it’s hard to rotate other team members onto the Jenkins team, which limits communication and collaboration.
Ultimately, the problem with these approaches is that they all fail to address the underlying cause of the problem: Continuous Integration and Deployment are separate concerns. When a single server is trying to perform both of these functions, many tensions arrive which are solvable by re-thinking the proper roles and workflows.
Can you Answer the Following Questions?
Which people, machines and code can deploy which applications to production?
Which people, machines and code have access to the production deployment credentials?
How are the access controls for #1 and #2 implemented and managed?
Knowing the answers to these questions is essential to passing a security audit or assisting in the analysis and remediation of a security incident. The reference architecture described below will allow an organization to answer these questions and be more secure while reducing friction and increasing velocity.
As you can see for the picture above Jenkins does not directly deploy to production, but rather simply adds artifacts to a repository. It has permission to trigger the release server, specifying the name and version of an artifact.
When the deployment server receives the deploy call, it first checks if the caller (person or client service) is authorized to perform the specified deploy operation. The mechanics of this authorization check can vary according to the implementation details of the deployment server:
If the Deployment Server exposes the deploy capability via a set of shell scripts, then the Deployment Server should validate that the SSH Key presented is trusted and map the identity of the caller to a user in the underlying operating system. Along with the OS user come a list of groups and sudoers permissions. OS permissions can be used to enforce who can run the script.
If the Deployment Server exposes the deploy capability via a set of Web Services, then the Deployment Server can be protected by a traffic authorization gatekeeper. The gatekeeper only allows authorized clients to invoke the deploy service.
Once the request is authorized, the deployment server retrieves the artifact and the deployment credentials (from a Secrets Server). Of course, the Secrets Server also performs an authorization check to ensure that the Deployment Server is authorized to access the specified credentials. Finally, using those credentials, the Deployment Server pushes the tested package into the designated environment (test, stage, production, etc).
By taking a step back and looking at the root cause of the problem – using Jenkins for more than just Continuous Integration – and adopting the reference architecture above – enterprises can eliminate this bottleneck in their deployment pipeline, be more secure, and ultimately more responsive to the needs of the business. At Conjur, we help organizations solve this problem through our 5 step “Continuous Secrets Delivery” process. You can see if for yourself by clicking below.