Software development has evolved a lot in the last few years and it will continue to do so with inculcation of cloud based technologies rising to an all time high. While this is happening, clients often ask what would it take to spin up new environments, how much time and money will this cost? Well, this is where DevOps has come into picture. Clients are looking for their infrastructure as source code which they can take and spin up as new environments quickly and with ease. In Paxcel, we have setup a dedicated team which is looking after this domain and are using various tools in domains like Continuous Integration, Continuous Delivery, Log Management and Repository Management.
I will bring up a case scenario wherein I will first define the problem and then the solution we designed for our client, Chaikin Analytics, using various tools like Vagrant, Puppet, Docker, Sonatype Nexus, Jenkins, Maven, GIT and Ruby Migrations.
Chaikin wanted to transform their whole infrastructure into code which could then be used to quickly spin up new environments with latest builds. In a nutshell, they wanted to automate their build deployment process.
Achieving Build Automation with Jenkins CI and Nexus
Our solution was two-fold. First, in order to cater to these requirements we opted to use Jenkins and Sonatype Nexus.
In this environment, developers could write code and commit to GitHub. Post-commit hook at the Github repo then trigger Jenkins to create build and push to Sonatype Nexus with help of a maven plugin. From here on, the actual VM where this build has to be deployed, uses a yum plugin, which makes the process much easier. We have configured this plug-in in Nexus and the web servers then fetch the build by running command like ‘yum ensure latest’ through Puppet module. We found that this works flawlessly: developers commit code and build is deployed in no time…cool !
Nexus not only helped us in achieving the automation, but also helped in hosting all third party JAR files, so as to give more control to admins on which version of jars developers should be using. Along with this, placeholders feature for previous deployments were a big plus in case a build required a roll back.
Infrastructure as Source Code
With first target of build automation achieved, our next effort was to port whole infrastructure as source code. For this we used tools like Vagrant, Puppet, Docker, Ruby Migrations, ElasticSearch’s Logstash.
Lets dig in more detail about how we used them.
We first opted for creating linux containers due to advantages they offer over a Virtual Machine. For creating those containers we opted for Vagrant, which helps us define the configurations of docker containers in single file. Once this is done, a user can take this file and run command ‘vagrant up’ which creates containers in no time. Next, we configured those containers with necessary RPMs and other settings; for this we opted for Puppet. We wrote Puppet module for each configuration which needs to be applied to a container. Vagrant has built-in support for Puppet, which we found to work flawlessly. The ‘vagrant up’ command now, apart from creating containers, also puppet provisioned the containers with necessary RPMs.
Until this point, we solved most of the problem areas. At this stage, the user needs to take a clone of repo and fire the command ‘vagrant up’ which, in turn, creates the necessary containers. The containers are then provisioned by Puppet where one of these modules would communicate with Sonatype Nexus for ensuring latest version of rpm is installed on machine.
One last thing we were left with were database changes. For eexample, if we had changes in code, which required schema change as well, we need to determine the best way to automate that. For this we opted for Ruby Migrations.
Using Ruby Migrations, the developer can write migration scripts for whatever change they want to make in database and then push it to the repo along with the code. The Puppet module will take care of applying those migrations into your database, so it works same way as it worked for builds.
With this done, all problem areas were addressed. All these wonderful tools have really helped us to represent true case of infrastructure as code and this workflow will only improve further with lot more promised on roadmaps of these tools.
Latest posts by Gaurav Vashishth (see all)
- Boxupp Talks “Build Automation” and “Infrastructure as Source Code” - December 12, 2014