Devops Principles

In this section, we will take a look at Devops, which is gaining wide popularity.

Devops is more of a movement or a culture, wherein the development team and operations work together without the corporate barriers. Managing the development team and operations separately in an organization hierarchy often gives rise to Developer vs Operations tussle. Though there is no single definition for Devops, there are a lot of processes and concepts which are usually associated with the Devops movement. Let us understand why do we need Devops and what are the challenges we face on a day to day basis?

Challenges faced:

1. Code shipped by development team, often would work in one environment, but not on others.
2. Development team would lack knowledge of deployment environments
3. Disparity of deployment environments could lead to a lot of issues. For instance, a database server might have different versions in different environments and this might cause issues during deployments, leading to delayed and painful releases.
4. Operations have little insight into development processes and internals; hence they only see their version of the story. As a result of separation of roles, we run into issues in deployment, blame games and quality issues and it would slow down the frequency of releases. These issues greatly impact business revenues in one way or the other.

How would Devops solve this problem?

The core solution lies in focusing on the end goal and our ultimate goal is that, we want to deliver the software to production with good quality. The barriers which come in way to achieve this goal should be torn down.

Devops brings together development and operations to work as one team. It brings about a change in mindset, allowing people to collaborate more, automate the processes and bring in parity in deployment environments. This leads to releases being stable and having good quality.

Processes associated with Devops:

There are few process aspects associated with the Devops movement to make it effective. They are listed down below:

1. Agile methodologies
2. Test driven development
3. Deployment processes
4. Branching and Release management Processes
5. Continuous Improvement
6. Automation using tools

1. Agile methodologies:

Use of Agile methodologies (there are lot of variations of Agile, which will not be covered in this book), which is a big deviation from the traditional water fall model of development, is a first definitive step. However, we will not go deep in this topic.

2. Test driven development:

Test driven development, is a very effective process in making sure that we develop quality code right from the beginning. Writing Unit test cases for code which we develop ensures that developers deliver quality code and also a sense of responsibility and ownership of quality. Quality is a collective responsibility of the entire team, and these processes will ensure that. There are a lot of unit test frameworks available today (like Junit, NUnit, Karma etc) and they are available for almost every programming language which we use today.

3. Deployment processes:

Deployment processes should be well defined and the team should adhere to that. Once the code is committed to a source control, code review, running of unit tests, security tests etc. should be performed on the code to ensure quality. Also, there should be gates which prevent bad quality code (which do not pass the adequate standards) from getting deployed. It should be the responsibility of the respective developer to fix bad code. Similarly, proper release notes should be published for every feature, to ensure deployment is smooth and hassle free.

Also, to ensure real Devops culture, each developer can also wear the hat of operations every now and then, for instance in every sprint (if Agile Scrum is followed) to understand the deployment processes. This way, every developer would take turns to deploy and would be cognizant of the environments, challenges etc. Understanding these would help them provide feedback and incorporate changes in the processes to make it better.

4. Branching and Release management Processes:

Branching strategy should be well defined and there are lot of distributed version control management systems (like Git), which allow geographically distributed development teams to collaborate and work. Code should be checked in very frequently and merged frequently (even many times a day). The Release management process should be well defined to allow frequent small releases (instead of one big release).

5. Continuous Improvement:

No process is fool proof and no process is perfect from day one. Also, a process which will work for one project, need not necessarily work well for the other. So, we need to have a continuous feedback loop, which should be immediately incorporated in the process life cycle and provide course correction.

6. Automation:

Automation of manual steps would lead to faster execution of the tasks and tasks being less error prone. It makes the end result more predictable. The commonly automated execution steps are:

• Build
• Code Reviews
• Unit Tests
• Integration Tests
• User Interface Tests
• Acceptance Tests
• Deployment

The above steps are collectively known as ‘Continuous Integration’ as illustrated in the above diagram.

There are various tools available for automating these steps and we can orchestrate these steps together into a pipeline, so that once the code is committed, all these steps are automatically run.

Deployment Automation:

Automation of deployment processes help in eliminating manual intervention and standardizes the process. Use of continuous integration and deployment processes help in having standard processes.

Infrastructure Provisioning:

'Infrastructure as a code' is a concept is which is gaining wide spread acceptance. Traditionally, Infrastructure setup is done manually and the steps are generally in the minds of few people who perform the deployment. Also, setting up new environments will be a Hercules task with a lot of legacy configurations, setup tasks and scripts. Often it consumes significant time and results in money drain.

Why is Infrastructure Provisioning important in Cloud?

With the advent of cloud, infrastructure is seen as a piece which needs to be setup and tore down quickly and on demand. Let us say, in a cloud environment, there are a huge number of services deployed, and we would need to scale out or scale down on the fly. We cannot manually configure each environment, since we will lose the ability to scale out quickly. Also, if there is a small change in environment configuration, or if we need to apply a small patch to software, manually applying that change would take a huge amount of time.

So we need a way to automate infrastructure provisioning. Infrastructure setup should be automated via programs or scripts and the entire datacenter could be setup this way. Any change to the environment, should be done through these scripts, which also should be version controlled. This way, the environment setup doesn’t depend on people, but rather would depend on automated processes.

This would help in cloud environments, where new environments (or virtual machines) should be provisioned on the fly. All we need to do is to run the scripts on new machines, and we get a new environment which will be an exact replica of existing environments. This will also eliminate human errors while setting up new environments.

Reference Implementations:

• Source Control revision management tools – Git, BitBucket
• Build Tools – Maven, Gradle
• Code Reviews Tools– SonarCube
• Unit Tests Tools – Junit, NUnit, Karma, Jasmine
• Integration Tests Tools – Citrus, Cucumber
• User Interface Tests Tools – Selenium
• Infrastructure deployment tools – Ansible, Chef, Puppet