Devtron: Empowering developers

With our migration to Kubernetes from EC2 machines, we are now successfully using Devtron to run our entire application stack and additional services, in this blog post we take a look at our migration journey of deployment from bespoke bash scripts to using GitOps

Back in 2019 we had one single monolith, the deployment process used to be:

1) Merge feature branch into integration branches, create PR that gets merged in master branch

2) Create new tag manually and set tag in version to be deployed on our deployment server

3) Manually run code_deploy.sh which is our main deployment script, this script contained code which fetched the tag and sshed into servers added to our production ASG and individually pull the code and restart gunicorn servers and took care of installing dependencies if any

4) env file has to be updated manually and copied onto each server before any deployment

5) Reverting a release was easy as we could simply change the tag version but we had to redo the entire process

Issues with older deployment system

Our monolith being based on Django and MySQL needed deployment as a whole, we faced various issues such as

Outdated Software Versions:

We are using CentOS and Ubuntu both in our ASG, Older EC2 instances were running outdated operating systems or software versions that lack necessary features

Incompatibility with Bash Scripts:

Bash scripts written for newer versions might not be compatible with older Bash versions, we have since updated our scripts to handle multiple operating systems

Limited Resources:

We could only deploy on ASG, nodes got added based on resource consumption, in Kubernetes this issue gets solved with Pods

Security Vulnerabilities:

There is no scanning of installed packages and libraries, we had to rely on fixed/pinned versions of python and js libraries, this caused lots of security issues

Dependency Issues:

Updating any python package was troublesome as we had to maintain multiple versions and resolve dependency conflicts

Networking Problems:

Deployment used to cause ssh-key mismatch errors as new nodes got deleted or added from the Auto Scaling Groups, using no-ssh hosts check was our only option

What is Devtron

We needed a simple way for developers to deploy any type of project which has a valid Dockerfile to deploy their project and manage resources easily without approaching the Devops team again and again, initially we thought to use teraform and helm to deploy new changes, but managing teraform and helm for hundreds to services with complex interdependencies proved to be a challenge for the devops team and this did not helped the dev team as they had to learn the internals of Kubernetes, also deploying changes with helm and manifest files can be error prone, we started looking at providing a simple GUI where devs can deploy and monitor services, thus we found Devtron to be helpful in this regard as it was also used by some of the other product based companies.

Devtron is a platform that binds continuous integration (CI), continuous deployment (CD), GitOps, security, observability, debugging and a lot more under a single user friendly UI

Key features of Devtron:

• Zero code software delivery workflow for Kubernetes
• Multi cloud deployment
• Easy DevSecOps integration
• Application debugging dashboard
• Enterprise-Grade security and compliances
• GitOps aware
• Operational insights

Developers can create a new application easily using Devtron Deployment Templates, with its latest update Devtron has even started deploying applications and cron jobs without need of any Dockerfile

Creating New Apps and Environments

There are 3 major pre-requisites before you can actually start using Devtron to create your pipelines, Here we take a look how easy is to deploy a new service and create new environment for testing or development:

• Adding Host URL
• GitOps Configuration
• Adding Container Registry

Adding Host URL: This is pretty straightforward. You just have to put your domain name, which in this case would be “http://devtron.searceinc.org”. You can also use a public IP address here if you didn’t point a domain name.

GitOps Configuration: Devtron uses GitOps configuration to store kubernetes configuration files of applications. You need to add your GitHub organisation name, username and Personal Access Token to set it up.

Adding Container Registry: This container registry would be used to store the images that we build using devtron. There are a plenty of options to choose from — Docker, Azure, Quay, GCR. In order to demonstrate the hassle-free integration with DockerHub, we are implementing the same for this blog.

Once everything is setup, you can create your own app easily.

Creating an application in Devtron:
An application in Devtron is nothing but a cumulation of your repository configuration, deployment template, pipelines etc. For this blog we are going to deploy a simple nginx web app that displays basic information.

Step 1:

Click on create “Create New>>Custom App”. Enter the App Name. (This field is devtron specific and not related to your deployment). Select the project “devtron-demo”, select blank app and click on “Create App”

Step 2:

Add the repo URL and put “/” for the checkout path. You might want to check “pull submodules recursively” if your repo has submodules. Save the configuration.

Step 3:

Select the container registry that you had integrated with Devtron earlier. For the container repository field, follow the format “username/repo-name”. Change the path for the Dockerfile based on it’s location in your repo. Save the configuration.

Step 4:

The Chart Type should be rollout deployment and the chart version should be latest. You might want to change the requests and limits for your resources depending on your application requirements or go with the default configuration provided. Save the configuration.

Step 5:

In the Workflow Editor click on “New Build Pipeline>>Continuous Integration”. Then Click on Advanced Options. Here you can give your pipeline a desired name. Enter the branch name from which you want your application to be built and change the “Trigger Build Pipeline” to “Manually” if required. You can ignore the Docker Build Arguments for now, we will cover it in a future blog. Click on create pipeline.

Step 6:

Now click on the small ‘+’ icon that you see next to your build pipeline. We are creating the deployment. Then Click on Advanced Options. Here you can give your pipeline a desired name. Select the environment and the namespace will get populated automatically. Set the deployment to Automatic. Click on Create Pipeline.

Step 7:

You will now see a workflow like this one.

Click on Build and Deploy in the top bar. Then click on select material in the build pipeline. Select the commit that you want to build an image from. Check ignore cache at the bottom and click on start build.

You can then click on the Details option in the build pipeline to see your build logs. Once the image is built successfully, you will see a screen like this and the image would have been pushed to Dockerhub.

Step 8:

If you recollect the deployment pipeline was set to Automatic and would have been triggered. Go to App Details section and you should be able to see your components being deployed and progressing. Once the deployment is successful, you should see a screen like this, signifying that your deployment is in healthy state.

You can go to the individual components, like the Pod, ReplicaSet, Service etc to check the Events or the Manifest in case the deployment fails for any of them.

Step 9:

Let’s check our application. We haven’t exposed it using an Ingress as of now so we will have to go inside the container and verify it’s working. Click on Pod, in the left pane and then hover over your pod name to click on terminal. Switch the session to bash and install the curl command using “apt-get install curl”. Now execute “curl localhost” to verify that your application is working as expected.

Hurray! We are getting the expected response, signifying that all components were deployed properly and are running as desired!

Step 10:

Let us clean-up now. We can now delete this deployment in one go just by deleting the deployment pipeline. The respective components would automatically get deleted from GKE.

Summary:

We have successfully built an image from our git repository, pushed the image to DockerHub, automated the deployment to EKS and we didn’t even touch the CLI once while doing the same and without any Devops help! Kubernetes is not just limited to simple deployments. Kubernetes has other components like config maps, secrets, build arguments, load balancing, ingress setup and a lot more to explore in terms of deployment strategies and Devtron has helped in simplification of these, using Devtron has reduced our deployment time from minutes to seconds