Ozlo

Case Study

Industry

Ozlo provides chat app

Challenge

Create a cost-effective solution to increase the throughput of the builds

The Client

Ozlo provides an chat app which can be described as "The friendly assistant who helps you get things done — find restaurants, discover recipes, order delivery, buy movie tickets and more!" They collect data across various sources such as Facebook, Youtube, Yelp etc. then helps to answers questions based on geo-location and user preferences.

The Challenge

Ozlo had a single huge test suite that validates the integrity of the all application components. Each pull request triggers build. As the team size continue to grow, builds were backlogged affecting developer productivity. The ask was to create a cost-effective solution to increase the throughput of the builds.

The Solution

The trivial solution to add more slave nodes would turn out to be expensive since they need high resource machines to satisfy the build requirements. We proposed to build a AWS ECS cluster which can provide containers that can act as Jenkins slave. We moved the share data on to AWS EFS and mounted the volume to the slave container, which saves time rather than waiting for files to download to complete.

All of this was completed using best practices policies and procedures and in partnership with nClouds partners and advisors. It was also important to ensure that Ozlo's team were in the loop and understood the process and timeframes.

The Technology & Methodology

We created an AWS EFS filesystem and referred it from the job which creates shared files, so that the AWS EFS is populated with all versions of shared files. We created a AWS CloudFormation template (using Troposphere) creates an AWS ECS cluster, Docker registry and 2 Auto Scaling groups 1 with a single reserved instance and 3 spot instances. We reserve 95% for CPU for each build job. We have defined scaling policies based on CPUReservation on AWS ECS cluster and scale up and down the AWS Spot Instance cluster. This enables slavepool to automatically scale up to 4x (configurable) based on demand for running multiple jobs.

Technology used included

  • Troposphere - To create AWS CloudFormation template using Python
  • AWS CloudFormation - To version control and automate infrastructure changes
  • AWS EFS - To store huge data chunks required for build
  • AWS ECS - To provide a pool of Jenkins slaves on demand
  • Auto Scaling Group - To enable cost saving by scaling down unused instances and scaling up to provide performance on demand

The Result

The wait time for build results has gone down over 3 times in case of multiple parallel builds. Of course, the max instances of autoscaling group can be adjusted to even more throughput. All this is done while keeping the costs low by allocating more machines only when there is demand.

Related Blog Posts

Contact Us Now

You can also email us directly at sales@nclouds.com for your inquiries or use the form below