RightScale Blog

Cloud Management Blog
RightScale 2016 State of the Cloud Report
Cloud Management Blog

Migrating to Docker: Getting the Flywheel Turning

Migrating to Docker: Getting the Flywheel Turning

The RightScale engineering team is moving the entire RightScale platform comprising 52 services and 1,028 cloud instances to Docker. This blog is the second in a series chronicling “Project Sherpa” and why we decided to migrate to Docker.

Deciding to migrate the entire RightScale platform to Docker containers was a big step, but we knew that the challenge would come in actually doing it. We wanted to schedule Project Sherpa as a short burst, but to avoid wasted time and effort, we needed to have a plan to efficiently bootstrap the entire engineering team in using Docker.

In the last 18 months, we had gone from experimenting with Docker, to using it in development, to releasing 7 greenfield production services in containers. Along the way we have figured out how to deal with some of the challenges like managing images, inputs, monitoring, and security. We also have taken the lessons learned in migrating the first services to Docker and developed our “2.0” design patterns.

We decided to take an agile approach with Project Sherpa, so we allocated two weeks to prepare before we actually kicked off the process of containerizing. We knew that there was a risk that the process might become — understatement alert — a bit chaotic, so we wanted to organize the chaos as much as possible.

Organizing the Teams

We started by creating a team of Sherpas. These are our Docker experts who can help others, along with our architects and scrum masters who lead the development teams. We had several planning and kickoff meetings with the Sherpas and set up a regular weekly Sherpa standup meeting. We also set up a Slack channel for the project.

Although our normal Dev teams will remain in place, we assigned an Ops person to advise each Dev team during the containerization process. A few of our lead Sherpas from Dev and Ops were designated as floating resources to assist teams as needed, pair with developers, and corral design decisions to ensure consistency. We also designated several developers to continue work on features required by customers as well as any hotfixes that might be needed while Project Sherpa was underway.

RightScale Docker Bible

A couple of our lead Sherpas who have been involved in the early Docker efforts created our version of a “Docker Bible.” The most important aspect of this documentation is that it provides “how to’s” that are specific to RightScale, codifies design patterns, and provides specific RightScale do’s and don’ts. This will be a living document that we add to as we go.    

Here is the Table of Contents for our Docker Bible:

  • Set up your laptop for Docker-based development
  • Package your app as a Docker image
    • Prepare to Dockerize your app (The ruby edition!)
    • Create an entrypoint script for your app
    • Determine inputs for your Docker image
      • Do’s and Don’ts of inputs
      • Where do inputs come from?
      • What should or should not be an input?
      • How do I name inputs?
    • Author a Dockerfile/.dockerignore
  • Debug your Docker app
  • Deliver your Docker app to Ops
    • Design Considerations
    • Containers are Processes
    • TL;DR - The short version
    • The Detailed Breakdown
      • Responsibilities of Hosts & containers
        • Responsibilities of the Host
        • Responsibilities of the Container
      • Things that Ops provides
      • Things that developers provide
    • The Checklist Ops of Ops-y Things to verify before calling things "done"
    • So... remind me again how does it all work together?
  • Continuously integrate and deploy your Docker app
    • How do I enable CI/CD for my application?
      • TL;DR
    • Detailed explanation
      • Use Travis boxes with sudo
      • Setting environment variables in the UI
      • Download Docker client binary
      • Add Git commit information to Dockerfile
        • Must have: image label with HEAD commit ref
        • Nice to have: commit log baked into image
      • Create Docker build script
        • Build Function
        • CI function
    • Parallelizing the build

 

Decisions, Decisions

During our prep period, we also made choices about how we would handle some common situations, such as how to run cron jobs inside containers (we built an app for that) or how to name, structure, and provide inputs all the way from developer laptops to staging and production. We know that as we dive into the process, more common situations will come up and we will surface and share those so that we can ensure a consistent approach.

We also discussed the issues of tech debt and feature creep. It would be easy to throw a bunch of other “nice to have” projects into the mix “since we’re in there...”, but we knew here be dragons, so we drew a line in the sand that we would focus on containerization as our top priority. However, we did identify a few areas of tech debt that would need to be addressed to move to Docker and assigned those projects out.

Initial Training

We challenged all of our developers to be ready to hit the ground running once we reached Day 1 of Project Sherpa. As an initial training for the people who were new to Docker, we held two “tech talks” to cover some of the basics. We assigned each developer a “homework” assignment to set up their environment and create a “Hello world” application. This ensured that everyone was set up correctly and would be ready to go once we hit “Day 1.

Dividing Up the Work

Our Sherpa team documented our list of 52 applications/services on our internal wiki and put a rough level of difficulty on each one (from Easy to Super Mega Crazy Hard). We assigned each service to a team, attempting to balance the level of effort among teams. Once the assignments were done, we empowered the individual Dev teams to be self-organizing. In almost all cases, the teams are pairing two developers for each service that needs to be containerized.

Scheduling

Determining the length of the project was a bit tricky, as such things usually are. We wanted to allow just enough time to containerize and to address the “must fix” tech debt items. We allocated three weeks to get to our “containerization done” milestone and one additional week to get everything wired together and into production. To get to the “containerization done” milestone, each individual service must be containerized, have passed all CI testing on a mini-environment, and have been handed off to Ops ready to go to staging.

Kicking It Off

Once we get through the first week of the project, we’ll re-calibrate the delivery schedule based on feedback from the developers working on each service. Along the way we plan to celebrate progress. Each time we hit “containerization done” for a service, we’ll ring the “RightScale celebration” gong.

To learn more about how we are using Docker, watch our on-demand webinar, Overcoming 5 Common Docker Challenges: How We Did It at RightScale, which covers:

  • Tips for securely producing high-quality, easy-to-manage Docker images
  • Options for multi-host networking between laptops and the cloud
  • Orchestrating Docker in production: service discovery, configuration, and auditability
  • Increasing host density with Docker
  • Dynamic monitoring and alerting for Docker containers

Watch the on-demand webinar.

This article was originally published on FierceDevOps.

RightScale 2016 State of the Cloud Report

Read More Articles on Migrating to Docker:
Migrating to Docker: Barely Controlled Chaos
Migrating to Docker: Why We’re Going All in at RightScale
Migrating to Docker: Halfway There
Migrating to Docker: The Second (Harder) Half
Migrating to Docker: The Final Results