It is a multi-cloud world with most enterprises using a portfolio of multiple clouds. Typically they use at least several public clouds as well as private clouds. The RightScale 2016 State of the Cloud Report found that AWS, Google Cloud Platform, Microsoft Azure, and IBM SoftLayer are among the top public cloud providers.
For companies that take a strategic approach to their cloud strategies, it's not as simple as choosing Cloud A and putting every workload in Cloud A. Instead, they need to consider the requirements for each application or workload and select the cloud that is the best fit. You will need to consider a variety of factors including features, costs, locations, security and compliance, performance, existing data centers you have in place for your private clouds, vendors that you have enterprise agreements or discounted pricing with, and more.
There are several key differences among public cloud vendors. In this blog I’ll drill down on container services and pricing, and in a separate post we go in depth on cloud storage.
With any of these clouds, you can install whatever container management or container hosts you want on top of the instances they provide but you are completely responsible for deploying and managing these solutions yourself. Three of these cloud providers — AWS, Azure, and Google — have a container-as-a-service offering. These containers services provide you with a ready-to-go managed solution that you can start using. SoftLayer doesn’t have a container service yet, so it is excluded from this comparison. Let’s look at each of the container services that AWS, Google, and Azure are providing and some of the key differences among them.
Amazon EC2 Container Service (ECS)
- General Availability (GA) in April 2015
- Custom scheduler or third party via API integration
- Integrates with existing services; IAM integration for permissions; CloudTrail integration for container logging; CloudFormation templates for launching clusters (with many examples)
- Uses regular EC2 instances for container hosts, with a lightweight agent for coordination
There are some interesting aspects to Amazon ECS. First, it uses a custom scheduler or cluster manager for the containers so it is ECS specific, but you can integrate any other system using the API. One of the really strong factors for AWS is that it has integrated ECS with the other services that it provides — in particular IAM — so you can manage permissions on ECS clusters using IAM.
AWS has also integrated CloudTrail, so you can get metrics from your containers and from the container hosts into CloudTrail and make decisions based on those. If you are using AWS CloudFormation, all of this is available through CloudFormation templates, so if you already have applications defined using CloudFormation, it's fairly easy to add ECS capabilities into those templates or to use CloudFormation templates to create new ECS clusters and host definitions. The main strength of AWS is the integration with its other services. If you're already using these services, bringing ECS into the fold is straightforward.
One detail that is true for AWS as well as the other clouds is that it uses regular instances as the container hosts. AWS deploys a lightweight agent onto those hosts, and the agent coordinates with the Amazon ECS system to do things like actually deploying containers.
Azure Container Service
- Preview in December 2015, expected General Availability (GA) early 2016
- Multiple orchestrators available: Apache Mesos, Docker Swarm
- Supported in Azure Resource Manager API; ARM templates available
- Currently no UI to manage clusters
Azure Container Service is the newest of the three container services. One of the interesting things about Azure is that it has multiple orchestration cluster management tools available for you to use by default. When you create a cluster in Azure, you can choose whether you want to use Apache Mesos or Docker Swarm. Apache Mesos is open source, and the Azure team is actively participating in the development of this tool to improve it and add more functionality, while Docker Swarm is the Docker-provided cluster management tool.
Just like the other providers, Azure has everything available via API, so if you want to use another tool, you can do that, but I’ll focus here on the ones that come out of the box. These orchestrators are supported in the Azure Resources Manager API. They do have ARM templates and these are somewhat similar to Amazon’s CloudFormation templates — templates that define your configurations and templates to help you stand it up. Currently, while Azure’s container service is in preview, there is no UI that you can use to manage a cluster, so this is something to keep in mind as you move forward with deciding which one to use. There is no UI if you need one, but we anticipate that should be coming with the GA release.
Google Container Engine (GKE)
- General Availability (GA) in August 2015
- Powered by Kubernetes; runs a Kubernetes master node outside of your project; container hosts run on instances inside your project
- Integrated with Google Cloud Logging for container metrics
- Provides a private Docker registry
- JSON-based declarative syntax for configuration
Google Container Engine is powered behind the scenes by Kubernetes, which is the open source cluster manager that Google has provided to the community. Some of the interesting things here if you are familiar with how Kubernetes works is that it runs a master node outside your project, and that node is what's responsible for coordinating what happens on all of the hosts. You do have access to it with the cube CTL command-line tool, and there is a bit of UI integration here as well, but that actual node is outside your project while the actual hosts that run the containers run on instances inside your project.
Similar to AWS, Google has integrated Google Container Engine with other things on its platform, which for Google primarily means Google Cloud Logging. One of the great things about Google is that it offers a private Docker registry by default. Again, this is something you could set up in AWS or on other providers, but you'd have to do the work yourself, whereas with Google it’s already provided.
Google also leverages a JSON-based syntax that you can use to define the behavior of what happens on the hosts. So you define in one file the declarative nature of what you want, and then Kubernetes handles the tracking of what's actually happening across all the hosts. Google Container Engine also has health checks built in; it can launch and terminate containers and different hosts as needed to make sure that whatever you've defined in the configuration is being satisfied.
A quick summary as you're looking to deploy on one of these services: There are two areas you should be most interested in and pay most attention to as you make your decision. The first is the state of the service. As I mentioned, AWS and Google are currently GA, while Azure will be GA soon. One of the most important factors in deciding which container service to use is what default orchestrator or cluster management engine they're using. If your technical team is already familiar with one of these engines or sees great promise in one of them and wants to stick with it, that can greatly sway your decision toward a specific container services. Again, all of these companies provide APIs, so you’re not actually limited to that engine, but that is the default one and will be the least effort to get going.
The second consideration in pricing. Amazon ECS is free, while Google Container Engine is free up to five nodes. If you have more than five nodes, Google charges $0.15 per cluster per hour. In both cases, you do still pay for the instances on which the containers run. We don’t currently have pricing information on Azure, but we expect that to come out soon.
Cloud Compute Pricing
In the majority of cases, more than 80 percent of your cloud infrastructure costs are for compute resources. Compute has a lot of discount mechanics that you can take advantage of to reduce your costs, so that's also an area where you can optimize. One difference is that the charging granularity differs among providers, as well as the minimum time they will charge you regardless of how long your instance is running. In AWS and SoftLayer that granularity is hourly, while Azure and Google are by the minute, although Google has a 10-minute minimum. For example, if your workload is to launch a lot of servers for 20-30 minutes and then turn them off again, in AWS and SoftLayer you will be charged for the full hour; in Azure and Google you're only going to be charged for that amount of time that you used it, which amounts to partial hours. So depending on your workloads, that could be an important consideration.
With AWS, On-Demand Instances are the most expensive, since you use what you need and pay per hour. Reserved Instances are an option where you make a commitment for 1 or 3 years and decide how much of it you want to pay upfront to determine the discount level, which can range up to 75 percent off. Scheduled Reserved Instances differ from Standard Reserved Instances. They have a 1-year term, are priced between 5 and 10 percent below On-Demand Instances, and they can only be used during specific times of day or night.
Spot Instances let you bid for and get an instance as long as your bid is higher than the current spot price, but Spot Instances have no guaranteed duration and may disappear at any time. The spot price varies but often amounts to a 50-90 percent lower cost than On-Demand Instances. Spot Blocks are another option, providing a defined-duration Spot Instance where you bid and request specific duration (up to 6 hours). This flat rate saves you up to 50 percent compared to On-Demand Instances and has a guaranteed duration.
Many enterprise customers have Microsoft Enterprise Agreements that include discounts for Azure ranging from 10-36 percent and upward. If you don't have a Microsoft Enterprise Agreement, you can still get pre-paid subscriptions, which also provide a discount over on-demand pricing.
Google automatically gives you the best price with its Sustained Use Discounts (SUD). The Google SUD discounts the on-demand prices based on how long you use a specific instance type or specific number of CPUs and RAM. If you only use an instance for 25 percent of the billing period, you'll get on-demand pricing. As you use it 50, 75, or 100 percent of the billing period, you’ll get successively higher discounts. With Google, you don’t need to pre-purchase or make decisions because these discounts are applied to your bill so that you automatically get the best price.
Google also has something comparable to AWS Spot Instances, called Preemptible Instances. Unlike AWS Spot Instances, which have a fluctuating price based on the latest bids, with Google Preemptible Instances you know the cost upfront. However, Google Preemptible Instances will never run longer than 24 hours.
SoftLayer has two pricing options: on-demand with pre-hour prices or a monthly commit. If you choose monthly commit, you'll get a cheaper rate.
One interesting aspect of both Google and SoftLayer is that you can have custom instance types. So if you need a specific configuration, you can get that on these clouds, but with AWS and Azure you may need to overprovision the RAM or CPUs to get the instance parameters that you want. Custom instance types can help you save money by creating the optimal combination of CPU and RAM for your needs.
Cloud providers are continually lowering their prices. AWS recently announced a price reduction, although it's only on specific instance types running Linux. As soon as AWS made the price reduction, Azure followed up with its own reduction. Azure has always maintained that it will match AWS prices. However, sometimes it can be difficult to compare apples to apples since instance sizes and specifications often don’t match exactly. Also, if you have a Microsoft Enterprise Agreement, you will get an additional discount, so it becomes a bit more complicated to compare.
Google, because of its Sustained Use Discounts, comes out as cheaper than on-demand prices compared to the other cloud providers. If you're using an instance 100 percent of the time, Google automatically gives you the best discounts, as opposed to your having to decide that and make a commitment upfront. Keep in mind that if you buy AWS Reserved Instances, the price depends on your region, your Reserved Instance type, and how long you've committed for, among other factors. Google is taking the strategy of keeping it simple with the concept of using whatever you want, charging you per minute, and giving you the best price based on that usage.
Keeping track of the various features and pricing structures of public clouds is cumbersome and complex, but tools such as Cloud Comparison can help simplify the process. In addition, the information above should provide you with a starting point to determine which options best suit the needs of your company.
To get more detail on the topics I covered here, watch our on-demand webinar, Compare Clouds: AWS vs. Azure vs. Google vs. SoftLayer.