RightScale Blog

Cloud Management Blog
RightScale 2014 State of the Cloud Report
Cloud Management Blog

Cloud Foundry Architecture and Auto-Scaling

Yesterday's blog post mostly covered the benefits of VMware's Cloud Foundry PaaS and how it fits with RightScale. Today I want to dive a little into the Cloud Foundry architecture and highlight how IaaS and PaaS really are complementary. I'm really hoping that more PaaS options will become available so we can offer our users a choice of PaaS software.

CloudFoundry Architecture

From a technical point of view I see two main innovations in Cloud Foundry. The first is that the software is released as an open source project with an Apache license, which gives users and third-parties access to make customizations and to operate Cloud Foundry on their own. The second is that Cloud Foundry is very modular and separates the data path from the control plane, i.e. the components that make user applications run from the ones that control Cloud Foundry itself and the deployment and scaling of user applications. The reason the latter innovation is significant is that it really opens up the door to innovate on the management of the PaaS as well as integrate it into existing frameworks such as RightScale's Dashboard. We're offering a free trial of RightScale cloud management with Cloud Foundry for dynamic server configuration.

Enough prelude, the pieces that make up Cloud Foundry are:

  • At the core the app execution engine is the piece that runs your application. It's what launches and manages the Rails, Java, and other language app servers. As your app is scaled up more app execution engines will launch an app server with your code. The way the app execution engine is architected is nice in that it is fairly stand-alone. It can be launched on any suitably configured server, then it connects to the other servers in the PaaS and starts running user applications (the app execution engines can be configured to run a single app per server or multiple). This means that to scale up the PaaS infrastructure itself the primary method is to launch more suitably configured app execution engines, something that is easy to do in a RightScale server array!
  • The request router is the front door to the PaaS: it accepts all the HTTP requests for all the applications running in the PaaS and routes them to the best app execution engine that runs the appropriate application code. In essence the request router is a load balancer that knows which app is running where. The request router needs to be told about the hostname used by each application and it keeps track of the available app execution engines for each app. Request routers are generally not scaled frequently, in part because DNS entries point to them and it's good practice to keep DNS as stable as possible, and also because a small number of request routers go a long way compared to app execution engines. It is possible, however to place regular load balancers in front of the request routers to make it easy to scale them without DNS changes.
  • The cloud controller implements the external API used by tools to load/unload apps and control their environment, including the number of app execution engines that should run each application. As part of taking in new applications it creates the bundles that app execution engines load to run an application. A nice aspect of the cloud controller is that it is relatively policy-free, meaning that it relies on external input to perform operations such as scaling how many app execution engines each application uses. This allows different management policies to be plugged-in.
  • A set of services provide data storage and other functions that can be leveraged by applications. In analogy with operating systems these are the device drivers. Each service tends to consist of two parts: the application implementing the service itself, much as MySQL, MongoDB, redis, etc. and a Cloud Foundry management layer that establishes the connections between applications and the service itself. For example, in the MySQL case this layer creates a separate logical database for each application and manages the credentials such that each application has access to its database.
  • A health manager responsible for keeping applications alive and ensuring that if an app execution engine crashes the applications it ran are restarted elsewhere.

All these parts are tied together using a simple message bus, which, among other things allows all the servers to find each other.

Auto-Scaling Cloud Foundry

"So, does it auto-scale"? seems to be the question everyone asks. (I wonder who started this auto-scaling business? ;-)) The answer is "no, but trivially so". There are actually two levels at which Cloud Foundry scales, whether automatically or not. The first is at the Cloud Foundry infrastructure level, e.g. how many app execution engines, how many request routers, how many cloud controllers, and how many services there are. The second level is at the individual application level and is primarily expressed in how many app execution engines are "running" the application (really, how many have the application loaded and are accepting requests from the request router).

The first level of scaling the Cloud Foundry infrastructure is the responsibility of the PaaS operator. The operator needs to monitor the load on the various servers and launch additional or terminate idle ones as appropriate. In particular, there should always be a number of idle app execution engines that can accept the next application or that can be brought to bear on an application that needs more resources. This level of scaling can be performed relatively easily manually or automatically in RightScale. The app execution engines can be placed in a server array and scaled based on their load. The second level of scaling is the responsibility of each application's owner. The nice thing about the modularity of Cloud Foundry is that it exposes the necessary hooks to adding external application monitoring and scaling decisions. It is also interesting that Cloud Foundry in effect exposes the resource costs and lets the application owner decide how much to consume - and pay for. This is in contrast to other systems that make it difficult to limit the resources other than by setting quotas at which point an application is suspended as opposed to simply running slower.

What we envision in working with Cloud Foundry is simple: RightScale will be able to monitor the various servers in the Cloud Foundry cluster, and determine for example when it’s “slack pool” of warm, ready-to-go app execution engines has dropped below a given threshold (or exceeded an idle threshold), and either boot new servers to add to the “slack pool” or de-commission unnecessary ones to save on cost, as appropriate.

PaaS and IaaS Synergy

The benefits of PaaS come from defining a constrained application deployment environment. That makes it necessary for many applications to "punch out" and leverage services outside of the PaaS framework. In some cases this may be a simple service, like a messaging server or a special form of data storage. In other cases it will end up being almost a reversed situation where a large portion of the application runs outside of the PaaS and the portions in the PaaS are really just complements or front-ends for the main system. Cloud Foundry makes it relatively easy to make outside services available to applications in the PaaS, but these outside applications still need to be managed. This is where an IaaS management framework like RightScale is great because it can bring the whole infrastructure under one roof.

Some examples for this punching out:

  • Databases from the SQL variety to NoSQL and other models. Accessing legacy databases as well as leveraging popular DB setups like our MySQL Manager, which provides master slave replication.
  • Different load balancers in front of the request routers, perhaps with extensive caching features, global load balancing, or other goodies. Examples would be Zeus, Squid and many others.
  • Legacy or licensed software, for example video encoding software or PDF generators.
  • Special back-end services, such as a telephony server.

If there's one thing I've learned about customers at RightScale it's the incredible variety of needs, architectures, and software packages that are in use. For this reason alone I see PaaS as another very nice tool in the RightScale toolbox.

Can you run Cloud Foundry without RightScale? Of course. It certainly runs on raw servers. They can PXE boot a base image and join the PaaS in one of the above server roles. However in a mixed environment it is much more beneficial to run the Cloud Foundry roles within a managed infrastructure cloud.

It seems obvious from the traditional SaaS/PaaS/IaaS cloud diagrams that these different layers were made to interoperate. And that’s what we’ve already seen our customers doing: combining PaaS and IaaS in ways that meet their needs. There are a number of PaaS solutions in the market with more on the horizon. We will continue to support as many as we can and to the extent that their architectures allow it, because cloud is a heterogeneous world and customers want choice. In the case of Cloud Foundry, we have a particularly open architecture that provides a compelling fit – and we’re excited to see where our joint customers take us together. Sign up for a free trial of RightScale with Cloud Foundry.

Comments

Can you please point me to the doc (describing the mechanism) and/or code (implementing measurement costing) pertaining to this statement? "Cloud Foundry in effect exposes the resource costs and lets the application owner decide how much to consume–and pay for."
Posted by williamlouth (not verified)   Ι   April 16, 2011   Ι   06:38 AM
What I meant by that statement is that as an application owner in CloudFoundry you get to decide how many resources are brought to bear to run your app. In particular, you get to decide how many instances are active, and presuming an instance-hour cost model this gives you control over the application cost. This means that you get to decide where you want to be on the cost/performance curve, e.g. whether you want to deploy more instances to improve performance (assuming this actually scales performance) or whether you'd rather save at the expense of performance. Also, if there is unwanted traffic hammering your application, you can decide not to scale-up and thus not to spend extra money on servicing these unwanted requests. This is in contrast to PaaS services where you pay essentially per-request and/or where you cannot easily control the scaling. Or where cost control comes in the form of a quota with the result that your app is "off the air" once the quota is reached.
What about the layer between CloudFoundry and Amazon EC2 for example? You indicate in your post that this is open? Do you use a cloud abstraction layer similar to deltaCloud for enabling this? Regarding Elastic Load Balancing, it's possible to configure the AWS ELB through its api. But who controls the instances in that case, is it the ELB or still the cloudFoundry cloud controller? Could you explain how it works thx for you answers!
Posted by Ludovic Havet (not verified)   Ι   June 20, 2011   Ι   02:05 AM
Here is another effort to explain the architecture http://virtualization-for-layman.blogspot.com/2011/05/vmware-cloudfoundry-architecture.html
Posted by sss (not verified)   Ι   May 24, 2011   Ι   08:34 AM
Nice explanation. It's tough to understand these complex technologies easily. Check my effort to explain cloud foundary in simple terms http://virtualization-for-layman.blogspot.com/2011/05/what-is-cloud-foundary.html
Posted by sandeep (not verified)   Ι   May 15, 2011   Ι   02:44 PM
[...] About ← RightScale Release: 25 ServerTemplates across 66 RightImages Cloud Foundry Architecture and Auto-Scaling → [...]
[...] Zend PHP Solution Pack represents its second major PaaS effort in as many weeks. Last week, it was announced as a foundational IaaS partner for VMware’s Cloud Foundry PaaS software that supports a variety of languages, including [...]

Post a comment