RightScale Blog

Cloud Management Blog
Cloud Management Blog

Why Do-It-Yourself Cloud Computing Management Is a Temporary Fad

I recently called up my buddy who used to be vice president of marketing at SugarCRM. I asked him if he ever encountered companies that were building their own CRM solutions internally. "No, that's dumb," he said. "That's why they came to Sugar, so they could use ours. It's too much work to do it yourself."

Building your own Salesforce.com? Yup, sounds like a lot of work. Yet here at RightScale, I see many companies trying to build their own cloud management solutions. Perhaps it is the DevOps mindset that has made cloud computing so popular: "If I can't get approval, I'll just do it myself on the side." Or perhaps it is because we are still in the early stages of cloud, and people are experimenting and discovering what is possible internally versus what is available in the market.

I did an informal poll of our sales team, and here's what they said were the top reasons companies try to make their own solutions rather than use a cloud management product:

  1. They want control, or the ability to highly customize their environment.
  2. PaaS and IaaS, as concepts, seem simple, easy to jump on. A "cloud computing management platform" seems like a complex paradigm to adopt.
  3. Because they can, and they want the challenge of exploring a new frontier.
  4. The cost of a cloud management solution is too high.

OK, so these appear to be valid reasons at first glance. But these statements are typically founded in misconceptions about cloud management solutions in general or RightScale in particular, which I'll address here:

Control: RightScale is not a PaaS service. We let you get into everything - perhaps more so than we should. Change the images if you must, run custom scripts against our API, and export usage data to include in your own data warehouse. Fifty-two percent of the servers running on RightScale are controlled by completely custom ServerTemplates, not ones we provide. Our product philosophy is to let you "get under the hood" if you need to - so please do.

Complexity: Cloud management is complex, and I don't argue that. What RightScale aims to do is provide a layer of abstraction that makes the difficult and mundane tasks, like auto-scaling, much easier. It is unfortunate that the term seems complex, because if anything, a cloud management solution can make managing your entire cloud infrastructure and applications so much easier.

Conquering the new frontier: You're being told by your boss to "Learn cloud now - just figure it out." You want to truly understand what's possible, how to build it, and deliver on expectations. As you start down this path, you cobble together some tools to accomplish your first foray into the cloud. Unfortunately, technologists have a tendency to "reinvent the wheel" as they continue along their path to the cloud. We're many steps ahead, and we're happy to share what we've already learned.

Cost: Netflix is a poster-child for DIY cloud, and has been forthcoming about its experience, which has helped grow this new paradigm. Netflix "designed its cloud architecture so that it has the option to move to an Amazon Web Services competitor" if needed, according to this NetworkWorld article. At a recent conference, Adrian Cockcroft, Cloud Architect for Netflix, mentioned that Netflix has 50+ engineers working on this cloud-independent solution. Doing some quick math, that's about $8.3 MM per year Netflix spends building and maintaining this platform. That could buy a lot of RightScale Enterprise Editions!

At the end of the day, we see many customers who come to us after they outgrow their own internal solutions. They eventually discover that there are just too many things to stitch together: configuration management, systems automation, monitoring, application automation, provisioning, user permissions, reporting...it goes on.

We have hundreds of employees and have spent many millions creating the most comprehensive cloud management platform in the world. And we designed our product to drive the same way no matter which cloud you choose. So while cloud management may seem like a fun weekend project to tackle, it's not - please don't try it at home.

Yes, Amazon is still the dominant cloud, but a tornado of new clouds is swirling. The next thing your boss will likely ask is, "So what if we wanted to use this other cloud instead?"

Update Feb 3, 2012: Since I published this post, I've received a lot of feedback regarding DIY in the cloud computing space.

A few of our customer developers pointed out that they actually appreciated learning the cloud through RightScale - it gave them both an understanding of the underlying IaaS cloud as well insight into ideal cloud management frameworks. Forbes ran articles on how this extensive cloud computing knowledge is in high demand in IT and is a ticket to the corner office, and we're starting to see RightScale listed as a required skill on some of these cloud job postings.

Next, I've heard from a few more larger companies who have built their own internal cloud management solution. They also cited approximately 50 engineers in their cloud computing groups, so it seems this is the sweet spot for development and maintenance of a robust internal solution. Let's not forget about the PaaS-like solutions we offer with our ServerTemplates in this regard - it is not just automated provisioning that these larger companies ultimately need to build.

I'm not saying you can't "do it yourself" in cloud computing (or in anything for that matter), I just want to encourage developers to avoid the trappings of #3 above - namely ignoring off-the-shelf solutions in the interest of personal discovery. It may work in the short term... until you hit one of the many walls that we've already had to plow through. At that point, you'll either have to scale the solution and team, or re-architect for a product that offers the necessary solutions already.

Update March 15, 2012: This post stirred a lot of pots, so I expanded it into an article on SYS-CON called "Do-It-Yourself Cloud Computing Management – Is It Worth It?" However you come down on DIY cloud management, I think you'll find more material worth mulling over.

Comments

The bit of the Netflix platform that matches up to what Rightscale does was built by one engineer, we added a second engineer recently, and have been able to scale and customize it in a very agile manner. Most of the engineers we have are building much more high level PaaS features, managing Cassandra data sources, integrating monitoring tools and providing 24x7 site reliability engineering.
It is important to distinguish massive cloud users like Netflix and other types of users. In the case of Netflix, they may have a very specific set of requirements that a general purpose management console/framework like Rightscale does not cover. In this case the flexibility of being able to do exactly what they need is much important than those 9MM (which are probably peanuts compared to what they are paying in terms of AWS). Among the other types of users, I can understand why some companies deploying private cloud may not be comfortable with a hosted SaaS management solution like Rightscale, preferring to keep everything behind the firewall. Finally, Rightscale is oriented toward system administrators, some of the 'homegrown' portals I have seen are more oriented towards end-users and self-service, tying them to internal auth services. Having said all that, Rightscale is the right choice for a lot of scenarios, so I also have to scratch my head when people want to create their own equivalent from scratch. Reminds me of mid nineties when everybody and their brother were building their own web servers :)
I think it depends who's doing what themselves. For some, IaaS is more interesting, but for experienced application developers PaaS is likely to be very interesting as it provides higher utilization and choices of isolation (i.e. virtual or paravirtual). I always wonder if RightScale is a PaaS, though... I think of a platform as a thing you can build other things on (a framework for development) and I think of a project as a resource pool of developer ideas. I know that industry designations referring to cloud delivery intend to shed light on service level agreements, so I understand why we're saying it's not really PaaS as we know it. I still wonder if RightScale is a SaaS dashboard managing IaaS across clouds or if it's evolving into a service catalog of platform services... and maybe wondering and continuous curiosity are good things since we're dealing with services and people change when they move around and surround themselves differently... therefore services change and solution stacks can reproduce themselves asexually as new people have problems you've solved with the same old solution you invented in the beginning. For RighScale access to resources across multiple clouds is key right now, but client requirements and services change over time and PaaSification could happen at some point couldn't it? You can already instantiate a cloudfoundry deployment from the multi-cloud marketplace so it will be interesting to see what developers might require as customers of RightScale-as-a-non-platform-service.
[...] from: Why Do-It-Yourself Cloud Computing Management Is a Temporary … Related Posts:Top 10 cloud computing stories of 2011 – Information Management … It’s [...]

Post a comment