Home / Blogs

Putting Cloud Providers to the Test - New Bitcurrent/Webmetrics Report

Recently Bitcurrent conducted a cloud performance test commissioned by my client Webmetrics, a business unit of Neustar. Bitcurrent describes itself as part blog, part analyst firm and part resource site for the web community. The research project was led by Alistair Croll, a well-known name in the space and program chair for the Cloud Connect conference.

You can’t read a technology story today without the “cloud” being mentioned. But there is precious little reputable, empirical research by third parties to help customers understand the term, and to help determine which company they should go with for cloud services.

Bitcurrent has produced a fascinating report, tracking the cloud performance of the leading providers in a number of ways. The full 50+ page report is only available via registration off the Webmetrics web site, but I’d be happy to send an abridged version to anyone who’d like to see it. Just drop a comment here, send me an email or ping me on Twitter—@cparente.

Here is some background on the report. Five companies were analyzed:

  1. Salesforce.com
  2. Google App Engine
  3. Terremark vCloud
  4. Amazon Elastic Cloud Computing (EC2)
  5. Rackspace Rackspace Cloud

Four different performance metrics were isolated:

  • A simple web request for a small, static object—this tested responsiveness
  • A request for a large, 2 Megabyte object—this tested network throughput
  • A request that triggered one million mathematically intensive calculations—this tested computing power
  • A request that triggered a search of 500,000 rows in a database for a string—this tested the back-end I/O (input/output)

I spoke at some length with Alistair about the report. He was jazzed about the report and said no one had ever measured the performance of cloud computing in this way. In particular, he stressed how the report uses histograms to illustrate the performance of each company. Histograms are a kind of graphical display that allowed for the display of the performance of all five companies for a specific request on a single graph. It’s as close as you can get to a distinct “fingerprint” for cloud performance:

Latency for delivery of 2 MB GIF

Alistair also had very nice words for Webmetrics underwriting the project and then staying completely hands off. This is from his blog:

At Bitcurrent, we try to remain nonpartisan. We want to contribute to the industry’s growth and understanding. So we were thrilled at Webmetrics/Neustar’s approach here: they financed the research into cloud performance, and let us use their monitoring tool to collect the data from our agents. They were supportive, and enthusiastic, but exercised no editorial control whatsoever over the content of the report. This kind of altruistic, good-for-the-market contribution is, in our opinion, the best kind of marketing, and we commend them for adopting it.

Just like we shoot for at my firm—great work for great clients. If you’re currently evaluating the cloud for business or academic reasons, this could be very useful information. Or, in the fine CircleID tradition, take some shots at it!

Ping me for a copy if interested.

By Christopher Parente, Founder, Content Marketing Agency

Filed Under


Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Co-designer of the TCP/IP Protocols & the Architecture of the Internet



IPv4 Markets

Sponsored byIPv4.Global

Domain Names

Sponsored byVerisign

Threat Intelligence

Sponsored byWhoisXML API

New TLDs

Sponsored byRadix

Brand Protection

Sponsored byCSC


Sponsored byVerisign


Sponsored byDNIB.com