Coho Data’s Xen Heritage

A couple of years ago, the core  of the XenSource team left Citrix with the conviction that a new generation of infrastructure software – developed to support massively scalable public clouds – could revolutionize enterprise computing.

We had been developing Xen and cloud infrastructure since 2005, and it was clear that one of the most profound differences between enterprise virtual infrastructure and the “stuff of the public cloud” was storage.   In the enterprise a mainframe-like addiction to expensive arrays and fiber channel switch networks was a mire of complexity. (EMC had wanted $1M to certify XenServer for Symmetrix – we politely declined). But Facebook was where the action was – a massively scalable system reliably serving hundreds of millions of users and billions of objects, built with open source software – MySQL, Memcache, Apache, and PHP – and commodity compute and flash-based storage.

Were there any lessons to be learned from Facebook?  Andy Warfield made the connection:  He booted a VM from a virtual hard disk file on a filer, by making it appear to be an iSCSI target.   But instead of a typical fat VHD, his file contained the bare bones of a VM, just enough to boot, and a new magic storage abstraction.  Once the bare-bones OS had booted and began to reference files in its file system Andy’s magic driver mapped these into – you guessed it – references to files in a memcached-like VM running on the hypervisor.   Using Xen’s v4v shared memory transport between these components allowed the VM to be constructed on-the-fly, at memory speed, from cached components, and the caching tier did what it needed to – objects that were used a lot sat in memory, and the rest automatically tiered: some in local flash, local disk, networked storage or even – you guessed it again – Amazon S3.


Fast forward a couple of months, and the “dream team” was assembled: Andy, Keir Fraser (the chief committer of Xen), Ramana Jonnala (our VP Eng for XenClient, and formerly a storage ace at Veritas), and a team of the world’s best virtualization, storage and security engineers. Coho Data was founded to deliver high performance Amazon-style storage to enterprises that will serve all of their data – block, file and object.  Their first product is the industry’s first flash-tuned scale-out storage architecture designed for the private cloud – that delivers predictable high performance of greater than 1M IOPS in ⅓ of a rack with pay-as-you-grow, public cloud capacity pricing.   Cloud-inspired storage for the enterprise is here.

Although it sounds easy, this has been one of the most complex of dev projects.  Enterprises don’t just want to store photos, so this system isn’t allowed to fail.  Fortunately the Coho team is one of the world’s best.   Xen – built by the same team, isn’t allowed to fail AWS, and this team has played a major role in making Xen so successful in public clouds today.   They are about to do the same for storage – this time for enterprise private clouds.   I’m genuinely privileged to be an advisor to this incredible group.

10,940 total views, 4 views today