VM Storage: We can build it, we have the technology

A post with Darren Gosyne, a Senior Escalation Engineer in our Support Team.

Colonel Steve Austin thinks Coho Data has the technology; the world's first bionic storage. Better than storage has been before. Better... stronger... faster.

Colonel Steve Austin thinks Coho Data has the technology; the world’s first bionic storage. Better than storage has been before. Better… stronger… faster.

One of the best things about being among the new kids on the block in the Enterprise Storage market is listening to users about their needs. The responses are not exactly surprising; they want a storage product that solves their problems, saves them money, and makes their lives easier. Their workloads aren’t Google’s workloads but they want something flexible, simple, and scalable.

 

Introducing SnapAttach

In recent weeks, we’ve been able to introduce a new feature in response to a customer’s feedback on their VM storage workflow needs. We’ve dubbed this new capability Coho SnapAttach. It’s another example of the programmable nature of our scale-out storage system; allowing us to quickly adapt, and provide more than a one-size-fits-all ownership experience.

 

“What’chu talkin’ ’bout, Willis?

Picture this. You frequently want to re-synchronize the data across several VMs. For example, your application team has completed the QA phase of their new web portal and wants to push these changes through to their UAT test groups.  The environments are all aligned, so the VM in QA is mirrored by several similar VMs in UAT, but the data needs to be refreshed.

SnapAttach

In the diagram above you can see they only need to update the “www” disk on the 3 UAT web servers. So how would you do this traditionally?  There’s a bunch of different approaches, but the two most common are:

Old Skool

Old Skool

  • Create a file share on QA, delete the contents of the www drives on the 3 destination VMs, copy the files over the network so the E drives all look alike. If we’re clever we’d think about things like doing a diff copy. Oh, and don’t forget the files’ permissions.

 

By the power of virtualization. I have the power!

By the power of virtualization. I have the power!

  • But we wouldn’t do that. We’d be clever and clone the VMDKs. We’d shut down the VMs, browse the datastore in vCenter and start copying and pasting VMDKs to the UAT VMs’ folders. Rename the new copies appropriately. Re-linking the VMDKs in the Edit Settings dialogue box of each UAT.  Or better yet, we’d be all over it with our “vmkfstools -i” l33t skills.
Time for another cup of joe

Time for another cup of joe

Either way, it’ll be a time consuming business. It might be fun the first few times but it’d get boring real quick.

What if running one command coordinated all this for you? What if that command used storage-offloaded cloning techniques to make this happen almost instantaneously? What if you could pass the task directly over to your application team so they could refresh disks themselves as frequently as they wanted?

This is where SnapAttach comes in…(Click to tweet)

We call it the Flux Capacitor SnapAttach. Fortunately it doesn’t require one point twenty-one jigawatts.

We call it the Flux Capacitor SnapAttach. Fortunately it doesn’t require one point twenty-one jigawatts.

Lights, Camera, Action

Here’s a demo of SnapAttach working. You have a number of VMs (Prod, Test, Dev) and the Prod VM contains two virtual disks (P1-OS and P2-Live Data). Test and Dev need a refresh of the Live Data as part of your daily activities. You want to clone Prod’s data, and replace the P2 disks of the Test and Dev VMs.

In SnapAttach we’ve included the ability to add the cloned VMDK(s) as a new disk(s), or replace an existing disk with the clone. This is controlled by specifying the destination VMs’ SCSI bus and ID, and designating a replace flag. A “next available” scsi id option is also included. SnapAttach is guest OS agnostic, but as you can see in the video, Windows happily mounts the disk, assigns a drive letter and even preserves the drive label. It will automagically add another SCSI controller if you’ve run out or if you designate a new bus id, and you can specific what type of controller you need. This sort of granularity is useful, for example VADP-style backups with Volume Shadow-copy Service (VSS) has special considerations around keeping enough IDs free.

Additionally, built into this feature is the ability to quiesce the source VM. If quiesce is specified, VMware’s guest tools are used to place the guest file systems (and in some cases, applications) in a consistent state prior to fast cloning the disk.  On Windows guests, this involves a VSS snapshot request to the guest VM.  If quiesce is set to false, the snapshot is taken immediately against the current state of the VM’s disks – this makes these operations significantly faster.

The SnapAttach API commands use a RESTful web service, interacting with a JSON-encoded namespace of our configuration and control APIs. Coho is committed to exposing features like this over a client-facing RESTful API.  We’ll continue to provide these to our customers as we work with them in their own environments, often delivering them as PowerShell or Python scripts.

 

<code>

To show an example, here’s a typical JSON input file for the command.  It doesn’t contain all the available options, but it should be enough for you to get the gist. Hopefully this helps demonstrate what’s starting to happen under the covers.

{
  "name": "example_snapshot_name",
  "quiesce": true,
  "source": {
    "vdisks": [
      {"lunid": "0:0"},
      {"lunid": "0:1"},
      {"lunid": "3:4"}
    ]
  },
  "targets": [
    {"vm": "a_vm_name",
      "vdisks": [
        {"lunid": "2:2"}
        {"lunid": "1:2"},
        {"lunid": "next_available"}
      ]
    },
    {"vm": "another_vm_name",
      "vdisks": [
        {"lunid": "2:2"},
        {"lunid": "1:2"},
        {"lunid": "0:4"}
      ]
    }
  ]
}

And I care because?

In the real world, you can imagine lots of serial processing scenarios – reducing the delay for tasks such as reporting, data mining, modelling etc. Maybe you need a point-in-time copy of your SQL transaction logs to run queries against the copy, to repeat multiple times a day.  Or you want a quick-and-dirty recovery option for a critical server that’s available online all the time (different hostname and IP address).  Building on the data refresh idea, maybe you have application updates that can be isolated on disk clones. Maybe there’s a suite of troubleshooting tools and scripts that you want attached to all your VMs, but copying it to every instance (and keeping them synced) made you think twice.  We’re only touching the surface; there’s a raft of different use-cases, and each environment will have its own special place for this tool.

This is just one example of Coho’s approach to re-invent the way storage is done. We believe in bridging those traditional boundaries and making more use of our scale-out storage across the enterprise. Our customer in question wanted to make their workflow more efficient and we were excited to help. So we built them a new API endpoint to our system, leveraging our file-level, space-efficient snapshot technology to clone point-in-time copies of the virtual disks and attach them to other VMs.

There’s a familiar sentiment in enterprise IT that we believe should be abolished; powerful, enterprise systems do not allow for customization. In a world where everything is next-gen, bigger than big, or the next whizzbang thing; it’s increasingly important to realize that technological advancement does not need to come at the expense of flexibility.  At Coho we’re rapier-focussed on solving genuine workload and workflow problems by delivering adaptable holistic storage solutions.

So, what can we build with you?


Interested in Coho Data? You should download our datasheet or  get the independent analyst review by ESG here.

7,811 total views, 3 views today