Tag Archive: VVols

Aug 08 2017

Learn about VVol Replication online and at VMworld

The ability to use VVol replication is new in vSphere 6.5 and there aren’t a whole lot of vendors that support it yet. Right now HPE is the only vendor that supports it on their 3PAR & Nimble platforms. You can see for yourself who supports it by going to the VVols’s Compatability Guide and selecting “VVols Storage Replication” in the Features window. As with anything VVol’s related, it’s on each partner to design their own implementations of any array capabilities related to VVols while staying within VMware’s VASA specification. As a result every partner will typically have a much different way of integrating with VVols.

Take Nimble for example, they actually supported replication in vSphere 6.0 by building it into their storage policies as a capability. With vSphere 6.5, replication was added as a separate SPBM Component that could be attached to a storage policy, this is the way 3PAR is doing it. They both are a means to the same end but they were just designed and implemented a bit differently. While the replication part is fairly straightforward and is automated through storage policies, the recovery operations can be a bit complicated as you pretty much have to rely on PowerCLI scripts to perform operations such as failover, failback and test failover.

HPE and VMware have published a few resources to help you with this and provided you with some sample PowerCLI scripts to get you started. The scripts are fairly vendor agnostic so they could be leveraged by any storage platform that supports VVol replication. Below are the links to some resources that will help you understand and implement VVol replication. In addition if you are attending VMworld be sure and check out session [STO3305BUS]: Replicating VMware VVols: A technical deep dive into VVol array based replication in vSphere 6.5 were I’ll be presenting with some technical folks from 3PAR & Nimble to give you plenty of great information including demo’s on VVol replication. In addition there is a HOL (SPL182701U) were you can experience VVols  first hand.

VVol Replication resources:
Share This:

Sep 15 2016

Top 10 reasons to start using VVols right now

VMware’s new storage architecture, Virtual Volumes (VVols), has been out for a year and a half now as part of the vSphere 6.0 release and adoption continues to be very light for a variety of reasons. This kind of adoption can be typical of any 1.0 release as users wait for it to mature a bit and better understand the differences of it compared to what they are used to using. VVols brings a lot of great benefits that many are unaware of so in this post I though I would highlight those to try and make a compelling use case to start using VVols right now.

top10list-crop10 -You don’t have to go all in

It’s not all or nothing with VVols, you can easily run it right alongside VMFS or NFS on the same storage array. Because VVols requires no up-front space allocation for it’s Storage Container, you will only be consuming space on your existing array for any VMs that are put on VVol storage. Since VVols is a dynamic storage architecture, whatever free space you have remaining on the array is available to your VVols Storage Container which is purely a logical entity unlike a VMFS volume which requires up-front space allocation.

You can easily move running VMs from VMFS to VVols using Storage vMotion and back again if needed or create new VMs on VVols. This allows you to go at your own pace and as you move VMs over you can remove VMFS datastores as they are emptied out which provides more available space to your Storage Container. Note that Storage vMotion is the only method to move existing VM’s to VVols and you cannot upgrade VMFS datastores to VVol Storage Containers.

9 – Gain early experience with VVols

VMware never keeps 2 of anything around that do the same thing, they always eventually retire the old way of doing it as it is double the development work for them. At some point VVols will be the only storage architecture for external storage and VMFS will be retired. How long did you wait to switch from ESX to ESXi or to switch from the vSphere C# client to the web client? Did you wait until the last minute and then scramble to learn it and struggle with it the first few months. Why wait, the sooner you start using it the sooner you will understand it and you can plan on your migration over time instead of waiting until you are forced to do it right away. By gaining early experience you will be ahead of the pack and can focus on gaining deeper knowledge of it over time instead of being a newbie who is just learning the basics. There are no shortage of resources available today to help you on your journey to VVols.

8 – Get your disk space back and stop wasting it

With VVols both space allocation and space reclamation is completely automatic and real-time. No storage is pre-allocated to the Storage Container or Protocol Endpoint, when VM’s are created on VVols storage they are provisioned thin by default. When VMs are deleted or moved space is automatically reclaimed as the array can see VMs as objects and knows which disk blocks they reside on. No more manually running time and resource intensive cli tools,  vmkfstools/esxcli to blindly try and reclaim space on the array from deleted or moved VMs. VVols is designed to allow the array to maintain a very thin footprint without pre-allocating space and carving up your array into silos like you do with VMFS and at the same time being able to reclaim space in real time.

7 – It’s available in all vSphere editions

VVols isn’t a vSphere feature that is licensed only in certain editions such as Enterprise Plus, it’s part of the vSphere core architecture and available in all vSphere editions. If you have vSphere 6.0 or higher you already have VVols capabilities and are ready to start using it. VVols is mostly under the covers in vSphere so it won’t be completely obvious that the capability is there. It is part of the same Storage Policy Based Management (SPBM) system in vSphere that VSAN uses and also presents itself as a new datastore type when you are configuring storage in vSphere.

6 – Let the array do the heavy lifting

Storage operations are very resource intensive and a heavy burden on the host. While server’s are commonly being deployed as SAN’s these days, a server really wasn’t built specifically to handle storage I/O like a storage array is. VMware recognized this early on which is why they created vSphere APIs for storage such as VAAI to offload resource intensive storage operations from the host to the storage array. VVols takes this approach to the next level, it shifts the management of storage tasks to vSphere and utilizing policy based automation storage operations are shifted to the storage array.

So things like thin provisioning and snapshots which can be done either on the vSphere side or the storage array side with VMFS are only done on the storage array side with VVols. How you do these things remains the same in vSphere but when you take a snapshot in the vSphere client you are now taking an array based snapshot. The VASA specification which defines VVols is basically just a framework and allows the array vendors to implement certain storage functionality, however they want to handle things within the array is up to each vendor. The storage array is a purpose built I/O engine designed specifically for storage I/O, it can do things faster and more efficiently, why not let it do what it does best and take the burden off the host.

5 – Start using SPBM right now

VSAN users have been enjoying Storage Policy Based Management for quite a while now and with VVols anyone with an external storage array can now use it as well. While Storage Policies have been around for even longer when first introduced in VASA 1.0, they were very basic and not all that useful. The introduction of VASA 2.0 in vSphere 6.0 was a big overhaul for SPBM and made it much more rich and powerful. The benefit of using SPBM is that makes it easier for vSphere and storage admins by automating storage provisioning and mapping storage array capabilities including features and hardware attributes directly to individual VMs. SPBM ensures compliance of defined policies so you can create SLA’s and align storage performance, capacity, availability and security features to meet application requirements.

4 – Snapshots don’t suck anymore

vSphere native VM snapshots have always been useful but they also have a dark side. One of the big disadvantages of vSphere snapshots is the commit process (deleting a snapshot) which can be very time and resource consuming, the more so the longer a snapshot is active. The reason for this is that when you create a vSphere snapshot, the base disk becomes read only and any new writes are deflected to delta vmdk files that are created for each VM snapshot. The longer a snapshot is active and the more writes that are made the larger the delta files grow, if you take multiple snapshots you create more delta files. When you delete a snapshot all those delta files have to be merged back into the base disk which can take a very long time and is resource intensive. As backup agents routinely take snapshots before doing a backup of a VM, snapshots are a pretty common occurrence.

With VVols the whole VM snapshot process changes dramatically, a snapshot taken in vSphere is not performed by vSphere but instead created and managed on the storage array. The process is similar in the fact that separate delta files are still created but the files are VVol snapshots that are array-based and more importantly what happens while they are active is reversed. When a snapshot of a VM on VVol-based storage is initiated in vSphere a delta VVol is created for each virtual disk that a VM has but the original disk remains Read-Write and instead the delta VVols contain any disk blocks that were changed while the snapshot is running. The big change occurs when we delete a snapshot, with VVols because the original disk is Read-Write, we can simply discard the delta VVols and there is no data to commit back into the original disk. This process can take milliseconds compared to minutes or hours that is needed to commit a snapshot on VMFS datastores.

The advantage of VVol array based snapshots are many, they run more efficiently on the storage array and you are not using up any host resources. In addition you are not waiting hours for them to commit, your backups will completed quicker and there is no chance of lost data from long snapshot chains trying to be written back into the base disk.

3 – Easier for IT generalists

Because storage array management shifts to vSphere through SPBM, IT generalists don’t really have to be storage admins as well. Once the initial array setup is complete and the necessary VVols components created, all the common provisioning tasks that you would normally do on the array to support vSphere are done through automation. No more creating LUNs, expanding LUNs, configuring thin provisioning, taking array based snapshots, etc., it’s all done automatically. When you create a VM, storage is automatically provisioned on the storage array and there is no more worrying about creating more LUNs, determining LUN sizes and increasing LUN sizes when needed.

With VVols you are only managing VMs and storage in vSphere, not in 2 places. As a result if you don’t have a dedicated storage admin to manage your storage array, an IT generalist or vSphere admin can do it pretty easily. Everything with SPBM is designed to be dynamic and autonomic and it really unifies and simplifies management in a good and efficient manner to reduce the overall burden of storage management in vSphere. Instead of having the added complexity of using vendor specific storage management tools, with VVols management becomes more simplified through the native vSphere interfaces.

2 – It unifies the storage protocols

VMFS and NFS storage are implemented pretty differently in vSphere and what VVols does is dictate a standardized storage architecture and framework across any storage protocol. There were several big differences with file and block implementations in vSphere, with block you presented storage to ESXi hosts as LUNs and with file as a mount point. You still do this to connect your array’s Protocol Endpoint to a host but the standard storage container that is presented to a host to store VMs is just that, a Storage Container, not a LUN or mount point anymore.

When it came to file systems with block an ESXI host would manage the file system (VMFS) and with file it was managed by the NAS array. With VVols there is no file system anymore, VVols are written natively to the array without a file system regardless of it’s a file or block array. And the biggest change is that VMs are written as files (VVols) to block storage in the same manner that file storage has been doing it all along. This creates a  unified and standardized storage architecture for vSphere and eliminates the many differences that existed before between file and block. VVols creates one architecture to bring them all and in the darkness bind them (LOTR reference).

1 – The VM is now a unit of storage

This is what it’s all about, the LUN is dead, because it’s all about the VM, ’bout the VM, no LUN. With VVols the VM is now a first class citizen and the storage array has VM-level visibility. Applications and VMs can be directly and more granularly aligned with storage resources. Where VMFS was very LUN-centric and static creating silos within an array, aligning data services to LUNs and utilizing pre-allocated and over provisioned resources. VVols is VM-centric and dynamic where nothing is pre-allocated, no silos are created and array data services are aligned at the more granular VM-level. This new operational model transforms storage in vSphere to simplify management, deliver better SLAs and provide much improved efficiency. VVols enables application specific requirements to drive provisioning decisions while leveraging the rich set of capabilities provided by storage arrays.

Share This:

Jun 16 2016

Observations and feedback on VMware VVols gathered from HPE Discover attendees

I was out at HPE Discover last week, I delivered a theater session on storage for vSphere which covered VVols and also staffed the VMware demo station which had a big focus on VVols. Throughout the event I probably spoke with at least 50 people and I wanted to share what I observed and heard from those conversations.

IMG_4306

Observation 1: Most people still don’t know what VVols is

I’d say 80% of the people that came to my demo station had heard of VVols but really did not have a basic understanding of what it is and the benefits that it provides. While my demo had a running vSphere environment with VVols configured, it seemed that the majority of my time with people seemed to be focused on presenting slides largely based on my VVols VMworld presentation last year to educate people on what VVols is. I probably spent at least 15-20 minutes per person explaining the concepts and benefits of VVols to people. This lack of knowledge seemed to exist both at the channel level and customer level.

Observation 2: Once people understood what VVols was all about they were excited about it

Almost everyone universally liked the benefits that VVols provides, the big ones were the changes to the snapshot mechanism, automated provisioning and reclamation, no more LUNs and silo’s, better efficiency and the ability to use storage policies.

Observation 3: Storage admins seemed OK with it but had some concerns

I talked to both sides of the fence, vSphere admins and storage admins and heard perspectives from each side. vSphere admins of course loved it as it empowers them with the ability to provision and manage storage resources. Most of the storage admins seemed OK with it as well despite giving up some level of control of provisioning and common storage tasks. A few concerns that storage admins had were around limiting VVol available space on the array, suppressing some of the array capabilities to vSphere and better visibility into VVol objects from the array side. There are some workarounds to address these that can be implemented on the array side.

Observation 4: While people thought the new snapshot mechanism is great, bulk snapshot management is not so great

With VVols there are two big changes to how snapshots work. The first is that there are no more vSphere managed snapshots, you still create snapshots the same way in the vSphere client but all VVol snapshots are actually array managed snapshots. The other big change is with the snapshot operation, with VMFS the base disk of a VM becomes read only and all changes are written to separate delta files. Once you delete a snapshot those delta files all have to be merged back into the base disk which is both resource and time intensive. With VVols the base disk remains read/write when a snapshot is taken, the delta files hold the original data when a change is made. When you delete a snapshot you can simply discard the delta files as the base disk already contains all the latest data, this is very quick and efficient.

Both these changes are great, now for the not so great, for people that want to do bulk VM snapshots (more than 1 VM), with VMFS you could do an array snapshot of the entire LUN, you can’t really do that with VVols though as each VVol is essentially its own LUN. You also can’t really do it in the vSphere Client either, you would have to snapshot each VM one-by-one which can be a pain in the butt especially if you are doing it to many VMs. You could try and script something with PowerCLI but it makes management more difficult. It would be nice if VMware could build a snapshot group feature into the vSphere client natively so you could take and manage snapshots of multiple VMs simultaneously.

Observation 5: People that are using VVols today had some concerns

I did run into a few people using VVols today and it ranged from some just testing it out to a few using it more large scale. Most people were OK with it but there were a few minor concerns. I had one person that had concerns around certificate expiration of the VASA Provider, if the cert were to expire your VASA Provider would essentially be unavailable which is not a good thing. You can manage certs for the VASA Provider on either the array side or the vSphere side, what they wanted to see is an alert mechanism/alarm that would let them know ahead of time so they were not surprised by a certificate expiring. Another concern was around protocol endpoints and the efficiency of using a single protocol endpoint on the storage array. I don’t feel this is a big concern area though as we have done extensive testing and found a single protocol endpoint to be sufficient. I brought them over to our product manager that had done some of the testing to get further re-assurance. The other concerns were around the lack of maturity of the new VASA spec (VVols 1.0) and lack of replication support. I think these will go away on their own with the next release of vSphere.

Observation 6: A lot of people are still on vSphere 5.x

I discovered a lot of people are still running vSphere 5.1 & 5.5 which prevents them from using VVols which requires vSphere 6.0. This also explains the lack of VVols understanding as it simply doesn’t exist in their world. The majority of them had upgrades planned in the next 6-9 months and they seemed excited to be finally able to start using VVols in their environment.

Share This:

Jan 18 2016

Customer adoption of VMware Virtual Volumes (VVols)

Tom Fenton recently published an article on Virtualization Review detailing the current state of VMware’s new Virtual Volume (VVol) storage architecture. In the article he polled a few vendors to find out what they are seeing as far as customer adoption of VVols. A few vendors responded including myself, both HDS & Dell did not have an accurate way to track adoption and where mainly relying on customer feedback. They are mainly seeing customers testing it out right now and using it in Dev/Test environments. HDS stated one of the limiters to VVol adoption is customers still on vSphere 5.5 and Dell stated customers are still trying to understand it better before diving in.

At HPE, we can track actual usage of VVol adoption via our array phone home capability which provides us with some usages stats on the array. In the article based on my feedback Tom wrote that we had seen at least 600 3PAR arrays with the VVols VASA Provider enabled within the array. More recent numbers puts that at around 720 arrays, but its important to note that this just means they have the potential to use VVols, not that they have VMs running on VVol storage. More detailed stats show that about 50 customers have created VMs on VVol storage. So this is pretty much inline with what other vendors are seeing which is pretty light adoption of VVols right now.

VVols has been available as part of vSphere 6 for almost a year now (March 2015), so why aren’t more people using it? There are probably a lot of reasons for this including:

  • Customers haven’t migrated to vSphere 6
  • Array firmware doesn’t support VVol
  • Lack of replication capabilities in VASA 2.0
  • Lack of knowledge/understanding of VVols
  • Limited scalability and feature support in some implementations
  • It’s essentially a 1.0 architecture

In my previous post on when customers would start adopting VVols I went into a lot more detail on the barriers/challenges to VVol adoption. I expect usage to pickup within the next year or so based on a number of factors:

  • VASA 3.0 with replication support in the next vSphere release
  • More arrays support for VVols
  • Increased scalability and more feature support
  • More mature implementation from VMware and array vendors
  • Better understanding of VVols and how to implement it

Until then I expect to see steadily increased usage of VVols, like any new technology or feature, adoption is almost always slow at first as customers are often cautious about jumping right in to something new. The same growing pains were apparent with VSAN as well when it was released as a 1.0 new storage architecture. If your array supports VVols I encourage you to definitely try it out and learn all you can about it as VVols is the future and at some point I expect VMFS to be phased out just like ESX was. If you are looking for resources to learn more about VVols be sure and check out my huge ever-growing VVols link collection and also my VMworld 2015 STO5888 session that VMware has made publicly available.

Share This:

Sep 29 2015

Want to learn more about VMware Virtual Volumes (VVols), here’s 3 great sessions to help you

VMware recently released about 50 VMworld 2015 sessions to the general public and within those there are 3 great sessions on VVols that will help you better understand the architecture and what VVols is all about. The sessions are all pretty technical which is good, one is from VMware and two of them are from storage partners (one of them is mine!).

The first session features Ken Werneburg and Patrick Dirks from VMware, Ken is a storage technical marketing guy and Patrick is on the engineering side so their is a lot of great technical deep dive content on VVols in this session.

STO4649 – Virtual Volumes Technical Deep Dive (Ken Werneburg, VMware – Patrick Dirks, VMware)

The second session features Andy Banta from SolidFire along with Ken Werneburg from VMware, it has a lot of cool real world analogies that explain how the components in VVols work.

STO5074 – Explaining Advanced Virtual Volumes Configurations (Ken Werneburg, VMware – Andy Banta, SolidFire)

The final session is mine and covers a wide range of topics on VVols including the architecture, benefits, migration, VAAI, implementation, thin provisioning and snapshots, backups and much more.

STO5888 – Top 10 Thing You MUST Know Before Implementing Virtual Volumes (Eric Siebert, HP)

And if after watching these you want to learn even more about VVols check out my huge Virtual Volumes link collection which also features links specific to each vendor.

 

Share This:

Sep 16 2015

New Virtual Volumes (VVols) technical papers from VMware

VMware has recently published 3 new technical papers that focus on their new Virtual Volumes (VVols) storage architecture that are definitely worth a read.

The first is a FAQ which does a good job summarizing important information around VVols and provides you a lot of quick facts around the architecture and implementation of VVols.

The next is a What’s New paper for Virtual Volumes, this one is about 6 months late as their is nothing new with VVols since the initial release in vSphere 6.0 in March but better late than never. This one is more a general tech paper on VVols and does an introduction to VVols and also covers the architecture and benefits of VVols.

The last one is a Getting Started Guide and is a walkthrough of how to use VVols from a vSphere perspective which covers things like how to make sure time is synced, creating a Storage Container, working with Storage Policies and assigning them to VMs. It’s designed to be used together with your array documentation that should cover the array side setup for VVols. The guide also covers some vSphere CLI commands for managing VVols components using escli.

Share This:

Jul 10 2015

Comparing Virtual Volume (VVol) limits to VMFS/NFS limits

I was going through some VVol documentation and found this comparison between VVol limits and VMFS/NFS limits in vSphere 6.0:

VMFS/NFS Limits#VVol Limits#
VMDK size64TBData VVol size64TB
Virtual Disks per host2,048VVols bound to a host4,096*
LUNs/NAS mounts per host256Protocol Endpoints per host256
Volume size64TBStorage Container size2 ^ 64**
Volumes per host256Storage Containers per host256
Adapter Queue depth32Adapter Queue depth32
Configured VASA Providers per host128
Configured VVol‐managed
storage arrays per ESXi host
64

* A host can see more than 4096 VVols, but can have only 4096 VVols bound at any given point in time (binding occurs when a VM is powered on)
** ridiculously large number

Some additional notes:

  • While multiple VVol Storage Containers are supported, it’s up to each vendor to decide what they want to support. Today many vendors only support a single Storage Container which encompasses an entire storage array.
  • While multiple VVol Protocol Endpoints are supported, it’s up to each vendor to decide what they want to support. Today most vendors only support a single Protocol Endpoint for the entire storage array.
  • The minimum size of a VVol is 1MB. Storage arrays must support at least 2TB VVols.
  • The maximum size of a data‐VVol is as large as whatever vSphere supports (62TB). The maximum size of a config‐VVol is currently 4GB. ESXi hosts will never try to create a virtual volume larger than what the array advertises as maximum.
  • The maximum number of VVols supported by a storage array is up to each vendor to decide what they want to support. The maximum number of VVols required by VMs in a cluster of ESXi hosts is the product of maximum number of virtual disks per VM (60), maximum number of snapshots per virtual disk (32), and maximum number of VMs per vCenter cluster (10,000). This make the theoretical maximum around 19 million total VVols.
  • The minimum number VVols a powered-on VM will have is 3 (config, swap, data) (swap goes away when VM is powered off). Each snapshot will add at least one additional VVol per virtual disk (plus an additional if memory state is selected). The maximum number of VVols a powered-on VM could have is around 2,000: 1 – config, 1 – swap, 60 – data, 1,920 – snapshots (60×32), 32 – memory state.
Share This:

Apr 24 2015

Another VVols webinar from Taneja Group featuring VMware speakers

Another VVols webinar coming up from Taneja Group who is doing a whole series of webinars on VVols, the first which was on April 2nd and I participated in was a panel discussion with several storage vendors (NetApp, HP, NexGen & Dell) giving their views and opinions on VVols. The next one on April 30th features two speakers from VMware: Juan Novella (Product Marketing Manager) and Ken Werneburg (Senior Technical Marketing Architect). It should be a good and informative session and it is the prelude to some additional Taneja deep dive webinars on VVols from individual vendors. I’ve signed on to represent HP at one which will be held in May. So click the link below and register and find out what VMware has to say about VVols.


Join us for the first in a series of fast-paced and informative 60-minute webinars, as we discuss with VMware one of the hottest topics in the datacenter: VMware vSphere Virtual Volumes (VVol). VVol is the industry’s first solution to enable native virtual machine-awareness across a broad range of SAN/NAS arrays. VVol is packaged as a feature in nearly all VMware vSphere Editions and is being embraced by storage partners at an unprecedented rate. IT professionals, especially those involved in datacenter operations, are showing great interest in implementing VVol in their own environments.

This moderated session features Juan Novella and Ken Werneburg, VVol experts from VMware, who will discuss VVol technology, the rapidly expanding ecosystem, and the impact this game-changing capability will have in the datacenter. Attendees will be encouraged to submit their questions during the session.

Panelists:
Juan Novella, Product Marketing Manager – Storage; VMware
Ken Werneburg, Senior Architect – Technical Marketing; VMware

VVols-disrupt

Share This:

Older posts «