Welcome to vSphere-land!

Today I wanted to highlight another white paper that I wrote for SolarWinds that is titled “Storage I/O Bottlenecks in a Virtual Environment”. I enjoyed writing this one the most as it digs really deep into the technical aspects of storage I/O bottlenecks. This white paper covers topics such as the effects of storage I/O bottlenecks, common causes, how to identify them and how to solve them. Below is an excerpt from this white paper, you can register and read the full paper over at SolarWinds website.

There are several key statistics that should be monitored on your storage subsystem related to bottlenecks but perhaps the most important is latency. Disk latency is defined as the time it takes for the selected disk sector to be positioned under the drive head so it can be read or written to. Once a VM makes a read or write to its virtual disk that request must follow a path to make its way from the guest OS to the physical storage device. A bottleneck can occur at different points along that path, there are different statistics that can be used to help pinpoint where the bottleneck is occurring in the path. The below figure illustrates the path that data takes to get from the VM to the storage device.

The storage I/O goes through the operating system as it normally would and makes its way to the device driver for the virtual storage adapter. From there it goes through the Virtual Machine Monitor (VMM) of the hypervisor which emulates the virtual storage adapter that the guest sees. It travels through the VMkernel and through a series of queues before it gets to the device driver for the physical storage adapter that is in the host. For shared storage it continues out the host on the storage network and makes its way to its final destination which is the physical storage device. Total guest latency is measured at the point where the storage I/O enters the VMkernel up to the point where it arrives at the physical storage device.

The total guest latency (GAVG/cmd as it is referred to in the esxtop utility) is measured in milliseconds and consists of the combined values of kernel latency (KAVG/cmd) plus device latency (DAVG/cmd). The kernel latency includes all the time that I/O spends in the VMkernel before it exits to the destination storage device. Queue latency (QAVG/cmd) is a part of the kernel latency but also measured independently. The device latency is the total amount of time that I/O spends in the VMkernel physical driver code and the physical storage device. So when I/O leaves the VMkernel and goes to the storage device this is the amount of time that it takes to get there and return. A guest latency value that is too high is a pretty clear indication that you have a storage I/O bottleneck that can cause severe performance issues. Once total guest latency exceeds 20ms you will notice the performance of your VMs suffer, as it approaches 50ms your VMs will become unresponsive.

Full paper including information on the key statistics related to storage I/O bottlenecks available here…

Continuing from my post on Monday which covered the white paper that I did for SolarWinds on the top 5 management challenges with virtualized environments, I wanted to highlight another of the white papers that focuses on one of those specific management challenges. This white paper is titled “Performance Management in a Virtual Environment” and covers how performance management differs from traditional physical environments, how to get started with performance management and knowing where to look and how to interpret the many statistics that are unique to virtual environments. The white paper also includes a table that details 13 key statistics that you need to pay attention to in virtual environments. Below is an excerpt from the first paper, you can register and read the full paper over at SolarWinds website.

So you’ve implemented virtualization and don’t know where to start when it comes to monitoring the performance of your virtual environment. In a traditional non-virtual environment you monitor performance through the guest operating system which is installed directly on the server hardware. Typically a centralized monitoring system relies on an agent installed on the guest OS or built-in components like Windows WMI to read performance statistics from the server. With virtualization this type of performance monitoring is no longer effective; the reason is the guest operating system is no longer seeing the physical hardware of the host. Instead it is seeing virtual hardware that is emulated by the hypervisor so performance statistics that are measured inside the guest OS are not an accurate reflection of the physical hardware of the host. As a result you need a monitoring application that is aware of the virtualization layer and can also measure the statistics that are unique to virtual environments.

Virtualization built-in performance monitoring tools like VMware’s vCenter Server can provide raw performance statistics for the virtual environment but doesn’t help you interpret them. The information returned by vCenter Server can be overwhelming and knowing what to look for and what the numbers mean can be difficult. Additionally vCenter Server is designed to mainly monitor and report at the virtualization layer and doesn’t extend to far into the guest OS layer so it does not provide a complete monitoring solution. There are hundreds of performance statistics that are generated by ESX/ESXi and vCenter Server that cover many different areas. Not all of these statistics are useful in most cases and if you tried to monitor them all you would be quickly overwhelmed. Some statistics are only useful in certain situations such as troubleshooting a resource bottleneck but there are others that can provide key indicators to the overall health of your vSphere environment and should be constantly monitored. Some statistics are specific to hosts and others only apply to virtual machines, the below table lists some of the more important statistics that you should focus on when monitoring vSphere.

Full paper including the table detailing key performance metrics available here…

I recently completed a series of technical papers for SolarWinds that highlight some of the challenges associated with virtualization. The first paper was titled “Top 5 Things You Need in a Virtualization Management Solution” and served as an introductory paper to the subsequent papers that cover the 5 things in more detail. Here are the 5 things that are covered:

• Performance Management
• Capacity Planning
• VM Sprawl
• Chargeback and Showback
• Storage I/O Bottlenecks

You might be used to vendor white papers that are focused on marketing and sales information and not much technical content. That’s not my writing style, my goal when writing white papers is to have at least 80% of the content be useful and educational content while still highlighting the vendors products in a more subtle manner.  Since my name is on the paper I want the reader to come away with both a thorough understanding of the topic from a technical perspective and also an understanding on how the vendor’s product relates to the topic. Below is an excerpt from the first paper, you can register and read the full paper over at SolarWinds website.

For any size data center, having a proper management solution is critical for ensuring an orderly, smooth running and problem free environment. Without one, your data center can quickly turn into a wild jungle with servers growing out of control without any regard for discipline or limits. A data center should be like a well-run city, with all parts of it communicating and interacting with each other. The mayor of that city should know about everything that happens within it so he can make the right decisions when needed to ensure everything runs smoothly. It shouldn’t be like a group of isolated villages that never speak to each other and have no idea what everyone else is doing. Having visibility into all aspects of your server environment is critical, but the amount of information that is monitored can easily be overwhelming, and not knowing how to interpret the information can make dealing with it difficult. Therefore, you need a management solution that can highlight the important information for you and also provide you with dashboards to make interpreting the information as easy as possible.

Virtualization technology has many benefits and most companies are now implementing it or planning to at some point. But the management of virtualized environments can be quite different from traditional physical environments, and the solutions designed to manage physical environments will not be as effective at managing a virtual environment. Therefore, implementing a management solution that is designed specifically for virtual environments is critical to ensure that you are monitoring the metrics and issues that are unique to virtual environments, and are able to keep up with the higher rate-of-change and can scale as your virtual environment grows.
In this white paper, we will look at the top 5 areas that you should be looking at when choosing a management solution for your virtual environment, and also cover the reasons why you need to use management solutions that are designed specifically for virtualization.

Full paper available here…

The polls will open next week for the annual VMware/virtualization top blog voting, so if you want to make sure your site is included make sure I have your blog listed on my vLaunchPad. This year will be a bit different, instead of just a top 25 we’ll also have categories kind of like they do at many awards shows. Some of the categories will be Best New Blog, Best Storage Blog, Blogger You Most Want To Meet, etc. I’m still playing with the categories so sound off in the comments if you have any ideas.

It’s always difficult to say goodbye, especially when you’ve been doing something you enjoy for many years.

I started writing for SearchVMware.com more than three years ago, in March 2008. Up to that point I had no writing experience, and I had never really thought of myself as a writer. On a whim, I responded to a post on TechTarget’s website about writing opportunities. They gave me a shot, and I found that I was pretty good at it, and since then I’ve written more than 100 articles on SearchVMware.com and more than 100 posts for the Virtualization Pro blog. My writing at TechTarget got me a lot of exposure and opportunities over the years, and it really helped my career develop and blossom.

Looking back at my writing over the years, I thought I would highlight some of the articles that I enjoyed doing the most and am most proud of. I tend to write pretty detailed and lengthy articles, because I like to give complete coverage to a product or feature, and as a result many of my articles were turned into series. My very first assignment was to write about VMware Converter, which ended up being a three-part series. I was a little green starting out, but one of the nice things about having editors is that they take your work and make it look even better.

Read the full article at searchvmware.com…

While part one of this vSphere 5 upgrade series dealt with the challenges that solution providers may need to deal with during a migration, but there are also benefits. Because of their simplistic nature and improved capabilities, the vSphere 5 features listed below can be assets during a customer upgrade:

vCenter Server appliance
VSphere 5 supports running vCenter Server as a pre-built Linux virtual appliance. This makes deploying and maintaining vCenter Server much easier and also means it’s no longer required to run it on a server running a Windows operating system (OS). The virtual appliance comes packaged with IBM’s DB2 Express and also supports only Oracle or DB2 for external databases. This will appeal to customers that mainly use Linux because vCenter Server doesn’t require the use of Microsoft products.

Solution providers can use a Web user interface (UI) for configuration, and it’s compatible with the new Flex based Web UI that is part of vSphere 5. Flex is a framework for web development from Adobe that enables rich functionality for Web browsers. This allows for better Web administration UI’s to be created so VMware can mimic the Windows based vSphere Client’s functionality through a Web browser. The previous web UI in vSphere 4 just used basic HTML and was not as feature-heavy as the new Flex based Web UI in vSphere 5.

Read the full article at searchsystemschannel.com…

Because of its new features and enhancements, many of your customers are likely anxious to start planning their vSphere 5 upgrade, and it’s important to know new caveats and issues before you let your customers dive right into it.

Solution providers can ensure a smooth transition to vSphere 5 by taking note of these important modifications and understanding what they mean to individual customer environments:

New licensing
Perhaps the biggest vSphere 5 deviation from previous versions is that your customers will need to deal with the new licensing model.

VSphere 4 licensing was fairly straightforward: as licenses were bought per CPU socket and you could run unlimited virtual machines (VMs) on the host. That model is no more with vSphere 5, and while licenses are still sold per CPU socket, each license comes with a fixed amount of virtual RAM (vRAM) that can be assigned to VMs. Depending on the environment, this could cause a customer to spend thousands of dollars in additional licenses to be compliant with new vSphere 5 licensing. The new model favors scale-out architectures that hold a greater amount of hosts that have smaller resource amounts. This means less VMs running on each host, which results in less vRAM usage per host. When you try and scale up with hosts that have large amounts of RAM, the additional license costs to match the amount of that RAM in a host can get very costly.

Read the full article at searchsystemschannel.com…

You learned about VMware Auto Deploy’s benefits in part one of our two-part series, but it’s also important to know the vSphere 5 feature’s nuts and bolts and how to set it up for customers.

Auto Deploy takes advantage of the Preboot Execution Environment (PXE) boot feature that is present in many physical network interface cards (NICs). This allows a server to boot from a remote image file using only a physical NIC and without local storage.

A server booting with PXE first obtains an IP address using Dynamic Host Configuration Protocol (DHCP) and then loads a boot image from a Trivial File Transfer Protocol (TFTP) server. Auto Deploy uses PXE booting to download an ESXi image file to a host and its components work to define which image a host should use, customizations and host-specific configuration information.

Auto Deploy relies on software depots that are used to store collections of vSphere Installation Bundles (VIB) and image files that are accessed remotely via HTTP to deploy or update hosts. VIB files are used to deploy the ESXi software and any hardware vendor customizations. The Auto Deploy Server uses the software depot to pull VIBs and image profiles when booting an ESXi host.

Read the full article at searchsystemschannel.com…

Although creating new hosts is a common server virtualization task, it is also a tedious process. Solution providers can use VMware Auto Deploy for vSphere 5 to do it much more efficiently.

Installing single hosts isn’t hard for solution providers but configuring multiple hosts can be time consuming. A VAR that has to create a new vSphere host without VMware Auto Deploy typically has to do the following:

1. Locate the installation media for ESX or ESXi and boot the server from it.

2. Follow the setup prompt to set configuration information.

3. Complete the installation and then reboot the server.

4. Verify that management console networking works properly.

5. Add the host to vCenter Server.

6. Configure networking, storage, security and other settings.

In doing all this, you risk making mistakes during the build process and could configure hosts inconsistently. Consistency is very important in a virtual environment both from a security and operational perspective. Once you’ve built and configured a host, another challenge is to back up host configuration data so if a problem occurs and you need to rebuild the host, you don’t have to restart from scratch.

Read the full article at searchsystemschannel.com…

The release of VMware Inc.’s vSphere 5 brings many exciting new features and enhancements to the virtualization platform, especially when it comes to storage. Two of the biggest new features in that area are Storage Distributed Resource Scheduler (DRS) and Profile-Driven Storage, which provide some much-needed control over storage resources.

In previous versions of vSphere, Distributed Resource Scheduler balanced VM workloads based on CPU and memory resource utilization. Storage DRS extends this capability to storage, enabling intelligent VM initial placement and load balancing based on storage I/O and capacity conditions within a cluster. Profile-Driven Storage, for its part, ensures that VMs are placed on storage tiers based on service-level agreements (SLAs), availability, performance and capabilities of the underlying storage platform. In this tip, we’ll examine both Storage DRS and the storage profile functionality in detail.

Storage DRS

Similar to the traditional DRS feature, Storage DRS uses a new type of cluster called a data store cluster, which is a collection of data stores that are aggregated into a single unit of consumption. By controlling all of the storage resources, Storage DRS allows intelligent placement of VMs that are powered on, as well as the shifting of workloads from one storage resource to another when needed to ensure optimum performance and avoid I/O bottlenecks. What this means in simpler terms is that, similar to vMotion’s movement of VMs from host to host, VMs can now be moved from data store to data store as well; the decision to move a VM from one data store to another is made by Storage DRS, which tells Storage vMotion to make the move.

Read the full article at searchvirtualstorage.com…

Selecting a virtualization hypervisor begins with an important choice: Do you need a hosted or bare-metal hypervisor? Once you decide which type of hypervisor you need, there are lots of factors to consider.

You want a virtualization hypervisor that’s compatible with your hardware, allows for simple management and gives you the performance your virtual infrastructure needs. You should also consider high availability, reliability and scalability. And of course, look into costs.

Here are eight considerations for choosing a virtualization hypervisor:

Performance
If you want high performance, a bare-metal virtualization hypervisor is really your only option. Bare-metal virtualization offers the least amount of resource overhead. Bare-metal virtualization hypervisors also have advanced resource controls that allow you to guarantee, prioritize and limit virtual machine (VM) resource usage.

Hosted hypervisors typically have no or limited resource controls, so VMs have to fight each other for resources. Unlike bare-metal virtualization, hosted hypervisors often have steep resource-overhead penalties, especially when operating system services, tools and applications are running on the guest operating system.

Read the full article at searchservervirtualization.com…

Once you choose the type of hypervisor that fits your needs, you need to choose the best hypervisor technology for your infrastructure. Hypervisor products from the major virtualization vendors have their pros and cons, so consider the features of these hosted and bare-metal virtualization hypervisors before you make your decision.

Bare-metal virtualization hypervisors

VMware ESX and ESXi
VMware has the most mature hypervisor technology by far, offering advanced features and scalability. However, VMware’s bare-metal virtualization hypervisor can be expensive to implement because of its higher licensing costs. The vendor does offer a free version of ESXi, but it’s very limited and has none of the advanced features of the paid editions. VMware also offers lower-cost bundles that can make hypervisor technology more affordable for small infrastructures.

Microsoft Hyper-V
Microsoft Hyper-V has emerged as a serious competitor to VMware ESX and ESXi. Hyper-V lacks many of the advanced features that VMware’s broad product line provides, but with its tight Windows integration, Microsoft’s hypervisor technology may be the best hypervisor for organizations that don’t require a lot of bells and whistles.

Reach the full article at searchservervirtualization.com…

VMware Tools are an optional set of drivers and utilities that improve the performance and manageability of virtual machines. But there’s a debate about whether the benefits of installing VMware Tools outweigh the potential virtual security risks that it introduces.

On one hand, VMware Tools replaces many of the guest operating system drivers that were designed for physical hardware. These optimized drivers can drastically improve performance and functionality (e.g., providing copy-and-paste capabilities between the host and VM). But the installation of VMware Tools also adds potential virtual security vulnerabilities to an otherwise secure infrastructure.

In this face-off, two experts debate the merits of running VMware Tools.

Creating unnecessary virtual security risks with VMware Tools
vs.
Realizing a virtual infrastructure’s potential with VMware Tools

Read the full article at searchvmware.com…

The iPad is becoming more and more popular in the enterprise, and not just for mobile workers. There is also a slew of iPad applications for VMware admins.

Many IT vendors see the iPad’s potential and are developing iPad apps that can manage their traditional hardware and software products. Xsigo Systems, for instance, has a very nice app called Xsigo XMS , which manages virtual I/O through the company’s XMS servers. There is also an iPad application called SiMU Pro that manages Cisco Systems Inc.’s Unified Computing System.

In addition, there are several iPad applications that can supplement the traditional VMware admin toolkit, including the vSphere Client and Secure Shell (SSH) applications. With the right iPad applications, VMware admins will reach a new level of management flexibility that’s not possible with traditional desktops and laptops.

TABLE OF CONTENTS

I. Top iPad applications for VMware management

II. Applications for remotely connecting to hosts and workstations

III. Top iPad applications for VMware networking

IV. General purpose iPad apps for VMware admins

I. TOP IPAD APPLICATIONS FOR VMWARE MONITORING AND MAINTENANCE
With the top iPad applications for infrastructure management, VMware admins can control basic functions, such as powering virtual machines (VMs) on and off and using vMotion.

These iPad apps mimic some of the functionality of the vSphere Client and service console, but they aren’t a full-fledge replacement. Even so, these iPad applications allow VMware admins to perform key virtualization tasks without a full-scale computer.

Read the full article at searchvmware.com…

VMware has released the Licensing Advisor tool that I talked about in this post for you to download and run in your environment to see how it complies with the new licensing model in vSphere 5. The download link and FAQ for the tool are available on VMware’s website.