Maximum Type Category vSphere 5.1 vSphere 5.5
Virtual disk size VM Storage 2TB minus 512 bytes 62TB
Virtual SATA adapters per VM VM I/O Devices NA 4
Virtual SATA devices per virtual SATA adapter VM I/O Devices NA 30
xHCI USB controllers VM I/O Devices 1 NA
Logical CPUs per host Host Compute 160 320
NUMA Nodes per host Host Compute 8 16
Virtual CPUs per host Host Compute 2048 4096
Virtual CPUs per core Host Compute 25 32
RAM per host Host Compute 2TB 4TB
Swap file size Host Compute 1TB NA
VMFS5 - Raw Device Mapping size (virtual compatibility) Host Storage 2TB minus 512 bytes 62TB
VMFS5 - File size Host Storage 2TB minus 512 bytes 62TB
e1000 1Gb Ethernet ports (Intel PCI‐x) Host Networking 32 NA
forcedeth 1Gb Ethernet ports (NVIDIA) Host Networking 2 NA
Combination of 10Gb and 1Gb Ethernet ports Host Networking Six 10Gb and Four 1Gb ports Eight 10Gb and Four 1Gb ports
mlx4_en 40GB Ethernet Ports (Mellanox) Host Networking NA 4
SR-IOV Number of virtual functions Host VMDirectPath 32 64
SR-IOV Number of 10G pNICs Host VMDirectPath 4 8
Maximum active ports per host (VDS and VSS) Host Networking 1050 1016
Port groups per standard switch Host Networking 256 512
Static/Dynamic port groups per distributed switch Host Networking NA 6500
Ports per distributed switch Host Networking NA 60000
Ephemeral port groups per vCenter Host Networking 256 1016
Distributed switches per host Host Networking NA 16
VSS portgroups per host Host Networking NA 1000
LACP - LAGs per host Host Networking NA 64
LACP - uplink ports per LAG (Team) Host Networking 4 32
Hosts per distributed switch Host Networking 500 1000
NIOC resource pools per vDS Host Networking NA 64
Link aggregation groups per vDS Host Networking 1 64
Concurrent vSphere Web Clients connections to vCenter Server vCenter Scalability NA 180
Hosts (with embedded vPostgres database) vCenter Appliance Scalability 5 100
Virtual machines (with embedded vPostgres database) vCenter Appliance Scalability 50 3000
Hosts (with Oracle database) vCenter Appliance Scalability NA 1000
Virtual machines (with Oracle database) vCenter Appliance Scalability NA 10000
Registered virtual machines vCloud Director Scalability 30000 50000
Powered-On virtual machines vCloud Director Scalability 10000 30000
vApps per organization vCloud Director Scalability 3000 5000
Hosts vCloud Director Scalability 2000 3000
vCenter Servers vCloud Director Scalability 25 20
Users vCloud Director Scalability 10000 25000
If you look at the VM resource configuration maximums over the years there have some big jumps in the sizes of the 4 main resource groups with each major vSphere release. With vSphere 5.0 the term “Monster VM” was coined as the number of virtual CPUs and the amount of memory that could be assigned to a VM took a big jump from 8 vCPUs to 32 vCPUs and from 255GB of memory to 1 TB of memory. The number of vCPUs further increased to 64 with the vSphere 5.1 release.
One resource that has stayed the same over the releases going back all the way to vSphere 3.0 (and beyond?) is the virtual disk size which has been limited to 2TB. With the release of vSphere 5.5, the maximum vCPUs and memory has stayed the same but the size of a virtual disk has finally been increased to 62TB. That increase has been long awaited as previously the only way to use bigger disks with a VM with using a RDM. So while the Monster VM may have not had it’s brains increased in this release, the amount it can pack away in its belly sure got a whole lot bigger!
vSphere 5.5 is now available for your downloading pleasure here, while you’re waiting for it to download be sure and check out my huge (and growing) vSphere 5.5 link collection to get you all the info you need to know how to use it. This is also the shortest release cycle from the previous version that VMware has done to date at 377 days from the release of vSphere 5.1. Also don’t forget the documentation, the first document I always check out is the latest Configuration Maximums document to see how much larger everything continues to grow.
Many virtual environments lack consistency and uniformity and are often made up of a diverse mix of hardware and software components. This diversity could include things like multiple vendors for hardware components as well as hypervisors from different vendors. Since budgets play a big factor in determining the hardware and software used in a virtual environment, often times there will be variety of equipment from different vendors. In other cases customers want to use best of breed hardware and software which typically means having to go to different vendors to get the configuration they desire.
For hardware it’s not uncommon to see different hardware vendors used across servers, storage and networking, and sometimes different hardware vendors within the same resource group (i.e. storage). When it comes to hypervisor software the most commonly used software is VMware vSphere but it’s not uncommon to see Microsoft’s Hyper-V inside the same data center as a majority of customers use Microsoft for their server OS which gives them access to Hyper-V. Another common mix for client virtualization is running Citrix XenDesktop on top of VMware vSphere. Whatever the reasons are for choosing different vendors to build out a virtual environment, it’s rare to find a datacenter that has the same brand hardware across servers, storage and networking and only a single hypervisor platform running on it.
As a result of this melting pot of different hardware and hypervisors, managing these heterogeneous virtual environments can become extremely difficult as management tools tend to be specific to a hardware or hypervisor platform. Having so many management tool silos can add complexity, increase administration overhead and increase costs. To help offset that we’ll cover 5 tips for managing heterogeneous virtual environments so you can do it more effectively.
1 - Group similar hardware together for maximum effectiveness
If you are going to use a mix of different brands and hardware models you can’t just throw it all together and expect it to work effectively and efficiently. Every hardware platform has its own quirks and nuances and because virtualization is very picky about physical hardware you should group similar hardware together whenever possible. When it comes to servers this is usually done at the cluster level so features like vMotion and Live Migration that move VMs from host to host can ensure CPU compatibility. Because AMD & Intel CPUs use different architectures you cannot move a running VM across hosts with different processor vendors. There are also limitations on doing this within processor families of a single CPU vendor that you need to be careful of. Some of these can be overcome using features like VMware’s Enhanced vMotion Compatibility (EVC) but it can still cause administration headaches.
Shared storage you can intermix more easily as many storage features built into the hypervisor like thin provisioning and Storage vMotion will work independently of the underlying physical storage hardware. With storage you can use different vendor arrays side by side in a virtual environment without any issues but you should be aware of the differences and limitations between file-based (NAS) and block-based (SAN) arrays. You can use file and block arrays together in a virtual environment but due to protocol and architecture differences you may run into issues with feature support and integration across file and block arrays. Therefore you should try and group similar storage arrays together as well so you can get take advantage of features and integration that may only work with within a storage array product family and within a storage protocol. By doing some planning up front and grouping similar hardware together you can increase efficiency, improve management and avoid incompatibility problems.
2 – Try and stick to one hypervisor platform for your production environment
I’ve seen surveys that say more than 65% of companies that have deployed virtualization are using multiple hypervisors. That number seems pretty high and what I question is how those companies are deploying multiple hypervisors. Someone taking a survey that has 100 vSphere hosts and 1 Hyper-V host can say they are running multiple hypervisors but the reality is they are using one primary hypervisor. When you start mixing hypervisors together for whatever reason you run into all sorts of issues that can have a big impact on your virtual environment. The biggest issues with this tend to be in these areas:
- Training – You have to make sure your staff is continually trained on 2 different hypervisor platforms
- Support – You need to have support contracts in place for 2 different hypervisor platforms
- Interoperability – There is some cross hypervisor interoperability between hypervisors using conversion tools but since they use different disk formats this can be cumbersome
- Management – Both VMware & Microsoft have tried to implement management of the other platform into their native management tools but it is very limited and not all that usable
- Costs – You are doubling your costs for training, support, management tools, etc
As a result it makes sense to use one hypervisor as your primary platform for most of your production environment. It’s OK to do one off’s and pockets here and there with a secondary hypervisor but try and limit that.
3 – Make strategic use of a secondary hypervisor platform
Sometimes it may make sense financially to intermix hypervisor platforms in your data center for specific use cases. Using alternative lower cost or free hypervisor platforms can help increase your use of virtualization while keeping costs down. If you do utilize a mix of hypervisors in your data center do it strategically to avoid complications and some of the issues I mentioned in the previous tip. Here are some suggestions for strategically using multiple hypervisor platforms together:
- Use one primary hypervisor platform for production and a secondary for your development and test environments. You may consider using the same hypervisor for production and test though so you can test for issues that may occur from running an application on a specific hypervisor before it goes into production.
- Create tiers based on application type and how critical it is. This can be further defined by the level of support that you have for each hypervisor. If you have 24×7 support on one and 9×5 support on another you’ll want to make sure you have all your critical apps running on the hypervisor platform with the best support contract.
- If you have remote offices you might consider having a primary hypervisor platform at your main site and using an alternate hypervisor at your remote sites.
These are just some of the logical ways to divide and conquer using a mix of hypervisors, you may find other ways that work better for you that gives you the benefits of both platforms without the headaches.
4 – Understand the differences and limitations of each platform
No matter if you’re using different hardware or hypervisor platforms you need to know what the capabilities and limitations of each platform which can impact availability, performance and interoperability. Each hypervisor platform tends to have its own disk format so you cannot easily move VMs across hypervisor platforms if needed. When it comes to features there are usually some requirements around using them and despite being similar they tend to be proprietary to each hypervisor platform.
When it comes to hardware CPU compatibility amongst hosts is a big one because moving a running VM from one host to another using vMotion or Live Migration requires both hosts to have CPUs that are the same manufacturer (i.e. Intel or AMD) as well as architecture (CPU family). With hypervisors there are some features unique to particular hypervisor platforms like the power saving features built into vSphere. Knowing the capabilities and limitations of the hardware and hypervisors that you use can help you strategically plan how to use them together more efficiently and help avoid compatibility problems.
5 – Leverage tools that can manage your environment as a whole
Managing heterogeneous environments can often be quite challenging as you have to switch between many different management tools that are specific to hardware, applications or a hypervisor platform. This can greatly increase administration overhead and decrease efficiency as well as limit the effectiveness of monitoring and reporting. When you have management silos in a data center you lack the visibility across the environment as a whole which can create unique challenges as virtual environments demand unified management. Compounding the problem is the fact that native hypervisor tools are only designed to manage a specific hypervisor so you need separate management tools for each platform. Some of the hypervisor vendors have tried to extend management to other hypervisor platforms but they are often very limited and more designed to help you migrate from a competing platform. The end result is a management mess that can cause big headaches and fuels the need for management tools that can operate at a higher level and that can bridge the gap between management silos.
SolarWinds delivers management tools that can stretch from apps to bare metal and can cover every area of your virtual environment. This provides you with one management tool that can manage multiple hypervisor platforms and also provide you with end to end visibility from the apps running in VM’s to the physical hardware that they reside on. Tools like SolarWinds Virtualization Manager deliver integrated VMware and Microsoft Hyper-V capacity planning, performance monitoring, VM sprawl control, configuration management, and chargeback automation; all in one affordable product that’s easy to download, deploy, and use. Take it even further by adding on with SolarWinds Storage Manager and Server and Application Monitor and you have a complete management solution from a single vendor that covers all your bases and creates a melting pot for your management tools to come together under a unified framework.
VMware recently updated a paper that covers Best Practices for running vSphere on iSCSI storage. This paper is similar in nature to the Best Practices for running vSphere on NFS paper that was updated not too long ago. VMware has tried to involve their storage partners in these papers and reached out to NetApp & EMC to gather their best practices to include in the NFS paper. They did something similar with the iSCSI paper by reaching out to HP and Dell who have strong iSCSI storage products. As a result you’ll see my name and Jason Boche’s in the credits of the paper but the reality is I didn’t contribute much to it besides hooking VMware up with some technical experts at HP.
So if you’re using iSCSI be sure and give it a read, if you’re using NFS be sure and give that one a read and don’t forget to read VMware’s vSphere storage documentation that is specific to each product release.
With vSphere 5.5 due out sometime later this month I thought I’d look back at the VMware release cycles for new versions over the years. If you look at the number of days that it took VMware to release a new major version of ESX/ESXi you’ll see that has been steadily dropping. With increased competition and more widespread adoption of virtualization gone are the days where VMware had the luxury of time to get new product versions out the door.
With such a short release cycle of just about 1 year now it must be a grueling task for VMware to maintain that pace especially with all the interoperability and compatibility between there ever growing product portfolio that they must maintain whenever they release a new version. It also makes it challenging for vendors to keep pace with VMware and support new features & APIs as well as major architectural changes like the end-of-life of the ESX hypervisor and the classic vSphere client.
You also have to be concerned about quality with such a fast pace, overall VMware has done a fairly decent job in that area but they have been bit in the butt a few times like with the infamous time-bomb bug. VMware will most likely stick with their one year release schedule that they have settled in to as it aligns with VMworld and trying to run any faster would likely be too much for them to handle.
Also check out my complete table of all the VMware release & build information.
ESX/ESXi version Release Date Days from prior release
2.5 11/29/2004 -
3.0 6/15/2006 563
3.5 12/10/2007 543
4.0 5/21/2009 528
4.1 7/13/2010 418
5.0 8/24/2011 407
5.1 9/10/2012 383
5.5 9/22/2013 377
Your complete guide to all the essential vSphere 5.5 links. Bookmark this page and keep checking back as it will grow as new links are added.
VMware vSphere 5.5 Product Feature Interactive Walkthroughs - (VMware)
VMware Compatibility Guide (VMware)
VMware Product Interoperability Matrixes (VMware)
VMware vSphere 5.5 Release Notes (VMware)
VMware What’s New Links
vSphere 5.5 Product Documentation — PDF and E-book Formats (vmware.com)
vSphere 5.5 Configuration Maximums (vmware.com)
vSphere Installation and Setup (vmware.com)
vSphere Upgrade (vmware.com)
vSphere vCenter Server and Host Management (vmware.com)
vSphere Virtual Machine Administration (vmware.com)
vSphere Host Profiles (vmware.com)
vSphere Networking (vmware.com)
vSphere Storage (vmware.com)
vSphere Security (vmware.com)
vSphere Resource Management (vmware.com)
vSphere Availability (vmware.com)
vSphere Monitoring and Performance (vmware.com)
vSphere Single Host Management (vmware.com)
vSphere Troubleshooting (vmware.com)
Main vSphere 5.5 download (vmware.com)
ESXi 5.5.0 (vmware.com)
vCenter Server 5.5.0 (vmware.com)
vSphere Replication 5.5 (vmware.com)
vSphere Data Protection 5.5.1 (vmware.com)
vCenter Orchestrator Appliance 5.5.0 (vmware.com)
Cisco Nexus 1000V Virtual Ethernet Module for vSphere 5.5.0 (vmware.com)
vCloud Networking and Security 5.5.0 (vmware.com)
vCenter Operations Manager Foundation 5.7.2 (vmware.com)
vSphere Big Data Extensions 1.0 (vmware.com)
vSphere App HA 1.0.0 (vmware.com)
vSphere Management Assistant 5.5 (vMA) (vmware.com)
vSphere PowerCLI 5.5 (vmware.com)
vSphere CLI 5.5 (vmware.com)
ESXi 5.5 Free Version – no more hard limit 32GB of RAM (ESX Virtualization)
Watch out with Hardware Version 10 in VMware ESXi 5.5! (Ivobeerens.nl)
Back To Basics: Managing VMware ESXi 5.5 Direct User Interface (DCUI) (Mike Laverick)
The vSphere Hypervisor 5.5-Web Client-HW Version 10-dilemma (running-system.com)
VMware vSphere 5.5 Physical Host Maximums (TechHead)
ESXi 5.5 introduces a new Native Device Driver Architecture Part 1 (Virtually Ghetto)
ESXi 5.5 introduces a new Native Device Driver Architecture Part 2 (Virtually Ghetto)
How to run Nested ESXi on top of a VSAN datastore? (Virtually Ghetto)
w00t! VMware Tools for Nested ESXi! (Virtually Ghetto)
Why is Promiscuous Mode & Forged Transmits required for Nested ESXi? (Virtually Ghetto)
The Good and the Bad of the new Native Driver Architecture in ESXi 5.5 (VMware Front Experience)
vSphere 5.5 Improvements Part 1 - The New Hotness in ESXi 5.5 (Wahl Network)
What’s New in vCloud Suite 5.5: vCenter Server and ESXi (WoodITWork)
The Monster VM didn’t get any smarter in vSphere 5.5 but it got a whole lot fatter (vSphere-land)
Configuration maximums that changed between vSphere 5.1 and 5.5 (vSphere-land)
No vMSC certification for vSphere 5.5 yet (vSphere-land)
Top 5 big & little enhancements in vSphere 5.5 (vSphere-land)
vSphere 5.5 - what I like from what’s new (Definit)
VMware Releases vSphere 5.5 – What’s New? (Jason Nash’s Blog)
vSphere 5.5 First Impressions (M80ARM)
VMworld 2013: VMware Announces vSphere 5.5 and vCloud Suite 5.5 (Petri)
Top 5 Features of VMware vSphere 5.5 (Petri)
vSphere 5.5: vSphere Ruby Console (Punching Clouds)
Read the vSphere 5.5 Release Notes!! (Technodrone)
What is new in VMware vSphere 5.5 (UP2V)
An overview of all VMworld 2013 announcements (UP2V)
vSphere 5.5 What’s New at VMworld 2013, NSX, SDDC, VSAN, vFlash (vClouds)
vSphere 5.5 My Top 10 New Features (vClouds)
vSphere 5.5 and VMworld Updates (Virtualised Reality)
What’s New in vSphere 5.5: Configs, The Web Client, vSANs (Virtualization Review)
Top 10 Most Important Features in vSphere 5.5 (Virtualization Software)
Whats new in vSphere 5.5 (Virtualize Tips)
VMTN Communities Roundtable Podcast - ESXi 5.5 What’s New? (VMTN Podcast)
A Summary of What’s New in vSphere 5.5 (VMware vSphere Blog)
What’s New in vSphere 5.5 Platform-Quick Reference (VMware Tech Paper)
vSphere 5.5 Improvements Part 2 - Pushing For A Software-Defined Data Center (SDDC) (Wahl Network)
What’s New in vCloud Suite 5.5: Introduction (WoodITWork)
vSphere 5.5: platform scalability (Yellow Bricks)
HA & DRS
VMware vSphere 5.5 Application High Availability – AppHA (ESX Virtualization)
vSphere App HA - Closer… But Not Yet (Technodrone)
An introduction to VMware AppHA (UP2V)
vSphere 5.5 Application HA, Advanced App Monitoring explained (vClouds)
A vSphere 5.5 Gem: Application High Availability (Virtualization Review)
High Availability in vSphere 5.5 Series – VMware vSphere App HA (Virtualization Software)
What’s New in vCloud Suite 5.5: vSphere App HA (WoodITWork)
What’s new in vSphere 5.5 for DRS? (Yellow Bricks)
vSphere 5.5 nuggets: High Availability Enhancement (Yellow Bricks)
Installing & Upgrading
vSphere 5.5 Install Pt. 1: Introduction (Derek Seaman)
vSphere 5.5 Install Pt. 2: SSO Reborn (Derek Seaman)
vSphere 5.5 Install Pt. 3: Upgrading vCenter (Derek Seaman)
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade (Derek Seaman)
vSphere 5.5 Install Pt. 5: SSL Deep Dive (Derek Seaman)
vSphere 5.5 Install Pt. 6: Certificate Template (Derek Seaman)
vSphere 5.5 Install Pt. 7: Install SSO (Derek Seaman)
vSphere 5.5 Install Pt. 8: Online SSL Minting (Derek Seaman)
vSphere 5.5 Install Pt. 9: Offline SSL Minting (Derek Seaman)
vSphere 5.5 Install Pt. 10: Replace SSO Certs (Derek Seaman)
vSphere 5.5 Install Pt. 11: Install Web Client (Derek Seaman)
vSphere 5.5 Install Pt. 12: Configure SSO (Derek Seaman)
vSphere 5.5 Install Pt. 13: Install Inventory Svc (Derek Seaman)
vSphere 5.5 Install Pt. 14: Create Databases (Derek Seaman)
vSphere 5.5 Install Pt. 15: Install vCenter (Derek Seaman)
vSphere 5.5 Install Pt. 16: vCenter SSL (Derek Seaman)
vSphere 5.5 Install Pt. 17: Install VUM (Derek Seaman)
vSphere 5.5 Install Pt. 18: VUM SSL & Misc. Config (Derek Seaman)
Installation of ESXi 5.5 in VMware Workstation (ESX Virtualization)
vCenter Server Appliance 5.5 (vCSA) – Installation and Configuration Video (ESX Virtualization)
VCSA 5.5 Installation and configuration – Part 2 (ESX Virtualization)
Installing and upgrading to ESXi 5.5 – Best practices and tips (ESX Virtualization)
vSphere 5.5 Host Upgrade using vSphere Update Manager (VUM) (Everything Should Be Virtual)
Back To Basics: Install VMware ESX 5.5 (Mike Laverick)
[Lab Post] Installing vCenter Server 5.5 on Windows Server 2012 R2 (mtellin.com)
Upgrade to vSphere 5.5? Take care about your backup software! (running-system.com)
Installing vSphere 5.5 tips and tricks pt 1 (SOSTech)
Installing vSphere 5.5 tips and tricks pt 2 (SOSTech)
Install and configure Orchestrator 5.5 (SOSTech)
Notes on Upgrading the VMware vCenter Server Appliance 5.1 to 5.5 (The Lone Sysadmin)
How to Upgrade the VMware vCenter Server Appliance 5.5 (The Virtual Way)
Quick Tip: Using the CLI to upgrade to a specific VM virtual hardware version in vSphere 5.5 (Virtually Ghetto)
How to upgrade to vSphere 5.5 (VMGuru.nl)
How to update your standalone host to ESXi 5.5 - beware of missing drivers and do NOT upgrade virtual hardware! (VMware Front Experience)
Update sequence for vSphere 5.5 and its compatible VMware products (2057795) (VMware KB Article)
Installing vCenter Server 5.5 best practices (VMwareKB TV)
Installing or upgrading ESXi 5.5 best practices (VMwareKB TV)
Installing VMware ESXi 5.5 using the Interactive Installer (VMwareKB TV)
Installing VMware vSphere 5.5 on a Windows platform using the Simple Install method (VMwareKB TV)
vCenter Server 5.5 – Simple vs Custom Installations (VMware vSphere Blog)
Upgrading vSphere vCenter Appliance 5.1 to 5.5 (vNinja)
Quick and Dirty ESXi 5.5 Upgrade (vNinja)
vSphere 5.5 is here! – KBs you need to know about (VMware Support Insider Blog)
Methods of installing ESXi 5.5 (2052439)
Methods for upgrading to ESXi 5.5 (2058352)
Installing or upgrading to ESXi 5.5 best practices (2052329)
Update sequence for vSphere 5.5 and its compatible VMware products (2057795)
Processor support discontinued in vSphere 5.5 (2057614)
MSCS support enhancements in vSphere 5.5 (2052238)
Additional SR-IOV support in vSphere 5.5 (2058280)
Required ports for vCenter Server 5.5 (2051575)
Hardware features available with virtual machine compatibility settings (2051652)
Hardware and software requirements for running AMD and NVIDIA GPUs in vSphere 5.5 (2058778)
Virtual Flash feature in vSphere 5.5 (2058983)
vSphere 5.5 Virtual Flash FAQ (2057157)
Setting up a Virtual Flash resource in vSphere 5.5 (2051647)
Disabling Virtual Flash in vSphere 5.5 (2051642)
vSphere 5.5 Virtual SAN requirements (2058424)
Methods for installing vCenter Server 5.5 (2053142)
Methods of upgrading to vCenter Server 5.5 (2053130)
Installing vCenter Server 5.5 best practices (2052334)
vCenter Server not listed in the inventory after installing or upgrading to vSphere 5.5 (2059528)
Configuring CA signed SSL certificates for vCenter Single Sign-On in vSphere 5.5 (2058519)
Creating and using a Service Principal Account in vCenter Single Sign-On 5.5 (2058298)
Reconfiguring the load balancer after upgrading a vCenter Single Sign-On High Availability deployment to version 5.5 (2058838)
VMware vCenter Server 5.x fails to start after upgrading Single Sign-On configuration from vSphere 5.1 to vSphere 5.5 (2058080)
Installing the vSphere 5.5 Web Client on a Microsoft Windows platform (2058262)
Logging in to vCenter Server 5.5 using vSphere Web Client or vSphere Client takes a long time (2058827)
Logging into the vSphere Web Client 5.5 fails with the error: Provided credentials are not valid. (2058796)
vSphere Client and vSphere PowerCLI might fail to connect to vCenter Server 5.5 due to a Handshake failure (2049143)
Cannot drag and drop virtual machines across hosts using the vSphere Web Client 5.5 (2059828)
Installing vSphere Update Manager 5.5 on a Microsoft Windows platform (2058410)
Upgrading to vSphere Update Manager 5.5 on a Microsoft Windows platform (2058423)
Upgrading VMware vSphere Update Manager from 5.0 or 5.1 to 5.5 fails with the error: Error 25113 (2059567)
Latency Sensitive Apps
VMware vSphere 5.5 Low Latency Applications Enhancements (ESX Virtualization)
Latency Sensitive Applications on vSphere 5.5 – Deployment and Best practices (ESX Virtualization)
VMware vSphere 5.5 Latency-Sensitivity Feature (TheSaffaGeek)
Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5 (VMware Tech Paper)
VMware vCloud Suite 5.5 Licensing, Pricing and Packaging (VMware.com)
What is new in VMware vCloud Suite 5.5 (UP2V)
Changes to VMware’s vCloud Suite 5.5 (Viktorious.nl)
VMWorld 2013: Understanding vCloud And vSphere Editions (Virtualization Software)
How to Get Your vSphere 5.5 Licenses (VMware.com)
License key requirements for new version of VMware products (VMware KB article)
VMware vSphere 5.5 Networking New Features (ESX Virtualization)
vSphere 5.5 - Changes on the Network Side (Jason Nash’s Blog)
VMworld 2013: What’s new in vSphere5.5: Networking (Mike Laverick)
Much awaited LACP enhancement - vSphere 5.5 (Stretch Cloud)
LACP Primer - vSphere 5.5 (Stretch Cloud)
Upgrading ESXi host to vSphere 5.5 with Cisco Nexus 1000V (Tom Fojta’s Blog)
Configuring LACP Support in vSphere 5 5 (VMwareTechPubs video)
Enhanced LACP Support in vSphere Distributed Switches (VMwareTechPubs video)
vSphere 5.5 Improvements Part 9 - Networking and VDS Razzle Dazzle (Wahl Network)
What’s New in vCloud Suite 5.5: vCloud Networking & Security (WoodITWork)
VMware NSX Product Walkthrough (VMware.com)
VMware NSX and the SDDC Revolution (digirati.me)
Video: VMware NSX 6.0 for vSphere - 01 Deploying the NSX Virtual Appliance (Hypervizor)
Video: VMware NSX 6.0 for vSphere - 02 Deploying the NSX Controllers (Hypervizor)
Video: VMware NSX 6.0 for vSphere - 03 Preparing Hosts and Configuring Unicast VXLAN (Hypervizor)
Video: VMware NSX 6.0 for vSphere - 04 Configuring a Logical Switch (Hypervizor)
Diagram: VMware NSX 6.0 for vSphere - System Architecture (Hypervizor)
Traffic Filtering and DSCP Marking in vDS 5.5 (Stretch Cloud)
VMware NSX Conversations (The Virtualization Practice)
The Perfect Storm – My thoughts on the VMware NSX launch (Think Cloud)
Introducing VMware NSX - The Platform For Network Virtualization (VMware Network Virtualization Blog)
VMware NSX Partners: Best-in-Class Services For Virtual Networks (VMware Network Virtualization Blog)
VMware NSX Virtualizes the Network to Transform Network Operations (VMware Network Virtualization Blog)
Seven reasons VMware NSX, Cisco UCS and Nexus are orders of magnitude more awesome together (VMware Network Virtualization Blog)
VMworld 2013: NET5847 - NSX: Introducing the World to VMware NSX (VMworld TV)
VMworld 2013: SEC5893 - Changing the Economics of Firewall Services in the Software-Defined Center - VMware NSX Distributed Firewall (VMworld TV)
vSphere 5.5 Improvements Part 8 – Network Virtualization with NSX (Wahl Network)
Announcing vSphere with Operations Management 5.5 - Increasing Performance and Availability for Your Business Critical Applications (VMware vSphere Blog)
What’s New in VMware vSphere with Operations Management 5.5? (VMware Data Sheet)
vSphere 5.5 Record Breaking Network Performance (Long White Clouds)
Greenplum Database Performance on VMware vSphere 5.5 (VMware Tech Paper)
Performance of vSphere Flash Read Cache in VMware vSphere 5.5 (VMware Tech Paper)
Performance Best Practices for VMware vSphere 5.5 (VMware Tech Paper)
SEsparse in VMware vSphere 5.5 (VMware Tech Paper)
IPv6 performance improvements in vSphere 5.5 (VMware Vroom! Blog)
vSphere Replication 5.5 Performance Findings (VMware Vroom! Blog)
VDI Benchmarking Using View Planner on VMware Virtual SAN (VSAN) (VMware Vroom! Blog)
A few cautionary notes about replication performance (VMware vSphere Blog)
VMware vSphere 5.5 Host Power Management (HPM) saves more power and improves performance (VMware Vroom! Blog)
VMware vSphere 5.5 Release Notes (vmware.com)
What’s New in VMware vSphere 5.5: Management APIs (Double Cloud)
VM Storage Policy APIs aka Storage Profile APIs will be available in vSphere 5.5 (Virtually Ghetto)
Exploring the vSphere Flash Read Cache (vFRC) APIs Part 1 (Virtually Ghetto)
Exploring the vSphere Flash Read Cache (vFRC) APIs Part 2 (Virtually Ghetto)
Exploring the vSphere Flash Read Cache (vFRC) APIs Part 3 (Virtually Ghetto)
Automate the migration from Virtual Standard Switch to vSphere Distributed Switch using PowerCLI 5.5 (Virtually Ghetto)
How to automate vFRC configurations using the command-line in ESXi (Virtually Ghetto)
How to add the missing ESXi 5.0 drivers to the ESXi 5.5 installation ISO (VMware Front Experience)
PowerCLI 5.5 What’s New-Overview (VMware PowerCLI Blog)
vSphere Security: Understanding ESXi 5.x Lockdown Mode (Definit)
vSphere 5.5 Hardening Guide - Public Draft Beta released (VMware Community)
Virtual Appliances getting more secure with vSphere 5.5 – Part 1 (VMware vSphere Blog)
Virtual Appliances getting more secure with vSphere 5.5 - Part 2 (VMware vSphere Blog)
Virtual Appliances getting more secure with vSphere 5.5 - Part 3 (VMware vSphere Blog)
Virtual Appliances getting more secure with vSphere 5.5 - Part 4 (VMware vSphere Blog)
Do *NOT* Install vSphere Replication 5.5 onto your 5.1 environment yet! (VMware vSphere Blog)
VMworld 2013: What’s new in vSphere Replication 5.5 (Mike Laverick)
vCenter Replication 5.5 appliance install (SOSTech)
What is new in vSphere Replication 5.5 (UP2V)
What is new in VMware Site Recovery Manager 5.5 (UP2V)
VMware Communities Roundtable Podcast – 249 – SRM 5.5 What’s New? (VMware Podcast)
VMware vSphere Replication 5.5 Overview (VMware Tech Paper)
VMworld 2013: BCO5129 - Protection for All - vSphere Replication & SRM Technical Update (VMworld TV)
What’s New With vSphere Replication and Site Recovery Manager 5.5 (VMware vSphere Blog)
vSphere 5.5 Improvements Part 6 - Site Recovery Manager (SRM) and vSphere Replication (Wahl Network)
What’s New in vCloud Suite 5.5: vSphere Replication and vCenter Site Recovery Manager (WoodITWork)
vSphere 5.5 Storage Enhancements Part 1: 62TB VMDK (Cormac Hogan)
Hot-Extending Large VMDKs in vSphere 5.5 (Cormac Hogan)
vSphere 5.5 Storage Profiles Are Now Storage Policies (Everything Should Be Virtual)
vSphere 5.5 Jumbo VMDK Deep Dive (Long White Clouds)
vSphere 5.5 Windows Failover Clustering Support (Long White Clouds)
VMworld 2013: What’s new in vSphere 5.5: Storage (Mike Laverick)
vSphere 5.5 UNMAP Deep Dive (VMware vEvangelist)
64TB VMDKs? Yes we can. (VM Today)
VMware’s Strategy for Software-Defined Storage (VMware CTO Blog)
What’s New in vSphere 5.5 Storage (VMware vSphere Blog)
Comparing VMware VSA & VMware Virtual SAN (VMware vSphere Blog)
VMworld 2013: STO5715-S - Software-defined Storage - The Next Phase in the Evolution of Enterprise Storage (VMworld TV)
vSphere 5.5 Improvements Part 3 – Lions, Tigers, and 62TB VMDKs (Wahl Network)
vSphere 5.5 nuggets: changes to disk.terminateVMOnPDLDefault (Yellow Bricks)
vSphere 5.5 nuggets: Change Disk.SchedNumReqOutstanding per device! (Yellow Bricks)
vCenter Server Links
How-to backup and restore VCSA 5.5 internal database (ESX Virtualization)
How to change the default password policies in vSphere 5.5 (ESX Virtualization)
vSphere 5.5 – Using vCenter Server Appliance (Everything Should Be Virtual)
VMware vCenter Server Appliance (VCSA) 5.5 deployment tips and tricks (ivobeerens.nl)
VMworld 2013: What’s New in vCenter 5.5 (Mike Laverick)
vCenter Server Appliance 5.5 – some basic information (running-system.com)
Upgrading to vSphere 5.5 and using default self-signed certificates? Read this first! (Viktorious.nl)
VMware vCenter Server 5.5: Installable vs Appliance (Virtual To The Core)
Administrator password expiration in new VCSA 5.5 (Virtually Ghetto)
vPostgres Database Backup in vCSA 5.5 (vNinja)
What’s New in vCloud Suite 5.5: vCenter Server and ESXi (WoodITWork)
Drag and drop vMotion not working with the 5.5 Web Client? (Yellow Bricks)
vSphere 5.5 nuggets: vCenter Server Appliance limitations lifted! (Yellow Bricks)
vCenter Server Single Sign-On
The Trouble With SSL Certificates and Upgrading to VMware SSO 5.5 (Long White Clouds)
Single Sign-On (SSO) Improvements in vSphere 5.5 (Petri)
Issues with authentication when running vSphere 5.5 with AD and SSO server on Windows Server 2012 (Shogan.tech)
What is new in VMware Single Sign-On 2.0 (UP2V)
A Look At vCenter 5.5 SSO RC Installation (VMware vEvangelist)
Allow me to introduce you to vCenter Single Sign-On 5.5 (VMware vSphere Blog)
vCenter Single Sign-On 5.5 Not Recognizing Nested Active Directory Groups (VMware vSphere Blog)
vCenter Single Sign-On “Cannot parse group information” Error (VMware vSphere Blog)
vCenter Single Sign-On 5.5 – Backwards Compatible? (VMware vSphere Blog)
vSphere 5.5 Improvements Part 7 – Single Sign On Completely Redesigned (Wahl Network)
Using Active Directory Integrated Windows Authentication with SSO 5.5 (Wahl Network)
What’s New in vCloud Suite 5.5: vCenter Server SSO (WoodITWork)
VMworld 2013: What’s New in vCloud Director 5.5 (Mike Laverick)
Fast Provisioning Enhancement in vCloud Director 5.5 (Tom Fojta’s Blog)
What is new in VMware vCloud Director 5.5 (UP2V)
VMware vCloud Director 5.5 new features (Virtual-blog)
Reports of vCloud Director’s death exaggerated (VMware ReThink IT Blog)
What’s New with VMware vCloud Director 5.5 (VMware Tech Paper)
A Summary of What’s New in vCloud Director 5.5 (VMware vSphere Blog)
vCloud Director Convergence and Transition Plan – What’s the Scoop? (VMware vSphere Blog)
What’s New in vCloud Suite 5.5: vCloud Director (WoodITWork)
vCloud Hybrid Service
How To Use VPN to Connect Multiple vCHS Clouds (Chris Colotti)
How To Move Workloads Between vCHS Virtual Private and Dedicated Clouds (Chris Colotti)
Understanding vCHS Resource Management – Virtual Private Cloud (Chris Colotti)
An introduction to VMware vCloud Hybrid Service (UP2V)
VMware vCloud Hybrid Service: Now Available to All U.S. Customers (VMware vCloud Blog)
Introducing The New vCloud Hybrid Service Marketplace (VMware vCloud Blog)
What Can I Run On vCloud Hybrid Service? (VMware vCloud Blog)
Introducing vCloud Hybrid Service at VMworld 2013 (VMworld TV)
vCloud Hybrid Service Demo at the VMware Booth, VMworld 2013 (VMworld TV)
vCloud Hybrid Service Overview Video (VMworld TV)
VMworld 2013: PHC5605-S - Everything You Want to Know About vCloud Hybrid Service - But Were Afraid to Ask (VMworld TV)
vFlash Read Cache Links
VMware vSphere Flash Read Cache 1.0 FAQ (VMware.com)
VMware vFlash Read Cache (vFRC) configuration steps (ESX Virtualization)
vSphere 5.5 – A Look at Flash Read Cache (Jason Nash’s Blog)
Introducing vSphere Flash Read Cache (Punching Clouds)
Nested ESXi 5.5 – Configure Virtual Flash? (SOSTech)
Introduction of VMware vSphere Flash Read Cache (UP2V)
vSphere 5.5 Flash Read Cache (vFRC) (vClouds)
Introducing Flash Read Cache to my lab – HP N36L (viktorious.nl)
Virtual Flash feature in vSphere 5 5 (VMware KB TV)
Performance of vSphere Flash Read Cache in VMware vSphere 5.5 (VMware Tech Paper)
What’s New in VMware vSphere Flash Read Cache (VMware Tech Paper)
vSphere Flash Read Cache - Single and Multiple Host Configuration (VMware TV)
vSphere Flash Read Cache - Configuring a VM to use Virtual Flash Read Cache (VMware TV)
vSphere Flash Read Cache - Interoperability with HA & DRS (VMware TV)
vSphere Flash Read Cache - Interoperability with vMotion (VMware TV)
Can you combine vSphere Host Cache and vFlash on a single SSD? (vNinja)
vSphere 5.5 Improvements Part 5 - vSphere Flash Read Cache (vFlash) (Wahl Network)
What’s New in vCloud Suite 5.5: VMware Virtual Flash (vFlash) (WoodITWork)
Introduction to vSphere Flash Read Cache aka vFlash (Yellow Bricks)
Frequently asked questions about vSphere Flash Read Cache (Yellow Bricks)
Something to know about vSphere Flash Read Cache (Yellow Bricks)
vSphere Flash Read Cache and esxcli (Yellow Bricks)
VMworld 2013: What’s New in vSphere 5.5 - The Virtual Machine - 62TB VMDK (Mike Laverick)
Quick Tip: New Hyper-V guestOS identifier in vSphere 5.5 (Virtually Ghetto)
What’s New in VMware Virtual SAN (VSAN) (VMware Tech Paper)
VSAN Public Beta Registration (VMware.com)
VSAN Public Beta Program Community Site (VMware.com)
What’s New in VMware Virtual SAN (VSAN) (Tech paper) (VMware.com)
Virtual SAN (VSAN) Beta Technical FAQ (VMware.com)
Virtual SAN Beta - Introduction (webinar slides) (VMware.com)
vSphere 5.5 Virtual SAN Public Beta Release Notes (VMware.com)
VMware Virtual SAN Datasheet (VMware.com)
Virtual SAN (VSAN) Proof of Concept (POC) Kit (VMware.com)
VSAN Design & Sizing Guide (VMware.com)
VMware VSAN Interactive Product Walkthrough (VMware.com)
VMware vSphere 5.5 Virtual SAN Hosted Beta (Lab Guide) (VMware.com)
VMware Communities Roundtable Podcast – 251 – VSAN (VMware.com)
VMworld 2013 session - STO5391 - VMware Virtual SAN (VMware TV)
Virtual SAN Beta Webinar - Introduction to VMware Virtual SAN (VSAN) (VMware TV)
Virtual SAN (VSAN) Beta Webinar - How to Install, Configure & Manage (video) (VMware TV)
Virtual SAN (VSAN) Setup (VMware TV)
How To Monitor Virtual SAN (VSAN) (VMware TV)
Deploying a Virtual Machine on Virtual SAN (VSAN) (VMware TV)
Changing a VM Storage Policy On-The-Fly with VSAN (VMware TV)
Virtual SAN (VSAN) & VM Availability (VMware TV)
Virtual SAN (VSAN) & vSphere HA Interoperability (VMware TV)
VSAN Part 1 - A first look at VSAN (Cormac Hogan)
VSAN Part 2 - What do you need to get started? (Cormac Hogan)
VSAN Part 3 - It is not a Virtual Storage Appliance (Cormac Hogan)
VSAN Part 4 – Understanding Objects and Components (Cormac Hogan)
VSAN Part 5 - The role of VASA (Cormac Hogan)
VSAN Part 6 - Manual or Automatic Mode (Cormac Hogan)
VSAN Part 7 - Capabilities and VM Storage Policies (Cormac Hogan)
VSAN Part 8 – The role of the SSD (Cormac Hogan)
VSAN Part 9 - Host Failure Scenarios & vSphere HA Interop (Cormac Hogan)
VSAN Part 10 - Changing VM Storage Policy on-the-fly (Cormac Hogan)
VMware VSAN configuration steps with nested ESXi hypervisors (ESX Virtualization)
VMware VSAN Video (ESX Virtualization)
Homelab Thoughts – VSAN (ESX Virtualization)
My completely ridiculous VSAN test (Gabe’s Virtual World)
VMworld 2013: What’s New in 5.5 – vSAN (Mike Laverick)
VMworld 2013: What’s New in VSAN 1.0 (Mike Laverick)
How does VMware VSAN help Horizon View? (My Virtual Cloud)
How is Horizon View configured for VMware VSAN? (My Virtual Cloud)
Why is Content-Based Read Cache (CBRC) so important for Horizon View and VSAN? (My Virtual Cloud)
VSAN Observer (Punching Clouds)
vSphere 5.5: Using RVC VSAN Observer Pt1 (Punching Clouds)
vSphere 5.5: Using RVC VSAN Observer Pt2 (Punching Clouds)
VMware’s VSAN in the Home Lab (Sean Crookston)
vSphere 5.5 VSAN, Introducing Your New Storage Vendor VMware (vClouds)
About VSAN capacity and VMDK placement (viktorious.nl)
VMware VSAN - Virtual SAN - How to configure (Virtual-blog)
Configure Disk Redundancy VMware VSAN - Virtual SAN (Virtual-blog)
VMware Virtual SAN Scalability Limits - VSAN (Virtual-blog)
STO5027 VMware Virtual SAN Technical Best Practices (virtualarchitect.nl)
How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi (Virtually Ghetto)
Additional steps required to completely disable VSAN on ESXi host (Virtually Ghetto)
VSAN and Storage Controllers (VMware vSphere Blog)
Virtual SAN & Disk Groups (VMware vSphere Blog)
Three Hosts, two VSAN Datastores! (VMware vSphere Blog)
vSphere 5.5 Availability and VSAN Public Beta (vNinja.net)
vSphere 5.5 Improvements Part 4 - Virtual SAN (VSAN) (Wahl Network)
In Which I Muse Over VMware’s Virtual SAN Architecture (Wahl Network)
What’s New in vCloud Suite 5.5: Virtual SAN (VSAN) (WoodITWork)
Introduction to VMware vSphere Virtual SAN (Yellow Bricks)
Frequently asked questions about Virtual SAN / VSAN (Yellow Bricks)
Virtual SAN news flash pt 1 (Yellow Bricks)
Testing vSphere Virtual SAN in your virtual lab with vSphere 5.5 (Yellow Bricks)
How do you know where an object is located with Virtual SAN? (Yellow Bricks)
How VSAN handles a disk or host failure (Yellow Bricks)
Virtual SAN and Data Locality/Gravity (Yellow Bricks)
Isolation / Partition scenario with VSAN cluster, how is this handled? (Yellow Bricks)
Initialized disks to be used by VSAN task completed successfully, but no disks added? (Yellow Bricks)
I created a folder on my VSAN datastore, but how do I delete it? (Yellow Bricks)
Be careful when defining a VM storage policy for VSAN (Yellow Bricks)
Designing your hardware for Virtual SAN (Yellow Bricks)
Pretty pictures Friday, the VSAN edition… (Yellow Bricks)
How to configure the Virtual SAN observer for monitoring/troubleshooting (Yellow Bricks)
VMware vSphere Virtual SAN design considerations… (Yellow Bricks)
Virtual SAN and Network IO Control (Yellow Bricks)
vSphere Metro Storage Cluster using Virtual SAN, can I do it? (Yellow Bricks)
4 is the minimum number of hosts for VSAN if you ask me (Yellow Bricks)
vSphere Data Protection
vSphere Web Client
vSphere 5.5 Web Client Workflows – Configure Software iSCSI (blog.shiplett.org)
vSphere 5.5 Web Client Workflows – Create Distributed Virtual Switch and Port Groups (blog.shiplett.org)
vSphere 5.5 Web Client Workflows – Add Hosts to DVS and Create VMkernel Ports (blog.shiplett.org)
vSphere 5.5 Web Client Workflows – Create a New Datastore (blog.shiplett.org)
vSphere 5.5 Web Client Workflows – SSO Configuration (blog.shiplett.org)
vSphere 5.5 Web Client Workflows – Licensing (blog.shiplett.org)
Few tips when working with vSphere 5.5 web client – video (ESX Virtualization)
vSphere Web Client 5.5: Drag & Drop cannot be used to start a vMotion (running-system)
My top 5 favorite enhancements to the new vSphere Web Client 5.5 (Virtually Ghetto)
5 Quick tips for the vSphere Web Client (VMwareTechPubs video)
Examining the User Experience in the new vSphere 5.5 Web Client [Video] (Wahl Network)
Well that’s a wrap, another VMworld in the bag for me. This is the 6th VMworld that I’ve attended so I’ve seen it evolve and change over the years and wanted to give my thoughts and feedback on it.
- Record attendance as usual, my thought on that in this post.
- Gone are the days where VMworld was like an Apple event, full of mystery and cool feature and new product unveilings. The keynotes back then were a lot more fun and enjoyable compared to today where almost everything that is announced you already know about. Things like vSAN, vVols, NSX and vFlash we’ve known about for a while now and were featured in prior VMworlds. So VMware is basically announcing things that they’ve already announced and in typical fashion many of the features you still have to wait for until a new release after 5.5 comes out. The end result is that I found that the keynotes were not that exciting, especially when we had the new guys (Pat, Carl, Robin) doing it, it just wasn’t the same as past years. Paul Maritz and Stephen Herrod had a special way of generating excitement and captivating the audience that will be missed.
- Also in the keynote VMware the OpenStack elephant briefly entered the room and quickly moved on, you can probably guess why.
- The streaming of the Day 1 keynote was very smooth, I prefer watching it from the hotel room where I can capture things more easily. I don’t know why VMware chose not to stream the Day 2 keynote, it would of been nice to give you the option of not fighting the crowds and watching it wherever you wanted to.
- VMware for gosh sakes, please count anyone that attends a VMworld regardless of how they got there towards alumni status. Right now if you went as a blogger, press, exhibitor or some other types of passes they don’t consider that as having attending the event and so it doesn’t count towards alumni status which you get when you have attended at least 3 VMworld events. If I’m there, I’m there OK, count it towards alumni, you’re making all those people that are losing out feel unwanted. It might seem like a trivial thing but it feels nice to be recognized as an alumni.
- Last year was pretty cramped but they did a good job this year with spreading things out and had things spread out further across some of the nearby hotels. It was nice being able to go downstairs from my room in the Marriott to see a session first thing in the morning.
- Thank you VMware for not making us eat outside this year in a park with no seating, pigeons and overflowing garbage cans. It was good to have lunch indoors this year and also split into both Moscone North and West.
- Storage and Networking were the big highlight areas for VMware at the show. On the networking side you had the NSX product which was the result of VMware’s Nicira acquisition, I don’t know a whole lot about it but the networking crowd seems to be fairly excited about it. The one interesting thing I noted on the keynote slide that showed 15+ partners that were working on NSX integration was that Cisco was missing from that list. That wasn’t by mistake either as it seems Cisco wants to sell their own networking solutions and not integrate with VMware on NSX. On the storage side you had all the new storage technologies (vSAN, vVols, vFlash, Virsto) that VMware has been talking about for a while now, so no real surprises there. However the only one that will be available in vSphere 5.5 is vFlash, vSAN will be a public beta for a period of time before it is fully supported (H1 2014). There was hardly any talk of vVols this year as VMware still has a lot of work to do to get that out the door. There was a lot of noise about vSAN though that was reflected everywhere at the show including sessions and in the Solution Exchange.
- Speaking of Virsto, it was mentioned in the keynote but there was hardly anything else on it at the show. I went to the one session that they had on it hoping to find out what VMware plans on doing with it but that was not covered at all. They basically just covered the architecture and how everything works as the product is today, what I want to know is how they are going to integrate it into vSphere. Today they sell the product just as it was before they bought it, I’m guessing that they are going to take some of the technologies from it and use them in their existing and future storage architecture. While what it does is cool, overall as the product is today it seems overly complicated with a lot of components which requires you to basically replace the existing vSphere storage architecture. I think it will be much better when VMware can blend it into their storage architecture.
- The announcement of vSphere 5.5 seemed to get lost in the show as the focus stayed on some of the big upcoming features. VMware has a paper on what’s new in vSphere 5.5 and it show be available sometime in September. As usual there are tons of little enhancements and features that don’t get much mention, so turn to the blogs to find out more about them. I’ll be publishing my vSphere 5.5 mega-link list soon that will have everything in one place for you.
- I thought the sessions went very smooth this year, VMware did a good job of organizing and scheduling them. It seemed like there were a lot more than last year which is a good thing as more sessions make the cut and you can pick and choose what you want to see.
- It sucks that VMware doesn’t give backpacks to everyone, they were actually pretty cool this year and being the 10th anniversary made them collectible. I can understand on keeping costs down but VMware should at least offer them for sale in the VMworld store for anyone that wants to buy one.
- I was pretty surprised to see the winners of the annual Best of VMworld awards that Tech Target runs each year at VMworld. I haven’t heard of many of the companies that won and were finalists. The winners this year seem to reflect that either big vendors aren’t innovating anymore or just not participating. I tend to believe the latter, you have to self-nominate your products and I think the bigger vendors don’t bother that much. Don’t get me wrong, there are plenty of small companies with cool products, but seriously a UPS power management tool is the best of the best in virtualization management tools for vSphere?
- I spotted Stephen Herrod at VMworld when I was talking with some people over by The Cube. He may not be VMware’s CTO anymore but it’s hard to turn your back on something that you have been passionate about for so many years so I can’t say I was surprised to see him. The folks at The Cube were quick to pounce on him and stick a microphone in his hand.
- Lots of great parties this year, I went to the SolarWinds vMixer, VMunderground, Veeam party and an HP party. My only gripe with the VMunderground this year was it was too far away and there were hardly any cabs around the location if you wanted to take one back.
- The Solutions Exchange was really crowded this year with steady traffic almost every day. Lots of energy in there and lots of great products so its always fun to walk around and take it all in.
All in all I think VMware did a excellent job executing VMworld this year and I look forward to going back next year. It always flies by so quickly and there are so many people that I wanted to bump into but didn’t so I’ll have to wait another year to see them.
Below are some of the Day 1 keynote slides with my comments on them:
I liked this slide that showed a visual representation of VMware innovation over the years. The standout innovative feature on that slide is vMotion which VMware made a special point of recognizing. Today we take vMotion for granted but you have to admit the first time you saw it after it was introduced it was pretty damn impressive.
I like the ‘Apps heart vSphere” theme, VMware had bumper stickers with that slogan all over that they were giving away. In the vSphere 5.5 release you can now virtualize any app, especially with the monster VM getting even bigger in vSphere 5.5. Of particular note VMware has paid some attention to the one app that has historically been difficult to virtualize, low latency apps. Those types of application are very sensitive to any type of latency and need to service requests in microseconds. In a virtualized environment any lag or performance bottleneck can impact those apps. VMware’s solution to this is to use CPU affinity which has been around for a while that can dedicate a CPU to an application and some additional resource settings to ensure that VMs serving low latency apps get top resource priority.
This slide shows some of the storage evolution with VMware, while some of the software features that fall into their Software Defined Storage category they have innovated, the VSA isn’t one of them. VSA’s have been around for many years before VMware released their own VSA.
These two slides illustrate VMware’s approach to Software Defined Storage, vSAN fits into that converged infrastructure pool with servers and local storage, the external pool below it will be vVols fits in one it’s released. The virtual data plane features many of VMware’s policy based storage controls, they are shifting to a more granular per-VM module across all their storage implementations. I suspect Virsto will surface here eventually as it already provides this today.
This slide illustrates the major storage pillars/features/architectures that VMware will have in vSphere. Note vFlash is really the only one that you can use in vSphere 5.5. vSAN will be a public beta until next year, vVols is still a ways out and Virsto is really just the same standalone product it always was before VMware acquired it.
These slides cover VMware’s new focus on network virtualization with NSX that completes their vision for the Software Defined Data Center. Now that compute is conquered, storage is mostly conquered they are moving on to the final piece of the SDDC puzzle.
These two slides illustrate how much the data center is being virtualized, 2010 marked the crossover of data centers having more VMs then physical servers, 2012 marked the crossover of data centers using more virtual network ports then physical network ports. Besides being more virtualized today I think two factors helped this trend, larger single adapter network bandwidth (10Gbe, 16Gb FC) and VM sprawl. The net effect is that vendors that sell physical network products need to shift their focus more and more to virtual network products if they want to sell products. While there will always be the need for physical networking much of the intelligence is being shifted from the physical side to the virtual side the same way it happened with storage.
The OpenStack elephant, VMware had to mention it as it is becoming increasingly popular in cloud data centers. They probably spent less than 30 seconds on this slide and just wanted to do the obligatory “yeah we work with OpenStack” and that’s it. Obviously VMware wants you to buy their own cloud, management and virtualization products but say if you do use OpenStack you can fit our products into it.
Finally VMware’s vCloud Hybrid Service, basically just vSphere and vCloud extending to VMware’s public hybrid cloud offerings. Also VMware introduced their vCloud Automation Center which is the end result of their Dynamic Ops acquistion, the end result is two separate products, vCloud and vCAC with vCAC being the more robust and advanced product. The whole VMware cloud offering is getting muddled and confusing and VMware constantly shifts strategies, licensing and product offerings for cloud.
This year VMware announced record attendance of over 22,000 people that attended VMworld. While their news release said over 22,000 at one of the keynotes it was stated their were 22,500 attendees. If you look at the graph below you can see every year is a record attendance for VMworld except for 2009 when it briefly dipped by 1500 attendees. Why the big dip in 2009? The shape of the economy probably played a factor that year which caused many IT departments to tighten their belts and limit travel and event expenses. Virtualization was also not as widely adopted back then as well which may of been a factor. The good news is that it bounced back in a big way the following year and has climbed every year. What I’d like to know is if VMware is counting their own employees and partner booth staff in their attendee numbers, I’m pretty confident that they are. It would be interesting to see a breakdown of all the different badge types that they issue for the show.
- VMworld 2004 - 1400
- VMworld 2005 - 3500
- VMworld 2006 - 6700
- VMworld 2007 - 10800
- VMworld 2008 - 14000
- VMworld 2009 - 12500
- VMworld 2010 - 17000
- VMworld 2011 - 19000
- VMworld 2012 - 21000
- VMworld 2013 - 22500
To provide some contrast to this, here’s how VMworld attendance stacked up against other popular tech conferences. Note while VMware is very open about the number of people that attend their annual conferences, many companies seem to not publicize that information.
- Oracle OpenWorld - 50,000
- Cisco Live 2013 - 20,000
- SAP SAPPHIRE NOW 2013 - 20,000
- EMC World 2013 - 15,000
- Microsoft TechEd 2013 -13,000
- HP Discover 2013 - 12,000
- CA World 2012 - 6,000
- IBM Edge 2013 - 4,700
- Dell World 2012 - 4,500
At some point I’m sure VMworld will peak out in attendance if I had to make a guess I’d say that next year may be the attendance tipping point. But who knows for sure, virtualization is still hot and VMworld these days is not just about virtualization but also all the technologies, products and services that interact with it which is quite a big list. We’ll have to wait and see what the future holds for VMworld.
Looking for a party to attend on Monday at VMworld, of course you are. SolarWinds is throwing their vMixer party again this year and I’ll be there so stop on by. If meeting me and other vExperts and bloggers is not compelling enough, there is also going to be a trivia contest where you can win cool prizes. The event is on Monday from 6:00pm - 9:00pm at Annabelle’s Bar and Bistro (68 4th St.) which is attached to the Mosser hotel right across the street from the Marriott Marquis. So what are you waiting for, register here and I look forward to meeting you there!
I was looking through the people that I follow on Twitter the other day and decided to tweet out the first people that I followed on Twitter related to virtualization. After I did that it seemed to catch on with many others tweeting the first people that they followed as well. A couple people asked how I found out the history of who I followed so I thought I’d show how I did it. It’s actually fairly simple, if you open your Twitter page using a web browser then click on the Following area which it shows how many people you are following you will see the list of people you follow in the order that you followed them starting from the most recent.
You then have to scroll all the way down to the bottom to see who you followed first, if you follow a large number of people it can take a while to get to the bottom. Your Twitter Following list may be displayed in historical order in other Twitter clients as well, I tried it in Janetter for Windows (much better than TweetDeck) and I was able to scroll to the bottom in a few seconds which was much quicker.
So I thought I’d share who I followed first related to virtualization, here are my first 25 in order starting with #1. Feel free to post your first 5/10/25 in the comments. Also here’s a Twitter list that I made for My First v25 so you can easily follow anyone on it that you might not be following already.
|5||Gabe Van Zanten||@gabvirtualworld|
Another year, another VMworld, this will be the 10th anniversary of VMworld since the first one held in San Diego in 2004. The first VMworld only had 1,400 attendees, this year I would expect that at least 22,000 people attend VMworld. For me this VMworld will be number 6, my first one was in 2008 in Las Vegas. I remember that experience fondly as it was more intimate back then when it was a lot smaller. I attended VMworld as a speaker that year as I was part of a group of VMTN moderators selected to participate in a VMTN Community Experts panel along with Thomas Bryant, Ken Cline, Steve Beaver, Tom Howarth and Edward Haletky. Over the years I learned many lessons about attending VMworld so I thought I’d pass that on to you:
- Set your priorities and expectations ahead of time, VMworld has plenty to offer and you’ll get as much out of it as you put into it. Sessions are at the bottom of the priority list for me, things like networking and going through the Solutions Exchange are at the top. If you make a schedule it will be challenging to keep it as there are plenty of distractions at VMworld. Be realistic and don’t try and cram your schedule so full that you stress yourself out and are rushing to get to everything. Relax, enjoy yourself and have fun.
- The sessions, so many to choose from, so little time, you’ll be lucky if you can attend more than a dozen of them. They’re just so much other stuff to do there its hard to find time to go to sessions. I recommend you pick a few that you really want to see and don’t sweat it if you miss some, remember they’re all recorded and you have plenty of time to see them after VMworld. I usually choose based on certain people that I want to hear speak and meet after the session. After all you can always hear the session later but you can’t meet the speaker after VMworld is over. This year again you have to register for sessions, but anyone can show up and you can usually get in if you wait in line until the session begins and there are still seats.
- The labs, always pretty hectic to get into them but they keep getting bigger and better each year. Gone for a while now are the instructor-led labs using on-site data centers, VMware has migrated everything to the cloud and it is all self-paced. Many of the labs are available 24×7 now thanks to VMware’s Project Nee, so you don’t need to feel the urgency to take them at VMworld. Still it’s always good to get some hands on so if you have some free time be sure and take a few.
- The parties, there are no shortage of them, my inbox always fills up with party invitations each year and trying to pick and choose which ones you want to attend can be even more challenging than picking which sessions to see. There is a pretty comprehensive list of them here. With VMworld starting a day earlier this year it opens up one more day for parties. Traditionally held on Sunday, the Welcome Reception in the Solutions Exchange is from 4-7pm on Sunday this year. There is lots of food, beer & vendors so its a good way to start the evening. Afterwards on Sunday is the annual big VMunderground warm-up party, this years it’s at the Terra Gallery from 8-11pm, you have to register to get in (its already closed) but if you try going later they may have some room for anyone to go in. This party grows every year and I think they had almost 2,000 tickets for it this year. The official (or unofficial) VMworld tweetup and flipcup tournament is in on Monday from 7:30pm - 10:30pm at Mr. Smith’s, more info and sign-up is available here. Tuesday seems to be the day every vendor throws a party, so pick the ones you want to go to and hop between them. Personally I’m going to try and make it to 2-3 of them including the Veeam party which is always great. Contact your vendors if you don’t have invites and they should be able to get you one. Wednesday is the big official VMworld party with Train & Imagine Dragons playing this year, unlike previous years where they played indoors in the same area as the keynotes are held, this year they are playing the the SF Giants stadium (AT&T Park). Finally on Wednesday evening if you are into stogies, a group of people are going to smoke them after 11:00pm at a local cigar bar, RSVP here. You can’t smoke anywhere in San Fran so if you like cigars come along.
- The networking, that’s what VMworld is all about, don’t be a hermit and don’t be afraid to talk to people. People like Mike Laverick, Jason Boche, Scott Lowe and Duncan Epping aren’t surrounded by security guards and are down to earth guys who will talk to anyone. So go say high, introduce yourself and have a conversation, you’ll be very glad you did so afterwards. Don’t know where to find people? Well parties are a good place to start, everyone seems to gravitate there. Do yourself a favor and get on twitter if you’re not already and you’ll know in real time whats going on. You can find out more about tweeting here and blogging here, the official VMworld hashtag is #vmworld. There is also the VMworld Hang Space which is a good place to socialize and meet the bloggers. Be sure and check out VMware’s social media & community guide which has a lot of good info in it.
- Plan your trip appropriately, you’re going to be on your feet a lot at VMworld, you better have comfortable shoes or you’re going to have real sore feet. Don’t by a pair of new spiffy shoes right before VMworld without breaking them in, you’ll regret it! Pack light if you can, you might want to being an extra bag, there are lots of prize giveaways and free swag all over the place so I can almost guarantee you’re going to go home with more than you came with. If you’re going to walk around with a back pack don’t stuff it too much, its going to get awfully heavy after wearing it a few hours. I travel light and don’t want a full laptop to lug around, I bring a netbook, iPad and iPhone and choose the one I want to carry for what I need to do at the time. Don’t forget power, especially for your iPhone, I carry 3 battery packs so I can charge it as needed without an outlet. Wi-fi coverage at Moscone is so-so and 3G in San Fran is always bad and will probably be even worse at VMworld due to an additional 5,000 or so iPhones/iPads all fighting for service.
- VMware makes a point to show off their talent at VMworld, this means those geeky developers that are normally locked up all day making the next version of vSphere are there and usually available to talk to. What better person to ask your HA question than someone who actually developed the feature. VMware has lots of other smart people there so be sure and check out the VMware booths in the Solutions Exchange to meet them. It’s not just VMware that has their smartest and brightest at the show though, most of the vendors have their best people there also so go by your favorite vendors and talk them up and get your questions answered.
- The Solutions Exchange is like a Super Walmart, everything you can possibly need for VMware products all under one roof, take your time, stroll around and I guarantee you’ll see many cool products that you probably never knew existed. VMware has an incredibly rich ecosystem of vendors that can help solve your pain points and enhance your environment. Do make a point of spending plenty of time there, besides learning a lot you’ll leave with pockets stuffed with vendor swag. Besides the Welcome Reception on Sunday there is the Hall Crawl on Tuesday from 4-6pm where you can get free booze from certain vendors that offer it.
- If this is your first time at VMworld or San Fran it can be a bit intimidating, especially when it comes to finding your way around. If you get there on Sunday try and pick up your badge then rather than fight the crowds on Monday. Walking around and trying to get your bearings can help, be sure and use the maps of the Moscone that are published on VMworld.com and in the docs you are given when you check in to see where everything is. If you need information don’t hesitate to ask someone, or even better trying tweeting it and you might get a quick answer. The more social you can be at VMworld especially if its your first time will really help you out as us virtualization folks are a friendly lot that don’t bite and are glad to help out a vComrade. Also be sure and download the VMworld mobile app for your phone or tablet.
- See San Francisco if you can, there is lots to see in the city, know how to get around ahead of time, BART and the public transportation are great for this. Go see the sea lions at Pier 39 (watch out for the Bush Man), take a boat trip to Alcatraz, see the Muir Woods, Golden Gate or go climb Coit Tower. I have a big list of things to do in San Francisco here.
- Know where to go after the action is over, once VMworld closes each day there are plenty of parties, after the parties are over many gather at popular spots. One such spot is the lobby bar of the Marriott Marquis hotel which is 2 blocks from the Moscone. I had many great late night conversations with others there each night after all the parties ended.
VMworld is four short days and will fly by before you know it, so be sure and make the most of it and soak up the incredible amount of knowledge that will be available both formally through labs and sessions and informally through talking to others. Enjoy the show, I hope to meet many of you there and I leave you with this: John Troyer takes a swim at VMworld 2011
I’ve been attending VMworld since 2008 and the official parties have always been a fun way to unwind for a bit and escape from virtualization. VMware has had a variety of different bands to play at the party over the years that are listed below:
- 2007 - Smash Mouth (Treasure Island)
- 2008 - DJ & Tainted Love (cover band) (Las Vegas Speedway)
- 2009 - Foreigner (Moscone)
- 2010 - INXS (Moscone)
- 2011 - Killers (Venetian)
- 2012 - Jon Bon Jovi & the Kings of Suburbia (Moscone)
- 2013 - Train and Imagine Dragons (AT&T Park)
I like a wide range of music but the only two bands that I really enjoyed seeing at VMworld were INXS and Foreigner which are in the classic rock era that I grew up with. Last year I was excited to find out that Jon Bon Jovi was playing which is another band I grew up with but the actual performance I thought was terrible as he played mostly cover songs from other bands and very few Bon Jovi songs. VMware is a big company and certainly has the deep pockets needed to hire bigger name bands, it certainly would be nice to see some bigger bands play at the VMworld party. Train is an OK band, I like a few of their songs but they are not a band I have a strong desire to see. To contrast this here’s the bands that have played at some other big tech conferences over the past few years:
- 2010 - Counting Crows
- 2011 - The Fray
- 2012 - Maroon 5
- 2013 - Bruno Mars
- 2011 - Paul McCartney
- 2012 - Sheryl Crow & Don Henley
- 2013 - Santana
Oracle Open World:
- 2010 - Black Eyes Peas & Don Henley
- 2011 - Sting
- 2012 - Pearl Jam
- 2013 - Maroon 5 & The Black Keys
- 2010 - Smash Mouth
- 2011 - Train
- 2012 - Weezer
- 2013 - Journey
If you’re going to VMworld this year be sure and check out my session STO5545 - The Top 10 Things You MUST Know About Storage for vSphere which will be on Tuesday, Aug. 27th from 5:00-6:00 pm. The session was showing full last week but they must have moved it to a larger room as it is currently showing 89 seats available. This session is crammed full of storage tips, best practices, design considerations and lots of other information related to storage. So sign up know before it fills up again and I look forward to seeing you there!
The focus on applications often gets lost in the shuffle when implementing virtualization which is unfortunate because if you look at any server environment its sole purpose is really to serve applications. No matter what hardware, hypervisor or operating system is used, ultimately it all comes down to being able to run applications that serve a specific function for the users in a server environment. When you think about it, without applications we wouldn’t even have a need for computing hardware, so it’s important to remember, it’s really all about the applications.
So with that I wanted to provide 5 tips for monitoring applications in a virtual environment that will help you ensure that your applications run smoothly once they are virtualized.
Tip 1 – Monitor all the layers
The computing stack consists of several different components all layered on top of each other. At the bottom is the physical hardware or bare metal as it is often referred to. On top of that you traditionally had the operating system like Microsoft Windows or Linux, but with virtualization you have a new layer that sits between the hardware and the operating system. The virtualization layer controls all access to physical hardware and the operating system layer is contained within virtual machines. Inside those virtual machines is the application layer where you install applications within the operating system. Finally you have the user layer that accesses the applications running on the virtual machines. Within each layer you have specific resource areas that need to be monitored both within and across layers. For example storage resources which are a typical bottleneck in virtual environments need storage management across layers so you get different perspectives from multiple viewpoints.
To have truly effective monitoring you need to monitor all the layers so you can get a perspective from each layer and also see the big picture interaction between layers. If you don’t monitor all the layers you are going to miss important events that are relevant at a particular layer. For example if you focus only on monitoring at the guest OS layer, how do you know your applications or performing as they should or that your hypervisor does not have a bottleneck. So don’t miss anything when you monitor your virtual environment, you should monitor the application stack from end to end all the way from the infrastructure to the applications and the users that rely on them.
Tip 2 – Pay attention to the user experience
So you monitor your applications for problems but that won’t necessarily tell you how well it’s performing from a user perspective. If you’re just looking at the application and you see it has plenty of memory and CPU resources and there are no error messages you might get a false sense of confidence that it is running OK. If you dig deeper you may uncover hidden problems, this is especially true with virtualized applications that run on shared infrastructure and multi-tier applications that span servers that rely on other servers to properly function.
The user experience is what the user experiences when using the application and is the best measure of how an application is performing. If there is a bottleneck somewhere between shared resources or in one tier of an application it’s going to negatively impact the user experience which is based on everything working smoothly. So it’s important to have a monitoring tool that can simulate a user accessing an application so you can monitor from that perspective. If you detect that the user experience has degraded many tools will help you pinpoint where the bottleneck or problem is occurring.
Tip 3 – Understand application and virtualization dependencies
There are many dependencies that can occur with applications and in virtual environments. With applications you may have a multi-tier application that depends on other services running on other VMs such as a web tier, app tier and database tier. Multi-tier applications are typically all or nothing, if any one tier is unavailable or has a problem the application fails. Clustering can be leveraged within applications to provide higher availability but you need to take special precautions to ensure a failure doesn’t impact the entire clustered application all at once. This may also extend beyond applications into other areas, for example if Active Directory or DNS is unavailable it may also affect your applications. In addition there are also many dependencies inside a virtual environment. One big one is shared storage, VMs can survive a host failure with HA features that bring them up on another host, but if your primary shared storage fails it can take down the whole environment.
The bottom line is that you have to know your dependencies ahead of time, you can’t afford to find them out when problems happen. You should clearly document what your applications need to be able to function and ensure you take that into account in our design considerations for your virtual environment. Something as simple as DNS being unavailable can take down a whole datacenter as everything relies on it. You also need to go beyond understanding your dependencies and configure your virtual environment and virtualization management around them. Doing things like setting affinity settings so when VMs are moved around they are either kept together or spread across hosts will help minimize application downtime and balance performance.
Tip 4 – Leverage VMware HA OS & application heartbeat monitoring
One of the little known functions of VMware’s High Availability (HA) feature is the ability to monitor both operating systems and applications to ensure that they are responding. HA was originally designed to detect host failures in a cluster and automatically restart VMs on failed hosts on other hosts in the cluster. It was further enhanced to detect VM failures such as a Windows blue screen by monitoring a heartbeat inside the VM guest OS through the VMware Tools utility. This feature is known as Virtual Machine (VM) Monitoring and will automatically restart a VM if it detects the loss of the heartbeat. To help avoid false positives it was further enhanced to detect I/O occurring by the VM to ensure that it was truly down before restarting it.
VMware HA Application Monitoring was introduced as a further enhancement to HA in vSphere 5 that took HA another level deeper, to the application. Again leveraging VMware Tools and using a special API that VMware developed for this you can now monitor the heartbeat of individual applications. VMware’s API allows application developers for any type of application, even custom ones developed in-house to hook in to VMware HA Application Monitoring to provide an extra level of protection by automatically restarting a VM if an application fails. Both features are disabled by default and need to be enabled to function, in addition with application monitoring you need to be running an application that supports it.
Tip 5 – Use the right tool for the job
You really need a comprehensive monitoring package that will monitor all aspects and layers of your virtual environment. Many tools today focus on specific areas such as the physical hardware or guest OS or the hypervisor. What you need are monitoring tools that can cover all your bases and also focus on your applications which are really the most critical part of your whole environment. Because of the interactions and dependencies with applications and virtual environments you also need tools that can understand them and properly monitor them so you can troubleshoot them more easily and spot bottlenecks that may choke your applications. Having a tool that can also simulate the user experience is especially important in a virtualized environment that has so many moving parts so you can monitor the application from end-to-end.
SolarWinds can provide you with the tools you need to monitor every part of your virtual environment including the applications. SolarWinds Virtualization Manager coupled with Server & Application Monitor can help ensure that you do not miss anything and that you have all the computing layers covered. SolarWinds Virtualization Manager delivers integrated VMware and Microsoft Hyper-V capacity planning, performance monitoring, VM sprawl control, configuration management, and chargeback automation to provide complete monitoring of your hypervisor.
SolarWinds Server & Application Monitor delivers agentless application and server monitoring software that provides monitoring, alerting, reporting, and server management. It only takes minutes to create monitors for custom applications and to deploy new application monitors with Server & Application Monitor’s built-in support for more than 150 applications. Server management capabilities allow you to natively start and stop services, reboot servers, and kill rogue processes. It also enables you to measure application performance from an end user’s perspective so you can monitor the user experience.
With SolarWinds Virtualization Manager and SolarWinds Server & Application Monitor you have complete coverage of your entire virtual environment from the bare metal all the way to the end user.