Updating VMware Virtual SAN HCL database offline

VMware offers certified compatibility guides which list System, I/O, Storage/SAN and Backup compatibility with VMware Infrastructure and previous versions of VMware ESX Server. VSAN Health Check leverage VMware Compatibility Guide database for various health checks stored on vCenter Server in place of VMware website.

VMware ships a copy of VMware compatibility guides database can be used for HCL Checks which was current when released. The VMware compatibility Guides database becomes outdated with time because of new certification with partners keep getting added to VMware Compatibility Guide database.  Hardware vendors regularly update their drivers and VMware adds certification for them. Therefore, it is critically important to keep the local copy up-to-date.

In one of my earlier post, I covered VSAN Hardware Compatibility list checker a very nice VMware Fling to verify the VSAN underline hardware.

VMWare VSAN HCL database can be automatically from VMware website if vCenter server can connect to Internet directly or via proxy. If you don’t have direct or proxy internet access, VMware VSAN HCL database can be downloaded manually and uploaded to vCenter Server. To download VMware VSAN HCL database manually, open the below URL in a web browser and save the content in a file with JSON extension.

http://partnerweb.vmware.com/service/vsan/all.json

Once you have VSAN updated HCL database JSON file, you can upload the same to vCenter Server using Web Client.

1 2

In this post we covered the steps to update VSAN HCL database offline. Thanks for Reading. Be social and share it in social media, if you fee worth sharing it. Happy Learning 🙂

 

VMware VSAN Network Design consideration

VMware Virtual SAN is a distributed shared storage solution that enables the rapid provisioning of storage within VMware vCenter. As Virtual SAN is a distributed shared storage, it is very much dependent on correctly configured network for Virtual Machines I/O and for communication between Virtual SAN Cluster nodes.  Because the majority of virtual machine I/O travels the network due to the distributed storage architecture, highly performing and available network configuration is critical to a successful Virtual SAN deployment.

In this post we will be covering few important points need to be considered from network perspective before VMware VSAN deployment.

Supported Network Interface Cards

In a VMware Virtual SAN hybrid configuration, Virtual SAN supports both 1 GB and 10 GB Network Interface Cards. If you have 1 GB Network Interface card installed on ESXi host than VMware requires this NIC to be dedicated only for Virtual SAN traffic. If a 10Gb NIC is used, this can be shared with other network traffic types. It is advisable to implement QoS using Network I/O Control to prevent one traffic to claim all the bandwidth. Considering the potential for an increased volume of network traffic between the hosts to achieve higher throughput, for Virtual SAN All Flash Configuration VMware supports only 10 GB Network Interface Card which can be shared with other network traffic types.

Teaming Network Interface Cards

Virtual SAN supports Route based on IP-hash load balancing, but cannot guarantee improvement in performance for all configurations. IP-hash performs the load balancing when Virtual SAN traffic type is among its many network traffic types. By design, Virtual SAN network traffic is not designed to load balanced across teamed network interface cards. NIC Teaming for VMware Virtual SAN Traffic is a way of making the Virtual SAN traffic network high available, where standby adapter take over the communication if primary adapter fails.

Jumbo Frame Support

VMware Virtual SAN supports Jumbo Frame. Even if use of Jumbo frame reduce CPU utilization and improve throughput, VMware recommends to configure jumbo frame only if the network infrastructure already supports it. As vSphere already use TCP segmentation offload (TSO) and large receive offload (LRO), Jumbo frame configured for Virtual SAN provides limited CPU and performance benefits. The biggest gains for Jumbo Frames will be found in all flash configurations.

Multicast Requirement

Multicast forwarding is a one-to-many or many-to-many distribution of network traffic. Rather than using the network address of the intended recipient for its destination address, multicast uses a special destination address to logically identify a group of receivers.

One of the requirements for VSAN is to allow multicast traffic on the VSAN network between the ESXi hosts participating in the VSAN cluster. Multicast is being used in discovering ESXi host and to keep track of changes within the Virtual SAN Cluster. Before deploying VMware Virtual SAN, testing performance of switch being used for Multicast is also very important. One should ensure a high quality enterprise switch is being used for Virtual SAN multicast traffic. Virtual SAN health services can also be leveraged to test Multicast performance.

Summary of network design considerations

  • Virtual SAN Hybrid Configuration support 1 GB and 10 GB network.
  • Virtual SAN All Flash Configuration support 10 GB network.
  • Consider implementing QoS for Virtual SAN Traffic using NIOC.
  • Consider Jumbo frame for Virtual SAN traffic if it is already configured in network infrastructure.
  • Consider NIC team for availability / redundancy for Virtual SAN traffic.
  • Multicast must be configured and functional between all hosts.

I hope this is informative for you. Thanks for Reading, be social and share it on social media if you feel it is worth sharing it.  Happy Learning … 🙂

VMware Virtual SAN Quiz

Questions mentioned in the quiz are for learning purpose only and are not from Exam perspective.

I hope this Quiz will be informative for you. Thanks for visiting this blog. Be social and share it in social media, if you feel worth sharing it.

Configuring Space Efficiency Technologies in VSAN 6.2

Virtual SAN 6.2 introduces space efficiency features optimized for modern all-flash storage. You can use space efficiency techniques to reduce the amount of space for storing data. Space efficiency features available in VSAN 6.2 are :

  • Deduplication and Compression on a Virtual SAN Cluster.
  • RAID 5 and RAID 6 erasure coding using Failure tolerance method policy attribute on VMs.

In this post, we will be covering the configuration of both the space efficiency features.

Deduplication and Compression

Enabling deduplication and compression can reduce the amount of physical storage consumed by as much as 7x. Environments with highly-redundant data such as full-clone virtual desktops and homogenous server operating systems will naturally benefit the most from deduplication. Deduplication and compression is a single cluster-wide setting that is disabled by default and can be enabled using a simple drop-down menu. This process does not incur virtual machine downtime and can be done online. The compression algorithm is applied after deduplication has occurred just before the data is written to the capacity tier.

When you enable deduplication and compression on a Virtual SAN all-flash cluster, redundant data within each disk group is reduced. Deduplication removes redundant data blocks, whereas compression removes additional redundant data within each data block. You can enable deduplication and compression as a cluster-wide setting, but they are applied on a disk group basis. The amount of storage reduction from deduplication and compression depends on the type of data stored and the number of duplicate blocks.

Deduplication and Compression Design Considerations :

  • Only available on ALL Flash Disk Groups.
  • On-disk format version 3.0 or later is required.
  • Storage-claiming method should be set to manual. Storage-claiming method to automatic after deduplication and compression has been enabled.
  • Capacity overhead for deduplication and compression is approximately five percent of total raw capacity.
  • Using policies with 100 percent proportional capacity reservations can make deduplication and compression less efficient.
  • Virtual SAN can eliminate duplicate data blocks within each disk group, but not across disk groups.

Procedure to configure Deduplication and Compression

Login to vSphere Web Client. Select the Cluster –> Manage –> Settings. Click on Edit to enable Deduplication and Compression.

Screenshot-18Select Enabled from the drop down list.Screenshot-19

Screenshot-20Once Deduplication and Compression is enabled. Virtual SAN performs a rolling reformat of every disk group on every host.
Screenshot-21It will take some time to get the duplication blocks of data removed. You can check the results of deduplication and compression by viewing the Deduplication and Compression Overview in the Virtual SAN Capacity monitor.
Screenshot-22

Screenshot-27

Configuring RAID 5 or RAID 6 Erasure Coding

You can use RAID 5 or RAID 6 erasure coding to protect against data loss and increase storage efficiency with same level of data protection as provided in RAID 1, while using less storage capacity.

storage-space-requirement

Below is the Host Requirements Based on Failure Tolerance Method. Due to the host or fault domain requirements of the RAID-5/6 (Erasure Coding) rule, Virtual SAN Stretched Cluster and 2 node configurations are not supported.

host-requirement

In VSAN 6.2, RAID5 and RAID6 technologies, there is no “parity host” and data parity can be on any capacity device.

parity-bit-scratered

RAID 5 and RAID 6 Design Consideration

  • On-disk format version 3.0 or later is required to support RAID 5 or RAID 6.
  • RAID 5/6 is not supported on stretched clusters.
  • RAID 5 or RAID 6 erasure coding is available only on all-flash disk groups.
  • Additional space savings by enabling deduplication and compression on the Virtual SAN cluster.

Procedure to configure RAID 5 and RAID 6.

RAID 5 and RAID 6 can be made available using VSAN policy. If FTT = 1 is configured in VSAN Policy, RAID 5  will get configure on the Virtual Machine disk.

Screenshot-24

Assign the VSAN Policy to the Virtual machine.

Screenshot-15

Screenshot-16

If FTT = 2, RAID 6 will be get configured on Virtual Machine disk.

Screenshot-25

Screenshot-26

This concludes the configuration of VSAN Storage efficiency techniques in VSAN 6.2. I hope this is informative for you. Thanks for reading !!!. Be social and share it in social media, if you feel worth sharing it.

 

 

 

Health and Performance Monitoring Service in VSAN 6.2

Being a VSAN administrator we generally use VSAN observer while troubleshooting VSAN Performance issues. VSAN observer was first introduced in 5.5 U1 GA and is a part of Ruby vSphere console ( RVC )  tool an interactive command line shell for vSphere management. VSAN observer allows to dig deep into VSAN’s performance, see the IOPS, latency, Outstanding I/Os and congestion’s and lot more. The information is provided at different layers in the Virtual SAN stack to help troubleshoot storage performance. Even VSAN Observer is a powerful tool it itself have some drawbacks.

Drawbacks of VSAN Observer :

  • VSAN Observer only provides real-time view of the system. It does not provide any historic performance data.
  • Not integrated with vSphere Web Client
  • VSAN Observers is a complex tool
  • Impacts vCenter Server as the tool is launched via RVC on vCenter Server.

To overcome these drawbacks, in VSAN 6.2 introduced Performance Service to get detailed understanding of VSAN performance. Virtual SAN performance service monitors the performance of your Virtual SAN environment, and investigate potential problems. The performance service collects and analyzes performance statistics and displays the data in a graphical format. You can use the performance charts to manage your workload and determine the root cause of problems. You can view detailed performance statistics for the cluster, and for each host, disk group, and disk in the Virtual SAN cluster. You also can view performance charts for virtual machines and virtual disks.

VSAN Performance service is disabled by default. Turn on Virtual SAN performance service to monitor the performance of Virtual SAN clusters, hosts, disks, and VMs. When you turn on the performance service, Virtual SAN places a Stats database object in the datastore to collect statistical data. The Stats database is a namespace object in the cluster’s Virtual SAN datastore. Performance and Health monitoring of VSAN Components can be performed on Cluster as well as on individual host.

Procedure to Enable VSAN Performance Service

  • Login to vSphere Web Client and navigate to Virtual SAN Cluster.
  • Click on Manage tab and click settings.
  • Select Health and Performance, click on Edit to edit the performance service  settings.

ScreenshotSelect the Turn On Virtual SAN performance service check box. Select a storage policy for the Stats database object and click OK. Screenshot-1

You can see Health and Performance service is enabled and status as Healthy and compliant.

Screenshot-2Performance and Health monitoring of VSAN Components can be performed on Cluster as well as on individual host. Select

  • Virtual SAN – Virtual Machine Consumption : Virtual SAN displays performance charts for clients running on the Cluster / Host, including IOPS, throughput, latency, congestions, and outstanding I/Os.
  • Virtual SAN – Backend : Virtual SAN displays performance charts for the cluster back-end operations, including IOPS, throughput, latency, congestions, and outstanding I/Os.

Screenshot-3

Click on Ask VMware to know more details on the counter. Once you click Ask VMware, it will open a Knowledge Base Article related to the counter.

Screenshot-4

Screenshot-7

Screenshot-8

Screenshot-9

Screenshot-10

This concludes the configuration of Health and Performance service in VSAN 6.2. I hope this is informative for you. Thanks for reading !!!. Be social and share it in social media, if you feel worth sharing it.

 

 

Deploy two Node VSAN Cluster using VSAN Configuration Wizard – VSAN 6.2

With VSAN 6.2, VMware simplified the process of deploying VSAN Stretched and VSAN two host cluster. VSAN Stretched cluster allows to protect virtual machines across data centers where as 2-node VSAN is an excellent solution for remote office / branch office (ROBO) configuration. In both the scenarios, a dedicated witness host is required.

Virtual SAN for ROBO use Fault Domains, where in this case the required failure zones are based on three nodes (two physical nodes and witness node). Witness node are used to provide an availability mechanism to virtual machines. Witnesses plays an important role in ensuring that more than 50% of the components of an object remain available. Instead of deploying a dedicated physical ESXi host to be a witness host, as well as consume a license for the witness host, Vmware deployed the VSAN witness appliance to take care of the witness requirements.  There are no licenses needed with the witness appliance, as it comes per-configured with a license. Of course, if you wish to use a physical host for the witness, you can do that too.

If you choose to use witness host as a physical server not a witness appliance, it is required to do the configuration manually on the host.

You should have

  • One flash
  • One capacity disk.
  • One Port Group to have VSAN Traffic enabled.
  • Witness host should not be part of VSAN Cluster

In Lab I have deployed a physical host to be used as Witness host. You can use the Configure Virtual SAN wizard to complete the basic configuration of your Virtual SAN cluster.

  • Navigate to an existing cluster in the vSphere Web Client.
  • Click the Manage tab and click Settings.
  • Under Virtual SAN, select General and click Configure to open the Configure Virtual SAN wizard.

ScreenshotChoose appropriate Disk Claiming technique to your environment.

  • Automatic: Claims all empty devices on the included hosts for Virtual SAN.
  • Manual : Requires manual claiming of the devices on the included hosts.

Choose Two host Virtual VSAN Cluster.  Screenshot-1Configuration Wizard will check if VSAN Traffic is allowed on atleast one portgroup on both the nodes of the cluster.Screenshot-2Mark disk to be used for Capacity / Cache Tier. Click Next.Screenshot-3 Select the node to be used as Witness Host.Screenshot-5Mark the disk for Cache / Capacity Tier on the witness host and click NEXT.Screenshot-6Click Finish to proceed further.Screenshot-8Once configured, one of the node will be marked as Preferred and other will be Secondary.Screenshot-9The virtual machine components will be scattered across nodes ensuring that more than 50% of the components of an object remain available in case of host failure.Screenshot-10

Screenshot-11

This conclude the deployment of VSAN 2 Node Cluster using VSAN Configuration wizard in VSAN 6.2. I hope this is informative for you. Thanks for reading !!!. Be social and share it in social media, if you feel worth sharing it.

Deploy VSAN Cluster using VSAN Configuration Wizard – VSAN 6.2

VMware announced VSAN 6.2 with vSphere 6.0 Update 2. With VSAN 6.2 VMware simplified the configuration of VSAN Cluster using VSAN Configuration wizard. Using VSAN Configuration Wizard VSAN Cluster can be deployed in few clicks.  Configuration wizard will check all the necessary configuration related to network and disk and prompt in case of any issues. In this post, I will be covering the process to create a VSAN Cluster using configuration wizard.

Procedure to deploy VSAN Cluster

Login to vSphere Web Client. Click on Cluster –> Manage –> Settings –> General. Click on Configure.ScreenshotSelect Disk Claiming option to Automatic / Manual depending on your requirement. As we are deploying a simple VSAN Cluster, Choose Do not configure under Fault Domain and Stretched cluster.Screenshot-1You should have SAN Traffic enabled on port group on all the host participating in VSAN Cluster.Screenshot-2Mark the disks connected to hosts for Cache Tier or Capacity Tier. Click Next.Screenshot-3Click on Finish.Screenshot-4

Screenshot-5

Screenshot-6

This conclude the deployment of VSAN Cluster using VSAN Configuration wizard in VSAN 6.2. I hope this is informative for you. Thanks for reading !!!. Be social and share it in social media, if you feel worth sharing it.