No Gravatar

This is the design that I constructed and implemented for my last companies Vsphere 4.0 Update 2 upgrade and hardware refresh for production virtual environment, I created two highly available vSphere clusters which I like to call “vClusters” using the latest HP blade technology with HP Virtual Connect and Flex-10. I was able to create a very dynamic system with 2 clusters which could easily be scaled to 4.

Hardware:

  • 2 HP Blade Chassis each equipped with 2 Flex-10 and 2 8gb Virtual Connect
  • Each Chassis is interconnected with 4 CX-4 stacking cables 2 per per Chassis side running between the Flex-10 modules
  • 18 Bl 460s G6 each with Intel Westmere Nehalems 32 nms 6 core procs each equipped with 48gb of memory
  • SAN 2 HP EVA 8400s
  • SAN Core Brocade 48000 (4GB director series)
  • Networking Core Cisco 6509s
  • 1 DataDomain DD 560

VMware Environment:

  • Licensing – All Enterprise Plus for dvs, host profiles, storage i/o (future), 12 core processors (future)
  • Each Cluster will hold 100-125 Virtual Machines with room for more than double the capacity
  • VMware thin provisioning (reduced storage by more than 200%)
  • Estimated capacity max per blade 30 VMS
  • 2 vClusters each with 8 servers 1 Server for HA reserved; fully automated DRS with DPM configured (not fully automated)
  • 2 Sandbox Servers Clustered with Private Virtual Honeypot
  • VMs each upgrades to virtual hardware 7 with VMware vmxnet 3
  • Vranger Pro 4.5
  • 4 resource pools per cluster
  1. Templates – CPU and Share Resources kept to a minimum. The templates are actually powered on VM’s why? Who likes patching ;)
  2. Delete – A resource pool with no resources mainly used to put VMs that are powered off and waiting to be deleted
  3. Prod – A resource pool with shares set to high for both CPU and Memory with expandable reservation
  4. Dev/Test – A resource pool with shares set to normal for both CPU and Memory with expandable reservations

Networking:

  • 80 gb uplinks to core router (Cisco 6509) 20 gb trunk per flex-10 module (2 flex-10) modules per chassis.
  • Flex-10 (Active/Active) 20 GB of networking to each blade with 20gb of networking between blades inter chassis (read about the configuration for Flex-10 and Virtual connect here)
  • dVs Fault Tolerance -Private Network – Non Routable only communicates within Blade Chassis
  • dVs Vmotion – Private Network – Non Routable only communicates within Blade Chassis
  • dVs Virtual Machines- Different Port groups each for different Vlans for Dev/Test/Prod
  • vS Service Console
    Note: In 4.1 I would change this design and route VMotion, and do mapped VLANS and 1 dVs for Vmotion/Service Console/Virtual Machines Dev/Virtual Machines Test…Id keep fault tolerance on a seperate private switch. However with the main dVs switch I would encorporate Network I/O control to effectively and dynamically utilize the 10gb pipe this would also solve the issue of the egress problem with flex-10 only controlling traffic one way.

Storage and Backup:

  • vRanger Pro 4.5 – Installed on VMs, configured to backup vClusters 50 VMs per hour very effective 50 vms per hour backup 100% success rate on backups 0 errors or troubleshooting. I honestly never thought that I would see the day after troubleshooting VCB for 2 years backups this good.
  • DD 560 set up with CIFS share for VMware backups, ESX boxes backup directly to DD560. Pre thin provision 40:1 compression ratio.
  • LUNS presented to each cluster with standard size of 500gb. sVMotion capability between clusters

Read More