Wow, that was quite a lengthy title for this post but I felt it described it accurately.   

This article is in response to a recent query I had regarding providing network resilience for Management and General VM networks on VMware ESXi 4.0 (also applies to ESXi 3.5) when you only have two physical NICs.  Please note that this article applies to ESXi and not ESX (minus the i).  As you probably know ESX (minus the i) has the requirement of an additional port for Service Console (SC) traffic.

In this example I am using a Dell PowerEdge R710 which actually comes with four x 1Gb ports on the back though for the purposes of these examples I have only connected two of the physical NICs ports.  Servers such as the HP Proliant DL360 G5/G6 as default only come with two x 1Gb ports and with tight budgets there may not be the luxury of purchasing additional network ports via a PCIe expansion card.

Following VMware’s best practice (see VI Performance Turning Guide, pg 7) of having a dedicated Management and VM Network port (and with the full fat version of ESX a Service Console (SC) port) each of which would also have a level of resilience in the form of an additional failover port(s) we would ideally need 4 physical network ports for a standard non iSCSI or vMotion enabled ESX host (see below):

Some people use VLAN’ing to separate their networks when presented with limited physical ports in a ESX host though often what you see is the configuration in the screenshot below where the Management and VM Networks are on separate physical NICs.  In the unlikely event that one of these fails then connectivity to that particular Port Group will be lost.

Rather than the two Port Groups sharing the two physical NICs and using VLAN’ing to separate the traffic of each there is any easier way of providing network resilience to a two NIC ESX host. 

First of all you have to add both the ‘Virtual Machine’ (VM Network) and ‘VMKernel’ (Management Network) to a single Virtual Switch.  Then to this Virtual Switch add the two physical adapters/NICs.  You should end up with a configuration below:

Next we want to specify which of the two physical NICs will be the ‘Active’ and ‘Standby’ Adapters for each of two Port Groups. It doesn’t really matter what order this is done in though for this example I am selecting the ‘Management Network’ after pressing the ‘Properties’ button (see screenshot above).

After selecting to edit the ‘Management Network’ and clicking on the ‘NIC Teaming’ tab you should see both NICs as being ‘Active Adapters’.  Take note of which NIC (eg: vmnic1) you are moving and using the ‘Move Down’ button make it a ‘Standby Adapter’ for the ‘Management Network’.

After making this change all of your ‘Management Network’ traffic will run over ‘vmnic0’ though should this connection fail due to network port  or cable failure then it will use ‘vmnic1’.

Next you want to change the NIC adapter Active/Standby order for the ‘Virtual Machine Network’.  Follow the same procedure as outlined above though…

…when you get to specifying which is the ‘Active’ and ‘Standby’ adapter ensure that these are the reverse of what you specified for the ‘Management Network’ port group (ie: ‘vmnic1’ should be the ‘Active’ and ‘vmnic0’ the ‘Standby’.

After completing this the ‘Management Network’ or the ‘Virtual Machine Network’ Port Group traffic are running over separate physical NIC ports though they can now withstand a single physical NIC failure.  In the event of this occurring both lots of traffic will be travelling over the same physical NIC port though this is much better than any service downtime and also gives you time to resolve the issue.

To eliminate another single point of failure it is a good idea to have each of the physical NICs ports patched into separate physical switches.