Deployment ESXi with duplicated MAC-addresses

After deploying new ESXi 4.1 hosts trough Altiris and adding them to the Virtual
Center Server, I saw that some of the hosts repeatedly were being disconnected
and connected again. Also the new hosts were not reachable from the network.

At first we thought of network problems although other servers in the datacenter did not
experience this sort of problems.

Further investigation led me to problems with the MAC-addresses. The routers were
giving messages stating “Host 00:25:B3:XX:XX:XX in vlan 250 is flapping between
port Gi4/8 and port Gi4/5” and ICMP-request(ping) to the hosts suffered from packet
loss. Having seen this symptoms before, I was sure that it had something to do with
duplicated MAC-addresses and it had to be a software-related problem because
hardware MAC-addresses are unique.

When checking the configuration of the ESXi hosts, I saw that the MAC-address of vmk0 (this is the virtual NIC for the management network) was the same on more than one
host. Unfortunately the GUI for ESXi management (the vSphere client) doesn’t provide the option to reset a MAC-address, so I had to get in the console of ESXi and try to set things straight from there.

Looking for the right commando’s and syntax I came across an article from VMware which
described my problem as a bug and offered a workaround (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111).
Quote:
Work around the issue by manually configuring the MAC address on the ESX host as follows:

  1. In the troubleshooting console run the following command: esxcfg-advcfg -s 1 /Net/FollowHardwareMac.
  2. Restart the server.

Unfortunately this workaround didn’t work for me. After a reboot the host still
had a spoofed MAC-address. I also tried setting the value to 0 so vmk0 would
get a VMware MAC-address, but that also didn’t do the trick.

Eventually there was nothing left to do, then to remove vmk0 and recreate it with the same
ip-parameters. This included two simple steps:
1. esxcfg-vmknic -d "Management Network"
2. esxcfg-vmknic -a -i 10.X.X.X -n 255.X.X.X "Management Network"

After completing the two steps above on all new ESXi hosts, all errors were gone and everything worked like a charm.