Windows Server 2016 LACP with Juniper EX Virtual Chassis

I’m not a Juniper guy or even a networking guy for that matter, but trying to help out a customer who has a few HPE DL380 boxes with Windows Server 2016 on them containing 4 NICs @ 1 gbit each, connected to Juniper EX series Virtual Chassis (stack). The customer needed more bandwith, so we tried to use the Windows Server NIC teaming feature, which has been around since Windows Server 2012.

The Windows side of things is very straight forward. I’ll show the details later down, first I needed to configure the switch.

Start with creating an Aggrated Link Group (LAG) using the command:

Juniper labels the LAG with ae followed by some number, I just took the next available. (You can list all existing with show interfaces ae and pressing tab. Or show lacp interfaces.) So in this example number 22 is used.

set interfaces ae22 aggregated-ether-options lacp active
set interfaces ae22 aggregated-ether-options lacp periodic fast
set interfaces ae22 unit 0 family ethernet-switching interface-mode access
set interfaces ae22 unit 0 family ethernet-switching vlan members Server
set interfaces ae22 description "To Server MyServer"

And for each of the interfaces:

set interfaces ge-1/0/4 description MyServer-NIC-A
set interfaces ge-1/0/4 speed 1g
set interfaces ge-1/0/4 link-mode full-duplex
set interfaces ge-1/0/4 ether-options 802.3ad ae11

Sadly I ran into an error :

[edit interfaces ge-1/0/4]
'unit 0'
logical unit is not allowed on aggregated links
error: xSTP configuration disallowed on interface ge-1/0/4 as it is part of an aggregate bundle
error: configuration check-out failed

Listing the configuration of the interfaces (show interfaces ge-1/0/4) didn’t show any unit 0.

admin@switch# show interfaces ge-1/0/4
description MyServer-NIC-A;
speed 1g;
link-mode full-duplex;
ether-options {
802.3ad ae22;
}

I found someone that suggested to create unit 0, and than deactivating it:

set interfaces ge-1/0/4 unit 0 family ethernet-switching
deactivate interfaces ge-1/0/4 unit 0

But still no luck. As I said, I’m completely new to Juniper.

Then I found that the switches were configured to use something called interfaces ranges. It’s quite a nice concept, in which you can group interfaces and give them a common configuration.

show interface-range

I found that some of the interfaces I wanted to use for my LAG were included in an interface range. Within the interface range a unit 0 was defined. All it did was set the vlan, but it was standing in the way of my efforts. So I removed my interfaces from the interface range.

delete interfaces interface-range RangeName member ge-1/0/4

And what do you know? commit check, didn’t trow any error.

Commiting the config may result in loosing connectivity, make sure you have another way of getting into the machine (iLO/iDRAC/IPMI/iBMC)

I had an iLO connection open, and could connect to the console of the machine.

Just open Server Manager, go to Local Server and click the ‘Disabled’ link just right of ‘NIC Teaming’. In the NIC Teaming window, click TASKS, New Team.

Select all Interfaces, choose LACP as Teaming mode: and Dynamic for Load Balancing Mode. Click OK.

After a few moments the Team will move from Failed to the OK status
admin@switch# run show interfaces terse | match ae22    
ge-1/0/4.0 up up aenet --> ae22.0
ge-2/0/4.0 up up aenet --> ae22.0
ge-3/0/4.0 up up aenet --> ae22.0
ge-4/0/4.0 up up aenet --> ae22.0
ae22 up up
ae22.0 up up eth-switch
Whohoo, Windows sees a 4 Gbps link

I found that since the customer uses static IP addresses, the wizard doesn’t migrate the IP address. I needed to set the static IP on the Team adapter. I ended up setting the default gateway again, since it didn’t save it in the configuration. The default gateway was prefilled from DHCP, but the gui didn’t save, while it was displayed… Took me a few minutes to realize that the missing ping reply was due to this, and not the config of the switch.

We didn’t push anything, but the machine just resumed its tasks and consumed 2 Gbps

The complete configuration as used on the Juniper switch:

(It should work without the created and disabled unit 0 on the interfaces, but I haven’t tried that)

All was done on Juniper EX3400 switches loaded with JUNOS 15.1 in a Virtual Chassis configuration

#Create the LAG
set interfaces ae22 aggregated-ether-options lacp active
set interfaces ae22 aggregated-ether-options lacp periodic fast
set interfaces ae22 unit 0 family ethernet-switching interface-mode access
set interfaces ae22 unit 0 family ethernet-switching vlan members Server
set interfaces ae22 description "To Server MyServer"

#Remove any interfaces from the interface-range
delete interfaces interface-range RangeName member ge-1/0/4
delete interfaces interface-range RangeName member ge-2/0/4
delete interfaces interface-range RangeName member ge-3/0/4
delete interfaces interface-range RangeName member ge-4/0/4

#Configure the first interface
set interfaces ge-1/0/4 description MyServer-NIC-A
set interfaces ge-1/0/4 speed 1g
set interfaces ge-1/0/4 link-mode full-duplex
set interfaces ge-1/0/4 unit 0 family ethernet-switching
deactivate interfaces ge-1/0/4 unit 0
set interfaces ge-1/0/4 ether-options 802.3ad ae22

#Configure the second interface
set interfaces ge-2/0/4 description MyServer-NIC-B
set interfaces ge-2/0/4 speed 1g
set interfaces ge-2/0/4 link-mode full-duplex
set interfaces ge-2/0/4 unit 0 family ethernet-switching
deactivate interfaces ge-2/0/4 unit 0
set interfaces ge-2/0/4 ether-options 802.3ad ae22

#Configure the third interface
set interfaces ge-3/0/4 description MyServer-NIC-C
set interfaces ge-3/0/4 speed 1g
set interfaces ge-3/0/4 link-mode full-duplex
set interfaces ge-3/0/4 unit 0 family ethernet-switching
deactivate interfaces ge-3/0/4 unit 0
set interfaces ge-3/0/4 ether-options 802.3ad ae22

#Configure the last interface
set interfaces ge-4/0/4 description MyServer-NIC-D
set interfaces ge-4/0/4 speed 1g
set interfaces ge-4/0/4 link-mode full-duplex
set interfaces ge-4/0/4 unit 0 family ethernet-switching
deactivate interfaces ge-4/0/4 unit 0
set interfaces ge-4/0/4 ether-options 802.3ad ae22

show | compare
commit check

Daisy Chaining VMware UMDS

I was wondering if one could daisy chain multiple VMware Update Manager Download Service (UMDS) appliances. The documentation doesn’t say a word about it. The only thing I found Googling this was one blog that say’s it can’t be done. But that blog was from 2014, now, 2018, let’s see…

I started with a Ubuntu 16.04 LTS server and used William Lam’s script to install the UMDS on top. It needed some more config:

(I tested this in Fusion virtual machines using vSphere 6.5 Update 1 (5969303))

Open the console of the first UMDS

sudo -i
mkdir /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
apt-get install openssh-server -y
ifconfig #This gives the IP address (Ubuntu in Fusion creates an ens33 interface)

Using a SSH (Windows: PuTTY / Mac: Termius) client, connect to the UMDS:

ssh vmninja@172.16.250.129

In the SSH session:

sudo -i
wget https://github.com/lamw/vghetto-scripts/raw/master/shell/install_umds65.sh
chmod +x install_umds65.sh
./install_umds65.sh /mnt/cdrom/umds/VMware-UMDS-6.5.0-5939545.tar.gz UMDSDB UMDS_DSN umdsuser VMware1!
/usr/local/vmware-umds/bin/vmware-umds -v
/usr/local/vmware-umds/bin/vmware-umds -G
/usr/local/vmware-umds/bin/vmware-umds -S --add-url https://vibsdepot.hpe.com/index.xml --enable-host --url-type HOST
/usr/local/vmware-umds/bin/vmware-umds -D

(In the example above I’ve added the HPE VibsDepot, to see if non-firstparty updates will get downloaded.)
It will now start downloading… It may take some time to complete, at the time of writing it was about 65 GB

Per William Lam’s suggestion for this test I used Python’s builtin webserver:

apt-get install python-minimal -y
cd /var/lib/vmware-umds
python -m SimpleHTTPServer 80

Using this as a foreground task, it shows all HTTP requests being received:

172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/HPQ/metadata-hpnmi-vmware55-bundle-2.3-6.zip HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/csco/__hostupdate20-consolidated-metadata-index__.xml HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/csco/csco-VEM-5.5.0-metadata.zip HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/vmw/__hostupdate20-consolidated-metadata-index__.xml HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/vmw/vmw-ESXi-5.5.0-metadata.zip HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:19] "GET /hostupdate/vmw/vmw-ESXi-6.0.0-metadata.zip HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:20] "GET /hostupdate/vmw/vmw-ESXi-6.5.0-metadata.zip HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:20] "GET /vaupgrade/bootstrap_index.xml HTTP/1.1" 200 -
172.16.250.150 - - [18/Feb/2018 05:28:20] "GET /vaupgrade/__valm-consolidated-index__.xml HTTP/1.1" 200 -

But first I needed to build a second UMDS, mostly the same as above, except:

  1. I didn’t add the HPE VibsDepot
  2. I pointed hostupdate.vmware.com and vapp-updates.vmware.com in the hosts file to localhost, by editing the hosts file to prevent it to try to download anything directly from VMware.com’s website.
    • For some reason it is not possible to remove these entries from UMDS’s config.
    • Nor to remove the updates for older versions. I didn’t need updates for any ESXi prior to 6.5.
  3. vmninja@ubuntu:~$ cat / etc/hosts
    127.0.0.1       localhost
    127.0.1.1       ubuntu
    127.0.0.1       hostupdate.vmware.com
    127.0.0.1       vapp-updates.vmware.com
    
    # The following lines are desirable for IPv6 capable hosts
    ::1     localhost ip6-localhost ip6-loopback
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    
  4. Instead of the HPE VibsDepot, I added the first UMDS as source:
    /usr/local/vmware-umds/bin/vmware-umds -S --add-url http://172.16.250.129/hostupdate/__hostupdate20-consolidated-index__.xml --enable-host --url-type HOST
    /usr/local/vmware-umds/bin/vmware-umds -S --add-url http://172.16.250.129/vaupgrade/__valm-consolidated-index__.xml --enable-va --url-type VA
    
  5. Start the download task:

    /usr/local/vmware-umds/bin/vmware-umds -D
    

After downloading was complete, I added the second UMDS to vCenter Update Manager:
UpdateManagerConfig

After clicking download now, the Non VMware Patches are visible in vCenter:
PatchRepository

So it seams to be possible to daisy chain UMDS. I have no idea about the supportability of this configuration, and if using something in production, use a real webserver!

This post was published originally on my personal blog https://vmninja.wordpress.com