how to deploy a nexus 1000v lab with a single esx host

How to Deploy a Nexus 1000v lab with a single ESX host.

By Robert Burns – CCIE Data Center #37856

*Slight variation works with VMware Workstation/Fusion also.

*Details of third party OS & application installation is beyond the scope of this guide.

Pre-requisites:

-Disable Windows firewalls on your client

-Java must be installed

Example Host List

Host IP VM Name Description

10.85.49.215 Bare Metal ESX host 10.85.49.216 RHEL62-Test-1 Redhat test VM 10.85.49.217 vCenter-5 vCenter server 10.85.49.218 ESX5-Nested-1 Nested ESX 5.1 (VM 1)

10.85.49.219 ESX5-Nested-2 Nested ESX 5.1 (VM 2) 10.85.49.220 N1000v VSM Management IP

1. Infrastructure setup

a. Install ESX bare metal host. Ensure you have > 75GB available space on your VMFS if

you plan on installing a vCenter VM with Update Manager.

b. Configure management network interface.

c. Install one WIN2K8/2012 VM (for vCenter) allocated 30GB or more for virtual disk. Next

install vCenter 5.1 on Windows VM. Alternately you can use the vCenter appliance.

d. On the vCenter server install VMware Update Manager (VUM).

e. Install two ESX 5.1 VMs following Nested ESX Instructions here:

http://www.vcritical.com/2011/07/vmware-vsphere-can-virtualize-itself/

i. Configured four vNICs each using the default vSwitch “VM Network” port group

for now.

ii. Boot Nested ESX VMs and configure Management network interfaces from the

VM console within VI Client.

f. Test all IP connectivity between vCenter and all three ESX hosts.

g. Add Nested ESX hosts to vCenter.

See Fig. 1

http://www.vcritical.com/2011/07/vmware-vsphere-can-virtualize-itself/

Tip: Put the Nested ESX hosts in their own Cluster for easier management.

Note: The two “ESX5-Nested-X” VMs below correspond to 10.85.49.218 & 10.85.49.219 hosts connected to vCenter in the “ESX-

Nested” cluster.

Fig. 1 – Infrastructure setup with Nested ESX VMs installed & added to vCenter.

2. Deploy the 1000v VSM

a. Download the latest 1000v SW from CCO.

*For ESX 5.1 and later you must use 1000v version Nexus1000v.4.2.1.SV2.1.1 or later.

http://software.cisco.com/download/release.html?mdfid=282646785&flowid=3090&softwa

reid=282088129&release=4.2%281%29SV2%281.1a%29&relind=AVAILABLE&rellifecycle=&r

eltype=latest

http://software.cisco.com/download/release.html?mdfid=282646785&flowid=3090&softwareid=282088129&release=4.2%281%29SV2%281.1a%29&relind=AVAILABLE&rellifecycle=&reltype=latest



b. Unzip the bundle and navigate to the Install_App folder & launch the application.

c. Start with the VSM Complete Installation – selecting Custom install.

d. Review the pre-reqs & click Next.

e. Enter the appropriate details for your vCenter

f. Enter the appropriate info for your VSM. Since I’m hosting the VSM pair on my single

bare metal ESX host, I’ve used the same ESX Host IP twice. Click Next when complete.

Notes:

- Recommend setting your Domain ID to something other than default 1.

- I’m setting my Management VLAN to what my physical switches use for

their native VLAN.

- I chose to use L2 mode for simplicity versus L3, but either will work.

- Do not migrate hosts at this time.

g. Review configuration and click Next. Be patient, deployment will take up to 15mins.

h. Next step will prompt to add additional Modules.

i. Select the hosts you wish to have the VEM agent installed. Click Next. Note: This method requires VUM to be previously installed. If not, you’ll need to manually install the VEM

agent vibs.

j. Review and then click Finish to proceed.

You can monitor the progress from the VI Client Recent Task log

k. The Install App hopefully completed successfully for all hosts.

**If the VEM installation fails, it likely points to a problem with VMware Update

Manager (VUM).

l. From the VI Client go to Home -> Inventory -> Networking and you should see your two

new hosts as part of the 1000v DVS. Ensure you click on the 1000v DVS in the left pane.

m. SSH into the VSM and and check the modules

N1000v(config)# show mod

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 0 Virtual Supervisor Module Nexus1000V active *

2 0 Virtual Supervisor Module Nexus1000V ha-standby

3 248 Virtual Ethernet Module NA ok

4 248 Virtual Ethernet Module NA ok

Mod Sw Hw

-- ------------------ ------------------------------------------------

1 4.2(1)SV2(1.1) 0.0

2 4.2(1)SV2(1.1) 0.0

3 4.2(1)SV2(1.1) VMware ESXi 5.1.0 Releasebuild-838463 (3.1)

4 4.2(1)SV2(1.1) VMware ESXi 5.1.0 Releasebuild-838463 (3.1)

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

2 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

3 02-00-0c-00-03-00 to 02-00-0c-00-03-80 NA

4 02-00-0c-00-04-00 to 02-00-0c-00-04-80 NA

Mod Server-IP Server-UUID Server-Name

--- --------------- ------------------------------------ --------------------

1 10.85.49.220 NA NA

2 10.85.49.220 NA NA

3 10.85.49.218 422954ef-1f4d-f096-2a47-4b64cd67b932 10.85.49.218

4 10.85.49.219 42299b1d-8226-47bc-f375-2432c7cbe87e 10.85.49.219

3. Migrate Test VM & Test Connectivity

a. Cold migrate (powered off) your Test VM to one the Nested ESX hosts. If you get any errors, you’ve likely done one of

the following:

- Didn’t modify your Nested ESX VM to Virtual Machine Version 9 prior to install

- Didn’t enable the “Expose NX/XD flag to guest” option in the Nested ESX VM.

b. Before we power it up we’re going to create a Port Profile for it on the 1000v.

N1000v(config)# port-profile type vethernet rhel-pp

N1000v(config-port-prof)# switchport mode access

N1000v(config-port-prof)# switchport access vlan 711

N1000v(config-port-prof)# state enabled

N1000v(config-port-prof)# no shut

N1000v(config-port-prof)# vmware port-group

c. Now change the virtual network binding of your test VM from the vSwitch to the 1000v port profile.

d. Power up your test VM, and verify the interface on the 1000v.

N1000v(config)# show interface virtual

-------------------------------------------------------------------------------

Port Adapter Owner Mod Host

-------------------------------------------------------------------------------

Veth1 Net Adapter 1 RHEL62-Test-1 3 10.85.49.218

N1000v(config)#

Assuming your networking & port profiles are setup correctly you should have connectivity to your Test VM.

4. Advanced Configuration (optional)

a. Now that we have basic connectivity, let’s add the remaining uplinks to your Nested ESX VEM hosts.

Select the host – Configuration – Networking – vSphere Distributed Switch tab – Manage Physical Adapters

b. Find the uplink port profile and click “Add NIC”. Add each of the 2 remaining NICs from each host.

c. Verify the uplinks on the 1000v. Your uplink port profile should be configured for mac pinning in which case you should

see two new Port channels automatically created.

N1000v(config)# show int brief

--------------------------------------------------------------------------------

Port VRF Status IP Address Speed MTU

--------------------------------------------------------------------------------

mgmt0 -- up 10.85.49.220 1000 1500

--------------------------------------------------------------------------------

Ethernet VLAN Type Mode Status Reason Speed Port

Interface Ch #

--------------------------------------------------------------------------------

Eth3/2 711 eth trunk up none 1000 1






--------------------------------------------------------------------------------

Port-channel VLAN Type Mode Status Reason Speed Protocol

Interface

--------------------------------------------------------------------------------

Po1 711 eth trunk up none a-1000(D) none

Po2 711 eth trunk up none a-1000(D) none

<snip>

5. Exercise - Determine which uplink your Test VM is utilizing

i. Find which module the VM is hosted by.

N1000v(config)# show int virtual

-------------------------------------------------------------------------------

Port Adapter Owner Mod Host

-------------------------------------------------------------------------------

Veth1 Net Adapter 1 RHEL62-Test-1 3 10.85.49.218

ii. Identify the Sub Group IDs of all uplinks on that host.

N1000v(config)# module vem 3 execute vemcmd show port

LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type

18 Eth3/2 UP UP FWD 305 1 vmnic1



49 Veth1 UP UP FWD 0 2 RHEL62-Test-1.eth0

305 Po1 UP UP FWD 0

<snip>

You can see from the output, SGID 1 = vmnic1, 2 = vmnic2 and 3 = vmnic3

iii. Find the VMs pinned Sub Group ID from the same ouput.

N1000v(config)# module vem 3 execute vemcmd show port






305 Po1 UP UP FWD 0

<snip>

From this we can see that the Test VM is assigned to SGID 2, which will use vmnic2 for external

communication.

6. Exercise – Force your Test VM to utilize vmnic1.

a. First determine what the SGID of vmnic1 is. From our previous output, this would be SGID “1”.

b. Configure either the port profile or the individual interface to “prefer” this Sub Group.

N1000v(config)# port-profile rhel-pp

N1000v(config-port-prof)# pinning id 1

c. Verify the change.

N1000v(config-port-prof)# module vem 3 execute vemcmd show port






305 Po1 UP UP FWD 0

<snip>

7. Explore & play with various features - ACLs, QoS, PVLANs, etc!

how to deploy a nexus 1000v lab with a single esx host

Documents