![Page 1: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/1.jpg)
Niles Pyelshak
Customer Support Engineer
March 17, 2015
Demystifying Unified Computing System (UCS) Interfaces for troubleshooting
Cisco Support Community
Expert Series Webcast
![Page 2: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/2.jpg)
Upcoming Expert Series Webcast
April 7th, 2015
SourceFire IDS mode integration on ISR G2 and ISR 4K routers will be reviewed in details during the webcast along with a live demo.
SourceFire integration on the ISR G2
and ISR 4K routers with Kureli Sankar
http://bit.ly/webcast-april7
![Page 3: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/3.jpg)
Finesse for UCCX, next generation agent desktop
and transitioning from CAD. Finesse general
capabilities, and how it fits into the UCCX
environment with Abhiram Kramadhati, Cisco
Technical Solutions Manager
Now through March 27th
Ask the Expert Events – Active
Join the discussion for these Ask The Expert Events:
http://bit.ly/knowledgesharing-mar17
Cisco Webex Meeting Server (CWMS):
Install/Configuration and Troubleshooting with Cisco
Experts: Amit Sing, Hari Haran S M, and Jyothi V.
![Page 4: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/4.jpg)
https://supportforums.cisco.com/expert-corner/top-contributors
Participate in Live
Interactive
Technical Events
and much more
http://bit.ly/1jlI93B
Become an Event Top Contributor
![Page 5: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/5.jpg)
Rate Content Now your ratings on documents, videos, and blogs count give points to the authors!!!
So, when you contribute and receive ratings you now get the points in your profile.
Help us to recognize the quality content in the community and make your searches easier. Rate content in the community.
https://supportforums.cisco.com/blog/154746
Encourage and acknowledge people who generously share
their time and expertise
![Page 6: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/6.jpg)
Cisco Support Community Expert Series Webcast
• Today’s featured expert is Cisco Support Engineer, TAC Niles Pyelshak
• Ask your questions now in the Q&A window
Niles Pyelshak Customer Support Engineer
Cisco TAC, Server Virtualization
Tuesday, March 17, 2015
Demystifying Unified Computing
System (UCS) Interfaces for
troubleshooting CCIE#44608
![Page 7: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/7.jpg)
Expert Series Webcast: Demystifying Unified Computing System (UCS) Interfaces for troubleshooting
Technical Expert Question Manager
Robert Burns
CCIE #37856
![Page 8: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/8.jpg)
Now through March 25th
Ask the Expert Event following the Webcast
Join the discussion for these Ask The Expert Events:
http://bit.ly/knowledgesharing-mar17
Niles will be continuing the discussion in an Ask the Expert
event. So if you have more questions, please visit the
Knowledge Center on the Cisco Support Community
https://supportforums.cisco.com/discussion/12451761/ ask-expert-demystifying-unified-computing-system-ucs- interfaces-troubleshooting
![Page 9: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/9.jpg)
If you would like a copy of the presentation slides, click the PDF file link in the chat box on the right or go to:
bit.ly/docslides-mar17
Thank You For Joining Us Today!
![Page 10: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/10.jpg)
Submit Your Questions Now! Use the Q & A panel to submit your questions
and the panel of experts will respond.
Please take a moment to
complete the survey at
the end of the webcast
![Page 11: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/11.jpg)
Niles Pyelshak
Customer Support Engineer
March 17, 2015
Cisco Support Community Deep Dive Expert Series Webcast
Demystifying Unified Computing System (UCS) Interfaces for troubleshooting
![Page 12: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/12.jpg)
Polling Question 1
Do you have Unified Computing System(UCS) in your Data Center?
a. I have UCS-B series but not C-Series
b. I have UCS-C Series but not B-Series
c. I don’t have UCS and have never used it.
![Page 13: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/13.jpg)
• What’s in a UCS Domain?
• End Host/Switch Mode
• UCS Unified Ports
• Chassis Connectivity
• Server Connectivity
• Traffic Flow
• UCS Interfaces
• HIF to NIF Pinning
• Fabric Failover
• Basic Network Troubleshooting
• Following the Path of a Packet
High Level Overview of UCS Life of a Packet and Troubleshooting
![Page 14: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/14.jpg)
Fabric Interconnect (FI)
• 10 GbE unified fabric switch • Connects up to 160 servers
UCS Fabric Extender (FEX)
• 10 GbE unified remote linecard • Flexible bandwidth connectivity
Compute (Blade and Rack)
• x86 industry standard • Patented extended memory
UCS Virtual Interface Card (VIC)
• CNA adapters • Flexible server I/O
Unified Management
System Components: High-level Overview
![Page 15: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/15.jpg)
UCS Fabric Interconnect
UCS Fabric Interconnect – UCS 6248
• 1RU
• 32 unified base ports and 1 expansion slot • Line rate – 960Gbps
UCS Fabric Interconnect – UCS 6296
• 2RU
• 48 unified base ports and 3 expansion slots • Line rate – 1920 Gbps
UCS Fabric Extender
UCS IO Module (IOM) – 2204 or 2208
• 4 or 8 10GbE fabric links (to Fabric Interconnect)
• 16 or 32 10GbE server links (to servers)
Nexus 2232PP or Nexus 2232TM
• 8 10GbE fabric links (to Fabric Interconnect)
• 32 10GbE server links (to servers)
UCS VIC Adapters
UCS VIC1240 plus Pass-through (PT) Expansion Card - Blades
• VIC1240: Up to 4 x 10 GbE
• PT: Expands VIC1240 up to 8 x 10GbE
• Up to 256 vPCIe
UCS VIC 1280 - Blades
• Up to 8 x 10GE ports • Up to 256 vPCIe
UCS VIC 1225 - Racks
• Up to 2 x 10GE ports • Up to 256 vPCIe
UCS Fabric Portfolio
2 3 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
CISCO UCS 6296UP
STAT
ID
CISCO UCS 6248UP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
STAT
ID
![Page 16: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/16.jpg)
Configuring Unified Ports
![Page 17: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/17.jpg)
• Configured on a per FI basis
• Slider based configuration
• Reboot is required for the new port personality to take into affect
• Recommendation is to configure GEM card, therefore GEM is only needed to be rebooted
Unified Port Screen
![Page 18: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/18.jpg)
Fabric Forwarding Mode of Operations
![Page 19: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/19.jpg)
• End-host mode (EHM): Default mode
• No spanning-tree protocol (STP); no blocked ports
• Admin differentiates between server and network ports
• Using dynamic (or static) server to uplink pinning
• No MAC address learning except on the server ports; no unknown unicast flooding
• Fabric failover (FF) for Ethernet vNICs (not available in switch mode)
• Switch mode: User configurable
• Fabric Interconnects behave like regular ethernet switches
• STP parameters are lock
Fabric Forwarding Mode of Operations Mode of Operations
![Page 20: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/20.jpg)
Putting the Components Together
6200
Fabric A
6200
Fabric B
B200 B250
CNA
I OM A CNA CNA
I OM
B
I OM
A
I OM
B
ETH 1 ETH 2
MGMT MGMT
Chassis 1 Chassis 20
Fabric Switch
Fabric Extenders
Uplink Ports
Compute Blades
Half / Full width
OOB Mgmt
Server Ports
Virtualized Adapters
Cluster
SAN A SAN B
![Page 21: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/21.jpg)
• Configured on a per FI basis
• Slider based configuration
• Reboot is required for the new port personality to take into affect
• Recommendation is to configure GEM card, therefore GEM is only needed to be rebooted
Unified Port Screen
![Page 22: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/22.jpg)
• Configured on a per FI basis
• Slider based configuration
• Reboot is required for the new port personality to take into affect
• Recommendation is to configure GEM card, therefore GEM is only needed to be rebooted
Unified Port Screen
![Page 23: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/23.jpg)
Putting the Components Together
6200
Fabric A
6200
Fabric B
B200 B250
CNA
I OM A CNA CNA
I OM
B
I OM
A
I OM
B
ETH 1 ETH 2
MGMT MGMT
Chassis 1 Chassis 20
Fabric Switch
Fabric Extenders
Uplink Ports
Compute Blades
Half / Full width
OOB Mgmt
Server Ports
Virtualized Adapters
Cluster
SAN A SAN B
![Page 24: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/24.jpg)
UCS Fabric Topologies Chassis Bandwidth Options
24
2x 4 Link 80 Gbps per Chassis
2x 8 Links 160 Gbps per Chassis
2x 2 Link 40 Gbps per Chassis
2x 1 Link 20 Gbps per Chassis
2208XP only
![Page 25: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/25.jpg)
• A IOM (sometimes called ‘Fabric Extender’) provides
• A 1GE switch used for internal management (1GE per slot)
• A number of 10G-KR sever facing links (HIF)
• A number of Fabric links (NIF)
• NIC cards on the servers use those HIF ports for external connectivity
• Each IOM provides a separate dedicated IO channel for internal management connectivity
• There is no local switching on IOMs – traffic is always switched by the FIs
What are those IOMs?
![Page 26: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/26.jpg)
Let’s go back in time a bit
26
Chassis Management
Controller
FLASH
EEPROM
DRAM
Control
IO
Chassis Signals
Switch
4 Physical ports to Fabric
Interconnect
8 internal backplane ports to blades
ASIC with some
funny name
Up to 20Gbps
per slot
Up to
80Gbps per
chassis
2104XP
Gen 1 IOM
![Page 27: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/27.jpg)
So this is what we had with the 2104XP
27
![Page 28: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/28.jpg)
2208
2208
220x-XP Architecture
Chassis Management
Controller
FLASH
EEPROM
DRAM
Control
IO
Chassis
Signals
Switch
Woodside ASIC
Internal backplane ports to blades
No Local Switching – ever!
Traffic goes up to FI
Fabric Ports to FI
2204
2204
Feature 2204-XP 2208-XP
ASIC Woodside Woodside
Fabric Ports (NIF)
4 8
Host Ports (HIF)
16 32
CoS 8 8
Latency ~ 500ns ~ 500ns
![Page 29: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/29.jpg)
• These interfaces (show int brief – NXOS shell) are backplane traces
• Eth x/y/z nomenclature where
• x = chassis number
• y = is always 1
• z = host interface port number
Blade Northbound Ports
2
9
![Page 30: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/30.jpg)
Server Connectivity
![Page 31: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/31.jpg)
`
VIC
M81K
R
Palo ASIC Port 0 Port 1
21
04
- B
21
04
- A
B200 M
1/M
2
B 2
30M
1/M
2
x16 Gen1
IOM 2104 with M81KR in M1/M2 Blades
UCS 5100 backplane has 4 passive KR lanes to each half width server slot
BW to half width slot
= Dual 10Gb
IOM 2104 with M81KR in M1/M2 Blades
![Page 32: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/32.jpg)
`
VIC
1280
Sereno ASIC Port 0 Port 1
22
08
- B
22
08
- A
B200 M
1/M
2
B 2
30M
1/M
2
x16 Gen1
IOM 2208 with VIC 1280 in M2 Blades
IOM 2208 has 4 KR lanes to each server slot
BW to half width slot
= Dual 4x10Gb
VIC 1280 has dual 4x10GB ports
IOM 2208 with VIC 1280 in M2 Blades
![Page 33: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/33.jpg)
Traffic Flow
![Page 34: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/34.jpg)
Inter-Fabric Traffic Example (1)
VM1
VM2
VNIC 0
6100 A 6100 B
VNIC 1
L2 Switching
ESX HOST
EHM EHM
![Page 35: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/35.jpg)
Inter-Fabric Traffic Example (2)
VM
1
VM
2
VNIC 0
6100 A 6100 B
VNIC 1
L2 Switching
ESX HOST
EHM EHM
![Page 36: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/36.jpg)
UCS Interfaces
![Page 37: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/37.jpg)
UCS Interfaces
Uplink(border) port
Server(fabric) port
Network(NIF) port
Host(HIF) port
VIF VIF VIF
UCS Fabric
Interconnects
IO Modules
VIC Adapter
Virtual Circuit
![Page 38: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/38.jpg)
Polling Question 1
What are the two types of VIFS?
a) DCE
b) VFC
c) Veth
d) HIF
![Page 39: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/39.jpg)
Virtual Interfaces (VN-TAG)
![Page 40: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/40.jpg)
• Virtual interfaces (VIFs) help distinguish between FC and Eth interfaces
• They also identify the origin server
• VIFs are instantiated on the FI and correspond to frame-level tags assigned to blade mezz cards
• A 6-byte tag (VN-Tag) is preprended by Palo and Menlo as traffic leaves the server to identify the interface
• VN-Tag associates frames to a VIF
• VIFs are ‘spawned off’ the server’s EthX/Y/Z interfaces
VN-Tag: Instantiation of Virtual Interfaces
4
0
![Page 41: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/41.jpg)
Adapter
Switch
10GE
A
Eth 1/1
IOM A
6200-A
Physical Cable
Virtual Cable
(VN-Tag)
Abstracting the Logical Architecture
Blade
10GE
A
vEth 1
IOM A
6200-A
vFC
1
Service Profile
(Server)
Cable
vNIC
1
vEth
1
6200-A
vHBA
1
vFC
1
(Server)
Dynamic, Rapid
Provisioning
State abstraction
Location
Independence
Blade or Rack
Logical Physical
vHBA
1
vNIC
1
![Page 42: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/42.jpg)
42
VN-Tag at the Adapter (Mezz Card) Level
![Page 43: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/43.jpg)
• Ethernet and FC are muxed on the same physical links concept of virtual interfaces (vifs) to split Eth and FC
• Two types of VIFs: veth and vfc
• Veth for Ethernet and FCoE; vfc for FC traffic
• Each EthX/Y/Z or Po interface typically has multiple vifs attached to it to carry traffic to and from a server
• To find all vifs associated with a EthX/Y/Z or Po interface, do this:
VIFs
![Page 44: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/44.jpg)
FCoE: a vfc bound to a veth
![Page 45: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/45.jpg)
Another Way to Find VIFs:
![Page 46: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/46.jpg)
4
6
![Page 47: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/47.jpg)
HIF to NIF Pinning
![Page 48: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/48.jpg)
IO Module HIF to NIF Pinning 2208XP – 1 Link
1-4
5-8
9-12
13-16
17-20
21-24
25-28
29-32
FEX Fabric Interconnect
Slot 1
Slot 2
Slot 3
Slot 4
Slot 5
Slot 6
Slot 7
Slot 8
One link
![Page 49: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/49.jpg)
IO Module HIF to NIF Pinning 2208XP – 2 Link
FEX Fabric Interconnect
29-32
1-4
5-8
9-12
13-16
17-20
21-24
25-28
Slot 1
Slot 2
Slot 3
Slot 4
Slot 5
Slot 6
Slot 7
Slot 8
Two links
![Page 50: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/50.jpg)
IO Module HIF to NIF Pinning 2208XP – 4 Link
FEX Fabric
Interconnect
29-32
1-4
5-8
9-12
13-16
17-20
21-24
25-28
Slot 1
Slot 2
Slot 3
Slot 4
Slot 5
Slot 6
Slot 7
Slot 8
Four links
![Page 51: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/51.jpg)
IO Module HIF to NIF Pinning 2208XP – 8 Link
FEX
29-32
1-4
5-8
9-12
13-16
17-20
21-24
25-28
Fabric
Interconnect
Slot 1
Slot 2
Slot 3
Slot 4
Slot 5
Slot 6
Slot 7
Slot 8
Eight links
![Page 52: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/52.jpg)
• What happens in a 4-link topology when you loose 1 link?
• Server interfaces pinned to that link go down*
• The remaining 3 links still pass traffic for the other blade servers
• To recover the failed servers’ vNICs, re-acknowledged of the chassis is required
• After a re-ack UCS falls back to 2 links with regards to blade to fabric port mapping
• That’s because the link count must be a power of 2!
IOM and Failover
* unless you enabled Fabric Failover
![Page 53: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/53.jpg)
HIF to NIF Pinning Port Channel Modes
![Page 54: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/54.jpg)
Port-channel pinning
54
![Page 55: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/55.jpg)
Fabric Port Channeling
![Page 56: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/56.jpg)
Fabric Failover
![Page 57: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/57.jpg)
Polling Question 2
What is the preferred failover?
a) Hardware (Fabric) Failover
b) Software Failover (OS, Hypervisor)
![Page 58: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/58.jpg)
• Fabric Failover is only applicable in EHM.
• NIC teaming software required to provide failover in Switch mode.
Fabric Failover
Recommendation:
End Host Mode
vNIC 1
10GE
10GE
vEth 1
OS / Hypervisor / VM
vEth 1
IOM IOM
PHY Adapter Cisco VIC – M81KR
Menlo – M71KR VIRT Adapter
6100-A 6100-B L1 L2
L1 L2
Physical Cable
Virtual
Cable
![Page 59: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/59.jpg)
Fabric Failover
![Page 60: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/60.jpg)
Fabric Failover
![Page 61: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/61.jpg)
Fabric Failover
![Page 62: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/62.jpg)
Following the Path of a Packet
![Page 63: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/63.jpg)
• UCSM top level : show service-profile circuit server <chassis#>/<slot#>
• 2.0(1) release has an improved output
Following the Path of a Packet
![Page 64: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/64.jpg)
Following the Path of a Packet
![Page 65: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/65.jpg)
VIF
Virtual Circuit
(VIF 1236)
UCS Fabric
Interconnects
IO Modules
VIC Adapter
Pins out uplink Eth 1/9
Following the Path of a Packet
![Page 66: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/66.jpg)
Following the Path of a Packet
![Page 67: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/67.jpg)
Following the Path of a Packet
![Page 68: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/68.jpg)
VIF
Virtual Circuit
(VIF 1236)
UCS Fabric
Interconnects
IO Modules
VIC Adapter
Interface 1/1/21
VNTAG 2
Pins out uplink Eth 1/9
Following the Path of a Packet
![Page 69: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/69.jpg)
Following the Path of a Packet
![Page 70: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/70.jpg)
VIF
Virtual Circuit
(VIF 1236)
UCS Fabric
Interconnects
IO Modules
VIC Adapter
Interface 1/1/21
(Pinned to 1/2)
VNTAG 2
Pins out uplink Eth 1/9
Server Port 1/2
Following the Path of a Packet
![Page 71: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/71.jpg)
Following the Path of a Packet
VIF
Virtual Circuit
(VIF 1236)
UCS Fabric
Interconnects
IO Modules
VIC Adapter
VNTAG 2
Pins out uplink Eth 1/9
Designated Receiver Eth 1/9
Interface 1/1/21
(Pinned to 1/2)
Server Port 1/2
![Page 72: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/72.jpg)
![Page 73: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/73.jpg)
C-Series Integration into UCSM
![Page 74: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/74.jpg)
CIMC
GE LOM PCIe Adapter
CPU Mem
OS or Hypervisor
C260M2, C460M2
C220M3, C240M3
C22M3, C24M3 all at CiCM
1.4(6) or higher
C200/C210/C250 NO
Mix of B & C Series is
supported (no B Series
required)
Nexus 2232 Nexus 2232
Mgmt Traffic
Data Traffic
C-Series UCSM Integration Single Wire Management with VIC1225 and UCSM 2.1
![Page 75: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/75.jpg)
UCSM 2.2: Direct Connect with VIC1225 (no FEX)
75
CIMC
GE LOM
CPU Mem
OS or Hypervisor
C260M2, C460M2
C220M3, C240M3
C22M3, C24M3 all at CiCM
1.4(6) or higher
C200/C210/C250 NO
PCIe Adapter
![Page 76: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/76.jpg)
Wrapping up
![Page 77: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/77.jpg)
Polling Question 3
Are B-series servers required to integrate rack mount servers into UCSM?
a) Yes
b) No
![Page 78: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/78.jpg)
• UCS is first and foremost a server, not a switch
• Gen 1 vs Gen 2 components
• 2208 and VIC 1280 allow port-channels
• VIC 1280 with 2208XP for maximum bandwidth
• End-host mode forwarding rules: dynamic pinning
Recap
• Preferred mode of operation should always be end-host mode
• Very much plug and play, scalability, L2 multipathing, fabric failover
• Switch mode: spanning-tree, practically no user configuration possible
• VIF, VN-TAG, HIF, NIFs, VETH, VFC, Border Interfaces
• Operational consistency: C-series integration
Fault Codes: http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ts/faults/reference/ErrMess/UCS_SEMs.html
![Page 79: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/79.jpg)
Submit Your Questions Now! Use the Q & A panel to submit your questions and our expert will respond
![Page 80: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/80.jpg)
Collaborate within our Social Media
Facebook- http://bit.ly/csc-facebook
Twitter- http://bit.ly/csc-twitter
You Tube http://bit.ly/csc-youtube
Google+ http://bit.ly/csc-googleplus
LinkedIn http://bit.ly/csc-linked-in
Instgram http://bit.ly/csc-instagram
Newsletter Subscription http://bit.ly/csc-newsletter
Learn About Upcoming Events
![Page 81: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/81.jpg)
Cisco has support communities in other languages!
Spanish https://supportforums.cisco.com/community/spanish
Portuguese https://supportforums.cisco.com/community/portuguese
Japanese https://supportforums.cisco.com/community/csc-japan
Russian https://supportforums.cisco.com/community/russian
Chinese
http://www.csc-china.com.cn
If you speak Spanish, Portuguese, Japanese, Russian or Chinese we invite you to participate and collaborate in your language
![Page 82: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/82.jpg)
More IT Training Videos and Technical Seminars on the Cisco Learning Network
View Upcoming Sessions Schedule
https://cisco.com/go/techseminars
![Page 83: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/83.jpg)
Please take a moment to complete the survey
Thank you for Your Time!
![Page 84: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/84.jpg)
![Page 85: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/85.jpg)
Basic Network Troubleshooting
![Page 86: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/86.jpg)
Check Vmware, Microsoft or UCS
configuration(VLANs, native etc..)
Do you see the VMs
MAC addresses?
Basic Network Troubleshooting: Through Isolation
Isolate two VMs(on
different hosts) or
two bare metal
installs on same the VLAN to the same
FI.
YES
NO
![Page 87: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/87.jpg)
Check local firewall settings on VMs. Arp could be failing. Could
be duplicate IP or MAC issue.
Can you ping
between both VMs?
Isolate VMs to
other fabric and run 3 previous steps
YES
NO
YES
![Page 88: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/88.jpg)
Check
configuration on
switch. Check
for VLANs being trunked over
interfaces, vPC,
STP. UCS disjoint setup.
NO Can you ping the VM
from the upstream L3 device?
Check arp table.
See if you have
duplicate IP or MAC. Check
broadcast
receiver UCS.
NO
YES
Pin VMs out different FI’s. Do you see the
VMs MAC addresses
on upstream switch?
YES
![Page 89: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/89.jpg)
NO Upstream routing
issue.
Can you ping inter-VLAN
between two UCS VMs or
bare metal install?
![Page 90: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/90.jpg)
Basic Network Troubleshooting: Identifying Duplicate IP Addresses
• Disable interface that is
intermittently dropping pings.
• Ping from a VM or L3 device that is in the same VLAN.
• Check to see if MAC address resolved during
ARP. If the MAC did resolve compare/contrast.
![Page 91: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/91.jpg)
• You can disable adapter or shutdown the host. Use MAC address table to see if MAC address moves to other vethernets or interfaces on the upstream switch
• Follow the vethernet or ethernet interface
Basic Network Troubleshooting: Identifying Duplicate MAC Addresses
![Page 92: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/92.jpg)
• Connect nxos : show platform fwm info mac <mac> <vlanID>
• A “Mac history” section was added in 2.0(1) release
• The “old_if_index” is helpful in cases were MAC is learned from different places same time (for example, duplication issue)
Basic Network Troubleshooting: MAC History
![Page 93: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/93.jpg)
VN-Tag at the Adapter (Mezz Card) Level
![Page 94: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/94.jpg)
CIMC
GE LOM PCIe Adapter
CPU Mem
OS or Hypervisor C200M2, C210M2,
C220M3, C240M3,
C250M2, C260M2 or
C460M2
Mix of B & C Series is
supported (no B Series
required)
Nexus 2232 Nexus 2232
2 LOM ports
exclusive CIMC connectivity
Adapter support:
Emulex CNA Qlogic CNA
Intel 10g NIC Broadcom 10g NIC
Cisco VIC
Mgmt Traffic
Data Traffic
C-Series UCSM Integration Before UCSM 2.1 or without VIC1225
![Page 95: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/95.jpg)
SPAN
![Page 96: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/96.jpg)
96
Let’s go back in time once more
The traces are there; we were just not using them …
![Page 97: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/97.jpg)
Following the Path of a Packet
• Connect nxos : show platform software enm internal info vlandb id <vlanID>
![Page 98: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/98.jpg)
UCS 6248 Hardware Diagram
98
10 Gig
12 Gig
12 Gig
10 Gig
Carmel 1
Unified Crossbar Fabric Sunnyvale
Carmel 2 Carmel 3
Carmel 4 Carmel 5 Carmel 6
Ca
rme
l
CP
U
Intel
Jasper Forest
PEX 8525
4 port PCIE
Sw itch
South
Bridge
12 Gig
12 Gig
Carmel 1
Carmel cpu
Sunny vale
Carmel 2
Mgmt
Console
Xcon1
Xcon2
DD
R3
x2
PCIE Dual Gig
PCIe x4 PCIe x4 PCIe x4
PCIe x8
PCIE Dual Gig
PCIE Dual Gig
0 1 0 1 N/C
0
1
Carmel 6
Memory
Flash
NVRAM
Serial
![Page 99: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/99.jpg)
• No slot based pinning
• No invalid link count for NIF ports (no “power of 2” rule)
Port-Channel Pinning
9
9
Pinned
to Po
2200-IOM VIC1200
adaptor with
DCE links in
Port-Channel
Gen-1 adaptor
with single 10G
link
![Page 100: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/100.jpg)
8 links, Discrète (default)
slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8
F
E X
8 links, Port-channel
F E
X
Fabric Interconnect
Fabric Interconnect
• Statically pinned to Individual fabric links, no dynamic re-pinning
• Failure of the Fabric port will bring the HIF down triggering a failover
• Statically pinned to logical link (port-channel)
• Failure of the Fabric port will not bring the HIF down no failover take place.
• Traffic re-hashed to available links
HIF to NIF Pinning Link Failure Connectivity from IOM to FI and Failover Behavior
![Page 101: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/101.jpg)
Fabric Failover
![Page 102: Cisco Support Community Expert Series Webcast · Tuesday, March 17, 2015 Demystifying Unified Computing System (UCS) Interfaces for troubleshooting CCIE#44608 . Expert Series Webcast:](https://reader033.vdocuments.us/reader033/viewer/2022050601/5fa8f552483b2a13c938d153/html5/thumbnails/102.jpg)
Adapter
Switch
10GE
A
Eth 1/1
IOM A
6200-A
Physical Cable
Virtual Cable
(VN-Tag)
Abstracting the Logical Architecture
102
Blade
10GE
A
vEth 1
IOM A
6200-A
vFC
1
Service Profile
(Server)
Cable
vNIC
1
vEth
1
6200-A
vHBA
1
vFC
1
(Server)
Dynamic, Rapid
Provisioning
State abstraction
Location
Independence
Blade or Rack
Logical Physical
vHBA
1
vNIC
1