1 asr9k arc base
DESCRIPTION
ASR9K BasicsTRANSCRIPT
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 1/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 1
ASR9K Overview
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Online Diagnostics
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 2/127
Cisco ConfidentialViking Hardware Overview 2
ASR 9000 At a Glance
Optimized for Aggregation
of Dense 10GE and 100GE
Designed for Longevity:
Scalable up to 400 Gbps ofBandwidth per Slot
Based on IOS-XR & ANA
for Nonstop Availability and
Manageability
Enables Network
Convergence of Business
and Residential Services
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 3/127
Cisco ConfidentialViking Hardware Overview 3
IOS-XR Modular OSAbsolute HW Redundancy
Broad Network ResiliencySchemes
Carrier Class
6.4Tbps Capable SystemEngineered for Nx100GSuperior H-QoS/Scale
Built to Last
Integrated VideoVideo Quality Monitoring
Local Ad-Insertion
Optimal Intelligent Multicast ForwardingVoD Streaming/Cache
Industry Leading Watts/GbpsPay as You Grow Power
Minimal Carbon Footprint
Power Reduction
High Density GE/10GExtensive L2VPN Services
Incorporated SynchEIPoDWDM Ready
Carrier Ethernet Focus
Complete EMS/NMS Support
Consistent ProvisioningModel Comprehensive OAM
Minimize Operational
Expense
Simplicity
ResilientConverged
VideoOptimized
Purpose
Built
“Green”
ASR 9000The (R)evolution Begins!
* Please refer to Road Map
for t im e line of featuresuppor t
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 4/127
Cisco ConfidentialViking Hardware Overview 4
Designed with Operators in Mind Software Optimized for Automation, Ease of Use
Simplicity
Manageable via Cisco ANA Toolkit
Cross-platform support of ASR9k, 7600, CRS-1
Standards-based northbound interface to carrierOSS
XR image mgmt, device inventory, fault
correlationCarrier Ethernet end-to-end service provisioning
Embedded ASR9k mgmt with XML over HTTP,SNMP
Complete OAM Coverage
802.3ah for link level fault detection
802.1ag draft 8.1 compliance
MPLS OAM/VCCV support
EtherSLA/Y.1731 Performance Monitoring
(future)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 5/127
Cisco ConfidentialViking Hardware Overview 5
IOS-XR Powers the Edge with ASR 9000A Fully Distributed, Microkernel-Based Architecture
Designed for Scale, HA, and PerformanceFor IP NGN Applications
Next Generation Architecture
Modular Components Applications Architecture
IS-IS
QoS
Distributed Middleware
BGP
Multicast
OAM
VLAN
MAC
Subs
OAM
VLAN
MAC
Subs
OAM
VLAN
MAC
Subs
LC1
Distributed Service Separation
LC2 LCn
MPLS Multicast
RIP BGP
OSPF ISIS
Manageability
Security
Forwarding
Base
Admin
Line Card
HostComposite
RoutingComposite
Microkernel-based design
Highly modular, highly extensible
‘Service-enabled’ blade architecture
Scale through distribution
Unique address tables per linecard
Process-level, stateful subscriber HA
Resilient
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 6/127
Cisco ConfidentialViking Hardware Overview 6
Cisco ASR 9000 Architecturally Superior by Design: Chassis & Fabric
Scales to 2x 200G Fabric / Slot
•
Per
-
slot power & cooling
ready for
200G
today
•
Backplane signal integrity
validated for
10G
SerDes
Half Terabit More Dense than Competition
•
ASR9k
: 8 slots *
20x
10GE
=
160x
10GE
•
“Box J”: 11 slots *
10x
10GE
=
110x
10GE
•
“Box A”: 10 slots *
10x
10GE
=
100x
10GE
500G
–
600G
Density Advantage for
ASR9k
Now
200GCard
Maximixed Slot Real Estate
Length x Width x HeightOptimized for 200 Gbps
with Daughtercard
Up to 8x NPUs per Card
Power-Up as You Grow
Modular 1500W Power Bricks
For Green Efficiency
PurposeBuilt
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 7/127Cisco ConfidentialViking Hardware Overview 7
Dual Fia
80 Gbps
Linecards per Chassis
Bandwidth per Slot
Bandwidth per Chassis
Linecard Density
8 LC + 2 RSP 4 LC + 2 RSP
180 Gbps 180 Gbps
2.8 Terabits 1.4 Terabits
10 slots 6 slots
40 Gbps 40 Gbps
22 Slots
80 Gbps 80 Gbps
6.4 Terabits 3.2 Terabits
400 Gbps 400 Gbps
20 LC + 2 RSP
200 Gbps 200 Gbps
550 Gbps
20 Terabits
400 Gbps
ASR 9000 System ScalabilityOutlasting the Future
0
25,000
50,000
2005 2006 2007 2008 2009 2010 2011 2012
P B / m o MobilityBusiness Internet
Business IP WANConsumer InternetConsumer IPTV/CATV
Source: Cisco Visual Networking Index — Forecast,2007 –2012
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 8/127Cisco ConfidentialViking Hardware Overview 8
Viking History
Viking was initially conceptualized as an L2 aggregation switchwith the Gladiator chassis (with Gracchus and Marcus line cards)in the Metro Ethernet Market with Goryeo taking the routing rolefitting into the SP edge.
When Goryeo was found to be no longer an advantageous
investment, Viking adapted to SP edge market.
This posed changes both in the chassis requirement when RedOctober (with Ryan, Ramius and Mancuso line cards) wasconceptualized and also in the multitude of feature and scalerequirements.
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 9/127© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 9
ASR9K Chassis
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)Online Diagnostics
Service Console
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 10/127Cisco ConfidentialViking Hardware Overview 10
Chassis Overview ASR-9000 10-slot system
10 slots: 8x linecards + 2x RSP
Half-rack: 17.38”w x 36.75”h x 28”d
Bandwidth (initial)
400Gbps backplane
180Gbps fabric 400G
40G/80G linecards N-100G
Carrier-class hardware redundancy
AC & DC systems
Pay-as-you-grow, modular power
Green emphasis throughout
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 11/127Cisco ConfidentialViking Hardware Overview 11
Hardware Subsystem: 10-slot PowerViking Power Details
750W power & cooling available per LC slot
FCS cards draw significantly less (~500W)
Max system draw 8.5KW, max FCS draw is <= 5KW
AC power supplies are 3kW, DC are 2kW or 1.5kW
Max power configs : 5+1 DC (or) 3+3 AC
FCS power configs: 3+1 DC (or) 2+2 AC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 12/127Cisco ConfidentialViking Hardware Overview 12
Hardware Subsystem: 10-slot powerPower Distribution (DC N:1 protection)
PS 0 P ow er
Di s t r i b u t i o
nB u s
PS 1
PS 2
PS 3
PS 4
PS 5
Feed A
Feed B
Feed A
Feed B
Feed A
Feed B
Feed A
Feed B
Feed A
Feed B
Feed A
Feed B
LC
LC
RSP
RSP
LC
LC
Fans
Fans
LC
Single power zone, onedistribution bus
All modules load share
2kW and 1.5kW supplies
Each power supply iswired to both ‘A’ and ‘B’feed
Feed failure doubles draw
on remaining feed supply failure increases
draw on remainingsupplies
Shelf 1 (bottom)
Shelf 0 (Top)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 13/127Cisco ConfidentialViking Hardware Overview 13
Hardware Subsystem: 10-slot powerPower Distribution (AC 1:1 protection)
PS 0 P ow er
Di s t r i b u t i o
nB u s
PS 1
PS 2
PS 3
PS 4
PS 5
LC
LC
RSP
RSP
LC
LC
Fans
Fans
LC
Single power zone, onedistribution bus
All modules load share
AC power supplies arerates @ 3KW ea.
‘A’ feed wired to top powershelf
‘B’ feed wired to bottom
power shelf
Feed A
Feed A
Feed A
Feed B
Feed B
Feed B
Shelf 1 (bottom)
Shelf 0 (Top)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 14/127Cisco ConfidentialViking Hardware Overview 14
10-slot chassis: 6kW redundant commons AC “1:1” protection vs. DC “N:1” protection
AC PS3kW
P ow er Di
s t r i b u t i onB
u s
AC PS3kW
AC PS3kW
AC PS
3kW
S YSTEM
LOAD
For 6KW AC, you need 4x 3KW powersupplies (feed failure takes downtwo of them, supply failure kills one)
Feed A
Feed A
Feed B
Feed B
Shelf 1 (bottom)
Shelf 0 (Top)
DC PS2kW
P ow er Di
s t r i b u t i onB
u s
DC PS2kW
DC PS2kW
DC PS
2kW
S YSTEM
LOAD
For 6KW DC, 4x 2KW power supplies(2kw each, feed failure has noimpact, so we protect only against a
supply failure)
Feed A & B
Shelf 1 (bottom)
Shelf 0 (Top)
Feed A & B
Feed A & B
Feed A & B
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 15/127Cisco ConfidentialViking Hardware Overview 15
Hardware Subsystem: 10-slot thermalsViking Cooling Details
Front-to-back airflow
750W cooling per LC slot
FCS cards significantly less
Dual “stacked” fan trays belowLCs
Variable speed fans
at normal ambient temperatures
lower power draw
lower noise profile
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 16/127Cisco ConfidentialViking Hardware Overview 16
Chassis Overview ASR-9000 6-slot system
6 slots: 4x linecards + 2x RSP
¼ rack: 17.38”w x 17.35”h x 28”d
Bandwidth
400Gbps backplane
180Gbps fabric 400G
40G/80G linecards Nx100G
Carrier-class hardware redundancy
AC & DC systems
Pay-as-you-grow, modular power
Green emphasis throughout
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 17/127Cisco ConfidentialViking Hardware Overview 17
Hardware Subsystem: 6-slotViking Power & Cooling Details
side-to-back airflow (no stageheating!)
750W cooling per LC slot
Dual vertical fan trays above cardcage
Tray power draw from 40W-300W
Temperature sensitive greenengineering
FCS draw is significantly less than
750W Variable speed fans
at normal ambient temperatures
lower power draw, lower noiseprofile
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 18/127Cisco ConfidentialViking Hardware Overview 18
Hardware Subsystem: 6-slot powerPower Distribution (AC 1:1 protection)
PS 0
P ow er D
i s t r i b u t i on
B u s
PS 1
PS 2
Feed A
Feed B
Feed C
LC
LC
LC
LC
RSP
RSP
Fans
Single power zone, one
distribution bus, allmodules load share
AC power supplies arerated @ 3KW ea.
Fully populated 40G
system requires <3kW Third source required for
redundancy for >3KWinstallations.
Power Entry Shelf
Fans
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 19/127Cisco ConfidentialViking Hardware Overview 19
Hardware Subsystem: 6-slot powerPower Distribution (DC N:1 protection)
PS 0
P ow er D
i s t r i b u t i on
B u s
PS 1
PS 2
Feed A
Feed B
Feed A
Feed B
Feed A
Feed B
LC
LC
LC
LC
RSP
RSP
Fans
Single power zone, onedistribution bus
All modules load share
2kW and 1.5kW supplies
Each power supply iswired to both ‘A’ and ‘B’feed
Power Entry Shelf
Fans
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 20/127Cisco ConfidentialViking Hardware Overview 20
ASR9K Power Calculator:
http://wwwin.cisco.com/sptg/crbu/products/asr9000/files/ASR_9000_Power_Model.xls
per-component values computed power values
Qty
Typical
(25C)Max (40C)
Ext. Temp
(55C)
Typical
(25C)
Max
(40C)
Ext. Temp
(55C)
9006 FanTray 0 100 275 375 0 0 0
9010 FanTray 2 200 300 600 400 600 1200
RSP 2 175 205 235 350 410 470
4xNP linecard 4 310 320 350 1240 1280 1400
8xNP linecard 4 565 575 630 2260 2300 2520
TOTALS 4250 4590 5590
To use: select 2 fan trays based on the chassis type. Enter the number of cards and RSPs
configured.
Max power values are tested values with full wire-rate traffic running on all ports, and worst-case
feature sets.
Note that it's the NUMBER OF NPs that matters, NOT the number of ports.
Specific example: the 8-port card with 4 NPs uses far far less power than the 8-port card with
8xNPs
Deployed "real world" systems are likely (but not guaranteed) to draw 10-33% less power than these numbers.
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 21/127© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 21
ASR9K System
Architecture
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)Online Diagnostics
Service Console
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 22/127Cisco ConfidentialViking Hardware Overview 22
Viking Architecture : SummaryCarrier Class Design
High Availability integral to system architecture
Not an afterthought
Modularized System Components – HW and SW
Isolate failure and faults to subsystem/component
HW based signaling for fabric failover
~ Zero packet loss on switchover Redundancy built-in throughout system
RSP, Fabric, Control Plane Ethernet, Chassis ControlBus, Power supplies, Fan Trays
Backplane Point-to-Point connectivity oriented
Limit single point of failuresIsolate faults
Highly Scalable System and dense QoS
Synchronous Ethernet Support
BITS/Network Timing Distribution
MAC
FIC
NPU
NPU
NPU
NPUMAC
MAC
FIC
NPU
NPU
NPU
NPUMAC
MAC
FIC
NPU
NPUMAC
MAC
FIC
NPU
NPU
NPU
NPUMAC
MAC
FIC
NPU
NPU
NPU
NPUMAC
FIC
CPU BITS/DTI
FIC
CPU BITS/DTI
FIC
CPU BITS/DTI
MAC
FIC
NPU
NPU
NPU
NPUMAC
CPUMAC
FIC
NPU
NPU
NPU
NPUMAC
CPU
CPU
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 23/127Cisco ConfidentialViking Hardware Overview 23
ASR9K SW Architecture Highlights
Scalable Forwarding Plane
Ethernet centric, multilayer switch design
Fully distributed packet processing
Two stage forwarding model
IOS XR Based
Sharing same XR release and L3 feature set with CRS-1
Carrier Class infrastructure
Offering S/W modularity, HA, Restartability, Fault Containment, NSF/SSO,SMU, ISSU
Focusing on Carrier Ethernet features
VPLS, EoMPLS, Bridge Domain, EFP, MSTP, Link Bundling, E-OAM, QinQ,IGMP Snooping
High throughput traffic management support
Hierarchical QoS, CBWFQ, WRED, 2R3C Policing
Security, Protection Features
L2,/L3 Security ACLs, CoPP, IP SLA, Mac Limiting, BPDU Filtering
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 24/127Cisco ConfidentialViking Hardware Overview 24
ASR9K Platform basic blocks
Switch Fabric
Route Processor
Switch Fabric
Di s t r i b u t e d
F or w a r d i n g
L i n e
C a r d
Route Processor
Di s t r i b u t e d
F or w a r d i n g
L i n e
C a r d
Line Cards carry forwardingengines, control plane processing
and integral I/O ports
Switch Fabric and Route Processorare combined on one RSP module.
Redundant Route-SwitchProcessors provide central point of
control for the router.
…
Di s t r i b u t e d
F or w a r d i n g
L i n e
C a r d
RSP also provides shared
resources for backplane Ethernet,timing and chassis control
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 25/127
Cisco ConfidentialViking Hardware Overview 25
Viking System Block Diagram
OctopusOctopus
CPU
8641D
GE
Switch
SystemTiming Bellagio 2
2Santa
Cruz
Octopus
Punt Path
Punt Path CPU
8641D
GE
Switch
SystemTiming Bellagio 2
2Santa
Cruz
Octopus
Punt Path
Punt Path
Backplane
Legend
Timing Plane
Data Plane
Arbitration Plane
Control Plane
Punt Path
CPU
8641D
GE
Switch
SystemTiming Bellagio 2
2Santa
Cruz
Octopus
Punt Path
Punt Path CPU
8641D
GE
Switch
SystemTiming Bellagio 2
2Santa
Cruz
Octopus
Punt Path
Punt Path
40x1GE
Ryan LC
NP3 NP3
CPU
8641
GE
PHY
NP3 NP3
BRG BRG
10Px
SFP10Px
SFP10Px
SFP
10Px
SFP
OctopusOctopus4x10GE
Mancuso
LC
NP3 NP3
CPU
8641
GE
PHY
NP3 NP3
BRG BRG
10G
PHY
XFP
OctopusOctopus8x10GE
Ramius
LC
NP3 NP3
CPU
8641
GE
PHY
NP3 NP3
BRG BRG
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
1 0 G P
H Y
S F P
10G
PHY
XFP
10G
PHY
XFP
10G
PHY
XFP
OctopusOctopus20x1GE
Thor LCCPU
8641
GE
PHY
10Px
SFP10Px
SFP
CPP
SERPSERP
CPP
PLIM ASIC
RSP1RSP0
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 26/127
Cisco ConfidentialViking Hardware Overview 26
Viking Logical Architecture Overview
O c t o p u s O c t o p u s
C P U
8 6 4 1 D
G E
S w i t c h
S y s t e m T i m i n g
B e l l a g i o 2 2 S a n t a
C r u z
O c t o p u s C P U
8 6 4 1 D
G E
S w i t c h
S y s t e m T i m i n g
B e l l a g i o 2 2 S a n t a
C r u z
O c t o p u s
B a c k p l a n e
Legend
Data Plane
Control Plane
C P U
8 6 4 1 D
G E
S w i t c h
S y s t e m T i m i n g
B e l l a g i o 2 2 S a n t a
C r u z
O c t o p u s
P u n t P a t h
C P U
8 6 4 1 D
G E
S w i t c h
S y s t e m T i m i n g
B e l l a g i o 2 2 S a n t a
C r u z
O c t o p u s
40 x 1 GE
Ryan LC
Trident
CPU
8641
GE
PHY
FPGA FPGA
10 x
SFP
10 x
SFP 10 x SFP
10 x SFP
O c t o p u s O c t o p u s
4 x 10 GE
Mancuso
LC CPU
8641
GE
PHY
FPGA FPGA
O c t o p u s O c t o p u s
8 x 10 GE
Ramius
LC CPU
8641
GE
PHY
FPGA FPGA
O c t o p u s O c t o p u s 20 x 1 GE
Thor LC
CPU
8641
GE
PHY
SPA
CPP
FPGA FPGA
CPP
PLIM ASIC
RSP 1 RSP 0
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
1 0 G E X F P
SPA SPA SPA
Trident Trident Trident Trident Trident Trident Trident Trident Trident Trident Trident
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 27/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 27
ASR9K RSP
(Route Switch Processor)
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)Online Diagnostics
Service Console
ASR 9K RSP
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 28/127
Cisco ConfidentialViking Hardware Overview 28
RSP2 RSP440
Processors 2 x 1.5GHz Freescale8641D CPU
Intel x86 Jasper Forest 4Core 2.27 GHz
RAM (userexpandable)
4GB @133MHz SDR
8GB
6GB (RSP440-TR) and12GB (RSP440-SE)version @1066MHz DDR3
Cache L1: 32KBL2: 1MB
L1: 32KB per CoreL2: 8MB shared
Primary persistentstorage
4GB 16GB - SDD
Secondary persistentstorage (HD/SSD)
30GB - HDD 16GB - SDD
USB 2.0 port No Yes
nV Cluster – EOBCports
No Yes, 2 x 1G/10G SFP+
Switch fabricbandwidth
184G/slot (with dualRSP)
440G/slot (with dual RSP)
ASR 9K RSP (Route/Switch Processors )
Overview
RSP440
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 29/127
Cisco ConfidentialViking Hardware Overview 29
Hardware Subsystem: RSPRoute/Switch Processor Details
High performance control plane processor
Redundant RP config
Active/Active fabric model
External interfaces
Console/AUX
2x 10/100/1000 Ethernet
Compact Flash
USB
BITS/DTI
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 30/127
Cisco ConfidentialViking Hardware Overview 30
Hardware Subsystem: RSPMain Board Components
CPU: FreeScale 8641D
Memory: 4GB
Hard Drive: 40G HDD
I/O controller: “Long Beach”
Fabric Interface: “Octopus”
Fabric Arbitration: “Bellagio”
Fabric Data path: “Santa Cruz”
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 31/127
Cisco ConfidentialViking Hardware Overview 31
CPU Complex Fabric Interface
Timing Domain
EOBC/Control Plane GE
Hardware Subsystem: RSPMain Board Components
CPU8641D
4GB MEM
Octopus
Long BeachI/O FPGA
SantaCruz 0
SantaCruz 1
HDD
Mgt Eth
CF card
Console
PuntFPGA
NVRAM Boot Flash
EtherSwitch
Clock TimeFPGA
BITS
Front Panel
BellagioArbitration
4G CF
Mgt Eth
Aux
Alarm
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 32/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 32
ASR9K Fabric
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)Online Diagnostics
Service Console
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 33/127
Cisco ConfidentialViking Hardware Overview 33
Hardware Subsystem: FabricFabric Architecture
Fabric is logically separate from LC/RSP
Physically resides on RSP
Similar to 7600, not like CRS/GSR
RSP requests access to fabric just like LCs do
Separate data and arbitration paths
Each LC/RSP has a fabric interface ASIC
1x Octopus 40G usable ethernet connectivity
2x Octopus 80G usable ethernet connectivity
Next-gen fabric interface chips in development
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 34/127
Cisco ConfidentialViking Hardware Overview 34
Hardware Subsystem: FabricFabric Architecture (single RSP shown)
Octopus
(LC)
Arbitration
Santa Cruz 1
Santa Cruz 0
Bellagio
Octopus(LC)
Octopus
(LC)
Octopus(RSP)
Octopus
(LC)
Octopus(LC)
Octopus
(LC)
Octopus(RSP)
Data
23G fabric channels
arbitration grant/request signals
?
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 35/127
Cisco ConfidentialViking Hardware Overview 35
RSP0
Hardware Subsystem: FabricFabric Architecture
Octopus(LC)
SantaCruz 0
Octopus(LC)
23G fabric channels
fabric data is active/active, arbiter is active/standby
SantaCruz 1
Bellagio
RSP1
SantaCruz 0
SantaCruz 1
Bellagio
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 36/127
Cisco ConfidentialViking Hardware Overview 36
RSP0
Hardware Subsystem: FabricScaling Fabric architecture for 80Gbps linecard
Octopus
SantaCruz 0
Octopus
Linecards are forwards/backwards compatible: fabric and backplane designed for 80G today
No changes to existing architecture, arbitration scheme stays the same
FCS fabric has enough switch fabric ports to support 80G/Linecard
Fabric and Backplane designed to support two Fabric Interface Devices seamlessly
SantaCruz 1
Bellagio
RSP1
SantaCruz 0
SantaCruz 1
Bellagio
Octopus
Dual-Octopus80G Linecard
Single-Octopus40G Linecard
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 37/127
Cisco ConfidentialViking Hardware Overview 37
(unicast fabric plane)
Hardware Subsystem: Fabric loadsharingUnicast loadsharing
Octopus
SantaCruz 0
Unicast traffic is sent across first available fabric link to destination (maximizes efficiency)
Each frame (or superframe) contains sequencing information
All destination octopus have re-sequencing logic
Additional re-sequencing latency is measured in nanoseconds
SantaCruz 1
Bellagio
(unicast fabric plane)
SantaCruz 0
SantaCruz 1
Bellagio
Octopus 4 3 2 1
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 38/127
Cisco ConfidentialViking Hardware Overview 38
(multicast fabric plane)
Hardware Subsystem: Fabric loadsharingMulticast loadsharing
Octopus
SantaCruz 0
Multicast traffic is hashed based on (S,G) info to maintain flow integrity
Very large set of multicast destinations preclude resequencing
Multicast traffic is non arbitrated – sent across a different fabric plane
Ratio of unicast credit to multicast credit is TBD
SantaCruz 1
Bellagio
(multicast fabric plane)
SantaCruz 0
SantaCruz 1
Bellagio
Octopus A1A2B1A3B2C1
Flows exit in-order
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 39/127
Cisco ConfidentialViking Hardware Overview 39
Hardware Subsystem: Fabric Active/Active Fabric Details
Each LC connects to and uses BOTH fabrics
One RSP provides enough bandwidth for 80G/slot
HW-based failure detection
very fast RSP failure notification (better MTTR)
Unicast traffic is loadbalanced across active links
Additional RSP provides more speedup, better hashing
No such thing as a single-RSP system (not sold) Always always always test, sell, and compete with both
RSPs installed and running!
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 40/127
Cisco ConfidentialViking Hardware Overview 40
Hardware Subsystem: FabricFabric Wiring & Scaling Details
10-slot chassis has 8 LC slots + 2 RSP slots
2x Santa Cruz fabric chips per RSP
Santa Cruz has 20 switch ports @ 20Gbps each
Each LC wired for two ports per Santa Cruz
40G linecards use one port for each Santa Cruz
80G linecards use two Santa Cruz ports each
6-slot chassis has several unwired ports
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 41/127
Cisco ConfidentialViking Hardware Overview 41
Hardware Subsystem: FabricFully Populated 40G/slot fabric subsystem: 10x octopus
Octopus(LC) Santa Cruz 0
Octopus(LC)
Santa Cruz 1
Santa Cruz 0
Santa Cruz 1
Octopus(LC)
Octopus(LC)
Octopus(LC)
Octopus(LC)
Octopus
(LC)
Octopus
(LC)
Octopus(RSP)
Octopus(RSP)
(bellagio & arbitration links are not shown)
RSP0
RSP1
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 42/127
Cisco ConfidentialViking Hardware Overview 42
Hardware Subsystem: FabricFully Populated 80G/slot fabric subsystem: 18x octopus
Octopus
Santa Cruz 0
Octopus
Santa Cruz 1
Santa Cruz 0
Santa Cruz 1
Octopus Octopus
Octopus
Octopus
Octopus(RSP)
Octopus(RSP)
Octopus
Octopus
Octopus
Octopus
Octopus
Octopus
Octopus
Octopus
Octopus
Octopus
80G
80G
80G
80G
80G
80G
80G
80G
40G40G
(bellagio & arbitration links are not shown)
RSP0
RSP1
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 43/127
Cisco ConfidentialViking Hardware Overview 43
Hardware Subsystem: Fabric Arbitration & Superframing Mechanism
20-port 20G crossbar switch w/ external arbitration
Scheduler granularity is per 10G (or 10x1) block
Ingress LC sends request for output LC
Arbiter replies with “credit” for egress LC Egress returns “credit” to LC when packet arrives
Arbitration not strictly required, but ...
Significantly improves switch efficiency
Intelligent “traffic cop”
Maintains state of available credit for each destination port
When no credit we don’t schedule for that port
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 44/127
Cisco ConfidentialViking Hardware Overview 44
Hardware Subsystem: Fabric Arbitration & Superframing Mechanism
Superframing significantly improves total throughput
multiple unicast frames from/to same destinations
0 (Empty)Min Sufficient
for Superframe
Max
Superframe
Max
MTU
Min reached (2)
Packet 1 Jumbo (1)
Packet 1 No superframing (1)
Packet 1 Max reached (3)Packet 2Packet 3
Packet 1Packet 2
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 45/127
Cisco ConfidentialViking Hardware Overview 45
Hardware Subsystem: Deep Fabric MathCounting the Bits…
each octopus acts and is wired to one “20G fabricchannels” on each santa cruz
raw bandwidth per “20G fabric channel” in the santa cruzfabric chip is:
8 parallel serial lines @ 3.125Mhz each, using 24/26 encoding.
8 * 3.125 * (24 / 26) 23.077 Gbps ~~> 23Gbps
there are two santa cruz chips per RSP:
two santa cruz per RSP: 23 * 2 == 46Gbps per RSP
there are two RSPs per chassis:
46Gbps/RSP * 2 92Gbps per octopus (in active/active mode)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 46/127
Cisco ConfidentialViking Hardware Overview 46
Superframes
With 32 byte DC3 hdr + FPOE + error control + 8-bye alignment overhead, smaller pkts areinefficient in the fabric. So unicast packets are grouped into Superframes < 2K bytes. Since therecould be huge combination of multicast destinations, multicast packets are neither put intoSuperframes nor are arbitrated through the VOQ mechanism.
1st packet DC3 fabric
header inclusive
2nd packet DC3 fabric headerinclusive
3rd packet DC3 fabric headerinclusive
Nth packet
8 bytes
3 FPOE bytes
(interpreted
by fabric)
Pad with zero to align to 8 bytes
4 byte CRC covers
entire frame.
Min. 1068
bytes to
guarantee
line rate
perfmnce
Word0
Word1
Word2
Sequencing
information in
32-byte DC3
header
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 47/127
Cisco ConfidentialViking Hardware Overview 48
Load Balancing from Active/Active
Parallel Paths in Redundant, Eight Line Card Chassis
0
1
14
15
0
1
15
0
1
15
0
1
15
RSP0
0
1
15
0
1
15
0
1
15
0
1
15
S C Z
O
c t oF r om-
F a b
First Line Card / Slot 0
Egress Path
14
14
14
14
14
14
O c t oF r om-
F a b
O c t oF r om-
F a b
O c t oF r o
m-
F a b
O c t o
T o-F a b
O c t o
T o-F a b
O c t o
T o-F a b
O c t o
T o-F a b
First Line Card / Slot 0
Ingress Path
Last Line Card / Slot 9
Egress Path
Last Line Card / Slot 9
Ingress Path
RSP1
S C Z
S C Z
S C Z
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 48/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 49
ASR9K Linecards
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)Online Diagnostics
Service Console
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 49/127
Cisco ConfidentialViking Hardware Overview 50
ASR 9000 Series Line Cards
Integrated L2 / L3 Services Dense Hierarchical QoS
High Density Service Scale
Distributed High Performance Control Plane
Modular Operating System - IOS-XR
Distributed / Flexible Forwarding Plane
Integrated Timing Support
Integrated Media Monitoring
Distributed OA&M Processing
Distributed Netflow
Online Insertion & Removal
40G Line Rate Performance
Pluggable Optical / Electrical Interfaces
Feature support may be software release dependent
A9K-40GE-B 40 Port GE SFP Line Card
A9K-4T-B 4 Port 10GE XFP Line Card
A9K-4T/8-B 8 Port 10GE XFP Line Card
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 50/127
Cisco ConfidentialViking Hardware Overview 51
L/B/E Li C d
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 51/127
Cisco ConfidentialViking Hardware Overview 52
-
Network Process Unit
STATS MEMORY
FRAME MEMORYLOOKUPMEMORY
TCAM
FIB MAC
NP complex
L/B/E Line CardsWhat’s the Difference?
Each NPU has Four Main memories:
– Lookup/Search Memory (RLDRAM): stores MAC, FIB, and Adjacencies Tables
– TCAM: classification (Vlan Tag (EVCs), QoS and Security ACL
– Stats QDR memory: interface and forwarding statistics, policers data, etc
– Frame memory: buffer memory for Queues
3 LC versions – low, base and extended - differ for size of memories
– TCAM, QDR and Frame memory sizes depend on LC version
Affects number of QoS queues and L2 sub-interfaces supported
– Search Memory is same
System level scale (unicast, multicast, MPLS label) adjacency and MAC
address) not affected by a mix of LCs
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 52/127
Cisco ConfidentialViking Hardware Overview 53
Metric Low Queue Medium Queue High Queue
MAC Addresses 512K 512K 512K
IPv4 Routes (total/per VRF) 1M/128K 1M/128K 1M/128K
ARP entries 32K 32K 32K
IPv6 Routes (total/per VRF) 512K/128K 512K/128K 512K/128K
VRFs 4k 4k 4k
L3 Subif/Port 4k 4k 4k
Bridge Domains 8k 8k 8k
MPLS Labels 128k 128k 128k
EFPs (L2 sub-int) 4k 16k 32k
Queues 8/port 64k/32k 256k/128k
Policers 8k 128k 256k
ASR 9000 Ethernet LinecardsCapability Comparison
D i f f e r e n t
C o m
m o n
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 53/127
Cisco ConfidentialViking Hardware Overview 54
How to Choose the Right Linecard ?
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 54/127
Cisco ConfidentialViking Hardware Overview 55
Typhoon LC
Part NumberTarget
Release
A9K-24x10GE-SE 4.2.0
A9K-24x10GE-TR 4.2.0
A9K-2x100GE-SE 4.2.0
A9K-2x100GE-TR 4.2.0
A9K-1x100GE-SE 4.2.2
A9K-1x100GE-TR 4.2.2
A9K-36x10GE-SE 4.2.2
A9K-36x10GE-TR 4.2.2
Part Number
Target
Release
A9K-RSP440-SE 4.2.0
A9K-RSP440-TR 4.2.0
A9K-MOD80-SE 4.2.0
A9K-MOD80-TR 4.2.0
A9K-MOD160-SE 4.2.1
A9K-MOD160-TR 4.2.1
A9K-MPA-2x10GE 4.2.1
A9K-MPA-4x10GE 4.2.0
A9K-MPA-8x10GE 4.3.1
A9K-MPA-20x1GE 4.2.0
A9K-MPA-1x40GE 4.2.3
A9K-MPA-2x40GE 4.2.1
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 55/127
Cisco ConfidentialViking Hardware Overview 56
Netflow Support
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 56/127
Cisco ConfidentialViking Hardware Overview 57
Licensable Features
L3 VPN Service Line Card LicenseEnables Provisioning of L3VPN Service Interfaces on the LC
Infrastructure VRF Line Card License
Enables Provisioning of L3 Interfaces in up to 8 VRFs on the LC Media Monitoring System License
Enabled Provisioning of Media Monitoring Points on any LC
Feature support may be software release dependent
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 57/127
Cisco ConfidentialViking Hardware Overview 58
Do I need a License ?
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 58/127
Cisco ConfidentialViking Hardware Overview 59
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 59/127
Cisco ConfidentialViking Hardware Overview 60
Linecards
Codename I/O BW Engine Comments
Ryan 40 x GE 40G 4 x Trident NPU
Ramius 8 x 10GE 40G 4 x Trident NPU 2:1 oversubscribed
Mancuso 4 x 10GE 40G 4 x Trident NPU
Thor 4 x SPA 20G 2 x CPP10 (Popeye)
Son-of-Thor 4 x SPA 40G 2 x CPP10 (Olive)
Odin 8 x 10GE 80G 8 x Trident NPU
Valhalla 16x10GE 80G 8 x Trident NPU 2:1 oversubscribed
FCS switch fabric provides ~80Gbps per LC slot
Roadmap to 200 Gbps with future switching silicon
RE D
O C T OBE R
H d S b t R d O t b Li d
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 60/127
Cisco ConfidentialViking Hardware Overview 61
Hardware Subsystem: Red October LinecardLinecard Details
40Gbps linecard Scalable architecture
-4xTen, 40xGE, “2 plus 20”
-80G linecard in 2H CY09
Carrier Ethernet/Transport focus
Flexible, microcoded architecture
Base & expanded memory options
additional NP memory -> more scale
Advanced IP software licence
better L3 routing scale, adv. features
Hard are S bs stem Red October Linecard
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 61/127
Cisco ConfidentialViking Hardware Overview 62
Hardware Subsystem: Red October LinecardRed October Board Components
Physical interfaces:8xTenGE, 4xTenGE, 40xGE, “2+20”
Forwarding/Queueing
“Trident” Network Processor
Header translation and interconnect
“Bridge”
Fabric Interface
“Octopus”
CPU: 8641D w/ 2GB memory
Internal GE for chip programming
Hardware Subsystem: Red October LC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 62/127
Cisco ConfidentialViking Hardware Overview 63via backplane
Hardware Subsystem: Red October LC8x10GE 2:1 oversubscribed Linecard : “Ramius”
10GE PHY
RSP 0
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
NetworkClocking
XFP3
XFP7
XFP2
XFP6
XFP
1
XFP5
XFP0
XFP4
Hardware Subsystem: Red October LC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 63/127
Cisco ConfidentialViking Hardware Overview 64via backplane
Hardware Subsystem: Red October LC4x10GE linerate Linecard: “Mancuso”
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
10GE PHY
10GE PHY
10GE PHY
10GE PHY
XFP
XFP
XFP
XFPNetworkClocking
Hardware Subsystem: Red October LC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 64/127
Cisco ConfidentialViking Hardware Overview 65via backplane
Hardware Subsystem: Red October LC40xGE linerate Linecard: “Ryan”
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
10xGE SFP
10xGE SFP
10xGE SFP
10xGE SFP
NetworkClocking
Hardware Subsystem
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 65/127
Cisco ConfidentialViking Hardware Overview 66
via backplane
Hardware Subsystem8x10GE line rate Linecard : “Odin”
10GE PHY
RSP 0
10GE PHY
10GE PHY
10GE PHY
Trident 0
Trident 1
RavenFPGA 0
Trident Memory
Trident Memory
Octopus 0
CPU8641D
CPU memory
2GB flash
gigEcontrolnetwork
RSP 1
NetworkClocking
XFP
XFP
XFP
XFP
Trident 2
Trident 3
Trident Memory
Trident Memory
10GE PHY
10GE PHY
10GE PHY
10GE PHY
Trident 4
Trident 5Raven
FPGA 1
Trident Memory
Trident Memory
XFP
XFP
XFP
XFP
Trident 6
Trident 7
Trident Memory
Trident Memory
Octopus 1
Hardware Subsystem
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 66/127
Cisco ConfidentialViking Hardware Overview 67
via backplane
Hardware Subsystem16x10GE 2:1 oversubscribed Linecard : “Valhalla”
10GE PHY
RSP 0
Trident 0
Trident 1
RavenFPGA 0
Trident Memory
Trident Memory
Octopus 0
CPU8641D
CPU memory
2GB flash
gigEcontrolnetwork
RSP 1
NetworkClocking
XFP
Trident 2
Trident 3
Trident Memory
Trident Memory
Trident 4
Trident 5Raven
FPGA 1
Trident Memory
Trident Memory
Trident 6
Trident 7
Trident Memory
Trident Memory
Octopus 1
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
10GE PHYXFP
Hardware Subsystem
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 67/127
Cisco ConfidentialViking Hardware Overview 68via backplane
Hardware Subsystem2x10GE + 20xGE linerate : “JetFire”
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
CPU memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
10GE PHY
10GE PHY
XFP
XFP
NetworkClocking
10xGE SFP
10xGE SFP
Hardware Subsystem: Red October Linecard
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 68/127
Cisco ConfidentialViking Hardware Overview 69
Hardware Subsystem: Red October LinecardSynchronous Ethernet support on existing HW
RSP has BITS input, DTI andcentralized Clock Distribution hardware
Full support for L1 Sync-E on linecards(XR 3.9)
Flexible time sourcing: Line cardscapable of recovering clock andsending to RSP and receiving Transmitclock from RSP
Frequency synchronization support viaSPAs also (CEoPS, Metronome ..)
Future Hardware capable ofIEEE1588-v2
via backplane
10GE PHY
RSP 0
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
10GE PHY
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GBflash
gigEcontrolnetwork
RSP 1
daughtercard
NetworkClocking
XFP
XFP
XFP
XFP
XFP
XFP
XFP
XFP
Hardware Subsystem: Trident Fwd Engine
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 69/127
Cisco ConfidentialViking Hardware Overview 70
Hardware Subsystem: Trident Fwd EngineTrident: 10Gbps forwarding/feature engine
10Gbps feature rich ethernet forwarding ASIC
Utilizes multiple cores for packet processing
(Similar to CPP/QuantumFlow Processor)
Multi-stage microcoded architecture
Integrated Traffic managers, Ethernet MACs
Multiple external memories, amount determines scale
Classification TCAMs, MAC/IP lookups,
Frame Buffers, Counters/stats
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 70/127
Cisco ConfidentialViking Hardware Overview 71
Trident NPU Block Diagram
Micro-EnginePipeline
1 5 G b i t / sX A UI +
G
i gE S GMI I MA C s ( or )
X A UI 1 0 G MA C s
Trident NPU Memory Interfaces
Ethernet Side Fabric Side
Host PCI-Express
HostSGMII
(“punt path”)
TMb
TMc
1 0 x G
i gE - or -2 x1 0 Gi gE
E t h
er n e t I n t er f a c e s
TMa
FrameMemory
TM ControlMemory
LookupTables
CountersStatisticsLookups
OptionalTCAM72
DRAM333MHz
DDR2/RLDRAM2
DRAM333MHz
DDR2/RLDRAM2
DRAM333MHz
DDR2/RLDRAM2
SRAM(36x1)
300MHzQDR
(72x1)300MHz
F a b r i cI n t er f a c e
S i x-l a
n eX A UI / 1 5 G b i t / s e c
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 71/127
Cisco ConfidentialViking Hardware Overview 72
Trident NPU Architecture Highlights
TM
FrameMemory
FromLink/SF/
Host
External Search Memory( RLDII, TCAM)
ControlMemory
Embedded searchMemory
keys searchresults
searchresults
ToLink/SF/
Host
TOP
parseTOP
Search ITOP
modifyTOP
Search IITOP
resolve
StatisticsMemory
TM TM
High-learn I/f
Loopback interface
256KB
256KB
FrameMemory
Trident NPU Single Chip, Programmable, up to 12Gbps w/ 64B packets, full duplex, wire speed networkprocessor (up to 12 Gbps ingress plus 12 Gbps egress per chip)
Traffic Manager Support WRED, total 32K queues (8 queues per subscriber x 4K subscribers) per TM
Up to 5 levels of traffic shaping hierarchy
ProgrammingModel
Pipeline programming model w/o parallel programming or multi-threading. Parallelization doneby HW
ProgrammingLanguage
CISC-like assembly instruction set with powerful bit manipulation instructions.
Macro and Assembly programming.
HW Assistance QoS (Traffic Manager)
Classification / Searching
Counter Updates for statistics maintenance
Hardware: Trident Network Processor
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 72/127
Cisco ConfidentialViking Hardware Overview 73
Multicore/Multistage processing complex
10Gbps+ full duplex forwarding/feature engine
External Memory Interfaces
EgressBQS
engine
64k
queues
IngressBQS
engine
32kqueues
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
PPE
High Speed on-chip memory
InterfaceMAC/PHY
switchfabric
interface
BQS : “buffering / queueing / scheduling”
fabric
RSP
RSP
Hardware Subsystem: Trident Fwd Engine
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 73/127
Cisco ConfidentialViking Hardware Overview 74
Hardware Subsystem: Trident Fwd EngineTrident: TCAM partitioning
256k entry TCAM on each NP, divided into tables:
96k entries @ 144b key size (QoS, ACL, iFIB)
16k entries @ 576b key size (QoS, ACL, iFIB)
96k for EFP classification
specific application determines which key size used
applications share space WITHIN a key-size table
i.e. no hard-partitioning between QoS/ACL/iFIBetc.
ingress & egress features share same TCAM space
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 74/127
Cisco ConfidentialViking Hardware Overview 75
show controller np all summary
RP/0/RSP1/CPU0:ASR9010-A#sh controller np summ all
Mon Apr 6 10:31:21.480 UTC
Node: 0/0/CPU0:
----------------------------------------------------------------
[total 4 NP] Driver - Version 10.26a Build 1010 ( Jan 28 2009, 12:00:00 )
NP 0 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 1 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 2 : Hardware Revision: v2 A1: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 3 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
Node: 0/5/CPU0:
----------------------------------------------------------------
[total 4 NP] Driver - Version 10.26a Build 1010 ( Jan 28 2009, 12:00:00 )
NP 0 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 1 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 2 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
NP 3 : Hardware Revision: v2 A1
: Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 75/127
Cisco ConfidentialViking Hardware Overview 76
show controller np counter np0 loc <>
RP/0/RSP1/CPU0:ASR9010-A#sh controller np counters np0 loc 0/0/cpu0
Mon Apr 6 10:34:11.161 UTC
Node: 0/0/CPU0:
----------------------------------------------------------------
Show global stats counters for NP0, revision v3
Read 7 non-zero NP counters:
Offset Counter FrameValue-------------------------------------------------------------------------------
25 PARSE_FABRIC_RECEIVE_CNT 19601
53 XAUI_TRAINING_PKT_DISCARD 1093
57 RESOLVE_INGRESS_L2_PUNT_CNT 29294
85 DIAGS 9786
197 PUNT_STATISTICS 447536
199 PUNT_DIAGS_RSP_ACT 9789
201 PUNT_DIAGS_RSP_STBY 9719
For detail explanation on what these counters mean, please look at: asr9k_ucode-cntrs.doc ( EDCS-778662)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 76/127
Cisco ConfidentialViking Hardware Overview 77
show controller np ports all loc <>
RP/0/RSP1/CPU0:ASR9010-A#sh controller np ports all loc 0/0/cpu0
Mon Apr 6 10:44:41.289 UTC
Node: 0/0/CPU0:
----------------------------------------------------------------
NP Bridge Fia Ports
-- ------ --- ---------------------------------------------------
0 1 0 TenGigE0/0/0/3
1 1 0 TenGigE0/0/0/2
2 0 0 TenGigE0/0/0/1
3 0 0 TenGigE0/0/0/0
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 77/127
Cisco ConfidentialViking Hardware Overview 78
sh controller fabric fia link-status loc <>
RP/0/RSP1/CPU0:ASR9010-A#sh controller fabric fia link-status loc 0/0/cpu0
Mon Apr 6 09:40:18.677 UTC
Fia to Arbiter sync status
ARB 0 SYNCED
ARB 1 SYNCED
Fia to Crossbar sync status
Fabric Link 0 SYNCED
Fabric Link 1 SYNCED
Fabric Link 2 SYNCED
Fabric Link 3 SYNCED
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 78/127
Cisco ConfidentialViking Hardware Overview 79
sh controller fabric fia stats loc <>RP/0/RSP1/CPU0:ASR9010-A#sh controller fabric fia stats loc 0/0/cpu0
FIA DDR Packet counters:
==========================
From Bridge#[0] 228791To Bridge #[0] 91984
From Bridge#[1] 955781
To Bridge #[1] 942233
FIA SuperFrame counters:
==========================
To Unicast Xbar[0] 39221
To Unicast Xbar[1] 34181
To Unicast Xbar[2] 694682
To Unicast Xbar[3] 34133
To MultiCast Xbar[0] 0
To MultiCast Xbar[1] 45666
To MultiCast Xbar[2] 289374
To MultiCast Xbar[3] 46303
From Unicast Xbar[0] 39225
From Unicast Xbar[1] 34181
From Unicast Xbar[2] 834384
From Unicast Xbar[3] 34133
From MultiCast Xbar[0] 0
From MultiCast Xbar[1] 86872From MultiCast Xbar[2] 0
From MultiCast Xbar[3] 4456
FIA Total Drop counters:
====================
Ingress drop: 0
Egress drop: 0
Total drop: 0
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 79/127
Cisco ConfidentialViking Hardware Overview 80
sh controllers fabric fia bridge stats loc <>UC - Unicast , MC - Multicast
LP - LowPriority , HP - HighPriority
Cast/ Packet Packet Error ThresholdPrio Direction Count Drops Drops
--------------------------------------------------------------------------------
Unicast Egress Stats
********************
UC HP Fabric to NP-0 0 0 0
UC LP Fabric to NP-0 25497 0 0
UC HP Fabric to NP-1 0 0 0
UC LP Fabric to NP-1 25497 0 0
UC HP Fabric to NP-2 172099 0 0
UC LP Fabric to NP-2 25497 0 0
UC HP Fabric to NP-3 456510 0 0
UC LP Fabric to NP-3 445408 0 0
----------------------------------------------------------------
UC Total Egress 1150508 0 0
Multicast Egress Stats
*********************
MC HP Fabric to NP-0 25424 0 0
MC LP Fabric to NP-0 0 0 0
MC HP Fabric to NP-1 25424 0 0MC LP Fabric to NP-1 0 0 0
MC HP Fabric to NP-2 25424 0 0
MC LP Fabric to NP-2 0 0 0
MC HP Fabric to NP-3 25424 0 0
MC LP Fabric to NP-3 0 0 0
---------------------------------------------------------------
MC Total Egress 101696 0 0
Cast/ Packet Packet
Prio Direction Count
--------------------------------------------------
sh controllers fabric fia bridge ddr-status
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 80/127
Cisco ConfidentialViking Hardware Overview 81
glocation <>
RP/0/RSP1/CPU0:ASR9010-A#sh controllers fabric fia bridge ddr-status loc 0/rsp0/cpu0
Fri Apr 17 08:37:38.032 UTC
Fia DDR# Status
-------- ------
0 SYNCED
1 SYNCED
RP/0/RSP1/CPU0:ASR9010-A#sh controllers fabric fia bridge ddr-status loc 0/0/cpu0
Fri Apr 17 08:38:45.050 UTC
Fia DDR# Status
-------- ------
0 SYNCED
1 SYNCED
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 81/127
Cisco ConfidentialViking Hardware Overview 82
sh controllers fabric fia bridge sync-status loc <>RP/0/RSP1/CPU0:ASR9010-A#sh controllers fabric fia bridge sync-status loc 0/0/cpu0
Fri Apr 17 08:33:15.429 UTC
Bridge# NP# Status
------- --- ------
0 0 SYNCED
0 1 SYNCED
1 2 SYNCED
1 3 SYNCED
RP/0/RSP1/CPU0:ASR9010-A#sh controllers fabric fia bridge sync-status loc 0/rsp0/cpu0
Fri Apr 17 08:42:49.929 UTC
Invalid Node. NP link status can be checked only on LCs
sh controllers fabric fia bridge flow-control loc
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 82/127
Cisco ConfidentialViking Hardware Overview 83
g0/0/cpu0
RP/0/RSP1/CPU0:ASR9010-A#sh controllers fabric fia bridge flow-control loc 0/0/cpu0
Fri Apr 17 08:41:53.502 UTC
UC - Unicast , MC - Multicast
LP - LowPriority , HP - HighPriority
Cast/ BackPressure Backpressure
Prio Direction Count
---------------------------------------------------
Ingress BackPressure Counters
******************************
FIA to bridge-0 0
FIA to bridge-1 0
All Bridge to NP-0 0
All Bridge to NP-1 0
All Bridge to NP-2 0
All Bridge to NP-3 0
Egress BackPressure Counters
****************************
Bridge-0 to FIA 0
Bridge-1 to FIA 0
UC LP NP-0 to Bridge0 0UC LP NP-1 to Bridge0 0
UC LP NP-2 to Bridge1 0
UC LP NP-3 to Bridge1 0
UC HP NP-0 to Bridge0 0
UC HP NP-1 to Bridge0 0
UC HP NP-2 to Bridge1 0
UC HP NP-3 to Bridge1 0
MC LP NP-0 to Bridge0 0
MC LP NP-1 to Bridge0 0
MC LP NP-2 to Bridge1 0
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 83/127
Cisco ConfidentialViking Hardware Overview 84
Extended (-E) Line Cards
Address the highest service scale requirements Address the highest buffering capacity requirements
Feature / Performance are virtually identical –B Line Cards
A9K-40GE-E 40 Port GE SFP Extended Line Card
A9K-4T-E 4 Port 10GE XCP Extended Line Card
A9K-4T/8-E 8 Port 10GE XCP Extended Line Card
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 84/127
Cisco ConfidentialViking Hardware Overview 85
Major Differences from –B Line Cards
Relative to -B, the Extended (-E) Line Cards provide~2x increase in Interface Density
~2x increase in Bridge Domain Density
~4x increase in Queue Density
~3x Packet Buffer Capacity
Relative to –B, the Extended Line Cards haveadditional High-Speed Memories / TCAM Resources
Otherwise the –E Line Cards are virtually identical to -B
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 85/127
Cisco ConfidentialViking Hardware Overview 86
Examples of Non-Differences From –B LCs
Layer 3 Route Scale Layer 2 Route Scale
Support of BGP
Support of L3VPNs
Simultaneous Support of L2 / L3
Support of Dense Hierarchical QoS
Forwarding Performance
Li C d K M i
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 86/127
Cisco ConfidentialViking Hardware Overview 87
Line Card Key Metrics
Metric -B Line Cards -E Line CardsMAC Addresses 1M 1M
IPv4 Routes 1M 1M
IPv6 Routes 256k 256k
VRFs 4k 4kEFPs 16k 32k
Bridge Domains 8k 8k
Queues 96k 384k
Policers 128k 256kDHCP Clients 128k 128k
Packet Buffer 50ms 150ms
Scale support may be software release dependent
Hardware Subsystem: SIP 20G (Thor)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 87/127
Cisco ConfidentialViking Hardware Overview 88
Next Gen SPA interface Processor
(SIP) for Viking : Dynamic Subscriber Management
Flexibility to include additional applicationslike Mobile Backhaul
20G SIP based on Quantum FlowProcessor (QFP)
Two SPA variants (at FCS) : 10 x 1GE,1 x 10GE
Target future support for ChOC / ATM /TDM SPAs ..
8641D Dual Core CPU and memory
Hardware Subsystem: SIP 20G (Thor)
Hardware Subsystem: Thor
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 88/127
Cisco ConfidentialViking Hardware Overview 89
QFP 0
Fabric
InterfaceSerpent
FPGAHyTop
FPGA
SPA
I/F
DRAM
Drakar
FPGA
CPU
SPA
I/F
HH SPA
PEX8509
HH SPA
HH SPA
HH SPA
yBlock Diagram w/ 4x Half-height SPAs
Ingress QFP complex
CPU daughtercard
SPA daughtercard
QFP Memory
Egress QFP complex
QFP 1
QFP Memory
Hardware Subsystem: QFP
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 89/127
Cisco ConfidentialViking Hardware Overview 90
• 40 Packet processors – 4 contexts (threads) each
1.2GHz processors with associated caches + DRAM packet memory
Single TCAM4 I/F (can cascade 1-4 devices)
C-language (supports stack)
HW assist for locks, PLU, hash LU, stats, WRED, policers, range lookup,
crypto, CRC
• Buffer/queue subsystem (CPP-BQS)
supports up to 128k HW queues per direction
256MB max packet buffer
• HW hierarchical scheduler
Fully configurable # of layers based on HQF
yQuantumFlow Processor (formerly CPP)
Viki C d
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 90/127
Cisco ConfidentialViking Hardware Overview 91
Viking Codenames
Ryan – 40x1GE Linerate LC for Red October/Samurai Ramius – 8x10GE Over-subscribed LC for Red October/Samurai
Mancuso - 4 x10GE Linerate LC for Red October/Samurai
Jetfire – 2x10GE + 20 GE Linerate LC for Red October/Samurai
Odin – 8x10GE Linerate LC for Red October/Samurai
Valhalla – 16x10GE Over-subscribed LC for Red October/Samurai
Maximus – RSP for Red October/Samurai
Red October – 10 Slot Chassis
Samurai – 6 Slot Chassis
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 91/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 92
ASR9K Life of aPacket
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)
Online Diagnostics
Service Console
P k t P th S h t
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 92/127
Cisco ConfidentialViking Hardware Overview 93
Packet Path Snapshot
SFP/XFP
OpticsNP3c
Bridge
FPGAOctopus
B a c k pl an e
Santa
Cruz
Santa
Cruz
SFP/XFP
OpticsNP3c
Bridge
FPGAOctopus
Moto 8641D
OctopusPunt
FPGA
Moto 8641D
Normal Path
Punt Path
32G DDR
per bridge
15G XAUI
per NP3
10G MAC
per PHY
2 x 23G
Chico
per LC
Punted to RSP if
FPOE says so
Life of a Packet
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 93/127
Cisco ConfidentialViking Hardware Overview 94
via backplane
Ingress Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
1
4
3
2
NetworkClocking
10GE PHY
10GE PHY
10GE PHY
10GE PHY
XFP
XFP
XFP
XFP
B id FPGA
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 94/127
Cisco ConfidentialViking Hardware Overview 95
Bridge FPGA
The Bridge FPGA is acting as a traffic adaptorbetween DDR interface of Octopus ASIC andXAUI-like interface of G8000 Networkprocessor
There are two data paths from the NPBridge3 FPGA:
Manipulates the header and does interface conversion, to theOctopus
This path handles all main data and also central punt data thatare routed
Local CPU through a switched GE link
used to process punt data locally or the packets may be sent tothe RSP card via the backplane GE or fabric link
Internal Packet Headers
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 95/127
Cisco ConfidentialViking Hardware Overview 96
Internal Packet Headers
Life of a Packet
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 96/127
Cisco ConfidentialViking Hardware Overview 97
RSP0
Fabric Packet Path
Octopus(LC)
Santa
Cruz 0
Octopus (LC)
SantaCruz 1
Bellagio
RSP1
SantaCruz 0
SantaCruz 1
Bellagio
1: Fabric Request
3: Fabric Grant
2: Arbitration
4: load-balanced
transmission across fabric links
5: credit return
Life of a PacketE P k P h
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 97/127
Cisco ConfidentialViking Hardware Overview 98
via backplane
Egress Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory2GB flash
gigEcontrolnetwork
RSP 1
daughtercard
4
1
2
35
10GE PHY
10GE PHY
10GE PHY
10GE PHY
XFP
XFP
XFP
XFP
NetworkClocking
Linecard backplane GE links
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 98/127
Cisco ConfidentialViking Hardware Overview 99
Linecard backplane GE links
The backplane GE links, one to each RSP card, areused primarily for control plane functions such asapplication image download, system configuration datafrom IOS-XR, statistics gathering and line card power-up and reset control.
sh controller backplane ethernet links loc <>
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 99/127
Cisco ConfidentialViking Hardware Overview 100
sh controller backplane ethernet links loc <>
RP/0/RSP1/CPU0:ASR9010-A#sh controller backplane ethernet links loc 0/rsp0/cpu0
Fri Apr 17 12:43:02.946 UTC
Node | Link State
----------------------------------
0/RSP0/CPU0 | Up (Local CPU)
0/RSP1/CPU0 | Up (Peer CPU)
0/0/CPU0 | Up
0/1/CPU0 | Down
0/2/CPU0 | Down
0/3/CPU0 | Down
0/4/CPU0 | Down
0/5/CPU0 | Up
0/6/CPU0 | Down
0/7/CPU0 | Down
RP/0/RSP1/CPU0:ASR9010-A#sh controller backplane ethernet links loc 0/rsp1/cpu0 Fri Apr 17 12:43:41.535 UTC
Node | Link State
----------------------------------
0/RSP1/CPU0 | Up (Local CPU)
0/RSP0/CPU0 | Up (Peer CPU)
0/0/CPU0 | Up
0/1/CPU0 | Down
0/2/CPU0 | Down
0/3/CPU0 | Down
0/4/CPU0 | Down
0/5/CPU0 | Up
0/6/CPU0 | Down
0/7/CPU0 | Down
Life of a PacketE M lti t P k t P th
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 100/127
Cisco ConfidentialViking Hardware Overview 101
via backplane
Egress Multicast Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory
gigEcontrolnetwork
RSP 1Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
mcast
Life of a PacketE M lti t P k t P th
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 101/127
Cisco ConfidentialViking Hardware Overview 102
via backplane
Egress Multicast Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
CPU8641D
2GB memory
RSP 1Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Subscriber
Non-subscriber
mcast
NewSub
Life of a PacketE M lti t P k t P th
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 102/127
Cisco ConfidentialViking Hardware Overview 103
Subscriber
via backplane
Egress Multicast Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
RSP 1Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Subscriber
Non-subscriber
mcast
Subscriber
Subscriber
Add replication only as new subscribersare added, prune when tree is empty...
Life of a PacketE M lti t P k t P th
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 103/127
Cisco ConfidentialViking Hardware Overview 104
via backplane
Egress Multicast Packet Path
RSP 0
Trident 0
Trident 1
Trident 2
Trident 3
BridgeFPGA 0
Trident Memory
BridgeFPGA 1
Trident Memory
Trident Memory
Trident Memory
Octopus
RSP 1
Non-subscriber
Non-subscriber
Subscriber
Subscriber
Subscriber
Subscriber
Non-subscriber
Non-subscriber
Non-subscriber
Subscriber
Non-subscriber
mcast
Subscriber
Subscriber
SubscriberSubscriber
Replicate intelligently and using need-based tree
saves bandwidth and processing cycles, while allowing for per-replicaservices delivery
Non-subscriber
(Pruned)(Pruned)
(leave)
Viking MulticastViking Multicast Forwarding Plane
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 104/127
Cisco ConfidentialViking Hardware Overview 105
LC2
Switch Fabric
LC1LC3
T0 T1 T3
Fabric Interface
B1B0
IGMP joinsMulticastSource
[1] IngressTrident RPF
check.
FGID : Fabric Group ID MGID : Multicast Group ID
[2] Switch Fabric Replicatesthe packet to the appropriate
Fabric Interface chips on theLine Cards using the FGID.
(Ex: 1110). Does not gothrough VOQ
[5] T Replicates the packet tothe local Interfaces using the
OIL
T1 T2T0 T3
R
R
RP
1 1 1
1 0
Fabric Interface Fabric Interface
B1B0 B1B0
[4] Bridge Replicates thepacket to the appropriate Tsusing the MGID. Has 32K
MGID entries w/ 2 bits for Tidentification
R
R
[3] Fabric Interface Replicatesthe packet to the appropriate
Bridges using the MGID.150KB Buffer for Multicast and
32K MGID entries.
[0] IGMP joins for
subscriberregistration.
Replication EnginesR
T0 T1 T2 T3T2
Viking Multicast Forwarding Plane
ASR9K Overview
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 105/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 106
ASR9K PowerManagement
ASR9K Overview
ASR9K Chassis
Architecture
RSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)
Online Diagnostics
Service Console
Power Management
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 106/127
Cisco ConfidentialViking Hardware Overview 107
Power Management
Power Management is a customer requirement that the featureavailable start from FCI (3.7.2) release.
Power Management enabled ASR9K chassis have the capability toprevent an FRU from being powered on, in the event that there isnot enough system power available to accommodate the FRU.
Power Management uses Power Monitor software feature whichallows user to monitor the power used and how much is available.
Power Management feature also alarms the user if there is notsufficient power or any other related events.
Power management feature is not available in 3.7.1 and onGladiator chassis.
System Overview
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 107/127
Cisco ConfidentialViking Hardware Overview 108
System Overview
PwrMon processIdentifies and communicates with power supply modules via I2C
Shelfmgr process
Allows or Denies LC power-up based on (1) estimated powerrequirements & (2) available power modules
Publishes Power Supply OIR events to internal consumers.
EnvMon process
Publishes power budget status to CLI
InvMgr processPublishes detailed power supply description to inventory
Feature Description
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 108/127
Cisco ConfidentialViking Hardware Overview 109
Feature Description
Inventory Description for Power Supplies
Indicate how many power module in system
Indicate power module source is AC or DC
RP/0/RSP0/CPU0:Samurai(admin)#show inventory… …
NAME: "power-module 0/PM0/SP", DESCR: "1.5kW DC Power Module"PID: A9K-1.5KW-DC , VID: V00, SN: TLD123950YS NAME: "power-module 0/PM1/SP", DESCR: "2kW DC Power Module"PID: A9K-2KW-DC , VID: V01, SN: TLD1214001S
==================
RP/0/RSP0/CPU0:RO(admin)#show inventory
… … NAME: "power-module 0/PM0/SP", DESCR: "3kW AC Power Module"PID: A9K-3KW-AC , VID: V00, SN: TLD123850NH NAME: "power-module 0/PM2/SP", DESCR: "3kW AC Power Module"PID: A9K-3KW-AC , VID: VOO, SN: TLD123650RA
Feature Description
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 109/127
Cisco ConfidentialViking Hardware Overview 110
Feature Description
Power Management Command Summary
Power Management Command Summary
Command Purpose
RP/0/RP0/CPU0:Router(admin)# show platform Displays information and status for eachnode in the system includes power supply
RP/0/RP0/CPU0:Router(admin)# show inventory To retrieve and display information aboutall the Cisco products that are installed inthe router , includes power supply
RP/0/RP0/CPU0:Router(admin)# show environmentpower-supply
Displays power supply capacity andcurrent information
RP/0/RP0/CPU0:Router (admin-config)#hw-modulepower [disable] location { location}
To power on a specified line card or
disable the node power-on feature
RP/0/RP0/CPU0:Router (admin-config)#hw-modulepower override location { location}
To allows a card to be forced to power up,regardless of an unprogrammed EEPROMvalue
Feature Description
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 110/127
Cisco ConfidentialViking Hardware Overview 111
Feature DescriptionChecking available power capacity
RP/0/RSP0/CPU0:VKG-3(admin)#show environment power-supply Mon Oct 12 12:25:05.399 UTC
R/S/I Modules Sensor Watts Status
0/PM0/*host PM 3000 Ok
0/PM1/*host PM 3000 Ok
0/PM2/*host PM 3000 Ok
Power Shelves Type: AC
Total Power Capacity: 9000WUsable Power Capacity: 9000WSupply Failure Protected Capacity: 6000WFeed Failure Protected Capacity: 3000W Worst Case Power Used: 2615W
Slot Max Watts---- ---------0/0/CPU0 375
0/1/CPU0 3750/RSP0/CPU0 2500/RSP1/CPU0 2500/7/CPU0 3750/FT0/SP 495 (default)0/FT1/SP 495 (default)
Worst Case Power Available: 6385WSupply Protected Capacity Available: 3385WFeed Protected Capacity Available: 385W
ASR9K Overview
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 111/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 112
Media and FileSystem
ASR9K Overview
ASR9K Chassis
ArchitectureRSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)
Online Diagnostics
Service Console
Devices On RSP
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 112/127
Cisco ConfidentialViking Hardware Overview 113
Devices On RSP
Device Type Quantity Size FS Type
Harddisk SAS 1 70GB QNX4
eUSB NAND 2 2GB/ea QNX4
C. Flash
(removable)
Compact
Flash
1 1GB DOS/QNX4
Flash NOR 2 64MB QNX FFSv3
NVRAM 1 512KB Cisco
Devices On LC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 113/127
Cisco ConfidentialViking Hardware Overview 114
Devices On LC
Device Type Quantity Size FS Type
eUSB NAND 1 2GB QNX4
Flash NOR 1 128MB QNX FFSv3
Partitions On RSP
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 114/127
Cisco ConfidentialViking Hardware Overview 115
Partitions On RSP
Device Partition Size Purpose
Harddisk /harddisk:
/harddiska:
/harddiskb:
Raw
70GB(total)
dumps/ etc.
Primary for kernel dumps
eUSB /disk0:
/disk0a:
/disk1:
/disk1a:
1.6GB
0.4GB
1.6GB
0.4GB
IOS-XR Packages
Golden Disk or Mirror Disk
C. Flash
(removable)
/compactflash: 1GB General Purpose Storage
Partitions On RSP (cont’d)
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 115/127
Cisco ConfidentialViking Hardware Overview 116
Partitions On RSP (cont d)
Device Partition Size Purpose
Flash 1 Raw
/bootflash:
Raw
43MB
MONLIB
MBI images
ROMMON/Firmware
Flash 2 Raw
/configflash:
Raw
28MB
MONLIB
OBFL data
Secondary for kernel dumps
NVRAM /nvram:
/nvram-raw:
Variables, logs
Reboot history; crashinfo,PCDS
Partitions On LC
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 116/127
Cisco ConfidentialViking Hardware Overview 117
Partitions On LC
Device Partition Size Purpose
eUSB /eusb0:
/eusb0a:
1.6GB
0.4GB
Local Kernel Dumps
Flash Raw/bootflash:
Raw
56MBMONLIB
ROMMON/Firmware/Kerneldumps
DOS-QNX4 File System Utility Comparison
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 117/127
Cisco ConfidentialViking Hardware Overview 118
DOS QNX4 File System Utility Comparison
DOS QNX4
Types 6, 4, 11, 12 77, 78, 79, 80
FileOrganization
Block-based Extent-based
Patition fdisk fdisk
Initialize mkdosfs dinit
fs check chkdosfs chkfsys
Verify media dcheck dcheck
Standard QNX utilities. Refer to QSSL web site.
CLI: show filesystem
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 118/127
Cisco ConfidentialViking Hardware Overview 119
CLI: show filesystem
RP/0/RSP0/CPU0:ios#show filesystem
File Systems:
Size(b) Free(b) Type Flags Prefixes
- - network rw qsm/dev/fs/tftp:
- - network rw qsm/dev/fs/rcp:
- - network rw qsm/dev/fs/ftp:
…
1644150784 1404023296 flash-disk rw disk1:
1644150784 1404023296 flash-disk rw disk0:
35645292544 35628865024 harddisk rw harddisk:
1022427136 1022418944 flash-disk rw compactflash:
8075067392 8073070080 harddisk rw harddiska:
7840202752 7838258688 harddisk rw harddiskb:
411041792 410914816 flash-disk rw disk1a:
411041792 410903552 flash-disk rw disk0a:
224256 165888 nvram rw nvram:
290816 290816 nvram rw nvram-raw:
44695552 23889200 flash rw bootflash:
28966912 28862044 rw configflash:
show media
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 119/127
Cisco ConfidentialViking Hardware Overview 120
show media
RP/0/RSP0/CPU0:VKG-3(admin)#sh mediaFri Apr 9 15:05:34.364 EST
Media Information for 0/RSP0/CPU0.Image Current Part
Mountpoint FsType FsType Size State DrvrPid Mirror Flags================================================================================/disk0: QNX4 QNX4 1.5G Mounted 0069666 Enabled/disk0a: QNX4 QNX4 0.4G Mounted 0069666/disk1: QNX4 QNX4 1.5G Mounted 0069676/disk1a: QNX4 QNX4 0.4G Mounted 0069676/compactflash: FAT16 FAT16 1.0G Mounted 0081977
/harddiska: QNX4 QNX4 7.5G Mounted 0049184/harddiskb: QNX4 QNX4 7.3G Mounted 0049184/harddisk: QNX4 QNX4 33.2G Mounted 0049184/lcdisk0: QNX4 (?) Not Present/lcdisk0a: QNX4 (?) Not Present
RP/0/RSP0/CPU0:VKG-3#sh media loc 0/0/cpu0Fri Apr 9 15:13:51.680 EST
Media Information for 0/0/CPU0.Image Current Part
Mountpoint FsType FsType Size State DrvrPid Mirror Flags================================================================================/lcdisk0: QNX4 QNX4 1.5G Mounted 0049185/lcdisk0a: QNX4 QNX4 0.4G Mounted 0049185
ASR9K Overview
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 120/127
© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialViking Hardware Overview 121
Online Diagnostics
ASR9K Overview
ASR9K Chassis
ArchitectureRSP
Fabric
Linecard
Life of a Packet
Power Management
Media and File Systems
Platform Fault Management (PFM)
Online Diagnostics
Service Console
What is online diagnostics
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 121/127
Cisco ConfidentialViking Hardware Overview 122
g
It defines a suite of tests for each card in system, although sometests test inter-card links
Tests run in background, faults reported to Platform Fault Manager
Tests can be enabled and disabled individually, test frequency canbe changed through configuration commands
Components Covered by FCI:
- Control path and data path links
- access to scratch registers for major ASICs and FPGAs/CPLDs
- integrity of FPD images
EOBC paths
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 122/127
Cisco ConfidentialViking Hardware Overview 123
p
EOBC heartbeat tests
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 123/127
Cisco ConfidentialViking Hardware Overview 124
Bi-directional;
Runs on all Line cards and Standby RSP
every 5 seconds, by default
Interval can be changed by changing configuration
User Interface
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 124/127
Cisco ConfidentialViking Hardware Overview 125
Generic user interface : GOLD-XR, same as on CRS Only admin users have privilege to access online diagnostic commands:
commands are in admin mode
Most tests running in the background by default
Show commands to see what tests are available, running frequency ofeach test, etc.:
“show diagnostic content location <node>”
Show commands to see tests result:“show diagnostic result location <node> [test <test-id> detail ]”
Change test running frequency or enable/disable a test:use admin configuration commands:“diagnostic monitor interval location <node> test <test-id> <day>
<hh:mm:ss.mille-sec>” “diagnostic monitor location <node> test <test-id> disable”
Sample screen shots
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 125/127
Cisco ConfidentialViking Hardware Overview 126
Sample screen shots
RP/0/RSP0/CPU0:ios(admin)#show diagnostic result location 0/rsp0/cpu0
Current bootup diagnostic level for RP 0/RSP0/CPU0: minimal
RP 0/RSP0/CPU0:
Overall diagnostic result: PASS
Diagnostic level at card bootup: minimal
Test results: (. = Pass, F = Fail, U = Untested)
1 ) PuntFPGAScratchRegister ---------> .
2 ) XbarInterfaceScratchRegister ----> .
3 ) ClkCtrlScratchRegister ----------> .
4 ) IntCtrlScratchRegister ----------> .
5 ) CPUCtrlScratchRegister ----------> .
6 ) XbarSwitchIdRegister ------------> .
7 ) EccSbeTest ----------------------> U
8 ) SrspStandbyEobcHeartbeat --------> U
9 ) SrspActiveEobcHeartbeat ---------> U
10 ) FabricLoopback ------------------> .
11 ) PuntFabricDataPath --------------> U
12 ) BootflashVerify -----------------> U
Referencias
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 126/127
Cisco ConfidentialViking Hardware Overview 127
Referencias
ASR9000/XR Understanding andtroubleshooting Fabric issues in the A9K https://supportforums.cisco.com/document/12135016/asr9000xr-understanding-and-troubleshooting-fabric-issues-a9k
ASR9000/XR: Troubleshooting packetdrops and understanding NP dropcounters
https://supportforums.cisco.com/docs/DOC-15552
7/17/2019 1 Asr9k Arc Base
http://slidepdf.com/reader/full/1-asr9k-arc-base 127/127