3 - ios xr software architecture v1.1

Cisco ASR9000 seriesIOS XR Software Architecture

2010 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialPresentation_ID 2

IOS XR Architecture Basics

Core OS CapabilitiesProtected process memory spacePreemptive multitasking

High AvailabilityProcess Restart

Local Packet Transport Service (LPTS)


Modular IOS != IOS XR

Modular IOS:

Ships today on Catalyst 6500 with Sup720 and Sup32

Based on the same IOS code with added Microkernel and IOS split into multiple processes.

Not everything as its own process (ie all Routing as one process), optimized for performance on existing hardware

IOS XR:

Ships today on CRS, ASR9000, XR12000 series

Complete rewrite of the code

Very modular, split into multiple processes and built for multi-terabit scaling and distributed operation

Features targeted for SP NGN router


Monolithic Kernel (BSD/Linux, NT) MMU with partial protection. Only

applications are protected

The Microkernel, the foundation of IOS XR

TRUE Microkernel (Mach, QNX) MMU with full protection for protected

Applications, drivers, and protocols

Process

Manager

K

Filesystem

Application Application

Kernel Filesystem Network Driver

DriverApplication

Feature Microkernel Monolithic Kernel

Preemptive scheduler with support for process priority Yes Yes

Protected memory architecture for application processes Yes Yes

Protected memory architecture for system processes Yes NO

Fault protection for application processes Yes Yes

Fault protection for Host Stack Yes NO

Fault protection for device drivers Yes NO

Fault protection for file system Yes NO

In Service SW Upgrade for application processes Yes Yes

In Service SW Upgrade for Network Drivers, File System Yes NO

System wide corruption -

Router Restart


IOS XR Software ArchitectureModular, Distributed Architecture

IOS XR Architecture Features

Real Time Deterministic Scheduling

Full Memory Protection

Light weight Microkernel

Restartability

Patchability

True Modularity

Distributed Processes/subsystems

Checkpointing for stateful recovery

IOS XR Architecture Benefits

Reliable architecture enabling highly available applications

Distributed to enable high level of scale limited only by hardware

Feature velocity due to modular software design


IOS XR Modular Software Packaging

Code base files are organized into components these are versioned and visible to the development engineer

Packages are unique sets of components and represent potential units of delivery

Packages are visible in the code base build infrastructure prevents illegal dependencies between packages

Packages can be grouped into composites for ease of delivery

SW is packaged and can be upgraded along these Composites:

Host includes Microkernel, Infrastructure code, platform independent forwarding code, host stack

Line Card Line card specific drivers and platform code

Routing - Support for static & dynamic unicast routing

Multicast - Support for Multicast protocols

MPLS MPLS, GMPLS, & UCP functionality

Mgmt XML, CWI

Security non-exportable security features

Line card

OSPF ISIS

RPL BGPRouting

Composite

OS

Admin

Base

Forwarding

Host

Composite

MPLSMulti-

cast

Mgmt Security


Protected Process Memory Space

Each process has a virtual memory spaceKernel maps virtual address to physical address (at page level)

Threads share the memory space

One process cannot corrupt anothers memoryProcess can only access virtual space

In IOS all processes shared same virtual space

Enables process restart

Communication between processes via controlled APIs

Limited use of shared memory

0x000000

OSPF

1

2

3

0x00000

0x10000

0x20000

0x100000

0x200000

0x300000

0x400000

0x500000

0x600000

0x700000

0x800000

0xa00000

0x900000


Preemptive Multitasking

Default priority is 10

Higher priority processes can interruptIn IOS, must wait for running process to finish

FIFO within same priority

Threads run while parent process is running

1010

10 16

16

62

50

50

50

50 50

10

10

10

10

16

16

16

16

62

50

10

10

1016

16

16

62

62

50

Sleeping

Ready

Waiting

505050


Example Process Priorities

RP/0/RP0/CPU0:TME#show process

JID TID Stack pri state HR:MM:SS:MSEC NAME

80 1 12K 63 Nanosleep 0:00:00:0006 wd-mbi

57 1 28K 10 Receive 0:00:03:0058 dllmgr

57 2 28K 10 Nanosleep 0:00:00:0048 dllmgr

57 3 28K 10 Receive 0:00:02:0527 dllmgr

57 4 28K 10 Receive 0:00:04:0537 dllmgr

57 5 28K 10 Receive 0:00:05:0847 dllmgr

281 1 40K 10 Receive 0:01:19:0038 qsm

281 2 40K 10 Receive 0:00:00:0000 qsm

281 5 40K 10 Receive 0:00:00:0002 qsm

281 7 40K 10 Nanosleep 0:43:16:0203 qsm

71 2 36K 10 Sigwaitinfo 0:00:00:0002 pkgfs

69 4 56K 12 Intr 0:00:00:0000 pcmciad

78 7 36K 10 Condvar 0:00:00:0000 syslogd_helper

59 2 56K 10 Sem 0:02:13:0504 eth_server

67 2 24K 10 Receive 0:00:00:0000 nvram

67 3 24K 10 Nanosleep 0:00:00:0000 nvram

59 1 56K 50 Receive 0:11:35:0873 eth_server

59 2 56K 10 Sem 0:02:13:0508 eth_server

559 5 56K 10 Sem 0:00:00:0000 eth_server



54 1 64K 10 Receive 0:00:00:0034 bfm_server

Highest

Priority


Process Restart

Microkernel includes minimal functionality

Non-kernel processes can be restarted

Critical to HA functions

Required for software patchingShut down old version of process

Start fixed version of process


Demonstrating Process RestartSame Job ID, New Process ID

RP/0/RSP0/CPU0:Nevada#show proc bgp

Fri Feb 23 17:31:24.271 UTC

Job Id: 140

PID: 3277026

Executable path: /disk0/asr9k-rout-3.9.1/bin/bgp

Instance #: 1

Version ID: 00.00.0000

Respawn: ON

Respawn count: 3

Max. spawns per minute: 12

Last started: Fri Feb 23 17:31:20 2001

Process state: Run (last exit due to SIGTERM)

Package state: Normal

Started on config: ipc/gl/ip-bgp/meta/speaker/0

core: MAINMEM

Max. core: 0

Placement: Placeable

startup_path: /pkg/startup/bgp.startup

Ready: 0.371s

Available: 2.790s

Process cpu time: 0.178 user, 0.034 kernel, 0.212 total

RP/0/RSP0/CPU0:Nevada#proc restart bgp

RP/0/RSP0/CPU0:Nevada#show proc bgp

Fri Feb 23 17:33:00.103 UTC

Job Id: 140

PID: 3293410

Executable path: /disk0/asr9k-rout-3.9.1/bin/bgp

Instance #: 1

Version ID: 00.00.0000

Respawn: ON

Respawn count: 4


Process RestartMicrokernel architecture enables restart of most processes

MicrokernelIOS XR

KernelBSD based routers

MonolithicIOS

Timers SchedulerTimers SchedulerTimers Scheduler

BGP OSPF

EIGRP ISIS

RIP VPN

SSH Telnet

Server

IPv4 Forwarding

ACLsLDP

TCP/IP Drivers

BGP OSPF

ISIS

RIP

VPN

SSHTelnet

Server

IPv4 Forwarding

ACLs

LDP

BGP OSPF

EIGRP ISIS

RIP VPN

SSH Telnet

Server

IPv4 Forwarding

ACLsLDP

TCP/IP DriversTCP/IP Drivers

Green areas

cannot restart


LPTS Local Packet Transport Service

Serves multiple functions with IOS-XR

Delivers packets from outside of the router to correct node in router

Protect RP/LC CPUs from excessive traffic (by using HWACLs/policers)

LC

Active RP Standby RP

LPTS

LC LC


LPTS Protects RPs and LC CPUsOnly forwards packets to registered ports

RP

LC

LC

LC

LC

LC

LC

LC


IOS XR LPTSDynamic Control Plane Protection

Local port Remote port Rate Priority

Any ICMP ANY ANY 1000 low

any 179 any any 100 medium

Router bgp

neighbor 202.4.48.99

!

any 179 202.4.48.99 any 1000 medium

202.4.48.1 179 202.4.48.99 2223 10000 medium

200.200.0.2 13232 200.200.0.1 646 100 medium

LC 1 IFIB TCAM HW EntriesL

PT

S

So

cke

t

BGP

LDP

SSHLC 2 IFIB TCAM HW Entries

mpls ldp

!

TCP Handshake

ttl_security

ttl

255

IFIB LPTS Internal FIB

DCoPP is an automatic, built in firewall for control plane traffic.

DCoPP is being made user configurable


LPTS Example BGPSession not yet configured

RP/0/RSP0/CPU0:Nevada# show lpts pifib hardware entry loc 0/0/CPU0

VRF ID : any

Destination IP : any

Source IP : any

Is Fragment : 0

Interface : any

M/L/T/F : 0/BGP4_FM/0/BGP-default

DestNode : 48

DestAddr : 48

SID : 9

L4 Protocol : TCP

TCP flag byte : any

Source port : Port:any

Destination Port : 179

Ct : 0x36450

Accepted/Dropped : 0/0

Lp/Sp : 1/0

# of TCAM entries : 1

HPo/HAr/HBu/Cir : 476734/2500pps/2500ms/2480pps

State : Entry in TCAM

Rsp/Rtp : 19/21

Any source

TCP, dest 179 is BGP

Send to RSP0

Police all unknown sessions

(collectively) to 2500pps

Traffic will be categorized by HW TCAM (same HW as transit ACLs)

Policing implemented by HW Policer (same HW as input policer)

Any IP address


Dynamic LPTS ExampleNew BGP Peer Configured (but not up)


Fri Feb 23 01:57:54.243 UTC

Source IP : 10.1.2.2

Is Fragment : 0

Interface : any

M/L/T/F : 0/IPv4_LISTENER/0/BGP-cfg-peer

DestNode : 48

DestAddr : 48

SID : 8

L4 Protocol : TCP

TCP flag byte : any

Source port : Port:any


Ct : 0x3649e


Lp/Sp : 1/255




Rsp/Rtp : 5/7

router bgp 100

address-family ipv4 unicast

!

neighbor 10.1.2.2

remote-as 999

address-family ipv4 unicast

route-policy pass in

route-policy pass out

Source IP is known

now that neighbor is

configured

2000 pps

BGP but other TCP

port isnt known


Dynamic LPTS ExampleBGP Session Up Specific Values Programmed into LPTS


Fri Feb 23 01:55:48.642 UTC

Source IP : 10.1.2.2

Is Fragment : 0

Interface : any

M/L/T/F : 0/IPv4_STACK/0/BGP-known

DestNode : 48

DestAddr : 48

SID : 7

L4 Protocol : TCP

TCP flag byte : any

Source port : Port:179


Ct : 0x364a2


Lp/Sp : 1/255




Rsp/Rtp : 5/7

1500 pps for this specific session

Both TCP ports are now known

3 - ios xr software architecture v1.1

Documents