© 2006 cisco systems, inc. all rights reserved.cisco public [email protected] 1 router software...

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Public [email protected] 1

Router Software Architecture – Now and Going Forward

Michael Beesley,

Engineering Director,

Routing Technology Group, Cisco Systems

[email protected]


Agenda

Overview of Router software components

IO Plane

Forwarding Plane

Control Plane

Constraints and requirements

Software component evolution

Summary


Overview of Router Software Components


IO Plane

Device drivers to manage IO hardware

Handles configuration, status reporting and statistics

Typical embedded application

Requires modest CPU and memory resources

Modest amount of software, with no major stability or maintenance concerns

Fairly low entropy in requirements – IO hardware evolution has slowed (possible exception is ring technology, RPR etc)

Will usually have dedicated CPU resources

Modularity at the per driver/per IO slot level


Forwarding Plane - Control

Embedded software to manage the forwarding/data plane hardware

Builds tables, trees, classification tables/TCAM entries etc required by the forwarding plane to process packets and apply features

Gathers statistics and state from forwarding plane hardware

Can require significant CPU and memory resources

Over time is a continually growing piece of software in size and complexity as hardware and feature set evolves

Will usually have dedicated CPU resources

Scale, performance and real time response are important


Forwarding Plane – Packet Processing

Depends on system and hardware architectureMay not exist – all hard coded silicon

Can be small amounts of custom microcode

Can be larger amounts of software in a higher level language (typically C with asm extenions)

Main job is to process packets and apply all configured features to packet flows

For stateful features where first packet processing is done in the forwarding plane, handles state creation, destruction and update of control plane

Some house keeping with regard to statistics, resource management may be required

As forwarding plane feature set expands, increases in size and complexity Processing power and memory resources depends greatly on scale and

performance of router being built Hardware assists will be included to offload heaviest work:

QoS scheduling, crypto, tree lookups, statistics management, classification etc


Control Plane

Largest and most challenging software component in a router:

CLI and other external management functions (SNMP, XML etc)

Routing protocols

Link layer protocols

Gateway to off box services (DHCP, SIP, Radius etc)

Must scale in terms of:

Physical and logical (subscriber) interface count

Routing protocol peers

Route and prefix counts

Off box services transactions per second

Very rich feature set with high feature velocity

Some real time(ish) requirements

Can easily be 20M~40M lines of source code


Constraints and Requirements


Constraints and Requirements

Reliable

5 9’s up time (3 minutes a year outage)

Must be very reliable and must have good redundancy/upgrade mechanisms

Scalable

In both enterprise and service provider, scale is going up in terms of peers, interfaces/subscribers and configured feature sets

Functional

Very rich feature set with some features (BFD, APS, Fast Reroute) pushing real timeliness of the system as networks converge onto packet based infrastructure

Long hardware lifetime with reluctance to upgrade

Due to power/cooling/lifetime, highest end CPUs typically not usable

Existing control plane software architectures can make using multi-core CPUs problematic


Software Component Evolution


IO Plane Evolution

Software redundancy to facilitate bug fixing and system software upgrade

Dual hw/sw complexes

Minimum Disruptive Restart of software/CPU

Scales with router size and IO, as such could offload some simple, repetitive but expensive tasks from control plane

Link layer keepalives

Media management (OAM, LMI etc)

Per IO module software modularity and separation

Allows different versions of the same driver to be running on the same line card, controlling different IO modules


Forwarding Plane Evolution

Redundancy and version translation between control software and packet processing code/microcode

Minimum Disruptive Restart of software/CPU

Fast restart of packet processing forwarding engine

Ample processing power, as such could offload some tasks from control plane

Routing protocol keepalives, BFD, etc

Resource exhaustion management (classification, prefixes, QoS queues etc)


Control Plane Evolution

Good hardware and software redundancy must be built in to the infrastructure to achieve carrier class

Clean separation between OS and application

Simple but expensive tasks must be offloaded:

To the IO plane (media/link layer protocol keepalives)

To the forwarding plane (routing protocol/BFD keepalives)

To dedicated hardware (key generation etc)

Multi core CPU usage (either SMP or master/slave)

To scale compute resources of control plane

All algorithms and data structures must be designed for robustness and scale


Control Plane Evolution (continued)

Consider separating some functions and moving them off box:Configuration

Provisioning

Element management/CLI

Some degree of modularity needed:Coarse grain modularity

Fine grain modularity

Monolithic application separated from an underlying OS

Scheduler choices:Pre-emptive

Co-operative

Two level scheduler, Co-operative on top of Pre-emptive

Must consider deprecating features to control code size/quality Might consider using more advanced programming languages


Summary


Summary

Modern routers are very complex hardware and software systems with demanding requirements and constraints

To achieve carrier class converged networks, router software and hardware architecture is going to have to evolve to better achieve scale and reliability

Some functions may be better implemented off box in a management or provisioning layer

Some efforts to limit feature sets and obsolete older less used features would improve system characteristics and reliability


Questions?


Thank You

© 2006 cisco systems, inc. all rights reserved.cisco public [email protected] 1 router software...

Documents

cisco systems

cisco public

forwarding plane hardware

control plane largest

forwarding plane feature

router software architecture

io plane device drivers

piece of software