a router architecture to achieve link rate throughput in suburban ad-hoc networks ronald pose carlo...

A Router Architecture to Achieve A Router Architecture to Achieve Link Rate Throughput in Suburban Link Rate Throughput in Suburban

Ad-Hoc NetworksAd-Hoc Networks

Ronald Pose Ronald Pose Carlo KoppCarlo Kopp

Muhammad Mahmudul IslamMuhammad Mahmudul Islam

School of Computer Science & Software EngineeringSchool of Computer Science & Software EngineeringMonash UniversityMonash University

OutlineOutline

Definition of the SAHNDefinition of the SAHN Required features & design steps for this systemRequired features & design steps for this system Survey of existing systemsSurvey of existing systems Possible hardware architecturePossible hardware architecture Required properties of RTOS for the SAHNRequired properties of RTOS for the SAHN Comparison of various RTOSsComparison of various RTOSs Initial development platform for the SAHNInitial development platform for the SAHN ReferencesReferences QuestionsQuestions

SAHN (1/3)SAHN (1/3)

Provides services not offered by commercial service providers• Bypass expensive infrastructure for broadbandBypass expensive infrastructure for broadband• Provide symmetric bandwidthProvide symmetric bandwidth• WLAN in inadequate wiring infrastructureWLAN in inadequate wiring infrastructure• Bypass ongoing service charges for Telco independent trafficBypass ongoing service charges for Telco independent traffic

Features multi-hop QoS routingFeatures multi-hop QoS routing• Security throughout all layersSecurity throughout all layers• Utilizing link states (e.g. available bandwidth, link stability, Utilizing link states (e.g. available bandwidth, link stability,

latency, jitter and security) to select suitable routeslatency, jitter and security) to select suitable routes• Avoid selfish routing strategy to avoid congestionAvoid selfish routing strategy to avoid congestion• Proper resource access control and managementProper resource access control and management


Ideal for cooperative nodes. Ideal for cooperative nodes. E.g. spread over a E.g. spread over a suburban area, connecting houses and businesssuburban area, connecting houses and business

Topology is quasi staticTopology is quasi static Uses wireless technologyUses wireless technology Symmetric broadband, multi Mbps bandwidthSymmetric broadband, multi Mbps bandwidth


No charges for SAHN trafficNo charges for SAHN traffic SAHN services SAHN services

run alongside run alongside

TCP/IPTCP/IP Conceived by Conceived by

Ronald Pose Ronald Pose

& &

Carlo Kopp 1997Carlo Kopp 1997

Application

Presentation

Session

Transport

Network

Data Link

Physical

TCP/UDP

IP

Application

Presentation

Session

Transport

Network

Data Link

Physical

TCP/UDP

IP

SAHN Routing

e.g. IEEE 802.11 variants

e.g. IEEE 802.11 variants

AUDIO

VEDIO

OTHER

Required Features for this SystemRequired Features for this System

Link rate throughputLink rate throughput ScalabilityScalability Low development costLow development cost

SAHN routing system should have the following SAHN routing system should have the following features to avoid system bottleneckfeatures to avoid system bottleneck

Design StepsDesign Steps

Identify time critical & non-time critical routing tasksIdentify time critical & non-time critical routing tasks Select appropriate interconnection fabricSelect appropriate interconnection fabric Design parallel processing engines for time critical Design parallel processing engines for time critical

taskstasks Select a real-time operating system which canSelect a real-time operating system which can

• manage the whole system efficientlymanage the whole system efficiently

• operate with small amount of memory and disk spaceoperate with small amount of memory and disk space

A scalable system achieving link rate throughput A scalable system achieving link rate throughput can be designed by the following stepscan be designed by the following steps

Routing Tasks in a SAHN NodeRouting Tasks in a SAHN Node

-Header separation from rest of the packet-Packet classification-Header decryption-Packet validation-Route cache lookup-Header & Checksum update-Encrypt header for the next hop-Prepend header with the rest of the packet-Switch packet to the outbound interface

-Route discovery with SAHN routing protocol-Route table/cache update-Load balancing-Error control-Switch control

Non TimeCritical

Processing(Slow Path)

TimeCritical

Processing(Fast Path)

..................................

..................................

..................................

..................................

..................................

..................................

..................................

..................................

..................................

switch fabric

switch fabric

Various Router ArchitecturesVarious Router Architectures

1stgeneration

router(Bus-based

backplane withSingle Processor)

2ndgeneration

router(Bus-based

backplane withMultiple Processors)

3rdgeneration

router(Switch-based

backplane with fullydistributed Processors)

4thgeneration

router(Switch fabric is

optical, multiclustered& multistaged)

Various router architectures

First Generation RouterFirst Generation Router

ConfigurationConfiguration a shared backplane busa shared backplane bus a CPUa CPU a shared pool of memorya shared pool of memory some line cards connected to the media. some line cards connected to the media. AdvantageAdvantage simple to implementsimple to implementDisadvantageDisadvantage data has to cross the shared bus several times, data has to cross the shared bus several times,

imposing a severe system bottleneck.imposing a severe system bottleneck.

Second Generation RouterSecond Generation Router

ConfigurationConfiguration both line cards and packet forwarding engines both line cards and packet forwarding engines

had fast processors with cache memorieshad fast processors with cache memories a shared backplane busa shared backplane busAdvantageAdvantage less load on shared busless load on shared busDisadvantageDisadvantage bus based architectures have traffic dependantbus based architectures have traffic dependant

throughputthroughput

Third Generation RouterThird Generation RouterConfigurationConfiguration both line cards and packet forwarding engines both line cards and packet forwarding engines

have fast processors with cache memorieshave fast processors with cache memories switch fabric instead of a shared busswitch fabric instead of a shared busAdvantageAdvantage switch fabric provides non-blocking switch fabric provides non-blocking

interconnection of time critical components interconnection of time critical components with much higher capacity and speed than a with much higher capacity and speed than a traditional backplane bustraditional backplane bus

DisadvantageDisadvantage costlycostly complicated designcomplicated design

Various Switch Fabrics (1/4)Various Switch Fabrics (1/4)

Shared medium switch fabricShared medium switch fabric like bus based backplanes of first generation routerslike bus based backplanes of first generation routers imposes bottleneck for inter-module traffic flowimposes bottleneck for inter-module traffic flow

Shared memory switch fabricShared memory switch fabric connects input and output ports to a central memory connects input and output ports to a central memory

pool in parallelpool in parallel input and output ports can have simultaneous read input and output ports can have simultaneous read

and write accesses to the shared memory pooland write accesses to the shared memory pool better throughput than a shared busbetter throughput than a shared bus performance limited by memory access timeperformance limited by memory access time


CPU, Central Buffer& Controller

Shar

ed B

US

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Ingr

ess

Port

s

Egr

ess

Port

s

Shar

ed M

emor

y


Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Ingr

ess

Port

s

Egr

ess

Port

s

Router designed with Shared BusSwitch fabric

Speed factorFor N input ports, N output ports and a port speed ofS pkt/sec, the shared bus should have speed equal toor greater than NS pkt/sec.

The router throughput depends upon the bus speed

Router designed with SharedMemory Switch fabric

Speed factorFor N input ports, N output ports and a port speed ofS pkt/sec, the shared memory should have accesstime less than or equal to 1/NS sec.

The router throughput depends upon memory speed

Various Switch Fabrics (3/4)Various Switch Fabrics (3/4)Distributed output buffered switch fabricDistributed output buffered switch fabric splits the shared memory into separate output splits the shared memory into separate output

buffers/output portbuffers/output port input ports have separate connections to their output input ports have separate connections to their output

buffersbuffers systems with large port counts require more memory & systems with large port counts require more memory &

complicated backplane layoutcomplicated backplane layoutSpace division/crossbar switch fabric with input buffersSpace division/crossbar switch fabric with input buffers all input and output ports are interconnected for unicast all input and output ports are interconnected for unicast

inter-port traffic flowinter-port traffic flow all input ports have their own buffersall input ports have their own buffers the speed of memory buffer need not to be more than the speed of memory buffer need not to be more than

that of its associated portthat of its associated port



Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Ingr

ess

Port

s

Egr

ess

Port

s


Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Line cardwith CPU

and Buffers

Ingr

ess

Port

s

Egr

ess

Port

s

Router designed with OutputBuffered Switch fabric

Speed factorFor N input ports, N output ports there are N2

independent paths between inputs and outputs anda total of N2 output buffers. The output buffersneed to operate at only port speed S pkt/sec.

The router throughput depends uponaccommodating memory size with quadratic N2

growth of interconnections.

Router designed with CrossbarSwitch fabric

Speed factorFor N input ports, N output ports there are N2

independent paths between inputs andoutputs. The input buffers need to operate atonly port speed S pkt/sec.

The router throughput can be scaledaccording to memory speed only as there isonly N input buffers instead of N2.

A Generic High-Speed Router ArchitectureA Generic High-Speed Router Architecture

Line Card

Line Card

Switch Fabric

ForwardingEngine

ForwardingEngine

Network Processorwith switchcontroller

Possible Router Architecture for SAHN (1/2)Possible Router Architecture for SAHN (1/2)

The SAHN router follows a hybrid approachThe SAHN router follows a hybrid approach A packet processing engine (PPE) is connected A packet processing engine (PPE) is connected

with each line cardwith each line card A central routing processing engine (RPE) A central routing processing engine (RPE)

performs the non-time critical tasks. performs the non-time critical tasks. The PPE forms the packet forwarding engine The PPE forms the packet forwarding engine

(PFE)(PFE) The PFEs are connected to each other through The PFEs are connected to each other through

a suitable switch fabrica suitable switch fabric

Possible Router Architecture for SAHN (2/2)Possible Router Architecture for SAHN (2/2)

Packet

Sw

itch

Fab

ric

Interfacecard

Memory shared by Interface card and PPE. Thereceived packets are stored here.

Tertiarycache for

RouteCache

Primarycache

for PPEcode

PPE Processor

Packet Forwarding Engine(PFE)

Encryption/Decryption Engine

Packet ProcessingEngine

HeaderInformation

Packet

Packet

FIFO

Packet RoutingEngine (PRE)

Main system memory.The complete routingtable is stored here.

Interfacecard

Memory shared by Interface card and PPE. Thereceived packets are stored here.

Tertiarycache for

RouteCache

Primarycache

for PPEcode

PPE Processor

Packet Forwarding Engine(PFE)

Encryption/Decryption Engine

Packet ProcessingEngine

HeaderInformation

Packet

Packet

FIFO

HeaderInformation

FIFO FIFO

Tertiarycache for

Headers toprocess

Primarycache

for RPEcode

RPE Processor

Inb

ou

nd

PF

E

Ou

tbo

und

PF

E

Upd

ated

cac

heen

trie

s

Upd

ated

cac

heen

trie

s

Properties of the RTOSProperties of the RTOS in SAHN (1/2) in SAHN (1/2)

Higher-priority tasks must have higher Higher-priority tasks must have higher preferencepreference

There should be support for fixed-priority pre-There should be support for fixed-priority pre-emptive scheduling for all of its tasksemptive scheduling for all of its tasks

Interrupt latency & the context switching time Interrupt latency & the context switching time should be as small as possibleshould be as small as possible

It must be cheapIt must be cheap Source code should be available to resolve Source code should be available to resolve

problems with the application codeproblems with the application code It has to be highly portable to various processor It has to be highly portable to various processor

familiesfamilies

Properties of the RTOSProperties of the RTOS in SAHN (2/2) in SAHN (2/2)

It must support multiple processors It must support multiple processors simultaneouslysimultaneously

Its image should fit in a small ROM/Flash-diskIts image should fit in a small ROM/Flash-disk It should have a familiar development It should have a familiar development

environment, possibly POSIX compliantenvironment, possibly POSIX compliant

Comparison Among Various RTOSs (1/3)Comparison Among Various RTOSs (1/3)

VxWorks ThreadX C Executive QNX Neutrino

RTLinux LynxOS Embedix RTAI

Target Supported

PowerPC, Coldfire, 68K, Intel Architecture, Intel StrongARM and XScale, ARM, SuperH, MIPS

PowerPC, Coldfire 68K; MCORE, ARM7, ARM9, ARM/Thumb, StrongARM, SH, TriCore, XScale, StarCore, ZSP, i960, V8xx, MIPS

PowerPC, 29K, 68K, ColdFire, i960, MIPS, SH, SPARC, V800, MIPS

PowerPC, x86, 175PowerPC0A, MIPS

PowerPC, x86, Alpha6, MIPS

PowerPC, PowerQUICC, PowerQUICC II, Intel Architecture, MIPS

PowerPC, ColdFire, Dragonball, ARM 7 & 9, StrongARM, SuperH , x86/IA32, MIPS

PowerPC, x86 (with and without FPU and TSC), ARM (StrongARM; ARM7: clps711x-family, Cirrus Logic EP7xxx, CS89712), MIPS

Development Host

Self-Hosted Self-Hosted UNIX, Solaris, Windows

Self-Hosted, Linux, Solaris, Windows, QNX4

Linux Solaris, SunOS, RS6000, LynxOS

Linux, Self-Hosted

Linux

Languages Supported

C, C++ C, C++ C, C++, Assembly

C, C++, Assembly, Java

C, C++ C,C++, Ada, Pascal, Java, Modula-2

C, C++ C, C++, PERL

Comparison Among Various RTOSs (2/3)Comparison Among Various RTOSs (2/3)VxWorks ThreadX C Executive QNX

NeutrinoRTLinux LynxOS Embedix RTAI

Min ROM/RAM required (KB)

15/5 2/1 5/1 64/varies 1500/4000 37/11 10/10 2000/2000

Typical context switch time/Interrupt Latency(µs)

10 1.7(40MHz ARM7)/0.5(200MHz PowerPC)

3/2 (100MHz)

1.95/4.3(Pentium 133),2.6/4.4(Pentium 100)

<30 (Interrupt Latency on 486/33MHz PC)

4 -19/14 7/15 4/20

Multitasking Strategy

Round-Robin, Time slice, Tasks can dynamically alter priorities, Rate monotonic Scheduling

Time slice, Fixed priority, Tasks can dynamically alter priorities

Time slice, Fixed priority, Tasks can dynamically alter priorities

Round-Robin, Time slice, Fixed Priority

One-shot, Periodic, FIFO, Rate monotonic

Round Robin, Time slice, Fixed priority, Tasks can dynamically alter priorities, Rate monotonic

One-shot, Periodic

Multiprocessor Support

Yes No No Yes Yes Yes Yes Yes

Source Code Included

No Yes No No Yes* No Yes* Yes*

Base price (USD)

$3000-$4000

$7500+ (Royalty Free)

$2500 (Royalty Free)

$3,995 (run time $50/ seat)

Free. (Royalty free).

$149. (Royalty Free)

Free*

*as per GNU Public Licensing agreement.

Comparison Among Various RTOSs (3/3)Comparison Among Various RTOSs (3/3)

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

5

4

3

2

1

QNX

VxWorks

Embedded NT

RTLinux

Pric

e

Rel

iabi

lity

Tim

ing

Des

ign

Past

Exp

erie

nce

OS

Age

Ease

of D

evel

opm

ent

Ove

rall

Initial Development PlatformInitial Development Platform

RTOSs providing more scalable properties tend to RTOSs providing more scalable properties tend to be far more expensive in terms of both upfront be far more expensive in terms of both upfront costs and recurring royalty/ licensing feescosts and recurring royalty/ licensing fees

some of them are not provided with source codes some of them are not provided with source codes RTLinux or RTAI are free under GNU Public RTLinux or RTAI are free under GNU Public

Licensing agreementLicensing agreement Some performance evaluation of ad-hoc routing Some performance evaluation of ad-hoc routing

protocols have been done with Linux systemprotocols have been done with Linux system

Initially we have decided to work with RTLinux or Initially we have decided to work with RTLinux or RTAI becauseRTAI because

ReferencesReferences• R. Pose and C. Kopp. Bypassing the Home Computing Bottleneck: The

Suburban Area Network. 3rd Australasian Comp. Architecture Conf.

(ACAC). February, 1998. pp.87-100.• A. Bickerstaffe, E. Makalic and S. Garic. CS honours theses. Monash

University. www.csse.monash.edu.au/\~rdp/SAN/. 2001• P. Misra. Routing Protocols for Ad Hoc Mobile Networks.

www.cis.ohio-state.edu/~jain/cis788-99/adhoc_routing/index.html. 02/07/2000

• Aweya James. IP Router Architectures: An Overview. Nortel Networks.

Ottawa, Canada, K1Y 4H7.

http://www.owlnet.rice.edu/\elec696/papers/weya99.pdf, 05/01/2003.• P. Newman, G. Minshall and L. Huston. IP switching and gigabit routers.

IEEE Communications Magazine. January, 1997.• Sayrafian Kamran. Overview of Switch Fabric Architectures,

http://www.zagrosnetworks.com. July, 2002.• A New Architecture for Switch and Router Design.

http://www.pmc-sierra.com/pressRoom/pdf/lcs\wp.pdf. 05/01/2003

Thank YouThank You

??

a router architecture to achieve link rate throughput in suburban ad-hoc networks ronald pose carlo...

Documents