patterns and performance of real-time high …schmidt/pdf/tao4.pdf · patterns and performance of...

Patterns and Performance of Real-timeObject Request Brokers

Douglas C. SchmidtAssociate Professor Elec. & Comp. Eng. [email protected] University of California, Irvinewww.ece.uci.edu/�schmidt/ (949) 824-1901

SponsorsNSF, DARPA, ATD, BBN, Boeing, Cisco, Comverse, GDIS, Experian, Global MT,

Hughes, Kodak, Krones, Lockheed, Lucent, Microsoft, Mitre, Motorola, NASA, Nokia,Nortel, OCI, Oresis, OTI, Raytheon, SAIC, Siemens SCR, Siemens MED, Siemens

ZT, Sprint, Telcordia, USENIX

High-performance, Real-time ORBs Douglas C. Schmidt

Motivation: the QoS-enabled Software Crisis

www.arl.wustl.edu/arl/� Symptoms

– Communication hardware getssmaller, faster, cheaper

– Communication software gets larger,slower, more expensive

� Culprits

– Inherent and accidental complexity

� Solution Approach

– Standards-based COTS Hardware &Software

UC Irvine 1


Problem: the COTS Hardware & Software Crisis

MISSIONMISSION--CRITICALCRITICAL

REALREAL--TIMETIMESYSTEMSSYSTEMS

COTSCOTSHARDWAREHARDWARE & &

SOFTWARESOFTWARE

� Context

– Adopting COTS hardware &software is increasingly essential forreal-time mission-critical systems

� Problems

– Inherent and accidental complexity– Integration woes


– Standards-based adaptive COTSmiddleware

UC Irvine 2


Context: Levels of Abstraction inInternetworking and Middleware

HTTPFTP

FDDIATMETHERNET

IP

FIBRE CHANNEL

UDP TCP

TELNET

DNS

LYNXOSLINUXWIN NT

CORBA

SOLARIS

CORBA SERVICES

CORBA

APPLICATIONS

VXWORKS

MIDDLEWARE ARCHINTERNETWORKING ARCH

RTP

UC Irvine 3


Problem: Lack of QoS-enabled Middleware

INFRASTRUCTUREMIDDLEWARE

DISTRIBUTIONMIDDLEWARE

COMMONMIDDLEWARE

SERVICES

APPLICATIONS

Cons

ConsConsEVENTEVENT

CHANNELCHANNEL

OPERATINGOPERATINGSYSTEMS SYSTEMS &&PROTOCOLSPROTOCOLS

HARDWAREHARDWARE DEVICESDEVICES

� Many applications require QoSguarantees

– e.g., avionics, telecom, WWW,medical, high-energy physics

� Building these applicationsmanually is hard and inefficient

� Existing middleware doesn’tsupport QoS effectively

– e.g., CORBA, DCOM, DCE, Java

� Solutions must be integratedhorizontally & vertically

UC Irvine 4


Candidate Solution: CORBA

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

DII ORBINTERFACE

ORBORB CORECORE GIOPGIOP//IIOPIIOP//ESIOPSESIOPS

IDLIDLSTUBSSTUBS

operation()operation()in args

out args + return value

CLIENTOBJECT(SERVANT)

OBJ

REF

STANDARD INTERFACE STANDARD LANGUAGE MAPPING

ORB-SPECIFIC INTERFACE STANDARD PROTOCOL

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

IDLSKELETON

DSI

OBJECT

ADAPTER

www.cs.wustl.edu/�schmidt/corba.html

Goals of CORBA

� Simplify distributionby automating

– Object location &activation

– Parametermarshaling

– Demultiplexing– Error handling

� Provide foundationfor higher-levelservices

UC Irvine 5

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Cav

eat:

Req

uire

men

ts/L

imita

tions

ofC

OR

BA

for

QoS

-ena

bled

Sys

tem

s

NE

TW

OR

KN

ET

WO

RK

OP

ER

AT

ION

SO

PE

RA

TIO

NS

CE

NT

ER

CE

NT

ER

HSM

HSM

AR

CH

IVE

AR

CH

IVE

SE

RV

ER

SE

RV

ER

AG

EN

TA

GE

NT

INT

ER

AC

TIV

EIN

TE

RA

CT

IVE

AU

DIO

AU

DIO

//VID

EO

VID

EO

AG

EN

TA

GE

NT

AR

CH

ITE

CT

UR

EA

RC

HIT

EC

TU

RE

SP

CS

PC

HA

RD

WA

RE

HA

RD

WA

RE

EM

BE

DD

ED

EM

BE

DD

ED

TA

OT

AO

MIB

MIB

AG

EN

TA

GE

NT

ww

w.c

s.w

ustl.

edu/

�

schm

idt/R

T-O

RB

.ps.

gz

Req

uire

men

ts

�

Loca

tion

tran

spar

ency

�

Per

form

ance

tran

spar

ency

�

Pre

dict

abili

tytr

ansp

aren

cy

�

Rel

iabi

lity

tran

pare

ncy

Lim

itatio

ns

�

Lack

ofQ

oSsp

ecifi

catio

ns

�

Lack

ofQ

oSen

forc

emen

t

�

Lack

ofre

al-t

ime

prog

ram

min

gfe

atur

es

�

Lack

ofpe

rform

ance

optim

izat

ions

UC

Irvi

neHigh-performance, Real-time ORBs Douglas C. Schmidt

Overview of the Real-time CORBA Specification

OS KERNEL

OS I/O SUBSYSTEM

NETWORK ADAPTERS

STANDARD

SYNCHRONIZERS

END-TO-END PRIORITY

PROPAGATION

ORB CORE

OBJECT ADAPTER

CLIENT

GIOP

PROTOCOL

PROPERTIES

THREAD

POOLSEXPLICIT

BINDING

NETWORK

OS KERNEL

OS I/O SUBSYSTEM

NETWORK ADAPTERS

operation()


in args

OBJECT

REF

OBJECT

(SERVANT)

STUBSSKELETON

Features

1. Portable priorities

2. End-to-end prioritypropagation

3. Protocol properties

4. Thread pools andbuffering

5. Explicit binding

6. Standard synchronizers

www.cs.wustl.edu/�schmidt/oorc.ps.gz

UC Irvine 7

High-perform

ance,R

eal-time

OR

Bs

Douglas

C.S

chmidt

Portable

Priorities

OR

B E

ND

SY

ST

EM

A

32

76

7

0 RTCORBA::Priority

0

255

OR

B E

ND

SY

ST

EM

B

0

31

Features

�

Designed

tosupport

heterogeneousreal-tim

eplatform

s

�

CO

RB

Apriorities

rangefrom

0

!

32767

�

Users

canm

apC

OR

BA

prioritiesto

nativeO

Spriorities

�

No

silverbullet,butrather

an“enabling

technique”

UC

Irvine8

High-performance, Real-time ORBs Do

End-to-End Priority Propagation

GLOBAL CORBA PRIORITY = 100

LYNXOS

PRIORITY

= 100

WINNT

PRIORITY

= 5

SOLARIS

PRIORITY

= 135

Current::priority(100)

to_native(100) => 100


to_native(100) => 5


to_native(100) => 135

SERVICECONTEXT

= 100ORBENDSYSTEM

A

ORBENDSYSTEM

B

ORBENDSYSTEM

C

(B) CLIENT

PROPAGATED

MODEL

SERVICECONTEXT

= 100

(A) SERVER

DECLARED

MODEL

ORBENDSYSTEM

A(3) CLIENT'S PRIORITY

IS NOT PROPAGATED

BY INVOCATION

(1) SERVER

PRIORITY

IS PRE-SET

ORBENDSYSTEM

B

(2) PRIORITY IS

EXPORTED IN IOR

Features

� Client priorities can propagate end-to-end

� Servers can also declare priority

UC Irvine

High-perform

ance,R

eal-time

OR

Bs

Douglas

C.S

chmidt

Configurable

ProtocolP

roperties

inte

rface

Pro

toco

lPro

pe

rties

{};

type

de

fstru

ct{

IOP

::Pro

fileId

pro

toco

l_typ

e;

Pro

toco

lPro

pe

rties

orb

_p

roto

col_

pro

pe

rties;

Pro

toco

lPro

pe

rties

tran

spo

rt_p

roto

col_

pro

pertie

s;}

Pro

toco

l;typ

ed

ef

seq

ue

nce

<P

roto

col>

Pro

toco

lList;

inte

rface

TC

PP

roto

colP

rop

ertie

s:

Pro

toco

lPro

pe

rties

{a

ttribu

telo

ng

sen

d_

bu

ffer_

size;

attrib

ute

lon

gre

cv_b

uffe

r_size

;a

ttribu

teb

oo

lea

nke

ep

_a

live;

attrib

ute

bo

ole

an

do

nt_

rou

te;

attrib

ute

bo

ole

an

no

_d

ela

y;};

Features

�

Selectand

configurecom

munication

protocols

–e.g.,T

CP

socketoptions

�

Supports

OR

Bprotocol

andtransportprotocol

configuration

�

Ordering

inP

roto

colL

istindicates

preferences

UC

Irvine10


Protocol Selection and Configuration

NETWORKOS KERNEL

ATMVME RTP

OS KERNEL

ORBCORE


2: PASS

OBJECT

REFERENCE

OBJECT ADAPTER

ATMVME TCP

4: INVOKE OPERATION 1: CREATE OBJECT

REFERENCE3: SELECT

PROTOCOL

� Protocol policies control protocol selection andconfiguration

– Order of protocols indicates protocolpreference

� Both server-side and client-side policiessupported

– Some policies control protocol selection,others control protocol configuration

– Some policies are exported to client in objectreference

UC Irvine


Thread Pools

SERVER ORB COREI/OTHREADS

Root POA

Thread Pool A

PRIORITY

10PRIORITY

35PRIORITY

50PRIORITY

20

Thread Pool B

DEFAULT

PRIORITY

DefaultThread Pool

S3DEFAULT

S1DEFAULT

S2DEFAULT

POA A

S410

S550

S650

S735 POA B

S8 S9 S10

POA C

S1120

S1225

S1315

� Pre-allocate threads and thread attributes

– Stacksize– Static threads and maximum threads– Default priority

UC Irvine


Buffering Requests

SERVER ORB COREI/OTHREADS

Thead Pool A

PRIORITY 10 PRIORITY 35 PRIORITY 20

Thead Pool B

Requests are bufferedwhen all threads arebusyBuffering can bespecified in terms of:

� Number of bytes

� Number of requests

When buffers are full:

� A transient exception isthrown to client

� Request is dropped byserver

� Request can bereissued later by client

UC Irvine


Explicit Binding

_validate_connection (out CORBA::PolicyListinconsistent_policies);

CLIENTORB CORE

P1-5 P10-20 P21-100

SERVERORB CORE

PRIORITY-BANDEDPRIVATE CONNECTIONS

P1-5 P10-20 P21-100

Features

� Enables pre-establishment of connections

– Priority-banded connections– Private connections– Protocol policies

UC Irvine


Standard Synchronizers

CLIENT

Mutexlock()

unlock()

try_lock()

ORB CORE

OBJECT

ADAPTER

OBJECT(SERVANT)

mutex3

mutex2

mutex4

mutex1

Features

� A portable Mutex API

– e.g., lock , unlock , try lock

� Necessary to ensure consistency between ORBand application synchronizers

– e.g., priority inheritance and priority ceilingprotocols

� Locality constrained

UC Irvine


Additional Information on Real-time CORBA

� Real-time CORBA 1.0 specification

– www.cs.wustl.edu/�schmidt/RT-ORB-std-new.pdf.gz

� Many papers at my Web site

– www.cs.wustl.edu/�schmidt/corba-research-realtime.html

� Upcoming OMG Real-time and Embedded CORBA Workshop

– www.omg.org/meetings/realtime/

� Real-time ORBs

– HighComm ! www.highcomm.com– ORB Express ! www.ois.com– TAO ! www.theaceorb.com

UC Irvine 16


Our Approach: The ACE ORB (TAO)

NETWORKNETWORK

ORBORB RUN RUN--TIMETIME

SCHEDULERSCHEDULER

operation()operation()

IDLIDLSTUBSSTUBS

IDLIDLSKELETONSKELETON

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK INTERFACENETWORK INTERFACE

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

OBJECTOBJECT((SERVANTSERVANT))

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK INTERFACENETWORK INTERFACE

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

ACEACE COMPONENTSCOMPONENTS

OBJOBJ

REFREF

REALREAL--TIMETIME ORBORB CORECOREIOPIOP

PLUGGABLEPLUGGABLEORBORB & & XPORTXPORTPROTOCOLSPROTOCOLS

IOPIOPPLUGGABLEPLUGGABLE

ORBORB & & XPORTXPORTPROTOCOLSPROTOCOLS

REALREAL--TIMETIME

OBJECTOBJECT

ADAPTERADAPTER

www.cs.wustl.edu/�schmidt/TAO.html

TAO Overview !

� An open-source,standards-based,real-time,high-performanceCORBA ORB

� Runs onPOSIX/UNIX,Win32, & RTOSplatforms– e.g., VxWorks,

Chorus, LynxOS

� Leverages ACE

UC Irvine 17


The ADAPTIVE Communication Environment (ACE)

PROCESSES//THREADSTHREADS

DYNAMICDYNAMIC

LINKINGLINKING

MEMORYMEMORY

MAPPINGMAPPING

SELECTSELECT//IO COMPIO COMP

SYSTEMSYSTEM

VV IPCIPCSTREAMSTREAM

PIPESPIPES

NAMEDNAMED

PIPESPIPES

CAPISS

SOCKETSSOCKETS//TLITLI

COMMUNICATIONCOMMUNICATION

SUBSYSTEMSUBSYSTEM

VIRTUAL MEMORYVIRTUAL MEMORY

SUBSYSTEMSUBSYSTEM

GENERAL POSIX AND WIN32 SERVICES

PROCESSPROCESS//THREADTHREAD

SUBSYSTEMSUBSYSTEM

FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR

SELF-CONTAINED

DISTRIBUTED

SERVICE

COMPONENTS

NAMENAME

SERVERSERVER

TOKENTOKEN

SERVERSERVER

LOGGINGLOGGING

SERVERSERVER

GATEWAYGATEWAY

SERVERSERVER

SOCKSOCK__SAPSAP//TLITLI__SAPSAP

FIFOFIFO

SAPSAP

LOGLOG

MSGMSG

SERVICESERVICE

HANDLERHANDLER

TIMETIME

SERVERSERVER

C++WRAPPER

FACADES

SPIPESPIPE

SAPSAP

CORBACORBA

HANDLERHANDLER

SYSVSYSVWRAPPERSWRAPPERS

SHAREDSHARED

MALLOCMALLOC

THE ACE ORBTHE ACE ORB

((TAOTAO))

JAWS ADAPTIVEJAWS ADAPTIVE

WEB SERVERWEB SERVER

MIDDLEWARE

APPLICATIONS

REACTORREACTOR//PROACTORPROACTOR


MANAGERSMANAGERS

STREAMSSTREAMS

SERVICESERVICE

CONFIGCONFIG--URATORURATOR

SYNCHSYNCH

WRAPPERSWRAPPERS

MEMMEM

MAPMAP

OS ADAPTATION LAYER

www.cs.wustl.edu/�schmidt/ACE.html

ACE Overview !

� A concurrentOO networkingframework

� Available inC++ and Java

� Ported toPOSIX, Win32,and RTOSs

Related work !

� x-Kernel

� SysVSTREAMS

UC Irvine 18


ACE and TAO Statistics� Over 50 person-years of effort

– ACE > 200,000 LOC– TAO > 200,000 LOC– TAO IDL compiler > 130,000

LOC– TAO CORBA Object Services >

150,000 LOC

� Ported to UNIX, Win32, MVS, andRTOS platforms

� Large user community

– �schmidt/ACE-users.html

� Currently used by dozens ofcompanies

– Bellcore, BBN, Boeing,Ericsson, Hughes, Kodak,Lockheed, Lucent,Motorola, Nokia, Nortel,Raytheon, SAIC, Siemens,etc.

� Supported commercially

– ACE ! www.riverace.com– TAO !

www.theaceorb.com

UC Irvine 19


Applying TAO to Avionics Mission Computing

REPLICATION

SERVICE

OBJECT REQUEST BROKER

1: SENSORS

GENERATE

DATA

FLIRGPS IFF

3:PUSH (EVENTS)

2: SENSOR PROXIES DEMARSHAL DATA

& PASS TO EVENT CHANNEL

3:PUSH (EVENTS)

EVENT

CHANNEL

HUD NavAir

FrameWTS

4: PULL(DATA)

www.cs.wustl.edu/�schmidt/JSAC-98.ps.gz

Domain Challenges

� Deterministic & statisticalreal-time deadlines

� Periodic & aperiodic processing

� COTS and open systems

� Reusable components

� Support platform upgrades

www.cs.wustl.edu/�schmidt/TAO-boeing.html

UC Irvine 20


Applying TAO to Distributed Interactive Simulations

www.cs.wustl.edu/�schmidt/Words99.ps.gz

Domain Challenges

� High scalability and groupcommunication

� High throughput and lowlatency

� “Interactive” real-time

� Multi-platform

hlasdc.dmso.mil/RTISUP/hla soft/hla soft.htm

UC Irvine 21

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

App

lyin

gTA

Oto

Em

bedd

edN

etw

ork

Ele

men

tMan

agem

enta

ndC

ontro

lM

AN

AG

EM

EN

T

CLI

EN

T

RIO

TA

ON

ET

WO

RK

O

PE

RA

TIO

NS

C

EN

TE

R

VS

I

MA

ST

ER

RIO

TA

OS

IGN

AL

ING

P

RO

CE

SS

OR

VS

I

MA

ST

ER

RIO

TA

OS

IGN

AL

ING

P

RO

CE

SS

OR

VS

I

SLA

VE

CO

NT

RO

L

PR

OC

ES

SO

R

TA

OR

IO

WU

GS

WU

GS

WU

GS

EN

DS

YS

TE

M A

UN

I

CL

IEN

T

TA

O

CO

NN

EC

T

RE

QU

ES

T

EN

DS

YS

TE

M A

UN

I

CL

IEN

TTA

O

AT

MS

WIT

CH

AT

MS

WIT

CH

AT

MS

WIT

CH

VS

I

SLA

VE

CO

NT

RO

L

PR

OC

ES

SO

R

TA

OR

IO

VS

I

SLA

VE

CO

NT

RO

L

PR

OC

ES

SO

R

TA

OR

IO

AD

D C

ON

NE

CT

ION GE

T S

TA

TS

EV

EN

T

PO

RT

DO

WN

AD

D C

ON

NE

CT

ION

GE

T S

TA

TS

AD

D C

ON

NE

CT

ION

CO

NN

EC

T

RE

QU

ES

T

INT

ER

-OR

B P

RO

TO

CO

L

Dom

ain

Cha

lleng

es

�

Hig

h-sp

eed

(20

Gbp

s)AT

Msw

itche

sw

/em

bedd

edco

ntro

llers

�

Low

-late

ncy

and

stat

istic

alre

al-t

ime

dead

lines

�

CO

TS

infr

astr

uctu

re,s

tand

ards

-bas

edop

ensy

stem

s,an

dsm

allf

ootp

rint

UC

Irvi


Optimization Challenges for QoS-enabled ORBs

ORBORBCORECORE

PLUGGABLEPLUGGABLEPROTOCOLSPROTOCOLS

PLUGGABLEPLUGGABLEPROTOCOLSPROTOCOLS

OS KERNELOS KERNEL

OS IOS I//O SUBSYSTEMO SUBSYSTEM

NETWORK INTERFACESNETWORK INTERFACES

OS KERNELOS KERNEL

OS IOS I//O SUBSYSTEMO SUBSYSTEM

NETWORK INTERFACESNETWORK INTERFACES

NETWORKNETWORK

ORBORBINTERFACEINTERFACE

operation()operation()

IDLIDLSTUBSSTUBS

OBJECTOBJECT

ADAPTERADAPTER


in argsin args


11

22

33

44

55

66

77

1)1) CLIENT MARSHALING CLIENT MARSHALING

2)2) CLIENT PROTOCOL CLIENT PROTOCOL

3)3) NETWORK LATENCY NETWORK LATENCY

4)4) SERVER PROTOCOL SERVER PROTOCOL

5) THREAD DISPATCHING

6) REQUEST DEMUXING

7) OPERATION DEMUXING

8) SERVANT DEMARSHALING


88

CLIENTCLIENT

IOPIOP IOPIOP

Key Challenges

� Alleviate priority inversionand non-determinism

� Reduce demultiplexinglatency/jitter

� Ensure protocol flexibility

� Specify QoS requirements

� Schedule operations

� Eliminate (de)marshalingoverhead

� Minimize footprint

UC Irvine 23


Problem: Optimizing Complex Software

DIAGNOSTIC STATIONSDIAGNOSTIC STATIONS

ATMATMMANMAN

ATM

LAN

ATM

LAN

MODALITIES

(CT, MR, CR) CENTRAL

BLOB STORE

CLUSTER

BLOB

STORE

DX

BLOB

STORE


Common Problems !

� Optimizing complex softwareis hard

� Small “mistakes” can be costly

Solution Approach (Iterative) !

� Pinpoint overhead viawhite-box metrics– e.g., Quantify and

VMEtro

� Apply patterns and frameworkcomponents

� Revalidate via white-box andblack-box metrics

UC Irvine 24


Solution 1: Patterns and Framework Components

ACCEPTORCONNECTOR

ABSTRACT

FACTORY

SERVANTCLIENT

OS KERNELOS KERNEL

ACTIVE

OBJECT

THREAD-SPECIFIC

STORAGE

SERVICE

CONFIGURATOR

STRATEGY

REACTORREACTOR

WRAPPER FACADESWRAPPER FACADES

www.cs.wustl.edu/�schmidt/ORB-patterns.ps.gz

Definitions

� Pattern

– A solution to a problem ina context

� Framework

– A “semi-complete”application built withcomponents

� Components

– Self-contained, “pluggable”ADTs

UC Irvine 25

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Sol

utio

n2:

OR

BO

ptim

izat

ion

Prin

cipl

eP

atte

rns

Defi

nitio

n

�

Opt

imiz

atio

npr

inci

ple

patte

rns

docu

men

tru

les

for

avoi

ding

com

mon

desi

gnan

dim

plem

enta

tion

prob

lem

sth

atca

nde

grad

eth

eef

ficie

ncy,

scal

abili

ty,a

ndpr

edic

tabi

lity

ofco

mpl

exsy

stem

s

Opt

imiz

atio

nP

rinci

ple

Pat

tern

sU

sed

inTA

O

#O

ptim

izat

ion

Prin

cipl

eP

atte

rn1

Opt

imiz

efo

rth

eco

mm

onca

se2

Rem

ove

grat

uito

usw

aste

3R

epla

cein

effic

ient

gene

ral-p

urpo

sefu

nctio

nsw

ithef

ficie

ntsp

ecia

l-pur

pose

ones

4S

hift

com

puta

tion

intim

e,e.

g.,p

reco

mpu

te5

Sto

rere

dund

ants

tate

tosp

eed-

upex

pens

ive

oper

atio

ns6

Pas

shi

nts

betw

een

laye

rsan

dco

mpo

nent

s7

Don

’tbe

tied

tore

fere

nce

impl

emen

tatio

ns/m

odel

s8

Use

effic

ient

/pre

dict

able

data

stru

ctur

es

UC

Irvi


ORB Latency and Priority Inversion Experiments

��

C 1C 0

Requests

C n

��

Client

...

...

2ATM Switch

Ultra 2 Ultra 2

I/O SUBSYSTEM

Server

Object Adapter

ORB Core

Servants RU

NT

IME

SCH

ED

UL

ER

www.cs.wustl.edu/�schmidt/RT-perf.ps.gz

Method

� Vary ORBs, hold OSconstant

� Solaris real-time threads

� High priority client C0

connects to servant S0

with matching priorities

� Clients C1 : : : Cn havesame lower priority

� Clients C1 : : : Cn

connect to servant S1

� Clients invoke two-wayCORBA calls that cube anumber on the servantand returns result

UC Irvine 27


ORB Latency and Priority Inversion Results

0

4

8

12

16

20

24

28

32

36

40

44

48

52

1 5 10 15 20 25 30 35 40 45 50

Number of Low Priority Clients

Late

ncy p

er

Tw

o-w

ay R

eq

uest

in M

illiseco

nd

s

CORBAplus High Priority CORBAplus Low Priority

MT-ORBIX High Priority MT-ORBIX Low Priority

miniCOOL High Priority miniCOOL Low Priority

TAO High Priority TAO Low Priority

Synopsis of Results

� TAO’s latency is lowest forlarge # of clients

� TAO avoids priority inversion

– i.e., high priority clientalways has lowest latency

� Primary overhead stemsfrom concurrency andconnection architecture

– e.g., synchronization andcontext switching

UC Irvine 28


ORB Jitter Results

1 5 10 15 20 25 30 35 40 45 50

TAO High Priority

TAO Low Priority

miniCOOL High Priority

miniCOOL Low Priority

MT-ORBIX High Priority

MT-ORBIX Low Priority

CORBAplus High Priority

CORBAplus Low Priority

0

50

100

150

200

250

300

Jitt

er in

mill

isec

on

ds

Number of Low Priority Clients

Definition

� Jitter ! standarddeviation fromaverage latency

Synopsis ofResults

� TAO’s jitter islowest and mostconsistent

� CORBAplus’ jitteris highest andmost variable

UC Irvine 29


Problem: Improper ORB Concurrency Models

OBJECTOBJECTADAPTERADAPTER

SERVANTSERVANT DEMUXERDEMUXER

SERVANTSERVANTSKELETONSSKELETONS

U

ORBORB CORECORE

FILTERFILTER

FILTERFILTER

FILTERFILTER

SERVANTSERVANTSKELETONSSKELETONS

U

2: enqueue(data)2: enqueue(data)

3: dequeue,3: dequeue,filterfilter

request,request,&enqueue&enqueue

4: dispatch4: dispatchupcall()upcall()

II//OO SUBSYSTEMSUBSYSTEM

THREADTHREAD

FILTERFILTER

1: recv()1: recv()

CONNECTION THREADSCONNECTION THREADS

Common Problems

� High context switching andsynchronization overhead

� Thread-level andpacket-level priorityinversions

� Lack of application controlover concurrency model

www.cs.wustl.edu/�schmidt/CACM-arch.ps.gz

UC Irvine 30


Problem: ORB Shared Connection Models

SERVERSERVERORBORB CORECORE



COMMUNICATIONCOMMUNICATION LINKLINK


SERVANTSSERVANTS

ONEONE TCPTCP

CONNECTIONCONNECTION

ORBORB CORECORE

SEMAPHORESSEMAPHORES

2: select()2: select()4: release()

3: read()B

OR

RO

WE

D T

HR

EA

D

BO

RR

OW

ED

TH

RE

AD

S

APPLICATION

5: dispatch_return()

1: invoke_twoway()

www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz

Common Problems

� Request-levelpriority inversions

– Sharing multiplepriorities on asingle connection

� Complex connectionmultiplexing

� Synchronizationoverhead

UC Irvine 31


Problem: High Locking Overhead

0

94

199

175

0

231216

599

0

100

200

300

400

500

600

700

TAO miniCOOL CORBAplus MT ORBIX

ORBs Tested

Use

r L

evel

Lo

ck O

per

atio

ns

per

Req

ues

t

client server

Common Problems

� Locking overhead affectslatency and jitter significantly

� Memory managementcommonly involves locking

www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz

UC Irvine 32


Solution: TAO’s ORB Endsystem Architecture

RR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR

HIGH-SPEED NETWORKINTERFACES(e.g., APIC, VME)

ZZ

EE

RR

OO

CC

OO

PP

YY

BB

UU

FF

FF

EE

RR

SSRTRT I/O I/OSUBSYSTEMSUBSYSTEM

RTRT OBJECTOBJECTADAPTERADAPTER

OO(1) (1) REQUESTREQUEST DEMUXERDEMUXER

CLIENTSCLIENTS

STUBSSTUBS

SERVANTSSERVANTS

SKELETONSSKELETONS

U

RTRT ORBORB CORECORE

REACTORREACTOR

((PP11))

REACTORREACTOR

((PP22))

REACTORREACTOR

((PP33))

REACTORREACTOR

((PP44))

SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER

PLUGGABLEPLUGGABLE PROTOCOLSPROTOCOLS

Solution Approach !

� Integrate scheduler into ORBendsystem

� Co-schedule threads

� Leader/followers thread pool

Principle Patterns !

� Pass hints, precompute,optimize common case,remove gratuitous waste,store state, don’t be tied toreference implementations &models

UC Irvine 33


Thread Pool Comparison Results

SERVANTSSERVANTS

ORBORB CORECORE

1: select()1: select()


5: dispatch upcall()5: dispatch upcall()

2: read()2: read()

3: enqueue()3: enqueue()

4: dequeue()4: dequeue()

II//OOTHREADTHREAD

WORKER THREADSWORKER THREADS

REQUESTREQUEST

QUEUEQUEUE

Worker Thread Pool

SERVANTSSERVANTS

ORBORB CORECORE

SEMAPHORESEMAPHORE

LEADERLEADER FOLLOWERSFOLLOWERS

1: select()1: select()3: release()3: release()


4: dispatch upcall()4: dispatch upcall()

2: read()2: read()

Leader/FollowerThread Pool

1 2 3 45

6

0

25

5075

100

0

500

1000

1500

2000

2500

3000

Pe

rfo

rma

nc

e Im

pro

ve

me

nt

Threads

Lo

ad

UC Irvine 34


Problem: Reducing Demultiplexing Latency

OP

ER

AT

ION1

OP

ER

AT

ION2

OP

ER

AT

IONK

2: DEMUX TO

I/O HANDLE

...

...

...POA1

OS I/O SUBSYSTEM

NETWORK ADAPTERS

SERVANT 1

5: DEMUX TO

SKELETON

6: DISPATCH

OPERATION

1: DEMUX THRU

PROTOCOL STACK

4: DEMUX TO

SERVANT

ORB COREORB CORE

ROOT POA3: DEMUX TO

OBJECT

ADAPTER

POA2 POAN

SERVANT N

...SKEL 1 SKEL 2 SKEL N

SERVANT 2

OS

KERNEL

LAYER

ORB

LAYER

SERVANT

LAYER

Design Challenges

� Minimize demuxing layers

� Provide O(1) operationdemuxing through all layers

� Avoid priority inversions

� Remain CORBA-compliant

www.cs.wustl.edu/�schmidt/POA.ps.gz

UC Irvine 35

High-perform

ance,R

eal-time

OR

Bs

Douglas

C.S

chmidt

Solution:

TAO

’sR

equestDem

ultiplexingO

ptimizations

.........

.........

IDL

IDL

SK

EL

S

KE

L 22

OB

JEC

T A

DA

PT

ER S

ER

VA

NT

S

ER

VA

NT

500

500

(A)

(A) L

AY

ER

ED

DE

MU

XIN

G L

AY

ER

ED

DE

MU

XIN

G,,

LIN

EA

R S

EA

RC

HL

INE

AR

SE

AR

CH

SE

RV

AN

T

SE

RV

AN

T 11

SE

RV

AN

T

SE

RV

AN

T 22

IDL

IDL

SK

EL

S

KE

L 11

OPERATIONOPERATION100100

OPERATIONOPERATION22.........


IDL

IDL

SK

EL

NS

KE

L N

............

.........

SERVANTSERVANT500::500::OPERATIONOPERATION100100





OB

JEC

T A

DA

PT

ER

.........

.........

IDL

IDL

SK

EL

S

KE

L 22

OB

JEC

T A

DA

PT

ER

(C) L

AY

ER

ED

DE

MU

XIN

G,

PE

RF

EC

T H

AS

HIN

G

SE

RV

AN

T

SE

RV

AN

T 11

SE

RV

AN

T

SE

RV

AN

T 22


OPERATIONOPERATION22.........


IDL

IDL

SK

EL

NS

KE

L N

search(o

bject k

ey)

hash

(operatio

n)

IDL

IDL

SK

EL

S

KE

L 11

(D) D

E-L

AY

ER

ED

AC

TIV

E D

EM

UX

ING

hash

(object k

ey)

search(o

peratio

n)

index

(object k

ey/o

peratio

n)

(B) L

AY

ER

ED

DE

MU

XIN

G,

DY

NA

MIC

HA

SH

ING

SE

RV

AN

T

SE

RV

AN

T 5

00

500

Dem

uxing

�

ww

w.cs.w

ustl.edu/�

schmidt/

fieeetc-97,C

OO

TS

-99

g.ps.gz

Perfecthashing

�

ww

w.cs.w

ustl.edu/�

schmidt/

gperf.ps.gz

UC

Irvine36


POA Demultiplexing Results

15

1015

2025

012345

Lat

ency

(us)

POA Depth

TransientPersistent

Synopsis of Results

� Active demux isefficient & predictablefor both transient andpersistent objectreferences.

Principle Patterns

� Precompute, passhints, usespecial-purpose &predictable datastructures, ignore refmodels

UC Irvine


Servant Demultiplexing Results

1 100 200 300 400 500

0

50

100

150

200

Lat

ency

(u

s)

No. of Objects

Active Demux

Dynamic Demux

Linear Demux

Synopsis of Results

� Linear demux iscostly

� Active demux ismost efficient &predictable

Principle Patterns

� Precompute, passhints, usespecial-purpose &predictable datastructures

UC IrvineH

igh-performance,

Real-tim

eO

RB

sD

ouglasC

.Schm

idt

Operation

Dem

ultiplexingR

esults

110

2030

4050

0 5

10

15

20

25

Latency (us)

No

. of M

etho

ds

Perfect H

ashing

Binary S

earch

Dynam

ic Hashing

Linear Search

Synopsis

ofResults

!

�

PerfectH

ashing–

Highly

predictable–

Low-latency

�

Others

strategiesslow

er

Principle

Patterns

!

�

Precom

pute,usepredictable

datastructures,rem

ovegratuitous

waste

UC

Irvine39

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

TAO

Req

uest

Dem

ultip

lexi

ngS

umm

ary

...

...

...

...

...

...

PO

AP

OA

11

SE

RV

AN

T

SE

RV

AN

T 11 O

RB

C

OR

EO

RB

C

OR

E

RO

OT

P

OA

AC

TIV

E

DE

MU

XIN

G

PO

A2

PO

AN

SE

RV

AN

T N

...

SK

EL

1S

KE

L 2

SK

EL

N

SE

RV

AN

T 2

AC

TIV

E

DE

MU

XIN

G

PE

RF

EC

TH

AS

HIN

G

Dem

ultip

lexi

ngS

tage

Abs

olut

eT

ime

(

�

s)1.

Req

uest

pars

ing

22.

PO

Ade

mux

23.

Ser

vant

dem

ux3

4.O

pera

tion

dem

ux2

5.P

aram

eter

dem

arsh

alin

gop

erat

ion

depe

nden

t6.

Use

rup

call

serv

antd

epen

dent

7.R

esul

tsm

arsh

alin

gop

erat

ion

depe

nden

t

UC

Irvi

ne


Real-time ORB/OS Performance Experiments

[P ]0

[P ]1

SCH

ED

UL

ER

0

RU

NT

IME

[P ]

I/O SUBSYSTEM

Server

0 nS1

nC

��Pentium II

S S0C 1C

...

...

Object AdapterServants

ORB Core

Client

...

[P]

Requests

Priority

...

[P ][P ]1

[P ]

n

n

www.cs.wustl.edu/�schmidt/RT-OS.ps.gz

Method

� Vary OS, hold ORBsconstant

� Single-processor IntelPentium II 450 Mhz, 256Mbytes of RAM

� Client and servant run onthe same machine

� Client Ci connects toservant Si with priority Pi

– i ranges from 1 : : : 50

� Clients invoke two-wayCORBA calls that cube anumber on the servant andreturns result

UC Irvine 41


Real-time ORB/OS Performance Results'&

$%

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 1 2 5 10 15 20 25 30 35 40 45 50

Low Priority Clients

Tw

o-w

ay R

equ

est

Lat

ency

, use

c

Linux

LynxOS

NT

Solaris

VxWorks

'&

$%

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

0 1 2 5 10 15 20 25 30 35 40 45 50


Tw

o-w

ay R

equ

est

Lat

ency

, use

c

Linux

LynxOS

NT

Solaris

VxWorks

High-priority Client Latency Low-priority Clients Latency

UC Irvine 42


Real-time ORB/OS Jitter Results'&

$%

0

2

10

20

30

40

50

0

500

1000

1500

2000

2500

3000

3500

Tw

o-w

ay J

itte

r, u

sec


LynxOSNTVxWorksLinuxSolaris

'&

$%

0 2

10

20

30

40

50

0

5000

10000

15000

20000

25000

30000

35000

40000

Tw

o-w

ay J

itte

r, u

sec


LynxOSLinuxSolarisNTVxWorks

High-priority Client Jitter Low-priority Clients Jitter

UC Irvine 43

High-perform

ance,R

eal-time

OR

Bs

Douglas

C.S

chmidt

Problem

:H

ard-codedO

RB

Messaging

andTransportP

rotocolsB

OA

RD

B

OA

RD

11

VM

EV

ME

15

53

15

53

BO

AR

D

BO

AR

D 22

�

GIO

P/IIO

Pare

notsufficient,e.g.:

–G

IOP

message

footprintmay

betoo

large–

TC

Placks

necessaryQ

oS–

Legacycom

mitm

entsto

existingprotocols

�

Many

OR

Bs

donotsupport

“pluggableprotocols”

–T

hism

akesO

RB

sinflexible

andinefficient

UC

Irvine44


One Solution: Hacking GIOP

� GIOP requests include fields that aren’t neededin homogeneous embedded applications

– e.g., GIOP magic #, GIOP version, byte order,request principal, etc.

� These fields can be omitted without any changesto the standard CORBA programming model

� TAO’s -ORBgioplite option save 15 bytesper-request, yielding these calls-per-second:

Marshaling-enabled Marshaling-disabledmin max avg min max avg

GIOP 2,878 2,937 2,906 2,912 2,976 2,949GIOPlite 2,883 2,978 2,943 2,911 3,003 2,967

� The result is a measurable improvement inthroughput/latency

– However, it’s so small (2%) that hacking GIOPis of minimal gain except for low-bandwidthlinks

UC Irvine


Better Solution: TAO’sPluggable Protocols Framework

CLIENT

STUBS SKELETONS

TCP

MULTICAST

IOP

VMEUDP

ORB MESSAGING COMPONENT

ORB TRANSPORT ADAPTER COMPONENT

ESIOP

REAL -TIME

IOPEMBEDDED

IOP

RELIABLE,BYTE-STREAM

ATMTCP

MEMORY

MANAGEMENT

CONCURRENCY

MODEL

OTHER

ORBCORE

SERVICES

COMMUNICATION INFRASTRUCTUREHIGH SPEED NETWORK INTERFACE

REAL -TIME I /O SUBSYSTEM

ORB MESSAGE

FACTORY

ORB TRANSPORT

ADAPTER FACTORY

OBJECT ADAPTER

GIOP GIOPLITE

ADAPTIVE Communication Environment (ACE)

OBJECT (SERVANT)operation (args)IN ARGS

OUT ARGS & RETURN VALUE

POLICY

CONTROL

Features

� Pluggable ORBmessaging andtransport protocols

� Highly efficient andpredictable behavior

Principle Patterns

� Replacegeneral-purposefunctions(protocols) withspecial-purposeones

UC IrvineH

igh-performance,

Real-tim

eO

RB

sD

ouglasC

.Schm

idt

CO

RB

AP

rotocolInteroperabilityA

rchitecture

OR

B M

ES

SA

GIN

G

CO

MP

ON

EN

T

OR

B T

RA

NS

PO

RT

AD

AP

TE

R C

OM

PO

NE

NT

TR

AN

SP

OR

T L

AY

ER

NE

TW

OR

K L

AY

ER

GIO

P

IIOP

TC

P

IP

VM

E

DR

IVE

R

AA

L5

AT

M

GIO

PL

ITE

VM

E-IO

P

ES

IOP

AT

M-IO

PR

ELIA

BLE

SE

QU

EN

CE

D

PR

OT

OC

OL C

ON

FIG

UR

AT

ION

S

ST

AN

DA

RD

CO

RB

A P

RO

GR

AM

MIN

G A

PI F

eatures

!

�

Presentation

layer–

e.g.,CD

R

�

Message

formats

–e.g.,G

IOP,

DC

E

�

Transportassum

ptions–

e.g.,TC

P,ATM

�

Object

addressing–

e.g.,IIOP

IOR

ww

w.cs.w

ustl.edu/�

schmidt/pluggable

protocols.ps.gz

UC

Irvine47


Embedded System Benchmark Configuration

MARSHALMARSHAL

PARAMSPARAMS

OBJECT OBJECT ((SERVANTSERVANT))

OBJECT ADAPTEROBJECT ADAPTER

ACTIVEACTIVE

OBJECTOBJECT

MAPMAP


VME BUSVME BUS

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

IDLIDLSTUBSSTUBS

ORB MESSAGINGORB MESSAGING

ORB TRANSPORTORB TRANSPORT

CLIENTCLIENT

INC

OM

ING

INC

OM

ING

CDRCDR

DATA COPYDATA COPY

DATA COPYDATA COPY

DMADMA COPY COPY



CDRCDR

DATA COPYDATA COPY

DEMARSHALDEMARSHAL

PARAMSPARAMS

DATA COPYDATA COPY

OU

TG

OIN

GO

UT

GO

ING

ORBORBCORECORE

obj->op (params)

VxWorks running on 200 Mhz PowerPC over 320 Mbps VME & 10Mbps Ethernet

UC Irvine 48 Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Eth

erne

t&V

ME

Two-

way

Late

ncy

Res

ults

010

0020

0030

0040

0050

0060

0070

0080

0090

00

void

shor

t

octe

t

long

stru

ct

shor

t seq

<oc

tet>

long

seq

<oc

tet>

shor

t seq

<lo

ng>

long

seq

<lo

ng>

Data Type

Lat

ency

(u

sec)

VM

E/G

IOP

lite

avg.

VM

E/G

IOP

avg

.

Eth

erne

t/GIO

Plit

e av

g.

Syn

opsi

sof

Res

ults

�

VM

Epr

otoc

olis

muc

hfa

ster

than

Eth

erne

t

�

No

appl

icat

ion

chan

ges

are

requ

ired

tosu

ppor

tV

ME

UC

Irvi

ne


Pinpointing ORB Overhead with VMEtro Timeprobes


RECVRECV

OBJECT (SERVANT)

ORBORBCORECORE



VME BUSVME BUS

OS KERNELOS KERNEL

VME DRIVERVME DRIVER

OS KERNELOS KERNEL

VME DRIVERVME DRIVER



ACTIVEACTIVE

OBJECTOBJECT

MAPMAP

GET OBJECTGET OBJECT

REFREF

IDLIDLSTUBSSTUBS

MARSHALMARSHAL

PARMATERSPARMATERS


SENDSEND




SENDSEND

II//O SENDO SEND

II//O RECVO RECV77

88


RECVRECV99

CLIENTCLIENT

DEMARSHALDEMARSHAL

PARAMETERSPARAMETERS

7799

II//O RECVO RECV


RECVRECV


RECVRECV

PARSE OBJECTPARSE OBJECT

KEY KEY

OBJECTOBJECT

DEMUXDEMUX

DEMARSHALDEMARSHAL


USER UPCALLUSER UPCALL


SENDSEND


SENDSEND

II//O SENDO SEND

MARSHALMARSHAL


OPERATIONOPERATION

DEMUXDEMUX

131311

22

44

661111

1010

1212

33

88INITIALIZATIONINITIALIZATION

33

55

44

11

22

66

1010

OUTGOINGOUTGOING INCOMINGINCOMING

INC

OM

ING

INC

OM

ING

OU

TG

OIN

GO

UT

GO

ING

POAPOA

DEMUXDEMUX55

� Timeprobes use VMEtro monitor,which measures end-to-end time

� Timeprobe overhead isminimal, i.e., 1 �sec

UC Irvine 50 Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

OR

B&

VM

EO

ne-w

ayO

verh

ead

Res

ults

200

200

198

200

203

205

212

226

257

309

51

49

50

50

49

51

50

51

52

56

24

23

23

22

23

23

22

23

23

23

68

66

66

66

66

66

67

66

69

72

0%10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

48

1632

6412

825

651

210

2420

48S

eque

nce

leng

th (

byte

s)

usecs

VM

E +

OS

ove

rhea

dO

RB

clie

nt p

re-w

rite

OR

B c

lient

pos

t-w

rite

OR

B s

erve

r ov

erhe

ad

Syn

opsi

sof

Res

ults

�

OR

Bov

erhe

adis

rela

tivel

yco

nsta

ntan

dlo

w

–e.

g.,�

110

�

secs

per

end-

to-e

ndop

erat

ion

�

Bot

tlene

ckis

VM

Edr

iver

and

OS

,not

OR

B

UC

Irvi

ne


Workstation Benchmark Configuration

MARSHALPARAMS

OBJECT (SERVANT)

OBJECT ADAPTER

ACTIVE

OBJECT

MAP

IDLSKELETON

NETWORKLINK

REAL-TIME I /OINTERFACE

DRIVER

IDLSTUBS

ORB MESSAGING

ORB TRANSPORT

CLIENT

BUFFER

MANAGEMENT

SEND MESSAGE

ORB MESSAGING

ORB TRANSPORT

DEMARSHALPARAMS

DISPATCH

MESSAGE

ORBCORE

obj->op (params)

DEMUX

REAL-TIME I /OINTERFACE

DRIVER

Debian Linux running on 400 Mhz workstation over Local IPC

UC Irvine 52 Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Bla

ckbo

xTw

o-w

ayLa

tenc

yR

esul

ts

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

void

shor

toc

tet

long

struc

t

small

seq<

octe

t>

large

seq<

octe

t>

small

seq<

long>

large

seq<

long>

small

seq<

struc

t>

large

seq<

struc

t>

Dat

a T

ype

Calls per Second

0.00

%

10.0

0%

20.0

0%

30.0

0%

40.0

0%

50.0

0%

60.0

0%

Performance Improvement (%)

IIOP

IIOP

/GIO

Plit

eU

IOP

UIO

P/G

IOP

lite

Per

form

ance

Incr

ease

Syn

opsi

sof

Res

ults

�

Loca

lIP

Cm

ore

effic

ient

than

TC

P/IP

over

loop

back

�

No

appl

icat

ion

chan

ges

are

requ

ired

tosu

ppor

tm

ultip

lepr

otoc

ols

UC

Irvi

ne


Client Whitebox Latency ResultsDirection Client Activities Absolute Time ( �s)Outgoing 1. Initialization 6.30

2. Get object reference 15.63. Parameter marshal 0.74 (param. dependent)4. ORB messaging send 7.785. ORB transport send 1.026. I/O 8.70 (op. dependent)7. ORB transport recv 50.78. ORB messaging recv 9.259. Parameter demarshal op. dependent

Xeon platform is quad-CPU 400 Mhz with 1 Gigabytes RAM

UC Irvine 54


Server Whitebox Latency Results on Xeon/NTDirection Server Activities Absolute Time ( �s)Incoming 1. I/O 7.0 (op. dependent)

2. ORB transport recv 24.83. ORB messaging recv 4.54. Parsing object key 4.65. POA demux 1.396. Servant demux 4.67. Operation demux 4.528. User upcall 3.84 (op. dependent)

Outgoing 9. ORB messaging send 4.5610. ORB transport send 93.6

UC Irvine 55


ORB & Transport Overhead Results

112

111

52 51

49 48

49 47

24 23

27 27

31

27

30

27

0

50

100

150

200

250

IIOP IIOP w/GIOPlite UIOP UIOP w/GIOPlite

Transport Protocol

To

tal T

ime

(use

cs)

OS and I/O ORB Transport Messaging

Synopsis of Results

� ORB overhead is relatively constant and low

– e.g., �49 �secs per two-way operation

� Bottleneck is OS and I/O operation

UC Irvine


Data Copies in the Pluggable Protocols

MARSHALMARSHAL

PARAMSPARAMS

OBJECT OBJECT ((SERVANTSERVANT))


ACTIVEACTIVE

OBJECTOBJECT

MAPMAP


VME BUSVME BUS

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

IDLIDLSTUBSSTUBS



CLIENTCLIENT

INC

OM

ING

INC

OM

ING

CDRCDR

DATA COPYDATA COPY

DATA COPYDATA COPY

DMADMA COPY COPY



CDRCDR

DATA COPYDATA COPY

DEMARSHALDEMARSHAL

PARAMSPARAMS

DATA COPYDATA COPY

OU

TG

OIN

GO

UT

GO

ING

ORBORBCORECORE

obj->op (params)

� Marshal parameters, data copy to CDR stream

� VME send, data copy from CDR stream to VMEbuffers

� DMA, data copy over VME Bus

� VME read, data copy to CDR stream

� Demarshal parameters, data copy to methodparameters

UC Irvine


One-Way Delayed Buffering Strategy

MARSHALPARAMS

OBJECT (SERVANT)

OBJECT ADAPTER

ACTIVE

OBJECT

MAP

IDLSKELETON

VME BUS

OS KERNEL

VMEDRIVER

OS KERNEL

VMEDRIVER

IDLSTUBS

ORB MESSAGING

ORB TRANSPORT

CLIENT

BUFFERED

DATA COPY

DMA COPY

ORB MESSAGING

ORB TRANSPORT

DEMARSHALPARAMS

DATA COPY

ORBCORE

obj->op (params)

DEMUX

� Copy params to new buffer

� Requests buffered in the Transport Adaptor

� Flush at byte count or timeout

� Send as one ORB message but multiplerequests

� Server demultiplexes individual requests

UC Irvine


Shared Buffer Strategy

MARSHALPARAMS

OBJECT (SERVANT)

OBJECT ADAPTER

ACTIVE

OBJECT

MAP

IDLSKELETON

VME BUS

OS KERNEL

VMEDRIVER

OS KERNEL

VMEDRIVER

IDLSTUBS

ORB MESSAGING

ORB TRANSPORT

CLIENT

BUFFERED

DATA COPY

DMA COPY

ORB MESSAGING

ORB TRANSPORT

DEMARSHALPARAMS

DATA COPY

ORBCORE

obj->op (params)

DEMUX

� Request free buffer

� Add to Send queue

� Return to Free pool

� Request free buffer

� Add to Revc queue

� Return to Free pool

UC Irvine


Problem: Overly Large Memory Footprint

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

DII ORBINTERFACE

ORBORB CORECORE GIOPGIOP//IIOPIIOP//ESIOPSESIOPS

IDLIDLSTUBSSTUBS

operation()operation()in args



OBJ

REF

STANDARD INTERFACE STANDARD LANGUAGE MAPPING

ORB-SPECIFIC INTERFACE STANDARD PROTOCOL

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

IDLSKELETON

DSI

OBJECT

ADAPTER

www.cs.wustl.edu/�schmidt/COOTS-99.ps.gz

� Problem

– ORB footprint is toobig for someembedded apps

� Unnecessary Features

– DSI, DII, & DynamicAny

– Interface Repository– Advanced POA

features– CORBA/COM

interworking

UC Irvine 60


Solution: Minimum CORBA

Component CORBA Minimum PercentageCORBA Reduction

POA 282k 207k 26.5ORB Core 347k 330k 4.8Dynamic Any 131k 0 100CDR Interpreter 69k 69k 0IDL Compiler 10k 11k 0Pluggable Protocols 15k 15k 0Default Resources 8k 8k 0

Total 862k 640k 25.8

Applying Minimum CORBA subsetting to TAO reduces memoryfootprint by �25% (on SPARC with EGCS) and increases ORB

determinism

UC Irvine 61


Problem: Providing QoS to CORBA Operations

RTRTOperationOperation

PERIODIC SCHEDULINGPERIODIC SCHEDULING//DISPATCHINGDISPATCHING

TT++PP

P:P: PERIOD PERIOD

C:C: REQUEST COMPUTATION TIME REQUEST COMPUTATION TIME

PP

CC CC

TIMETIME

TT TT++22PP

struct RT_Info {struct RT_Info { Time worstcase_exec_time_; Time worstcase_exec_time_; Period period_; Period period_; Criticality criticality_; Criticality criticality_; Importance importance_; Importance importance_;};};

� Design Challenges

– Specifying/enforcing QoSrequirements

– Focus on Operations upon Objects

� Rather than on communication channels orthreads/synchronization

– Support static and dynamic scheduling


– Servants publish resource (e.g., CPU)requirements and (periodic) deadlines

– Most clients are also servants

UC Irvine 62


Solution: TAO’s Real-time Static Scheduling Service

RT_INFO

REPOSITORY

OFF-LINE

SCHEDULER

DEPENDS UPON =EXECUTES AFTER

4: ASSESS

SCHEDULABILITY

5: ASSIGN OS THREAD

PRIORITIES AND

DISPATCH QUEUE

ORDERING

SUBPRIORITIES

6: SUPPLY

PRIORITIES

TO ORB

3: POPULATE

RT_INFO

REPOSITORY

RUN-TIME

SCHEDULER

Priority/SubpriorityTable Per

Mode

MODE 0

MODE 1

MODE 2

MODE 3

CURRENT

MODE

SELECTOR

struct RT_Info { Time worstcase_exec_time_; Period period_; Criticality criticality_; Importance importance_;};

1: CONSTRUCT CALL

CHAINS OF RT_OPERATIONS

2: IDENTIFY THREADS

RT

Operation

RT

Operation

RT

Operation

I/O SUBSYSTEM

ORB CORE

OBJECT ADAPTER

www.cs.wustl.edu/�schmidt/TAO.ps.gz

UC Irvine 63


TAO’s RT Event Service Architecture

0 1 2 3 4

1: push (event)

Supplier ProxiesSupplier

2: push (event)

3: push (event, consumer)


5: enqueue (event, consumer)

6: dequeue (event, consumer)

7: push (event)

8: push (event)

Consumer

Consumer Proxies

Dispatcher

Priority Queues

Run-Time Scheduler

Event Correlation

Subscription & Filtering

Consumer

ConsumerEvent Channel

Dispatching Module

Features !

� Integrated with RTScheduler

� Stream-basedarchitecture– Enhance pluggability

� Source and type-basedfiltering

� Event correlations– Conjunctions (A+B+C)– Disjunctions (AjBjC)


UC Irvine 64


Visualizing Periods for Avionics Operations

UC Irvine 65


Example: Applying TAO to Real-time Avionics

I/O Facade

SensorProxy

SensorProxy

SensorProxy

SensorProxy

I/O Facade I/O Facade

2: Demarshaled data

High LevelAbstraction

Low LevelAbstraction

1: I/O via interrupts

AircraftSensors

� Synopsis

– Typical Interactions

� I/O arrives

� Proxies demarshaldata

� Facades process data– Advantages:

� Efficient control flow

� Clean layeredarchitecture

– Disadvantages:

� Coupled layers� Inflexible scheduling

UC Irvine 66


Forces/Domain Characteristics� I/O driven

– Periodic processing requirements

� Complex dependencies

– e.g., I/O Facades depend on multiple sensor proxies

� Real-time constraints

– Deterministic and statistical deadlines– Static scheduling (e.g., rate monotonic)

� Single-Processor (VxWorks)

– Single address space– No distribution requirements (yet)

UC Irvine 67


Candidate Solution: COS Event Service

push()push()

CONSUMERCONSUMER

CONSUMERCONSUMER

EVENTEVENT

CHANNELCHANNEL

CONSUMERCONSUMER

SUPPLIERSUPPLIER

SUPPLIERSUPPLIER

push()push()push()push()

push()push()

push()push()

www.cs.wustl.edu/�schmidt/report-doc.html

� Features

– Decoupledconsumers andsuppliers

– Transparent groupcommunication

– Asynchronouscommunication

– Abstraction fordistribution

– Abstraction forconcurrency

UC Irvine 68


Applying the COS Event Service to Real-time Avionics

� TAO is currently used at Boeing for avionics mission computing

– Initial flight dates are mid-summer 1998

� Extensive benchmarks demonstrate it is possible to meet stringentperformance goals with real-time CORBA

– e.g., for Boeing, target latency for CORBA oneway operations is150 �secs for 100 Mhz PowerPC running over MVME 177 boards

� Technology transfer to commercial vendors via OMG RT SIG andDARPA Quorom program & OCI

UC Irvine 69


Overview of Avionics Mission ComputingConsumers

I/O Facade

SensorProxy

SensorProxy

SensorProxy

SensorProxy

1: I/O via interrupts

I/O Facade I/O Facade

EventChannel

2: push (demarshaled data)

AircraftSensors

3: push (demarshaled data)

Suppliers

� Typical Interactions

– I/O arrives– Proxies demarshal

data– Proxies push to

channel– EC pushes to facades– Facades process data

� Advantages:

– Anonymousconsumers/suppliers

– Group communication– Asynchronous pushes

UC Irvine 70


Issues Not Addressed by COS Event Service� No support for complex event dependencies

– Consumer-specified event filtering– Event correlations (e.g., waiting for events A and B before

pushing)

� No support for real-time scheduling policies

– Priority-based dispatching (e.g., which consumer is dispatchedfirst)

– Priority-based preemption policies and mechanisms– Interval timeouts for periodic processing– Deadline timeouts for “failed” event dependencies

UC Irvine 71


TAO’s Event Service Architecture

Subscription& Filtering

EventCorrelation

DispatchingModule

EVENTCHANNEL

Consumer

ConsumerConsumer

Supplier SupplierSupplier

push (event)

push (event)

ConsumerProxies

SupplierProxies

PriorityTimers

EventFlow

� Features

– Stream-based architecture

� Enhance pluggability– Subscription/filtering

� Source and type-basedfiltering

– Event correlations

� Conjunctions (A+B+C)

� Disjunctions (AjBjC)

UC Irvine 72


Collaborations in the RT Event Channel

Subscription& Filtering

SupplierProxies

PriorityTimers

EventCorrelation

DispatchingModule

ConsumerProxies

EVENTCHANNEL

Object Ref

Subscription Info Publish Types

CorrelationSpecs

RT_Info

Object Ref

Timeout Registration

Consumer

Supplier

CONNECT_PUSH

SUPPLIER

CONNECT_PUSH

CONSUMER Source IDEvent TypeTimestamp

Data

EVENT

STRUCTURE

www.cs.wustl.edu/�schmidt/events tutorial.html

� Well-defined eventstructure

– CORBA Anys areinefficient

� Augmented COSinterfaces:

– Extra QoS structureto connect suppliersand consumers

UC Irvine 73


Real-Time Event Dispatching with TAO’s Event Service

0 1 2 3 4

1: push (event)

Supplier ProxiesSupplier

2: push (event)



5: enqueue (event, consumer)

6: dequeue (event, consumer)

7: push (event)

8: push (event)

Consumer

Consumer Proxies

Dispatcher

Priority Queues

Run-Time Scheduler

Event Correlation

Subscription & Filtering

Consumer

ConsumerEvent Channel

Dispatching Module

� Features

– Run-time scheduler

� Determines event priority– 2-level priority queues

� Preemption groups

� Priority queues– Dispatcher

� Encapsulatesconcurrency policy

UC Irvine 74


Real-time Event Channel Dispatching Experiments

0 1 2 3 4

(B) RTU Dispatching

Dispatcher

Run-TimeScheduler

3:3: DEQUEUE DEQUEUE

REQUEST REQUEST

4:4: DISPATCH DISPATCH

REQUEST REQUEST

1:1: INCOMING INCOMING

REQUEST REQUEST

2:2: ENQUEUE ENQUEUE

REQUESTREQUEST

0 1 2 3 4

(C) Threaded Dispatching

Run-TimeScheduler


REQUEST REQUEST


REQUESTREQUEST


REQUEST REQUEST


REQUEST REQUEST

Dispatcher

(A) FIFO Dispatching

Dispatcher


REQUEST REQUEST


REQUEST REQUEST


REQUESTREQUEST


REQUEST REQUEST

www.cs.wustl.edu/�schmidt/oopsla.ps.gz

UC Irvine 75


Multi-Threaded Dispatching

UC Irvine 76


Single-Threaded Dispatching

UC Irvine 77


Dimensions of ORB Extensibility

1 POSIX,POSIX, WWININ32,32, RTOSRTOSSS,, MVSMVS

3 DLLDLLSS

2

strategy2

strategy1

strategy3 profile2

profile1

profile3

profile4

1. Extensible to retargeting on new platforms

2. Extensible via custom implementation strategies

3. Extensible via dynamic configuration of custom strategies

UC Irvine 78


Applying Patterns to Develop Extensible ORBs

ACCEPTORCONNECTOR

ABSTRACT

FACTORY

SERVANTCLIENT

OS KERNELOS KERNEL

ACTIVE

OBJECT

THREAD-SPECIFIC

STORAGE

SERVICE

CONFIGURATOR

STRATEGY

REACTORREACTOR

WRAPPER FACADESWRAPPER FACADES

www.cs.wustl.edu/�schmidt/ORB-patterns.ps.gz

� Factories produce Strategies

� Strategies implementinterchangeable policies

� Concurrency strategies useReactor and Active Object

� Acceptor-Connectordecouple transport fromGIOP operations

� Service Configurator permitsdynamic configuration

UC Irvine 79


Addressing ORB Portability and Typesafety Challenges

� Problem

– Building an ORB using low-level system APIs is hard

� Forces

– Low-level APIs are tedious to program– Low-level APIs are error-prone– Low-level APIs are non-portable

� Solution

– Apply the Wrapper Facade pattern to encapsulate low-level OSprogramming details

UC Irvine 80


Enhancing Portability and Typesafety with the WrapperFacade Pattern

1: method_k()

2: function_k()

clientclient

FunctionsFunctions

function_1()...function_n()

WrapperWrapperFacadeFacade

method_1()...method_m()

www.cs.wustl.edu/�schmidt/wrapper-facade.ps.gz

� Intent

– Encapsulates low-levelsystem calls within type-safe,modular, and portable classinterfaces

� Forces Resolved

– Avoid tedious, error-prone,and non-portable system APIs

– Create cohesive abstractions– Don’t compromise

performance

UC Irvine 81


Using the Wrapper Facade Pattern in TAO

'&

$%

POSIXPOSIXPthreadsPthreads

BSDBSDSocketsSockets

WinSockWinSock

SocketSocketMacrosMacros Other OSOther OS

System CallsSystem Calls

pthread_create()pthread_create()

pthread_mutex_*pthread_mutex_*

socket(), bind(),socket(), bind(),

recv(), send()recv(), send()

gettimeofday(),gettimeofday(),

select(), etc.select(), etc.

SunSoft IIOP's ORB CoreSunSoft IIOP's ORB Core

GENERALGENERAL

POSIXPOSIX &&WWININ3232

SERVICESSERVICES

'&

$%

TAO's ORB CoreTAO's ORB Core

spawn()spawn()

acquire()acquire()

open(),open(),close(),close(),

recv(), send()recv(), send()

dlopen()dlopen()

dlsym()dlsym()

COMMUNICATIONCOMMUNICATION

SUBSYSTEMSUBSYSTEM

VIRTUALVIRTUAL

MEMORY MEMORY

SUBSYSTEMSUBSYSTEM

GENERALGENERAL

POSIXPOSIX && WWININ3232

SERVICESSERVICES


SUBSYSTEMSUBSYSTEM

THREADTHREAD

WRAPPERSWRAPPERS

DYNAMIC

LINKING

SELECT/IO COMP

SOCKETS/TLI

ACEWRAPPER

FACADES

handle_events()

SunSoft IIOP TAO

� TAO’s wrapper facades are based on the ACE framework

� The Wrapper Facade pattern substantially increased portability andreduced the amount of ad hoc code

UC Irvine 82


Addressing ORB Demuxing/Dispatching Challenges� Problem

– ORBs must process many different types of events simultaneously

� Forces

– Multi-threading may not be available– Multi-threading may be inefficient– Multi-threading may be inconvenient– Tightly coupling general event processing with ORB-specific logic

is inflexible

� Solution

– Use the Reactor pattern to decouple generic event processingfrom ORB-specific processing

UC Irvine 83


Enhancing Demuxing with the Reactor Pattern

Initiation DispatcherInitiation Dispatcher

handle_events()register_handler(h)remove_handler(h)

select (handles);foreach h in handles loop table[h].handle_event(type)end loop

Event HandlerEvent Handler

handle_event(type)get_handle()

handlers

HandleHandle ownsuses

notifies

ConcreteConcreteEventEvent

HandlerHandler

Synchronous EventSynchronous EventDemultiplexerDemultiplexer

select()

1 N

www.cs.wustl.edu/�schmidt/Reactor.ps.gz

� Intent

– Decouplessynchronous eventdemuxing/dispatchingfrom event handling

� Forces Resolved

– Demuxing eventsefficiently within onethread

– Extending applicationswithout changingdemux infrastructure

UC Irvine 84


Using the Reactor Pattern in TAO'&

$%

ORB COREORB CORE

1: RUN EVENT LOOP

OBJECTOBJECT

ADAPTERADAPTER

APPLICATIONAPPLICATION

2: GET REQUEST

3: BLOCK FOR CONNECTION 6: INCOMING

MESSAGE

5: DISPATCH

SERVANTSERVANT

7: UPCALL

GIOPGIOP

EngineEngine4: select()

serverserver

endpointsendpoints'

&

$%

ORB COREORB CORE

1: RUN EVENT LOOP

OBJECTOBJECT

ADAPTERADAPTER

APPLICATIONAPPLICATION

3: handle_input()

4: DISPATCH

SERVANTSERVANT

5: UPCALL

ACTIVE OBJECT MAPACTIVE OBJECT MAP

2: select()

ReactorReactor

ConnectionConnectionHandlerHandler



SunSoft IIOP TAO

� The Reactor pattern and ACE Reactor are widely used

UC Irvine 85


Addressing ORB Endpoint Initialization Challenges

� Problem

– The communication protocol used between ORBs is oftenorthogonal to its connection establishment and service handlerinitialization protocols

� Forces

– Low-level connection APIs are error-prone and non-portable– Separating initialization from processing increases software reuse

� Solution

– Use the Acceptor-Connector pattern to decouple passive/activeconnection establishment and GIOP connection handlerinitialization from the subsequent ORB interoperability protocol(e.g., IIOP)

UC Irvine 86


Enhancing Endpoint Initialization with theAcceptor-Connector Pattern'

&

$%

CREATE CREATE &&

ACTIVATEACTIVATE


peer_stream_

open()

EventEventDispatcherDispatcher

NO

TIFIE

S

NO

TIFIE

S

Strategy StrategyAcceptorAcceptor

peer_acceptor_

handle_input()

Connection HandlerConnection Handler

'&

$%

HANDLE ASYNC

CONNECTION COMPLETION

CREATE &

ACTIVATE

ConnectionHandler

peer_stream_

open()

EventDispatcher

StrategyConnector

connect(sh, addr)

complete()

Connection Handler

Acceptor Structure Connector Structure

� Intent

– Decouple connection establishment and service handlerinitialization from subsequent service processing

UC Irvine 87


Using the Acceptor-Connector Pattern in TAO

'&

$%

ORB COREORB CORE1: lookup()

2: connect() 4: accept()

5: read()/write()

5: read()/write()clientclient

endpointendpoint

clientclientendpointendpoint

clientclientendpointendpoint

serverserverendpointendpoint

listenerlistenerendpointendpoint

serverserverendpointendpoint

3: select()3: select()

SERVERCLIENT

'&

$%

GIOPGIOP

HandlerHandler

Strategy StrategyConnectorConnector

Cached Cached

ConnectConnect

StrategyStrategy


ReactorReactor

ORBORB CORECORE

SERVERSERVERCLIENTCLIENT

5:5: REQUEST REQUEST//RESPONSERESPONSE

2: connect()2: connect() 3: accept()3: accept()

0..N

6: DISPATCH()

4: CREATE & ACTIVATE

1: operation()

ConnectionHandler

ConnectionHandler

ConnectionHandler

Concurrency

Strategy

StrategyAcceptor

SunSoft IIOP TAO

� Forces Resolved

– (1) Improve portability and reuse and (2) avoid common mistakes

UC Irvine 88


Addressing ORB Concurrency Challenges

� Problem

– Multi-threaded ORBs are needed since Reactive ORBs are ofteninefficient, non-scalable, and non-robust

� Forces

– Multi-threading can be very hard to program– No single multi-threading model is always optimal

� Solution

– Use the Active Object pattern to allow multiple concurrent serveroperations using an OO programming style

UC Irvine 89


Enhancing ORB Concurrency withthe Active Object Pattern

ProxyProxy

Future m1()Future m2()Future m3()

HIDDENHIDDENFROMFROM

CLIENTSCLIENTS

VISIBLEVISIBLETOTO

CLIENTSCLIENTS

2: enqueue(M1)

ActivationActivationQueueQueue

enqueue()dequeue()

ServantServant

n1

loop { m = act_queue.dequeue() if (m.guard()) m.call()}

SchedulerSchedulerdispatch()enqueue()

m1()m2()m3() 4: m1()

11 11

1: enqueue(new M1)

3: dispatch()

MethodMethodRequestRequestguard()call()

M1M1

M2M2

M3M3

www.cs.wustl.edu/�schmidt/ Act-Obj.ps.gz

Intent

� Decouple thread ofrequest execution fromthread of requestreception

Forces Resolved

� Allow blockingoperations

� Permit flexibleconcurrency strategies

UC Irvine 90


Using the Active Object Pattern in TAO'&

$%

select()select()

2: NOTIFY

ORB CORE

serverendpoint

REACTIVE

1: ARRIVAL

3: READ

serverendpoint

select()

2: NOTIFY

ORB CORE

serverendpoint

1: ARRIVAL

3: READ

serverendpoint

THREAD-PERCONNECTION

'&

$%

ORB CORE

LEADER

FOLLOWERS

2: read()

Reactor

1: select()

3: release()

server endpoint

DISPATCHER

server endpoint

SunSoft IIOP TAO

� TAO supports several variants of Active Objects (e.g.,Thread-per-Connection, Thread-per-Request, Thread Pool, etc.)

UC Irvine 91


Reducing Lock Contention and Priority Inversions withthe Thread-Specific Storage Pattern

� Problem

– It is important to minimize the amount of locking required toserialize access to resources shared by an ORB

� Forces

– Locks increase performance overhead– Locks increase potential for priority inversion– Different concurrency schemes yield different locking costs

� Solution

– Use the Thread-Specific Storage pattern to maximizethreading-model flexibility and minimize lock contention andpriority inversion

UC Irvine 92


Minimizing ORB Locking withthe Thread-Specific Storage Pattern

THREAD ATHREAD A THREAD BTHREAD B

1: ACE_OS::thr_getspecific(key)

2: get_state(key)

ORB THREAD-SPECIFIC STATE

Reactor

Acceptor

Connector

Reactor

Acceptor

Connector

THREAD-SPECIFICOBJECT TABLES

INDEXED BY KEY

� Forces Resolved

– Minimizes overhead and priority inversion

Intent

� Allows multiplethreads to use onelogically globalaccess point toretrieve ORBthread-specificdata withoutincurring lockingoverhead for eachaccess

UC Irvine 93


Using Thread-Specific Storage in TAO

CLIENTCLIENT ORBORB CORECORE

20 10 5 1

HZ HZ HZ HZ

CONNECTOR

20 10 5 1

HZ HZ HZ HZ

CONNECTOR

20 10 5 1

HZ HZ HZ HZ

CONNECTOR

SERVER ORB CORE

REACTOR

(20 HZ)

REACTOR

(10 HZ)

REACTOR

(5 HZ)

REACTOR

(1 HZ)

CC

OO

NN

NN

EE

CC

TT

22

AA

CC

CC

EE

PP

TT

OO

RR

CC

OO

NN

NN

EE

CC

TT

11

CC

OO

NN

NN

EE

CC

TT

33

II//OO SUBSYSTEMSUBSYSTEMII//OO SUBSYSTEMSUBSYSTEM

CC

OO

NN

NN

EE

CC

TT

22

AA

CC

CC

EE

PP

TT

OO

RR

CC

OO

NN

NN

EE

CC

TT

11

C

O

N

N

E

C

T

3

C

O

N

N

E

C

T

2

A

C

C

E

P

T

O

R

C

O

N

N

E

C

T

1

C

O

N

N

E

C

T

3

C

O

N

N

E

C

T

2

A

C

C

E

P

T

O

R

C

O

N

N

E

C

T

1

C

O

N

N

E

C

T

3

COMMUNICATION LINK

CLIENT APPLICATION

STUB STUB STUB

P1 P22 PP33 PP44

www.cs.wustl.edu/�schmidt/TSS-pattern.ps.gz

UC Irvine 94


Addressing ORB Flexibility Challenges� Problem

– Real-world ORBs must be flexible to satisfy the requirements ofmany different types of end-users and applications

� Forces

– Ad hoc schemes for ORB flexibility are too static andnon-extensible

– Flexibility often has many (related) dimensions

� Solution

– Use the Strategy pattern to support multiple transparently“pluggable” ORB strategies

UC Irvine 95


Enhancing ORB Flexibility with the Strategy Pattern

StrategyStrategy

algorithm_interface()algorithm_interface()

ConcreteConcreteStrategy AStrategy A

algorithm_interface()algorithm_interface()

STRATEGY

ConcreteStrategy B

algorithm_interface()

ConcreteStrategy C

algorithm_interface()

Context

context_interface()

� Intent

– Factor out similarityamong algorithmicalternatives

� Forces Resolved

– Orthogonally replacebehavioral subsetstransparently

– Associating statewith an algorithm

UC Irvine 96


Using the Strategy Pattern in TAO

'&

$%

ORB COREORB CORE

OBJECTOBJECT ADAPTERADAPTER

DEMUXING

CODE

CONCURRENCY

CODE if (do_thread) // take lock...

...

if (do_thread) // releaselock...

CONNECTION

MANAGEMENT

CODE

if (do_thread)

// thread...

else

// single-threaded

'&

$%

Thread-Thread-SpecificSpecificConnectConnectStrategyStrategy

Cached CachedConnectConnectStrategyStrategy

Reactive ReactiveConcurrencyConcurrency

StrategyStrategy

Threaded ThreadedConcurrencyConcurrency

StrategyStrategy

SERVANT SERVANT 11 SERVANT NSERVANT N

IDLIDL

SKEL SKEL 22IDLIDL

SKEL NSKEL N

ORB COREORB CORE

SERVERSERVERCLIENTCLIENT

...... ............

SE

RV

AN

TS

ER

VA

NTN

::N

::O

PE

RA

TIO

NO

PE

RA

TIO

NKK

SE

RV

AN

TS

ER

VA

NTN

::N

::O

PE

RA

TIO

NO

PE

RA

TIO

N11

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

KK

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

22

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

11

(B) ACTIVE DEMUXING STRATEGY

index(object key)......

......

(A) PERFECT HASHING

DEMUXING

STRATEGY

OP

ER

AT

ION

OP

ER

AT

ION

KK

OP

ER

AT

ION

OP

ER

AT

ION

22

......

OP

ER

AT

ION

OP

ER

AT

ION

11

hash(operation)

hash(object key) OBJECTOBJECT

ADAPTERADAPTER

IDLIDL

SKEL SKEL 11

SERVANT SERVANT 22

ORB COREORB CORE

ORB COREORB CORE

StrategyStrategyAcceptorAcceptor

StrategyStrategyConnectorConnector

SunSoft IIOP TAO

UC Irvine 97


Addressing ORB Configurability Challenges

� Problem

– Aggressive use of Strategy pattern creates a configurationnightmare

� Forces

– Managing many individually configured strategies is hard– It’s hard to ensure that groups of semantically compatible

strategies are configured

� Solution

– Use the Abstract Factory pattern to consolidate multiple ORBstrategies into semantically compatible configurations

UC Irvine 98


Centralizing ORB Configurability withthe Abstract Factory Pattern

AbstractAbstract

FactoryFactory

make_product_A()

make_product_B()

AbstractAbstract

Product_AProduct_A

ConcreteConcrete

Factory_1Factory_1

make_product_A()

make_product_B()

ConcreteConcrete

Factory_2Factory_2

make_product_A()

make_product_B()

AbstractAbstract

Product_BProduct_B

Product_B1Product_B1

Product_A2Product_A2

Product_B2Product_B2

Product_A1Product_A1

� Intent

– Integrate allstrategies used toconfigure an ORB

� Forces Resolved

– Consolidatescustomization ofmany strategies

– Ensuressemantically-compatiblestrategies

UC Irvine 99


Using the Abstract Factory Pattern in TAO

Dispatching

Strategy

Demuxing

Strategy

ORB

Server

Abstract

Factory

Concurrency

Strategy Thread-

per-

Connection

FIFO

Dispatching

Perfect

Hashing

Avionics AvionicsConcreteConcreteFactoryFactory

Rate-based Rate-based

DispatchingDispatching

Thread- Thread-

per-per-

PriorityPriority

ActiveActive

DemuxingDemuxing

Medical MedicalImagingImagingConcreteConcreteFactoryFactory

UC Irvine 100


Addressing ORB Dynamic Configurability Challenges

� Problem

– Prematurely committing ourselves to a particular ORBconfiguration is inflexible and inefficient

� Forces

– Certain ORB configuration decisions can’t be made efficientlyuntil run-time

– Forcing users to pay for components they don’t use is undesirable

� Solution

– Use the Service Configurator pattern to assemble the desiredORB components dynamically

UC Irvine 101


Enhancing Dynamic ORB Extensibilitywith the Service Configurator Pattern

init()fini()suspend()resume()info()

ServiceService

DispatchingDispatchingServiceService

ServiceServiceRepositoryRepository

services

ConcurrencyConcurrencyServiceService

DemuxingDemuxingServiceService

www.cs.wustl.edu/�schmidt/Svc-Conf.ps.gz

Intent

� Decouples ORB strategiesfrom time when they areconfigured

Forces Resolved

� Reduce resource utilization

� Support dynamic(re)configuration

UC Irvine 102


Using the Service Configurator Pattern in TAO

dynamic ORB Service_Object * avionics_orb:make_orb() "-ORBport 2001"

Priority-basedPriority-basedDispatchingDispatching

TAOTAOPROCESSPROCESS

DLLDLLSS

Thread-per Thread-perRateRate

ConcurrencyConcurrency

AvionicsAvionicsConcreteConcreteFactoryFactory

PerfectPerfectHashingHashing

Service ServiceRepositoryRepository

ActiveActiveDemuxingDemuxing

MedicalMedicalImagingImagingConcreteConcreteFactoryFactory

FIFOFIFODispatchingDispatching

Thread-perThread-perConnectionConnection

ConcurrencyConcurrency

svc.confFILE

UC Irvine 103


Quantifying the Benefits of Patterns

1 − 5 6 − 10 > 10MVG Range

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

% M

etho

ds in

Ran

ge

SunSoft IIOPTAO

Macabe Complexity Metric Scores forTAO and SunSoft IIOP

� Statistics

– Patterns greatlyreduce codecomplexity

� e.g., Most TAOcomponents have

v(G) < 10

– TAO components aresubstantially smallerthan SunSoft IIOP

� e.g., connectionmanagementreduced by a factorof 5

UC Irvine 104


Lessons Learned Developing QoS-enabled ORBs

� Avoid dynamic connection management

� Minimize dynamic memory managementand data copying

� Avoid multiplexing connections fordifferent priority threads

� Avoid complex concurrency models

� Integrate ORB with OS and I/Osubsystem and avoid reimplementingOS mechanisms

� Guide ORB design by empiricalbenchmarks and patterns

'&

$%'

&

$%

UC Irvine 105


Concluding Remarks

� Researchers and developers of distributed, real-time applicationsconfront many common challenges

– e.g., service initialization and distribution, error handling, flow control,scheduling, event demultiplexing, concurrency control, persistence, faulttolerance

� Successful researchers and developers apply patterns,frameworks, and components to resolve these challenges

� Careful application of patterns can yield efficient, predictable,scalable, and flexible middleware

– i.e., middleware performance is largely an “implementation detail”

� Next-generation ORBs will be highly QoS-enabled, though manyresearch challenges remain

UC Irvine 106 Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Cur

rent

Sta

tus

ofTA

O

SC

HE

DU

LIN

G

PR

OP

ER

TY

EV

EN

T/

NO

TIF

ICA

TIO

NT

RA

DIN

G

NA

MIN

GC

ON

CU

RR

EN

CY

TIM

E

A/V

ST

RE

AM

ING

NE

TW

OR

K

OR

B Q

OS

INT

ER

FA

CE

RID

LS

TU

BS

RID

LS

KE

LE

TO

N

CL

IEN

T

OS

KE

RN

EL

OS

I/

O S

UB

SY

ST

EM

NE

TW

OR

K A

DA

PT

ER

S

OS

KE

RN

EL

OS

I/

O S

UB

SY

ST

EM

NE

TW

OR

K A

DA

PT

ER

S

GIO

P/R

IOP

OB

JE

CT

(SE

RV

AN

T)

LO

GG

ING

CORBA SERVICES

IDL

CO

MP

ILE

R

RE

AL-T

IME

OR

B C

OR

E

AC

EC

OM

PO

NE

NT

S

LIF

EC

YC

LE

RE

AL-T

IME

OB

JE

CT

AD

AP

TE

R

UC

Irvi

ne


Synopsis of TAO’s Pattern-Oriented ORB Design

'&

$%

DIIDII ORBORBINTERFACEINTERFACE

IDLIDLSTUBSSTUBS

PORTABLEPORTABLE

OBJECTOBJECT

ADAPTERADAPTER

IDLIDLSKELETONSKELETON DSIDSI

CLIENTCLIENT

ORB COREORB CORE

SERVERSERVER

StrategyConnector

Cached

Connect

Strategy

CLIENT

5: REQUEST/RESPONSE

2: connect() 3: accept()

GIOP

Handler0..N

6:6: DISPATCH DISPATCH()()

4:4: CREATE CREATE && ACTIVATE ACTIVATE

1: operation()1: operation()


Reactor Reactor

StrategyAcceptor

StrategyAcceptor

ConnectionHandler

ConnectionHandler

ConnectionHandler

ConnectionHandler

IDL COMPILER

'&

$%

......... ..................

SE

RV

AN

TS

ER

VA

NTN

::N

::O

PE

RA

TIO

NO

PE

RA

TIO

NMM

SE

RV

AN

TS

ER

VA

NTN

::N

::O

PE

RA

TIO

NO

PE

RA

TIO

N11

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

MM

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

22

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

11

OBJECT ADAPTER

...

...IDL

SKEL 2

(A) LAYERED DEMUXING,PERFECT HASHING

SERVANT 1 SERVANT 2

OP

ER

AT

IONM

OP

ER

AT

ION2

...

OP

ER

AT

ION1

IDL

SKEL K

hash(operation)

IDL

SKEL 1

(C) DE-LAYERED ACTIVE

DEMUXING

hash(object key) index(object key/operation)

(B) LAYERED DEMUXING,DYNAMIC HASHING

SERVANT N

ORB CORE

OBJECT ADAPTER

ORB CORE

ORBCORE

GIOP/IIOP

DII ORBINTERFACE

operation()

IDLSTUBS OBJECT

ADAPTER

IDLSKELETON DSI

in argsin args


CLIENTCLIENT OBJECTOBJECT((SERVANTSERVANT))

UC Irvine 108


Summary of TAO Research ProjectCompleted work

� First POA and first deployedreal-time CORBA schedulingservice

� Pluggable protocols framework

� Minimized ORB Core priorityinversion and non-determinism

� Reduced latency via demuxingoptimizations

� Co-submitters on OMG’sreal-time CORBA spec

Ongoing work

� Dynamic/hybrid scheduling

� Distributed QoS, ATM I/OSubsystem, & open signaling

� Implement CORBA Real-time,Messaging, and Fault Tolerancespecs

� Tech. transfer via DARPAQuorum program andwww.theaceorb.com

– Integration with Flick IDLcompiler, QuO, TMO, etc.

UC Irvine 109


Summary: Real-time Optimizations in TAO

NETWORK

OS KERNEL

OS I/O SUBSYSTEM

NETWORK INTERFACES

ORBINTERFACE

ORBCORE

operation()

IDLSTUBS

OBJECT

ADAPTER

IDLSKELETON

in args


CLIENT

GIOP

OBJECT(SERVANT)

CONCURRENCY

MODELS

TRANSPORT

PROTOCOLS

I/OSUBSYSTEM

NETWORK

ADAPTER

PRESENTATION

LAYER

SCHEDULING,DEMUXING, &DISPATCHING

DATA COPYING

& MEMORY

ALLOCATION

CONNECTION

MANAGEMENT

OS KERNEL

OS I/O SUBSYSTEM

NETWORK INTERFACES

OBJ

REF

UC Irvine 110


Next Steps: New TAO Features and Optimizations

STANDARD

SYNCHRONIZERS

END-TO-END

PRIORITY

PROPAGATION

ORB CORE

OBJECT

ADAPTER


THREAD

POOLS

EXPLICIT

BINDING

NETWORK

OS KERNEL

NETWORK INTERFACE

OS I/O SUBSYSTEM

OS KERNEL

NETWORK INTERFACE

OS I/O SUBSYSTEM

PLUGGABLEPROTOCOLS &

PROPERTIES

GIOP GIOP

Forthcoming Features

� CORBA ComponentModel (CCM)

� Real-time and MinimumCORBA

� CORBA Messaging

� Fault-Tolerant CORBA

� Notification Service

www.cs.wustl.edu/

�schmidt/TAO-status.html

UC Irvine 111


Next Steps: Integrating QoS-EnabledCORBA Component Model with TAO

NETWORK

ORB RUN-TIMEIDLSTUBS

IDLSKELETON

OS KERNEL

HIGH-SPEED

NETWORK INTERFACE

REAL-TIME I/OSUBSYSTEM

OS KERNEL

HIGH-SPEED

NETWORK INTERFACE

REAL-TIME I/OSUBSYSTEM

ACE COMPONENTS

REAL-TIME ORB COREIOP

PLUGGABLEORB & XPORTPROTOCOLS

IOPPLUGGABLE

ORB & XPORTPROTOCOLS

QoS

ADAPTATION

HOME

COMPONENT

IMPLEMENTATION

CA

LL

BA

CK

S

REAL-TIME POASCHEDULER

IDL

SKELS

CLIENToperation ()

in args


REAL-TIME

PORTABLE OBJECT ADAPTER

CO

RB

A S

ER

VIC

ES

(SE

CU

RIT

Y,

EV

EN

TS

...)

Features

� Select optimalcommunication reflectively

� Re-factor component QoSaspects into their containers

� Dynamically load/unloadcomponentimplementations

UC Irvine 112


Next Steps: Integrating TAO with ATM I/O Subsystem

REACTOR

(20 HZ)

REACTOR

(10 HZ)

REACTOR

(5 HZ)

REACTOR

(1 HZ)

ORB CORE

APIC ATM DRIVER

I/OSUBSYSTEM

PER-VC

SOCKET

QUEUES

APIC PACING

QUEUES

Z

E

R

O

C

O

P

Y

B

U

F

F

E

R

S

�schmidt/RIO.ps.gz

Features

� Vertical integration of QoSthrough ORB, OS, and ATMnetwork

� Real-time I/O enhancementsto Solaris kernel

� Provides rate-based QoSend-to-end

� Leverages APIC features forcell pacing and zero-copybuffering

UC Irvine 113


Next Steps: Strategized Scheduling Framework ssttrruucctt RRTT__IInnffoo {{ wwcc__eexxeecc__ttiimmee__;; ppeerr iioodd__;; ccrr iittiiccaalliittyy__;; iimmppoorr ttaannccee__;; ddeeppeennddeenncciieess__;; }};;

SSCCHHEEDDUULLIINNGG

SSTTRRAATTEEGGYY

RRTT__IINNFFOO

RREEPPOOSSII TTOORRYY

OOFFFF--LL II NNEE

SSTTRRAATTEEGGIIZZEEDD

SSCCHHEEDDUULLEERR

OOFFFF--LLIINNEE

OONN--LLIINNEE

RRTT__IINNFFOO

RREEPPOOSSII TTOORRYY

RRUUNN--TTIIMMEE

SSCCHHEEDDUULLEERR

11.. SSPPEECCIIFFYY RRTT__OOPPEERRAATTIIOONN

CCHHAARRAACCTTEERRIISSTTIICCSS AANNDD

DDEEPPEENNDDEENNCCIIEESS

55.. AASSSSEESSSS SSCCHHEEDDUULLAABBIILL IITTYY

33.. AASSSSII GGNN SSTTAATTIICC PPRRIIOORRIITTYY AANNDD SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY

44.. MMAAPP SSTTAATTIICC PPRRIIOORRIITTYY,, DDYYNNAAMMII CC SSUUBBPPRRII OORRII TTYY,, AANNDD

SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY II NNTTOO DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY

AANNDD DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

66.. AASSSSIIGGNN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE CCOONNFFIIGGUURRAATTIIOONN

22.. PPOOPPUULLAATTEE

RRTT__IINNFFOO

RREEPPOOSSIITTOORRYY

77.. SSUUPPPPLLYY DDIISSPPAATTCCHHIINNGG QQUUEEUUEE

CCOONNFFIIGGUURRAATTIIOONN TTOO TTHHEE OORRBB

RRTTOOppeerraattiioonn



II //OO SSUUBBSSYYSSTTEEMM

OORRBB CCOORREE

OOBBJJEECCTT AADDAAPPTTEERR

99.. SSUUPPPPLLYY SSTTAATTIICC PPOORRTTIIOONNSS OOFF

DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY AANNDD

DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

TTOO TTHHEE OORRBB

((SSCCHHEEDDUULLEERR''SS OOUUTTPPUUTT

IINNTTEERRFFAACCEE))

((SSCCHHEEDDUULLEERR''SS IINNPPUUTT

IINNTTEERRFFAACCEE))

88.. CCOONNFFIIGGUURREE QQUUEEUUEESS BBAASSEEDD

OONN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE

CCOONNFFIIGGUURRAATTIIOONN

1100.. DDYYNNAAMMIICC QQUUEEUUEESS AASSSSIIGGNN

DDYYNNAAMMIICC PPOORRTTIIOONNSS OOFF

DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

((AANNDD PPOOSSSSIIBBLLYY

DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY)) OORRBB EENNDDSSYYSSTTEEMM

www.cs.wustl.edu/�schmidt/dynamic.ps.gz

UC Irvine 114


Next Steps: Open ATM Signaling & Control

MANAGE

MIBTAO

RIO

APIC

NETWORK OPERATIONS

CENTER - ENDSYSTEM

GET STATSRECONFIG

SWITCH STATUS

CHANGE

RIO

APIC

TAO objB

objA

LOW LATENCY

CONTROLHIGH - BANDWIDTH

VIDEO/AUDIO DATA

ENDSYSTEM A

ENDSYSTEM B

HIGH - BANDWIDTH

BULK DATA

ENDSYSTEM C

HIGH -BANDWIDTH

VIDEO/AUDIO

MULTICAST

NETWORK

SERVICE

PROCESSOR

SPC

WUGS

SPC

SPC

SPC

SPC

SPC

SC

CLASSIFIER /ROUTER

ACTIVE NETWORK

RESOURCE CONTROLLER

NATIVE ATM BYPASSES THROUGH APIC

ATM PORT INTERCONNECT CONTROLLER (APIC)

USER

KERNEL

NSP

HIGH -PERFORMANCE

NETWORK ELEMENT

WEIGHTED

FAIR

QUEUING

IP FLOW

CLASSIFICATION

AND ROUTING

TAO

WUGS

OBJECT

SIGNALING

SERVERTAO

RESOURE

MANAGER

SWITCH

MANAGER

SWITCH CONTROLLER

HIGH -PERFORMANCE NETWORK ELEMENT

QOSMANAGER

TAO

PORT

CONTROLLER

SMART

PORT

CARD

UC Irvine 115

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Nex

tSte

ps:

Ada

ptiv

eM

iddl

ewar

e(

e.g.

,Q

uO/T

AO

)

INF

RA

ST

RU

CT

UR

EIN

FR

AS

TR

UC

TU

RE

MID

DL

EW

AR

EM

IDD

LE

WA

RE

DIS

TR

IBU

TIO

ND

IST

RIB

UT

ION

MID

DL

EW

AR

EM

IDD

LE

WA

RE

QQOO

S

S A

DA

PT

IVE

AD

AP

TIV

EL

AY

ER

LA

YE

R C

OM

MO

NC

OM

MO

NS

ER

VIC

ES

SE

RV

ICE

S

OP

ER

AT

ING

SY

ST

EM

OP

ER

AT

ING

SY

ST

EM

&& P

RO

TO

CO

LS

PR

OT

OC

OL

S

op

erati

on

()op

erati

on

()in

args

in args

ou

t a

rgs

+ r

etu

rn

valu

eou

t a

rgs

+ r

etu

rn

valu

eC

LIE

NT

CL

IEN

TO

BJE

CT

OB

JE

CT

((SE

RV

AN

TS

ER

VA

NT))

OP

ER

AT

ING

SY

ST

EM

OP

ER

AT

ING

SY

ST

EM

&& P

RO

TO

CO

LS

PR

OT

OC

OL

S

WIR

EL

ES

SW

IRE

LE

SS//W

IRE

LIN

E N

ET

WO

RK

WIR

EL

INE

N

ET

WO

RK

CL

IEN

T H

OS

TC

LIE

NT

H

OS

TS

ER

VE

R H

OS

TS

ER

VE

R H

OS

T

RESOURCE

RESOURCE

MANAGERS

MANAGERS

RESOURCE

RESOURCE

MANAGERS

MANAGERS

OB

JO

BJ

RE

FR

EF

STATUS

STATUS

COLLECTORS

COLLECTORS

BANDWIDTH

BANDWIDTH

CONTROL

CONTROL

PROPERTY

PROPERTY

MANAGERS

MANAGERS

POLICY

POLICY

MANAGERS

MANAGERS

QO

S M

EC

HA

NIS

MS

QO

S M

EC

HA

NIS

MS

...

QO

S M

AN

AG

ER

S

...

INF

RA

ST

RU

CT

UR

EM

IDD

LE

WA

RE

DIS

TR

IBU

TIO

NM

IDD

LE

WA

RE

QO

S A

DA

PT

IVE

LA

YE

R C

OM

MO

NS

ER

VIC

ES

ww

w.d

ist-

syst

ems.

bbn.

com

/pap

ers/

Key

The

mes

�

Dec

oupl

efu

nctio

nal

path

from

QoS

path

�

Em

phas

ize

inte

grat

ion

and

confi

gura

tion

UC

Irvi

ne

Hig

h-pe

rform

ance

,R

eal-t

ime

OR

Bs

Do

Nex

tSte

ps:

Aud

io/V

ideo

Str

eam

ing

End

-poi

ntFl

ow d

ata

Ada

ptor

Stre

am

(Sou

rce)

Flow

dat

a

Ada

ptor

Obj

ects

And

Man

agem

ent

Obj

ect

Con

trol

Inte

rfac

e

OR

B C

OR

E

POA

End

-poi

nt

Stre

am

Con

trol

Inte

rfac

e

Obj

ect

Stre

am

Stre

am POA

Con

trol

(Sin

k)

MU

LT

IME

DIA

STR

EA

M

ww

w.c

s.w

ustl.

edu/

�

schm

idt/a

v.ps

.gz

Effi

cien

cy

�

Soc

kets

for

data

tran

sfer

toge

thig

hpe

rfor

man

ce

Fle

xibi

lity

�

Use

sC

OR

BA

for

cont

rol

mes

sage

san

dpr

oper

ties

UC

Irvi

ne


Next Steps: Distributed Interactive Simulations

NETWORKNETWORKOPERATIONSOPERATIONS

CENTERCENTER

HSMHSM

ARCHIVEARCHIVE

SERVERSERVER

AGENTAGENT

INTERACTIVEINTERACTIVE

AUDIOAUDIO//VIDEOVIDEO

AGENTAGENT ARCHITECTUREARCHITECTURE

SPCSPC

HARDWAREHARDWARE

EMBEDDEDEMBEDDED

TAOTAO

MIBMIB

AGENTAGENT

UC Irvine 118


Web URLs for Additional Information� These slides: �schmidt/TAO4.ps.gz

� More information on CORBA: �schmidt/corba.html

� More info on ACE: �schmidt/ACE.html

� More info on TAO: �schmidt/TAO.html

� TAO Event Channel: �schmidt/JSAC-98.ps.gz

� TAO static scheduling: �schmidt/TAO.ps.gz

� TAO dynamic scheduling: �schmidt/dynamic.ps.gz

� ORB Endsystem Architecture: �schmidt/RIO.ps.gz

� Pluggable protocols: �schmidt/pluggable protocols.ps.gz

UC Irvine 119


Web URLs for Additional Information (cont’d)

� Network monitoring, visualization, & control: �schmidt/NMVC.html

� Performance Measurements:

– Demuxing latency: �schmidt/COOTS-99.ps.gz

– SII throughput: �schmidt/SIGCOMM-96.ps.gz

– DII throughput: �schmidt/GLOBECOM-96.ps.gz

– ORB latency & scalability: �schmidt/ieee tc-97.ps.gz

– IIOP optimizations: �schmidt/JSAC-99.ps.gz

– Concurrency and connection models: �schmidt/RT-perf.ps.gz

– RTOS/ORB benchmarks:

�schmidt/RT-OS.ps.gz

�schmidt/words-99.ps.gz

UC Irvine 120

patterns and performance of real-time high …schmidt/pdf/tao4.pdf · patterns and performance of...

Documents