december 16th, 2002 composable distributed real-time systems krishnakumar b [email protected]...

December 16th, 2002

Composable Distributed Real-time Systems

Krishnakumar [email protected]

Department Of Computer ScienceWashington University, St.Louis, MO

Advisor: Dr. Christopher D. GillFunded by DARPA PCES under Contract to

Boeing (F33615-00-C-3048) &Washington University in St.Louis (F33615-00-C-

1697)

2

Krishnakumar B Composable Distributed Real-Time Systems

Washington University in St.Louis

Outline of Talk• Components, Component

Models• Middleware Composition

Challenges• Solutions• Future work• Questions?

InterfaceRepository

IDLCompiler

ImplementationRepository

ClientOBJREF

Object(Servant)

in argsoperation()out args +

return

DIIIDL

STUBSORB

INTERFACE

IDLSKEL

DSI

Object Adapter

ORB CORE GIOP/IIOP/ESIOPS

3



Project Synopsis• Context

– Developing distributed real-time systems using components

• Problems– Configuration information tied to

implementation– Context sensitive QoS

specification– No validation of QoS properties– Ad hoc deployment

• Solution Approach– Specify configuration as meta-

data – Decouple QoS specification from

context dependencies– Validate configuration– Standardize deployment

Packaging & Deployment

Component Properties

Display

Database

Print

FileComponent Assembly

Database

Display

Print

File

File Properties

Database Properties

4



Overview of Components• Component

– Encapsulated part of a software system

– Implements a specific service or set of services

• Components vs. Objects– Multiple views per

component – Transparent navigation

between views– Extensibility without

inheritance– Higher-level Execution

Environment

FileComponent

FILE*File

UnixFile

FileStream

FileConnector

Different Views of aComponent

Client A

Client B

Client C

Client D

Navigation

5



Properties of Component Models• Component types, properties, inter-connections, interfaces• Component run-time infrastructure• Component Identity• Underlying Object Model

Features EnterpriseJavaBeans

Microsoft.NET

CORBAComponent

Model

Basic Type JavaBean .NETComponent

CORBAComponent

Transport RMI/IIOP SOAP IIOP

Object Model JVM CLR CORBA

Component Identification

JNDI GUID UUID

Unit of deployment JAR Files .NET Assembly

SoftPkg

6



CORBA Component Model• Component Model

specification from OMG• Builds on strengths of

CORBA• Key elements

– Component• Facets• Receptacles• Event Sources• Event Sinks

– ComponentHome– Container– Component Server– ORB Services

Facets

ComponentServer

Container

ComponentHome

ORB Middleware

Operating System

Transaction Persistence Events Security QoS

Component

Container

ComponentHome

Component Component ComponentReceptacle

External Interfaces

7



Component Integrated ACE ORB (CIAO)

• CCM implementation based on TAO

• QoS-aware Containers• Introspection interfaces• QoS assurance

mechanisms • Separate developer roles

– Application developer– Component

developer– QoS provisioners– ORB configurators

• Specify QoS as metadata

Client Application Server

Component Assembly

RT-ORB

in args

out args + return value

Operation ()

QoSMechanism

Plug ins

QoSMechanism

Plug ins

ClientConfiguration

Aggregate

QoSAdaptation

Container

CORBAComponent

ComponentHome

Real-time POA

QoS PropertyAdaptor

QoS Policies

Re

flect

QoSAdaptation

QoSAdaptation

QoSMechanism

Plug ins

NamedPolicy

Aggregate

NamedPolicy

Aggregate

ObjectReference

QoSAdaptation

QoSMechanism

Plug ins

QoS Policies

Component ConnectionSpecifications

Component & Home Impls

Deployment&

ConfigurationMechanism

PackagingTool

Assembly

ToolComponentAssemblyPackage

Home Properties Component Properties

Deployment

Tool

AssemblyDescriptor

CORBAComponent

Package

softpkgDescriptor

CORBAComponentPackage

Component Interconnection

Definitions

CORBAComponentPackage

8



Motivating Application• CORBA Event Channel

– Typical interactions:• I/O arrives• Proxies demarshal

data• Facades process data

• Problems– Numerous

configuration options– Configuration coupled

with implementation– No configuration

validation• Potential run-time

failure

9



Problems & SolutionsProblem Solution Contribution

Configuration tied with functionality

Specify configuration information as meta-data

Configuration decoupled from functionality

Implicit dependencies of QoS properties on context

Make context explicit as part of packaging

Well-defined dependencies

Invalid configuration resulting in run-time failure

Validate configuration meta-data prior to deployment

Static validation of configuration

Ad hoc manual deployment

Define standard mechanisms for deployment

Automated deployment

10



Configuration of Components• Context

– Configuration of distributed real-time systems

• Problem– Overwhelming options– Configuration tangled

with functionality• Strong Coupling

– Ad hoc specification• Interoperability

problems

– Premature finalization• Integration problems

ECDispatching ECDispatchingThreads

ECFiltering

ECSupplierFiltering

ECTimeout

ECScheduling

ECProxyPushConsumerCollection

ECProxySupplierLock

ECConsumerControl

ECSupplierControl

ECConsumerControlPeriod

ECSupplierControlPeriod

ECProxyConsumerLock

EventChannel

11



Solution: Configuration as Meta-data• Meta-data

– Describes systemic properties

• Describe configuration as meta-data

• Separate configuration from functionality

• Use meta-language to describe meta-data– XML provides a basis– Ample scope for

• Customization • Extension

– Increased interoperability


ECFiltering

ECSupplierFiltering

ECTimeout ECScheduling

ECProxyPushConsumerCollection

ECConsumerControl ECSupplierControl

ECConsumerControlPeriod ECSupplierControlPeriod

ECProxySupplierLockECProxyConsumerLock

XML !XML !

EventChannel

12



Solution Applied in CIAO• CIAO uses ACEXML library• Based on Simple API for XML

(SAX) v2.0– Callback-based API– Low memory footprint

• Document Type Definition (DTD)

• Validating Parser• Features

– DTD Validator– XML Namespaces– Locator– Filters

• Extensible– Pluggable Content

Handlers

Content Handler

ACEXML Parser

Client2: create

Default Handler

1: extend

4: register

3: instantiate

5: read

<?xml version="1.0"?> <doc> <para>Hello, world!</para>

</doc>

6: callback

XML File

13



Lessons Learned• Specifying configuration as meta-data

decouples– Configuration information from functionality– Act of configuration from functionality

• XML is very useful for describing meta-data– Customizable– Location transparent

• Allows out-of-band fetching of configuration information

• Interoperable

14
















15



Context Aware QoS Specification• Context

– Designing Components

• Multiple QoS assurance levels

• Different underlying middleware

• Problem– Developers exposed

to• Different QoS

features• Different

implementations

– Dependencies implicit

Vendor A’s ORB

Vendor B’s ORB

Thread Pool dispatch

Reactive dispatch

Component

16



Solution: Package Dependencies• Identify dependencies

– External– Internal

• Make dependencies explicit

• Describe dependencies extensibly– Scope for

standardization– Third-party

integration• Check dependencies

during component installation

Component A

ORB AComponent B

ORB B

libEvent.solibNotify.so

libSecurity.so libPersistence.so

<<depends>> <<depends>>

<<depends>>

<<depends>>

<<depends>>

<<depends>>

<<depends>> <<depends>>

<<depends>>

17



Solution Applied in CIAO (1/2)• CIAO captures dependencies using XML Software Package descriptors• Specification at multiple levels of granularity

– Individual implementation of same component – Component level– Package Level

<softpkg name="EventChannelPkg" version="0.1"> <pkgtype>CORBA Component</pkgtype> <title>EventChannel</title> <description>A simple Event Channel test </description> <dependency type="DSO" ><localfile name="libTAO.so"/> </dependency> <implementation id="EventChannel" > <os name="Linux" version="4,0,0,0" /> <processor name="x86" /> <compiler name="gcc" /> <programminglanguage name="C++" /> <propertyfile> <fileinarchive name="eventchannel.cpf"/></propertyfile> <code type="DSO" > <fileinarchive name="libService.so"/> <entrypoint>createEventChannel</entrypoint> </code> </implementation></softpkg>

18



Solution Applied in CIAO (2/2)• META-INF

– Well-known descriptor file

• ACEXML parses META-INF• CIAO

– Uses Pluggable Handlers features of ACEXML

– Implements SoftPkgHandler

• Dependencies checked during installation

SoftPkgHandler

ACEXML Parser

CIAO

2: create

DefaultHandler

1: extend

4: register

5: read

<softpkg> <implementation id="EventChannel"> <code type="DSO" > <fileinarchive name="libService.so"/> </code> </implementation></softpkg>

6: callback

META-INF

ACEXML Validator

3: instantiate

19



Lessons Learned• Dependency checking reduces run-time

surprises– Missing files within the package– Local environment files needed

• Standard format needed for interoperability– Different ORBs may use different methods to

specify• Dispatching models Thread pools, Reactive• Concurrency mechanisms Mutex, Semaphores• Different levels of filtering events

20
















21



Component Misconfiguration• Context

– Integration of application from COTS components

• Problem– Large number of

components• Check individually• Check overall system

– Manual integration • Error-prone• Doesn’t scale

ECConsumerControl ECConsumerControlPeriod

EventChannel


== 0== “mt”

== “reactive” == 1

22



Solution: Configuration Validation• Validate component

configuations– XML allows

specification of Document Type Definitions (DTD)

– Check configuration to ensure conformance to DTD

• QoS configuration validation should be done for– Individual components– Groups of components– Application wide in an

end-to-end fashion

ECConsumerControl ECConsumerControlPeriod

EventChannel


== “mt”

== “reactive”

>= 1

>=1000

23



PropertyFile

Solution Applied in CIAO• Component Property Files (cpf)

– Supports specifying simple types, structs, sequences, valuetypes

– Set of valid configurations • Validation done at deployment

– Minimize run-time failure• Minimize run-time exceptions• Allow static QoS provisioning<?xml version="1.0" encoding="UTF-8" ?><!DOCTYPE properties SYSTEM "properties.dtd"><properties> <struct name="ECProperties" type=“EC_Property"> <description> Configuration for a Simple Event Channel</description> <simple name="ECDispatching" type="string"> <value>mt</value> </simple> <simple name="ECDispatchingThreads" type=“long"> <value>2</value> </simple> <simple name="ECProxyConsumerLock" type="string"> <value>basic</value></simple> <simple name="ECConsumerControl" type="string"> <value>reactive</value></simple> <simple name="ECConsumerControlPeriod" type="long"> <value>50000</value> </simple> </struct></properties>

EC_Property

ECDispatching

ECDispatchingThreads

ECConsumerControl

ECConsumerControlPeriod

ECProxyConsumerLock

ACEXML Validator

SyntacticValidation

Configuration Validation

24



Lessons Learned• How to parse Hedge Automata

– Hedge is a sequence of trees– An XML document is a “Hedge”

• Configuration validation helps reduce “pilot error”

• Currently ORB configuration is intricately tied to the Service Configurator– Any configurable entity in the ORB is a

ACE_Service_Object– Only way to locate entities is from the

Service_Repository• Properties parsed from XML have to converted

back to text to be re-parsed by Service Configurator

25
















26



Deployment• Context

– Deploying an application built using COTS components

• Problem– Complex applications

• Large no. of components

– Micro-management of components

• Difficulty in reasoning complete end-to-end behaviour

– Manual deployment inherently error-prone

– Ad hoc scripts no better

QoSProperties

Checksum

Version Dependencies

List of FilesQoS Specs ComponentInterconnections

ImplementationDLL DLL

QoSProperties

Checksum



27



Solution : Component Packages• Packaging

– Logical grouping – Software Package

• Single Component

– Component Assembly• Multiple components

– Groups components with

• Implementation• Meta-data• Dependencies

• Packages are deployed as compressed (.ZIP) files

Meta-data

Component Component

Meta-data

Implementation

QoSProperties

Checksum


DLL DLL

List of FilesQoS Specs ComponentInterconnections

Assembly

Component Server

Deployment Application

Component Server Component Server

Assembly Assembly


Meta-data

QoSProperties

Checksum


28



Package Descriptors (1/2)• Package descriptors• CORBA Component Descriptor (.ccd)

– Describes a single component• Interfaces, ports, QoS policies• QoS properties described via (.cpf) files

• CORBA Software Descriptor (.csd)– Describes implementations of a single component– Useful in deploying a single component

<corbacomponent> <componentrepid repid="IDL:BookStore:1.0" /> <componentkind> <entity> <servant lifetime="component" /> </entity> </componentkind> <security rightsfamily="corba" rightscombinator="secallrights"> </security> <threading policy="multithread" /> <componentfeatures name="BookStore" repid="IDL:BookStore:1.0"> <inheritscomponent repid="IDL:Acme/Store:1.0" /> <ports> <provides providesname="book_search" repid="IDL:BookSearch:1.0" facettag="1"> <uses usesname="fedex_rates" repid="IDL:ShippingRates:1.0" /> <emits emitsname="low_stock" eventtype="StockRecord"><eventpolicy policy="normal" /></emits> <publishes publishesname="offer_alert" eventtype="SpecialOffer"> <eventpolicy policy="normal" /> </publishes> </ports> </componentfeatures></corbacomponent>

29



Package Descriptors (2/2)• CORBA Assembly descriptor (.cad)

– Describes a collection of components & their properties– Describes partitioning of the group of components– Describes dependencies among components

<componentassembly id="ZZZ123"> <description>Example assembly"</description> <componentfiles> <componentfile id="A"> <fileinarchive name="ca.ccsd"/> </componentfile> <componentfile id="B"> <fileinarchive name="cb.ccsd"/> </componentfile> </componentfiles> <partitioning> <homeplacement id="AaaHome"> <usagename>Example home for A components</usagename> <componentfileref idref="A"/> <componentimplref idref="an A impl"/> <homeproperties> <fileinarchive name="AHomeProperties.cpf"/> </homeproperties> <componentproperties> <fileinarchive name="defaultAProperties.cpf"/> </componentproperties> </homeplacement> </partitioning></componentassembly>

30



Deployment Hierarchy• Packages are read by

Deployment agent• Descriptor of package is

read from META-INF• Unpacking of packages in

safe directory– Avoid trojan

implementations• Action determined by

contents of META-INF • Set of Helper classes

– ComponentInstallation– AssemblyFactory– Assembly– ServerActivator

• Used in bootstrapping deployment

AssemblyFactory Assembly

Deployment Application

ServerActivator

ComponentInstallation

ComponentServer

Container

CCMHome

CCMObject

<<instantiates>>

<<instantiates>>

<<instantiates>>

<<instantiates>>

<<instantiates>>

31



Lessons Learned• Packaging makes component management

easier– Provides logical view of application subsytems– Aggregates groups of components– Provides COTS components in the literal sense

• Standardizing deployment– Solves problem once– Allows sharing of components– Reduce footprint

• Lazy component loading using Virtual Component design pattern

32



Future Work• Complete integration of

Packaging & Deployment framework with CIAO

• Decouple ORB configuration from Service Configurator

• Configuration of ORB from structs and sequences

• Allow imposing policies in a hierarchical fashion

• Measure overhead• Increase user-friendliness of

the Deployment framework

Client Application Server

Component Assembly

RT-ORB

in args

out args + return value

Operation ()

QoSMechanism

Plug ins

QoSMechanism

Plug ins

ClientConfiguration

Aggregate

QoSAdaptation

Container

CORBAComponent

ComponentHome

Real-time POA

QoS PropertyAdaptor

QoS Policies

Re

flect

QoSAdaptation

QoSAdaptation

QoSMechanism

Plug ins

NamedPolicy

Aggregate

NamedPolicy

Aggregate

ObjectReference

QoSAdaptation

QoSMechanism

Plug ins

QoS Policies

Component ConnectionSpecifications

Component & Home Impls

Deployment&

ConfigurationMechanism

33



Timeline

5/02

1 Apr, 20021 Apr, 2002

15 Jan, 200315 Jan, 2003

6/02 7/02 8/02 9/02 10/02 11/02 12/02 1/03

4/02XML Parser

11/6URL fetcher

6/02Namespace support

9/02Exception infrastructure

23/9Obstack improvements

10/02Exception Support

10/02XML Validator

10/02Component Packages

10/02QoS Specif ications

11/02Deployment Infrastructure

12/02Defense

1/03Integration

Non-software artifact

To be implemented

Implementation complete

Implemented byNanbor Wang

8/02WORDS 2003 paper

Component Packages refers to:1. Ability to package components as assemblies.2. Support for reading and modifying assemblies.

Deployment Infrastructure refers to:1. Ability to install a component(s) as assemblies.2. Setting up and tear down of a componenthierarchy using assemblies.

12/02Configuration Validation

34



Concluding Remarks• Summary Of Benefits

– Specify configuration as meta-data – Decouple QoS specification from context dependencies– Validate configurations– Standardize deployment

• Programming effort – ACEXML

• Common infrastructure - 8K lines• Parser Enhancements - 3K lines• Validator - 6K lines and growing• Available under $ACE_ROOT/ACEXML

– Packaging & Deployment • 4K lines • Available under $ACE_ROOT/ACEXML/compass

• Availability– cvs co –r Validator ACE_wrappers/ACEXML– cvs co –r CIAO ACE_wrappers

• First release early 2003

35



Acknowledgements• Dr. Chris Gill• Dr. Ron Cytron• Dr. David Levine• Dr. Doug Schmidt• All past and present members of the DOC

group

december 16th, 2002 composable distributed real-time systems krishnakumar b [email protected]...

Documents