Evaluating the Correctness and Effectiveness of a Middleware QoS
Configuration Process in DRE Systems
Institute for Software Integrated Systems
Dept of EECS, Vanderbilt UniversityNashville, TN, USA
Amogh Kavimandan, Anantha Narayanan, Aniruddha Gokhale, Gabor
www.dre.vanderbilt.edu/~gokhalePresented at ISORC 2008, Orlando, FL
May 5-7, 2008
Distributed Real-time and Embedded (DRE) Systems• DRE systems traits:
• Composed from diverse, complex sub-systems
• Stringent requirements on resources
• Multiple, simultaneous QoS requirements at local- (sub-system) and global- (application) level
• Heterogeneous platforms
• Increasing scale
2
• Trend towards component-based application development
• Functionality realized via composition, deployment & configuration of application components on execution platforms
Specification
Composition
Deployment Planning
Configuration
ComponentImpl Impl Impl
Properties Properties
ComponentImpl Impl Impl
Properties Properties
compose
DRE Systems Software Development Processes• DRE systems illustrate the
following (non-exhaustive) development stages:
• Specification – functional description, interface definition, implementation
3
Specification
Composition
Deployment Planning
Configuration
ComponentImpl Impl Impl
Properties Properties
ComponentImpl Impl Impl
Properties Properties
Application Packages
Assembly
ComponentComponent
ComponentComponent
compose
create
DRE Systems Software Development Processes• DRE systems illustrate the
following (non-exhaustive) development stages:
• Specification – functional description, interface definition, implementation
• Composition – functional integration, hierarchical organization & packaging
4
Specification
Composition
Deployment Planning
Configuration
ComponentImpl Impl Impl
Properties Properties
ComponentImpl Impl Impl
Properties Properties
Application Packages
Assembly
ComponentComponent
ComponentComponent
compose
Deployment Plan
create
System Resource Profile
Bridge
Firewall
Computing Resources
Network Topology
Infrastructure
DRE Systems Software Development Processes• DRE systems illustrate the
following (non-exhaustive) development stages:
• Specification – functional description, interface definition, implementation
• Composition – functional integration, hierarchical organization & packaging
• Deployment – computing resource allocation, node placement
5
Specification
Composition
Deployment Planning
Configuration
ComponentImpl Impl Impl
Properties Properties
ComponentImpl Impl Impl
Properties Properties
Application Packages
QoS Design
Service Levels
Preserve priority
Event Filtering
Prioritize invocations
Assembly
ComponentComponent
ComponentComponent
compose
Deployment Plan
create
System Resource Profile
Bridge
Firewall
Computing Resources
Network Topology
Infrastructure
DRE Systems Software Development Processes• DRE systems illustrate the
following (non-exhaustive) development stages:
• Specification – functional description, interface definition, implementation
• Composition – functional integration, hierarchical organization & packaging
• Deployment – computing resource allocation, node placement
• Configuration – choosing right set of parameters for hosting infrastructure (e.g., middleware) for actuating application QoS requirements
6Focus on DRE systems middleware QoS configuration
CCM Platform ThreadPool
Allow_request_buffering;max_buffered_requests;stacksize;max_buffer_size;allow_borrowing;
PriorityModelPolicypriority_model;priority_value;
BandedConnectionlow;high;
Laneslong static_threads; long lane_priority;long dynamic_threads;
RTECFactoryec_dispatching;ec_dispatch_thrds;ec_filterting;ec_scheduling;
PublisherBehaviorpublisher_control;pub_cntrl_period;publisher_lock;
FilterGroup
SubscriberBehaviorsubscriber_control;sub_cntrl_period;subscriber_lock;
Observer
System QoS RequirementsMultiple service levels
Preserve invocation priority
On-demand event delivery
Publisher/subscriber/topic event filtering
Prioritize service invocations
Reactive event control
Group events
Middleware QoS Configuration Process
ProxyCollection
7
• What vs how - Middleware platforms provide what is required to achieve system QoS not always how it can be achieved• No centralized orchestrator to realize QoS
from options providing individual QoS control
• Non-trivial to manually perform QoS configuration activity• Choose appropriate configuration
mechanisms in an application-specific manner, particularly for large applications
• Middleware does not prevent developers from choosing semantically invalid configurations to achieve QoS
Middleware QoS Configuration: Hard Challenges
Lack of effective QoS configuration tools result in QoS policy mis-configurations that are hard to analyze & debug
Automated Middleware QoS Configuration: Hard Challenges
8
• Use DRE system QoS requirements to automated middleware QoS configurations
• Different DRE systems (e.g., shipboard computing environment, emergency response services) exhibit variability in domain-specific QoS requirements
• How do we deal with this variability?
• Must deal with plethora of configuration mechanisms of hosting middleware platforms for muliple m/w platforms
• How do we bridge the gap between domain-specific requirements and configuration mechanisms?
Control Center
SystemResourceManager
Tool must address the variability and bridge this gap
– Shields developers from configuration semantics
• Automated translation using model transformation to generate system
QoS configurations– Reusable, one-step translation– Encodes best practices in QoS mapping
QoS Policy Modeling
request/response
publish/subscribe
brokered notification
reliable messaging
QoS Configurations
Metamodel
Target Model
instanceof
Graph Rewriting Rules
G G’ G G’
G G’ G G’ G G’
Requirements Metamodel
Source Model
instanceof
Transformation Engine
Model Transformation
Non-functional Analysis
DRE Developer
System QoS Configuration
Evolution
System-level QoS
requirements
Requirements Metamodel
QoS Configurations
Metamodel
QoS Mapping
• QUality of service pICKER (QUICKER):
• Domain-independent QoS modeling languages – express system QoS in terms of requirements semantics
– Easier to model, evolve, lesser modeling effort QoS Policy Modeling
request/response
publish/subscribe
brokered notification
reliable messaging
9
Solution Approach: QUICKER
ISORC 2007: General idea of QUICKER
RTAS 2008: Model transformation algorithms for CCM/RT
QoS Policy Modeling
request/response
publish/subscribe
brokered notification
reliable messaging
Validation & Verification
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
10
ISORC 2008: Evaluating QUICKER
1. Is the transformation process correct?• We use structural
correspondence to prove the transformations correct
2. Are the generated artifacts correct?• We use model
checking
3. Do the generated configurations deliver the desired QoS?• We use empirical
validation
QoS Configurations
Metamodel
Target Model
instanceof
Graph Rewriting Rules
G G’ G G’
G G’ G G’ G G’
Requirements Metamodel
Source Model
instanceof
Transformation Engine
Model Transformation
Non-functional Analysis
DRE Developer
System QoS Configuration
Evolution
System-level QoS
requirements
Requirements Metamodel
QoS Configurations
Metamodel
QoS Mapping
QoS Policy Modeling
request/response
publish/subscribe
brokered notification
reliable messaging
Focus on the correctness & effectiveness of our QoS configuration process
• Verification framework• Specify correctness properties at meta-level• Add annotations for each instance (correspondence rules)• Use annotations to automatically verify whether the instances
satisfy the correctness properties
• We do not attempt to prove the general correctness of the transformation itself
Source Meta
Target Meta
Source Model
Target Model
CorrectnessSpecification
Model Transformation
CorrectnessChecker
AnnotationsCertificate
(1) Verifying the Correctness of QoS Mapping Algorithms
11
RealTimeConfiguration
-priority_model : Policy-default_priority : long
PriorityModelPolicy
10..1
StructuralCorrespondence-bursty_client_requests : bool-fixed_priority_service_execution : bool
RTRequirements
11 1 1
• Add cross links to identify corresponding elements• Rules specify correspondence conditions for selected types• At the end of the transformation, the instance models are checked
if they satisfy all the correspondence conditions
Input Model Output Model
CorrespondenceRules
crosslink
(1) Verifying the Correctness of QoS Mapping Algorithms
12
13
Container
COMPONENTEXECUTORS
ComponentHome
POA
CallbackInterfaces
I nt e
r na
lIn
terf
ac e
s
Eve
ntS
inks
Face
t s
Rec
ep
t acl
esE
ven
tS
ou
r ce
s
ComponentReference
Co
mp
on
en
t C
on
t ex
t
COMPONENT SERVER 1
Container
COMPONENTEXECUTORS
ComponentHome
POA
CallbackInterfaces
I nte
r na
lI n
t erf
ace
s
Ev
ent
Sin
ksF
ace
t s
Re
ce
pt a
cle
sE
v en
tS
ou
rce
s
ComponentReference
Co
mp o
nen
t C
on
t ex
t
COMPONENT SERVER 2
ORB
End-to-End PriorityPropagation
ThreadPools
Portable PrioritiesProtocol Properties
Priority Band
• Dependencies may span beyond “immediate neighbors”, e.g.,
– application execution path
– components belonging to separate assemblies
• Empirically validating configuration changes slows down development & QA process considerably
• Several iterations before desired QoS is achieved (if at all)
Container
COMPONENTEXECUTORS
ComponentHome
POA
CallbackInterfaces
I nt e
r na
lIn
terf
ac e
s
Eve
ntS
inks
Face
t s
Rec
ep
t acl
esE
ven
tS
ou
r ce
s
ComponentReference
Co
mp
on
en
t C
on
t ex
t
COMPONENT SERVER 1
Assembly 1 Assembly n
PriorityModel
Priority Band
(2) Verifying the Generated QoS Configurations
Comm<Lanes, PModel>
Ana1<Bands>
Ana2<Bands>
Ana3<Bands>
Flt1<Bands>
Flt2<Bands>
Flt3<Bands>
Gzm1<Bands>
Gzm2<Bands>
Gzm3<Bands>
Depends on
options of dependent component(s) triggers detection of potential mismatches• e.g., dependency between Gizmo invocation priority & Comm lane priority
14
• Leveraging Bogor model checking framework• Dependency structure
maintained in Bogor used to track dependencies between QoS options of components, e.g.:
– Analysis & Comm are connected
– Gizmo & Comm are dependent
• Change(s) in QoS
Detect mismatch if either values change
(2) Verifying the Generated QoS Configurations
15
• Representation of middleware QoS options in Bogor model-checker
• BIR extensions allow representing domain-level concepts in a system model
• QUICKER defines new BIR extensions for QoS options
– Allows representing QoS options & domain entities directly in a Bogor input model
– e.g., CCM components, Real-time CORBA lanes/bands are first-class Bogor data types
• Reduces size of system model by avoiding multiple low-level variables to represent domain concepts & QoS options
(3) Verifying the Generated QoS Configurations
16
• Representation of properties (that a system should satisfy) in Bogor
• BIR primitives define language constructs to access & manipulate domain-level data types, e.g.:
– Used to define rules that validate QoS options & check if property is satisfied
• Automatic generation of BIR of DRE system from QUICKER-generated output models
Model interpreters auto-generate Bogor Input Representation of a system from its model
(3) Verifying the Generated QoS Configurations
of correctness of QUICKER’s QoS configuration
process• Verified the generated QoS configurations through model-checking
• QUICKER toolchain provides
• QoS requirements modeling languages
• QoS mapping algorithms for mapping requirements to middleware QoS options
• We discussed verification
Model Transformation
Non-functional Analysis
DRE Developer
System QoS Configuration
Evolution
System-level QoS
requirements
Requirements Metamodel
QoS Configurations
Metamodel
Validation & Verification
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
ComponentS2
S3S4
S1
QoS Configurations
Metamodel
Target Model
instanceof
Graph Rewriting Rules
G G’ G G’
G G’ G G’ G G’
Requirements Metamodel
Source Model
instanceofQoS Policy Modeling
request/response
publish/subscribe
brokered notification
reliable messaging
QoS Mapping
Transformation Engine
20
Concluding Remarks
QUICKER can be downloaded from www.dre.vanderbilt.edu/CoSMIC/
• Verified the correctness of QoS Mapping Algorithms through structural correspondence• Empirically validated the configurations by applying the QUICKER process to
representative DRE system case study• Future work based on capturing variability in requirements & middleware
Questions?
21
1-5 10-20 21-100
BACKUP
deve
lops
QoS Design
Multi-service Levels
Preserve invocn. priority
Event delivery on-demand
Event Filtering
Prioritize invocations
Component
Impl Impl Impl
Properties Properties
deploys
Application Specification
Platform Specification
Overview of QUICKER: Specifying QoS Requirements• Challenge 1. QoS
requirements specification
• DRE developers are domain experts who understand domain-level issues, system QoS specification must be expressible at the same level of abstraction
23
24
• Challenge 1. QoS requirements specification
• DRE developers are domain experts who understand domain-level issues, system QoS specification must be expressible at the same level of abstraction
• Large gap between what is required (by the application) and how it can be achieved (by the middleware platform)
• Configurations can not be reused; difficult to scale to large-scale systems
deve
lops
QoS Design
Multi-service Levels
Preserve invocn. priority
Event delivery on-demand
Event Filtering
Prioritize invocations
Component
Impl Impl Impl
Properties Properties
deploys
Application Specification
Platform Specification
Overview of QUICKER: Specifying QoS Requirements
25
• Application requirements are expressed as QUICKER QoS policy models
• QUICKER captures policies in a platform-independent manner
– Specifying QoS is tantamount to answering questions about application; rather than using low-level mechanisms (such as type of publisher proxy collection, event dispatching mechanism etc.) to achieve QoS
Require support for request buffering.
Gizmo 1Gizmo 2Gizmo 3
Filter 1Filter 2Filter 3
Analysis 1Analysis 2Analysis 3
Comm Ground
Science Agent
Requests executed at the invocation priority.
Support varying service levels for clients.
Requires concurrency support in order to handle all Gizmo requests.
Overview of QUICKER: Specifying QoS Requirements
• Representation at multiple levels of granularity– e.g., component- or assembly-level
26
• Application requirements are expressed as QUICKER QoS policy models
• QUICKER captures policies in a platform-independent manner
– Specifying QoS is tantamount to answering questions about application; rather than using low-level mechanisms (such as type of publisher proxy collection, event dispatching mechanism etc.) to achieve QoS
Requirement
OutEventPort
InEventPort
RequiredRequestPort
ProvidedRequestPort
RequestConnectorEventConnector
*
-src0..*
-dst0..*
*
*-src0..1
-dst0..*
*
Component
ComponentAssembly
1
0..*
1
0..*
1
0..*
1
0..*
1
0..*
-src*
-dst1
-src*
-dst1
-src
*
-dst
1
-src
*
-dst
1
Overview of QUICKER: Specifying QoS Requirements
27
• Benefits of QUICKER QoS policy modeling
• QoS policy specifications can be containment inherited, reused
• QoS policy inherited by all contained objects
• More than one connections can share QoS policies
• Scalable, flexible QoS policy models
Overview of QUICKER: Specifying QoS Requirements
28
Pub-side filtering Sub-side filteringEvent-type
filtering
Dispatching Scheduling
Event Grouping Pub-proxy collection
Sub-proxy collection
Event TimeoutObserverPub-disconnect
control
Sub-disconnect control
// Get the correct bands from the <server_declared_obj>. policies[0] = server_declared_obj->_get_policy (RTCORBA::PRIORITY_BANDED_CONNECTION_POLICY_TYPE); RTCORBA::PriorityBandedConnectionPolicy_var
bands_policy = RTCORBA::PriorityBandedConnectionPolicy ::_narrow (policies[0]); RTCORBA::PriorityBands_var bands = bands_policy->priority_bands ();
// Set the proper bands at the object level. Note that a new // object is returned. object = client_propagated_obj->_set_policy_overrides(policies,
CORBA::SET_OVERRIDE);
iterator:
COPY_ON_READ
COPY_ON_WRITE
DELAYED
IMMEDIATE
dispatching:
REACTIVE
PRIORITY
MT
scheduling:
NULL
PRIORITY
bands:
low_prio
high_prio
fltrgrp:
DISJUNCTION
CONJUNCTION
LOGICAL_AND
TPool:
stacksize
lane_borrowing
request_bufferring
lanes:
static_thrds
dyna_thrds
• Challenge 2. QoS realization
• Very large configuration space providing high degree of flexibility and configurability
• Semantic compatibility of QoS configurations enforced via low-level mechanisms – tedious, error-prone
• Prune middleware configuration space, instantiate configuration set selected, validate their values
Overview of QUICKER: Realizing System QoS
29
• Mapping application QoS policies onto configuration options using model transformation developed in GReAT
• Semantic translation algorithms specified in terms of input & output languages
– e.g., rules that translate multiple application service requests & service level policies to corresponding QoS options
• Transformation output as system model – allow for further analysis & translation
• Simplifies application development & enhances traceability
PriorityModel
Lane Lane Lane
Pro
vid
er
Ser
vice
Req
ues
t
Provider Service Levels
Level 1
Level 2
Level 3
Multiple Service Requests Service Levels
Priority Model PolicyThread Pool Lanes
Overview of QUICKER: Realizing System QoS
30
Challenge 2 Resolved: Realizing System QoS• Algorithm shown is RT-CCM QoS mapping
that uses• Application structural properties to
automatically deduce configurations– Lines 7-16 show thread resource
allocation scheme– Line 27 shows client-side QoS
configurations• QoS policies specified used for the
remaining configurations– Service invocation profile used to
assign thread resources– Line 28 resolves priority dependency
of connected components
# of client components and interface operations used to calculate # of threads required
Comm Ground
Gizmo 1 Filter 1 Analysis 1
Gizmo 2 Filter 2 Analysis 2Science Agent
Gizmo 3 Filter 3 Analysis 3
31
• Challenge 3. QoS options dependency resolution of application sub-systems
• Configurations of connected components may exhibit dependency relationship
– e.g., server-side priority model and client-side priority bands must match
• Manually tracking dependencies between components is hard
Overview of QUICKER: Realizing System QoS