![Page 1: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/1.jpg)
Monitoring IVHM Systems using a Monitor-Oriented Programming
Framework
S. Ghoshal, S. Manimaran - QSI
G. Rosu, T. Serbanuta, G. Stefanescu - UIUC
![Page 2: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/2.jpg)
IVHM System Analysis
• IVHM systems pose significantly higher safety and dependability requirements than most other systems
• Formal analysis of IVHM systems is therefore highly desirable …
• … but also challenging, due to their highly integrated nature (different technologies, hardware, software, sensors, etc.) and combined complexity
![Page 3: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/3.jpg)
Overview
• Our Approach
• MOP (University of Illinois at Urbana)
• TEAMS (Qualtech Systems Inc.)
• Project Research Plan
• Conclusion and Future Work
![Page 4: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/4.jpg)
Our Approach to IVHM Analysis
• Separation of concerns
1. State “health” assessment, or diagnosis
2. Temporal behaviors of state sequences
Steering / Recovery
IVHM System
Model-basedobservation
Temporal behavior monitor
Violation /Validation
Abstract events/statesTEAMS
MOP
![Page 5: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/5.jpg)
Overview
• Our Approach
• MOP (University of Illinois at Urbana)
• TEAMS (Qualtech Systems Inc.)
• Project Research Plan
• Conclusion and Future Work
![Page 6: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/6.jpg)
Monitoring-Oriented Programming (MOP)http://fsl.cs.uiuc.edu/mop
- proposed in 2003 –RV’03, ICFEM’04, RV’05, CAV’05, TACAS’05,
CAV’06, CAV’07, OOPSLA’07, ICSE08, …
ERE LTL ptLTL ptCaRet
logic plugins
…
…JavaMOP
BusMOP
MOP CFG
lang
uage
s
![Page 7: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/7.jpg)
What is MOP?
• Framework for reliable software development– Monitoring is basic design discipline
• … rather than “add on” grafted onto existing code
– Recovery allowed and encouraged– Provides to programmers and hides under the hood a
large body of formal methods knowledge/techniques• Monitor synthesis algorithms
– Generic for different languages and application domains
![Page 8: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/8.jpg)
Example: Correct and efficient sorting
Heapsort
O(n log(n))
Monitor ifvector issorted
yes
O(n)
Insertionsort
no
O(n2)provablycorrect
Works in MOP
We have an efficient and provably correct sorting algorithm!We avoided proving heap sort correct, which is hard!
Need to show it does not
destroy the multiset
![Page 9: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/9.jpg)
MOP Example: “Authentication before use”
Execution 1, correct
authenticate
begin end
use
begin end
Execution 2, incorrect
![Page 10: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/10.jpg)
class Resource {/*@ class-scoped SafeUse() { event authenticate : after(exec(* authenticate())) event use : before(exec(* access())) ptltl : use -> <*> authenticate }@*/
void authenticate() {...}void access() {...}...}
MOP Example: “Authentication before use”
![Page 11: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/11.jpg)
MOP Example: “Enforce authentication before use”
Execution 1, correct
authenticate
begin end
use
begin end
Execution 2, incorrect but corrected
Call authenticate()
![Page 12: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/12.jpg)
class Resource {/*@ class-scoped SafeUse() { event authenticate : after(exec(* authenticate())) event use : before(exec(* access())) ptltl : use -> <*> authenticate violation { @this.authenticate(); } }@*/void authenticate() {...}void access() {...}...}
MOP Example: “Enforce authentication before use”
![Page 13: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/13.jpg)
/*@ class-scoped SafeClose() { event openRegKey : after(exec(* openRegKey())) event closeHandle : before(exec(* closeHandle())) event closeRegKey : before(exec(* closeRegKey())) ere : any* openRegKey closeHandle validation { @this.closeRegKey(); return; } }@*/
Method openRegKey should be followed
by closeRegKey, not by closeHandle
MOP Example: “Correcting method matching”
![Page 14: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/14.jpg)
/*@ class-scoped FileProfiling() { [ int count = 0; int writes = 0;] event open : after(call(* open(..))) {writes = 0;} event write : after(call(* write(..))) {writes ++;} event close : after(call(* close(..))) ere : (open write+ close)* violation { @RESET; } validation { count ++; File2.log(count + ": " + writes); } }@*/
MOP Example: ProfilingHow many times a file is open, written to, and then closed?
![Page 15: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/15.jpg)
Fail Fast Iterators
Vector v = new Vector();v.add(new Integer(1));Iterator i = v.iterator();v.add(new Integer(2));
• Following code throws exception in Java:
• No exception raised if one uses Enumeration instead of Iterator– Java language decision, showing that properties
referring to sets of objects are important
![Page 16: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/16.jpg)
MOP Example:Safe Enumeration
• Basic safety property:– If nextElement() invoked on an enumeration
object, then the corresponding collection (vector) is not allowed to change after the creation of the enumeration object
![Page 17: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/17.jpg)
/*@global validation SafeEnum(Vector v, Enumeration+ e) { event create<v,e> : after(call(Enumeration+.new(v, ..))) returning e event updatesource<v> : after(call(* v.add*(..))) \/ … event next<e> : before(call(Object e.nextElement())) ere : create next* updatesource+ next)}@*/
MOP Example:Safe Enumeration
AspectJ code generated from the above: ~700 LOC
![Page 18: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/18.jpg)
MOP Example:Safe Locking Policy
Each lock should be released as many times as it was acquired
![Page 19: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/19.jpg)
/*@method-scoped SafeLock(Lock l) { event acquire<l> : before(call(* l.acquire())) event release<l> : before(call(* l.release()))
cfg : S -> epsilon | S acquire S release}@*/
MOP Example:Safe Locking
![Page 20: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/20.jpg)
MOP Approach to Monitoring
Keep the following distinct and generic:• specification formalisms• event definitions• validation handlers
![Page 21: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/21.jpg)
MOP Distinguished Features: Extensible logic framework
• Observation: no silver-bullet logic for specs• MOP logic plugins (the “How”): encapsulate
monitor synthesizers; so far we have plugins for– ERE (extended regular expressions), PtLTL (Past-time
LTL), FtLTL (Future-time LTL), ATL (Allen temporal logic), JML (Java modeling language), PtCaRet (Past-time Call/Return), CFG (Context-free grammars)
• Generic universal parameters– Allow monitor instances per groups of objects
![Page 22: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/22.jpg)
MOP Distinguished Features: Configurable monitors
Working scope– Check point: check spec at defined place– Method: within a method call– Class: check spec everywhere during obj lifetime– Interface: check spec at boundaries of methods– Global: may refer to more than one object
Running mode– Inline: shares resources with application– Outline: communicates with application via sockets– Offline: generated monitor has random access to log
![Page 23: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/23.jpg)
MOP Distinguished Features: Decentralized Monitoring/Indexing
• The problem: how to monitor a universally quantified specification efficiently!
create<v,e>udatesource<v>next<e>
create next* updatesource+ next
( v,e)
![Page 24: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/24.jpg)
Decentralized Monitoring
Monitor instances(one per parameter instance)
Mp1
Mp2
Mp3
… Mp1000
![Page 25: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/25.jpg)
Indexing …
• The problem: how can we retrieve all needed monitor instances efficiently?
Mp1
Mv,e1
Mv,e2
… Mp1000
udatesource<v>
Naïve implementation very inefficient (both time- and memory-wise)
![Page 26: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/26.jpg)
MOP – Grigore Rosu 26
MOP’s Decentralized Indexing
• Monitors scattered all over the program
• Monitor states piggybacked to object states
• Weak references
SafeEnum events
create<v,e>udatesource<v>next<e>
![Page 27: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/27.jpg)
MOP: Evaluation• More than 100 program-property pairs
– Dacapo benchmark, Tracematches benchmark, Eclipse, …
• Overhead < 8% in most cases; close to hand-optimized
4.52.21.71.52.80.06.54.75.46.74.43.5xalan
0.00.08.05.413.725.40.00.011.344.80.00.0pmd
0.10.00.61.10.00.30.00.00.00.00.00.5lusearch
1.11.72.23.20.00.31.81.20.51.90.21.6luindex
0.20.42.30.00.60.00.30.20.50.80.00.6jython
0.00.01.41.40.00.82.10.01.20.90.03.3hsqldb
0.00.01.00.51.50.80.00.00.01.50.61.2fop
2.42.23.13.01.53.80.53.71.40.02.84.1eclipse
0.00.00.00.50.00.04.83.60.00.00.00.0chart
0.05.80.00.00.00.41.10.00.00.01.50.0antlr
ClosedReaderLeakingSyncHasNextHashMapSafeIteratorSafeEnum
Overhead in % MOP monitors VS. hand-optimized monitors
![Page 28: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/28.jpg)
MOP: Evaluation (cont.)
• Even significantly faster than logic specific solutions
N/A63.520.211.151.2KABCReweave
N/A124.323.921.2438.7KAproveHashSet
N/A15.23.33.39.9KWekaHashtable
N/A4522322101.4KCerRevSimNullTrack
708415091360.19.5KjHotDrawSafeEnum
21933546.6021.1KajHotDrawListener
PQLTracematchesMOPHand Optimize
d
LOCProgramProperty
Results for Tracematches benchmarks, Overhead in %
![Page 29: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/29.jpg)
Overview
• Our Approach
• MOP (University of Illinois at Urbana)
• TEAMS (Qualtech Systems Inc.)
• Project Research Plan
• Conclusion and Future Work
![Page 30: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/30.jpg)
QSI’s TEAMS
• Model-based diagnosis system– TEAMS model = dependency model capturing
relationships: failure modes observable effects
• QSI’s TEAMS Tool Set– TEAMS Designer: help create models– TEAMS-RT: processing data in real time– TEAMATE: infers health status + optimal tests– TEAMS-RDS: remote diagnostic server
![Page 31: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/31.jpg)
TEAMS Designer
• Help users create models– (model can also be imported)– Capture component and data
dependency + other aspects that allow efficient diagnosis
• Model = hierarchical multi-layered directed graph– Node: physical component– Test-point: “observation” node– Edge: cause-effect dependency
![Page 32: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/32.jpg)
Overview
• Our Approach
• MOP (University of Illinois at Urbana)
• TEAMS (Qualtech Systems Inc.)
• Project Research Plan
• Conclusion and Future Work
![Page 33: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/33.jpg)
Project Objectives
1. Develop tools, techniques and ultimately an integrated framework for IVHM system monitoring, control and verification
2. Show that runtime verification and monitoring can play a crucial role in the development of safe, robust, reliable, scalable and operational IVHM systems
![Page 34: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/34.jpg)
Project Plan
• TEAMS: capture system “health”
• MOP: generate and integrate monitors
• Integrated system: check IVHM system at runtime, steering if failures are detected
Steering / Recovery
IVHM System
Model-basedobservation
Temporal behavior monitor
Violation /Validation
Abstract events/statesTEAMS
MOP
![Page 35: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/35.jpg)
What is done: TEAMS side Case study: B-737 Autoland
• With data provided by Celeste M. Belcastro and Kenneth Eure, a model for B-737 is being developed
![Page 36: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/36.jpg)
What is done: MOP side Case study: B-737 Autoland
• Two new logic plugins– Context-free patterns– Past-time LTL with Calls/Returns– (still missing timed logic plugins)
• Improved monitor garbage collection– Current MOP more than an order of
magnitude faster than other RV systems
![Page 37: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/37.jpg)
Overview
• Our Approach
• MOP (University of Illinois at Urbana)
• TEAMS (Qualtech Systems Inc.)
• Project Research Plan
• Conclusion and Future Work
![Page 38: Monitoring IVHM Systems using a Monitor-Oriented Programming Framework](https://reader034.vdocuments.us/reader034/viewer/2022051516/56812f9c550346895d95174c/html5/thumbnails/38.jpg)
Conclusion and Future Work
• Discussed initial steps towards integrated framework for IVHM system monitoring, control and verification– Separation of concerns
• Observation / diagnosis of system “health”• Monitoring of temporal behaviours
• A lot to be done– Complete TEAMS model for B-737 autoland– Automate integration of TEAMS and MOP