goal models framework monitoring and diagnosing · adaptive monitoring: can adjust monitoring...
TRANSCRIPT
![Page 1: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/1.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Monitoring and DiagnosingSoftware Requirements
(Wang, McIlraith, Yu, Mylopoulos)
presented by Michalis Famelis
January 8, 2009
![Page 2: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/2.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
A framework for monitoring and diagnosing failures of asoftware system to fulfil its requirements.
![Page 3: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/3.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
• Monitor software for requirements compliance
• Logs
• Goal models
• Denial of goals
• Diagnosis as a SAT problem
• Application to SOA systems
![Page 4: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/4.jpg)
Goal Models
![Page 5: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/5.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Goal models
• Obtained by requirements analysis or by reverseengineering
• We assume the existence of traceability links
• Graphs with AND and OR decompositions of goals intosubgoals and tasks
• Also contain, additional contribution links(+ + S ,−− S ,+ + D,−− D,++,−−)
• Annotated with preconditions, effects (postconditions)
• and monitoring switches
![Page 6: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/6.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Example: Squirrel Mail
![Page 7: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/7.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Squirrel Mail Annotations
![Page 8: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/8.jpg)
Framework Overview
![Page 9: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/9.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Overview
Adaptive monitoring: Can adjust monitoring granularityaccording to diagnosis using the monitoring switches
![Page 10: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/10.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Monitoring
• Monitoring of an instrumented system for a number ofsessions
• Logging of the truth values of preconditions and effects
• Logging the occurence of a task (predicate occa(αi , t))
![Page 11: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/11.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Example log for Squirrel Mail
URLEntered(1), occa(a1, 2), correctFormLoaded(3),¬wrongIMAP(4), occa(a2, 5), correctKeyEntered(6),
occa(a3, 7), occa(a4, 8), occa(a5, 9),¬webmailStarted(10), occa(a7, 11),¬emailSent(12).
![Page 12: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/12.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
The formula
• Diagnosis reduced to the satisfiability of a propositionalformula Φ
• Φ = Φlog ∧ Φdeniability ∧ ΦgoalModel
• Φlog encodes the log data as mentioned earlier
• ΦgoalModel encodes the goal model (decompositions andcontribution links)
• Φdeniability encodes how tasks or goals are denied withregard to preconditions and effects
• The formula can be extended with ΦdomainConstraints
![Page 13: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/13.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Encoding Deniability
• Task/Goal Denial“If a task/goal occurence was logged and either itsprecondition didn’t hold before it or its effect didn’thappen after it, it is denied”
• Task/Goal Session Denial“If a task/goal is denied once, it is denied for the wholesession”
• Explanation closure (fluents)“If the truth value of a literal is changed at some point,then some task/goal that has it as an effect must haveoccured”
![Page 14: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/14.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
About Diagnoses
• Diagnosis: a set D of deniability predicates over all thegoals and tasks in the goal model, such that D ∪ Φ issatisfiable
• Core Diagnosis: a set CD of deniability predicates overall the tasks in the goal model, such that CD ∪ Φ issatisfiable
• Participating Diagnostic Component (PDC): adeniability predicate over some task in the goal modelsuch that PDC ∪ Φ
![Page 15: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/15.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Algorithms
• 2 alternative algorithms to encode an annotated goalmodel into a propositional formula, with (A2) and without(A1) log preprocessing
• Log preprocessing is a scalability optimization to avoidexponential growth of Φ by encoding goal/task denialaxioms only for timesteps that the goal/task actuallyoccurs
• 2 alternative algorithms for doing diagnosing, using a SATsolver
• The first algorithm (A3) finds all Core Diagnoses• The other (A4) is more scalable because it only bothers to
find all Participating Diagnostic Components
![Page 16: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/16.jpg)
The Framework in Action
![Page 17: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/17.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Notes on Implementation
• Implemented in Java (11 classes, roughly 5 KLOC)
• SAT4J solver as a java library
• Implementation of monitoring specification andinstrumentation code with Aspect Oriented Programming(AspectJ)
• Goal Driven Instrumentation:• “what” to monitor from the goal model• “where” to insert the monitors from the traceability links
![Page 18: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/18.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Contribution Links Experiment
• Aim to compare the performance of A3 and A4
• For a denied goal made of n tasks, A3 can return up to 2n
Core Diagnoses, A4 only up to n Participating DiagnosticComponents
• But the presence of contribution links, as they act asconstraints that imporve the efficiency of the SAT solver
• It was confirmed by experiment that with morecontribution links the performance gap between A3 andA4 became smaller
![Page 19: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/19.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
The ATM simulation
![Page 20: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/20.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Experimenting with the ATMsimulation
• A first set of experiments dealt with the tradeoff betweenmonitoring granularity and diagnostic precision
• A second set explored the issues of scalability of theframework for large goal models
• For both sets, A4 (PDC) rather A3 was used
• The optimized encoding algorithm (A2) was used for thefirst set and both encoding algorithms for the second set
![Page 21: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/21.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
First set of ATM experiments
• 5 experiments to monitor increasingly many elements ofthe goal model, ie increasing monitoring granularity.
• The experiment showed that the number of PDC returnedby the diagnostic component is inversely proportional tomonitoring granularity
• Also, with greater granularirty the overall load is increased
• But the overall time to diagnose may not increase (morefine grained, less PDCs to find, whereas more coarse, morePDCs to report)
• Benefit by being able to make better diagnoses
![Page 22: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/22.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Second set of ATM experiments
• 20 experiments to assess the scalability of the framework
• Larger models were obtained by cloning the ATM goalmodel
• The experiments were run with and without logpreprocessing for increasingly large goal models.
• Without log preprocessing, the time to encode anddiagnose increased exponentially to the size of the GM
• With log preprocessing, the framework scaled well, (almostlinearly)
![Page 23: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/23.jpg)
Application to SOA
![Page 24: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/24.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Service Oriented Architecture
• A three layer software architecture
• Business Process Layer (top): treats services as blackboxes
• Component Layer (middle): gives the business logic of theBPL services
• Infrastructure Layer (bottom): contains the back-endinfrastructure
![Page 25: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/25.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
ATM SOA global goal model
![Page 26: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/26.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Monitoring and DiagnosingMulti-layer systems
• Hierarchical monitoring and diagnosis
• We can monitor on each isolated layer or as a global model
• As higher level depend on the correct functioning of lowerleverls, we can treat leafs as black boxes that decomposeto root level goals in lower levels
• Tradeoff between monitor granularity and diagnosticprecision as noted before
![Page 27: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/27.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Evaluation
• 20 experiment sets of 3 experiments (for the 3 SOA levels)
• Each set dealt with a larger global model (more BusinessProcesses)
• Used algorithms A2 and A4
• The experiments showed that the framework scaled well(almost linearly)
• Also, the experiments confirmed again thegranularity/precision tradeoff: the framework was mostefficient at the BPL and less efficient at the IL
![Page 28: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/28.jpg)
Conclusion
![Page 29: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/29.jpg)
introduction
goal models
frameworkoverview
in action
application toSOA
conclusion
Conclusions
• Robust, scalable framework for monitoring and diagnosisof the fulfullment of the requirements of a system, basedon its goal model
• A prototype that can be extended to be used forindustrial-scale software systems
• The framework can be extended to also handlenon-functional requirements, failures by wrong domainassumptions and malicious attacks and different levels oftraceability granularity, or the non-existence of traceability
![Page 30: goal models framework Monitoring and Diagnosing · Adaptive monitoring: Can adjust monitoring granularity according to diagnosis using the monitoring switches. introduction goal models](https://reader034.vdocuments.us/reader034/viewer/2022051911/6001bd2061aa2758be76cc03/html5/thumbnails/30.jpg)
Questions?