modelling and multi-objective optimization of quality attributes in variability-rich software
DESCRIPTION
Modelling and Multi-Objective Optimization of Quality Attributes in Variability-Rich Software. Rafael Olaechea, Steven Stewart, Kryzstof Czarnecki, Derek Rayside University of Waterloo. Customizing Variability-Rich Software. - PowerPoint PPT PresentationTRANSCRIPT
1
Modelling and Multi-Objective Optimization of Quality Attributes in Variability-Rich Software
Rafael Olaechea, Steven Stewart, Kryzstof Czarnecki, Derek Rayside
University of Waterloo
2
Customizing Variability-Rich Software
• Stakeholders need to customize software functionality to their specific needs
• Conflicting Quality Requirements:– Cost– Reliability– Performance– Binary footprint
3
MobilePhone
ConnectivityPasswordProtection
Bluetoothcost = 35
perform= 500cost = 50
perform = 300
cost = 10perform = 20Security = 1
USB Wificost = 85
perform = 725
Mobile Phone Product Line
Variability is modeled explicitly using feature models.Quality Attributes can be considered using attributed feature models.
4
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Software Product Derivation Process
5
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Software Product Derivation Process
6
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Mobile Phone
Connectivity
Password Protection
Bluetooth
cost = 35perform=
500
cost = 50perform
= 300
cost = 10perform = 20Security = 1
USB Wifi
cost = 85perform = 725
Attributed Feature Models with Quality Attributes
Software Product Derivation Process
7
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Mobile Phone
Connectivity
Password Protection
Bluetooth
cost = 35perform=
500
cost = 50perform
= 300
cost = 10perform = 20Security = 1
USB Wifi
cost = 85perform = 725
Attributed Feature Models with Quality AttributesConfigured Product
Mobile Phone
ConnectivityPassword Protection
USB
Software Product Derivation Process
8
Software Product Derivation Process
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Mobile Phone
Connectivity
Password Protection
Bluetooth
cost = 35perform=
500
cost = 50perform
= 300
cost = 10perform = 20Security = 1
USB Wifi
cost = 85perform = 725
Attributed Feature Models with Quality Attributes
Configured Product(s)
Mobile Phone
ConnectivityPassword Protection
USB
Add one or more Objectives
9
Software Product Derivation Process
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Mobile Phone
Connectivity
Password Protection
Bluetooth
cost = 35perform=
500
cost = 50perform
= 300
cost = 10perform = 20Security = 1
USB Wifi
cost = 85perform = 725
Attributed Feature Models with Quality Attributes
Configured Product(s)
Mobile Phone
ConnectivityPassword Protection
USB
Add one or more Objectives
Multi-objectiveOptimization
10
Software Product Derivation Process
VariabilityModelling Measurement
ProductDerivation
Domain Analysis
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Feature Model
Mobile Phone
Connectivity
Password Protection
Bluetooth
cost = 35perform=
500
cost = 50perform
= 300
cost = 10perform = 20Security = 1
USB Wifi
cost = 85perform = 725
Attributed Feature Models with Quality Attributes
Configured Product(s)
Mobile Phone
ConnectivityPassword Protection
USB
Add one or more Objectives
Multi-objectiveOptimization
Optimally
11
Mobile Phone
ConnectivityPassword Protection
BluetoothUSB Wifi
Optimally Configured Software Products
Pareto-Front of Optimal Products: A product is Pareto-Optimal if no other product is better than it in at least one quality attribute and at least as good as it in all other quality attributes
30 50 70 90 110 130 150 170 190250
450
650
850
1050
1250
1450
1650Performance vs. Cost
Cost
Perf
orm
ance
12
Clafer Language to Express Attributed Feature Models
• Lightweight Textual Modelling Language• Combines feature modelling with:– Types for each feature– Inheritance– Modularizing information about each feature
• Aggregation functions to define product-level quality attributes
• Supports top-level goals
13
Expressing Attributed Feature Models in ClaferMoo
abstract Featureperformance : integercost : integer
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
14
Expressing Attributed Feature Models in ClaferMoo
abstract Featureperformance : integercost : integer
abstract SecurityFeature : Featuresecurity : integer
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
15
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneMobile Phone
Connectivity
Password Protection
BluetoothUSB
16
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor ConnectivityMobile Phone
Connectivity
Password Protection
BluetoothUSB
17
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
18
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
19
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
20
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer [total_performance = sum Feature.performance ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
21
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer [total_performance = sum Feature.performance ]total_cost : integer
[total_cost= sum Feature.cost ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
22
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer [total_performance = sum Feature.performance ]total_cost : integer
[total_cost= sum Feature.cost ]total_security : integer [ total_security = sum SecurityFeature .security]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
23
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer[total_performance = sum Feature.performance ]
..total_security : integer
[ total_security = sum SecurityFeature .security]
MyPhone : MobilePhone
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
24
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer[total_performance = sum Feature.performance ]
..total_security : integer
[ total_security = sum SecurityFeature .security]
MyPhone : MobilePhone[ PasswordProtection ]
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
25
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer[total_performance = sum Feature.performance ]
..total_security : integer
[ total_security = sum SecurityFeature .security]
MyPhone : MobilePhone[ PasswordProtection ]
<< max MyPhone.total_performance >><< min MyPhone.total_cost >>
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
26
Expressing Attributed Feature Models in ClaferMoo
abstract MobilePhoneor Connectivity
Bluetooth : Feature[performance = 300 cost = 50 ]
USB : Feature[performance = 500 cost = 35 ]
PasswordProtection : SecurityFeature ? [ security = 1 performance = 20 cost = 10 ]
total_performance : integer[total_performance = sum Feature.performance ]
..total_security : integer
[ total_security = sum SecurityFeature .security]
MyPhone : MobilePhone[ PasswordProtection ]
<< max MyPhone.total_performance >><< min MyPhone.total_cost >>
Mobile Phone
Connectivity
Password Protection
BluetoothUSB
MyPhone : MobilePhoneConnectivity
USBPasswordProtectiontotal_cost = 45total_performance = 525total_security = 1
ClaferMoo Implementation Architecture
Conjunctive Normal Form + Objectives
ClaferTranslator
Alloy +Moolloy
Extension
Kodkod +Moolloy
Extension
SAT
Clafer Feature Models+ Quality Objectives
Alloy+ Objectives
28
Methodology and Research Questions
• Can real-world attributed Feature Models be expressed in ClaferMoo?– Mechanical translation of Attributed feature models into
Clafer.• How scalable is the current implementation of
ClaferMoo ?– Measure time to compute Pareto-Front of optimal
products for partial configurations.• Used dataset from 9 published attributed feature
models by Siegmund, Apel et al
29
Evaluating Scalability of Implementation• For each attributed feature model:– Randomly generate a set of 10
satisfiable partially-configured products• Feature are either:
– selected– not selected – unconstrained
30
Evaluating Scalability of Implementation• For each attributed feature model:– Randomly generate a set of 10
satisfiable partially-configured products• Feature are either:
– selected– not selected – unconstrained
– Optimize over quality attributes and measure running times to compute the Pareto-Front, repeating 20 times
MyPhone : MobilePhone[ PasswordProtection !USB ]
<< max MyPhone.total_performance >><< min MyPhone.total_cost >>
31
Experimental Evaluation on Published Models
Variant-Rich Software
Features Quality Attributes Objectives
Berkeley DB II 12 Footprint, Price, Reliability
Prevayler 5 Footprint
ZipMe 8 Footprint
Berkeley DB I 9 Footprint
Apache 9 Performance
PKJab 11 Footprint
LinkedList 18 Footprint
SQLite 85 Footprint
UMLViolet 100 Footprint
32
Experimental Evaluation on Published Models
Variant-Rich Software
Pareto Front Size
Time to compute Pareto-Front
Median Pareto Front Size for Partial Configurations
Median Time to Compute PF for Partial Configurations
Berkeley DB II 6 3.8 min. 2.5 46 s.
Prevayler 1 0.1 s. 1 59 ms.
ZipMe 2 0.1 s. 1.5 50 ms.
Berkeley DB I 1 0.8 s. 1 0.4 s.
Apache 2 0.2 s. 1 90 ms.
PKJab 2 0.1 s. 1 78 ms.
LinkedList 2 4.6 s. 1 0.7 s.
SQLite N/A > 11 min N/A > 18 min
UMLViolet Timeout N/A N/A N/A
33
Limitations and Future Work
• Limitations– Most models found in published literature were single-
objective– Feature Interactions were not considered in our
experiments– Method scales to small to medium variability models
• Future Work– Will test different back-ends for multi-objective reasoning– Plan on applying to a dataset to be provided by our
industrial partner
34
Conclusions
• Multi-objective optimization and explicitly modelling quality attributes helps stakeholders see the different alternatives
• Language combining inheritance and types with feature models helps represent quality attributes and optimize over them