![Page 1: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/1.jpg)
1
Sustainability Evaluation
of Software Architectures
Heiko Koziolek Industrial Software Systems
ABB Corporate Research, Germany
![Page 2: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/2.jpg)
2 2
![Page 3: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/3.jpg)
3
• Technology – GUI: Microsoft Visual Basic
– Middleware: Classic OPC
– OS: Windows XP
• Functionality (select.) – Basic alarm management
– No system integration
– No online updates
• Code & Documentation – Defined APIs and layers
– Design rationale present
– Up-to-date documentation
• Technology – GUI: Microsoft WPF
– Middleware: OPC UA
– OS: Windows 7
• Functionality (select.) – Rich alarm management
– Multi-system integration
– Online updates
• Code & Documentation – Some layer violations
– Some design rationale lost
– Outdated documentation
2004 2011
![Page 4: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/4.jpg)
Measuring sustainability?
4
![Page 5: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/5.jpg)
5
83%
70%
54%
40%
17%
16%
15%
11%
5%
Experience
Prototyping
Scenarios
Checklists
Simulation
Questionaires
Metrics
Others
Math. models
[Source: AliBabar & Gorton 2009, survey with 88 architects]
State of practice: Techniques for architecture evaluation
![Page 6: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/6.jpg)
Research Questions
• RQ1: Scenario-based methods & sustainability evaluation?
• RQ2: What kind of architecture-level metrics?
• RQ3: Implications for industry & research?
6
![Page 7: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/7.jpg)
Search
• 14 conferences, 12 journals, 10 books
• Included only methods/metrics
from renowned venues
• Excluded dormant or unvalidated
methods/metrics
7
![Page 8: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/8.jpg)
Research Questions
• RQ1: Scenario-based methods & sustainability evaluation?
• RQ2: What kind of architecture-level metrics?
• RQ3: Implications for industry & research?
8
![Page 9: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/9.jpg)
9
Scenario-based Methods
• SAAM (5%)
• ATAM (5%)
• ARID
• SAAMER
• PASA
• SBAR
• SAAMCS
• ISAAMCR
• ALMA (1%)
• ESAAMI
• SAAF
• ASAAM
• SALUTA
• SACAM
• DOSAM
• SBAR
Practitioners
applying
the method
[AliBabar &
Gorton2009]
dormant
![Page 10: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/10.jpg)
Sustainability Evaluation Criteria
Change scenario specification
Ripple effect analysis
Variation point analysis
Architecture improvement
Tradeoff analysis Existing tools &
artifacts
![Page 11: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/11.jpg)
Sustainability Evaluation Criteria
Change scenario specification
Ripple effect analysis
Variation point analysis
Architecture improvement
Tradeoff analysis Existing tools &
artifacts
![Page 12: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/12.jpg)
12
Sustainability evaluation ATAM ALMA
Change
scenario spec.
Ripple effect
analysis
Variation point
analysis
Architecture
improvement
Tradeoff
analysis
Existing
artifacts/tools
Return on
Investment
![Page 13: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/13.jpg)
13
Sustainability evaluation ATAM ALMA
Change
scenario spec.
Quality attribute
scenario template
Top-down, bottom-up,
no template
Ripple effect
analysis
Manual, based on
experience
Manual, based on
experience
Variation point
analysis
n/a n/a
Architecture
improvement
Modifiability Tactics,
Styles
n/a
Tradeoff
analysis
Utility tree with
stakeholder preferences
n/a
Existing
artifacts/tools
No support No support
Return on
Investment
Cost: 30-70 person days
Benefit: not quantified
Cost: unknown
Benefit: not quantified
![Page 14: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/14.jpg)
Findings RQ1: Scenarios
• ATAM: more features
• ALMA: more specific for sustainability
• Manual ripple effect analysis
• No reverse engineering
• No tool integration
• Return on Investment unknown
• In practice: use best of both methods
14
![Page 15: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/15.jpg)
Research Questions
• RQ1: Scenario-based methods & sustainability evaluation?
• RQ2: What kind of architecture-level metrics?
• RQ3: Implications for industry & research?
15
![Page 16: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/16.jpg)
Architectural metrics categories
Similarity of Purpose
Encapsulation Independent
Compil. & Testing
Acyclic Dependencies
Size 16
[Sarkar2007]
![Page 17: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/17.jpg)
Architectural metrics categories
Similarity of Purpose
Encapsulation Independent
Compil. & Testing
Acyclic Dependencies
Size 17
[Sarkar2007]
![Page 18: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/18.jpg)
18
Architecture Metrics
Similarity of
Purpose
Encapsulation
Independent
Compilation
& Testing
Acyclic
Dependencies
Size
![Page 19: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/19.jpg)
19
Architecture Metrics
Similarity of
Purpose
Concept Domination Metric, Concept Coherency Metic,
API Function Usage Index, Concern Diffusion over Arch. Components,
Concern Scope, Concern Overlap
Encapsulation Ratio of Cohesive Interactions, Import Coupling, Export Coupling,
Modularization Quality, Afferent Couplings, Efferent Couplings,
Component-level Interlacing Between Concerns,
Lack of Concern-based Cohesion, Module Interaction Index,
Non-API Function Closedness Index, Implicit Dependency Index,
Base class fragility index, Inheritance-based inter-module coupling,
Not-programming-to-interfaces Index, Association-induced coupling,
State Access Violation Index, Entropy of an architectural slicing,
Architecture Slicing Cohesion, Decision Volatility
Independent
Compilation
& Testing
Cumulative Component Dependency, Average Cumulative Component
Dependency, Normalized Cumulative Component Dependency,
Coupling of a module, Intra-module coupling of a module, Cohesion of a
module, Abstractness, Instability, Distance from the Main Sequence,
Module Interaction Stability Index, Normalized Testability Dependency
Metric, Plugin Pollution Index, Change impact, Independence Level
Acyclic
Dependencies
Package Dependency Cycles, Cyclic Dependencies Index, Layer
Organization Index, Excessive Structural Complexity
Size Module Size Boundness Index, Module Size Uniformity Index
![Page 20: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/20.jpg)
20
Architecture Metrics
Similarity of
Purpose
Concept Domination Metric, Concept Coherency Metic,
API Function Usage Index, Concern Diffusion over Arch. Components,
Concern Scope, Concern Overlap
Encapsulation Ratio of Cohesive Interactions, Import Coupling, Export Coupling,
Modularization Quality, Afferent Couplings, Efferent Couplings,
Component-level Interlacing Between Concerns,
Lack of Concern-based Cohesion, Module Interaction Index,
Non-API Function Closedness Index, Implicit Dependency Index,
Base class fragility index, Inheritance-based inter-module coupling,
Not-programming-to-interfaces Index, Association-induced coupling,
State Access Violation Index, Entropy of an architectural slicing,
Architecture Slicing Cohesion, Decision Volatility
Independent
Compilation
& Testing
Cumulative Component Dependency, Average Cumulative Component
Dependency, Normalized Cumulative Component Dependency,
Coupling of a module, Intra-module coupling of a module, Cohesion of a
module, Abstractness, Instability, Distance from the Main Sequence,
Module Interaction Stability Index, Normalized Testability Dependency
Metric, Plugin Pollution Index, Change impact, Independence Level
Acyclic
Dependencies
Package Dependency Cycles, Cyclic Dependencies Index, Layer
Organization Index, Excessive Structural Complexity
Size Module Size Boundness Index, Module Size Uniformity Index
![Page 21: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/21.jpg)
Example: Module Interact. Idx
𝒎
API Non-API
𝑀𝐼𝐼 𝒎 = 𝐾𝑒𝑥𝑡(𝑓
𝑎)𝑓𝑎∈ 𝑓1 𝑎 … 𝑓𝑛
𝑎
𝐾𝑒𝑥𝑡(𝑚)
=1 + 1
3 ≈ 0.67
call call call
21
[Source: Sarkar2007 IEEE TSE]
![Page 22: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/22.jpg)
Example: Comparing Systems
22
[Source: Sarkar2007 IEEE TSE]
![Page 23: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/23.jpg)
Findings RQ2
• >40 architecture-level source code metrics in literature
• Measures for architecture and modularization quality
• Success factors for application: – Mix of metrics
– Combination with class-level / process metrics
• Limitations: – Empirical validation
– Tool support
23
![Page 24: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/24.jpg)
Research Questions
• RQ1: Scenario-based methods & sustainability evaluation?
• RQ2: What kind of architecture-level metrics?
• RQ3: Implications for industry & research?
24
![Page 25: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/25.jpg)
Findings RQ3: Implications
• More empirical research
• Method integration
• Effective formal models
• Codify experiences
• Explore other
approaches (simulation?)
25
![Page 26: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/26.jpg)
26
Conclusions
• Sustainability Evaluation of Architectures – change scenarios:
basic support for sustainability criteria
– architectural code metrics: measure modularization, lack empirical evidence
• Research opportunities – combined methods
– empirical research
![Page 27: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/27.jpg)
27
![Page 28: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/28.jpg)
28
Existing surveys
• Surveys on scenario-
based method
– Dobrica 2002
– Ali Babar 2004
– Kazman 2005
– Breivold 2010
– ...
• Surveys / papers
on code metrics
– McCabe 1976
– Halstead 1977
– Chidamber 1994
– Riaz 2009
– ...
28
![Page 29: Sustainability Evaluation of Software Architectures: A Systematic Review](https://reader033.vdocuments.us/reader033/viewer/2022051109/547b8a185906b5a4798b45fe/html5/thumbnails/29.jpg)
ATAM ALMA
Goals Sensitivity & Tradeoff
Analysis
Change impact analysis,
maint. effort prediction
Architectural
description
Process, data-flow, uses,
physical & module view
Any
Process
support
Comprehensive Limited
Tool support
n/a n/a
Knowledge
repository
Recommended n/a
Validation > 20 industrial case
studies documented
7 industrial case studies
(before 2004) 29