![Page 1: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/1.jpg)
Software: Good, Bad, or Just a Matter of Taste?
Arie van Deursen Delft University of Technology, The Netherlands
SBES 2012, Natal, Brazil, 27 September 2012
1
![Page 2: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/2.jpg)
Acknowledgements
• Marcio Delamaro & CBSoft organizers
• Joint work with Eric Bouwers & Joost Visser Software Improvement Group
• Co-workers TU Delft Software Engineering Research Group
2
![Page 3: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/3.jpg)
3
A. van Deursen and T. Kuipers. Source-Based Software Risk Assessments. ICSM, 2003.
www.sig.eu
Collect detailed technical findings about software-intensive systems
Translate into actionable information for high-level management
Using methods from academic and self-funded research
![Page 4: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/4.jpg)
Focus on Software Architecture
The organizational structure of a software system including components, connectors,
constraints, and rationale
-- Kogut & Clemens, 1994
“Architectures allow or preclude nearly all of a system’s quality attributes.”
-- Clements et al, 2005
4
![Page 5: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/5.jpg)
Early versus Late Evaluations
• Early: Making design decisions
– Design space exploration
– What-if analysis
• Late: Analyzing as-implemented architecture
– Drift & erosion
– Actual suitability
5
van Deursen, Hofmeister, Koschke, Moonen, Riva: Symphony: View-Driven Software Architecture Reconstruction. WICSA 2004
L. Dobrica and E. Niemela. A survey on software architecture analysis methods. IEEE Transactions on Software Engineering 2002.
![Page 6: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/6.jpg)
The Software Risk Assessment Process
6
ICSM 2009
![Page 7: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/7.jpg)
Lord Kelvin
Meten = Weten
Medição = Conhecimento
To measure = To know
“when you cannot measure it, (…), your knowledge is of
a meagre and unsatisfactory kind”
7
![Page 8: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/8.jpg)
8
Country Size Brazil - NL
![Page 9: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/9.jpg)
9
Population Brazil - NL
![Page 10: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/10.jpg)
10
Gross Domestic
Product
![Page 11: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/11.jpg)
11
GDP Per Capita
![Page 12: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/12.jpg)
12
Brazil – NL Football Wins
4-3
![Page 13: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/13.jpg)
13
IT Labor Force?
![Page 14: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/14.jpg)
14
IT Profitability?
![Page 15: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/15.jpg)
15
IT Quality??
![Page 16: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/16.jpg)
Software metrics for project management? Beware!
E. Bouwers, J. Visser, and A. van Deursen. Getting what you Measure Communications of the ACM, May 2012
16
![Page 17: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/17.jpg)
Pittfall 1:
Metric in a bubble:
No context, no goal
17
![Page 18: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/18.jpg)
Pittfall 2:
Metrics galore: Measuring everything
18
![Page 19: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/19.jpg)
Pittfall 3:
One-track metric: Narrow-minded
19
![Page 20: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/20.jpg)
Pittfall 4:
Treating the metric: Fighting symptoms
20
![Page 21: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/21.jpg)
21
ISO25010 -- The Many Faces of Software Quality
![Page 22: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/22.jpg)
22
What to measure?
Heitlager, Kuipers, Visser. A Practical Model for Measuring Maintainability. QUATIC 2007 J. Visser. SIG/TÜViT Evaluation Criteria Trusted Product Maintainability:
Guidance for producers Version 4.2, 2012
![Page 23: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/23.jpg)
Metric Value: Good or Bad?
• Benchmark of many industrial systems
– All metrics of interest collected
• Determine thresholds based on percentage of systems with given value.
– E.g.: 90% of systems have McCabe <= 15
– McCabe > 15 suggests high risk.
23
Tiago L. Alves, Christiaan Ypma, Joost Visser. Deriving metric thresholds from benchmark data. ICSM 2010.
![Page 24: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/24.jpg)
2009: Re-assessing Architectural Properties Study of 40 risk assessments Rethinking architectural properties Outcome: Metrics refinement wanted
24
Eric Bouwers, Joost Visser, Arie van Deursen: Criteria for the evaluation of implemented architectures. ICSM 2009
![Page 25: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/25.jpg)
Information Hiding
25
Things that change at the same rate belong together
Things that change quickly should be insulated from things that change slowly
Kent Beck. Naming From the Outside In. Facebook Blog Post, September 6, 2012.
![Page 26: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/26.jpg)
Encapsulation Metrics?
Which software architecture metrics can serve as indicators for the success of encapsulation of
an implemented software architecture?
26
Eric Bouwers, Arie van Deursen, and Joost Visser. Quantifying the Encapsulation of Implemented Software Architectures
Technical Report TUD-SERG-2011-033-a, Delft University of Technology, 2012
![Page 27: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/27.jpg)
Approach
• Identification of candidate metrics
• Quantitative approach (repository mining): – Which metric is the best predictor of good
encapsulation?
• Qualitative approach: – Is the selected metric useful in a late architecture
evaluation context?
27
![Page 28: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/28.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
Module (size) Component Module dependency
Lifted (comp) dependency
C1 C2
C3
28
![Page 29: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/29.jpg)
Literature Study: Candidate Metrics
29
![Page 30: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/30.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Commit in version repository results in change set 30
![Page 31: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/31.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set I: modules { A, C, Z } Affects components C1 and C3 31
![Page 32: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/32.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set II: modules { B, D, E } Affects components C1 only
Local change 32
![Page 33: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/33.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set III: modules { Q, R, U } Affects components C2 only
Local change 33
![Page 34: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/34.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set IV: modules { S, T, Z } Affects components C2 and C3
Non-Local change 34
![Page 35: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/35.jpg)
Observation 1: Local Change-Set Series are Good
• Combine change sets into series
• The more local changes in a series, the better the encapsulation worked out.
35
![Page 36: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/36.jpg)
Observation 2: Metrics may change too
• A change may affect the value of the metrics.
• Cut large set of change sets into sequence of stable change-set series.
36
![Page 37: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/37.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set I: modules { A, C, Z } Affects components C1 and C3 37
![Page 38: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/38.jpg)
U Z
C
E
A B
R
X
S
Y
P
T
Q
D
C1 C2
C3
Change set I: modules { A, C, Z } The Change Set may affect metric outcomes!! 38
![Page 39: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/39.jpg)
Solution: Stable Period Identification
39
![Page 40: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/40.jpg)
Approach
• Identify 10 long running open source systems
• Determine metrics on monthly snapshots
• Determine stable periods per metric: – Metric value
– Ratio of local change in this period
• Compute (Spearman) correlations [0, .30, .50, 1]
• Assess significance (p < 0.01)
• [ Assess project impact ]
• Interpret results
40
![Page 41: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/41.jpg)
Systems Under Study
41
![Page 42: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/42.jpg)
Results
42
![Page 43: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/43.jpg)
Best Indicator for Encapsulation: Percentage of Internal Code
Module types:
1. Internal
2. Inbound
3. Outbound
4. Transit
43
Eric Bouwers, Arie van Deursen, Joost Visser. Dependency Profiles for Software Architecture Evaluations. ICSM ERA, 2011.
![Page 44: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/44.jpg)
Threats to Validity
Construct validity • Encapsulation ==
local change? • Commit == coherent? • Commits too small? • Commits too large? • Architectural (meta)
model appropriate?
External • Open source, Java • IC behaves same on other
technologies Internal validity • Stable periods: Length, nr,
volume • Monthly snapshots Reliability • Open source systems • All data available
44
![Page 45: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/45.jpg)
Shifting paradigms
• Statistical hypothesis testing:
Percentage of internal change is valid indicator for encapsulation
• But is it of any use?
• Can people work with?
• Shift to pragmatic knowledge paradigm
45
![Page 46: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/46.jpg)
Software Risk Assessments
46
![Page 47: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/47.jpg)
Using “Percentage of Internal Change” For Evaluation Purposes
Setting
• SIG has included %IC in its revised maintainability model
• Used in risk assessments and monitors of many industrial systems
• Frequent discussion between clients, consultants, suppliers, and analysts.
47
![Page 48: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/48.jpg)
Evaluation Approach
• SIG consultants and analysts report any IC-related experience to investigator
• Collect “stories” of actual use in (anonymized) memos.
– 6 months, 50 memos so far
• Analyze using grounded theory
48
Michaela Greiler, Arie van Deursen, Margaret-Anne D. Storey: Test confessions: A study of testing practices for plug-in systems. ICSE 2012: 244-25
![Page 49: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/49.jpg)
Preliminary Findings (1)
[ Still collecting, analyzing, and digesting ]
• Metrics added require componentization / architectural view. – Documenting it is a useful process
– Low %IC values trigger discussion
• Important to see metric in full suite (avoid “one-track method”)
49
![Page 50: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/50.jpg)
Preliminary Findings (2)
Different “architectures” exist:
1. In the minds of the developers
2. As-is on the file system
3. As used to compute the metrics
• These should be the same!
• Should [3] capture [1] or [2]?
50
![Page 51: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/51.jpg)
Preliminary Findings (3)
• Is low %IC fixable?
• Requires (substantial) refactoring:
– Smaller components and interfaces,
– Fewer dependencies.
• Consultant’s wish list: “what-if analysis”:
– See effects of changing architecture
• Major refactoring for one of the systems monitored planned based on %IC insights
51
![Page 52: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/52.jpg)
Encapsulation Can be Measured!
Module types:
1. Internal
2. Inbound
3. Outbound
4. Transit
52
And doing so, leads to meaningful discussions.
![Page 53: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/53.jpg)
Software: Good, Bad, or Just a Matter of Taste?
Ultimately a Matter of Engineering Tradeoffs
Yet there is a difference between right and wrong
53
![Page 54: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/54.jpg)
Software Metrics Matter
• A key tool for informed decision making
• Benchmarking, monitoring, and interpretation are key
• Pitfalls are plentiful, in research as in practice
54
![Page 55: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/55.jpg)
(Open Source) Repositories: A Research Gold Mine
• Today: Metrics validation.
• GitHub! GitHub! GitHub!
• Relentless identification of threats to validity esstential: “Ameaças à validade”!
55
![Page 56: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/56.jpg)
Paradigm: Pragmatism
• Evaluate based upon the possibilities of action
• Calls for rigorous qualitative studies capturing reality in rich narratives
• Rich palette of empirical methods needed
56
![Page 57: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/57.jpg)
The Final: July 13, 2014
57
![Page 58: Software: Good, Bad, or Just a Matter of Taste?arie/papers/presentations/sbes2012.pdf · Just a Matter of Taste? ... Eric Bouwers & Joost Visser Software Improvement Group ... Software:](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a84db057f8b9ac96a8bf5dd/html5/thumbnails/58.jpg)
The Final: July 13, 2014
58
Brazil vs Netherlands