ranking refactoring suggestions based on historical volatility
DESCRIPTION
Ranking Refactoring Suggestions based on Historical Volatility Nikolaos Tsantalis Alexander Chatzigeorgiou University of Macedonia Thessaloniki, Greece. 15th European Conference on Software Maintenance and Reengineering ( CSMR 2011). Design Problems. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/1.jpg)
Ranking Refactoring Suggestions
based on Historical Volatility
Nikolaos Tsantalis Alexander Chatzigeorgiou
University of Macedonia
Thessaloniki, Greece
15th European Conference on Software Maintenance and Reengineering (CSMR 2011)
![Page 2: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/2.jpg)
Design Problems
non-compliance with design principles
excessive metric values
lack of design patterns
violations of design heuristics
Fowler’s bad smells
![Page 3: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/3.jpg)
Design Problems … can be numerous
0102030405060708090
1.3 1.3.1 1.3.2 1.3.3 1.3.4 1.3.5 1.4 1.4.1 1.4.2 1.4.3
Num
ber o
f Sm
ells
Versions
JFlex
Long Method
Feature Envy
State Checking
050
100150200250300350400
Num
ber o
f Sm
ells
Versions
JFreeChart
Long Method
Feature Envy
State Checking
![Page 4: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/4.jpg)
Motivation
• Are all identified design problems worrying?
• Example: Why would it be urgent to improve a method suffering from Long Method if the method had never been changed?
• Need to define (quantify) the urgency to resolve a problem
• One possible source of information: Past code versions
•Underlying Philosophy: code fragments that have been subject to maintenance tasks in the past, are more likely to undergo changes → refactorings involving the corresponding code should have a higher priority.
![Page 5: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/5.jpg)
Goal
• To rank refactoring suggestions based on the urgency to resolve the corresponding design problems
• The ranking mechanism should take into account:
• the number of past changes
• the extent of change
• the proximity to the current version
![Page 6: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/6.jpg)
Inspiration
![Page 7: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/7.jpg)
Forecasting in Financial Markets vs. Software
Financial Markets
• Trends in volatility are more predictable than trends in prices
• Volatility is related to risk and general stability of markets
• defined as the relative rate at which prices move up and down
• time: trading days
Software – Preventive Maintenance
• Risk lies in the decision to invest on resolving design problems
•volatility based on changes involving code affected by a smell
• time: successive software versions
![Page 8: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/8.jpg)
Code Smell Volatility
software versionsi-1 i+1i
transitioni transitioni+1
extent of changei-1,i
extent ofchangei,i+1
volatilityi+1
σ
![Page 9: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/9.jpg)
Forecasting Models
• Random Walk (RW)
tt 1ˆ
• Historical Average (HA)
t
iit t 1
11
ˆ
• Exponential Smoothing (ES)
ttt a ˆ)1(ˆ 1
• Exponentially-Weighted Moving Average
t
ijttt t
a1
111
ˆ)1(ˆ
![Page 10: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/10.jpg)
Examined Smells
• Detection tool: JDeodorant
• Identified smells:
• Long Method
• Feature Envy
• State Checking
![Page 11: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/11.jpg)
Long Method
int i; int product = 1; for(i = 0; i < N; ++i) { product = product *i; }
System.out.println(product);
Pieces of code with large size, high complexity and low cohesion
int i; int sum = 0; for(i = 0; i < N; ++i) { sum = sum + i; } System.out.println(sum);
![Page 12: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/12.jpg)
What to look for
The presence of Long Method implies that it might be difficult to maintain the method
→ perform refactoring if we expect that the intensity of the smell will change
Previous versions: detect changes in the implementation of the method that affect the intensity of the smell
change
![Page 13: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/13.jpg)
Long Method
int i; int sum = 0; int product = 1; for(i = 0; i < N; ++i) { sum = sum + i; product = product *i; } System.out.println(sum);System.out.println(product);
int i; int sum = 0; int product = 1;int sumEven = 0; for(i = 0; i < N; ++i) { sum = sum + i; product = product *i; if(i%2 == 0) sumEven += i; } System.out.println(sum);System.out.println(product);System.out.println(sumEven);
Version i Version i+1
Extend of Change: number of edit operations to convert methodi to methodi+1
![Page 14: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/14.jpg)
Feature Envy
A method is “more interested in a class other than the one it actually is in”
m(Target t) { t.m1(); t.m2(); t.m3();}
m() { m1(); m2(); m3();}
![Page 15: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/15.jpg)
Feature Envy
The Intensity of the smell is related to the number of “envied” members
m(Target t) { t.m1(); t.m2(); t.m3();}
Extend of Change: variation in the number of “envied” members
Version i Version i+1
envies 3 members
m(Target t) { t.m1(); t.m2(); t.m3(); t.m4();} envies 4
members
![Page 16: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/16.jpg)
State Checking
State Checking manifests itself as conditional statements that select an execution path based on the state of an object
doStateA();
switch(type) {case STATE_A:
break;case STATE_B:
break;}
doStateB();
![Page 17: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/17.jpg)
What to look for
State Checking: implies a missed opportunity for polymorphism
if (state == StateA) { . . . . . .}else if (state == StateB) { . . . . . .}else if (state == StateC) { . . . . . .}
+. . .. . .. . .
+ (additional statements)
. . .
. . .
. . .
![Page 18: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/18.jpg)
State Checking
The intensity of the smell is primarily related to the number of conditional structures checking on the same states
Version i Version i+1
1 cond. structure
2 cond. structures
Extend of Change: variation in the number of conditional structures
![Page 19: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/19.jpg)
Application
1. Calculate past volatility values (for each refactoring opportunity)
2. Estimate future volatility3. Rank suggestions according to this estimate
![Page 20: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/20.jpg)
Evaluation
• Goal: To compare the accuracy of the four examined models
• performed along two axes:
• direct comparison of forecast accuracy (RMSE)
• comparison of rankings produced by each model and according to the actual volatility
• Context: two open source projects
• JMol: 26 project versions (2004 ..)
• JFreeChart: 15 project versions (2002 ..)
![Page 21: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/21.jpg)
JMol
020406080100120140160180200
0
100
200
300
400
500
600
KSLO
C
#cla
sses
#classes KSLOC
0
0.0005
0.001
0.0015
0.002
0.0025
0.003
0.0035
0.004
0.0045
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Exte
nt o
f Cha
nge
(Fea
ture
Env
y)
Exte
nt o
f Cha
nge
(Sta
te C
heck
ing)
Transitions between software versions
State Checking Feature Envy
![Page 22: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/22.jpg)
JFreeChart
020406080100120140160180200
0
100
200
300
400
500
600
700
KSLO
C
#cla
sses
#classes KSLOC
0
0.01
0.02
0.03
0.04
0.05
0.06
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Exte
nt o
f Cha
nge
Transitions between software versions
Long Method
![Page 23: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/23.jpg)
Comparison of Forecast Accuracy
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
1 2 3 4 5 6 7 8 9 10 11 12
RMSE
Transitions between software versions
EWMA
ES
HA
RW
N
iiiN
RMSE1
2ˆ1
both consider the average of all historical values
Long Method / JFreeChart
![Page 24: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/24.jpg)
0
0.001
0.002
0.003
0.004
0.005
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
RMSE
Transitions between software versions
EWMA
ES
HA
RW
Comparison of Forecast Accuracy
N
iiiN
RMSE1
2ˆ1
Random Walk is being favored by successive versions with zero volatility
Peaks in RMSE when versions with zero volatility are followed by abrupt change
Feature Envy / JMol
![Page 25: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/25.jpg)
Comparison of Forecast Accuracy
Random Walk
Historical Average
Exponential Smoothing
EWMA
Long Method (JFreeChart)
0.032646 0.031972 0.032176 0.032608
Feature Envy (JMol)
0.003311 0.003295 0.003309 0.003301
State Checking (JMol)
0.052842 0.052967 0.053051 0.053879
Overall RMSE for each smell and forecasting model
• Simplicity and relatively good accuracy of HA appropriate strategy for ranking refactoring suggestions
• HA achieves the lowest error for Long Method and Feature Envy
• more sophisticated models that take proximity into account do not provide higher accuracy
![Page 26: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/26.jpg)
Ranking Comparison
• Forecasting models extract the anticipated smell volatility for future software evolution
• Therefore, estimated volatility for the last transition can be employed as ranking criterion for refactoring suggestions
• Evaluation:
Rankings produced by each model
Rankings produced by actual volatility in the last transition
Compare
![Page 27: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/27.jpg)
Ranking Comparison
• To compare the similarity between alternative rankings (of the same set) we used Spearman’s footrule distance
ABCDEF
ABCDEF
NFr = 0
ABCDEF
FEDCBA
NFr = 1
S
i
SiiFr
12121 )()(,
ABCDEF
ACBEFD
NFr = 0.333
![Page 28: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/28.jpg)
Ranking Comparison - Spearman’s footrule
(Long Method / JFreeChart)
Random Walk
Historical Average
Exponential Smoothing
EWMA
Actual 0.6220 0.3255 0.5334 0.3238
Random Walk
Historical Average
Exponential Smoothing
EWMA
Actual 0.0096 0.0210 0.0199 0.0213
Random Walk
Historical Average
Exponential Smoothing
EWMA
Actual 0.07 0.13 0.14 0.13
(Feature Envy / JMol)
(State Checking / JMol)
high frequency of changes
low frequency of changes
low frequency of changes
![Page 29: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/29.jpg)
Conclusions
Refactoring suggestions can be ranked:
• according to design criteria
• according to past source code changes (higher priority for pieces of code that have been the subject of maintenance)
Simple forecasting models, such as Historical Average
• lowest RMSE error
• similar rankings to those obtained by actual volatility (frequent changes)
Future Work #1: Analyze history at a more fine-grained level
Future Work #2: Combination of structural and historical criteria
![Page 30: Ranking Refactoring Suggestions based on Historical Volatility](https://reader035.vdocuments.us/reader035/viewer/2022070405/56814037550346895daba139/html5/thumbnails/30.jpg)
Thank you for your attention
15th European Conference on Software Maintenance and Reengineering (CSMR 2011)