what programmers say about refactoring tools? an empirical investigation of stackoverflow 
DESCRIPTION
Talk on the Workshop of Refactoring ToolsTRANSCRIPT
Gustavo Pinto, Fernando KameiFederal University of Pernambuco
{ghlp, fkk}@cin.ufpe.br
What Programmers Say About Refactoring Tools?
An Empirical Investigation of StackOverflow
ProgrammersResearchers
ProgrammersResearchers
What they think about?
ProgrammersResearchers
What they think about?
ProgrammersResearchers
What they think about?
1.5M Users3.4M Questions6.8M Answers40GB of data
A TON of information(with some quality)
1.5M Users3.4M Questions6.8M Answers40GB of data
But, what about Refactoring Tools?
A TON of information(with some quality)
Filter questions by “refactoring” and “tools” on the body, or
subject or tag name.
Store the results in a single MySQL table.
Manual filter to remove false positives
The final data!
3.4M Questions
754 Q>2K A
from July 31, 2008 to July 31, 2012:
● 324 Questions● 1,135 Answers● 1,254 Users
Filter questions by “refactoring” and “tools” key-word, on the
body, or subject or tag name.
Store the results in a single MySQL table.
Manual filter to remove false positives
The final data!
3.4M Questions
754 Q>2K A
from July 31, 2008 to July 31, 2012:
● 324 Questions● 1,135 Answers● 1,254 Users
Filter questions by “refactoring” and “tools” key-word, on the
body, or subject or tag name.
Store the results in a single MySQL table.
Manual filter to remove false positives
The final data!
3.4M Questions
754 Q>2K A
from July 31, 2008 to July 31, 2012:
● 324 Questions● 1,135 Answers● 1,254 Users
2x
Research Questions
● RQ1: What are the most desirable features in refactoring tools?
● RQ2: What are the barriers to adoption of refactoring tools?
● RQ3: How is interest in refactoring tools related to programmer expertise?
● RQ4: Does interest in refactoring tools increase over the years?
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)a. PHP » 29%b. JavaScript » 23%c. Python » 23%d. Ruby » 11%e. Perl » 9%f. R » 1%
g. Perl » 1%h. Matlab » 1%
RQ1: What are the most desirable features in refactoring tools?
StackOverflow Question #9424189
“I am looking for a python tool that will allow me to specifically rename and move
modules.”
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)
“I would prefer a tool that just makes suggestions about possible refactorings: names the refactoring, optionally provides a short description of it (great for learning purposes),
highlights the code section and lets me do the refactoring myself”
RQ1: What are the most desirable features in refactoring tools?
StackOverflow Question #785667
RQ1: What are the most desirable features in refactoring tools?
“I would like to rename a table column in MySql and also have the name updated in any triggers, stored procs, etc. that reference the column. [...] If a tool exists that can perform such a
refactoring, it would be great if it could generate an SQL script that applies these changes.”
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)3. Refactoring for databases (12 questions ≈ 3.71%)
a. Automatic simplification of SQLb. Refactoring table or views structurec. Fixing SQL vulnerabilities
StackOverflow Question #1644271
RQ1: What are the most desirable features in refactoring tools?
“I have inherited significant amounts of Groovy code, and I have found it difficult to maintain [..]. I have tried to manually convert some pieces to Java, it's been a pain. Are you aware of any tools
or plugins that help with this conversion?”
StackOverflow Question #5302103
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)3. Refactoring for databases (12 questions ≈ 3.71%)4. Multi-language Refactorings (9 questions ≈ 2.77%)
a. All questions related to JVM languages
RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (52 questions ≈ 16%) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)
2. Lack of trust (39 questions ≈ 12%)
RQ2: What are the barriers to adoption of refactoring tools?
33% are from novice users
In order of interest:
1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)
What refactoring tool do you use to do X in
language Z?
64%
49%
RQ2: What are the barriers to adoption of refactoring tools?
33% are from novice users
In order of interest:
1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)
What refactoring tool do you use to do X in
language Z?
64%
49%
Barriers or lack of experience?
RQ2: What are the barriers to adoption of refactoring tools?
1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)
2. Lack of trust (12 % of Q)
In order of interest:
Sometimes the tool will change the meaning of your code, without you expecting it.
StackOverflow Question #1372059
RQ2: What are the barriers to adoption of refactoring tools?
1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)
2. Lack of trust (12 % of Q)
In order of interest:
Sometimes the tool will change the meaning of your code, without you expecting it.
StackOverflow Question #1372059
“It is not good to decrease readability in favor of atomicity”
StackOverflow Question #5302103
RQ3: How is interest in refactoring tools related to programmer expertise?
Or.. do only hackers talk about refactoring tools?
RQ3: How is interest in refactoring tools related to programmer expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novicesA: 41% comes from seniors
Novices Seniors
RQ3: How is interest in refactoring tools related to programmer expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novicesA: 41% comes from seniors
Novices Seniors
Refactoring tool is a topic that interest all kind of users.
RQ3: How is interest in refactoring tools related to programmer expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novicesA: 41% comes from seniors
Novices Seniors
Also, we ran a Person Correlation between reputation x questions and reputation x answers, but no significant correlation was found.
RQ4: Does interest in refactoring tools increase over the years?
Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1
Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1
● Yes, but very little.
RQ4: Does interest in refactoring tools increase over the years?
Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1
Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1
RQ4: Does interest in refactoring tools increase over the years?
Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1
Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1
● Yes, but very little.● An increment of 0.56 questions per trimester.● A decrement of -8.13 answers per trimester.
RQ4: Does interest in refactoring tools increase over the years?
Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1
Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1
● Yes, but very little.● An increment of 0.56 questions per trimester.● A decrement of -8.13 answers per trimester.
-0.43 without outliers
Conclusions● We provided a set of desired features, as well as common
cause for barriers adoption.○ Insights for researchers and tool vendors
● We observed that novice users produce more questions, and seniors provide the answers.○ Also, most of the barriers were reported by novice users
● We identified a relative small interest in refactoring tools ○ Mean of 6 questions / month, sd of 4.45
Gustavo Pinto, Fernando KameiFederal University of Pernambuco
{ghlp, fkk}@cin.ufpe.br
What Programmers Say About Refactoring Tools?
An Empirical Investigation of StackOverflow