130321 zephyrin soh - on the effect of exploration strategies on maintenance tasks

ExplorationStrategies

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Strategy Identification

Results

Conclusion andFuture Work

Conclusion

Threats to Validity andFuture Work

On the Effect of Exploration Strategies onMaintenance Tasks

Zephyrin Soh, Yann-Gael Gueheneuc, Giuliano AntoniolBram Adams, Foutse Khomh

Department of Computer and Software EngineeringEcole Polytechnique de Montreal, Quebec, Canada

March 21, 2013

Pattern Trace Identification, Detection, and Enhancement in JavaSOftware Cost-effective Change and Evolution Research Lab

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Outline

Introduction

Empirical Study

Results

Conclusion and Future Work

2 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

IntroductionMotivating Example (1/3)

Program ExplorationI Developers always explore the program when performing

a maintenance task i.e., move from a program entity toanother.

I Types of program entities (e.g., Java): project, package,file, class, attribute, method, resource (XML file, etc.)

I Consider an exploration graph where:

I Node: a program entityI Edge: moving from a program entity to another.

3 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

3 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

3 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

I Consider an exploration graph where:I Node: a program entity

I Edge: moving from a program entity to another.

3 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

I Consider an exploration graph where:I Node: a program entityI Edge: moving from a program entity to another.

3 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

4 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

4 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

4 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

4 / 30

We focus on the program entity oftype class ⇒ Containment principle

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

4 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

5 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

5 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

5 / 30

A graph represents a program exploration

(a) Graph 1 (b) Graph 2

I Is there any difference between exploration graphs?

I How you find the difference?

I Why there is difference?

I How much is the difference?

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

IntroductionContext (1/2)

Program Exploration

Two main concepts:

I Interaction event (IE): Each developer’s action on anentity (through IDE) e.g., select a file Foo.java throughpackage explorer.

I Interaction history (IH): a sequence of interactionevents.

Many tools used to gather IH: Mylyn, SmartGroups,CodingTracker.

6 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Objectives of Program ExplorationI Developers look at the relevant program entities

I Developers perform on average 19.31 navigation actionsbetween two modifications [1]⇒ They spend time finding where and how to performa change on a program.

Research on Program Exploration

I Reduce the developers’ search effort (when searchingthe relevant entities to perform a given task).

I Improve the developers’ productivity.

[1] D. Rothlisberger, SmartGroups: Focusing on Task-Relevant SourceArtifacts in IDEs, ICPC 2011

7 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Objectives of Program ExplorationI Developers look at the relevant program entities

I Developers perform on average 19.31 navigation actionsbetween two modifications [1]⇒ They spend time finding where and how to performa change on a program.

Research on Program Exploration

I Reduce the developers’ search effort (when searchingthe relevant entities to perform a given task).

I Improve the developers’ productivity.

[1] D. Rothlisberger, SmartGroups: Focusing on Task-Relevant SourceArtifacts in IDEs, ICPC 2011

7 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalProblem (1/1)

Main Question

How can we help developers based on the way they explore aprogram?

First StepI Investigate if developers have different exploration

strategiesI Exploration strategies (ES): the way (how) the

developers navigate through the program entities

I Study some cause/effect relationships around ES

8 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalProblem (1/1)

Main Question

8 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalProblem (1/1)

Main Question

8 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalBenefits (1/1)

Usefulness of the Exploration Strategies

Exploration strategies can help to:

I Evaluate a developer’s performance: a specific strategycan tend to be more efficient.

I Improve our knowledge on developers’ comprehension orprogramming process: a top-down or bottom-upcomprehension can be related to a specific strategy

I Characterize developers’ expertise: the experienceddevelopers can tend to follow a specific strategy thatcan be used to help inexperienced ones

I Develop new exploration features for IDE: a strategycan highlight the need of a specific feature.

9 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalBenefits (1/1)

9 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalBenefits (1/1)

9 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

GoalBenefits (1/1)

9 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Empirical StudyResearch Questions (1/1)

Questions

Do developers follow specific exploration strategieswhen performing maintenance tasks?

Do specific exploration strategies affect the main-tenance time?

Do specific exploration strategies affect the explo-ration effort?

Does extensive work results to a specific explo-ration strategy?

10 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Questions

10 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Questions

10 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Questions

10 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Empirical StudyData (1/3)

Data CollectionI Four open-source projects: ECF, Mylyn, PDE, Eclipse

PlatformI We focus on IHs build by Mylyn

I Mylyn is an industrial tool provided as an Eclipse’splug-in (also CodingTracker)

I Mylyn’s IHs are available: all contributions to Mylynmust be made using Mylyn

11 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Example: Mylyn’s IHs as attachment (XML file) to a bug

11 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

11 / 30

Mylyn’s Interaction History

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

11 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

11 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Parsing Interaction Histories

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

12 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Parsing StructureHandleI Identify the type and parts of program entity

I Parsing ⇒ A graph representation of an interactionhistory

I Remember: We focus on the program entity of typeclass

13 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Parsing StructureHandleI Identify the type and parts of program entity

I Parsing ⇒ A graph representation of an interactionhistory

I Remember: We focus on the program entity of typeclass

13 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Empirical StudyMetrics (1/3)

14 / 30

I NumEvent(Classes) =∑

e∈Classes NumEvent(e)

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

15 / 30

I NumRevisits(c) = number of times a class c is revisited

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

I NumEdit: total number of edit events on all classes andtheir contentsNumEdit(e) =

∑eventBe NumEdit(event)

NumEdit(Classes) =∑

e∈Classes NumEdit(e)

I Exploration Effort: edit ratio which is related to theidentification of the relevant entitiesEditRatio(Classes) = NumEdit(Classes)

NumEvent(Classes)

I Class Level Duration (CLD): total duration spent on allclasses and their contentsDuration(event) = EndDate(event)− StartDate(event)Duration(e) =

∑eventBe Duration(event)

CLD = Duration(Classes) =∑

e∈Classes Duration(e)

I Overall Duration (OD): total duration spent on an IHOD =

∑event∈IH Duration(event)

16 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

NumEvent(Classes)

16 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

NumEvent(Classes)

16 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

NumEvent(Classes)

16 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Empirical StudyStrategy Identification (1/3)

Gini Inequality IndexI Used in econometrics to measure the inequality of

income among a population

I Gini ∈ [0, 1]: 0 (perfect equality), 1 (maximalinequality)

I We measure how much classes are (un)equally revisited

I NumRevisits = income of a class⇒ Apply Gini [2] on NumRevisits

Gini(IH) = 12n2µ

n∑i=1

n∑j=1| NumRevisit(eni )− NumRevisit(enj ) |

[2] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?,Technical report, Department of Economics, Dalhouse University, 2004

17 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Problem 1: Gini Threshold

How to know if classes are (un)equally revisited?⇒ Define a Gini Threshold

I If Gini(IH) < Threshold⇒ Classes are almost equally revisited⇒ EES: Extended Exploration Strategy

I If Gini(IH) ≥ Threshold⇒ Some classes are more revisited than others (core vs.periphery)⇒ CES: Centralized Exploration Strategy

18 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Problem 2: Cut-Off Point

For CES, core vs. periphery classes⇒ Define a COP

I Ordered the classes by NumRevisits{(C1, 9), (C2, 2), (C3, 4), (C4, 16), (C5, 11)}⇒ (C4, 16), (C5, 11), (C1, 9), (C3, 4), (C2, 2)

I COP: a maximum difference between NumRevisit oftwo consecutive classes⇒ COP = 5: Between C4-C5, and C1-C3

I More than one COP: the one that provides more coreclasses⇒ Between C1-C3

I Core classes = {C4,C5,C1}Peripheral classes = {C3,C2}

19 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

19 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

19 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

19 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (1/4)

Problem 1: Manual ValidationI Build an Oracle

I Random set of heterogenous interaction historyI Sample size: proportionally distributed among projectsI 95%± 10 confidence level

I Use Graphviz to generate the exploration graphs

I Subjects: Three PhD students with Java Experience

Validation ProcessI Explain the meaning of the graphs

I Subjects’ labels of a graph: C, E or D (Doubt)

I Final label: CES (or EES) if at less two C (or E)Undecided if completely different labels or two D

20 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (1/4)

Problem 1: Manual ValidationI Build an Oracle

I Random set of heterogenous interaction historyI Sample size: proportionally distributed among projectsI 95%± 10 confidence level

I Use Graphviz to generate the exploration graphs

I Subjects: Three PhD students with Java Experience

Validation ProcessI Explain the meaning of the graphs

I Subjects’ labels of a graph: C, E or D (Doubt)

I Final label: CES (or EES) if at less two C (or E)Undecided if completely different labels or two D

20 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (2/4)

Compromise ThresholdI F-Measure: maximize both precision and recall

21 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (2/4)

Compromise ThresholdI F-Measure: maximize both precision and recall

21 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (3/4)

Observation 1: Developers follow mostly the EES whenperforming a maintenance task.

I Methodical developers do not reinvestigate methods asfrequently as opportunistic developers [3]

I CES ⇒ opportunistic developers

I EES ⇒ methodical developers

[3] M. P. Robillard et al., How effective developers investigate source code: Anexploratory study, IEEE TSE, 2004

22 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (3/4)

Observation 1: Developers follow mostly the EES whenperforming a maintenance task.

I Methodical developers do not reinvestigate methods asfrequently as opportunistic developers [3]

I CES ⇒ opportunistic developers

I EES ⇒ methodical developers

[3] M. P. Robillard et al., How effective developers investigate source code: Anexploratory study, IEEE TSE, 200422 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (4/4)

Confounding FactorI Developers navigate through relationships between

classes and key binding ⇒ The architecture can affectthe ES

I Classes (and relations) involved in an IH = part of thesystem used to perform a task

I If IHs A and B have almost the same part of a system⇒ same ESA and B different parts ⇒ Different ES

I Number of common classes (NCC) between A and B tocapture their common part

I Compute NCC for each pair of IHNo difference between the NCC of the IHs with thesame ES and the NCC of the IHs with different ES⇒ The architecture does not affect the ES

23 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (4/4)

23 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (4/4)

23 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (4/4)

23 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ1 (4/4)

23 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ2 (1/1)

Observation 2: The EES is on average 69.39% less timeconsuming than the CES.

I Methodical developers performs tasks two times fasterthan opportunistic ones [3]

I CES ⇒ read source code in details [3]

I EES ⇒ look at explicit program entity(ies) [3]

24 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ2 (1/1)

Observation 2: The EES is on average 69.39% less timeconsuming than the CES.

I Methodical developers performs tasks two times fasterthan opportunistic ones [3]

I CES ⇒ read source code in details [3]

I EES ⇒ look at explicit program entity(ies) [3]

[3] M. P. Robillard et al., How effective developers investigate source code: Anexploratory study, IEEE TSE, 200424 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ3 (1/1)

Observation 3: An EES requires more exploration effortthan a CES.

I Methodical developers are successful developerscompared to opportunistic developers [3]

I CES ⇒ code modifications in one place [3]

I EES surprising: IH edits vs. source code repository edits

25 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ3 (1/1)

Observation 3: An EES requires more exploration effortthan a CES.

I Methodical developers are successful developerscompared to opportunistic developers [3]

I CES ⇒ code modifications in one place [3]

I EES surprising: IH edits vs. source code repository edits

25 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ4 (1/2)

Expand the Work on Many DaysI Open-source project: developers are volunteers⇒ they work on their spare time.

I Number of working days spent on change request

I Unbalanced proportion of IH: the more days developerswork on a change task, the more time they spend onprogram entities

26 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ4 (1/2)

Expand the Work on Many DaysI Open-source project: developers are volunteers⇒ they work on their spare time.

I Number of working days spent on change request

I Unbalanced proportion of IH: the more days developerswork on a change task, the more time they spend onprogram entities26 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ4 (2/2)

Observation 4: Developers usually adopt a CES whenperforming maintenance tasks spanning on more thanthree days.

I Developers refresh their mind by (re)understanding theentities that they explored before

I Mylyn task context feature sometimes push developersto (re)explore entities already explored in previousworking sessions

27 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

ResultsRQ4 (2/2)

Observation 4: Developers usually adopt a CES whenperforming maintenance tasks spanning on more thanthree days.

I Developers refresh their mind by (re)understanding theentities that they explored before

I Mylyn task context feature sometimes push developersto (re)explore entities already explored in previousworking sessions27 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Conclusion and Future WorkConclusion (1/1)

28 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

28 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

28 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

28 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

Conclusion and Future WorkThreats to Validity and Future Work (1/1)

Threats to Validity and Future Work

I Construct validity: An IH can be a part of developers’work

I Conclusion validity: The time recorded can bedifferent to the “real” time spent⇒ Perform an experiment to collect a data

I Internal validity: We use only Mylyn’s IH⇒ Use the IH from other tools

I External validity: Our subject projets are Eclipse-basedprojects⇒ Use other systems

29 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

29 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

29 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

29 / 30

Zephyrin Soh et al.

Introduction

Motivating Example

Context

Problem

Benefits

Empirical Study

Research Questions

Metrics

Results

Conclusion

30 / 30

Thanks for your attention!

130321 zephyrin soh - on the effect of exploration strategies on maintenance tasks

future work conclusion

future work3

future work4

future workmarch

future work2

program entity edge

eect of exploration

exploration graph

Technology

subcommittee on hydrology ( soh)

hyongsok (tom) soh - stanford university · updated:...

soh, ignou - learning skills · soh, ignou director (soh)...

soh fw12-lookbook

subcommittee on harmonization (soh) update

soh cah toa

navy safety and occupational health (soh) program … ·...

130321 blug - uffe sorensen - posted

2012 soh dinner night photos

desmond soh, crossroads international church

kaelyn yunci soh - moehaiggirls.moe.edu.sg/qql/slot/u240/hgs...

soh 2014 annual report

soh wei ni

130705 zephyrin soh - how developers spend their effort...

4. mr. soh brand

trinity university - soh 2015-16

nap i 130321

ubd edi soh presentation

soh jin lee

9.6 sine and cosine. chief soh-cah-toa soh cah toa