csmr11a.ppt

Antipatterns andComprehension

Abbes, Khomh,Gueheneuc, and

Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions

Conclusion andFuture Work

Questions

References

An Empirical Study of the Impact of TwoAntipatterns on Program Comprehension

Marwen Abbes, Foutse Khomh,Yann-Gael Gueheneuc, and Giuliano Antoniol

DGIGL, Ecole Polytechnique de Montreal, Quebec, CanadaE-mails: {marwen.abbes, yann-gael.gueheneuc,

giuliano.antoniol}@polymtl.ca, [email protected]

CSMR 2011February 3, 2011

Pattern Trace Identification, Detection, and Enhancement in JavaSOftware Cost-effective Change and Evolution Research Lab



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Outline

Introduction

Related Work

Experimental Study

Study Results

Discussions

Conclusion and Future Work

Questions

2 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Introduction

OO ParadigmI There is many benefits in using OO for analysis, design,

coding.

I These benefits include better understandability, easiermodifications, greater productivity, and simpler reuse.

I These benefits rely mainly on intuition instead ofobservations or experimental results.

3 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Introduction

Empirical StudiesI There exist few empirical studies confirming the benefits

of OO.

I It is only recently that researchers started to study the“OO claims” empirically.

I Obtained results show that OO does not always bringthe expected benefits.

4 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Introduction

Empirical StudiesI The literature suggests many explanations for the

apparent lack of benefits of OO:I OO notions (objects, polymorphism, inheritance,

patterns, and so on) are not easy to learn and useadequately.

I The use of OO impacts negatively the understandabilityof systems.

I Antipatterns deteriorate the quality of OO systems andthem harder to maintain.

I Therefore, the benefits of OO require further empiricalstudies, in particular to show its effectiveness andefficacy or lack thereof.

5 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Introduction

GoalI Antipatterns are conjectured in the literature to

decrease the quality of OO systems.

I Yet, despite the many studies on antipatterns, fewstudies empirically investigated the impact ofantipatterns on program comprehension.

I We want to gather quantitative evidence on the relationbetween antipatterns and program comprehension (orlack thereof).

I We perform a quantitative analysis of the impact of twoantipatterns on developers’ performance duringcomprehension and maintenance tasks.

6 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Related Work

Antipatterns and Quality

I Webster’s first book on quality-assurance problems [1].I Brown’s 40 antipatterns [2].

I Antipatterns are bad practices when solving designproblems.

I Antipatterns are conjectured in the literature todecrease the quality of systems.

I Fowler’s 22 code smells and refactorings [3] to improvethe design of existing code.

7 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Related Work

Antipatterns and DetectionI Many approaches exist for specifying and detecting

code smells and antipatterns, e.g., [4, 5, 6].

I We detect antipatterns using DEX, which stems fromour method DECOR (recall of 100% and an averageprecision greater than 60%) to locate and remove themin the systems that we study.

8 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Related Work

Antipatterns and Software Evolution and MaintenanceI Several works have studied the impact of antipatterns

on evolution phenomena.

I Vaucher et al. [7] studied in the versions of twodifferent systems the evolution of Blobs:

I They tracked the occurrences of the Blob.I They observed that Blobs are, in few cases, created by

design as the best solution to a particular problem.

I Olbrich et al. [8] studied occurrences of the Blob andShotgun Surgery:

I They tracked the occurrences of these antipattersn.I They observed that they are more change-prone than

other classes.

I These works suggest that antipatterns impact programevolution and maintenance.

9 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Related Work

Antipatterns and UnderstandabilityI Several works have studied the impact of antipatterns

on evolution phenomena.

I Deligiannis et al. [9] conducted a controlled experimenton the impact of Blob on the maintainability of twosystems.

I They concluded that the maintainability OO systemsdepends on good practices and design.

I They did not assess the impact of Blob on thedevelopers’ ease of understanding.

I Du Bois et al. [10] studied the impact onunderstandability of the decomposition of a Blob.

I They found that students had more difficultiesunderstanding the original Blob than otherdecompositions.

I Their study did not reveal any objective notion of“optimal” understandability.

10 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

In a NutshellI We perform 3 experiments to assess subjects’

comprehension of source code in the presence of twoantipatterns:

I Experiment 1 includes one and only one Blob.I Experiment 2 includes one and only one Spaghetti Code.I Experiment 3 includes both antipatterns.

I In each experiment, we assign two systems to eachsubject: one containing one occurrence of one (or both)antipattern(s) and one without any occurrence.

I We then measure and compare the subjects’performances for both systems.

11 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

HypothesesI H0Blob : There is no statistically significant difference

between the subjects’ average performance whenexecuting comprehension tasks on the source code ofsystems containing one occurrence of the antipatternBlob and their average performance with source codewithout any antipattern.

I If we reject the previous null hypothesis, then:I Either, E1Blob : the subjects’ average performance is

better when executing comprehension tasks on systemscontaining no occurrence of the Blob;

I Or, E2Blob : the subjects’ average performance is betterwhen executing comprehension tasks on systemscontaining one occurrence of the Blob;

I We have two similar null hypotheses H0SpaghettiCode andH0Blob+SpaghettiCode and sets of alternative hypotheses forthe other two experiments.

12 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

ObjectsI We choose three systems for each experiment, all

developed in Java.

Numbers of ReleaseExperiments Systems Classes SLOCs dates

YAMM 0.9.1 64 11,272 19991 JVerFileSystem 167 38,480 2008

AURA 95 10,629 2008GanttProject 2.0.6 527 68,545 2008

2 and 3 JFreeChart 1.0.13 989 302,844 2009Xerces 2.7.0 740 233,331 2008

Table: Object Systems

I We perform each experiment on 3 systems, because asingle system could be intrinsically easier/more complexto understand.

13 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

ObjectsI We use DECOR to ensure that each system has at least

one occurrence of antipattern.

I We validate the detected occurrences manually.

I For each system, we select randomly a subset of classesresponsible for managing a specific task to limit the sizeof the source code given to the subjects.

I For example, in JFreeChart, we choose the source codeof the classes responsible for editing and displaying theproperties of a plot.

I We refactor each subset of each system to remove allother occurrences of (other) antipatterns to reducepossible bias.

14 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

Independent and Dependent VariablesI The independent variable is the presence of the

occurrence(s) of the specific antipattern(s).

I Our dependent variables measure the subjects’performance, in terms of effort, time spent, andpercentage of correct answers.

15 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

Dependent Variable: Effort

Figure: TLX

16 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

Dependent Variable: Time

Figure: Timer.

17 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

Dependent Variable: Correct AnswersI We compute the percentage of correct answers for each

question by dividing the number of correct elementsfound by the subject by the total number of correctelements that they should have found.

18 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

SubjectsI Each experiment was performed by 24 anonymous

subjects, S1 to S24.

I Some subjects were enrolled in the M.Sc. and Ph.D.programs in Computer and Software engineering atEcole Polytechnique de Montreal or in ComputerScience in Universite de Montreal.

I Others were professionals working for softwarecompanies in the Montreal area, recruited through theauthors’ industrial contacts.

I All subjects were volunteers and could withdraw at anytime, for any reason.

19 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

QuestionsI Category 1: Finding a focus points:

I Question 1: Where is the code involved in theimplementation of this behavior?

I Question 2: Which type represents this domainconcept or this UI element or action?

I Category 2: Expanding a focus points:I Question 1: Where is this method called or this type

referenced?I Question 2: What data can we access from this object?

I Category 3: Understanding a code subset:I Question 1: How are these types or objects related?I Question 2: What is the behavior that these types

provide together and how is it distributed over thesetypes?

20 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

DesignI Our design is a 2×3 factorial design:

I We have three different systems, each with twopossibilities.

I For each combination, we prepare a set ofcomprehension questions, making up treatments.

I We have six different groups of subjects, each oneaffected to each one treatment.

With Antipattern(s) Without Antipattern(s)System 1 S3, S7, S9, S11, S12, S18, S21, S24 S1, S5, S8, S10, S15, S16, S20, S22System 2 S1, S2, S6, S14, S15, S17, S20, S22 S4, S7, S9, S11, S13, S18, S19, S23System 3 S4, S5, S8, S10, S13, S16, S19, S23 S2, S3, S6, S12, S14, S17, S21, S24

Table: Experimental Study

21 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Experimental Study

Analysis Method

I We use the (non-parametric) Mann-Whitney test tocompare sets of dependent variables and assess whethertheir difference is statistically significant.

I We also test the hypotheses with the (parametric)Student’s t-test.

I We use the Cohen d effect size, which indicates themagnitude of the effect of a treatment on thedependent variables.

I We use ANOVA to assess the dependence between thesix sets of dependent variables, as we have six differentgroups affected to the different treatments.

22 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Study Results

Summary of the Collected Data

Systems C. A.T. (s.) A.E. A.C.A.System 1: B 261 43.42 71%

YAMM B 149 27.99 71%System 2: B 251 43.30 67%

JVerFileSystem B 206 33.68 79%System 3: B 189 33.31 58%

Aura B 271 42.55 66%

System 1: S 190 47.31 67%GanttProject S 194 53.44 51%

System 2: S 215 39.27 52%Xerces S 182 34.48 52%

System 3: S 195 42.37 52%JFreeChart S 218 45.84 45%

System 1: BS 187 45.36 44%GanttProject BS 107 27.32 83%

System 2: BS 184 42.83 73%Xerces BS 140 29.33 86%

System 3: BS 208 48.68 43%JFreeChart BS 138 31.27 78%

23 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Study Results

p-values and Cohen’s d effect size results

TimesM.-W. t-Test ANOVA Cohen

p p p dExperiment 1 0.46 0.30 0.02 0.18Experiment 2 0.89 0.97 0.87 0.01Experiment 3 <0.01 <0.01 <0.01 1.54

EffortsM.-W. t-Test ANOVA Cohen

p p p dExperiment 1 0.25 0.17 0.29 0.24Experiment 2 0.64 0.57 0.41 0.09Experiment 3 <0.01 <0.01 0.01 1.20

AnswersM.-W. t-Test ANOVA Cohen

p p p dExperiment 1 0.41 0.21 0.58 0.27Experiment 2 0.26 0.24 0.60 0.35Experiment 3 <0.01 <0.01 <0.01 1.61

24 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

25 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Discussions

ResultsI The results of Experiment 1 show an increase in

subjects’ average time and effort on systems with Bloband a decrease in their average percentages of correctanswers.

I Experiment 2 reveals no significant difference betweensystems without/with a Spaghetti Code.

I Experiment 3 shows strong statistically significantdifferences between systems without/with a Blob and aSpaghetti Code.

I ANOVA tests confirm that these differences aresignificant across the six groups of subjects.

I Cohen’s d effect size values (for efforts, times, andpercentages of correct answers) are large; suggesting astrong relation.

26 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Discussions

Mitigating VariablesI We investigate if three mitigating variables impacted

our results: knowledge of Java, Eclipse, and in softwareengineering.

I We use 5-level Likert scales (bad, neutral, good,excellent, expert).

I As the groups are non-equal in terms of size, we usedthe Mann-Whitney test.

I We found no significant differences between thedifferent levels; p-values are high, expressing a lack ofcorrelation.

I We perform an ANOVA test to assess the impact of themitigating variables on the three measured variables,which shows that they do not impact our results.

27 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Discussions

Threats to Validity

Some threats limit the validity of our study results:

I Construct Validity: Measurement errors, subjectivity,“seriousness” of the antipattern occurrences.

I Internal Validity: Learning, selection, measures (again),diffusion of the treatments.

I External Validity: Generalisability.

28 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References


ConclusionI Antipatterns are conjectured in the literature to

negatively impact the quality of systems.

I We performed three experiments to study whethersystems with antipattern(s) are more difficult tounderstand than systems without any antipattern.

I Each experiment was performed with 24 subjects andon three different Java systems.

I We measured the subjects’ performance in terms ofefforts, times, and percentages of correct answers.

29 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References


ConclusionI Collected data showed that the occurrence of one

antipattern in the source code of a system does notsignificantly make its understandability.

I The combination of two occurrences of antipatternsimpacted negatively and significantly understandability.

I Developers can cope with antipatterns in isolationbut combinations thereof should be avoided,possibly through detection and refactorings.

I Consequently, developers and quality assurancepersonnel should be wary with growing numbers ofantipatterns in their systems.

30 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References


Future WorkI We want to investigate whether these

statistically-significant differences are due to the densityof antipatterns in the system andU-or to theoccurrences of specific antipatterns.

I We also plan to replicate this study in other contexts,with other subjects, other questions, other antipatterns,and other systems.

31 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

Any Question?

32 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

References

B. F. Webster, Pitfalls of Object Oriented Development,1st ed. M & T Books, February 1995. [Online].Available:www.amazon.com/exec/obidos/ASIN/1558513973

W. J. Brown, R. C. Malveau, W. H. Brown, H. W.McCormick III, and T. J. Mowbray, Anti Patterns:Refactoring Software, Architectures, and Projects inCrisis, 1st ed. John Wiley and Sons, March 1998.[Online]. Available: www.amazon.com/exec/obidos/tg/detail/-/0471197130/ref=ase theantipatterngr/103-4749445-6141457

M. Fowler, Refactoring – Improving the Design ofExisting Code, 1st ed. Addison-Wesley, June 1999.

33 / 37

www.amazon.com/exec/obidos/ASIN/1558513973

www.amazon.com/exec/obidos/tg/detail/-/0471197130/ref=ase_theanti patterngr/103-4749445-6141457





Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

References

Naouel Moha, Y.-G. Gueheneuc, L. Duchien, andA.-F. L. Meur, “DECOR: A method for the specificationand detection of code and design smells,”Transactionson Software Engineering (TSE), 2009. [Online].Available: http://www-etud.iro.umontreal.ca/˜ptidej/Publications/Documents/TSE09.doc.pdf

M. J. Munro, “Product metrics for automaticidentification of “bad smell” design problems in javasource-code,” in Proceedings of the 11th InternationalSoftware Metrics Symposium. IEEE Computer SocietyPress, September 2005. [Online]. Available: http://doi.ieeecomputersociety.org/10.1109/METRICS.2005.38

34 / 37

http://www-etud.iro.umontreal.ca/~ptidej/Publications/Documents/TSE09.doc.pdf

http://www-etud.iro.umontreal.ca/~ptidej/Publications/Documents/TSE09.doc.pdf

http://doi.ieeecomputersociety.org/10.1109/METRICS.2005.38

http://doi.ieeecomputersociety.org/10.1109/METRICS.2005.38



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

References

R. Marinescu, “Detection strategies: Metrics-based rulesfor detecting design flaws,” in Proceedings of the 20th

International Conference on Software Maintenance.IEEE CS Press, 2004, pp. 350–359.

Stephane Vaucher, Foutse Khomh, Naouel Moha, andY.-G. Gueheneuc, “Prevention and cure of softwaredefects: Lessons from the study of god classes,” inProceedings of the 16th Working Conference on ReverseEngineering (WCRE), G. Antoniol and A. Zaidman, Eds.IEEE Computer Society Press, October 2009, 10 pages.[Online]. Available: http://www-etud.iro.umontreal.ca/˜ptidej/Publications/Documents/WCRE09b.doc.pdf

35 / 37

http://www-etud.iro.umontreal.ca/~ptidej/Publications/Documents/WCRE09b.doc.pdf

http://www-etud.iro.umontreal.ca/~ptidej/Publications/Documents/WCRE09b.doc.pdf



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

References

S. Olbrich, D. S. Cruzes, V. Basili, and N. Zazworka,“The evolution and impact of code smells: A case studyof two open source systems,” in Third InternationalSymposium on Empirical Software Engineering andMeasurement, 2009.

D. Ignatios, S. Ioannis, A. Lefteris, R. Manos, andS. Martin, “A controlled experiment investigation of anobject oriented design heuristic for maintainability,”Journal of Systems and Software, vol. 65, no. 2,February 2003.

B. D. Bois, S. Demeyer, J. Verelst, T. Mens, andM. Temmerman, “Does god class decomposition affectcomprehensibility?” in Proceedings of the IASTEDInternational Conference on Software Engineering.IASTED/ACTA Press, 2006, pp. 346–355.

36 / 37



Antoniol

Introduction

Related Work

ExperimentalStudy

Study Results

Discussions


Questions

References

References

37 / 37

csmr11a.ppt

Technology

benets of oo

benetsdiscussions of

thequestions oo

foutse khomh

referencesobtained results

experimental results

referencesthese benets

discussions yanngal