csmr13b.ppt

ACRE: AnAutomated AspectCreator for TestingC++ Applications

Etienne Duclos,Sebastien Le

Digabel,Yann-Gael

Gueheneuc andBram Adams

Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

ACRE: An Automated Aspect Creator forTesting C++ Applications

Etienne Duclos, Sebastien Le Digabel,Yann-Gael Gueheneuc and Bram Adams

Department of Computer and Software EngineeringGERAD and Department of Mathematics and Industrial Engineering

Ecole Polytechnique de Montreal, Quebec, Canada

CSMRMarch 07, 2013

Pattern Trace Identification, Detection, and Enhancement in JavaGroupe d’Etudes et de Recherche en Analyse des Decisions



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Problem

2 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Outline

Background

Related Work

ApproachDSLACREHypotheses

ResultsH1H2H3

Threats to Validity

Conclusion

3 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Background

Aspect-Oriented ProgrammingI Based on the concept of separation of concerns

I Used with OOP thanks to a weaver

I Different execution models available

I Two major languages: AspectJ and AspectC++

TestingI Memory Management

I Difficult in C++ [Boehm 12], high risk of memory leak

I Invariant TestingI Allow to verify that a property is always true

I Interference Bug TestingI Very common bug, and one of the most difficult to

remove [Software Quality Research Group 12]

4 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Related Work

AOP in testI AOP used in Java for unit and integration

testing [Xu 04, Knauber 04, Li 09]

I AOP used to test embedded C++ programs[Pesonen 05, Pesonen 06, Metsa 07]

I AOP used for memory management in C++ programs[Sioud 06]

I Some disadvantages for using AOP in test[Metsa 07, Farhat 10]

5 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Approach

Domain Specific LanguageI DSL used to describe the test to create

I Embedded in the source code, but does not modify itsbehavior nor its compilation

I Four types of aspects supported : counter, checking,timing and logging

I Few simple rules (see Table II named DSL Synthesis)

6 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Approach

ACRE: What is it ?I Automated aspeCt cREator

I Input: C++ source code containing aspect definitions

I Output: generated aspects

ACRE: How does it work ?I Implements a parser for each aspect type

I Easy to add new types of aspect

ACRE: Where to find ?

http://web.soccerlab.polymtl.ca/~ducloset/ACRE

I Under the GPL license

7 / 16

http://web.soccerlab.polymtl.ca/~ducloset/ACRE



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

8 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Approach

Hypotheses

I H1: Generated aspects can be used to do memorytesting for C++ programs.

I H2: Generated aspects can be used to do invarianttesting for C++ programs.

I H3: Generated aspects can be used to help findinginterference bugs in C++ programs.

Tested program: NOMAD

I C++ program [Audet 06, Le Digabel 11] used in bothresearch and industry

I 49 classes, 48k LOC

I Under the LGPL license, downloaded more than 3,000times since 2008 http://www.gerad.ca/nomad

9 / 16

http://www.gerad.ca/nomad



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Results: H1I H1: Generated aspects can be used to do memory

testing for C++ programs

I Bug report: memory leak on NOMAD

Injected lines

//// name : EVPCG//// type : coun t e r//// c l a s s : E v a l Po i n t//// namespace : NOMAD

10 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Generated counter aspect for memory testing

aspect EVPCG{pub l i c : s t a t i c i n t Eva l Po in tCoun t ;po intcut Eva l Po in tCounted ( ) =

”NOMAD: : Eva l Po i n t ” ;

adv ice Eva l Po in tCounted ( ) : s l i c e s t r u c t {c l a s s Eva l Po in tCount {

pub l i c : Eva l Po in tCount ( ){EVPCG : : Eva l Po in tCoun t++;}

pub l i c : ˜ Eva l Po in tCount ( ){EVPCG : : Eva l Po in tCount −−;}

} Eva l Po i n t c o un t e r ;} ;adv ice e x e c u t i o n ( ”% main ( . . . ) ”) : a f t e r ( ){

p r i n t f ( ”F i n a l count o f Eva l Po i n t :%d\n ” , Eva l Po in tCoun t ) ;

i f ( Eva l Po in tCoun t > 0)p r i n t f ( ”Memory l e a k ! ! \ n ”) ;

}} ;

11 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Results: H1

Figure: NOMAD output with a counter aspect weaved

I Memory leak founded

I Leak fixed by the NOMAD team of developers

I Generated aspects can be used to do memory testingfor C++ programs ,

12 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Results: H2

I H2: Generated aspects can be used to do invarianttesting for C++ programs

I Goal: to verify a crucial mathematical point of theMADS algorithm

I (Un)fortunately no bug found for the 37 tested problems

I A developer from the NOMAD team mutated NOMADsource code to have an error

I Bug founded on the mutated version

I Generated aspects can be used to do invariant testingfor C++ programs ,

13 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Results: H3

I H3: Generated aspects can be used to help findinginterference bugs in C++ programs

I Goal: to verify the parallel version of NOMAD

I Timing aspects give access times for both read andwrite events

I Times used by Bhattacharya et al. approach[Bhattacharya 12]

I Result: it is possible that an interference bug occurs

I Generated aspects can be used to help findinginterference bugs in C++ programs ,

14 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

Threats to ValidityI Internal validity : Founded bugs may be caused by the

added aspects

I External validity : Approach used only under Linux

I Reliability validity : All the programs are available onthe Internet

Future WorkI Generate more types of aspects

I Extend ACRE to test other languages

I Use ACRE on bigger programs

15 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

ConclusionI Allow the use of AOP to do memory, invariant and

interference testing in C++ programs

I No modifications in the source code of the testedprogram

I Allow the use of aspect without knowledge in AOP

Thank you for your attention ,Any questions ?

16 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

C. Audet & J.E. Dennis, Jr.Mesh Adaptive Direct Search Algorithms for ConstrainedOptimization.SIAM Journal on Optimization, vol. 17, no. 1, pages188–217, 2006.

Yosi Ben-Asher, Eitan Farchi & Yaniv Eytani.Heuristics for Finding Concurrent Bugs.In Proceedings of the 17th International Symposium onParallel and Distributed Processing, pages 288.1–, 2003.

17 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Neelesh Bhattacharya, Olfat El-Mahi, Etienne Duclos,Giovanni Beltrame, Giuliano Antoniol, SebastienLe Digabel & Yann-Gael Gueheneuc.Optimizing Threads Schedule Alignments to Expose theInterference Bug Pattern.In Proceedings of the 4th edition of the Symposium onSearch Based Software Engineering (SSBSE), 2012.

Hans-J. Boehm.Advantages and Disadvantages of Conservative GarbageCollection, 2012.tire de http://www.hpl.hp.com/personal/Hans_

Boehm/gc/issues.html, consulte le 06 juin 2012.

18 / 16

http://www.hpl.hp.com/personal/Hans_Boehm/gc/issues.html

http://www.hpl.hp.com/personal/Hans_Boehm/gc/issues.html



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Orit Edelstein, Eitan Farchi, Evgeny Goldin, Yarden Nir,Gil Ratsaby & Shmuel Ur.Framework for testing multi-threaded Java programs.Concurrency and Computation: Practice andExperience, pages 485–499, 2003.

Michael D. Ernst, Jeff H. Perkins, Philip J. Guo,Stephen Mccamant, Carlos Pacheco, Matthew S.Tschantz & Chen Xiao.The Daikon system for dynamic detection of likelyinvariants.In Science of Computer Programming, 2006.

19 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Salam Farhat, Greg Simco & Frank J. Mitropoulos.Using Aspects for Testing Nonfunctional Requirementsin Object-Oriented Systems.In Proceedings of the IEEE SoutheastCon 2010, pages356–359, 2010.

Pallavi Joshi, Mayur Naik, Chang-seo Park & KoushikSen.CalFuzzer: An Extensible Active Testing Framework forConcurrent Programs.In Proceedings of the 21st International Conference onComputer Aided Verification, pages 675–681, 2009.

20 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Peter Knauber & Johannes Schneider.Tracing Variability from Implementation to Test UsingAspect-Oriented Programming.International Workshop on Software Product LineTesting (SPLiT 2004), Boston, Massachusetts, USA,2004.

S. Le Digabel.Algorithm 909: NOMAD: Nonlinear Optimization withthe MADS algorithm.ACM Transactions on Mathematical Software, vol. 44,pages 1–15, 2011.

21 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Xiaofei Li & Xusheng Xie.Research of software testing based on AOP.In Proceedings of the 3rd International Conference onIntelligent Information Technology Application, pages187–189, 2009.

Jani Metsa, Mika Katara & Tommi Mikkonen.Testing Non-Functional Requirements with Aspects: AnIndustrial Case Study.In Proceedings of the Seventh International Conferenceon Quality Software, pages 5–14, 2007.

Madanlal Musuvathi, Shaz Qadeer & Thomas Ball.CHESS: A Systematic Testing Tool for ConcurrentSoftware, 2007.

22 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

Jani Pesonen, Mika Katara & Tommi Mikkonen.Production-Testing of Embedded Systems with Aspects.In Haifa Verification Conference, pages 90–102, 2005.

Jani Pesonen.Extending Software Integration Testing Using Aspects inSymbian OS.In Proceedings of the Testing: Academic & IndustrialConference on Practice and Research Techniques, pages147–151, 2006.

Aymen Sioud.Gestion de cycle de vie des objets par aspects pourC++.Master’s thesis, Universite de Quebec a Chicoutimi,2006.

23 / 16



Digabel,Yann-Gael


Background

Related Work

Approach

DSL

ACRE

Hypotheses

Results

H1

H2

H3

Threats to Validity

Conclusion

References

UOIT Software Quality Research Group Facultyof Science.Concurrency Anti-pattern Catalog for Java, 2012.tire de http:

//faculty.uoit.ca/bradbury/concurr-catalog/,consulte le 06 juin 2012.

Guoqing Xu, Zongyuan Yang, Haitao Huang, QianChen, Ling Chen & Fengbin Xu.JAOUT: Automated Generation of Aspect-Oriented UnitTest.In Proceedings of the 11th Asia-Pacific SoftwareEngineering Conference, pages 374–381, 2004.

24 / 16

http://faculty.uoit.ca/bradbury/concurr-catalog/

http://faculty.uoit.ca/bradbury/concurr-catalog/

csmr13b.ppt

Technology

sbastien lee digabel

applications etienne

eapproachdsl yanngal

anautomated aspectcreator

c boehm

aspect typeh1h2easy

oop thanks

concept of separation