tutorial - computer science & eggay/sbst2016/slides/tutorial.pdfgordon fraser, university of...
TRANSCRIPT
![Page 1: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/1.jpg)
Gordon Fraser, University of Sheffield, UKAndrea Arcuri, Simula Research Labs, Norway
Gordon Fraser, University of Sheffield
TUTORIAL
![Page 2: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/2.jpg)
AcknowledgementsAndrea ArcuriJosé CamposBenjamin FriedrichFlorian GrossJuan Pablo GaleottiAlessandra GorlaMat HallFitsum Meshesha KifitewMerlin LangYanchuan LiEva MayPhil McMinn
Andre MisDaniel MuthAnnibale PanichellaDavid PatersonJeremias RoesslerJose Miguel RojasKaloyan RusevSina ShamshiriSebastian SteenbuckAndrey TarasevichMattia VivantiThomas White
![Page 3: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/3.jpg)
Why should you care?
• Are you writing Java code?Learn how to use EvoSuite to help you testing
• Are you doing research on testing?Learn how to use EvoSuite for experimentsLearn how to extend EvoSuite
• Are you doing research on SBST?Hear about our experiences in developing an SBST tool
![Page 4: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/4.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 5: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/5.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 6: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/6.jpg)
History
μTest
class Foo { int bar(int x) { return 2 * x; }}
class Foo { int bar(int x) { return 2 * x; }}
void test() { f = new Foo(); y = f.bar(10);
}
+
assert(y==20
Mutation-based Generation of Tests and Oracles, ISSTA 2010
![Page 7: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/7.jpg)
LocalDate date = new LocalDate(2010, 7, 15);
assertEquals(date.getDayOfMonth(), ...);date.plusYears(1);assertEquals(date.getYear(), 2011);
assertEquals(date.size(), 3);assertEquals(date.getValue(YEAR), 2010);assertEquals(date.getValue(MONTH_OF_YEAR), 7);assertEquals(date.getValue(DAY_OF_MONTH), 15);assertEquals(date.getLocalMillis(), ...);assertEquals(date, date);assertEquals(date.compareTo(date), 0);assertEquals(date.getYearOfCentury(), ...);assertEquals(date.getYear(), 2010);assertEquals(date.getWeekyear(), ...);assertEquals(date.getMonthOfYear(), 7);assertEquals(date.getWeekOfWeekyear(), ...);assertEquals(date.getDayOfWeek(), ...);
![Page 8: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/8.jpg)
LocalDate date = new LocalDate(2010, 7, 15);
assertEquals(date.getDayOfMonth(), ...);date.plusYears(1);assertEquals(date.getYear(), 2011);
assertEquals(date.size(), 3);assertEquals(date.getValue(YEAR), 2010);assertEquals(date.getValue(MONTH_OF_YEAR), 7);assertEquals(date.getValue(DAY_OF_MONTH), 15);assertEquals(date.getLocalMillis(), ...);assertEquals(date, date);assertEquals(date.compareTo(date), 0);assertEquals(date.getYearOfCentury(), ...);assertEquals(date.getYear(), 2010);assertEquals(date.getWeekyear(), ...);assertEquals(date.getMonthOfYear(), 7);assertEquals(date.getWeekOfWeekyear(), ...);assertEquals(date.getDayOfWeek(), ...);
assertEquals(date.size(), 3);
![Page 9: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/9.jpg)
assertEquals(date.getYear(), 2011);
assertEquals(date.getDayOfMonth(), ...);
assertEquals(date.getDayOfMonth(), ...);
date.plusYears(1);
assertEquals(date.size(), 3); assertEquals(date.getValue(YEAR), 2011);assertEquals(date.getValue(MONTH_OF_YEAR), 7); assertEquals(date.getValue(DAY_OF_MONTH), 15); assertEquals(date.getLocalMillis(), ...); assertEquals(date, date); assertEquals(date.compareTo(date), 0); assertEquals(date.getYearOfEra(), ...); assertEquals(date.getYearOfCentury(), ...); assertEquals(date.getWeekyear(), ...); assertEquals(date.getMonthOfYear(), 7); assertEquals(date.getWeekOfWeekyear(), ...); assertEquals(date.getDayOfWeek(), ...);
![Page 10: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/10.jpg)
assertEquals(date.getYear(), 2011);
assertEquals(date.getDayOfMonth(), ...);
assertEquals(date.getDayOfMonth(), ...);
date.plusYears(1);
assertEquals(date.size(), 3); assertEquals(date.getValue(YEAR), 2011);assertEquals(date.getValue(MONTH_OF_YEAR), 7); assertEquals(date.getValue(DAY_OF_MONTH), 15); assertEquals(date.getLocalMillis(), ...); assertEquals(date, date); assertEquals(date.compareTo(date), 0); assertEquals(date.getYearOfEra(), ...); assertEquals(date.getYearOfCentury(), ...); assertEquals(date.getWeekyear(), ...); assertEquals(date.getMonthOfYear(), 7); assertEquals(date.getWeekOfWeekyear(), ...); assertEquals(date.getDayOfWeek(), ...);
![Page 11: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/11.jpg)
LocalDate date = new LocalDate(2010, 7, 15);date.plusYears(1);assertEquals(date.getYear(), 2011);assertEquals(date.getValue(YEAR), 2011);
![Page 12: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/12.jpg)
LocalDate date = new LocalDate(2010, 7, 15);date.plusYears(1);assertEquals(date.getYear(), 2011);
![Page 13: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/13.jpg)
History
April 9, 2010“Evolutionary Generation of Whole Test Suites,” 11th International Conference on Software Quality (QSIC 2011)
![Page 14: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/14.jpg)
Stats
• 6,865 commits
• 229,889 LOC
• 2,420 tests
![Page 15: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/15.jpg)
@Test
public void test() {
}
int x = 2;int y = 2;int result = x + y;assertEquals(4, result);
![Page 16: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/16.jpg)
@Test
public void test() {
}
DateTime var3 = var1.toDateTime(var2);
DateTime var4 = var3.minus(var0);
TimeOfDay var2 = new TimeOfDay();
YearMonthDay var1 = new YearMonthDay(var0);
int var0 = 10
DateTime var5 = var4.plusSeconds(var0);
![Page 17: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/17.jpg)
DateTime var3 = var1.toDateTime(var2);
DateTime var4 = var3.minus(var0);
TimeOfDay var2 = new TimeOfDay();
YearMonthDay var1 = new YearMonthDay(var0);
int var0 = 10
DateTime var5 = var4.plusSeconds(var0);
![Page 18: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/18.jpg)
![Page 19: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/19.jpg)
Crossover
![Page 20: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/20.jpg)
Mutation
![Page 21: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/21.jpg)
Mutation
![Page 22: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/22.jpg)
Fitness
public int gcd(int x, int y) { int tmp; while (y != 0) { tmp = x % y; x = y; y = tmp; } return x;}
![Page 23: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/23.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 24: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/24.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 25: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/25.jpg)
Getting EvoSuite
http://www.evosuite.org/downloads
• Jar release - for command line usage
• Maven plugin
• IntelliJ plugin
• Eclipse plugin
• Jenkins plugin
![Page 26: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/26.jpg)
Testing a Class
• Demo - command line
• Main options: -projectCP -class -criterion
![Page 27: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/27.jpg)
Properties• -Dproperty=value
• Search budget (s) -Dsearch_budget=60
• Assertion generation-Dassertions=false -Dassertion_strategy=all
• Minimisation (length and values) -Dminimize=false
• Inlining-Dinline=false
![Page 28: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/28.jpg)
EvoSuite Sandbox
• Demo - Nondeterministic class
• Runtime library to execute tests
![Page 29: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/29.jpg)
Testing multiple classes
Demo:
• Target / prefix
• Continuous
• Maven
• Jenkins
• IntelliJ
![Page 30: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/30.jpg)
Experiment Exercise
• EvoSuite by default uses a combination of different coverage criteria.
• RQ1: Does the combination lead to larger test suites than just using branch coverage?
• RQ2: Does the combination lead to better test suites than just using branch coverage?
![Page 31: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/31.jpg)
Setup
• Download and unzip: http://evosuite.org/files/tutorial/Tutorial_Experiments.zip
• Maven project, but zip includes compiled code and dependencies(mvn compile dependency:copy-dependencies)
• Where is EvoSuite? export EVOSUITE=“java -jar /path/to/evosuite.jar”
![Page 32: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/32.jpg)
Treatments
• Default criteria combination: $EVOSUITE -class tutorial.Person
• Only branch coverage: $EVOSUITE -class tutorial.Person -criterion branch
• Resulting data: evosuite-report/statistics.csv
![Page 33: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/33.jpg)
Generating Data
http://evosuite.org/files/tutorial/Tutorial_Experiments.zip
• Commons Options: -prefix tutorial -Dsearch_budget=20 -Doutput_variables=configuration_id,TARGET_CLASS, Size,Length,MutationScore
• Treatment 1: $EVOSUITE -Dconfiguration_id=Default <common options>
• Treatment 2: $EVOSUITE -Dconfiguration_id=Branch -criterion branch <common options>
![Page 34: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/34.jpg)
Cluster Experiments
• Demo - Sun Grid Engine
![Page 35: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/35.jpg)
Analysing Data
• R analysis_scripts/analyse.R
• Pythonanalysis_scripts/analysis.py analysis_scripts/plots.py
Libs: easy_install numpy easy_install matplotlib easy_install pandas easy_install scipy
![Page 36: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/36.jpg)
Experiment Results
• RQ1: Does the combination lead to larger test suites than just using branch coverage?
• RQ2: Does the combination lead to better test suites than just using branch coverage?
![Page 37: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/37.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 38: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/38.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 39: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/39.jpg)
Building EvoSuite
• Git repository: git clone https://github.com/EvoSuite/evosuite.git
• Mavenmvn package (mvn -DskipTests package)
• Where is EvoSuite now? master/target/evosuite-master-1.0.4-SNAPSHOT.jar
• Why is the jar file so huge?
![Page 40: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/40.jpg)
Module Structure
• master
• client
• runtime
• standalone-runtime
• plugins
• generated
• shaded
![Page 41: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/41.jpg)
Testing EvoSuite
public class NullStringSystemTest extends SystemTestBase {
@Testpublic void testNullString() {
EvoSuite evosuite = new EvoSuite();
String targetClass = NullString.class.getCanonicalName();
Properties.TARGET_CLASS = targetClass;
String[] command = new String[] { "-generateSuite", "-class", targetClass };
Object result = evosuite.parseCommandLine(command);GeneticAlgorithm<?> ga = getGAFromResult(result);TestSuiteChromosome best = (TestSuiteChromosome) ga.getBestIndividual();System.out.println("EvolvedTestSuite:\n" + best);
int goals = TestGenerationStrategy.getFitnessFactories(). get(0).getCoverageGoals().size(); // assuming single fitness function
Assert.assertEquals("Wrong number of goals: ", 3, goals);Assert.assertEquals("Non-optimal coverage: ", 1d, best.getCoverage(), 0.001);
}}
Example Test:
![Page 42: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/42.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
![Page 43: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/43.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
![Page 44: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/44.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
package org.evosuite.ga.operators.crossover;
import org.evosuite.ga.Chromosome;import org.evosuite.ga.ConstructionFailedException;
public class MiddleCrossOver extends CrossOverFunction { @Override public void crossOver(Chromosome parent1, Chromosome parent2) throws ConstructionFailedException {
// TODO }}
![Page 45: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/45.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
@Test public void testSinglePointCrossOver() throws ConstructionFailedException { DummyChromosome parent1 = new DummyChromosome(1, 2, 3, 4, 5, 6); DummyChromosome parent2 = new DummyChromosome(7, 8, 9, 10);
MiddleCrossOver xover = new MiddleCrossOver();
xover.crossOver(parent1, parent2);
assertEquals(Arrays.asList(1, 2, 3, 9, 10), parent1.getGenes()); assertEquals(Arrays.asList(7, 8, 4, 5, 6), parent2.getGenes()); }
![Page 46: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/46.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
@Overridepublic void crossOver(Chromosome parent1, Chromosome parent2) throws ConstructionFailedException { int middle1 = parent1.size()/2; int middle2 = parent2.size()/2; Chromosome t1 = parent1.clone(); Chromosome t2 = parent2.clone(); parent1.crossOver(t2, middle1, middle2); parent2.crossOver(t1, middle2, middle1);}
![Page 47: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/47.jpg)
Extending EvoSuite
• (Artificial) Example: Middle point crossover
public enum CrossoverFunction { SINGLEPOINTRELATIVE, SINGLEPOINTFIXED, SINGLEPOINT, COVERAGE, MIDDLE }
![Page 48: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/48.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 49: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/49.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 50: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/50.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to test my own Java code?
• Yes, of course
![Page 51: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/51.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to implement my ideas on unit test generation?
• Yes, of course
![Page 52: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/52.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to study developer behaviour?
• Yes, of course
![Page 53: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/53.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to generate unit tests for my experiment on X?
• Yes, of course
![Page 54: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/54.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to build a unit test generator for a different programming language?
• EvoSuite is 90% JVM handling code
• Would need to reimplement representation, search operators, fitness functions, test execution, …
![Page 55: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/55.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to create an Android testing tool?
• Android uses Java / Dalvik bytecode
• Can also compile to Java bytecode
• How to handle Android dependencies?
![Page 56: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/56.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to create a GUI testing tool?
• If you want to test Java/Swing applications…
• But whole test suite optimisation may not be the right choice
![Page 57: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/57.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to create a web app testing tool?
• If it’s based on JEE, unit testing already works (JEE support is not complete yet)
• System testing…see GUI testing
![Page 58: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/58.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to implement a non-test generation SBSE tool?
• GA implementation is quite test specific
• Using for other purposes would need refactoringBut then, is it better than using existing generic GA libraries?
• If the tool uses Java, why not?
![Page 59: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/59.jpg)
When to use and not to use EvoSuite
• Should I use EvoSuite…
• …to implement a tool that requires tests?
• E.g., specification mining, fault localisation, program repair, GI, …
• Sure, integrating EvoSuite should be easy
![Page 60: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/60.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 61: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/61.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 62: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/62.jpg)
1. Java
…is a weird languageand never ceases to surprise me
My personal enemy: Java Generics
Bytecode over sourcecode - yes!
![Page 63: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/63.jpg)
2. Corner Cases
The more corner cases you cover
...the more can go wrong
...the more new corner cases you will find
...the slower EvoSuite becomes
![Page 64: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/64.jpg)
2. Corner Cases
• Constant Seeding: +5%
• Virtual FS: +1.4%
• Mocking +4.7%
• JEE support: +3%
• DSE: +1.2%
![Page 65: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/65.jpg)
3. Developers
…some really care only about coverage
…others don’t care about coverage:"I wouldn’t normally in real life be aiming for 100% coverage. I’d probably end up with fewer tests without this tool but I couldn’t tell you if they would be all the right tests.”
…do not want their tests to be generated
…hate ugly tests
…don’t like waiting
Talk to them!
![Page 66: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/66.jpg)
3. Developers
public class Example {
private Example() {}
// …}
![Page 67: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/67.jpg)
4. Testing
Testing randomised algorithms is difficult
Make the implementation deterministic
Always use LinkedHashSet over HashSet, LinkedHashMap over HashMap
Java reflection is not deterministic
Avoid static state (e.g. singletons)
![Page 68: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/68.jpg)
4. Testing
EvoSuite uses one central random number generator
Any change will affect something at a completely different part of the program
Change seeds frequently during testing to find flaky tests
![Page 69: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/69.jpg)
5. Documentation
I don’t comment my code
Students struggle
I spend more time explaining things than it would take me to implement them
![Page 70: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/70.jpg)
6. Tool Comparisons
Reviewers want to see them
I don’t like doing them
It’s impossible to make them fair
Contact tool authors
Report bugs
Make your own tools usable
![Page 71: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/71.jpg)
7. Open Source
“The source code will be released under an open source library (most likely GPL2) at a later point, as soon as a number of refactorings are completed.” — FSE’11 tool paper appendix
Public GitHub repo: 2015
It will never be clean enough, just release it!
![Page 72: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/72.jpg)
8. Licensing
License matters
Google will not touch GPL
BSD, MIT - do you want others to become rich with your idea?
Gnu Lesser Public License, Apache
![Page 73: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/73.jpg)
9. Tool Papers
The first one will be cited
The rest no one will cite
It shouldn’t be this way
![Page 74: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/74.jpg)
10. Robustness
Creating a robust tool…
…is a huge effort
…it will never be finished
EvoSuite is a black hole swallowing all my time!
![Page 75: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/75.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 76: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/76.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 77: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/77.jpg)
Stuff we are working on...
• Increasing coverage…
• Readability optimisation
• Better environment handling
• Mocking and private reflection
• Finding out how developers benefit most from using test generation
• User studies, replications
![Page 78: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/78.jpg)
Method Names
@Test(timeout = 4000) public void testFooReturningFalse() throws Throwable { StringExample stringExample0 = new StringExample(); boolean boolean0 = stringExample0.foo(""); assertFalse(boolean0); }
@Test(timeout = 4000) public void test3() throws Throwable { StringExample stringExample0 = new StringExample(); boolean boolean0 = stringExample0.foo(""); assertFalse(boolean0); }
![Page 79: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/79.jpg)
Variable Names @Test(timeout = 4000) public void testFooReturningFalse() throws Throwable { StringExample stringExample0 = new StringExample(); boolean boolean0 = stringExample0.foo(""); assertFalse(boolean0); }
@Test(timeout = 4000) public void testFooReturningFalse() throws Throwable { StringExample invokesFoo = new StringExample(); boolean resultFromFoo = invokesFoo.foo(""); assertFalse(resultFromFoo); }
![Page 80: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/80.jpg)
Variable Namespublic class Foo { public void foo() { StringExample sx = new StringExample(); boolean bar = sx.foo(""); }}
@Test(timeout = 4000) public void testFooReturningFalse() throws Throwable { StringExample sx = new StringExample(); boolean bar = sx.foo(""); assertFalse(bar); }
![Page 81: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/81.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 82: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/82.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 83: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/83.jpg)
1. SBST is Slow
• Fitness evaluation means executing tests
• Executing tests is slow
• How to reduce the number of fitness evaluations?
• How to improve search operators?
• Can we use ML to predict test execution results?
![Page 84: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/84.jpg)
2. OO Guidance
• Object oriented code has a terrible search landscape
• Complex dependency objects are a problem
• Include dependency objects in fitness functions?
• Better testability transformations?
• Better fitness functions?
![Page 85: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/85.jpg)
3. New Features
• Integration testing
• Concurrent code
• GUI handling code
• Database dependent code
• Prioritising tests
![Page 86: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/86.jpg)
4. SBST Usability
• Assertion/contract testing code?
• Coverage isn’t a great objective
• Usability as optimisation goal
• Study developers using SBST tools
![Page 87: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/87.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 88: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/88.jpg)
Outline• History
• Using EvoSuite
• Extending EvoSuite
• When to use and not to use EvoSuite
• Lessons learned building an SBST tool
• Things we are working on
• Ideas for future work
![Page 89: Tutorial - Computer Science & Eggay/sbst2016/slides/tutorial.pdfGordon Fraser, University of Sheffield, UK Andrea Arcuri, Simula Research Labs, Norway Gordon Fraser, University of](https://reader030.vdocuments.us/reader030/viewer/2022011801/5b05ab177f8b9a5c308bd2f3/html5/thumbnails/89.jpg)
Online Tutorials
• Using EvoSuite on the command line: http://www.evosuite.org/documentation/tutorial-part-1/
• Using EvoSuite with Maven: http://www.evosuite.org/documentation/tutorial-part-2/
• Running experiments with EvoSuite: http://www.evosuite.org/documentation/tutorial-part-3/
• Extending EvoSuite: http://www.evosuite.org/documentation/tutorial-part-4/