objecves - washington and lee universitysprenkle/cs209/in_class/13-inclass.pdf · 2016-11-08 ·...

11/8/16

1

Objec&ves• Tes&ng

Oct12,2016 Sprenkle-CSCI209 1

NoSilverBullet:EssenceandAccidentsofSoHwareEngineering“Ofallthemonstersthatfillthenightmaresofourfolklore,noneterrifymorethanwerewolves,becausetheytransformunexpectedlyfromthefamiliarintohorrors.Forthese,oneseeksbulletsofsilverthatcanmagicallylaythemtorest.“ThefamiliarsoHwareproject,atleastasseenbythenontechnicalmanager,hassomethingofthischaracter;itisusuallyinnocentandstraighSorward,butiscapableofbecomingamonsterofmissedschedules,blownbudgets,andflawedproducts.Soweheardesperatecriesforasilverbullet--somethingtomakesoHwarecostsdropasrapidlyascomputerhardwarecostsdo.“But,aswelooktothehorizonofadecadehence,weseenosilverbullet.Thereisnosingledevelopment,ineithertechnologyorinmanagementtechnique,thatbyitselfpromisesevenoneorder-of-magnitudeimprovementinproduc&vity,inreliability,insimplicity.Inthisar&cle,Ishalltrytoshowwhy,byexaminingboththenatureofthesoHwareproblemandtheproper&esofthebulletsproposed.” Oct10,2016 Sprenkle-CSCI209 2

by Frederick P. Brooks, Jr., 1986

11/8/16

2

WhoisFredBrooks?• UNCProfessor• TuringAwardwinner• “ThemostimportantsingledecisionIevermadewastochangetheIBM360seriesfroma6-bitbytetoan8-bitbyte,therebyenablingtheuseoflowercasele_ers.Thatchangepropagatedeverywhere.”

Oct10,2016 Sprenkle-CSCI209 3

Tradi&onalSoHwareEngineeringProcess:WaterfallModel

Oct10,2016 Sprenkle-CSCI209 4

Requirements

Design

Implementa&on

Integra&on

Acceptance

Release/Maintenance

Validate at each stepGoal: A stage is 100% complete before moving to next step

11/8/16

3

FeedbackinWaterfallModel

Oct10,2016 Sprenkle-CSCI209 5

• Problems may be revealed in later stages

• What happens if problems aren’t revealed until Acceptance?

Requirements

Design

Implementa&on

Integra&on

Acceptance

Release/Maintenance

Itera&veDesign

Oct10,2016 Sprenkle-CSCI209 6

Design

Evaluate Implement

Get feedback from users/clients

11/8/16

4

SpiralModel•  Idea:smallerprototypestotest/fix/throwawayØ  Findingproblemsearly

costsless•  Ingeneral…

Ø  Breakfunc&onalityintosmallerpieces

Ø  Implementmostdepended-onorhighest-priorityfeaturesfirst

Oct10,2016 Sprenkle-CSCI209 7

Design

ImplementEvaluate

Prototypes

Radial dimension: cost[Boehm 86]

Prototypes• Purpose/Dimensions

Ø Func&onalityØ  Interac&onØ  Implementa&on

• Fidelity:Ø Low:omitsdetailsØ High:closertofinishedprojectØ Mul&-dimensional

•  Breadth:%offeaturescoveredØ Onlyenoughfeaturesforcertaintasks

• Depth:degreeoffunc&onalityØ Limitedchoices,cannedresponses,noerrorhandling

Oct10,2016 Sprenkle-CSCI209 8

From Nielsen, �Usability Engineering

11/8/16

5

LowFidelityPrototypes• Media:Paper• Examples:storyboard,sketches,flipbook,flowdiagram

Oct10,2016 Sprenkle-CSCI209 9

HighFidelityPrototypes• Media:Flash,HTML(non-interac&ve),PowerPoint,Video

• Examples:Mockups,WizardofOz

Oct10,2016 Sprenkle-CSCI209 10

Virtual Peer for Autistic Children

http://www.articulab.justinecassell.com/projects/samautism/index.html

11/8/16

6

HowtoImplementanEffec&veSolu&on• Understandtheproblem(interactwithpeople)• Understandexternalconstraints(interactwithpeople)

• Designaneffec&vesolu&ontotheproblem• Whiledesigningthesolu&on,designsometeststoverifythattheproblemissolved(andremainssolved)

• Codetheeffec&vesolu&ontotheproblem• Teachotherteammembersaboutyoursolu&ontotheproblem(interactwithpeople)

Oct10,2016 Sprenkle-CSCI209 11

SpiralModelSteps• Designa{method,class,package}•  Implementthe{method,class,package}• Testthe{method,class,package}• Fixthe{method,class,package}• Deploythe{method,class,package}• Getfeedback

Ø Probablywillrequiremodifica&onstodesignØ Mayevenneedtorollbackapreviousversion

• RepeatOct10,2016 Sprenkle-CSCI209 12

11/8/16

7

AgileDevelopmentFramework:Scrum• TheScrumframeworkin30seconds

Ø Productownercreatespriori&zedwishlist,aproductbacklog

Ø Teamworksinasprint,usually2-4weeks• Duringplanning,teampicksasubsetofwishlist,asprintbacklog,anddecideshowtoimplementthosepieces

• DailyScrum:teammeetsdailytoassessitsprogressØ ScrumMasterkeepstheteamfocusedonitsgoal

•  Atendofsprint,workshouldbepoten&allyshippable:Ø readytohandtoacustomer,putonastoreshelf,orshowtoastakeholder

•  Thesprintendswithasprintreviewandretrospec&veØ Repeatsprint

Oct10,2016 Sprenkle-CSCI209 13

https://www.scrumalliance.org/why-scrum

SOFTWARETESTINGPROCESS

Oct12,2016 Sprenkle-CSCI209 14

11/8/16

8

ABadRoleModel

Oct12,2016 Sprenkle-CSCI209 15http://imgur.com/HBSbn

MicrosoHTes&ng• Beyondtheirinternaltes&ng…

Ø 5millionpeoplebetatestedØ 60+yearsofperformancetes&ngØ 1Billion+Office2007sessions

• S&ll,usersfoundcorrectness,stability,robustness,andsecuritybugs

Oct12,2016 Sprenkle-CSCI209 16

11/8/16

9

Type1Bugs:Compile-Time

• SyntaxerrorsØ Missingsemicolon,parentheses

• Compilerno&fiesoferror• Cheap,easytofix

Oct12,2016 Sprenkle-CSCI209 17

Type2Bugs:Run-Time

• Usuallylogicerrors• Expensivetolocate,fix

Oct12,2016 Sprenkle-CSCI209 18

11/8/16

10

Aside:Objec&onsto“Bug”Terminology• “Bug”

Ø Soundslikeit’sjustanannoyance• Cansimplyswataway

Ø Minimizespoten&alproblemsØ Hidesprogrammer’sresponsibility

• Alterna&vetermsØ DefectØ Fault

Oct12,2016 Sprenkle-CSCI209 19

SoHwareTes&ngProcess

• TestSuite:setoftestcases

Oct12,2016 Sprenkle-CSCI209 20

Input Program Output

Test Case

Program Under Test

ExpectedOutput ?

pass or fail

11/8/16

11

SoHwareTes&ngProcess

• Testerplaysdevil’sadvocateØ Hopestorevealproblemsintheprogramusing“good”testcases

Ø Be_ertesterfindsthanacustomer!

Oct12,2016 Sprenkle-CSCI209 21

Input Program Output

How is testing different from debugging?

HowWouldYouTestaCalculatorProgram?

• Whattestcases:inputandexpectedoutput?

Oct12,2016 Sprenkle-CSCI209 22

Numerical Answer

adds, subtracts, multiplies, divides

Operands, operators, expected

output

Input CalculatorProgram Output

11/8/16

12

ExampleTestCasesforCalculatorProgram

•  BasicFunc&onalityØ Addi&onØ  Subtrac&onØ Mul&plica&onØ DivisionØ Orderofopera&ons

•  InvalidInputØ  Le_ers,not-opera&on

characters(&,$,…)

•  “Tricky”CasesØ Divideby0Ø Nega&veNumbersØ  Longsequencesof

operands,operatorsØ  VERYlarge,VERYsmall

numbers

Oct12,2016 Sprenkle-CSCI209 23

TypesofTes&ng(Non-Exhaus&ve)•  Black-boxtes&ng

• White-boxtes&ng

•  Non-func&onaltes&ng

•  Acceptancetes&ng

Oct12,2016 Sprenkle-CSCI209 24

Ideas or definitions of any of these?

11/8/16

13

TypesofTes&ng(Non-Exhaus&ve)•  Black-boxtes&ng

Ø  Testfunc2onality(e.g.,thecalculator)

Ø NoknowledgeofthecodeØ  Examplesoftes&ng:

boundaryvalues

• White-boxtes&ngØ HaveaccesstocodeØ Goal:executeallcode

•  Non-func&onaltes&ngØ  Performancetes&ngØ Usabilitytes&ng(HCI)Ø  Securitytes&ngØ  Interna&onaliza&on,

localiza&on

•  Acceptancetes&ngØ  Customerteststodecide

ifacceptsproduct

Oct12,2016 Sprenkle-CSCI209 25

LevelsofTes&ng• Unit

Ø TestsminimalsoHwarecomponent,inisola&onØ Forus,Class-leveltes&ngØ Web:Webpages(H_pRequest)

•  Integra&onØ Testsinterfaces&interac&onofclasses

• SystemØ Teststhatcompletelyintegratedsystemmeetsrequirements

• SystemIntegra&onØ Testsystemworkswithothersystems,e.g.,third-partysystems

Oct12,2016 Sprenkle-CSCI209 26

Cost increases

11/8/16

14

UNITTESTING

Oct12,2016 Sprenkle-CSCI209 27

WhyUnitTest?• Verifycodeworksasintendedinisola&on• Finddefectsearlyindevelopment

Ø EasiertotestsmallpiecesØ Lesscostthanatlaterstages

Oct12,2016 Sprenkle-CSCI209 28

11/8/16

15

WhyUnitTest?• Verifycodeworksasintendedinisola&on• Finddefectsearlyindevelopment

Ø EasiertotestsmallpiecesØ Lesscostthanatlaterstages

• Asapplica&onevolves,newcodeismorelikelytobreakexis&ngcodeØ Suiteof(small)testcasestorunaHercodechangesØ Alsocalledregressiontes&ng

Oct12,2016 Sprenkle-CSCI209 29

SomeApproachestoTes&ngMethods• Typicalcase

Ø Testtypicalvaluesofinput/parameters• Boundarycondi&ons

Ø Testatboundariesofinput/parametersØ Manyfaultslive“incorners”

• Parametervalida&onØ Verifythatparameterandobjectboundsaredocumentedandchecked

Ø Example:pre-condi&onthatparameterisn’tnull

Oct12,2016 Sprenkle-CSCI209 30

➥  All black-box testing approaches

11/8/16

16

AnotherUseofUnitTes&ng: Test-DrivenDevelopment

• Adevelopmentstyle,evolvedfromExtremeProgramming

•  Idea:writetestsfirstwithoutcodebias• TheProcess:

1.  Writeteststhatcode/newfunc&onalityshouldpass• Likeaspecifica&onforthecode(pre/postcondi&ons)• Alltestswillini&allyfail

2.  Writethecodeandverifythatitpassestestcases• Knowyou’redonecodingwhenyoupassalltests

Oct12,2016 Sprenkle-CSCI209 31

What assumption does this make?

How do you know you’re “done” in traditional development?

SoHwareTes&ngIssues• Howshouldyoutest?HowoHen?

Ø CodemaychangefrequentlyØ Codemaydependonothers’codeØ Alotofcodetovalidate

• Howdoyouknowthatanoutputiscorrect?Ø ComplexoutputØ Humanjudgment?

• Whatcausedacodefailure?

Oct12,2016 Sprenkle-CSCI209 32

➥  Need a systematic, automated, repeatable approach

11/8/16

17

Characteris&csofGoodUnitTes&ng

• AutomaEc• Thorough• Repeatable• Independent

Oct12,2016 Sprenkle-CSCI209 33

Why are these characteristics ofgood (unit) testing?

Characteris&csofGoodUnitTes&ng• AutomaEc

Ø Sinceunittes&ngisdonefrequently,don’twanthumansslowingtheprocessdown

Ø Automateexecu&ngtestcasesandevalua&ngresultsØ  Input:intestitselforfromafile

• ThoroughØ Coversallcode/func&onality/cases

• RepeatableØ Reproduceresults(correct,failures)

•  IndependentØ TestcasesareindependentfromeachotherØ Easiertotracefaulttocode

Oct12,2016 Sprenkle-CSCI209 34

11/8/16

18

JUNIT

Oct12,2016 Sprenkle-CSCI209 35

JUnitFramework• Aframeworkforunittes&ngJavaprograms

Ø SupportedbyEclipseandotherIDEsØ DevelopedbyErichGammaandKentBeck

• Func&onalityØ Writetests

•  Validateoutput,automa&callyØ Automateexecu&onoftestsuitesØ Displaypass/failresultsoftestexecu&on

•  StacktracewherefailsØ Organizetests,separatefromcode

Oct12,2016 Sprenkle-CSCI209 36

Kent Beck

Erich Gamma

But,yousEllneedtocomeupwiththetests!

11/8/16

19

Aside:Framework

Oct12,2016 Sprenkle-CSCI209 37

A framework is a basic conceptual structure used to solve or address complex issues.

This very broad definition has allowed the term to be used as a buzzword, especially in a software context.

Tes&ngwithJUnit• Typicalorganiza&on:

Ø Setoftes&ngclassesØ Tes&ngclassespackagedtogetherinatests package• Separatepackagefromcodetes&ng

• AtestclasstypicallyØ FocusesonaspecificclassØ Containsmethods,eachofwhichrepresentsanothertestoftheclass

Oct12,2016 Sprenkle-CSCI209 38

testsCDTestDVDTestMediaItemTest

11/8/16

20

StructureofaJUnitTest1.  Setupthetestcase(op&onal)

Ø  Example:Crea&ngobjects

2.  Exercisethecodeundertest3.  Verifythecorrectnessoftheresults4.  Teardown(op&onal)

Ø Example:reclaimcreatedobjects

Oct12,2016 Sprenkle-CSCI209 39

Annota&ons• Tes&nginJUnit4:usesannotaEons• Providedataaboutaprogramthatisnotpartofprogramitself

• Havenodirecteffectonopera&onofthecode• Exampleuses:

Ø @Override:methoddeclara&onisintendedtooverrideamethoddeclara&oninparentclass•  Ifmethoddoesnotoverrideparentclassmethod,compilergenerateserrormessage

Ø Informa&onforthecompilertosuppresswarnings(@SupressWarnings)

Oct12,2016 Sprenkle-CSCI209 40

11/8/16

21

TestsareMethods• Markyourtes&ngmethodwith@Test

Ø From org.junit.Test

• Conven&on:Methodnamedescribeswhatyou’retes&ng

Oct12,2016 Sprenkle-CSCI209 41

public class CalculatorTest {

@Testpublic void addTest() {

…}

}

A method to test the “add” functionality

Class for testing the Calculator class

AssertMethods• Varietyofassertmethodsavailable•  Iffail,throwanexcep&on• Otherwise,testkeepsexecu&ng• Allstatic void• Example: assertEquals(Object expected, Object actual)

Oct12,2016 Sprenkle-CSCI209 42

@Testpublic void addTest() {

… assertEquals(4, calculator.add(3, 1));

}

11/8/16

22

AssertMethods• Touseasserts,needsta2cimport:

Ø static allowsustonothavetouseclassname

• MoreexamplesØ assertTrue(boolean condition)Ø assertSame(Object expected, Object actual)

•  Refer to same object Ø assertEquals(double expected, double actual, double delta)

Oct12,2016 Sprenkle-CSCI209 43

import static org.junit.Assert.*;

ExampleUsesofAssertMethods

Oct12,2016 Sprenkle-CSCI209 44

@Testpublic void testEmptyCollection() {

Collection collection = new ArrayList(); assertTrue(collection.isEmpty());}

@Testpublic void testPI() {

final double ERROR_TOLERANCE = .01;assertEquals(Math.PI, 3.14, ERROR_TOLERANCE);

}

Will fail if ERROR_TOLERANCE = .001

assertEquals(double expected, double actual, double delta)

11/8/16

23

SetUp/TearDown• MaywantmethodstosetupobjectsforeverytestintheclassØ CalledfixturesØ Ifhavemul&ple,noguaranteesfororderexecuted

Oct12,2016 Sprenkle-CSCI209 45

@Beforepublic void prepareTestData() { ... }

@Beforepublic void setupMocks() { ... }

@Afterpublic void cleanupTestData() { ... }

Executed before each test method

ExampleSetUpMethod

Oct12,2016 Sprenkle-CSCI209 46

@Before Executed before each test methodCan use testCD in test methods

private CD testCD;

@Beforepublic void setUp() {

testCD = new CD("CD title", 100, 1997, "CD Artist", 11);

}

11/8/16

24

Expec&nganExcep&on• HandlingErrorCases

Ø Some&mesanexcep&onistheexpectedresult

Oct12,2016 Sprenkle-CSCI209 47

@Test(expected=IndexOutOfBoundsException.class)public void testIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0);}

Add an “expected” attribute:

Test case passes iff exception thrown

SetUp/TearDownForClass• MaywantmethodstosetupobjectsforsetoftestsØ Executedoncebeforeanytestinclassexecutes

Oct12,2016 Sprenkle-CSCI209 48

@BeforeClasspublic static void setupDatabaseConnection() { ... }

@AfterClasspublic static void teardownDatabaseConnection() { ... }

11/8/16

25

JUNITINECLIPSE

Oct12,2016 Sprenkle-CSCI209 49

UsingJUnitinEclipse• Eclipsecanhelpmakeourjobeasier

Ø Automa&callyexecutetests(i.e.,methods)Ø Wecanfocusoncomingupwithtests

Oct12,2016 Sprenkle-CSCI209 50

11/8/16

26

UsingJUnitinEclipse•  InEclipse,gotoyourMediaItem project• CreateanewJUnitTestCase(underJava)

Ø UseJUnit4• Addjunittobuildpath

Ø Putinpackagemedia.testsØ Name:DVDTestØ ChoosetotestDVD class

•  Select setUp and tearDown• Selectmethodstotest

• RuntheclassasaJUnitTestCaseOct12,2016 Sprenkle-CSCI209 51

Example• TestmethodthatgetsthelengthoftheDVD

Ø Revise:AddcodetosetUp methodthatcreatesaDVD

• NotesØ Replayingallthetestcases:rightclickonpackageØ FastViewvsDetachedØ Hint:CTL-Spacebartogetauto-completeop&ons

Oct12,2016 Sprenkle-CSCI209 52

11/8/16

27

UnitTes&ng&JUnitSummary• UnitTes&ng:tes&ngsmallestcomponentofyourcodeØ Forus:classanditsmethods

• JUnitprovidesframeworktowritetestcasesandruntestcasesautoma&callyØ EasytorunagainaHercodechanges

• JUnitResourcesavailablefromCoursePage’s“Resource”Link,underJavaØ APIØ Tutorials

Oct12,2016 Sprenkle-CSCI209 53

Project1:Tes&ngPrac&ce• Given:aCar classthatonlyhasenoughcodetocompile

• Yourjob:Createagoodsetoftestcasesthatthoroughly/effec2velytestCar classØ FindfaultsinmyfaultyversionofCar classØ Start:lookatcode,thinkabouthowtotest,setupJUnittests

Ø Wri_enanalysisofprocess

Oct12,2016 Sprenkle-CSCI209 54

11/8/16

28

Project1:Tes&ngPrac&ce• 1st:EmailmeandyourteammatewiththenameofyourteamØ Iwillcreatearepositorythatthepaircanworkontogether

Oct12,2016 Sprenkle-CSCI209 55

LookingAhead• MoreTes&ng!• Extracreditassignment

Oct12,2016 Sprenkle-CSCI209 56