objecves - washington and lee universitysprenkle/cs209/in_class/13-inclass.pdf · 2016-11-08 ·...
TRANSCRIPT
11/8/16
1
Objec&ves• Tes&ng
Oct12,2016 Sprenkle-CSCI209 1
NoSilverBullet:EssenceandAccidentsofSoHwareEngineering“Ofallthemonstersthatfillthenightmaresofourfolklore,noneterrifymorethanwerewolves,becausetheytransformunexpectedlyfromthefamiliarintohorrors.Forthese,oneseeksbulletsofsilverthatcanmagicallylaythemtorest.“ThefamiliarsoHwareproject,atleastasseenbythenontechnicalmanager,hassomethingofthischaracter;itisusuallyinnocentandstraighSorward,butiscapableofbecomingamonsterofmissedschedules,blownbudgets,andflawedproducts.Soweheardesperatecriesforasilverbullet--somethingtomakesoHwarecostsdropasrapidlyascomputerhardwarecostsdo.“But,aswelooktothehorizonofadecadehence,weseenosilverbullet.Thereisnosingledevelopment,ineithertechnologyorinmanagementtechnique,thatbyitselfpromisesevenoneorder-of-magnitudeimprovementinproduc&vity,inreliability,insimplicity.Inthisar&cle,Ishalltrytoshowwhy,byexaminingboththenatureofthesoHwareproblemandtheproper&esofthebulletsproposed.” Oct10,2016 Sprenkle-CSCI209 2
by Frederick P. Brooks, Jr., 1986
11/8/16
2
WhoisFredBrooks?• UNCProfessor• TuringAwardwinner• “ThemostimportantsingledecisionIevermadewastochangetheIBM360seriesfroma6-bitbytetoan8-bitbyte,therebyenablingtheuseoflowercasele_ers.Thatchangepropagatedeverywhere.”
Oct10,2016 Sprenkle-CSCI209 3
Tradi&onalSoHwareEngineeringProcess:WaterfallModel
Oct10,2016 Sprenkle-CSCI209 4
Requirements
Design
Implementa&on
Integra&on
Acceptance
Release/Maintenance
Validate at each stepGoal: A stage is 100% complete before moving to next step
11/8/16
3
FeedbackinWaterfallModel
Oct10,2016 Sprenkle-CSCI209 5
• Problems may be revealed in later stages
• What happens if problems aren’t revealed until Acceptance?
Requirements
Design
Implementa&on
Integra&on
Acceptance
Release/Maintenance
Itera&veDesign
Oct10,2016 Sprenkle-CSCI209 6
Design
Evaluate Implement
Get feedback from users/clients
11/8/16
4
SpiralModel• Idea:smallerprototypestotest/fix/throwawayØ Findingproblemsearly
costsless• Ingeneral…
Ø Breakfunc&onalityintosmallerpieces
Ø Implementmostdepended-onorhighest-priorityfeaturesfirst
Oct10,2016 Sprenkle-CSCI209 7
Design
ImplementEvaluate
Prototypes
Radial dimension: cost[Boehm 86]
Prototypes• Purpose/Dimensions
Ø Func&onalityØ Interac&onØ Implementa&on
• Fidelity:Ø Low:omitsdetailsØ High:closertofinishedprojectØ Mul&-dimensional
• Breadth:%offeaturescoveredØ Onlyenoughfeaturesforcertaintasks
• Depth:degreeoffunc&onalityØ Limitedchoices,cannedresponses,noerrorhandling
Oct10,2016 Sprenkle-CSCI209 8
From Nielsen, �Usability Engineering
11/8/16
5
LowFidelityPrototypes• Media:Paper• Examples:storyboard,sketches,flipbook,flowdiagram
Oct10,2016 Sprenkle-CSCI209 9
HighFidelityPrototypes• Media:Flash,HTML(non-interac&ve),PowerPoint,Video
• Examples:Mockups,WizardofOz
Oct10,2016 Sprenkle-CSCI209 10
Virtual Peer for Autistic Children
http://www.articulab.justinecassell.com/projects/samautism/index.html
11/8/16
6
HowtoImplementanEffec&veSolu&on• Understandtheproblem(interactwithpeople)• Understandexternalconstraints(interactwithpeople)
• Designaneffec&vesolu&ontotheproblem• Whiledesigningthesolu&on,designsometeststoverifythattheproblemissolved(andremainssolved)
• Codetheeffec&vesolu&ontotheproblem• Teachotherteammembersaboutyoursolu&ontotheproblem(interactwithpeople)
Oct10,2016 Sprenkle-CSCI209 11
SpiralModelSteps• Designa{method,class,package}• Implementthe{method,class,package}• Testthe{method,class,package}• Fixthe{method,class,package}• Deploythe{method,class,package}• Getfeedback
Ø Probablywillrequiremodifica&onstodesignØ Mayevenneedtorollbackapreviousversion
• RepeatOct10,2016 Sprenkle-CSCI209 12
11/8/16
7
AgileDevelopmentFramework:Scrum• TheScrumframeworkin30seconds
Ø Productownercreatespriori&zedwishlist,aproductbacklog
Ø Teamworksinasprint,usually2-4weeks• Duringplanning,teampicksasubsetofwishlist,asprintbacklog,anddecideshowtoimplementthosepieces
• DailyScrum:teammeetsdailytoassessitsprogressØ ScrumMasterkeepstheteamfocusedonitsgoal
• Atendofsprint,workshouldbepoten&allyshippable:Ø readytohandtoacustomer,putonastoreshelf,orshowtoastakeholder
• Thesprintendswithasprintreviewandretrospec&veØ Repeatsprint
Oct10,2016 Sprenkle-CSCI209 13
https://www.scrumalliance.org/why-scrum
SOFTWARETESTINGPROCESS
Oct12,2016 Sprenkle-CSCI209 14
11/8/16
8
ABadRoleModel
Oct12,2016 Sprenkle-CSCI209 15http://imgur.com/HBSbn
MicrosoHTes&ng• Beyondtheirinternaltes&ng…
Ø 5millionpeoplebetatestedØ 60+yearsofperformancetes&ngØ 1Billion+Office2007sessions
• S&ll,usersfoundcorrectness,stability,robustness,andsecuritybugs
Oct12,2016 Sprenkle-CSCI209 16
11/8/16
9
Type1Bugs:Compile-Time
• SyntaxerrorsØ Missingsemicolon,parentheses
• Compilerno&fiesoferror• Cheap,easytofix
Oct12,2016 Sprenkle-CSCI209 17
Type2Bugs:Run-Time
• Usuallylogicerrors• Expensivetolocate,fix
Oct12,2016 Sprenkle-CSCI209 18
11/8/16
10
Aside:Objec&onsto“Bug”Terminology• “Bug”
Ø Soundslikeit’sjustanannoyance• Cansimplyswataway
Ø Minimizespoten&alproblemsØ Hidesprogrammer’sresponsibility
• Alterna&vetermsØ DefectØ Fault
Oct12,2016 Sprenkle-CSCI209 19
SoHwareTes&ngProcess
• TestSuite:setoftestcases
Oct12,2016 Sprenkle-CSCI209 20
Input Program Output
Test Case
Program Under Test
ExpectedOutput ?
pass or fail
11/8/16
11
SoHwareTes&ngProcess
• Testerplaysdevil’sadvocateØ Hopestorevealproblemsintheprogramusing“good”testcases
Ø Be_ertesterfindsthanacustomer!
Oct12,2016 Sprenkle-CSCI209 21
Input Program Output
How is testing different from debugging?
HowWouldYouTestaCalculatorProgram?
• Whattestcases:inputandexpectedoutput?
Oct12,2016 Sprenkle-CSCI209 22
Numerical Answer
adds, subtracts, multiplies, divides
Operands, operators, expected
output
Input CalculatorProgram Output
11/8/16
12
ExampleTestCasesforCalculatorProgram
• BasicFunc&onalityØ Addi&onØ Subtrac&onØ Mul&plica&onØ DivisionØ Orderofopera&ons
• InvalidInputØ Le_ers,not-opera&on
characters(&,$,…)
• “Tricky”CasesØ Divideby0Ø Nega&veNumbersØ Longsequencesof
operands,operatorsØ VERYlarge,VERYsmall
numbers
Oct12,2016 Sprenkle-CSCI209 23
TypesofTes&ng(Non-Exhaus&ve)• Black-boxtes&ng
• White-boxtes&ng
• Non-func&onaltes&ng
• Acceptancetes&ng
Oct12,2016 Sprenkle-CSCI209 24
Ideas or definitions of any of these?
11/8/16
13
TypesofTes&ng(Non-Exhaus&ve)• Black-boxtes&ng
Ø Testfunc2onality(e.g.,thecalculator)
Ø NoknowledgeofthecodeØ Examplesoftes&ng:
boundaryvalues
• White-boxtes&ngØ HaveaccesstocodeØ Goal:executeallcode
• Non-func&onaltes&ngØ Performancetes&ngØ Usabilitytes&ng(HCI)Ø Securitytes&ngØ Interna&onaliza&on,
localiza&on
• Acceptancetes&ngØ Customerteststodecide
ifacceptsproduct
Oct12,2016 Sprenkle-CSCI209 25
LevelsofTes&ng• Unit
Ø TestsminimalsoHwarecomponent,inisola&onØ Forus,Class-leveltes&ngØ Web:Webpages(H_pRequest)
• Integra&onØ Testsinterfaces&interac&onofclasses
• SystemØ Teststhatcompletelyintegratedsystemmeetsrequirements
• SystemIntegra&onØ Testsystemworkswithothersystems,e.g.,third-partysystems
Oct12,2016 Sprenkle-CSCI209 26
Cost increases
11/8/16
14
UNITTESTING
Oct12,2016 Sprenkle-CSCI209 27
WhyUnitTest?• Verifycodeworksasintendedinisola&on• Finddefectsearlyindevelopment
Ø EasiertotestsmallpiecesØ Lesscostthanatlaterstages
Oct12,2016 Sprenkle-CSCI209 28
11/8/16
15
WhyUnitTest?• Verifycodeworksasintendedinisola&on• Finddefectsearlyindevelopment
Ø EasiertotestsmallpiecesØ Lesscostthanatlaterstages
• Asapplica&onevolves,newcodeismorelikelytobreakexis&ngcodeØ Suiteof(small)testcasestorunaHercodechangesØ Alsocalledregressiontes&ng
Oct12,2016 Sprenkle-CSCI209 29
SomeApproachestoTes&ngMethods• Typicalcase
Ø Testtypicalvaluesofinput/parameters• Boundarycondi&ons
Ø Testatboundariesofinput/parametersØ Manyfaultslive“incorners”
• Parametervalida&onØ Verifythatparameterandobjectboundsaredocumentedandchecked
Ø Example:pre-condi&onthatparameterisn’tnull
Oct12,2016 Sprenkle-CSCI209 30
➥ All black-box testing approaches
11/8/16
16
AnotherUseofUnitTes&ng: Test-DrivenDevelopment
• Adevelopmentstyle,evolvedfromExtremeProgramming
• Idea:writetestsfirstwithoutcodebias• TheProcess:
1. Writeteststhatcode/newfunc&onalityshouldpass• Likeaspecifica&onforthecode(pre/postcondi&ons)• Alltestswillini&allyfail
2. Writethecodeandverifythatitpassestestcases• Knowyou’redonecodingwhenyoupassalltests
Oct12,2016 Sprenkle-CSCI209 31
What assumption does this make?
How do you know you’re “done” in traditional development?
SoHwareTes&ngIssues• Howshouldyoutest?HowoHen?
Ø CodemaychangefrequentlyØ Codemaydependonothers’codeØ Alotofcodetovalidate
• Howdoyouknowthatanoutputiscorrect?Ø ComplexoutputØ Humanjudgment?
• Whatcausedacodefailure?
Oct12,2016 Sprenkle-CSCI209 32
➥ Need a systematic, automated, repeatable approach
11/8/16
17
Characteris&csofGoodUnitTes&ng
• AutomaEc• Thorough• Repeatable• Independent
Oct12,2016 Sprenkle-CSCI209 33
Why are these characteristics ofgood (unit) testing?
Characteris&csofGoodUnitTes&ng• AutomaEc
Ø Sinceunittes&ngisdonefrequently,don’twanthumansslowingtheprocessdown
Ø Automateexecu&ngtestcasesandevalua&ngresultsØ Input:intestitselforfromafile
• ThoroughØ Coversallcode/func&onality/cases
• RepeatableØ Reproduceresults(correct,failures)
• IndependentØ TestcasesareindependentfromeachotherØ Easiertotracefaulttocode
Oct12,2016 Sprenkle-CSCI209 34
11/8/16
18
JUNIT
Oct12,2016 Sprenkle-CSCI209 35
JUnitFramework• Aframeworkforunittes&ngJavaprograms
Ø SupportedbyEclipseandotherIDEsØ DevelopedbyErichGammaandKentBeck
• Func&onalityØ Writetests
• Validateoutput,automa&callyØ Automateexecu&onoftestsuitesØ Displaypass/failresultsoftestexecu&on
• StacktracewherefailsØ Organizetests,separatefromcode
Oct12,2016 Sprenkle-CSCI209 36
Kent Beck
Erich Gamma
But,yousEllneedtocomeupwiththetests!
11/8/16
19
Aside:Framework
Oct12,2016 Sprenkle-CSCI209 37
A framework is a basic conceptual structure used to solve or address complex issues.
This very broad definition has allowed the term to be used as a buzzword, especially in a software context.
Tes&ngwithJUnit• Typicalorganiza&on:
Ø Setoftes&ngclassesØ Tes&ngclassespackagedtogetherinatests package• Separatepackagefromcodetes&ng
• AtestclasstypicallyØ FocusesonaspecificclassØ Containsmethods,eachofwhichrepresentsanothertestoftheclass
Oct12,2016 Sprenkle-CSCI209 38
testsCDTestDVDTestMediaItemTest
11/8/16
20
StructureofaJUnitTest1. Setupthetestcase(op&onal)
Ø Example:Crea&ngobjects
2. Exercisethecodeundertest3. Verifythecorrectnessoftheresults4. Teardown(op&onal)
Ø Example:reclaimcreatedobjects
Oct12,2016 Sprenkle-CSCI209 39
Annota&ons• Tes&nginJUnit4:usesannotaEons• Providedataaboutaprogramthatisnotpartofprogramitself
• Havenodirecteffectonopera&onofthecode• Exampleuses:
Ø @Override:methoddeclara&onisintendedtooverrideamethoddeclara&oninparentclass• Ifmethoddoesnotoverrideparentclassmethod,compilergenerateserrormessage
Ø Informa&onforthecompilertosuppresswarnings(@SupressWarnings)
Oct12,2016 Sprenkle-CSCI209 40
11/8/16
21
TestsareMethods• Markyourtes&ngmethodwith@Test
Ø From org.junit.Test
• Conven&on:Methodnamedescribeswhatyou’retes&ng
Oct12,2016 Sprenkle-CSCI209 41
public class CalculatorTest {
@Testpublic void addTest() {
…}
}
A method to test the “add” functionality
Class for testing the Calculator class
AssertMethods• Varietyofassertmethodsavailable• Iffail,throwanexcep&on• Otherwise,testkeepsexecu&ng• Allstatic void• Example: assertEquals(Object expected, Object actual)
Oct12,2016 Sprenkle-CSCI209 42
@Testpublic void addTest() {
… assertEquals(4, calculator.add(3, 1));
}
11/8/16
22
AssertMethods• Touseasserts,needsta2cimport:
Ø static allowsustonothavetouseclassname
• MoreexamplesØ assertTrue(boolean condition)Ø assertSame(Object expected, Object actual)
• Refer to same object Ø assertEquals(double expected, double actual, double delta)
Oct12,2016 Sprenkle-CSCI209 43
import static org.junit.Assert.*;
ExampleUsesofAssertMethods
Oct12,2016 Sprenkle-CSCI209 44
@Testpublic void testEmptyCollection() {
Collection collection = new ArrayList(); assertTrue(collection.isEmpty());}
@Testpublic void testPI() {
final double ERROR_TOLERANCE = .01;assertEquals(Math.PI, 3.14, ERROR_TOLERANCE);
}
Will fail if ERROR_TOLERANCE = .001
assertEquals(double expected, double actual, double delta)
11/8/16
23
SetUp/TearDown• MaywantmethodstosetupobjectsforeverytestintheclassØ CalledfixturesØ Ifhavemul&ple,noguaranteesfororderexecuted
Oct12,2016 Sprenkle-CSCI209 45
@Beforepublic void prepareTestData() { ... }
@Beforepublic void setupMocks() { ... }
@Afterpublic void cleanupTestData() { ... }
Executed before each test method
ExampleSetUpMethod
Oct12,2016 Sprenkle-CSCI209 46
@Before Executed before each test methodCan use testCD in test methods
private CD testCD;
@Beforepublic void setUp() {
testCD = new CD("CD title", 100, 1997, "CD Artist", 11);
}
11/8/16
24
Expec&nganExcep&on• HandlingErrorCases
Ø Some&mesanexcep&onistheexpectedresult
Oct12,2016 Sprenkle-CSCI209 47
@Test(expected=IndexOutOfBoundsException.class)public void testIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0);}
Add an “expected” attribute:
Test case passes iff exception thrown
SetUp/TearDownForClass• MaywantmethodstosetupobjectsforsetoftestsØ Executedoncebeforeanytestinclassexecutes
Oct12,2016 Sprenkle-CSCI209 48
@BeforeClasspublic static void setupDatabaseConnection() { ... }
@AfterClasspublic static void teardownDatabaseConnection() { ... }
11/8/16
25
JUNITINECLIPSE
Oct12,2016 Sprenkle-CSCI209 49
UsingJUnitinEclipse• Eclipsecanhelpmakeourjobeasier
Ø Automa&callyexecutetests(i.e.,methods)Ø Wecanfocusoncomingupwithtests
Oct12,2016 Sprenkle-CSCI209 50
11/8/16
26
UsingJUnitinEclipse• InEclipse,gotoyourMediaItem project• CreateanewJUnitTestCase(underJava)
Ø UseJUnit4• Addjunittobuildpath
Ø Putinpackagemedia.testsØ Name:DVDTestØ ChoosetotestDVD class
• Select setUp and tearDown• Selectmethodstotest
• RuntheclassasaJUnitTestCaseOct12,2016 Sprenkle-CSCI209 51
Example• TestmethodthatgetsthelengthoftheDVD
Ø Revise:AddcodetosetUp methodthatcreatesaDVD
• NotesØ Replayingallthetestcases:rightclickonpackageØ FastViewvsDetachedØ Hint:CTL-Spacebartogetauto-completeop&ons
Oct12,2016 Sprenkle-CSCI209 52
11/8/16
27
UnitTes&ng&JUnitSummary• UnitTes&ng:tes&ngsmallestcomponentofyourcodeØ Forus:classanditsmethods
• JUnitprovidesframeworktowritetestcasesandruntestcasesautoma&callyØ EasytorunagainaHercodechanges
• JUnitResourcesavailablefromCoursePage’s“Resource”Link,underJavaØ APIØ Tutorials
Oct12,2016 Sprenkle-CSCI209 53
Project1:Tes&ngPrac&ce• Given:aCar classthatonlyhasenoughcodetocompile
• Yourjob:Createagoodsetoftestcasesthatthoroughly/effec2velytestCar classØ FindfaultsinmyfaultyversionofCar classØ Start:lookatcode,thinkabouthowtotest,setupJUnittests
Ø Wri_enanalysisofprocess
Oct12,2016 Sprenkle-CSCI209 54
11/8/16
28
Project1:Tes&ngPrac&ce• 1st:EmailmeandyourteammatewiththenameofyourteamØ Iwillcreatearepositorythatthepaircanworkontogether
Oct12,2016 Sprenkle-CSCI209 55
LookingAhead• MoreTes&ng!• Extracreditassignment
Oct12,2016 Sprenkle-CSCI209 56