testing in agile with coaching agile journeys and xbosoft
TRANSCRIPT
Tes$nginAgile(agile)
Philip Lew
©XBOSo4,Inc.2017AllRightsReserved
1 @xboso4@philiplew
AgileTes$ng–WhattoDo?
• Today,we’llcoverwhatyoudoin‘AgileTes$ng’,whichisamaQeroffocusand$ming
• Asopposedtowaterfallwhereso4warewas‘readyfortes$ng’,weare‘readyfortes+ng’(doingtes$ngac$vi$es)allthe$me,butwhatexactlyarewedoingwhen?
©XBOSo4,Inc.2017AllRightsReserved 2@xboso4@philiplew
A4erWorkingWithManyofOurClientsinBothWaterfallandAgile
• I’vefoundthatfocusand$mingarethekeydifferences– Eitherintheirsuccess– Orintheirfailure
• Notrocketscience• Mostlynochangesintechnology,butchangesinfocusand$ming
©XBOSo4,Inc.2017AllRightsReserved 3
“Having the Right People In the Right Place Doing the Right Thing At the Right Time.”
@xboso4@philiplew
Meet Your Instructor
4©XBOSo4,Inc.2017AllRightsReserved
Phil Lew • Telecommunications consultant and network designer • Team Lead, Data warehousing product development • Software product manager, BI product • COO, large IT services company • CEO, XBOSoft, software QA and testing services
Relevant specialties/Research • Software quality process improvement • Software process measurement • Software quality in use / UX design
Where to find me • @philiplew • @xbosoft • www.xbosoft.com • [email protected]
@xboso4@philiplew
TodayisAllAboutYou
©XBOSo4,Inc.2017AllRightsReserved 5
• Everwonderhowdotes$ngprac$cesdifferwhetheryouronanagileversusawaterfallteam?
• Doyous$llhavestep-by-steptestscriptsorjusthavegeneraltestscenariosoruserstories.
• Doyouhaveatestplan?• Whatdoyoudoasand‘agile’tester?• Whyareyouhere?
@xboso4@philiplew
Se_ngExpecta$ons…
• Interac$ve-askques$onsviaGoto-WebinarPanel
• Iwon’treadtheslides…• Slidesforyouasatake-away
– Sessionisrecordedandavailablea4erwards
6©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew
So4wareisNew• So4waredevelopmentisnew–notlongenoughtoknowhowtodoit.
• Wearenotbuildingdoorsormakingtoilets.• Whenwedonotknowwhatwearedoing,thebeststrategyistodo‘liQlebyliQle’approachandgetfeedbackassoonaspossible.
• Weusethesametechniqueinoureverydaylives,andAgileisanapplica$onofitinso4ware.
©XBOSo4,Inc.2017AllRightsReserved 7@xboso4@philiplew
WhatGotUsHere
• Smallerteams• Fasteritera$ons• Listeningtotheuser
– Con$nuousbeta– Datacollec$on&analy$cs
©XBOSo4,Inc.2017AllRightsReserved 8
• Communica$on• Workingsmarter• Analysis,adap$onandimprovement
1. Changes in technology (mobile, cloud) 2. Changes in business models 3. Many failures…
@xboso4@philiplew
AgileProblems
ResistancetoChange
DIstrust
RequirementsChurn
FrozenRequirements
AgileDoing–NotBeing
ProcessInconsistency LackTest
Automa$on
Retrospec$vesNotValuable
AgileFall
LackCustomer-User
Understanding
AgileFailures–Why?
©XBOSo4,Inc.2017AllRightsReserved 9
AgileSuccess
Execu$veSupport
UserInvolvement
ScopedValue
SkilledPlayers
AgileProcess
Proficiency
ClearBusinessObjec$ves HighUseof
Automa$on
ConsistentDev,Integra$onandReleasePrac$ces
Collabora$veBehaviors
FullFocus
AgileSuccess–Why?
©XBOSo4,Inc.2017AllRightsReserved 10
©XBOSo4,Inc.2017AllRightsReserved 11@xboso4@philiplew
Be-Do-HaveParadigm
Be
Do
Have@xboso4@philiplew
©XBOSo4,Inc.2017AllRightsReserved 12
AgileBe-Do-Have
Be
Do
Have
©XBOSo4,Inc.2017AllRightsReserved
Process • Iterative (sprints) • Daily standups • Face to face communication • Post mortem – end of sprint • Delivery meeting – end of sprint • Planning meeting – before sprint • Self organizing
People • Communicative • Collaborative/Cooperative • Flexible and willing • Knowledgeable-multi • Initiative/responsible • Responsive
Results • Speed • Quality
@xboso4@philiplew 13
AgileQuality
• Nopolice• Sharedteamresponsibility• Tes$ngisdonebyeveryone?
– Tes$ngisdonebyqualifiedteammembers– Developerscouldtakeonsometes$ngtasks
• Developmentisdonebyeveryone?– Developmentisdonebyqualifiedteammembers– Testerscouldtakeonsomedevelopmenttasks
©XBOSo4,Inc.2017AllRightsReserved 14@xboso4@philiplew
Team members do what they can do and what is needed
©XBOSo4,Inc.2017AllRightsReserved 15
OneoftheMainAgileFlavors:Scrum
1 2
3
@xboso4@philiplew
AGILEPLANNINGThingstothinkaboutanddoduringthiscri$calphase…
©XBOSo4,Inc.2017AllRightsReserved 16
1
@xboso4@philiplew
TesterPairingandSwapping• WithTestSpecialists
– Security– Performance– Usability– Automa$on
• WithDomainExperts– Businessperspec$ve– Auto-correct/complete/verifystories– Benefitsinbothdirec$ons
©XBOSo4,Inc.2017AllRightsReserved 17
1
@xboso4@philiplew
Tes$ngisCri$calPartofthePlan
• Tes$ngisnotana4erthought– Clarifystorieswithcustomerandenduser
• Thinkfromtheirpointofview– Iden$fytes$ngac$vi$estosupportdevelopment
• Whiteboxtes$ng• Integra$ontes$ng
– Backloggrooming• Improveandclarifystories
18©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
WorkloadManagement
• “Sustainable”-thisAgileprinciplehelpstopreventtestteamburnout
• Sometes$ngtaskscanbeoutsidethesprints(suchastestautoma$oncodedevelopment),– sometesterscanworkoutsidethesprintsontheselong-termprojects
• Properes$ma$onofworkavoidspressureontestersduringtheendoftheitera$on
• Properes$ma$onalsoincludeses$ma$onoftes$ng
1
©XBOSo4,Inc.2017AllRightsReserved 19@xboso4@philiplew
SizetheJobProperlyandPriori$ze• SomeAgileprojectsritualizesprint$medeadlines,whilenotfollowingotherrulesrelatedtosustainableworkload
• Whentheteamcon$nuallyover-commits,thetestteamgetssqueezedonthelastweekendofeverysprint
• Resolvethischallengethroughrisk-basedtes$ng• Reducethescopeoftes$ngbasedonrisk
– Movelow-riskteststonextsprintor– Don’tdoatall
1
©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew 20
PlanningforTechnicalDebt
• Agileteamsshouldhavea“fixbugsfirst” approachtopreventalargebacklogofbugs– Reduce‘$metofix’beyondthesprintlength
• Periodicstabiliza$onor‘fixdefects’sprintsshouldbeplanned
• Withpropermetrics,tes$ngcanhelpmeasureandmanagetechnicaldebt
• Plan$mefordefectverifica$on
1
©XBOSo4,Inc.2017AllRightsReserved 21@xboso4@philiplew
UnderstandingTestBasis• Testbasisisconstantlychangingeachitera$onandsome$meswithin– Stories– Acceptancetests– Environment– Productusage
• Youmayhaveadifferentfocusindifferentitera$ons
22©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
SprintBacklog–SprintPlanning• Eachbacklogentrybrokendownintotasks
• Tasksincludedevelopment,tes$ng,environmentconfigura$ons,dataprepara$on,anddocumenta$on
• Allplannedtogether
23©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
UserStories
• Ideallyreviewedbybusinessandtechnicalstakeholdersearlyinthesprint
• Testersshouldbeinvolvedinthesereviews• Highqualityuserstorieswithclearexpectedresultsleadtoearlydefectremovalanddefectpreven$on
• Testersshouldinspectforcompletenessandclarity– DoIunderstandwhatI’dbetes$ngfor?
1
©XBOSo4,Inc.2017AllRightsReserved 24@xboso4@philiplew
StoryGrooming• Clarifydescrip$onvialanguageandexamples
• Predictandavoidalternateinterpreta$ons• Elicitacceptancetestsfromproductownerorcustomer/enduser
• Predictalternateorerroneousflows• Relatenewstoriestoalreadyunderstoodconceptsandfunc$ons
25©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
Infrastructure
• Developingandmaintainingtakes$me,some$mesoverlookedandnotsmall– Tools– Frameworks– Data– Environments
26©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
TestDataandEnvironments
• Automa$ngtestdatachoresiscri$cal– Generateandcapture– Maintainandorganize– Validate– Changeproceduresandaccess– Regulatorycompliance
27©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
TestWhatonWhatEnvironment• Developmentenvironment
– Unit/component– Sta$ccodeanalysis
• Integra$onenvironment– Incrementalintegra$on– Regression
28©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
• QAenvironment– Smoke– Func$onal– Usability– Regression– UA
• Stagingenvironment– Performance,stressandload– EndtoEnd– System– Security
Func$onalTes$ng
• Canshi4focusdependingontheitera$onandfocusoftheitera$on– Stories
• Crossfunc$onal• Storyinterac$ons• Experiencebasedandexploratory
29©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
NonFunc$onalTes$ng• O4enforgoQenun$ltheend
– Usability– Security– Performance– Sta$c-Whitebox– Inspec$ons
30©XBOSo4,Inc.2017AllRightsReserved
1
@xboso4@philiplew
ReviewTests
• Inaddi$ontorequirements,businessandtechnicalstakeholderscanreviewtestcondi$ons(acceptancecriteria)– Reducesthenumberoffalseposi$vesandfalsenega$vesassociatedwithtes$ng
– Improvestestresults– Increasesconfidenceinquality– Gaininsight/reviewwhat’simportantandwhat’snot
1
©XBOSo4,Inc.2017AllRightsReserved 31@xboso4@philiplew
PlanningforChange
• Agiletes$ngrequireschangeinmindsetandchangeinfocusintechnicalskills
• Testersarebestequippedwhentheyknowhowtoprogramoratleastunderstandthelanguagebeingused
• Duetoshortitera$onsandcompressedregression,Agiletes$ngrequirestestautoma$on
• Testersarebestequippediftheycaneffec$velyimplementtestautoma$on
1
©XBOSo4,Inc.2017AllRightsReserved 32@xboso4@philiplew
DURINGTHESPRINT
©XBOSo4,Inc.2017AllRightsReserved 33
2
@xboso4@philiplew
Gowhereyou’rewantedANDneeded…
DuringtheSprint–WhatDoYouDo?
• Pairing• TestDrivenDevelopment• AutomatedTes$ng• ExploratoryTes$ng• Con$nuousIntegra$on
34©XBOSo4,Inc.2017AllRightsReserved
2
@xboso4@philiplew
TesterPairing–HeatoftheSprint• Withtestdomainexperts
– Security– Performance– Usability– Automa$on
• Withothertesters– “ExtremeTes$ng”--XT(XP)
• Withprogrammers– Whitebox– Programminglogicversustes$nglogic
©XBOSo4,Inc.2017AllRightsReserved 35
2
@xboso4@philiplew
TesterPairing–HeatoftheSprint• Withdevelopers
– Whitebox– Programminglogicversustes$nglogic– Helpdeveloperslearnhowtowritegoodunittestsforcode
©XBOSo4,Inc.2017AllRightsReserved 36
2
@xboso4@philiplew
TestDrivenDevelopment• Definethetest
– Thisiswheretesterscanhelpindesignandcri$queofdatadriventestsandunittests
– AcceptanceTestDrivenDevelopment– BehaviorDrivenDevelopment
• Design/Implementthetest• Developcodetopassthetest
37©XBOSo4,Inc.2017AllRightsReserved
2
@xboso4@philiplew
AutomatedUnitTes$ng
• Automatedunittestslimitregressionrisk• Canberunaso4enaseveryfewhours,oronce/day
• Combineautomatedunittestswithsta$ccodeanalysisandcodecoveragetomaximizeeffec$venessandcompleteness– Don’tusuallyfinddefects
2
©XBOSo4,Inc.2017AllRightsReserved 38@xboso4@philiplew
AutomatedFunc$onalTes$ng
• Automatedfunc$onaltes$ngcanbeintegratedwithcon$nuousintegra$on,automatedunittes$ng,codecoverage,andsta$canalysisframeworks
• Furtherminimizesregressionriskandincreasestes$ng‘velocity’.
• Selenium,amongstmanyotheropensourcetoolscandothis.
2
©XBOSo4,Inc.2017AllRightsReserved 39@xboso4@philiplew
Sta$cAnalysis
• Sta$ccodeanalysiscaniden$fymanyproblemspriortotes$ng– Complexity– Deadcode– Reusability
• Somesta$canalysiscaniden$fysecurityandmaintainabilityproblemsthattes$ngcannotiden$fyeasily
2
©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew 40
ExploratoryTes$ng• Notfreeformdowhateveryouwant• Structuredexperiment• Time-boxed• Purposefulandfocused
– Charter
©XBOSo4,Inc.2017AllRightsReserved 41
Design
Execu$onLearning
2
@xboso4@philiplew
ExploratoryTes$ngCharters• Mission,whatyousetouttodo• Whatyouwanttoaccomplish,learn/find• Ensuresharedunderstandingofscope• Examples:
– Failures,errors,andrecovery– Endtoendsystem– Onefunc$on/workflow– Emergingbehaviors(analy$cs)– Non-func$onal
• Security–aQacks• Performance
©XBOSo4,Inc.2017AllRightsReserved 42
2
@xboso4@philiplew
ExploratoryTes$ngTime-Boxes• Varieddependingonscopeandmission
– Typical–2hourstohalfday– Short–1hourorless– Long–1dayormore
©XBOSo4,Inc.2017AllRightsReserved 43
2
@xboso4@philiplew
ExploratoryTes$ng–MaximizingOutput
• Duringthesession,takenoteson– Decisionsyoumade– Whatyoudid– Whatyoudiscovered
• A4erthesession– Stopatthedesignated$me-box– Reviewfindings
• Whatworkedandwhatdidn’t• Whatdidwelearnandshouldwedoanothersame/diff
©XBOSo4,Inc.2017AllRightsReserved 44
2
@xboso4@philiplew
Con$nuousIntegra$on
• Detectintegra$onproblemsimmediatelya4ercodecheck-in
• Brokenbuildscanbereducedoreliminated• Combinecon$nuousintegra$onwithautomatedunittes$ngandautomatedfunc$onaltes$ngasthefounda$onofyour‘smoke’regression.
2
©XBOSo4,Inc.2017AllRightsReserved 45@xboso4@philiplew
THERETROSPECTIVE
©XBOSo4,Inc.2017AllRightsReserved 46
3
@xboso4@philiplew
Tes$ngImprovements
• Wherewas$mewasted?• Whatwasnottestable?• Whattasksfellofftheplateandwhy?• Whattasksweretotallyoverlooked?• Isautoma$ondoingwhatwewant?
3
©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew 47
AnalyzeDefects
• Whatnewdefectswereintroduced– Whatarethesourcesofdefects– Wherewerethedefectsintroduced
• Commoncauses• Howtoavoid?
3
©XBOSo4,Inc.2017AllRightsReserved 48@xboso4@philiplew
Collabora$on
• Withclient/user-Didwedeliverwhatwasexpected?
• Withdevelopers• Withbusinessanalysts• Withotherteams• Whatinforma$onwasmissed/notcommunicatedandwhy?
3
©XBOSo4,Inc.2017AllRightsReserved 49@xboso4@philiplew
AgileTrust
• EveryonetalksabouthowAgilerequires‘trust’.
• Trustisespeciallyneededinretrospec$veswhenyoutalkaboutyourdirtylaundry.
• Butwhatistrust?• Whatdoestrustmeantoyou?• Howdoyoucreateanenvironmentfortrust?
@xboso4@philiplew©XBOSo4,Inc.2017AllRightsReserved 50
3
@xboso4@philiplew©XBOSo4,Inc.2017AllRightsReserved 51
1 2
3
TheEnd4
Successfully‘GoingAgile’• Agileisnotatechniqueortechnology• Youcanbeagilewithoutabook,withoutacoach,justlikeyoucanloseweightifyouwantto,withoutWeightWatchers
• Withadiet,youwantalifestylechange– Notjusta3weekeffort,butforlongtermhealth– Some$mesyouhavesetbacks
• WithAgile,youwantaworkstylechange
©XBOSo4,Inc.2017AllRightsReserved 52@xboso4@philiplew
Summary–Shi4YourThinking• Agilepresentsasignificantshi4in$mingofwhenandwhatyoudoasatester
• Testersshouldembracethesechanges(opportuni$es)– Learnnewskills– Takeonadifferentmindsetincollabora$on– “Everyoneisresponsibleforquality”
©XBOSo4,Inc.2017AllRightsReserved 53@xboso4@philiplew
Summary-AsaTester1. Agiledoesn’tmean‘notestplan’asyou’ve
seen,butit’sadifferenttestplan.2. Beflexibleintheheatofthesprint,followthe
planand“Gowhereyouarewantedandneeded.”
3. Agilegivesyoutheopportunityforates$ngretrospec$vethatyoudidnothavebefore.
– Useittoimprove.
©XBOSo4,Inc.2017AllRightsReserved 54@xboso4@philiplew
Thanks! For those knowledge seekers, download:
Agile Metrics Agile Test Plan Agile 7 Habits
https://xbosoft.com/resources/white-papers/
Philip Lew @philiplew [email protected]
©XBOSo4,Inc.2017AllRightsReserved 55@xboso4@philiplew