Surviving the Top Ten Challenges of Software Testing

Surviving the Top TenChallenges of Software

TestingRandall W. Rice, CQA, CSTE, FLMI

Rice Consulting Services, Inc.405-692-7331

www.riceconsulting.comSolving Your Testing Problems

RCS© 1997, Rice Consulting Services, Inc.

Introduction and Basis ofResearch

These findings are the results ofsurveying over 1,000 testers from allmajor industries and geographicregions.Research was conducted over a threeyear period.

RCS2

Major Finding

Most testing problems are not technical, but human innature.Attempts at solving testing problems in mostcompanies are aimed at technical approaches.

RCS3

How Testing Tests Testers

Many testers wear two faces:Tester One- The person that understands how

to test software, is good at findingdefects, and enjoys testing

- Tester one is generally happywhen only performing tests.

RCS4

The “Other” Tester

Tester Two- The person that must fight the

political battles that surroundtesting

- Tester two is generally unhappywhen dealing with political issues.

RCS5

Challenge Ten: Is Testing aNatural-born Instinct?

Central Issue - Lack ofeducation in testingThe large majority of testershave never had formal trainingin software testing.Many testers and theirmanagement are unaware oftraining sources for testing.

RCS6

Surviving the Top Ten Challenges of Software Testing

Solutions for Training

Obtain formal training in testingtechniquesCertification- CSTE (Certified Software

Test Engineer)Attending conferencesReading books and articles

RCS7

Challenge Nine: Whose Sideare Testers On?

Central Issue - Lack ofteamwork“Us versus them” mentalitySeen most often inorganizations withindependent test teams.

RCS8

Solutions to the TeamworkChallenge

The goal is to get to “Us andthem”Each person on the team canhave a role in testing- Developers: unit and

structural testing- Testers: independent testing- Users: business-oriented

testing- Management: to support

testing activitiesRCS

9

Challenge Eight: TestingWithout Tools

Central Issue - Lack ofunderstanding of the usefulness oftest toolsWithout upper managementsupport, it is difficult to acquire testtools.A process is needed for the tools.People should be trained in the useof test tools.People must accept and apply thetools.

RCS10

Solutions for Acquiring andUsing Test Tools

Identify a “champion” for obtainingtest tools.Base the case for test tools in costsvs. benefits.Have a basic testing process inplace.Train people in tool usage.Measure the benefits.

RCS11

Challenge Seven: ManagersJust Don’t Understand Testing

Central Issue - Management is notfully aware of what is required foreffective testing.Management is often focused onthe product, not the process.This leads to lack of managementsupport and involvement fortesting.

RCS12

Surviving the Top Ten Challenges of Software Testing

Solutions to EducatingManagement in Testing Issues

Cultural change is needed.Focus your message tomanagement on:- reducing the cost of rework- meeting the project schedule

The benefits of testing must relateto these two things to bepersuasive.

RCS13

Challenge Six: Would the RealCustomer Please Stand Up?

Central Issue #1 - A lack ofunderstanding of who thecustomer really is. We mustunderstand the differencebetween:- Customers- Users

Central Issue #2 - Lack ofcustomer involvement intesting.

RCS14

Solutions to Identifying andInvolving the Customer in Testing

Involve the customer and usersthroughout the project byperforming reviews andinspections.Include users on the system testteam.Perform user acceptancetesting.Understand the differencebetween the customer andusers.

RCS15

Challenge Five: Too MuchWork - Too Little Time

Central Issue - Arbitraryschedules and estimates donot allow enough time fortesting.Realism: There is neverenough time to test everypossible combinations oftest conditions.

RCS16

Solutions to the Time Crunch

Base schedules andestimates on measurabletesting activities.- Scripts to be executed- Cases to be tested- Requirements to be tested

Have contingency plans forschedule slippage.Integrate automated testingtools to the project.

RCS17

Challenge Four: OvercomingThrowing Stuff Over the Wall

Central Issue - Developersdepending on testers to findbugs.Seen most often whenindependent test teams areused in an organization.

RCS18

Surviving the Top Ten Challenges of Software Testing

Solutions to OvercomingThrowing Stuff Over the Wall

Developers must takeownership and responsibility forthe quality of their work.Quality control is most effectivewhen performed at the point ofcreation.Train developers to becomeexcellent testers.Get management support fordeveloper responsibility forquality.

RCS19

Challenge Three: Hitting aMoving Target

Central Issue - Software andsystems that undergo rapidand/or constant changeSoftware developmenttechniques such as RapidApplication Development canproduce a new version of thesoftware very quickly.You can’t perform a four-weektest every day manually.

RCS20

Solutions for Hitting a MovingTarget

The testing process mustaccommodate change.Focus on testablerequirements.Use automated testingtools.Manage the rate anddegree of change.

RCS21

Challenge Two: Fighting aLose-Lose Situation

Central Issue - Testers areoften the “bad guys” when theyfind defects, yet if the softwaregoes live with problems, testersare blamed for the defects.Since independent testers haveno control over the softwaredevelopment process, theycannot guarantee softwarequality.

RCS22

Solutions for Fighting a Lose-Lose Situation

The perception of testing mustchange.- Testers are paid to find

defects- Each defect found is one

more the customer or userwill not find

Testers are not to blame forbottlenecks. It is management’sresponsibility to have anefficient process.

RCS23

Challenge One: Having to Say“No”

Central Issue - Testers areoften the bearer of bad news.Problem: Management doesn’tlike to hear bad news.

RCS24

Surviving the Top Ten Challenges of Software Testing

Solutions for Having to Say“No”

Most responsibility is onmanagement to:- have a quality software

development process in place.- have contingency plan in place in

case of problems.- understand that testing is only an

evaluation activity.- accept the honest facts.

Keep the test results objective

RCS25

Summary - Applying theLessons Learned

Most testing problems seem tobe human in nature.“Quality is everyone’s job, but it’smanagement’s responsibility.”Dr. Deming.There are three key parts of thetesting picture: Process, Peopleand Tools.

RCS26