module02-reporting.ppt

8/19/2019 Module02-Reporting.ppt

http://slidepdf.com/reader/full/module02-reportingppt 1/74

Reporting andanalyzing bugs

How to communicate

efficiently to theprogrammer



Some vocabularyAn error (or mistake) is something peoplemakeA fault is the result of an error:inaccurate re uirements te!t" erroneousdesign" buggy source code etc#

$aults of omission are difficult todetect

A fault won%t yield a failure without theconditions that trigger it#

&!ample: if the program yields ' ' * onthe +,th time you use it" you won%t see

the error before or after the +,th use#



Some vocabulary-he failure is the program%sactual incorrect or missing

behavior under the error.triggering conditions#A failure occurs when a fault e!ecutes

An incident is a characteristic ofa failure that helps you recognizethat the program has failed#



/ocabulary e!ampleHere%s a defective program

0123- A0123- 42R01- A 5 4

6hat is the error7 6hat is the fault76hat is the critical condition76hat will we see as the incident of thefailure7



4ug reporting-esters report bugs to programmers2roblem Report forms are commonly used

0f the report is not clear andunderstandable" the bug will not get fi!ed-o write a fully effective report youmust:

&!plain how to reproduce the problemAnalyze the error so that it can be describedwith a minimum number of steps6rite a report that is complete" easy tounderstand" and non.antagonistic



6hat kind of error toreport7

Report all following types ofproblems" but keep straight in yourmind" and on the bug report" whichtype you%re reporting#

8oding &rror: -he program doesn%t do whatthe programmer would e!pect it to do#9esign 0ssue: 0t%s doing what theprogrammer intended" but a reasonablecustomer would be confused or unhappy withit#

ore on the ne!t slide;



6hat kind of error toreport7

Re uirements 0ssue: -he program is welldesigned and well implemented" but itwon%t meet one of the customer%sre uirements#9ocumentation 5 8ode ismatch: Reportthis to the programmer (via a bug report)and to the writer (usually via a memo ora comment on the manuscript)#Specification 5 8ode ismatch: Sometimesthe spec is right< sometimes the code isright and the spec should be changed#



4ug ReportsA bug report is a tool that you use tosell the programmer on the idea ofspending her time and energy to fi! abug#4ug reports are your primary workproduct as a tester# -his is what peopleoutside of the testing group will mostnotice and most remember of your work#-he best tester isn%t the one who findsthe most bugs or who embarrasses themost programmers# -he best tester is theone who gets the most bugs fi!ed#



Selling 4ugs-ime is in short supply# 0f you wantto convince the programmer to spendhis time fi!ing your bug" you mayhave to sell him on it#Sales revolves around twofundamental ob=ectives:

otivate the buyer ( ake him 6A1- to fi!the bug#)>vercome ob=ections (?et past hise!cuses and reasons for not fi!ing thebug#)



otivating the 4ug$i!er

Some things that will often makeprogrammers want to fi! the bug:

0t looks really bad#0t looks like an interesting puzzle andpi ues the programmer%s curiosity#0t will affect lots of people#?etting to it is trivially easy#0t has embarrassed the company" or a buglike it embarrassed a competitor#

anagement (that is" someone withinfluence) has said that they really wantit fi!ed#



otivating the 4ug $i!6hen you run a test and find a failure"you%re looking at a symptom" not at theunderlying fault# @ou may or may nothave found the best e!ample of afailure that can be caused by theunderlying fault#-herefore you should do some follow.upwork to try to prove that a defect:

is more serious than it first appears.is more general than it first appears.



ook for follow.uperrors

6hen you find a coding error" you havethe program in a state that theprogrammer did not intend and probablydid not e!pect# -here might also bedata with supposedly impossible values#-he program is now in a vulnerablestate# Beep testing it and you might

find that the real impact of theunderlying fault is a much worsefailure" such as a system crash orcorrupted data#



-ypes of follow.uptesting

/ary the behaviour (change theconditions by changing what the

test case does)/ary the options and settings ofthe program (change theconditions by changing somethingabout the program under test)#/ary the software and hardwareenvironment#



+# /ary @our 4ehaviourBeep using the program after you see theproblem#4ring it to the failure case again (and again)#0f the program fails when you do C" then do Cmany times# 0s there a cumulative impact7-ry things that are related to the task thatfailed# $or e!ample" if the programune!pectedly but slightly scrolls the display

when you add two numbers" try tests that affectadding or that affect the numbers# 9o C" seethe scroll# 9o @ then do C" see the scroll# 9oD" then do C" see the scroll" etc# (0f thescrolling gets worse or better in one of these

tests" follow that up" you%re getting usefulinformation for debugging#)



+# /ary @our 4ehaviour-ry things that are related to the failure # 0fthe failure is une!pected scrolling after adding"try scrolling first" then adding# -ry repaintingthe screen" then adding# -ry resizing the display

of the numbers" then adding#-ry entering the numbers more uickly or changingthe speed of your activity in some other way#Also try other exploratory testing techniques #$or e!ample" you might try some interferencetests# Stop the program or pause it =ust as theprogram is failing# >r try it while the programis doing a background save# 9oes that cause dataloss corruption along with this failure7



'# /ary >ptions andSettings0n this case" the steps to achieve the failure are taken as given# -ry toreproduce the bug when the program is in a different state:

8hange the values of environment variables#8hange how the program uses memory#8hange anything that looks like it might be relevant that allows you to change as an option#

$or e!ample" suppose the program scrolls une!pectedly when you add two numbers#aybe you can change the size of the program window" or the precision (or

displayed number of digits) of the numbers



E# /ary the8onfigurationA bug might show a more serious failureif you run the program with lessmemory" a higher resolution printer"more device interrupts coming in etc#

0f there is anything involving timing" usea really slow (or very fast) computer"link" modem or printer" etc##

0f there is a video problem" try otherresolutions on the video card# -rydisplaying 38H more (less) comple!images#

1ote that this is not configurationtesting



E# /ary the8onfiguration

6e are interested in whether there is aparticular configuration that will show thebug more spectacularly#

Returning to the e!ample (une!pectedscrolling when you add two numbers)" trythings like:

9ifferent video resolutions9ifferent mouse settings if you have a wheelmouse that does semi.automated scrollingAn 1-S8 (television) signal output instead of atraditional (C?A or S/?A" etc#) monitor output#



4ug 1ew to -his/ersion70n many pro=ects" an old bug (from a

previous release of the program) might notbe taken very seriously if there weren%tlots of customer complaints#

0f you know it%s an old bug" check its history#-he bug will be taken more seriously if it isnew#@ou can argue that it should be treated as newif you can find a new variation or a newsymptom that didn%t e!ist in the previousrelease# 6hat you are showing is that the newversion%s code interacts with this error in newways# -hat%s a new problem#



otivating the 4ug$i!:Show it is ore

?eneral

ook for configuration dependence4ugs that don%t fail on theprogrammer%s machine are much lesscredible (to that programmer)# 0f theyare configuration dependent" the reportwill be much more credible if itidentifies the configuration dependencedirectly (and so the programmer startsout with the e!pectation that it won%tfail on all machines#)



8onfigurationdependence0n the ideal case (standard in many

companies)" test on ' machines9o your main testing on achine +# aybe thisis your powerhouse: latest processor" newestupdates to the operating system" fancy printer"video card" 3S4 devices" huge hard disk" lotsof RA " cable modem" etc#6hen you find a defect" use achine + as yourbug reporting machine and replicate on achine'# achine ' is totally different# 9ifferentprocessor" different keyboard and keyboarddriver" different video" barely enough RA "slow" small hard drive" dial.up connection witha link that makes turtles look fast#



8onfigurationdependenceSome people do their main testing on theturtle and use the power machine forreplication#6rite the steps" one by one" on the bug format achine +# As you write them" try them on

achine '# 0f you get the same failure"you%ve checked your bug report while youwrote it# (A valuable thing to do#)0f you don%t get the same failure" you havea configuration dependent bug# -ime to dotroubleshooting# 4ut at least you know thatyou have to#



3ncorner your cornercases6e test at e!treme values becausethese are the most likely places to

show a defect# But once we find the defect,we don’t have to stick with extreme value tests.

-ry mainstream values# -hese are easysettings that should pose no problemto the program# 9o you replicate thebug7 0f yes" write it up" referringprimarily to these mainstreamsettings# -his will be a very credible

bug report#



3ncorner your cornercases

0f the mainstream values don%t yieldfailure" but the e!tremes do" then do sometroubleshooting around the e!tremes#

0s the bug tied to a single setting (a truecorner case)7>r is there a small range of cases7 6hat is it70n your report" identify the narrow range thatyields failures# -he range might be so narrowthat the bug gets deferred# -hat might be theright decision# @our reports help the companychoose the right bugs to fi! before a release"and size the risks associated with theremaining ones#



>vercoming >b=ections:

Analysis of the

$ailure

-hings that will make programmersresist spending their time on the bug:

-he programmer can%t replicate the defect#Strange and comple! set of steps re uiredto induce the failure#1ot enough information to know what stepsare re uired" and it will take a lot ofwork to figure them out#-he programmer doesn%t understand thereport#3nrealistic (e#g# Fcorner caseG)0t%s a feature#



1on.Reproducible&rrors

Always report non.reproducibleerrors# 0f you report them well"programmers can often figure out theunderlying problem#@ou must describe the failure asprecisely as possible# 0f you can

identify a display or a message wellenough" the programmer can oftenidentify a specific point in the codethat the failure had to pass through#




6hen you realize that you can%treproduce the bug" write down everythingyou can remember# 9o it now" before youforget even more#As you write" ask yourself whetheryou%re sure that you did this step (orsaw this thing) e!actly as you are

describing it# 0f not" say so# 9rawthese distinctions right away# -helonger you wait" the more you%ll forget#




aybe the failure was a delayed reactionto something you did before starting thistest or series of tests# 4efore you

forget" note the tasks you did beforerunning this test#8heck the bug tracking system# Are theresimilar failures7 aybe you can find apattern#

$ind ways to affect timing of the programor devices" slow down" speed up#-alk to the programmer and5or read thecode#



1on.Reproducible bugsare reproducible

$ailures occur under certainconditions

0f you know the conditions" youcan recreate a failure0f you don%t know the criticalconditions" you cannot recreatethe failure6hat are some reasons you cannotreproduce a failure7



Reasons for non.

reproducible bugsSome problems have delayed effects:a memory leak might not show up untilafter you cut and paste ', times#

stack corruption might not turn into astack overflow until you do the sametask many times#a wild pointer might not have an easilyobservable effect until hours after itwas mis.set#

0f you suspect that you have time.delayedfailures" use tools such as videotape"capture programs" debuggers" debug.loggers" or memory meters to record along series of events over time#



Reasons for non.reproducible bugs-he bug depends on the value of a hidden input

variable# 0n any test" there are the variablesthat we think are relevant" and there iseverything else# 0f the data you think arerelevant don%t help you reproduce the bug" askwhat other variables were set" and what theirvalues were#Some conditions are hidden and others areinvisible# @ou cannot manipulate them and so

it is harder to recognize that they%represent# @ou might have to talk with theprogrammer about what state variables or flagsget set in the course of using a particularfeature#



Reasons for non.

reproducible bugsSome conditions are catalysts# -hey makefailures more likely to be seen# &!ample:low memory for a leak< slow machine for a

race# 4ut sometimes catalysts are moresubtle" such as use of one feature thathas a subtle interaction with another#Some bugs are predicated on corrupteddata# -hey don%t appear unless there areimpossible configuration settings in theconfig files or impossible values in thedatabase# 6hat could you have doneearlier today to corrupt this data7



Reasons for non.

reproducible bugs-he bug might appear only at a specifictime of day or day of the month or year#

ook for week.end" month.end" uarter.

end and year.end bugs" for e!ample#2rograms have various degrees of datacoupling# 6hen two modules use the samevariable" oddness can happen in thesecond module after the variable ischanged by the first# 0n some programs"interrupts share data with main routinesin ways that cause bugs that will onlyshow up after a specific interrupt#



Reasons for non.

reproducible bugs-he program may depend on one version ofa 9 # A different program loads adifferent version of the same 9 into

memory# 9epending on which program is runfirst" the bug appears or doesn%t#-he bug depends on you doing relatedtasks in a specific order#

-he bug is caused by an error in error.handling# @ou have to generate a previouserror message or bug to set up theprogram for this one#



Reasons for non.

reproducible bugs-he program might be showing aninitial state bug" such as:

-he bug appears only the first timeafter you install the program (soit happens once on every machine#)-he bug appears once after you load

the program but won%t appear againuntil you e!it and reload theprogram#



Reasons for non.

reproducible bugs@ou forgot some of the details of thetest you ran" including the criticalone(s) or you ran an automated test that

lets you see that a crash occurred butdoesn%t tell you what happened#-he bug depends on a crash or e!it of anassociated process#-he program might appear only under apeak load" and be hard to reproducebecause you can%t bring the heavilyloaded machine under debug control(perhaps it%s a customer%s system)#



Reasons for non.

reproducible bugs>n a multi.tasking or multi.usersystem" look for spikes in backgroundactivity#-he bug occurred because a devicethat it was attempting to write to orread from was busy or unavailable#

0t might be caused by keyboardkeybounce or by other hardware noise#



Reasons for non.

reproducible bugs-he apparent bug is a side.effect of ahardware failure#

A flaky power supply creates irreproduciblefailures#>ne prototype system had a high rate ofirreproducible firmware failures#&ventually" these were traced to a problemin the building%s air conditioning# -he

test lab wasn%t being cooled" no fan wasblowing on the unit under test" andprototype boards in the machine ran veryhot# -he machine was failing at hightemperatures#



0ncomprehensible bugreports

2rogrammers will not spend time ona bug if the bug report:

Has a strange and comple! set ofsteps re uired to induce the failure#9oes not have enough information toknow what steps are re uired" and it

will take a lot of work to figurethem out#0s hard to understand#



Reporting &rrorsAs soon as you run into a problem in thesoftware" fill out a 2roblem Report form# 0n awell written report" you:

&!plain how to reproduce the problem#Analyze the error so you can describe it in aminimum number of steps#0nclude all the steps#

ake the report easy to understand#Beep your tone neutral and non.antagonistic#Beep it simple: one bug per report#0f a sample test file is essential toreproducing a problem" reference it and attachthe test file#



-he 2roblem Report$orm

A typical form includes many of thefollowing fields

Problem report number : must be uni ueReported by : original reporter%s name#Some forms add an editor%s name#Date reported : date of initial reportProgram (or component) name : the visible

item under testRelease number : like Release '#,Version (build) identifier : like version8 or version ',,,, ,'a




Configuration(s) : h5w and s5wconfigurations under which the bug wasfound and replicated

Report type : e#g# coding error" designissue" documentation mismatch"suggestion" ueryCan reproduce : yes 5 no 5 sometimes 5unknown# (3nknown can arise" for e!ample"

when the configuration is at a customersite and not available to the lab)#Severity : assigned by tester# Somevariation on small 5 medium 5 large



-he 2roblem Report$ormPriority : assigned by programmer5pro=ect

managerProblem summary : +.line summary of theproblemKey ords : use these for searching later"anyone can add to key words at any timeProblem description and ho to reproduceit : step by step reproduction descriptionSuggested fi! : leave it blank unless youhave something useful to sayStatus : -ester fills this in# >pen 5closed 5 resolved



-he 2roblem Report$ormResolution : -he pro=ect manager owns this field#8ommon resolutions include:

Pending : the bug is still being worked on#"i!ed : the programmer says it%s fi!ed# 1ow you should checkit#Cannot reproduce : -he programmer can%t make the failurehappen# @ou must add details" reset the resolution to2ending" and notify the programmer#Deferred : 0t%s a bug" but we%ll fi! it later##s Designed : -he program works as it%s supposed to#$eed %nfo : -he programmer needs more info from you# She has

probably asked a uestion in the comments#Duplicate : -his is =ust a repeat of another bug report (CR&$it on this report#) 9uplicates should not close until theduplicated bug closes#&ithdra n : -he tester withdrew the report#




Resolution version : build identifierResolved by : programmer" pro=ectmanager" tester (if withdrawn bytester)" etc#Resolution tested by : originatingtester" or a tester if originatorwas a non.tester

Change history : date.stamped list ofall changes to the record" including

name and fields changed#



-he 2roblem Report

$ormComments : free.form" arbitrarily long field"typically accepts comments from anyone onthe pro=ect# -esters" programmers" techsupport (in some companies) and others havean ongoing discussion of reproductionconditions" etc#" until the bug is resolved#8losing comments (why a deferral is >B" orhow it was fi!ed for e!ample) go here#

-his field is especially valuable for recordingprogress and difficulties with difficult orpolitically charged bugs#6rite carefully# Iust like e.mail and usenetpostings" it%s easy to read a =oke or a remark asa flame# 1ever flame#



0mportant 2arts of the

Report:2roblem Summary-his one.line description of the

problem is the most important partof the report#

-he pro=ect manager will use it inwhen reviewing the list of bugs thathaven%t been fi!ed#&!ecutives will read it when reviewingthe list of bugs that won%t be fi!ed#-hey might only spend additional timeon bugs with FinterestingG summaries#



2roblem Summary-he ideal summary gives the reader enoughinformation to help her decide whether toask for more information# 0t should

include:A brief description that is specific enoughthat the reader can visualize the failure#A brief indication of the limits ordependencies of the bug (how narrow or broad

are the circumstances involved in this bug)7Some other indication of the severity (not arating but helping the reader envision theconse uences of the bug#)



8an @ou Reproduce -he

4ug7@ou may not see this on your form" but youshould always provide this information#

1ever say it%s reproducible unless you haverecreated the bug# (Always try to recreate thebug before writing the report#)0f you%ve tried and tried but you can%trecreate the bug" say F1oG# -hen e!plain whatsteps you tried in your attempt to recreate it#0f the bug appears sporadically and you don%t

yet know why" say FSometimesG and e!plain#You may not be able to try to replicate somebugs. &!ample: customer.reported bugs where thesetup is too hard to recreate#



How to Reproduce the

4ug#$irst" describe the problem# 9on%t rely on thesummary to do this . some reports will printthis field without the summary#1e!t" go through the steps that you use torecreate this bug#

Start from a known place (e#g# boot theprogram)-hen describe each step until you hit the bug#13 4&R -H& S-&2S# -ake it one step at a time#

0f anything interesting happens on the way"describe it# (@ou are giving people directionsto a bug# &specially in long reports" peopleneed landmarks#)




4ug9escribe the erroneous behaviour and" ifnecessary" e!plain what should havehappened# (6hy is this a bug7 4e clear#)

ist the environmental variables (e#g#configuration) that are not coveredelsewhere in the bug tracking form#0f you e!pect the reader to have anytrouble reproducing the bug (specialcircumstances are re uired)" be clearabout them#




4ug0t is essential to keep the descriptionfocused-he first part of the description should bethe shortest step.by.step statement of howto get to the problem#Add F1otesG after the description such as:

8omment that the bug won%t show up if you dostep C between step @ and step D#8omment e!plaining your reasoning for runningthis test#8omment e!plaining why you think this is aninteresting bug#8omment describing other variants of the bug#



Beeping the Report

Simple0f you see two failures" write tworeports#8ombining failures creates problems:

-he summary description is typically vague# @ousay words like FfailsG or Fdoesn%t workGinstead of describing the failure more vividly#-his weakens the impact of the summary#-he detailed report is typically lengthened andcontains comple! logic like: F9o this unless

that happens in which case don%t do this unlessthe first thing" and then the testcase of thesecond part and sometimes you see this but ifnot then thatG#



Beeping the Report

Simple&ven if the detailed report isrationally organized" it is longer(there are two failures and two setsof conditions" even if they arerelated) and therefore moreintimidating#@ou%ll often see one bug get fi!ed

but not the other#6hen you report related problems onseparate reports" it is a courtesyto cross.reference them#



Beeping it Simple:

&liminate 3nnecessarySteps (+)Sometimes it%s not immediately obvious what steps can bedropped from a long se uence of steps in a bug#

Look for critical steps -- Sometimes the first symptomsof a failure are subtle.

@ou have a list of the steps you took to show the error#@ou%re now trying to shorten the list# ook carefullyfor any hint of a failure as you take each step .. A fewthings to look for:

&rror messages (you got a message +, minutes ago# -heprogram didn%t fully recover from the error" and theproblem you see now is caused by that poor recovery#)

9elays or une!pectedly fast responses#9isplay oddities" such as a flash" a repainted screen"a cursor that =umps back and forth" multiple cursors"misaligned te!t" slightly distorted graphics" etc#



Beeping it Simple:

&liminate 3nnecessarySteps (')Sometimes the first indicator that the system isworking differently is that it sounds a littledifferent than normal#An in.use light or other indicator that a device is

in use when nothing is being sent to it (or a lightthat is off when it shouldn%t be)#9ebug messagesJturn on the debug monitor on yoursystem (if you have one) and see if5when a messageis sent to it#

0f you%ve found what looks like a critical

step" try to eliminate almost everything elsefrom the bug report# ?o directly from that stepto the last one (or few) that shows the bug# 0fthis doesn%t work" try taking out individualsteps or small groups of steps#



2ut /ariations After

the ain ReportSuppose that the failure looks differentunder slightly different circumstances#$or e!ample" suppose that:

-he timing changes if you do two additionalsub.tasks before hitting the finalreproduction step-he failure won%t show up or is much lessserious if you put something else at aspecific place on the screen-he printer prints different garbage(instead of the garbage you describe) if youmake the file a few bytes longer



0tKs not a bug" it%s a

featureAn argument over whether something is oris not a bug is really an argument aboutthe oracle you should use to evaluate

your test results# An oracle is the principle or mechanismby which you recognize a problem#L eets the specificationL or L eets there uirementsL is a heuristic oracle#0f you know it%s LwrongL but you donKthave a mismatch to a spec" what can youuse7



Some useful oracle

heuristicsConsistent ith 'istory : 2resent functionbehaviour is consistent with past behaviour#Consistent ith our %mage : $unction

behaviour is consistent with an image thatthe organization wants to pro=ect#Consistent ith Comparable Products :$unction behaviour is consistent with thatof similar functions in comparable products#Consistent ith Claims

: $unction behaviouris consistent with documented or advertisedbehaviour#



Some useful oracle

heuristicsConsistent ith ser s *!pectations :$unction behaviour is consistent withwhat we think users want#Consistent ithin Product : $unctionbehaviour is consistent withbehaviour of comparable functions orfunctional patterns within the

product#Consistent ith Purpose : $unctionbehaviour is consistent with apparentpurpose#



&diting 4ug ReportsSome groups have a second tester(usually a senior tester) reviewreported defects before they go tothe programmer# -he second tester:

checks that critical information ispresent and intelligiblechecks whether she can reproduce the bug

asks whether the report might besimplified" generalized or strengthened#0f there are problems" she takes thebug back to the original reporter#



Assignment +6e will do bug report editing andevaluating for Assignment +

-he purpose of this assignment is togive you e!perience editing bugswritten by other people#-his task will give you practice

thinking about what a professionalreport should be" as well asanalyzing bugs before reporting them



&diting 4ugs>pen >ffice is installed at 2rism@ou can also download and install in yourown machine$or the assignment" you will need to =ointhe MA pro=ect using your cs accountRead the instructions athttp:55 a#openoffice#org5issueNhandling5pro=ectNissues#html Read the bug entry guidelines athttp:55 a#openoffice#org5issueNhandling5bugNwritingNguidelines#html

http://qa.openoffice.org/issue_handling/project_issues.html


http://qa.openoffice.org/issue_handling/bug_writing_guidelines.html








6hat to do$ind * bug reports in 0ssue -racker aboutproblems with >pen >ffice that appear tohave not yet been independently verified#

$or each report" review and replicate thebug" and add comments as appropriate in theAdditional 8omments field to the report#-he hard copy report that you will submitshould list the bug numbers# -he markerwill read the comments you filed on the bugreport# 0n addition" for each bug" reporton what was done well" what was done poorlyand what was missing that should have beenthere in the bug report#



0ssues to address in

your report0s the summary short (about *,.O,characters) and descriptive78an you understand the report7

As you read the description" do youunderstand what the reporter did78an you envision what the program did inresponse7

9o you understand what the failure was70s it obvious where to start (whatstate to bring the program to) toreplicate the bug7




your report0s it obvious what files to use (if any)70s it obvious what you would type70s the replication se uence provided as a

numbered set of steps" which tell youe!actly what to do and" when useful" whatyou will see79oes the report include unnecessaryinformation" personal opinions or

anecdotes that seem out of place70s the tone of the report insulting7 Areany words in the report potentiallyinsulting7




your report9oes the report seem too long7 -ooshort7 9oes it seem to have a lot ofunnecessary steps7 (-his is your first

impressionJyou might be mistaken# Afterall" you haven%t replicated it yet# 4utdoes it >>B like there%s a lot ofe!cess in the report7)9oes the report seem overly general(F0nsert a file and you will seeG P whatfile7 6hat kind of file7 0s there ane!ample" like F0nsert a file likeblah#foo or blah'#feeG7)



Replicate the Report8an you replicate the bug79id you need additional information orsteps7

9id you get lost or wonder whether you haddone a step correctly7 6ould additionalfeedback (like" Fthe program will respondlike this###G) have helped79id you have to guess about what to done!t79id you have to change your configurationor environment in any way that wasn%tspecified in the report7



Replicate the Report9id some steps appear unnecessary7 6erethey unnecessary79id the description accurately describethe failure79id the summary accurate describe thefailure79oes the description include non.

factual information (such as thetester%s guesses about the underlyingfault) and if so" does this informationseem credible and useful or not7



$ollow.3p -estsAre there follow.up teststhat you would run on this

report if you had the time76hat would you hope to learnfrom these tests7

How important would thesetests be7



-esterKs evaluation9oes the description include non.factual information (such as thetester%s guesses about the underlying

fault) and if so" does this informationseem credible and useful or not79oes the description include statementsabout why this bug would be importantto the customer or to someone else7

-he report need not include suchinformation" but if it does" it should becredible" accurate" and useful#

module02-reporting.ppt

Documents