take the pain (and weekends) out of test case management

© 2011 Seapine Software, Inc. All rights reserved.

Take the Pain (and Weekends)

Out of Test Case Management

Peter Varhol

Solutions Evangelist

Seapine Software

© 2011 Seapine Software, Inc. All rights reserved.

Agenda

Why we need test case management

How you know you’ve outgrown the spreadsheet

Requirements, testing, and defects

Practical steps for test case management

Summary and questions

© 2011 Seapine Software, Inc. All rights reserved.

Why We Need Test Case

Management

• Administration

• Keeping track of test cases and associated data is a

BIG job

© 2011 Seapine Software, Inc. All rights reserved.

Why We Need Test Case

Management

• Tracking

• How do you know when testing is complete?

• And how much longer it has to go?

• Test case data and results tells us where we are

© 2011 Seapine Software, Inc. All rights reserved.

Why We Need Test Case

Management

• Reporting

• Other stakeholders need to know about schedules,

quality, and trends

© 2011 Seapine Software, Inc. All rights reserved.

Why We Need Test Case

Management

• Analysis

• Are we there yet?

• How long will it take to get there?

© 2011 Seapine Software, Inc. All rights reserved.

How You Know You’ve Outgrown

Your Spreadsheet

• A common way to manage test cases is with a

spreadsheet

• A general-purpose tool for a specific job

• Still a lot of manual effort

• How to know when it’s time to

change

© 2011 Seapine Software, Inc. All rights reserved.

How You Know You’ve Outgrown

Your Spreadsheet

• You divide your attention between Sunday football

games and updating your test case spreadsheet

• Managing the spreadsheet is a full-time job

• So is testing

© 2011 Seapine Software, Inc. All rights reserved.

How You Know You’ve Outgrown

Your Spreadsheet

• You know precisely which tests were executed today

and what the results were

• But don’t have a clue as to how many of the actual

application requirements you’ve verified

© 2011 Seapine Software, Inc. All rights reserved.

How You Know You’ve Outgrown

Your Spreadsheet

• Management asks when testing will be finished, and it

takes you a day to (sort of) provide an answer

• You can’t perform ad hoc queries with a

spreadsheet

© 2011 Seapine Software, Inc. All rights reserved.

How You Know You’ve Outgrown

Your Spreadsheet

• I’ll tell you where to read the rest of them at the end of

the presentation

© 2011 Seapine Software, Inc. All rights reserved.

Requirements, Test Management,

and Defects

• It’s the circle of (testing) life

• Test cases flow from requirements

• Defects flow from (failed) test cases

• Requirements with defects haven’t been met

• Defects may become feature requests that turn into

requirements

• Measuring each in a vacuum gives an incomplete

picture

© 2011 Seapine Software, Inc. All rights reserved.

Requirements, Test Management,

and Defects

• Linking requirements to tests to defects enables:

• The ability to flow changes through the process

• Demonstrating why individual defects mean certain

requirements aren’t met

• Knowing what tests to re-execute when a defect is

fixed.

• Much more . . .

• But doing so manually is time-consuming and error-

prone

© 2011 Seapine Software, Inc. All rights reserved.

Essentials of Test Case

Management

• Culmination of requirements and test plan

• The test plan guides testing strategies that are

implemented in test cases

• Test cases test requirements

• They must be written to provide unambiguous

evidence on whether or not the requirement is met

© 2011 Seapine Software, Inc. All rights reserved.

Essentials of Test Case

Management

• Biggest challenges in test case management

• Keeping the data up to date

• Obtaining actionable information from the data

• Sharing information with stakeholders

© 2011 Seapine Software, Inc. All rights reserved.

Essentials of Test Case

Management

• What you need to know about your test cases

• Exact requirement they test

• Any changes to the test case

• When executed

• Pass/fail/blocked status

• Notes on execution issues

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test Management

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test

Management

• Don’t skimp on the test plan

• This is your roadmap to quality

• Take extra time to map your quality needs to your

testing strategy

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test

Management

• Automate data collection, analysis, and reporting

• Your time is too valuable

• You’ll discover information that will otherwise fall

through the cracks

• Automation reduces

human error

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test

Management

• Maintain links to requirements and defects

• An integrated solution keeps links current

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test

Management

• Define reports and track trends

• Reports enable information sharing

• Trend analysis improves prediction

© 2011 Seapine Software, Inc. All rights reserved.

Steps to Effective Test

Management

• Use your data

• Focus on the day-to-day details

• But don’t lose sight of the big picture

• Good test management means higher quality

© 2011 Seapine Software, Inc. All rights reserved.

Summary

• Managing test cases leads to high quality

• And take less time

• Spreadsheets work up until a point

• Then become part of the problem

• Test cases, requirements, and defects should be

managed together

• Automation brings efficiency and accuracy to test case

management

© 2011 Seapine Software, Inc. All rights reserved.

Thank you