semester 1, 2003 week 7 cse9020 / 1 software testing and quality assurance with thanks to shonali...

Semester 1, 2003 Week 7

CSE9020 / 1

Software Testing and Quality Assurance

With thanks to Shonali Krishnaswamyand Sylvia Tucker

Semester 1, 2003 Week 7

CSE9020 / 2

Reading and ReferencesReading and References

“Software Testing - A Craftsman’s Approach”, (1995) by Paul C. Jorgensen

“The Complete Guide to Software Testing”, (1988), by Bill Hetzel

Semester 1, 2003 Week 7

CSE9020 / 3

Presentation OutlinePresentation Outline

Introduction

Terminology

Levels of Testing

Defect Fixing Strategies

Test Process

Bottom line

Semester 1, 2003 Week 7

CSE9020 / 4

Testing and QualityTesting and Quality

Hetzel - “Testing is the process of establishing confidence that a program or system does what it is supposed to do”

Quality <=> “Meets requirements”

Quality is not tangible

Semester 1, 2003 Week 7

CSE9020 / 5

Testing and Quality Testing and Quality

The purpose of testing

is to make quality visible

Testing is the measurement

of software quality

Semester 1, 2003 Week 7

CSE9020 / 6

TerminologyTerminology

Test Plan: A document describing the testing approach

– Description of the system to be tested

– Objectives of testing

Semester 1, 2003 Week 7

CSE9020 / 7

Terminology - Test Plan Terminology - Test Plan

– Project team

– Test environment

– Resource requirements

– Test Schedule

Semester 1, 2003 Week 7

CSE9020 / 8

TerminologyTerminology

Test Procedure: A document defining the steps required to carry out the test plan (or part of it) - that is, execute a set of test cases

– Test case name, objective

– Description of input required

– Description of expected output

Semester 1, 2003 Week 7

CSE9020 / 9

Terminology - Test ProcedureTerminology - Test Procedure

Some aspects to watch :

– Commands or Macros to run the test case

– Location and format of the actual output

– Assumptions, Conditions

Semester 1, 2003 Week 7

CSE9020 / 10

TerminologyTerminology

Test Case: A specific set of test data and the corresponding expected results for a particular test objective.

– Invalid and Valid test cases

Test Script: The code written to implement the test case

Semester 1, 2003 Week 7

CSE9020 / 11

Levels of TestingLevels of Testing

Unit Testing:

Testing of individual components of a system (sometimes called Basic Functionality Testing)

Semester 1, 2003 Week 7

CSE9020 / 12

Levels of TestingLevels of Testing

System Testing:

Testing in a wider scope - a group of components and the sequence of operations

Semester 1, 2003 Week 7

CSE9020 / 13

Levels of TestingLevels of Testing

Acceptance Testing:

Testing to determine if a system is ready for implementation

(or, - would the ‘user’ buy this ?

- would I buy it ? )

Semester 1, 2003 Week 7

CSE9020 / 14

Automated vs. Manual TestingAutomated vs. Manual Testing

Manual

– Every test case is executed manually

– Comparison of actual and expected is done manually

Semester 1, 2003 Week 7

CSE9020 / 15

Automated vs. Manual TestingAutomated vs. Manual Testing

Manual– Suitable for interactive tests

– Tedious and time consuming

– Simpler

Semester 1, 2003 Week 7

CSE9020 / 16

Automated vs. Manual TestingAutomated vs. Manual Testing

Automated– A test harness runs the test scripts

– Analysis of the results is automated and the results are logged

– Testing effort is reduced and it is more efficient in the longer term

Note: The actual test scripts are not generated automatically

Semester 1, 2003 Week 7

CSE9020 / 17

Boundary Value AnalysisBoundary Value Analysis

Guidelines:– Break up the system in to components that

have to be tested– Determine the boundary values for the

components– Design test cases based on the boundary

values

Limitation: Works well with independent variables (or components) with well defined boundaries

Semester 1, 2003 Week 7

CSE9020 / 18

Equivalence ClassesEquivalence Classes

Motivations: Completeness with minimal or no redundancies

An equivalence class is a partition of a set

Guidelines:– Identify equivalence classes– Design test cases for each equivalence class

Example: A variable can take only values from 0 to 9 inclusive.

Semester 1, 2003 Week 7

CSE9020 / 19

Decision TablesDecision Tables

Condition/Action rules are presented in a tabular form

Test Cases: Conditions are inputs, Actions are outputs

1 2 3 4 5

Fixed rate account T T F F F

Variable rate accountF F T T F

Consumption < 100 KWH T F T F

Consumption >= 100 KWH F T F T

Minimum monthly charge X

Schedule A billing X X

Schedule B billing X

Other treatment X

Semester 1, 2003 Week 7

CSE9020 / 20

Defect Fixing StrategiesDefect Fixing Strategies

Document Defects– full and complete information so the defect

can be reproduced– Pass it back to the developer

Fix – Document the fix– Pass it back to the tester

Re-test, regression tests

Semester 1, 2003 Week 7

CSE9020 / 21

Test ProcessTest Process

Develop a test plan (and agree on it)

Specify the test procedure

Perform boundary value analysis /

equivalence classes etc.

Semester 1, 2003 Week 7

CSE9020 / 22

Test ProcessTest Process

Design and implement test cases

Run the tests

Analyse and document results

Fix Defects then Run the tests etc etc -and agree that the fixes have worked

Semester 1, 2003 Week 7

CSE9020 / 23

Testing Web SitesTesting Web Sites

Are essentially client/server applications Consider:

– Interaction between html pages– internet connections– firewalls– applications that run in web pages (client)

• applets• javascript• .net approach

Semester 1, 2003 Week 7

CSE9020 / 24

Testing Web SitesTesting Web Sites

– applications that run on the server• cgi scripts• database interfaces• logging applications• dynamic page generators

– variety of servers, browsers (versions)– expected loads on the server, expected

performance on client, and server• investigate web testing tools

http://www.softwareqatest.com/qatweb1.html

Semester 1, 2003 Week 7

CSE9020 / 25

Testing Web SitesTesting Web Sites

Who is the target audience?– Browsers, connection speed

• intranet (fast) or internet (may be sloooooow!)

– Usability guidelines• Pages should be 3-5 screens max., otherwise

provide internal links within the page• Use a consistent layout, navigation• Make pages browser-independent, or generate

for a specific browser• Include page owner, revision date, contact link• NO dead-end pages!

Semester 1, 2003 Week 7

CSE9020 / 26

Testing Web SitesTesting Web Sites

On-going – Performance Testing– Link Testing– Security Testing– HTML validation (browsers change!)

Semester 1, 2003 Week 7

CSE9020 / 27

In Summary...In Summary...

Why? Who? When? What? How?

Semester 1, 2003 Week 7

CSE9020 / 28

Bottom LineBottom Line

Testing is expensive

Testing is necessary

You are never going to test everything– use Risk Analysis to determine where

testing should be focused

Semester 1, 2003 Week 7

CSE9020 / 29

Bottom LineBottom Line

Your IE Project is not complete until your client has completed the Acceptance Testing to their satisfaction, and signed-off the project!

Semester 1, 2003 Week 7

CSE9020 / 30

Bottom LineBottom Line

Most Essential

Make sure that– the system– and the group members

maintain versioning control

(link Software Testing and Quality Assurance with with Risks and Ethics)

Semester 1, 2003 Week 7

CSE9020 / 31

More MaintenanceMore Maintenance

Pilot Entry:

Left inside main tyre almost needs replacement

Mechanic Entry:

Almost replaced left inside main tyre

Pilot Entry:

Test flight OK, except autoland very rough

Mechanic entry

Autoland not installed on this aircraft