semester 2, 2003 week 7 cse9020 / 1 software testing and quality assurance with thanks to shonali...

Semester 2, 2003 Week 7

CSE9020 / 1

Software Testing and Quality Assurance

With thanks to Shonali Krishnaswamyand Sylvia Tucker


CSE9020 / 2

Reading and ReferencesReading and References

“Software Testing - A Craftsman’s Approach”, (1995) by Paul C. Jorgensen

“The Complete Guide to Software Testing”, (1988), by Bill Hetzel


CSE9020 / 3

Presentation OutlinePresentation Outline Introduction

Terminology

Levels of Testing

Types of Testing

Defect Fixing Strategies

Test Process

Bottom line


CSE9020 / 4

Testing and QualityTesting and Quality

Hetzel - “Testing is the process of establishing confidence that a program or system does what it is supposed to do”

Quality <=> “Meets requirements”

Hetzel proposes that Quality is not tangible


CSE9020 / 5

Testing and Quality Testing and Quality

The purpose of testing

is to make quality visible(? And hence tangible ?)

Testing is the measurement

of software quality (including inspection as a testing technique)


CSE9020 / 6

TerminologyTerminology

Test Plan: A document describing the testing approach

– Description of the system to be tested

– Objectives of testing


CSE9020 / 7

Terminology - Test Plan Contents Terminology - Test Plan Contents

– Project team

– Test environment

– Resource requirements

– Test Schedule


CSE9020 / 8


Test Procedure: A document defining the steps required to carry out some part of the test plan - that is, execute a set of test cases

– Test case name, objective

– Description of input required

– Description of expected output


CSE9020 / 9

Terminology - Test ProcedureTerminology - Test Procedure

Some aspects that need attention:

– Commands or Macros to run the test case

– Location and format of the actual output

– Assumptions, Conditions


CSE9020 / 10


Test Case: A specific set of test data and the corresponding expected results for a particular test objective.

– Invalid and Valid test cases

Test Script: The code written to implement the test case


CSE9020 / 11

Levels of TestingLevels of Testing

Unit Testing:

Testing of individual components of a system (sometimes called Basic Functionality Testing)


CSE9020 / 12


System Testing:

Testing in the wider scope - a integrated group of components and a matching sequence of operations


CSE9020 / 13


Acceptance Testing:

Testing to determine if a system is ready for implementation

(or, - will the ‘user’ commit to use this ? )


CSE9020 / 14

Types of Testing:Functional

Types of Testing:Functional

For each component, test to confirm that the functional requirements have been satisfied.

A necessary part of all testing


CSE9020 / 15

Black Box TestingBlack Box Testing

The components being examined as being closed – not available to be inspected.

Like when you buy a watch, or a DVD player. Then internals are ‘a black box’


CSE9020 / 16

Glass Box TestingGlass Box Testing

As opposed to black box testing For each component, the internals are

examined, and then a decision is made as to the best way of making up a detailed test specification


CSE9020 / 17

Boundary Value AnalysisBoundary Value Analysis

Guidelines:– Break up the system in to components that

have to be tested– Determine the boundary values for these

components– Design test cases based on the boundary

values

Limitation: Works well with independent variables (or components) with well defined boundaries


CSE9020 / 18

Equivalence ClassesEquivalence Classes

Motivations: Completeness with minimal or no redundancies

An equivalence class is a partition of a set

Guidelines:– Identify equivalence classes– Design test cases for each equivalence

class

Example: A variable can take only values from 0 to 9 inclusive.


CSE9020 / 19

Decision TablesDecision Tables

Condition/Action rules are presented in a tabular form

Test Cases: Conditions are inputs, Actions are outputs

1 2 3 4 5

Fixed rate account T T F F F

Variable rate accountF F T T F

Consumption < 100 KWH T F T F

Consumption >= 100 KWH F T F T

Minimum monthly charge X

Schedule A billing X X

Schedule B billing X

Other treatment X


CSE9020 / 20

Automated vs. Manual TestingAutomated vs. Manual Testing

Manual– Every test case is executed manually

– Comparison of actual and expected is done manually


CSE9020 / 21


Manual– Suitable for interactive tests

– Tedious and time consuming

– Simpler


CSE9020 / 22


Automated– A test harness runs the test scripts

– Analysis of the results is automated and the results are logged

– Testing effort is reduced and may be more efficient in the longer term

Note: The actual test scripts are not generated automatically


CSE9020 / 23

Defect Fixing StrategiesDefect Fixing Strategies

Document Defects– full and complete information so the defect can be

reproduced– Pass it back to the developer

Fix – Document the fix– Pass it back to the tester

Re-test, regression tests(this is where automated testing can be effective)


CSE9020 / 24

Test ProcessTest Process

Develop a test plan (and agree on it)

Specify the test procedure

Perform boundary value analysis /

equivalence classes etc.


CSE9020 / 25

Test ProcessTest Process

Design and implement test cases

Run the tests

Analyse and document results

Fix Defects, Run the tests, compare the results, and agree that the fixes have worked


CSE9020 / 26

Testing Web SitesTesting Web Sites

Are essentially client/server applications Consider:

– Interaction between .html pages– internet connections– firewalls– applications that run in web pages (client)

• applets• javascript• .net approach


CSE9020 / 27


– applications that run on the server• cgi scripts• database interfaces• logging applications• dynamic page generators

– variety of servers, browsers (versions)– expected loads on the server, expected

performance on client, and server• investigate web testing tools

http://www.softwareqatest.com/qatweb1.html


CSE9020 / 28


Who is the target audience?– Browsers, connection speed

• intranet (fast) or internet (may be sloooooow!)

– Usability guidelines• Pages should be 3-5 screens max., otherwise

provide internal links within the page• Use a consistent layout, navigation• Make pages browser-independent, or generate

for a specific browser• Include page owner, revision date, contact link• NO dead-end pages!


CSE9020 / 29


On-going – Performance Testing– Link Testing– Security Testing– HTML validation (browsers change!)


CSE9020 / 30

In Summary...In Summary...

Why? Who? When? What? How?


CSE9020 / 31

Bottom LineBottom Line

Testing is expensive

Testing is necessary

You are never going to test everything– use Risk Analysis to determine where

testing should be focused


CSE9020 / 32


Your Case Study Project is not complete until your client has completed the Acceptance Testing to their satisfaction, and then signed-off the project !


CSE9020 / 33


Most Essential

Make sure that– the system– and the group members

maintain versioning control

(link Software Testing and Quality Assurance with Risks and Ethics)


CSE9020 / 34

More MaintenanceMore Maintenance

Pilot Entry:

Left inside main tyre almost needs replacement

Mechanic Entry:

Almost replaced left inside main tyre

Pilot Entry:

Test flight OK, except autoland very rough

Mechanic entry

Autoland not installed on this aircraft

semester 2, 2003 week 7 cse9020 / 1 software testing and quality assurance with thanks to shonali...

Documents