semester 1, 2003 week 7 cse9020 / 1 software testing and quality assurance with thanks to shonali...
TRANSCRIPT
Semester 1, 2003 Week 7
CSE9020 / 1
Software Testing and Quality Assurance
With thanks to Shonali Krishnaswamyand Sylvia Tucker
Semester 1, 2003 Week 7
CSE9020 / 2
Reading and ReferencesReading and References
“Software Testing - A Craftsman’s Approach”, (1995) by Paul C. Jorgensen
“The Complete Guide to Software Testing”, (1988), by Bill Hetzel
Semester 1, 2003 Week 7
CSE9020 / 3
Presentation OutlinePresentation Outline
Introduction
Terminology
Levels of Testing
Defect Fixing Strategies
Test Process
Bottom line
Semester 1, 2003 Week 7
CSE9020 / 4
Testing and QualityTesting and Quality
Hetzel - “Testing is the process of establishing confidence that a program or system does what it is supposed to do”
Quality <=> “Meets requirements”
Quality is not tangible
Semester 1, 2003 Week 7
CSE9020 / 5
Testing and Quality Testing and Quality
The purpose of testing
is to make quality visible
Testing is the measurement
of software quality
Semester 1, 2003 Week 7
CSE9020 / 6
TerminologyTerminology
Test Plan: A document describing the testing approach
– Description of the system to be tested
– Objectives of testing
Semester 1, 2003 Week 7
CSE9020 / 7
Terminology - Test Plan Terminology - Test Plan
– Project team
– Test environment
– Resource requirements
– Test Schedule
Semester 1, 2003 Week 7
CSE9020 / 8
TerminologyTerminology
Test Procedure: A document defining the steps required to carry out the test plan (or part of it) - that is, execute a set of test cases
– Test case name, objective
– Description of input required
– Description of expected output
Semester 1, 2003 Week 7
CSE9020 / 9
Terminology - Test ProcedureTerminology - Test Procedure
Some aspects to watch :
– Commands or Macros to run the test case
– Location and format of the actual output
– Assumptions, Conditions
Semester 1, 2003 Week 7
CSE9020 / 10
TerminologyTerminology
Test Case: A specific set of test data and the corresponding expected results for a particular test objective.
– Invalid and Valid test cases
Test Script: The code written to implement the test case
Semester 1, 2003 Week 7
CSE9020 / 11
Levels of TestingLevels of Testing
Unit Testing:
Testing of individual components of a system (sometimes called Basic Functionality Testing)
Semester 1, 2003 Week 7
CSE9020 / 12
Levels of TestingLevels of Testing
System Testing:
Testing in a wider scope - a group of components and the sequence of operations
Semester 1, 2003 Week 7
CSE9020 / 13
Levels of TestingLevels of Testing
Acceptance Testing:
Testing to determine if a system is ready for implementation
(or, - would the ‘user’ buy this ?
- would I buy it ? )
Semester 1, 2003 Week 7
CSE9020 / 14
Automated vs. Manual TestingAutomated vs. Manual Testing
Manual
– Every test case is executed manually
– Comparison of actual and expected is done manually
Semester 1, 2003 Week 7
CSE9020 / 15
Automated vs. Manual TestingAutomated vs. Manual Testing
Manual– Suitable for interactive tests
– Tedious and time consuming
– Simpler
Semester 1, 2003 Week 7
CSE9020 / 16
Automated vs. Manual TestingAutomated vs. Manual Testing
Automated– A test harness runs the test scripts
– Analysis of the results is automated and the results are logged
– Testing effort is reduced and it is more efficient in the longer term
Note: The actual test scripts are not generated automatically
Semester 1, 2003 Week 7
CSE9020 / 17
Boundary Value AnalysisBoundary Value Analysis
Guidelines:– Break up the system in to components that
have to be tested– Determine the boundary values for the
components– Design test cases based on the boundary
values
Limitation: Works well with independent variables (or components) with well defined boundaries
Semester 1, 2003 Week 7
CSE9020 / 18
Equivalence ClassesEquivalence Classes
Motivations: Completeness with minimal or no redundancies
An equivalence class is a partition of a set
Guidelines:– Identify equivalence classes– Design test cases for each equivalence class
Example: A variable can take only values from 0 to 9 inclusive.
Semester 1, 2003 Week 7
CSE9020 / 19
Decision TablesDecision Tables
Condition/Action rules are presented in a tabular form
Test Cases: Conditions are inputs, Actions are outputs
1 2 3 4 5
Fixed rate account T T F F F
Variable rate accountF F T T F
Consumption < 100 KWH T F T F
Consumption >= 100 KWH F T F T
Minimum monthly charge X
Schedule A billing X X
Schedule B billing X
Other treatment X
Semester 1, 2003 Week 7
CSE9020 / 20
Defect Fixing StrategiesDefect Fixing Strategies
Document Defects– full and complete information so the defect
can be reproduced– Pass it back to the developer
Fix – Document the fix– Pass it back to the tester
Re-test, regression tests
Semester 1, 2003 Week 7
CSE9020 / 21
Test ProcessTest Process
Develop a test plan (and agree on it)
Specify the test procedure
Perform boundary value analysis /
equivalence classes etc.
Semester 1, 2003 Week 7
CSE9020 / 22
Test ProcessTest Process
Design and implement test cases
Run the tests
Analyse and document results
Fix Defects then Run the tests etc etc -and agree that the fixes have worked
Semester 1, 2003 Week 7
CSE9020 / 23
Testing Web SitesTesting Web Sites
Are essentially client/server applications Consider:
– Interaction between html pages– internet connections– firewalls– applications that run in web pages (client)
• applets• javascript• .net approach
Semester 1, 2003 Week 7
CSE9020 / 24
Testing Web SitesTesting Web Sites
– applications that run on the server• cgi scripts• database interfaces• logging applications• dynamic page generators
– variety of servers, browsers (versions)– expected loads on the server, expected
performance on client, and server• investigate web testing tools
http://www.softwareqatest.com/qatweb1.html
Semester 1, 2003 Week 7
CSE9020 / 25
Testing Web SitesTesting Web Sites
Who is the target audience?– Browsers, connection speed
• intranet (fast) or internet (may be sloooooow!)
– Usability guidelines• Pages should be 3-5 screens max., otherwise
provide internal links within the page• Use a consistent layout, navigation• Make pages browser-independent, or generate
for a specific browser• Include page owner, revision date, contact link• NO dead-end pages!
Semester 1, 2003 Week 7
CSE9020 / 26
Testing Web SitesTesting Web Sites
On-going – Performance Testing– Link Testing– Security Testing– HTML validation (browsers change!)
Semester 1, 2003 Week 7
CSE9020 / 27
In Summary...In Summary...
Why? Who? When? What? How?
Semester 1, 2003 Week 7
CSE9020 / 28
Bottom LineBottom Line
Testing is expensive
Testing is necessary
You are never going to test everything– use Risk Analysis to determine where
testing should be focused
Semester 1, 2003 Week 7
CSE9020 / 29
Bottom LineBottom Line
Your IE Project is not complete until your client has completed the Acceptance Testing to their satisfaction, and signed-off the project!
Semester 1, 2003 Week 7
CSE9020 / 30
Bottom LineBottom Line
Most Essential
Make sure that– the system– and the group members
maintain versioning control
(link Software Testing and Quality Assurance with with Risks and Ethics)
Semester 1, 2003 Week 7
CSE9020 / 31
More MaintenanceMore Maintenance
Pilot Entry:
Left inside main tyre almost needs replacement
Mechanic Entry:
Almost replaced left inside main tyre
Pilot Entry:
Test flight OK, except autoland very rough
Mechanic entry
Autoland not installed on this aircraft