important points to note -...
TRANSCRIPT
Important Points to Note • All Participating colleges are requested to mute your telephone lines during the webinar session.
• Participants are requested to make note of questions if any and post it to the instructor using the
chat option during the webinar session .
• Question & Answer(Q&A) is scheduled at the end of the session and each college is given a
chance to ask 2-3 questions. Answers to the questions posted by colleges through chat option
will also be answered during the Q&A.
• Instructor may initiate the relevant Polls at regular intervals during the session. Colleges are
requested to gain inputs from all the participants and provide your responses to these polls
initiated by instructor.
• Session Feedback will be initiated at the end of the webinar. Colleges are requested to get the
collective opinion from the participants and provide your responses for the feedback
questionnaire.
Session Rationale and ObjectiveSession Name: Software Testing
Session Rationale: The purpose of learning this Session is to make the learner understand that various types of
testing which is an important phase in software development process. Also, testing will help the team to produce
defect free and quality code.
Session Objective:
At the end of this Session, you will be able to:At the end of this Session, you will be able to:
• Develop test plans to carry out Unit testing
• Perform unit and integration testing
• Understand the code review guidelines and checklist and apply them to produce
quality code
Session PlanSession Name: Testing
ContentLearning Aid /
Methodology
Trainer
Approach
Learner
Activity
Learning Outcome
(Skill, Competency)
Objective of Testing
and Test plans
generation
Presentation, Scenario
Explains,
Discusses,
facilitates
Participates,
Absorbs, RespondsAbility to generate Test plans
5
generation facilitates
Recording / Logging
a DefectPresentation, Hands-
on
Explains,
Discusses,
facilitates
Participates,
Absorbs, RespondsAbility to log defects during Unit testing
Different types of
Testing Techniques Presentation
Explains,
Discusses
Participates,
Absorbs, RespondsAbility to understand various testing types
Testing – What Is it?
• A process of demonstrating that errors are not present?
OR
• A way of establishing confidence that a program • A way of establishing confidence that a program does what it is supposed to do?
OR
• A means of achieving an error-free program by finding all errors?
Testing is…
• A process of executing a program with the
intent of finding errors
• A “DESTRUCTIVE”, yet creative process
Why Testing?
• Verifies that all requirements are implemented correctly (both for positive and negative conditions)
• Identifies defects before software deployment
• Helps improve quality and reliability• Helps improve quality and reliability
• Makes software predictable in behavior
• Reduces incompatibility and interoperability issues
• Helps marketability and retention of customers
Cost of Ineffective Testing
• Time– Late Releases– Projects need to be reworked or abandoned
• Money– Budget over-runs– Budget over-runs– Defects are 100 to 1000 times more costly to find and
repair after deployment
• Quality– Developers unsure of product quality– Products released with undiscovered or unresolved defects
Testing Lifecycle
Iteration nDevelopment Lifecycle
Project Planning
Requirements Capture
Implementation
Build 0 Build 1 Build 2
Analysis & Design
Test Lifecycle
Plan Test
Design Test
Setup Test Bed
Execute TestEvaluate Test
Defect Tracking/Enhancement Request
Testing Lifecycle
Requirements Capture
Analysis
Test planning & Scenario Design
Test Case
The Scenarios Design and the Test Case Development could normally start in parallel with the Development Cycle.Test Execution Synchs up with the Development Cycle during the functional Testing Phases
Test Case Development
Test Execution
Test Result Analysis
Test Cycle Closure
Defect Fixing Cycle
Defects
New Version
Testing Lifecycle
• Test requirements identification– Validate for testability
• Test Planning and Scenario design– Develop Test Objectives– Develop Test Objectives
– Identify Test Items
– Resources and Schedules
• Test Case Development– Test Case Specification
– Pre-requisites, post-requisites and Acceptance criteria
Testing Lifecycle (contd.)
• Prepare Test Bed– Test Data Preparation– Test Environment Setup
• Test Execution– Run Tests and Validate Results– Run Tests and Validate Results– Bug Reporting– Bug fixes and retesting
• Test Result Analysis– Defect Analysis– Determining Test Set Coverage and Effectiveness
Start Testing Early
• There are two basic forms of testing
– Verification Testing
– Validation Testing
• Verification: “Are we building the right thing?”
• Validation: “Are we building it right?”
Testing Stages – the V-Model
Requirement Analysis
Functional Specifications
User Acceptance Testing
System Testing
High-Level Design
Detailed Design
Code
Integration Testing
Unit Testing
Verification ValidationReviews
Testing Techniques
• Static Testing - Verification
• Dynamic Testing - Validation
– White Box Testing– White Box Testing
– Black Box Testing
Testing Techniques (contd.)
• Static Testing/Reviews– Scope
• Requirements Feasibility
• Design and Technical Architecture
• Program specifications and Code• Program specifications and Code
• Test Plans, Test Cases and Data
• User Documents
– Modes• Reviews
• Inspections
• Code Walkthroughs
Testing Techniques (contd.)
• Static Testing– Advantages
• Capture defects early, so saves rework cost
• Checklist-based approach
• Focuses on coverage• Focuses on coverage
• Group perspective
• High probability of finding defects
– Disadvantages• Cannot test data dependencies
• High skill levels required
Testing Techniques (contd.)
• Dynamic Testing
– White-box/Glass-box Testing
• Unit Testing• Unit Testing
• Integration/Interface Testing
– Black-box Testing
• System Testing
• Acceptance testing
White-Box/Glass-box Testing
• Structural Testing process
• Program Logic-driven Testing
• Design-based Testing• Design-based Testing
• Examines the internal structure of program
White-Box Testing – Advantages
• High Code coverage – Exhaustive (thorough)
path testing
• Program logic is tested• Program logic is tested
• Internal boundaries are tested
• Testing is algorithm specific
Black-box Testing – Advantages
• User’s perspective
• Focus on features, not implementation
• Big-picture approach• Big-picture approach
Levels of Testing
• Unit Testing
• Integration Testing
• Function Testing• Function Testing
• Performance Testing /System Testing
• Acceptance Testing
• Installation Testing
Testing Stages – the V-model
Requirement Analysis
Functional Specifications
User Acceptance Testing
System Testing
High-Level Design
Detailed Design
Code
Integration Testing
Unit Testing
Verification ValidationReviews
Unit Testing (1 of 2)• Each individual unit of code is tested to ensure that it performs its intended
functionality
• Unit tests are done on their respective modules by Software Engineer who has written code
• Unit tests are created using some techniques which ensure that all logical paths of the code unit are tested and maximum number of errors can be detected by executing these tests
• The Unit Test Plan (UTP) prepared towards the end of the design phase is used for Unit Testing the software or code
Unit Testing (2 of 2)
• Any defects found during unit testing are logged in the Defect Tracking System (DTS) and they are tracked till the defects are removed from the code
• Test Case: A set of inputs, execution preconditions, and expected outcomes developed for a particular objective, such as to exercise a outcomes developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement
• Precondition for a test case: A precondition for a test case is any activity that is required to be completed in order for the stage to be set for the test case execution. A precondition is one without which the test procedure cannot be executed
Documenting Test Cases (1 of 2)
• Very often test plans contain hundreds of test cases and so it is essential to keep documentation of test cases clear and in simplest terms as possible
Typical Test Plan Template
SlNo
Test case name
Test Procedure Pre-condition (Input
Condition)
Expected Result
Reference to Detailed Design / Spec Document
• A test case name should be of the following format.
<Module Name>_<Function Name>_<Test Procedure>,
where
– Module Name is the name of the module the test case tests
– Function Name is the name of the function or functionality the test case tests
– Test Procedure is a term or word which briefly represents what the test case is trying to do
Documenting Test Cases (2 of 2)
• Test Procedure (Condition to be tested): Explains briefly but clearly, what the test case is doing.
• Input Condition: Input Condition contains details of any specific preconditions
• Expected Result: sometimes known as Pass/Fail Criteria
Identifying Test Cases
• Boundary Value Analysis
• Equivalence Partitioning
• Logic Coverage
• Random Generation
Boundary Value Analysis (1 of 5)
• A boundary value is one which indicates the border (or the limit) of a value
• Test cases that explore boundary values have the highest payoff in terms of detecting bugs, as the most common errors occur at the boundary levelthe boundary level
• Boundary value test cases are those which test the function or field using values that – Lie on the boundary
– Just above the boundary
– Just Below the boundary
• For example if an input condition specifies that
the range of values of the input variable items
must be from 0 to 100, the boundary values
would be -1, 0,1,99,100 and 101
Boundary Value Analysis (2 of 5)
would be -1, 0,1,99,100 and 101
/************************************************** **********/* Function: fnFindGrade * Description: Given the percentage score of studen t, * assigns the grade of the student.* Criteria for Grades:* A - 80 to 100* B+ - 73 to 79* B – 65 to 72* C – 55 to 64
Boundary Value Analysis (3 of 5)
* B – 65 to 72* C – 55 to 64* D – 0 to 54* Z – for invalid grades (Score <0 or score >1 00)* Input Parameters: * int iPercentScore - Percentage scored by the stude nt* Char acGrade[] – Array containing the grade assign ed* Returns:None*************************************************** **********/
void fnFindGrade (int iPercentScore, char acGrade[]) {if (iPercentScore >=80 && iPercentScore <=100) {
strcpy(acGrade,”A”);}else if (iPercentScore >=73 && iPercentScore <=79) {
strcpy(acGrade,”B+”);}else if (iPercentScore >=65 && iPercentScore <=72) {
strcpy ( acGrade ,”B”);
Boundary Value Analysis (4 of 5)
strcpy ( acGrade ,”B”);}else if (iPercentScore >=55 && iPercentScore <=64) {
strcpy(acGrade,”C”);}else if (iPercentScore >=0 && iPercentScore <=54) {
strcpy(acGrade,”D”);}else {
strcpy(acGrade,”Z”);}
}
Boundary Value Analysis (5 of 5)Sl No Test case name Test Procedure Pre-condition Expected Result
Reference to Detailed Design / Spec Document
1fnFindGrade_MinusOne
Call fnFindGrade with iPercentScore = -1
None “Z” should be assigned to grade (Negative Test case)
fnFindGrade
2 fnFindGrade_0 Call fnFindGrade with iPercentScore = 0
None Grade “D” should be assigned
fnFindGrade
3 fnFindGrade_1 Call fnFindGrade with iPercentScore = 1
None Grade “D” should be assigned
fnFindGrade
4 fnFindGrade_99 Call fnFindGrade with iPercentScore = 99
None Grade “A” should be assigned
fnFindGrade
5 fnFindGrade_100 Call fnFindGrade with iPercentScore = 100
None Grade “A” should be assigned
fnFindGrade
6 fnFindGrade_101 Call fnFindGrade with iPercentScore = 101
None “Z” should be assigned to grade (Negative test case)
fnFindGrade
Equivalence Partitioning (1 of 3)
• This consists of dividing all possible inputs into a set of classes, where either all inputs that fall into a given class are valid or all are invalid. Then selecting a few test cases from each class is sufficient
• For example if an input condition specifies that the range of values of the • For example if an input condition specifies that the range of values of the input variable items must be from 1 to 999, one valid equivalence class would be 1<=items<=999 and 2 invalid equivalence classes would be items<1 and items>999
• The values chosen for Equivalence partition should not be a boundary value
Equivalence Partitioning(2 of 3)Sl No Test case name Test Procedure Pre-
conditionExpected Result Reference to Detailed Design /
Spec Document
1 fnFindGrade_D20 Call fnFindGrade with iPercentScore = 20
None Grade “D” should be assigned
fnFindGrade
2 fnFindGrade_D48 Call fnFindGrade with iPercentScore = 48
None Grade “D” should be assigned
fnFindGrade iPercentScore = 48 assigned
3 fnFindGrade_C59 Call fnFindGrade with iPercentScore = 59
None Grade “C” should be assigned
fnFindGrade
4 fnFindGrade_B71 Call fnFindGrade with iPercentScore = 71
None Grade “B” should be assigned
fnFindGrade
Equivalence Partitioning(3 of 3)Sl No Test case name Test Procedure Pre-
conditionExpected Result Reference to Detailed Design /
Spec Document
5 fnFindGrade_A90
Call fnFindGradewith iPercentScore = 90
None Grade “A” should be assigned fnFindGrade
6 fnFindGrade_InvalidMinus
Call fnFindGradewith iPercentScore =
None “Z” should be assigned to grade fnFindGrade_InvalidMinus
30 -30 (Negative test case) fnFindGrade
7 fnFindGrade_Invalid300
Call fnFindGradewith iPercentScore=300
None “Z” should be assigned to grade (Negative test case) fnFindGrade
Logic Coverage (1 of 3)• This technique aims to generate enough test cases so that an appropriately
defined coverage criterion is met
• Criterion: Every statement in the program must be executed at least once, every branch in the program must be executed at least once, or every path in the program must be executed at least once
• Example:• Example:– The User Interface for searching the address book should be very friendly and should
use the following search criteria
– Can search the address book based on one or more of the following fields:• E-mail Id of the person
• Name of the person
• Employee Number of the person
– Can accept one or more fields for search criteria
– Accepts partial values for E-mail and Name
Logic Coverage (3 of 3)Sl No Test case name Test Procedure Pre-condition Expected Result
Reference to Detailed Design / Spec Document
1 AddrBook_All_BlankAll the fields are kept blank and click on 'search'
NoneAddress Book must display an error message and prompt the user to enter atleast one field (Negative test case)
Address Book Module
2 AddrBook_EmpNo_OkType in an Employee Number and then click on 'search'
NoneAddress Book must fetch only one entry of the person with that employee number
Address Book Module
Type in an invalid Employee Number Address Book must fetch zero records and
3 AddrBook_EmpNo_FailType in an invalid Employee Number and then click on 'search'
NoneAddress Book must fetch zero records and display that the record is not found (Negative test case)
Address Book Module
4 AddrBook_Email_FullType in a full email-id (aniket_sharma) and then click on 'search'
NoneAddress Book must fetch only one entry of the person with the corresponding email-id
Address Book Module
5 AddrBook_Email_PartialType in a partial but valid email-id (aniket) and then click on 'search'
NoneAddress Book must fetch one or more records where email-id begins with same letters
Address Book Module
6 AddrBook_Email_FailType in an invalid email-id (kkgghh) and then click on 'search'
NoneAddress Book must fetch zero records and display that the record is not found (Negative test case)
Address Book Module
Random Generation
• Data is generated randomly either using a tool or
manually. This is the simplest method but not
the most efficient
Scenario
Problem Description: Online Balance Enquiry
The User interface design for balance enquiry is given. PIN number can take at least 4 digits
starting from 1000; Balance amount is displayed if valid PIN number is entered by the user.
Otherwise, it displays appropriate error message.
Identify and document test cases for a given user interface design using the techniques
learnt.
Otherwise, it displays appropriate error message.
PIN Number:
Balance Amount: ………………………………………………………
Submit
Recording / Logging a Defect
• Any defect found in code or document must be recorded
• Recording of defects will ensure that the defects detected
are not lost and also ensures that they are tracked till the are not lost and also ensures that they are tracked till the
defects are removed completely
• Defect data can also be used to do an analysis, which will
help improve the process of writing code and testing
Defect Tracking System• Most software companies have a dedicated system, for logging and
tracking defects
• Most defect tracking systems can also do detailed analysis of the defects to help a project take corrective action in due course of time.
• In the absence of a Defect Tracking System, an MS Excel sheet (spreadsheet) is maintained to log and track defects
• When testing or code review is done and defects are found, defects are logged
• A defect log captures some of the following information:
– Defect Number or Id: The number or Id which identifies the defect
– Submitted by: Person who found the defect
– Description: A detailed description of the defect
– Detected Stage: The stage at which defect was detected
• Example: Unit Testing, Code Review etc.
A sample defect tracking Excel sheet (2 of 2)
• Example: Unit Testing, Code Review etc.
– Assigned to: The developer who has to remove this defect
– Type of defect: Type of defect tells about the nature of the defect
• Example: Coding Standards related, Logical Error
– Injected Stage: The stage of software life cycle where this defect might have been introduced
• High Level Design, Detailed Design, Coding etc
– Action Taken: To be fixed or fixed
– Notes: Additional notes such as a guideline for fixing the bug could be included
Testing Stages – the V-model
Requirement Analysis
Functional Specifications
User Acceptance Testing
System Testing
High-Level Design
Detailed Design
Code
Unit Testing
Integration Testing
Verification ValidationReviews
Integration Testing
• Combining and testing multiple components
together
• Integration of modules, programs and • Integration of modules, programs and
functions
• Tests Internal Program interfaces
• Tests External interfaces for modules
Integration TestingBanking System
CRM HR Financial External interface
External System
Java Based Based on C, C++ VB based
Online banking Loans Account OpeningInter module interface
Intra module interface Unit Module
Approach for testing interfaces
Intra-Module Inter-Module External
Test Techniques White Box Black Box Black Box
Test Environment Setup Development environmentIsolated development environment
Simulate live environment
Test ApproachCheck for parameter / procedure called.Database updation
Identify pre-process and approach to Interface.Identify the post-process to validate the interface
Identify external / internal interface and validate against external system
Techniques based on order of Integration
• Big-bang Integration (non-incremental)
• Random Incremental Integration
• Top-Down Integration• Top-Down Integration
• Bottom-up Integration
Top-down vs. Bottom-up
• Top-down Integration– Advantage
• Key Interface defects trapped earlier
– Disadvantage• Core functionality tested late in the cycle• Core functionality tested late in the cycle
• Bottom-up Integration– Advantage
• Core functionality tested early
– Disadvantage• Key Interface defects trapped late in the cycle
Testing Stages – the V-model
Requirement Analysis
Functional Specifications
User Acceptance Testing
System Testing
High-Level Design
Detailed Design
Code
Integration Testing
Unit Testing
Verification ValidationReviews
System Testing – What is it?
• A Black-box Testing technique
• Specifications-based Testing
• Typically independent team testing
• Simulated environment testing
• Live/Simulated user data• Live/Simulated user data
• Tests the whole system
• Functional and non-functional requirements tested
• Business transaction-driven testing
• Compatibility errors uncovered
• Performance limitations uncovered
System Testing – Different Types
• Functional testing (Sanity / Regression )
• Performance and Scalability testing
• Load/Stress testing
• Volume Testing
• Usability testing• Usability testing
• Configuration testing
• Installability testing
• Disaster and Recovery testing
• Reliability / Availability testing
• Statistical Testing based on Usage Models
Functional Testing – Sanity Test
• Very basic minimal number of tests to verify the product for the feature / protocol compliance
• Could be a subset of regression test used to validate only high level functionality
• Typically an initial testing effort to determine if a new • Typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort
• Sanity testing will be performed in scenarios when cursory testing is sufficient to prove that the application is functioning properly
Why Sanity Testing?
• Regression testing is more time taking and more effort is required whereas Sanity testing is quicker focusing on important functionality of software
• Saves time as only selected high level functionality areas are only tested. Bugs found this way would take
• Saves time as only selected high level functionality areas are only tested. Bugs found this way would take almost triple the time when found through regression testing
• Efficient in scenarios wherein releases comes in pretty often with very few changes
Functional Testing – Regression
• Re-execution of one or more tests in subsequent build of the application/product to ensure– Revisiting and testing all prior bug-fixes in response to a
new fix/enhancement
Re-testing all programs that might be affected by the – Re-testing all programs that might be affected by the fix/enhancement
– Hidden Bugs are uncovered
• The baseline for Regression Testing grows with every build
Regression Testing
• Should Cover– Business Process depending on criticality
– User friendliness
– Cross functional dependencies– Cross functional dependencies
• Activities– System appreciation
– Preparation of Test case repository
– Automate test cases
– Execute Regression test suite
Without Regression Testing – Hidden Bugs
PASS PASS? PASS?Test Requirement 1
UnknownUnknown
CBA
Software Builds
FixFAIL FAIL PASS
PASS BUG? BUG?
Requirement 1
Test Requirement 2
Test Requirement 3
NewProblem
HiddenBug
Fix
With Regression Testing – No Hidden Bugs
PASS PASS PASSTest Requirement 1
CBA
Software Builds
FixFAIL FAIL PASS
PASS BUG PASS
Test Requirement 2
Test Requirement 3
NewProblem
Fix
Fix
Acceptance Testing
• Demonstrates that the system meets mutually agreed Acceptance criteria
– Critical Requirements
– Minimum Performance level– Minimum Performance level
– Maximum Defect Detection Rate
• Typically, a sub-set of System testing
Alpha / Beta Testing
• Forms of Acceptance testing
• Testing in the production environment
• Alpha testing is performed by end users within • Alpha testing is performed by end users within
a company but outside development group
• Beta testing is performed by a sub-set of
actual customers outside the company
Performance Testing
• Number of concurrent users accessing at any point in given time
• System’s performance under high volume of data
• Stress testing for systems, which are being scaled up to • Stress testing for systems, which are being scaled up to larger environments or implemented for the first time
• Operational intensive transactions. (Most frequently used transactions)
• Volume intensive transactions (for both volume and stress testing)
Installation Testing
• Basic installation
• Installation of various configurations
• Installation on various platforms• Installation on various platforms
• Regression testing of basic functionality
When do you stop testing?
• When time runs out
• Specific test case design methods
• Target test coverage attained
• Certain number of errors found*
• Error detection rate drops below a threshold*• Error detection rate drops below a threshold*
• Predicted density of errors left drops below a threshold
Sequence test cases in order of priority
* Infosys Process Capability Baseline
Test Artifacts
• Test Plan
• Test Environment/Bed
• Test Case
• Test Data• Test Data
• Test Tools
• Test Scripts
• Test Log
• Bug Reports
Traits of a good Tester
• Destructive creativity
• Detective skills
• Appreciating the users’ perspective • Appreciating the users’ perspective
• Adapt to and Understand Requirements change
• A skeptical, but not hostile attitude
• An eagerness to embrace new technologies
Test Engineers
• Role - Execute tests, record test scripts, maintain statistics and metrics, check test data setup, test environment setup, execute re-tests
• Knowledge - Understanding of the importance of testing, awareness of tools, how to progress against a plan
• Knowledge - Understanding of the importance of testing, awareness of tools, how to progress against a plan
• Skills - Observation, accuracy, methodical, co-ordination, problem solver
• Experience - Following instructions, problem reporting and solving, and relevant testing tools
Test Automation
• Good candidates:– Complex and time-consuming tests
– Tests requiring a great deal of precision
– Tests involving many simple, repetitive tests– Tests involving many simple, repetitive tests
– Tests involving many data combinations
• Lower priority candidates:– One-time only tests
– Testing peripheral devices
– Subjective assessment tests (look and feel based)
Types of Testing Tools
• The software testing tools broadly of the following type:
– Functional / regression–– Load /Stress: Performance testing under high load
/ stress
– Web Testing: testing the Web Sites
– Data Generation: Generation of test data
Benefits
• Reduces manpower and time
• More coverage within the same testing time.
• Testing products which are very difficult to • Testing products which are very difficult to
test manually. Example, Load Tests
• Can look inside the software for memory
leaks, redundant code etc.
Test Tools (Client Server)Category Tool Vendor
Source Code Testing
BoundsChecker Numega
Pure Coverage Rational
Purify Rational
Jprobe Sitraka Software
ATTOLCoverage ATTOL Software
QA Run Compuware
Functional Testing
QA Run Compuware
Win Runner Mercury
Team Test Rational
QA Center Compuware
Performance Testing
OptimizeIT VM Gear
Load Runner Mercury
QA Load Compuware
Test Tools (Web Applications)
Category Tool Vendor
Functional Testing Rational Robot Rational
Silk Test Segue Software
Performance Testing Silk Performer Segue Software
Web Load Radview
Link & HTML Testing NetMechanic Monte Sano
Linkbot Tetranet
Site Inspector
Doctor HTML Imageware
Hands-on
Problem Description: Online Balance Enquiry
Implement the Balance Enquiry module in C and Perform Unit Testing and Log Defects.
Implement the Balance Enquiry module by writing a function. The data structure can
be 2-D array for storing set of pin numbers and the corresponding balance amount. be 2-D array for storing set of pin numbers and the corresponding balance amount.
Accept a pin number from the user, if it is found in the array, then display the
corresponding balance amount. If not, display suitable error message. After writing
the source code, test it and log the defects in defect tracking system.