metrics for manual testing

Metrics – Manual Testing

Anup Panigrahi

1

Introduction

Metrics collected during various phases of testing has been described

for the different phases in the following slides.

1) Requirement Phase

2) Test Design Phase

3) Test Execution Phase

2

Requirement Phase

3

Requirement PhaseFormula:

(Total number of Original Requirements + Cumulative number of requirements changed (till date) + Cumulative number of requirements added (till date) + Cumulative number of requirements deleted (till date)) / (Total number of Original Requirements)

Description:

This metric is calculated taking added, deleted, changed requirements into consideration.

This helps to identify the exact requirements that are required for further design activities.

4

Requirement Stability Index

(based on Effort)Formula:

((Estimated Effort for the completed/ongoing stages) / (Estimated Effort for the completed/ongoing stages - Effort Estimated for Changes till that stage)))

Description:

This metric is used to calculate the Requirement stability index based on the effort spent on the requirement analysis

5

Requirements Leakage Index

Formula:

(No. of missed Requirements / No. of Initial Requirements)

Description:

This metric helps to identify the requirements that have possibly been missed out during the requirement analysis phase.

6

Test Design Phase

7

Test Case Preparation Productivity

Formula:

(No of Test Cases /scripts) / (Effort spent for Test Case/Script Preparation)

Description:

Test case preparation productivity is used to calculate the number of test cases prepared and the effort spent for the same.

8

Test Execution Phase

9

Test Case Pass Percentage

Formula:(Total Number of test cases or scripts executed and passed / Total Number of test cases or scripts prepared)*100

Description:The metric calculates the test case pass percentage which helps to determine the functional coverage of the business components

10

Test Case Execution Percentage

Formula:(Total number of test cases or scripts executed/Total number of test cases or scripts prepared)*100

Description:The metric calculates the test case execution percentage, from which the execution efficiency can be measured

11

Defect Summary

• Description:Defect summary gives the status of the defects, whether it falls in the following categories: Submitted (New/Open), Re/Assigned, Fixed, Verified, Closed, Rejected, Duplicate, Deferred.

12

Defect Discovery Rate

Formula:Total number of defects found in application /Number of test cases or scripts executed.

Description:This metric is useful to calculate the rate of identifying defects based on the test cases executed. This metric helps to evaluate the efficiency of the team in identifying the defects.

13

Defect Severity

Description:Defect Severity helps to classify the defects based on the following categories: Critical, Serious, Important, Non Critical, Out Of Scope. Defects are given importance based on the severity. For example, Critical and Serious defects has to be given immediate attention , so that these defects does not affect the business functionalities.

Module wise severity is also calculated which gives the defect severity for each module.

i.e., System Crash is a Critical Defect (Severity 1)

14

Defect Density

Description:This metric shows the defect population in each module. Based on this metric, attention is paid to the module which has more defects as well as high defect density percentage. This metric also helps to compare the defects spread across all

modules and take necessary action accordingly.

15

Defect Rejection RatioFormula:

(Defects Rejected/Defects Raised)*100

Description:This metric is used to calculate the percentage of defects that has been rejected among the overall defects raised. Defects have to be verified and analyzed before logging to avoid high rejection ratio.

Notes:

We should not consider the defects those are rejected due to Business reasons, or sometime rejection happens as the Business seeing no scope to fix it, or Browser Issue, Cancelled, Out of Scope,

Region Issue, etc.

16

Thank You!

17