product quality: metrics, verification, validation, testing
TRANSCRIPT
Arab Open University2nd Semester, 2006-2007
M301
Unit 5.3Product Quality: Metrics,
Verification, Validation, Testing [email protected]
Reem AlAttas © 2
Topic Road Map
Introduction to software qualityVerification, validation and testingMeasuring software quality factorsTesting techniquesFormal technical reviews
Reem AlAttas © 3
What is software quality?
Software that is fit for its purpose and is of sufficiently high quality is said to be of appropriate quality, i.e. in conformance to the requirements and expectations of the customer.
Reem AlAttas © 4
Requirements
There are three general types of requirements that have an effect on software product quality: Product operation requirements how the product will
be used by the end user? Product revision requirements how the product will be
changed? Product transition requirements how the product will
be modified for different operating environments?
Reem AlAttas © 5
Product operation requirements
1. Correctness: that attribute which determines how well a system fulfils the customer’s overall objectives;
2. Reliability: that attribute which determines system can be expected to perform its intended function;
3. Efficiency: that attribute which determines the level of computing resources required perform its function;
4. Integrity: that attribute which determines how well the data is secured;
5. Usability: that attribute which determines how easy the system is to use.
Reem AlAttas © 6
Product revision requirements
1. Maintainability: that attribute which determines how easily bugs can be found and fixed.
2. Flexibility: that attribute which determines the effort required to modify an operational system.
3. Testability: that attribute which determines how easily the system can be tested to show that the customer’s requirements have been met.
Reem AlAttas © 7
Product transition requirements
1. Portability: that attribute which determines how easily the system can be used on another machine
2. Reusability: that attribute which determines how easy it is to reuse some of the software to make future developments more cost-effective
3. Interoperability: that attribute which determines the effort required to couple one system to another.
Reem AlAttas © 8
Primary Software Quality Factors (SQFs)
1. Correctness.
2. Integrity.
3. Maintainability.
4. Usability.
Reem AlAttas © 9
Correctness Measure
The popular measure for assessing correctness is defects per thousand lines of code (defects per KLOC)
Defect (bug): verified lack of conformance to requirements.
Reem AlAttas © 10
Integrity Measure
Integrity is measured by considering the proportion of ‘attacks’ on a product as opposed to bona fide uses.
Reem AlAttas © 11
Example
Reem AlAttas © 12
Topic Road Map
Introduction to software quality Verification, validation and testingMeasuring software quality factorsTesting techniquesFormal technical reviews
Reem AlAttas © 13
Verification
The process of checking that each system description is selfconsistent, and that different system descriptions are consistent and complete with respect to each other.
Reem AlAttas © 14
Validation
The process of checking that each system description is consistent with the customer’s requirements.
Reem AlAttas © 15
Consistency
Two system descriptions are consistent simply means that, where they describe the same part of the system, they do not contradict each other.
Consistency with a customer requirement means that the requirement is met.
Reem AlAttas © 16
Completeness
Every aspect of the customer’s requirements must be met by the system description.
More difficult task than showing consistency.
Reem AlAttas © 17
Testing
The process of executing a software system, or part of a software system, in order to check that it meets its requirements.
It is any form of validation or verification that operates on program code.
Reem AlAttas © 18
Test Procedure
Select some test data.Predict the results that should be
expected.Check whether the software conforms to
our expectations.
Reem AlAttas © 19
Role of Testing
Help finding and fixing bugs.Improve the customer’s confidence in a
software product.Improve the testing process itself.
A cost–benefit analysis of ‘bugs found’ versus ‘time taken’ can be performed to determine when testing should stop.
Reem AlAttas © 20
Types of Testing
1. Usability testing.2. Developmental testing: checks that developmental
activities have been carried out correctly.3. Requirements-based testing: checks that a system
meets the customer’s requirements.4. Regression testing: occurs during developmental
testing and system maintenance, and checks that fixing one bug has not introduced others.
Reem AlAttas © 21
Developmental testing stages
Unit testing, in which units of functionality (e.g. The classes in an object-oriented system) are tested in isolation.
Integration testing, in which previously unit-tested units are tested together.
System testing, in which the completed system is tested against the customer’s requirements.
Reem AlAttas © 22
Requirements testing stage
Acceptance testing, which is performed by the customer, and after which (all being well) the system is accepted as complete.
Reem AlAttas © 23
Topic Road Map
Introduction to software qualityVerification, validation and testing Measuring software quality factorsTesting techniquesFormal technical reviews
Reem AlAttas © 24
Measuring Quality The quality of a software product
can be measured by obtaining values for the eleven software quality factors (SQFs)
Obtaining such values is not a trivial task, as there is no obvious way of evaluating a product against these factors.
What we must do is to find properties of software products which are measurable, and use these to infer the values for the SQFs.
1. Correctness2. Reliability3. Efficiency4. Integrity5. Usability
6. Maintainability7. Flexibility8. Testability
9. Portability10. Reusability11. Interoperability
Reem AlAttas © 25
Software Metrics
1. Accuracy2. Auditability3. Communication commonality4. Completeness5. Complexity6. Conciseness7. Consistency8. Data commonality9. Execution efficiency10. Expandability
11. Generality12. Hardware independence13. Instrumentation14. Modularity15. Operability16. Robustness17. Security18. Self-documentation19. System independence20. Traceability21. Training
Reem AlAttas © 26
The Dependence of the SQFs on the Metrics
Reem AlAttas © 27
Ex. The Relationship Between Reliability and the Metrics
F value for the SQF, reliability. m measured values for the metrics which relate to the SQF,
reliability. c weightings indicating the relative importance of the metrics with
respect to the SQF, reliability.
Reem AlAttas © 28
Measuring System Complexity
1. Lines-of-code Metric: measures the complexity of a method by counting the number of lines-of-code.
More LOC more errors. Should comment lines be included in the line
count?
Reem AlAttas © 29
… Measuring System Complexity
2. McCabe’s Cyclomatic-complexity Metric: measures the complexity of a method by counting the number of independent paths through a method body. A method body starts and completes its execution defines one
independent path. The number of independent paths through a piece of code can be
calculated by counting the number of decision points. Each if statement adds a decision point. Each while, do-while and for loop adds a decision point. Each switch statement adds one decision point for each of the cases it
tests for, excluding any default case. Each try statement adds one decision point per catch block, but any
finally block is not a decision point, as it will always be executed. The operators && and || each add a decision point.
Reem AlAttas © 30
… Measuring System Complexity
A cyclomatic-complexity of ten or more should be a hint that a method body should be restructured into two or more simpler methods.
Reem AlAttas © 31
Modern Object-oriented Metrics Depth of inheritance tree (DIT): its number of ancestors.
Coupling between objects (CBO): the number of has-a relationships the class has with other classes.
Number of children (NOC): the number of children for that class.
Response for a class (RFAC): the size of the response set for the class, which consists of all the methods of that class together with all the methods of other classes called by those methods.
Lack of cohesion in methods (LCOM): its cohesiveness.
Weighted methods per class (WMPC): its complexity of behavior the sum of the cyclomatic complexities of each method of the class.
Reem AlAttas © 32
Topic Road Map
Introduction to software qualityVerification, validation and testingMeasuring software quality factors Testing techniquesFormal technical reviews
Reem AlAttas © 33
Testing Techniques
Black-box testing we choose test cases by looking at the specification (i.e. requirements and high-level design) of the system to be tested.
White-box testing we choose test cases by looking at the detail of the implementation of the system to be tested.
Reem AlAttas © 34
A strategy for black-box testing (using partitioning)
1. For each method in the class, determine the input data space.
2. Partition the input data space into sub-domains.
3. Test all sub-domains given by the case analysis.
Reem AlAttas © 35
Step 1
Reem AlAttas © 36
Step 2
Reem AlAttas © 37
Step 3
Reem AlAttas © 38
White-box Testing Techniques
Basis-path testing.Loop testing.
Reem AlAttas © 39
Basis-path testing
Ensures that all reachable statements in a method are tested at least once. Select data which will exercise the straight-through
path, i.e., loop and if conditions evaluate to false, and only default cases selected in switch statements.
Then, find data that deviates from the straight-through path at the first decision point.
The process is then repeated for each subsequent decision point in the program, varying the flow in each case.
Reem AlAttas © 40
Loop testing
Used to test loop constructs. In Java, there are three ways in which
loops can occur: Simple loops. Nested loops. Concatenated loops.
Reem AlAttas © 41
Simple loops forms
Test data should be found for the following cases: the loop is skipped entirely the loop is passed through exactly once the loop is passed through more than once, maximum n times
m times, where 3 < m < n – 2 pass through the loop n – 1, n and n + 1 times.
Reem AlAttas © 42
Nested loops form
Approach: Start with the innermost loop. Work outwards. Repeat step 2 until all loops are tested.
Reem AlAttas © 43
Concatenated loops
Concatenated loops can be tested separately using sequential simple loop tests
Reem AlAttas © 44
Topic Road Map
Introduction to software qualityVerification, validation and testingMeasuring software quality factorsTesting techniques Formal technical reviews
Reem AlAttas © 45
Inspections (Formal Technical Reviews)
A formal, efficient, and economical method of finding errors in design and code.
TMA5 – Q3
Thank You!