verification and validation: a quick introduction
DESCRIPTION
Verification and Validation: A Quick Introduction. Authors Massood Towhidnejad Mike Rowe David Dampier Sponsored In Part by NSF Grant EEC-0080502 And published on www.swenet.org. Learning Objectives. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/1.jpg)
Verification and Validation: A Quick Introduction
AuthorsMassood Towhidnejad
Mike Rowe David Dampier
Sponsored In Part by NSF Grant EEC-0080502 And published on www.swenet.org
![Page 2: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/2.jpg)
Learning Objectives1. The student will be able to define both verification and
validation, and to know the difference between them.2. The student will be able to identify the appropriate activities
and their classification as either verification or validation.3. The student will be able to distinguish between verification
and validation, given a set of activities.4. Given a set of software artifacts, the student will understand
the appropriate verification or validation activity to that artifact.
5. Given a set of projects in different categories, the student will be able to determine which category of projects would require more V&V activity, and which would require less.
6. Given a set of defects and the phases of development in which they were found, the student will be able to estimate the relative cost of correcting errors early or late in the lifecycle.
![Page 3: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/3.jpg)
Why Invest in Quality? Cost effective Provides competitive edge Essential for business survival Essential for international marketing Helps to retain customers, and
increase profits Hallmark of world class business
![Page 4: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/4.jpg)
Ways to Improve Quality Prevention of Defects
• Process Improvement• Complexity Reduction• Risk Management• Causal Analysis
Detection and Correction of Defects• Verification• Validation• Rework• Causal Analysis
![Page 5: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/5.jpg)
Verification
Verification – Are we building the product right?
Verification is any checking process conducted on software artifacts in an attempt to determine if they work as specified by the designers of the system.
Includes reviews, inspections, walkthroughs, unit testing and integration testing.
![Page 6: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/6.jpg)
Validation
Validation – Are we building the right product?
Validation is the process of evaluating software artifacts during the software development process in an attempt to determine if the system works as required by the customers. Any evaluation activity that involves the customer can be used for validation purposes.
Includes program reviews, system testing, customer acceptance testing.
![Page 7: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/7.jpg)
Verification vs. Validation Verification
• Main purpose is to detect defects in the artifacts of the system under development.
Validation• Main purpose is to show that the system
under development meets user needs, requirements, and expectations.
![Page 8: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/8.jpg)
Verification & Validation Techniques Static Methods
• Techniques applied to artifacts without execution.
Dynamic Methods• Techniques applied to artifacts through
execution. Mathematically Based Methods
![Page 9: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/9.jpg)
Static: Reviews Walkthroughs
• Code verification• Document
ConOps, SRS validation STEP, SAD, SDD verification
Inspections• Code verification• Document Audits verification
Program Reviews• Customer involved validation• No customer verification
![Page 10: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/10.jpg)
Effectiveness of Static Verification More than 60% of program defects can be
detected by program inspections. More than 90% of program defects may be
detectable using more rigorous mathematical program verification.
The defect detection process is not confused by the existence of previous defects.
![Page 11: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/11.jpg)
Dynamic: Testing (Verification) Unit Test (Detailed Design):
• Testing the individual software modules, components, or units.
Integration Testing (Architectural Design): • After unit test, the system is put together in
increments. Integration testing focuses on the interfaces between software components (OO thread-based, cluster-based testing)
System Testing (Requirements Spec): • One goal of system testing is to ensure that
the system functions as specified in the specification.
![Page 12: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/12.jpg)
Dynamic: Testing (Validation)
System Testing (Requirements Spec):• Another goal of system testing is to ensure
that the system functions as the client expected in a controlled environment.
User Acceptance Test (ConOps):• A set of formal tests run for the client, and
specified by the client. When the system passes these tests, the software has been accepted by the client as meeting the requirements.
![Page 13: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/13.jpg)
Verification and Validation in the Development Lifecycle
CodeExecute Unit
Tests
White-Box &Black-Box
Testing
Black-BoxTesting
RequirementsAnalysis
ExecuteSystem Tests
Execute Integration
TestsDesign
Validate the System
Verify Design
Verify Implementation
Reviews
![Page 14: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/14.jpg)
Mathematics-Based Verification
Verification is based on mathematical arguments which demonstrate that a program is consistent with its specification.
Programming language semantics must be formally defined.
The program must be formally specified.
![Page 15: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/15.jpg)
Why Inspection is important? Relative cost to fix a defect Phase in which found Cost ratio
(hours)– Requirement 1– Design 3-6– Coding 10– Testing 15-70– Operation 40-1000
Data derived from Capers Jones.
![Page 16: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/16.jpg)
Costs of Finding and Fixing Late
Delays in identifying and fixingdefects gets geometrically more
expensive as the lifecycle progresses!
$1
$10
$100
$1000
Cost
of F
ixin
g a
Def
ect
Defects
Time in (Phase of) Development
![Page 17: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/17.jpg)
Cost of Quality Includes all costs of quality-related
activities. Quality costs =
+Prevention costs+Detection and Appraisal costs+Failure costs
• Internal failure costs• External failure costs
Cost of Quality (COQ) – refers to the cost of correcting defects once found.
![Page 18: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/18.jpg)
Quality Cost Components Direct Cost
– Reviews/inspections– Unit testing– System testing– Acceptance testing– Test planning and
design– Computer time– Resources (terminals,
staffs, etc.)
Indirect Cost– Rework– Recovery– Corrective action cost– Failures– Analysis meeting– Debugging– Retesting– Legal fees
![Page 19: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/19.jpg)
Verification or… Validation? Reviews
Unit Testing Integration Testing System Testing Acceptance Testing
![Page 20: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/20.jpg)
Verification or… Validation?
Reviews Either
Unit Testing
Integration Testing System Testing Acceptance Testing
![Page 21: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/21.jpg)
Verification or… Validation?
Reviews Either
Unit Testing Verification
Integration Testing
System Testing Acceptance Testing
![Page 22: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/22.jpg)
Verification or… Validation?
Reviews Either
Unit Testing Verification
Integration Testing Verification
System Testing
Acceptance Testing
![Page 23: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/23.jpg)
Verification or… Validation?
Reviews Either
Unit Testing Verification
Integration Testing Verification
System Testing Validation
Acceptance Testing
![Page 24: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/24.jpg)
Verification or… Validation?
Reviews Either
Unit Testing Verification
Integration Testing Verification
System Testing Validation
Acceptance Testing Validation
![Page 25: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/25.jpg)
Balancing Trade-Offs What is the real requirement? What are the expected benefits of
V & V vs.– cost of increasing V&V activity– additional time required– technical feasibility– risk of not finding defects
» delivery» operation
“When is it good enough?”(Bach, 1997)
![Page 26: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/26.jpg)
Summary Verification and Validation are both
important to the quality of software. Each is different:
• Verification determines that what we are doing is done correctly.
• Validation determines whether we are doing the right thing.
Sometimes, less quality is okay. Some software systems require less V&V, as the cost is not supportable. Some require more no matter what it costs.
The earlier defects are discovered, the less it will cost to correct them.
![Page 27: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/27.jpg)
Backup Slides
![Page 28: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/28.jpg)
• Design • Coding
• System Requirements• Contractor Management
• Evaluate Requirements• Evaluate Design
• Monitor Tests• Integration
TestedProducts
Discrepancies
Subs
Project Manager
IV&V
• Testing
Independent Verification and Validation (IV&V)
![Page 29: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/29.jpg)
Static and Dynamic Verification & Validation
Formalspecification
High-leveldesign
Requirementsspecification
Detaileddesign Program
Prototype Dynamicvalidation
Staticverification
Sommerville, p. 421
![Page 30: Verification and Validation: A Quick Introduction](https://reader035.vdocuments.us/reader035/viewer/2022070500/56816854550346895dde6834/html5/thumbnails/30.jpg)
Test Case Development in the V Model
Code Execute Unit
Tests
Test cases designed and executed by developers
Test cases designedand executed by
independent testers
RequirementsAnalysis
ExecuteSystem Tests
Execute Integration
TestsDesign
Validate Requirements
Verify Design
Verify Implementation