![Page 1: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/1.jpg)
Barbara Russo SwSE - Software and Systems Engineering research group
�1
Verification and Validation
![Page 2: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/2.jpg)
�2
• Software products is imperfect as it is created by human beings
• Verification and Validation techniques are methods to ensure the final product quality
Verification and Validation
![Page 3: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/3.jpg)
�3
• Check the consistency of an implementation with a specification
• It is about “How” i.e., the process of building • Are we building the product right?" (B.
Boehm)
• Example: A music player plays (it does play) the music when I press Play
Verification
![Page 4: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/4.jpg)
�4
• Check consistency between two descriptions (roles) of the system at different stages of the development process; • UML class diagram and its code implementation • Specification document and UML class diagram
• Chain of Two Roles: • Specification ⇒ Implementation (Specification) ⇒ Implementation …
Verification
![Page 5: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/5.jpg)
�5
• Check the degree at which a software system fulfills user’s requirements
• It is about “What” - the product itself • Are we building the right product ? (B.
Boehm)
• Example: A music player plays a song (it does not show a video) when I press Play
Validation
![Page 6: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/6.jpg)
�6
• Requirements are goals of a software system • Specifications are solutions to achieve
requirements • Software that matches requirements ⇒
useful software • Software that matches specifications ⇒
dependable software
Usefulness vs. dependability
![Page 7: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/7.jpg)
�7
• Degree at which a software system complies with its specifications
• Specifications are prone to defects as they have been written by human beings, but • Verification does not discuss the correctness of
the specifications
Dependability
![Page 8: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/8.jpg)
�8
.
![Page 9: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/9.jpg)
�9
• Software analysis and software reviews are verification processes to examine a software artefact and to approve it
• Software review is a systematic inspection of software to find and resolve defects • Performed manually • Documents like requirements, system designs,
codes, test plans and test cases
Software reviews
![Page 10: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/10.jpg)
�10
• Validation involves stakeholders' judgment
• Exercise: Discuss a validation technique
Verification vs. Validation
![Page 11: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/11.jpg)
�11
• Acceptance testing: customers verify and validate user stories (requirements)
• alpha testing: performed by users in a controlled environment. Capture operational profiles decided by the organisation
• beta testing: performed by users in a their own environment. Capture real operational profiles
Examples of validation techniques
![Page 12: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/12.jpg)
�12
• Verification manly focuses on dependability and concerns four software properties: • Correctness: consistency with specification • Reliability: statistical approximation to correctness;
probability that a system deviates from the expected behaviour
Verification
![Page 13: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/13.jpg)
�13
• Robustness: being able to maintain operations under exceptional circumstances of not full-functionality
• Safety: robustness in case of hazardous behaviour (e.g., attacks)
Verification
![Page 14: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/14.jpg)
�14
• Consistency: Specification vs specification, no conflicts
• No ambiguity: open to interpretations, uncertainty
• Adherence to standards: consistency with benchmarks
Specification Self-consistency
![Page 15: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/15.jpg)
�15
• How can we check whether our software satisfies any of the dependability properties?
• For example, correctness: given a set of specifications and a program we want to find some logical procedure (e.g., a proof) to say that the program satisfies the specifications
Checking dependability
![Page 16: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/16.jpg)
�16
Some problems cannot be solved by any computer program (Alan Turing)
Undecidability of problems
![Page 17: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/17.jpg)
�17
Given a program P and an input I, it is not decidable whether P will eventually halt when it runs with that input I or it runs forever
The halting problem
![Page 18: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/18.jpg)
�18
• Rephrasing: there is no infallible algorithm that can check a property of interest • For example, correctness
The halting problem
![Page 19: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/19.jpg)
�19
• Undecidability implies that given a program P and a verification technique T, we do not know whether the technique can verify the program in finite time
• ... and even when checking is feasible it might be very expensive
Checking a program
![Page 20: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/20.jpg)
�20
• Thus, verification techniques are inaccurate when checking dependability properties
• We need to speak of optimistic and pessimistic inaccuracy of a verification technique
Inaccuracy of verification techniques
![Page 21: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/21.jpg)
�21
• Technique that verifies a property S can return TRUE on programs that does not have the property (FALSE POSITIVE)
Optimistic Inaccuracy
![Page 22: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/22.jpg)
�22
• Testing is an optimistic technique for correctness
• It returns that a program is correct even if no finite number of tests can guarantee correctness
Example
![Page 23: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/23.jpg)
�23
• Technique that verifies a property S can return FALSE on programs that have the property (FALSE NEGATIVE)
• Also called conservative technique
Pessimistic Inaccuracy
![Page 24: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/24.jpg)
�24
• Automatic testing is pessimistic for correctness as it typically uses rules (not heuristics!) • Software that does not adhere to such rules is
not correct.
Example
![Page 25: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/25.jpg)
�25
Accuracy: confusion matrix
Pred. TRUE
Pred. FALSE
TRUE TP FN
FALSE FP TN
Predictedbythetechnique
Truth
![Page 26: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/26.jpg)
�26
• In complex system, a direct verification can be infeasible
• Often this happens when properties are related to specific human judgements, but not only
Substituting principle
![Page 27: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/27.jpg)
�27
• Substituting a property with one that can be easier verified • Constraining the class of programs to verify • Separate human judgment from objective
verification
Substituting principle
![Page 28: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/28.jpg)
�28
• “Race condition": interference between writing data in one process and reading or writing related data in another process (e.g., an array accessed by different threads)
• To avoid race conditions: testing the integrity of shared data • It is difficult as it is checked at run time • Substitution principle: adhere to a protocol of
serialisation
Example - correctness
![Page 29: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/29.jpg)
�29
• When group of objects or states can be transmitted as one entity and then at arrival reconstructed into the original distinct objects
Serialisation
![Page 30: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/30.jpg)
�30
• An object can be represented as a sequence of bytes that includes the object's data as well as information about the object's type and its types of data
• After a serialised object has been written into a file, it can be read from the file and deserialised: the type information and bytes that represent the object and its data can be used to recreate the object in memory
Java object serialisation
![Page 31: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/31.jpg)
�31
• The ObjectOutputStream class contains the method public final void writeObject(Object x) throws IOException
• The method serialises an Object and sends it to the output stream
Java object serialisation
![Page 32: Verification and Validation - unibzrusso/SRT/Testing/02.ValidationVerification.pdf · Verification and Validation. 2 • Software products is imperfect as it is created by human beings](https://reader035.vdocuments.us/reader035/viewer/2022062414/5f023ba77e708231d4033cbf/html5/thumbnails/32.jpg)
�32
• Similarly, the ObjectInputStream class contains the method for deserialising an object: public final Object readObject() throws IOException, ClassNotFoundException
• This method retrieves the next Object out of the stream and deserialises it
Java object serialisation