![Page 1: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/1.jpg)
2014-2015
Verification and ValidationPart 15 : A Resume
Burkhart WolffDépartement Informatique
Université Paris-Sud / Orsay
![Page 2: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/2.jpg)
26/11/14 B. Wolff - GLA - Resume 2
What is Software Engineering ?
" Methods, techniques and tools for# design: requirement analysis, models,
specifications# development: programmation, integration# validation: prototypes, testing# verification: formal proof of required properties# maintenance: reusability, improvements
" A slightly longer answer:
![Page 3: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/3.jpg)
26/11/14 B. Wolff - GLA - Resume 3
What is Software Engineering ?
" ... slightly longer answer:
The discipline of software engineering was created to address poor quality of software, get projects exceeding time and budget under control, and ensure that software is built systematically, rigorously, measurably, on time, on budget, and within specification. [Wikipedia [en]]
" Or much shorter:
SE addresses the problems of
« Development in the Large » ... so for teams with 100 or 1000 of developers, and budgets of sometimes billions of dollars.
![Page 4: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/4.jpg)
26/11/14 B. Wolff - GLA - Resume 4
What is Software Engineering ?
" ... slightly longer answer:
Reminder: What is it, when I talk about " Or much shorter:
SE addresses the problems of
« Development in the Large » ... so for teams with 100 or 1000 of developers, and budgets of sometimes billions of dollars.
![Page 5: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/5.jpg)
26/11/14 B. Wolff - GLA - Resume 5
What is Software Engineering ?
" ... slightly polemically:
Reminder: What is it, when I talk about Software Engineering ? Writing:
! “Write-once, throw away” programs ?! Programs written by a small team with 50 KLoCs?! Our LRI Website-Service
OR:
! programs in industrial context with large institutions (states, companies) commissioning it ?
![Page 6: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/6.jpg)
26/11/14 B. Wolff - GLA - Resume 6
Answers to the Problem of Software Engineering
" Development Processes
" Standards on Processes, Documentation, and organization
" Process-support# Version management
(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges
![Page 7: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/7.jpg)
26.11.14 B. Wolff - GLA - Resume 7
2013-14 L2-GL - Intro 12
One way to view Software Engineering Project
MANAGEMENT PROCESS
Development Mgt. Risc Management
ConfigurationManagement
« PeopleWare »(Staff, Sub-contractors)
PROCESSUS QUALITE
QUALITYASSURANCE
QUALITYCONTROL
METRICS
FEASIBILITY STUDIES (BEFORE PROJET)
SPECIFICATION & DESIGN
PRODUCTION
INTEGRATION & VALIDATION
EXPLOITATION
TECHNICAL PROCESS
Development
Maintenance & Support
![Page 8: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/8.jpg)
26/11/14 B. Wolff - GLA - Resume 8
How can software be «built systematically»?" A Variant: The VPM3-Model (Daimler)
![Page 9: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/9.jpg)
26/11/14 B. Wolff - GLA - Resume 9
Answers to the Problem of Software Engineering
" Development Processes
" Standards on Processes, Documentation, and organization
" Process-support# Version management
(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges# Verification and Validation Techniques
![Page 10: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/10.jpg)
26.11.14 B. Wolff - GLA - Resume 10
Some Relevant Standards (Recall)
" Many attempts to control development processes andsoftware products by standards (norms)
" Some attempts to assure and certify software quality.# Most serious and relevant (in France):# DO 178B (Avionics)# ISO 29119 (Software Test) # ISO 15408 « Common Criteria» for computer
security certification requiring formal models as well as proof techniques for EAL 6 and EAL 7.
![Page 11: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/11.jpg)
26/11/14 B. Wolff - GLA - Resume 11
Answers to the Problem of Software Engineering
" Development Processes
" Standards on Processes, Documentation, and organization
" Process-support# Version management
(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges# Verification and Validation Techniques
![Page 12: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/12.jpg)
26/11/14 B. Wolff - GLA - Resume 12
Concepts of Central. Version Control(Recall)
" Working copises(in user space)
" Repository(on the server-side)
" update:syncing with therepository
" commit: creating a new revision of a document(involves new registration, inclusion in documents, consistency checks)
" operations lock, checkout, import, ...
![Page 13: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/13.jpg)
26/11/14 B. Wolff - GLA - Resume 13
Build Management: A Build-Server (Recall)
"
![Page 14: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/14.jpg)
26/11/14 B. Wolff - GLA - Resume 14
Build Management: A Build-Server (Recall)
"
![Page 15: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/15.jpg)
26/11/14 B. Wolff - GLA - Resume 15
The Problem of Software Engineering
" In a large process, automatic support formerges of documents and code becomes vital
" This raises the problem of consistency of products:documents and code
" Answers we know of:# . . .
![Page 16: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/16.jpg)
26/11/14 B. Wolff - GLA - Resume 16
Answers to the Problem of SE
" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges
(from type-checking in UML and code,to MISRA code style checkers)
# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification
![Page 17: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/17.jpg)
26/11/14 B. Wolff - GLA - Resume 17
Answers to the Problem of SE
" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges
(from type-checking in UML and code,to MISRA code style checkers)
# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification
![Page 18: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/18.jpg)
26/11/14 B. Wolff - GLA - Resume 18
A first Glance to an Example: Bank
Opening a bank account. Constraints:" there is a blacklist" there is a present of 15 euros in the initial account" account numbers must be distinct.
![Page 19: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/19.jpg)
26/11/14 B. Wolff - GLA - Resume 19
A first Glance to an Example: Bank (2)
" context Compte:inv unique: Compte.allInstances->isUnique(c | c.no)
context Banque::ouvrirCompte(nomC: String) : Integer
pre: Personne.allInstances->forall(p | p.nom <> nomC)post: Personne.allInstances->isUnique(p | p.nom = nomC
and p.oclIsNew() and )
and Compte.allInstances ->isUnique(c | c.titulaire.nom = nomC and c.oclIsNew() and c.solde = 15)and Personne.allInstances ->includesAll(Personne.allInstances@pre)and Compte.allInstances ->includesAll(Compte.allInstances@pre)
![Page 20: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/20.jpg)
26/11/14 B. Wolff - GLA - Resume 20
Answers to the Problem of SE
" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges
(from type-checking in UML and code,to MISRA code style checkers)
# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification
![Page 21: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/21.jpg)
26/11/14 B. Wolff - GLA - Resume 21
Static Analysis (Recall)
" Modern Static Analysis Methodsare typically a combination of the following techniques# Data-Flow, Control-flow, Patterns, (“Classics”)# Symbolic Execution (“forward analysis”)# Deductive Verification (wp like, “backw. ana.”)# Model-Checking# Abstract Interpretation # Predicate Abstraction,# Interpolant Construction
![Page 22: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/22.jpg)
26/11/14 B. Wolff - GLA - Resume 22
Static Analysis (Recall)
" Modern Static Analysis Methodsare typically a combination of the following techniques# Data-Flow, Control-flow, Patterns, (“Classics”)# Symbolic Execution (“forward analysis”)# Deductive Verification (wp like, “backw. ana.”)# Model-Checking# Abstract Interpretation # Predicate Abstraction,# Interpolant Construction
![Page 23: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/23.jpg)
26/11/14 B. Wolff - GLA - Resume 23
Answers to the Problem of SE
" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges
(from type-checking in UML and code,to MISRA code style checkers)
# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification
![Page 24: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/24.jpg)
26/11/14 B. Wolff - GLA - Resume 24
Testing Methods (Recall)
" A taxonomy on types of tests# Static Test / Dynamic (Runtime) Test # Structural Test / Functional Test# Statistic Tests
" Functional Test; Link to UML/OCL# Dynamic Unit Tests, Static Unit Tests,
# Coverage Criteria
" Structural Tests# Control Flow and Data Flow Graphs
# Tests and executed paths. Undecidability.
# Coverage Criteria
![Page 25: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/25.jpg)
26/11/14 B. Wolff - GLA - Resume 25
Testing Methods (Recall)
" It is an approximation to full verification (for ex. by proof)" Main emphasis: finding bugs early,
# either in the model# or in the program# or in both
" A systematic test is:# process programs and specifications
and to compute a set of test-cases under controlled conditions.
# ideally: testing is complete if a certain criteria,the adequacy criteria is reached.
![Page 26: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/26.jpg)
26/11/14 B. Wolff - GLA - Resume 26
Testing Methods (Recall)
" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges
(from type-checking in UML and code,to MISRA code style checkers)
# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification
![Page 27: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/27.jpg)
26/11/14 B. Wolff - Ingé. 2 - Proof-Based Verification II 27
Hoare – Logic: A Proof System for Programs
" Hoare revisited (i):
" ... this part is actually highly deterministic
![Page 28: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/28.jpg)
26/11/14 B. Wolff - Ingé. 2 - Proof-Based Verification II 28
Hoare – Logic: A Proof System for Programs
" Hoare revisited (ii):
" ... this part needs some work, and some new ideas.
![Page 29: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/29.jpg)
26/11/14 B. Wolff - Ingé. 2 - Proof-Based Verification II 29
Tools: gwhy and Squareroot
![Page 30: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/30.jpg)
26/11/14 B. Wolff - GLA - Resume 30
Resume : Test vs. Proof
Test
# Requires Testability of Programs (initializable,reproducible behaviour, sufficient control over non-determinism)
# Can be also Work-Intensive !!!
# Requires Test-Tools, whose details can be very complicated
# Requires a Formal Specification
# Makes Test-Hypothesis, which may be hard to justify !
![Page 31: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/31.jpg)
26/11/14 B. Wolff - GLA - Resume 31
Resume : Test vs. Proof
Deductive Verification
# Needs knowledge / assumptions on the precise execution of the language and the the environment(may be difficult in practice ... gcc -o3 !!!)
# Is work-intensive and intellectually expensive !!!
# Requires Tools, whose details can be complicated
# Requires a Formal Specification
# Currently restricted to abstract models or realistic C with code-size 10kloc.)
![Page 32: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/32.jpg)
26/11/14 B. Wolff - GLA - Resume 32
An Example for an Extended Static Checking (Combi-) Method: SAL Annotations
# Due to heavy machinery(heuristics, patterns, abstract interpretation ...)
the annotation of loops with invariantsis not necessary by the user for the SALlanguage.
The programmer has just to provide contracts.
Since 2006, Microsoft annotates the entireWindows and Word code-base with SAL.
![Page 33: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/33.jpg)
26/11/14 B. Wolff - GLA - Resume 33
An Example for an Extended Static (Combi-) Checking Method: SAGE
# ... a white box fuzz - tester.
![Page 34: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/34.jpg)
26/11/14 B. Wolff - GLA - Resume 34
Static Analysis : (recall)
" Lots of Methods with the same Goal:
More automation, better prediction of “real errors”
(i.e. less false-positives)better explanation of “real errors”
(very difficult)less constraints on the supported language subset
(“real” C, “real” Java) trend to “dirty languages”such as JavaScript
![Page 35: Verification and Validationwolff/teach-material/2017-18/L3-GLA/part-XV-Resu… · 26.11.14 B. Wolff - GLA - Resume 10 Some Relevant Standards (Recall) " Many attempts to control development](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f079ad57e708231d41dd11a/html5/thumbnails/35.jpg)
26/11/14 B. Wolff - GLA - Resume 35
Global Summary
" Current Research and Trends:
# Combination of Methods
# Test and Proof have a lot in common
# A lot of theory needs to combined with heavy technology