ch. 61 verification. ch. 62 outline what are the goals of verification? what are the main approaches...
TRANSCRIPT
![Page 1: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/1.jpg)
Ch. 6 1
Verification
![Page 2: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/2.jpg)
Ch. 6 2
Outline• What are the goals of verification?• What are the main approaches to
verification?– What kind of assurance do we get
through testing?– How can testing be done systematically?– How can we remove defects
(debugging)?• What are the main approaches to
software analysis?– informal vs. formal
![Page 3: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/3.jpg)
Ch. 6 3
Need for verification
• Designers are fallible even if they are skilled and follow sound principles
• Everything must be verified, every required quality, process and products– even verification itself…
![Page 4: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/4.jpg)
Ch. 6 4
Properties of verification
• May not be binary (OK, not OK)– severity of defect is important– some defects may be tolerated
• May be subjective or objective– e.g., usability
• Even implicit qualities should be verified– because requirements are often incomplete– e.g., robustness
![Page 5: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/5.jpg)
Ch. 6 5
Approaches to verification
• Experiment with behavior of product– sample behaviors via testing– goal is to find "counterexamples"– dynamic technique
• Analyze product to deduce its adequacy– analytic study of properties– static technique
![Page 6: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/6.jpg)
Ch. 6 6
Testing and lack of "continuity"
• Testing samples behaviors by examining "test cases"
• Impossible to extrapolate behavior of software from a finite set of test cases
• No continuity of behavior– it can exhibit correct behavior in
infinitely many cases, but may still be incorrect in some cases
![Page 7: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/7.jpg)
Ch. 6 7
Verification in engineering
• Example of bridge design• One test assures infinite correct
situations
![Page 8: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/8.jpg)
Ch. 6 8
procedure binary-search (key: in element; table: in elementTable; found: out Boolean) is
beginbottom := table'first; top := table'last; while bottom < top loop
if (bottom + top) rem 2 ≠ 0 then middle := (bottom + top - 1) / 2;
else middle := (bottom + top) / 2;
end if;if key ≤ table (middle) then
top := middle;else
bottom := middle + 1;end if;
end loop;found := key = table (top);
end binary-search
if we omit thisthe routineworks if the elseis never hit!(i.e. if size of table is a power of 2)
![Page 9: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/9.jpg)
Ch. 6 9
Goals of testing
• To show the presence of bugs (Dijkstra, 1987)
• If tests do detect failures, we cannot conclude that software is defect-free
• Still, we need to do testing– driven by sound and systematic
principles
![Page 10: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/10.jpg)
Ch. 6 10
Goals of testing (cont.)
• Should help isolate errors– to facilitate debugging
• Should be repeatable– repeating the same experiment, we
should get the same results• this may not be true because of the effect
of execution environment on testing• because of nondeterminism
• Should be accurate
![Page 11: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/11.jpg)
Ch. 6 11
Theoretical foundations of testing
![Page 12: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/12.jpg)
Ch. 6 12
Definitions (1)
• P (program), D (input domain), R (output domain)– P: D R (may be partial)
• Correctness defined by OR D R– P(d) correct if <d, P(d)> OR– P correct if all P(d) are correct
![Page 13: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/13.jpg)
Ch. 6 13
Definitions (2)
• FAILURE– P(d) is not correct
• may be undefined (error state) or may be the wrong result
• ERROR (DEFECT)– anything that may cause a failure
• typing mistake• programmer forgot to test “x = 0”
• FAULT– incorrect intermediate state entered by
program
![Page 14: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/14.jpg)
Ch. 6 14
Definitions (3)
• Test case t – an element of D
• Test set T– a finite subset of D
• Test is successful if P(t) is correct• Test set successful if P correct for
all t in T
![Page 15: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/15.jpg)
Ch. 6 15
Definitions (4)
• Ideal test set T– if P is incorrect, there is an element of
T such that P(d) is incorrect
• if an ideal test set exists for any program, we could prove program correctness by testing
![Page 16: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/16.jpg)
Ch. 6 16
Test criterion
• A criterion C defines finite subsets of D (test sets)– C 2D
• A test set T satisfies C if it is an element of CExampleC = {<x1, x2,..., xn> | n 3 i, j, k, ( xi<0 xj=0
xk>0)}<-5, 0, 22> is a test set that satisfies C<-10, 2, 8, 33, 0, -19> also does<1, 3, 99> does not
![Page 17: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/17.jpg)
Ch. 6 17
Properties of criteria (1)• C is consistent
– for any pairs T1, T2 satisfying C, T1 is successful iff T2 is successful• so either of them provides the “same”
information
• C is complete– if P is incorrect, there is a test set T of
C that is not successful• C is complete and consistent
– identifies an ideal test set– allows correctness to be proved!
![Page 18: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/18.jpg)
Ch. 6 18
Properties of criteria (2)
• C1 is finer than C2– for any program P
• for any T1 satisfying C1 there is a subset T2 of T1 which satisfies C2
![Page 19: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/19.jpg)
Ch. 6 19
Properties of definitions
• None is effective, i.e., no algorithms exist to state if a program, test set, or criterion has that property
• In particular, there is no algorithm to derive a test set that would prove program correctness– there is no constructive criterion that
is consistent and complete
![Page 20: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/20.jpg)
Ch. 6 20
Empirical testing principles
• Attempted compromise between the impossible and the inadequate
• Find strategy to select significant test cases– significant=has high potential of
uncovering presence of error
![Page 21: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/21.jpg)
Ch. 6 21
Complete-Coverage Principle
• Try to group elements of D into subdomains D1, D2, …, Dn where any element of each Di is likely to have similar behavior– D = D1 D2 … Dn
• Select one test as a representative of the subdomain
• If Dj Dk for all j, k (partition), any element can be chosen from each subdomain
• Otherwise choose representatives to minimize number of tests, yet fulfilling the principle
![Page 22: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/22.jpg)
Ch. 6 22
Complete-Coverage Principle
example of a partition
![Page 23: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/23.jpg)
Ch. 6 23
Testing in the smallWe test individual modules• BLACK BOX (functional) testing
– partitioning criteria based on the module’s specification
– tests what the program is supposed to do
• WHITE BOX (structural) testing– partitioning criteria based on module’s
internal code– tests what the program does
![Page 24: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/24.jpg)
Ch. 6 24
White box testing
derives test cases from program code
![Page 25: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/25.jpg)
Ch. 6 25
Structural Coverage Testing
• (In)adequacy criteria – If significant parts of program
structure are not tested, testing is inadequate
• Control flow coverage criteria– Statement coverage– Edge coverage– Condition coverage– Path coverage
![Page 26: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/26.jpg)
Ch. 6 26
Statement-coverage criterion
• Select a test set T such that every elementary statement in P is executed at least once by some d in T– an input datum executes many
statements try to minimize the number of test cases still preserving the desired coverage
![Page 27: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/27.jpg)
Ch. 6 27
Exampleread (x); read (y);if x > 0 then
write ("1");else
write ("2");end if;if y > 0 then
write ("3");else
write ("4");end if;
{<x = 2, y = 3>, <x = - 13, y = 51>, <x = 97, y = 17>, <x = - 1, y = - 1>}covers all statements
{<x = - 13, y = 51>, <x = 2, y = - 3>} is minimal
![Page 28: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/28.jpg)
Ch. 6 28
Weakness of the criterion
if x < 0 then x := -x;
end if;z := x;
{<x=-3} covers allstatements
it does not exercise the case when x is positiveand the then branch isnot entered
![Page 29: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/29.jpg)
Ch. 6 29
Edge-coverage criterion
• Select a test set T such that every edge (branch) of the control flow is exercised at least once by some d in Tthis requires formalizing the concept of the control graph, and how to construct it– edges represent statements– nodes at the ends of an edge represent entry into
the statement and exit
![Page 30: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/30.jpg)
Ch. 6 30
G G1 2 G1
G1
G1
G 2
I/O, assignment, or procedure call
if-then-else if-then
while loop
two sequential statements
Control graph construction rules
![Page 31: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/31.jpg)
Ch. 6 31
Simplification
a sequence of edges can be collapsed into just one edge
. . .n n nnn k-1 k1 2 3
n1n
k
![Page 32: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/32.jpg)
Ch. 6 32
beginread (x); read (y);while x ≠ y loop
if x > y then x := x - y;
else y := y - x;
end if;end loop;gcd : = x;
end;
Exemple: Euclid's algorithm
![Page 33: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/33.jpg)
Ch. 6 33
Weaknessfound := false; counter := 1;while (not found) and counter < number_of_items loop
if table (counter) = desired_element then found := true;
end if;counter := counter + 1;
end loop;if found then
write ("the desired element is in the table");else
write ("the desired element is not in the table");end if;
test cases: (1) empty table, (2) table with 3 items, second ofwhich is the item to look fordo not discover error (< instead of ≤ )
![Page 34: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/34.jpg)
Ch. 6 34
Condition-coverage criterion
• Select a test set T such that every edge of P’s control flow is traversed and all possible values of the constituents of compound conditions are exercised at least once– it is finer than edge coverage
![Page 35: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/35.jpg)
Ch. 6 35
Weakness
{<x = 0, z = 1>, <x = 1, z = 3>} causes the execution of all edges, but fails to expose the risk of a division by zero
if x ≠ 0 then
y := 5; else
z := z - x; end if;if z > 1 then
z := z / x; else
z := 0; end if;
![Page 36: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/36.jpg)
Ch. 6 36
Path-coverage criterion
• Select a test set T which traverses all paths from the initial to the final node of P’s control flow– it is finer than previous kinds of
coverage– however, number of paths may be too
large, or even infinite (see while loops)• additional constraints must be provided
![Page 37: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/37.jpg)
Ch. 6 37
The infeasibility problem• Syntactically indicated behaviors
(statements, edges, etc.) are often impossible– unreachable code, infeasible edges, paths,
etc.
• Adequacy criteria may be impossible to satisfy – manual justification for omitting each
impossible test case– adequacy “scores” based on coverage
• example: 95% statement coverage
![Page 38: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/38.jpg)
Ch. 6 38
Further problem
• What if the code omits the implementation of some part of the specification?
• White box test cases derived from the code will ignore that part of the specification!
![Page 39: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/39.jpg)
Ch. 6 39
Black box testing
derives test cases from specifications
![Page 40: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/40.jpg)
Ch. 6 40
The specification
The program receives as input a record describing an invoice. (A detailed description of the format of the record is given.) The invoice must be inserted into a file of invoices that is sorted by date. The invoice must be inserted in the appropriate position: If other invoices exist in the file with the same date, then the invoice should be inserted after the last one. Also, some consistency checks must be performed: The program should verify whether the customer is already in a corresponding file of customers, whether the customer’s data in the two files match, etc.
![Page 41: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/41.jpg)
Ch. 6 41
Did you consider these cases?
• An invoice whose date is the current date• An invoice whose date is before the current date
(This might be even forbidden by law)This case, in turn, can be split into the two following subcases: • An invoice whose date is the same as that
some existing invoice • An invoice whose date does not exist in any
previously recorded invoice• Several incorrect invoices, checking different types of
inconsistencies
![Page 42: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/42.jpg)
Ch. 6 42
Systematic black-box techniques
• Testing driven by logic specifications (pre and postconditions)
• Syntax-driven testing• Decision table based testing• Cause-effect graph based testing
![Page 43: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/43.jpg)
Ch. 6 43
Logic specification of insertion of invoice record in a file
for all x in Invoices, f in Invoice_Files{sorted_by_date(f) and not exist j, k (j ≠ k and f(j) =f(k)}
insert(x, f)
{sorted_by_date(f) and for all k (old_f(k) = z implies exists j (f(j) = z)) and for all k (f(k) = z and z ≠ x) implies exists j (old_f(j) = z) andexists j (f(j). date = x. date and f(j) ≠ x) implies j < pos(x, f) andresult x.customer belongs_to customer_file andwarning (x belongs_to old_f or x.date < current_date or ....)}
![Page 44: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/44.jpg)
Ch. 6 44
TRUE implies sorted_by_date(f) and for all k old_f(k) = z implies exists j (f(j) = z) and for all k (f(k) = z and z ≠ x) implies exists j (old_f(j) = z)
and(x.customer belongs_to customer_file) implies resultand not (x.customer belongs_to customer_file and ...)
implies not resultandx belongs_to old_y implies warningandx.date < current_date implies warningand....
Apply coverage criterion to postcondition…Rewrite in a more convenient way…
![Page 45: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/45.jpg)
Ch. 6 45
Syntax-driven testing (1)
• Consider testing an interpreter of the following language
<expression> ::= <expression> + <term>|<expression> - <term> | <term>
<term> ::= <term> * <factor> | <term> / <factor> | <factor>
<factor> ::= ident | ( <expression>)
![Page 46: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/46.jpg)
Ch. 6 46
Syntax-driven testing (2)
• Apply complete coverage principle to all grammar rules
• Generate a test case for each rule of the grammar– note, however that the test case
might also cover other rules• Note: the specification is formal,
and test generation can be automated
![Page 47: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/47.jpg)
Ch. 6 47
Decision-table-based testing
“The word-processor may present portions of text in three different formats: plain text (p), boldface (b), italics (i). The following commands may be applied to each portion of text: make text plain (P), make boldface (B), make italics (I), emphasize (E), super emphasize (SE). Commands are available to dynamically set E to mean either B or I (we denote such commands as E=B and E=I, respectively.) Similarly, SE can be dynamically set to mean either B (command SE=B) or I (command SE=I), or B and I (command SE=B+I.)”
![Page 48: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/48.jpg)
Ch. 6 48
P
B
I
E
SE
E = B
E = I
SE = I
SE = B
SE = B + I
p b i b i b i b,i b,iaction
*
*
*
* *
*
*
* * *
*
*
*
*
*
![Page 49: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/49.jpg)
Ch. 6 49
Cause effect graphsB I P E E = B SE E = I SE = B SE = I SE = B + I
b
i
p
A N D
O R
A N DO
R
The AND/OR graph represents the correspondence betweencauses and effects
![Page 50: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/50.jpg)
Ch. 6 50
Further constraints“Both B and I exclude P (i.e., one cannot ask both forplain text and, say, italics for the same portion of text.) E and SE are mutually exclusive.”
a
b
c
e
a
b
c
a
b
c
i
o
a
b
a
b
r m
at mostone
at leastone
one and onlyone
requires masks
![Page 51: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/51.jpg)
Ch. 6 51
B I P E E = B SE E = I SE = B SE = I SE = B + I
m
m b
i
p
m m
A N D
O R
A N DO
R
X m Y = X implies not Y
![Page 52: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/52.jpg)
Ch. 6 52
Coverage criterion
• Generate all possible input combinations and check outputs
• May reduce the number by going backwards from outputs– OR node with true output:
• use input combinations with only one true input
– AND node with false output: • use input combinations with only one false
input
![Page 53: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/53.jpg)
Ch. 6 53
Testing boundary conditions
• Testing criteria partition input domain in classes, assuming that behavior is "similar" for all data within a class
• Some typical programming errors, however, just happen to be at the boundary between different classes
![Page 54: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/54.jpg)
Ch. 6 54
Criterion
• After partitioning the input domain D into several classes, test the program using input values not only “inside” the classes, but also at their boundaries
• This applies to both white-box and black-box techniques
![Page 55: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/55.jpg)
Ch. 6 55
The oracle problemHow to inspect the results of test
executions to reveal failures• Oracles are required at each stage
of testing• Automated test oracles are
required for running large amounts of tests
• Oracles are difficult to design - no universal recipe
![Page 56: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/56.jpg)
Ch. 6 56
Testing in the large
• Module testing– testing a single module
• Integration testing– integration of modules and subsystems
• System testing– testing the entire system
• Acceptance testing– performed by the customer
![Page 57: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/57.jpg)
Ch. 6 57
Module testing
• Scaffolding needed to create the environment in which the module should be tested– stubs
• modules used by the module under test
– driver• module activating the module under test
![Page 58: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/58.jpg)
Ch. 6 58
Testing a functional module
PROCEDURE UNDER TEST DRIVERSTUB
CALL CALL
ACCESS TO NONLOCAL VARIABLES
![Page 59: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/59.jpg)
Ch. 6 59
Integration testing
• Big-bang approach– first test individual modules in isolation– then test integrated system
• Incremental approach– modules are progressively integrated
and tested• can proceed both top-down and bottom-up
according to the USES relation
![Page 60: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/60.jpg)
Ch. 6 60
Integration testing and USES relation
A
B C
D E
If integration and testproceed bottom-uponly need drivers
Otherwise, if we proceedtop-down only stubs areneeded
![Page 61: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/61.jpg)
Ch. 6 61
M1 M2
2,1 2,2M M
Example
M1 USES M2 and M2 IS_COMPOSED_OF {M2,1, M2,2}
CASE 1Test M1, providing a stub for M2 and a driver for M1 Then provide an implementation for M2,1 and a stub for M2,2
CASE 2Implement M2,2 and test it by using a driver, Implement M2,1 and test the combination of M2,1 and M2,2 (i.e., M2) by using a driverFinally, implement M1 and test it with M2, using a driver for M1
![Page 62: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/62.jpg)
Ch. 6 62
Testing OO programs
• New issues– inheritance– genericity– polymorphism– dynamic binding
• Open problems still exist
![Page 63: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/63.jpg)
Ch. 6 63
Inheritance
Personnel
Consultant Employee
Manager Administartive_Staff Technical_Staff
![Page 64: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/64.jpg)
Ch. 6 64
How to test classes of the hierarchy?
• “Flattening” the whole hierarchy and considering every class as a totally independent component – does not exploit incrementality
• Finding an ad-hoc way to take advantage of the hierarchy
![Page 65: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/65.jpg)
Ch. 6 65
A sample strategy
• A test that does not have to be repeated for any heir
• A test that must be performed for heir class X and all of its further heirs
• A test that must be redone by applying the same input data, but verifying that the output is not (or is) changed
• A test that must be modified by adding other input parameters and verifying that the output changes accordingly
![Page 66: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/66.jpg)
Ch. 6 66
Separate concerns in testing
• Testing for functionality is not enough• Overload testing• Robustness testing• Regression testing
– organize testing with the purpose of verifying possible regressions of software during its life—that is, degradations of correctness or other qualities due to later modifications
![Page 67: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/67.jpg)
Ch. 6 67
Testing concurrent and real-time systems
• Nondeterminism inherent in concurrency affects repeatability
• For real-time systems, a test case consists not only of input data, but also of the times when such data are supplied
![Page 68: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/68.jpg)
Ch. 6 68
Analysis
![Page 69: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/69.jpg)
Ch. 6 69
Analysis vs. testing
• Testing characterizes a single execution
• Analysis characterizes a class of executions; it is based on a model
• They have complementary advantages and disadvantages
![Page 70: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/70.jpg)
Ch. 6 70
Informal analysis techniques
Code walkthroughs• Recommended prescriptions
– Small number of people (three to five)– Participants receive written documentation from the
designer a few days before the meeting– Predefined duration of meeting (a few hours)– Focus on the discovery of errors, not on fixing them– Participants: designer, moderator, and a secretary– Foster cooperation; no evaluation of people
• Experience shows that most errors are discovered by the designer during the presentation, while trying to explain the design to other people.
![Page 71: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/71.jpg)
Ch. 6 71
Informal analysis techniques
Code inspection• A reading technique aiming at
error discovery• Based on checklists; e.g.:
– use of uninitialized variables; – jumps into loops; – nonterminating loops; – array indexes out of bounds; – …
![Page 72: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/72.jpg)
Ch. 6 72
Correctness proofs
![Page 73: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/73.jpg)
Ch. 6 73
A program and its specification (Hoare
notation)
{true}begin
read (a); read (b);x := a + b;write (x);
end{output = input1 + input2}
proof by backwards substitution
![Page 74: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/74.jpg)
Ch. 6 74
Proof rules
Claim1, Claim2
Claim3
Notation:If Claim 1 and Claim 2 have been proven, one can deduce Claim3
![Page 75: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/75.jpg)
Ch. 6 75
Proof rules for a language
{F1}S1{F2}, {F2}S2{F3}
{F1}S1;S2{F3}
sequence
{Pre and cond} S1 {Post},{Pre and not cond} S2 {Post}{Pre} if cond then S1 ; else S 2 ; end if; {Post}
if-then-else
while-do{I and cond} S {I}{I} while cond loop S; end loop; {I and not cond}
I loop invariant
![Page 76: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/76.jpg)
Ch. 6 76
Correctness proof
• Partial correctness– validity of {Pre} Program {Post}
guarantees that if the Pre holds before the execution of Program, and if the program ever terminates, then Post will be achieved
• Total correctness– Pre guarantees Program’s termination
and the truth of Post
These problems are undecidable!!!
![Page 77: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/77.jpg)
Ch. 6 77
Example{input1 > 0 and input2 > 0}begin
read (x); read (y);div := 0;while x =y loop
div := div + 1;x := x - y;
end loop;write (div); write (x);
end;{input1 = output1 * input2 + output2 and 0 =output2 < input2 }
![Page 78: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/78.jpg)
Ch. 6 78
Invention of loop invariant
• Difficult and creative step• Cannot be constructed automatically• In the example
input1 = div * y + x and x =0 and y = input2
![Page 79: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/79.jpg)
Ch. 6 79
Programs with arrays
{Pre} a(i) := expression; {Post}
Pre denotes the assertion obtained from Post by substituting every occurrence of an indexedvariable a(j) by the term
if j = i then expression else a(j);
![Page 80: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/80.jpg)
Ch. 6 80
Example{n =1}i := 1; j := 1;found := false;while i =n loop
if table (i) = x thenfound := true;i := i + 1
elsetable (j) := table (i);i := i + 1; j := j + 1;
end if;end loop;n := j - 1;{not exists m (1 =m =n and table (m) = x) andfound =exists m (1 =m =old_n and old_table (m) = x)}
old_table, old_n constants denoting the values of table and of nbefore execution of the program fragment
![Page 81: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/81.jpg)
Ch. 6 81
Correctness proof
• Can be done by using the following loop invariant{(j =i) and (i =old_n + 1) and (not exists m (1 =m < j and table (m) = x)) and (n = old_n) andfound = exists m (1 =m < i and
old_table(m) = x)}
![Page 82: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/82.jpg)
Ch. 6 82
Correctness proofs in the large
• We can prove correctness of operations (e.g., operations on an abstract data type)
• Then use the result of the proof in proving fragments that operate on objects of the ADT
![Page 83: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/83.jpg)
Ch. 6 83
Examplemodule TABLE;exports
type Table_Type (max_size: NATURAL): ?;no more than max_size entries may be stored in a table; user modules must guarantee thisprocedure Insert (Table: in out TableType ;
ELEMENT: in ElementType);procedure Delete (Table: in out TableType;
ELEMENT: in ElementType);function Size (Table: in Table_Type) return NATURAL;provides the current size of a table…
end TABLE
![Page 84: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/84.jpg)
Ch. 6 84
{true}Delete (Table, Element);{Element Table};
{Size (Table) < max_size}Insert (Table, Element){Element Table};
Having proved these
We can then prove properties of programs using tablesFor example, that after executing the sequence
Insert(T, x);Delete(T, x);
x is not present in T
![Page 85: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/85.jpg)
Ch. 6 85
An assessment ofcorrectness proofs
• Still not used in practice• However
– may be used for very critical portions– assertions may be the basis for a
systematic way of inserting runtime checks– proofs may become more practical as
more powerful support tools are developed– knowledge of correctness theory helps
programmers being rigorous
![Page 86: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/86.jpg)
Ch. 6 86
Symbolic execution
• Can be viewed as a middle way between testing and analysis
• Executes the program on symbolic values
• One symbolic execution corresponds to many actual executions
![Page 87: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/87.jpg)
Ch. 6 87
Example(1)
Consider executing the following fragmentwith x=X, y=Y, a=A
x := y + 2;if x > a then
a := a + 2;else
y := x + 3;end if;x := x + a + y;
1
2 3
4
x := y + 2
a := a + 2y := x + 3
x := x + a + y
x > a
![Page 88: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/88.jpg)
Ch. 6 88
Example(2)
• When control reaches the conditional, symbolic values do not allow execution to select a branch
• One can choose a branch, and record the choice in a path condition
• Result:<{a = A, y = Y + 5, x = 2 * Y + A + 7}, <1, 3, 4>, Y + 2 ≤ A> execution
pathpath condition
![Page 89: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/89.jpg)
Ch. 6 89
Symbolic execution rules (1)
• read (x) – removes any existing binding for x and
adds binding x = X, where X is a newly introduced symbolic value
• Write (expression)– output(n) = computed_symbolic_value
(n counter initialized to 1 and automatically incremented after each output statement)
symbolic state: <symbolic_variable_values, execution_path, path_condition>
![Page 90: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/90.jpg)
Ch. 6 90
Symbolic execution rules (2)
• x:= expression – construct symbolic value of expression,
SV; replace previous binding for x with x = SV
• After execution of the last statement of a sequence that corresponds to an edge of control graph, append the edge to execution path
![Page 91: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/91.jpg)
Ch. 6 91
Symbolic execution rules (3)• if cond then S1; else S2; endif
• while cond loop…endloop– condition is symbolically evaluated
• eval (cond)
– if eval (cond) true or false then execution proceeds by following the appropriate branch
– otherwise, make nondeterministic choice of true or false, and conjoin eval (cond) (resp., not eval (cond)) to the path condition
![Page 92: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/92.jpg)
Ch. 6 92
Programs with arrays
• Let A1 be the symbolic value of array a when statement a(i)= exp is executed
• Then, after execution of the statement, a receives the new symbolic value A2, denoted as A2 = A1<i, exp>, a shorthand for– for all k if k = i then A2(k) = exp
else A2(k) = A1(k)
![Page 93: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/93.jpg)
Ch. 6 93
Symbolic execution of concurrent programs
message_reception
ok_parity
ok_paritynot ok_parity
channel channel channel
send_nack send_ack send_ack
m := f()
Predicate:
Predicate:
Predicate:
1
2
2
3
3
![Page 94: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/94.jpg)
Ch. 6 94
Assumptions
• Simplifying assumption: no more than one token in a place
• A sequence of atomic steps can be modeled by a firing sequence– this resolves the nondeterminism that is due
to several transitions being enabledThe triple <symbolic_variable_values,
execution_path, path_condition> can be used to model the symbolic state of the interpreter (execution_path is the firing sequence)
![Page 95: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/95.jpg)
Ch. 6 95
Symbolic execution and testing
• The path condition describes the data that traverse a certain path
• Use in testing:– select path– symbolically execute it– synthesize data that satisfy the path
condition• they will execute that path
![Page 96: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/96.jpg)
Ch. 6 96
Example (1)found := false; counter := 1;while (not found) and counter < number_of_items loop
if table (counter) = desired_element then found := true;
end if;counter := counter + 1;
end loop;if found then
write ("the desired element exists in the table");else
write ("the desired element does not exist in the table");
end if;
![Page 97: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/97.jpg)
Ch. 6 97
Example (2)
1
2
34
6
5
7
8 9
write "the desired element exists in the table"write "the desired element does not exist in the table"
found := true
counter := counter + 1
![Page 98: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/98.jpg)
Ch. 6 98
Model checking
• Correctness verification, in general, is an undecidable problem
• Model checking is a rather recent verification technique based on the fact that most interesting system properties become decidable (i.e., algorithmically verifiable) when the system is modeled as a finite state machine
![Page 99: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/99.jpg)
Ch. 6 99
Principles
• Describe a given system—software or otherwise—as an FSM
• Express a given property of interest as a suitable formula
• Verify whether the system’s behavior does indeed satisfy the desired property– this step can be performed automatically– the model checker either provides a proof that
the property holds or gives a counterexample in the form of a test case that exposes the system’s failure to behave according to the property
![Page 100: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/100.jpg)
Ch. 6 100
P1,P2
P4, P2P1, P5
P3
P3P3
P6, P2 P1, P7P4, P5P3
P6, P5 P7, P4
t1 t2
t3 t2 t1
t2 t3
t5t6
t5t6
t4t1
t4
FSM representing markings of a PN
![Page 101: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/101.jpg)
Ch. 6 101
P
P
P
t
tP
t
1
3
1
3
4
6
5
P
P
P
2
5
7
t
t
t
2
4
6
The original PN
![Page 102: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/102.jpg)
Ch. 6 102
Properties and proofs
• Property to be verified given through a formula (in temporal logic)
• In the example, one can prove– there is always a computation that
allows the left process to enter the critical region
– there is no guarantee that the left process accesses the shared resource unless it already owns it
![Page 103: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/103.jpg)
Ch. 6 103
Why so many approaches to testing and analysis?
• Testing versus (correctness) analysis• Formal versus informal techniques• White-box versus black-box techniques• Techniques in the small/large• Fully automatic vs. semiautomatic
techniques (for undecidable properties)• …
view all these as complementary
![Page 104: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/104.jpg)
Ch. 6 104
Debugging
• The activity of locating and correcting errors
• It can start once a failure has been detected
• The goal is closing up the gap between a fault and failure– memory dumps, watch points– intermediate assertions can help
![Page 105: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/105.jpg)
Ch. 6 105
Verifying other qualities
![Page 106: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/106.jpg)
Ch. 6 106
Performance
• Worst case analysis– focus is on proving that the system
response time is bounded by some function of the external requests
vs. average behavior• Standard deviation• Analytical vs. experimental
approaches
![Page 107: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/107.jpg)
Ch. 6 107
Reliability (1)
• There are approaches to measuring reliability on a probabilistic basis, as in other engineering fields
• Unfortunately there are some difficulties with this approach
• Independence of failures does not hold for software
![Page 108: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/108.jpg)
Ch. 6 108
Reliability (2)
• Reliability is concerned with measuring the probability of the occurrence of failure
• Meaningful parameters include:• average total number of failures
observed at time t: AF(t)• failure intensity: FI(t)=AF'(t)• mean time to failure at time t:
MTTF(t)=1/FI(t)
• Time in the model can be execution or clock or calendar time
![Page 109: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/109.jpg)
Ch. 6 109
Basic reliability model
• Assumes that the decrement per failure experienced (i.e., the derivative with respect to the number of detected failures) of the failure intensity function is constant– i.e., FI is a function of AF
FI(AF) = FI0 (1 - AF/AF∞)
where FI0 is the initial failure intensity and AF∞ is the total number of failures
• The model is based on optimistic hypothesis that a decrease in failures is due to the fixing of the errors that were sources of failures
![Page 110: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/110.jpg)
Ch. 6 110
Logarithmic model
• Assumes, more conservatively, that the decrement per failure of FI decreases exponentially FI(AF) = FI0 exp ( - AF)
: failure intensity decay parameter
![Page 111: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/111.jpg)
Ch. 6 111
FI
AF
AF0
AF
Basic model
Logarithmic model
Model comparison
![Page 112: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/112.jpg)
Ch. 6 112
AF
AF
t
Basic model
Logarithmic model
Model comparison
![Page 113: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/113.jpg)
Ch. 6 113
Verifying subjective qualities
• Consider notions like simplicity, reusability, understandability …
• Software science (due to Halstead) has been an attempt
![Page 114: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/114.jpg)
Ch. 6 114
Halstead's software science
• Tries to measure some software qualities, such asabstraction level, effort, …
• by measuring some quantities on code, such as1, number of distinct operators in the program 2, number of distinct operands in the program- N1, number of occurrences of operators in the
program- N2, number of occurrences of operands in the
program
![Page 115: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/115.jpg)
Ch. 6 115
McCabe's source code metric
• Cyclomatic complexity of the control graph– C = e - n + 2 p
• e is # edges, n is # nodes, p is # connected components
• McCabe contends that well-structured modules have C in range 3 .. 7, and C = 10 is a reasonable upper limit for the complexity of a single module– confirmed by empirical evidence
![Page 116: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/116.jpg)
Ch. 6 116
Goal-question-metric (GQM)
• Premise– software metrics must be used to analyze
software qualities, not to evaluate people – quality evaluation must be of end product,
intermediate products, and process– metrics must be defined in the context of
a complete and well-designed quality improvement paradigm (QIP)
![Page 117: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/117.jpg)
Ch. 6 117
GQM
• Not concerned with measuring a single quantity or group of quantities– e.g., cyclomatic complexity
• A method that is intended to lead from a precise definition of the objectives of measuring qualities (the goals) to the quantities (the metrics) whose measures are used to verify the achievement of such qualities.
![Page 118: Ch. 61 Verification. Ch. 62 Outline What are the goals of verification? What are the main approaches to verification? –What kind of assurance do we get](https://reader037.vdocuments.us/reader037/viewer/2022110320/56649cc45503460f9498df67/html5/thumbnails/118.jpg)
Ch. 6 118
The method
• Define goal precisely—Example:– Analyze the information system with the
purpose of estimating the costs from the point of view of the manager in the context of a major software house
• Define suitable set of questions aimed at achieving the stated goal
• Associate a precise metric with every question