cmpt 473 software quality assurance intro to testingwsumner/teaching/473/02-testing.pdf · intro to...
TRANSCRIPT
![Page 1: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/1.jpg)
Intro to Testing
CMPT 473Software Quality Assurance
Nick Sumner
![Page 2: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/2.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
![Page 3: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/3.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
– Measurement – Testing provides a metric of software quality
![Page 4: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/4.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
– Measurement – Testing provides a metric of software quality
e.g. for requirements / criteria R1, R2, R3, R4
Each test T can check a requirement
![Page 5: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/5.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
– Measurement – Testing provides a metric of software quality
e.g. for requirements / criteria R1, R2, R3, R4
Each test T can check a requirement
T1 → R1, R2
![Page 6: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/6.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
– Measurement – Testing provides a metric of software quality
e.g. for requirements / criteria R1, R2, R3, R4
Each test T can check a requirement
T1 → R1, R2
T2 → R3
![Page 7: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/7.jpg)
Why Do We Test?
● Recall: What role did testing play in the process we saw last time?
– Measurement – Testing provides a metric of software quality
e.g. for requirements / criteria R1, R2, R3, R4
Each test T can check a requirement
T1 → R1, R2
T2 → R3
T3 → R4
![Page 8: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/8.jpg)
8
But What is Testing?
Reasoning about behavior is hard/subtle.
![Page 9: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/9.jpg)
9
But What is Testing?
Reasoning about behavior is hard/subtle.
Running a program is easy (easier)….
![Page 10: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/10.jpg)
10
But What is Testing?
Reasoning about behavior is hard/subtle.
Running a program is easy (easier)….
Testing (informally):
Running the program to see if it behaves as expected
![Page 11: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/11.jpg)
11
But What is Testing?
Reasoning about behavior is hard/subtle.
Running a program is easy (easier)….
Simple idea, but...
● More than half of development cost
● Still cheaper than not testing
● Testing well is hard
Testing (informally):
Running the program to see if it behaves as expected
![Page 12: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/12.jpg)
12
Ideas?
Run a program on all inputs:
for test in allPossibleInputs: run_program(test)
![Page 13: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/13.jpg)
13
Ideas?
Run a program on all inputs:
Why not?
for test in allPossibleInputs: run_program(test)
![Page 14: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/14.jpg)
14
Ideas?
Run a program on all inputs:
Why not?
Maybe select a few tests:
import random.samplefor test in sample(allPossibleInputs, 100): run_program(test)
for test in allPossibleInputs: run_program(test)
![Page 15: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/15.jpg)
15
Ideas?
Run a program on all inputs:
Why not?
Maybe select a few tests:
Why not?
for test in allPossibleInputs: run_program(test)
import random.samplefor test in sample(allPossibleInputs, 100): run_program(test)
![Page 16: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/16.jpg)
16
Ideas?
Run a program on all inputs:
Why not?
Maybe select a few tests:
Why not? A primitive example of fuzz testing.
for test in allPossibleInputs: run_program(test)
import random.samplefor test in sample(allPossibleInputs, 100): run_program(test)
![Page 17: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/17.jpg)
17
Need A Bit More Care
Testing:
● Dynamically examines (runs) a program
![Page 18: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/18.jpg)
18
Need A Bit More Care
Testing:
● Dynamically examines (runs) a program
● Considers specific software under test
![Page 19: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/19.jpg)
19
Need A Bit More Care
Testing:
● Dynamically examines (runs) a program
● Considers specific software under test
● Run test cases from a test suite that targets specific quality goals
![Page 20: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/20.jpg)
20
Need A Bit More Care
Testing:
● Dynamically examines (runs) a program
● Considers specific software under test
● Run test cases from a test suite that targets specific quality goals
● Identifies differences between observed behavior and expected behavior
![Page 21: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/21.jpg)
21
Need A Bit More Care
Testing:
● Dynamically examines (runs) a program
● Considers specific software under test
● Run test cases from a test suite that targets specific quality goals
● Identifies differences between observed behavior and expected behavior
We can use this framework to refine how we test
![Page 22: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/22.jpg)
22
Targeting Quality Objectives
● Functional
– Does the program provide expected output for a given input?
e.g. …
![Page 23: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/23.jpg)
23
Targeting Quality Objectives
● Functional
– Does the program provide expected output for a given input?
e.g. Correct Output. All features present. Interface design.
![Page 24: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/24.jpg)
24
Targeting Quality Objectives
● Functional
– Does the program provide expected output for a given input?
e.g. Correct Output. All features present. Interface design.
● Nonfunctional
– Are output independent goals met?
e.g. …
![Page 25: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/25.jpg)
25
Targeting Quality Objectives
● Functional
– Does the program provide expected output for a given input?
e.g. Correct Output. All features present. Interface design.
● Nonfunctional
– Are output independent goals met?
e.g. Performance, Scalability, Security, Documentation
![Page 26: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/26.jpg)
26
Targeting Quality Objectives
● Functional
– Does the program provide expected output for a given input?
e.g. Correct Output. All features present. Interface design.
● Nonfunctional
– Are output independent goals met?
e.g. Performance, Scalability, Security, Documentation
We'll start this semester by looking at functional goals.
![Page 27: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/27.jpg)
27
Subtle Terminology
● Fault / Defect– Flaws in static software (e.g. incorrect code)
![Page 28: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/28.jpg)
28
Subtle Terminology
● Fault / Defect– Flaws in static software (e.g. incorrect code)
● Failure– An observable, incorrect behavior as compared to expected results
![Page 29: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/29.jpg)
29
Subtle Terminology
● Fault / Defect– Flaws in static software (e.g. incorrect code)
● Failure– An observable, incorrect behavior as compared to expected results
● Error / Infection– Incorrect internal state (not yet observed)
![Page 30: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/30.jpg)
30
Subtle Terminology
● Fault / Defect– Flaws in static software (e.g. incorrect code)
● Failure– An observable, incorrect behavior as compared to expected results
● Error / Infection– Incorrect internal state (not yet observed)
● Latent Defect– Unobserved defects in delivered software that testing did not expose
![Page 31: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/31.jpg)
31
Subtle Terminology
● Fault / Defect– Flaws in static software (e.g. incorrect code)
● Failure– An observable, incorrect behavior as compared to expected results
● Error / Infection– Incorrect internal state (not yet observed)
● Latent Defect– Unobserved defects in delivered software that testing did not expose
The later a defect is found,the more it costs to fix. Why?
![Page 32: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/32.jpg)
32
A Simple Example
void toUppercase(char *str) { for (int i = 0, e = strlen(str) - 1; i < e; ++i) { if (isletter(str[i]) && islower(str[i])) { str[i] = str[i] - 32; } } printf(“%s\n”, str);}
![Page 33: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/33.jpg)
33
A Simple Example
● What is a fault in this program?
void toUppercase(char *str) { for (int i = 0, e = strlen(str) - 1; i < e; ++i) { if (isletter(str[i]) && islower(str[i])) { str[i] = str[i] - 32; } } printf(“%s\n”, str);}
![Page 34: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/34.jpg)
34
A Simple Example
● What is a fault in this program?
● What is a test case that has a failure?
void toUppercase(char *str) { for (int i = 0, e = strlen(str) - 1; i < e; ++i) { if (isletter(str[i]) && islower(str[i])) { str[i] = str[i] - 32; } } printf(“%s\n”, str);}
![Page 35: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/35.jpg)
35
A Simple Example
● What is a fault in this program?
● What is a test case that has a failure?
● What is a test case that does not have a failure?
void toUppercase(char *str) { for (int i = 0, e = strlen(str) - 1; i < e; ++i) { if (isletter(str[i]) && islower(str[i])) { str[i] = str[i] - 32; } } printf(“%s\n”, str);}
![Page 36: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/36.jpg)
36
A Simple Example
● What is a fault in this program?
● What is a test case that has a failure?
● What is a test case that does not have a failure?
What exactly do we mean by test case?
void toUppercase(char *str) { for (int i = 0, e = strlen(str) - 1; i < e; ++i) { if (isletter(str[i]) && islower(str[i])) { str[i] = str[i] - 32; } } printf(“%s\n”, str);}
![Page 37: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/37.jpg)
37
Test Cases
Test cases need
● Input to provide the program
● Expected output or behavior to check for correctness
![Page 38: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/38.jpg)
38
Test Cases
Test cases need
● Input to provide the program
● Expected output or behavior to check for correctness
But where does the expected behavior come from?
● An oracle
![Page 39: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/39.jpg)
39
Test Oracles
● In general, a means of deciding whether a test passes or fails (was the behavior expected or not)
![Page 40: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/40.jpg)
40
Test Oracles
● In general, a means of deciding whether a test passes or fails (was the behavior expected or not)
● Sometimes very simple
– How are unit tests evaluated?
![Page 41: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/41.jpg)
41
Test Oracles
● In general, a means of deciding whether a test passes or fails (was the behavior expected or not)
● Sometimes very simple
– How are unit tests evaluated? (Test Drivers!)
![Page 42: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/42.jpg)
42
Test Oracles
● In general, a means of deciding whether a test passes or fails (was the behavior expected or not)
● Sometimes very simple
– How are unit tests evaluated? (Test Drivers!)
● Sometimes tricky
– Is result strictly specified? (content,order, timing,...)
– Is the program deterministic?
![Page 43: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/43.jpg)
43
Test Oracles
● In general, a means of deciding whether a test passes or fails (was the behavior expected or not)
● Sometimes very simple
– How are unit tests evaluated? (Test Drivers!)
● Sometimes tricky
– Is result strictly specified? (content,order, timing,...)
– Is the program deterministic?
● Sometimes requires a person
– Expensive and undesirable
– “Does this software meet my needs?”
![Page 44: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/44.jpg)
44
Coverage / Adequacy
Recall: can't look at all possible inputs.
![Page 45: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/45.jpg)
45
Coverage / Adequacy
Recall: can't look at all possible inputs.
Need to determine if a test suite covers / is adequate for our quality objectives.
![Page 46: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/46.jpg)
46
Coverage / Adequacy
Recall: can't look at all possible inputs.
Need to determine if a test suite covers / is adequate for our quality objectives.
● Sufficiently addresses criteria
● Lack of failures provides enough confidence that the software is acceptable
![Page 47: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/47.jpg)
47
Coverage / Adequacy
Recall: can't look at all possible inputs.
Need to determine if a test suite covers / is adequate for our quality objectives.
● Sufficiently addresses criteria
● Lack of failures provides enough confidence that the software is acceptable
Key Idea:
● Find a smaller test suite that is representative of our goals
![Page 48: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/48.jpg)
48
Approaches
● Test until you run out of time
● Test until you run out of money
![Page 49: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/49.jpg)
49
Approaches
● Test until you run out of time
● Test until you run out of money
● Identify redundant inputs based on the specification
![Page 50: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/50.jpg)
50
Approaches
● Test until you run out of time
● Test until you run out of money
● Identify redundant inputs based on the specification
● Identify redundant inputs based on program structure
![Page 51: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/51.jpg)
51
Approaches
● Test until you run out of time
● Test until you run out of money
● Identify redundant inputs based on the specification
● Identify redundant inputs based on program structure
● Identify poorly tested areas by measuring how well your tests identify potential bugs
![Page 52: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/52.jpg)
52
Approaches
● Test until you run out of time
● Test until you run out of money
● Identify redundant inputs based on the specification
● Identify redundant inputs based on program structure
● Identify poorly tested areas by measuring how well your tests identify potential bugs
No approach covers everything you want!
Need to combine them for a balanced approach toward the desired goals.
![Page 53: CMPT 473 Software Quality Assurance Intro to Testingwsumner/teaching/473/02-testing.pdf · Intro to Testing CMPT 473 Software Quality Assurance Nick Sumner. Quiz What are three perspectives/roles](https://reader034.vdocuments.us/reader034/viewer/2022042709/5f5213656238b429c202e6f7/html5/thumbnails/53.jpg)
53
Next Up...
Revisit the basics of unit testing.