Download - WARNING
WARNING
These slides are not optimized for printing or exam preparation. These are for lecture delivery only.
These slides are made for PowerPoint 2010. They may not show up well on other PowerPoint versions. You can download PowerPoint 2010 viewer from here.
These slides contain a lot of animations. For optimal results, watch in slideshow mode.
But it worked in my
machine!Our
tester
Your software
Quality Assurance: Testing and Beyond
CS2103/T, Lecture 9, Part 1, [Oct 18, 2013]
Validation Verification
[Quality Assurance]
Validation Verification
[Quality Assurance]
Are we building the correct product?
Are we building the product correctly?
Validation Verification
[Quality Assurance]
Are we building the correct product?
Are we building the product correctly?
Validation Verification
[Quality Assurance]
Are we building the correct product?
Are we building the product correctly?
FAIL!
Validation Verification
[Quality Assurance]
Are we building the correct product?
Are we building the product correctly?
FAIL! FAIL!
Validation Verification
[Quality Assurance]
Buggy requirements Buggy code
FAIL! FAIL!
Validation Verification
[Quality Assurance]
Two QA activities for a PC version of a ‘Snakes and ladders’ game:a) Ensure that the game allows players to decide
who throws the die first.b) When more than one piece is on a square,
ensure that it is the piece that belongs to the current player that will be moved.
(Are we building the correct product?)
(Are we building the product correctly?)
snakes {a|b}e.g. snakes a
77577tinyurl.com/questionSMS
Which one is more related to verification than validation?
Validation Verification
[Quality Assurance]
Validation Verification
[Quality Assurance]
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
But it worked in my
machine!
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
System specification
Requirements specification
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
Product
System specification
Requirements specification
Validation Verification
[Quality Assurance]
Product
E.g. Acceptance testing E.g. System testing
Validation Verification
[Quality Assurance]
Product
E.g. Acceptance testing E.g. System testing
Validation Verification
[Quality Assurance]
Product
E.g. Acceptance testing E.g. System testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
Product
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
GU
I
Logic
Automated API tester
Manual testing
(or automate)
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
GU
I
Logic
Automated API tester
Manual testing
(or automate)
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
GUI testing
GUI testing
GUI testing
GUI testing
GUI testing
GUI testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
GUI testing
Validation Verification
E.g. Acceptance testing E.g. System testing
Back-end
GUI
[Quality Assurance]
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Using EclEmma coverage tool on TEAMMATES
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Statement coverage
Function coverage
Condition coverage
Exit/entry coverage
Branch coverage
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Statement coverage
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Statement coverage
Path coveragevoid readInput(){
while ( hasInput() ) input += readFromKeyboard();
}
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Statement coverage
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
void readInput(){while ( hasInput() )
input += readFromKeyboard();}
Path coverage
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Path coveragevoid readInput(){
while ( hasInput() ) input += readFromKeyboard();
}
Control Flow Graphs
Validation Verification
E.g. Acceptance testing E.g. System testing
[Quality Assurance]
← How much of this is being tested?
Statement coverage
Function coverage
Condition coverage
Exit/entry coverage
Branch coverage
Path coverage
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
E.g. System testing
just testing?
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
E.g. System testing
Other QA techniques
just testing?
[Quality Assurance]
Other QA techniques
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Code reviews • static analysis • formal methods• …
Other QA techniques
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Using CodeProAnalytix static analysis tool on TEAMMATES
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
SUT doubleAmount(int):int
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Text case 1: Input: 23 Expected output: 46
Text case 2: Input: 10 Expected output : 20
Text case n: Input: -2 Expected output : -4
…
Testi
ng
SUT doubleAmount(int):int
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Mathematically prove if input = x
then, output = 2x
Testi
ng
SUT doubleAmount(int):int
Text case 1: Input: 23 Expected output: 46
Text case 2: Input: 10 Expected output : 20
Text case n: Input: -2 Expected output : -4
…
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Validation Verification
E.g. Acceptance testing
[Quality Assurance]
Other QA techniques
Code reviews • static analysis • formal methods• …
Have the ability to relate these concepts to the project: validation, verification, system testing, acceptance testing , code coverage, GUI testing, other QA techniques.
For example, the ability to explain how the above was done in the project, or why they were not done and how you could have done them.
You are welcome to, but not required to, use • Automation of GUI testing• Using tools to measure coverage• Path analysis• Apply other QA techniques
If all tests pass and those tests achieve100% path coverage (all possible execution paths through the code have been tested), can the software still have bugs?
(Discuss with neighbors before submitting)
buggy {yes|no}e.g. buggy yes
77577tinyurl.com/questionSMS