WARNING

These slides are not optimized for printing or exam preparation. These are for lecture delivery only.

These slides are made for PowerPoint 2010. They may not show up well on other PowerPoint versions. You can download PowerPoint 2010 viewer from here.

These slides contain a lot of animations. For optimal results, watch in slideshow mode.

But it worked in my

machine!Our

tester

Your software

Quality Assurance: Testing and Beyond

CS2103/T, Lecture 9, Part 1, [Oct 18, 2013]

Validation Verification

[Quality Assurance]

Validation Verification

[Quality Assurance]

Are we building the correct product?

Are we building the product correctly?

Validation Verification

[Quality Assurance]

Are we building the correct product?

Are we building the product correctly?

Validation Verification

[Quality Assurance]

Are we building the correct product?

Are we building the product correctly?

FAIL!

Validation Verification

[Quality Assurance]

Are we building the correct product?

Are we building the product correctly?

FAIL! FAIL!

Validation Verification

[Quality Assurance]

Buggy requirements Buggy code

FAIL! FAIL!

Validation Verification

[Quality Assurance]

Two QA activities for a PC version of a ‘Snakes and ladders’ game:a) Ensure that the game allows players to decide

who throws the die first.b) When more than one piece is on a square,

ensure that it is the piece that belongs to the current player that will be moved.

(Are we building the correct product?)

(Are we building the product correctly?)

snakes {a|b}e.g. snakes a

77577tinyurl.com/questionSMS

Which one is more related to verification than validation?

Validation Verification

[Quality Assurance]

Validation Verification

[Quality Assurance]

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

But it worked in my

machine!

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

System specification

Requirements specification

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

Product

System specification

Requirements specification

Validation Verification

[Quality Assurance]

Product

E.g. Acceptance testing E.g. System testing

Validation Verification

[Quality Assurance]

Product

E.g. Acceptance testing E.g. System testing

Validation Verification

[Quality Assurance]

Product

E.g. Acceptance testing E.g. System testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

Product

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

GU

I

Logic

Automated API tester

Manual testing

(or automate)

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

GU

I

Logic

Automated API tester

Manual testing

(or automate)

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

GUI testing

GUI testing

GUI testing

GUI testing

GUI testing

GUI testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

GUI testing

Validation Verification

E.g. Acceptance testing E.g. System testing

Back-end

GUI

[Quality Assurance]

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Using EclEmma coverage tool on TEAMMATES

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Statement coverage

Function coverage

Condition coverage

Exit/entry coverage

Branch coverage

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Statement coverage

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Statement coverage

Path coveragevoid readInput(){

while ( hasInput() ) input += readFromKeyboard();

}

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Statement coverage

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

void readInput(){while ( hasInput() )

input += readFromKeyboard();}

Path coverage

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Path coveragevoid readInput(){

while ( hasInput() ) input += readFromKeyboard();

}

Control Flow Graphs

Validation Verification

E.g. Acceptance testing E.g. System testing

[Quality Assurance]

← How much of this is being tested?

Statement coverage

Function coverage

Condition coverage

Exit/entry coverage

Branch coverage

Path coverage

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

E.g. System testing

just testing?

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

E.g. System testing

Other QA techniques

just testing?

[Quality Assurance]

Other QA techniques

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Code reviews • static analysis • formal methods• …

Other QA techniques

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Using CodeProAnalytix static analysis tool on TEAMMATES

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

SUT doubleAmount(int):int

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Text case 1: Input: 23 Expected output: 46

Text case 2: Input: 10 Expected output : 20

Text case n: Input: -2 Expected output : -4

…

Testi

ng

SUT doubleAmount(int):int

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Mathematically prove if input = x

then, output = 2x

Testi

ng

SUT doubleAmount(int):int

Text case 1: Input: 23 Expected output: 46

Text case 2: Input: 10 Expected output : 20

Text case n: Input: -2 Expected output : -4

…

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Validation Verification

E.g. Acceptance testing

[Quality Assurance]

Other QA techniques

Code reviews • static analysis • formal methods• …

Have the ability to relate these concepts to the project: validation, verification, system testing, acceptance testing , code coverage, GUI testing, other QA techniques.

For example, the ability to explain how the above was done in the project, or why they were not done and how you could have done them.

You are welcome to, but not required to, use • Automation of GUI testing• Using tools to measure coverage• Path analysis• Apply other QA techniques

If all tests pass and those tests achieve100% path coverage (all possible execution paths through the code have been tested), can the software still have bugs?

(Discuss with neighbors before submitting)

buggy {yes|no}e.g. buggy yes

77577tinyurl.com/questionSMS