basic concepts and unit testing - 219343 software testing
TRANSCRIPT
Outline Basic concepts Testing principles Unit testing
Basic concepts and unit testing219343 Software Testing
Jittat Fakcharoenphol
Kasetsart University
November 7, 2008
Outline Basic concepts Testing principles Unit testing
Outline
1 Basic concepts
2 Testing principles
3 Unit testing
Outline Basic concepts Testing principles Unit testing
Testing1
Testing in not sexy.
When projects fail, testing is the reason.
In Europe in ancient times bearer of bad news got killedFollowing legends, even bearer of good news died ...
1Most of the materials on basic concepts are taken from slides developed forthe course in 2007 mostly by Uwe Guhl. (See the course homepage for thatsemester.) Thanks, Uwe!
Outline Basic concepts Testing principles Unit testing
Testing1
Testing in not sexy.
When projects fail, testing is the reason.
In Europe in ancient times bearer of bad news got killedFollowing legends, even bearer of good news died ...
1Most of the materials on basic concepts are taken from slides developed forthe course in 2007 mostly by Uwe Guhl. (See the course homepage for thatsemester.) Thanks, Uwe!
Outline Basic concepts Testing principles Unit testing
Testing1
Testing in not sexy.
When projects fail, testing is the reason.
In Europe in ancient times bearer of bad news got killed
Following legends, even bearer of good news died ...
1Most of the materials on basic concepts are taken from slides developed forthe course in 2007 mostly by Uwe Guhl. (See the course homepage for thatsemester.) Thanks, Uwe!
Outline Basic concepts Testing principles Unit testing
Testing1
Testing in not sexy.
When projects fail, testing is the reason.
In Europe in ancient times bearer of bad news got killedFollowing legends, even bearer of good news died ...
1Most of the materials on basic concepts are taken from slides developed forthe course in 2007 mostly by Uwe Guhl. (See the course homepage for thatsemester.) Thanks, Uwe!
Outline Basic concepts Testing principles Unit testing
Definitions
”Testing is the process of establishing confidence that aprogram or system does what it is supposed to.” (Hetzel,1973)
”Testing is demonstrating that a system is fit for purpose.”(Evans et al. 1996)
”Testing is the process of executing a program or system withthe intent of finding errors.” (Myers, 1979)
”Testing is the process consisting of all life cycle activitiesconcerned with checking software and software-related workproducts.” (Gelperin and Hetzel, 1988)
Outline Basic concepts Testing principles Unit testing
Definitions
”Testing is the process of establishing confidence that aprogram or system does what it is supposed to.” (Hetzel,1973)
”Testing is demonstrating that a system is fit for purpose.”(Evans et al. 1996)
”Testing is the process of executing a program or system withthe intent of finding errors.” (Myers, 1979)
”Testing is the process consisting of all life cycle activitiesconcerned with checking software and software-related workproducts.” (Gelperin and Hetzel, 1988)
Outline Basic concepts Testing principles Unit testing
Definitions
”Testing is the process of establishing confidence that aprogram or system does what it is supposed to.” (Hetzel,1973)
”Testing is demonstrating that a system is fit for purpose.”(Evans et al. 1996)
”Testing is the process of executing a program or system withthe intent of finding errors.” (Myers, 1979)
”Testing is the process consisting of all life cycle activitiesconcerned with checking software and software-related workproducts.” (Gelperin and Hetzel, 1988)
Outline Basic concepts Testing principles Unit testing
Definitions
”Testing is the process of establishing confidence that aprogram or system does what it is supposed to.” (Hetzel,1973)
”Testing is demonstrating that a system is fit for purpose.”(Evans et al. 1996)
”Testing is the process of executing a program or system withthe intent of finding errors.” (Myers, 1979)
”Testing is the process consisting of all life cycle activitiesconcerned with checking software and software-related workproducts.” (Gelperin and Hetzel, 1988)
Outline Basic concepts Testing principles Unit testing
Statements
“Program testing can be used to show the presence of bugs,but never to show their absence!” (Dijkstra 1969)
“In most cases ‘what you test in a system is much moreimportant than ‘how much you test” (Craig 2002)
“Prioritise tests so that, when ever you stop testing, you havedone the best testing in the time available.” (ISEB testingfoundation course material 2003)
Outline Basic concepts Testing principles Unit testing
Statements
“Program testing can be used to show the presence of bugs,but never to show their absence!” (Dijkstra 1969)
“In most cases ‘what you test in a system is much moreimportant than ‘how much you test” (Craig 2002)
“Prioritise tests so that, when ever you stop testing, you havedone the best testing in the time available.” (ISEB testingfoundation course material 2003)
Outline Basic concepts Testing principles Unit testing
Statements
“Program testing can be used to show the presence of bugs,but never to show their absence!” (Dijkstra 1969)
“In most cases ‘what you test in a system is much moreimportant than ‘how much you test” (Craig 2002)
“Prioritise tests so that, when ever you stop testing, you havedone the best testing in the time available.” (ISEB testingfoundation course material 2003)
Outline Basic concepts Testing principles Unit testing
Goals
How many testers does it take to change a light bulb?
None.
Testers just noticed that the room was dark. Testers dont fixthe problems, they just find them.
Outline Basic concepts Testing principles Unit testing
Goals
How many testers does it take to change a light bulb?
None.
Testers just noticed that the room was dark. Testers dont fixthe problems, they just find them.
Outline Basic concepts Testing principles Unit testing
Goals
How many testers does it take to change a light bulb?
None.
Testers just noticed that the room was dark. Testers dont fixthe problems, they just find them.
Outline Basic concepts Testing principles Unit testing
Goals
Goals can vary. Define your mission.
Find important bugs fast.
Outline Basic concepts Testing principles Unit testing
Specific goals
Goal of Testing is to establish the ground for the acceptance of thesoftware by the customer based on the specification through
1 High test coverage
2 Low number of non critical defects There should be no criticaldefect
3 Statements concerning software quality
Outline Basic concepts Testing principles Unit testing
Specific goals
Goal of Testing is to establish the ground for the acceptance of thesoftware by the customer based on the specification through
1 High test coverage
2 Low number of non critical defects
There should be no criticaldefect
3 Statements concerning software quality
Outline Basic concepts Testing principles Unit testing
Specific goals
Goal of Testing is to establish the ground for the acceptance of thesoftware by the customer based on the specification through
1 High test coverage
2 Low number of non critical defects There should be no criticaldefect
3 Statements concerning software quality
Outline Basic concepts Testing principles Unit testing
Specific goals
Goal of Testing is to establish the ground for the acceptance of thesoftware by the customer based on the specification through
1 High test coverage
2 Low number of non critical defects There should be no criticaldefect
3 Statements concerning software quality
Outline Basic concepts Testing principles Unit testing
Goals: high test coverage
Completeness. Ensure all requirements are implemented.
Total scope must be tested at least once (of course hopefullysuccessful)
Critical scope. Ensure that critical requirements areimplemented and work fine.All high prioritized requirements must be tested successfully
Outline Basic concepts Testing principles Unit testing
Goals: high test coverage
Completeness. Ensure all requirements are implemented.Total scope must be tested at least once (of course hopefullysuccessful)
Critical scope. Ensure that critical requirements areimplemented and work fine.
All high prioritized requirements must be tested successfully
Outline Basic concepts Testing principles Unit testing
Goals: high test coverage
Completeness. Ensure all requirements are implemented.Total scope must be tested at least once (of course hopefullysuccessful)
Critical scope. Ensure that critical requirements areimplemented and work fine.All high prioritized requirements must be tested successfully
Outline Basic concepts Testing principles Unit testing
Goals: how number of non critical defects
At the end the final version of the application:
should have no critical defects any moreshould have only a small number of tolerable defects
Demand on Testing is therefore to detect as much criticaldefects as soon as possible idea is to fix them during theTesting phase
The acceptance criteria should determine what the customerexpects. A contract should contain acceptance criteriaconcerning severity level and priority level of bugs.
Outline Basic concepts Testing principles Unit testing
Goals: how number of non critical defects
At the end the final version of the application:
should have no critical defects any moreshould have only a small number of tolerable defects
Demand on Testing is therefore to detect as much criticaldefects as soon as possible idea is to fix them during theTesting phase
The acceptance criteria should determine what the customerexpects. A contract should contain acceptance criteriaconcerning severity level and priority level of bugs.
Outline Basic concepts Testing principles Unit testing
Goals: how number of non critical defects
At the end the final version of the application:
should have no critical defects any moreshould have only a small number of tolerable defects
Demand on Testing is therefore to detect as much criticaldefects as soon as possible idea is to fix them during theTesting phase
The acceptance criteria should determine what the customerexpects. A contract should contain acceptance criteriaconcerning severity level and priority level of bugs.
Outline Basic concepts Testing principles Unit testing
Goals: Statements concerning software quality
Is it possible to install the software?
Is it possible to operate the software, is it compatible?
Fulfils the software the expected functionality?
Do the interfaces work?
Is it possible to use the software optimal(Software-ergonomics, usability, end user needs)
Does the software run steadily, with high performance, failproof?
Fulfils the software special cultural features (Multilingualism,English / metric system, weight units)?
Is the software safe / secure?
Outline Basic concepts Testing principles Unit testing
Tools/processes needed
Defect management (issue trackers)
Release management
Outline Basic concepts Testing principles Unit testing
Tools/processes needed
Defect management
(issue trackers)
Release management
Outline Basic concepts Testing principles Unit testing
Tools/processes needed
Defect management (issue trackers)
Release management
Outline Basic concepts Testing principles Unit testing
Tools/processes needed
Defect management (issue trackers)
Release management
Outline Basic concepts Testing principles Unit testing
Testing principles2
Principles provide a general guideline.
2Materials from Graham, van Veenendall, Evans, and Black, Fundamentalsof Software Testing: ISTQB Certification, Thomson, 2007.
Outline Basic concepts Testing principles Unit testing
Testing shows presence of defect
Outline Basic concepts Testing principles Unit testing
Exhaustive testing is impossible
Outline Basic concepts Testing principles Unit testing
Early testing
Outline Basic concepts Testing principles Unit testing
Example: The Triangle Test3
Given the lengths of three sides of a triangle, determine thetype of that triangle.
EquilateralIsoscelesScalene
3Taken from Black, Pragmatic Software Testing, Wiley, 2007
Outline Basic concepts Testing principles Unit testing
The code that can be tested
Consider this code:
package lect02;
public class Triangle1 {
static void checkType(int a, int b, int c) {if((a==b) && (b==c))
System.out.println("Equilateral");
else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");
elseSystem.out.println("Scalene");
}
public static void main(String [] argv) {int a,b,c;
// .... read inputcheckType(a,b,c);
}}
How can we test thisprogram, in particularmethod checkType?
Automatically? Veryvery difficult.Because?
It is hard to check theoutput of the method.
Outline Basic concepts Testing principles Unit testing
The code that can be tested
Consider this code:
package lect02;
public class Triangle1 {
static void checkType(int a, int b, int c) {if((a==b) && (b==c))
System.out.println("Equilateral");
else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");
elseSystem.out.println("Scalene");
}
public static void main(String [] argv) {int a,b,c;
// .... read inputcheckType(a,b,c);
}}
How can we test thisprogram, in particularmethod checkType?
Automatically? Veryvery difficult.Because?
It is hard to check theoutput of the method.
Outline Basic concepts Testing principles Unit testing
The code that can be tested
Consider this code:
package lect02;
public class Triangle1 {
static void checkType(int a, int b, int c) {if((a==b) && (b==c))
System.out.println("Equilateral");
else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");
elseSystem.out.println("Scalene");
}
public static void main(String [] argv) {int a,b,c;
// .... read inputcheckType(a,b,c);
}}
How can we test thisprogram, in particularmethod checkType?
Automatically?
Veryvery difficult.Because?
It is hard to check theoutput of the method.
Outline Basic concepts Testing principles Unit testing
The code that can be tested
Consider this code:
package lect02;
public class Triangle1 {
static void checkType(int a, int b, int c) {if((a==b) && (b==c))
System.out.println("Equilateral");
else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");
elseSystem.out.println("Scalene");
}
public static void main(String [] argv) {int a,b,c;
// .... read inputcheckType(a,b,c);
}}
How can we test thisprogram, in particularmethod checkType?
Automatically? Veryvery difficult.Because?
It is hard to check theoutput of the method.
Outline Basic concepts Testing principles Unit testing
The code that can be tested
Consider this code:
package lect02;
public class Triangle1 {
static void checkType(int a, int b, int c) {if((a==b) && (b==c))
System.out.println("Equilateral");
else if((a==b) ||(b==c) ||(a==c))System.out.println("Isosceles");
elseSystem.out.println("Scalene");
}
public static void main(String [] argv) {int a,b,c;
// .... read inputcheckType(a,b,c);
}}
How can we test thisprogram, in particularmethod checkType?
Automatically? Veryvery difficult.Because?
It is hard to check theoutput of the method.
Outline Basic concepts Testing principles Unit testing
After some fix
package lect02;
public class Triangle {
enum Type { Equilateral, Isosceles, Scalene }
static Type checkType(int a, int b, int c) {if((a==b) && (b==c))
return Type.Equilateral;else if((a==b) ||(b==c) ||(a==c))return Type.Isosceles;
elsereturn Type.Scalene;
}}
Outline Basic concepts Testing principles Unit testing
Practice
Find as many interesting test cases for this method.Write in this form:
Tester action and data Expected result
We will use this example to talk about good test set again.
Outline Basic concepts Testing principles Unit testing
Practice
Find as many interesting test cases for this method.Write in this form:
Tester action and data Expected result
We will use this example to talk about good test set again.
Outline Basic concepts Testing principles Unit testing
Unit testing: classic example4
John: John works hard. He codes everyday. The project deadline is tomorrow.
He types in about two hundred new lines per hour, and thinks that after 6 hours
and roughly a thousand new lines added the program would work flawlessly.
Betty: Betty works hard. She codes everyday. The project deadline is
tomorrow. She types in about one hundred new lines per hour, and keeps
testing each method she adds. She does not proceed to write new codes unless
all previously written pieces work correctly.
Guess who will go to bed earlier?
4Materials regarding unit testing concepts are from Alberto Savoia’s slidesand George Necular’s software engineering course
Outline Basic concepts Testing principles Unit testing
Unit testing: classic example4
John: John works hard. He codes everyday. The project deadline is tomorrow.
He types in about two hundred new lines per hour, and thinks that after 6 hours
and roughly a thousand new lines added the program would work flawlessly.
Betty: Betty works hard. She codes everyday. The project deadline is
tomorrow. She types in about one hundred new lines per hour, and keeps
testing each method she adds. She does not proceed to write new codes unless
all previously written pieces work correctly.
Guess who will go to bed earlier?
4Materials regarding unit testing concepts are from Alberto Savoia’s slidesand George Necular’s software engineering course
Outline Basic concepts Testing principles Unit testing
Unit testing: classic example4
John: John works hard. He codes everyday. The project deadline is tomorrow.
He types in about two hundred new lines per hour, and thinks that after 6 hours
and roughly a thousand new lines added the program would work flawlessly.
Betty: Betty works hard. She codes everyday. The project deadline is
tomorrow. She types in about one hundred new lines per hour, and keeps
testing each method she adds. She does not proceed to write new codes unless
all previously written pieces work correctly.
Guess who will go to bed earlier?
4Materials regarding unit testing concepts are from Alberto Savoia’s slidesand George Necular’s software engineering course
Outline Basic concepts Testing principles Unit testing
Developer Testing Revolution
Developer testing is a key component in a hot paradigm:Agile/eXtreme Programming.
The Developer Testing Trinity:
TestTest early and oftenTest well
Outline Basic concepts Testing principles Unit testing
Developer Testing Revolution
Developer testing is a key component in a hot paradigm:Agile/eXtreme Programming.
The Developer Testing Trinity:
TestTest early and oftenTest well
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Good reasons for developer testing
Reduces unit-level bugs
Forces you to slow down and think
Improves design
Makes development faster
Tests are good documentation
Tests constrain features
Tests allows safe refactoring and reduce the cost of change
Tests defend against other programmers
Tests reduce fear
Outline Basic concepts Testing principles Unit testing
Goals
Does the code do what I want?
Does the code do what I want all the time?
Can I depend on it?
Also: get a document for the code.Plus: Always correct documentation for your intention.
Outline Basic concepts Testing principles Unit testing
Goals
Does the code do what I want?
Does the code do what I want all the time?
Can I depend on it?
Also: get a document for the code.Plus: Always correct documentation for your intention.
Outline Basic concepts Testing principles Unit testing
Goals
Does the code do what I want?
Does the code do what I want all the time?
Can I depend on it?
Also: get a document for the code.Plus: Always correct documentation for your intention.
Outline Basic concepts Testing principles Unit testing
Goals
Does the code do what I want?
Does the code do what I want all the time?
Can I depend on it?
Also: get a document for the code.
Plus: Always correct documentation for your intention.
Outline Basic concepts Testing principles Unit testing
Goals
Does the code do what I want?
Does the code do what I want all the time?
Can I depend on it?
Also: get a document for the code.Plus: Always correct documentation for your intention.
Outline Basic concepts Testing principles Unit testing
Test your code
It is your code, and your responsibility
Do it for your current colleagues
Do it for future generation of colleagues
Do it for yourself
Outline Basic concepts Testing principles Unit testing
Test early and often
Outline Basic concepts Testing principles Unit testing
Test early and often
Outline Basic concepts Testing principles Unit testing
Heaven!
Every class has unit tests
The tests are executed many times each day
The tests are thorough, up to date, and easy to maintain andanalyze
In this class, we shall aim for that.
Outline Basic concepts Testing principles Unit testing
Heaven!
Every class has unit tests
The tests are executed many times each day
The tests are thorough, up to date, and easy to maintain andanalyze
In this class, we shall aim for that.
Outline Basic concepts Testing principles Unit testing
Heaven!
Every class has unit tests
The tests are executed many times each day
The tests are thorough, up to date, and easy to maintain andanalyze
In this class, we shall aim for that.
Outline Basic concepts Testing principles Unit testing
Heaven!
Every class has unit tests
The tests are executed many times each day
The tests are thorough, up to date, and easy to maintain andanalyze
In this class, we shall aim for that.
Outline Basic concepts Testing principles Unit testing
Heaven!
Every class has unit tests
The tests are executed many times each day
The tests are thorough, up to date, and easy to maintain andanalyze
In this class, we shall aim for that.
Outline Basic concepts Testing principles Unit testing
A unit
What’s a unit?
A single method/function/procedureA collection of related methods/functions/procedues
Ideal world: independent, self-sufficient, standalone
Real world: lots of dependence
Outline Basic concepts Testing principles Unit testing
A unit
What’s a unit?
A single method/function/procedureA collection of related methods/functions/procedues
Ideal world:
independent, self-sufficient, standalone
Real world: lots of dependence
Outline Basic concepts Testing principles Unit testing
A unit
What’s a unit?
A single method/function/procedureA collection of related methods/functions/procedues
Ideal world: independent, self-sufficient, standalone
Real world: lots of dependence
Outline Basic concepts Testing principles Unit testing
A unit
What’s a unit?
A single method/function/procedureA collection of related methods/functions/procedues
Ideal world: independent, self-sufficient, standalone
Real world:
lots of dependence
Outline Basic concepts Testing principles Unit testing
A unit
What’s a unit?
A single method/function/procedureA collection of related methods/functions/procedues
Ideal world: independent, self-sufficient, standalone
Real world: lots of dependence
Outline Basic concepts Testing principles Unit testing
Basic structure
Setup
Create initial statesInitialize method parametersStore pre-execute values
Execute code
Compare results
Outline Basic concepts Testing principles Unit testing
Basic structure
Setup
Create initial statesInitialize method parametersStore pre-execute values
Execute code
Compare results
Outline Basic concepts Testing principles Unit testing
Basic structure
Setup
Create initial statesInitialize method parametersStore pre-execute values
Execute code
Compare results
Outline Basic concepts Testing principles Unit testing
Practice: Triangle
Write junit test cases for Triangle.
Outline Basic concepts Testing principles Unit testing
Discussion: test cases for Triangle
What are your test cases?
Outline Basic concepts Testing principles Unit testing
Test-driven development
Traditional steps:design, code, test, design, code, test, ...
TDD:test, code, refactor, test, code, refactor, ...
We’ll discuss more about TDD later. For now, let’s do it alittle.
Outline Basic concepts Testing principles Unit testing
Test-driven development
Traditional steps:design, code, test, design, code, test, ...
TDD:test, code, refactor, test, code, refactor, ...
We’ll discuss more about TDD later. For now, let’s do it alittle.
Outline Basic concepts Testing principles Unit testing
Test-driven development
Traditional steps:design, code, test, design, code, test, ...
TDD:test, code, refactor, test, code, refactor, ...
We’ll discuss more about TDD later.
For now, let’s do it alittle.
Outline Basic concepts Testing principles Unit testing
Test-driven development
Traditional steps:design, code, test, design, code, test, ...
TDD:test, code, refactor, test, code, refactor, ...
We’ll discuss more about TDD later. For now, let’s do it alittle.
Outline Basic concepts Testing principles Unit testing
TDD: Triangle
See demo.
Outline Basic concepts Testing principles Unit testing
Practice: Median