cs61b l11 testing & static (1)garcia / yelick fall 2003 © ucb 2003-09-19 dan garcia (ddgarcia)...
TRANSCRIPT
CS61B L11 Testing & Static (1) Garcia / Yelick Fall 2003 © UCB
2003-09-19 Dan Garcia
(www.cs.berkeley.edu/~ddgarcia)
Kathy Yelick (www.cs.berkeley.edu/~yelick)
inst.eecs.berkeley.edu/~cs61b/www.ucwise.org1 Handout: notes
Computer Science 61BData Structures and Advanced Programming
Lecture 11 – Testing
CS61B L11 Testing & Static (2) Garcia / Yelick Fall 2003 © UCB
Testing Overview• Purpose of testing: Provide evidence that
specification matches implementation • Why not prove that the program works?
– Because it’s impossible in general– Because it’s infeasible in practice for most
programs– But there is interesting work here (OSQ, etc.)
• Testing is hard– It cannot produce proof of correctness.– It can be tedious.– It is psychologically difficult
• Testing is not debugging– Testing is to ensure there are no bugs,– Debugging is tracking down a known problem
CS61B L11 Testing & Static (3) Garcia / Yelick Fall 2003 © UCB
Command Loop Testing• One way to testing a program is to write a
“driver” in the main method, e.g.,Enter month: __3___
Enter day: __4___
What method: __t__
Result is: “tomorrow is 3/5”
• What’s wrong with this?– Hard to keep track of which tests you have run– A lot of work to redo testing
• Don’t do this! Instead write test code.– More work the first time, but less in the long run
CS61B L11 Testing & Static (4) Garcia / Yelick Fall 2003 © UCB
Regression Testing• Regression Testing: repeated testing, to make
sure that your program has not “regressed”: – It worked yesterday, but not today (“bit rot”)– Because I fixed something else, or added a feature,
…• Write a program to test your program
– More work to write than the interpreter, but – Easy to run after that
• Have someone else write test code if possible– Do not cut-and-paste from solution to test code
• Example: Titanium has 2 sets of tests– Nightly “cron” job checks that compiler not broken– Before a major release, larger set of tests run on
multiple machines
CS61B L11 Testing & Static (5) Garcia / Yelick Fall 2003 © UCB
Testing Principles• Want tests to be easy to understand:
Testing tomorrow method: tomorrow on 3/4 is 3/5
• Self-checking if possible: All tomorrow tests PASSED
• How much test code?– At least 1x-3x more test code than “actual” code– More is not always better: not 2 pages, but the
right 2 pages• Divide cases into normal, abnormal,
boundaries– Boundary cases in the interface – Branches (and loop bounds) in the code
CS61B L11 Testing & Static (6) Garcia / Yelick Fall 2003 © UCB
Black-box vs. Glass-box Testing
• Black box tests derived from the specification– Usually done in a separate class, e.g., DateTester– Test all public methods– Use the specification to select cases: normal,
exceptional, etc.– Do not test things that fail to satisfy the “requires”
• Glass box tests derived from an implementation.– Usually done inside the class, in the main method java IslamicDate -- runs main– Can test private methods, e.g., gcd– Can test repOk on illegal representation values– Design tests looking at the code:
» branches and loops: test bounds» test all paths through the code
CS61B L11 Testing & Static (7) Garcia / Yelick Fall 2003 © UCB
Choosing Test Cases• Consider testing contains1MoreThan
– Test different outputs (only 2: true, false)– Test for possible crashing
» Nulls, substring/array/Vector bounds, 0-divide
– Note re-use of test code across versions. » Useful in project 1 (although some behavior differs)
– Test boundariesPlacement of character (first, last, middle)No character inserted: different/sameLength of strings (s1, shorter, longer, same)
•Glass box testing (which could still be done “outside”) would include loop boundaries, recursion cases, etc.
Most errors occur at the boundaries!
CS61B L11 Testing & Static (8) Garcia / Yelick Fall 2003 © UCB
Testing Classes• These ideas are fine for (static) methods, but
how do we test classes?– Abstraction prevents you from seeing inside
objects
• Idea: – test all methods that build different objects
» constructors, mutators, methods that return objects of the class
– Using accessors/observers» All methods and return (or modify) a different type
• What if constructors & observers are buggy? – Answer: It’s only a bug if you can write a program
to see it.
CS61B L11 Testing & Static (9) Garcia / Yelick Fall 2003 © UCB
Example: IslamicDateTester
write IslamicDate(int,int)
test IslamicDate(int,int)
write equals
test equals
write toString
test toString
write other constructors
test other constructors
Assume IslamicDate(int,int) is OK
Start with most important/simplest method: make sure it doesn’t crash
Next do most important accessors
write tomorrows
test tomorrowsGroup related methods for testing(check non-mutation)
Assume equals/toString OK
write daySpans
check daySpans
CS61B L11 Testing & Static (10) Garcia / Yelick Fall 2003 © UCB
Test Framework for IslamicDate
• Define a separate class for black-box testingpublic class DateTester
public static int testBasicConstructor ()
public static int testEquals ()
public static int testToString ()
public static int testOtherConstructors ()
public static int testTomorrows ()
public static int testDaySpans ()
public static void main (String [ ] args)
Checks that the simplest constructor doesn’t crash
Checks equals using basic constructor
Checks toString using basic constructor
Checks other constructors using toString and equals
Checks tomorrow and makeTomorrow using above
Checks recDaySpan and iterDaySpan using above
Calls all the other test routines
CS61B L11 Testing & Static (11) Garcia / Yelick Fall 2003 © UCB
Abstraction Helps in Testing• The previous slides assume tight abstraction:
– That no one can see inside and modify internals
• Public fields complicate the picture:– Include all values of public non-final fields in
creating the set of “all objects”– Include all public fields (final or not) in the set of all
“observers”
• Design principle: narrow interfaces:– Methods: keep the number of input parameters
small– Classes, avoid public fields, except when necessary– This simplifies testing (and use) of your code
CS61B L11 Testing & Static (12) Garcia / Yelick Fall 2003 © UCB
Testing repOk• repOk cannot be throughly tested outside
the class (can’t get false out) Use glass-box testing: IsalamicDate date1 = new IslamicDate(1,1);for (int month = 1; month <= 2; month++) { date1.myMonth = month; for (int day = -1; day <=32; day = (day==0 ? 30+month%2 : day+1)) { date1.myDay = day; showVerbose("Testing repOk on " + date1); if (date1.repOk()) { System.out.println(“Unexpected repOk->true ” + “for ” + date1); allPassed = false; } }} Note use of loops for generating tests
Avoid using same logic as in the other methods
CS61B L11 Testing & Static (13) Garcia / Yelick Fall 2003 © UCB
Aside: Using repOk• If systems (your program) are going to fail,
you want it to happen as soon as possible• Place repOk calls at then end of all
constructors and mutators using assert: assert <boolean> [ : <String>]
• Example: public IslamicDate(int month, int day) { myMonth = month; myDay = day; assert repOk() : "repOk failed on IslamicDate(int,int)"; }
CS61B L11 Testing & Static (14) Garcia / Yelick Fall 2003 © UCB
More on Assert• Assert only works with Java 1.4 and
beyond– New keyword “assert” causes problems for old code
• Compile with javac –source 1.4 <JavaFileName>
• Run with java –ea <ClassName>
• ea = enable assertions• No cost for assert statement when the are
not enabled• See more variations at:
http://java.sun.com/j2se/1.4.2/docs/guide/lang/assert.html
CS61B L11 Testing & Static (15) Garcia / Yelick Fall 2003 © UCB
StaticDefinitions:
– Static: associated with a class, not its objects– Non-static: associated with an object of a class
1. Non-static fields: instance variables– Each object in the class gets its own copy
2. Static fields: class variables – One variable for all the objects in the class
3. Non-static methods– Operate on an object– Can access “this” and instance variables
4. Static methods– Do not operate on an object
CS61B L11 Testing & Static (16) Garcia / Yelick Fall 2003 © UCB
Non-StaticAccount kathy =
new Account (100);
Account mike =
new Account (150);
mike.balance() 150
myBalancekathy
myBalancemike
100 150
During the call to balance on the Mike object, there is a “this” variable
this
CS61B L11 Testing & Static (17) Garcia / Yelick Fall 2003 © UCB
Static Field/Method Example• Seen static/final fields for constants• Add a non-final one (rare!) to the Account
classpublic class Account {
private static int ourCount = 0; private int myBalance;}
Add to all constructors
public Account (int bal) { myBalance = bal; ourCount++; }
Keep track of the number of Accounts that have been created
Add an observer
public static int getCount () { return ourCount; }
CS61B L11 Testing & Static (18) Garcia / Yelick Fall 2003 © UCB
Static
ourCount
0
Before any Account objects are created:
1
kathymyBalance
100
myBalancemike
150
Account kathy =
new Account (100);
2Account mike =
new Account (150);Account.getCount() 2
CS61B L11 Testing & Static (19) Garcia / Yelick Fall 2003 © UCB
StaticNon-static fields: instance variables
– Common case; accessed by “this” in non-static methods
1. Static fields: class variables – Shared by all objects; accessed as– classname.fieldname – object.fieldname (legal but not recommended)
2. Non-static methods– Common case; manipulate non-static fields (uses this)
3. Static methods– Do not operate on an object; no “this” or non-static
fields can be accessed
CS61B L11 Testing & Static (20) Garcia / Yelick Fall 2003 © UCB
PRS Question• Assume we have a Fraction containing public methods: public double toDouble ( ) { … } public static int gcd (int x, int y) { … } public boolean repOk () { … }
• And a separate FractionTester class with: Fraction f1 = new Fraction (…);• How many of the following will cause a CT or RT error?
1: Fraction.toDouble()2: Fraction.gcd(12,8)3: f1.toDouble()4: f1.gcd(12,8)5: assert (f1.toDouble() > 0);6: assert Fraction.repOk();7: assert f1.repOk();
0: none1: 2: 3: 4: 5: 6: 7: all
CS61B L11 Testing & Static (22) Garcia / Yelick Fall 2003 © UCB
PRS Answer• Assume we have a Fraction containing: public double toDouble ( ) { … } public static int gcd (int x, int y) { … } public boolean repOk () { … }
• And a separate FractionTester class with: Fraction f1 = new Fraction (…);• How many of the following will cause a CT or RT error?
1: Fraction.toDouble()2: Fraction.gcd(12,8)3: f1.toDouble()4: f1.gcd(12,8)5: assert (f1.toDouble() > 0);6: assert Fraction.repOk();7: assert f1.repOk();
0: none1: 2: 3: 4: 5: 6: 7: all
CS61B L11 Testing & Static (23) Garcia / Yelick Fall 2003 © UCB
Administrivia• Quiz
– Review session Saturday 2-4pm in 306 Soda– Not on the quiz: Exceptions, repOk, invariants,
lab3– For TRUE & FALSE questions, you will be graded
#right – #wrong
• No homework this week or next week– Start the project now!– Watch newgroup, errata.txt in proj1,
announcements
CS61B L11 Testing & Static (24) Garcia / Yelick Fall 2003 © UCB
Testing Classes• Testing in the ideal world
Use all “observer” methods in the class that return (or modify) a different type
Use constructors, plus mutators, and any method that returns the type to construct:
IslamicDate (all constructors)tomorrowmakeTomorrow
Set of all possible objects of a given type
equalstoStringrecDaysBetweeniterDaysBetweendayOfYear
Check that all of these return the expected value
All of this is done using the previous rules for checking normal, unusual, and boundary inputs
CS61B L11 Testing & Static (25) Garcia / Yelick Fall 2003 © UCB
Two More Categories of Testing
• Liskov distinguishes between– Modular testing
»One class at a time – Integration testing
»Testing multiple classes together»E.g., the entire program
• Integration testing is even harder!– The test cases are harder to design (too large)– Debugging at this level is also hard– Reduce the errors in this phase by doing more
modular testing.