black box (finite state machine) testing design for testability coverage measures random testing...
TRANSCRIPT
![Page 1: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/1.jpg)
• Black box (Finite State Machine) testing• Design for testability• Coverage measures• Random testing• Constraint-based testing• Debugging and test case minimization• Using model checkers for testing• Coverage revisited (“small model property”)
Topics in Testing We’ll Cover
![Page 2: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/2.jpg)
Testing Project• Start with: interface and some definitions• You– Implement (with intentional bugs)– Provide some (possibly VERY minimal) unit tests– Exchange programs with two other students– Submit a test plan for colleagues’ code– Present & defend test plan (bonus points)– Build a tester for your colleagues’ code
• Must include some random testing or SPIN testing by final checkin– Test! (and submit bug tickets)– Submit bug fixes, evaluations, mini-reports at milestones– Turn in final test report
![Page 3: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/3.jpg)
Testing Project• Your submitted code should contain two “subtle” bugs– Make sure your unit tests do not find the bug– Mail me: test cases and descriptions for the bugs• Subject MUST be: CS362W11 BUGS [onidid]
– You get to debug programs in lots of classes and in real life – in this class you get to bug a program intentionally, for once in your life
• I will see which testers find which bugs (over a sample of submitted programs)
![Page 4: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/4.jpg)
Testing Project• Grading criteria–Design/implementation project/tester– Effectiveness of the tester–Presentation of test plan (extra credit)–Quality of test/bug reports• Can I figure out how you tested the system?• Can I figure out what wasn’t tested?
– Coverage, plus information on input restrictions• Can I figure out how reliable you think the code
is? What are key strengths/weaknessess?–How “interesting” and hard-to-find your
intentional bugs are
![Page 5: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/5.jpg)
Testing Project
Expectations• You can program in C• You can figure out possibly poorly
written specifications (a key softwareengineering skill)
• Or know when to ask someone who can!• You can (learn to) use makefiles /
build system
“he who learns to play the harp learns to play by playing it”- Aristotle, Metaphysics, Book IX
![Page 6: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/6.jpg)
Testing ProjectOh, right.What are we implementing and testing?A card game
Well defined notion of correctness“Specifications” not intended for implementation,
but intended to be unambiguousGames are a good example of formal specifications:
no one wants to have to “make up” the rule or interpret ambiguity in the middle of playing a game
![Page 7: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/7.jpg)
Testing ProjectWhat are we implementing and testing?
Dominion (+ expansions)Card game published
by Rio Grande gamesPlayers start with
their own draw decksBuy cards to add to
decks (using cards)Some cards worth
points if in deck
Play thiscards &you candraw acard andgain twoactions(playing acard is anaction)
![Page 8: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/8.jpg)
Schedule• April 7: First code checkin– Initial dominion implementation, expected to perhaps
be buggy/incomplete; not frozen• April 21: Test plan due• April 28: BUGS DUE• May 5: First test report (brief) due• May 26: Second test report (brief) due• June 2: “Late Midterm”• June 7: FINAL TEST REPORT DUE, PROJECT
COMPLETED, FINAL CODE FREEZE, TESTER CODE DUE
![Page 9: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/9.jpg)
Grading & Other Admin StuffProject: 75%Late midterm: 15%(Possibly-in-class) quizzes/exercises/homework: 10%
My office hours: Tuesday, 10-11KEC 3067
TAs: Sheela Surisettysurisets @onid.oregonstate.eduOffice Hours: W 3-4
![Page 10: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/10.jpg)
Assignment 1• Get on beaversource (use your onid account)– http://beaversource.oregonstate.edu– Become my “friend”– Ask to join the class project (CS362S11)• I’ll show you how, now
![Page 11: Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization](https://reader035.vdocuments.us/reader035/viewer/2022070401/56649f1a5503460f94c2f928/html5/thumbnails/11.jpg)
Assignment 1• Using svn, check out the code in the dominion
directory– Compile it using make all– Look over the code, and the specifications (the rules
in the rules subdirectory)
• Due as of first class next week