For More Information

References

These references are related to the Construx Professional Software Tester Boot Camp semi-nar.

[Bach03b] James Bach, “Exploratory Testing Explained”, April 2003, at http://www.satisfice.com/articles/et-article.pdf

[Beizer90] Boris Beizer, Software Testing Techniques, 2nd Ed., Van Nostrand Reinhold, 1990

[Binder00] Robert Binder, Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison Wesley, 2000

[Boehm81] Barry Boehm, Software Engineering Economics, Prentice Hall, 1981

[Brilliant90] Susan S. Brilliant, John C. Knight, Nancy G. Leveson, “Analysis of Faults in an N-version Software Experiment”, IEEE Transactions on Software Engineering, V16, N 2, February, 1990

[Brownlie92] Robert Brownlie, et al. “Robust Testing of AT&T PMX/StarMAIL Using OATS, AT&T Technical Journal, Vol. 71, No. 3, May/June 1992, pp. 41-47.

[Buwalda04] Hans Buwalda, “Soap Opera Testing”, Better Software Magazine, February 2004, available at http://www.logigear.com/downloads/ You will have to register on the site and an email will be sent to you containing a link to the article.

[Carver01] Jeff Carver, “Improving Software Inspections by Using Reading Techniques” in A Quantitative Approach to Software Management and Engineering, available at www.cms.umd.edu/class/fall2001/cmsc735/index.html

[Chilenski94] John Chilenski, Steven Miller, "Applicability of Modified Condition/Decision Coverage to Software Testing", Software Engineering Journal, September, 1994

[Conte86] S D Conte, H E Dunsmore, V Y Shen, Software Engineering Metrics and Models, Benjamin/Cummings, 1986

[Fagan86] Michael Fagan, “Advances in Software Inspections”, IEEE Transactions on Soft-ware Engineering, Vol 12, No 7, July, 1986

[Frankel90] Eric Frankel, course notes from SE-516 Software Quality Assurance at Seattle University, Seattle, WA, 1990

[Gannsle98] Jack Ganssle, “Faster, Better Code”, in the Break Points section of The Embed-ded Report, Miller Freeman, August, 1998. At www.embedded.com/98/9808br.htm

[Gatlin04] Kang Su Gatlin, “The Trials and Tribulations of Debugging Concurrency”, ACM Queue, October 2004, pages 66-73

[Gause89] Donald Gause, Gerald Weinberg, Exploring Requirements: Quality Before De-sign, Dorset House, 1989

[Grady94] Robert B. Grady, “Successfully Applying Software Metrics”, IEEE Computer September 1994

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 1

For More Information

[Gries71] David Gries, Compiler Construction for Digital Computers, Wiley, 1971

[Hetzel88] Bill Hetzel, The Complete Guide to Software Testing, 2nd Ed., Wiley, 1988

[Hoffman94] Doug Hoffman, “So Little Time, So Many Cases”, available at http://www.cs.bsu.edu/homepages/dmz/cs639/So%20little%20time,%20so%20many%20cases.ppt

[Horgan94] Joseph R. Horgan, Saul London, and Michael R. Lyu, “Achieving Software Quality with Testing Coverage Measures”, IEEE Computer, September 1994, pages 60-69, 1994

[Horrocks99] Horrocks, Ian. Constructing the user interface with statecharts. Read-ing, MA: Addison-Wesley, 1999. [Jensen74] Kathleen Jensen, Nicklaus Wirth, Pascal User Manual and Report, 2nd Ed., Springer-Verlag, 1974

[Jones86] Capers Jones, Programmming Productivity, McGraw-Hill, 1986

[Jones96] Capers Jones, Applied Software Measurement, 2nd Ed., McGraw-Hill, 1996

[Kaner93] Cem Kaner, Jack Faulk, Hung Quoc Nguyen, Testing Computer Software, 2nd Ed., International Thompson Computer Press, 1993

[Kaner00a] Cem Kaner, “Architectures of Test Automation”, August 2000, available at http://www.kaner.com/testarch.html

[Kaner02a] Cem Kaner, James Bach, and Bret Pettichord, Lessons Learned in Software Testing: A Context Driven Approach, Wiley, 2002

[Kaner03a] Cem Kaner, “Cem Kaner on Scenario Testing”, Software Testing and Quality Engineering, September/October 2003, available at http://www.kaner.com/pdfs/ScenarioSTQE.pdf

[Kuhn02] Richard D. Kuhn, and Michael J Reilly, “An Investigation of the Applicability of Design Experiments to Software Testing,” 27th NASA/IEEE Software Engineering Work-shop, NASA Goddard Space Flight Center, 4-6 December 2002. Available at http://csrc.nist.gov/staff/kuhn/kuhn-reilly-02.pdf

[Larson75] R R Larson, “Test Plan and Test Case Inspection Specification”, IBM Corp., Tech. Report TR21.585, April 4, 1975

[McCabe76] T J McCabe, "A Complexity Measure", IEEE Transactions on Software Engi-neering, Vol 2 No 4, December, 1976

[McConnell98] Steve McConnell, seminar material for Software Project Survival, Construx Software, Bellevue, WA, 1998

[Meyer88] Bertrand Meyer, Object Oriented Software Construction, Prentice-Hall, 1988

[Mugridge05] Rick Mugridge and Ward Cunningham, Fit (Framework for Integrated Tests) for Developing Software, Prentice Hall, 2005

[Myers79] Glenford Myers, The Art of Software Testing, Wiley, 1979

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 2

For More Information

[Phadke89] Madhav S. Phadke, Quality Engineering Using Robust Design, Prentice Hall, 1989

[Phadke97] Madhav S. Phadke, Planning Efficient Software Tests, Crosstalk, October 1997, at http://www.stsc.hill.af.mil/crosstalk/1997/10/planning.asp

[Phadke03] Madhav S. Phadke, “Design Of Experiment for Software Testing”, January 2003, at http://www.isixsigma.com/library/content/c030106a.asp

[Pressman96] Roger Pressman, Software Engineering: A Practitioners Approach, 4th Ed, McGraw Hill, 1996

[Rapps82] S Rapps, E J Weyuker, "Data Flow Analysis Techniques for Test Data Selection", Sixth International Conference on Software Engineering, Tokyo, Japan, September, 1982

[Robertson06] Suzanne Robertson and James Robertson, Mastering the Requirements Proc-ess, 2nd Edition, Addison-Wesley, 2006

[RTCA92] ____, Software Considerations in Airborne Systems and Equipment Certification, Document RTCA/DO-178B, RTCA, Inc. December, 1992

[Rubin94] Jeffrey Rubin, Handbook of Usability Testing, Wiley, 1994

[Wallace01] Delores R. Wallace and D. Richard Kuhn, "Failure Modes in Medical Device Software: An Analysis of 15 years of Recall Data”, International Journal of Reliability, Qual-ity and Safety Engineering, Vol. 8, No. 4, 2001

[Weinberg71] Gerald Weinberg, The Psychology of Computer Programming, Van Nostrand, 1971

[Wiegers03] Wiegers, Karl E. Software requirements. 2nd edition. Redmond, Wash.: Microsoft Press, 2003.

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 3

For More Information

Additional Sources

[Andrews06] Mike Andrews and James A. Whittaker, How to Break Web Software, Addison Wesley, 2006

[Andrews] T. Andrews, S. Qadeer, S. K. Rajamani, J. Rehof, and Y. Xie, “Zing!”, available at: http://www.research.microsoft.com/zing

[Astels03] David Astels, Test-Driven Development: A Practical Guide, Prentice Hall PTR, 2003

[Austin96] Robert D Austin, Measuring and Managing Performance in Organizations, Dorset House Publishing, 1996

[Bach99a] James Bach, “A Low-Tech Testing Dashboard”, presentation at Star ’99 East, at http://www.satisfice.com/presentations/dashboard.pdf

[Bach99b] James Bach, “General Functionality and Stability Test Procedure”, document for testing the functionality and stability of a software application for the purpose of certifying it for Windows 2000, at http://www.satisfice.com/tools/procedure.pdf

[Bach99c] James Bach, “Heuristic Risk-Based Testing”, Software Testing and Quality Engi-neering November 1999, at http://www.satisfice.com/articles/hrbt.pdf

[Bach00] Jonathan Bach, “Session-Based Test Management”, Software Testing and Quality Engineering, November 2000, available at http://www.satisfice.com/articles/sbtm.pdf

[Bach01a] James Bach, “Boost Your Testing Superpowers”, presentation at Star ’99 East, at http://www.satisfice.com/articles/boost.shtml simple and cheap testing tools

[Bach01b] James Bach, “What is Exploratory Testing”, www.stickyminds.com column, at http://www.satisfice.com/articles/what_is_et.shtml

[Bach02] James Bach, Rapid Software Testing, course notes, Fall 2002, at http://www.testing-education.org/coursenotes/bach_james/cm_200204_rapidtesting/index.html

[Bach03a] James Bach, “Heuristic Test Strategy Model”, April 2003, at http://www.satisfice.com/tools/satisfice-tsm-4p.pdf

[Bach03b] James Bach, “Heuristics of Software Testability”, April 2003, at http://www.satisfice.com/tools/testable.pdf

[Bach03c] Jonathan Bach, “Testing in Session: A Method to Measure Exploratory Testing”, slides of a presentation to Washington Software Association QA SIG, May 13, 2003, avail-able at http://www.qasig.org/presentations/Session-Based%20Test%20Management.pdf

[Bach04] James Bach and P Schroeder, “Pairwise Testing: a Best Practice that Isn’t”, 22nd Annual Pacific Northwest Software Quality Conference, Portland, October 2004, at http://www.pnsqc.org/proceedings/pnsqc2004.pdf/

[Bach04] James Bach, “Reasons to Repeat Tests”, 2004, available at http://www.satisfice.com/repeatable.shtml

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 4

For More Information

[Beck02] Kent Beck and Erich Gamma, “Junit: A Cook’s Tour”, at http://junit.sourceforge.net/doc/cookstour/cookstour.htm

[Beck02] Kent Beck, “Simple Smalltalk Testing: With Patterns”, at http://www.xprogramming.com/testfram.htm

[Beck03] Kent Beck, Test-Driven Development, By Example , Addison Wesley, 2003, see also articles at http://www.junit.org/news/article/index.htm

[Beizer95] Boris Beizer, Black Box Testing , Wiley, 1995

[Black99] Rex Black, Managing the Testing Process, Microsoft Press, 1999

[Black04] Rex Black, Critical Testing Processes: Plan, Prepare, Perform, Perfect, Addison Wesley, 2004

[Boehm01] Barry Boehm and Victor R. Basili, “Software Defect Reduction Top 10 List”, IEEE Computer, January 2001, available at www.cs.umd.edu/projects/SoftEng/ESEG/papers/82.78.pdf

[Boehm04] Boehm, Barry and Richard Turner, 2004. Balancing Agility and Discipline: A Guide for the Perplexed, Boston, Mass.: Addison Wesley, 2004.

[Broekman03] Bart Broekman and Edwin Notenboom, Testing Embedded Software, Addi-son Wesley, 2003.

[Buwalda99] Hans Buwalda and Maartje Kasdorp, “Getting Automated Testing Under Con-trol”, Software Testing and Quality Engineering, November/December 1999, available at http://www.logigear.com/downloads/ You will have to register on the site and an email will be sent to you containing a link to the article.

[Buwalda02] Hans Buwalda, Dennis Janssen and Iris Pinkster, Integrated Test Design and Automation Using the Test Frame Method, Addison Wesley, 2002

[Bybro03] Mattias Bybro, “A Mutation Testing Tool for Java Programs”, Master’s Thesis, 2003, available at http://www.nada.kth.se/~karlm/a_mutation_testing_tool_for_java.pdf

[Cockburn00] Alistair Cockburn, Writing Effective Use Cases, Addison-Wesley, 2000.

[Cohen97] D. M. Cohen et al, “The AETG system: An Approach to Testing Based on Com-binatorial Design”, IEEE Transactions on Software Engineering, Vol. 23, No. 7, July 1997

[Copeland03] Lee Copeland, A Practitioner’s Guide to Software Test Design, Artech House Publishers, 2003

[Craig02] Rick D. Craig and Stefan P. Jaskiel, Systematic Software Testing, Artech House Publishers, 2002

[Culbertson02] Robert Culbertson, Chris Brown and Gary Cobb, Rapid Testing, Prentice Hall PTR, 2002

[Davis03] Noopur Davis and Julia Mullaney, “The Team Software ProcessSM (TSPSM) In Practice: A Summary of Recent Results”, SEI Technical Report CMU/SEI-2003-TR-014, September 2003, available at http://www.sei.cmu.edu/pub/documents/03.reports/pdf/03tr014.pdf

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 5

For More Information

[DeLano97] David DeLano and Linda Rising, “System Test Pattern Language“, 1997, at http://members.cox.net/risingl1/articles/systemtest.htm

[Dustin99] Elfriede Dustin, Jeff Rashka, and John Paul, Automated Software Testing: Intro-duction, Management and Performance, Addison Wesley, 1999

[English06] Ryan English, “What Lies Beneath: Hunt Down Security Vulnerabilities with Penetration Testing”, Better Software Magazine, May 2006, available at http://www.stickyminds.com/bettersoftware/downloads/BS%208_5%20Final%20Web.pdf (this link is to the whole magazine; the article is on page 26)

[Fagan76] Michael Fagan, “Design and Code Inspections to Reduce Errors in Program De-velopment”, IBM Systems Journal, Vol 15, No 3, 1976. Available at http://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf .

[Feathers02] Michael C. Feathers, “Working Effectively with Legacy Code”, available at http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf

[Feathers02] Michael C. Feathers, “The Self-Shunt Unit Testing Pattern”, May 2001, avail-able at http://www.objectmentor.com/resources/articles/SelfShunPtrn.pdf

[Feathers05] Michael C. Feathers, Working Effectively with Legacy Code, Prentice Hall, 2005

[Fewster99] Mark Fewster and Dorothy Graham, Software Test Automation, Addison-Wesley, 1999

[Gamma02] Erich Gamma, and Kent Beck, “Junit Test Infected: Programmers Love Writing Tests”, at http://junit.sourceforge.net/doc/testinfected/testing.htm

[Grady99] Grady, Robert B. 1999. “An Economic Release Decision Model: Insights into Software Project Management.” In Proceedings of the Applications of Software Measurement Conference, 227-239. Orange Park, FL: Software Quality Engineering.

[Hammell04] Thomas Hammell, with Russell Gold and Tom Snyder, “Getting Started with Test Driven Development”, JavaWorld December 2004, at http://www.javaworld.com/javaworld/jw-12-2004/jw-1206-tdd_p.html

[Havelund00] Klaus Havelund and Grigore Rosu, “Java PathExplorer – a Runtime Verifica-tion Tool”, 2000, an experimental tool for verifying Java programs. Developed by NASA Ames Research Center. Available at http://www.softwarequalitymethods.com/SQM/Papers/DarkerSIdeMetricsPaper.pdf

[Havelund04] Klaus Havelund and Grigore Rosu, “Java Path Explorer – A Runtime Verifi-cation Tool”, at http://webcourse.cs.technion.ac.il/236801/Winter2004-2005/ho/WCFiles/Java-Path-Explorer.pdf

[Hayes04] Linda Hayes, The Automated Testing Handbook, Software Testing Institute, 2004

[Hendrickson00] Elizabeth Hendrickson and Grant Larson, “Architecture Achilles Heel Analysis”, at http://www.testing.com/test-patterns/patterns/Architecture-Achilles-Heels-Analysis.pdf

[Hendrickson06] Elisabeth Hendrickson, “Rigorous Exploratory Testing”, April 19, 2006, at http://www.qualitytree.com/ruminate/041906.htm

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 6

For More Information

[Hoffman98] Douglas Hoffman, “A Taxonomy for Test Oracles”, Quality Week 1998, at http://www.softwarequalitymethods.com/Papers/OracleTax.pdf

[Hoffman99] Douglas Hoffman, “Heuristic Test Oracles”, Software Testing and Quality Engineering, March/April 1999, at http://www.softwarequalitymethods.com/Papers/STQE%20Heuristic.pdf

[Hoffman00a] Douglas Hoffman, “The Darker Side of Metrics”, 2000, at http://www.softwarequalitymethods.com/Papers/DarkMets%20Paper.pdf

[Hoffman00b] Douglas Hoffman, “Mutating Automated Tests”, 2000, at http://www.softwarequalitymethods.com/Papers/MutatingAutoTests.pdf

[Howard05] Michael Howard, David LeBlanc, and John Viega, 19 Deadly Sins of Software Security. McGraw-Hill, 2005.

[Humphrey91] Humphrey, Watts S., Terry R. Snyder, and Ronald R. Willis. 1991. “Soft-ware Process Improvement at Hughes Aircraft.” IEEE Software 8, no. 4 (July): 11–23.

[Humphrey00b] Watts Humphrey, The Personal Software ProcessSM (PSPSM), Software Engineering Institute, 2000, download at http://www.sei.cmu.edu/pub/documents/00.reports/pdf/00tr022.pdf

[Humphrey00c] Watts Humphrey, The Team Software ProcessSM (TSPSM), Software Engi-neering Institute, 2000, download at http://www.sei.cmu.edu/pub/documents/00.reports/pdf/00tr023.pdf

[Hunt03] Andrew Hunt and David Thomas, Pragmatic Unit Testing, In Java with JUnit, The Pragmatic Bookshelf, www.pragmaticprogrammer.com , 2003

[Hunt03] Andrew Hunt and David Thomas, Pragmatic Unit Testing, In C# with NUnit, The Pragmatic Bookshelf, www.pragmaticprogrammer.com , 2003

[Jones05a] Capers Jones, “Software Engineering: The State of the Art in 2005”, 2005, avail-able at http://www.compaid.com/caiInternet/casestudies/capers-stateofart2005.pdf

[Jones05b] Capers Jones, “The Impact of Poor Quality and Canceled Projects on the Soft-ware Labor Shortage”, 2005, available at http://www.compaid.com/caiInternet/casestudies/capers-waste05.pdf

[Kaner95] Cem Kaner, “Software Negligence and Testing Coverage”, 1995, available at http://www.kaner.com/pdfs/negligence_and_testing_coverage.pdf

[Kaner00b] Cem Kaner, “Rethinking Software Metrics”, Software Testing and Quality En-gineering March/April 2000, available at http://www.kaner.com/pdfs/rethinking_sw_metrics.pdf

[Kaner00c] Cem Kaner, “Measurement of the Extent of Testing”, Pacific Northwest Soft-ware Quality Conference 2000, available at http://www.pnsqc.org/proceedings/pnsqc00.pdf - the paper is at pages 108-144 and the slides at pages 145-172 in the proceedings document

[Kaner01] Cem Kaner, “Pattern: Scenario Testing”, online at Brian Marick’s web site , http://www.testing.com/test-patterns/patterns/pattern-scenario-testing-kaner.html .

[Kaner02a] Cem Kaner, James Bach, and Bret Pettichord, Lessons Learned in Software Testing: A Context Driven Approach, Wiley, 2002

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 7

For More Information

[Kaner02b] Cem Kaner, Black Box Software Testing (Professional Seminar), 2002, avail-able at http://www.testing-education.org/coursenotes/kaner_cem/cm_200204_blackboxtesting/index.html .

[Kaner03b] Cem Kaner, “What IS a Good Test Case?”, STAR East 2003, available at http://www.testingeducation.org/articles .

[Kaner04] Cem Kaner, Walter P Bond, and Pat McGee, “High Volume Test Automation”, Keynote address at STAR East 2004, slides available at http://www.kaner.com/pdfs/HVAT_STAR.pdf .

[Kaner05] Cem Kaner, James Bach, Black Box Software Testing, 2005. This course includes video lectures, slides, readings etc. Available at http://www.testing-education.org/BBST/index.html

[Kim00] Sunwoo Kim, John A. Clark, and John A. McDermid, “Class Mutation: Mutation Testing for Object Oriented Programs”, 2000, available at http://www-users.cs.york.ac.uk/~jac/papers/ClassMutation.pdf

[Kimberland04] Kelly Kimberland, “Microsoft’s Pilot of TSP Yields Dramatic Results”, February 2004, available at http://www.sei.cmu.edu/publications/news-at-sei/features/2004/2/feature-1-2004-2.htm

[Kit95] Edward Kit, Software Testing in the Real World, Addison-Wesley, 1995

[Kohl05] Jonathan Kohl, “Conventional Software Testing on a Scrum Team”, article on In-formit.com, September 30, 2005, at http://www.informit.com/articles/printerfriendly.asp?p=412981&rl=1 a professional tester joins a Scrum team

[Kohl06a] Jonathan Kohl, “Test Driven Development from a Conventional Software Testing Perspective, Part 1”, article on Informit.com, April 14, 2006, at http://www.informit.com/articles/printerfriendly.asp?p=462520&rl=1 a conventional tester with some programming skills pairs with a developer to learn TDD

[Kohl06b] Jonathan Kohl, “Test Driven Development from a Conventional Software Testing Perspective, Part 2”, article on Informit.com, April 21, 2006, at http://www.informit.com/articles/printerfriendly.asp?p=463938&rl=1 a conventional tester with some programming skills pairs with a developer to learn TDD

[Kohl06c] Jonathan Kohl, “Test Driven Development from a Conventional Software Testing Perspective, Part 3”, article on Informit.com, May 4, 2006, at http://www.informit.com/articles/printerfriendly.asp?p=466663&rl=1 a conventional tester with some programming skills pairs with a developer to learn TDD

[Kolawa99] Adam Kolawa, “Mutation Testing: A New Approach to Automatic Error-Detection”, 1999, at http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=2011

[Koomen99] Tim Koomen, Martin Pol, Test Process Improvement, Addison-Wesley, 1999

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 8

For More Information

[Koved03] Tim Koved, “SPADE and SABER: Improving Systems Through Error Reduc-tion”, talk for Microsoft, 2003, at http://research.microsoft.com/projects/SWSecInstitute/slides/koved.pdf

[Koziol94] Jack Koziol, David Litchfield, Dave Aitel, and Chris An, The Shellcoder's Hand-book: Discovering and Exploiting Security Holes, Wiley, 2004

[Lamport94] Leslie Lamport, “TLA - The Temporal Logic of Actions”, information avail-able at http://research.microsoft.com/users/lamport/tla/tla.html

[Larus04] James R Larus, Thomas Ball, Manuvir Das, Robert DeLine, Maneul Fahndrich, Jon Pincus, Sriram K Rajamani, and Ramanathan Venkatapathy, “Righting Software”, IEEE Software May/June 2004, pages 92-100

[Ledgard03] Josh Ledgard, “Software Testing 6: Good Tests for Bad Parameters”, at http://blogs.msdn.com/jledgard/archive/2003/11/03/53722.aspx

[Leffingwell97] Leffingwell, Dean, 1997. “Calculating the Return on Investment from More Effective Requirements Management,” American Programmer, 10(4):13-16.

[Lewis00] William E. Lewis, Software Testing and Continuous Quality Improvement, Auer-bach, 2000

[Li04] Kanglin Li and Mengqi Wu, Effective Software Test Automation: Developing an Automated Software Testing Tool, Sybex, 2004

[Li05] Kanglin Li and Mengqi Wu, Effective GUI Test Automation: Developing an Auto-mated GUI Testing Tool, Sybex, 2005

[Link02] Johannes Link, Unit Testing in Java, Morgan Freeman, 2002

[Long01] Johnny Long, Google Hacking for Penetration Testers, Syngress Publishers, 2001

[Loveland05] Scott Loveland, Geoffrey Miller, Richard Prewitt, Jr, Michael Shannon, Soft-ware Testing Techniques: Finding the Defects that Matter, Charles River Media, 2005

[McCaffrey06] James McCaffrey, “Create a Simple Mutation Testing System with the .NET Framework”, MSDN Magazine, April 2006. Available at http://msdn.microsoft.com/msdnmag/issues/06/04/MutationTesting/default.aspx

[MacKinnon01] Tim Mackinnon, Steve Freeman, Philip Craig, “Endo Testing: Unit Testing with Mock Objects”, in Extreme Programming eXamined, Addison Wesley, 2001, and at http://www.connextra.com/aboutUs/mockobjects.pdf

[McMahon06] Chris McMahon, “Old School Meets New Wave”, Better Software Magazine, June 2006, pages 28-32, (on testing middleware) at http://www.stickyminds.com/bettersoftware/docserver.asp?dt=digitalmagazine&ti=22

[Maguire93] Steve Maguire, Writing Solid Code, Microsoft Press, 1993 – not on testing per se, but on good coding techniques

[Mandl85] Robert Mandl, “Orthogonal Latin Squares: An Application of Experiment Design to Compiler Testing”, Communications of the ACM, Vol. 128, No. 10, October 1985, pp. 1054-1058.

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 9

For More Information

[Marick97a] Brian Marick, “How to Misuse Code Coverage”, 1997, available at http://www.testing.com/writings/coverage.pdf

[Marick97b] Brian Marick, “Classic Testing Mistakes”, presented at Star ‘97, available at http://www.testing.com/writings/classic/mistakes.pdf .

[Marick01] Brian Marick, “A Short Catalog of Test Ideas for …..”, at http://www.testing.com/writings/short-catalog.pdf

[Marick02] Brian Marick, “Bypassing the GUI”, STQE magazine, September/October 2002, pages 41-47. Available at http://www.testing.com/writings/bypassing-the-gui.pdf

[Mays90] R. G. Mays, C. L. Jones, G. J. Holloway, and D. P. Studinski, “Experiences With Defect Prevention”, IBM Systems Journal, Vol 29, No 1, 1990 http://www.research.ibm.com/journal/sj/291/ibmsj2901C.pdf

[Miller00] Barton P. Miller, David Koski, Chin Pheow Lee, Vivekananda Maganty, Ravi Murthy, Ajitkumar Natarajan, Jeff Steidl, “Fuzz Revisited: A Re-Examination of the Reliabil-ity of Unix Utilities and Services”, 2000. Available at http://www.opensource.org/advocacy/fuzz-revisited.pdf

[Moore02] Ivan Moore and Sebastian Palmer, “Making a Mockery”, in Proceedings of XP2002: 3rd International Conference on eXtreme Programming and Flexible Processes in Software Engineering. Available at http://ciclamino.dibe.unige.it/xp2002/atti/Moore-Palmer--MakingaMockery.pdf

[Mosley02] Daniel J Mosley and Bruce A. Posey, Just Enough Software Test Automation, Prentice Hall PTR, 2002

[Nagle04] Carl J Nagle, “Test Automation Frameworks”, available at http://www.safsdef.sourceforge.net/DataDrivenTestAutomationFrameworks.htm Also open source frameworks downloadable from http://safsdev.sourceforge.net/Default.htm .

[Neerumalla06] Bala Neerumalla, “New SQL Truncation Attacks And How To Avoid Them”, MSDN Magazine, November 2006, available at http://msdn.microsoft.com/msdnmag/issues/06/11/SQLSecurity/default.aspx

[Nguyen01] Hung Q. Nguyen, Bob Johnson, and Michael Hackett, Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems, Second Edition, Wiley, 2003

[Nyman04] Noel Nyman, “In Defense of Monkey Testing”, available at http://www.softtest.org/sigs/material/nnyman2.htm

[Offutt95] A. Jefferson Offutt, “A Practical System for Mutation Testing: Help for the Common Programmer”, Twelfth International Conference on Testing Computer Software, June 1995, available at http://ise.gmu.edu/~offutt/rsrch/papers/practical.pdf

[Offutt00] A. Jefferson Offutt and Roland H Untch, “Mutaiton 2000: Uniting the Orthogo-nal”, Mutation2000 Conference, October 2000, available at http://ise.gmu.edu/~offutt/rsrch/papers/mut00.pdf

[One00] Aleph One, “Smashing the Stack for Fun and Profit”, available at http://insecure.org/stf/smashstack.html .

[Perry95] William E. Perry, Effective Methods for Software Testing, Wiley, 1995

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 10

For More Information

[Pettichord01] Bret Pettichord, “Success with Test Automation”, 2001, at http://www.io.com/~wazmo/succpap.htm

[Pierce01] Bill Pierce, “Diagnose Common Runtime Problems with hprof”, JavaWorld, De-cember 2001, at http://www.javaworld.com/javaworld/jw-12-2001/jw-1207-hprof_p.html

[Reimer04] Darrell Reimer, Edith Schonberg, Kavitha Srinivas, Harini Srinivasan, Bowen Alpern, Robert D. Johnson, Aaron Kershenbaum, Larry Koved, “SABER: Smart Analysis-Based Error Reduction”, ISSTA ‘04, at ACM website with digital library subscription. See also talk on SABER by Larry Koved in web references section.

[Riersone01] Leanna Rierson, Kelly Hayhurst, and Dan Veerhusen, “Modified Condi-tion/Decision Coverage (MC/DC): An Interactive Video Teletraining Course”, FAA, May 2001, at http://www.javaworld.com/javaworld/jw-12-2001/jw-1207-hprof_p.html

[Robinson00] Harry Robinson, “Intelligent Test Automation”, Software Testing and Quality Engineering September/October 2000, and at http://www.geocities.com/model_based_testing/intelligent.pdf

[Robinson04a] Harry Robinson, “Things That Find Bugs in the Night”, original article posted on StickyMinds.com, at http://www.stickyminds.com/pop_print.asp?Objectid=7331&ObjectType=COL

[Robinson04b] Harry Robinson, “Obstacles and Opportunities for model-based testing in an industrial software environment”, as a text document at http://www.geocities.com/harry_robinson_testing/ObstaclesAndOpportunities.pdf

and as PowerPoint slides at

http://www.geocities.com/harry_robinson_testing/ECMDSE_Robinson.pdf

[Robinson05] Harry Robinson, “Model Based Testing”, slides from tutorial at Star East 2005 at http://us.share.geocities.com/harry_robinson_testing/stareast_2005_mbt_tutorial.ppt#256,1,Model-BasedTesting

[Santos06] Pablo Santos and Francisco J. Garcia, “Distributed Unit Testing”, Dr Dobbs Por-tal, October 2006, on an extension to NUnit to support distributed unit testing, at http://www.ddj.com/dept/debug/193104810;jsessionid=5UUMFWO45ODMAQSNDLOSKHSCJUNN2JVN?_requestid=613571 and link to the source code at pnunit.codicesoftware.com

[Schneider00] Andy Schneider, “JUnit Best Practices”, JavaWorld December 2000, at http://www.javaworld.com/javaworld/jw-12-2000/jw-1221-junit_p.html

[Shore04] Jim Shore, “Fail Fast”, IEEE Software, September/October 2004, at http://martinfowler.com/ieeeSoftware/failFast.pdf on assertions and using them to fail on null values, etc.

[Shull02a] Shull, et al, 2002. “What We Have Learned About Fighting Defects,” Proceed-ings, Metrics 2002. IEEE; pp. 249-258.

[Shull02b] Shull, Forrest and Roseanne Tesoriero, 2002. “What We Have Learned About Fighting Defects, Results of the METRICS02 workshop”, available at CeBASE http://www.cebase.org/www/frames.html?/www/researchActivities/defectReduction/non-eWorkshop/what_we_have_learned_about_fight.asp .

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 11

For More Information

[Simmons00] Erik Simmons, “When Will We Be Done Testing? Software Defect Arrival Modeling Using the Weibull Distribution”, Pacific Northwest Software Quality Conference, 2000 at http://www.pnsqc.org/proceedings/pnsqc00.pdf - the paper is at pages 194-210 and the slides at pages 211-243 in the proceedings document

[Slutz98] Don Slutz, “Massive Stochastic Testing of SQL”, Proceedings of the Very Large Database Conference 1998, at http://www.vldb.org/conf/1998/p618.pdf

[Spec#] Microsoft Research, “SpecSharp (or Spec#)” , information at http://research.microsoft.com/specsharp

[Spin] ACM, “On-The-Fly, LTL Model Checking with SPIN”, information at http://spinroot.com/spin/whatispin.html

[SPMN98a] Software Program Managers Network, The Little Book of Testing, Volume I, Overview and Best Practices, Software Program Managers Network, 1998. Downloadable from the SPMN website, http://www.spmn.com/products_guidebooks.html

[SPMN98b] Software Program Managers Network, The Little Book of Testing, Volume II, Implementation Techniques, Software Program Managers Network, 1998. Downloadable from the SPMN website, http://www.spmn.com/products_guidebooks.html

[Spuler94] David A. Spuler, C++ and C Debugging, Testing and Reliability, Prentice Hall, 1994

[Stobie05] Keith Stobie, “Too Darned Big to Test”, ACM Queue, February 2005, pages 30-37.

[Thevenod-Fosse93] Pascale Thevenod-Fosse and Helene Waeselynk, “STATEMATE Ap-plied to Statistical Software Testing”, ACM ISSTA (International Symposium on Software Testing and Analysis, 1993, pages 99-109. (Available in the ACM Digital Library if you subscribe)

[Thomas02] Dave Thomas and Andy Hunt, “Learning to Love Unit Testing”, STQE maga-zine, January/February 2002, pages 32-47. Available at http://www.pragmaticprogrammer.com/articles/stqe-01-2002.pdf

[Thomas02] Dave Thomas and Andy Hunt, “Mock Objects”, IEEE Software, May/June 2002, pages 22-24. Available at http://www.pragmaticprogrammer.com/articles/may_02_mock.pdf

[UKSMA00] United Kingdom Software Metrics Association, “Quality Standards Defect Measurement Manual, Release 1.a”, October 1000. at http://www.uksma.co.uk/public/defstan1a.pdf

[VanDeursen01] Arie van Deursen, Leon Moonen, Alex van den Bergh, and Gerard Kok “Refactoring Test Code”. at : http://homepages.cwi.nl/~arie/papers/xp2001.pdf

[VanDoren00] Edmond VanDoren, “Cyclomatic Complexity”. Article on SEI website at : http://www.sei.cmu.edu/str/descriptions/cyclomatic_body.html

[Whittaker03a] James A Whittaker, How to Break Software, Addison-Wesley, 2003

[Whittaker03b] James A Whittaker and Herbert H Thompson, How to Break Software Secu-rity, Addison-Wesley, 2003

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 12

For More Information

[Williams04] Yuan Laurie Williams, “Mutation Testing”, 2004, six powerpoint slides, at http://agile.csc.ncsu.edu/testing/MutationTesting.pdf

[Willis98] Willis, Ron R., et al, 1998. “Hughes Aircraft’s Widespread Deployment of a Con-tinuously Improving Software Process,” Software Engineering Institute/Carnegie Mellon University, CMU/SEI-98-TR-006, May 1998. available at http://www.sei.cmu.edu/pub/documents/98.reports/pdf/98tr006.pdf

[Yu04] Yuan Yu and Tom Rodeheffer, “RaceTrack: Detecting Potential Races in Managed Code”, 2004, at http://research.microsoft.com/research/sv/racetrack/

[Zeller02] Andreas Zeller and Ralf Hildebrandt, “Simplifying and Isolating Failure–Inducing Input”, IEEE Transactions on Software Engineering, Vol 28, No 2, February 2002, at http://www.st.cs.uni-sb.de/papers/tse2002/

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 13

For More Information

Organizations

Quality Assurance Forum, 17 St Catherine’s Road, Ruislip Middlesex HA4 7RX, UK

American Society for Quality Control (ASQC), 611 East Wisconsin Avenue, Milwaukee, WI, 53202

IEEE Computer Society, PO Box 80452, Worldway Postal Center, Los Angeles, CA 980080

ANSI/IEEE Std 829-1998 Software Test Documentation

ANSI/IEEE Std 1008-1987 Software Unit Testing

ANSI/IEEE Std 1012-1986 Software Verification & Validation Plans

available through IEEE Standards Sales in New Jersey (201) 981-0060

IEEE International Test Conference (ITC)

IEEE European Design and Test Conference (ED&TC)

Software Quality Association (South Australia) Inc, http://www.sqa.asn.au

Journal of Software Testing, Verification and Reliability (Wiley Interscience)

Washington Software Association QA SIG www.qasig.org

Web Application Security Consortium http://www.webappsec.org/ an international group who produce best-practice security standards for the World Wide Web.

Open Web Application Security Project (OWASP) http://www.owasp.org/index.jsp is dedi-cated to finding and fighting the causes of insecure software.

Pacific Northwest Software Quality Conference, usually in October in Portland (http://www.pnsqc.org)

Seattle Area Software Quality Assurance Group (www.sasqag.org) has monthly free meetings on fourth Thursdays at Construx in Bellevue, WA. Quarterly $99 training days are held lo-cally in Puget Sound area. Prior talks are stored on website.

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 14

For More Information

Interesting Web Sites

http://www.Construx.com

Here are the general sites for testing informatiton, testing gurus, and forums. www.qaforums.com - Software Testing and Quality Assurance discussions site www.stickyminds.com

Site for software test managers, testers, and QA professionals to gather information and provide resources for one another – website attached to Better Software Magazine

Better Software Magazine – can sign up for a free subscription at www.BetterSoftware.com/APFLBL http://www.sqa-test.com/toolpage.html http://www.softwareqatest.com/ - information on automated testing tools http://www.testingfaqs.org/

home page for access to test tools lists in many categories – GUI test drivers, unit test tools, static analysis tools, test design tools and many others

www.badsoftware.com – site hosted by Cem Kaner and David Pels www.compinfo-center.com/tpsw12-t.htm info on software testing and links to other sites www.csc.liv.ac.uk/~mrw SW Testing Teacher’s page. Goofy picture but has useful links www.faqs.org/faqs/software-eng/testing-faq FAQ’s about testing www.grove.co.uk/Site_Links.html Software Testing Links www.io.com/~wazmo/qa.html

Brett Pettichord has put together a great list of links to articles and sites about SW Testing

www.jamesbach.com Information about testing methodologies and more www.kaner.com Cem’s writings, courses, and links to his other sites www.mccabe.com McCabe and Associates – QA consulting firm with products and processes www.sqatester.com New site with testing info, tester idea exchange areas, job postings and more www.testingstuff.com – extensive collection of testing resources www.sqa-test.com

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 15

For More Information

Automated Testing Specialists – great links to articles on test automation, SW testing sites, and tools

http://www.testing.com/ Brian Marick’s testing site http://www.csst-technologies.com/hplinks.htm

- software testing related links page

Testdrivendevelopment-subscribe@yahoogroups.com

Test Driven Development mailing list

www.javaworld.com/channel_content/jw-testing-index.shtml

JavaWorld.com’s Testing Article Listing page:

Software Testing and Related Magazines Software Testing Journal “Software Testing Verification and Reliability” from www.interscience.wiley.com/ipages/0960-0833www.soft.com Software Research, Inc has Testing Techniques Newsletter (TNN Online)

Testing Techniques Newsletter, On-Line Edition (TTN-Online)

http://www.soft.com Email: ttn@soft.com

To request your free subscription or propose any type of article send Email to "ttn@soft.com". TO SUBSCRIBE: Send Email to "ttn@soft.com" and include in the body of your letter the phrase "subscribe ".

www.softwaremag.com Online software magazine – has industry news Better Software Magazine – website is www.Stickyminds.com – see above Test Patterns Software Testing Patterns page on Brian Marick’s website – has links to further sites http://www.testing.com/test-patterns/patterns/ Testing Tools sites www.opensourcetesting.org – site that lists open source testing tools members.fotunecity.com/mailz/tester.html – testing tool for creating, printing and running tests www.assess.com

Assessment Systems Corporation has books, software and various automated testing tools

www.autotestco.com/html/index.thm introducing automated tools to your team www.csst-technologies.com

CSST technologies provides products and services for testing client-server applications

www.ddj.com

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 16

For More Information

Dr Dobb’s website of software tools www.ict.co.uk/radstar1.cfm methodology plus tools www.iplbath.comp20.htm IPL Software Testing Products Library www.optimizeit.com

Offers OptimizeIt, a performance testing and enhancement tool for Java and JavaBeans

www.rational.com Rational Software’s site. Info about Software testing tools (now owned by IBM) www.segue.com Segue offers awide range of testing tools and related services www.soft.com

Software Research, Inc offers testing tools, including capture/playback, test management, code coverage, and source-code analysis

www.sqa-test.com/toolpage.html links to a number of test tool companies. Excellent site www.stellarlogic.com/SLChome.asp Stellar Logic Corporation provides tools, services and information www.testcompress.com information on McCabe TestCompress automated testing software www.webmastersolutions.com load testing and website monitoring services www.fraps.com Fraps is a universal Windows application that can be used with all games using DirectX or OpenGL technology. In its current form Fraps performs many tasks and can best be described as:

Benchmarking Software - See how many Frames Per Second (FPS) you are getting in a corner of your screen

Screen Capture Software - Take a screenshot with the press of a key! Realtime Video Capture Software - Fraps can capture audio and video up to

1152x864 and 100 frames per second! http://www.sasqag.org/pastmeetings/19%20Jan%202006%20d.pdf presentation called ‘Load/Performance Type Testing Tools at a Price You Can Afford’ by Cordell Vail and Joe Towns. They work at an organization without a lot of money, and searched for tools that cost less and found one that worked for them. This is a recording with audio of the presentation. Software Testing and Related Organizations http://hissa.nist.gov/

National Institute of Standards and Technology, Software Quality Group. Articles on Software Quality. Links to related sites

http://www.nist.gov/director/prog-ofc/report02-3.pdf "The Economic Impacts of Inadequate Infrastructure for Software Testing" from NIST

www.center.org Software Development Forum’s center for information, connection and education www.ondaweb.com/sti

Software Testing Iinstitute (STI). Articles and book suggestions for testers. Industry and profession overview. Also has discussion forum.

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 17

For More Information

www.icstest.comICSTEST International Conference on Software Testing is an annual event that is a forum for presentations, tutorials, discussions, and exchange of experience on software testing

www.qaiusa.comQuality Assurance Institute site has info on SW Testing, consulting, education, assessments and certification programs

www.sasqag.org Seattle Area Software Quality Assurance Group (SASQAG) has links, membership info, certification info, and past and future meeting info

www.siia.net Software and Information Industry Assocation has info on conferences, etc. www.softwareqatest.com Software QA/Test Resource Center has FAQ’s resources lists tools, etc. www.sqe.com/stareast/index.html Tester conference site - STAR – Software Testing Analysis and Review www.ssq.org Society for Software Quality www.stagroup.com

STA group offers excellent classes on software testing and automation. Based in the Seattle area.

www.stqe.net A resource for forums, publications, book reviwes and other informaiton about software testing

www.testingtraining.com Software Testing Center offers training, including online training. Based in California.

www.wsa1.org Washington Software Alliance provides resources for WA software industry. Has regular meetings. Hosts testing SIG – its website is www.qasig.org

Links to useful freeware, shareware, and cheapware programs for testing:

http://www.zdnet.com

www.tucows.com

www.shareware.com

www.pcmagazine.com

www.cnet.com

www.qadownloads.com

www.softpanorama.org

http://www.xprogramming.com/software.htm links to over 82 unit testing frameworks librar-ies for different languages

Web Testing and Related Sites

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 18

For More Information

Bad Web Sites

www.entropy8.com

This company is actually in the business of building web sites!

www.websitesthatsuck.com

featuring really bad websites

www.worstoftheweb.com

links to bad websites; but I think they’re mainly objecting to the content

http://hebb.cis.uoguelph.ca

this site is ugly. Try going into Deb Stacey’s page

Web Tools IEHttpHeaders tool, which help uncover what is being sent between pages. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker. http://www.blunck.infno/iehttpheaders.html

Paros http://www.parosproxy.org/225235.html helps uncover what is being sent between pages. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker

SPIKE Proxy http://linux.softpedia.com/get/Internet/Proxy/SPIKE-Proxy-10461.shtml tests parameter manipulation and CGI buffer overflow. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker.

SSLDigger is available on the Foundstone website http://foundstone.com/ – go to resources, then free tools. It allows you to test an SSL-enabled web server to determine which encryption algorithms it supports. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker.

Wget is included with most Linux and BSD distributions. It’s a simple yet powerful com-mand-line tool for accessing, downloading, or mirroring Web server content

cURL http://curl.haxx.se/ , also http://curl.haxx.se/libcurl is a command line tool that is also a pen tester. It has similar functionality to Wget.

Blackwidow – http://softbytelabs.com/Frames.html a web spider or crawler tool. 30-day free trial is available, tool costs 39.95 after that.

Cygwin, http://www.cygwin.com which is a Unix environment for Windows. Pro-vides, for example, the grep utility on a Windows system. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker.

The Regulator – http://regex.osherove.com/ helps create search expressions for grep. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker.

FITScanner is available on the CD that comes with the book How To Break Software Secu-rity, by James Whittaker and Herbert Thompson

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 19

For More Information

Nikto, http://www.cirt.net/code/nikto.shtml a tool which helps to find known vulner-abilities in a web server.

Wikto http://www.sensepost.com/research/wikto adds to Nikto the Google Hacking Database GHDB and using the Google search engine to case your client. The database is at http://johnny.ihackstuff.com

Stunnel http://stunnel.org allows you to set up a tunnel to a machine using Secure Sockets Layer. Stunnel is the “Universal SSL Wrapper” – it can be both a server and a client

IISLockdown, http://www.microsoft.com/technet/security/tools/locktool.mspx a tool for locking down servers. Also on the CD that comes with the book, How To Break Web Software, by Mike Andrews and James Whittaker.

TextPad – http://www.textpad.com/products/index.html and http://www.textpad.com/add-ons/syna2g.html basic product isn’t free, add ons are free

A useful text editor which can display and edit almost any file, and you can get free syntax definition files, so that TextPad so it appropriately highlights and indents documents (like Perl programs)

Cookie Pal – http://www.kburra.com/cpal.html allows users more fine grained control over what cookies they will accept or reject

Cookie Crusher - http://www.thelimitsoft.com/cookie/ allows users more fine grained con-trol over what cookies they will accept or reject

http://www.securityspace.com/s_survey/data/man.200507/cookieReport.html

http://www.dutchduck.com/faq/faqs.aspx link to FAQ pages on cookies

http://www.across.si/papers/session_fixation.pdf paper on session fixation

BBCode http://en.wikipedia.org/wiki/BBCode

Examples of things to filter for http://ha.ckers.org/xss.html

http://www.ngssoftware.com/papers/advanced_sql_injection.pdf for more information on SQL injection techniques

chroot command for Apache servers http://www.linux.com/article.pl?sid=04/05/24/1450203

buffer overflows

“Smashing the Stack for Fun and Profit”, available at http://insecure.org/stf/smashstack.html

http://www/securityfocus.com/archive/1/317142/2003-03-28/2003-04-03/0

http://blogs.msdn.com/michael_howard/

http://msdn.microsoft.com/security/securecode/columns/default.aspx

UTF-8 encoding

http://en.wikipedia.org/wiki/UTF-8

http://www.unicode.org/standard/standard.html

encoder/decoder – Napkin

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 20

For More Information

http://www.0x90.org/releases/napkin/

RainForrestPuppy, a pioneer of Web application security testing

http://www.wiretrip.net/rfp/

checklist for locking down an application and Microsoft SQL Server

http://www.securitymap.net/sdm/docs/windows/mssql-checklist.html

Ethereal (a network monitoring tool) http://www.ethereal.com/

J0hnny (of Google hacking fame http://johnny.ihackstuff.com/index.php?module=prodreviews

HTTPrint identifies web server and version by differences in responses to requests http://net-square.com/httprint/

SiteDigger from Foundstone http://www.foundstone.com/resources/proddesc/sitedigger.htm executes Google searches to see if your site is vulnerable to known Web server bugs

BugTraq site that lists security vulnerabilites of web servers www.securityfocus.com

CERT site that lists security vulnerabilites of web servers www.cert.org

Brutus www.hoobie.net/brutus/brutus-download.html a tool for brute force hacking of authentication

Information on Cross-Site Tracing http://www.cgisecurity.com/whitehat-mirror/WH-WhitePaper_XST_ebook.pdf

Information on modifying an Apache server to remove weak ciphers

http://httpd.apache.org/docs/2.0/ssl/ssl_howto.html

mod-ssl http://wwww.modssl.org/docs/2.8/

Information on modifying an IIS server to remove weak ciphers

http://support.microsoft.com/?kbid=245030

www.msw.com.au

they sell various Web tools, including SiteMapper, a program that maps web sites, and SubmitWolfPRO, a Web site submission tool

www.tali.com

HTML Power Tools for Windows

www.webmasterfree.com

freeware tools for the Web, and news

www.xmlspy.com

XML Spy is an XML editor. Free trial version available

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 21

For More Information

HTML Validation and Link Checking Sites / Software

http://cq-pan-cqu.edu.au/validate

Location of Web Techs, a free online HTML validator

http://html.about.com/cs/linkverifiers/index.thm

a link to link verifiers

www.validator.w3.org

location of an HTML validator by W3C

www.arealvalidator.com

location of A Real Validator, HTML validation software with a 30 day trial version

www.htmlvalidator.com

free download of CSE Validator Lite, an HTML validator

Game of Life: http://hensel.lifepatterns.net/ website for downloadable versions of the Game of Life used in lab

MC/DC

http://www.validatedsoftware.com/code_coverage_tools.html link to site listing coverage tools for use with RTCA DO-178B testing requirements – i.e. these tools can do MC/DC coverage

Testing checklists

See ‘Common Software Errors’ in Testing Computer Software by Cem Kaner et al (Appendix A – 74 pages!)

Attacks from How To Break Software by James Whittaker (see handout)

Test Catalog by Brian Marick from Craft of Software Testing, available at : www.testing.com/writings/short-catalog.pdf (see handout)

Josh Ledgard’s group’s listing of bugs to look for: http://blogs.msdn.com/jledgard/archive/2003/11/03/53722.aspx (see handout)

From James Bach – it’s titled “Heuristic Test Strategy Model”, but it lists areas to think about when designing testing:

http://www.satisfice.com/tools/satisfice-tsm-4p.pdf

Coverage A short document on coverage, mostly white box, with more types than we cover in the seminar

http://www.bullseye.com/coverage.html

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 22

For More Information

NCover - A free coverage tool for the .NET environment – does statement coverage only: http://ncover.org/site/

Test coverage for Java

Clover http://www.thecortex.net/clover/ Clover is a commercial application that is free for noncommercial activities

JCover http://www.codework.com/JCover/product.html

Simian: this tool does similarity analysis in almost any text file, finding duplications of code http://www.redhillconsulting.com.au/products/simian/

Vil – does code metrics in the .NET environment http://www.1bot.com/

Keith Stobie’s talk at WSA QA SIG September 2005 ‘It’s Too Darn Big To Test’

http://www.qasig.org/presentations/BigSysTestWSAv3.pdf

FIT Information

FIT website http://fit.c2.com Documentation on using FIT is here, also example source code

You also need the FitLibrary from http://sourceforge.net/projects/fitlibrary and POI from http://jakarta.apache.org/poi

FitNesse is at www.fitnesse.org . FitNesse runs on a web server, which makes it easy to share Fit test tables among many people working on a project. Chapter 27 in the book on Fit discusses FitNesse.

Talk at NetObjectives on Lean-Agile System Testing, January 2007, includes slides on FitNesse at http://www.netobjectives.com/events/download/latesting0701_ppt.pdf

Two free sources of Combinatorial Testing tools, and one commercial source:

Jenny will do pairs, triplets, etc. You say what you want with parameters. Written by Bob Jenkins (free, open source, public domain). It covers all n-tuples of features and supports restrictions. It can extend an existing test suite. It always uses pseudorandom methods to pro-duce test cases. 20 dimensions of 10 features each, all pairs, requires 195 test cases. It’s writ-ten in C. http://burtleburtle.net/bob/math/jenny.html

AllPairs by James Bach. It’s written in PERL. (free, open source, GPL). It can only cover all pairs of features. It doesn't support any restrictions. It takes as input a tab-delimited table listing the actual parameter values of the attributes you want to test. The output is a table suitable for dumping into Excel listing the test cases, parameter value by parameter value. It also produces an index of pairs saying which test cases cover each pair. 20 dimensions of 10 features each, all pairs, requires 230 testcases. http://www.satisfice.com/tools/pairs.zip

Ward Cunningham provides further discussion and the source code of a Java program to gen-erate all pairs combinations at http://fit.c2.com/wiki.cgi?AllPairs

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 23

For More Information

There is a listing of available tools at http://www.pairwise.org/tools.asp

and some references to effectiveness of pairwise at http://www.pairwise.org/results.asp

AETG from Telcordia (commercial, $6000 for two seats for a year). It's web-based. It can cover all pairs (or triples or arbitrary n-tuples) of features. It supports restrictions, disallowing certain feature combinations. It can extend an existing test suite. It can often use deterministic methods (as opposed to pseudorandom) to generate test cases. 20 dimensions of 10 features each, all pairs, requires 180 test cases. http://aetgweb.argreenhouse.com/

Orthogonal Arrays are another way to do all pairs test cases

Here’s a website with a comprehensive catalog of orthogonal arrays: http://www.research.att.com/~njas/oadir/index.html

and this company sells tools that will generate orthogonal arrays

http://www.phadkeassociates.com

Model Based Testing

Model-based testing website: www.model-based-testing.org

Papers on model based testing: http://www.geocities.com/model_based_testing/online_papers.htm

http://www.geocities.com/harry_robinson_testing/ObstaclesAndOpportunities.pdf

Mutation Testing Mutation Testing references and tools

http://ise.gmu.edu/~offutt/mujava/ MuJava, a mutation testing tool for Java

http://ise.gmu.edu/~offutt/rsrch/mut.html about Mothra, a mutation testing tool for For-tran. It’s available free for research and educational use but not for commercial use. The page also links to papers on the theory of mutation testing.

Site listing mutation testing tools: http://www.xpdeveloper.com/xpdwiki/Wiki.jsp?page=MutationTestingTools

Mutation testing for Java

Jester http://jester.sourceforge.net Jester performs random mutations on the source code being tested; it then verifies if your tests still pass.

Tools for recording what happens on the screen

Good for recording when you’re doing exploratory testing, in case you don’t exactly remember the set of steps that led to a problem showing up

Not free, not expensive: Camtasia www.techsmith.com

Free: CamStudio www.camstudio.org

5 -Professional Software Testing Boot Camp References.doc ( (02/15/07) Page 24