ten things every software tester should know; european efforts in tester certification

Ten Things every Software Tester Should Know; European Efforts in

Tester Certification

Michael Kress

Associate Technical Fellow

Boeing

Chair – American Society for Quality Software Division

Ten Things every Software Tester Should Know; European Efforts in Tester Certification

• What world do you live in?• The gospel according to Bach* –

– This world? Or this?:– Complete specs Implicit specs– Quantifiable criteria Meaningful criteria– Protected schedule risk driven iterations– Early involvement good relationships– Zero defects Good enough– Complete test coverage enough information

– * James Bach 13 ICSQ, Dallas, Oct 6, 2003


• 1. Know the specified requirements– How many requirements are there?– Know when the requirements are mature and

complete.– Do Requirements progress tracking.

– How do you count and track requirements?– “The…system shall accept the user input, and compute…, and

store in locations 1,2 and 3, and send the output to….”



complete.– Do Requirements progress tracking.– How do you count and track requirements?– “The…system shall accept the user input, and compute…, and




complete.– Do Requirements progress tracking.– How do you count and track requirements?– “The…system shall accept the user input, and compute…, and


• Answer: 1 “shall” statement, 6 requirements


• 2. Know what the customer really wants– Walk in the user’s shoes– Know who ALL the stakeholders are.

• Customer• Regulatory authorities• Actual user• System maintainer• Help desk

– Customer does not want a ¼” drill; he wants a ¼” hole– Why are most milkshakes sold in the morning?– What do people want in a cell phone?


• 3. Know Stakeholder expectations– Customer is not always the ultimate user.– Management desires a secure system, but– Users may find security measures a nuisance

• Frequently changing passwords• Multiple passwords• 3 strikes and you’re out (wrong passwords)• Cumbersome encryption features

– Both customer and user features need attention


• 4. Know the interface requirements:

• “All interfaces shall be specified”

• “All interfaces shall be specified with inputs and outputs. Specifically for each external interface, the following should be specified: control data, parity checks, data correlation and constraints, frequency, priority, and timing. In addition, each input and output should include: source and destination, unit of measure, limits and ranges, scale factors, accuracy and precision, and frequency of arrival or delivery.”


• 5. Know your product’s capabilities– Not all features are created equal– What is most important to users?– What features are rarely used?– What causes lockup? Consequences; Recovery– Ship with known faults? Help desk/menus,


• 6. Know the language of testers

– Test strategies White/black box– Bottom up/top down Drivers/stubs– Input space partition test Error seeding– Mutation testing Structural coverage– Statement, decision condition coverage Test case explosion– Requirements based testing Error prediction– Test case design (boundary/mid range) Domain testing– Complexity (McCabe, Halstead) Function testing– Defect, bug, error, fault Clean room– External/internal faults Fault masking– Ad hoc testing Dead/deactivated code– Defect containment/leakage 6 sigma for test– Constructive Software QA Analytical QA– V-model (Boehm) Regression testing


• A source code statement reads: (IF A<=200 and B>=30 THEN C=1000. Which of the following sets of test case inputs would be selected to perform condition coverage for this statement?

• Test Case 1 Test Case 2

• a) A= 200, B= 30 A=199, B= 31

• b) A= 50, B= 15 A= 250, B= 45

• c) A= 75, B= 50 A= 150, B= 10

• d) A= 0, B= 0 A= 200, B= 30


• Answer b) is correct.

• To have condition coverage, each variable has to take on all possible conditions. Answer b) has test case inputs that cover the possible conditions. In answer a) the condition of variable A>200 and B<30 is not covered. In answers c) and d) the condition of variable A>200 is not covered.


• 7. Know the importance of Configuration Control

– What you start with may not be what you end with.

– What about early test versions?

– Where to begin retest?

– When to start over?

– Regression tests


• 8. Know your timetable

– Plan your tests– Track your progress– Module, H/S Integ, System V&V, Acceptance

tests


• 9. Know your budget

– Money spent vs. work accomplished– Earned Value Tracking

• Budgeted cost of work Scheduled (BCWS)

• Budgeted cost of work Performed (BCWP)


• 10. Know your goals

• How good is “good enough”?• Quantify your result

– No major stakeholder errors present– Will you satisfy ??? % of the market?

• Acceptance vs. “characterization tests• Are you ready to deliver?

ASQF - Arbeitskreis Software-Qualitat und FortbildiungEnglish - Association for Software-Quality and Training)

• ASQF is the European equivalent of the American Society for Quality – Software Division.

• • ASQF is the author, custodian administrator for a wide variety

of software quality events, initiatives, and European personnel certification programs.

• • Host for International Assessor Certification Scheme Board.• Organizer of the 5th SPICE Conference, Austria April 2005• Organizer of the 3rd World Congress for Software Quality

Munich, Sept 2005

ASQF - Arbeitskreis Software-Qualitat und FortbildiungEnglish - Association for Software-Quality and Training)

• Founder of International Software Testing Qualifications Board (ISTQB)*

• • International participants• Defined standardized Body of Knowledge• International Core curriculum• Accreditation Standards•• *A NOT-FOR-PROFIT ENTERPRISE; SANCTIONED AND

SPONSORED BY THE EUROPEAN SOFTWARE PROCESS IMPROVEMENT TRAINING INITIATIVE.

ASQF - Arbeitskreis Software-Qualitat und FortbildiungEnglish - Association for Software-Quality and Training

• The European Organization for Quality – Software Group is the authorizing “umbrella” organization for the International Software Testing Qualification Board

• Member nations:• • Sweden, Germany, Austria, Finland, Holland,

Switzerland, Poland


Vision • Ensuring professional quality and topicality of the scheme• Generation and definition of an international core syllabus for every qualification of the above scheme• Definition and maintenance of examination structure• and regulations• Definition and maintenance of accreditation guidelines• Definition and maintenance of certification guidelines• Definition and maintenance of the “ideal” pass rate• Monitoring compliance of the national boards to ISTQB• procedures and guidelines• Admission and expulsion of national boards into and• from ISTQB


• THE ASQ SOFTWARE DIVISION IS COLLABORATING WITH ASQF IN BRINGING THE TESTER CERTIFICATION PROGRAM ( OR ONE LIKE IT) TO THE UNITED STATES UNDER THE ENDORSEMENT OF THE ASQ CERTIFICATION AUTHORITIES.


• For more information:

• Dr. Bernd Hindel– [email protected]

• Robert Treffny– Robert [email protected]

ten things every software tester should know; european efforts in tester certification

Documents

track requirements

software tester

requirements progress

tester certification1

european efforts

tester certificationwhat

specified requirementshow

user input