“the future of software testing” iso 29119: the new ...€¦ · iso 29119: the new...
Post on 22-Jun-2020
27 Views
Preview:
TRANSCRIPT
ISO 29119: The New International Software
Testing StandardStuart Reid,
Testing Solutions Group,UK
Europe’s Premier Software Testing EventWorld Forum Convention Centre, The Hague, Netherlands
WWW.QUALTECHCONFERENCES.COM
“The Future of Software Testing”
ISO/IEC 29119:
The New International Software Testing Standard
Stuart Reid
sreid@testing-solutions.com
Testing Solutions Groupwww.testing-solutions.com
EuroSTAR 2008 Workshop
Workshop Goals
• Introduce ISO 29119
• Delegate groups to:
– Review the overall scope & structure
– Review and debug the test processes
– Brainstorm the test techniques (part 4)
• What techniques do we want in it?
– Define the ongoing review process
Who Needs Standards?
What are standards?
• “Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationally recognized industrial, professional, trade associations or governmental bodies”
- ISO
– Guidelines documents as they are not compulsory unless mandated by an individual or an organization
– Agreements because they often reflect a certain level of consensus
Motivation for ISO 29119
• Lacking in current standards:
– Organizational Testing not covered
• e.g. Test Policy and Organizational Test Strategy
– no Project Test Management
– BS 7925 only covers unit testing
– common functional techniques missing
– poor coverage of non-functional testing
• Conflicts in definitions, processes & procedures
– plethora of standards to be replaced by one
• e.g. IEEE 829, IEEE 1008, BS 7925-1/-2, IEEE 1028
– practitioners do not know which standard to follow
Documentation
Part 3
TestingTechniques
Part 4
Processes
Part 2
Concepts & Vocabulary
Part 1
BS 7925-2 IEEE 829IEEE 1008
BS 7925-1
BS 7925-2
ISO 29119 – Scope & Structure
May 08
May 09
May 10
May 11
May 12
…
ISO 29119 - Timeline
Working Draft (WD)Committee Draft (CD)Final Committee Draft (FCD)Final Draft International Standard (FDIS)Final International Standard (FIS)
Parts 2 & 3
Parts 1 & 4
WD
CD
FCD
FDIS
FIS
WD
CD
FCD
FDIS
FIS
Organizational Test Strategy Process
Project Test Management Process
Test Level Process
Organizational Test Policy Process
Test Execution Cycle
Organizational Test Policy
Organizational Test Strategy
Strategise, Plan and Manage Test Project
Strategise, Plan and Manage Test Level
Design & Execute Tests
Manage Test Environment
Report Defects
Organizational Test Processes
Organizational Test Strategy Process
Create/Maintain Test Strategy
Test Policy
IT ProjectManagement Policy
Test Strategy
Organizational Test Policy Process
Create/Maintain Test PolicyIT Policy
Management PolicyTest Policy
Test Policy Test Policy Updates
Project Test Management Process
OrganizationalTest Strategy
Comments on Organizational Test Strategy usage
Manage Conformance
with Test Policy(TP4)
Organizational Test Policy
[Test Policy not working effectively]
[Test Policy working effectively]
Create/Update Test Policy
(TP1)
Organizational Test PolicyProcess
Organizational Test Policy Issue Log
Organizational Test Policy Update Log
Gain Consensus
on Test Policy(TP2)
Approved Organizational
Test Policy
Publish Test Policy
(TP3)
Review Test
Policy(TP5)
[major revision required]
•Views of primary stakeholders;•Knowledge of the current test practice within the organization;and, where available:• Organization’s
Mission Statement;
• IT Policy;• IT Project
Management Policy;
• Quality Policy;• Current Test
Policy;• Test Policy
Update Log. [Suitable period has elapsed:• since Test Policy was published; or• since Test Policy was last reviewed; or• major organizational change]
Comments onOrganizational
Test Policy usage
Draft OrganizationalTest Policy
[Test Policy changes not required]
Project Test Management Process
Comments onOrganizationalTest Strategy usage
ProjectTestCompletionReport
Create/Maintain Project Test
Plan(TM1)
Monitor and Control(TM2)
ReportTest Project Completion
(TM3)
ProjectTest Plan
ProjectTestStatusReports
Project Test Plan Updates
Project Test Management Process
TestStrategy
Organizational Test Strategy Process
Project Management Plan,Product Documentation,Software Development Plan, Regulatory Standards
Level Test Plan,Level Test Status Reports
Project Test Plan
Level Test CompletionReport
Control Directives
Test Level Process
• Organizational Test Policy• Organizational Test Strategy• Regulatory Standards
• Project Management Plan• Applicable product documentation
(e.g. System Requirements, Test Item Specifications)• Software Development Plan• Project and Product Risks• Project Test Plan Updates
Create/Maintain Project Test Plan
Plan Creation of
Project Test Plan(TM1.2) Identify &
Analyze Risks
(TM1.3)
Design Project Test
Strategy(TM1.5)Determine
Staffing and
Scheduling(TM1.6)Document
Project Test Plan(TM1.7)
Schedule, Staffing Profile
Project Test Strategy
Analyzed Risks
Project scope
Identify Risk
Mitigation Approaches
(TM1.4)
Gain Consensus on Project Test Plan(TM1.8)
Approved Project Test Plan
Draft Project Test Plan
Project Test Plan Publish
Project Test Plan(TM1.9)
Understand Project Context(TM1.1)
Mitigation Approaches
Fundamental Test Process
ProjectTestPlan
Test Level
Status Report
Level Test CompletionReports
Control Directives
Project Test Management Process
ReportAnomalies
(TL6)
ReportTest Level Completion
(TL8)
Level Test Plan
TestSpecification
TestResults
[Issuenoticed]
[No issues noticed]
Anomaly Report
Test Level Process
Set-up & Maintain TestEnvironment
(TL4)
Plan Testing (TL1)
RunTests(TL5)
Test Environment Readiness Report
Specific TestEnv’t Requirements
Level Test CompletionReport
Design & BuildTests(TL3)
Test Execution Cycle
Close Down Testing(TL7)
Control Directives
Monitor and Control(TL2)
Test Process Info
Asset Info
Measures
Level Test Plan Updates
Project Management Plan,Product Documentation,Software Development Plan, Regulatory Standards
Test Activity Descriptions Example – Design & build tests
• Each testing activity is described using a standard template (following ISO 24774), for example:
Purpose
The purpose of Design and Build Tests is to design and document test cases that will be executed during testing.
Entry Criteria
•the Test Plan has been agreed
•the Test Item Specification(s) is available
Inputs
•Test Plan
•Test Item Specification(s)
•System Requirements
Roles
•Test Designer
Tasks
1.Analyse the test item specification(s) to identify specification items;
2.Determine the test conditions by identifying which specification items to test using the selection criteria from the Test Plan;
3.Derive test coverage items to be exercised by applying the test case design technique(s) to the test conditions;
4.Derive test cases by selecting input values to exercise the test coverage items and corresponding expected results;
5.Create a test set by rationalising the test cases (i.e. removing redundancy, duplication, etc.);
6.Derive a test script by ordering the test cases within the test set using their pre- and post-conditions;
7.Identify and document any specific test environment requirements that are not already included in the Test Plan.
Outputs
•Test Specification
•Specific Test Environment Requirements
Outcomes
•the Test Specification has been documented.
Exit Criteria
•the Test Specification has been agreed.
Part 4: Techniques
• Test case design techniques
– Static testing techniques
• Reviews - inspections & walkthroughs, etc.
– Dynamic testing techniques
• Black-box, white-box, etc.
– Non-functional testing techniques
• Security, performance, usability, etc.
• Test measurement techniques (e.g. coverage)
• Appendices
– Examples of each technique, at each level of testing
– Test technique effectiveness
– Mappings to existing standards
Your turn....
• Session 1– Choose the 1st topic you wish to ‘review’
– Create a group around the topic
– Review the topic
– Report back to the other groups
• Break
• Session 2– Choose your 2nd topic
• it can be the same as your 1st topic
– Create a group around the topic
– Review the topic
– Report back
Topics
1. Review the overall scope & structure
2. Four-tier process model
3. Organizational testing (policy & strategy)
4. Project Test Management
5. Creating (and Maintaining) a Test Plan
6. Fundamental Test Process
7. Test techniques (part 4)
8. The ongoing review process
Conclusions
• We will consider the workshop feedback
• Do you want to be involved?– Join the working group
• through your national standards body
• 6 day meetings, every 6 months
• contribute between meetings
– Contribute materials
– Review drafts
– May 2008 meeting - Berlin• 18 participants representing 12 countries
– November 2008 (Seoul) – last week...
– May 2009 (India)
For more information
• sreid@testing-solutions.com – if interested in trialling the standard on a project, reviewing drafts or writing examples
• http://www.jtc1-sc7.org/ – access to official documents released by WG 26
top related