risk-based testing - a common language for project stakeholders
TRANSCRIPT
© 2003 Insight Consulting Ltd. Version 1.01
Risk-based testing - a commonlanguage for project stakeholders
Mike RussellInsight Consulting Ltd.
e-mail: [email protected]: http://www.insight.ie
Insight Consulting Ltd.114 GranitefieldDun LaoghaireCo. DublinIreland
© 2003 Insight Consulting Ltd. Version 1.02
Learning objectives
To identify perception and communication problems betweentesting and other project stakeholders
To discuss how risk-based testing can help solve these problemsand improve the (perceived) added value of testing
To outline how to implement improvements such as risk-basedtesting in a practical way
© 2003 Insight Consulting Ltd. Version 1.03
Agenda
Key challenges facing testingWhat is risk-based testing?How does it address the challenges?Getting started
© 2003 Insight Consulting Ltd. Version 1.04
Perceptions of testing
Testing
Software
System
Quality
Software
System
(Sign-off )
© 2003 Insight Consulting Ltd. Version 1.05
Perceptions of testing - 2
Project Stakeholders include customers, users, seniormanagement, project management, development, etc.
Squeeze on testingSkills required?The ‘sign-off problem’...
….lack of appreciation of ‘added value’ of software testingmorale and staff turnoverdifficult in justifying testing resourcesless effective testing
© 2003 Insight Consulting Ltd. Version 1.06
Origins of these perceptions
A lack of a clear understanding of the objectives and role oftestingA lack of involvement in influencing testingPoor visibility of testingNot understanding or being able to interpret the results of testingLinking testing to a narrow definition of quality that is notaligned with the primary objective of the project
Underlying theme = Poor communication
© 2003 Insight Consulting Ltd. Version 1.07
Agenda
Key challenges facing testingWhat is risk-based testing?How does it address the challenges?Getting started
© 2003 Insight Consulting Ltd. Version 1.08
Risk Management
Risk = Impact x Likelihood SEI model:
• identify• analyse• plan• track• control• communicate
Likelihood
Impact
Hi
Lo
Lo Hi
!
© 2003 Insight Consulting Ltd. Version 1.09
Three types of software risk
Project Riskresource constraints, external
interfaces, supplierrelationships, contract
restrictions
Process Riskvariances in planning and
estimation, shortfalls instaffing, failure to trackprogress, lack of quality
assurance and configurationmanagement
Primarily a managementresponsibility
Planning and the development processare the main issues here.
Product Riskbusiness critical features,
complexity, design and codequality, non-functional issues,
requirements instability
Requirements risks are the most significant risksreported in risk assessments.
Testers are mainlyconcerned withProduct Risk
© 2003 Insight Consulting Ltd. Version 1.010
Risk-based testing - key elements
Risk identification and analysisRisk-based testing strategy (test planning)Design, prioritisation and review of test cases (logicaldesign part of test specification - ‘what’)Prioritised execution of test cases on the basis of risk(execution)Risk-based tracking and controlRisk-based test reportingRisk-based decision on release
© 2003 Insight Consulting Ltd. Version 1.011
Risk-based reporting*
Progress through the test plan
today Plannedend
residual risksof releasing
TODAY
Res
idua
l Ris
ks
start
all risks‘open’ atthe start
* From ‘Risk-based reporting’ by Paul Gerrard
© 2003 Insight Consulting Ltd. Version 1.012
Benefit & objectives based test reporting
Open
Closed
Ris
ks
Open
Open
Closed
Closed
Open
Obj
ectiv
e
Obj
ectiv
e
Obj
ectiv
e
Obj
ectiv
e
Bene
fit
Bene
fit
Bene
fit
Bene
fit
Bene
fit
Benefits available for releaseO
bjec
tive
Bene
fit
Closed
© 2003 Insight Consulting Ltd. Version 1.013
Agenda
Key challenges facing testingWhat is risk-based testing?How does it address the challenges?Getting started
© 2003 Insight Consulting Ltd. Version 1.014
Improving the communication
Risk Management well established practice withmanagement and customersRisks can be quantifiedLanguage of risk is easily understood and embraced
risk is sexy (relatively!)Easier to solicit input from stakeholdersStatus and reporting easily understoodTesting becomes aligned with core project objectives,addressing risks and informing as benefits becomeavailable
© 2003 Insight Consulting Ltd. Version 1.015
Addressing the challenges
Role of testingto provide good clear test evidence on risk tostakeholders
Moment of involvementImproved visibility and influence
through number of interfaces and interactions andmeaningful (understandable) reporting
© 2003 Insight Consulting Ltd. Version 1.016
Addressing the challenges -2
… and better testing by using more knowledge to focustesting with available resources
All facilitated by improved communication through use of thecommon language of risk
© 2003 Insight Consulting Ltd. Version 1.017
Stakeholder benefitsCustomer/users
opportunity to influencebetter focus on their concerns
Project Managerbetter use of testing resourcesimproved collaboration between teamsuseful information for tracking and release decision
Developmentrisks useful input to improve design and developmentclearer understanding of development test responsibilities
© 2003 Insight Consulting Ltd. Version 1.018
Stakeholder benefits - 2
Testearly involvementmanage the squeeze on testingthe ‘sign-off problem’ disappears‘added value’ of testing more obvious (even quantifiable)and easier to communicate
© 2003 Insight Consulting Ltd. Version 1.019
Agenda
Key challenges facing testingWhat is risk-based testing?How does it address the challenges?Getting started
© 2003 Insight Consulting Ltd. Version 1.020
RBT sample improvement actionsHold a risk identification workshop with customer, projectmanagement, development and testPiggy-back on existing risk management process
strengthen link between product risks and testingPrioritise test cases on basis of known risks
get developers and users to reviewDevelop a risk-based testing strategy for system testEnhance weekly test reporting on basis on risks addressedand risks remainingRevise test schedule to run high risk tests first
© 2003 Insight Consulting Ltd. Version 1.021
Improvement actions - how?Simple action plan with prioritised actions, estimated effort, who andwhenPilotDefine the process step in 1 page (what to do) with 1-2 page proceduraldetail only if required (how to do it) and supporting templates/checklists
training should not be an objectiveobjective is communication/consensus and reminding people what todo and how to do it
Train/coach/facilitate as requiredTrack and measure resultsRe-plan next set of actions - iterate
© 2003 Insight Consulting Ltd. Version 1.022
Summary
The language of risk and risk-based testing provides not only for more effective testing but for improved communication and collaboration between testing and other project stakeholders. Justifying the ‘added value’ of testing becomes less of an issue.
© 2003 Insight Consulting Ltd. Version 1.023
References
www.insight.ie - Our website containing testing news, links, opinions, etc.www.evolutif.co.uk - Systeme Evolutif’s website containing a number ofarticles on risk-based E-Business testing
‘Risk-based E-Business Testing - Paul Gerrard and Neil Thompson,Artech House, ISBN: 1580533140; 1st edition (August 2002) - see alsowww.riskbasedtesting.com
www.stickyminds.com - SQE website containing a number of relatedarticles‘Making Process Improvement Work: A Concise Action Guide forSoftware Managers and Practitioners’, Neil S. Potter, Mary E. Sakry,Addison-Wesley Pub Co; ISBN: 0201775778, March 2002. [see alsowww.processgroup.com for articles and templates on Practical SPI].