value based requirements engineering and prioritization percolating ‘value’ into the sdlc...
TRANSCRIPT
Value Based Requirements EngineeringAnd Prioritization
Percolating ‘Value’ into the SDLC (Software Development Lifecycle)
Nupul Kukreja15th October, 2012
Agenda• The ‘science’ of decision making – what, why
and how?• The role of decision making in Value Based
Requirements Engineering & Prioritization (VBRE/RP)
• Importance of ‘Planning’• VBRP – what, why & how?• Understanding VBRP in CS577• VBRP in Action
3
Agenda: VBSE 4+1 View
Theory-W:SCS Win-Win
Dependency Theory
Utility Theory
DecisionTheory
ControlTheory
1. Protagonist goals3a. Solution Exploration7. Risk, opportunity, change management
2. Identify SCS2a. Results chains
3b, 5a, 7b. Cost/schedule/ performance tradeoffs3b, 7a. Solution Analysis
5a, 7b. Options, solution development & analysis
3. SCS Value Propositions (Win Conditions)
4. SCS expectations management
5a, 7b. Prototyping
6, 7c. Refine, execute, monitor & control plans
6a, 7c. State measurement, prediction correction; Milestone synchronization
5a. Investment analysis, Risk analysis
5. SCS WinWin Negotiation
Decision Making – What?Definition:*
• Decision making can be regarded as the mental processes (cognitive process) resulting in the selection of a course of action among several alternative scenarios
• Every decision making process produces a final choice
• The output can be an action or an opinion of choice.
*Courtesy: http://en.wikipedia.org/wiki/Decision_making
Decision Making – Making a ChoiceWhich apartment do you want to live in?
Making a Choice – How?• So, just how did you choose your apartment?• What all did you ‘see’ / evaluate before signing
the lease?• Examples of possible criteria:– Rent per month– Location i.e., quality of area– Proximity to campus– Presence of a good looking neighbor – …and probably several others
Analyzing the ‘Problem’• You have the two key ingredients necessary for a
decision problem:– A list of criteria– A set of alternatives
• Problem: Given a list of criteria AND a set of alternatives select the alternative that best suits the given criteria
Decision Analysis
Criteria
Alternatives
Rank ordered set of alternatives
Apartment Selection
Rank ordered set of apartments
Decision Making – Why Bother?• Converts ‘art’ into science– Adds rigor to the act of decision making
• Justification of choice of action i.e., why you chose what you chose?– Helps decrease legal liability
Ex.: Why did Company A select the bid/tender of Company X over Y? (It’s not only cost )
• Documents the institutional memory that lead to that particular decision (i.e., helps trace back to provide justification of decision)
Decision Making – How do you do it?
• If it is a well studied science (it is) does anything exist ‘out there’ to help practice this ‘science’?
• There are various techniques that can be employed:– Multi-attribute utility theory (ISE 562 in Fall)– Analytical Hierarchy Process (AHP)– Simple Additive Weighting (SAW)– TOPSIS (Technique of Ordered Preference by
Similarity to Ideal Solution)– …and many more (
http://en.wikipedia.org/wiki/MCDA#MCDM_Methods)
Decision Problem – Look and Feel
Criterion 1
Criterion 2
Criterion 3 … Criterion
N
Alternative 1 S11 S12 S13 … S1N
Alternative 2 S21 S22 S23 … S2N
Alternative 3 S31 S32 S33 … S3N
… … … … … …
Alternative N SN1 SN2 SN3 … SNN
Sij = Measure of how well Alternative ‘i‘ does on Criterion ‘j’
Almost always has a matrix-like representation:
When considering multiple criteria in decision making the problem is often referred to as
“Multiple Criteria Decision Analysis” (MCDA) or “Multiple Criteria Decision Making” (MCDM)
Okay, Great! But what has any of it got to do with VBSE?
• Everything!! • How do you decide:
Which requirement to implement first?Which component to design first?Which architectural style/pattern to use?Which requirements to test first?Which test-cases form part of the regression
suite?What all to prototype first?For which component to write the “first line of
code”?
ControlTheory
Wait! But I’ve already done all of this before and I got it right too!
• Probably! However…• Were you able to justify your choice i.e., how
you came about choosing a specific option?– Was there ever a need to do so?
• Just “how” did you decide?– Perhaps intuition, gut feel, domain knowledge,
tossing a coin, do as the Romans do…• Yes, a simple 1-10 works or even MoSCoW
(Must, Should, Could, Would – have) but for relatively simple problems
Short Detour: Planning
Purpose of ‘Planning’*• Why do we do it?– Reduce risk & uncertainty– Supporting better decision making– Establishing trust (i.e. frequent delivery)– Conveying expectations
• Planning is a ‘quest for value’– Attempt to find an optimal solution of the overall
product development question: What should be built?
*Agile Estimation & Planning – Mike Cohn
A Good Plan• One that stakeholders find sufficiently reliable• One that can be used as basis for decision
making– Approximate time to market– An idea of the set of features
• Made more precise as project moves on• Is a ‘living’ artifact showing the current status
of the project to avoid last minute surprises• Is planned around ‘value’
Planning for ‘Value’*• Stakeholders needs must be understood• Just delivering features is not important
– Necessary to balance scope, schedule, cost and value of features comprising the release
• Various factors impacting prioritization– Financial value of having the features– Cost of development– Amount/significance of new knowledge
• Product (what)• Project (how)
– Reducing Risk (Business/Technical)• Schedule• Cost• Functionality
*Agile Estimation & Planning – Mike Cohn
Decision Making and Planning for Value
• Decision analysis techniques can be ‘overloaded’ to perform requirements prioritization…
• …the rank-ordered output of decision making techniques could also rank-order requirements!
• Requirements prioritized against the goals/needs/values of the stakeholders VBRP
The ‘Dimensions’ of Value• Value is a multi-dimensional quantity, a simple 1-10 may
not cut it (too many ties – Must have, Must-must haves, must-must-must haves etc.)
• Value lies in the eyes of the beholder…and so does its dimensions!
• Dimensions are ‘hidden’ within the ‘expected benefits’ of the various stakeholders
• The benefits serve as the goals/objectives/criteria on which to prioritize the requirements
• Point to consider: Are all dimensions* equally important?(*dimensions = benefits = criteria = goals = objectives)
Prioritizing Criteria/Value Dimensions
• All criteria may NOT be equally important• But just HOW do you prioritize the criteria
themselves?– A simple 1-10 (Yes, it works )– A highly involved process like multi-attribute utility
theory (MAUT) to get the utility functions for each criteria which shows the risk attitude of each criterion
– Or another interesting technique - Project Success Sliders*
* Radical Project Management by Rob Thomsett
Project Success Sliders
Image: Radical Project Management by Rob Thomsett
25%
50%
75%
Success Criteria
(defined at start of project)
Project Success Sliders
Image: Radical Project Management by Rob Thomsett
25
50
100
75
25
75
50
1
2
4
3
1
3
2
It’s ‘subjective’ but an extremely effective tool to understand the importance of expectations and their relative tradeoffs!
They can (and are) interpreted on a relative scale i.e., twice as important, half as much etc.,
Putting It All TogetherYou have:
1.Relative ranking of all criteria2.List of requirements (or win conditions)
What else do you need?• Scores! How well each requirement/win-
condition does on each criteriaYou STILL need something else…• The decision analysis ALGORITHM!
Overall Score
Criteria
Alternatives
Criterion 1
Criterion 2
Criterion 3 … Criterion
N
W1 W2 W3 … WN
S(A1) Alternative 1 S11 S12 S13 … S1N
S(A2) Alternative 2 S21 S22 S23 … S2N
S(A3) Alternative 3 S31 S32 S33 … S3N
… … … … … … …
S(AN) Alternative N SN1 SN2 SN3 … SNN
Decision Analysis Algorithm: Given a set of criteria with possible weights and the measure of how the alternatives rank on the respective criteria find the most optimal alternative
The scale of the scores could be absolute, relative, 1-10 (Likert
Scale), Fibonacci... …must be consistent across the
matrix
Algorithms for Practicing VBRP• There are various algorithms in literature to choose from
along with those from MCDA/M:– Kano Analysis– Planning Poker– 100-point assignment technique– Simple Additive Weighting– Quality Function Deployment (House of Quality)– Cost of Delay– Weiger’s Prioritization– Theory-W (Business value vs. Technical Feasibility)– TOPSIS– …and quite a few more
What are we using in CS577?• TOPSIS: Technique of Ordered Preference by Similarity
to Ideal Solution (MCDM Technique)• Integrated in Winbook for prioritizing MMFs and win
conditions• We’ll be using Business Value, Technical Feasibility and
Relative Penalty as the criteria against which to rank the requirements. You may add more (e.g., risk, cost, effort, time – anything you deem appropriate!)
• MMFs Prioritized against stakeholders’ value propositions/goals/objectives
• Success Sliders for prioritizing value propositions
27
Ideal Alternative (S’)
TOPSIS Primer
Criterion 1
Criterion 2
Alternative 1
Alternative 2Non-Ideal Alternative (S*)
Aim: Rank order alternatives by their ‘closeness to ideal’ and ‘distance from non-ideal’Criterion: Has ‘direction of preference’ i.e. more/less of the criterion is preferredIdeal: Best score for each criterion Non-ideal: Worst score for each criterion
Refinements to TOPSIS/VBRP• Prerequisite Handling (not yet integrated in
Winbook)– Specifying MMF/WC prerequisites and factoring in
prioritization– Simple solution: Priority of item less than all those
in its prerequisite graph • Hierarchical Prioritization (partially integrated)– Prioritizing high level MMFs w.r.t. project goals– Prioritizing WCs w.r.t. MMFs– Prioritizing test cases w.r.t. WCs and so on
Points To Note• TOPSIS is ONE way of practicing VBRP• You could even perform COTS tradeoff analyses using
it. (It’s a decision problem!)• Use the VBRP technique that is best suited to the
situation – each technique has its pros and cons. Choose wisely.
• Some prioritization techniques do not lend themselves to a typical spreadsheet like analysis
• The output of some techniques could be bucketed (categorized into the MoSCoW buckets) or ordinal (explicitly rank ordered)
Conclusion• Tools and techniques for prioritization are only
one side of the coin…• …negotiations and discussions are the other.
The latter must be held for the former to be of any value. Using the tool will NOT guarantee VBRP
• There will be a Homework on VBRP posted tonight/tomorrow, due next week. Watch the class website.