1 2/7/2014. cognitive biases in the risk matrix william siefert, m.s. eric d. smith boeing systems...
TRANSCRIPT
![Page 1: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/1.jpg)
104/10/23
![Page 2: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/2.jpg)
Cognitive Cognitive Biases Biases in thein theRisk MatrixRisk Matrix
William Siefert, M.S. Eric D. SmithBoeing Systems Engineering Graduate ProgramMissouri University of Science and [email protected]© 2007 Smith
William Siefert, M.S.
Consequence
Lik
elih
ood
1
1
2
3
4
5
2 3 4 5
Low Risk
Moderate Risk
High Risk
![Page 3: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/3.jpg)
304/10/23
“Fear of harm ought to be proportional not merely to the gravity of the harm, but also to the probability of the event.”
Logic, or the Art of Thinking
Antoine Arnould, 1662
Consequence x Likelihood = Risk
![Page 4: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/4.jpg)
404/10/23
Risk graphing
Moderate Risk
Low Risk
High Risk
Consequence of bad outcome
Lik
elih
ood
of b
ad o
utco
me
Hyperbolic curves
Hyperbolic curvesin log-log graph
iso-risk lines
![Page 5: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/5.jpg)
504/10/23
5 x 5 Risk “Cube”
Consequence
Lik
elih
ood
1
1
2
3
4
5
2 3 4 5
Low Risk
Moderate Risk
High Risk
Original
Current
Objective vs.
Subjectivedata
"Campfire conversation"
piece
![Page 6: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/6.jpg)
604/10/23
Present Situation• Risk matrices are recognized by industry
as the best way to: consistently quantify risks, as part of arepeatable and quantifiable risk management process
• Risk matrices involve human: Numerical judgment
Calibration – location, gradationRounding, Censoring
Data updatingoften approached with under confidence often distrusted by decision makers
![Page 7: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/7.jpg)
704/10/23
Goal
• Risk Management improvement and better use of the risk matrixConfidence in correct assessment of probability and value
Avoidance of specific mistakesRecommended actions
![Page 8: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/8.jpg)
804/10/23
Heuristics and BiasesDaniel Kahneman won the Nobel Prize in Economics in 2002 "for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty.“
Similarities between cognitive bias experiments and the risk matrix axes show that risk matrices are susceptible to human biases.
![Page 9: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/9.jpg)
904/10/23
Anchoring• First impression dominates all further thought• 1-100 wheel of fortune spun• Number of African nations in the United Nations?
Small number, like 12, the subjects underestimated Large number, like 92, the subjects overestimated
• Obviating expert opinion• The analyst holds a circular belief that expert
opinion or review is not necessary because no evidence for the need of expert opinion is present.
![Page 10: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/10.jpg)
1004/10/23
Heuristics and BiasesPresence of cognitive biases – even in extensive and vetted analyses – can never be ruled out.
Innate human biases, and exterior circumstances, such as the framing or context of a question, can compromise estimates, judgments and decisions.
It is important to note that subjects often maintain a strong sense that they are acting rationally while exhibiting biases.
![Page 11: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/11.jpg)
1104/10/23
Terminology
Subjective Parameters
Likelihood (L) Consequence (C)
Subjective Probability, π(p)
Utility (negative), U-(v)
Shown on:
Ordinate, Y axis Abscissa, X axis
Objective Parameters
Objective Probability, p
Objective Value, v
![Page 12: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/12.jpg)
1204/10/23
5 x 5 Risk “Cube”
Consequence
Lik
elih
ood
1
1
2
3
4
5
2 3 4 5
Low Risk
Moderate Risk
High Risk
Original Objective vs.
Subjectivedata
"Campfire conversation"
piece
![Page 13: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/13.jpg)
1304/10/23
Likelihood
1. Frequency of occurrence is objective, discrete
2. Probability is continuous, fiction "Humans judge probabilities poorly" [Cosmides
and Tooby, 1996]
3. Likelihood is a subjective judgment(unless mathematical)
'Exposure' by project manager timeless
![Page 14: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/14.jpg)
1404/10/23
Consequence, C
• Objective Consequence determination is costly• Range of consequence• Total life-cycle cost• Mil-Std 882d
$ damage Human
impact
Environment Law
Catastrophic > $1M Death, Disability irreversible damage
Violate
Critical: $1M - $200K
Hospitalization to >= 3 personnel
Reversible damage
Violate
Marginal: $200K-$10K
Loss of work days; injury
Mitigation
damage
Negligible: $10K-$2K No lost work day; injury
Minimal damage
![Page 15: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/15.jpg)
1504/10/23
Case Study
• Industry risk matrix data1412 original and current risk points (665)
Time of first entry knownTime of last update known
Cost, Schedule and Technical knownSubject matter not known
• Biases revealedLikelihood and consequence judgment
![Page 16: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/16.jpg)
1604/10/23
Magnitude vs. Reliability [Griffin and Tversky, 1992]
• Magnitude perceived more valid• Data with outstanding magnitudes but
poor reliability are likely to be chosen and used
• Suggestion: Data with uniform source reliability
Speciousness of data
• Observation: risk matrices are magnitude driven, without regard to reliability
![Page 17: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/17.jpg)
1704/10/23
Expected Distribution for original risk points in Risk Matrix?
Consequence
Lik
elih
ood
1
1
2
3
4
5
2 3 4 5
Low Risk
Moderate Risk
High Risk
Bivariate NormalUniform:
![Page 18: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/18.jpg)
1804/10/23
1. Estimation in a Pre-Define Scale Bias • Response scale effects judgment [Schwarz, 1990]• Two questions, random 50% of subjects:
• Please estimate the average number of hours you watch television per week:
____ ____ __X_ ____ ____ ____ 1-4 5-8 9-12 13-16 17-20 More
• Please estimate the average number of hours you watch television per week:
____ ____ __X_ ____ ____ ____ 1-2 3-4 5-6 7-8 9-10 More
![Page 19: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/19.jpg)
1904/10/23
Likelihood: Logistic, 1412, 665, and Normal
0
50
100
150
200
250
300
350
400
1 2 3 4 5Likelihood
Coun
ts
Logistic distribution1412 (scaled) original L counts 665 original L countsNormal distribution
![Page 20: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/20.jpg)
2004/10/23
Likelihood Marginal Distribution of Original Points1 2 3 4 5
58 272 754 288 40
Normal distribution with μ= 3.0, σ= 0.78
38 330 676 330 38
∆ = actual – normal
20 - 58 78 - 42 2
4,362 dfN (Χ2 = 22, Logistic
0
100
200
300
400
500
600
700
800
900
1 2 3 4 5
LikelihoodNormalLogistic
Χ2 >~ 10 reject H0
H0 = Normal
![Page 21: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/21.jpg)
2104/10/23
Effect of Estimation in a Pre-Defined Scale
Likelihood count in original risks
0
100
200
300
400
500
600
700
800
1 2 3 4 5
Consequence count in original risks
0
100
200
300
400
500
600
700
1 2 3 4 5
‘People estimate probabilities poorly’ [Cosmides and Tooby, 1996]
Consequence/Severity amplifiers
![Page 22: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/22.jpg)
2204/10/23
Effect of Estimation in a Pre-Defined Scale
Likelihood count in original risks
0
100
200
300
400
500
600
700
800
1 2 3 4 5
Consequence count in original risks
0
100
200
300
400
500
600
700
1 2 3 4 5
‘People estimate probabilities poorly’ [Cosmides and Tooby, 1996]
Consequence/Severity amplifiers
![Page 23: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/23.jpg)
2304/10/23
Severity Amplifiers
• Lack of control• Lack of choice• Lack of trust• Lack of warning• Lack of understanding• Manmade• Newness• Dreadfulness• Personalization• Recallability• Imminency
![Page 24: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/24.jpg)
2404/10/23
5 x 5 Risk Matrix
Consequence
Lik
elih
ood
1
1
2
3
4
5
2 3 4 5
Low Risk
Moderate Risk
High Risk
![Page 25: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/25.jpg)
2504/10/23
Situation assessment• 5 x 5 Risk Matrices seek to increase
risk estimation consistency
• Hypothesis: Cognitive Bias information can help improve the validity and sensitivity of risk matrix analysis
![Page 26: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/26.jpg)
2604/10/23
Prospect Theory
• Decision-making described with subjective assessment of:
Probabilities Values
and combinations in gambles
• Prospect Theory breaks subjective decision making into:
1) preliminary ‘screening’ stage, probabilities and values are subjectively assessed
2) secondary ‘evaluation’ stage combines the subjective probabilities and utilities
![Page 27: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/27.jpg)
2704/10/23
Humans judge probabilities poorly*
0.00.0
1.0
1.0
Ideal Estimate
Typical Estimate
Real Probability
Su
bje
ctiv
e P
rob
ab
ility
We
igh
ting
![Page 28: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/28.jpg)
2804/10/23
Subjective Probability, π(p) • small probabilities
overestimated• large probabilities
underestimated
π(p) = ( pδ) / [pδ + (1- p)δ] (1/ δ)
p = objective prob.0 < δ ≤ 1
When δ =1, π(p) = p = objective probability
0.0
1.0
1.0Objective probability, p0.0
Subjective probability,
π(p)
usual value for δ:δ = 0.69 for lossesδ = 0.61 for gains
![Page 29: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/29.jpg)
2904/10/23
Gains and losses are not equal*
Gains
Losses
ObjectiveValue
Reference Point
SubjectiveWorth
![Page 30: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/30.jpg)
3004/10/23
Subjective Utility • Values considered from reference
point established by the subject’s wealth and perspective Framing
• Gains and losses are subjectively valued
1-to-2 ratio.• For gains:
U+(v) = Ln(1 + v) • For losses:
U-(v) = -(μ)Ln(1 – cv)μ = 2.5
c = constantv = objective value
Gains
Losses
ObjectiveValue, v
Reference Point
SubjectiveUtility, U(v)
![Page 31: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/31.jpg)
3104/10/23
Implication of Prospect Theory for the Risk Matrix
0.00.0
1.0
1.0Actual probability
Estimated probability
Loss
es
Val
ue
Util
ity
CEO, Company Ownership
Viewpoint
Engineer, Non-Ownership
Viewpoint
General Tendencies for
Engineers
Pro
babi
lity
Risk Cube
Severity
![Page 32: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/32.jpg)
3204/10/23
ANALYSES AND OBSERVATIONS OF INITIAL DATA
Impediments for the appearance of cognitive biases in the industry data:
1) Industry data are granular while the predictions of Prospect Theory are for continuous data
2) Qualitative descriptions of 5 ranges of likelihood and consequence
non-linear influence in the placement of risk datum points
Nevertheless, the evidence of cognitive biases emerges from the data
![Page 33: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/33.jpg)
3304/10/23
2. Diagonal Bias• Anticipation of
later moving of risk points toward the origin
• Risk points withdrawn from the origin upward and rightward along the diagonal
Diagonal Bias in Original Points
0
1
2
3
4
5
1 2 3 4 5
Consequence
Lik
elih
oo
d
Regression on 1412 Original Points
Intercept Slope R
2.2 0.22 0.22
![Page 34: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/34.jpg)
3404/10/23
3. Probability Centering Bias
• Likelihoods are pushed toward L = 3
• Symmetric to a first order
LIkelihood Marginal Distribution of Original Points
0
1
2
3
4
5
6
-1 0 1 2 3 4 5 6
Consequence
Lik
elih
oo
d
![Page 35: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/35.jpg)
3504/10/23
Open Risks by Month
0
10
20
30
40
50
60
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77
Series2
Linear (Series2)
![Page 36: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/36.jpg)
3604/10/23
Implication of Prospect Theory for the Risk Matrix
0.00.0
1.0
1.0Actual probability
Estimated probability
Loss
es
Val
ue
Util
ity
CEO, Company Ownership
Viewpoint
Engineer, Non-Ownership
Viewpoint
General Tendencies for
Engineers
Pro
babi
lity
Risk Cube
Severity
![Page 37: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/37.jpg)
3704/10/23
3a. Asymmetrical Probability Bias• Subjective probability
transformation• π(p) predicts that
likelihood data will be pushed toward L = 3 Large probabilities
translated down more than small probabilities are translated up
Reduced amount of large subjective probabilities, comparatively
0.0
1.0
1.0Objective probability, p0.0
Subjective probability,
π(p)
1 2 3 4 5
58 272 754 288 40
![Page 38: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/38.jpg)
3804/10/23
4. Consequence Bias
• Consequence is pushed higher
• Engineer identifies with increased risk to entire corporation
• 'Personal' corporate risk
Consequence Marginal Distribution of Original Points
-1
0
1
2
3
4
5
6
0 1 2 3 4 5 6
Consequence
Lik
elih
oo
d
![Page 39: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/39.jpg)
3904/10/23
Statistical Evidence for Consequence Bias Max at C = 4
C = 1 significantly less than C = 5 countsC = 2 significantly less than C = 4
Consequence Original Data Points
1 2 3 4 5
20 145 538 599 110
0
100
200
300
400
500
600
700
1 2 3 4 5
Normal distribution comparison: χ2 = 600, df = 4 0.0 probability
Consequence smoothed
Consequence increased, → , by Amplifiers
H0 = Normal
![Page 40: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/40.jpg)
4004/10/23
Consequence translation
Gains
Losses
ObjectiveValue
Reference Point
SubjectiveWorth
![Page 41: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/41.jpg)
4104/10/23
Likelihood mitigation recommendations• Engineers and Management
1. Technical risk highest priority
2. Schedule risk communicated well by management
3. Cost risk likelihood less frequently communicated by management.
Higher cognizance of cost risk will be valuable at the engineering level
Likelihood mitigation
1. Technical
2. Schedule
3. Cost
![Page 42: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/42.jpg)
4204/10/23
Consequence Mitigation• Engineers:
1. Schedule consequences effect careers
2. Technical consequences effect job performance reviews
3. Cost consequences are remote and associated with management
Higher cognizance of cost risk will be valuable at the engineering level
Consequence mitigation
1. Schedule
2. Technical
3. Cost
![Page 43: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/43.jpg)
4304/10/23
CONCLUSION• First time that the effects of cognitive biases
have been documented within the risk matrix• Clear evidence that probability and value
translations, as likelihood and consequence judgments, are present in industry risk matrix data
• Steps 1) the translations were predicted by prospect theory,
2) historical data confirmed predictions
• Risk matrices are not objective number grids• Subjective, albeit useful, means to verify that risk
items have received risk-mitigating attention.
![Page 44: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/44.jpg)
4404/10/23
Data Collection Improvements
• Continuum of data from Risk management to(Issue management)Opportunity management
• Different databasesyears of data in each
• TimeWaterfall Risk charts
![Page 45: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/45.jpg)
4504/10/23
Suggestions for risk management improvement
• Objective basis of risk:Frequency data for Probability$ for Consequence
• Long-term, institutional rationality
• Team approach
• Iterations
• Public review
• Expert review
• Biases and errors awareness
![Page 46: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/46.jpg)
4604/10/23
Future work
• Confirmation of the presence of probability biases, and value biases in risk data from other industries or companies
• Real world effects on industry from using biased risk mitigation data $’s, not risk pointsSequential ramificationsProspect Theory risk gambles
• Inform decision makers about how cognitive biases affect risk assessment
![Page 47: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/47.jpg)
4704/10/23
![Page 48: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/48.jpg)
4804/10/23
References• L. Cosmides, and J. Tooby, Are humans good intuitive
statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty, Cognition 58 (1996), 1-73.
• D. Kahneman, and A. Tversky, Prospect theory: An analysis of decision under risk, Econometrica 46(2) (1979), 171-185.
• Nobel, "The Bank of Sweden Prize in Economic Sciences in memory of Alfred Nobel 2002," 2002. Retrieved March, 2006 from Nobel Foundation: http://nobelprize.org/economics/laureates/2002/index.html.
• N. Schwarz, Assessing frequency reports of mundane behaviors: Contributions of cognitive psychology to questionaire construction, Review of Personality and Social Psychology 11 (1990), 98-119.
• A. Tversky, and D. Kahneman, Advances in prospect theory: Cumulative representation of uncertainty, Journal of Risk and Uncertainty 5 (1992), 297-323.
![Page 49: 1 2/7/2014. Cognitive Biases in the Risk Matrix William Siefert, M.S. Eric D. Smith Boeing Systems Engineering Graduate Program Missouri University of](https://reader035.vdocuments.us/reader035/viewer/2022070305/5514a72e55034640138b4c14/html5/thumbnails/49.jpg)
4904/10/23
• Comments !
• Questions ?