ABA Risk Management Forum

April 28-30, 2010

Best Practices: Managing Judgment Risk

Presented by: Tyler D. Nunnally

Founder & CEOUpside Risk

What is Judgment Risk?

Behavioral Economics 101

Judgment Bias

Risk Appetite

Best Practices: Managing Judgment Risk

Presentation Topics

‘92 Summer Olympic Games (Barcelona, Spain)

‘93 Nunnally International Trade, Inc. (Prague, Czech Republic)

‘03 University of St Andrews (St Andrews, Scotland)

‘04 Global Business Consulting (Barcelona, Spain)

‘06 Oxford Risk Research & Analysis Ltd (Oxford, England)

’09 Upside Risk (Atlanta, Georgia, USA)

Presenter Background

PART I: Judgment Risk

Key Risk Factors in the Decision-Making Process

What is Judgment Risk?

RiskProbability

of Occurrence

Judgment BiasX

Severityof

Likely Impact

Risk Appetite

“JUDGMENT RISK”

What is Judgment Risk?

Alan Greenspan commenting on how human factor risks caused forecasting models to fail in subprime crisis

“I do not say that the current systems of risk management or econometric forecasting are not in large measure soundly rooted in the real world… But these models do not fully capture what I believe has been, to date, only a peripheral addendum to business-cycle and financial modeling—the innate human responses that result in swings between euphoria and fear that repeat themselves generation after generation with little evidence of a learning curve.

Source: Alan Greenspan, “We will never have a perfect model of risk”, Financial Times, March 16, 2008.

“This, to me, is the large missing ‘explanatory variable’ in both

risk-management and macroeconometric models. Current practice is to introduce notions of “animal spirits”, as John Maynard Keynes put it, through “add factors”. That is, we arbitrarily change the outcome of our model’s equations. Add-factoring, however, is an implicit recognition that models, as we currently employ them, are structurally deficient; it does not sufficiently address the problem of the missing variable”

…Forecasters’ concerns should be not whether human response is rational or irrational, only that it is observable and systematic”.

Source: Alan Greenspan, “We will never have a perfect model of risk”, Financial Times, March 16, 2008.

What is Judgment Risk?

PART II:

Behavioral Economics 101

Combines scientific disciplines of psychology and economics

“Bounded rationality” introduced by Herbert Simon (’78 Nobel Prize)

Prominence of study led by Kahneman & Tversky (’02 Nobel Prize)

Heuristics and biases

Behavioral Economics 101

The Way People Make Decisions

SYSTEM 1

(Intuitive) SYSTEM 2

(Reason)

Automatic Spontaneous Unconscious Instinctive “Gut feel”

Thoughtful Controlled Informed Deductive Analytical

Decision-Making Under Risk and Uncertainty

Diagnosis: person scans, appraises and questions available information

Assessment: risk versus rewards analysis made of available options

Action: commitment to chosen course of action

Adjustment: evaluate decision outcomes and adjust accordingly

Diagnosis ▼

Assessment ▼

Action ▼

Adjustment

Behavioral Economics in the Mainstream

NY Times Business Best Sellers in 2009

PREDICTABLY IRRATIONAL, by Dan Ariely

FREAKONOMICS, by Steven D. Levitt and Stephen J. Dubner

SWAY, by Ori Brafman and Rom Brafman

NUDGE, by Richard H. Thaler and Cass R. Sunstein

“Perhaps the greatest challenge facing behavioral economics is demonstrating its applicability in the real world”

Source: Levitt, Steven and List, John “Homo Economicus Evolves,” Science, February 15, pp. 909–

10.

What the Critics Say…

• Public Policy

• Consumer Choice

• Upside Risk’s approach: Risk Management

“Real World” applications

“Risk management is a form of engineering: it uses science, but ultimately depends on judgement… The ultimate protection against risk is good judgement and alertness: your own and that of your colleagues”.

Source: Risk Management, November 2005

PART III:

Judgment Bias

Probabilities & Statistics: Sample Size Bias

Suppose the mean credit score of all senior citizens is 650. You haveselected a random group of 50 senior citizens. The first person thatyou review has a credit score of 800.

What do you think the mean credit score of this group of 50 seniorcitizens will be?

A. 598B. 650C. 653D. 725

Miscalculating Probabilities & Statistics

Decision traps people fall into when judging risk:

• Overestimate/underestimate real value• Overestimate/underestimate chances of success • Overestimate/underestimate chances of failure • Overweight small probabilities• Underweight large probabilities• Failure to adapt to changing business conditions• Erroneous forecasts, estimates and projections

Heuristics: Availability

Perception of how “risky” something is often determined by how readily examples come to mind from memory. Bias occurs when probability assumptions become inflated by recent events.

US AIRLINE INDUSTRY PASSENGER REVENUES 1999-2004

Risk Exposure: - Overly cautious - Lost opportunities - Falling profits

Behavioral Biases: Confirmation Bias

Suppose you and a colleague disagree whether it would be a good idea tolaunch a new product. You believe that launching the product would be amistake. Your colleague believes that, if introduced, the product would be abig success.

In this situation, what would you do?

A. Seek a third opinion from someone who I know would agree with my positionB. Seek out information that supports my positionC. Seek out information that contradicts my colleague’s positionD. Seek out information that could either support or contradict my position

Booms and Busts: Subprime Crisis

U.S. House Prices: 1988 - 2008

Key Risk Factors: - Time discounting- Confirmation bias- Herding- Overconfidence

PART IV:

Risk Appetite

Risk Preferences

Risk Preference↓

Risk Appetite

Risk Averse↓

Risk Avoiding

Risk NeutralRisk Prone

↓Risk Seeking

ConservativeDecisions

Middle of the Road Decisions

AggressiveDecisions

A person’s level of comfort in taking risk often determines business decisions

Risk-Taking Behavior

Risk-taking behavior has a significant bearing on business outcomes

Risk Aversion Behavior:

Cause decision delays

Failure to take any action

Reliance on others to make decisions

Risk Seeking Behavior:

Ignore underlying evidence

Lack of objectivity between options

Confirmation bias

Risk Appetite Metrics: Sunk Costs

You have a significant sum invested in a project to develop a new product. If successfully completed, the project could have a considerable financial impact on your company. However, the project is overdue and over-budget. To complete the project it will now cost double the original projected cost. You must now decide to make the additional investment in order to complete the project, or to terminate it altogether.

To complete the project, what probability of success would be required before you would make an additional investment? Please choose one of the following probabilities:

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Excessive Risk Appetite and Catastrophic Risk

FDIC Troubled Bank List: 2007 – 2009

Risk Aversion and Falling Profits

Prospect Theory

People generally dislike loss twice as much as they like gains

People become riskseeking in the faceof loss and risk aversein the face of gains

Risk Domains

RISKDOMAINS

FinancialHealth/Safety

Recreation Ethical Social

For instance, a heavy drinker and smoker may take health risks, but he is not necessarily the same guy who makes high risk investments with his 401(k).

A persons’ risk propensities are compartmentalized and divided into separate domains

PART V: Best Practices:

Managing Judgment Risk

“The best behavioral risk management strategy is to try and arrest problem development earlier in the behavioral stage”.

Source: Rudy M. Yandrick, Behavioral Risk Management, 1996

Judgment Risk Indicator: Individual MetricsKey Risk Factor Category Risk Exposure Judgment Bias Sample Size Bias Probabilities and Statistics Low Base Rate Bias Probabilities and Statistics Very High Conjunction Fallacy Probabilities and Statistics Very High Gamblers Fallacy Probabilities and Statistics Very Low Overconfidence (Perception) Behavioral Bias High Time Discounting Behavioral Bias Very High Overconfidence (Actual) Behavioral Bias Moderate Confirmation Bias Behavioral Bias Low Illusion Of Validity Behavioral Bias Very High Status Quo Bias Behavioral Bias Very Low Herding Behavioral Bias Moderate Framing Effects Behavioral Bias Moderate OVERALL High Risk Appetite Status Quo Choice Dilemmas High Sunk Cost Loss Choice Dilemmas High Sunk Cost Gain Choice Dilemmas Low Perception Investment Decisions Low Benefits Investment Decisions Moderate Behavioral Bias Investment Decisions Moderate Willingness to Pay Risk vs. Reward Very High Low Stakes Lottery Choice Moderate High Stakes Lottery Choice Moderate OVERALL High

Judgment Risk Indicator: Organizational Metrics

Scatter Plot of Z-scores

-3.00

-2.00

-1.00

0.00

1.00

2.00

3.00

-3.00 -2.00 -1.00 0.00 1.00 2.00 3.00

Risk Appetite

Jud

gm

ent

Bia

s

Creative Risk Mitigation

First line of defense in hiring practices

Education: Professional Development

Personnel Evaluation

Due diligence & compliance

Benchmarking

Banking Peer Group Risk Appetite Benchmark

Tyler D. Nunnally, Founder & CEO

Upside RiskAtlanta, Georgia U.S.A.

Phone: +1 (404) 320 6047Email: tnunnally@upside-risk.com

www.Upside-Risk.com