robust decision making
Post on 05-Jan-2016
28 Views
Preview:
DESCRIPTION
TRANSCRIPT
Robust Decision Making
Robert Lempert
RAND
HDGC Seminar
February 13, 2004
211-15-03
How Should Climate-Change Uncertainties Be Characterized for Decisionmakers?
Key climate change uncertainties include “Basecase” emissions Behavior of perturbed climate system Value our descendants place on impacts of climate
change Costs of abatement with future technology
Climate-change decisionmakers must understand uncertainties to make effective choices
311-15-03
Analytic Tools Often Vital to Clarify Thinking, But Can Illuminate Trees Rather Than Forest
Analytic tools often vital in improving complicated decisions: Can successfully summarize vast quantities of information Help address flaws in human reasoning
Traditional analytic methods assume well-characterized risks and policy choices based on predictions
But strategic decisions can go awry if decision-makers assume risks are well-characterized when they are not
Uncertainties are underestimated Strategies can be brittle Misplaced concreteness helps blind decision-makers to surprise
Predict Act
411-15-03
Global Scenario Group offers three families of sustainability scenarios Conventional worlds Barbarization Great Transformations
These scenarios Capture a wide range of factors which may affect the
future Attempt to make an argument for a particular risk-
management strategy
Scenarios Capture and Communicate Information About Future, But Hard to Link to Actions
511-15-03
Should Analysts Put Probabilities on Scenarios
Such as Those Developed by SRES?
Pros Necessary to make
policy Others will provide
likelihood estimates if experts don’t
Cons Little evidence to support
judgments about probabilities
Arguing over likelihoods distracts from reaching consensus on near-term actions
Desire for concreteness driving IPCC towards placing probabilities on scenarios
611-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
711-15-03
Climate Change is a Problem of Decisionmaking Under Deep Uncertainty
Deep uncertainty is: When we do not know, and/or key parties to the decision do not agree
on, the system model, prior probabilities, and/or “cost” function
Under conditions of deep uncertainty, decision-makers: Often seek robust strategies, ones which perform reasonably well
compared to the alternatives across a wide range of plausible futures, evaluated with a range of values
Robust strategies are often (but not always) adaptive, that is they evolve over time in response to new information
Often use choice of strategy, not additional information, to reduce uncertainty
811-15-03
Robust Decisionmaking (RDM)
Robust decisionmaking Is an iterative, analytic process that identifies
• Strategies that perform well over a wide range of futures
• Remaining vulnerabilities of these strategies Made possible by advances in computational
capabilities Characterizes uncertainties most important to the
choice among strategies
911-15-03
Four Key Elements of Robust Decision Making
Consider large ensembles (hundreds to millions) of scenarios
Seek robust, not optimal strategies Achieve robustness with adaptivity Design analysis for interactive exploration of a
multiplicity of plausible futures
10
11-15-03
Consider Ensembles of Many Scenarios
On the occasion of the 1893 World Columbian Exposition, 74 experts wrote essays predicting what the United States would look like in 1993
Most were wrong
But some were strangely close to the truth
11
11-15-03
Use Robustness Criteria to Judge Alternative Strategies
Under deep uncertainty, decision makers often seek robust strategies that work reasonably well over a wide range of plausible futures
We measure robustness according to degree of “regret,” which is defined as the difference between
the performance of a strategy in a given future, and
the performance of the best strategy in that future
12
11-15-03
Combine Human and Machine Capabilities
Landscape ofplausible futures
Alternative strategiesX
Ensemble of scenarios
Robuststrategies
13
11-15-03
Exploratory Modeling Software Supports This Process
Exploratory modeling software enables users to navigate through large numbers of scenarios and Formulate rigorous policy arguments based on these explorations
CARsTM is java-based exploratory modeling software that: Links to virtually any type of model and/or data Supports interactive use of search and visualization to create, explore,
compare, and understand large scenario ensembles
Tools to draw Tools to draw meaning from meaning from
informationinformationUsers
Tools to Tools to represent represent
informationinformation
14
11-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
15
11-15-03
Example Application of Robust Decisionmaking
The RDM approach employed a simple method of representing information “Toy” systems-dynamics model with 41 input parameters
representing uncertainties about• future economic, demographic, and environmental trends• values and capabilities of future decisionmakers
Simple agent-based model of future decisionmakers Four value functions based loosely on UN Human Development
Index, which reflects interests of a range of stakeholders Near-term strategies affect “decoupling” rate
Example: What choice of near-term actions will help ensure Example: What choice of near-term actions will help ensure strong economic growth and a healthy environment over the strong economic growth and a healthy environment over the
course of the 21course of the 21stst century? century?
16
11-15-03
Visualizations Capture Key Relationships Among Plausible Futures
Landscape of plausible futures helps illuminate key challenges to ensuring strong economic growth and a healthy environment over the
course of the 21st century.
Economic growth rate
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
India since 1960
U.S. 1890-1930
U.S. since 1950
U.S. in 20th century
China since 1960
Brazil since 1980
Russia since 1993
Conventional World scenario
Barbarization scenario
Great Transition scenario
Decoupling rate
17
11-15-03
Compare “Fixed” Near-Term Strategies Across Scenarios
Near Term
Choose policies
Assume near-term policy continues until changed by future generations
Future decision-makers recognize
and correct our mistakes
Future
18
11-15-03
Look for Robust Strategies
XLandscape of
plausible futures
Alternative strategies
Ensemble of scenarios
Robuststrategies
19
11-15-03
Strategies Should Be RobustAcross Multiple Measures of “Goodness”
Use measures inspired by UN’s Human Development Index (HDI)• Discounted, average rate of improvement in GDP/capita,
longevity, and environmental quality (but no education level) time series
• Four different weightings
N$: North GDP/capita and longevity
W$: Global GDP/capita and longevity
NG: North GDP/capita, longevity, and environmental quality
WG: Global GDP/capita, longevity, and environmental quality
20
11-15-03
Speeding Decoupling Performs Well in Many Futures Using North HDI Measure
Slight speed-upSlight speed-up
1.0 2.0 3.0 4.0–1.0
0
5.0
0
N$ W$
NG WG
1.0
3.0
4.0
2.0
Conventionalworld scenario
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
Economic growth rate
Decoupling Rate
No regretMild
A lotOverwhelming
21
11-15-03
But Often Fails for Global Green Measure
1.0 2.0 3.0 4.0–1.0
0
1.0
5.0
3.0
4.0
2.0
0
N$ W$
NG WG
Conventionalworld scenario
Economic growth rate
Decoupling rate
No regretMild
A lotOverwhelming
Slight speed-upSlight speed-up
22
11-15-03
Exploration DemonstratesNo “Fixed” Strategy Is Robust
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
No regretMild
A lotOverwhelming
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
Economic growth rate
Decouplingrate
N$ W$
NG WG
Stay the CourseStay the Course Crash EffortCrash Effort
23
11-15-03
Design and Examine Additional Strategies
XLandscape of
plausible futures
Alternative strategies
Ensemble of scenarios
Robuststrategies
24
11-15-03
Start with a Milestone, but Evaluate Progress Early and Modify Milestone If Necessary (Safety Valve)
NODoes the carrying capacity change?
Choose policies to maximize
utility
Determine best policy to meet milestone
Select near-term milestone
YES
Is milestone achievable with
current approach?
Relax milestone
Present Future
YES
NO
Implement policy
25
11-15-03
“Safety Valve” Strategy Appears Highly Robust
Safety valveSafety valve
Economic growth rate (%)
Dec
ou
plin
g r
ate
(%)
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
N$ W$
NG WG
Economic growth rate (%)1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
No regretMild
A lotOverwhelming
N$ W$
NG WG
+
WorstCase
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
26
11-15-03
Even Simple Scenario Generator Implies a High Dimensional Uncertainty Space
Uncertainties Levers
Economic Parameters (N&S)14 parameters
Demographic Parameters8 parameters
Environment Parameters (N&S)7 parameters
Future Generations (N&S)10 parameters
6 parameters
Measures Relationships
4 measures 14 state equations
27
11-15-03
RDM Employs an Iterative Process
Suggest candidate robust Suggest candidate robust strategystrategy
Initial choice is contingent on Initial choice is contingent on probability weighting across probability weighting across futuresfutures
Characterize breaking scenariosCharacterize breaking scenarios i.e., clusters of futures where i.e., clusters of futures where
strategy performs poorly strategy performs poorly independent of assumed independent of assumed weightingsweightings
Identify tradeoffs among well-Identify tradeoffs among well-hedged strategieshedged strategies
28
11-15-03
Scanning Across All Scenarios Suggests a Candidate Robust Strategy
No increase
Stay the course
All North
All South
Mostly North
North & some South
Policy
Regret
29
11-15-03
Analytic Tools Generate “Narrative” Scenarios
RDM identifies low-dimensional, easy-to-interpret regions where candidate strategy performs poorly Used Friedman and Fisher’s (1999) Patient Rule Induction Method (PRIM) “Low Global Decoupling” scenario is defined by 3 of 41 parameters Scenario suggests important data for consideration by decisionmakers
-0.03 -0.00288 0.03
0.0004 0.00812 0.04
-0.01 0.0139 0.05
North's Innovation Rate
Difference in InnovationRate bet. the N. and S.
North's Economic
Growth Rate
1950-99 (U.S.)
1960-99 (India)
1963-99 (Brazil)
1978-99 (China)
1993-99 (Russia)
1890-1930 &
1890-1930 (U.S.) 1950-99 (U.S.)
30
11-15-03
RDM Analysis Helps Policymakers Focus on a Small Number of Key Tradeoffs
Assessment of adaptive “milestone” sustainability strategies over two computer-generated scenarios
0.000 0.002 0.004 0.006 0.008 0.010 0.012
Regret in SV01.005.002 Satisficing Futures
0.00
0.02
0.04
0.06
0.08
Reg
ret
in L
ow
Glo
bal
Dec
ou
pli
ng
Fu
ture
s
SV02.010.015
Safety valve strategyMilestone strategy
M12
SV01.010.015
SV01.005.002
M12
SV02.005.015
M22
M0XM13
Regret in SV01.005.002 “Satisficing” Futures
Regret in low-global- decoupling futures
31
11-15-03
Analysis Ends by Characterizing Uncertainties which Drive Policy Choices
1:100 1:10 1:1 10:1 100:1
Stringent milestonesand lax cost constraints(SV01-1%-1.5%)
SV02-1%-1.5%
SV01-0.5%-0.2%
SV01-1%-1.5%RobustRegions}
SVab-x%y%a = N milestoneb = S milestonex% = N cost thresholdy% = S cost threshold
Lax/lax
Stringent/stringentStringent/lax
Lax milestonesand lax cost constraints(SV02-1%-1.5%)
Stringent milestonesand stringent cost constraints(SV01-0.5%-0.2%)
Relative Odds of A Low Decoupling Future
ExpectedRegret
32
11-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
33
11-15-03
Different Methods Appropriate in Different Circumstances
Scenario Planning
Robust DecisionsPredict-Then-Act
Uncertainty
Com
plex
ity
Hedging OpportunitiesWell-
characterizedDeep Many
Few
Low
High
34
11-15-03
Robust Decision Making Adds Another Means to Characterize Uncertainty for Decisionmakers
Information about future characterized by identifying robust strategies and their vulnerabilities
Complicated technology supports simple operational concept
Focus on alternative policies may require Closer coordination between analyst and
decisionmakers
Changes in process in organizations that use analysis
35
11-15-03
36
11-15-03
Landscape ofplausible futures
Alternative strategiesX
Ensemble of scenarios
Robuststrategies
What About Surprises?
37
11-15-03
The Advisory Panel Suggested Several
Potentially Stressing Surprises
Rapid technological advance that eliminates emissions
Plague that decimates population for twenty years
Future generations whose values (utility) are completely disconnected from concern about the environment
38
11-15-03
“Safety Valve” Strategy Is Still Robust, Even with Surprises
–1.0
0
1.0
5.0
3.0
4.0
2.0
No surprise
1.0 2.0 3.0 4.00–1.0
0
1.0
5.0
3.0
4.0
2.0
Population surprise
–1.0
0
1.0
5.0
3.0
4.0
2.0
Technological surprise
1.0 2.0 3.0 4.00–1.0
0
1.0
5.0
3.0
4.0
2.0
Value surprise
N$ W$
NG WG
Economic growth rate
Rate of change in emissions intensity
top related