robust decision making
DESCRIPTION
Robust Decision Making. Robert Lempert RAND. HDGC Seminar February 13, 2004. How Should Climate-Change Uncertainties Be Characterized for Decisionmakers?. Key climate change uncertainties include “Basecase” emissions Behavior of perturbed climate system - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/1.jpg)
Robust Decision Making
Robert Lempert
RAND
HDGC Seminar
February 13, 2004
![Page 2: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/2.jpg)
211-15-03
How Should Climate-Change Uncertainties Be Characterized for Decisionmakers?
Key climate change uncertainties include “Basecase” emissions Behavior of perturbed climate system Value our descendants place on impacts of climate
change Costs of abatement with future technology
Climate-change decisionmakers must understand uncertainties to make effective choices
![Page 3: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/3.jpg)
311-15-03
Analytic Tools Often Vital to Clarify Thinking, But Can Illuminate Trees Rather Than Forest
Analytic tools often vital in improving complicated decisions: Can successfully summarize vast quantities of information Help address flaws in human reasoning
Traditional analytic methods assume well-characterized risks and policy choices based on predictions
But strategic decisions can go awry if decision-makers assume risks are well-characterized when they are not
Uncertainties are underestimated Strategies can be brittle Misplaced concreteness helps blind decision-makers to surprise
Predict Act
![Page 4: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/4.jpg)
411-15-03
Global Scenario Group offers three families of sustainability scenarios Conventional worlds Barbarization Great Transformations
These scenarios Capture a wide range of factors which may affect the
future Attempt to make an argument for a particular risk-
management strategy
Scenarios Capture and Communicate Information About Future, But Hard to Link to Actions
![Page 5: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/5.jpg)
511-15-03
Should Analysts Put Probabilities on Scenarios
Such as Those Developed by SRES?
Pros Necessary to make
policy Others will provide
likelihood estimates if experts don’t
Cons Little evidence to support
judgments about probabilities
Arguing over likelihoods distracts from reaching consensus on near-term actions
Desire for concreteness driving IPCC towards placing probabilities on scenarios
![Page 6: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/6.jpg)
611-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
![Page 7: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/7.jpg)
711-15-03
Climate Change is a Problem of Decisionmaking Under Deep Uncertainty
Deep uncertainty is: When we do not know, and/or key parties to the decision do not agree
on, the system model, prior probabilities, and/or “cost” function
Under conditions of deep uncertainty, decision-makers: Often seek robust strategies, ones which perform reasonably well
compared to the alternatives across a wide range of plausible futures, evaluated with a range of values
Robust strategies are often (but not always) adaptive, that is they evolve over time in response to new information
Often use choice of strategy, not additional information, to reduce uncertainty
![Page 8: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/8.jpg)
811-15-03
Robust Decisionmaking (RDM)
Robust decisionmaking Is an iterative, analytic process that identifies
• Strategies that perform well over a wide range of futures
• Remaining vulnerabilities of these strategies Made possible by advances in computational
capabilities Characterizes uncertainties most important to the
choice among strategies
![Page 9: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/9.jpg)
911-15-03
Four Key Elements of Robust Decision Making
Consider large ensembles (hundreds to millions) of scenarios
Seek robust, not optimal strategies Achieve robustness with adaptivity Design analysis for interactive exploration of a
multiplicity of plausible futures
![Page 10: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/10.jpg)
10
11-15-03
Consider Ensembles of Many Scenarios
On the occasion of the 1893 World Columbian Exposition, 74 experts wrote essays predicting what the United States would look like in 1993
Most were wrong
But some were strangely close to the truth
![Page 11: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/11.jpg)
11
11-15-03
Use Robustness Criteria to Judge Alternative Strategies
Under deep uncertainty, decision makers often seek robust strategies that work reasonably well over a wide range of plausible futures
We measure robustness according to degree of “regret,” which is defined as the difference between
the performance of a strategy in a given future, and
the performance of the best strategy in that future
![Page 12: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/12.jpg)
12
11-15-03
Combine Human and Machine Capabilities
Landscape ofplausible futures
Alternative strategiesX
Ensemble of scenarios
Robuststrategies
![Page 13: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/13.jpg)
13
11-15-03
Exploratory Modeling Software Supports This Process
Exploratory modeling software enables users to navigate through large numbers of scenarios and Formulate rigorous policy arguments based on these explorations
CARsTM is java-based exploratory modeling software that: Links to virtually any type of model and/or data Supports interactive use of search and visualization to create, explore,
compare, and understand large scenario ensembles
Tools to draw Tools to draw meaning from meaning from
informationinformationUsers
Tools to Tools to represent represent
informationinformation
![Page 14: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/14.jpg)
14
11-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
![Page 15: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/15.jpg)
15
11-15-03
Example Application of Robust Decisionmaking
The RDM approach employed a simple method of representing information “Toy” systems-dynamics model with 41 input parameters
representing uncertainties about• future economic, demographic, and environmental trends• values and capabilities of future decisionmakers
Simple agent-based model of future decisionmakers Four value functions based loosely on UN Human Development
Index, which reflects interests of a range of stakeholders Near-term strategies affect “decoupling” rate
Example: What choice of near-term actions will help ensure Example: What choice of near-term actions will help ensure strong economic growth and a healthy environment over the strong economic growth and a healthy environment over the
course of the 21course of the 21stst century? century?
![Page 16: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/16.jpg)
16
11-15-03
Visualizations Capture Key Relationships Among Plausible Futures
Landscape of plausible futures helps illuminate key challenges to ensuring strong economic growth and a healthy environment over the
course of the 21st century.
Economic growth rate
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
India since 1960
U.S. 1890-1930
U.S. since 1950
U.S. in 20th century
China since 1960
Brazil since 1980
Russia since 1993
Conventional World scenario
Barbarization scenario
Great Transition scenario
Decoupling rate
![Page 17: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/17.jpg)
17
11-15-03
Compare “Fixed” Near-Term Strategies Across Scenarios
Near Term
Choose policies
Assume near-term policy continues until changed by future generations
Future decision-makers recognize
and correct our mistakes
Future
![Page 18: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/18.jpg)
18
11-15-03
Look for Robust Strategies
XLandscape of
plausible futures
Alternative strategies
Ensemble of scenarios
Robuststrategies
![Page 19: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/19.jpg)
19
11-15-03
Strategies Should Be RobustAcross Multiple Measures of “Goodness”
Use measures inspired by UN’s Human Development Index (HDI)• Discounted, average rate of improvement in GDP/capita,
longevity, and environmental quality (but no education level) time series
• Four different weightings
N$: North GDP/capita and longevity
W$: Global GDP/capita and longevity
NG: North GDP/capita, longevity, and environmental quality
WG: Global GDP/capita, longevity, and environmental quality
![Page 20: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/20.jpg)
20
11-15-03
Speeding Decoupling Performs Well in Many Futures Using North HDI Measure
Slight speed-upSlight speed-up
1.0 2.0 3.0 4.0–1.0
0
5.0
0
N$ W$
NG WG
1.0
3.0
4.0
2.0
Conventionalworld scenario
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
Economic growth rate
Decoupling Rate
No regretMild
A lotOverwhelming
![Page 21: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/21.jpg)
21
11-15-03
But Often Fails for Global Green Measure
1.0 2.0 3.0 4.0–1.0
0
1.0
5.0
3.0
4.0
2.0
0
N$ W$
NG WG
Conventionalworld scenario
Economic growth rate
Decoupling rate
No regretMild
A lotOverwhelming
Slight speed-upSlight speed-up
![Page 22: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/22.jpg)
22
11-15-03
Exploration DemonstratesNo “Fixed” Strategy Is Robust
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
No regretMild
A lotOverwhelming
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
Conventionalworld
scenario
Economic growth rate
Decouplingrate
N$ W$
NG WG
Stay the CourseStay the Course Crash EffortCrash Effort
![Page 23: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/23.jpg)
23
11-15-03
Design and Examine Additional Strategies
XLandscape of
plausible futures
Alternative strategies
Ensemble of scenarios
Robuststrategies
![Page 24: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/24.jpg)
24
11-15-03
Start with a Milestone, but Evaluate Progress Early and Modify Milestone If Necessary (Safety Valve)
NODoes the carrying capacity change?
Choose policies to maximize
utility
Determine best policy to meet milestone
Select near-term milestone
YES
Is milestone achievable with
current approach?
Relax milestone
Present Future
YES
NO
Implement policy
![Page 25: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/25.jpg)
25
11-15-03
“Safety Valve” Strategy Appears Highly Robust
Safety valveSafety valve
Economic growth rate (%)
Dec
ou
plin
g r
ate
(%)
1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
N$ W$
NG WG
Economic growth rate (%)1.0 2.0 3.0 4.0
–1.0
0
1.0
5.0
3.0
4.0
2.0
0
No regretMild
A lotOverwhelming
N$ W$
NG WG
+
WorstCase
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
U.S. in 19thcentury
U.S. since 1950
U.S. in 20thcentury
![Page 26: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/26.jpg)
26
11-15-03
Even Simple Scenario Generator Implies a High Dimensional Uncertainty Space
Uncertainties Levers
Economic Parameters (N&S)14 parameters
Demographic Parameters8 parameters
Environment Parameters (N&S)7 parameters
Future Generations (N&S)10 parameters
6 parameters
Measures Relationships
4 measures 14 state equations
![Page 27: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/27.jpg)
27
11-15-03
RDM Employs an Iterative Process
Suggest candidate robust Suggest candidate robust strategystrategy
Initial choice is contingent on Initial choice is contingent on probability weighting across probability weighting across futuresfutures
Characterize breaking scenariosCharacterize breaking scenarios i.e., clusters of futures where i.e., clusters of futures where
strategy performs poorly strategy performs poorly independent of assumed independent of assumed weightingsweightings
Identify tradeoffs among well-Identify tradeoffs among well-hedged strategieshedged strategies
![Page 28: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/28.jpg)
28
11-15-03
Scanning Across All Scenarios Suggests a Candidate Robust Strategy
No increase
Stay the course
All North
All South
Mostly North
North & some South
Policy
Regret
![Page 29: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/29.jpg)
29
11-15-03
Analytic Tools Generate “Narrative” Scenarios
RDM identifies low-dimensional, easy-to-interpret regions where candidate strategy performs poorly Used Friedman and Fisher’s (1999) Patient Rule Induction Method (PRIM) “Low Global Decoupling” scenario is defined by 3 of 41 parameters Scenario suggests important data for consideration by decisionmakers
-0.03 -0.00288 0.03
0.0004 0.00812 0.04
-0.01 0.0139 0.05
North's Innovation Rate
Difference in InnovationRate bet. the N. and S.
North's Economic
Growth Rate
1950-99 (U.S.)
1960-99 (India)
1963-99 (Brazil)
1978-99 (China)
1993-99 (Russia)
1890-1930 &
1890-1930 (U.S.) 1950-99 (U.S.)
![Page 30: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/30.jpg)
30
11-15-03
RDM Analysis Helps Policymakers Focus on a Small Number of Key Tradeoffs
Assessment of adaptive “milestone” sustainability strategies over two computer-generated scenarios
0.000 0.002 0.004 0.006 0.008 0.010 0.012
Regret in SV01.005.002 Satisficing Futures
0.00
0.02
0.04
0.06
0.08
Reg
ret
in L
ow
Glo
bal
Dec
ou
pli
ng
Fu
ture
s
SV02.010.015
Safety valve strategyMilestone strategy
M12
SV01.010.015
SV01.005.002
M12
SV02.005.015
M22
M0XM13
Regret in SV01.005.002 “Satisficing” Futures
Regret in low-global- decoupling futures
![Page 31: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/31.jpg)
31
11-15-03
Analysis Ends by Characterizing Uncertainties which Drive Policy Choices
1:100 1:10 1:1 10:1 100:1
Stringent milestonesand lax cost constraints(SV01-1%-1.5%)
SV02-1%-1.5%
SV01-0.5%-0.2%
SV01-1%-1.5%RobustRegions}
SVab-x%y%a = N milestoneb = S milestonex% = N cost thresholdy% = S cost threshold
Lax/lax
Stringent/stringentStringent/lax
Lax milestonesand lax cost constraints(SV02-1%-1.5%)
Stringent milestonesand stringent cost constraints(SV01-0.5%-0.2%)
Relative Odds of A Low Decoupling Future
ExpectedRegret
![Page 32: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/32.jpg)
32
11-15-03
Outline
Robust decision making
Example of robust decisionmaking as a means of characterizing uncertainty
Conclusions
![Page 33: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/33.jpg)
33
11-15-03
Different Methods Appropriate in Different Circumstances
Scenario Planning
Robust DecisionsPredict-Then-Act
Uncertainty
Com
plex
ity
Hedging OpportunitiesWell-
characterizedDeep Many
Few
Low
High
![Page 34: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/34.jpg)
34
11-15-03
Robust Decision Making Adds Another Means to Characterize Uncertainty for Decisionmakers
Information about future characterized by identifying robust strategies and their vulnerabilities
Complicated technology supports simple operational concept
Focus on alternative policies may require Closer coordination between analyst and
decisionmakers
Changes in process in organizations that use analysis
![Page 35: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/35.jpg)
35
11-15-03
![Page 36: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/36.jpg)
36
11-15-03
Landscape ofplausible futures
Alternative strategiesX
Ensemble of scenarios
Robuststrategies
What About Surprises?
![Page 37: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/37.jpg)
37
11-15-03
The Advisory Panel Suggested Several
Potentially Stressing Surprises
Rapid technological advance that eliminates emissions
Plague that decimates population for twenty years
Future generations whose values (utility) are completely disconnected from concern about the environment
![Page 38: Robust Decision Making](https://reader035.vdocuments.us/reader035/viewer/2022081516/56813aee550346895da3614a/html5/thumbnails/38.jpg)
38
11-15-03
“Safety Valve” Strategy Is Still Robust, Even with Surprises
–1.0
0
1.0
5.0
3.0
4.0
2.0
No surprise
1.0 2.0 3.0 4.00–1.0
0
1.0
5.0
3.0
4.0
2.0
Population surprise
–1.0
0
1.0
5.0
3.0
4.0
2.0
Technological surprise
1.0 2.0 3.0 4.00–1.0
0
1.0
5.0
3.0
4.0
2.0
Value surprise
N$ W$
NG WG
Economic growth rate
Rate of change in emissions intensity