decision making under deep uncertainty robert lempert director rand pardee center for longer range...

Decision Making Under Deep UncertaintyDecision Making Under Deep Uncertainty

Robert LempertDirector

RAND Pardee Center for Longer Range Global Policy and the Future Human Condition

July 8, 2009

2

Traditional Planning Methods Can Illuminate Trees Rather Than ForestTraditional Planning Methods Can

Illuminate Trees Rather Than Forest

Traditional analytic methods characterize uncertainties as a prelude to assessing alternative decisions

Traditional analytic methods characterize uncertainties as a prelude to assessing alternative decisions

Predict Act

Climate change confronts decisionmakers with deep uncertainty, where

– They do not know, and/or key parties to the decision do not agree on, the system model, prior probabilities, and/or “cost” function

Decisions can go awry if decisionmakers assume risks are well-characterized when they are not

– Uncertainties are underestimated

– Competing analyses can contribute to gridlock

– Misplaced concreteness can blind decision-makers to surprise

Climate change confronts decisionmakers with deep uncertainty, where

– They do not know, and/or key parties to the decision do not agree on, the system model, prior probabilities, and/or “cost” function

Decisions can go awry if decisionmakers assume risks are well-characterized when they are not

– Uncertainties are underestimated

– Competing analyses can contribute to gridlock

– Misplaced concreteness can blind decision-makers to surprise

3

Forecasting the Unpredictable Can Contribute to Bad Decisions


Gross national product (trillions of 1958 dollars)

2.22.0

1.81.6

1.4

1.2

1.0.8

.6

.4

.2

0180

Energy use (1015 Btu per year)

0

Historical trend

continued1970

1920 19291940

19501960

1910

1973

19731900

1890

20 40 60 80 100 120 140 160

1975 Scenarios• In the early 1970s forecasters made projections of U.S energy use based on a century of data

• In the early 1970s forecasters made projections of U.S energy use based on a century of data

4



Gross national product (trillions of 1958 dollars)

2.22.0

1.81.6

1.4

1.2

1.0.8

.6

.4

.2

0180

Energy use (1015 Btu per year)

0

Historical trend

continued1970

1920 19291940

19501960

1910

1973

19731900

1890

20 40 60 80 100 120 140 160

2000 Actual

1990

19801977

1975 Scenarios2000 Actual

1990

19801977


… they all were wrong


… they all were wrong

5

OutlineOutline

• Robust Decision Making (RDM)

• Climate vulnerability and response option analysis for Inland Empire Utilities Agency (IEUA)

• Observations



• Observations

6

New Technology Allows Computer to Serve As “Prosthesis for the Imagination”

New Technology Allows Computer to Serve As “Prosthesis for the Imagination”

• Robust Decision Making (RDM) is a quantitative decision analytic approach that

– Characterizes uncertainty with multiple, rather than single, views of the future

– Evaluates alternative decision options with a robustness, rather than optimality, criterion

– Iteratively identifies vulnerabilities of plans and evaluates potential responses

• Robust Decision Making (RDM) is a quantitative decision analytic approach that

– Characterizes uncertainty with multiple, rather than single, views of the future

– Evaluates alternative decision options with a robustness, rather than optimality, criterion

– Iteratively identifies vulnerabilities of plans and evaluates potential responses

Candidate strategy

Identify vulnerabilities

Assess alternatives for ameliorating vulnerabilities

• RDM combines key advantages of scenario planning and quantitative decision analysis in ways that

– Decision makers find credible

– Contribute usefully to contentious debates

• RDM combines key advantages of scenario planning and quantitative decision analysis in ways that

– Decision makers find credible

– Contribute usefully to contentious debates

7

RDM Has Effectively Addressed Many Types of Decisions Under Deep Uncertainty

RDM Has Effectively Addressed Many Types of Decisions Under Deep Uncertainty

Energy, Environment, and Climate Change

• Long-Range Natural Resource Management

• Renewable portfolios standards

• Center on climate change decision making

National Security

• Terrorism Insurance

• Force procurement and deployment

• Pre-conflict shaping strategies

Commercial-Sector Applications

• Electric utilities’ strategies under deregulation

• Product and technology planning in the auto industry

8

Compare Alternative Approaches to Managing Catastrophic Event with Unknown ProbabilityCompare Alternative Approaches to Managing Catastrophic Event with Unknown Probability

• Consider town on shore of pristine lake

– Lake can switch abruptly to undesirable and potentially irreversible eutrophic state at unknown pollution concentration

• Citizens must decide how much pollution to emit

– Gain small utility from emitting pollution to lake and lose significant utility if lake goes eutrophic

– Deeply uncertain about location of concentration threshold

• Alternative decision approaches include:

– Optimum expected utility

– Precautionary principle

– Robust decision making

• Consider town on shore of pristine lake

– Lake can switch abruptly to undesirable and potentially irreversible eutrophic state at unknown pollution concentration

• Citizens must decide how much pollution to emit

– Gain small utility from emitting pollution to lake and lose significant utility if lake goes eutrophic

– Deeply uncertain about location of concentration threshold

• Alternative decision approaches include:

– Optimum expected utility

– Precautionary principle

– Robust decision making

Robert J. Lempert and Myles T. Collins., 2007: “Managing the Risk of Uncertain Threshold Response: Comparison of Robust, Optimum, and Precautionary Approaches” Risk Analysis 27 (4), 1009–1026

9

Use Simple Simulation Model of Lake System to Assess Consequences of Town’s Decisions

Use Simple Simulation Model of Lake System to Assess Consequences of Town’s Decisions

•Three “policy levers” describe town’s citizens’ adaptive strategy

– Initial pollution emissions (L0)

– Maximum yearly increase in emissions (L)

– Safety margin (S) – buffer between pollution emissions and estimate of critical threshold (Xcrit)

•Over time, citizens learn true value of critical threshold– Observations increasingly accurate as level of pollution approaches unknown

threshold

Nutrients in LakeNatural Emissions

Nutrient sink

Recycling when eutrophicLt = f(L0,L,S)

AnthropogenicEmissions

Learning

€

Lt = f L0,ΔL,S( )

Lempert and Collins (2007)

10

Well Characterized Uncertainty Suggests An Optimal Strategy

Well Characterized Uncertainty Suggests An Optimal Strategy

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.3 0.4 0.5 0.6 0.7 0.8 0.9

Xcrit

Probability Density

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.3 0.4 0.5 0.6 0.7 0.8 0.9

Xcrit

Probability Density

Optimal Strategy w Uncertainty

Linit 0.37

Safety Margin 3.0

L 0.11

Mean PVU 13.

Probability distribution

for critical threshold

Xcrit


11

Robust Strategies Trade Some Optimal Performance for Less Sensitivity to Assumptions

Robust Strategies Trade Some Optimal Performance for Less Sensitivity to Assumptions

• Represent uncertainty about critical threshold with set of multiple, plausible distributions

• Define expected regret of strategy s contingent on distribution i

where strategy s regret is

• Compared to optimal strategy, a robust strategy has small weighted average of best and worst expected regret

• Represent uncertainty about critical threshold with set of multiple, plausible distributions

• Define expected regret of strategy s contingent on distribution i

where strategy s regret is

• Compared to optimal strategy, a robust strategy has small weighted average of best and worst expected regret

€

ρi X( )

€

R s,i = Rs x( )ρ i x( )dxx

∫

€

Rs x( )=Maxs'pvU s' x( )[ ]− pvU s x( )

€

Vs = zR s,best + 1− z( )R s,worst

€

0 ≤ z <1


12

Find Vulnerabilities of Optimal StrategyFind Vulnerabilities of Optimal Strategy

Regret of Strategy A over different values of Xcrit

0

5

10

15

20

25

30

0.3 0.4 0.5 0.6 0.7 0.8 0.9

Xcrit

Regret of Strategy A

30773

Vulnerabilities

Present value utility of optimum strategy


13

Compare Regret Over Range of FuturesCompare Regret Over Range of Futures

A

B

C

D

E

F

G

0

1

2

3

4

5

6

7

8

9

0 20 40 60 80 100 120 140

Expected Regret over Strategy A's Vulnerable Futures (z=0)

Expected Regret over Initial Priors (z=1)A

B

C

D

E

F

G

0

1

2

3

4

5

6

7

8

9

0 20 40 60 80 100 120 140

Expected Regret over Strategy A's Vulnerable Futures (z=0)

Expected Regret over Initial Priors (z=1)

Optimal Strategy


14

Town’s Citizens Have More Robust Options Than Strategy A

Town’s Citizens Have More Robust Options Than Strategy A

0

2

4

6

8

10

12

14

16

18

20

0.01 0.1 1 10 100

Odds of Strategy A Priors

Expected Regret

A

BC

D

E

G

F

0

2

4

6

8

10

12

14

16

18

20

0.01 0.1 1 10 100

Odds of Strategy A Priors

Expected Regret

A

BC

D

E

G

F

Optimal Strategy

PotentialRobust Strategy


15

OutlineOutline



– What impacts may climate change have on IEUA’s current plans?

– What should IEUA do in response?

• Observations



– What impacts may climate change have on IEUA’s current plans?

– What should IEUA do in response?

• Observations

16

Climate Change Poses Significant Planning Challenge for Water Managers

Climate Change Poses Significant Planning Challenge for Water Managers

• Climate change will likely have large but uncertain impacts on supply and demand for water

• “Stationarity is dead”

– Most agencies already include climate (often implicitly) in many decisions

– Amidst all the uncertainty one thing we do know for sure -- tomorrow’s climate will not be like the past’s

• Relaxing this assumption poses key challenges– How do you adjust plans based on uncertain climate projections?

– How do you communicate these plans, especially when uncertain long-term benefits require near-term costs?

• Climate change will likely have large but uncertain impacts on supply and demand for water

• “Stationarity is dead”

– Most agencies already include climate (often implicitly) in many decisions

– Amidst all the uncertainty one thing we do know for sure -- tomorrow’s climate will not be like the past’s

• Relaxing this assumption poses key challenges– How do you adjust plans based on uncertain climate projections?

– How do you communicate these plans, especially when uncertain long-term benefits require near-term costs?

17

Conducted Vulnerability and Options Analysis for Inland Empire Utilities Agency (IEUA)

– IEUA currently serves 800,000 people

• May add 300,000 by 2025

– Water presents a significant challenge

David G. Groves, Debra Knopman, Robert J. Lempert, Sandra H. Berry, and Lynne Wainfan, Presenting Uncertainty About Climate Change to Water Resource Managers, RAND TR-505-NSF, 2007.

18

– Current water sources include:

• Groundwater 56%

• Imports 32%

• Recycled 1%

• Surface 8%

• Desalter 2%



• May add 300,000 by 2025


Groves et. al. (2007)

19

– Current water sources include:

• Groundwater 56%

• Imports 32%

• Recycled 1%

• Surface 8%

• Desalter 2%

Focus of IEUA’s 20 year plan



• May add 300,000 by 2025



20

Model

Performance of plans

IEUAPlans

System data &climate forecasts

We Built a Model to Assess Performance of IEUA Plans in Different Future States of World

We Built a Model to Assess Performance of IEUA Plans in Different Future States of World

– Model projects future water supply and demand for IEUA service area

• Consistent with IEUA management plans and assumptions

• Reflect plausible trends of climate change

– Model projects future water supply and demand for IEUA service area

• Consistent with IEUA management plans and assumptions

• Reflect plausible trends of climate change

#

#

%

##

##

#

#

`

%

#

#

# #

#

#

#

#

`

#

#

##

#

#%

##

`

# #

%

$

$

Based on WEAP software toolGroves et. al. (2007)

21

GCMs Project Plausible Temperature and Precipitation Ranges for Southern California

GCMs Project Plausible Temperature and Precipitation Ranges for Southern California

– Derived from forecasts from 21 GCMs with A1B emissions scenario

– Each forecast weighted by ability to reproduce past climate and level of agreement with other forecasts

– Derived from forecasts from 21 GCMs with A1B emissions scenario

– Each forecast weighted by ability to reproduce past climate and level of agreement with other forecasts

(Tebaldi et al.)

9

8

7

6

54

3

2

1

-1 0 1 2 3Change in summer temperature (deg C) from 2000 - 2030

Temperature

1

2

3

4 56

7

8

9

-30 -20 -10 0 10 20Percent change in winter precipitation from 2000 - 2030

Precipitation


22

Generate Future Weather Sequences by Resampling Historic Local Climate Records

Generate Future Weather Sequences by Resampling Historic Local Climate Records

KNN method produces hundreds of local weather sequences

– Daily and monthly variability that matches historic Chino climate

– Temperature and precipitation trends that match climate model forecasts

KNN method produces hundreds of local weather sequences

– Daily and monthly variability that matches historic Chino climate

– Temperature and precipitation trends that match climate model forecasts

(Yates et al.)

1980 2000 2020 2040 2060Year

Wetter Neutral

Drier Historical

IEUA

Precipitation

1980 2000 2020 2040 2060Year

Hotter

Warmer

Neutral

Historical

IEUA

Temperature


23

ModelPerformance

of plans

IEUA Plans

System data & climate forecasts

Model Assess Performance of IEUA Plans in Many Different Scenarios

Model Assess Performance of IEUA Plans in Many Different Scenarios

Temp: +1.6Temp: +1.6ooC Precip: -10%C Precip: -10%

Scenario BPlan suffers shortages in adverse future climate

00

5050

100100

150150

200200

250250

300300

350350

400400

20052005 20102010 20152015 20202020 20252025 20302030

YearYear

An

nu

al

su

pp

ly (

taf)

An

nu

al

su

pp

ly (

taf)

Recycled

GroundwaterGroundwater

Local SuppliesLocal Supplies

ImportsImports

Dry-year yieldDry-year yieldSurplusSurplus

ShortageShortage

Temp: +0.7Temp: +0.7ooC Precip: +3%C Precip: +3%

Scenario APlan generates surpluses in benign future climate

00

5050

100100

150150

200200

250250

300300

350350

400400

20052005 20102010 20152015 20202020 20252025 20302030

YearYear

An

nu

al

su

pp

ly (

taf)

An

nu

al

su

pp

ly (

taf)

RecycledRecycled

GroundwaterGroundwater

ImportsImports

SurplusSurplus

Local SuppliesLocal Supplies


24

Many Uncertain Factors Could Impact the Performance of Current IEUA PlanMany Uncertain Factors Could Impact the Performance of Current IEUA Plan

Natural Processes

• Future temperatures

• Future precipitation

• Changes in groundwater processes

Performance of Management Strategies

• Development of aggressive waste-water recycling program

• Implementation of groundwater replenishment

Costs of Future Supplies and Management Activities

• Imported supplies

• Water use efficiency


25

Planners in S. California, for Instance, Face aRange of Possible Future Climate ConditionsPlanners in S. California, for Instance, Face aRange of Possible Future Climate Conditions

Summer-time temperature change(2000- 2030)

+.1C +2.1C0

Likely range

Results based on statistical summary of 21 of the world’s best Global Climate Models

Winter-time precipitation change (2000 - 2030)

+8%-19% 0

Likely range

No change Hotter

WetterMuch drier


26


Natural Processes











27

0 1.0 2.0 3.0 4.0PV shortage cost ($ billions)

2.5

3.0

3.5

4.0

PV supply cost

($ billions) Scenario A

Scenario B

• Adverse climate

• $3.4 billion in supply cost

• $1.9 billion in shortage cost

Current IEUA 2005 Urban Water Management Plan

• Benign climate

• $3.3 billion in supply cost

• $0 in shortage cost

“Scenario Maps” Help Decision Makers Visualize How Plans Evolve Over Many Futures


David G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry: Preparing for an Uncertain Climate Future: Identifying Robust Water Management Strategies, RAND DB-550-NSF, 2008.

28


2.5

3.0

3.5

4.0

PV supply cost

($ billions)

Current IEUA Plan

(200 Scenarios)




29


2.5

3.0

3.5

4.0

PV supply cost

($ billions)

Current IEUA Plan

$3.75 billion cost threshold

Current plan generates high costs in 120 of 200 Scenarios




30

Discover Key Scenarios in Ensembles of Many Model Runs

Discover Key Scenarios in Ensembles of Many Model Runs

1. Ran the model 200 times under different combinations of uncertain factors (e.g. temperature and precipitation trends and others)

2. Used statistical algorithms to identify conditions that lead to 2005 UWMP to perform poorly

3. These factors become key driving forces for “policy-relevant” scenarios

1. Ran the model 200 times under different combinations of uncertain factors (e.g. temperature and precipitation trends and others)

2. Used statistical algorithms to identify conditions that lead to 2005 UWMP to perform poorly

3. These factors become key driving forces for “policy-relevant” scenarios

0

10

20

30

40

50

60

70

80

Number of futures

2.5 3 3.5 4 4.5 5 5.5 6 6.5

NPV total costs ($ billions)

UWMP Forever

Nu

mb

er o

f ru

ns

High Cost(120 runs)

Statistical analysis suggests factors that

contribute most to these undesirable

outcomes


31


2.5

3.0

3.5

4.0

PV

su

pp

ly c

ost

($

bil

lio

ns)

Current IEUA Plan

Statistical Analysis Suggests Key Factors That Create Vulnerabilities for Existing PlanStatistical Analysis Suggests Key Factors

That Create Vulnerabilities for Existing Plan

Natural Processes










These three factors explain 70% of vulnerabilities of IEUA’s current plans


32

Response Options May Help IEUA Address These Vulnerabilities

Response Options May Help IEUA Address These Vulnerabilities


33

Can Quantify Some, But Not All, Of These CostsCan Quantify Some, But Not All, Of These Costs

Costs increase over time

Average Cost

0 200 400 600 800 1000 1200

Saved through efficiency

Recycled

Stormwater Replenishment*

Groundwater

Recycled Replenishment*

Imported (Tier 1)

Imported Replenishment*

Imported (Tier 2)

Desalted Groundwater

Shortages

Cost in 2005 ($/AF)* includes the cost of spreading


34

Should IEUA Act Now or Later to Reduce Potential Climate Vulnerabilities?

Should IEUA Act Now or Later to Reduce Potential Climate Vulnerabilities?

Act now to Act now to augment augment

2005 Plan?2005 Plan?

NO

Monitor, and take Monitor, and take additional action additional action if supplies drop if supplies drop

too lowtoo low

In 2015, 2020, 2025, ….

YESImplement Implement additional additional efficiency, efficiency,

recycling, and recycling, and replenishmentreplenishment

In 2015, 2020, 2025, ….

Monitor, and take Monitor, and take additional action additional action if supplies drop if supplies drop

too lowtoo low


35

Compare Nine Strategies Over200 Scenarios Reflecting Key Uncertainties

Compare Nine Strategies Over200 Scenarios Reflecting Key Uncertainties

0 40 8060 100 12020

Static options

Update options

Number of Scenarios (PV Costs > $3.75 billion)

Current Plan forever

Current Plan + DYY and recycling

Current Plan + replenishment

Current Plan with updates

Current Plan + replenishment with updates

Current Plan + efficiency

Current Plan + efficiency with updates

Current Plan + DYY and recycling with updates

Current Plan + all enhancements


36

Just Allowing IEUA’s Current Plan to UpdateReduces Vulnerability Substantially

Just Allowing IEUA’s Current Plan to UpdateReduces Vulnerability Substantially

0 40 8060 100 12020

Static options

Update options

From 120Down to 30

Number of Scenarios (PV Costs > $3.75 billion)

Current Plan forever

Current Plan + DYY and recycling

Current Plan + replenishment








37

Acting NowReduces Future Vulnerabilities Even More








0 20 403010

Static options

Update options

Number of Scenarios(PV Costs > $3.75 billion)


38









0 20 403010

Static options

Update options

Number of Scenarios(PV Costs > $3.75 billion)

Implementation becomes

more challenging

This analysis helped IEUA decide to make more near-term efficiency investments, and to monitor performance and adapt

as needed down the roadGroves et. al. (2008)

39

OutlineOutline



• Observations



• Observations

40

Conducted Elicitations Among IEUA’s Planners and Community to Estimate

Likelihood of Achieving Goals

Conducted Elicitations Among IEUA’s Planners and Community to Estimate

Likelihood of Achieving Goals

0

.01

.02

.03

.04

Density

40 50 60 70 80Recycling

0

.01

.02

.03

Density

80 90 100 110 120 130GW

Recycling Replenishment

Goal GoalMissgoal

Missgoal

Probability of meeting UWMP goals

Meet Goals

Miss Goals Groves et. al. (2007)

41


Natural Processes











42

Meet recycling goal

Meet replenishment goal

Future climate

New conservation

Percolation decrease

Climate on imports

Miss ExceedMeet

Miss ExceedMeet

Drier Wetter

-5% +20%

-20% 0%

Weak Strong

Explains 70% of high cost cases

Analysis Suggests Factors That Cause Severe Shortages for IEUA’s 20 Year Plan

Analysis Suggests Factors That Cause Severe Shortages for IEUA’s 20 Year Plan

Climate-related uncertainties facing IEUA Climate-related uncertainties facing IEUA


43

RDM Enables Effective Planning Based on Multiple Views of FutureRDM Enables Effective Planning

Based on Multiple Views of Future

• Use many scenarios to imagine the future

– Not a single forecast

• Seek robust strategies that do well across many scenarios assessed according to several values

– Not optimal strategies

• Employ strategies that evolve over time in response to changing conditions

– Not "fixed" strategies

• Use computer as “prosthesis for the imagination”

– Not a calculator

• Use many scenarios to imagine the future

– Not a single forecast

• Seek robust strategies that do well across many scenarios assessed according to several values

– Not optimal strategies

• Employ strategies that evolve over time in response to changing conditions

– Not "fixed" strategies

• Use computer as “prosthesis for the imagination”

– Not a calculator

44

More InformationMore InformationDavid G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry: Preparing for an Uncertain Climate

Future: Identifying Robust Water Management Strategies, RAND DB-550-NSF, 2008.


Groves, David G, David Yates, Claudia Tebaldi, 2008: “Developing and Applying Uncertain Global Climate Change Projections for Regional Water Management Planning,” Water Resources Research, 44(12): W12413


David G. Groves and Robert J. Lempert, 2007: A new analytic method for finding policy-relevant scenarios, Global Environmental Change 17, 73-85.

Robert J. Lempert, Steven W. Popper, Steven C. Bankes, 2003: Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, RAND MR-1626-RPC, Aug.

www.rand.org/ise/projects/improvingdecisions/

David G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry: Preparing for an Uncertain Climate Future: Identifying Robust Water Management Strategies, RAND DB-550-NSF, 2008.


Groves, David G, David Yates, Claudia Tebaldi, 2008: “Developing and Applying Uncertain Global Climate Change Projections for Regional Water Management Planning,” Water Resources Research, 44(12): W12413


David G. Groves and Robert J. Lempert, 2007: A new analytic method for finding policy-relevant scenarios, Global Environmental Change 17, 73-85.

Robert J. Lempert, Steven W. Popper, Steven C. Bankes, 2003: Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis, RAND MR-1626-RPC, Aug.

www.rand.org/ise/projects/improvingdecisions/

45

Thank you!

46

We Also Evaluated How This Analysis Affected Policy-Makers’ Views

We Also Evaluated How This Analysis Affected Policy-Makers’ Views

• Four IEUA workshops presented modeling results to participants including:

– Agency professional managers and technical staff

– Local elected officials

– Community stakeholders

• “Real-time” surveys measured participants’

– Understanding of concepts

– Willingness to adjust policy choices based on information presented

– Views on RDM

• Four IEUA workshops presented modeling results to participants including:

– Agency professional managers and technical staff

– Local elected officials

– Community stakeholders

• “Real-time” surveys measured participants’

– Understanding of concepts

– Willingness to adjust policy choices based on information presented

– Views on RDM


47

First Three Workshops Compared Alternative Approaches to Uncertainty

First Three Workshops Compared Alternative Approaches to Uncertainty

Workshop design approximates on-going laboratory experiments

Compared three approaches

• Traditional qualitative scenarios

• Probabilistic forecasts

• RDM with Scenario discovery


48

RDM Scenarios More Useful, But More Difficult to Understand


Questionnaire item Traditional Scenarios

Scenario Discovery

Provides results that can be used in planning

Agree somewhat

Agree strongly

Provides information on how to improve plan

Agree somewhat

Agree somewhat

Is easy to explain to decisionmakers

Agree somewhat

Disagree strongly

• Traditional scenarios – Gave IEUA much of the information they needed

– Emphasized the importance of achieving goals in IEUA’s plan

• Scenario Discovery – Provided more useful information

– Sparked discussion of adaptive strategies






49



Questionnaire item Traditional Scenarios

Scenario Discovery

Provides results that can be used in planning

Agree somewhat

Agree strongly

Provides information on how to improve plan

Agree somewhat

Agree somewhat

Is easy to explain to decisionmakers

Agree somewhat

Disagree strongly









50

Fourth (Adaptive Strategy) Workshop Compared Different Presentations of RDM

Fourth (Adaptive Strategy) Workshop Compared Different Presentations of RDM

Participants reported:– RDM helped support comparison of climate-related risks and choice among

plans

– Preference for scatter plot over histogram scenario displays

After the workshop:– 35% said consequences of bad climate change now appeared “more

serious” than before

– 40% thought the likelihood of of bad climate change outcomes for the IEUA was “greater” than before

– 75% though the ability of IEUA planners to plan for and manage effects was “greater” than before

Overall, analysis increased:– Perceived likelihood of serious climate impacts

– Confidence that IEUA could take effective actions to reduce its vulnerability to climate change

– Support for near-term efficiency enhancements to current IEUA plan

Participants reported:– RDM helped support comparison of climate-related risks and choice among

plans

– Preference for scatter plot over histogram scenario displays

After the workshop:– 35% said consequences of bad climate change now appeared “more

serious” than before

– 40% thought the likelihood of of bad climate change outcomes for the IEUA was “greater” than before

– 75% though the ability of IEUA planners to plan for and manage effects was “greater” than before

Overall, analysis increased:– Perceived likelihood of serious climate impacts

– Confidence that IEUA could take effective actions to reduce its vulnerability to climate change

– Support for near-term efficiency enhancements to current IEUA planGroves et. al. (2008)

51

ObservationsObservations• Analysis suggests IEUA’s current long-range plans:

– Vulnerable to climate change

– Can be made more resilient by near-term conservation, attention to storm intensity, and effective future monitoring and updating

• Measurements suggest

– RDM analysis effectively shifted views on seriousness of climate challenges and appropriate responses, but requires more work to be easily understood by policy-makers

– Importance of linking effective response options with presentation of climate uncertainty

• Currently using this approach to help several major water agencies include climate in their long-range plans

• Analysis suggests IEUA’s current long-range plans:

– Vulnerable to climate change

– Can be made more resilient by near-term conservation, attention to storm intensity, and effective future monitoring and updating

• Measurements suggest

– RDM analysis effectively shifted views on seriousness of climate challenges and appropriate responses, but requires more work to be easily understood by policy-makers

– Importance of linking effective response options with presentation of climate uncertainty

• Currently using this approach to help several major water agencies include climate in their long-range plans

decision making under deep uncertainty robert lempert director rand pardee center for longer range...

Documents

quantitative decision

blind decisionmakers

s energy use

alternative approaches

s forecasters

response option analysis

emitting pollution

projections of u