Uncertainty and Integrative Issues

Bryan C. Pijanowski and Marianne HubnerJanuary 6, 2005

INTEGRATIVESpatial and temporal scales

Uncertainty analysisFeedbacks and tipping points

Practices of scienceBroader impacts

CLIMATE DYNAMICS

Regional Local

LAND COVERNPP SIMULATIONS

LAND USE CHANGE

Case Studies Models Role Playing

Games

Crops Rangeland Remote Sensing

CaseStudies

Human Systems

Global Climate

Presentation

• Provide overview of uncertainty discussion thus far• Review some issues related to “Practices of

Science”• Summarize several integrative questions that was

in the proposal that we need to keep in mind once models begin to be coupled

UncertaintyToward A Rubric and Paper

Publication Issues

• Focus on very basic issues and concepts in each of our fields. Use examples from our study if available now but not the emphasis

• Target journal: Proceedings of the National Academy of Sciences

• Target audience; for grad classes in any of our courses in geography, ecology, geosciences. More of a synthesis paper rather than a single advancement paper

• Model paper: Peters et al (with Pielke; Oct 2004; nonlinear patterns in climate-land modeling)

Potential/Working Title

• Treatment of uncertainty in analysis and modeling of climate-land interactions at regional scales

1. Description of the “system”

• What is the system and what do we know about it now (use stuff from previous proposals; a nice diagram of the system from IPCC would be nice). Needs to be short and sweet, direct. Use our systems diagram from the proposal.

2. Definitions

• Definitions of uncertainty vary across disciplines. We need to describe what these definitions are and relate these to the system that we are trying to study. Perspectives thus far:– Systems Engineering/Artificial Intelligence perspective – Cognitive Scientists (learning and culture)– From Statistical perspective (how it differs from error) (Marianne).

See NIST standards site; measurement error in terms of Type A and Type B Error.

– From Science/Policy (climate) perspective • Regional scale

– Climate perspective (size; meaning)

Toward a Definition (?)

Donald Rumsfeld - US Secretary of Defense

"Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns - the ones we don't know we don't know".

2. Definitions

National Institute of Standards and Technology (USA)

2. Definitions

2. Definitions

2. Definitions

2. Definition (problems)

• Relying solely on a quantitative definition of uncertainty and equating it with statistical error has several problems:– You need an adequate sample size to reliably estimate error– We will have some situations with an N=1, giving degrees of

freedom of 0– We will need to deal with projections which are not likely to

include important factors that will occur in the future– Narrative descriptions will assign qualitative levels of

uncertainty which cannot be incorporated into quantitative assessments (maybe use fuzzy sets)

– It inadequately quantifies uncertainty of future events that are not present in historical data that would be used to estimate error

2. Definition of Uncertainty

Statistical

Qualitative

Unknown and Not

Understood

UncertaintyUncertainty

Incorrectly Assumed

3. Sources of uncertainty

#1. Data (lack of data, data representation of process or state, inherent error (accuracy and precision), abundance of certain types of data; level of data completeness; lack of adequate time series of data; changing data standards across time and space; purpose of data collection and its subsequent application to problem; integrating data at different spatial scales)

#2. Scientific Understanding (Lack of rigorous scientific investigations at the interstices; multidisciplinary nature of work; terminology and paradigm challenges; lack of integrative theories that encompass biophysical and social sciences (Panarchy)

3. Sources of uncertainty

#3. Model (what is a model and why do we use them; simple to complex representation of reality; history versus future; reduced-form versus structural model types; predictable drivers and unknown drivers; representation of all drivers versus controlling drivers; exogenous factors; global parameters versus use of location specific variables (e.g., LAI); defining ‘initial states/conditions/boundary conditions; propagation of error through system of models; temporal trajectories of futures- what the error signal looks like. See figure). How we communicate the value of our models to policy analysts using terms like validation, verification. Lack of adequate descriptions of model assumptions.

3. Sources of uncertainty

#4. Policy (linkage to policy and action/behavior; veridicality and model transparency; strength; enforcement/penalties; policy tradeoffs; consensus seeking versus truth seeking; policy formulation without quantitative analysis).

3. Sources of uncertainty

4. Methods to Address Uncertainty

• Data Analysis Practices (per Hubner)– Examine variance across spatial and temporal scales (use GIS to

produce maps that integrate data and provide visualization products)

– Identify outliers in data; determine if error or contains high value information

– Examine pdfs (tests for kurtosis, hetereoskadistic variances)– Aggregation of data at different spatial scales using homogenous

locations of smaller scale variable (see Figure)– Apply multiple data sources to increase confidence– Combine/compare data collected using different methods at

different scales (aerial photography and on the ground surveys of ownership boundaries)

– Scaling up using remote sensing– Need for long-term monitoring (LTER and NEON)

4. Methods to Address Uncertainty

• Modeling Practices– Use of surrogates to model general processes internally and exogenously– Clearly articulate model assumptions and model strengths and weaknesses– Never believe our models; use models to test ideas, confirm our understanding– Determine how data errors and amount of system entropy (disorder) affects

model performance– Capturing feedbacks and nonlinear relationships– Use of qualitative approaches (expert judgment; RPS; Turing tests) to integrate

with quantitative methods (parameter sweeping; identifying endpoints; scenario identification)

– Model performance assessment: trends, historical fit; rates, location (kappa) and patterns (FRAGSTATS)

– Combing the use of reduced form and structural models (use structural to understand process that can help inform reduced form models)

– Examining model behavior (characterize dynamics in a general way; looking at feedbacks and tipping points)

– Model ensembles (see Figures; executing many different models using millions of runs; Monte Carlo and stochastic simulation; Introduce concept of output space

4. Methods to Address Uncertainty

• Policy– Articulating model assumptions, strengths and limitations– Past model use, failures and successes and ability to rely

on model for current policy formulation– Diffusion of innovations (some are leaders, others are

laggards)– Purpose of model development versus its application to

policy problem (often a mismatch)– Identifying risk and quantifying uncertainty in outcomes– Model “clairvoyance”– Using technology to visualize outcomes– Identifying, using expert judgment, risk and uncertainty

space related to model output space.– Use of terms validation, verification, model sensitivity

analysis and other terms used to describe model quality

Futures Based on Unexpected

Technological Revolution

Futures Based on

Unexpected Political Event

Sustainable Futures

Marginally Sustainable Futures

Unsustainable Futures

All Futures (Future Space)

Time dimension (rate/trajectory)

Spa

tial d

imen

sion

(allo

catio

n/pa

tche

s)

One model “future”(best historical fit)

All futures described by the ensemble of models

“Worst case” scenario

Desired outcomePolicy “distance”

Bookends?

Practices of Science

Practices of Science in Multidisciplinary Research

• We need to recognize that we all have different practices of science

• We all judge different practices of science differently from our discipline

• To close gaps in our understanding at the interstices, we need to blend our practices

• A true test of success? If we change our value and our own practice of science to embrace new practices

A Need for a Project “Motto”

System wide Questions

System wide Approaches

Activity 6. Exploring Climate-Land Feedbacks.A culmination of the project efforts will be feedback experiments to determine the impact

of the climate change on land use, and land use on climate. Once the RCMs, LUC models, and productivity simulations have been developed, the following coupled systems will be examined:

Coupled system #1: Static land use, as a “control.” The climate model will be set to respond to an increase of atmospheric carbon dioxide. This system will serve as a reference for analysis..

Coupled system #2: Land use updated on decadal time steps (two extremes and the most plausible scenarios). The climate and land use models would set to feedback on a decadal basis: the climate would respond to land cover changes over the previous decade, and the land use/cover would then respond to changing climatic conditions as interpreted through the local climate analysis and productivity simulations. The adjusted land cover would then be an input to the climate model, which would continue its execution. It is expected that this feedback exchange would be executed for years 2010, 2020 and 2030.

Coupled system #3: Land cover updated seasonally, and land use by decade (two extremes and the most plausible scenarios). This coupling is identical to #2 except land cover is updated seasonally.

The results from these experiments will be systematically compared across scenarios and across time steps, to examine spatial-temporal dynamics.

System Wide Questions

• Are the feedbacks between land use change and climate linear or non-linear? What interactions appear to have negative feedbacks? Positive feedbacks?

• Are there tipping points that cause one system to change state? What methods are useful to determine tipping points? What are general patterns characteristic of tipping points?

• What are the important spatial and temporal scales of interactions? To what degree does the climate response lag behind land use change, and vice versa? What are the important measures to consider when scaling up from case studies to the region?

• What are the key factors driving the dynamics? What components of the climate-land use system appear to be tightly coupled and which loosely coupled? Is there hidden order within complexity that can be understood and described?

• What is the nature of perturbations? How frequent are changes to the system introduced? Are the perturbations surprises or introduced with some level of predictability?

Base reference with IGBP

Change 10% of IGBPclumped

Change 10% of IGBP

with annual updates

Change 10% of IGBP

w/seasonal & annual updates

Seasonal Dynamics Based On:LAISingle scattering albedo

Change 10% of IGBP

dispersed

Change 10% of IGBP

with annual updates

Change 10% of IGBP

w/seasonal & annual updates

No change vs pattern of LC change

Coupling of 1st order changes by pattern of

change

1st and 2nd order coupling of land and

climate

Comparing 2nd order couplings comparing

land patterns and climate interactions

CLIP East Africa – Michigan State University

Matrix of Climate Land Interaction Simulations

February 7, 2003 Draft

Discussion Points

• Who wants to participate in paper?• Who will be the core writers (Pijanowski lead)?• Timing (skeletal draft complete by end of Feb

2005)• Submission to PNAS by May 2005

time

Measure of change

Extrapolation of

historical change

Lower limits of possible future trajectories

Uppe

r lim

its o

f

poss

ible

futu

re

traje

ctor

ies

Breadth of possible futures