© crown copyright met office an introduction to long-range forecasting emily wallace nov 2012
TRANSCRIPT
© Crown copyright Met Office
An Introduction to Long-range Forecasting
Emily Wallace Nov 2012
© Crown copyright Met Office
Content
• How is long-range forecasting possible?
• predictability vs chaos, drivers of predictability, what is predictable?
• Dynamical seasonal prediction systems
• Initialisation, coupled modelling, assessing uncertainty
• Hindcasts
• Bias correction, model climatology, skill assessment
• Products
• Standard products, bespoke products
© Crown copyright Met Office
How is it possible?
© Crown copyright Met Office
Predictability and chaos
‘chaos’
dry wetThe distribution is analogous to the climatology of a meteorological variable (here, rainfall).
The ball drops can be seen as values corresponding to individual years.
• The precise bin in which a ball falls cannot be predicted (‘chaos’).
• If many drops are made, the ‘distribution’ of balls in the bins can be described.
Predictability and chaos
wetdry
‘chaos’
large-scale influences
© Crown copyright Met Office
Individual ball drops are analogous to individual forecasts, all with similar starting points.
The prediction consists in quantifying the difference between the two distributions (climatology and forecast).
Example of large-scale influence:ocean temperatures
• The precise bin in which a ball falls still cannot be predicted (‘chaos’).
• The tilt of the table changes the shape of the distribution (‘predictability’).
• If many drops are made, the new distribution of balls in the bins can be described.
© Crown copyright 2011
time
pred
icta
bilit
y
ICs
SSTs, surface, etc
external forcings
days months years
Sources of predictability: Initial conditions (pressure, temperature, etc)Boundary conditions (SST, soil moisture, etc); External forcing (emissions, solar etc)
Seasonal: probabilistic forecast
© Crown copyright 2011
Example of forcing: sea surface temperature anomalies
The forcing pattern is large scale and slow-varying in time.The impact is also large scale.
© Crown copyright 2011
Teleconnections: typical La Niña impacts
© Crown copyright 2011
Teleconnections: typical El Niño impacts
• ‘climate’ (seasonal averages), not ‘weather’ (conditions on specific days)
• large-area averages, not localised events
• range of outcomes, with probabilities attached to them (risk)
What is predictable at long range?
How are long-range predictions done?
• statistical methods – using empirical relationships derived from historical records
• dynamical methods – using dynamical (climate) models
Statistical models – using empirical relationships derived from historical records
Statistical models are…
• Cheap – equations are far less complex than dynamical models
But…
• Require a long, good quality, observational dataset to train the model on
• Will produce poor predictions if the assumptions change
• Easy to over-tune
© Crown copyright Met Office
Seasonal forecasting with dynamical models
A seasonal forecasting system requires:
• definition of starting point (initial conditions; data assimilation)
• model of the climate system
• description of uncertainties (ensembles)
Dynamical methods: seasonal forecasting systems
© Crown copyright Met Office
For seasonal forecasting, assimilation of ocean state is important
Tropical Atmosphere Ocean array (TAO)
ARGO floats
SST
Subsurface ocean
Climate simulation Vs. seasonal prediction
© Crown copyright Met Office
Climate model
Observed state of ocean/atmosinitialisation time
Synchronised to real world
Forecast from initialisation time
Climate model
Climate simulationArbitrary or climatology start
A seasonal forecasting system requires:
• definition of starting point (initial conditions; data assimilation)
• model of the climate system
• description of uncertainties (ensembles)
Dynamical methods: seasonal forecasting systems
Coupled and uncoupled seasonal forecast systems http://www.wmolc.org “Data ->system configuration”
• Coupled (1-tier) systems
• Model: Includes interactive 3D ocean model
• Initial conditions: atmosphere + 3D ocean
• GPCs: Exeter, ECMWF, Washington, Toulouse, Melbourne, Montreal, Tokyo, Beijing
• Forecast range: typically 6 months +
• Uncoupled (2-tier) systems
• Model: atmosphere only + prescribed SST. Atmosphere ‘forced’ with predicted (or persisted) SST anomalies. No 2-way atmosphere/ocean interaction
• Initial conditions: atmosphere (usually) + SST
• GPCs: Moscow, Seoul, CPTEC, Pretoria
• Forecast range: typically 3-5 months
© Crown copyright Met Office
A seasonal forecasting system requires:
• definition of starting point (initial conditions; data assimilation)
• model of the climate system
• description of uncertainties (ensembles)
Dynamical methods: seasonal forecasting systems
Uncertainty type 1: initial condition uncertainty
© Crown copyright Met Office
Climate model
Data assimilation: ocean/atmos
• Run model forecasts from many slightly different initial conditions• Forecasts may evolve differently • Collectively, the ensemble estimates the range of uncertainty stemming from sensitivity to initial conditions
forecastEnsemble prediction
Uncertainty type 2: uncertainty in model formulation• When climate models are developed choices must be
made on schemes to represent physical processes
• e.g. Convection scheme, radiation scheme...
• Forecasts from the same basic model and same initial state may give different forecasts when different physics schemes are used.
• Choice of physics scheme is often centre dependent
• Model formulation uncertainties are addressed by:
• Stochastically perturbing model variables (and/or tuneable physics parameters) as the model runs
• Combining ensembles from different modelling centres. Typically each centre will have made different choices in model formulation. Thus multi-model: e.g. LC-LRFMME, EUROSIP, APCC
© Crown copyright Met Office
Addressing model formulation uncertainties
© Crown copyright Met Office
Uncertainty in initial atmospheric
state Uncertainty in future atmospheric
state
Ensemble forecast from model 1 explores part of the future uncertainty (from initial
condition uncertainty)
Ensemble forecast from model 2 (i.e. perturbed physics), run from same set of initial states, typically explores additional future uncertainties (from model
formulation uncertainty)
Including representation of model formulation uncertainties gives better sampling of the true uncertainty.
Example ensemble predictionsMet Office GloSea4 system
• Initial condition uncertainty (lagged analysis)
• 21 different initial ocean/atmos states used (daily lag)
• Model formulation uncertainty
• Stochastic (kinetic energy) perturbations to model wind field as the model runs
© Crown copyright Met Office
22nd Feb‘11
23rd
Mar’1121st Feb‘11
Forecast
2 perturbed runs from daily start dates ->14 runs to (7 months) each week, after 3 weeks we have a 42 member ensemble
GloSea4 ensemble prediction of Nino3.4 SST anomaly from March 2010
© Crown copyright Met Office
Ensemble ‘postage’ stamps
© Crown copyright Met Office
Ensemble mean: reinforces commonalities, masks uncertainties
© Crown copyright Met Office
Qualities of the ensemble mean
• Considered the ‘most likely’ single (deterministic) prediction.
• Usually lies near centre of the ensemble distribution
• Picks out the dominant signal:
• Commonalities across the members ‘reinforce’
• Differences across members tend to cancel
Important ‘but’…..
• Quantitative information on uncertainty is removed by the averaging process
Seasonal prediction systems
© Crown copyright Met Office
Climate model
Initialisation:Current state of ocean/atmos
forecast
Ensemble generation
© Crown copyright Met Office
Hindcasts: Correcting model climatology
Hindcasts for bias correction
• Seasonal forecasts have biases. To adjust for biases we generate a set of retrospective forecasts (hindcasts) that describe the ‘climatology’ of the model
• Model climatologies are defined over all retrospective years and all members
• For GloSea4: 14 hindcast years, 12 members = 168 realisations of each season.
• Note: most systems have more ensemble members in the real-time forecast than in the hindcast set.
© Crown copyright Met Office
Hindcast: a collection of forecasts of the past (1996-2009)
Bias corrected forecastobs
Forecastmembers
obs
Hindcastmean
Forecastmembers
All Hindcast runs:~ 12 members14 years (96-09)
Hcst mean
Raw forecast
© Crown copyright 2011
• PROBLEM:
• What does “climate” mean under climate change?
Calculating anomalies: the importance of the reference period
Does it m
atter?
© Crown copyright 2011
Ensemble-mean forecast for the average temperature anomaly over MAM 2011
1996-2009
1981-2010
1971-2000
Reference period
Wallace and Arribas, 2012, MWR
© Crown copyright Met Office
Hindcasts: Generating bias corrected probabilistic forecasts
Probabilistic forecasts and bias correction e.g. precipitation forecast
© Crown copyright Met Office
wet
dry
Observed climatology,
Lower tercile (obs)
Upper tercile (obs)
Model climatology, e.g. wet bias
Upper tercile (model)
Lower tercile (model)
Ensemble member
Member is counted as a prediction of the average (obs) tercile category
Generating probability forecasts from the ensemble
• An estimate of the forecast probability of an event is the proportion of the ensemble members that predict the event
© Crown copyright Met Office
Category No. Members that predict category
Fraction of total ensemble members
Forecast probability
above 5 5/9 55%
average 3 3/9 33%
below 1 1/9 11%
Seasonal prediction systems
© Crown copyright Met Office
Climate model
Initialisation:Current state of ocean/atmos
forecast
Ensemble generation
Retrospective forecasts
(hindcasts)Skill
assessment(verification)
Forecast bias correction
© Crown copyright Met Office
Hindcasts: Assessing skill
Skill of seasonal forecasting systems
• Skill is assessed on the hindcast (covering a number of past years)
• Can (and should) be done in several ways:
• Statistical assessment of skill
• Process based assessment
What is in Malaysia for me?What’s in Malaysia for me?
© Crown copyright 2007
Statistical skill of forecast products, estimated from hindcasts:http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/gpc-outlooks/glob-seas-prob-skillROC curves Reliability
diagram
Looking like an idiot
Look
ing
wel
l pre
pare
d
© Crown copyright 2011
• Forecasts are generated monthly using data from GloSea4 and ECMWF
• Skill (linear correlation) of 6-month forecasts from March to September is detailed below
Skill (linear correlation)
Mar Apr May Jun Jul Aug Sep
TS 0.26 0.49 0.59 0.33 0.55 0.50 0.42
ACE 0.14 0.25 0.74 0.61 0.56 0.46 0.17
Perfect forecasts would have a skill of 1.0
Deterministic skill assessmentSkill of tropical storm seasonal forecast 1987–2009
Skill of seasonal forecasting systems
• Skill is assessed on the hindcast (covering a number of past years)
• Can (and should) be done in several ways:
• Statistical assessment of skill
• Process based assessment
© Crown copyright 2011
Time
Lat
itud
e
Colours: 5-day average rainfall in mm/day, 10°W-10°E
Red line: Timing of monsoon onset , early July
Time
Lat
itud
e
Mean observed rainfall (TRMM1998-2010) GloSea4 mean rainfall (1996-2009), 25 April start dates
•Good agreement between observed and GloSea4 temporal evolution of monsoon and onset timing
•Some skill in predicting late/early onset (ROC score ~0.6)
Seasonal forecasts with GloSea4 of timing of monsoon onset over Sahel
Improving ENSO forecasts
Obs The westward extension of Nino is a common error
in many climate models. It affects remote regions.
High-res model has better ENSO pattern and
teleconnections
Low resolution
High resolution
© Crown copyright 2011
ENSO teleconnections: precipitation
JJA
DJF
Forecast (E-L) Observed (E-L)
Skilful reproductions in the tropics – even for rainfall
Red = El Nino is drier Blue = El Nino is wetter
© Crown copyright Met Office
Examples of simple forecast products:http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/gpc-outlooks
© Crown copyright 2007
‘Raw’ products
© Crown copyright Met Office
Examples of bespoke forecast products and information
GloSea4 Forecast probabilities for 2011 Short Rains (Sep-Nov)
Early onset: Late onset:
Courtesy of Michael Vellinga
Observations for 2011
Plots courtesy of Lizzie Good Courtesy of Michael Vellinga
Forecast products
Deterministic forecasts
• Provides a best estimate and forecast range (±1 stdev interval) for:
• Numbers of named storms
• ACE index
• During the following 6 months
Probabilistic forecasts
• Probability distributions
• Exceedance of thresholds (to aid assessment of risk)
• Help to quantify and communicate the inherent uncertainties in the forecast.
Public forecast
Tailored products
© Crown copyright Met Office
Summary
How is long-range forecasting possible?
• Large scale forcing that evolve slowly can make climate predictable
Dynamical seasonal prediction systems
• Must include: initialisation, a climate model, and a way to assess uncertainty
Hindcasts
• Due to model biases hindcasts are needed for correction of forecasts, They are also used to assess forecast quality, and can lead to model improvements
Products
• We are developing new and exciting bespoke products and would like to make further developments with you
Questions and answers