probabilistic prediction. uncertainty in forecasting all of the model forecasts i have talked about...
TRANSCRIPT
![Page 1: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/1.jpg)
Probabilistic Prediction
![Page 2: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/2.jpg)
Uncertainty in Forecasting
• All of the model forecasts I have talked about reflect a deterministic approach.
• This means that we do the best job we can for a single forecast and do not consider uncertainties in the model, initial conditions, or the very nature of the atmosphere. These uncertainties are often very significant.
• Traditionally, this has been the way forecasting has been done, but that is changing now.
![Page 3: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/3.jpg)
• The work of Lorenz (1963, 1965, 1968) demonstrated that the atmosphere is a chaotic system, in which small differences in the initialization, well within observational error, can have large impacts on the forecasts, particularly for longer forecasts.
• In a series of experiments found that small errors in initial conditions can grow so that all deterministic forecast skill is lost at about two weeks.
A Fundamental Issue
![Page 4: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/4.jpg)
Butterfly Effect: a small change at one place in a complex system can have large effects elsewhere
![Page 5: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/5.jpg)
Not unlike a pinball game
![Page 6: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/6.jpg)
Uncertainty Extends Beyond Initial Conditions
• Also uncertainty in our model physics.
• And further uncertainty produced by our numerical methods.
![Page 7: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/7.jpg)
Probabilistic NWP• To deal with forecast uncertainty, Epstein (1969)
suggested stochastic-dynamic forecasting, in which forecast errors are explicitly considered during model integration.
• Essentially, uncertainty estimates were added to each term in the primitive equation.
• This stochastic method was not computationally practical, since it added many additional terms.
![Page 8: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/8.jpg)
Probabilistic-Ensemble NWP• Another approach, ensemble prediction, was
proposed by Leith (1974), who suggested that prediction centers run a collection (ensemble) of forecasts, each starting from a different initial state.
• The variations in the resulting forecasts could be used to estimate the uncertainty of the prediction.
• But even the ensemble approach was not possible at this time due to limited computer resources.
• Became practical in the late 1980s as computer power increased.
![Page 9: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/9.jpg)
Ensemble Prediction
• Can use ensembles to estimate the probabilities that some weather feature will occur.
•The ensemble mean is more accurate on average than any individual ensemble member.
•Forecast skill of the ensemble mean is related to the spread of the ensembles
•When ensemble forecasts are similar, ensemble mean skill is higher.•When forecasts differ greatly, ensemble mean forecast skill is less.
![Page 10: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/10.jpg)
T
The true state of the atmosphere exists as a single point in phase space that we never know exactly.
A point in phase space completely describes an instantaneous state of the atmosphere.For a model, a point is the vector of values for all parameters (pres, temp, etc.) at all grid points at one time.
An analysis produced to run a model like the eta is in the neighborhood of truth. The complete error vector is unknown, but we have some idea of its structure and magnitude.
e
Chaos drives apart the forecast and true trajectories…predictability error growth.
EF can predicted the error magnitude and give a “probabilistic cloud” of forecasts.
12hforecast
36hforecast
24hforecast
48hforecast
T
48hverification
phasespace
![Page 11: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/11.jpg)
e
a
uc
j
tg
n
2 1 0 1 2 34
2
0
2
4
6
7
-2.96564
Core ,i 2
Cent ,1 2
32.5 ,Core ,i 1 Cent ,1 1
M
T
T
Analysis Region
48h forecast Region
12hforecast
36hforecast
24hforecast
phasespace
![Page 12: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/12.jpg)
A critical issue is the development of ensemble systems that provide probabilistic
guidance that is both reliable and sharp.
![Page 13: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/13.jpg)
Elements of a Good Probability Forecast
• Reliability (also known as calibration) – A probability forecast p, ought to verify with relative
frequency p.– Forecasts from climatology are reliable (by definition), so
calibration alone is not enough.
![Page 14: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/14.jpg)
Elements of a Good Probability Forecast
• Sharpness (a.k.a. resolution) – The variance or confidence interval of the
predicted distribution should be as small as possible.
Probability Density Function (PDF)for some forecast quantity
SharpLessSharp
![Page 15: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/15.jpg)
Early Forecasting Started Probabilistically
• Early forecasters, faced with large gaps in their nascent science, understood the uncertain nature of the weather prediction process and were comfortable with a probabilistic approach to forecasting.
• Cleveland Abbe, who organized the first forecast group in the United States as part of the U.S. Signal Corp, did not use the term “forecast” for his first prediction in 1871, but rather used the term “probabilities,” resulting in him being known as “Old Probabilities” or “Old Probs” to the public.
• A few years later, the term ‘‘indications’’ was substituted for probabilities and by 1889 the term ‘‘forecasts’’ received official sanction (Murphy 1997).
![Page 16: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/16.jpg)
“Ol Probs”
Professor Cleveland Abbe, who issued the first public“Weather Synopsis and Probabilities” on February 19, 1871
•Cleveland Abbe (“Ol’ Probabilities”), who led the establishment of a weather forecasting division within the U.S. Army Signal Corps,
•Produced the first known communication of a weather probability to users and the public.
![Page 17: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/17.jpg)
History of Probabilistic Prediction
• The first operational probabilistic forecasts in the United States were produced in 1965. These forecasts, for the probability of precipitation, were produced by human weather forecasters and thus were subjective predictions. The first objective probabilistic forecasts were produced as part of the Model Output Statistics (MOS) system that began in 1969.
![Page 18: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/18.jpg)
Ensemble Prediction
• Ensemble prediction began an NCEP in the early 1990s. ECMWF rapidly joined the club.
• During the past decades the size and sophistication of the NCEP and ECMWF ensemble systems have grown considerably, with the medium-range, global ensemble system becoming an integral tool for many forecasters.
• Also during this period, NCEP has constructed a higher resolution, short-range ensemble system (SREF) that uses breeding to create initial condition variations.
![Page 19: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/19.jpg)
NCEP Global Ensemble System• Begun in 1993 with the MRF (now GFS)• First tried “lagged” ensembles as basis…using runs of various
initializations verifying at the same time.• Subsequently, they used the “breeding” method to find perturbations
to the initial conditions of each ensemble members.• Breeding adds random perturbations to an initial state, lets them
grow, then reduce amplitude down to a small level, lets them grow again, etc.
• Give an idea of what type of perturbations are growing rapidly in the period BEFORE the forecast.
• Does not include physics uncertainty.• Coarse spatial resolution..only for synoptic features.
![Page 20: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/20.jpg)
NCEP Global Ensemble TodayAt 00Z:• T254L64 high resolution control) out to 7 days, after which this run
gets truncated and is run out to 16 days at a T170L42 resolution• T62 control that is started with a truncated T170 analysis • 10 perturbed forecasts each run at T62 horizontal resolution. The
perturbations are from five independent breeding cycle.
At 12Z:• T254L64 control out to 3 days that gets truncated and run at
T170L42 resolution out to 16 days• Two pairs of perturbed forecasts based on two independent breeding
cycles (four perturbed integrations out to 16 days.
![Page 21: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/21.jpg)
Variety of Ways to View Ensembles and Their Output
![Page 22: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/22.jpg)
![Page 23: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/23.jpg)
![Page 24: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/24.jpg)
![Page 25: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/25.jpg)
![Page 26: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/26.jpg)
The Thanksgiving Forecast 200142h forecast (valid Thu 10AM)
13: avn*
11: ngps*
12: cmcg*
10: tcwb*
9: ukmo*
8: eta*
Verification
1: cent
7: avn
5: ngps
6: cmcg
4: tcwb
3: ukmo
2: eta
- Reveals high uncertainty in storm track and intensity- Indicates low probability of Puget Sound wind event
SLP and winds
![Page 27: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/27.jpg)
Box andwhiskers
![Page 28: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/28.jpg)
Major Global Ensembles• NCEP GEFS (Global Ensemble Forecasting
System): GFS, 21 members every 6 hr, T254 (roughly 70 km resolution), 64 levels
http://www.esrl.noaa.gov/psd/map/images/ens/ens.html)
• Canadian CEFS: GEM Model, 21 members, 100 km grid spacing, 0 and 12Z
• ECMWF: 51 members, 62 levels, 0 and 12Z, T639 (roughly 27 km)
• http://www.ecmwf.int/products/forecasts/d/charts/medium/eps/
![Page 29: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/29.jpg)
Major International Global/Continental Ensembles
Systems• North American Ensemble Forecasting
Systems (NAEFS): Combines Canadian and U.S. Global Ensembles:
http://www.meteo.gc.ca/ensemble/naefs/EPSgrams_e.html
![Page 30: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/30.jpg)
NCEP Short-Range Ensembles (SREF)• Resolution of 32 km• Out to 87 h twice a day (09 and 21 UTC
initialization)• Uses both initial condition uncertainty (breeding)
and physics uncertainty.• Uses the NMM and Regional Spectral Models
and recently the WRF model (21 total members)• http://www.emc.ncep.noaa.gov/SREF/• http://www.emc.ncep.noaa.gov/mmb/SREF/FCST/COM_US/web
_js/html/mean_surface_prs.html
![Page 31: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/31.jpg)
SREF Current System
Model Res (km) Levels Members Cloud Physics ConvectionRSM-SAS 32 28 Ctl,n,p GFS physics Simple Arak-SchubertRSM-RAS 32 28 n,p GFS physics Relaxed Arak-Schubert
Eta-BMJ 32 60 Ctl,n,p Op Ferrier Betts-Miller-JanjicEta-SAT 32 60 n,p Op Ferrier BMJ-moist prof
Eta-KF 32 60 Ctl,n,p Op Ferrier Kain-FritschEta-KFD 32 60 n,p Op Ferrier Kain-Fritsch
with enhanced detrainment
PLUS
* NMM-WRF control and 1 pert. Pair* ARW-WRF control and 1 pert. pair
![Page 32: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/32.jpg)
British Met Office MOGREPS
• 24 members, 18 km
![Page 33: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/33.jpg)
Ensemble Post-Processing
• Ensemble output can be post-processed to get better probabilistic predictions
• Can weight better ensemble members more.• Correct biases• Improve the width of probabilistic distributions
(pdfs)
![Page 34: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/34.jpg)
BMA (Bayesian Model Averaging) is One Example
![Page 35: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/35.jpg)
![Page 36: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/36.jpg)
There is a whole theory on using probabilistic information for
economic savings
C= cost of protection
L= loss if bad event event occurs
Decision theory says you should protect if the probability of
occurrence is greater than C/L
![Page 37: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/37.jpg)
Critical Event: sfc winds > 50kt
Cost (of protecting): $150K
Loss (if damage ): $1M
Hit
FalseAlarm
Miss
CorrectRejection
YES NO
YES
NO
Forecast?
Obs
erve
d?
Decision Theory Example
Deterministic Observation ProbabilisticCase Forecast (kt) (kt) Cost ($K) Forecast 0% 20% 40% 60% 80% 100%
1 65 54 150 42% 150 150 150 1000 1000 10002 58 63 150 71% 150 150 150 150 1000 10003 73 57 150 95% 150 150 150 150 150 10004 55 37 150 13% 150 0 0 0 0 05 39 31 0 3% 150 0 0 0 0 06 31 55 1000 36% 150 150 1000 1000 1000 10007 62 71 150 85% 150 150 150 150 150 10008 53 42 150 22% 150 150 0 0 0 09 21 27 0 51% 150 150 150 0 0 0
10 52 39 150 77% 150 150 150 150 0 0Total Cost: 2,050$ 1,500$ 1,200$ 1,900$ 2,600$ 3,300$ 5,000$
$150K $1000K
$150K $0K
Deterministic Observation ProbabilisticCase Forecast (kt) (kt) Cost ($K) Forecast 0% 20% 40% 60% 80% 100%
1 65 54 150 42% 150 150 150 1000 1000 10002 58 63 150 71% 150 150 150 150 1000 10003 73 57 150 95% 150 150 150 150 150 10004 55 37 150 13% 150 0 0 0 0 05 39 31 0 3% 150 0 0 0 0 06 31 55 1000 36% 150 150 1000 1000 1000 10007 62 71 150 85% 150 150 150 150 150 10008 53 42 150 22% 150 150 0 0 0 09 21 27 0 51% 150 150 150 0 0 0
10 52 39 150 77% 150 150 150 150 0 0Total Cost: 2,050$ 1,500$ 1,200$ 1,900$ 2,600$ 3,300$ 5,000$
Cost ($K) by Threshold for Protective Action
Optimal Threshold = 15%
![Page 38: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/38.jpg)
The Most Difficult Part: Communication of Uncertainty
![Page 39: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/39.jpg)
Deterministic Nature?
• People seem to prefer deterministic products: “tell me exactly what is going to happen”
• People complain they find probabilistic information confusing. Many don’t understand POP.
• Media and internet not moving forward very quickly on this.
![Page 40: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/40.jpg)
![Page 41: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/41.jpg)
![Page 42: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/42.jpg)
Icons are not effective in providing probabilities
![Page 43: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/43.jpg)
Even worse…they use the same icons for likely rain and rain as they do for chance
rain. Also, they used “likely rain” for 70% on this page and “chance rain” for
70% in the example on the previous page
![Page 44: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/44.jpg)
And a “slight” chance of freezing drizzle reminds one of a trip to
Antarctica
![Page 45: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/45.jpg)
Commercial sector
is no better
![Page 46: Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that](https://reader031.vdocuments.us/reader031/viewer/2022032313/56649e555503460f94b4d3be/html5/thumbnails/46.jpg)
A great deal of research and development is required to
develop effective approaches for communicating probabilistic
forecasts which will not overwhelm people and allow them to get value out of them.