bob livezey and marina timofeyeva noaa/nws/ocwws/climate services division
Post on 24-Feb-2016
47 Views
Preview:
DESCRIPTION
TRANSCRIPT
Diagnosis of Skill Variability of CPC Long-Lead Diagnosis of Skill Variability of CPC Long-Lead Seasonal Forecasts: Implications for Seasonal Forecasts: Implications for
Production and UseProduction and Use
Bob Livezey and Marina TimofeyevaNOAA/NWS/OCWWS/Climate Services Division
Climate Diagnostics and Prediction WorkshopOctober 21, 2004
Madison WI
OutlineOutline
• IntroductionIntroduction• Skill StratificationsSkill Stratifications• ResultsResults• Conclusions and LessonsConclusions and Lessons
IntroductionIntroduction
• Users should only care about the performance of forecasts that can potentially benefit their decision process
• Livezey (1990): There are non-random subsets of seasonal forecasts that were skillful enough to be useful
Seasonal Temperature Forecast Skill Seasonal Temperature Forecast Skill 1960s to 80s1960s to 80s
All Seasons 8.3
Winter 12.6Spring 8.6Summer 9.3Fall 2.8
Seasonal Temperature Forecast Skill Seasonal Temperature Forecast Skill 1960s to 80s1960s to 80s
All Seasons 8.3
Winter 12.6Spring 8.6Summer 9.3Fall 2.8
Introduction (Cont.)Introduction (Cont.)
• This talk will make the point :
– That was made by Livezey (1990)– That there are many non-random subsets of forecasts that do
not have useful skill – That it is critical for this information to be shared with
potential users– That skill analyses with different stratifications are highly
informative– That seasonal forecast production should be dominantly
objective– That long-lead seasonal forecasts of non-trend signal should
only be done on appropriate opportunities, not routinely
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts– 1/3 EC’s scored as ‘hits’
• Displays and Stratifications– Summed over all forecasts for each lead for three overlapping
seasons at a time (DJF to FMA, FMA to AMJ, etc.) – Stratified further by strong ENSO years vs. other years– Mapped for appropriate combinations of leads
TemperatureTemperature
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:FMA,MAM,AMJ ENSO:FMA,MAM,AMJOther:FMA,MAM,AMJ
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:AMJ,MJJ,JJA ENSO:AMJ,MJJ,JJAOther:AMJ,MJJ,JJA
TemperatureTemperature
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:JJA,JAS,ASO ENSO:JJA,JAS,ASOOther:JJA,JAS,ASO
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:ASO,SON,OND ENSO:ASO,SON,ONDOther:ASO,SON,OND
TemperatureTemperature
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:OND,NDJ,DJF ENSO:OND,NDJ,DJFOther:OND,NDJ,DJF
-100
1020304050
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:DJF,JFM,FMA ENSO:DJF,JFM,FMAOther:DJF,JFM,FMA
ResultsResults
• Seasonal Temperature Die-Aways:– Moderate to high national-scale skill confined to
Fall/Winter strong ENSO years at short to medium leads– Otherwise skill is dominantly modest and level with lead
(derived from biased climatologies, ie long-term trend)– Worst forecasts are for
• Fall/Winter at short to medium leads for non strong-ENSO years: Forecasters subtracting value
• Summer/Fall at medium to long leads for strong ENSOs: No remedy except science advances
– Short-lead forecasts are better now than for the 1960s-80s, ~14 vs ~8 overall, ~20 vs ~13 for the winter
TemperatureTemperature
-120 -110 -100 -90 -80 -70
3035
4045
-20 20 60
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other: FMA-AMJ (0.5-12.5 lead)
TemperatureTemperature
-120 -110 -100 -90 -80 -70
3035
4045
-20 20 60
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other:JJA-ASO (0.5-12.5 lead)
TemperatureTemperature
-120 -110 -100 -90 -80 -70
3035
4045
0 10 30
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other: OND-DJF (6.5-12.5 lead)
TemperatureTemperature
-120 -110 -100 -90 -80 -70
3035
4045
-20 20 60
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
ENSO: DJF-FMA (0.5-5.5)
ResultsResults
• Seasonal Temperature Maps:– Without a strong ENSO, justification for OND-
DJF forecasts is questionable– For certain regions/seasons/situations skill is
unambiguously useful even for undisciplined, occasional users
PrecipitationPrecipitation
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:FMA,MAM,AMJ ENSO:FMA,MAM,AMJOther:FMA,MAM,AMJ
-10
-5
0
510
1520
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:AMJ,MJJ,JJA ENSO:AMJ,MJJ,JJAOther:AMJ,MJJ,JJA
PrecipitationPrecipitation
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:OND,NDJ,DJF ENSO:OND,NDJ,DJFOther:OND,NDJ,DJF
-10-505
1015
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
All:DJF,JFM,FMA ENSO:DJF,JFM,FMAOther:DJF,JFM,FMA
ResultsResults
• Seasonal Precipitation Die-Aways:
– Barely useable national-scale skill entirely confined to Fall/Winter strong ENSO years in short to medium leads
– Otherwise skill is statistically indistinguishable from zero
– Short-lead forecasts overall seem to be no better now than for the 1960s-80s (but at least are now made two-weeks earlier)
PrecipitationPrecipitation
-120 -110 -100 -90 -80 -70
3035
4045
-20 0 20 40
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other: OND-DJF (0.5-12.5 lead)
PrecipitationPrecipitation
-120 -110 -100 -90 -80 -70
3035
4045
-10 0 5
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other: FMA-AMJ (0.5-12.5 lead)
PrecipitationPrecipitation
-120 -110 -100 -90 -80 -70
3035
4045
-5 5 15 25
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
Other:JJA-ASO (0.5-12.5 lead)
PrecipitationPrecipitation
-120 -110 -100 -90 -80 -70
3035
4045
-20 40 80
AL
AK
AZAR
CA
CO
CT
DE
FL
GAHI
ID
IL IN
IA
KS
KY
LA
ME
MD
MA
MI
MN
MS
MO
MT
NE
NV
NH
NJ
NM
NY
NC
ND
OH
OK
OR
PARI
SC
SD
TN
TX
UT
VT
VA
WA
WV
WI
WY
ENSO: DJF-FMA (0.5-6.5)
ResultsResults
• Seasonal Precipitation Maps:– Justification for long-lead forecasts is
questionable for at least three of six seasonal groups examined
– For certain regions/seasons/situations skill is unambiguously useful even for undisciplined, occasional users
Conclusions and Conclusions and RecommendationsRecommendations
• There are non-random subsets of seasonal forecasts that are skillful enough even for undisciplined, occasional users
• These skills exclusively reflect ENSO and trend signals
• There are many non-random subsets of forecasts that do not have useful skill
• We must share this information with potential users
• Skill analyses with different stratifications are highly informative
• Seasonal forecast production should dominantly rely on objective exploitation of ENSO and trend with subjective modification only after rigorous criteria are met
• Routine production of certain long-lead forecasts should be terminated (can be accommodated by transition to separate trend and high frequency forecasts with latter issued on forecast of opportunity basis only)
top related