ulrich damrath

Long-term trends of the quality of COSMO-EU forecast expressed as the universal score COSI and

others scores for temperature 2 m and wind 10 m- related to

A global score for COSMO models -recommendations from

“The Score Meeting

of 20th August 2007 in Zurich”

Ulrich Damrath

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach 2009

2

Outlook

• The COSMO-Index COSI

• Time series of the index and ist parts @ DWD starting @ 2003

• Time series of RMSE-values starting @ 2000

• Time series of BIAS-values starting @ 1994


3

Approach

Application of a score like the UK-index - basic values

n

2

ofn

1 = rmsvwe VV

n

2of tt

n

1 = rmse

2p

2f

r

r1 SS

chance""T

chance""R

ETS

Continuous elements

Wind

Skill score for wind and continuous elements related to persistence

Skill score for categorical elementsrelated to chance


4

Approach

Application of a score like the UK-index - final score

iii

ii

SSww

1S

The Index, I, is normalised so that the value is equal to 100 on 31st

March 2000.

Thus, the Index is defined as:

100S

SI

0

where: S0 is the value of S on 31st March


5

Scores in MOVI and UKMO

• Continuous parameters: Reduction of varianceRV = 1 – (RMSE prog / RMSE ref)2

where ref = persistence

• Categorical parameters: ETS• ETS = (R – „chance“) / (T –“chance“)

R= number of obs events correctly forecast

T = number of events which were either observed or

forecasted

global score S like

COSMO-index COSI = S/S0 x100

iii

ii

SSww

1S


6

Parameters

• total cloud amount [threshold: 0-2, 3-6, 7-8 temperature [t2m, later: tmin, tmax]

• 10m- windvector

• precipitation [thresholds: 0.2, 2, 10 mm/6h]


7

Verification frequency

• Every 3h• T2m, 10m-wind and cloudiness:

• @ 00, 03,…, 18, 21 UTC

later on: tmin & tmax over 12h

• 6h-sums: precipitation


8

Persistenceone day before forecast day

• Cloud cover wind and temperature

• 6h-sums: precipitation

Verification frequency

03 1206 15 1809 21 24

1206 18 24

Forecast day

03 1206 15 1809 21 24


10

Final definition of the score: 1

isteptimeatjelementforScoreSkillsk

jelementforScoreSkillSK

skSK

ji

j

ijij

:

:

8

1

,

8

1,

Continouos elements, sum over 8 time steps: (start: vv=3, end: vv=24, step: 3)Equal weights for all forecast times!


11

mcategoryin

isteptimeatjelementforScoreSkillsk

jelementforScoreSkillSK

skNM

SK

mji

j

N

imji

M

lj

:

:

*

1

,,

1,,

1

Final definition of the score - 2Categorical elements, sum over N time steps: (start: vv=vvb, end: vv=vve, step: vvs)Cloud cover: vvb=3, vve=24 vvs=3Precipitation: vvb=6, vve=24 vvs=6Equal weights for all forecast times and categories!


12

EN

jj

E

SKN

SK1

1

Final definition of the score - 3

Sum over all elements: Equal weights for all elements!

COSMO-index COSI = S/S0 x100: This is not defined up to now because there is no agreementconcerning the date for S0 . But it seems to be no major problem..


13

German list:10015 Helgoland............. 10028 St Peter-Ording........10035 Schleswig............. 10055 Fehmarn................10091 Arkona................ 10113 Norderney..............10129 Bremerhaven........... 10131 Cuxhaven...............10147 Hamburg Fuhlsbuettel.. 10161 Boltenhagen............10170 Rostock-Warnemuende.... 10184 Greifswald.............10200 Emden-Flugplatz....... 10224 Bremen.................10270 Neuruppin.............. 10291 Angermuende............10315 Muenster/Osnabrueck.... 10321 Diepholz...............10338 Hannover............... 10348 Braunschweig...........10361 Magdeburg.............. 10379 Potsdam................10382 Berlin-Tegel........... 10384 Berlin-Tempelhof.......10385 Berlin-Schoenefeld..... 10393 Lindenberg.............10400 Duesseldorf............ 10427 Kahler Asten...........10453 Brocken................ 10460 Artern.................10469 Leipzig-Schkeuditz..... 10480 Oschatz................10488 Dresden-Klotzsche...... 10496 Cottbus................10499 Goerlitz............... 10501 Aachen.................10506 Nuerburg-Barweiler..... 10513 Koeln/Bonn.............10532 Giessen................ 10544 Wasserkuppe............10548 Meiningen.............. 10554 Erfurt-Bindersleben....10557 Neuhaus a.R. .......... 10567 Gera-Leumnitz..........10577 Chemnitz............... 10578 Fichtelberg............10609 Trier-Petrisberg....... 10616 Hahn...................10637 Frankfurt/M.-Flughafen. 10655 Wuerzburg..............10685 Hof.................... 10688 Weiden.................10708 Saarbruecken-Ensheim... 10738 Stuttgart-Echterdingen.10763 Nuernberg.............. 10776 Regensburg.............10788 Straubing.............. 10805 Lahr...................10836 Stoetten............... 10852 Augsburg...............10870 Muenchen, Flughafen.... 10895 Fuerstenzell...........10929 Konstanz............... 10946 Kempten................10962 Hohenpeissenberg.......


14

COSI@DWD

T 2mV 3.22V3.19 COSMO-EU SSO


15

COSI@DWD - Scores for day 1


16



17



18

COSI@DWD - Scores for different forecast times


19

COSI@DWD - Cloud cover for day 1


20



21



22

COSI@DWD - Wind vector for day 1


23

RMSE - Wind speed for vv=12


24



25



26



27



28

COSI@DWD - Temperature for day 1


29

RMSE - Temperature


30

BIAS - Temperature


31

RMSE - Temperature


32

BIAS - Temperature


33



34



35

COSI@DWD - Precipitation for day 1


36



37



38

Summary• Replacement of LM with COSMO-EU lead at first to remarkable

deterioration of COSI.• Main effect: worse results concerning wind which is the element

that contributes most to the score (not in the sense of the weight but in the sense of it has a higher value compared to the other elements)

• A lot of model changes lead to a remarkable improvement of COSI.• Changes of cloud microphysics brought essential improvement of

temperature forecasts.

• The new diagnostic scheme for T2m amplified this effect.

• The new SSO-scheme was a jump!

• Now COSI is better for all days compared with the beginning of calculation of the score.

ulrich damrath

Documents