cosmo general meeting 2011 wg5 parallel session 5 september 2011

Federal Department of Home Affairs FDHAFederal Office of Meteorology and Climatology MeteoSwiss

Verification resultsat MeteoSwiss in the year 2011

Francis Schubiger, Vanessa Stauch,Tanja Weusthoff, Pirmin Kaufmann

MeteoSwiss

COSMO General Meeting 2011

WG5 Parallel Session 5 September 2011

2 Verification results at MeteoSwiss in 2011

COSMO GM / WG5 Parallel Session, 05.09.2011

Content

• results of the operational verification in 2011

• separate verification of all eight daily COSMO-2 forecasts

• neighborhood verification (also weather type dependant)

• COSMO-2 03 UTC forecasts up to +48h

• extension of Beth Ebert‘s package: -> „fuzzy in time“

• Operational (COSMO-2, COSMO-7, COSMO-LEPS [mean], IFS):• Surface (3h-steps SYNOP and 1h-steps SMN [Swiss Met Net])

• psred, ps, T2m, Td2m, 10m-wind, cloud cover, gusts,precipitation (1h, 12h), global radiation

• Upper-air (TEMP)• T, RH, wind,

• Quasi-operational (COSMO-2, COSMO-7, IFS):• precipitation with RADAR (neighborhood/fuzzy verification, Ebert’s

package) also weather-type dependant• Monitoring: SMN (swiss SYNOPs) , RADAR, windprofilers• Ongoing in experimental modus:

• windprofiler over Switzerland• fluxes at Payerne

• New / Next developments:• VAD (locations of swiss radars)

=> All these packages should be replaced by VERSUS 2

Verification of COSMO-2, COSMO-7, COSMO-LEPS and IFS

T2m: mean diurnal cycle (first 24h forecasts)domain Switzerland (hourly SYNOP‘s)

V. Stauch, P. Kaufmann

COSMO-7

COSMO-2

Autumn 2010

Winter 2010/2011

Spring 2011

Summer 2011

T2m: mean diurnal cycle Spring 2011COSMO-7 vs IFSSYNOP‘s over COSMO-7 domain

COSMO-7

T2m: mean diurnal cycle Spring 2011COSMO-7 vs COSMO-LEPSSYNOP‘s over COSMO-7 domain

COSMO-7

COSMO-LEPS

Global radiation (inclined): Spring 2011COSMO-7 mean diurnal cycleSYNOP‘s over Switzerland

Global radiation (horizontal): Spring 2011COSMO-7 mean diurnal cycleSYNOP‘s over Switzerland

Precipitation: Spring 2011geographical distribution

precipitation12h sums

frequency bias(1 mm/12h)

Spring 2011

COSMO-7+24 & +48h

Precipitation (12h-sums +12 to +24h):Spring 2011 over Switzerland (SYNOP‘s)frequency bias: COSMO-7 & COSMO-2

observed frequency

V. Stauch

Precipitation (12h-sums +36 to +48h):Spring 2011 over Switzerland (SYNOP‘s)frequency bias: COSMO-7 & IFS

observed frequency

V. Stauch

Precipitation (12h-sums +12 to +24h):Spring 2011 over Switzerland (SYNOP‘s)COSMO-7 & COSMO-2

V. Stauch

for both models mean over 9 gridpoints foreach station

Precipitation (12h-sums):Spring 2011 over Switzerland (SYNOP‘s)COSMO-7 & IFS

V. Stauch

Hourly precipitation: diurnal cycleSwiss SYNOPs (1h acc) CH-radarcomposit (3h acc)

03 09 15 21 00

fcst: +12..+24h fcst: +3..+6hRADAR

COSMO-7

COSMO-2

COSMO-7

COSMO-2

COSMO-7

03 09 15 21 00

fcst: +12..+24h fcst: +3..+6h

V. Stauch

• 2m-temperature:

• cold bias in winter and warm bias in summer• positive bias more pronounced during nighttime• COSMO-2 ~0.5 K warmer (higher values) than COSMO-7

• 2m-dewpoint:• negative bias (~1 K) in the summer and autumn period

• 10m-windspeed:• negative bias along the coast and on mountains, positive bias inland

• total cloudiness:• mean daily cycle not well represented (mainly overestimation during night

and underestimation during day)• precipitation:

• overestimation (except in Summer ~ no bias) most pronounced in Winter (over COSMO-2 domain: 30-35% ; over Swizerland ~45%)

• higher amounts in COSMO-2 / higher amounts over Alpine area• low amounts (0.1 mm/12h): overestimated in Winter (over the Alps all

seasons)• high amounts (10 mm/12h): underestimated in COSMO-7 (over full domain

by ~30% in summer, 20% in winter)

Verification of surface weather parameters

TEMPS verification: Windspeed +24hWinter 10/11

COSMO-7

COSMO-2

all TEMPsCOSMO-2domain

• temperature: cold bias (~ 0.5 K) in winter and warm bias (~ 0.5 K) in summer from ground up to 300 hPa

• windspeed: positive bias in PBL (up to +1 m/s in winter) and slight negative bias above 400 hPa

Verification with vertical profiles: main results

Separate verification of the eight daily COSMO-2 forecasts• questions of interest:

• which is the best forecast for the afternoon/evening convection ? – is it the latest forecast or perhaps a forecast with a longer „spin up“ time ?

• are there significant differences between the eight forecasts ?

• period:COSMO-2 forecasts from 1 May to 31 July 2011• Verification with hourly obs over Switzerland

COSMO-2: precipitation diurnal cycleforecasts of 00, 03, 06 and 09 UTC

Vanessa Stauch

COSMO-2: precipitation diurnal cycleforecasts of 12, 15, 18 and 21 UTC

Vanessa Stauch

COSMO-2: precipitation diurnal cycleall eight daily forecasts

Vanessa Stauch

COSMO-2: 2m-temperature diurnal cycleall eight daily forecasts

Vanessa Stauch

COSMO-2: 2m-temperature biasall eight daily forecasts

Vanessa Stauch

COSMO-2: 10m-wind speed diurnal cycleall eight daily forecasts

Vanessa Stauch

results for 20103h accumulated precipitation sumsover the domain of the swiss radar composit

models: COSMO-2 and COSMO-7for all 8 daily forecast runs, precipitation sums from +3 to +6h

observation precipitation estimates of the swiss radar composit

in case of a missing value, the full date will not be evaluated

Neighborhood verification for precipitation

COSMO-2 COSMO-7 COSMO-2 - COSMO-7

goodbadCOSMO-7 better COSMO-2 better

Neighbourhood Verification, January–December 2010 3h sums (+3 ..+6h) Fractions Skill Score (top) and Upscaling (bottom)

Fractions Skill Score

Upscaling

T. Weusthoff

FSS, COSMO-2 minus COSMO-7, 2010

Winter Spring

AutumnSummer

numbers = FSS-Score of COSMO-2

colours = differences COSMO-2 minus COSMO-7

COSMO-2 better

COSMO-7 better

• COSMO-2: similar skill in all seasons

• COSMO-2 better than COSMO-7 on almost all scales

• COSMO-2 better than COSMO-7 especially in Winter and Summer

Tanja Weusthoff

COSMO-7 better COSMO-2 better

differences in Fractions Skill Score for weather-type dependant verif

COSMO-2 minus COSMO-7

YEAR 2010

Summary neighbourhood Verifikation precipitation in 2010

• COSMO-2 better than COSMO-7 on all scales,differences become less with increasing leadtime

• good forecast of the spatial structure on higher scales

• The skill of the models varies for different weather types and the differences between COSMO-2 and COSMO-7 varies also:- best skill: Autumn and Spring, south to northwest weather types- greatest difference COSMO-2 minus COSMO-7: Summer and Winter, north- and east types, convective cases

Tanja Weusthoff

COSMO-2 03 UTC up to +48h:Comparison COSMO-2 to COSMO-7• period (full year): 1 June 2010 – 31 May 2011• Main interest:

• advance of COSMO-2 as compared to COSMO-7• question: “are the COSMO-2 forecasts at +48h

still better than those of COSMO-7” ?• Verification with:

• SYNOP over Switzerland (hourly):comparison +0 to +24h vs +24 to +48h

• Swiss RadarComposit: neighborhood Verifikation for four 12h-Summen (0..+12, +12..+24, +24..+36, +36..+48h)

• TEMPs for +21h and +45h

Verification SYNOP-CH: 2m-temperatureCOSMO-2 vs COSMO-7

COSMO-2COSMO-7

advance of COSMO-2 also in the second day

V. Stauch

Verification SYNOP-CH: precipitation COSMO-2 vs COSMO-7 [+0 to +24h]

Frequency bias (mm/h)

COSMO-2COSMO-7

0.1 mm/h

10 mm/h

1.0 mm/h

V. Stauch

COSMO-2COSMO-7

0.1 mm/h

10 mm/h

1.0 mm/h

V. Stauch

Verification SYNOP-CH: precipitation COSMO-2 vs COSMO-7 [+24 to +48h]

Frequency bias (mm/h)

Verification with swiss radarcomposit

second forecast day

first forecast day+0.. +12h

+24..+36h

+12..+24h

+36..+48h

daytime: 03-15 UTC nighttime: 15-03 UTC

T. Weusthoff

COSMO-2 COSMO-7 COSMO-2 COSMO-7

FRACTIONS SKILL SCORE: COSMO-2 – COSMO-7precipitation sums of 12h

+ 0 .. + 12h

+ 24 .. + 36h

+ 12 .. + 24h

+ 36 .. + 48h

T. Weusthoff

COSMO-2 better

COSMO-7 better

UPSCALING: COSMO-2 – COSMO-7precipitation sums of 12h; score: ETS (equitable threat score)

+ 0 .. + 12h

+ 24 .. + 36h

+ 12 .. + 24h

+ 36 .. + 48h

T. Weusthoff

COSMO-2 better

COSMO-7 better

3. „Fuzzy in Time“

• Extension of the spatial window with a window in time volume (dx * dy * dt)

• Evaluation of the forecasts in this volume• Time-window ntm = [1h,3h,5h,7h,9h]

T. Weusthoff

dymodel

observation

e.g. FSS

fraction blue pxiels model = 12/75 (dt=3), 5/25 (dt=1)

fraction blue pxiels obs = 12/75 (dt=3), 3/25 (dt=1)

T. Weusthoff

00 01 02 03 04 05 06 07 08 09 10 11 12 13 1420 21 22 23

hourly accumulated precipitation

Evaluation of 3h accumulations, time window ntm = 5 (t0 +-2h)

00 UTC 03 UTC 06 UTC 08 UTC

22 UTC 02 UTC

01 UTC 05 UTC

3 hourly accumulated precipitation

Evaluation of 3h accumulations, time window ntm = 1

09 UTC00 UTC 03 UTC 06 UTC

23 UTC 01 UTC 02 UTC 04 UTC

T. Weusthoff

FSS for different time-windowsCOSMO-2, July 2010

T. Weusthoff

COSMO GM / WG5 Parallel Session, 05.09.2011 T. Weusthoff

FSS for different time-windowsCOSMO-7, July 2010

FSS for different time-windows COSMO-2 minus COSMO-7, July 2010

T. Weusthoff

Summary „fuzzy in time“

• FSS increases on all scales with increasing time-window• greatest effect for small spatical scales• lowest effect for high threshods

• Both models show a similar increase• difference COSMO-2 minus COSMO-7 stays equal, resp.

becomes littler for high time-windows

• For Upscaling the influence of a time tolerance is relatively low und restricted on low thresholds ( effect of the avergaing)

Application of time-windows on the gridsclae would make sense; simultaneous application with space tolerance brings no great change

T. Weusthoff

cosmo general meeting 2011 wg5 parallel session 5 september 2011

Documents

lrp2010 wg5 fundamental interactions nathal severijns (...

cosmo-clm2 : current status and · cosmo-clm experiment...

wg5 report to tc 20101127-draftv3

european regional reanalysis activities · cosmo-rea6,...

issues for wg5, october 16 th 2012 michael kobel: ippog...

cosmo general meeting, 5 – 9 september 2011, rome, italy...

wg5 activities pp versus2 - cosmo model · avt data policy...

dubrovnik - ewglam/srnwp 8-11/10/ 2007 cosmo strategy for...

wg5 highlights - heavy flavours

overview of canpolin wg5: ecosystems

cosmo 2011 the phase ii simple dark matter search · cosmo...

cosmo-sreps cosmo priority project

the new features of fortran 2003 jtc1/sc22/wg5 n1579 iso/iec...

wg5 summary - argonne national laboratory

afad 2018 summary of wg5 - indico.ibs.re.kr

cosmo adhesives -...

wg5 summary viewgraphs - aps.anl.gov

iec tc101 wg5 pt2 meeting -...

xiii cosmo general meeting rome, 5-8 september 2011

first experiments with cosmo-1 - cosmo-model.org · first...