validation of h saf precipitation products - cnripwg/meetings/saojose-2012/... · 2016. 6. 7. ·...

Validation of H‐SAF Precipitation Products

Bożena Łapeta, Silvia Puca and Precipitation Validation Team

Satellite Remote Sensing Centre, Institute of Meteorology and Water Management, Kraków, Poland

(In the presentation materials from H‐SAF PVR documents were used)

• H‐SAF prcipitation products validated• Validation with ground data:

– methodology– results

• Conclusions

Presentation Overview

NOAA Training on “New and Emerging Technologies, Sensors, and Datasets for Precipitation” 15‐17 October 2012, Sao Jose dos Campos, Brazil

The PRECIPITATION PRODUCT VALIDATION GROUP is composed by 24 experts in hydrology, rain gauge data, radar data, and meteorology coming from 8 countries.

Precipitation Products Validation Group (PPVG)Coordination : DPC (Italy)

Italy DPC Silvia PucaCoordination [email protected]

Belgium IRM Emmanuel Roulin Pierre Baguis

[email protected]@oma.be

Bulgaria NIMH Gergana Kozinarova [email protected] Georgy Koshinchanov [email protected]

Germany BfG Peter Krahe [email protected] Hungary OMSZ Kereney Judit [email protected] DPC Silvia Puca [email protected]

Gianfranco Vulpiani [email protected]

Emanuela Campione [email protected]

Alexander Toniazzo [email protected]

Uni. Ferrara

Federico Porcù [email protected]

Lisa Milani [email protected]

CIMA Simone Gabellani [email protected] Rebora [email protected]

Poland IMWM Bozena Łapeta [email protected] Iwanski [email protected]

Slovakia SHMÚ Jan Kanak [email protected] Jurasek [email protected]

Luboslav Okon [email protected]

Turkey ITUTSMS

ahmet oztopalIbrahim Sonmez

[email protected]@dmi.gov.tr

Product Resolution (Europe) Cycle (Europe) Timeliness

Precipitation rate at ground by MW conical scanners (H‐01) 10 km (with CMIS)

15 km (with other GPM)

6 h (with CMIS only)

3 h (with full GPM)

15 min

Precipitation rate at ground by MW cross‐track scanners (H‐02)

10 km 6h 5 min

Precipitation rate at ground by GEO/IR supported by LEO/MW(H‐03)

8Km 15min 5min

Cumulated rain 3 and 24 h (H‐05)

10 km

(from merged MW + IR) 3 h 15 min

Precipitation products


H‐SAF validation methodology1. The Common Validation is the result of the validation activities done by all the

Institutes involved in the HPPVG:– both rain gauges (4100 posts) and radar data (40 C band radars) are used;– it is based on statistical scores evaluated on multi‐categorical and continuous

statistics;– the statistical scores are monthly averages;– the same up‐scaling techniques by all the institutes (if proposed by developers).

2. Specific validationEach Institute in addition to the common validation methodology has developed aspecific validation methodology based on its own knowledge and experience.

– lightning data, numerical weather prediction and nowcasting product– case studies: convective/stratiform precipitation, day/night, land/ocean


Comparison between the satellite data and ground data were done on the satellite product native grid.There are several approach to bring the observations comparable:• to compare untransformed data, e.g. comparing areal data to

observations at a nearest gauge station.• to upscale the reference observations to areal averages corresponding

to the resolution of the precipitation products but in an equal‐area map projection (interpolation of RG data; averaging of radar data within the product pixel)

How to compare satellite products with ground data?


Radar and rain gauge instruments provide many measurements within a single satellite IFOV, those measurements were averaged following the satellite antenna pattern of AMSU‐B, SSMI and SEVIRI.

How to combine satellite products with ground data?

Gaussian filter used to average ground data within satellite H‐02 pixels


Rain Gauges network (4100)

I will put the most actual map (from Jakub)


Data Sources RaingaugesInstrument characteristics Telemetric and mechanic

Time domain (near real time/ case studies)

Near real time, case studies

Time resolution (15 min, 30 min)

10 – 30 min (telemetric),3 – 24 h (mechanic)

Spatial distribution (whole national territory/ limited area)

Whole national territory

Number of station (please attach a map)

~390 mechanic (RMI) + 12 telemetric (RMI) + 4160 telemetric (SETHY)

Operational/ for research only

Operational (RMI) + research (other networks)

Data quality check

Telemetric: automatically checked / mechanic: autom. + manually checked

Radar network (40 C‐band)


Data Sources RaingaugesInstrument characteristics Telemetric and mechanic

Time domain (near real time/ case studies)

Near real time, case studies

Time resolution (15 min, 30 min)

10 – 30 min (telemetric),3 – 24 h (mechanic)

Spatial distribution (whole national territory/ limited area)

Whole national territory

Number of station (please attach a map)

~390 mechanic (RMI) + 12 telemetric (RMI) + 4160 telemetric (SETHY)

Operational/ for research only

Operational (RMI) + research (other networks)

Data quality check

Telemetric: automatically checked / mechanic: autom. + manually checked

H-SAF validation methodology

Main steps of the common validation:

• Selection of satellite pixels falling into the region of interest;• Taking into account quality index information (to be implemented);• Radar: selection of the radar data synchronous with the satellite ones;• Rain gauge: selection of the radar data synchronous with the satellite ones and

spatial interpolation of rain gauge data;• Up‐scaling of ground data at the resolution of the native satellite grid, or

nearest‐neighbour matching;• Statistical score calculation.

The common validation is now performed by means of a unified software developed at DPC, Italy

(contact person: A. Rinollo, [email protected])

H-SAF Precipitation Products Validation and Application, Training Course 6th IPWG Workshop Sao Jose dos Campos, Brazil 15-19 October,

MC statistic:– ACCURACY– POD – FAR – BIAS– ETS– OR– HSS

CS statistic:‐ Number of points‐ observed Mean rain (rate or cumulated)‐ Satellite Mean rain (rate or cumulated)‐ Observed Maximum rain (rate or cumulated)‐ Satellite Maximum rain (rate or cumulated)‐Mean error‐Multiplicative bias‐Mean absolute error‐ Root mean square error‐ correlation coefficient‐ Standard deviation

Plots:‐ Scatter plot‐ Probability density function

Common methodology

Period: January 2009‐March 2010


The PP Validation System

comparison national radar and rain gauge data with precipitation products on satellite native grid using common software

ITALY

‐DPC

ITALY‐Uni. Fe

POLAND‐IMWM

HUNGARY‐HMS

BELGIUM‐RMI

GERMANY‐BFG

TURKEY‐ITU, TSMS

SLOVAKIASHMU

• evaluation of the monthly continuous scores and contingency tables for the precipitation classes

• evaluation of PDF producing numerical files called ‘DIST’ files and plots

The PP validation leader collect all the validation files (MC, CS and DIST files), verify the consistency of the results and evaluate the monthly common statistical

results

• numerical files called ‘CS’ and ‘MC’ files• numerical files called ‘DIST’ files and plots

BULGARY‐NIMH


Further information about validation technique, ground data quality calculation, ground data interpolation and upscaling, and unified software can be found in:

S. Puca et al., The validation service of the Hydrological SAF geostationary and polarsatellite precipitation products, NHESS, 2012 (submitted paper).

A. Rinollo et al., A common protocol for the validation of satellite rainfall estimations using radar data over the European territory, NHESS, 2012 (submitted paper).

S. Puca et al., The Hydrological SAF validation service of geostationary and polar products,Proc. EUMETSAT conference, 2012 (in progress).

A. Rinollo et al., A quality index for radar‐based rainfall estimation and the impact of its introduction on the validation of H‐SAF satellite precipitation products, Proc. EUMETSAT conference, 2012 (in progress).

Some references on validation


Example of H‐01 product


Results for H‐01

‐18‐16‐14‐12‐10‐8‐6‐4‐202

Spring 2009

Summer 2009

Autumn 2009

Winter 2009/10

H01‐reference [mm/h]

H‐01 ME

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Results for H‐01

‐16‐14‐12‐10‐8‐6‐4‐202

Spring Summer Autumn Winter

H01‐refernce mm/h

ME Radars

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Variability with season• For heavy and medium precipitation the performance are rather similar across seasons. • For light precipitation summer is substantially worse, and autumn better.• The FAR is rather high in all seasons, whereas the POD is better in summer and worse in winter.

Variability with precipitation type (or intensity)• Heavy and medium precipitation have very similar performances through all seasons and geographical areas.

• For light precipitation there is a substantial degradation in spring and winter, and very substantial in summer.

Summary H‐01


Results for H‐02

‐18

‐16

‐14

‐12

‐10

‐8

‐6

‐4

‐2

0

Spring 2009

Summer 2009

Autumn 2009

Winter 2009/10


H‐02 ME

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Results for H‐02

‐14

‐12

‐10

‐8

‐6

‐4

‐2

0


H02‐refernce mm/h

ME Radars

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Variability with season• For heavy and medium precipitation the performance are rather similar across seasons.• For light precipitation autumn is substantially worse, and winter better. • The FAR is rather high in all seasons, whereas the POD is better in summer and worse in

winter. In summer POD is higher than FAR.

Variability with precipitation type (or intensity)• Heavy and medium precipitation have very similar performances through all seasons

and geographical areas.• For light precipitation there is a substantial degradation in spring and summer, and

very substantial in autumn.

Summary H‐02


Results for H‐03

‐18

‐16

‐14

‐12

‐10

‐8

‐6

‐4

‐2

0

Spring 2009

Summer 2009

Autumn 2009

Winter 2009/10


H‐03 ME

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Results for H‐03

‐18‐16‐14‐12‐10‐8‐6‐4‐20


H03‐refernce [m

m/h]

ME Radars

> 10 mm/h

1‐10 mm/h

10 mm/h

1‐10 mm/h

Variability with season• For heavy and medium precipitation the performance are rather similar across seasons• For light precipitation summer is substantially worse, and winter better. • The FAR is rather high in all seasons, whereas the POD is better in summer and worse in

winter.

Variability with precipitation type (or intensity)• Heavy and medium precipitation have very similar performances through all seasons and geographical areas.

• For light precipitation there is a substantial degradation in spring and autumn, and very substantial in summer.

Summary H‐03


Example of H‐05 3h product


Example of H‐05 24h product


Results for H‐05


Variability with season• The performance are rather similar across seasons, however summer is better

and winter worse. • The FAR is rather high in all seasons, whereas the POD is better in summer.• POD values are higher for 24 h accumulated product than for 3h one

Summary H‐05


Variability with geographical areaIt may be observed that the performances are rather consistent across the various geographical areas, especially for heavy (> 10 mm/h) and medium (1‐10 mm/h) precipitation. This has been favoured by the adoption of a common validation methodology across the various participating Institutes. Even for inner lands and coastal areas the performance are rather similar.

Variability with validation toolThe performances resulting from validation by radar and those by rain gauges are rather similar. This is very important because User should not mind about which tool has been used for the validation: the information on the performance is regarded as a property of the product, not of the ground truth.

Conclusions


Cases Study

High RG measurements connected with heavy precipitation.

Days when H01 and H02 products were available more or less at the same time (max time span ‐ 30 min);

11 May 2009 ‐ convective precipitation on the frontline moving across Poland.

The meteorological situation resulted in heavy convective precipitation that occurred in the afternoon, at the South of Poland. The 6 hour cumulated precipitation measured at the SYNOP stations exceeded 60 mm.


11 May 2009


0

10

20

30

40

50

60

70

0 10 20 30 40 50 60 70

H‐03

rain rate [m

m/h]

RG rain rate [mm/h]

11 May 2009


Parameter H‐01 [mm/h]

H‐02 [mm/h] H‐03 [mm/h]

Max RG 45.6 20.7 79.2

Max SAT 51.5 15.0 28.9

Mean RG 3.6 2.5 3.3

Mean SAT 4.7 1.8 2.4

ME 1.1 ‐0.7 ‐1.0

St.Dev 5.9 3.2 5.9

RMSE 6.0 3.3 6.0

RMSE % 4.6 2.3 3.3

11 May 2009


0

5

10

15

20

25

00:12

00:57

01:42

02:27

03:12

03:57

04:42

05:27

06:12

06:57

07:42

08:27

09:12

09:57

10:42

11:27

12:12

12:57

13:42

14:27

15:12

15:57

16:42

17:27

18:12

18:57

19:42

20:27

21:12

21:57

22:42

23:27

Rain ra

te [m

m/h]

Time UTC

(49.7 N; 19.41 E)H03 RG

11 May 2009


Temporal variability of RG and H‐03 rain rates (selected posts)

11 May 2009


Case study ‐ conclusions

H‐02 and H‐03 tends to underestimate high precipitation values while for H‐01 over‐estimation has been found.

The quality of H‐SAF rain rate products in convective precipitation intensity estimation found for those cases is very good: the Mean Error varies from ‐1.0 mm/h to 1.1 mm/h and RMSE is equal 4.6%, 2.3% and 3.3% respectively for H‐01, H‐02 and H‐03. The best results were obtained for H‐02.


Conclusions

The ability of H‐SAF rain rate products in convective precipitation recognition was found to be very good for H‐01 and quite good for H‐02. For H‐03, values of POD and FAR are the same. It should be also mentioned that ACC values are very high for all products.

The spatial distribution of convective precipitation was well described by H‐01 and H‐02 products, however, the size of precipitation area was slightly overestimated. On the other hand, the maximum values of rainfall were properly localised by H‐01.

The H‐03 product seems to be too rainy ‐ the precipitation area was significantly overestimated. Moreover, distribution of rainfall intensity was found to be too homogeneous and the spots with heavy rainfall were missed. These features can be also seen in the results obtained for the analysis of temporal variability of rain rate performed for selected H‐03 pixels


Thank you for your attention!


validation of h saf precipitation products - cnripwg/meetings/saojose-2012/... · 2016. 6. 7. ·...

Documents