copernicus global land operations cryosphere … · europe (72n 11w – 35n 50e) with 0.005 degree...

Copernicus Global Land Operations – Lot 2 Date Issued: 08.05.2017 Issue: I1.02

Copernicus Global Land Operations

“Cryosphere and Water” ”CGLOPS-2”

Framework Service Contract N° 199496 (JRC)

QUALITY ASSESSMENT REPORT

SNOW COVER EXTENT

COLLECTION 500M CONTINENTAL EUROPE

VERSION 1.0

Issue I1.02

Organization name of lead contractor for this deliverable: ENVEO IT GmbH

Book Captain: Gabriele Schwaizer (ENVEO)

Contributing Authors: Elisabeth Ripper (ENVEO)


Document-No. CGLOPS2_QAR_SCE500-CEURO-500m_V1 © C-GLOPS Lot1 consortium

Issue: I1.02 Date: 08.05.2017 Page: 2 of 31

Dissemination Level PU Public X

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential, only for members of the consortium (including the Commission Services)




Document Release Sheet

Book captain: Gabriele Schwaizer Sign Date

Approval: Roselyne Lacaze Sign Date

Endorsement: Mark Dowell Sign Date

Distribution: Public




Change Record

Issue/Rev Date Page(s) Description of Change Release

03.03.0217 All First Issue I1.01

08.05.2017 All Update after external review I1.02




TABLE OF CONTENTS

1 Background of the document .................................................................................................. 9

1.1 Executive Summary ..................................................................................................................... 9

1.2 Scope and Objectives ................................................................................................................... 9

1.3 Content of the document............................................................................................................. 9

1.4 Related documents .....................................................................................................................10

1.4.1 Applicable documents ...........................................................................................................................10

1.4.2 Input .....................................................................................................................................................10

1.4.3 Output ..................................................................................................................................................10

1.4.4 External documents...............................................................................................................................10

2 Review of Users Requirements .............................................................................................. 11

3 Quality Assessment Method .................................................................................................. 12

3.1 Overvall procedure .....................................................................................................................12

3.2 Satellite Reference Products .......................................................................................................14

3.2.1 Landsat scenes ......................................................................................................................................14

3.2.2 Very high resolution optical satellite data ..............................................................................................17

4 Results ................................................................................................................................... 19

5 Conclusions ............................................................................................................................ 28

6 Recommendations ................................................................................................................. 30

7 References ............................................................................................................................. 31




List of Figures

Figure 3.1: Validation concept developed in the ESA QA4EO project SnowPEx (lead: ENVEO) for

the validation of the SCE product with snow reference products from high resolution optical

satellite data........................................................................................................................... 14

Figure 3.2: Overview on distribution of Landsat acquisitions selected for the generation of

reference snow maps. The fill color of each scene indicates the year of the acquisition. Black

outlined scenes show the footprint of a scene. A red outlined scene shifted towards the south-

west of a basic footprint represents a further scene available for this location of the year as

indicated by the fill color. ........................................................................................................ 16

Figure 3.3: Overview on spatial distribution of very high resolution optical satellite data (red

outlines) provided courtesy by the Copernicus Data Warehouse during the EU FP7 project

CryoLand for validation purposes. The fill color of each scene indicates the year of the

acquisition. Black outlined scenes show the footprint of a scene. A red outlined scene shifted

towards the south-west of a basic footprint represents a further scene available for this

location of the year as indicated by the fill color. .................................................................... 18




List of Tables

Table 3.1: Band numbers, wavelengths and resolution of Landsat 5 Thematic Mapper (TM),

Landsat 7 Enhanced Thematic Mapper plus (ETM+), Landsat 8 Operational Land Imager

(OLI) and Thermal Infrared Sensor (TIRS), and Sentinel-2 Multi-Spectral Instrument (MSI). . 15

Table 4.1: Summary of statistical measures resulting from the validation of the SCE product over

continental Europe with snow maps from each the total area of 110 selected Landsat scenes

and 44 very high resolution optical satellite scenes. Results are generated by ENVEO during

the ESA QA4EO project SnowPEx, and the EU FP7 project CryoLand. ................................. 19


continental Europe with 110 selected Landsat scenes, separating the total non-forested and

total forested areas classified from each Landsat scene. Results are generated by ENVEO

during the ESA QA4EO project SnowPEx. ............................................................................. 20

Table 4.3: Statistical measures of the fractional snow cover extent (snow on ground) vs. Landsat

snow algorithms with the same quantity displayed from January to July (independent of the

year). Results are generated by ENVEO during the ESA QA4EO project SnowPEx. ............. 21

Table 4.4: Statistical measures derived from the validation of the SCE product over continental

Europe with reference snow maps generated by three different algorithms from Landsat

scenes. Only values for the total area covered by each scene are provided in the table. R =

correlation coefficient. Results are generated by ENVEO within the ESA QA4EO project

SnowPEx. .............................................................................................................................. 22

Table 4.5: Statistical measures derived from the validation of the continental European fractional

snow cover products with snow maps generated manually from very high resolution optical

satellite data. Analyses are performed by ENVEO within the EU FP7 project CryoLand. ....... 25

Table 4.6: Validation results of fractional SE products detecting snow on ground vs. LS snow

algorithms for different fraction sections: 0-25 %, 26-50 %, 51-75%, 76-100 %. Results are

generated by ENVEO during the ESA QA4EO project SnowPEx. .......................................... 27

Table 5.1: Compliance matrix for the continental European SCE product. .................................... 29




List of Acronyms

ABBREV. MEANING

ATBD Algorithm Theoretical Basis Document

ESA European Space Agency

ETM+ Enhanced Thematic Mapper plus

EU European Union

FP7 7th Framework Program of the European Community for research,

technological development and demonstration activities

FSC Fractional Snow Cover

MODIS Moderate Imaging Spectroradiometer

MSI Multi-Spectral Instrument

NASA National Aeronautics and Space Administration

NDSI Normalized Difference Snow Index

NDVI Normalized Difference Vegetation Index

NIR Near Infrared

NRT Near-Real Time

OLI Operational Land Imager

QA4EO Quality Assessment for Earth Observation

QAR Quality Assessment Report

RMSE Root Mean Square Error

SCE Snow Cover Extent

SRTM Shuttle Radar Topography Mission

SWIR Shortwave Infrared

TIR Thermal Infrared

TIRS Thermal Infrared Sensor

TM Thematic Mapper

VHR Very High Resolution

VIS Visible




1 BACKGROUND OF THE DOCUMENT

1.1 EXECUTIVE SUMMARY

The Global Land (GL) Component in the framework of Copernicus is earmarked as a component of

the Land service to operate “a multi-purpose service component” that will provide a series of bio-

geophysical products on the status and evolution of land surface at global scale. Production and

delivery of the parameters are to take place in a timely manner and are complemented by the

constitution of long term time series.

The Copernicus Global Land Service contains a Snow Cover Extent (SCE) Near Real Time (NRT)

product extending continental Europe at 0.005 degree resolution. The SCE is derived from medium

resolution optical satellite data, currently from Terra MODIS (Moderate resolution Imaging

Spectroradiometer). The spectral capabilities of the sensor are used in combination with a digital

elevation model, a transmissivity map describing the transmissivity of forest canopy, and surface

classification maps to detect the fraction of snow cover per pixel in percentage. The usage of the

transmissivity map allows also in forested areas to detect the snow on the ground.

This Quality Assessment Report describes the validation activities and results performed so far for

the Continental European SCE product at 0.005 degree resolution.

1.2 SCOPE AND OBJECTIVES

The document presents the results of the quality assessment of the central European SCE500 V1

product.

The quality assessment is performed on the SCE product from 2000 to 2014, covering continental

Europe (72N 11W – 35N 50E) with 0.005 degree resolution using snow maps generated from high

resolution optical satellite data acquired between 2000 and 2014 over different surface types and

at different snow conditions over Europe.

The objective is to check that the overall product performance meets the quality requirements

defined by users.

1.3 CONTENT OF THE DOCUMENT

This document is structured as follows:

• Chapter 2 recalls the users requirements, and the expected performance

• Chapter 3 describes the methodology for quality assessment, the metrics and the criteria of

evaluation

• Chapter 4 presents the results of the analysis




• Chapter 5 summarizes the main conclusions of the study

• Chapter 6 makes recommendations based upon the results

1.4 RELATED DOCUMENTS

1.4.1 Applicable documents

AD1: Annex I – Technical Specifications JRC/IPR/2015/H.5/0026/OC to Contract Notice 2015/S

151-277962 of 7th August 2015

AD2: Appendix 1 – Copernicus Global land Component Product and Service Detailed Technical

requirements to Technical Annex to Contract Notice 2015/S 151-277962 of 7th August 2015

AD3: IGOS, 2007: Cryosphere Theme Report 2007. WMO/TD-No 1405. 100 pp.

AD4: GCOS, 2006: Systematic observation requirements for satellite-based products for climate.

WMO/TD Report No. 1338, September 2006.

1.4.2 Input

Document ID Descriptor

CGLOPS2_SSD Service Specifications of the Global Component of

the Copernicus Land Service.

CGLOPS2_SVP Service Validation Plan of the Global Land Service

CGLOPS2_ATBD_SCE500-

CEURO-500m_V1

Algorithm Theoretical Basis Document of the

SCE500-CEURO-500m_V1 product

1.4.3 Output

Document ID Descriptor

CGLOPS2_PUM_SCE500-

CEURO-500m_V1

Product User Manual summarizing all information about the

SCE500-CEURO-500m_V1 product

1.4.4 External documents

ED1: Malnes, E., Buanes, A., Nagler, T., Bippus, G., Gustafsson, D., Schiller, C., Metsämäki, S.,

Pulliainen, J., Luojus, K., Larsen, H.E., Solberg, R., Diamandi, A., Wiesmann, A. (2015). User

requirements for the snow and land ice services – CryoLand. The Cryosphere, 9, 1191–1202,

doi:10.5194/tc-9-1191-2015.




2 REVIEW OF USERS REQUIREMENTS

According to the applicable documents [AD3] and [AD4], and the external document [ED1], the

user’s requirements relevant for the Snow Cover Extent product for continental Europe are:

• Definition: Continental European Snow Cover Extent (SCE)

• Geometric properties:

o Spatial resolution: Minimum: 500 m, Target: 100 m

o Grid / Projection: Geographic coordinates (latitude/longitude), WGS84 datum,

WGS84 ellipsoid

• Geographical coverage: Continental Europe (72°N / 11°W – 35°N / 50°E)

• Accuracy requirements:

o Geometric accuracy: Target: Sub-pixel location error

o Thematic accuracy: Minimum: 15 %, Target: 5 %

• Temporal requirements:

o Time period: Full year

o Temporal frequency: Minimum: Daily, Target: 12 hours

o Delivery time: Minimum: 48 hours after image acquisition, Target: 12 hours after

image acquisition

According to the applicable documents [AD3] and [AD4], and the external document [ED1], the

user’s requirements relevant for the Snow Cover Extent product for continental Europe are:

• Definition: Northern hemispheric Snow Cover Extent (SCE)

• Geometric properties:

o Spatial resolution: Minimum: 4 km, Target: 500 m, 100 m in complex terrain

o Grid / Projection: Geographic coordinates (latitude/longitude), WGS84 datum,

WGS84 ellipsoid

• Geographical coverage: Northern hemisphere (85°N / 180°W – 35°N/180°E)

• Accuracy requirements:

o Geometric accuracy: Minimum: 1/3 IFOV, Target: IFOV 1 km (100 m in complex

terrain)

o Thematic accuracy: Minimum: 15 %, Target: 5 %

• Temporal requirements:

o Time period: Full year

o Temporal frequency: Minimum: Daily, Target: 12 hours

o Delivery time: Minimum: 24 hours after image acquisition, Target: 12 hours after

image acquisition




3 QUALITY ASSESSMENT METHOD

3.1 OVERVALL PROCEDURE

The snow cover extent (SCE) product for continental Europe is generated daily from medium

resolution optical satellite data. The applied algorithm [CGLOPS2_ATBD_SCE500-CEURO-

500m_V1] detects the fraction of snow cover per pixel, given in percentage of snow ranging

between 0 % and 100 %. The applied algorithm corrects for forest canopy, and thus provides in

forested areas information on the snow on ground.

The validation of the snow cover extent product for continental Europe is based on the

intercomparison of the product with reference snow maps from high resolution optical satellite data

The validation is performed on a pixel by pixel basis, following the concept and recommendations

established during the ESA QA4EO project SnowPEx (lead by ENVEO). To describe the product

quality, statistical measures derived from the intercomparison of the product with reference snow

information are used. For the validation of the continental European SCE product with snow maps

from Landsat scenes, the total area covered by each Landsat scene is used, but also different

surface classes, such as forested / non-forested areas, plains, and mountains are discriminated.

The swath width of very high resolution (VHR) optical satellite data also used for validation is in

most cases significantly smaller than that of Landsat images. Thus, for the validation of the

continental European SCE product with snow maps from VHR optical satellite data only the total

coverage of the particular VHR scene is used.

For the validation of the SCE product with snow maps from high resolution optical satellite data all

snow maps are prepared as fraction of the snow cover extent. For a pixel-by-pixel intercomparison

it is mandatory that all information is available in the same map projection and grid size. Thus, the

reference snow maps from high resolution optical satellite data are resampled to the grid size of

the SCE product. Pixels classified as water bodies, clouds, or invalid pixel information in either the

SCE product or the reference snow product are excluded from the validation process. For all other

areas, a pixel-by-pixel intercomparison is applied calculating the following statistical measures for

the total area and for particular surface classes to describe the product performance:

• The fully snow covered area of each individual product is derived by the number of

equivalent fully snow covered pixels (SCE = 100%) (Nequ_fse), which is given according to:

𝑁𝑒𝑞𝑢_𝑓𝑠𝑒 = ∑ ∑𝑆𝐸100(𝑖, 𝑗)

100

𝑥

𝑖=0

𝑦

𝑗=0

Eq. 1

where the 𝑆𝐸100 indicates only pixels with full snow cover in the SCE products.




• The total snow covered area of each individual product is derived by the number of

equivalent snow covered pixels (Nequ_se), which is given according to

𝑁𝑒𝑞𝑢_𝑠𝑒 = ∑ ∑𝐹𝑆𝐶(𝑖, 𝑗)

100

𝑥

𝑖=0

𝑦

𝑗=0

Eq. 2

where the fractional snow coverage is in per cent (values between 1 to 100 %).

• For determining the Bias between two snow products, the pixels specified by 𝑁𝑢𝑖 are used

as calculation basis:

𝐵𝐼𝐴𝑆 = 1

𝑁𝑢𝑖 ∑ ∑(𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗))

𝑥

𝑖=0

𝑦

𝑗=0

Eq. 3

• The root-mean-square error, RMSE, between two snow products is calculated using all

pixels suitable for inter-comparison (Nui) according to

𝑅𝑀𝑆𝐸 = √1

𝑁𝑢𝑖 ∑ ∑(𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗))²

𝑥

𝑖=0

𝑦

𝑗=0

Eq. 4

• In addition to the RMSE, the unbiased RMSE is applied using the same input dataset (Nui)

and is described by

𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑 𝑅𝑀𝑆𝐸 =

Eq. 5 = √

1

𝑁𝑢𝑖 ∑ ∑((𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇

) − (𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹 ))²

𝑥

𝑖=0

𝑦

𝑗=0

• The correlation coefficient calculation between two products (EXT = SCE in Product, REF =

Reference snow map, e.g. from Landsat) includes only the valid pixels for the inter-

comparison (𝑁𝑢𝑖) and is given by

𝐶𝑜𝑟𝑟𝐶𝑜𝑒𝑓 =

Eq. 6 = ∑ ∑ (𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇

𝑥𝑖=0

𝑦𝑗=0 ) (𝐹𝑆𝐶𝑅𝐸𝐹 (𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹

)

√∑ ∑ (𝐹𝑆𝐶𝐸𝑋𝑇(𝑖, 𝑗) − 𝐹𝑆𝐶𝐸𝑋𝑇 )²𝑥

𝑖=0𝑦𝑗=0 ∑ ∑ (𝐹𝑆𝐶𝑅𝐸𝐹(𝑖, 𝑗) − 𝐹𝑆𝐶𝑅𝐸𝐹

)²𝑥𝑖=0

𝑦𝑗=0

where FSC

is the average fractional snow cover value.




The concept for the product validation with snow maps from high resolution optical satellite data is

shown in Figure 3.1.

Figure 3.1: Validation concept developed in the ESA QA4EO project SnowPEx (lead: ENVEO) for the

validation of the SCE product with snow reference products from high resolution optical satellite

data.

3.2 SATELLITE REFERENCE PRODUCTS

3.2.1 Landsat scenes

Archived high resolution optical satellite data from the Landsat satellite family, including Landsat 5

TM and Landsat 7 ETM+, are used to generate reference snow maps. This data base will be

extended in future with data from Landsat 8 OLI/TIRS and Sentinel-2 MSI. The different sensors

have very similar spectral band characteristics (Table 3.1). Thus, established methods and

algorithms can be applied with only very minor adaptations on data of all these sensors. The

Sentinel-2 sensor does not feature thermal infrared bands, which are useful for automated cloud

screening. Thus, areas obscured by clouds or cloud shadows in Sentinel-2 scenes are currently

masked manually. The development of new methods for automated cloud screening from Sentinel-

2 is ongoing, and can be included in the processing chain as soon as a method has been

established. Landsat images used for the processing of reference snow maps are level "L1T"

products, meaning the scenes are radiometrically and geometrically corrected by the data provider

U.S. Geological Survey (USGS), using ground control points and a digital elevation model for the

orthorectification.




Table 3.1: Band numbers, wavelengths and resolution of Landsat 5 Thematic Mapper (TM), Landsat 7

Enhanced Thematic Mapper plus (ETM+), Landsat 8 Operational Land Imager (OLI) and Thermal

Infrared Sensor (TIRS), and Sentinel-2 Multi-Spectral Instrument (MSI).

Spectral

characteristic

Landsat 5 TM Landsat 7 ETM+ Landsat 8 OLI/TIRS Sentinel-2 MSI

Bd. No.

Spectral Band-width

(micro-metres)

Resolution (meters)

Bd. No.

Spectral Band-width

(micro-metres)

Resolution (meters)

Bd. No.

Spectral Band-width

(micro-metres)

Resolution (meters)

Bd. No.

Spectral Band-width

(micro-metres)

Resolution (meters)

Deep blue and violets (coastal blue, aerosol)

1 0.43 - 0.45

30 1 0.433 - 0.453

60

Visible (VIS)

Blue 1 0.45 - 0.52

30 1 0.45 - 0.52

30 2 0.45 - 0.51

30 2 0.458 - 0.523

10

Green 2 0.52 - 0.60

30 2 0.52 - 0.60

30 3 0.53 - 0.59

30 3 0.543 - 0.578

10

Red 3 0.63 - 0.69

30 3 0.63 - 0.69

30 4 0.64 - 0.67

30 4 0.650 - 0.680

10

Near Infrared (NIR)

5 0.698 - 0.713

20

6 0.773 - 0.793

20

7 0.773 - 0.793

20

4 0.76 - 0.90

30 4 0.77 - 0.90

30 5 0.85 - 0.88

30 8 0.785 - 0.900

10

8A 0.855 - 0.875

20

9 0.935 - 0.955

60

Shortwave Infrared (SWIR)

5 1.55 - 1.75

30 5 1.55 - 1.75

30 6 1.57 - 1.65

30 11 1.565 - 1.655

20

7 2.08 - 2.35

30 7 2.09 - 2.35

30 7 2.11 - 2.29

30 12 2.100 - 2.280

20

Panchromatic (PAN)

8 0.52 - 0.90

15 8 0.50 - 0.68

15

Cirrus / clouds 9 1.36 - 1.38

30 10 1.360 - 1.390

60

Thermal Infrared (TIR)

6 10.40 - 12.50

120* (30) 6 10.40 - 12.50

60** (30) 10 10.60 - 11.19

100*** (30)

11 11.50 - 12.51

100*** (30)

* TM Band 6 was acquired at 120-meter resolution, but products processed before February 25, 2010 are resampled to

60-meter pixels. Products processed after February 25, 2010 are resampled to 30-meter pixels.

** ETM+ Band 6 is acquired at 60-meter resolution. Products processed after February 25, 2010 are resampled to 30-

meter pixels

*** TIRS bands are acquired at 100 meter resolution, but are resampled to 30 meter.




The generation of the reference snow maps from Landsat data includes four major steps:

(1) the pre-processing of the data in the original map projection and grid size

(2) reprojection of the pre-processed data into the map projection of the SCE products

(3) apply selected algorithms to generate high resolution reference snow maps

(4) aggregate high resolution reference snow maps to grid size of SCE products

Currently, 110 Landsat scenes acquired between 2000 and 2011 have been used to generate

reference snow maps. An overview on the spatial and temporal distribution of these scenes is

shown in Figure 3.2.

Figure 3.2: Overview on distribution of Landsat acquisitions selected for the generation of reference

snow maps. The fill color of each scene indicates the year of the acquisition. Black outlined scenes

show the footprint of a scene. A red outlined scene shifted towards the south-west of a basic

footprint represents a further scene available for this location of the year as indicated by the fill

color.

The pre-processing of the Landsat scenes includes radiometric calibration (Chander, Markham,

and Helder 2009) and topographic correction (Ekstrand, 1996) of all used bands in order to reduce

topographically induced illumination effects. The map projection of the delivered Landsat images is

Universal Transverse Mercator (UTM) on WGS84 ellipsoid and is maintained until the pre-




processing is finished. The origin coordinates of all Landsat scenes refer to the upper left corner of

the upper left pixel. This pixel reference remains unchanged for all high-resolution snow products.

The original Landsat pixel size is 30 m x 30 m and will also be preserved until pre-processing is

finished.

The topographically corrected top of atmosphere reflectance at 30 m x 30 m grid size resulting

from the pre-processing is reprojected (bilinear) into the map projection of the SCE products

(geographic coordinates / WGS84). Based on the reprojected topographically corrected top of

atmosphere reflectance at 30 m x 30 m, three snow detection algorithms are applied on each

Landsat scene to assess also the quality of the reference snow maps:

• Dozier and Painter 2004: thresholds applied on NDSI and NIR band, resulting in maps

showing binary snow cover on ground

• Klein, Hall, and Riggs 1998: thresholds applied on NDVI and NDSI, resulting in maps

showing binary snow cover on ground

• Salomonson and Appel 2004, 2006: linear model weighting the NDSI, resulting in maps

showing fractional information on viewable snow cover (in forested areas on top of forest

canopy)

The resulting high-resolution snow maps from Landsat are aggregated to the SCE product

resolution.

3.2.2 Very high resolution optical satellite data

Beside the freely available Landsat data set, a set of 44 very high resolution (VHR) optical satellite

data, such as SPOT-5, Quickbird or WorldView-1/-2, acquired between 2004 and 2013 was

courtesy provided by the Copernicus Data Ware House during the EU FP7 project CryoLand for

validation purposes. These VHR optical satellite data (0.50m – 10m resolution) cover only small

areas, and only images acquired at cloud-free conditions were selected as reference scenes.

Snow maps from these VHR optical satellite data were generated manually by the CryoLand

partners, and used in the same way as Landsat based snow maps as reference snow maps to

validate the continental European SCE product. An overview on the spatial distribution of these

VHR satellite scenes is shown in Figure 3.3.




Figure 3.3: Overview on spatial distribution of very high resolution optical satellite data (red outlines)

provided courtesy by the Copernicus Data Warehouse during the EU FP7 project CryoLand for

validation purposes. The fill color of each scene indicates the year of the acquisition. Black outlined

scenes show the footprint of a scene. A red outlined scene shifted towards the south-west of a basic

footprint represents a further scene available for this location of the year as indicated by the fill

color.




4 RESULTS

In this chapter, the detailed validation results derived from the pixel-by-pixel comparison of the

continental European SCE product with snow maps from high-resolution optical satellite data are

presented.

All reference snow maps are reprojected to the map projection and aggregated to a fractional snow

cover extent at the resolution of the continental European SCE product (0.005 degrees). For the

pixel-by-pixel intercomparison, the statistical measures described in detail in Section 3.1 are

calculated. Table 4.1 shows a summary of the validation results of the continental European SCE

product compared on a pixel-by-pixel basis with the 110 selected Landsat scenes of 2000 – 2011

and the 44 VHR optical satellite scenes acquired between 2004 and 2013. Most of the selected

Landsat scenes used for the generation of reference snow maps for the validation of the Pan-

European SCE product were acquired between January and May. For the validation with snow

maps from Landsat data the results are presented separately for each snow algorithm applied on

all the Landsat scenes.


continental Europe with snow maps from each the total area of 110 selected Landsat scenes and 44

very high resolution optical satellite scenes. Results are generated by ENVEO during the ESA

QA4EO project SnowPEx, and the EU FP7 project CryoLand.

Snow maps from Landsat data Snow maps from VHR

satellite data

MEAN Salomonson Klein Dozier Manual mapping

unbiased RMSE 15,32 15,92 15,61 15,36

BIAS 0,15 -0,75 0,82 3,02

Correlation Coefficient 0,74 0,72 0,73 0,74

RANGE MIN MAX MIN MAX MIN MAX MIN MAX

unbiased RMSE 2,38 26,69 1,70 38,33 0,45 30,12 0,00 38,75

BIAS -19,61 13,76 -33,28 28,84 -17,10 98,47 -13,45 44,64

Correlation Coefficient 0,03 0,96 0,06 0,97 0,04 0,97 0,05 0,92

The mean unbiased RMSE of all scenes is about 15%, independently which approach was used

for the generation of the reference snow maps. Also the mean Correlation Coefficient is very

similar, around 0.73, for the validation with all reference data sets. Only the mean Bias shows

larger discrepancies depending on the snow detection approach used for the generation of

reference snow maps. Nevertheless, if the range of these statistical measures are investigated in

more detail, there are indeed partly large differences. The unbiased RMSE shows ranges between

about 0% and 39%, the Bias shows a large variance ranging from about –33% to about 98%, and




also the Correlation Coefficients reflect these differences ranging between 0.03 and 0.97. Thereby,

each of the maximum and minimum statistical measures are derived for different reference scenes.

The validation of the continental European SCE product with reference snow maps from Landsat

data has also been separately analysed with respect to forested and non-forested areas, as

forested areas are very challenging for mapping snow correctly. It has to be considered that

different snow algorithms provide different thematic information on snow in forest, which can either

be the snow on top of the forest canopy, also called “viewable snow”, or the snow on the ground.

For validation activities in forested areas only snow maps from those algorithms providing the

same thematic information as the SCE product (snow on the ground) are used. In non-forested

areas, the results of all snow algorithms can be intercompared. A summary of the validation results

for non-forested areas and forested areas, respectively, derived from intercomparison of the SCE

product with the reference snow maps from Landsat data, is shown in Table 4.2. The agreement of

the SCE product with reference snow maps from Landsat scenes is slightly higher in non-forested

areas. Also in forested areas, the performance of the SCE product is still acceptable, although

some outliers are found. Some cases with larger discrepancies between the SCE product and the

reference snow products are found for forested and non-forested areas, with unbiased RMSE

values up to 35% in non-forested areas, and up to 40%. in forested areas, but there are neither

particular spatial nor temporal patterns identifiable. For the interpretation of the results one should

keep in mind that also the performance of the reference snow maps would need a detailed

evaluation.


continental Europe with 110 selected Landsat scenes, separating the total non-forested and total

forested areas classified from each Landsat scene. Results are generated by ENVEO during the ESA

QA4EO project SnowPEx.

Class Total non-forested area Total forested area

MEAN Salomonson Klein Dozier Klein Dozier

unbiased RMSE 13,47 14,36 13,69 17,73 17,42

BIAS -0,50 -1,04 0,33 -0,46 1,51

Correlation Coefficient 0,77 0,74 0,76 0,60 0,62

RANGE MIN MAX MIN MAX MIN MAX MIN MAX MIN MAX

unbiased RMSE 1,03 24,35 1,72 34,98 0,46 26,28 0,32 39,50 0,00 32,18

BIAS -16,98 12,52 -27,52 26,33 -14,82 99,55 -35,93 29,41 -22,78 97,67

Correlation Coefficient 0,04 0,98 0,02 0,98 0,01 0,98 003 0,93 0,02 0,94

The statistical measures unbiased RMSE and Bias resulting from the validation of the SCE product

with snow maps generated by the Dozier and the Klein approaches from Landsat scenes acquired

between January and July are illustrated in the scatterplots in Table 4.3. The mean values of the




unbiased RMSE and Bias shown in these plots are slightly different to the values shown in Table

4.1, as only a particular time frame is considered for generating these mean values.

Table 4.3: Statistical measures of the fractional snow cover extent (snow on ground) vs. Landsat

snow algorithms with the same quantity displayed from January to July (independent of the year).

Results are generated by ENVEO during the ESA QA4EO project SnowPEx.

Statistical

measure Dozier Klein

Bias

Unbiased

RMSE

The individual validation results of the SCE product with snow maps generated by different

algorithms from the selected Landsat scenes, using each the total area covered by the Landsat

scene, are provided in Table 4.4. The detailed validation results resulting from snow maps from

VHR satellite data, also each valid for the total area covered by the particular VHR scene, are

listed in Table 4.5.




Table 4.4: Statistical measures derived from the validation of the SCE product over continental

Europe with reference snow maps generated by three different algorithms from Landsat scenes. Only

values for the total area covered by each scene are provided in the table. R = correlation coefficient.

Results are generated by ENVEO within the ESA QA4EO project SnowPEx.

Date

Landsat scene No. of pixels used

for inter-

comparison

Salomonson Klein Dozier

Sensor

Path Row R Bias unbiased

RMSE R Bias

unbiased RMSE

R Bias unbiased

RMSE

28.02.2000 LE7 185 26 132751 0,8 1,15 25,43 0,8 -8,03 26,23 0,82 -1,82 25,44

07.03.2000 LE7 193 27 132782 0,94 -4,14 16,76 0,94 -1,39 15,71 0,95 -0,93 14,56

07.03.2000 LE7 193 28 99664 0,92 1,85 17,93 0,92 -0,49 17,98 0,92 1,14 17,8

21.03.2000 LE7 195 27 126719 0,96 0,18 8,27 0,96 -0,37 8,4 0,97 -0,16 7,91

21.03.2000 LE7 195 28 101387 0,95 1,75 14,92 0,95 0 14,55 0,95 0,94 14,7

27.03.2000 LE7 173 33 110122 0,93 5,31 16,51 0,94 4,72 15,71 0,94 4,64 15,77

30.03.2000 LE7 194 19 128724 0,91 3,92 13,7 0,92 -0,62 12,79 0,89 4,24 14,43

30.03.2000 LE7 194 20 153546 0,92 -6,1 15,46 0,92 0,91 14,89 0,91 -4,58 15,12

17.05.2000 LE7 194 13 242312 0,88 4,35 19,16 0,78 12,32 26,56 0,89 1,97 18,45

21.05.2000 LE7 190 12 228796 0,86 -6,46 20,42 0,78 -10,73 24,9 0,85 -6,39 20,83

22.05.2000 LE7 181 15 152224 0,28 -19,61 26,69 0,27 -33,28 38,33 0,26 -17,1 30,12

07.03.2001 LE7 188 23 161916 0,89 -0,27 18,54 0,91 9,39 17,72 0,88 1,24 19,98

28.03.2001 LE7 191 12 269166 0,08 -3,23 6,61 0 1,09 4,99 0 1,09 4,99

29.03.2001 LE7 182 25 160308 0,94 4,7 14,26 0,93 8,15 16,62 0,95 3,59 14,2

04.05.2001 LE7 194 12 195320 0,85 -1,67 17,49 0,77 -7,41 20,81 0,84 -2,29 17,98

12.01.2002 LE7 173 33 129845 0,88 3,37 14,73 0,88 3,53 14,97 0,88 3,51 14,94

04.03.2002 LE7 194 27 130909 0,96 -0,65 13,4 0,96 -1,97 13,97 0,96 -1,98 13,32

04.03.2002 LE7 194 28 130614 0,94 0,22 16,41 0,94 -1,38 16,66 0,94 -0,85 16,28

13.03.2002 LE7 193 27 147399 0,92 -0,63 17,84 0,92 0,57 17,98 0,92 0,19 17,83

13.03.2002 LE7 193 28 144679 0,93 -0,5 15,84 0,93 0,5 15,97 0,93 -0,07 15,81

05.04.2002 LE7 178 14 234555 0,21 7,01 10 0,2 -0,63 4,73 nan 98,47 7,35

09.04.2002 LE7 174 21 164594 0,85 -1,92 21,64 0,88 8,31 20,13 0,83 -2,36 24,33

13.04.2002 LE7 170 21 56478 0,9 -0,36 16,92 0,93 -5,24 14,17 0,89 -1,43 17,83

27.05.2002 LE7 190 11 156070 0,89 -2,91 9,91 0,91 -1,87 9,09 0,91 -1,75 8,97

29.05.2002 LE7 188 12 237853 0,72 3,14 10,25 0,7 2,96 10,79 0,75 2,38 9,44

30.05.2002 LE7 195 12 265463 0,86 2,48 10,83 0,87 2,15 10,4 0,87 2,11 10,32

14.12.2002 LE7 173 33 115084 0,94 5,39 15,17 0,94 5,46 15,97 0,94 5,26 15,73

10.01.2003 LE7 202 33 124431 0,79 -7,05 16,64 0,67 -11,84 25,96 0,8 -6,56 18,47

18.01.2003 LE7 194 28 80024 0,9 -2,55 20,91 0,9 -3,15 20,82 0,91 -2,89 20,42

24.01.2003 LE7 172 33 121635 0,95 -4,04 15,15 0,94 -4,12 15,53 0,94 -4,09 15,5

03.02.2003 LE7 178 25 169585 0,33 -1,01 10,46 0,12 2,68 10,03 0,13 2,41 10,22

07.02.2003 LE7 174 36 7206 nan -0,64 2,5 nan -0,25 1,7 nan -0,02 0,45

07.02.2003 LE7 190 29 63009 0,92 1,11 17,62 0,9 -8,06 21,05 0,9 -2,02 20




Date


for inter-

comparison


Sensor


RMSE R Bias

unbiased RMSE

R Bias unbiased

RMSE

08.02.2003 LE7 197 30 88504 0,92 -0,57 10,22 0,91 1,13 10,89 0,93 -0,2 9,86

09.02.2003 LE7 172 21 180808 0,64 -5,34 16,74 0,18 9,82 20,24 0,5 -4,2 23,35

12.02.2003 LE7 193 23 113259 0,73 3,17 18,35 0,67 -10,87 18,47 0,68 4,49 23,78

14.02.2003 LE7 191 27 113591 0,61 8,2 21,81 0,61 3,1 21 0,66 -1,01 18,32

15.02.2003 LE7 198 27 133319 0,79 -0,12 2,38 0,73 -0,36 3,62 0,76 -0,08 2,64

16.02.2003 LE7 189 22 119611 0,53 -7,11 13,22 0,32 3,72 11,68 0,46 -3,41 16,59

19.02.2003 LE7 194 19 149364 0,75 -12,64 21,67 0,73 0,78 22,56 0,74 0,19 22,13

19.02.2003 LE7 194 25 88771 0,75 8,11 25,34 0,76 -3,26 25,09 0,75 6,68 26,56

19.02.2003 LE7 194 26 150273 0,85 5,87 21,02 0,87 -1,91 20,01 0,86 2,21 20,86

19.02.2003 LE7 194 27 112814 0,46 2,54 16,43 0,43 -1,78 15,64 0,49 -1,63 14,85

19.02.2003 LE7 194 28 126248 0,94 -1,1 16,68 0,94 -2,58 16,95 0,94 -1,38 15,9

21.02.2003 LE7 192 27 94449 0,72 9,59 22,49 0,73 6,21 21,24 0,76 1,61 19,32

24.02.2003 LE7 197 25 162640 0,92 -1,22 12,45 0,92 1,61 12,64 0,94 0,03 10,66

26.02.2003 LE7 195 27 107653 0,84 5,82 23,91 0,84 -0,14 23,71 0,87 1,59 21,39

26.02.2003 LE7 195 28 69767 0,93 -0,11 17,24 0,94 -1,88 16,7 0,94 -0,91 16,3

27.02.2003 LE7 186 26 158302 0,57 -3,55 21,46 0,44 7,65 22,89 0,54 0,41 22,85

27.02.2003 LE7 186 27 141617 0,58 1,63 12,99 0,58 -2,08 12,32 0,56 -0,61 13,43

27.02.2003 LE7 186 28 133052 0,93 -2,94 14,14 0,91 -5,59 15,93 0,93 -2,16 14,33

27.02.2003 LE7 186 29 91336 0,85 -5,54 22,31 0,85 -9,22 22,83 0,84 -2,92 23,31

27.02.2003 LE7 186 30 127432 0,9 1,02 19,33 0,88 -5,85 20,78 0,9 -0,04 19,59

27.02.2003 LE7 186 31 97801 0,92 1,38 18,22 0,92 4,97 19,09 0,92 1,76 18,9

27.02.2003 LE7 186 32 48518 0,93 -1,59 11,99 0,92 -2,64 13,33 0,93 -1,63 12,13

08.03.2003 LE7 169 34 105942 0,96 -4,11 13,81 0,96 -3,93 13,76 0,96 -3,92 13,76

08.03.2003 LE7 185 21 145969 0,3 10,39 15,39 0,23 -1,26 10,08 0,3 6,07 17,57

12.03.2003 LE7 181 32 86814 0,86 -1,29 9,21 0,81 -2,79 12,53 0,89 -1,04 8,55

15.03.2003 LE7 186 17 152573 0,56 -18,33 18,95 0,56 -4,16 18,16 0,53 -8,9 19,24

15.03.2003 LE7 186 20 176128 0,49 7,94 21,76 0,59 -3,96 18,46 0,5 5,41 23,32

23.03.2003 LE7 194 28 143183 0,96 -1,05 12,37 0,97 -0,35 11,93 0,96 -0,93 12,3

24.03.2003 LE7 185 23 171462 0,74 -0,13 15,43 0,74 7,02 17,99 0,62 -5,34 18,06

24.03.2003 LE7 185 31 132405 0,94 0,22 11,7 0,94 -1,65 12,05 0,94 0,76 12,05

28.03.2003 LE7 181 21 186386 0,59 11,89 21,08 0,52 -4,86 18,45 0,59 12,48 26,12

28.03.2003 LE7 181 22 184319 0,53 -0,17 21,99 0,5 10,27 20,69 0,48 1,35 25,69

28.03.2003 LE7 181 23 173309 0,88 1,66 19,95 0,9 9,25 18,67 0,86 -0,33 22,77

28.03.2003 LE7 181 24 166692 0,79 -1,36 9,76 0,82 -2,81 11,41 0,74 -0,1 10,37

01.04.2003 LE7 177 14 261073 0,21 -8,68 8,87 0,13 0,59 3,81 0,17 0,32 4,46

01.04.2003 LE7 193 27 141321 0,93 0,83 15,09 0,93 -1,26 15,51 0,93 0,41 15,12

09.04.2003 LE7 185 13 201437 0,09 -1,48 4,96 0,23 0,44 3,28 0,16 0,36 3,49




Date


for inter-

comparison


Sensor


RMSE R Bias

unbiased RMSE

R Bias unbiased

RMSE

10.04.2003 LE7 192 12 278368 0,07 -0,03 3,49 0,11 0,43 3,15 0,09 0,41 3,19

17.04.2003 LE7 193 13 269038 0,24 -2,47 7,32 0,29 0,73 5,48 0,25 0,13 6,3

17.04.2003 LE7 193 14 163481 0,73 13,76 18,54 0,74 1,16 16,35 0,73 11,69 21,92

11.05.2003 LE7 169 30 136848 0,85 -1,48 9,52 0,86 -1,32 9,45 0,86 -1,24 9,26

14.05.2003 LE7 174 13 2159 0,05 1,83 4,6 0,06 0,6 4,53 0,06 0,62 4,55

11.10.2003 LT5 192 27 122304 0,85 1,43 23,05 0,86 0,16 22,84 0,86 0,98 22,7

18.10.2003 LT5 193 27 112554 0,87 2,34 17,8 0,88 1,58 17,68 0,87 2,48 18,12

25.10.2003 LT5 194 28 113400 0,92 -1,29 18,35 0,92 -3,36 18,62 0,92 -2,35 18,36

30.05.2004 LT5 200 16 158380 0,91 3,15 14,82 0,91 2,77 14,64 0,92 2,65 14,48

30.05.2004 LT5 200 17 162872 0,92 -3,14 18,4 0,92 -2,71 18,33 0,92 -2,6 18,24

10.06.2004 LT5 197 13 192733 0,96 -0,89 12,86 0,96 -0,42 12,76 0,96 -0,37 12,72

05.04.2005 LT5 194 28 135212 0,9 0,89 17,18 0,9 0,61 17,72 0,89 1,38 18,24

17.04.2005 LT5 198 11 131059 0,62 0,29 10,45 0,6 -1,07 10,74 0,62 -0,82 10,91

25.04.2005 LT5 190 14 203426 0,85 2,93 15,68 0,8 -4,13 17,66 0,84 0,7 16,87

30.04.2005 LT5 193 14 131271 0,9 6,19 17,38 0,86 -8,42 20,72 0,9 6,94 18,59

16.05.2005 LT5 193 11 266156 0,81 0,21 8,56 0,78 -0,43 9,23 0,78 -0,05 9,44

16.05.2005 LT5 193 12 279181 0,41 0,36 6,31 0,33 -0,34 6,54 0,4 -0,2 6,56

15.04.2006 LT5 195 17 73472 0,18 11,19 18,94 0,19 2,14 16,49 0,21 2,88 16,59

30.04.2006 LT5 188 12 254569 0,75 2,99 14,56 0,65 6,63 16,55 0,71 3,63 16,09

03.05.2006 LT5 193 11 200192 0,88 -5,7 18,38 0,85 -6,56 20,82 0,85 -4,3 21,11

05.05.2006 LT5 191 12 220061 0,84 -10,2 22,97 0,72 -19,5 30,66 0,84 -7,79 23,22

05.05.2006 LT5 191 13 235105 0,74 11,97 21,28 0,54 28,84 32,94 0,77 5,42 20,85

08.05.2006 LT5 196 13 206648 0,91 5,59 19,1 0,81 11,86 27,12 0,91 3,45 18,67

11.06.2006 LT5 194 27 157030 0,61 4,83 17,57 0,6 5,17 18,48 0,62 4,53 17,37

13.06.2006 LT5 192 27 157021 0,75 -3,32 14,22 0,73 -3,46 15,15 0,75 -2,86 14,03

20.06.2006 LT5 193 27 141747 0,6 -3,1 12,62 0,6 -2,86 12,55 0,61 -2,58 12,12

29.05.2007 LT5 194 12 250863 0,94 -2,36 9,73 0,93 -2,09 10,6 0,95 -1,43 8,96

12.06.2009 LT5 193 28 153201 0,83 -2,58 13,34 0,82 -2,35 13,4 0,83 -2,18 13,23

07.03.2010 LT5 181 28 126737 0,95 2,73 14,36 0,92 4,48 17,49 0,94 3,26 14,91

23.03.2010 LT5 181 24 154306 0,71 -4,5 20,75 0,72 -13,12 20,25 0,69 -5,15 22,7

23.03.2010 LT5 181 26 156872 0,88 -12,8 17,95 0,88 -12,74 19,88 0,88 -11,81 19,32

04.04.2010 LT5 169 23 52340 0,63 4,74 16,33 0,55 8 17,9 0,58 6,72 17,9

04.04.2010 LT5 185 31 134551 0,86 -2,08 10,45 0,83 -2,32 11,68 0,86 -1,8 10,21

04.04.2010 LT5 185 32 129838 0,73 -2,55 11,16 0,69 -2,96 12,95 0,74 -2,25 10,95

29.03.2011 LT5 186 20 124324 0,03 11,91 12,31 0,08 -0,52 4,53 0,04 5,13 13,53

29.03.2011 LT5 186 21 69851 0,58 7,43 11,61 0,76 -0,68 8,09 0,69 0,69 10,36

08.05.2011 LT5 194 12 237292 0,92 -0,71 16,96 0,86 -6,61 22,4 0,93 0,51 17,03




Date


for inter-

comparison


Sensor


RMSE R Bias

unbiased RMSE

R Bias unbiased

RMSE

14.05.2011 LT5 188 12 228286 0,92 -1,52 17,21 0,88 -6,79 21,32 0,92 -1,6 17,73

21.08.2011 LT5 193 27 106941 0,57 -0,87 6,99 0,56 -0,81 6,84 0,56 -0,79 6,83

22.09.2011 LT5 193 28 108787 0,92 -3,22 13,55 0,9 -5,04 16,64 0,92 -2,8 13,91

Table 4.5: Statistical measures derived from the validation of the continental European fractional

snow cover products with snow maps generated manually from very high resolution optical satellite

data. Analyses are performed by ENVEO within the EU FP7 project CryoLand.

Acquisition Date

Scene Centre

(Lat/Lon) Satellite

Number of pixels used for

intercomparison

Correlation Coefficient

Bias Unbiased

RMSD

11.02.2004 59.65N/10.85E SPOT5 14238 nan 2.82 8.40

07.03.2004 61.76N/11.22E SPOT5 23235 nan 5.80 13.19

10.03.2004 66.04N/18.94E SPOT5 29475 nan 0.59 4.10

12.03.2004 64.39N/12.32E SPOT5 23863 nan 1.16 5.50

12.03.2004 65.21N/13.11E SPOT5 20699 nan 1.30 5.85

28.03.2004 62.84N/13.95E SPOT5 22986 nan 2.45 9.25

01.04.2004 64.43N/13.14E SPOT5 26480 nan 0.43 3.51

03.04.2004 68.63N/16.67E SPOT5 22956 nan 2.00 8.11

09.04.2004 59.96N/17.71E SPOT5 14048 nan -0.08 1.23

10.09.2004 59.56N/7.66E SPOT5 11735 nan 0.00 0.00

09.02.2005 47.99N/6.53E SPOT5 3952 0.83 6.96 15.85

09.02.2005 47.99N/6.53E SPOT5 12285 0.78 13.13 23.44

30.12.2005 45.60N/14.12E SPOT5 2796 nan 7.95 14.66

11.12.2008 37.83N/43.84E SPOT5 7053 nan 2.99 14.20

24.11.2009 43.67N/40.97E SPOT5 5554 0.38 12.96 23.27

24.11.2009 43.67N/40.97E SPOT5 15322 0.81 14.21 27.59

06.01.2010 38.82N/40.50E SPOT5 6207 nan 5.17 17.53

10.03.2010 51.78N/8.20E SPOT5 18110 0.88 11.19 18.12

24.12.2010 51.17N/4.25W SPOT5 18139 0.66 -13.45 15.16

25.12.2010 43.19N/4259E SPOT5 6255 nan 44.64 38.75

05.01.2011 49.64N/3.53E SPOT5 13598 0.85 -2.79 14.08

14.03.2011 39.30N/43.16E SPOT5 15978 0.54 -1.76 9.14

25.03.2011 42.71N/19.97E SPOT5 6656 0.90 1.03 19.03

11.04.2011 46.08N/7.31E SPOT5 15208 0.85 3.64 23.76

11.04.2011 45.11N/6.95E SPOT5 14280 0.83 7.17 25.56




Acquisition Date

Scene Centre

(Lat/Lon) Satellite

Number of pixels used for

intercomparison

Correlation Coefficient

Bias Unbiased

RMSD

01.10.2011 61.48N/8.78E Worldview2 1377 nan 0.00 0.07

31.01.2012 49.37N/20.25E SPOT5 6976 nan 4.63 11.98

31.01.2012 47.52N/24.29E SPOT5 4599 nan 4.73 11.37

31.01.2012 49.37N/20.25E SPOT5 6976 nan 4.63 11.98

03.02.2012 52.20N/4.28W SPOT5 16629 0.86 -3.89 21.78

03.02.2012 51.94N/3.63W SPOT5 17061 0.86 4.08 23.00

08.02.2012 48.42N/15.80E SPOT5 5676 nan 6.77 13.38

19.02.2012 54.93N/2.87W SPOT5 18919 0.70 -1.57 10.91

16.03.2012 43.20N/18.84E SPOT5 5465 nan 4.50 13.19

27.03.2012 41.80N/23.63E SPOT5 5509 0.92 5.43 18.31

17.04.2012 61.30N/8.94E Worldview1 209 nan -0.81 0.95

01.05.2012 61.54N/9.09E Worldview1 1074 0.83 -3.65 14.63

05.05.2012 61.30N/8.94E Quickbird 288 0.86 -5.33 15.77

30.05.2012 61.48N/8.78E Worldview2 787 0.78 -9.83 13.76

09.12.2012 43.05N/1.08W SPOT5 17171 0.86 -0.01 23.87

03.01.2013 43.04N/0.87E SPOT5 11002 0.82 10.41 26.03

14.01.2013 37.47N/3.97W SPOT5 15857 0.76 10.81 22.22

03.02.2013 42.99N/0.63W SPOT5 11403 0.65 -13.15 32.16

02.03.2013 45.67N/7.11E SPOT5 16814 0.67 -7.84 28.58

For the validation of the SCE product with reference snow maps from Landsat data also different

snow conditions are considered. The snow maps are therefore intercompared with respect to the

fraction of snow cover extent per pixel. The same statistical analyses as shown before for the total

area and different surface classes are used for this kind of validation. The unbiased RMSE and the

Bias values resulting from this validation of the SCE product with fractional snow cover maps from

all 110 selected Landsat scenes are illustrated in the scatterplots in Table 4.6. The results of the

comparison of the SCE product with Dozier snow maps from Landsat scenes shows an

underestimation of the SCE product in the 0-25 % range (bias 9.9 %) compared to the reference

snow product, and a tendency to overestimate the snow cover slightly towards higher snow cover

fractions (bias -9.28 % in the 75-100 % range) compared to the reference snow maps. The same

tendency is also apparent in the validation with the Klein snow maps from Landsat data.




Table 4.6: Validation results of fractional SE products detecting snow on ground vs. LS snow

algorithms for different fraction sections: 0-25 %, 26-50 %, 51-75%, 76-100 %. Results are generated

by ENVEO during the ESA QA4EO project SnowPEx.

Dozier Klein

FSC Unbiased RMSE Bias Unbiased RMSE Bias

0 -

25 %

26 -

50

%

51 -

75

%

76 -

100

%




5 CONCLUSIONS

The continental European SCE product has so far been validated with snow maps generated from

a selection of 110 Landsat scenes acquired between 2000 and 2011, and from 44 selected VHR

optical satellite data acquired between 2004 and 2013 over different surface classes and at

different snow conditions all over Europe. The validation activities were performed in the EU FP7

project CryoLand and the ESA QA4EO project SnowPEx, each under the lead of ENVEO. The

SCE product has been validated on a pixel-by-pixel comparison using statistical measures to

describe the performance of the product, following the validation protocols developed within the

QA4EO project SnowPEx, and agreed by the international snow community.

The mean resulting unbiased Root Mean Square Error derived from the validation of the SCE

product with all reference snow maps is 15%, independently of the applied snow algorithm for the

generation of the reference snow map. The mean Bias derived for validation with Landsat snow

maps is within ± 0.8 %, while the validation with VHR snow maps results in a higher mean Bias of

about 3%. The mean correlation coefficient is in all validation cases about 0.73. Anyway, for

particular validation cases the resulting statistical measures are higher or lower, respectively.

In case of Landsat based reference snow maps, generated with snow algorithms resulting in the

same thematic information as the SCE product, namely snow on ground, also the performance

over non-forested and forested areas has been analysed separately. In forested areas, the

resulting unbiased RMSE is about 17%, while in non-forested areas it is about 14%, with mean

Bias values of about ±1.5%. The derived correlation coefficient is about 0.76 for non-forested

areas, and about 0.60 for forested areas.

Additionally, the agreement of the fraction of snow cover of the SCE product with the aggregated

fractional snow cover from the selected Landsat scenes is assessed. In case of less fractional

snow cover (0% - 25%), the SCE products underestimates the derived reference snow cover, while

for areas largely covered by snow (76% - 100%), the SCE products overestimate the derived

reference snow cover from Landsat data.

The performance of the SCE product is compared with the requirements defined by the users. The

resulting compliance matrix is shown in Table 5.1.




Table 5.1: Compliance matrix for the continental European SCE product.

Requirements Minimum Target Value Range

(Min/Max) Compliant Comments

Thematic

accuracy:

15 % 5 % Unbiased

RMSE: 15 %

Bias:

± 0.8 %

Unbiased

RMSE:

0% / 39%

Bias: -3% /

98%

Partly Reference snow maps need

also validation, performance

depends on surface class

and snow conditions

Time period: Full year Full year 2000 –

present

Yes

Temporal

frequency

Daily 12 hours Daily Yes

Delivery time: 48 hours

after image

acquisition

12 hours

after image

acquisition

12 hours

after image

acquisition

Yes Depends on the availability

of the NRT satellite data

used as input




6 RECOMMENDATIONS

The validation results of the continental European SCE product at 0.005 degree resolution with

snow maps from high resolution optical satellite data indicates that the minimum thematic accuracy

requirement of 15% defined by users can on average be achieved. But, for particular surface or

snow conditions, the differences between the SCE product and the snow maps used as reference

data are larger. For the interpretation of the quality assessment of the SCE product for continental

Europe one should keep in mind that also the snow maps from (very) high resolution optical snow

maps can include some misclassifications.

When reference snow maps from high resolution optical satellite data are generated, the following

facts should be considered:

1. For the generation of a reference snow map from high resolution optical satellite data, a

snow mapping algorithm designed particularly for the spectral capabilities of the satellite

data should be used. The reference snow maps from Landsat data used so far for the

validation activities are based on algorithms originally developed for medium resolution

optical satellite data from Terra MODIS. In the snow community, it is quite common to just

adapt these approaches for Landsat data, as the sensors have similar spectral

characteristics.

2. The generated reference snow maps from high resolution optical satellite data need to be

validated themselves. As snow cover is a highly dynamic parameter which can change

within a few hours from fully snow covered to snow free, the availability of reference data

acquired over the same region and at the same time as a Landsat scene is very limited.

Thus, this task is very challenging and needs a very extensive planning and preparation

phase.

Also, the validation of the continental European SCE product at 0.005 degree resolution with in-situ

snow observations for single points can be critical, as the station of the snow measurement does

not necessarily reflect the snow conditions of a pixel covering an area of about 500m x 500m. Such

a validation of the SCE product has been performed during the QA4EO project SnowPEx by SYKE

for different surface classes within particular snow climate classes after Sturm, Holmgren, and

Liston (1995), but needs to be interpreted very carefully. A publication of these results is in

preparation.

Nevertheless, the current validation results of the continental European SCE product will in future

be extended with further snow maps from high resolution optical satellite data and in-situ

observations, depending on the availability of such data.




7 REFERENCES

Chander, Gyanesh, Brian L. B.L. Markham, and Dennis L. D.L. Helder. 2009. “Summary of Current Radiometric Calibration Coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI Sensors.” Remote Sensing of Environment 113: 893–903.

Dozier, J., and T. H. Painter. 2004. “Multispectral and Hyperspectral Remote Sensing of Alpine Snow Properties.” Annual Review of Earth and Planetary Sciences 32: 465 – 494.

Klein, Andrew G, Dorothy K Hall, and George A Riggs. 1998. “Improving Snow Cover Mapping in Forests through the Use of a Canopy Reflectance Model.” Hydrological Processes 12: 1723–

44.

Salomonson, V.V., and I. Appel. 2006. “Development of the Aqua MODIS NDSI Fractional Snow Cover Algorithm and Validation Results.” IEEE Transactions on Geoscience and Remote Sensing 44 (7): 1747–56. doi:10.1109/TGRS.2006.876029.

Salomonson, V.V, and I Appel. 2004. “Estimating Fractional Snow Cover from MODIS Using the Normalized Difference Snow Index.” Remote Sensing of Environment 89 (3): 351–60.

doi:10.1016/j.rse.2003.10.016.

Sturm, M., J. Holmgren, and G. E. Liston. 1995. “A Seasonal Snow Cover Classification System for Local to Global Applications.” Journal of Climate 8 (5): 1261–83. doi:10.1175/1520-

0442(1995)008<;1261:ASSCCS>;2.0.CO;2.

copernicus global land operations cryosphere … · europe (72n 11w – 35n 50e) with 0.005 degree...

Documents