validation report for the northern high latitude avhrr l3...

Validation Report for theNorthern High Latitude AVHRR

L3 Sea and Sea Ice SurfaceTemperature product

OSI203a

Version 2.0

STEINAR EASTWOOD, ANETTE LAUEN BORG AND ATLE SØRENSEN

NORWEGIAN METEOROLOGICAL INSTITUTE

19. February 2019

EUMETSAT Ocean and Sea Ice SAFHigh Latitude Processing Centre

Validation Report for NHL L3 SST/ISTproduct SAF/OSI/CDOP/met.no/TEC/RP/117

This page is intentionally left blank.

Version: 2.0 Page: 2 of 25



Document Change Record

Version Date Change Description Responsible

1.0 26-04-2011 Validation of extended 5km productSteinar

Eastwood

2.0 draft 14-06-2017 MajorExtending product with IST and iceprobabilities

SteinarEastwood

2.0 21-08-2018 MajorAdded figure 2, 6 and 7. Added twilightmargin numbers in table 5 and 6.

SE

2.0 19-02-2019 Minor Added AVHRR and IST in title. SE




Table of Contents1 Introduction.................................................................................................................5

1.1 Scope...................................................................................................................5

1.2 Overview.............................................................................................................5

1.3 Glossary...............................................................................................................5

1.4 Applicable documents.........................................................................................6

1.5 Acknowledgements.............................................................................................6

2 Validation data............................................................................................................7

2.1 In situ drifter data................................................................................................7

2.2 Source of in situ data...........................................................................................7

2.3 Sea ice data..........................................................................................................8

2.4 Satellite data........................................................................................................8

2.5 Target accuracy....................................................................................................8

3 Validation method.....................................................................................................11

3.1 SST validation methods.....................................................................................11

3.2 IST validation methods......................................................................................11

3.3 Ice and water probabilities validation method...................................................12

4 Validation results.......................................................................................................13

4.1 L3 NHL SST validation results.........................................................................13

4.2 L3 NHL IST validation results..........................................................................13

4.3 L3 NHL ice and water probabilities validation results......................................13

5 Discussion.................................................................................................................18

5.1 L3 NHL SST product........................................................................................18

5.2 L3 NHL IST production....................................................................................18

5.3 L3 NHL ice and water probabilities product.....................................................19

6 Conclusion.................................................................................................................20

7 References.................................................................................................................21




1 Introduction

1.1 Scope

The purpose of this report is to document the level of agreement between theEUMETSAT Ocean and Sea Ice Satellite Application Facility (OSI SAF) L3 5kmNorthern High Latitude (NHL) Sea and Sea Ice Surface Temperature (SST/IST)product and in situ observations, the so called OSI203a product.

1.2 Overview

The EUMETSAT OSI SAF is producing a range of operational airsea interfaceproducts, namely: wind, sea ice characteristics, Sea Surface Temperatures (SST), seaice surface temperature (IST) and radiative fluxes, Surface Solar Irradiance (SSI) andDownward Longwave Irradiance (DLI). More details on the products and OSI SAFproject are available at http://www.osisaf.org.

SST, SSI and DLI products from the OSI SAF are produced using geostationary andpolar orbiting satellites and are available in level 2 and level 3 formats, with differenttimeliness depending on the production setup.

A specific L3 High Latitude SST/IST product is produced covering the NorthernHigh Latitudes north of 50N, OSI203a. This NHL L3 SST/IST product is derivedfrom the OSI SAF L2 High Latitude SST/IST product (OSI205), which is based onMETOP AVHRR polar orbiter data received through EUMETCast. OSI203a is a12hourly product. It is available on NetCDF format through the OSI SAF HighLatitude FTP server (ftp://osisaf.met.no/prod) and EUMETCast. See alsohttp://osisaf.met.no for product monitoring, validation , news messages and otherinformation.

This report is separated in chapters describing the in situ validation data, thevalidation method and the results obtained. More details on the OSI SAF NHLSST/IST product itself is available in the Product User Manual ([RD.1] ).

1.3 Glossary

Acronym Description

AVHRR Advanced Very High Resolution Radiometer

CMS Centre de Méteorologie Spatiale

DLI Downward Longwave Irradiance

DMI Danish Meteorological Institute

GRIB Gridded Binary Format

HDF Hierarchical Data Format

HL High Latitudes


ftp://osisaf.met.no/prod



Acronym Description

HIRLAM High Resolution Limited Area Model

LML Low and Mid Latitudes

METNO Norwegian Meteorological Institute

MODTRAN Moderate Resolution Transmittance model

MSG Advanced Very High Resolution Radiometer

NOAA National Oceanic and Atmospheric Administration

NAR Near Atlantic Regional

NWC Nowcasting

RMDCN Regional Meteorological Data Communication Network

SAF Satellite Application Facility

SMHI Swedish Meteorological and Hydrological Institute

SSI Surface Solar Irradiance

SST Sea Surface Temperature

1.4 Applicable documents

[RD.1] Northern High Latitude L3 SST/IST Product User Manual, v3.0.

[RD.2] Validation report for the L2 SST/IST product, OSI205, v.1.1.

[RD.3] OSI SAF CDOP2 Product Requirement Document, v3.7.

[RD.4] The recommended GHRSST Data Specification (GDS), v2.0.

1.5 Acknowledgements

The work and validation of the sea ice and water probabilities has been done partlythrough the EUMETSAT Federated Activity on cloud and ice masking in Polarconditions.




2 Validation dataThe validation of the Northern High Latitude SST/IST product is performed using amatchup database (MDB) built specifically for high latitudes. The MDB is built bycollocating in situ drifting buoy observations with the OSI203b product. In addition,the OSI203a contains probabilities of water and ice, which are validated bycomparison with sea ice charts and the OSI SAF sea ice concentration product. Thevalidation data are presented here.

2.1 In situ drifter data

In situ drifting buoy data are used to validate both SST and IST in the OSI203bproduct. For SST validation the water temperature observations from drifting buoysare used. These water temperature (labeled Tw) observations are usually done at 2050cm depth.

For IST validation the air temperature observations (labeled TTT) from drifting buoysplaced on the sea ice are used. The air temperature observations are not measuring thesame quantity as the IST, which is a surface skin temperature. The difference betweenair and skin surface temperature can be significant. But these air temperatureobservations are the only available in situ source with reasonable good coverage inboth time and space and therefore suitable for routinely validation of the IST product.This is also why the target accuracy values are as high as they are (see 2.5). This isfurther discussed in the validation report for OSI205, [RD.2] .

2.2 Source of in situ data

Separate sources for in situ data has been used for SST and IST drifter data.

For SST validation drifting buoy data from the Copernicus Center for Marine Services(CMEMS) In Situ Thematic Assembly Center (TAC) has been used. The In Situ TACcollects in situ observations from various sources, including drifting buoys. Thedrifting buoy data from the In Situ TAC are formatted on NetCDF files by Ifremerand provided at this site:

Up to 20160714:

ftp://ftp.ifremer.fr/ifremer/cersat/projects/myocean/ssttac/insitu/data/

From 20160715:

ftp://ftp.ifremer.fr/ifremer/cersat/projects/myocean/ssttac/insitu/data/v2/

For IST validation drifting buoy data received locally through GTS at MET Norwayhas been used. The GTS stream of data provides global drifter data and all northernhemisphere data poleward of 50N have been collected for this validation.


ftp://ftp.ifremer.fr/ifremer/cersat/projects/myocean/sst-tac/insitu/data/v2/

ftp://ftp.ifremer.fr/ifremer/cersat/projects/myocean/sst-tac/insitu/data/



2.3 Sea ice data

For validation of the sea ice and water probabilities in OSI203a two sources of seaice data have been used. The first source of validation is the sea ice charts from theNorwegian Ice Service. These charts are manually drawn charts based mainly SARdata, in addition to optical data from MODIS and AVHRR and passive microwavedata from AMSR2. These charts are drawn daily on working days, and providepolygons of areas where the sea ice concentration is in certain ranges. They cover theNorth Atlantic region from East Greenland to Novaja Zemlja. An example of an icechart is shown in Figure 1. The sea ice charts have been collected from CMEMS,where they are provided as 1km gridded products.

Since the Norwegian Ice Service charts do not cover the whole are of interest, the OSISAF sea ice concentration product has also been used for this validation, to also coverthe inner Arctic Ocean.

2.4 Satellite data

As input data to the level 3 OSI203a processing, the level 2 OSI205 SST/ISTproduct is used. OSI205 product files back to 20160401 have been used to produceOSI203a products for the period 20160401 to 20170215, so almost one year ofdata.

The OSI205 product provides SST and IST fields, in addition to probabilities ofwater and ice. The OSI205 data are on swath format which are gridded and averagedto the 5 km 12 hourly OSI203a product.

2.5 Target accuracy

The required accuracy of the SST and IST products are defined as monthly mean biasand standard deviation of the surface temperatures values compared with in situmeasurements. Three requirement levels are defined in the PCR v3.7 [RD.3] :

• Threshold – The model user community gain no improved model performanceusing data of worse quality than this.

• Target – This is an intermediate quality level, between the two extremes(Threshold and Optimal), at which the product quality aim at.

• Optimal – The model user community can not gain from improvements in theST quality beyond this level.

The validation of the OSI203a product will be compared with the target accuracyrequirement.

The IST accuracy requirements are split in two: 1) requirements for validation againstin situ IR radiometers, and 2) requirements for validation statistics against in situbuoy data. This is discussed further in [RD.2] . All threshold accuracies are given in




Table 1.

Treshold, std/bias, K Target, std/bias, K Optimal, std/bias, KRadiometer Buoy Radiometer Buoy Radiometer Buoy

SST -- 1.5/1.5 -- 1.0/0.7 -- 0.3/0.1

IST 3.0/2.5 4.0/4.5 2.0/1.5 3.0/3.5 0.8/0.5 1.0/0.8

Table 1: SST and IST temperature quality requirements thresholds (from [RD.2]).

The required accuracy for the sea ice and water probabilities are defined as monthlyprobability of detection and false alarm ratio.

The required accuracy for the sea ice and water probabilities are defined as monthlyprobability of detection (PoD) and false alarm ratio (FAR). The quality requirementsare given in Table 2.

Treshold requirement Target requirement Optimal requirement

PoD FAR PoD FAR PoD FAR

Ice andwater prob 0.65 0.40 0.80 0.20 0.90 0.10

Table 2: Ice and water probabilities quality requirements thresholds (from [RD.2]).




A

1st L (left): CMEMS ice concentration

1st R (right): OSI SAF ice concentration

2nd L: probability of water

2nd R: probability of ice

3rd L: fraction of water from PPS2014

3rd R: fraction of ice from PPS2014

4th left: land mask

White = undefined or land

Figure 1: Examples of sea ice validation data and probability fields from OSI-203-a product2017.03.15 12UTC. Legend in 1st left applies to all images, except land mask. In land maskred=land, green=sea, blue=ice cape. White=clouds/no data.




3 Validation methodFor validation of OSI203a the data described in Chapter 2 has been used. Thevalidation methods are described below.

3.1 SST validation methods

The central part of the SST validation is building of the validation matchup data base(MDB). The MDB has been built by collocating the L3 NHL SST product with theSST in situ data. In this collocation a set of rules have been applied, amongst other tomake the matchups as representative as possible:

1. Matchups have maximum 1 hour in time difference.

2. In situ observations are matched to the SST product pixel (5km pixel size)they lie within, labeled the center pixel.

3. A 5x5 box of pixels around this center pixel are kept.

4. Only matchups with cloud free (that is valid SST) center pixel are kept.

The SST validation is performed mainly by calculating monthly statistics for thedifference between the center SST pixel and in situ SST values, SSTcSSTinsitu.Standard mean difference (bias) and standard deviation are used. In addition, themargin of the bias and standard deviation when compared with the accuracyrequirement is calculated, in percentage.

A set of filters are applied to the matchups before the statistics are calculated. This ispartly to only validate the data that users are advised to use quantitatively, partlybecause the quality control of the in situ observations is limited.

• quality_level >= 3

• abs(SSTcSSTclimatology) <= 10ºC

• abs(SSTcSSTinsitu) <= 3.0ºC

• MeteoFrance CMS buoy black list is applied

3.2 IST validation methods

For the IST validation the methods are very similar as for SST validation, except thata different in situ data set is used. The same rules 14 are applied as for SST, exceptusing IST instead.

For the quality control, the following filters are applied for IST:

• quality_level >= 3

• abs(ISTcISTinsitu) <= 10.0ºC

This last filter has a much less strict threshold. This is because the ISTinsitu value isless representative of the ice surface temperature, than for the SST equivalent.




3.3 Ice and water probabilities validation method

To validate the ice and water probabilities the CMEMS and OSI SAF sea iceconcentration data sets described in section 2.3 are used. These ice concentrationproducts are regridded to the OSI203a product grid using nearest neighborinterpolation. Examples are shown in Figure 1. The probabilities and iceconcentration field are then compared grid point by grid point and summarized in acontingency table, as seen in Table 3. Validation statistics are calculated based on thiscontingency table with number of matches. To compare probabilities with iceconcentrations, both data sources are classified. Ice is defined as ice probability >=99% and ice concentration > 95%, water as water probability >= 99% and iceconcentration = 0%. The mix in between is not used in the comparison.

OSI-203-a

Pice >= 99% Pwater >= 99% Pmix

CMEMS/OSI SAF

icechart/pro

duct

Ice (90-100%) pice_ice pwat_ice

Water (0-5%) pice_wat pwat_wat

Mix (6– 89%)

Table 3: Contigency table for calculating validation statistics for ice and waterprobabilities. The entries in the table are number of matches. The mix entries are notfilled in, as they are not used in the statistics.

The statistical measures used are probability of detection (PoD) and false alarm ratio(FAR). The equations for calculating PoD and FAR for ice and water classificationare given below:

PoD ice= pice_icepice_ice+pwat_ice

FAR ice= pice_watpice_wat+pice_ice

PoDwater= pwat_watpwat_wat+pice_wat

FARwater= pwat_icepwat_ice+pwat_wat

When there are less than 50 data points in the denominator of these expression, theyare not presented, as the values are then not regarded as statistically significant.




4 Validation resultsThe validation results are split in three sections. First one section with the results forthe SST validation, one for the IST validation and one for the ice probabilityvalidation.

4.1 L3 NHL SST validation results

The validation experiment has been run with almost one year of data (2016.05 to2017.03) to assess the quality of the NHL 12hourly SST product. The results are splitin daytime and night time validation, using the mean time of each data point todecide the time. Daytime is defined as solar zenith angle less than 85 degrees, nighttime as solar zenith angle more than 95 and twilight as between 85 and 95 degrees.Positions of the SST validation data are shown in Figure 6.

The monthly SST validation results are presented in Table 5. For daytime and nighttime bias and standard deviation are shown, with accuracy target requirement andmargin. For twilight there is no target requirement, but margin results using the sametarget requirement are still shown.

Table 5 shows that the SST validation results are mostly within the target requirementfor both bias and standard deviation, both at daytime and night time. The exception isthat bias is slight above target requirement at daytime in January 2017 and standarddeviation is slightly above target requirement during night in June and August 2016.

The validation has also been done per quality level, and the time series of monthlyresults are presented in Figure 4 and Figure 5. The numbers are given in chapter 8.The overall statistics per quality level for the full period are presented in Table 4.These are the numbers that are used in the so called Sensor Specific Error Statistics(SSES), as defined by GHRSST in the Data Specification document [RD.4] .

4.2 L3 NHL IST validation results

The validation for IST was run over the same period as for the SST validation usingthe same methods and definitions. The monthly IST validation results are presented inTable 6. Table 6 shows that for daytime the IST validation results are within theaccuracy target requirement for standard deviation for all months. There is asystematic cold bias and the results are around the target requirement, above for May,August, September, October and March. The positions of the IST validation data areshown in Figure 7.

For nighttime, the IST validation results are worse than for daytime. The standarddeviation is around the target requirement; outside the requirement with maximum0.2ºC, and well within the threshold requirement. The bias is outside the targetrequirement with a systematic cold bias, though within or at the threshold requirementfor all months.




The validation has also been done per quality level, and the overall statistics perquality level for the full period are presented in Table 4. The numbers for the monthlystatistics are given in chapter 8.

The distribution of the difference between IST and Tair is shown in Figure 2. Thisdistribution peaks at 4.5°C, with a more dominant tail towards colder bias. If the coldbias for IST was only due to failing cloud masking, the peak would be closer to 0°C.When the L2 input IST data are compared with in situ IR measurements ([RD.2] ), thecold bias is indeed closer to 0°C. So, we can expect that the large cold bias observedfor IST is at least partly caused by the fact the Tair obs do not well represent the IST.

SST Bias Std dev Num obs

All -0.56 0.94 24821

CL=2 -0.90 1.01 1615

CL=3 -0.73 0.97 5809

CL=4 -0.44 0.89 12025

CL=5 -0.32 0.75 4033

IST Bias Std dev Num obs

All -4.23 3.04 24563

CL=2 -4.75 2.90 2414

CL=3 -4.52 3.06 7157

CL=4 -4.0 3.05 14536

CL=5 -3.20 2.32 189

Table 4: SST/IST error statistics per quality level for OSI-203a, day and night time together.


Figure 2: Distribution of difference between satellite IST and in situ Tair.




Bias Std deviationDate Nobs Bias Req Margin Std Req Margin

Day time

201605 890 -0.44 0.7 37.00 0.73 1.0 27.06201606 962 -0.32 0.7 54.44 0.90 1.0 9.88201607 1403 -0.25 0.7 63.63 0.92 1.0 7.90201608 1924 -0.26 0.7 63.27 0.91 1.0 8.82201609 2074 -0.44 0.7 37.08 0.87 1.0 12.82201610 1619 -0.51 0.7 26.91 0.74 1.0 25.71201611 931 -0.60 0.7 14.37 0.87 1.0 12.83201612 571 -0.71 0.7 -1.00 0.83 1.0 16.90201701 467 -0.76 0.7 -7.96 0.82 1.0 18.32201702 471 -0.67 0.7 3.73 0.73 1.0 26.90201703 383 -0.55 0.7 21.05 0.63 1.0 37.47

201605 85 -0.34 0.7 52.00 0.88 1.0 11.57201606 37 -0.58 0.7 17.57 1.06 1.0 -6.34201607 52 -0.67 0.7 4.48 0.98 1.0 2.09201608 541 -0.30 0.7 56.90 1.08 1.0 -7.78201609 1184 -0.48 0.7 31.25 0.96 1.0 3.64201610 1463 -0.53 0.7 24.00 0.80 1.0 20.01201611 1485 -0.57 0.7 18.93 0.84 1.0 15.73201612 1218 -0.65 0.7 6.88 0.76 1.0 24.34201701 680 -0.58 0.7 16.66 0.81 1.0 18.76201702 377 -0.58 0.7 16.71 0.80 1.0 19.62201703 163 -0.63 0.7 10.68 0.57 1.0 43.31

Twilight

201605 158 -0.74 0.7 -6.41 1.05 1.0 -4.52201606 173 -0.68 0.7 3.23 1.14 1.0 -13.90201607 230 -0.74 0.7 -5.38 1.27 1.0 -27.09201608 357 -0.27 0.7 61.41 1.29 1.0 -28.62201609 318 -0.64 0.7 8.46 1.09 1.0 -8.89201610 168 -0.92 0.7 -31.49 1.08 1.0 -7.66201611 221 -0.89 0.7 -26.61 1.00 1.0 0.15201612 228 -0.57 0.7 18.78 0.88 1.0 11.73201701 136 -0.84 0.7 -19.79 0.82 1.0 17.68201702 67 -0.59 0.7 16.03 0.70 1.0 30.11201703 43 -0.66 0.7 5.95 0.71 1.0 29.48

Night time

Table 5: Monthly SST validation results for 2016.05 – 2017.03, using quality levels 3,4 and5. The requirements and margins are only for daytime and nighttime, and not shown fortwilight.




Bias Std deviationDate Nobs Bias Req Margin Std Req Margin

Day time

201605 574 -3.04 3.5 13.11 2.46 3.0 17.84201606 645 -3.87 3.5 -10.66 2.31 3.0 23.07201607 136 -4.62 3.5 -31.92 1.76 3.0 41.47201608 434 -3.35 3.5 4.21 1.90 3.0 36.62201609 410 -3.03 3.5 13.33 2.25 3.0 24.97201610 148 -3.03 3.5 13.57 2.96 3.0 1.35201611 0201612 0201701 0201702 61 -4.39 3.5 -25.33 2.56 3.0 14.70201703 190 -2.37 3.5 32.18 2.69 3.0 10.25

201605 0201606 0201607 0201608 0201609 144 -4.73 3.5 -35.23 3.09 3.0 -2.84201610 2068 -4.45 3.5 -27.27 3.04 3.0 -1.30201611 3179 -4.60 3.5 -31.38 3.13 3.0 -4.21201612 3724 -4.19 3.5 -19.63 3.17 3.0 -5.54201701 3408 -4.32 3.5 -23.48 3.07 3.0 -2.17201702 1769 -4.07 3.5 -16.22 3.01 3.0 -0.33201703 300 -3.99 3.5 -14.13 2.87 3.0 4.20

Twilight

201605 0201606 0201607 0201608 73 -3.12 3.5 10.83 2.39 3.0 20.27201609 768 -4.27 3.5 -22.11 2.78 3.0 7.44201610 1136 -4.37 3.5 -24.88 3.03 3.0 -1.04201611 356 -3.48 3.5 0.57 3.59 3.0 -19.67201612 174 -1.77 3.5 49.33 3.21 3.0 -7.07201701 284 -2.48 3.5 29.12 3.06 3.0 -1.94201702 576 -2.63 3.5 24.91 3.21 3.0 -6.92201703 699 -3.92 3.5 -12.14 3.16 3.0 -5.18

Night time

Table 6: Monthly IST validation results for 2016.05 - 2017.03, using quality levels 3, 4 and 5. Therequirements and margins are only for daytime and nighttime, and not shown for twilight.




Figure 3: Validation statistics for ice and water probabilities, using CMEMS ice charts(upper two) and OSI SAF ice concentration (lower two) for validation. The purple lines aresea ice PoD, the green lines are water PoD, the red are sea ice FAR and the light blue arewater FAR. There are logarithmical scale on the number plots, which are the second andfourth plots.




2016

05

2016

06

2016

07

2016

08

2016

09

2016

10

2016

11

2016

12

2017

01

2017

02

2017

03

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

Monthly daytime SST statistics per quality level

SD CL=2

CL=3

CL=4

CL=5

Bias CL=2

CL=3

CL=4

CL=5

Month

Bia

s a

nd

sta

nd

ard

de

via

tion

Figure 4: Monthly daytime SST bias and standard deviation per quality level.

2016

05

2016

06

2016

07

2016

08

2016

09

2016

10

2016

11

2016

12

2017

01

2017

02

2017

03

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

Monthly night time SST statistics per quality level

SD CL=2

CL=3

CL=4

CL=5

Bias CL=2

CL=3

CL=4

CL=5

Month

Bia

s a

nd

sta

nd

ard

de

via

tion

Figure 5: Monthly night time SST bias and standard deviation per quality level.




Figure 6: Position of SST validation data.

Figure 7: Position of IST validation data.



4.3 L3 NHL ice and water probabilities validation results

The validation for ice and water probabilities was run over the same period as the SSTvalidation. The monthly validation results for ice and water probabilities are presentedin Figure 3, using both the ice charts from CMEMS (upper two panels) and the OSISAF sea ice concentration (lower two panels) for validation. Both PoD and FAR areshown in Figure 3. In each of the two sets of panels the first is panel is the PoD andFAR validation results and the second panel the number of data point used in themonthly values. During October and November there are too few collocated data withthe ice charts to present validation results, as the sea ice has retreated and the METNorway ice charts do not cover the whole Arctic. The comparison with OSI SAF seaice concentration has data coverage for the full period.

The comparison with ice charts and OSI SAF ice concentration product show thesame tendency. Water probabilities are always within requirement, both for PoD andFAR. The sea ice probabilities are within requirements for parts of the year, exceptduring late summer to early winter. The PoD for sea ice is outside (below) therequirement of 0.8 in October and January when compared with the ice charts and inOctober when compared with OSI SAF sea ice concentration. The FAR for sea ice isoutside (higher than) the requirement of 0.2 when compared with ice charts inOctober and November, and in August to October when compared with OSI SAF seaice concentration.




5 Discussion

5.1 L3 NHL SST product

Table 5 shows that in general the daytime validation results are a bit better than thenight time results, both for bias and std deviation. However, the number of validationpoint vary a lot since the length of day and night differ a lot at high latitudes duringthe year. The bias is always negative, and standard deviation between 0.5 and 0.9.This indicates that the cloud masking is probably not optimal, as undetected cloudsleaves a cold bias and increases the std deviation.

There seems to be a tendency for the summer months to have higher standarddeviation and less cold bias in the SST product. This could be due to diurnal warmingof the surface layer and influencing the validation. This has not been furtherinvestigated here.

Regarding twilight validation results, Table 5 shows that the twilight validation resultsare usually worst than both daytime and night time, sometimes slight better thannighttime. Cloud masking is also more difficult during twilight conditions, withvisible channels giving less information and the important 3.7um channel still beingaffected by reflected solar radiation.

The SST validation results per quality level in Table 4 show that the quality is best forquality level 5 and decreases with quality level, as expected. This is also the overalltrend in the monthly validation results per quality level in Figure 4 and Figure 5, withsome variations for some of the months.

5.2 L3 NHL IST production

As for the SST validation, Table 6 shows that the number of validation data for theIST validation vary a lot. For the IST validation not all months have validation resultsfor daytime or nighttime. This is during periods with either polar night (November toJanuary) or midnight sun (May to August).

At daytime the performance is within accuracy requirements, as the cloud and icemasking has a sufficient quality in polar regions during daytime. There is a significantnegative bias and higher standard deviation that for the SST validation. But this isexpected when comparison is done against air temperature measurements from buoyson the ice, where it is not know if the measurement is done in free air or buried undersnow.

At night time the validation results show a more negative bias and higher standarddeviation. The results are not within the accuracy requirement (which is the same atdaytime and night time). The degradation in quality during night time is somethingthat must be expected in polar conditions, due to the difficulty with cloud and icemasking in polar night condition. This issue of cloud masking in polar conditions has




been further studied in a federated activity between the OSI SAF and NWC SAF, andthis study came to the same conclusion (see Eastwood et al, 2017).

Regarding twilight validation results, Table 6 shows that the twilight results are worsethan for daytime and comparable to nighttime results when we look at those monthswhere the number of validation data are comparable. As for SST, the cloud masking ismore difficult during twilight than at daytime.

The IST validation results per quality level in 4 show that the quality is best forquality level 5. The bias decreases with quality level, as expected, while the standarddeviation is almost constant for level 4, 3 and 2. The quality level settings might notbe optimal, but it is also difficult to use air temperature observations to conclude onthis. Quality level 5 is not used at night time, due to high uncertainty in the cloudmasking over sea ice during night time.

5.3 L3 NHL ice and water probabilities product

Figure 3 shows that there for water detection, the probabilities are good. For sea ice,there is a significant variation in PoD and FAR for the sea ice probability over theyear. There are several conditions that changes with season for sea ice, and conditionsthat makes it more difficult for during parts of the year:

1. The area covered by sea ice vary through the year, with maximum in Marchand minimum in September.

2. Only probability data with solar zenith angle less than 80 degrees are used inthe L3 product, due to higher uncertainties in the during twilight conditions.

3. The surface of the sea ice changes with time of year and area; from thin newice in autumn/early winter, to snow covered ice in winter and spring and wetice/snow and melting ponds during summer.

4. Average cloud cover in polar regions is high, and higher during summer thanwinter.

1. and 2. lead to a large variation in number of available data with probabilitiesthrough the year, while 3. gives changes in sea ice albedo.

The sea ice probability seems to not work optimally under late summer to early winterconditions, with high false alarm ratio and reduction in probability of detection for seaice, caused probably by a combination of the points listed above.

Still, the overall results are good.




6 ConclusionThe 5km NHL SST/IST product OSI203a has been validated and compared with theaccuracy target requirements.

For SST, the validation shows that the OSI203a is mostly within the targetrequirement of |.0°C standard deviation and 0.7°C bias on a monthly basis, both atdaytime and nighttime. So the SST part of the product is ready for distribution.

For IST, the validation shows that at daytime the OSI203a is within the targetrequirement of 3.0°C standard deviation for all months, and within or close to thetarget of 3.5°C bias on a monthly basis. At night time the OSI203a is not within thetarget requirement, but within the threshold requirement of 4.0°C standard deviationand within or close to the threshold requirement of 4.5°C bias.

For the ice and water probabilities, the validation shows that the OSI203a is withinthe target requirement for all month for water probability and for parts of the year forsea ice probability. During late summer to early winter the product is not withinrequirement. Still, the overall results are good, and we suggest that the sea iceprobability part of the product is distributed as for water with a warning

The overall results for the OSI203a product are good, and we suggest that theproduct is distributed with a notification in the Product User Manual and product filemeta data that night time IST quality is lower and the quality of the sea iceprobabilities are lower during the period from late summer to early winter.




7 References

Eastwood, S., A. Dybbroe, R. Scheirer, N. Håkansson and Ø. Godøy (2017): OSISAF/NWCSAF Federated activity on cloud and ice masking in polar conditions Final Report. 21 pages.




8 Appendix A: Quality level validationThe tables used to generate Figure 4 and Figure 5 are presented below.


SD Bias NumDate CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5201605 0.87 0.73 0.70 0.84 -0.71 -0.56 -0.38 -0.52 102 250 547 93201606 1.02 0.87 0.84 1.26 -0.88 -0.42 -0.20 -0.77 159 350 550 62201607 1.09 0.97 0.81 1.07 -0.60 -0.35 -0.10 -0.86 148 399 858 146201608 0.95 0.94 0.82 1.17 -0.57 -0.55 -0.09 -1.01 157 519 1314 91201609 0.89 0.87 0.82 0.98 -0.81 -0.77 -0.28 -0.56 177 598 1334 142201610 0.86 0.78 0.69 0.69 -1.08 -0.80 -0.41 -0.33 105 476 930 213201611 0.76 0.86 0.86 0.57 -0.92 -0.90 -0.45 -0.28 86 342 508 81201612 0.92 0.80 0.77 0.62 -0.98 -0.97 -0.48 -0.02 66 304 228 39201701 0.85 0.80 0.80 0.46 -1.17 -1.05 -0.65 -0.19 55 193 214 60201702 0.88 0.75 0.70 0.43 -1.06 -1.31 -0.89 -0.22 36 63 217 191201703 0.88 0.68 0.65 0.40 -1.08 -0.98 -0.77 -0.25 19 63 133 187

SD Bias NumDate CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5201605 0.54 0.88 0.83 -0.73 -0.50 0.06 0 9 47 29201606 0.82 1.21 0.62 -0.38 -0.87 -0.11 0 8 20 9201607 0.89 1.08 0.39 -0.99 -0.70 -0.29 0 8 34 10201608 0.85 1.21 1.07 0.97 -1.05 -0.59 -0.41 0.03 5 49 341 151201609 0.96 1.04 1.08 0.63 -1.86 -0.89 -0.71 -0.13 3 102 582 500201610 0.97 0.81 0.85 0.66 -0.86 -0.80 -0.66 -0.32 8 77 793 593201611 0.57 0.99 0.87 0.69 -1.54 -0.90 -0.62 -0.38 17 142 836 507201612 0.87 0.98 0.78 0.56 -1.84 -0.87 -0.70 -0.46 14 102 805 311201701 0.25 0.93 0.85 0.52 -0.28 -0.95 -0.61 -0.32 6 91 415 174201702 0.89 0.77 0.76 0.63 -1.29 -1.33 -0.72 -0.24 8 63 129 185201703 0.47 0.57 0.65 0.37 -1.11 -0.85 -0.83 -0.39 3 25 61 77

Day time

Night time

Table 7: Monthly SST validation statistics per quality level.

SD Bias NumDate CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5201605 1.71 2.12 2.48 3.07 -2.42 -3.53 -2.97 -2.25 62 117 424 33201606 1.98 2.56 2.20 0.68 -3.60 -4.16 -3.75 -3.65 125 201 432 12201607 0.79 1.94 1.49 -3.54 -4.55 -4.70 57 75 61 0201608 1.95 2.18 1.30 0.23 -3.98 -3.72 -2.86 -3.15 145 249 182 3201609 2.66 2.24 2.24 1.32 -4.49 -2.82 -3.25 -4.73 144 250 147 13201610 2.59 2.84 2.72 -4.69 -2.20 -4.20 59 87 61 0201611 2 1 0 0201612 0 0 0 0201701 0 0 0 0201702 1.29 2.90 1.09 -4.26 -4.28 -5.12 1 9 44 8201703 1.46 3.16 3.58 1.79 -2.15 -1.19 -2.37 -2.67 13 27 56 107

SD Bias NumDate CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5 CL=2 CL=3 CL=4 CL=5201605 0 0 0 0201606 0 0 0 0201607 0 0 0 0201608 0 0 0 0201609 4.18 2.82 2.85 -4.86 -6.05 -3.62 23 66 78 0201610 2.95 3.20 2.91 -5.69 -4.88 -4.20 221 773 1295 0201611 2.87 3.01 3.12 -5.30 -5.36 -4.28 226 936 2243 0201612 3.22 2.96 3.23 -4.57 -4.53 -4.05 304 1060 2664 0201701 2.87 2.97 3.09 -4.85 -4.60 -4.21 285 965 2443 0201702 3.48 3.18 2.96 -4.91 -4.42 -3.97 89 375 1394 0201703 3.00 2.96 2.79 -2.92 -4.75 -3.70 18 83 217 0

Day time

Night time

Table 8: Montly IST validation statistic per quality level.

validation report for the northern high latitude avhrr l3...

Documents