Developing Operational Soil Moisture Processing Chains: The Experience with European Scatterometers and Synthetic

Aperture Radars

Wolfgang Wagnerww@ipf.tuwien.ac.at

Department of Geodesy and Geoinformation (GEO)Vienna University of Technology (TU Wien)

www.ipf.tuwien.ac.at

Main Expertise

Development of Level 2 and Level 3 processing chains for• C-band Scatterometers (25 km)

– ERS scatterometer (1991-2011)– METOP ASCAT (2006-2020+)

• C-band Synthetic Aperture Radars (SARs)– ENVISAT ASAR (2002-2012)

- Global Monitoring (GM) mode (1 km)- Wide Swath (WS) mode (150 m)

– Sentinel-1 (launch 2013/14)

Level 2 and Level 3 parameters• Soil Moisture

– Surface soil moisture at 25 km (ASCAT) and 1 km (ASAR GM) scale– Soil Water Index (indicator of profile soil moisture) at 25 km

• Freeze/thawing at 25 km• Water bodies at 150 m

Our Approach

Work globally (or at least continental) Use simple but robust models

• Detailed error characterisation and quality flags• Calibration of Level 2/3 algorithms using historic data archives

NRT 25 km ASCAT Soil Moisture Product

Descending orbits on 8/10/2012

NRT Processing by EUMETSAT within H-SAF

NRT 25 km ASCAT Soil Water Index (SWI) Product

Profile Soil Moisture Anomalies on 8/10/2012

NRT Processing chain built up within geoland2;possible extension in GIO Global Land

ASCAT Processing Infrastructure

EU M ETSA TLevel 1 N R T Processing

EU M ETSA TLevel 2 N R T Processing

Level 0N R T

Level 1N R T

Level 2N R T

ZA M GLevel 3a N R T

Processing

Level 3aN R T

IM PortugalLevel 3b N R T

Processing

Level 3bN R T

Level 1 25 km ASC AT backscatter trip letsLevel 2 25 km ASC AT surface soil m oistureLevel 3a 1 km disaggregated ASC AT /ASAR soil m oistureLevel 3b 25 km ASC AT Soil W ater Index (SW I)

Level 0Archive

EU M ETSA TLevel 1

R eprocessing

Level 1Archive

TU W ienLevel 2

R eprocessing

Level 2Archive

U sers

N R T D ata A rchive

U sers

N R T D ata A rchive

U sers

N R T D ata A rchive

U sers

N R T D ata A rchive

Leve l 2 M odel

Param eters

U sers

U sers

M ETO P A SC A T

TU W ienASAR

Processing

Leve l 3a M odel

Param eters

ASAR Level 1 A rchive

EN VISA T A SA R

TU W ienAlgorithm

D evelopm ent

C N ESSoftw are

Engineering

N R T O perationsR eprocessing

A lgorithm& Softw are D evelopm ent

1 km ASAR Soil Moisture

ESA SHARE Project

ASAR WS Water Bodies Product

June 2007 September 2007

Water surface extent in Ob floodplain based on 150 m ASAR WS data

ASAR Processing Infrastructure

So far only in-house• 200 600 Terabyte• 12 processors

Global processing of 1 km ASAR GM data doable

“Only” regional products for 150 m ASAR WS

Technical Challenges• Version control• Quality control• Several projections• Resampling• Processing speed• Visualisation• etc.

Our Experiences with Users

We have several hundreds of data users• Users, including scientific users, come only when operational large-scale

processing capabilities have been put in place Users can live with missing and inaccurate data when well documented Only few users can pay for the data What is a viable business model?

Higher or

sec-ondary educa-

tion24%

Non profit organ-isation

7%

Private com-pany6%Public

body9%Re-

search organ-isation

47%

(No Information Provided)

8%

Organisation Type

Agriculture14%

Climate31%

Disasters5%

Ecosystems 8%

Undefined11%

Water24%

Weather8%

Societal Benefit Area

Users of our ECV soil moisture product released in June 2012

Our Experiences with Science Data Quality

It takes much too long to pass on scientific innovations to the operations The experience gained by working globally is totally different from

working only locally• Local problems may be irrelevant in the global picture • Algorithmic failures may become apparent that have never been predicted by

the theory

Areas of Unexplained

Dry Soil Backscatter Behaviour

Our Experiences with Service Development Models

In earth observation a „common“ service development model is• ATBD and scientific prototype are developed by the scientist• Operational software is written by software engineers• Service is run by operational agency

This approach might work well if abundant funding is available In case of limited resources the problems are

• Very long development cycles (several years)• Neither the software engineers nor the operational agency can build up

enough expertise about the algorithms and products• Scientists do not know whether their algorithms run correctly or not• In case of problems responsibilities are not clear• Development teams may quickly break up under new funding schemes

EO service programmes designed with the vision „to get rid of the scientists“ in the operational phase are bound to fail!

Applying Service Development Models as in NWP

NRT Processing

Facility

Data Archive

Monitoring Facility

NRT Users

Reprocessing Facility

Off-line Users

Global Testing Facility

YesAlgorithms ok?

No

YesOperations ok?

No

R&D Platforms

Satellite Data

Software Flow Data Flow

NRT Processing

Off-Line Processing

Legend

Scientists must be able to test their algorithms without too many constraints!

Large Data Volumes: Rethinking EO Service Models

New EO satellites will acquire an unprecedented data volume• Sentinel-1

– 1.2 Terabyte per day just for Level 0 for one satellite– Tens of Petabyte required for complete mission

Challenges• Data distribution• Reprocessing• Quality control• Visualisation• etc.

New approaches arerequired• Bringing the software to

the data• Massive parallel processing• New cooperation models between science, industry, and users

Planned dissemination network for the Sentinel satellites (ESA)

Wrapping Up

The high technological potential of EO is still only partially tapped Successful EO missions must not stop short of delivering just „images“ With the US system of environmental satellites being at the risk of

collapse, Europe has now a window of opportunity to become the agenda setter in this domain

In order to seize this opportunity Europe must address the current structural deficits in the governance of GMES and planning of the EO data processing capabilities

As of today Europe lacks the processing infrastructure to cope with the huge data volumes generated by novel satellite missions such as Sentinel-1

Processing facilities must offer tens of Petabytes storage, massive parallel processing capabilities and high speed communication networks.

New EO service development models are urgently needed GPOD, CEMS & Co are the pathfinders

An Austrian Initiative

Earth Observation Data Centrefor Water Resources Monitoring

a cooperation initiated by

EODC Water

EODC-Water Objectives

Strategic goals• Establish Austria as a player in the operational EO ground segment and

service market• Strengthen its academic, institutional and commercial organisations by

coordinating, focusing, and reinforcing their expertise and capabilities Overall aims

• Provide unique EO products for improved monitoring of water resources • Operate Collaborative Ground Segment components for the Sentinel satellites• Apply service standards as commonly employed by NWP centres• Focus on fully automatic and continental to global scale products• Establish the logistics and seamless ITC infrastructure system

Key products & services• Pre-processing products for Sentinel-1, Sentinel-2 and Sentinel-3 • Global medium-resolution hydrological parameters

– Soil moisture, snow, water bodies, glaciers, …• Service & Support