EO Data Preservation and Valorisation at ESA

CCI Collocation meeting, ESA-ESRIN, 29 September 2015

Mirko Albani, Henri Laur (ESA)

EO missions in orbit

Heritagemissions

Futuremissions

Flyingmissions

Heritage3rdpartymissions:Landsat,ALOS,Nimbus,Seasat,J-ERS,etc.

EO missions in orbit

è Assets on ground: the data

Noprogramme(funding)forheritagemissions=nodatapreservaHon/valorisaHon/discoverability

Long Term Data Preservation Programme (LTDP)

The current programme [2013-2017] addressing the heritage missions data preservation is: The Long Term Data Preservation Programme (LTDP):

à Part of ESA’s mandatory activities (i.e. “ESA General Budget”), it addresses the preservation of ESA’s assets in science data from space (not only EO data);

à LTDP funds the data preservation, some data valorisation and data accessibility of ESA & Third Party Missions (TPM) EO “Heritage Missions”, starting 5 years after end of satellite operations.

ESA Long Time Data Series (excerpt)

Data from EO missions in red are covered in the current LTDP Programme

ESA Heritage EO Data at ESA

à still a considerable user interest

•  ERS and Envisat data (the main ESA heritage data) à still a major interest from users on heritage EO data

•  Data value & utilization increase if part of long time data series

Statistics at mid 2015

Sentinel

user stats not

included

New users registered by ESA in 2015

Heritage EO Missions: à on-going activities at ESA

•  Overall ESA archive completeness review; heritage ESA & TPM missions data assessed for completeness, availability of provenance & context information, format, naming convention & metadata definition.

•  Recovery of unique data still on heritage media via dedicated transcriptions or to fill identified gaps in master datasets.

•  Level-0 data consolidation according to sensor specific procedures and alignment to ESA EO standard for data archival; collection of all documentation and supporting SW tools and archiving at ESA in a dedicated “Preservation System”.

•  Assessment of processing baseline for each mission instrument data set for possible alignment with current/future missions data (i.e. build long time data series); full data set reprocessing (Level-1).

•  Provision of discovery/access online through ESA systems.

Example #1: à ERS-1/2 Level-0 LBR Instruments Data Consolidation & Master Dataset Generation (1/2)

Figure: ERS-1 ERAC LBR gaps visible in the limit of visibility of ground stations

Phase 1 (Completed): à Consolidated master datasets generated for all ERS-1/2 Low Bit Rate (LBR) instruments (except ATSR); gaps identified.

Phase 2 (On-going): •  Completed ERS-1 data transcription from heritage media to fill gaps •  Refinement of Phase 1 master datasets plus ATSR & SAR Level 0 data

Figure: gaps filled with ERS-1 reprocessed data for ERAC and EMWC products

Example #1: à ERS-1/2 Level-0 LBR Instruments Data Consolidation & Master Dataset Generation (2/2)

Example #2: à SEASAT SAR data recovery: à Greenland glaciers as seen by 3 generations of radar missions

Comparison of images from Seasat on 16 August 1978, ERS-2 on August 1996 and Sentinel-1 on 20 August 2014 to map the retreat of two large glaciers in southeast Greenland over a 36-year period.

Retreat has been estimated at a rate of about 180 m/yr (top glacier) and 61.5 m/yr (bottom glacier)

Seasat L-band SAR ESA data holdings (European coverage Jul-Oct 1978) fully recovered and accessible online for the first time ever. SEASAT, ALOS and JERS-1 data aligned to generate a long time data series in L-band.

Example #3: à ESA AVHRR data holdings status and activities

Status: • Volume: ~23 TB (NOAA 7-19), mix of Level 0, Level 1A and Level 1B products, • Includes 1Km data set acquired during the “1Km” project started in Apr 1992 and ended in Sep 1996 (23 stations worldwide), • Preserved in the past on a best effort basis and archived at ESA facilities: part ingested in robotic tape libraries, part on heritage media, • Not accessible to users at this time via ESA dissemination systems.

LTDP programme impact:

à Re-transcription and consolidation of ESA AVHRR data assets à AVHRR curation project under the auspices of the GSCB aimed at:

ü Compiling an overall inventory of AVHRR data over Europe, ü Generating a “Coherent European AVHRR Level-1 dataset” based on the

combination of data from different European holders: free and open access for science,

ü Planned completion end 2016.

Nimbus-7 •  Coastal Zone Color Scanner (CZCS) •  Nov 1978 – Jun 1986

Marine Observation Satellites MOS-1 & MOS-1b •  MESSR (Multi-Spectral Electronic Self-Scanning Radiometer) •  VTIR (Visible and Thermal Infrared Radiometer) •  Feb 1987 – Nov 1995 & Feb 1990 – April 1996

Adeos-1 •  OCTS (Ocean Colour and Temperature Scanner) •  Aug 1996 – Jun 1997

SPOT-1 & SPOT-2 •  Feb 1986 – Dec 1990 & Jan 1990 – Jun 2009 •  HRV (High Resolution Visible) & HRVIR (High Resolution Visible IR)

Is this the right priority order for CCI needs?

Heritage datasets addressed in the coming months

Future EO Data Preservation and Valorisation Scenario

… in order to enable Earth Resources Management providing heritage data as a service to a global community of users

Objective:

à discoverability and accessibility of “good quality, reliable, curated and interoperable heritage EO data and their associated information”

Areas of activities:

1.   Prevent the loss of heritage EO data and of its associated knowledge

2.   Valorise heritage EO data through creation of knowledge based e.g. on harmonization & combination with new data à long time data series

3.   Removing barriers preventing the access to all (old and new) EO data

Data & Knowledge Digital Preservation à future approach at ESA

•  Preserve ESA EO Heritage missions data & knowledge ensuring authenticity and understandability in the long term (Authority Entity);

à with the assumption/awareness that one can never anticipate the exploitation value of the data asset in the future

•  Plan and implement preservation activities in all EO mission phases to capture knowledge at EO data set generation;

•  Establish a coordinated and harmonised approach across ESA and in cooperation with EO data owners in the Member States, to preserve European Space Science data;

•  Prevent loss of non-ESA EO data at preservation risk and judged of European long-term interest.

Data Valorisation & Discoverability à future approach at ESA

•  Valorise ESA EO Heritage Data: ü  Ensure the full content of the ESA EO archives is fit for contemporary purpose

thus enabling multi-disciplinary applications & broader data use, ü  Improve data quality through algorithms evolution, data reprocessing, data

validation and traceability (QA4EO), ü  Build long time series of coherent data and data

collections for specific applications, ü  In cooperation with other EO data owners and users

in the Member States.

•  Make sure the full content of the ESA EO heritage archives is maintained discoverable, accessible and usable in the long term through state of art technologies and innovative tools & services

Generate Fundamental Level 1 Data Records based on the combination of archived data assets (heritage and current) from National & ESA EO Missions

Implement new concept like “Users to the data” à Applications Platforms (see ESA Ground Segment Evolution Strategy)

ESA UNCLASSIFIED – For Official Use

Proposed at next ESA Council at Ministerial Level

à LTDP+

Why:ESAheritagedatasetsrepresentaunique,valuable,independentandstrategicresourceownedbyESAMemberStates.

How:Incoopera:onacrossESADirectorates(EOP,SRE,HSO)andwithdataownersandusersinMemberStatesthroughacoopera:ve,harmonisedandsharedapproach.

1.Datapreserva:on 3.Datadiscoverability2.Dataimprovement

LTDP+

Ensurecon*nuousqualityimprovementofEOheritagedataforcon*nuitywithcurrent&futuremissionsformul*-disciplinaryapplica*onsandbroaderdatause

LongTermDataPreserva9on

EnsuredataaccessibilitytousersrelyingonlatestICTasexpectedbyusers.Ensuremaximumpossibleexploita*on,includingthroughApplica*onsPlaBorms.

MissiontoHeritageData

SecurepreservaHonofallESAheritagedataassetsandtheirassociatedinforma*onPreventlossofdatafrom(non-ESA)missionsatpreserva*onriskandjudgedofEuropeaninterest

Envisat ERS-1/-2 EuropeanassetsfromLandsat,ALOS,AVHRR,etc…GOCE[2017-2021]

LTDP+willbeproposedatnextC/MIN(December2016)aspartoftheESAGeneralBudget

Thank you for your attention !!!

LTDP+

“Valorize the past being inspired by the present to better shape the future: from Earth Observation towards Earth

Resources Management”

Contact: Mirko.Albani@esa.int