a multi-satellite, multi-sensors and multi- temporal ... · muscate processing current status 14...

MUSCATE: A MULTI-SATELLITE, MULTI-SENSORS AND MULTI-

TEMPORAL VERSATILE PLATFORM DESIGNED

FOR THEIA

Joëlle DONADIEU

CNES – Toulouse, France

On behalf of main authors:

C L’HELGUEN, P.LASSALLE, P.KETTIG - CNES

T.SEGUR, D.CLESSE, N.NEROME - CAPGEMINI

WHAT IS THEIA ?

2

• THEIA: French national multi-agency organisation for the studies of continental surfaces

• THEIA promotes the use of satellite data by scientific community and institutional actors.

MUSCATE OVERVIEW

MUSCATE = Initially designed as THEIA processing facility of HR

optical satellite products

3 Families of Satellites:

• SPOT 1 to 5: Spot World Heritage project

• LANDSAT 5-7-8

• SENTINEL-2 (A & B)

Already more than30 years of data !

MUSCATE OVERVIEW

4

MUSCATE = Initially designed as THEIA processing facility of HR

optical satellite products

3 Families of Satellites:

• SPOT 1 to 5: Spot World Heritage project

• LANDSAT 5-7-8

• SENTINEL-2 (A & B)

3 Types of Products (CEOS standard):

• Level 1C: ortho-rectified product in TOA reflectance

• Level 2A: level 1C product in surface reflectance

• Level 3A : temporal synthesis of level 2A products

But designed to be evolutive :

• New Satellites

• New type of Products

DataCatalog

ProcessingOrchestrator(PHOEBUS)

MUSCATE ARCHITECTURE

5

Production facility

Distribution facility

Acquisition facility

Exchange space

THEIA shared services – Websitetheia.cnes.fr

Archiving

SPOT, LANDSAT, SENTINEL2 DATA

Administrators

MUSCATE ARCHITECTURE

6

Production facility

Distribution facility

Catalog HMI PHOEBUS HMIAcquisition facility

Exchange space

Processing on Cluster

PHOEBUS orchestrator

Administrator / Catalog

Catalogdata base

THEIA shared services – Websitetheia.cnes.fr

Archiving

SENTINEL2

DataCatalog

ProcessingOrchestrator(PHOEBUS)

MUSCATE: PRODUCTION FACILITY

EXAMPLE OF SENTINEL 2 PROCESSING

7

MAJA algorithm (multi-temporal) :

• Clouds (and shadows) detection,

• Aerosol Optical Thickness (AOT) & Water Vapour estimation,

• Atmospheric effects Correction

• Cirrus correction



8






Initialisation phase (backward)

t0

t0 + 5t0 + 10

t0 + 15t0 + 20

t0 + 25

t0t0 + 5

t0 + 10t0 + 15

t0 + 20t0 + 25

Level 1C

t0

Level 2A



9






Initialisation phase

Nominal phase (recursive)

t0

Level 2A

t0 + 5

Level 1C

t0

Level 2A

t0 + 5

Level 2A



Retrieve of Level-1C products on PEPS platformand ingestion in MUSCATE

catalog

MAJA L2 processing(atmo. correction)

Dispatch to MUSCATE

distribution facility

Archiving

MUSCATE catalog database

REAL

PREDICTED – available inputs

PREDICTED – unavailable inputs

LEGENDE

Level 1C

Level 2A

MUSCATE Catalog(tasks list with all available

inputs)

Level-1C

Level-2A

Level-2A already avail.

PHOEBUS Orchestrator

L2 processing T1L2 processing T2L2 processing T3PHOEBUS pooling

Processingstart

PHOEBUS pooling

Processingstart

End of processing

End of processing

HPC Cluster

10

End of processing

Processingstart

PHOEBUS pooling

Snow L2B processing

Level 2B Snow

Level-2B Snow

11

Snow detection from Sentinel2 Level-2A products

on selected mountain areas defined by THEIA users

MUSCATE: PRODUCTION FACILITY EXAMPLE OF SENTINEL 2 PROCESSING

0

250

500

750

1000

2014 2016 2018 2020 2022 2024 2026

Vo

lum

e in

To

Years

Input

Output

Total

MUSCATE PROCESSING CAPACITY

12

MUSCATE : an Ambitious Project …

• Automatically process up to 2000 products a day (SPOT / LANDSAT /

SENTINEL-2)

• Offer reprocessing capabilities

• Cumulative Volume of MUSCATE data ~1 PBSE

NTI

NEL

-2A

SEN

TIN

EL-2

B

~1 PetaByte in 2024

MUSCATE PROCESSING CAPACITY

13

MUSCATE : an Ambitious Project …

… with “limited budget”!

• Use of CNES HPC centre (resources shared among projects)

• Currently

• Limited to 150 CPU

• 0,5 PB Common storage means

• Extensive re-use of CNES software components

MUSCATE PROCESSING CURRENT STATUS

14

MUSCATE : ramp-up phase in 2018

• Hardware architecture upgraded this year

• Resources and parameters tuning

• Sentinel2A& 2B Level 2A automatically produced on geographic area defined by THEIA users (equivalent to 6 times France area)

• Automatically processed up to 900 SENTINEL-2 L2A products a day with increase production to 1350 products / day

• Cumulative MUSCATE data ~ 140 000 generated data

SEN

TIN

EL-2

A

SEN

TIN

EL-2

B

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

18/08/16 26/11/16 06/03/17 14/06/17 22/09/17 31/12/17 10/04/18 19/07/18

S2-N2B

S2-N2A

SWH-L1C

OUTLOOK AND CHALLENGES

15

Host of new scientific processing chains

Temporal synthesis Level-3A

1st batch : July 2018

Automatic production : end of 2018


16

Landsat 8 production :

on France area

Level 2A products (projected on S2 grid) : september 2018

Level 2B-snow products on mountain area

Reprocessing of level 2A Sentinel2A&2B products :

with MAJA 3.1

Using CAMS and GRIB meteo data

Autumn 2018

Venµs production :

Level 2A products

Using CAMS and GRIB meteo data

Level 3A products


17

Muscate – SAR

Integration of new processing chains in CNES HPC for

FORM@TER datapole to distribute Sentinel1 interferometry

products

Coregistered SLCs, interferograms and spectral diversity corrections

Stack of coregistered interferograms Velocity maps

Time


18

Muscate cloud compatible

Virtualization of MUSCATE environment (compatible

DIAS/AWS/…)

Promote THEIA processing chains in other contexts and other

large areas of interests

Provide massive processing capability deployable outside

CNES in the frame of partnerships and collaborations

a multi-satellite, multi-sensors and multi- temporal ... · muscate processing current status 14...

Documents