designing the user experience for earth observation data ...designing the user experience for earth...

Designing the User Experience for Earth Observation Data Services in the Cloud

Christopher Lynnes, NASA*

*U.S. Civil Servant 1

https://ntrs.nasa.gov/search.jsp?R=20190032056 2020-06-14T11:53:43+00:00Z

Earth Observing System Data and Information System (EOSDIS)

EOSDIS

Applications

Capture and clean

Education

Process

Archive

Transform*

Distribute Research

*Subset, reformat, reproject

Commercial

2

Distributed Active Archive Centers (DAACs)

ASF: Alaska Satellite FacilityLP: Land ProcessesPO: Physical OceanographyOB: Ocean BiologyORNL: Oak Ridge National LaboratoryNSIDC: National Snow and Ice Data CenterGHRC: Global Hydrology Research CenterASDC: Atmospheric Science Data CenterCDDIS: Crustal Dynamics Data and Information SystemSEDAC: Socio-Economic Data and Applications CenterLAADS: Level 1 and Atmosphere Archive and DIstribution SystemGES DISC: Goddard Earth Sciences Data and Informations Services Center

ASF DAAC Synthetic Aperture Radar

PO.DAAC Physical Oceanography

NSIDC DAAC Cryosphere

LPDAAC Land Processes

GHRC Severe Weather

ORNLBiogeochemical Dynamics

ASDCRadiation BudgetTroposphere

LAADSAtmosphere

OB.DAAC Ocean Biosphere

SEDAC Human Interactions in Global Change

CDDISCrustal Dynamics

GES DISCAtmosphere ModelingHydrology

3

Data Volumes are Growing...

4

Cloud Data Hosting for Data-Proximal Computing

Data Archive

Data Analysis

5

Rethinking the User Experience

6

How Data Distribution Largely Works Today

RepositoryMetadataCommon DAAC

Repository

Staging

Subset Regrid Reformat

order

data

notification

query

results

Analysis

7

Yes, this mode will still be available in the cloud, but...

8

...you’re gonna need a bigger disk.

9

How Data Distribution Largely Works Today

RepositoryMetadataCommon DAAC

Repository

Staging

Subset Regrid Reformat

order

data

notification

query

results

Analysis

10

Working toward a seamless User Experience...

search data

request data with transformations

analyze transformed data

RepositoryMetadataCommon

Subset Regrid Reformat

Staging

query results

staging location

DAAC-managed

Cloud Repository

API

Jupyter Notebook API = Application Program Interface

API

11

1. Easy-to-use API

2. Fast data transformations for synchronous response

3. Generous staging space and longevity

4. Analysis- and cloud-friendly data accessa. Cloud-friendly output formats

i. Cloud-optimized GeoTIFF

ii. zarr

b. OPeNDAP* in the Cloud

What Do We Need to Get There?

*OPeNDAP = Open-Source Project for a Network Data Access Protocol12

One More Seam to Target...

13

Distribution from Multiple DAACs

RepositoryMetadataCommon

DAAC Repository

Staging

Subset Regrid Reformat

order

data

notification

query

results

Analysis

DAAC Repository

Staging

Subset Regrid Reformat

notification

data

14

Data Co-location

ASF: Alaska Satellite FacilityLP: Land ProcessesPO: Physical OceanographyOB: Ocean BiologyORNL: Oak Ridge National LaboratoryNSIDC: National Snow and Ice Data CenterGHRC: Global Hydrology Research CenterASDC: Atmospheric Science Data CenterCDDIS: Crustal Dynamics Data and Information SystemSEDAC: Socio-Economic Data and Applications CenterLAADS: Level 1 and Atmosphere Archive and DIstribution SystemGES DISC: Goddard Earth Sciences Data and Informations Services Center

ASF DAAC Synthetic Aperture Radar

PO.DAAC Physical Oceanography

NSIDC DAAC Cryosphere

LPDAAC Land Processes

GHRC Severe Weather

ORNLBiogeochemical Dynamics

ASDCRadiation BudgetTroposphere

LAADSAtmosphere

OB.DAAC Ocean Biosphere

SEDAC Human Interactions in Global Change

CDDISCrustal Dynamics

GES DISCAtmosphere ModelingHydrology

15

Data Co-location

ASF DAAC Synthetic Aperture Radar

PO.DAAC Physical Oceanography

NSIDC DAAC Cryosphere

LPDAAC Land Processes

GHRC Severe Weather

ORNLBiogeochemical Dynamics

ASDCRadiation BudgetTroposphere

LAADSAtmosphere

OB.DAAC Ocean Biosphere

SEDAC Human Interactions in Global Change

CDDISCrustal Dynamics

GES DISCAtmosphere ModelingHydrology

16

● Migration to cloud based on prioritization by DAACs

● Iterative development of data transformation services

● Community engagement○ Conferences and Workshops○ User Working Groups○ Beta testing○ <your idea / input here>

EOSDIS in the Cloud Going Forward...

17