designing the user experience for earth observation data ...designing the user experience for earth...
TRANSCRIPT
Designing the User Experience for Earth Observation Data Services in the Cloud
Christopher Lynnes, NASA*
*U.S. Civil Servant 1
https://ntrs.nasa.gov/search.jsp?R=20190032056 2020-06-14T11:53:43+00:00Z
Earth Observing System Data and Information System (EOSDIS)
EOSDIS
Applications
Capture and clean
Education
Process
Archive
Transform*
Distribute Research
*Subset, reformat, reproject
Commercial
2
Distributed Active Archive Centers (DAACs)
ASF: Alaska Satellite FacilityLP: Land ProcessesPO: Physical OceanographyOB: Ocean BiologyORNL: Oak Ridge National LaboratoryNSIDC: National Snow and Ice Data CenterGHRC: Global Hydrology Research CenterASDC: Atmospheric Science Data CenterCDDIS: Crustal Dynamics Data and Information SystemSEDAC: Socio-Economic Data and Applications CenterLAADS: Level 1 and Atmosphere Archive and DIstribution SystemGES DISC: Goddard Earth Sciences Data and Informations Services Center
ASF DAAC Synthetic Aperture Radar
PO.DAAC Physical Oceanography
NSIDC DAAC Cryosphere
LPDAAC Land Processes
GHRC Severe Weather
ORNLBiogeochemical Dynamics
ASDCRadiation BudgetTroposphere
LAADSAtmosphere
OB.DAAC Ocean Biosphere
SEDAC Human Interactions in Global Change
CDDISCrustal Dynamics
GES DISCAtmosphere ModelingHydrology
3
Data Volumes are Growing...
4
Cloud Data Hosting for Data-Proximal Computing
Data Archive
Data Analysis
5
Rethinking the User Experience
6
How Data Distribution Largely Works Today
RepositoryMetadataCommon DAAC
Repository
Staging
Subset Regrid Reformat
order
data
notification
query
results
Analysis
7
Yes, this mode will still be available in the cloud, but...
8
...you’re gonna need a bigger disk.
9
How Data Distribution Largely Works Today
RepositoryMetadataCommon DAAC
Repository
Staging
Subset Regrid Reformat
order
data
notification
query
results
Analysis
10
Working toward a seamless User Experience...
search data
request data with transformations
analyze transformed data
RepositoryMetadataCommon
Subset Regrid Reformat
Staging
query results
staging location
DAAC-managed
Cloud Repository
API
Jupyter Notebook API = Application Program Interface
API
11
1. Easy-to-use API
2. Fast data transformations for synchronous response
3. Generous staging space and longevity
4. Analysis- and cloud-friendly data accessa. Cloud-friendly output formats
i. Cloud-optimized GeoTIFF
ii. zarr
b. OPeNDAP* in the Cloud
What Do We Need to Get There?
*OPeNDAP = Open-Source Project for a Network Data Access Protocol12
One More Seam to Target...
13
Distribution from Multiple DAACs
RepositoryMetadataCommon
DAAC Repository
Staging
Subset Regrid Reformat
order
data
notification
query
results
Analysis
DAAC Repository
Staging
Subset Regrid Reformat
notification
data
14
Data Co-location
ASF: Alaska Satellite FacilityLP: Land ProcessesPO: Physical OceanographyOB: Ocean BiologyORNL: Oak Ridge National LaboratoryNSIDC: National Snow and Ice Data CenterGHRC: Global Hydrology Research CenterASDC: Atmospheric Science Data CenterCDDIS: Crustal Dynamics Data and Information SystemSEDAC: Socio-Economic Data and Applications CenterLAADS: Level 1 and Atmosphere Archive and DIstribution SystemGES DISC: Goddard Earth Sciences Data and Informations Services Center
ASF DAAC Synthetic Aperture Radar
PO.DAAC Physical Oceanography
NSIDC DAAC Cryosphere
LPDAAC Land Processes
GHRC Severe Weather
ORNLBiogeochemical Dynamics
ASDCRadiation BudgetTroposphere
LAADSAtmosphere
OB.DAAC Ocean Biosphere
SEDAC Human Interactions in Global Change
CDDISCrustal Dynamics
GES DISCAtmosphere ModelingHydrology
15
Data Co-location
ASF DAAC Synthetic Aperture Radar
PO.DAAC Physical Oceanography
NSIDC DAAC Cryosphere
LPDAAC Land Processes
GHRC Severe Weather
ORNLBiogeochemical Dynamics
ASDCRadiation BudgetTroposphere
LAADSAtmosphere
OB.DAAC Ocean Biosphere
SEDAC Human Interactions in Global Change
CDDISCrustal Dynamics
GES DISCAtmosphere ModelingHydrology
16
● Migration to cloud based on prioritization by DAACs
● Iterative development of data transformation services
● Community engagement○ Conferences and Workshops○ User Working Groups○ Beta testing○ <your idea / input here>
EOSDIS in the Cloud Going Forward...
17