Data Catalog Interoperability

Searching Simulation and Observational Climate Data Using OGC CSW

Kyle Halliday

June 19, 2006

UCRL-PRES-222084

GO-ESSP 2

Overview

Project Overview / Goal Obstacles General Approach Concrete Approach Implementation Related & Future Work

GO-ESSP 3

Project Overview / Goal

Observational (satellite) Data located in catalog at GMU / NASA Ames

Simulation Data located in catalog at PCMDI

Goal—Develop a unified interface for scientists to

use to browse and search catalogs

—Browse API: THREDDS XML—Search API: OGC CSW

GO-ESSP 4

Problem with Searching

Observational data and Simulation data are different—Observational data is measured by instruments and

sensors.—Simulation data is generated by models for a given

experiment/scenario in a series of runs and averaged over some frequency.

Common Denominator—Time, Latitude, Longitude

Conclusion: A meaningful search interface must know about these differences, either beforehand or on-the-fly.

GO-ESSP 5

Many Pegs, No Holes

Adapter design pattern

Client Application

Client Application

Data Catalog

Data Catalog

GO-ESSP 6

Many Pegs, Square Holes

Adapter design pattern

Client Application

Client Application

Data Catalog

Data Catalog

GO-ESSP 7

OGC CSW Adapter

OGC CSW Adapter

Square Pegs, Square Holes

Adapter design pattern

Client Application

Client Application

Data Catalog

Data Catalog

GO-ESSP 8

Abstract Approach

Client SoftwareClient Software Data CatalogData Catalog

whatDoesYourDataLookLike()

likeThis

buildInterface()

searchCatalog()

searchResults

whatOperationsDoYouSupport()

{whatDoesYourDataLookLike, searchCatalog}

GO-ESSP 9

Concrete Approach

OGC CSW exposes the following methods: GetCapabilities

—retrieve service metadata (which operations are supported, description of catalog, contact info, etc)

DescribeRecord—Learn about the catalog’s information

model GetDomain

—Dynamically retrieve information about the data range of a particular parameter

GO-ESSP 10

Concrete Approach

GetRecords—Search the catalog, returning search

results and associated metadata GetRecordById

—Search catalog, returning search results that match the given ID

CSW also supports transactional requests, for adding, updating, and deleting records

GO-ESSP 11

Implementation – Virtual Organization

GMU (Solaris) (laits.gmu.edu)Globus 4.0.1 with GMU Certs.

GMU (Mac)(geobrain.laits.gmu.edu)Globus 4.0.1 with GMU

Certs.

GMU CA center

Ames ipg05 (Linux)(ipg05.ipg.nasa.gov)

Globus 4.0.1 with IPG Certs.GMU LAITS VONASA IPG VO

GMU (Linux)(data.laits.gmu.edu)

Globus 4.0.1 with GMU Certs.

IPG CA center

NASA SGT (Linux)(arao2.sgt-inc.com)Globus 3.2 with CEOS

Certs.

NASA (Linux)(former.intl-interfaces.net)Globus 3.0 with CEOS Certs.

CEOS VO

Authentication among different VO

LLNL esg2 (Linux)(esg2.llnl.gov)

Globus 4.0.1 with ESG Certs.

LLNL ESG VO

ESG CA center

GO-ESSP 12

Implementation – Adapter at LLNL

GetRecords

GetRecordsResponse

ESG Metadata Catalog

ESGQueryService

ESG Publishing Scripts

CSW Plugin WCS

OGC Testbed Data

Data publishing

client

server

GMU GCSF

GO-ESSP 13

Related Work – CDAT THREDDS/OPeNDAP Browser

GO-ESSP 14

Related Work – CDAT THREDDS/OPeNDAP Browser

Tree View generated from THREDDS XML

Tree View generated from THREDDS XML

GO-ESSP 15

Related Work – CDAT THREDDS/OPeNDAP Browser

File Metadata drawn from OPeNDAP

File Metadata drawn from OPeNDAP

GO-ESSP 16

Related Work – CDAT THREDDS/OPeNDAP Browser

OPeNDAP server supports HTTP Basic

authentication

OPeNDAP server supports HTTP Basic

authentication

GO-ESSP 17

Future Work – Advanced Search

GO-ESSP 18

Links

PCMDI— http://www-pcmdi.llnl.gov

GMU LAITS— http://geobrain.laits.gmu.edu/

NASA Ames— http://www.nasa.gov/centers/ames/

IPCC ESG Data Portal— http://esg.llnl.gov/portal

IPCC ESG Data Portal THREDDS Base— http://climate.llnl.gov/dap/ipcc4/?thredds

IPCC ESG Data Portal OPeNDAP Base— http://climate.llnl.gov/dap/ipcc4/

CDAT + Documentation— http://cdat.sf.net

PyDAP OPeNDAP Server— http://pydap.org/

OGC CSW 2.0 Specification— http://portal.opengeospatial.org/files/?artifact_id=5929&version=2

ISO19115/ISO19119 Application Profile for CSW 2.0— https://portal.opengeospatial.org/files/?artifact_id=8305