page 1 csiss center for spatial information science and systems wgiss-37 meeting,2014 cwic...

CSISS Center for Spatial Information Science and Systems

WGISS-37 meeting,2014

CWIC OpenSearch Implementation

Yuanzheng Shao, Lingjun Kang, Archie Warnock{yshao3, lkang3}@gmu.edu

[email protected]

WGISS-37 CWIC Project, April 15, 2014


Outline

• CWIC OpenSearch introduction• Data provider analysis• CWIC OpenSearch architecture• CWIC OpenSearch interface design • CWIC OpenSearch interaction


CWIC OpenSearch IntroductionWGISS37 - CWIC OpenSearch Implementation


• CWIC provides a common access point to search and access satellite data from the CEOS agencies

• Unified query interface and response achieved across all integrated data center

• CWIC enriched existing data center by implementing both CSW interface and OpenSearch interface

• CWIC does not harvest any data or metadata

CWIC OpenSearch Introduction


Data Provider AnalysisWGISS37 - CWIC OpenSearch Implementation


• Heterogeneity among existing data providers– Different web portal– Different query language and queryables– Different metadata model – Different data access method

Data Provider Analysis - Heterogeneity


Data Provider Analysis – 1/3

Data provider

Search criteria Native query

methodData

access URLDatasets number b

CSW & OS Support

spatial temporal Entry ID CSW OS

NASA ECHO O O R SOAP & OS Download 1704 N Y

Brazil INPE R O R Web API Order 15 N N

NOAA GHRSST

O O R CSW Download 51 Y Y

CCMEO O O O a CSW & OS Order 2 Y Y

USGS/LSI O O R SOAP Download and order

38 N N

a The numbers vary over time due to data provider update their records in GCMD.

b CCRS provides two CSW endpoint for RadarSat-1 and 2 respectively.

Required (R) Optional (O) Not Applicable (NA)

Refer CWIC-DOC-12-004r3




Data provider

Support Date

Support Time

Support Single

Temporal point a

Support Multiple

TOI bTemporal Operator Chronological

sorting

NASA ECHO

Yes No Yes Yes Before, After, Between Ascending

Brazil INPE Yes Yes Yes No Overlaps Descending

NOAA GHRSST

Yes Yes Yes Yes Between, EqualTo, GreaterThan, GreaterThanEqualTo, LessThan, LessThanEqualTo

Descending

CCMEO Yes Yes Yes Yes EqualTo, LessThan, GreaterThan, LessThanEqualTo, LessThanOrEqualTo, GreaterThanEqualTo, GreaterThanOrEqualTo

Descending

USGS/LSI Yes Yes Yes No Overlaps Ascending




Data provider

Support BoundingBox

Support Polygon

Support multiple AOI Coordinates

Order Coordinate Projection Spatial Operator

NASA ECHO Yes Yes Yes N/A EPSG:4326 Equal, Touch, Within, Contains, Relate

Brazil INPE Yes No No N/A EPSG:4326 BBOX

NOAA GHRSST

Yes Yes Yes long lat EPSG:4326 BBOX, Contains, Disjoint, Equals, Intersects, Overlaps, Within

CCMEO Yes Yes Yes long lat EPSG:4326 BBOX, Equals, Overlaps, Disjoint, Intersects, Touches, Crosses, Within, Contains

USGS/LSI Yes No No N/A EPSG:4326 BBOX


• In order to discover data from multiple data centers, users need to – deal with different web portal– manage multiple user account information– get familiar with different query interface– understand different metadata model

Data Provider Analysis - Challenges


CWIC OpenSearch ArchitectureWGISS37 - CWIC OpenSearch Implementation


CWIC Architecture Design

• Mediator-wrapper

– Mediator

• Receive a query from data client

• Dispatch request to relevant wrapper

• Assemble and return the query results

– Wrapper

• Translate global query language to native query language

• Execute data inventory search

• Convert native metadata model to global metadata model

• Return response to mediator


CWIC OpenSearch Architecture

CWIC Mediator

CWIC OpenSearch Service Interface

Mediator Core

IDN

Legend

CWIC – Wrapper

CWIC – Mediator

OGC CSW Protocol

Java API

Native Protocol

OpenSearch API

CWIC OpenSearch Clients

OpenSearch Interface

USGS Landsat

Connector

USGS Landsat

Connector


NASA ECHO Connector

NASA ECHO Connector


Brazil INPE Connector

Brazil INPE Connector


CCMEO Connector

CCMEO Connector

USGS LSI Brazil INPE

NASA ECHO

CanadaCCMEO


NOAA GHRSST Connector

NOAA GHRSST Connector

NOAA GHRSST


CWIC OpenSearch Design WGISS37 - CWIC OpenSearch Implementation


• CWIC Best Practice• OGC 10-032r8

– OGC OpenSearch Geo and Time Extensions• OpenSearch Draft 5

– http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_5

• ATOM Syndication Format– http://tools.ietf.org/search/rfc4287

CWIC OpenSearch Design - Standards


• Aim– generating OpenSearch description document dynamically

based on the dataset identifier. Instruct the granule-level querying

– Customized response for each dataset, hard-coded the dataset identifier in the query template

• Parameter– datasetId: used to specify dataset identifier (optional)– clientId: used to specify client identifier (mandatory)

• Sample OSDD request (RESTful request)– http://cwic.wgiss.ceos.org/opensearch/datasets/

Landsat_8/osdd.xml?clientId=chromeTest

CWIC OpenSearch Design - OSDD


• Dataset Identifier (mandatory)– The dataset identifier (datasetId) parameter is used to

specify the dataset identifier which could be retrieved from IDN as DIF Entry ID. This parameter is a required parameter. The client can not specify more than one dataset in single request

– Required namespace: • xmlns:cwic="http://cwic.wgiss.ceos.org/opensearch/extensions/1.0/“

• Client Identifier (optional)– The client identifier (clientId) parameter is used to identify

user information

CWIC OpenSearch Design – Request (datasetId)


• Temporal Extension (optional)– Adopted OGC 10-032r8– Required namespace:

• xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"

• Parameter Values– time:start - a string describing the start of the temporal

interval to search (bigger or equal to)– time:end - a string describing the end of the temporal interval

to search (smaller or equal to).• Format

– Character string with the start of the temporal interval according to RFC-3339

• date only: YYYY-MM-DD • date-time: YYYY-MM-DDTHH:MI:SSZ

CWIC OpenSearch Design – Request (temporal)


• GEO Extension (optional)– Adopted OGC 10-032r8– Required namespace:

• xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/"

• Parameter Value– geo:box - the bouding box specifying the area of interest

• Format– the box is defined by "west, south, east, north" coordinates

of longitude, latitude, in an EPSG:4326 decimal degrees.

CWIC OpenSearch Design – Request (GEO)


• Used namespace in ATOM Response

CWIC OpenSearch Design – ATOM Response

Namespace URL

xmlns http://www.w3.org/2005/Atom

xmlns:opensearch http://a9.com/-/spec/opensearch/1.1/

xmlns:dc http://purl.org/dc/elements/1.1

xmlns:georss http://www.georss.org/georss/10

xmlns:geo http://a9.com/-/opensearch/extensions/geo/1.0/

xmlns:time http://a9.com/-/opensearch/extensions/time/1.0/

xmlns:cwic http://cwic.wgiss.ceos.org/opensearch/extensions/1.0/


• <feed> element


Element Value

title Fixed value: CWIC OpenSearch Response

updated Current system date/time

author Fixed value: <author><name>CEOS WGISS Integrated Catalog (CWIC) - CWIC Contact - Web: http://wgiss.ceos.org/cwic</name><email>[email protected]</email></author>

id Fixed value: http://cwic.wgiss.ceos.org/opensearch/granules.atom

link Provide the “self” link corresponding to the request

link Provide the “search” link corresponding to the OSDD request

opensearch:totalResults The number of search results available for the current search.

opensearch:startPage Replaced with the page number of the set of search results desired by the search client.

opensearch:itemsPerPage The number of search results returned per page.

opensearch:Query Defines a search query that can be performed by search clients.


• <feed>/<entry> element


Element Value

title granule name

id Fixed URL prefix followed by granule identifier. For example: http://cwic.wgiss.ceos.org/opensearch/granules.atom?uid=NAVO-L2P-AVHRR19_L.20091128-AVHRR19_L-NAVO-L2P-SST_s2348_e2352-v01.nc.bz2

updated granule update time

author Fixed value for each data provider

georss:box bounding box of specific entry

dc:date temporal range of specific entry

summary a text construct that conveys a short summary, abstract, or excerpt of an entry.

link see next slide


• <feed>/<entry>/<link> element


Type of link

Definition rel value mime-type

data link representing a data file or other science data resource;

enclosure application/x-netcdf, application/x-hdf, etc.

metadata file with (usually) structured information about corresponding data files

via text/xml

browser image of the data typically used for making data request decisions

icon image/jpeg, image/pngetc.

order granule ordering url enclosure text/html


• <feed>/<entry>/<link> for OPeNDAP


Type of link

Definition rel value

mime-type

Onlink urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Onlink enclosure text/html

Thumbnail urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Thumbnail icon image/png (for GHRSST)

Document urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Document via text/xml

TDS urn:x-esri:specification:ServiceType:TDS enclosure application/x-bzip2

FTP urn:x-esri:specification:ServiceType:FTP enclosure application/x-bzip2

OPeNDAP urn:x-esri:specification:ServiceType:OPeNDAP enclosure text/html

LAS urn:x-esri:specification:ServiceType:LAS icon image/png (for GHRSST)

Download urn:x-esri:specification:ServiceType:Download enclosure application/x-bzip2

WMS/WCS urn:x-esri:specification:ServiceType:WMSurn:x-esri:specification:ServiceType:WCS

enclosure text/xml


CWIC OpenSearch InteractionsWGISS37 - CWIC OpenSearch Implementation


CWIC OpenSearch Sequence Diagram


Register

Dataset

DatasetEntr

yId

Data Provider

Dataset Query

EntryId of

Dataset

DatasetId EntryId Pair

EntryIdDatasetIdGCMD Title

OpenSearch Query

(EntryId)

Data Provider

Query

Query Result

OpenSearch ATOM Result

CEOS International Dictionary Network

CWIC

CWIC OpenSearch Client

CWIC OpenSearch Interactions


Questions?

page 1 csiss center for spatial information science and systems wgiss-37 meeting,2014 cwic...

Documents