1 echo and edg status sept 12, 2006 beth weinstein, [email protected] yonsook enloe,...
TRANSCRIPT
2
Outline
What is ECHO?ECHO Background - How did we get here?Why use ECHO?Current Functionality Current Partners ParticipationUpcoming activities
3
ECHO Mission/Vision Statement
ECHO Mission• ECHO’s mission is to enable a global marketplace of Earth
Observation resources that will make Earth Observation data utilization more efficient and will spark innovation. ECHO provides Earth Observation communities with the ability to publish, discover, access and integrate directory and inventory level data and services through community-developed user interfaces.
ECHO Vision • ECHO will…
•be highly recognized, trusted and valued by the Earth Observation community
•be a critical building block in distributed information, modeling, decision support and public access systems
•have a low cost of participation to encourage broad community involvement
4
ECHO as Middleware
ECHO is middleware for a service-oriented enterpriseIts capabilities can be accessed through publicly available APIs• Based on industry standards for performing web-based
computing• Defined in the Web Services Definition Language (WSDL) and
are accessible through Simple Object Access Protocol (SOAP).
Custom clients are developed to interact with ECHO’s Earth science metadata and extended services registry• Clients written in most contemporary programming
languages are isolated from the underlying technologies • Clients may call the ECHO web services much like a local
function call.
5
ECHO Background – How did we get here?
ECHO initiated as an enhancement to EOSDIS in response to:• User feedback on complexity and limitations of the “system-wide”
view of EOSDIS data provided by EOS Data Gateway (EDG).•Belief that the community could and would develop better client
capabilities tailored to their needs.• Evolving NASA Earth science vision of multiple, distributed,
heterogeneous data and service providers.• Availability of emerging technologies (e.g. web services).
Response was development of ECHO as enabling infrastructure.• “Externalized” metadata and made it accessible via APIs that
supported development of custom clients.• Extensible architecture that allows standard client and provider
interfaces to be added.• Support for data services.
Centralized “clearinghouse” model based on industry feedback.• Driven by performance and availability requirements.
6
Why Use ECHO?
Open system provides Earth science data and services to large, diverse pool of users enabling scientific community interaction and collaboration Control in the hands of the data partner• Automate mapping between your metadata and ECHO
catalog metadata • Control visibility and access to your contributed resources• Select the best spatial search approach for your data• Check the history of orders and provide status on open
ordersUsers search for collection and inventory-level data• Search and order data through a customized user interface• Directly access online data and/or order data on media
ECHO offers high system availability• 99% system availability• Even if your system is down, ECHO users can still search
your metadata
7
ECHO Capability Today
User Registration and LoginMetadata ingest, validation, and reconciliationSearching is accurate and flexible - support for Cartesian, geodetic and orbital data description and subsequent search by these and the following mechanisms• Spatial Search (e.g. line, polygon, multipolygon, circle)• Temporal Range Searches• Keyword Searches• Numeric Searches (e.g. cloud cover percentage)• Product Specific Attribute Searches
Open interfaces (web service based) for human-machine or machine-to-machine clients (legacy interfaces are being phased out).Data Access• Direct On-line Access• Brokering of Orders• Price Quotes• Subscriptions
Interoperability with other systems (OGC/NSDI Client support)Service Catalog based on web services standards
8
ECHO Schedule
ECHO 7.0 operational•Browse Data Insert, Update, and Deletes•Multiple Collections and Groups for Access Control Rules•Spatial Query Based on Lat/Lon Point
Mar 2006
ECHO 8.0 operational (Phase out of XML APIs. Web Service APIs are primary interface mechanism.)
•Web Services API•Asynchronous Queries•Web Service brokering
Oct 2006
ECHO 9.0 operational (Removal of XML APIs. Web Service APIs are only interface mechanism.)
•Improved performance•More Comprehensive Error Handling•Enhanced Security•Metrics
Apr 2007
ECHO 10.0 operational•EOSDIS Evolution Items
4Q 2007
9
ECHO Data Partner Status
ECHO’s Current Holdings (Aug 2006) from 10 Data Partners• Collections 2,408• Granules 66 million• Browse 14 million
All NASA ECS DAACs are actively participating in ECHO (GES, LARC, LP, NSIDC)• Atmospheric Composition and Dynamics, Global Precipitation, Ocean
Biology, Ocean Dynamics, Solar Irradiance• Radiation Budget, Clouds, Aerosols and Tropospheric Chemistry• Land Processes• Snow and Ice, Cryosphere and Climate
V0 DAACs are participating (ASF, GES, JPL, ORNL, SEDAC, PO)• Synthetic Aperture Radar (SAR), Sea Ice, Polar Processes,
Geophysics• Biogeochemical Dynamics, Ecological Data, Environmental Processes• Oceanic Processes, Air-Sea Interactions• Population, Sustainability
MODIS Data Processing System (MODAPS) is operationalJAXA CEOP is in test mode
10
Data Provider Support in ECHO
FTP Push for submitting metadataDTDs that govern required xml format available on echo website for metadata submissionRequired and optional Collection level and product (granule) metadata Browse metadata and imagery storageAutomated processes for ingesting metadata files. Metadata update and reconciliation capabilitiesPartner test system to test metadata and perform validation before ingesting into operational systemECHO Ops team support for data providers
11
ECHO Client Partner Status
Variety of client types • Web-based clients• Installed applications• User interactive• machine-to-machine
Variety of purposes• General purpose (e.g. Geospatial and
temporal searching)• Customized user interfaces (e.g.
Targeted to smaller communities, Facilitates specific tasks)
• Back-end harvesting tools (e.g. Supports client-side caching of key information)
• Additional value-added processing by clients (e.g. Subset, Stitch, Resample, Reproject, Reformat)
• Middleware componentsNASA ROSES ACCESS solicitation client will be selected in Fall 2006.
12
Custom Clients and ECHO
There are many different ways to use ECHO.• Can use a subset of ECHO web service APIs to use only the desired ECHO
functionality (search, order, access, extended services...)• Can search through entire ECHO metadata or service registries or a
subset • Can search using full search criteria or limited search criteria or product
specific attributes• Can customize the search results returned to subset of data or service
metadata or list of data access urls of result set• Application-specific server can harvest ECHO metadata of interest and
cache it for value added processing by its own dedicated clientClients can use ECHO Web Service APIs to register an extended service in the ECHO Service registry and access this service through SOAPCase example: OPeNDAP Group built a limited client access to ECHO Service Registry in less than one month. This client access was used by Matlab client and can be used by other OPeNDAP enabled clients. This client invokes a search that searches for only data served by OPeNDAP servers. Client queries by space & time, and is returned set of metadata & OPeNDAP urls of result set.Level of effort to build client to query ECHO using web services is very small.
13
EDG Transition to ECHO’s WIST Client
Warehouse Inventory Search Tool (WIST) • ECHO client being developed by NASA ESDIS• General search and order interface• Will offer all EDG functionality • Public access to current ECHO operational version
WIST releases • Preoperational WIST has been deployed for public access. While it has been
tested with ECHO, it is considered preoperational because not all of the EDG functionality has been included yet.
• Next version of WIST released in November 2006 will include subsetting and ASTER On Demand support.
• Operational WIST 1.0 will be available in April 2007 when ECHO Version 9.0 goes operational.
• ECHO Partners (e.g. NASA DAACs) will continue to test and evaluate WIST as new releases are made.
EDG to WIST transition• Goal is to start the EDG-to-WIST transition plan in summer 2007 and turn off
EDG in late 2007.• ECHO must meet criteria (e.g. search performance, available, up-to-date
metadata) before EDG is turned off. • EDG and WIST will run in parallel for a period of time.
14
International Partner/ECHO Activities
CEOP (Coordinated Enhanced Observing Period) program • Plans to use ECHO and OPeNDAP enabled clients and servers• OPeNDAP enabled clients can directly query ECHO• Satellite data used by CEOP will be represented in ECHO
JAXA is currently evaluating ECHO through its CEOP activityIsrael Space Agency will become an ECHO Data Partner. Also expect to be a client partner and service partner later.IRE RAS (Russia) will become an ECHO data partner, client partner, and service partner.ESA is implementing a prototype to access ECHO from INFEO through ECHO’s web services APIs.Dundee (Scotland) considering becoming an ECHO data partner and client partner.Continuing discussions with other international partners. Future discussions with GEO partners also expected.We welcome additional partners!
15
OPeNDAP/ECHO Activities
CEOP program • CEOP will ingest metadata repesenting data of interest (from
JAXA, NASA, ESA, and Eumetsat) into ECHO. • WTF-CEOP developing extensions to OPeNDAP based tools to
provide access to satellite data to the CEOP science community.
• CEOP Satellite Data Server will provide access to satellite data served through a WCS to OPeNDAP enabled clients.
OPeNDAP prototype• Direct search and access of ECHO through the web service
APIs by OPeNDAP clients• Matlab client was demoed in July 2006 with operational
capability expected when ECHO 8.0 is operational. • This capability was demoed at the WTF-CEOP session
yesterday.
16
ECHO & GEO Tasks
ECHO can be a WGISS contribution to GEO – initial task list
Task DA 06-06: Spatial Data InfrastructuresAdvocate use of existing Spatial Data Infrastructure components as
institutional and technical precedents, where appropriate, including standard protocols and interoperable system interfaces, among other components.Task AR 06-05: GEO Clearinghouse
Initiate development of a publicly accessible, network-distributed clearinghouse, subject to GEOSS interoperability specifications to date, and including an inventory of existing data, metadata, and pre-defined common products.Task WA 06-05 In Situ Water Resource Monitoring
Initiate the creation of a coordination mechanism within GEO for global in-situ water observations, including ocean observations, and advocate synergy and sharing of infrastructure among observing systems.Task WA 06-07 Capacity Building Program in Latin America Task WA 07-P2 Satellite Water Measurements
17
IDN (GCMD)/ECHO Activities
GCMD and ECHO are working together to share information from its registries and give users a more unified experience when interacting with the two systemsGCMD Portal to ECHO data operational in 2Q 2007
18
ECHO Contact Information
ECHO International contact– Yonsook Enloe [email protected] ECHO Operations (Ops)• [email protected]
Visit the ECHO Project Website• http://eos.nasa.gov/echo• Holdings Summary, Upcoming Functionality, APIs and DTDs• Reference materials and tools• Real-time systems status monitor• Operations metrics updated weekly
Join ECHO Mailing List: [email protected] Schedule bi-lateral telecons to discuss potential collaboration!