a services oriented architecture for water resources data
DESCRIPTION
A Services Oriented Architecture for Water Resources Data. David R. Maidment Center for Research in Water Resources University of Texas at Austin. A Services Oriented Architecture for Water Resources Data. CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server - PowerPoint PPT PresentationTRANSCRIPT
A Services Oriented Architecture for Water Resources Data
David R. MaidmentCenter for Research in Water Resources
University of Texas at Austin
A Services Oriented Architecture for Water Resources Data
• CUAHSI and WATERS• WaterML and WaterOneFlow • Hydrologic Information Server• Modeling services
A Services Oriented Architecture for Water Resources Data
• CUAHSI and WATERS• WaterML and WaterOneFlow • Hydrologic Information Server• Modeling services
Ocean Sciences
What is CUAHSI?
• CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc
• Formed in 2001 as a legal entity
• Program office in Washington (5 staff)
• NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities
Earth Sciences
AtmosphericSciences
UCAR
CUAHSI
Unidata
HISNational Science Foundation
Geosciences Directorate
CUAHSI Member Institutions
112 US Universities as of September 2007
HISTeam
WATERSTestbed
WATERS Network Information System
NSF has funded work at 11 testbed sites, each with its own science agenda. HIS supplies the
common information system
Minnehaha Creek
SuperComputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, CZEN
GEON
Government:USGS, EPA,
NCDC
Industry:ESRI, Kisters,
MicrosoftCUAHSI HIS
HIS Team and its Cyberinfrastructure Partners
HIS Team:Texas, SDSC,Utah, Drexel,
Duke
SupercomputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC
Industry:ESRI, Kisters,
Microsoft
HISTeam
WATERSTestbed
WATERS Network Information System
CUAHSI HIS
HIS, WATERS and the CUAHSI Community
SupercomputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC, USDA
Industry:ESRI, Kisters,
OpenMI
HISTeam
WATERS Testbed
WATERS Network Information System
CUAHSI HIS
International Partners
CSIRO and Bureau of MeteorologyAustralian Water Resources Information System
European CommissionWater database design and model integration (OpenMI)
HIS Goals• Hydrologic Data Access – providing better
access to a large volume of high quality hydrologic data across the nation
• Support for Observatories – integrating local observations by academic investigators with hydrologic data for a region
• Advancement of Hydrologic Science – modeling and analysis of “hydrology in a dynamic earth”
• Hydrologic Education – bringing more data into the classroom
A Services Oriented Architecture for Water Resources Data
• CUAHSI and WATERS• WaterML and WaterOneFlow • Hydrologic Information Server• Modeling services
DefinitionThe CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using web services so that they operate as a connected whole.
Services Oriented Architecture• Service-oriented Architecture (SOA) is an
architectural design pattern that concerns itself with defining loosely-coupled relationships between producers and consumers.
• A major focus of Web services is to make functional building blocks accessible over standard Internet protocols that are independent from platforms and programming languages.
• The Web Services Description Language (WSDL, pronounced 'wiz-dəl' or spelled out, 'W-S-D-L') is an XML-based language that provides a model for describing Web services.
(from Wikipedia)
Defined by the World Wide Web Consortium (W3C)
Web Pages and Web Serviceshttp://www.safl.umn.edu/ http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx?
Uses Hypertext Markup Language (HTML)Uses WaterML
(an eXtended Markup Language for water data)
Rainfall & SnowWater quantity
and quality
Remote sensing
Water Data
Modeling Meteorology
Soil water
Water Data Web Sites
NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454
Time series of streamflow at a gaging station
USGS has committedto supporting CUAHSI’sGetValues function
Observation Stations
Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System
USGS National Water Information System NOAA Climate Reference Network
Map for the US
Water Quality Measurement Sites in EPA Storet
Substantial variation in data availability from states
Data from Bora Beran, Drexel University
Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)
Geographic Integration of Storet and TCEQ Data in HIS
Observations CatalogSpecifies what variables are measured at each site, over what time interval,
and how many observations of each variable are available
Point Observations Information ModelData Source
Network
Sites
Variables
Values{Value, Time, Qualifier, Offset}
USGS
Streamflow gages
Neuse River near Clayton, NC
Discharge, stage (Daily or instantaneous)
206 cfs, 13 August 2006• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value• An offset allows specification of measurements at various depths in water
http://www.cuahsi.org/his/webservices.html
GetSites
GetSiteInfo
GetVariables
GetVariableInfo
GetValues
Locations
Variable Codes
Date Ranges
WaterML and WaterOneFlow
GetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
Client
STORET
NAMNWIS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML
WaterOneFlow• Set of query functions • Returns data in WaterML
NWISNWIS
ArcGISArcGIS
ExcelExcel
NCARNCAR
UnidataUnidata
NASANASAStoretStoret
NCDCNCDC
AmerifluxAmeriflux
MatlabMatlabAccessAccess JavaJava
FortranFortran
Visual BasicVisual Basic
C/C++C/C++
Some operational services
CUAHSI Web ServicesCUAHSI Web Services
Data SourcesData Sources
ApplicationsApplications
Extract
Transform
Load
http://www.cuahsi.org/his/
GetValues for Dissolved Oxygenat Minnehaha Creek
http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx?WSDL
A Services Oriented Architecture for Water Resources Data
• CUAHSI and WATERS• WaterML and WaterOneFlow • Hydrologic Information Server• Modeling services
Hydrologic Information Server• Supports data discovery,
delivery and publication– Data discovery – how do I
find the data I want?• Map interface and
observations catalogs• Metadata based Search
– Data delivery – how do I acquire the data I want?
• Use web services or retrieve from local database
– Data Publication – how do I publish my observation data?
• Use Observations Data Model
Water Resource Regions and HUC’s
National Hydrography Dataset
NHDPlus for Region 17E
NHDPlus Reach Catchments ~ 3km2
About 1000 reach catchments in each 8-digit HUC
Average reach length = 2km 2.3 million reaches for continental US
Reach Attributes
• Slope• Elevation• Mean annual flow
– Corresponding velocity• Drainage area• % of upstream
drainage area in different land uses
• Stream order
Observations CatalogSpecifies what variables are measured at each site, over what time interval,
and how many observations of each variable are available
Hydrologic Information Server
Microsoft SQLServer Relational Database
Observations Data Geospatial Data
GetSites
GetSiteInfo
GetVariables
GetVariableInfo
GetValues
DASH – data access system for hydrologyWaterOneFlow services
ArcGIS Server
Minnehaha Creek Experimental Sitehttp://his.safl.umn.edu/DASH/
Water Temperature (°C)
Dissolved Oxygen (mg/L) Depth (m)
Dissolved Oxygen (% saturation)
Data Heterogeneity
• Syntactic mediation– Heterogeneity of
format– Use WaterML to get
data into the same format
• Semantic mediation– Heterogeneity of meaning– Each water data source
uses its own vocabulary– Match these up with a
common controlled vocabulary
– Make standard scientific data queries and have these automatically translated into specific queries on each data source
• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them
Objective
NWIS
NARR
NAWQANAM-12
request
request
request
request
request
requestrequest
request
request
return
return
return
return
return
returnreturn
return
return
What we are doing now …..
Michael PiaseckiDrexel University
Semantic MediatorWhat we would like to do …..
NWIS
NAWQA
NARR
generic request
GetValues
GetValues
GetValues
GetValues
GetValues
GetValuesGetValues
GetValues
GetValues HODM
Michael PiaseckiDrexel University
Hydroseekhttp://www.hydroseek.org
Supports search by location and type of data across multiple observation networks including NWIS and Storet
A Services Oriented Architecture for Water Resources Data
• CUAHSI and WATERS• WaterML and WaterOneFlow • Hydrologic Information Server• Modeling services
• Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive
• Software standards for model linking• Uses model core as an “engine”• http://www.openMI.org
OpenMI Conceptual Framework
VALUES
All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data
Space, L
Time, T
Variables, V
D
An application of the data cube to integrate simulation modelsJon Goodall, Duke University
Typical model architectureApplication
User interface + engineEngine
Simulates a process – flow in a channelAccepts inputProvides output
ModelAn engine set up to represent a particular location e.g. a reach of the Thames
Engine
Output data
Input data
Model application
Run
Write
Write
Read
User interface
Accepts Provides
Rainfall(mm)
Runoff(m3/s)
Temperature(Deg C)
Evaporation(mm)
Accepts Provides
Upstream Inflow(m3/s)
Outflow(m3/s)
Lateral inflow(m3/s)
Abstractions(m3/s)
Discharges(m3/s)
River Model
Linking modelled quantities
Rainfall Runoff Model
Data transfer at run time
Rainfall runoff
Output data
Input data
User interface
River
Output data
Input data
User interface
GetValues(..)
Models for the processes
River(InfoWorks RS)
Rainfall(database)
Sewer(Mouse)
RR(Sobek-Rainfall
-Runoff)
Data exchange3 Rainfall.GetValues
River(InfoWorks-RS)
Rainfall(database)
Sewer(Mouse)
2 RR.GetValues
7 RR.GetValues
RR(Sobek-Rainfall
-Runoff)
1 Trigger.GetValues
6 Sewer.GetValues
call
data
4
5 8
9
Coupling the HIS with Models using OpenMI
ODM
ObservationsData Model
WaterOneFlow Web Services
Water Markup Language
WOF WaterML
MODFLOW HEC-RAS Others
OpenMI
SWATHSFP“academic models”
Linking the HIS with HEC-RAS via OpenMI
HIS WaterOneFlow Web Service forNWIS Realtime
Streamflow
A HEC-RAS Model
Workflow “trigger”
Using real-time streamflow from the HIS as a boundary condition for a HEC-RAS simulation.
SOAP-based Web Service for Models
HEC-RAS
USGSNWIS
WSDL
WSDLSimulation Model
Database
OpenMI Workflow
In an effort to build cyberinfrastructure for the hydrologic sciences, we are extending OpenMI to utilize models as web services.
Extending OpenMI for Distributed Computing
Connects to remote database via web
services
Connects to remote model via web services
Goal: To allow a modeler to create a workflow from OpenMI components that wrap web services.
Model linkage designed on client machine
Conclusion
The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.
For more information: http://www.cuahsi.org/his.html