cuahsi wateroneflow web services
DESCRIPTION
CUAHSI WaterOneFlow Web Services. By Tim Whiteaker CE 394K.2 Hydrology 1 February 2007. Outline. What are Web Services What is WaterOneFlow What is WaterML How do we use HIS Analyst to access WaterOneFlow. 16. 4. Math library. Program. Programs Use Functions. - PowerPoint PPT PresentationTRANSCRIPT
CUAHSI WaterOneFlowWeb Services
By Tim WhiteakerCE 394K.2 Hydrology
1 February 2007
Outline
• What are Web Services• What is WaterOneFlow• What is WaterML• How do we use HIS Analyst to access
WaterOneFlow
Programs Use Functions• Functions are pieces of code that perform
a specific task
Program
Mathlibrary
AddMultiplySquareRoot
Functions:
What’s the square root
of 16?
16
4
result = Math.SquareRoot(16)
Web Services Let Computers Share Functions
Program
Mathlibrary
AddMultiplySquareRoot
Functions:What’s the square root of 16
The answer is 4
• Local programs take advantage of remote resources
• Works just like local functionresult = Service.SquareRoot(16)
ServiceClient
How Do Web Services Work
• SOAP (Simple Object Access Protocol) is a protocol for exchanging messages over a network
• WSDL (Web Services Description Language) is a language for describing what a web service can do
How Do Web Services Work
Client
Server
SOAP WSDL(This is what I
can do)
SOAP is like speaking the same languageWSDL is like a contract
CUAHSI Web Services (WaterOneFLow)
• Standard mechanism for flow of hydrologic data between hydrologic data servers (databases) and users.
• Provides data access to– USGS NWIS– EPA STORET– NCEP North American Model (NAM)– NASA MODIS– Daymet– NWS ASOS
http://water.sdsc.edu/waterOneFlow/
Water Data Web Sites
NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454
Time series of streamflow at a gaging station
Observation Stations
Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System
USGS National Water Information System NOAA Climate Reference Network
Map for the US
WaterOneFlow• Consistent Query Format • Consistent Data Format
WaterOneFlow Methods
• (Almost) the same for all networks• Methods
– GetSiteInfo – Give me info about a site– GetVariableInfo – Give me info about a
variable– GetValues – Give me a time series for a
variable at a site
GetSiteInfo
• Input– Network name and
site code– Authorization token
• Output– Site name– Site location– Variables measured at
the site• Variable code• Period of record
response = service.GetSiteInfo("NWIS:08158000", "")
where
GetVariableInfo
• Input– Network name and
variable code– Authorization token
• Output– Variable name– Variable description– Variable units
response = service.GetVariableInfo("NWIS:00060", "")
what
GetValues
• Input– Network name and
location (e.g., site code)
– Network name and variable code
– Start datetime– End datetime– Authorization token
• Output– Site info– Variable info– Time series
response = service.GetValues("NWIS:08158000", "NWIS:00060", _ "2006-12-31T01:00:00", "2006-12-31T08:00:00", "")
where what
when
The “Where” in GetValues
• Works for sites, or geographic locations• NetworkName:SiteCode
– NWIS:08158000
• GEOM:POINT(Longitude Latitude)– GEOM:POINT(-113 35)
• GEOM:BOX(WLon SLat,ELon NLat)– GEOM:BOX(-108 45,-107 46)
Locations
Variable Codes
Date Ranges
Extract – Transform – Load
GetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
Client
STORET
NAMNWIS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML – The Output of WaterOneFlow
• Standardized for all data sources• Formats
– Object– XML (for programs that can’t handle objects)
• Defined in XML
XML – A Primer
• The Extensible Markup Language (XML) is a W3C-recommended general-purpose markup language that supports a wide variety of applications. – Wikipedia
• XML represents data• XML is both human and machine readable
<site> <name>Mansfield Dam</name></site>
XML Structure
<site></site>
<site></site>
<site />
OR
OR
element
start tag end tag
• Example – Let’s describe a streamflow site
element name = “site”
Elements can have Children
<site> <name>Mansfield Dam</name></site>
element
child element
element text
Elements can have Attributes
<site> <name>Mansfield Dam</name> <siteCode network=“NWIS”>08154510</siteCode></site>
attribute name attribute value
XML Nesting
<site> <name>Mansfield Dam</name> <siteCode network=“NWIS”>08154510</siteCode> <location> <latitude>30.39</latitude> <longitude>97.91</longitude> </location></site>
WaterML
• XML for describing water data• Defined at
• WaterOneFlow returns data in WaterML format
http://water.sdsc.edu/waterOneFlow/documentation/schema/
Point Observations Information ModelData Source
Network
{Value, Time, Qualifier}
USGS
Streamflow gages
Neuse River near Clayton, NC
Discharge, stage (Daily or instantaneous)
206 cfs, 13 August 2006
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value
Sites
Variables
Values
http://www.cuahsi.org/his/webservices.html
WaterML Response
SiteInfoResponseType
VariablesResponseType
TimeSeriesResponseType
WaterML Key Elements
• Response Types
– SiteInfo
– Variables
– TimeSeries
• Key Elements– site– sourceInfo– seriesCatalog– variable– timeSeries
• values
– queryInfo
GetValues
GetVariableInfo
GetSiteInfo
SiteInfoResponseType
• Namespaces• queryInfo• site
Network
SitesVariables
queryInfo
• Parameters sent to service• URLs called (if external resource)
queryInfo – Another Example
userparameters
query URL
site
• siteInfo – info about the site• seriesCatalog – catalogs of variables
measured
siteInfo
• Name• Site Code• Location
geoLocation
• geogLocation – geographic coordinates– LatLon point– LatLon box
• localSiteXY – projected coorindates
European Petroleum Survey Groupi.e., NAD 83, etc.
seriesCatalog
name web service location
original data sourceseries (variables)
series
• variable – what is measured• valueCount – how many measurements• variableTimeInterval – when is it measured
variable
• variableCode – global identifier• variableName• units
Sites
VariablesValues
variableTimeInterval
• TimeIntervalType – date range• TimeSingleType – single measurement• TimePeriodRealTimeType – last n days
ISO Time
• International Organization for Standardization• 1998-03-01T14:30:00 =
March 1, 1998, at 2:30 in the afternoon
SiteInfo Response, RecapqueryInfo
site
name
code
location
seriesCatalog
variables
what
how many
when
VariablesResponseType
• variable – same as in series element• Code, name, units Sites
VariablesValues
Additional variable Children
• variableDescription• valueType (enumeration)
– Field Observation– Sample– Model Simulation Result– Derived Value
• generalCategory – e.g., “water quality”• sampleMedium – e.g., “water column”
-- Many are optional
TimeSeriesResponseType
• queryInfo• timeSeries
– sourceInfo – “where”– variable – “what”– values
Sites
Variables
Values
sourceInfo• SiteInfoType
– Same as siteInfo element– code, name, location
• DataSetInfoType– For data continuous in space– LatLonPointType– LatLonBoxType
variable
• Same as previous variable element• code, name, units, etc.
values
• Each time series value recorded in value element
• Timestamp, plus metadata for the value, recorded in element’s attributes
ISO Time
valuequalifier
value Metadata Examples
• qualifiers• censorCode (lt, gt, nc)• qualityControlLevel (Raw, QC’d, etc.)• methodID• offset
– offsetValue– offsetUnitsAbbreviation– offsetDescription– offsetUnitsCode
TimeSeries Response, RecapqueryInfo
location
variable
values
Conclusions about WaterML
• Consistent Format• Includes lots of information• How do I use it?
HIS Analyst
HIS Analyst
• Desktop apps that use WaterOneFlow• Examples for various application
environments• HydroObjects – for apps that aren’t web
service enabled• Workbook available at:
http://www.cuahsi.org/his/docs/HIS-workbook-20061130.pdf
HIS Workbook
• Ingesting NWIS Data into Excel• Ingesting STORET Data into Excel• Ingesting Weather and Streamflow Data
into ArcGIS• Plotting MODIS Data with Matlab• Ingesting NWIS Data using VB.Net• Ingesting NWIS Data Using Java
Excel Demo
Weather Downloader
Historical Weather Data
Forecasted Weather Data
Streamflow
Other
Recap
• Computers share functionality through Web Services
HIS Analyst
Client WaterOneFlowWeb Service
GetSiteInfoGetVariableInfoGetValues
WaterMLTime SeriesRepository