Download - Data Management Updates
Data Management Updates
Kristen Gunthardt, EPA OWNate Booth, USGS CIDA
NWQMC February 1, 2010
Topics• Water Quality Data Exchange• Portal Update• Towards a Multidisciplinary Data Sharing FRamework
What is WQX?• WQX defines the framework by which EPA compiles water
quality monitoring data in the STORET Data Warehouse
• WQX is governed by a standardized format, so all data must comply with this format
• The WQX format allows anybody to share data regardless of what the original source of the data was
• WQX provides a common suite of data elements that we can use to share data across sources – NWIS Water Quality and STORET Warehouse data
Today’s Status• 37 State agencies have successfully flowed data via WQX or WQX
Web since 2007 (LINK)
• Over 80 Tribal organizations have successfully flowed data via WQX or WQX Web since 2007
• Other states and tribes continue to come on-line, and/or have been funded through EPA Exchange Network grant dollars to transition to WQX
• STORET Helpdesk assistance, grant funding, as well as individual consultation and training facilitate the transition to WQX
• EPA Office of Water and Office of Environmental Information continue to partner to provide tools for all data providers
1960’s 1970’s 1980’s 1990’s 2000’s
Legacy STORET
Data Integration Timeline
WATSTORE
Modern STORET
NWIS
NWISWeb
QW WebServices
STORET warehouse
Data copied
U.S. Water Data Portal Project: Integrating Water Information
• The United States Geological Survey (USGS) and the United States Environmental Protection Agency (USEPA) have enhanced water quality monitoring data access
• Common data standards and web services improve on the historical approach
• Water managers and the public will access integrated water quality monitoring data from multiple agencies through a singular data portal
What is a web service?•Computer-to-computer data sharing•Uses Input parameters and outputs XML•Can be used in multiple ways by many applications
USGS
EPA
Internet (XML)
•For more information, please visit: http://qwwebservices.usgs.gov/ and http://www.epa.gov/storet/web_services.html
9
Components of Longer Term Data Integration
Other Data Partners
Integrated water quality monitoring data is made possible through:• Web services technology• Standardized metadata• Compatible search parameters• Common vocabularies
Gulf of Maine+Sharing a Variety of Data for the
Northeast Coastal and Ocean Data Partnership
Learn more at www.necodp.org
• Further Integration– Common spatial frameworks (NHD)– Common analytical method metadata (NEMI)
• Improve the federated dataset • A single web portal for water-quality data
– Unified downloads
Future Work
USGS / USEPA Water-Quality Data ExchangeLower PotomacHydrologic Unit 02070011161 USGS, 169 EPA stream sites
USGS EPA
150M water quality observations over last 100 yrs
Towards an International Multidisciplinary Water Data Sharing Framework
• A common information model for the entire hydrologic cycle
• Data Integration Framework • Open Geospatial Consortium Hydrology
Domain Working Group• National and International partners
Credit: David J. Schwab
Helping the World to CommunicateGeographically
Hydrology Domain Working Group
• A joint working group of the OGC and WMO constituted as an OGC Domain Working Group.
• Brings together interested parties to develop and promote the technology for greatly improving the way in which water information is described and shared.
• Co-chaired by representatives nominated by the OGC TC and the World Meteorological Organisation’s (WMO) Commission for Hydrology (CHy).
• Current Co-Chairs: Ilya Zaslavsky (SDSC), Ulrich Looser (GRDC) and David Lemon (CSIRO)
• > 50 Participants, > 30 Organisations
Courtesy: David Lemon, CSIRO
Meets every 3 months
Teleconferences most weeks
WaterML Version 2 standard being proposed
Vote for adoption 3-6 months later
Jointly with World Meteorological Organization
Evolving WaterML into an International Standard
November 2009
• CUAHSI is a consortium representing 125 US universities
• Supported by the National Science Foundation Earth Science Division
• Advances hydrologic science in nation’s universities
• Includes a Hydrologic Information System project
http://www.cuahsi.org
Courtesy: David Maidment
19
What have we learned?
• We have custom-built a very large scale services-oriented architecture and a sophisticated user interface to it– A much simpler and more general pattern has emerged
based on existing Open Geospatial Consortium (OGC) information exchange standards and extensions to them
• We have exposed a very large volume of information– It needs to be carefully organized to be most useful
Courtesy: David Maidment
20
Open Geospatial Consortium Web Service Standards
• Map Services
• Web Map Service (WMS)• Web Feature Service (WFS)• Web Coverage Service
(WCS)• Catalog Services for the Web
(CS/W)
• Observation Services
• Observations and Measurements Model
• Sensor Web Enablement (SWE)
• Sensor Observation Service (SOS)
These standards have been developed over the past 10 years ….…. by 400 companies and agencies working within the OGC
Towards an International Multidisciplinary Water Data Sharing Framework
• Open Geospatial Consortium Hydrology Domain Working Group Interoperability Experiments – Academic, Government, Industry– Groundwater– Surface Water– Flood Forecasting– Met/Oceans
Groundwater Data Exchange Experiment
Test and enhance OGC standards for water observations
Exchange groundwater well characteristics and water levels with Canada
Start with Lake Superior Basin
Groundwater Data Portal
Groundwater Data Portal
Groundwater Data Portal
Surface Water Interoperability Experiment (3 use cases)
Cross border – share streamflow data across regional boundaries (Europe)
Forecasting (Deltares, USGS, NOAA)– share streamflow and rating curve for flood forecasting
Global Runoff (Kisters) – Real-time calculations of streamflow volume to oceans
Surface Water Interoperability Experiment – USGS gages
ACWI SOH, IWRSS
Courtesy: Kelli Page, GLOS
Met / Oceans
Credit: David J. Schwab
Great Lakes Restoration Initiative –GLOS backbone· Support cross-
agency and cross-discipline data analysis and modeling through a data standards-based virtual observatory
Nested watershed models and sensors
· Hierarchy of watershed water-quality models:· SPARROW (spatial variability, annual time-step)· HSPF (daily time step) – 2 ag watersheds, 1 on LM· Sensor derived surrogates (~real-time) – 30 sites / 8
on LM
· Common basin-wide hydrologic model that can test climate change scenarios
· Evolve data standards to couple to in-lake models and observations
U.S. Department of the InteriorU.S. Geological Survey
Kristen Gunthardt
USEPA Office of Water
Email: [email protected]
Thank you
Nate Booth, USGS
USGS Center for Integrated Data Analytics (CIDA)
Ph: 608-821-3822
Email: [email protected]