Data Management Updates

Kristen Gunthardt, EPA OWNate Booth, USGS CIDA

NWQMC February 1, 2010

Topics• Water Quality Data Exchange• Portal Update• Towards a Multidisciplinary Data Sharing FRamework

What is WQX?• WQX defines the framework by which EPA compiles water

quality monitoring data in the STORET Data Warehouse

• WQX is governed by a standardized format, so all data must comply with this format

• The WQX format allows anybody to share data regardless of what the original source of the data was

• WQX provides a common suite of data elements that we can use to share data across sources – NWIS Water Quality and STORET Warehouse data

Today’s Status• 37 State agencies have successfully flowed data via WQX or WQX

Web since 2007 (LINK)

• Over 80 Tribal organizations have successfully flowed data via WQX or WQX Web since 2007

• Other states and tribes continue to come on-line, and/or have been funded through EPA Exchange Network grant dollars to transition to WQX

• STORET Helpdesk assistance, grant funding, as well as individual consultation and training facilitate the transition to WQX

• EPA Office of Water and Office of Environmental Information continue to partner to provide tools for all data providers

1960’s 1970’s 1980’s 1990’s 2000’s

Legacy STORET

Data Integration Timeline

WATSTORE

Modern STORET

NWIS

NWISWeb

QW WebServices

STORET warehouse

Data copied

U.S. Water Data Portal Project: Integrating Water Information

• The United States Geological Survey (USGS) and the United States Environmental Protection Agency (USEPA) have enhanced water quality monitoring data access

• Common data standards and web services improve on the historical approach

• Water managers and the public will access integrated water quality monitoring data from multiple agencies through a singular data portal

What is a web service?•Computer-to-computer data sharing•Uses Input parameters and outputs XML•Can be used in multiple ways by many applications

USGS

EPA

Internet (XML)

•For more information, please visit: http://qwwebservices.usgs.gov/ and http://www.epa.gov/storet/web_services.html

9

Components of Longer Term Data Integration

Other Data Partners

Integrated water quality monitoring data is made possible through:• Web services technology• Standardized metadata• Compatible search parameters• Common vocabularies

Gulf of Maine+Sharing a Variety of Data for the

Northeast Coastal and Ocean Data Partnership

Learn more at www.necodp.org

• Further Integration– Common spatial frameworks (NHD)– Common analytical method metadata (NEMI)

• Improve the federated dataset • A single web portal for water-quality data

– Unified downloads

Future Work

USGS / USEPA Water-Quality Data ExchangeLower PotomacHydrologic Unit 02070011161 USGS, 169 EPA stream sites

USGS EPA

150M water quality observations over last 100 yrs

Towards an International Multidisciplinary Water Data Sharing Framework

• A common information model for the entire hydrologic cycle

• Data Integration Framework • Open Geospatial Consortium Hydrology

Domain Working Group• National and International partners

Credit: David J. Schwab

Helping the World to CommunicateGeographically

Hydrology Domain Working Group

• A joint working group of the OGC and WMO constituted as an OGC Domain Working Group.

• Brings together interested parties to develop and promote the technology for greatly improving the way in which water information is described and shared.

• Co-chaired by representatives nominated by the OGC TC and the World Meteorological Organisation’s (WMO) Commission for Hydrology (CHy).

• Current Co-Chairs: Ilya Zaslavsky (SDSC), Ulrich Looser (GRDC) and David Lemon (CSIRO)

• > 50 Participants, > 30 Organisations

Courtesy: David Lemon, CSIRO

Meets every 3 months

Teleconferences most weeks

WaterML Version 2 standard being proposed

Vote for adoption 3-6 months later

Jointly with World Meteorological Organization

Evolving WaterML into an International Standard

November 2009

• CUAHSI is a consortium representing 125 US universities

• Supported by the National Science Foundation Earth Science Division

• Advances hydrologic science in nation’s universities

• Includes a Hydrologic Information System project

http://www.cuahsi.org

Courtesy: David Maidment

19

What have we learned?

• We have custom-built a very large scale services-oriented architecture and a sophisticated user interface to it– A much simpler and more general pattern has emerged

based on existing Open Geospatial Consortium (OGC) information exchange standards and extensions to them

• We have exposed a very large volume of information– It needs to be carefully organized to be most useful

Courtesy: David Maidment

20

Open Geospatial Consortium Web Service Standards

• Map Services

• Web Map Service (WMS)• Web Feature Service (WFS)• Web Coverage Service

(WCS)• Catalog Services for the Web

(CS/W)

• Observation Services

• Observations and Measurements Model

• Sensor Web Enablement (SWE)

• Sensor Observation Service (SOS)

These standards have been developed over the past 10 years ….…. by 400 companies and agencies working within the OGC

Towards an International Multidisciplinary Water Data Sharing Framework

• Open Geospatial Consortium Hydrology Domain Working Group Interoperability Experiments – Academic, Government, Industry– Groundwater– Surface Water– Flood Forecasting– Met/Oceans

Groundwater Data Exchange Experiment

Test and enhance OGC standards for water observations

Exchange groundwater well characteristics and water levels with Canada

Start with Lake Superior Basin

Groundwater Data Portal

Groundwater Data Portal

Groundwater Data Portal

Surface Water Interoperability Experiment (3 use cases)

Cross border – share streamflow data across regional boundaries (Europe)

Forecasting (Deltares, USGS, NOAA)– share streamflow and rating curve for flood forecasting

Global Runoff (Kisters) – Real-time calculations of streamflow volume to oceans

Surface Water Interoperability Experiment – USGS gages

ACWI SOH, IWRSS

Courtesy: Kelli Page, GLOS

Met / Oceans

Credit: David J. Schwab

Great Lakes Restoration Initiative –GLOS backbone· Support cross-

agency and cross-discipline data analysis and modeling through a data standards-based virtual observatory

Nested watershed models and sensors

· Hierarchy of watershed water-quality models:· SPARROW (spatial variability, annual time-step)· HSPF (daily time step) – 2 ag watersheds, 1 on LM· Sensor derived surrogates (~real-time) – 30 sites / 8

on LM

· Common basin-wide hydrologic model that can test climate change scenarios

· Evolve data standards to couple to in-lake models and observations

U.S. Department of the InteriorU.S. Geological Survey

Kristen Gunthardt

USEPA Office of Water

Email: Gunthardt.Kristen@epamail.epa.gov

Thank you

Nate Booth, USGS

USGS Center for Integrated Data Analytics (CIDA)

Ph: 608-821-3822

Email: nlbooth@usgs.gov