Internal USGS Real-Time Site OperationsFrom DCP assignment to managing site metadata

STIWG Meeting Austin TexasMay 7, 2014Brian Reece

Richard Pardee

An Inside StoryWe only see each others public operations

Internal operations and managing historical operational records are critical to historical data interpretation

We have a lot to learn from each other about how we do business internally

Overview

• National Water Information System (NWIS)• Platform Assignment And Scheduling System– What is it?– Brief history 1998 - 2012– June 2012 upgrade

• Site Information Management System– What is it?– Integrated workflows– Data, metadata, history, workforce, safety

National Water Information System (NWIS)

web delivery

Archive

groundwater,

surface water,

water qualitySites

OPERATIONS

NWISWeb – What is it?• Provides access to select USGS data

– Real-time data– Site information– Current and historical

• Streamflow– Daily means– Daily, monthly, and annual statistics– Peak flows– Field Measurements including channel geometry

• Ground-water levels• Water-quality

• Does not include all data stored in NWIS

http://waterdata.usgs.gov/nwis

What is PASS

• Richard Pardee is the best!• Component of our National Water Information

System (NWIS)• Centrally manages more than 12,000 USGS

DCPID assignments in a very distributed organization

• Reporting system to NOAA DCS• Configurations are shared with other agencies

Some PASS History

• An application is only as good as the other application it depends on

• 1998 – 2011– Completely compartmentalized – Not NWIS aware, Not DCS

Aware, in fact most people weren’t even aware it existed.– Unknown systems depended on it and it was unknown what

systems it depended on• 2011 - USGS changing databases from Ingres to Oracle

– Small operational databases / systems tend to get left behind as orphans

Upgrading PASS

• Released June 2012• Integrated system documented and nurtured

interconnections– NWIS, DCS, SIMS,

• Manages configurations and notifications and is “organizationally” aware

• In the workflow

How PASS works

• User perspective / administrator perspective• Focus on the workflow, communication and logging

PASS Issues

• Manual updates by Richard to DADDS• Richard “shares” with MS Water Science Center• Need a direct webservice to this information

internally and better sharing externally• DECODES repository• Dependency on nightly DCS download and

parsing of a text file• In a nutshell, more integration

What is SIMS

• Site Information Management System• Centrally managed site operational details and

history• Metadata, documentation, and various site

operational details • Several systems depend on SIMS, SIMS

depends on several other systems• True interdependence

Some SIMS History• Initially released to support the

USGS Annual Water Data Report - ADR (2006)

• Maintained operational details to support the ADR

• 2007 - Expanded to related real-time field operational details

• 2008 to present - Expanded to organization, workforce and site documentation tool

Organizational

• NWIS is the foundation with 1.6 million sites and associated hydrologic data

• SIMS manages organizational details about modern / active sites– Water Science Center (WSC) with primary responsibility– WSC office with primary responsibility– Person / operator with primary responsibility– Field trip(s) site is a part of– How to safely visit and measure location– Detailed historical information

Let’s go live