U.S. Department of the InteriorU.S. Geological Survey

FY2010 Data Integration:Services, Capacity, Framework.

September 28, 2009

1

A Request for Funds

$500K in funding from FY2010 Bureau Shared Costs to complete short-term, targeted efforts that benefit USGS Science Data Integration.

2

Funding Request Outcomes

Deliver immediate benefit to solve an existing data integration challenge.

Create an infrastructure that can be leveraged.

Demonstrate a methodology and/or solutions architecture that can be repeated/replicated for other data.

Create an environment that allows future innovative applications to access USGS data.

3

Small Investment, Big Pay-OffOne Use Case:

~4,000 USGS scientists are licensed for ArcGIS If ½ of those scientists save one pay period a

year by reducing their manual efforts manipulating data

~$8M of scientists’ time can be refocused on our science mission (based on $100K annual salary)

4

From a Scientist’s Perspective:

Data Integration is the ability to: Discover the existence of data relevant to a study. Access and evaluate that data. Download and use data in an analytical

framework. Make the results available for others use.

These abilities are required by our scientists to do their job.

Enabling these abilities requires: Services (Mostly Computer Services & Tools) Capacity (Skills and Training) Consistent framework (Solutions Architecture)

5

Council for Data Integration: On Services, Framework, Capacity

Build Web services and tools to discover, expose, and analyze data.

Build a framework to foster data management by providing solutions architecture, standards, data stewardship, data hosting, and publishing.

Build skills and capacity through training, education, workforce development, and cultural change.

6

The Council Approach:

Focus on targeted efforts that yield short-term benefits to the USGS science mission

Leverage existing capabilities within USGS Leverage all USGS assets from bottom to top

Be sustainable – By incorporating into existing efforts By supporting a longer-term strategy

7

Build upon existing efforts of science centers with targeted, tangible products

Empower scientists with data and tools

Seek substantial return on investment

Augment existing fundamental building blocks for data integration

8

The Council Approach:

The Council for Data Integration Responsibilities:

1. Lead the development and implementation of the USGS data integration strategy

2. Provide recommendations for implementation of data integration guidelines

3. Promote Bureau-wide data integration

Membership: AD for Geospatial Information, Chief Scientists, key scientists and managers

Community of Practice Teams: Address technical issues

9

Council Members

Betty AdrianSteve Aulenbach (NEON)Lee Allison (AZ)Leslie ArmstrongDeb BakerSky BristolFrank D’ErchiaTerry D’ErchiaJacque FahsholtzSteve Fancy (NPS)Robert FisherDave GovoniGreg GuntherLauren HayDonn HolmesHarry HouseRichard Huffine

Kate KaseTim KernKeith KirkKaren KlimaLinda LeakeFran LightsomBob MatthiasScott McEwenMike McHaleMike MulliganPete MurdochJC NelsonCallie OblingerBob PierceBrian ReeceAlyssa RosemartinCarma San Juan

Peter SchweitzerSharon ShinJenifer SieverlingChris SkinnerMark SoggeDave SollerPat Stingley (DOI)Bob SwansonJune ThormodsgardRoland VigerBrian Wee (NEON)Mingzhen WeiJake WeltzinSandy Williamson

Kevin Gallagher (Co-Chair)

Linda Gundersen (Co-Chair)

Gladys CotterDave FerdererLen GaydosMarty Goldhaber

Earl GreeneKevin HopeBruce JonesCheryl MorrisStan PonceRoger Sayre

Richard SignellSteve Tessler

Community of Practice

Council for Data Integration Members

10

CDI Accomplishments in 5 months

Chartered Council 5 Meetings (including 1 face-to-face) Teams

Program Planning Demonstration Project Metadata Data Publishing (initial focus on data.gov)

Wiki and document repository High value opportunities Draft 5-Year Program Plan

11

CDI – Wiki and Document Repository

Council for Data Integration

Community met in Denver, September 15-17, 2009

Meeting Goals• Build community by sharing stories about data integration efforts• Identify high value data integration opportunities

Agenda Highlights• Climate Change: CEN, NCCWRC• Ecosystems: GAM, WLCI, WERC, NPN• Powell Center• Data Blast: 20 posters and live demos• Partners: AZ State Geologist, NPS, DOI, NEON

Outcome• Proposed six high value opportunity areas

Council Face-To-Face

13

Over 25 High Value Opportunities Defined in 6 Overarching Themes

WebServices

Tools for Using Data

GeospatialAvailability

Of CorporateData Assets

ArchitectureFor Data

Integration

MetadataRegistry& Data

Publishing

CulturalChanges and

WorkforcePlanning

… that will leverage resources to augment and extend existing science center data integration efforts to all USGS scientists

14

FY2010 Funding Request Details:

Make demonstrable progress towards implementing three high value areas: Exposing “Corporate” Data Organizing Science Models and Outputs Preserving and Accessing Project Data while

Training Scientists

15

USGS Corporate Data

User Story As a scientist, I need a way to find, get and use

USGS corporate data to explore new scientific discoveries and conclusions. I need a way to make sure I am getting the best available data and to access them in a way that I do not have to spend time transforming them into a format that I can use.

Challenge USGS scientists (and many more external

customers) manually download, reformat, and load data into ArcGIS and TNM.

16

Expose USGS Corporate Data - (Services and Framework)

Approach Develop web services to directly access USGS

“corporate” data with ArcMap and TNM.

Benefit & Impact Scientists save valuable time. USGS “corporate” data is readily available via

standard web services. A framework/methodology for implementing web

services is established which can be replicated.

17

Science Modeling Data

User Story As a scientific modeler, I want a way to quickly access major

modeling inputs (e.g., downscaled GCMs) in a form that will drive my modeling application. I also want a way to store, organize, and share the model outputs so that other scientists can access and use them and I can have a permanent online reference to these important artifacts.

Challenge: USGS scientists need to share and publish project

data and modeling outputs that do not currently have an established outlet.

18

Approach Leverage current hosting capabilities to serve as a

repository for standardized modeling inputs and outputs. Use standard data cataloging techniques to document and expose best available data for discovery and access.

Benefit & Impact Scientists’ time is saved by reducing data preparation time for

established modeling/analysis workflows. Model outputs are referenced, easily discovered, and

accessed online. Solutions architecture is portable to other efforts.

19

Modeling Hosting & Access Portal – (Services and Framework)

Data Management & Upload Tools

User Story As a USGS scientist, I have data that I have created or

enhanced from other data, and I need a way to post this data online, document it so that others can understand it, share it with colleagues before it is published, and publish it electronically as a permanent data resource. I want to be able to integrate my project data with USGS corporate data and with other project data so that I can work with a broad range of data assets using the tools with which I am familiar.

Challenge

DOI and USGS need to compile, organize and manage data assets. Scientists and technology specialists need training and support to implement these tools.

20

Approach

Create a database upload and registry tool for non-corporate databases. Provide training and support (e.g. swat team) to educate and assist.

Benefit & Impact Project data that does not fit into a major USGS system will

not be lost. Project data that might not otherwise be accessible will be

available to all scientists. Scientists will receive valuable data management and tools

training.

21

Data Management & Upload Tools – (Services and Capacity)

Accountability: If funded the Council will - Focus on clear deliverables and develop

methodologies that can be replicated.

Track and report on progress.

Create a single account to track expenditures.

Redirect GIO resources to augment.

Leverage existing efforts in science centers.

22

Funding Request Outcomes

Deliver immediate benefit to solve an existing data integration challenge.

Create an infrastructure that can be leveraged.

Demonstrate a methodology and/or solutions architecture that can be repeated/replicated for other data.

Create an environment that allows future innovative applications to access USGS data.

23

A shared passion for …data integration

24