A Proposed Short Courseon Data Stewardship

Scott HausmanDeputy Director

NOAA’s National Climatic Data Center

Preparing Scientists to Steward Their Data

Introduction

• Background– NESDIS Workforce Development– NCDC-ESIP-CICS Collaboration

• Purpose– Forum on Short Course Concept

• Overview– Challenge: NOAA Perspective– Proposal: Short Course– Discussion: Your Thoughts– Conclusion: Way Ahead

21/6/102010 ESIP Federation Winter Meeting

NESDIS

Challenge: NOAA Perspective

31/6/102010 ESIP Federation Winter Meeting

To accomplish it’s mission and vision, NOAA needs data!

• VISION: An informed society that uses a comprehensive understanding of the role of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions

• MISSION: To understand and predict changes in Earth’s environment and conserve and manage coastal and marine resources to meet our Nation’s economic, social, and environmental needs

• CORE VALUES: People, Integrity, Excellence, Teamwork, and Ingenuity Science, Service, and Stewardship

Challenge: NOAA Perspective

• Environmental Data Management at Environmental Data Management at NOAA, NRC Report, 2007NOAA, NRC Report, 2007

• Data Access & Archive Requirements Data Access & Archive Requirements Working Group (DAARWG)Working Group (DAARWG)

• Environmental Data Management Environmental Data Management Committee (EDMC) EstablishedCommittee (EDMC) Established

• Global Earth Observation – Integrated Global Earth Observation – Integrated Data Environment Data Environment (GEO-IDE)(GEO-IDE)

• Comprehensive Large Array-data Comprehensive Large Array-data Stewardship System (CLASS)Stewardship System (CLASS)

Data Stewardship is a Top Priority

41/6/102010 ESIP Federation Winter Meeting

Definition of Data Stewardship: “Maintaining the science integrity and long term utility of scientific records.” John Jensen, NOAA/NESDIS/NCDC

Data Management

Challenge: NOAA Perspective

• NPOESS/GOES-RNPOESS/GOES-R• National MesonetNational Mesonet• Network of NetworksNetwork of Networks

– Private SectorPrivate Sector– Local/State MesonetsLocal/State Mesonets– USIEOS/IOOSUSIEOS/IOOS– GEOSSGEOSS

• Phased Array RadarPhased Array Radar• Next Generation of Next Generation of

SatellitesSatellites

Explosion of Data SourcesExplosion of Data Sources

51/6/102010 ESIP Federation Winter Meeting

Challenge: NOAA Perspective

61/6/102010 ESIP Federation Winter Meeting

Projected Growth in Data VolumeProjected Growth in Data Volume

(reanalysis)

Challenge: NOAA PerspectiveRescue of Physical RecordsRescue of Physical Records

71/6/102010 ESIP Federation Winter Meeting

Over 10 terabytes of climate data now digitized‣ 54 million weather and environmental images online‣ Hundreds of millions of records digitized now online ‣ International data access and rescue activities‣ 84 current NOAA climate/environmental rescue projects

July 1st, 1842 hourly weather data from Washington, DC, imaged and digitized

through the CDMP Program

Imaged Records Example:Glacial Pairs – Muir Glacier, Alaska

19411941 20042004

Challenge: NOAA PerspectiveWorkforce LimitationsWorkforce Limitations

81/6/102010 ESIP Federation Winter Meeting

Data Stewardship

Many of our data managers are geophysical scientists or IT professionals.

Proposal: Short Course

• Short Course on Data Stewardship• When: Summer 2011, 1-2 Weeks• Where: Asheville, NC (NCDC)• Potential Curriculum

– Stewardship Responsibilities– Metadata Concepts– Format Standards– OASIS Reference Model– Applications and Tools– Archives– Lab Exercises

91/6/102010 ESIP Federation Winter Meeting

Proposal: Short CourseContinuum of Data Stewardship ResponsibilitiesContinuum of Data Stewardship Responsibilities

101/6/102010 ESIP Federation Winter Meeting

• CollectionCollection– Observing System DesignObserving System Design– Instrument HistoryInstrument History

• Data ProcessingData Processing– Software Version ControlSoftware Version Control– Format StandardsFormat Standards

• UsersUsers– Reporting ErrorsReporting Errors– Linking Results to Source DataLinking Results to Source Data

Proposal: Short CourseStewardship as the Art & Science of Data ManagementStewardship as the Art & Science of Data Management

111/6/102010 ESIP Federation Winter Meeting

ARTStewarding

Ethics

SCIENCEData AdminStandards

Proposal: Short Course

• Theory– Lifecycle Management– OAIS RM– Ethics

• Interoperability– File Format Standards– Naming Conventions– Metadata Standards

• Config Management

• Preservation– Provenance– Context– Fixity– Reference– Retention

• Security and Integrity• Access and Discovery• Traceability

121/6/102010 ESIP Federation Winter Meeting

Basic ConceptsBasic Concepts

Discussion Forum

– Should there be an area of focus?– What topics should be included?– What materials already exists?– Who should teach?– Who should attend?– When during the year should the course be held?– How long should the course be?

Questions to Consider

131/6/102010 ESIP Federation Winter Meeting

Conclusion: Way AheadTimeline

141/6/102010 ESIP Federation Winter Meeting

February 2010 Establish

Working Group

June 2010 Define Course

July 2010 Send Invitations to

Prospective Instructors

October 2010 Publicize Event

February 2011 Registration Deadline

April 2011 Course

Materials Due

June 2011 Short Course

Scott HausmanDeputy DirectorNOAA’s National Climatic Data Center151 Patton Avenue, Room 557Asheville, NC 28801-5002

828.271.4848Scott.Hausman@noaa.gov

http://www.ncdc.noaa.gov

151/6/102010 ESIP Federation Winter Meeting

Thank you for sharing your insights!Thank you for sharing your insights!