U.S. Department of the InteriorU.S. Geological Survey

Tutorials on Data Management

Lesson 5: Preserve

CC

imag

e b

y N

AS

A G

odd

ard

Sp

ace

Flig

ht

Cen

ter

on F

lickr

Preserve

Provided by DataONE

Lesson Topics

Maintain Data Records Requirements Retention Schedules Evaluation of Data Archiving Data Management Activities

Preserve

Provided by DataONE

Learning Objectives

After completing this lesson, the participant will be able to: Know the issues of how to maintain data Determine Records Requirements and Retention

Schedules Evaluate Data and

Data Quality Be aware of current Data

Management activities

CC

imag

e b

y E

PI2

oh

o

n F

lickr

Preserve

Provided by DataONE

The Data Lifecycle

Preserve

Provided by DataONE

Maintain Data

Determine records requirements Establish retention

schedule Publish availability Update/modify data

Provided by Tom Chatfield, BLM

CC

imag

e b

y S

ea

ttle

Mun

icip

al A

rch

ives

on

F

lickr

Preserve

Provided by DataONE

Determine Records Requirements

Ensure compliance with administrative & statutory recordkeeping requirements

Support with Official Agency Records Designation Documentation (OARDD) BLM Manual Handbook 1270-1

Determine records access category Protect data from unauthorized use

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Establish Retention Schedule

Review the General Records Schedule to ensure a record schedule has been identified. If not, work with State Records Manager to develop and approve new schedule.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Publish Availability

Submit documentation to state or federal repositories. If data was meant for public access may need publication to Federal Register, internet, etc.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Update/Modify Data

Data sets are periodically modified as new or updated information becomes available through additional field work, continued research, etc.

Update metadata to record improvement and inform known users

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Evaluate Data

It is advisable to periodically evaluate whether data remain useful, and to use that evaluation to identify any improvements or course corrections in management procedures that would make the data more useful. How much is the data used, and have the programs supplemented it with additional data sets (or stretched data definitions to fit their needs)? Does continued use, without fixes, increase risk to a agency program?

Evaluation of such data issues can be scheduled into parallel tasks to "evaluate" business processes and supporting software applications. Evaluate the data to ensure the current business needs/requirements are being met. Evaluate quality control procedures, acceptable quality levels, and quality control results. Modifications may be required for the data being collected, quality control procedures or acceptable quality levels. Plan and execute data quality assessments per the schedule recorded in the metadata file for this data. Solicit problems from users, and include them in your quality assessment. Update the metadata file to include quality assessment results. Inform known users of this data of quality assessment results. Inform applications developers of recurring quality problems that might be addressed through software changes. Fulfill other reporting requirements that may have been placed on you, such as reporting under the Information Quality Act.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Evaluate Your Data

What’s the ‘magic number’? Are business needs still being met? Are the quality levels still within the acceptable

ranges? Are users periodically contacted regarding

possible problems? Are scheduled quality assessments recorded as

part of the metadata?

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

What is Data Quality?“fitness for intended use” Conformity to standards Completeness Currency Consistency Accuracy Relevance Complaints or Other Known/ Suspected

Problems

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

What is Data Quality?

How to Evaluate the Quality of BLM Data (Nov 2002) Conformity- What the data represents, how it was collected, when was it

collected, etc. Completeness- Results cannot be trusted using incomplete data because

you cannot be sure what the missing values have revealed. Currency- Very much a matter of judgment by SMEs. Is data recent enough

to support decision? Consistency- Are the intended operations on certain data items actually

happening the way to assume? Accuracy- Can you determine if a stored value is right? Must be repeatable Relevance- Is data to broad or narrow for our intended purposes? Additional reference- DOI OCIO Data Quality Management Guidelines

August 2008

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Archive Data

Authority Disposal of DATA and Paper Records are

regulated by the National Archives and Records Administration (NARA)

NARA Regulations are published in the 36 CFR Chapter XII

Electronic Records Management is published under PART 1234 of the 36 CFR Chapter XII.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Electronic Records vs. Data

Electronic Records include numeric, graphic, and text information, which may be recorded on any medium capable of being read by a computer and which satisfies the definition of a record. This includes: Data base Data base management system Data file Electronic information system Electronic mail system Electronic recordkeeping system

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Significant?

Data are maintained in support of significant ongoing projects in support of the basic mission of the USGS.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

20/53 – Record CopyThe “record” copy of a report which is used for making a specific decision, such as a land use plan, an environmental impact assessment, or land use adjudication, is filed as a hard copy with the related project or case file.

20/52 – Record CopyThe “record copy" of the output products of a layer or combination of themes which is used for analysis or for making a specific decision, such as a land use plan, an environmental impact assessment, or land use adjudication, is filed as hard copy with the related case file.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

When? Cutoff EOFY in which the data are created or

significantly altered by the BLM.

Where? Transfer copy of data to NARA at the EOFY

and transfer in accordance with 36 CFR 1228.188, along with appropriate documentation (see items 52e & 53e).

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Contact a Records Specialist

NEVER dispose of

records and data

without contacting

your local or state

Records Experts

Provided by Tom Chatfield, BLM

CC

imag

e b

y S

ea

ttle

Mun

icip

al A

rch

ives

on

F

lickr

Preserve

Provided by DataONE

Who is Responsible?

Project Manager/Lead in consultation with: Data Steward Records Administrator GIS Specialist

Physical archive may be carried out by the local systems administrator or GIS coordinator

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Archive or Backup?

Archive and Backup are both important steps in protecting your data, but they are not the same process.

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Simple Data Archiving Process Flow Chart FLOW CHART THAT’S USGS?

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

National Activities Affecting DataManagement Federal Level Activities DOI Level Activities USGS Activities

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Federal Level Activities

Geospatial Platform Enterprise Architecture

Geospatial Line of Business OMB Circular A-16 Supplemental Guidance Data Life Cycle

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

What is an Enterprise Architecture?

Enterprise Architecture improves the Business by developing and providing business-driven solutions for improved business processes, data, applications, and technology optimization

What is an Enterprise Architecture? Models the way the organization currently works Provides decision-makers an understanding of the

entire organization Helps identify ways to cut costs and increase

effectiveness

Provided by Tom Chatfield, BLM

NEVER COMPLETE !

Preserve

Provided by DataONE

Geospatial Line of Business(GeoLoB)The Federal Enterprise Architecture (FEA) effort has several ongoing activities which cross all Federal Agencies.

The FEA GeoLoB is one of those efforts.

GeoLoB Current workgroups include: Lifecycle Management, Performance Management,

Grants & Contracts, Technical Architecture, Common Services, and Geoenabled Business.

Support to this group for BLM comes through EGIM

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

A-16 Supplemental Guidance

Designed to provide processes to clarify the ‘general guidance’ of OMB Circular A-16

Contains Four Sections: Roles and Responsibilities Geospatial Lexicon Data Life Cycle Theme Principles (What is a ‘theme”?)

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Federal Data Life Cycle Effort

Developed to manage data itself as a capital asset

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Federal Geographic Data Committee(FGDC)Executive Order 12906 requires Federal Agencies to use FGDC Standards. If no FGDC standard exists, we must create one and then propose it at the FGDC level. EXAMPLES: Cadastral Data Content Standard National Hydrography Data (NHD) Watershed Boundary Dataset (WBD)

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

DOI EA Activities

Data Committees Data Quality Management Guide/Addendum Authoritative Data Source Guidance/Cookbook

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

DOI Level Data Committees

Data Advisory Committee (DAC) Architecture committee with a dataset/data management

specific focus. Policy, guidance, and training are developed with input from

the bureaus.

Enterprise Geospatial Information Management (EGIM) Architecture committee with a geospatial dataset/data

management specific focus. Produced the Geospatial Blueprint using the Methodology

for Business Transformation (MBT). Activities have recently shifted to implementation of the

Blueprint.Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Data Quality Management Guide andAddendum DOI has issued a Data Quality Management

Guidelines Document The EGIM and DAC are currently working on a

Geospatially focused Addendum to the Guidelines

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

Authoritative Data Source andCookbook ADS- the acronym is also sometimes used to

mean Authoritative Data System. Definition- One single source or system for a

data set. Roads, Hydrography, Land Ownership What is it about?

Improving data re-use, exchange, and consolidation Improving discovery, understanding, and data access Improving information quality

Provided by Tom Chatfield, BLM

Preserve

Provided by DataONE

USGS National Level Activities

????

Preserve

Provided by DataONE

Determine your records requirement Establish retention schedules There are ways of evaluating data and data quality Know the difference between archiving and backup There are a number of data management activities

at the Federal, DOI, and USGS level

Summary

Preserve

Provided by DataONE

1. Chatfield, T., Selbach, R. February, 2011. Data Management for Data Stewards. Data Management Training Workshop. Bureau of Land Management (BLM).

Resources