data dictionary - ncrst-sepp home page
TRANSCRIPT
DATA DICTIONARY
UNDERSTANDING & STRUCTURING
AVAILABLE GEODATA
Memphis‐in‐May NCRST‐SEPP WorkshopMay 7th, 2009
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Duplication of data collection
• Lack of effective data sharing
• Incoherent terminology across data collections
• Incoherent information management practices
• Poor and incoherent utilization of data collected
Statement of Problem ?
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
The data dictionary provides geographicinformation system (GIS) data filedescriptions and metadata, andresource information for eachenvironmental assessment area in auser-friendly format.
Data dictionary (or system catalog) isa database about the data.
A tool for recording, coordinating andprocessing information about the datathat an organization uses.
A central catalog for metadata.
Data Dictionary:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Increase the best use of Available Data
• Influence other organizations thebest usage within available resources
• Strengthen the Multi Criteria Decisions
• Each partner / user maintains its own data store fully documented with their outputs(with standard metadata)
Data Competencies
Evaluation of Data Quality For Planning Purposes
Decision Making Strategies
Data Information
Data Management &
Leadership
PROJECT PERSONNEL
Modified from : http://www.citymatch.org/data_index.php
Goals:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Gathering and understanding availablegeodata is not simple. The process islengthy, requires communication, earlydata exchanges, and people skilled atsorting out complex data.
With the SEPP, the geodata useful fortransportation corridor planning is beingcatalogued and organized accordingsource, category, and applicability.
As result, the Data Dictionary contains notonly a metadata, but all necessaryinformation to rapidly familiarize the userswith the data available (date, format,storage, software required, contact person,projects associate with, etc)
Data Dictionary:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Data Dictionary: Cycle
“It’s a continuous process”
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Hypothesis: Earlier integration of local data is ideal.Local plans and issues may be reflected in results andpossible opposition may be avoided.
Challenge: Integrating “best available data” fromFederal, State and local “spheres” is the biggestchallenge. Organizing the data and developing a“multi‐scale” data dictionary is a must!
Federal data Moderate to low detail data. Very welldocumented, distriduted nationally, widely used.
State data Moderate to highly detailed. Widelyused with decent metadata. Reuse of value‐addedversions of federal data is common.
Local data Highly detailed data. Produced forinternal use as needed. Not typically distributed sonormally does not incorporate proper metadatadocumentation.
Federal Data
State Data
Local Data
Data Dictionary:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Project Needs
Data Access
• One important application of the data dictionary is to provide access to a glossary of the scientific terms that exist in a data collection
• It allows data managers to identify and address data problems prior to adding the update to the archive
Applications:
Aggregation & Consolidation
Process
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Data Source Ease of Availability
Documents
Tabular Data
Vector Data
Raster Data
Documents
Tabular Data
Vector Data
Raster Data
Data Flow:
Adapted from : http://www.premier‐international.com/Solutions_Data_Migration_Solutions.aspx
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Unfortunately, most organizations don't know much about the data pool and itsreuse until much effort has been wasted, and the application implementationtimeline is in jeopardy
• Leverage Work from Prior Projects
• The Qualitative Problem Sometimes using free data online could
cause delays in the application implementations
• The Quantitative Approach Managing huge data sometimes couldcause unanticipated data cleansing, causing cost and time overruns and risking
delays.
Data Quality:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Data Hierarchy levels:
• U.S.
• Regions and Divisions
• State
• County
• County Subdivision
• Place (or place part)
• Census tract
• Block group
• BlockDifficulty in availability
• Unfortunately, getting data of interest in detail is a great pain.
• Generally the larger the geographic area, the more topics and time periods of data you can find for ex. Data from National Wetlands Inventory in the following slide.
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Data Availability:
Missing data
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Hydric data coverage from SSURGO isnot very satisfactory in Desoto Countyas evident from the picture.
Data Availability:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Management Advantages:1
improve control and knowledge about the data resource and provide a hold on the data.
5allows accurate assessment of cost and time scale to effect any changes.
2reduces the clerical load of database administration, and gives more control
3aid the recording, processing, storage and destruction of data and associated documents.
4reduced data redundancy
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Organized DataData dictionary has served as referencing document for the dataprocessing
• Journal Publication
The MSU team is currently working on paper entitled “Structuringand integrating best‐available geodata to add efficiency in multi‐scale EIA in transportation planning”
Deliverables:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Medium Scale: “Federal, State and MPO data”Identifying feasible alignments
Proposed Alignment B3 of I‐269 Aerial image: 1999
Alternative B3 – Why was it rejected in the EIS?
Need for and Importance of Integrating Local Future Development Planning
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Medium Scale: “Federal, State and MPO data”Identifying feasible alignments
Proposed Alignment B3 of I‐269 Overlay of Future Planned Developments Aerial image: 2004
Need for and Importance of Integrating Local Future Development Planning
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Medium Scale: “Federal, State and MPO data”Identifying feasible alignments
Proposed Alignment B3 of I‐269Aerial image: 20072007“Highly Detailed” Image Shows Recent Development
3” Multi‐spectral image data Provided by Desoto CountyShows High Detail of LocalData!
Need for and Importance of Integrating Local Future Development Planning
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
• Data Dictionary is a building block with which effective,sustainable digital preservation strategies can be implemented
• Data Dictionary is implementation independent and is useful toany organization committed to the long‐term preservation ofdigital materials
• Data content standards improve use and accessibility
– Data and metadata are easier to understand
– Data and products may be more readily used by many end users
Conclusion:
M S U - N C R S T - S E P P M E M P H I S - I N - M A Y W O R K S H O P M a y 6 t h – 8 t h , 2 0 0 9 M e m p h i s - T N
DATA DICTIONARY UNDERSTANDING & STRUCTURING AVAILABLE GEODATA
Acknowledgements
NCRST-SEPP research sponsored by the U.S. Department of TransportationResearch and Innovative Technology Administration (USDOT RITA) underCooperative Agreement DTOS59-07-H-0004, “Streamlining TransportationCorridor Planning Processes and Validating the Application of CommercialRemote Sensing and Spatial Information (CRS&SI) Technologies forEnvironmental Impact Assessments”