datamanagement&$intro$to$rpeople.tamu.edu/~alawing/materials/essm689/datamanagement.pdf ·...
TRANSCRIPT
![Page 1: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/1.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Data Management & Intro to R
A. Michelle Lawing Ecosystem Science and Management Texas A&M University College Sta,on, TX 77843 [email protected]
![Page 2: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/2.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Michener and Jones 2011
![Page 3: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/3.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
3
Barriers to Synthesis
• Data not preserved – Tiny propor,on of ecological and evolu,onary data are readily
available
• Dispersed, isolated repositories – Each community has its own; disconnected; underu,lized
• Lack of so\ware interoperability
• Heterogeneous data – Many data formats, metadata formats, and varying seman,cs
![Page 4: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/4.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Dispersed data
Global Biodiversity Informa,on Facility (GBIF) Downloaded 01/26/2015
![Page 5: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/5.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Data diversity • Biological
– e.g., Gene, Organism, Popula,on, Species, Community, Biome, Ecosystem
• Environmental – e.g., Atmospheric, Chemical,
Ecological, Hydrological, Oceanographic, Physical
• Social – e.g., Land use, human popula,on
• Economic – e.g., trade, ecosystem services,
resource extrac,on
![Page 6: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/6.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Biodiversity data heterogeneity Space Time Taxa
![Page 7: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/7.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
“Dark” data in the long tail
Heidorn 2008
![Page 8: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/8.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Data Heterogeneity Heterogeneity High Low
• Tight coupling • Simple subsetting • Explicit semantics
• Loose coupling • Hard subsetting • Limited semantics
Volume Low High
![Page 9: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/9.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
![Page 10: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/10.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
![Page 11: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/11.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Knowledge Network for Biocomplexity Data Distribu,on
Data until: 07 Oct 2011 Total: 25,191 data sets
![Page 12: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/12.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
So\ware diversity
GMN
![Page 13: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/13.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Solu,ons
• Preserve data
• Adopt standards
• Create networks
• Create interoperable so\ware
![Page 14: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/14.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Metadata and data heterogeneity • Every community has
– many data schemas • one for each project and person
– many data formats • ASCII, NetCDF, HDF, GeoTiff, ...
– many metadata schemas • Biological Data Profile, Darwin Core, Dublin Core, Ecological Metadata Language (EML), Open GIS schemas, ISO Schemas, ...
• Accep,ng this heterogeneity is cri,cal
![Page 15: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/15.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Column metadata
![Page 16: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/16.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Morpho
Wizard to create metadata
![Page 17: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/17.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Morpho highlights • Create metadata in EML format • Manage data in EML packages • Save, publish, and share data
• Search for data • Mul,-‐language
– English, Spanish, Chinese, French, Portuguese, Japanese • Export data and metadata • Cross-‐plajorm, and open source
Morpho
![Page 18: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/18.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
Kepler
DMP-Tool
Plan
Collect
Assure
Describe
Preserve
Discover
Integrate
Analyze
![Page 19: DataManagement&$Intro$to$Rpeople.tamu.edu/~alawing/materials/ESSM689/DataManagement.pdf · ESSM689$Quan,tave$Methods$in$ Ecology,$Evolu,on$and$Biogeography$ Ecosystem$Science$and$Management|$](https://reader034.vdocuments.us/reader034/viewer/2022042019/5e76a7f5d255000eae0399fb/html5/thumbnails/19.jpg)
ESSM 689 Quan,ta,ve Methods in Ecology, Evolu,on and Biogeography
Ecosystem Science and Management | Texas A&M University (c) 2015, A. Michelle Lawing
How do we harness the long tail?
• Efficient data federa,on – Focus on individual contributors
• Late binding in informa,cs systems – Loose coupling – Schema-‐less storage
• Central search for discovery
• Interoperable so\ware