making the case for metadata at srs-nsf national science foundation division of science resources...

Making the Case for Metadata at SRS-NSF

National Science Foundation

Division of Science Resources Statistics

Jeri Mulrow, Geetha Srinivasarao, and John Gawalt

FedCASIC Workshops, BLSMarch 17, 2010

National Science FoundationDivision of Science Resources Statistics

www.nsf.gov/statistics/1

1 9 8 4

Today’s Talk

• A bit about SRS

•Historical perspective of data and metadata dissemination

• Metadata users and their metadata needs

• Standardization efforts

• Challenges and future vision

A bit about the Division of Science Resources

Statistics (SRS)

• Federal Statistical agency within NSF

• 11 periodic data collections on the U.S. Science and Engineering enterprise

• Data dating back to the 1950s

Historical Perspective of SRS data and metadata dissemination

• 1950s – early 1990s paper only

• Detailed statistical tables withminimum metadata as footnotes

• Publications included Highlights about the survey Scope and method of survey Questionnaire Cover letters

Example -- 1950s publication

1990’s thru 2000’s

• 1992 – electronic format

• Detailed statistical tables in spreadsheetswith minimum metadata as footnotes

• Kept paper, added electronic text Survey Methodology, Limitations to the data, Definitions, Historical revisions, List of tables

• PDF added Questionnaire, Cover letters, Instructions

Example --1993 PDF

Example – 1991 Electronic spreadsheet

Example – 1991 text

• Source data tables in Excel with footnotes

• HTML / PDF Highlights of the survey Links to references Survey description

• PDF Survey Questionnaire Instructions Definitions

Example – 2007 Excel spreadsheet

Example -- 2007 SIRD1

Example – 2007 HTML

Example – 2007 PDF

BUT THAT’S NOT ALL

• Electronic databases Create and download your own customized

aggregate tables

• Public use files Access to some microdata series

Metadata in WebCASPAR ….

Metadata in WebCASPAR

• Variable specific metadata available under Info link

• Metadata not tightly integrated with the data itself – does not get downloaded with the data

WebCASPAR Taxonomy

• Survey specific taxonomies

•NCES IPEDS Classification of Instructional program codes (CIP)

• Integrated taxonomy for querying across surveys

http://webcaspar.nsf.gov/

Metadata in SESTAT

• Metadata Explorer is separate from the data Individual variable information

Description Question Domain/Availability – history Valid response categories Keywords

•Metadata is not tightly integrated with the data itself – it does not get downloaded with the data

https://sestat.nsf.gov/sestat/sestat.html

Example -- Public Use file

Summary – Where are we?

• Different surveys have evolved differently Varying levels of details/metadata

• Not in an standardized structure

Hodge-podge

Metadata Users & Their Metadata Needs

• Not a one-to-one relationship, but many-to-many

• They occur at all stages of the survey process

Process Data

Define research objectives

Choose mode of collection Choose sampling frame

Construct and pretest questionnaire Design and select sample

Develop Survey Instrument Develop Sample Design

Survey Process

Source: Survey Methodology (2009) Groves, Fowler, Couper, Lepkowski, Singer & Tourangeau.

Recruit and measure sample

Code and edit data

Make postsurvey adjustments

Perform analysis

Define Scope

Collect Data

Disseminate Data

Define Scope

Users Metadata

Data User GeneralSurvey Manager TopicSubject Matter Expert Population of interestStatistician Other data sourcesSurvey Methodologist SpecificRespondent Frame options Sample design options Historical info/data User needs Federal Register notices

Develop Survey Instrument

Users Metadata

Data User QuestionsSurvey Manager Answer choicesSubject Matter Expert Definition of termsStatistician InstructionsSurvey Methodologist Logic flow of questionsRespondent Cognitive work Validity assessments Reliability assessments Functionality testing Alternative questions Instrument design specs – paper, web, CATI

Develop Sample Design

Users Metadata

Data User Population of interestSurvey Manager Sampling frame / Universe specsSubject Matter Expert Update scheduleStatistician Sample design specs Desired criteria Sample selection techniques Historical information on performance of designs Estimation methods

Collect Data

Users Metadata

Data User Variable names and formatsSurvey Manager Variable data typesSubject Matter Expert Physical storageStatistician Tables and relationshipsDatabase Administrators Mapping of questions toSoftware Developers variables and definitions Logic flow of questions Response rates over time Paradata Cover letter

Process Data

Users Metadata

Data User Item response ratesSurvey Manager Zero vs. null vs. missingSubject Matter Expert Edit specificationsStatistician Imputation specificationsDatabase Administrators Recode specificationsSoftware Developers Data table specifications Changes across survey cycles

Data Dissemination and Publication

Users Metadata

Data User History of changesSurvey Manager Methodology reportSubject Matter Expert Public use files with Statistician documentationDatabase Administrators Author/contact source Software Developers Who can access whatArchivist Type of product Content format URL; Keywords Relationships Metadata schema

Who are the Metadata Users?

• Data users Basic & advanced Analysts General public

• Respondent• Survey Manager• Survey Methodologist• Statistician• Subject Matter Expert• Software Developer• Database Administrator• Archivist

Need for Standardization of Metadata

is Apparent

is Critical

Standardization Efforts

• Dublin Core

• SDMX (aggregate level)

• DDI 3.0 (record level)

Recent SRS Efforts

• Data Repository (Oracle)

• Inclusion of some metadata

• SAS/ACCESS User Interface for internal users

• Evaluating external user interfaces

SRS Efforts -- Working with Commercial Contractors

• Requirements for Data / Metadata delivery

• Examples document

• Standard contracting language

• Checklist

SRS AdoptedBasic Operating Procedures

• Using Oracle to store microdata and metadata

• Collecting metadata in whatever format

• Keeping it all organized

Challenges

• Getting all the players on the same page Many different users Many different uses Many different providers Many different products Many different formats

• Cost

• Keeping it all straight

Near Future Vision

SRS Data Repository

Data and Metadata

Taxonomy Efforts

Data & Metadata

Dissemination

Analytic tools

DDI 3.0, SDMX…

Near Future Vision

SRS Data Repository

Data and Metadata

Taxonomy Efforts

Data & Metadata

Dissemination

Analytic tools

DDI 3.0, SDMX…

Paradata

Thank you!

making the case for metadata at srs-nsf national science foundation division of science resources...

minimum metadata

s paper

survey links

survey scope

s early

list of tables pdf

footnotes html pdf highlights

detailed statistical

Documents

nair, devi geetha; rasilo, paavo; arkkio, antero

faculty of medicine postgraduate handbook · geetha devi...

bhagwad geetha

geetha timber tamil nadu india

geetha dhyanam malayalam with meaning

ashtavakra geetha

4. civil - a modified - k. geetha

dr. geetha n. - university of mysorebannariamman institute...

geetha nokia

lina mukhopadhyay & geetha durairajan evaldepteflu@gmail 20...

evidenced-based case management practice, part...

geetha madhuri to get hitched today

geetha saara thaalaattu

the bhagavad geetha simplified (spiritual)-115

geetha portfolio-2014

harithaharam with smt.puli geetha, muncipal chair person ·...

presentation by geetha thoopal, addl. member (finance...

baskaran, geetha (2016) a domain transformation approach for...

geetha reddy

geetha vahini