howard university october 10,2012 the integrity of data management responsible conduct of research...
DESCRIPTION
OMB Guidelines for Federal Statistical Agencies 16 Federal Statistical Agencies adhere to Office of Management and Budget Guidelines to maintain high standards of performance Development of concepts and methods Planning and design of surveys and other means of collecting data Collection of data Processing and editing of data OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, February 22, 2002, (67 FR ) 3TRANSCRIPT
Howard UniversityOctober 10,2012
The Integrity of Data ManagementResponsible Conduct of Research
Workshop
Peggy Carr, Ph.D.Associate Commissioner
National Center for Education Statistics
1
Overview of Statistical and Research Standards
2
OMB Guidelines for Federal Statistical Agencies
16 Federal Statistical Agencies adhere to Office of Management and Budget Guidelines to maintain high standards of performanceDevelopment of concepts and methodsPlanning and design of surveys and other
means of collecting dataCollection of dataProcessing and editing of data
OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, February 22, 2002, (67 FR 8752-8460) 3
Best Practices for the Conduct of Research
4
Best Practices for the Conduct of Research
1. Conceptualize Research Problem2. Operationalize Study Design3. Engage in Peer Review and Networking
Activities4. Publish and Publish Early/Often5. Be Ethical
5
1. Conceptualize Research Problem
6
Literature Review
Statement of Problem
Hypotheses Research Question
Definition of Terms, Concepts, and
Constructs
ConceptualOperational
2a. Operationalize Study DesignThe Concept of Study Design
Hypothesis 1
Explained Variance
Unexplained Variance
Sampling Error
Measurement Error
Validity
Reliability
Validation of ConstructsFace – Content validation across judgesCriterion – Predictive and concurrentConstruct – Methodological approaches to validation
8
Jealousy Envy
2b. Operationalize Study Design
2c. Operationalize Study DesignReliability of Measures
Test and Retest – stability over timeEquivalency – consistency across formsInternal – internal consistency
9
UNRELIABLE RELIABLE
2c. Operationalize Study DesignResearch Design
Internal Validity – refuting rival plausible hypothesesExternal Validity – generalizability of findingsChoice of Designs – experimental, quasi, survey,
correlational
10
Quality Assurances of DataData Checks (e.g., out of range variance, criterion
reference checks, replication)
Thoughtful and Appropriate Data AnalysisThe nature of your dataLimitations of statistical technique
Discuss Findings and Conclusions with ColleaguesRelate back to literature reviewImplications for future researchFlaws and limitations
11
NEXT
2c. Operationalize Study Design
Trend in fourth-grade NAEP mathematics average scores
12
2c. Operationalize Study Design
SOURCE: The Nation’s Report Card, 4th Grade NAEP: Mathematics, 1990-2011.
13
2c. Operationalize Study DesignTrend in fourth-grade NAEP mathematics percentile scores
SOURCE: The Nation’s Report Card, 4th Grade NAEP: Mathematics, 1990-2011.
Oprah discussing education and research
3. Engage in Peer Review and Networking Activities
Find a specialty and stick with it
Attend professional meetings and conferences – share your research
Keep an open mind and learn from others
15
4. Publish and Publish Early/OftenStart your publication record early in peer
reviewed journalsAs first, second, or third author
Publish and/or present oftenRegardless of professional focusAnnually
16
5. Be EthicalBe committed and follow through
Adhere toSubjects’ RightsInstitutional Review BoardConfidentiality and Privacy Rights
17
Questions?
18