1
MODULE
Introduction
whatworks.ed.gov
2
MODULE Introduction
This module provides an introduction to the What Works Clearinghouse (WWC) Group Designs training.
The objectives of this module are to:
Clarify the purpose of this training and explain the certification options Provide background information on the WWC Describe logistics related to training Identify resources to support learning the WWC Group Design
Standards Provide an overview of the topics covered in the training
Training Purpose The WWC offers this training to inform the public about our design
standards, increase transparency, and promote the use of rigorous research.
The purpose of this training is to: • Describe the WWC Group Design Standards• Explain how to review a study using the WWC Group Design Standards• Prepare people for the optional group design certification activities
This training is not a comprehensive training on designs or methods.
3
Who Should Take the WWC Group Design Training?
Those interested in assessing the credibility of research evidence and interpreting research findings.• People trying to make evidence-based decisions. • Those interested in developing rigorous studies to meet WWC Group Design
Standards.
People who want to receive a certificate of completion for viewing the first six modules.
People who want to pursue one of two certification options through the WWC:1) Becoming certified in the WWC Group Design Standards.2) Becoming a certified reviewer for the WWC.
4
What Works Clearinghouse: Who, What, How, and Why
Who we are: An initiative of the Institute of
Education Sciences (IES) at the U.S. Department of Education
What we do: Evaluate and summarize the rigorous
research on education interventions
How we do it: Using a common framework:
The WWC Group Design Standards
Why we do it: Provide educators with
the information to make evidence-based decisions
5
Examples of WWC Products
6
Intervention reportssummarize all the
rigorous evidence on a specific intervention to
support evidence-baseddecision making.
Practice guides help educators address classroom challenges by
offering concrete suggestions and
summarizing the evidence base associated with the
suggested practices.
Quick reviewscontain the WWC’s
assessment of a single publication that is receiving media
attention.
Training Logistics
The online training is open to the public and accessible at all times.
You do not have to complete the online training in one sitting.
You can return to the training and start any module at any time.
You can work through training modules at your own pace.
You may print each module to view offline.
You should proceed through the modules sequentially, as later modules may revisit and build upon the concepts from earlier modules.
7
Training Logistics
Use the button in the middle of the screen to continue moving through the training or use the menu on the left to navigate through the modules.
Use the buttons on the top right to share the training with others or save your progress.
When saving your progress, you will obtain a unique passcode (PIN), which you can use to resume the training when you return.
You must complete all of the modules in the training series (and view them through the WWC website) to obtain your certificate of completion.
The “print certificate” link will appear after you complete the final module.
8
Trainee ResourcesTrainees can access several resources to supplement their learning during this training:
Procedures and Standards Handbooks: The WWC Procedures Handbook and the WWC Standards Handbook document all the procedures and standards in this training and include information about how the WWC reviews, documents, and reports on studies.
Review protocols: Review protocols describe the scope and parameters of each review.
Glossary: Glossary terms are underlined and bolded in red. You can access the full glossary through the Help Desk menu tab on the WWC homepage or from the Resources menu on the training page.
FAQ: You can access a list of questions and answers through the FAQ link in the Resources menu.
Help: You can submit technical questions about the training environment or questions about the content covered in the training through the WWC help desk.
Access these resources and more through the WWC website: whatworks.ed.gov.
9
What Is the Value of the WWC Group Design Standards?
• The WWC developed and implemented a set of standards for reviewing and synthesizing education research because:
There is a substantial amount of research on education interventions. The quality of that research varies widely. There were no common rules or guidelines for developing research studies.
• The WWC standards provide a set of common guidelines to help the public:
Decipher the information from different studies with different findings. Identify key elements in understanding quality research.
10
What Are the WWC Group Design Standards? The standards provide a common framework
to assess the credibility of research evidence on the effectiveness of interventions.
The WWC standards are sets of criteria used to assess research and assign a rating that reflects a study’s internal validity.
Upon reviewing an eligible study, the WWC assigns one of three ratings to it.
After reviewing studies using the standards, the WWC summarizes its findings in different publications and products.
11
Does Not Meet WWC Group
Design Standards
Meets WWC Group Design
Standards Without
Reservations
Design Standards
Meets WWC Group Design
Standards With
Reservations
Group Design Standards Framework
12
Training Module Content
Each module includes:
A summary of the objectives for the module.
Detailed information on the module topic, including examples.
“Knowledge Checks” to reinforce content and test what you’ve learned.
13
Training Module Overview
In addition to this introductory module, the WWC Group Design Standards Training contains several other modules:
14
Required Modules forCertificate of Completion
Additional Training Modules
ReportingCluster-Level AssignmentStudy Review Guide
MODULE
MODULE
8Group DesignsAttritionBaseline EquivalenceConfounding FactorsOutcome MeasuresSystematic Review Process
MODULEMODULEMODULEMODULEMODULE
12345
MODULE 6
MODULE 9
7
Module 1 Overview: Group Designs
The Group Designs module will answer:
What types of research designs can we review using the WWC Group Design Standards? (randomized controlled trials [RCTs] and quasi-experimental designs [QEDs])
What counts as an eligible RCT and an eligible QED study?
What does the WWC consider well-executed randomization?
15
Module 2 Overview: Attrition
The Attrition module will answer:
What is attrition and how does the WWC define attrition?
How does the WWC assess attrition for RCTs?
Why is attrition important?
How does the WWC calculate attrition rates?
How does the WWC apply the attrition standards to RCTs?
How can we recognize what the WWC does and does not consider to be attrition?
16
Module 3 Overview: Baseline Equivalence
The Baseline Equivalence module will answer:
How does the WWC define baseline equivalence?
When and how to apply the WWC equivalence standard?
How does the WWC determine the characteristics that it uses to assess baseline equivalence?
17
Module 4 Overview: Confounding Factors
The Confounding Factors module will answer:
How does the WWC define a confounding factor?
What are examples of confounding factors?
How do confounding factors affect the WWC’s review of a study?
18
Module 5 Overview: Outcome Measures
The Outcome Measures module will answer:
How does the WWC define an outcome measure?
Why does the WWC evaluate the outcome measures used to demonstrate impacts in studies?
What is the WWC standard for outcome measures and what requirementsmust a study’s outcome measures meet?
How does the WWC apply the outcome measure standard?
19
Module 6 Overview: Systematic Review Process
The Systematic Review Process module will answer:
What are the steps of the systematic review process?
How does the WWC use the systematic review process to evaluate education research?
How does the WWC use review protocols to guide the parameters of each review?
How does the WWC identify and define a study?
20
Module 7 Overview: Reporting
The Reporting module will answer:
How does the WWC report on the effectiveness of an intervention?
How does the WWC define and calculate effect size and the improvement index?
What are the statistical adjustments the WWC makes during a study review?
How does the WWC characterize study findings based on the effect size and statistical significance of effects for a domain?
21
Module 8 Overview: Cluster-Level Assignment
The Cluster-Level Assignment module will answer:
How does the WWC identify eligible cluster designs?
How do compositional changes affect the rating of cluster RCTs?
What is the standards framework the WWC uses to review cluster studies?
22
Module 9 Overview: Study Review Guide
The Study Review Guide module will answer:
What is the purpose of the study review guide?
How do reviewers access the study review guide on the WWC website?
What are the major components of the study review guide?
How do reviewers use the study review guide to enter information from a study?
23
24
INTRO MODULE
Conclusion
25
MODULE Introduction
You can access all the resources mentioned in this module through the WWC website, whatworks.ed.gov.
The full slide deck for this module is available on the WWC website. To receive a certificate of completion for viewing these training modules, you
must view the videos on the WWC website.
Thank you!