evidence based practice university of utah
DESCRIPTION
Presentation Prepared by Christian Sabey March 2009. Evidence Based Practice University of Utah . Training School Psychologists to be Experts in Evidence Based Practices for Tertiary Students with Serious Emotional Disturbance/Behavior Disorders US Office of Education 84.325K H325K080308. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/1.jpg)
+Evidence Based Practice University of Utah
Training School Psychologists to be Experts in Evidence Based Practices for Tertiary Students with Serious Emotional Disturbance/Behavior Disorders
US Office of Education 84.325K
H325K080308
Presentation Prepared by
Christian Sabey
March 2009
![Page 2: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/2.jpg)
+
Research in Special Education: Scientific Methods and Evidence-
Based Practice Exceptional Children Vol. 71, No. 2,
pp.137-148Council for Exceptional Children
![Page 3: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/3.jpg)
+ Council for Exceptional Children (CEC)
CEC was first established in 1922 by school administrators and faculty at the Teachers College of Colombia University
Its purpose is to improve the success of children with disabilities and gifted/talented children
It is the largest professional organization in the country devoted to special education (SPED)
Focus is on teachers and administrators (applied research)
![Page 4: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/4.jpg)
+ What is the issue?
In light of No Child Left Behind (NCLB), teachers are required to use scientifically validated practices
Disagreement in the field of SPED over what should be accepted as scientifically validated practice Randomized clinical trials (RCT) are the “gold standard” for
establishing effectiveness RCT are not widely utilized in SEPD research Other, less rigorous, methodologies are used in SPED
research
What evidence should be accepted to scientifically validate a practice?
![Page 5: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/5.jpg)
+ CEC Division of Research Task Force
AssumptionsDifferent types of research questions are
important in establishing the effectiveness of a practice
Different types of questions require different types of methodologies
![Page 6: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/6.jpg)
+ Methodologies Being Used in SPED
CEC Division or Research identified four types of methodologies being used in SPED research:1. Experimental group2. Correlational 3. Single subject4. Qualitative designs
(Each of these methodologies was addressed individually in an article in the issue that this article came from)
![Page 7: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/7.jpg)
+ Rational for Multiple Methodologies
National Academy of Science Questions must guide research methods
What is happening? (description) Is there a systematic effect? (cause) Why or how is it happening (mechanism)
Complexity of SPED RCT is for hard sciences (physics, chemistry, biology)
“We [education researchers] do our science under conditions that physical scientists would find intolerable” (Berliner 2002)
Variability in subject characteristics (ex. Comorbid conditions)
SPED services occur across several different contexts Random assignment is often impossible or unethical
![Page 8: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/8.jpg)
+
History of SPED research SPED research has historically drawn from multiple
academic disciplines like psychology, sociology and anthropology, which each use different methodologies
More than one methodology is important to SPED research
Institute of Education Science (IES) RCT is gold standard Different methodologies are important for addressing
different questions Other methodologies allow researchers to work in
naturalistic settings (applied research)
Rational for Multiple Methodologies Continued
![Page 9: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/9.jpg)
+
Given that multiple methodologies are used in SPED research, what can be done to distinguish between shoddy research and high quality research?
![Page 10: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/10.jpg)
+ Quality Indicators of Research Methodology
Researchers in SPED need to establish quality indicators for each methodology that is used in SPED research
Establishing quality indicators for each methodology would: Assist researchers in the design of their experiments Provide a standard for reviewers to evaluate research findings Help consumers to determine how useful research findings
may be Other organizations have already made steps in this
direction American Psychological Association (APA) Division 12 APA Division 16 Society for the study of School Psychology
![Page 11: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/11.jpg)
+ Quality Indicators of Research Methodology Continued
What Works Clearinghouse (WWC) has developed a method specific evaluation tool called the Design and Implementation Assessment Device (DIAD) Used to conduct an extremely detailed evaluation of a
research article Used to guide researchers using a particular methodology
Quality indicators for research in SPED have not been identified and articulated (as of 2005)
![Page 12: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/12.jpg)
![Page 13: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/13.jpg)
+ Evidence-Based Practice In Education
Search for evidence-based practice originated in the medical field as early as the mid 19th century
The search came to fruition in the 1990s in an effort to bridge the gap between what was known and what was practiced in the medical field
Educators followed suit by recognizing the gap between research and practice and looking for ways to fill the gap
Two types of organizations emerged to fill this need Research synthesis organizations Professional associations
![Page 14: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/14.jpg)
+ Research Synthesis Organizations
![Page 15: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/15.jpg)
+ Cochrane Collaborationwww.cochrane.org
![Page 16: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/16.jpg)
+ Campbell Collaborationwww.campbellcollaboration.org
![Page 17: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/17.jpg)
+ University of London Institute of Education Evidence for Policy and Practice Information and Co-
ordinating Centre (EPPIC)http://eppi.ioe.ac.uk/cms/
![Page 18: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/18.jpg)
+ Institute of Education ScienceWhat Works Clearinghouse
http://ies.ed.gov/ncee/wwc
![Page 19: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/19.jpg)
+ Research Synthesis Organizations Continued
Most research synthesis organizations choose to limit the evidence supporting a practice to RCT or rigorously controlled quasi-experimental designs.
WWC suggests that qualitative research could be used to indicate that a practice is “promising” but is not sufficient to determine if a practice is efficacious
![Page 20: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/20.jpg)
+ Professional Associations
![Page 21: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/21.jpg)
+ Professional Association Standards Many professional associations have established their
own standards for what will be considered evidence based practice
![Page 22: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/22.jpg)
+ APA Division 12
Well-established Two well-conducted group design studies by different
researchers Nine well-conducted single-subject designs
Probably efficacious Two group design studies by same investigator At least three single-subject design studies
![Page 23: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/23.jpg)
+ CEC Division of Early Childhood
Established a process for identifying recommended practices Literature review Focus groups
Experts Practitioners Families
![Page 24: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/24.jpg)
+ American Speech-Language Hearing Association
Adapted a level system from the Center for Evidence-based Medicine Level I – Meta-analyses including at least one randomized
experimental design or well-designed randomized control studies
Level II – Controlled studies without randomization and quasi-experimental designs
Level III – Well designed nonexperimental studies (correlational studies, case studies)
Level IV – Expert committee report, consensus conference, and clinical experience of respected authorities
![Page 25: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/25.jpg)
+ CEC Division of Learning Disabilities and Division of
Research Alerts
Literature review by a single expert from the field
![Page 26: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/26.jpg)
+ Where do we go from here?
Pressure on the Department of Education to demonstrate that there are effective educational practices available
Pressure to demonstrate that investing money in educational research is a good investment
It is important that “science” not be defined by one method
Research in the field of education should be viewed as a continuum rather than a fixed point
![Page 27: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/27.jpg)
+ Stage 1
Observation Exploration Flexible methodology
Correlational methods Qualitative methods
![Page 28: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/28.jpg)
+ Stage 2
Controlled laboratory or classroom experiments Observational studies of classrooms Teacher-researcher collaborative experiments
Qualitative Single-subject Quasi-experimental RCT
![Page 29: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/29.jpg)
+ Stage 3
Well-controlled RCT in naturalistic settings to prove the effectiveness of interventions that were promising in Stage 2.
Single-subject designs may be included in this stage
![Page 30: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/30.jpg)
+ Stage 4
Determine what factors facilitate the adoption of effective practices in typical school settings with typical teachers.
![Page 31: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/31.jpg)
+
Randomized Clinical Trial
Control Group
Quasi-Experimental
Single-Subject
Correlational
Qualitative
Quality Indicators
Quality Indicators
Quality Indicators
Quality Indicators
Quality Indicators
Quality Indicators
Strength of evidence of effectiveness
Weak
Strong WeakStrong
Weak
Strong
Continuum of Evidence
![Page 32: Evidence Based Practice University of Utah](https://reader035.vdocuments.us/reader035/viewer/2022062411/568168b3550346895ddf800a/html5/thumbnails/32.jpg)
+ Conclusion
SPED as a field needs to clarify The match between research questions and methodologies The guidelines of a methodology that make it “high quality” How to use the research findings from each methodology as
scientific evidence for establishing effective practice