becoming an evidenced-based practitioner: reading research 101

Handout for the Therapro webinar Becoming an Evidence Based Practitioner: Reading Research 101presented by Kate Barlow, OTD, MS, OTR/L - September 14, 2021

Copyright © 2021 Kate Barlow. All rights reserved. No part of this material shall be reproduced or transmitted in any form or by any means without permission from the author.

Becoming an Evidenced-Based Practitioner: Reading Research 101

Kate Barlow, OT, OTD, [email protected]

Kate Barlow, OT, OTD, OTRAssociate Professor at American International CollegeCDC’s Act Early Ambassador for MassachusettsFounder of the International Interprofessional Mentorship Group https://libguides.aic.edu/[email protected]

LLeeaarrnniinngg OObbjjeeccttiivveess::

Identify ways to access online research articles

1Understand the basic statistics used in a research article

2Analyze a research article using a free analysis tool

3 My personal goal for this webinar is that you will learn how to appraise research and feel comfortable doing so.



Poll #1Dr. Kizzmekia Corbett is a key developer behind the Moderna vaccine.

What is Evidenced-Based Practice (EBP)?

The latest definition by Zimolag, French and Patterson in 2002:

Defines evidenced-based occupational therapy as the client-

centered enablement of occupation, based on client information and a critical

review of relevant research, expert consensus and experience. This definition

of evidenced-based occupational therapy recognizes the range of sources and

scope of evidenced available to occupational therapists including: research

evidence, information provided by the client for determining occupational

priorities and capacities, knowledge that occupational therapists have gained

from past experiences (as cited in Sudsawad, 2006, p.657)

Why do we need EBP?

•High quality care•Gap between research and practice•Improves effectiveness and efficiency of therapy•Challenge old practices with new research•Research provides the proof that treatments are effective for reimbursement

(Hissong, Lape, & Bailey, 2015, p.127)



“The most important reason to implement EBP is that it improves the quality of the services you provide.”

(Brown, 2017,p.7)

Where do we start? Quantitative Research

POWER

WWhhaatt aarree tthhee LLeevveellss ooff EEvviiddeennccee??

I- High Quality Randomized Controlled Trial (RCT), Systematic Reviews of RTC’s, Meta-Analysis

II- Small-Scale RCT’s, non-randomized studies with a control group

III- Non-randomized studies with one group (no control group)

IV- Descriptive Studies (single subject designs, case studies & surveys)

V- Expert Opinion, Literature Reviews

LLeevveell II EEvviiddeennccee

HHiigghh QQuuaalliittyy RRCCTT’’ss-- A RCT with a great enough N to find statistical significanceSSyysstteemmaattiicc RReevviieeww ooff hhiigghh qquuaalliittyy RRCCTT’’ss (an expansive literature review of RCT’s in a subject area that can be repeated due to publication of search terms in each data base). “Synthesizing data”. Cochrane. MMeettaa--AAnnaallyyssiiss-- a statistical analysis of multiple research studies. The highest level of evidence. Yields the greatest statistical power. Researchers have inclusion and exclusion criteria and also must account for publication bias. Very time consuming.



LLeevveell IIII EEvviiddeennccee

SSmmaallll--ssccaallee RRCCTT’’ss (N too small so results are not statistically significant)

NNoonn--rraannddoommiizzeedd ssttuuddiieess wwiitthh aa ccoonnttrrooll ggrroouupp.. Examples: ❖ Cohort studies with a

control group❖ Case-control pretest-posttest

designs with a control group WWhhaatt iiss aa CCoohhoorrtt ssttuuddyy??

A longitudinal study that follows a group over time that performs cross-section data analysis at intervals through time.

WWhhaatt iiss aa ccaassee ccoonnttrroolllleedd

pprreetteesstt-- ppoosstttteesstt ssttuuddyy??

An observational study that compares two groups, one with a disease or intervention and one without. There is no randomization.

LLeevveell IIIIII EEvviiddeennccee

NNoonn--rraannddoommiizzeedd ssttuuddiieess wwiitthhoouutt aa ccoonnttrrooll ggrroouupp.. Examples:

❖Pretest-post-test designs❖Longitudinal studies,

such as a Time Series study❖Cross Sectional studies



WWhhaatt aarree TTiimmee SSeerriieess SSttuuddiieess??

A type of longitudinal study that captures data at specific time intervals, usually equally spaced and behavior is traced over time.

WWhhaatt aarree CCrroossss SSeeccttiioonnaall SSttuuddiieess??A type of study that collects data on a group or population at one specific point in time.

LLeevveell IIVV EEvviiddeennccee

DDeessccrriippttiivvee SSttuuddiieess.. Examples: ❖Single Subject Design❖Correlational Studies❖Case Studies❖Normative Studies❖Survey Studies

LLeevveell VV EEvviiddeennccee

❖Expert Opinion❖Literature Reviews❖Lab research



RReeaaddiinngg aa RReesseeaarrcchh PPaappeerr

❖Statistical Significance❖Clinically Significant❖Effect Size❖Power

SSttaattiissttiiccaall SSiiggnniiffiiccaannccee

Indicates the degree to which the changes can be attributed to the treatment rather than chance.In social sciences, the usual convention is that 5 out of 1000 occurrences of a phenomenon being caused by chance is a reasonable number to accept. This implies you are 95% certain that the improvement is caused from the treatment. This is the ccoonnffiiddeennccee lleevveell. The confidence level (probability level) is expressed as pp << 00..0055 SSttaattiissttiiccaall SSiiggnniiffiiccaannccee

The tests most commonly used to

determine statistical significance are

Pearson’s chi-square test and the T-Test.

If the results of the tests are P=

anything less than 0.05, than the

results are considered significant



If results are statistically significant, a researcher may decide that results can be generalized to a similar population.

SSttaattiissttiiccaall SSiiggnniiffiiccaannccee-- NNeeeedd ttoo kknnooww

(Hissong, Lape & Bailey, 2015)

CClliinniiccaallllyy SSiiggnniiffiiccaanntt

❖A change that would be regarded by clinicians as meaningful and important. ❖The practical meaning of the study results. ❖Clinical significance is not the same as statistical significance.

Effect Size

❖The magnitude of the treatment effect.

❖The effect size is independent from the sample size.

TThhee iinntteerrpprreettaattiioonn ooff CCoohheenn’’ss dd ffoorr eeffffeecctt ssiizzee

Cohen's dd is the mathematical equation completed: M1 minus M2

divided by the standard deviation

SSmmaallll eeffffeecctt ssiizzee

MMeeddiiuumm EEffffeecctt SSiizzee

LLaarrggee EEffffeecctt SSiizzee

0.2- 0.4 0.5- 0.7 0.8 or greater



PPoowweerr--The ability of a study to detect a difference or relationship.

PPoowweerr is based on 3 things:1. sample size2. effect size3. Alpha (confidence) level

(Brown, 2017)

PPoowweerr❖ The larger the study, the more powerful it is.❖ If you have a large effect size, you have

greater power.❖ Power can be manipulated by changing the

confidence level. If you change the confidence level to 90% (instead of 95%), there is a greater chance of reaching statistical significance.

(Brown, 2017)

HHyyppootthheessiiss TTeessttiinngg

TYPE I ERRORSWhen a hypothesis is accepted, yet it is actually false. Changing the confidence (alpha) level to 90%, can result in a type I error.

TYPE II ERRORSWhen the hypothesis is rejected, but it is actually trueThis occurs when the sample size is too small.

Brown, 2017



WWhhaatt iiss tthhee DDiiffffeerreennccee iinn PPuurrppoossee bbeettwweeeenn QQuuaalliittaattiivvee aanndd QQuuaannttiittaattiivvee RReesseeaarrcchh??

QQUUAANNTTIITTAATTIIVVEE

Theory TestingEstablish factsShow relationshipsPredict outcomesGeneralize findings

QQUUAALLIITTAATTIIVVEE

Get an understanding of people, culture, environments or groupsDescribe viewsDescribe experiencesDescribe beliefs

Hissong, Lape & Bailey, 2015, pg.35

WWhhaatt iiss tthhee DDiiffffeerreennccee iinn SSuubbjjeeccttss bbeettwweeeenn QQuuaalliittaattiivvee aanndd QQuuaannttiittaattiivvee RReesseeaarrcchh??

QQUUAANNTTIITTAATTIIVVEE

RecruitedInclusion CriteriaRandomly assigned to a groupLarge subject group Subjects represent populationSubjects have limited interaction with researchers

QQUUAALLIITTAATTIIVVEE

Small subject groupSubjects may not represent population Subjects selected because of roles, culture, or environmentSubjects have intense contact with researchers over an extended period of time

Hissong, Lape & Bailey, 2015, pg.36

TToommlliinn && BBoorrggeettttoo((22000077))

1. Meta-Synthesis of Qualitative Studies

2. Group Qualitative studies with more rigor (Prolonged engagement, Triangulation of data & Stakeholder checking)

3. Group Qualitative Studies with less rigor

4. Qualitative studies with a single informant

MMiixxeedd--MMeetthhooddss SSttuuddyy

a rreesseeaarrcchh approach whereby researchers collect and analyze both quantitative and qualitative data within the same study.



BBaarrrriieerrss ttoo EEBBPP❖Lack of access to

research data bases❖Lack of time to

conduct literature review❖Lack of skills to

conduct literature review❖Lack of skills to

appraise research❖Lack of skills to

apply research

LLaacckk ooff aacccceessss ttoo rreesseeaarrcchh ddaattaa bbaasseess

❖Take an OT/PT/SLP student (free)

❖Contact your alma mater (small fee)

❖Join national associations (AOTA, APTA, ASHA) (fee)

❖https://scholar.google.com/ (free)

LLaacckk ooff ttiimmee ttoo ccoonndduucctt lliitteerraattuurree rreevviieeww

❖Journal Club at work

❖Request productivity adjustment at work

❖Use Cochrane database

❖Read published Critically Appraised Topic (CATs) papers

LLaacckk ooff sskkiillllss ttoo ccoonndduucctt lliitteerraattuurree rreevviieeww && aapppprraaiissee rreesseeaarrcchh

❖Webinars! (Thank you for coming here today!)

❖Journal club at work

❖ Request local professors to provide presentations for work site in exchange for taking students

❖Take a course



DDaattaa SSeeaarrcchh// CCoolllleeccttiioonn LLooggDDaattaabbaassee SSeeaarrcchh

DDaatteessSSeeaarrcchh TTeerrmmss

YYiieelldd ##

AArrttiiccllee FFoouunndd ((iinncclluuddee FFiirrsstt aauutthhoorr,, eett aall.. ((YYeeaarr)) TTiittllee aanndd wweebb aaddddrreessss..

LLeevveell ooff EEvviiddeennccee LLaacckk ooff

sskkiillllss ttoo aappppllyy rreesseeaarrcchh

❖Find a mentor!

❖Journal Club

❖Community of Practice Groups

❖Attend conferences within your area of expertise

Analyzing the Research you Find

CCAASSPP CChheecckklliissttsshttps://casp-uk.net/casp-tools-checklists/

RRaattiinngg SSccaalleess::Allows the reader to evaluate the validity of the research.Examples:

❖ PEDro Scale❖ Alberta❖ CASP

VVooccaabbuullaarryy

BBaasseelliinnee EEqquuaalliittyyThe control group and the experimental group are equivalent in characteristics prior to the intervention.

IInncclluussiioonn CCrriitteerriiaaThe criteria in a clinical trial’s protocol that prospective subjects must meet to be eligible for participation in a study.

EExxcclluussiioonn CCrriitteerriiaaIn the context of a clinical trial, any characteristic in the study protocol that would preclude a potential subject’s participation in the study.




MMaattuurraattiioonnRefers to the subjects’ growth development or changes that occur naturally over time

AAttttrriittiioonnThe subjects who drop out of the study-can be due to illness, a move, inconvenience of the study or death

RReessppoonnssee RRaatteeThe number of surveys returned divided by the total number in the sample. (You want a response rate higher than 20%.) Most studies’ response rate falls between 30-60%.


BBlliinnddiinngg“Techniques to reduce experimental bias by keeping the subjects and/or investigators ignorant of group assignments and research hypothesis”

LLiikkeerrtt SSccaallee“A summative scale based on responses to a set of statements for which respondents are asked to rate their degree of agreement or disagreement”

GGeenneerraalliizzaabbiilliittyy“The quality of research that justifies inference of outcomes to groups or situations other than those directly involved in the investigation”

(Portney & Watkins, 2015)



Let’s Practice!

LLeevveellss ooff EEvviiddeennccee ffoorr QQuuaannttiittaattiivvee RReesseeaarrcchh

I- High Quality Randomized Controlled Trial (RCT), Systematic Reviews of RTC’s, Meta-Analysis

II- Small-Scale RCT’s, non-randomized studies with a control group

III- Non-randomized studies with one group (no control group)

IV- Descriptive Studies (single subject designs, case studies & surveys)

V- Expert Opinion, Literature Reviews



NNaattiioonnaall SSuurrvveeyy ooff OOccccuuppaattiioonnaall TThheerraappyy PPrraaccttiittiioonneerrss’’ IInnvvoollvveemmeenntt iinn RReessppoonnssee ttoo IInntteerrvveennttiioonnSusan M. Cahill, Beatriz McGuire, Nathaniel D. Krumdick, Michelle M. Lee

MMEETTHHOODD.. We mailed a survey to a random sample of 1,000 practitioners from the American Occupational Therapy Association’s Early Intervention and School Systems Special Interest Section.

RREESSUULLTTSS.. Of 295 returned surveys (29.9% response rate), 19 were excluded because of missing or incomplete data. Three-quarters of respondents (77.6%) reported that their districts implemented RtI. Two-thirds of respondents (66.3%) indicated that lack of resources limited their involvement in RtI; two-thirds (67%) said that district guidelines that describe expectations for practitioners’ involvement would help increase their participation. Many respondents cited the need for continuing education and supported moving from a caseload to a workload model.

AA MMiixxeedd--MMeetthhooddss SSttuuddyy EExxaammiinniinngg DDeevveellooppmmeennttaall MMiilleessttoonneess aanndd PPaarreennttaall EExxppeerriieenncceess iinn GGhhaannaaKKaattee BBaarrllooww && SSttaacceeyy RReeyynnoollddss

BBaacckkggrroouunndd:: Ghana currently lacks a systematic method for identifying children with developmental delays, and there do not appear to be any culturally sensitive assessment tools available. The objectives of this mixed methods research study were to explore normal developmental milestones and parental practices in the Ghanaian population. The study also aimed to identify and describe culturally specific factors that should be accounted for on assessments used in the Ghanaian culture.

MMeetthhooddss:: Twenty-eight Ghanaian parents were interviewed and asked to participate in a developmental milestone picture-identification task. Quantitative and qualitative data analyses were performed.

RReessuullttss:: The interviews revealed several cultural factors that may need to be accounted for on a standardized assessment for Ghanaian children. Some gross motor and self-care skills (e.g., crawling and toilet training) were reported to develop earlier in Ghanaian children compared to children in the United States.

FFuunnccttiioonnaall CCaappaacciittyy aanndd SSeellff--EEsstteeeemm ooff PPeeooppllee WWiitthh CCeerreebbrraall PPaallssyy Sandra Martina Espín-Tello; Heather Olivia Dickinson; Manuel Bueno-Lozano; María Teresa Jiménez-Bernadó; Ana Luisa Caballero-Navarro

MMEETTHHOODD.. We conducted a cross-sectional observational study of 108 people with CP, ages 16–65 yr, who were residents of Spain. Self-esteem was captured using the Rosenberg Self-Esteem Scale (RSES), and functional capacity using the Barthel Index (BI). Sociodemographic characteristics were recorded. The relationship between the RSES score and the BI score was analyzed using linear regression.

RREESSUULLTTSS.. RSES scores increased significantly as BI scores increased (regression coefficient = 0.047, 95% confidence interval [0.017, 0.078], p = 0.003). People with a higher level of education, active employment, and independent living arrangements tended to have better functional capacity and higher self-esteem.

TThhee SSIITTEE pprrooggrraamm aatt IIUUPPUUII:: AA ppoosstt--sseeccoonnddaarryy pprrooggrraamm ffoorr iinnddiivviidduuaallss wwiitthh iinntteelllleeccttuuaall ddiissaabbiilliittiieess Pat Rogana, Jean Updikeb, Gwen Chesterfieldcand Steve Savagec

AAbbssttrraacctt. Exemplary transition services for students with intellectual disabilities aged 18–21 should be anchored in community based rather than school basedsettings. A growing number of community-based transition programs are now located on college and university campuses throughout the United States, spurred by the national Think College initiative (http://www.thinkcollege.net/ ). This article describes the SITE (Skills for Independence, Transition and Employment) program located at Indiana University- Purdue University, Indianapolis (IUPUI), an urban campus setting. The program, a partnership between IUPUI and the Indianapolis Public Schools (IPS), focuses on transition from school to adult life and offers a 1–2 year post-secondary education opportunity. The article begins with a brief discussion of college-based transition programs for young adults with intellectual disabilities. The SITE program and how it was developed is then described. Finally, we discuss the development of the Indiana Partnership for Post-secondary Education and Careers. The article concludes with future directions for program development and systems change.



SSeennssoorryy--BBaasseedd AApppprrooaacchheess iinn IInntteerrvveennttiioonn ffoorr CChhiillddrreenn WWiitthh AAuuttiissmm SSppeeccttrruumm DDiissoorrddeerr:: IInnfflluueenncceess oonn OOccccuuppaattiioonnaall TThheerraappiissttss’’ RReeccoommmmeennddaattiioonnss aanndd PPeerrcceeiivveedd BBeenneeffiittss Sandra Thompson-Hodgetts; Joyce Magill-Evans

MMEETTHHOODD.. Occupational therapists working with children with ASD (N = 211 from 16 countries) completed an online survey addressing their work experience, training, use of sensory-based approaches, and beliefs and perceptions about the effects of the approaches. Linear regression was used to determine predictors of use of and beliefs about sensory-based approaches.

RREESSUULLTTSS.. Most respondents (98%) used sensory-based approaches for children with ASD and would recommend the approaches for 57% of the children they treated. Having a mentor who promoted sensory-based approaches and practicing outside North America and Australia predicted greater use and perceived effectiveness of these approaches. Less than 5 yr of occupational therapy experience predicted less use of the approaches.

EEffffeeccttiivveenneessss ooff aa 1100--WWeeeekk TTiieerr--11 RReessppoonnssee ttoo IInntteerrvveennttiioonn PPrrooggrraammiinn IImmpprroovviinngg FFiinnee MMoottoorr aanndd VViissuuaall––MMoottoorr SSkkiillllss iinn GGeenneerraall EEdduuccaattiioonnKKiinnddeerrggaarrtteenn SSttuuddeennttss Alisha M. Ohl, Hollie Graze, Karen Weber, Sabrina Kenny,Christie Salvatore, Sarah Wagreich

MMEETTHHOODD.. We recruited 113 students in six elementary schools. Two general education kindergarten classrooms at each school participated in the study. Classrooms were randomly assigned to the intervention and control groups. Fine motor skills, pencil grip, and visual–motor integration were measured at the beginning of the school year and after the 10-wk intervention.

RREESSUULLTTSS.. The intervention group demonstrated a statistically significant increase in fine motor and visual–motor skills, whereas the control group demonstrated a slight decline in both areas. Neither group demonstrated a change in pencil grip.

TThhee pprroommoottiioonn ooff ppoossiittiivvee mmeennttaall hheeaalltthh ffoorr nneeww mmootthheerrss dduurriinngg CCoovviidd--1199KKaattee BBaarrllooww aanndd AAvviirriill ((AAppppllee)) SSeeppuullvveeddaa

ABSTRACT

The Covid-19 pandemic has caused increased hardship for new mothers and their young children. Increased isolation, lack of in-person doctor visits and decreased interaction within the community, has pregnant and postpartum women in need of additional support. Occupational therapists often work with infants and their mothers due to feeding concerns. Difficulty with early feeding adds additional stress on the mother–infant dyad relationship. This case study describes a mother’s traumatic experience giving birth during Covid-19 and the occupational therapy intervention provided to the mother–infant dyad. In order to improve outcomes for the mother and child with feeding concerns, this case study is a call to action for paediatric occupational therapy practitioners to include the promotion of positive mental health of mothers in their practice, particularly during the pandemic.

IImmpplleemmeennttiinngg tthhee SSaammee SSiittee MMooddeell iinn OOccccuuppaattiioonnaall TThheerraappyy FFiieellddwwoorrkk:: SSttuuddeenntt aanndd FFiieellddwwoorrkk EEdduuccaattoorr PPeerrssppeeccttiivveess KKaattee GG.. BBaarrllooww,, MMiicchhaaeell SSaalleemmii,, && CCoouurrttnneeyy TTaayylloorr

The Same Site Model of fieldwork service delivery provides the opportunity for the occupational therapy student to complete their Level I and Level II fieldwork at the same site. Due to limited research on the Same Site Model, a survey study was conducted to explore fieldwork educator and student perceptions on using the model. A 10-question survey, with 8 Likert questions and 2 open ended questions was designed by 2 Academic Fieldwork Coordinators. A total of 116 surveys were completed by occupational therapy students (N=45) and fieldwork educators (N=71) on their preference of the Same Site Model, along with their perceptions of the student’s preparedness, communication skills and ability to interact with clients during their Level II fieldwork. Out of the 116 returned surveys, only 24 indicated having the opportunity to use the Same Site Model. The findings of this current study revealed a moderate positive correlation (r=0.407) at 0.01 level of significance (p<0.05) suggesting that although only 20% of the sample population had experience using the Same Site Model, the majority (83%) of occupational therapy students and fieldwork educators who did participate in the Same Site Model would prefer it again in the future. In addition, although not all respondents felt the Same Site Model should be considered best practice, 50% of all the returned surveys reported that the Same Site Model should at least be considered when placing students.



PPoosstt--sseeccoonnddaarryy TTrraannssiittiioonn SSeerrvviicceess aanndd tthhee RRoollee ooff tthhee OOccccuuppaattiioonnaall TThheerraappiisstt

KKaattee BBaarrllooww aanndd NNeeiill GGiillee

AAbbssttrraacctt:: Post-secondary transition services are federally mandated, yet there are great differences in implementation. The purpose of this study was to explore the current practices in post-secondary transition services and to understand how occupational therapists are involved in transition services. Public school administrators from 11 school districts were interviewed, all of whom work in transition services. Triangulation of data, member and stakeholder checks were all completed for validation and to ensure trustworthiness. Analysis of the interviews revealed four assertions:

a) Districts use a variety of personnel as the transition coordinator b) Transition assessments need to be continuous and more comprehensive c) More training is needed for the entire transition process d) Occupational therapists are underutilized in transition services

From these assertions, one global theme emerged: There is a need for increased training on the overall process to deliver best practice transition services, including timeline, assessments, agency involvement, and training for parents and staff. This study also discussed the importance of parent involvement for student success. Future research is recommended on the implementation of the pre-employment transition services and the process of evaluating students throughout their transition years.

TThhaannkk yyoouu..QQuueessttiioonnss??

References

Brown, C. (2017). The Evidence-Based Practitioner: Applying Research to Meet Client Needs. F.A. Davis Company.

Hissong, A., Lape, J., & Bailey, D. (2015). Research for the Health Professional (3rd ed.). F.A. Davis Company.

Portney, L., & Watckins, M. (2015). Foundations of clinical research: Applications to practice (3rd ed.) F. A. Davis Company.

Sudsawad, P. (2006). Definition, evolution, and implementation of evidence-based practice in occupational therapy. In G. Kielhofher (Ed.) Research in occupational therapy; Methods of inquiry for enhancing practice (656-662). F.A. Davis Company.

Tomlin, G., & Borgetto, B. (2011). Research Pyramid: A new evidence-based practice model for occupational therapy. American Journal of Occupational Therapy, 65, 189–196. doi: 10.5014/ajot.2011.000828

becoming an evidenced-based practitioner: reading research 101

Documents