evaluation results
DESCRIPTION
Evaluation Results. Ohio’s SAMHSA Garrett Lee Smith Grant. Evaluation of Ohio’s Infrastructure for Statewide Mental Health Check Ups. Responsivity Continuous Quality Improvement Do-Study-Reflect-Plan. School Climate. Capability (Stakeholder perception of innovation adoption). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/1.jpg)
Evaluation Results
Ohio’s SAMHSA Garrett Lee Smith
Grant
![Page 2: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/2.jpg)
School Climate
Evaluation of Ohio’s Infrastructure for Statewide Mental Health Check Ups
ReceptivityFamily motivation togive consent, complete referral
Program Providers
Youthand
Families
Activities consent,
screening, referral
OutputConsents
and screenings
offered “Counts”
OutcomesConsents granted;Referrals completed
ResponsivityConsent Climate Referral Climate
ResponsivityContinuous Quality Improvement Do-Study-Reflect-Plan
Capability(Stakeholder perception of innovation adoption)
![Page 3: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/3.jpg)
Consents
NOT Returned
3,791 31% 8,434 69% Count Row %
Returned No
Count Row %
Returned YesConsents Returned = 12,232 (48%)
25,662 100% 3,791 15% 13,4308,434 33%
Count Row %
Count Row %
Returned No
Count Row % Not ReturnedReturned Yes
Consents Distributed*
*October 1, 2006- June 30, 2008
52%Count Row %
Total Offered
![Page 4: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/4.jpg)
Consents
4
We need to improve consent rates by applying what we have learned in Year 1 & 2
*Barriers, Challenges and Strategies Newsletter
![Page 5: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/5.jpg)
What Works & What Doesn’t*Barriers, Challenges and Strategies Newsletter
5
Strategies Proven to Work• Sending consent form home with begin-year registration• Sending consent from a classroom combined with prevention educationStrategies that are Not Effective• Mailing consents• Incentives
![Page 6: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/6.jpg)
Education + Mental Health = Higher Consent Rates
One-Sample Test
57.742 20925 .000 .137 .13 .14
145.418 10516 .000 .668 .66 .68
SOS
Consents
t df Sig. (2-tailed)Mean
Difference Lower Upper
95% ConfidenceInterval of the
Difference
Test Value = 0
Case Summaries
N
3111
5379
8490
382
1645
2027
3493
7024
10517
ConsentsNo
Yes
Total
No
Yes
Total
No
Yes
Total
SOS0
1
Total
Consents•Of consents returned:
•TeenScreen YES = 63.4%•SOS YES= 81.2%
![Page 7: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/7.jpg)
Project Totals*
1.Screenings Offered 25,6622.Consents Returned 12,232 3.Screenings 7,6584.Clinical Interviews 1,9365.Referrals Made 1,238
* Oct 2006 – June 2008
![Page 8: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/8.jpg)
8
Program Outputs and Outcomes
1.Offer Screening2.Obtain Parental Consent3.Conduct Screenings4.Conduct Clinical Interviews5.Refer for Counseling
![Page 9: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/9.jpg)
9
Consents by Program Type
TeenScree
n SOSJuvenile Justice
Consents Offered 85% 13% 2%Consents Distributed & Not Returned 79% 21% 0%Consents Returned Yes 63% 81% 99%Consents Returned No 37% 19% 1%
![Page 10: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/10.jpg)
Screens by Program Type
TeenScreen SOS
Juvenile Justice
TeenScreen
Screened 5,577 2,001 593
Screened Positive 29% 9% 48%
Screened Negative 71% 91% 52%
The TeenScreen Tool is more sensitive and finds more positive youth.
![Page 11: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/11.jpg)
11
Clinical Interview by Program Type
TeenScreen SOSJuvenile Justice
Screened Positive 1601 191 286Interviews Completed 1746* 198* 276Youth Identified for Emergency Care 27 0 2*More interviews completed than positive due to youth interviewed due to debriefing results
![Page 12: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/12.jpg)
12
Clinical Interview Results
TeenScreen and SOS programs completed 100% of clinical interviews of youth positive
TeenScreen programs identified 27 youth that needed emergency care
![Page 13: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/13.jpg)
13
Referrals by Program Type
TeenScreen % SOS %Juvenile Justice %
Total referred after clinical interview 1084 131 196 Completed Referrals 552 51% 41 31% 144 73%
![Page 14: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/14.jpg)
Crisis Referrals: 27 (2.3%)
18 Accepted by youth and parent 4 Accepted by parent not youth 1 Accepted by youth not parent 4 rejected by youth and parent
![Page 15: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/15.jpg)
Does Follow-up Aid Intervention?
1238 Referrals made, 990 received first follow-up call 680 First appointments made 576 First appointments kept (85%) 34 First appointments missed (5%) 68 Made Second appointments (9 missed the 2nd appt.)
Of 34 known missed first appointments, 5 Missed due to lack of interest 1 Missed due to provider not accepting insurance 3 Had transportation concerns 25 Did not report or reported other reasons
![Page 16: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/16.jpg)
Referrals Rejected: 116
72 (62%) received follow-up call 4 made and kept an appointment 68 made no appointment
44 (38%) not called
![Page 17: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/17.jpg)
Referrals by type of site
525 76.2% 160 23.2% 4 .6%
4 100.0%
3 100.0%
1 100.0%
7 87.5% 1 12.5%
31 75.6% 10 24.4%
221 97.8% 5 2.2%
Community Mental HealthCenter Outpatient Svs
Emergency Rm
Hospital-Based PsychClinic (outpatient)
Inpatient Unit
Mobile Crisis
Other
School-Based Srvcs
Referraltype
Count Row %
School
Count Row %
Juvenile Justice
Count Row %
Mental Health Agency
Type of site (numeric)
![Page 18: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/18.jpg)
18
System Motivation
Perception of Screening
Referral Climate
Screening Climate
Referral Completion
Parent
Youth
![Page 19: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/19.jpg)
19
Perception of Screening (Innovation Adoption)
![Page 20: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/20.jpg)
20
Perception of Screening Questionnaire
Stakeholders’ Perceptions of Compatibility Complexity Observability
![Page 21: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/21.jpg)
Stakeholder PerceptionsCorrelations
1 .159* .776** .149
.010 .000 .083
261 261 261 136
.159* 1 .411** -.110
.010 .000 .204
261 261 261 136
.776** .411** 1 .199*
.000 .000 .020
261 261 261 136
.149 -.110 .199* 1
.083 .204 .020
136 136 136 1164
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Relative advantage mean
Complexity mean
Observability mean
Appointment Yes
Relativeadvantage
meanComplexity
meanObservability
meanAppointment
Yes
Correlation is significant at the 0.05 level (2-tailed).*.
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 22: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/22.jpg)
22
Referral Climate
Youth Parent
![Page 23: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/23.jpg)
23
Referral Climate Items Provided choices and options Understood me Conveyed confidence Listened to me Encouraged questions Tried to understand how I see things
before making suggestions
![Page 24: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/24.jpg)
Referral Climate
Correlations
1 -.090 -.126
.376 .237
1164 98 90
-.090 1 .385**
.376 .000
98 180 115
-.126 .385** 1
.237 .000
90 115 150
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Appointment Yes
Mean - Youth RCQ
Mean - Parent RCQ
AppointmentYes
Mean -Youth RCQ
Mean -Parent RCQ
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 25: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/25.jpg)
Referral ClimateCorrelations
1.000 .261** .087 -.168** -.076
. .000 .112 .002 .170
180 115 180 180 180
.261** 1.000 .081 .093 -.055
.000 . .208 .151 .399
115 150 144 144 144
.087 .081 1.000 .019 .455**
.112 .208 . .673 .000
180 144 261 261 261
-.168** .093 .019 1.000 .354**
.002 .151 .673 . .000
180 144 261 261 261
-.076 -.055 .455** .354** 1.000
.170 .399 .000 .000 .
180 144 261 261 261
Correlation Coefficient
Sig. (2-tailed)
N
Correlation Coefficient
Sig. (2-tailed)
N
Correlation Coefficient
Sig. (2-tailed)
N
Correlation Coefficient
Sig. (2-tailed)
N
Correlation Coefficient
Sig. (2-tailed)
N
Mean - Youth RCQ
Mean - Parent RCQ
Relative advantage mean
Complexity mean
Observability mean
Kendall's tau_b
Mean -Youth RCQ
Mean -Parent RCQ
Relativeadvantage
meanComplexity
meanObservability
mean
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 26: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/26.jpg)
26
Referral Climate Findings
Lowest Rated Youth Item“The person who talked with me
showed confidence that I can make changes if I want to”
![Page 27: Evaluation Results](https://reader035.vdocuments.us/reader035/viewer/2022062217/56812daf550346895d92d9f5/html5/thumbnails/27.jpg)
27
Referral Climate Questioner Findings
Lowest Rated Adult Item“The person who talked with me
understands how I see things with respect to seeing a counselor”