feasibility and effectiveness of a wraparound-specific...
TRANSCRIPT
![Page 1: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/1.jpg)
Wraparound Evaluation & Research Team 2815 Eastlake Avenue East Suite 200 ⋅ Seattle, WA 98102
www.depts.washington.edu/wrapeval
FidelityEHR 2100 Calle de la Vuelta, C-202 ⋅ Santa Fe, NM 87505
www.fidelityehr.com
Feasibility and Effectiveness of a Wraparound-Specific
Electronic Health Record Eric J. Bruns, PhD Alyssa N. Hook, BS Isabella Esposito, BS Elizabeth Parker, PhD
University of Washington Wraparound Evaluation & Research Team
Kelly L. Hyde, PhD FidelityEHR
30th Annual Research & Policy Conference on Child, Adolescent and Young Adult Behavioral Health
March 6, 2017
![Page 2: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/2.jpg)
Acknowledgments
• FidelityEHR – Founder & CEO Kelly L. Hyde, PhD – Formerly Social TecKnowledgy – Mission:
• “To support empowerment, engagement and healthy outcomes through innovations in technology for families and communities.”
– TMS-WrapLogic rebranded in January 2016 • This study funded by the National Institute of
Mental Health (R42-MH95516; PI Bruns)
![Page 3: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/3.jpg)
Research Hypothesis: Health Information Technology (HIT) can facilitate efficiency, fidelity, positive outcomes
![Page 4: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/4.jpg)
NIMH Small Business Technology Transfer (STTR) Study
Three phases: Phase 1: Program elements of FidelityEHR Phase 2: User Experience Testing: Determine if
FidelityEHR is feasible and usable Phase 3: Determine if transitioning from paper to
FidelityEHR impacts Wraparound implementation by providers and outcomes for youth and families
![Page 5: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/5.jpg)
FidelityEHR Highlighted Features
• Secure, web-based login • User friendly interface • Customizable Workflows • High Fidelity Wraparound-based Plan of Care • Contact/Progress Notes, Critical Incident Tracking • Progress Monitoring plus Assessment Builder • Secure Messaging and Scheduling • Report Builder for program and system decision support • CANS Builder, Algorithms, T-COM Reports
![Page 6: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/6.jpg)
FidelityEHR Record Navigation and and Workflow
![Page 7: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/7.jpg)
FidelityEHR Plan of Care
![Page 8: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/8.jpg)
FidelityEHR Core Assessments
![Page 9: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/9.jpg)
FidelityEHR CANS Assessment
![Page 10: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/10.jpg)
FidelityEHR CANS TCOM REPORTS
![Page 11: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/11.jpg)
Research Aims
• Is FidelityEHR feasible, acceptable, and contextually appropriate in the “real world” of wraparound implementation?
• Comparing care coordinators randomly assigned to EHR vs. continued services as usual (SAU), how does FidelityEHR affect: – Wraparound supervision? – Wraparound practice? – Teamwork and Alliance? – Wraparound Fidelity? – Parent Satisfaction?
![Page 12: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/12.jpg)
Theory of Positive Impact EHR Components
•Information management: e.g., family, team, plan, providers, services, billing
•Fidelity support: e.g., Workflow pane, reminders, alerts, supervisor reports
•Standardized assessment: clinical alerts, treatment recommendations
•Feedback of information via dashboard reports on fidelity, services, progress, outcomes
•Supervisor, manager, administrative reports: e.g., services, costs, satisfaction, fidelity, outcomes, placements
Impact on Staff/Teams
•Availability of information
•Transparency and efficiency
•Better collaboration and teamwork
•Adherence to elements of high-fidelity Wraparound
•More frequent progress review
•Decision-making based on objective data
•More focused, directive, data-informed supervision
•Staff more satisfied and self-efficacious
•Admin/manager-level accountability
Paths to Family Outcomes
• Goal clarity • Team
communication and consensus
• Better problem-solving
• Greater treatment alliance
• Family and team better engaged, hopeful, and satisfied
• Shorter self-correction cycles
• More effective treatment
• Reduced staff turnover
Outcomes
• Families retained in services
• Greater social support
• Greater progress and reduction in top problems
• Reduced youth emotional and behavioral problems
• Improved youth functioning
• Reduced out of home/ community placement
• Reduced costs to systems
![Page 13: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/13.jpg)
Staff and family data were collected from two agencies
Site 1 • Wraparound organization in
rural area in SE US • Staff in study: 3 Supervisors 26 Facilitators
Site 2 • Agency providing multiple
services including traditional Wraparound and other Wraparound-based treatment tracks in a mixed urban/rural region of a Midwestern state.
• Staff in study: 2 Supervisors 5 Facilitators
![Page 14: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/14.jpg)
Study Flow (CONSORT Diagram)
![Page 15: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/15.jpg)
Facilitator Demographics
EHR n = 18 (%)
SAU n = 13 (%)
Male 9 (39%) 2 (15%)
Female 11 (61%) 11 (85%)
White 12 (67%) 10 (77%)
African American 5 (28%) 2 (15%)
Hispanic 0 1 (8%)
Other 1 (6%) 0
![Page 16: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/16.jpg)
RESULTS: Usability
![Page 17: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/17.jpg)
63.9
48.1 50.6 54.6 58.3
0
10
20
30
40
50
60
70
80
90
100
Field-Based TestingSept. 2015 (n=7)
Implementation Wave 1Feb. 2016 (n=12)
Implementation Wave 2June 2016 (n=12)
Implementation Wave 3Jan. 2017 (n=12)
EHR usability ratings in marginal range but slowly increased over time
SOURCE: Bangor, A., Kortum, P., & Miller, J. (2009). An empirical evaluation of the system usability scale.
• The System Usability Scale (SUS) provides a quick and easy understanding of a user’s subjective rating of a product’s usability
• 12 facilitators completed the SUS over the course of one year (Site 1) • 3 facilitators completed the SUS at 6 months only (Site 2)
Acceptable usability
Marginal usability Low: 50-62
High: 63-70
Unacceptable usability
Site 1 (n=12; 4 waves of UX data)
Site 2 (n=3; 1 UX assessment)
![Page 18: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/18.jpg)
• The distribution indicates more than half of the users (61%) rated FidelityEHR with Marginal or Acceptable usability after 6 months of use
0
1
2
4
6
4
1
0 0
1
2
3
4
5
6
7
0-20 21-30 31-40 41-50 51-60 61-70 71-80 81-100
Num
ber o
f Use
rs
Distribution of SUS Scores for both agencies
SUS Score (n=18)
The distribution of scores indicate a range of opinions on usability
Acceptable usability
Marginal usability Low: 50-62 High: 63-70
Unacceptable usability
![Page 19: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/19.jpg)
Facilitators newly hired and trained on system report higher usability ratings
• Facilitators trained on FidelityEHR as part of their onboarding process report higher ratings for usability than facilitators in the research study
Acceptable usability
Marginal usability Low: 50-62
High: 63-70
Unacceptable usability
62.2 56.8
52.2
0
10
20
30
40
50
60
70
80
90
100
Newly-Hired Facilitators(n=43)
EHR Group Facilitators(n=14)
SAU Group Facilitators(n=8)
![Page 20: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/20.jpg)
Staff report EHR aligns well with Wraparound service setting
• System Acceptability & Appropriateness scale (SAAS) gauges satisfaction, utility, and fit with service context of technology
17%
17%
6%
67%
72%
78%
39%
61%
67%
72%
17%
11%
17%
61%
39%
33%
28%
0% 20% 40% 60% 80% 100%
Satisfied with current version
Satisfied with ease of use
Satisfied with content of system
Comfort interacting with system
Compatible with service setting
Fits with treatment modality
Fits with approach to service delivery
Acce
ptab
ility
Appr
opria
tene
ss
Staff (n=18) rate the degree to which they agree with each item at 6 months
Not at all Moderately Extremely
![Page 21: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/21.jpg)
Qualitative feedback: Strengths of the system
• “Can quickly pull reports” • “Can more easily make changes on the fly” • “Better direction of where to go in supervision” • “More aware of looking at needs and progress” • “Great to be able to work remotely” • “Families are better at understanding their outcomes” • “Overall, love the system compared to the old one… Keeps us
focused on particular needs & outcomes, more organized with monitoring”
• “Tasks flow from strategies which link to needs” • “System is overall good… just need to work out kinks”
![Page 22: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/22.jpg)
Qualitative feedback: Needs for system improvement
• “Contact logs take a lot of clicks… and we use it the most” • “Team meeting reminders aren’t consistent” • “Core assessments don’t all display in supervision” • “Plan of Care is too long – can’t just print one page (e.g.,
assessments) … need POC report builder” • “Tedious to add and delete strategies” • “Can’t sort contact logs by dates”
![Page 23: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/23.jpg)
Qualitative feedback: Change is hard, and transition to EHR must be done strategically
• “First weeks were hard – challenging to have conflicting answers from supervisors… hard because things weren’t sorted out”
• “Hard to learn all at once – had a lot of workarounds” • “Would have been better to have earlier trainings, and a better
user’s manual” • “Took a long time to transition… couldn’t breathe til March” • “EHR was added to the CAFAS, Suicidal Ideation/BX assessment,
assessments asking families at EVERY team meeting how they do and how they feel, Protective Risk Factors Survey, etc. there is too much… we are overwhelmed with requirements”
• “Starting to get the hang of it but study data will be impacted because we weren’t using the system to its maximum capacity … just trying to get by”
![Page 24: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/24.jpg)
RESULTS: Changes in Practice
![Page 25: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/25.jpg)
Supervisors report small differences in supervision activities by group
• After six months of FidelityEHR use, Wraparound Supervisors report how much time they spent on certain activities in supervision with Facilitators
• Reviewing Plans of Care and Skills Coaching & Training take up approximately one-third of supervision
5.0%
5.8%
8.1%
8.1%
11.5%
5.8%
6.9%
9.6%
9.6%
15.8%
13.8%
5.0%
6.1%
6.6%
7.2%
7.7%
8.2%
10.1%
10.1%
10.9%
12.3%
15.7%
0% 5% 10% 15% 20%
Facilitator's Professional Role
Case Conceptualization
Supervisory Relationship
Crisis Assessment/Management
Administrative Tasks
Facilitator Personal Support
Reviewing Progress Toward Needs
Natural Support Engagement
Youth & Family Engagement
Skills Coaching & Training
Reviewing Plans of Care
EHR SAU
![Page 26: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/26.jpg)
Supervisors report more time reviewing progress toward needs for EHR staff (p<.01)
• EHR group spends more time reviewing progress toward needs compared to the SAU group
10.1%
6.9%
0%
3%
6%
9%
12%
Reviewing Progress Toward Needs
Perc
enta
ge o
f tim
e sp
ent i
n su
perv
ision
EHR SAU
![Page 27: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/27.jpg)
Facilitators report shifts in practice throughout the course of EHR use
• The Current Assessment Practice Evaluation – Revised (CAPER) was administered to facilitators on a biweekly basis for eight months to assess the degree to which their practice was influenced by reviewing assessment data
![Page 28: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/28.jpg)
Attitudes toward standardized measures higher for SAU group at 6 months
• At the 6-month follow-up, SAU facilitators reported improved attitudes toward the reliability and validity of standardized measures and had more positive opinions about using standardized measures compared to EHR group
Positive Attitudes
Negative Attitudes
* Indicates item is significantly different; p<.05
2.8 3.4 3.2
2.9 3.4
3.1 3.0 3.3 3.2
2.8
3.6 3.3
0
1
2
3
4
5
BCJ PQ PC BCJ PQ* PC*
EHR (n = 18) SAU (n = 13)
Baseline 6 Months
![Page 29: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/29.jpg)
Fidelity to Wraparound • Caregivers completed the WFI-EZ after four months of Wraparound services • No difference found for total fidelity • Marginal difference (*p=.1) in favor of EHR found for Strength/Family Driven
69.7
57.9
71.7 74.4 79.7
69.4 66.1 62.4
73.6 74.7 71.4 68.8 67.8 65.6 73.8 75.3 77.6
72.0
0102030405060708090
100
EffectiveTeamwork
Natural Supports Needs-Based Outcomes-Based Strength &Family Driven
Total Fidelity
Fide
lity
Scor
e
EHR (n=42) SAU (n=23) National Mean
*
![Page 30: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/30.jpg)
Facilitator satisfaction with Wraparound practice is high
• Clinician Satisfaction Index measured general feelings about using the Wraparound process
• Both groups report high job satisfaction; scores in SAU group declined slightly between baseline and six months
58.29 62.15
58.29 59.08
0
10
20
30
40
50
60
70
EHR (n=17) SAU (n=13)
Baseline 6 Months
High Satisfaction
Low Satisfaction
![Page 31: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/31.jpg)
RESULTS: Impact on Youth & Family Experiences
![Page 32: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/32.jpg)
No significant differences in Caregiver Satisfaction
• Caregivers are generally satisfied with services
High Satisfaction
Low Satisfaction
1.2
0.9 1.0 1.1
1.4
0.9 1.0
1.2 1.4
1.1 1.2 1.2
-2.00
-1.00
0.00
1.00
2.00
C1. Satisfied withWraparound
C2. Satisfied withyouth's progress
C3. Family madeprogress toward needs
C4. More confident aboutability to care for youth
EHR SAU National Mean
![Page 33: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/33.jpg)
Caregivers in both groups report positive working alliance with facilitators
• Working Alliance Inventory (WAI) quantifies the degree to which team members work collaboratively and connect emotionally with items such as, “My Facilitator and I trust one another”
SOURCE: Hanson, W. E., Curry, K. T., & Bandalos, D. L. (2002). Reliability generalization of working alliance inventory scale scores.
High Alliance
Low Alliance
23.8 23.6 25.4
24.2 24.4 26.0 26.6 25.7
0
4
8
12
16
20
24
28
Goal Task Bond Total Working Alliance
EHR (n=42) SAU (n=23)
![Page 34: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/34.jpg)
Caregivers report a positive team climate for both groups
• The Team Climate Inventory (TCI) assesses team interactions and performance with items such as, “We have a ‘we are in it together’ attitude”
• Both groups report positive team climate
Positive Team
Climate
Negative Team
Climate
17.1 17.3
13.0 12.9
17.6 18.0
13.1 13.0
0
5
10
15
20
Vision Participative Safety Task Orientation Support for Innovation
EHR (n=42) SAU (n=23)
Max score: 15
Max score: 20
![Page 35: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/35.jpg)
DISCUSSION & IMPLICATIONS
![Page 36: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/36.jpg)
Successful EHR Implementation is becoming a science in and of itself
• Studies of successful EHR implementation have consistently recognized the importance of thoughtful planning and training in the implementation process: – Timing training to coincide with implementation – Targeting training to users’ needs – Providing knowledgeable on-site support
![Page 37: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/37.jpg)
Eight Domains of Successful EHR Implementation
![Page 38: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/38.jpg)
Limitations Borne of Implementation and Study Challenges
• Rapid training and implementation cycles • Staff-level randomization within
supervisors/programs – Disruptions to routines – Supervisors having to supervise differently
depending on staff
• System still being improved in response to feedback
![Page 39: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/39.jpg)
Discussion: User Experiences
• Staff report EHR aligns with Wraparound service setting • Marginal usability reported overall • User opinions ranged from low to high
– Typical patter of “eager adopters” vs “laggers” – Staff saw strengths of the EHR, but also experienced multiple
“kinks” during study to be addressed by development team • Usability scores increase over time
– Those who experienced the software as “part of their job” or trained as part of onboarding were more satisfied
– Those who had to “change practice” and/or do different things from their colleagues less satisfied
![Page 40: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/40.jpg)
User confidence levels over time during EHR adoption
Confidence and Usability starts high, typically declines, then increases again
![Page 41: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/41.jpg)
EHR Implementation Confidence and Adoption Similar to Usability Ratings Found in Study
63.9
48.1 50.6 54.6
020406080
100
Field-BasedTesting
ImplementationWave 1
ImplementationWave 2
ImplementationWave 3
![Page 42: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/42.jpg)
Discussion: Impact on Practice & Implementation
• Few significant findings: – EHR group spends more time reviewing progress toward
needs compared to the SAU group – EHR group had marginally better fidelity in one area
(Strengths and Family Driven) – Both groups demonstrated significantly improved use of
assessment and feedback • Side effect of investment in EHR agency-wide in these sites?
– SAU facilitators report more positive opinions about using standardized measures at 6 months
![Page 43: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/43.jpg)
Implications
• Rigorous study provided opportunity for substantial improvements FidelityEHR System – Staff viewed system as appropriate to wraparound context,
but change was hard and improvements were needed • Modest but positive shifts in some proximal outcomes
(supervision, use of data, fidelity) and lack of negative impact on satisfaction, teamwork, staff job satisfaction could be viewed favorably given the challenges
• Wraparound-specific EHR in wraparound worthy of continued development and research
![Page 44: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/44.jpg)
Discussion: Next Steps
• Complete analysis on youth and family outcomes
• More rigorous grant with: – Updated FidelityEHR system featuring revamped
“responsive design” – More time / resources for implementation
support – Longer follow-up
![Page 45: Feasibility and Effectiveness of a Wraparound-Specific ...depts.washington.edu/wrapeval/sites/default/files...Secure, web-based login • User friendly interface • Customizable Workflows](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f540cdbcc2088536c0f1655/html5/thumbnails/45.jpg)
Wraparound Evaluation & Research Team 2815 Eastlake Avenue East Suite 200 ⋅ Seattle, WA 98102
www.depts.washington.edu/wrapeval
FidelityEHR 2100 Calle de la Vuelta, C-202 ⋅ Santa Fe, NM 87505
www.fidelityehr.com
For more information:
[email protected] [email protected] www.wrapinfo.org www.FidelityEHR.com