if you build it, will they come? · 2019-06-05 · if you build it, will they come? moving...

@lauradesveaux @NoahIvers

If you build it, will they come?

Moving recipients from apathy to action

Laura Desveaux PhD, PT

[email protected]

lauradesveaux

Noah Ivers MD, PhD

[email protected]

NoahIvers

#AF2019

mailto:[email protected]

mailto:[email protected]

Agenda#AF2019

• Introductions

• Review of theory

• Understanding failure and success

• Engaging end users

• Group activity → Practical strategies

• The role of context

• Group activity → Supportive coaching

#AF2019@lauradesveaux @NoahIvers

RECOGNIZE OUR ASSUMPTIONS

3


BEFORE WE BUILD IT, WE NEED TO KNOW HOW IT WORKS

4

A&F Best Practices

#AF2019@lauradesveaux @NoahIvers 5

Brown et al, Imp Sci 2019; 14:40.


Learning from failure

2 x 2 FACTORIAL TRIAL

Top 25%

- Frame

Mean

- Frame

Top 25%

+ Frame

Mean

+ Frame

Does the design of the report improve performance?

Learning from Failure


RESULTS OF LTC FACTORIAL

TRIAL?

9


…BUT ENGAGEMENT WAS SUBOPTIMAL

10



ENGAGEMENT

11

60%REGISTERED

38%

REACH23%

14%

33%

5%

Primary Care LTC


ENGAGEMENT DOESN’T JUST HAPPEN

12

A&F Best Practices

CoherenceCognitive

Participation(Collective)

ActionReflexive Monitoring


Strategies to Increase Engagement

Understand workflow

• Where is the drop-off?

• Where can you intervene?Consider the

evidence

• What has worked?

• How was it operationalized?Seek user feedback

• What do they say?

• How do they react?



Select techniques and draft content

Recruit users and non-users

Co-create new content

Pretest content

Finalize operational details

Priority is to gauge user response vs. feedback

Bravo et al, JMIR Hum Factors 2018; 5:3.

1

2

3

4

5



Remember:

User preference ≠ Scientific evidence

Consider context



Vaisson et al, Curr Oncol 2019 [In press]



Minor changes can make a small difference in engagement

Anticipated Regret

(n=2642)

Material Incentive

(n=2777)

Problem Solving

(n=2812)

Primary outcome: Any SAR logins during trial

SAR accessed (%) 21.8% 20.3% 19.5%

Adjusted RR [ 95% CI ] 1.07 [0.97-1.18] 0.95 [0.87-1.05] 0.87 [0.79-0.96]

Secondary outcome: Total number of SAR unique log-ins

Adjusted RR [ 95% CI ] 1.09 [0.95-1.25] 0.89 [0.77-1.02] 0.81 [0.71-0.93]

Secondary outcome: Adherence to cancer screening

Breast ca screening aRR [95% CI] 1.000 [0.997-1.002] 1.000 [0.997-1.003] 0.998 [0.995-1.001]

Cervical ca screening aRR [95% CI] 1.002 [0.998-1.003] 1.000 [0.998-1.002] 1.003 [1.001-1.006]

Colon ca screening aRR [95% CI] 1.000 [0.998-1.002] 1.000 [0.998-1.002] 1.000 [0.998-1.002]


Interactive Activity

What makes you engage?



Group Activity:

Designing Feedback


Interactive Activity1. Pair up with someone you don’t know

2. Explain your job & responsibilities (1 min each)

3. Guess feedback parameters for your partner (5 min)

→ Topics, Delivery, Source

4. Write down your ideal feedback parameters (5 min)

5. Discuss alignment and rationale

6. Would you engage with the feedback? Why?

7. Identify themes (if able)


What we’ve learned

How do you best deliver feedback?

It depends.



What is meaningful engagement?


• Goal must be clearly stated

• Utility must align with recipient goals

• Perceptions of feedback influence engagement

• Approach to practice influences perceptions of

feedback

MEANINGFUL CONTENT >> A STRONG VISUAL




ESTABLISH MEANINGFUL CONTENT FIRST

…THEN CONSIDER USABILITY AND RELATIVE ADVANTAGE


Brown et al, Imp Sci 2019; 14:40.



Threats to meaningful engagement:

The data sucks

My patients are different

I don’t know what to do with the data

1

2

3

How do primary care physicians engage with A&F?


Tenants of Feedback Intervention TheoryHysong et al, BMJ Qual Saf 2017; 26:4.


Bridging the gap:

Facilitated Feedback


Facilitated Feedback

R2C2 → an evidence-based model on delivering

effective feedback to physicians

29

Rapport Building

Explore Reactions

Explore Content

Coach for Change

Sargeant et al, Acad Med 2015; 90(12).



Group Activity:

Feedback Coaching



1. Get back in your pairs

2. Designate one person as the coach and the other as

the recipient (who is now a family doctor)

3. Discuss the following feedback


High Dose Opioids Antibiotic prescribing rate

You Average

20

40

50

30

10

%

High Dose Opioids Antibiotics


Group Facilitation

41

Cooke et al, Imp Sci 2018; 13:136.

Precursors

Relationship building

Question choice

Usability


Group Facilitation

42

Cooke et al, Imp Sci 2018; 13:136.

• Navigate through reactions to data

• Understand their data

• Create a plan for change

Note: A “coaching-oriented approach” with prompts is essential


Positive Deviance Approach

• Identify organizations or individuals that are performing

well

• Highlight key strategies/practices that enable high

performance

• Share strategies with feedback recipients

43

Lawton et al, BMJQS 2014; 23.


‘Beneficent Persuasion*’: How will data → QI?

Motivation (goal-commitment, social comparison, regret)

i.e., address outcome expectancy with relative risk reduction rather than number needed to treat

Cochrane 2011 Mar 16;(3):CD006776

Capability (self-efficacy, graded-entry, coaching)

i.e., connect trigger to desired action and plan for barriers

Psychol Health 2005, 20:143-160

Opportunity (addressing barriers, co-interventions)

i.e., make the desired action easier or no effort at all

NEJM. 2007 Sep 27;357(13):1340-4

*Ann Fam Med. 2010 May; 8(3): 260–26445


Design must

be fit for

purpose


Be clear

about the

purpose

PURPOSE FUELS APPETITE


Help people

understand

their score and

make plans to

improve


Be realistic

about your

goals


Help people

improve

together by

changing

processes, not

trying harder

if you build it, will they come? · 2019-06-05 · if you build it, will they come? moving...

Documents