preliminary findings from a randomized mixed-methods cross-over study to compare the performance of...
DESCRIPTION
2012 Summer Medical Editor's Meeting: Dominick Frosch, PhDTRANSCRIPT
![Page 1: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/1.jpg)
Preliminary findings from a randomized mixed-methods cross-over study to
compare the performance of DVD and web-based patient decision support
Dominick L. Frosch, PhD
Associate Investigator, PAMFRI Associate Professor of Medicine, UCLA
July 31, 2012
![Page 2: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/2.jpg)
Knowledge gaps in decision support
n The internet is increasingly becoming the primary medium for providing patients decision support.
n Few data are available to guide optimal design of internet decision support and ensure comparable effectiveness compared to video.
n A handful of studies have examined the efficacy of internet decision support, but only one study (2003) directly compared internet and video decision support.
![Page 3: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/3.jpg)
Review of Video and Internet Presentation
n 98% of participants assigned to the Video group viewed the tape.
n 54% of those assigned to Internet viewed the
entire presentation of 47 slides. 40% never reviewed any part of the presentation.
Frosch, Kaplan & Felitti, JGIM, 2003.
![Page 4: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/4.jpg)
PSA Knowledge
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Pre Post
VideoInternet
p<.001
![Page 5: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/5.jpg)
PSA Knowledge: Video & Internet Completers
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Pre Post
Video I-Net Comp.
![Page 6: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/6.jpg)
Research questions
n How do internet and video decision support compare in terms of: – Increases in patient knowledge – User acceptability – Decision quality
n What proportion of web-based decision support do individuals review and how much time do they spend with it?
![Page 7: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/7.jpg)
Study design
Figure 1. Research design Primary care
patients (N=90)
Complete baseline quantitative survey - 1
Review DVD
Review Health
Crossroads
Complete quantitative survey - 2
Review DVD
Review Health
Crossroads
Complete quantitative survey - 3
RA probes content selection
RA probes content selection
![Page 8: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/8.jpg)
Study measures
n Decision/disease specific knowledge n Quantitative user ratings of decision support n Decision quality n Role preferences n Click-stream (para) data on HC use n Think-out loud while using HC n In-depth interviews following each review of
decision support
![Page 9: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/9.jpg)
Study setting and sampling
n Palo Alto Medical Foundation in Silicon Valley, CA.
n Participants are active primary care patients at PAMF who have/are: – Diabetes – Benign Prostatic Hyperplasia – Overdue to consider colon cancer screening
n Participants are randomly selected from primary care panels and invited by mail/phone.
![Page 10: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/10.jpg)
Participant demographics
N=30
Age 64.6 (SD=12.9)
Gender 63.3% Male
Marital status 76.7% married or living with partner
Ethnicity 83.3% Caucasian 10.0% Asian 6.7% Latino
Education 80.0% College graduate or more
Household income 85.7% >$50,000
Health literacy score (range 0-8) 7.9 (SD=.35)
![Page 11: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/11.jpg)
How much do patients view on the Health Crossroads?
50.00% 51.00%
45.00%
54.00% 50.00%
36.00%
66.00%
46.00%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
All BPH CRC Diabetes
DVD first WWW first
![Page 12: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/12.jpg)
How much time do patients spend on the Health Crossroads?
25.2 26.7
16.6
32.2
21.0 17.6 17.9
27.4
33
45
31
24
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
All BPH CRC Diabetes
DVD first WWW first Video length
![Page 13: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/13.jpg)
How many videos in the Health Crossroads do patients click on?
1.5 1.4 1.8
1.4
2.5
0.8
5.0
1.8
0.0
1.0
2.0
3.0
4.0
5.0
6.0
All BPH (25) CRC (27) Diabetes (19)
DVD first WWW first
![Page 14: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/14.jpg)
Time spent watching videos in the Health Crossroads
1.6
1.3
2.0
1.4
2.4 2.6
3.1
1.6
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
All BPH CRC Diabetes
DVD first WWW first
![Page 15: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/15.jpg)
Reflections on using Health Crossroads
“I didn’t even look at the videos but just the presence of the videos seemed like it was getting in my way, it was just filling up space, and you know I didn’t come looking for a video…these days not knowing what’s going to happen when you click on a video, if you’re going to get an advertisement, and uh, and then you know, something you know a video, will take 10 minutes to say something that you can read in a minute.” MALE, 54, CRC "My reaction is to take the DVD only because everybody likes to watch a DVD, but the website is better. But see I was forced to read the website. If I had the DVD, I will actually put the DVD in and play it. I would go to [the website]; I don't know if I would do as intensive reading as I did here, um, um. There has to be a catch but I don't know what the catch is...There has to be follow-up. If there's no follow up, then they might just flip through [the website] and jot down and see a picture and read a line, but not really read the whole thing." MALE, 61, DIABETES
![Page 16: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/16.jpg)
Changes in knowledge - Combined
65.0%
82.0% 79.0%
56.0%
68.0%
77.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
Baseline First review Second review
All
DVD first WWW first
![Page 17: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/17.jpg)
Changes in knowledge - BPH
40.0%
73.0%
63.0%
43.0% 48.0%
60.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
Baseline First review Second review
BPH
DVD first WWW first
![Page 18: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/18.jpg)
Changes in knowledge – CRC Screening
78.0%
95.0% 98.0%
55.0%
80.0%
92.0%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
Baseline First review Second review
CRC
DVD first WWW first
![Page 19: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/19.jpg)
Changes in knowledge – Diabetes
80.0% 80.0% 80.0%
71.0%
73.0%
78.0%
66.0%
68.0%
70.0%
72.0%
74.0%
76.0%
78.0%
80.0%
82.0%
Baseline First review Second review
Diabetes
DVD first WWW first
![Page 20: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/20.jpg)
Understanding how patients use HC
“It wasn’t too busy, you know um, I didn’t get bored, I could easily find something you know, some websites are too much you know, this was simple here’s all the different tests you can take and these are all the procedures, I liked the chart you know the summary thing cause you forget and with the summary then you don’t have to click from page to page cause you have it all right there so that’s probably what I liked best was the summary chart.” FEMALE, 52, CRC "I liked [the website] better because you could read something if weren't sure about it you can go back and look at it." "In any given amount of time, you can absorb just so much information. So the DVD is a little too fast, but that can't be helped. So the website is a very good supplement." MALE, 85, BPH
![Page 21: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/21.jpg)
Predictors of Health Crossroads knowledge gain
Variable R2 Change Beta Age .10 -.17 Education .21 .28 Proportion of pages viewed
.05 .18
Number of videos clicked .10 .40
Cumulative R2 = .46
![Page 22: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/22.jpg)
Which medium do patients prefer?
21.4%
64.3%
14.3%
43.8%
37.5%
18.7%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
Prefer video Equal ratings Prefer Health Crossroads
DVD first WWW first
![Page 23: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/23.jpg)
Preferences for Health Crossroads
"Maybe I shouldn't say I would never use the DVD. If [my doctor] gave me the DVD as a package and said view the DVD, you know, and then go to the website, you know, I would do that. As I was watching [the DVD], I was thinking that I would never go back to this as a tool. I would watch it once...and be done with it." MALE, 66, DIABETES "I am not an internet person. I don't sit there and look at this stuff all day long, but I can read it. I can go back to it. I can access it immediately with a mouse, and get what I want. [I] don't have to watch the whole thing again to figure out where in the picture it was and I have to go find this out. So I like that first website very much." MALE, 85, DIABETES “The DVD you didn’t have any choice you just had to watch whatever came up but this one you could choose what you wanted to see.” FEMALE, 64, DIABETES "Everybody sits at a computer and it's just so easy you know, but you get a DVD in mail and go "Oh I want to watch that" and you throw it on your desk, but you never do it. But everyone sits at a computer and it's just "click" and we're there." FEMALE, 52, CRC
![Page 24: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/24.jpg)
Conclusions
n Data appear to confirm that DVD may be the superior option, for ensuring that patients are exposed to all necessary information.
n Knowledge gains appear to be greater for DVD.
n More viewing of videos embedded in HC appears to drive greater knowledge gains.
n Some suggestions that having both mediums is valuable for patients – start with DVD, but use HC to reinforce knowledge gains.
![Page 25: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/25.jpg)
Limitations and next steps
n Findings are preliminary, based on 1/3 of ultimate sample of 90 participants.
n Study is exploratory, not designed or powered for null-hypothesis significance testing.
n Non-naturalistic laboratory study, but qualitative data give us insight into how people might use decision support at home.
n Participant recruitment is currently 85% complete. Analysis will be complete by year-end.
![Page 26: Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support](https://reader030.vdocuments.us/reader030/viewer/2022020207/554b899fb4c90574668b4fde/html5/thumbnails/26.jpg)
Questions and suggestions?