technology-based education and intervention tools in medical education: lessons and methods for the...
TRANSCRIPT
Technology-based education and intervention tools in medical
education: Lessons and methods for the University community
Scott M. Strayer, MD, MPHAssociate Professor
Department of Family MedicineUniversity of Virginia Health System
My Colleagues and Co-Authors
Sandra Pelletier, PhDCrista Warniment, MDKaren Ingersoll, PhDSteve Heim, MDJohn Schorling, MD, MPH
Preventable Causes of Death
The most common causes of disease, disability, premature death, and health care burden in the US can be directly attributed to 4 health risk behaviors: Smoking tobacco Risky use of alcohol Unhealthy diet Physical inactivity
1. McGinnis JM, Foege W. Actual causes of death in the United States.JAMA. 1993;270:2207-2212.2. Mokdad AH, Marks JS, Stroup DF, Gerberding JL. Actual causes ofdeath in the United States, 2000. JAMA. 2004;291:1238-1245.
Health Behavior Counseling
Obesity In surveys of patients who had a routine checkup with a
physician in the past 12 months, only 14% of respondents reported receiving advice from their physician to lose weight, even though approximately 34% of the sample was classified as overweight and 19% as obese (Sciamanna et al., 2000).
Moreover, only 16% of overweight individuals and 40% of obese individuals reported receiving advice from their physician to lose weight during their checkup within the past twelve months (Loureiro, 2006).
Health Behavior Counseling
Alcohol Use Only 23% of binge drinkers are counseled about
alcohol use (Am J Prev Med 2003;24(1):71–74)
A survey of family physicians and internists found that only 64.9% of respondents screened 80-100% of their patients for alcohol abuse or dependence during the initial visit and a mere 34.4% screened that many patients during an annual visit
Only 20% said treatment resources for early problem drinkers was adequate
72% preferred not to counsel patients themselves (Spandorfer & Turner 1999)
Health Behavior Counseling
Smoking Cessation Smoking cessation counseling only occurs at
23 to 46% of primary care visits Only 35% of physicians assist with smoking
cessation attempts Less than 10% arrange follow-up for smoking
patients
• Thorndike AN, Rigotti NA, Stafford RS, Singer DE. National patterns in the treatment of smokers by physicians. JAMA 1998; 279:604-608
• Lewis CE, Clancy C, Leake B, Schwartz JS. The counseling practices of internists. Ann Intern Med 1991; 114:54-58.
• Goldstein MG, DePue JD, Monroe AD, et al. A population-based survey of physician smoking cessation counseling practices. Prev Med 1998; 27:720-729.
Barriers
Physicians Competing demands Conflicting recommendations Lack of training
Patients Lack of knowledge Fear of discomfort Cost
Office Poor reimbursement Lack of systems
Opportunities for Intervention
Most people visit a primary care doctor about three times per year.
Even 2-3 minute interventions are effective, especially when followed up with telephone, e-mail, nurse calls, referrals, 1-800 numbers, etc.
Many primary care providers provide 2-3 minute health promotion/behavior interventions at every outpatient visit.
Stange, KC, Woolf, SH, Gjeltema K. One minute for prevention: The power of leveraging to fulfill the promise of health behavior counseling. Am J Prev Med, 2002; 22:320-323.
Integrating the Behavioral Theories
Strayer SM, et al. Development and evaluation of an instrument for assessing brief behavioral change interventions. Patient Educ Couns (2010), doi:10.1016/j.pec.2010.04.012
Research Design
RCT with 3rd year family medicine clerkship students All students received smoking cessation workshop based on 5 A’s, Stages of Change and
MI Randomization by paired rotation blocks to receive either a reminder card or a handheld
computer tool - the Educational Smoking Mobile Intervention Tool (E-SMOK-I.T.)
Students assessed pre and post study for smoking cessation counseling knowledge and self-reported behavior and comfort using a previously validated survey instrument
Students assessed for appropriate smoking cessation counseling using video-taped standardized patient interview using an instrument that was developed to assess combined smoking cessation counseling interventions
2 independent observers assessed key smoking cessation counseling activities using videotapes and rating form
Focus groups held with E-SMOK-I.T. tool users post clerkship
Study Flow Diagram
Educational Smoking Intervention Tool (E-SMOK-I.T.)
Main Screen Assessment Screen
E-SMOK-I.T. (cont).
Assist Screen Arrange screen
Analysis
Frequency and descriptive analysesInter-rater reliability for videotape
assessmentPre/post survey analysis using mixed
ANOVASmoking cessation counseling skills using
Independent t-tests
Results
116 students completed the study and had complete video observations (N=63, control, N=53 intervention).
Mean age for participants was 26 and 52% were male.
62% reported handheld computer literacy as intermediate on a 3 point scale.
Average Inter-rater reliability was 0.82 (Intraclass correlation)
Results
Overall smoking cessation counseling behaviors, knowledge and comfort increased among all participants (p<.001)
Paper-based group performed better than the handheld tool group (69% vs 62%; t=2.318, p=.022) at end of clerkship
No difference between groups at end of academic year (61% vs. 59%; t=.621, p=.538)
Focus groups of each rotation block revealed several possible reasons for results.
Pre-post Survey Results
Knowledge pre/post increase 0.71 (F=15.54, p<0.001)Behavior pre/post increase 7.026 (F=194.45, p<0.001)Comfort pre/post increase 5.93 (F=163.620, p<0.001)
Knowledge Behavior Comfort0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
9.00
11.98
6.95
9.71
18.97
12.89
Pre Post
Me
an
Su
mm
ary
Sc
ore
Su
rve
y S
ub
ca
t-e
go
rie
s
Counseling MI Spirit Retention0
10
20
30
40
50
60
70
69
60 6162
51
59
Paper-Based
ESMOK-I.T
Me
an
Pro
po
rtio
n C
orr
ec
t C
ou
ns
elin
g
Sk
ills
Counseling** p<.022, MI Spirit** p=.004, Retention p=.538
Smoking Cessation Counseling Skills
Focus Group Themes
Discomfort using at point of care or patient not receptive: “I think I might have used it more if I were more comfortable
with all the other aspects of being in that room.”
“… but the patients were either like I don’t want to quit or they knew what they needed.”
Forgot or Gossip about tool: “I was afraid of it … I heard that there were problems with it
with info-retriever I was afraid about that”
Focus Group Themes
Would have used if more review / training: “It might be useful if I spent more time learning how to use it.”
Not enough time, smoking not priority: “The times that it really gets brought up it is a long
appointment you don’t have enough time to spend on smoking”
Tool used as educational reference “I did like how you could go to related things inside the
program. But mostly I went through it to review it.”
“Looked at it once or twice but not with patient”
Conclusions
Smoking cessation counseling by 3rd year medical students was improved using a combination of workshop and a supplementary reference tool.
Our hypothesis that the point of care tool would enhance skills significantly did not occur (unlike with our studies in practicing physicians)
Barriers specific to the 3rd year training period need to be addressed in order for a point of care tool to be effective in this setting
SBIR Phase 2 Contract
Phase II Research Aims
Conduct user-centered design evaluations to validate design and usability of new functionality
Assess the ability of QuitAdvisorMD to influence smoking cessation as measured by
The increase in clinician initiated smoking cessation counseling The increase in ability of clinicians to provide appropriate smoking
cessation counseling based on the PHS guidelines (and in particular, physician performance of the 5 A’s of: asking patients about smoking status, advising patients to quit smoking, assessing patient readiness to quit smoking, assisting patients with their quit attempts or in giving motivational interventions if they are not ready to quit smoking, and arranging follow-up for all patients who are smokers)
The increase in patient quit attempts.
Cross-study Themes
Content is a valued reference/resource (AMIT, QA, ACS)
Users liked the software as an educational tool (AMIT, ACS)
Need more practice and training (ACS, AMIT, QA)
Discomfort using computers with patients (AMIT, ACS)
Lack of time (ACS, QA, AMIT)
Conclusions
The content for computer-assisted counseling tools was appealing to all levels of learners
Many users see this software as an educational tool
All levels of learners requested more training and practice
Discomfort using computers with patients affects learners and even some practicing physicians
Lack of time for counseling is a persistent barrier
Topics for Discussion
Should health information technology tools (e.g. electronic medical records) be evaluated with learners as well as practitioners for official certification?
What are the best methods for evaluating health information technology tools with learners?
How would you feel as a patient if you were being interviewed by a medical student or resident using one of our tools described in our studies?