RFT 2003-02 Evaluation of clinical interventions in community pharmacy
Final Report
This project was funded by the Australian Government Department of Health and Ageing as part of the Third Community Pharmacy Agreement Research and Development Program, which is managed by The
Pharmacy Guild of Australia
2
The Research Team
• Gregory Peterson• Peter Tenni • Helen Kruup• Omar Hasan• Brita Pekarsky• James Reeve • Michael Roberts • Roger Rumble • Julie Stokes
3
RFT 2003-02: Evaluation of clinical interventions in community pharmacy
• Clinical Intervention– Where a pharmacist identifies, or is presented
with, an actual or potential drug related problem and he or she recommends an action to be taken to resolve or prevent the problem
4
Outline of Today’s Presentation
• Methods– Recruitment, training– Evaluation of value
• Results– Frequency, Types, Drugs involved– Economic Analysis
• Conclusions
• Where to from here?
Method
s
Conclusion
sR
esultsR
esults
5
Overview of Methods
Method
s
6
Pharmacy Recruitment and Enrolment
• 250 WiniFRED® pharmacies invited to participate
• 75 enrolled for the project, only 52 possible due to software, hardware or location issues
• Arms– Remuneration– Intervention
Prompt– Observation
Method
s
7
Remuneration Randomisation
Method
s
• Crossover design
8
PROMISe Observers
• Seven observers, each visiting 3 pharmacies 9 times in 3 weeks– 21 pharmacies “observed”
• Assist with documentation– Identify opportunities for documentation
– Aid classification/documentation process
• Time some events– Investigation of problems, phone calls
to doctors, discussions with patients
Method
s
9
Automatic Intervention Prompt
• Related to antiplatelet prophylaxis for vascular events in diabetic patients
• Activated when oral antidiabetic agents were dispensed
• 31 of 52 pharmacies randomised to receive prompt
Method
s
10
WiniFRED Interface Training
• Three training evenings plus initiation visits to each site
Method
s
11
Online Classification Training
• All pharmacists who indicated that they would participate were required to complete an on-line training package– 20 scenarios to be classified– 2 case-based clinical skills assessments
• Pharmacist demographics questionnaire completed at this point
• 20 scenarios re-classified after 3 weeks of use of the system
Method
s
12
PROMISe Data Collection
• Pharmacy Demographics– Daily workload and staffing– Entrepreneurial orientation– Prescriptions dispensed
• Pharmacist Demographics– Clinical skills– Job satisfaction
• Clinical Intervention Parameters– Patient demographics – Drug involved and other
drugs taken by patient– Type of problem– Action taken,
Recommendation made– Acceptance of
recommendation– Reactive or proactive– Time taken– Documenting pharmacist’s
rating of clinical significance
Method
s
13
Data Collection
• Initially planned for 3 weeks
• Extended to 4, then 8 weeks to obtain sufficient numbers of interventions
• Loss of interest from many pharmacies once observation phase was over and project team left Melbourne.
Method
s
14
Assessment of Value:Probability and Severity Considered
• Considerations: • How Addressed in Method
1. The nature and severity of the potential consequence(s) had the intervention not occurred
2. The probability that the consequence(s) will occur before the intervention
3. The probability that the consequence(s) will occur despite (after) the intervention
4. The degree to which the intervention can be attributed to the pharmacist
1. Consequences table•Economic and non-economic parameters for each level of severity, •validated by experts, •multiple consequences (positive and negative) possible
2,3. Panel members considered probability and severity for each consequence selected before and after the intervention
4. Panel members provided a value for attribution
Method
s
15
Method
s
Consequences Table
16
Assessment of Value
The Value of Clinical Interventions (VOCI)Economic Assessment System
AttributabilityAttributable Probability Reduction
Parameters of Consequence(s)
Value of Intervention Described as:
•Days of Loss of Poor Health•Cost of Admissions Avoided•Number of Admissions Avoided•Number of Days in Hospital Avoided•Number and Cost of GP Consults Avoided•Number and Cost of Specialist Consultations Avoided•Cost of Investigations Avoided
Probability of Consequence
BeforeIntervention
Probability of Consequence
AfterIntervention
- X =
X
Attributable Probability Reduction
=
AttributabilityAttributable Probability Reduction
Parameters of Consequence(s)
Value of Intervention Described as:
•Days of Loss of Poor Health•Cost of Admissions Avoided•Number of Admissions Avoided•Number of Days in Hospital Avoided•Number and Cost of GP Consults Avoided•Number and Cost of Specialist Consultations Avoided•Cost of Investigations Avoided
Probability of Consequence
BeforeIntervention
Probability of Consequence
AfterIntervention
- X =
X
Attributable Probability Reduction
=
PROMISe Economic Simulation
and Extrapolation
Model
Method
s
17
UTAS Server
I NTERNETINTERNET INTERNETINTERNET
INTERNETINTERNET
INTERNETINTERNET
Data Base and Analysis
Interface
Data Base and Analysis
Interface
Assessment of Value
• 16 Clinical Assessors in 4 panels– 2 physicians, 6 GPs, 8 pharmacists
• Secure internet access to intervention details• Each panel assessed the same set of 51
common interventions and a “panel specific” set of 60 randomly selected interventions
– 51 common interventions and 240 randomly selected interventions were assessed
Method
s
18
Clinical Panel Intervention Display
Method
s
19
Clinical Panel Selection of Consequences
Method
s
20
Economic Analysis: Derivation of Main Value Indicators
Method
s
21
Sources of Information
Results
5. PROMISe SQL Database (~13,000 interventions and ~430,000
prescriptions)
3. PROMISe Pharmacist Demographics (125)
6. Clinical Panel Assessments(16 members, ~290 interventions)
7. PROMISe Pharmacists’ Feedback
(~80)
1. PROMISe Pharmacy Demographics (52)
2. Non- PROMISe WiniFRED Pharmacy Demographics (~40)
8. Non-PROMISe Pharmacists’ Opinions (~400 phone interviews)
4. Direct Observation Visits
(63 visits)
22
Method
sR
esults
23
PROMISe Pharmacy Demographics (n = 52)
• Entrepreneurial Orientation– ~15% more innovative (self determined from responses to 2
questions)
• Location and Size– No different to Non-PROMISe and non-WiniFRED pharmacies
• Date of QCPP Accreditation– More Innovators (accredited before December 1999)
• 5/48;10.4% cf 2.5% in Victoria
• IT Facilities and resources– Used to determine attitudes to skills in IT area
• Daily Staffing levels– Used for workload analysis and simulations
Results
24
PROMISe Pharmacist Demographics (n= 125)
• Gender, Age, Registration year– Younger age group (80/125; 64% <40yo)
• Practice Profile– 17/122; 14% accredited for medication reviews (cf ~5%)
• Scenario Classification Score (Before and after study)– Improved from 76% to 83% post trial
• Scored well for– Job Satisfaction (83%), – Professional Integrity (77%), – Change Readiness (73%)
• Clinical Skills– Good range of scores
Results
25
PROMISe Database: Non Clinical Interventions
• 11,493 Non-Clinical (Brand Substitution interventions) from 305,519 scripts (average rate of 3.7%)
• Under-utilised by pharmacists in study, still extrapolates to ~$15M pa
Results
26
Clinical Interventions
Results
• 2396 interventions from 435,520 scripts
0.55 interventions per 100 scripts
0.00
0.20
0.40
0.60
0.80
1.00
1.20
21/04/2005
28/04/2005
5/05/2005
12/05/2005
19/05/2005
26/05/2005
2/06/2005
9/06/2005
16/06/2005Clinical Interventions per 100 Prescriptions
Cumulative Clinical Intervention Rate (per 100prescriptions)Poly. (Clinical Interventions per 100 Prescriptions)
27
Clinical Interventions
Results
Observers Present
Project Team PresentRemuneration
• Decline in recording of interventions
28
Clinical Interventions: CategoriesR
esults
29
Clinical Interventions: Actions
Results
• Investigation and discussion with patient common (71%)
• 18% contact with prescriber
30
Clinical Interventions: Recommendations Results
31
Clinical Interventions: Acceptance of Recommendations
Results
• Dose, Drug or Education category interventions highly accepted
32
Clinical Interventions: Proactive vs Reactive
• Drug selection, Dose problems more likely to be proactive
• Education and Toxicity less likely to be proactive (direct patient requests)
Results
33
Clinical Interventions: Clinical Significance (Pharmacist reported)
• More likely to be drug selection or toxicity problems and result in referral to GP
Results
34
Clinical Interventions: Drugs Involved- Numbers
Results
35
Clinical Interventions: Drugs Involved- Rates
• Skewed by intervention prompt
Results
36
Clinical Interventions for particular groups of drugs : Antidiabetic Agents
• Skewed by intervention prompt
Results
37
Clinical Interventions: Drugs Involved- Rates
Results
38
Results of Randomisation
Results
52 Pharmacies Enrolled
22 Ever Observed 30 Unobserved
12 Paid in Phase 1,3
18 Paid in Phase 2,3
11 Paid inPhase 1,3
11 Paid inPhase 2,3
9 Aspirin Prompt
5 No Aspirin Prompt
6* No Aspirin Prompt
2 No Aspirin Prompt
5 Aspirin Prompt
9 Aspirin Prompt
7 Aspirin Prompt
9 Aspirin Prompt
*1 pharmacy converted to prompt after 1 week
39
Clinical Interventions: Effect of Remuneration
Results
• Effect of remuneration in first two weeks of study (univariate)
• Small impact when payment instituted (ameliorated reduction cf 20% reduction from Phase 1 to 2)
1.09
0.61
0.71
0.81
0
0.2
0.4
0.6
0.8
1
1.2
Phase 1 Phase 2
Clin
ica
l In
terv
en
tio
n R
ate
Paid
Unpaid
25% reduction
14%
40
Clinical Interventions: Effect of Observation
• Observation significant (unadjusted)– 1.02 vs 0.46 for all
three phases (ever observed)
– 2.02 vs 0.8 for Phase 1 (observed days)
Results
41
Aspirin Intervention Prompt Effectiveness
Total194/7895 = 2.46
Aspirin prompt 193/4174 = 4.63 No prompt
1/ 3721 = 0.03
Ever observed0.03
Never observed0
Ever observed157/2128 = 7.34
Never observed37/2046 = 1.81
• Aspirin interventions per 100 diabetic patients• Only 7 in phase 3 (when aspirin prompt switched off)
Results
42
Aspirin Interventions: Observer Effect
0
20
40
60
80
100
120
140
160
180
Nu
mb
er o
f A
spir
in In
terv
enti
on
s
Never ObservedEver Observed
Phase 1: Observers Present Phase 2: Observers Absent
12.6
1.84
2.3
1.3
Phase 3: Prompt off
Results
43
Clinical Interventions: Effect of Intervention Prompt
• Significant effect in first half of study on other interventions as well
Results
44
Aspirin Intervention Prompt
• No aspirin interventions without prompt
• Observation had a marked effect on increasing rate
• Increased rate of other interventions as well
• No residual effect of prompt – 7 after prompt turned off
• ?Fatigue to prompt after 3-4 weeks
Results
45
Clinical Interventions: Multivariate Analysis of Effects Within Phases
Results
2,384 Pharmacy Days where more than 20 prescriptions dispensedMean Intervention Rate 0.74 (Standard Deviation 1.78)
Phase 1
1.01 (1.79)
Phase 20.88 (1.73)
Phase 30.52 (1.76)
Observed2.02 (2.26)
Observed2.40 (2.87)
Unobserved0.78 (1.59)
Unobserved0.52 (1.76)
Unobserved0.80 (1.61)
Paid2.29 (2.43)
Unpaid1.68 (2.00)
Paid0.98 (1.94)
Unpaid0.65 (1.27)
Paid2.05 (2.29)
Unpaid2.67 (3.29)
Paid0.66 (1.37)
Unpaid0.91 (1.80)
Paid0.52
(1.76)
Aspirin 2.45
(2.59)
No Aspirin
1.56 (1.27)
Aspirin 1.95
(2.17)
No Aspirin
1.29 (1.69)
Aspirin1.31
(2.29)
No Aspirin
0.48 (1.05)
Aspirin0.66
(1.32)
No Aspirin
0.63 (1.18)
Aspirin 2.90
(2.51)
No Aspirin
1.20(1.81)
Aspirin 3.07
(3.58)
No Aspirin
1.10 (0.70)
Aspirin0.74
(1.40)
No Aspirin
0.56 (1.33)
Aspirin1.27
(2.12)
No Aspirin
0.26 (0.52)
Aspirin0.81
(2.98)
No Aspirin
0.42 (1.10)
Phase, Prompt, Observation independently significant, payment not
46
Clinical Interventions: Multivariate Analysis of Effects Within Phases
Results
• No relationship effects when analysed by phase• Phase One
– Prompt significant (F=8.87; p = 0.003)– Observation significant (F=26.4; p <0.001)– Payment not significant (F=2.41; p = 0.121)
• Phase Two– Prompt significant (F=14.6; p < 0.001)– Observation significant (F=18.4; p <0.001)– Payment not significant (F=0.05; p = 0.813)
• Phase Three– Prompt significant ( F= 10.1; p = 0.001)
47
Economic Analysis Overview
Results
• Involves complex simulation of outcomes based on variable assumptions regarding:– The rate of interventions in observed vs non-observed
days– The rates of interventions on busy and less busy days– The rates of interventions with and without the
intervention prompt– The proportion of interventions performed that were
actually documented (recording rate)
• Study design allows for estimate of opportunity for intervention
48
Economic Analysis - Current Value
Results
All Australian pharmacies
232M scripts pa
Average Value of Interventions in
PROMISe data• 0.22 days in hospital• 1.23 consultations• $290 in total costs• 44 days of poor health
Value of interventions in all Australian pharmacies
• 262,424 days in hospital• 1.48M consultations• $349M in total costs• 53M days of poor health
PROMISe Sample
52 Pharmacies for 8 weeks
2396 Interventions435,000 prescriptions
PROMISe intervention data
2373 Interventions420,152 scripts
PROMISe Assessed Sample
291 Interventions1779 Assessments
16 AssessorsPROMISe
Assessed Sample
291 Interventions
Clinical Assessment
Process
1.6M interventions
49
Economic Analysis: Assumptions in Current Value Simulation
Results
• Recording rate on observed days 90%– Higher rate reduces final value
• Recording rate on Unobserved Days 50%– Higher rate reduced final value
• Attribution Rate 75%– Lower rate reduces final value
• Intervention Rate 0.69 per 100 scripts
50
Economic Analysis: Varying Assumptions in Current Value Simulation
Results
51
Effect of Activity (Workload)
13.1
7.8
4.8 4.8
3.4
0
2
4
6
8
10
12
14
1 (6) 2 (8.7) 3 (11.1) 4 (14.1) 5 (18.8)
Quintile of Activity (Scripts per hour)
Inte
rve
nti
on
ra
te p
er
10
00
pre
sc
rip
tio
ns
Increased workload decreased the intervention rate by 75% for all interventions (even in the same pharmacies)
34/52; 65%
Results
Representation of pharmacies across each quintile
45/52; 87% 46/52; 88% 47/52; 90% 42/52; 81%
52
Effect of Activity and Intervention Prompt
0.611.11
0.840.750.97
12.5
7.3
4.2 4.2
3.2
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
1 2 3 4 5
Quintile of Activity
Inte
rve
nti
on
ra
te p
er
10
00
pre
sc
rip
tio
ns
Aspirin Rate
Other Rate
Uptake of educational prompt was relatively resistant to increased workload
Results
53
Economic Analysis: Assessing the Increased Opportunity for Intervention by Improving Staffing levels
Results
• Based on intervention rate from second lowest quintile (45 of 52 pharmacies represented ; 87%)
• 28 hours of extra pharmacist time per 1000 prescriptions (~Cost $1300; ~ savings $910)
54
Economic Analysis: Assessing the Increased Opportunity for
Intervention by an Aspirin Prompt (or similar)
Results
• Based on intervention rate achieved in Aspirin pop-up arm with observation– Observation
mimics an education/ incentive program
• Additional $319M
55
Economic Analysis: Assessing the Increased Opportunity for
Intervention by Optimal Identification
Results
• Additional $606M
• Based on 2.08 ints/100 scripts
• Achievable with suitable motivation…
56
Motivators and Barriers for clinical interventions
• Key motivators were identified – Work environment– Clinical knowledge/continuing education– Professional satisfaction– Information continuity – Remuneration
• Barriers to documentation– Lack of time– Forgetfulness– Workflow restrictions– Software concerns
Results
57
Potential to increase interventions
• In the feedback on project, participants indicated that they would like to be able to carry out more interventions– Adequate staffing and staff mix– Continuing education– Identification of recordable incidents could be
further optimised
Results
58
Conclusions
• Current value of clinical interventions is high
• Considerable scope for increasing intervention rate (and value) with educational techniques– Up to threefold
Conclusion
s
59
Conclusions
Conclusion
s
60
Potential Roll-out Strategies for PROMISe
• Integrated “whole solution” approach– Repository model, – Feedback based on information received
(education and quantified), – Pharmacist access to individual results and
examples– “Push” information to pharmacies
• Incentive payment structure associated with targets for interventions
Conclusion
s
61
Recommendations
• Explore additional educational alerts• Expand economic analysis to other datasets• More detailed economic analysis to evaluate
different types of interventions• Identify factors associated with increased
intervention rates – Workload
• Obtain more representative information – Larger sample for longer period
• Simplify and modify classification system for more widespread use
Conclusion
s