epi235: epi methods in hsr
DESCRIPTION
EPI235: Epi Methods in HSR. April 17, 2007 L5 Program Evaluation with Longitudinal Data 1: Applications (Dr. Schneeweiss) Various examples of applications in Health Services Research. Strengths and limitations of time series analysis . Background reading: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/1.jpg)
1
EPI235: Epi Methods in HSR
April 17, 2007 L5
Program Evaluation with Longitudinal Data 1: Applications (Dr. Schneeweiss)
Various examples of applications in Health Services Research. Strengths and limitations of time series analysis .
Background reading: •Soumerai SB, Avorn J, Gortmaker S, Ross‑Degnan D. Payment restrictions for prescription drugs in Medicaid: Effects on therapy, cost, and equity. New Engl J Med 1987;317:550‑556.•Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin TJ. Effects of Medicaid drug-payment limits on admission to hospitals and nursing homes. N Engl J Med 1991;325:1072-1077
![Page 2: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/2.jpg)
2
![Page 3: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/3.jpg)
3
![Page 4: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/4.jpg)
4
![Page 5: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/5.jpg)
5
What is Program Evaluation?
![Page 6: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/6.jpg)
6
Program evaluation
What is program evaluation?
Design options:
Randomized trial
Cohort study
Quasi-experiment
What means quasi-experiment?
- implementation vs. evaluation perspective
- unintended randomization
Quasi-experiment = pre-post comparison?
= longitudinal design
= repeated measure design
Program Evaluation
![Page 7: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/7.jpg)
7
![Page 8: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/8.jpg)
8
![Page 9: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/9.jpg)
9
![Page 10: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/10.jpg)
10
![Page 11: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/11.jpg)
11
![Page 12: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/12.jpg)
12
Observational Design: Simple pre-post comparisons
Time
Intervention
Assumptions for causal inference:
1. The pre experience represents the post experience had there been no intervention
Outcome rate
![Page 13: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/13.jpg)
13
Observational Design: Simple pre-post comparisons
Time
Intervention
Threat to causal inference:
Single pre-post estimates are averages of an underlying trend independent of the Intervention
Outcome rate
![Page 14: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/14.jpg)
14
25% cost sharing in Quebec
0.0
0.7
1.4
2.1
# o
f e
ss
en
tia
l dru
gs
pe
r d
ay
ObservedPredicted (--- 95% CI)
11-93 8-94 8-95 8-96 8-97
Elderly (n=70801)
Policy Initiated
Tamblyn, JAMA 2000
![Page 15: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/15.jpg)
15
Observational Design: no concurrent controls
Time
Intervention
Assumptions for causal inference:
1. Close temporal relation
2. Extrapolation of baseline trend is equal to the counterfactual experienceSchneeweiss, Health Policy 2000
![Page 16: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/16.jpg)
16
Fixed ¢50 cost sharing in SC
Nelson, Med Care 1984
![Page 17: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/17.jpg)
17
Observational Design: With concurrent controls
Assumptions for causal inference:
1. …
2. Control trend is equal to the counterfactual experience of intervention group
Time
Intervention
Intervention group
Control group
Schneeweiss, J Clin Epi 2002
![Page 18: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/18.jpg)
18
3 prescription caps in NH:
80%
85%
90%
95%
100%
Baseline Cap After Cap
% o
uts
ide
nu
rsin
g h
om
e
New Jersey
New Hampshire
Soumerai NEJM 1991
![Page 19: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/19.jpg)
19
Longitudinal quasi-experimental evaluation designs
Strength Weaknesses
Controls for between-person
confounding
Immediate causal inference
Intuitive graphs – easy to understand
Violation of basic assumption
Co-intervention or long-term trend
Slow or delayed effect
Regression to the mean
Reactiveness
Correlated data
Within-person confounding?
![Page 20: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/20.jpg)
20
… or randomization?
Time
Intervention
Intervention group
Control group
R
Assumptions for causal inference:
1. Subjects comply with their assigned ‘treatment’ = policy
![Page 21: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/21.jpg)
21
Complex statistical analyses are less convincing for decision makers
![Page 22: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/22.jpg)
22
Formulary delisting in BC
![Page 23: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/23.jpg)
23
More Examples:
![Page 24: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/24.jpg)
24
More Examples:
Prescription drug useSurgical site infectionsContraindicated drug use
![Page 25: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/25.jpg)
25
Medicaid prior authorization: Use
Smalley NEJM 1995
![Page 26: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/26.jpg)
26
Medicaid prior authorization: $$
Smalley NEJM 1995
![Page 27: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/27.jpg)
27
Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery)Solid diamonds = surgical site infection rate after cesarean section
Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods.
Weinberg et al.: Arch Intern Med 2001
Reducing surgical site infections after C-section: Hospital A
![Page 28: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/28.jpg)
28
Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery)Solid diamonds = surgical site infection rate after cesarean section
Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods.
Weinberg et al.: Arch Intern Med 2001
Reducing surgical site infections after C-section: Hospital B
![Page 29: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/29.jpg)
29
The Intervention:
'Dear Doctor' letters concerning interactions between cisapride and a series of drugs.
A letter in 1995 described a risk of prolonged QT intervals and serious ventricular arrhythmia in patients who received macrolide antibiotics and imidazole antifungals in conjunction with cisapride.
A June 1998 letter that expanded the list of contraindicated comedications had wider distribution than an earlier one, was accompanied by substantial Internet and media coverage, and was complemented by an effort to inform large pharmacy dispensing information organizations of the warnings against concurrent use of the named drugs.
FDA Risk Management: Effectiveness of a ‘Dear Doctor’ letter
![Page 30: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/30.jpg)
30
Weatherby LB et al. PharmEpi Drug Safety 2001
![Page 31: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/31.jpg)
31
![Page 32: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/32.jpg)
32
Lecture on LPUs:
Weatherby LB et al. Clin Pharm Ther 2002
![Page 33: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/33.jpg)
33
The Policy Model
![Page 34: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/34.jpg)
34
Policy Model: reference pricing
![Page 35: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/35.jpg)
35
The Clinical Model
![Page 36: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/36.jpg)
36
More on the Clinical Model:
![Page 37: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/37.jpg)
37
Clinical Model: reference pricing
![Page 38: EPI235: Epi Methods in HSR](https://reader035.vdocuments.us/reader035/viewer/2022062804/56814c48550346895db94d44/html5/thumbnails/38.jpg)
38
Policy Model vs. Clinical Model