1 epi235: epi methods in hsr may 3, 2007 l10 outcomes and effectiveness research 4: hmo/network (dr....
Post on 21-Dec-2015
213 views
TRANSCRIPT
1
EPI235: Epi Methods in HSR
May 3, 2007 L10
Outcomes and Effectiveness Research 4: HMO/Network (Dr. Schneeweiss)
Methodologic issues in benchmarking physician and hospital performance. Analysis of patient data clustered in physicians and in clinics using hierarchical (multi-level) models. Reporting benchmarking results.
Background reading: Austin PC, Goel V, van Walraven C. An introduction to multilevel regression models. Can J Public Health 2001;92:150-154.Austin PC, Tu JV, Alter DA: Comparing hierarchical modeling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: Should we be analyzing cardiovascular outcomes data differently? Am Heart J 2003;145:27-35.Carey K: A multilevel modeling approach to analysis of patient costs under managed care. Health Econ 2000;9:435-446 .
2
3
4
Pennsylvania Consumer Report
www.phc4.org
5
6
7
8
9
10
11
Quality improvement by quality measurement in NY
0
1
2
3
4
5
6
7
8
9
1989 1990 1991 1992
Year
Ris
k-ad
just
ed m
ort
alit
y .
<50 OPs
50-100 OPs
100-150 OPs
>150 OPs
Hannan ED et al. JAMA 1995
12
Risk factor reporting in the NY Bypass Reporting System (%)
Risk factor 1989 1990 1991
Renal insufficiency 0.4 0.5 2.8
CHF 1.7 2.9 7.1
COPD 6.9 12.4 17.4
Unstable Angina 14.9 21.1 21.8
Low EF (<40%) 18.9 23.1 22.2
13
Multilevel Modeling in Health Services Epidemiology or Hierarchical Modeling or Random Effects Modeling (mixed effects…) or Variance Components Analysis Major issue in health services research: Data clustered at multiple levels: Patients clustered by physician (provider) Physicians clustered by facility (group practice, hospital) Hospitals clustered by region Classical Epi examples: Patients clustered in studies (meta-analysis) Nutrients, food items, and meals
14
Covariates: patient level (severity, comorbidity), provider level (training, experience), facility level (teaching hospital, practice guidelines), regional level (salary structure)
Cluster sizes may vary substantially Example: Patient costs increase with age Outcome: annual costs per patient Exposure: patient age Covariates: Patient level (1): gender Physician level (2): gender, practice volume -> Two level model What do we want to estimate?
1. Patient-level (covariate) effects adjusted for clustering 2. Cluster specific measures of utilization or outcomes adjusted for patient-
level risk factors (case-mix adjustment) HSR: #2 = benchmarking
15
Aggregated analysis versus individual level analysis Aggregated analysis results in loss of power Individual level analysis ignores the structure of nesting within organizations: -> Ordinary least squares regression (OLS):
individual outcome and covariate data are mixed with higher level covariates.
Such 'intra-class correlation' violates the OLS assumption of random errors, i.e. independent observations with common variance In nested data the error term is composed of an individual-level and an group-level variance component. Disregarding the correlation of observations leads to underestimates of OLS standard errors and thus overestimates of the significance -> false 'positive' results. Multilevel modeling is most efficient and provides unbiased estimates of standard errors.
No Clusters
Clusters but No Clustering
Clusters with much Clustering
Extreme Clustering: Pure Lumps
16
17
CABG Mortality among 55 hospitals
MLwiN®
L e v e l o n e m o d e l :
iii exy 10 i = i n d i v i d u a l s ; j = p r o v i d e r L e v e l t w o : V a r i a t i o n i n i n t e r c e p t i s p r e d i c t e d b y
jj u 000
S u b s t i t u t e i n l e v e l o n e m o d e l :
ijjijjij euxy 010
V a r i a n c e c o m p o n e n t s ! F i x e d p a r a m e t e r
R a n d o m p a r a m e t e r
T h e m o d e l i s m a d e t w o - l e v e l b y a l l o w i n g t h e l i n e s f o r a l l h o s p i t a l s t o v a r y a r o u n d t h e m e a n i n t e r c e p t (
0 ) b y a n a m o u n t u 0 j , w h i c h i s n o r m a l l y d i s t r i b u t e d a r o u n d t h e m e a n .
T h e s e d e p a r t u r e s f r o m t h e a v e r a g e a r e k n o w n a s t h e l e v e l 2 r e s i d u a l s . T h e y a r e i n t e r p r e t e d a s t h e h o s p i t a l e f f e c t .
T h e t r u e l e v e l 2 r e s i d u a l s a r e u n k n o w n , b u t w e c a n e s t i m a t e t h e m w i t h t h e i r 9 5 % c o n f i d e n c e l i m i t s . T h e s e e s t i m a t e s c a n b e p l o t t e d f o r a g r a p h i c a l c o m p a r i s o n o f h o s p i t a l s .
18
19
20
Mixed effect models: Patient level covariates = random effects, provider level covariates = fixed effects SAS for continuous (linear) outcomes: proc mixed data=temp; class doc_id; model cost= pat_age pat_sex doc_sex doc_volume /solution; random intercept /subject* = doc_id; * complete independence is assumed between subjects random intercept age /subject* = doc_id; Non-linear mixed effects models: e.g. binary outcomes, count outcomes SAS V8: proc nlinmixed Expansion to more than two levels. MLwin can handle up to 15 levels.
21
SAS data format:
Patient characteristics Hospital char. Patient Hospital DCG Age gender Teaching MC
1 1 … … M 1 0.56 2 1 M 1 0.56 3 1 F 1 0.56 4 1 M 1 0.56 5 1 F 1 0.56 6 1 M 1 0.56 7 1 M 1 0.56 8 1 M 1 0.56 9 1 M 1 0.56 10 1 M 1 0.56 11 1 M 1 0.56 1 2 M 0 0.78 2 2 F 0 0.78 3 2 M 0 0.78 1 3 M 1 0.60 2 3 M 1 0.60 3 3 M 1 0.60 4 3 M … 0.60 5 3 M 0.60 6 3 F … 7 3 … 8 3 9 3 10 3 11 3 12 3 13 3 14 3
22
23
24
25
26
What is the best comparator for benchmarking?1) Average of all institutions 2) Stratified by institution characteristics
Austin et al. Am Heart J 2004
27
Class Exercise Background: Some previous literature has shown that hospitals that initiate education programs for patients in management of chronic stable angina have shown improved outcomes. However, hospitals that have initiated such programs tend to differ from facilities that do not implement such programs. Objective: To evaluate a group outpatient educational intervention to improve outcomes in stable angina. Specific Aims: (1) To determine whether a group outpatient education program on the management of chronic stable angina improves patient health as measured by the Seattle Angina Questionnaire (SAQ). (2) To determine what patient characteristics and facility characteristics are associated with the success of such an intervention. Resources: 30 hospitals have agreed to participate. The intervention/ non-intervention selection can only be done on a facility level, because if some patients are receiving special group education, all physicians will want their patients to participate. Therefore, we cannot randomize on a patient level! The facilities will not be self-selecting into the intervention/ non-intervention arms. Rather, the researcher (you) must decide how to allocate the educational intervention. Primary data collection on patient health status (SAQ) will be used to measure outcomes to assess the effect of the intervention. Complete during and post-intervention data, both administrative and primary data are available for all facilities. Individual participation in the group education program (i.e. percentage of sessions attended) will be recorded for those individuals in the intervention facilities. Patient characteristics will also be available for some variables, including age, gender, race, and presence of supplemental insurance. Patients’ incomes will not be known, but their zipcode will be. Facility characteristics will be available for a large number of variables including size, teaching hospital status, financial ownership status, average demographics of patients, presence of cardiology programs, staff specialties, competitive environment, expenditures per patient and average distance of patients’ zip codes to the hospital.
28
Dealing with clustering
Generalized estimating equations (GEE) SAS: proc genmod
Random effects modeling (multi-level modeling)
29
Effect of physician specialty on patient outcomes
Attending cardiology specialty
Austin et al. Am Heart J 2003
30
Teaching hospital
31
GEE vs. random effects model
Twisk, Eur J Epi, 2004
Expl.: Risk factors of increased cholesterol levels
32
-6 -4 -2 0 2 4 6
-6-4
-20
24
6
Variance of Z= 1
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
-6-4
-20
24
6
Variance of Z= 2
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
-6-4
-20
24
6
Variance of Z= 3
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
-6-4
-20
24
6
Variance of Z= 4
x
Expecte
d V
alu
e Y
GEE
Rand. intercept
33
-6 -4 -2 0 2 4 6
0.0
0.2
0.4
0.6
0.8
1.0
Variance of Z= 1
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
0.2
0.4
0.6
0.8
Variance of Z= 2
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
0.2
0.4
0.6
0.8
Variance of Z= 3
x
Expecte
d V
alu
e Y
-6 -4 -2 0 2 4 6
0.2
0.4
0.6
0.8
Variance of Z= 4
x
Expecte
d V
alu
e Y
GEE
Rand. intercept
34
Final Note: Parameter Interpretation
Marginal parameters are often thought to be more appropriate for policy questions, “What is the difference in the rate of illness in the treated population versus the untreated?”
Conditional parameter estimates closer to individual effects, “What is the effect of treatment in an individual person?”
Conditional parameter are more clinically relevant?