robust methods for health-related quality-of-life assessment

IntroductionMethodology

SimulationsApplicationConclusion

Robust Methods for Health-relatedQuality-of-life Assessment

Ian McCarthy

Baylor Scott & White HealthCenter for Clinical Effectiveness

Utah Health Services Research ConferenceApril 30, 2014

Robust Methods for Health-related Quality-of-life Assessment



Background

Cost- and comparative-effectiveness studies becomingincreasingly important

Require assessment of health-related quality-of-life (HRQoL)outcomes and quality-adjusted life-years (QALYs)

Common approach first collapses the multi-dimensionalHRQoL profile into a one-dimensional QALY (Drummondet al., 2005; Brazier et al., 2002; Brazier & Ratcliffe, 2007)

EQ-5DSF-6DHUI




Problem

Loss of information when reducing HRQoL profile into QALY, withpotentially biased and inconsistent marginal effects estimates(Mortimer & Segal, 2008; Devlin et al., 2010; Parkin et al., 2010;Gutacker et al., 2012):

1 Floor and ceiling effects not present in the underlying domainsbut imposed by the scoring algorithm.

2 Nonlinearities in the relationship between the outcome andindependent variables which are difficult to approximate usingthe summary score.




Current Study

1 Monte Carlo study showing the bias of the estimatedcoefficients when relying solely on QALYs or some othersummary score based on several ordered outcome variables.

2 Propose new two-step methodology that first estimatescoefficients in each HRQoL domain and then transforms thecoefficients and marginal effects into the QALY domain basedon predicted values from the first-stage regressions.




Estimating QALYsMarginal Effects: Standard ApproachMarginal Effects: Proposed Methodology

The SF-6D

Developed by John Brazier and other, the SF-6D is formed from asubset of questions from the SF-36 or SF-12 and is a commonHRQoL outcome intended to provide a general measure of apatient’s health status (Brazier et al., 2002; Brazier & Ratcliffe,2007).

Six dimensions/domains of health: (Physical functioning, rolelimitations, social functioning, pain, mental health, andvitality)

Each domain characterized numerically with a range ofintegers. Best value is 1, and worst value ranges from 4 to 6.

Scoring algorithm developed in Brazier et al. (2002) andBrazier & Ratcliffe (2007) for calculating a population-basedindex score from the SF-6D questionnaire





Scoring the SF-6D

Physical Functioning (PF)PF=2 or PF=3 -0.035PF=4 -0.044PF=5 -0.056PF=6 -0.117Role Limitations (RL)RL=2 or RL=3 or RL=4 -0.053Social Functioning (SF)SF=2 -0.057SF=3 -0.059SF=4 -0.072SF=5 -0.087Pain (P)P=2 or P=3 -0.042P=4 -0.065P=5 -0.102P=6 -0.171Mental Health (MH)MH=2 or MH=3 -0.042MH=4 -0.100MH=5 -0.118Vitality (V)V=2 or V=3 or V=4 -0.071V=5 -0.092Combination of Domains“Most Severe” -0.061





Focus on QALYs

By far the most common methodology for estimatingcoefficients and ultimately marginal effects is to first reducethe multi-dimensional health profile to a one-dimensionalQALY (Austin et al., 2000; Austin, 2002; Richardson &Manca, 2004; Manca et al., 2005; Basu & Manca, 2012)

Recent literature on how best to accommodate distributionalfeatures somewhat specific to QALYs (Austin, 2002; Basu &Manca, 2012), including a censored least absolute deviationmodel and a Beta MLE approach





First Stage Regression

1 Estimate an ordered probit model separately for each domain,d = 1, ..., 6, with the follow-up HRQoL response (yid ,t1)modeled as a function of person-specific variables (xi ),baseline HRQoL response (yid ,t0), and treatment status (Ti ).

2 Form predicted probabilities of every possible response, j , ineach domain, d , denoted p̂d

j .

The regression results provide a predicted (marginal) probability foreach of 31 possible outcomes for each person.





“Most Severe” Category

1 Defined as any one of the following (Brazier et al., 2002): 4or more in the physical functioning, social functioning, mentalhealth, or vitality domains; 3 or more in the role limitationdomain; or 5 or more in the pain domain

2 Since the probabilities, Pdij , are potentially correlated across

domains, the probability of a “most severe” health status canbe calculated following the principle of inclusion and exclusionfor probability:

P (A1 ∪ A2 ∪ ... ∪ AN) = P (A1) + ...+ P (AN) +

N∑n=2

(−1)n+1P (∩ n events) .





Estimate QALYs

Q̂ALY i = 1− 0.035×(

P̂PFi2 + P̂PF

i3

)− 0.044× P̂PF

i4 − 0.056× P̂PFi5 − 0.117× P̂PF

i6

− 0.053×(

P̂RLi2 + P̂RL

i3 + P̂RLi4

)− 0.057× P̂SF

i2 − 0.059× P̂SFi3 − 0.072× P̂SF

i4 − 0.087× P̂SFi5

− 0.042×(

P̂Paini2 + P̂Pain

i3

)− 0.065× P̂Pain

i4 − 0.102× P̂Paini5 − 0.171× P̂Pain

i6

− 0.042×(

P̂MHi2 + P̂MH

i3

)− 0.100× P̂MH

i4 − 0.118× P̂MHi5

− 0.071×(

P̂Vi2 + P̂V

i3 + P̂Vi4

)− 0.092× P̂V

i5

− 0.061× P̂ (Most Severe) .




Marginal Effects on QALYsTreatment Effects with Selection

Data Generating Processes

The D × 1 vector of latent HRQoL values, y∗i , is simulated asfollows:

y∗i = γ + βx ′i + εi , where

ε ∼ N (0D×1, ID×D) ,

x ∼ U[0, 1],

γ = ID×1, and

β = 1.5× ID×1.

Discrete HRQoL values are generated based on the value of thelatent value, y∗id , relative to the Jd × 1 vector of threshold values ineach domain.





Simulated QALY Distributions

010

2030

4050

Fre

quen

cy

.4 .6 .8 1SF-6D Index Score

010

2030

40F

requ

ency

.2 .4 .6 .8 1SF-6D Index Score

010

2030

4050

Fre

quen

cy


020

4060

80F

requ

ency

.3 .4 .5 .6 .7 .8SF-6D Index Score

050

100

150

200

Fre

quen

cy






Monte Carlo Results

Model Incremental Effect St. Dev. Mean % Bias Lower % Bias Upper % Bias RMSE

DGP 1:True Effect 0.070 0.002Two-stage Approach 0.070 0.003 -0.73% -11.85% 11.64% 0.0827OLS 0.073 0.004 3.79% -8.89% 17.18% 0.0828Beta MLE 0.077 0.004 9.49% -4.84% 25.44% 0.0830Beta QMLE 0.075 0.004 6.27% -6.66% 19.96% 0.0829DGP 2:True Effect 0.093 0.003Two-stage Approach 0.092 0.005 -0.64% -12.62% 11.48% 0.1041OLS 0.089 0.005 -3.84% -15.36% 8.39% 0.1043Beta MLE 0.142 0.010 52.57% 28.34% 76.59% 0.1115Beta QMLE 0.102 0.006 10.14% -4.26% 25.24% 0.1043DGP 3:True Effect 0.076 0.003Two-stage Approach 0.075 0.005 -1.34% -15.60% 15.21% 0.0916OLS 0.065 0.004 -15.02% -29.91% -1.40% 0.0923Beta MLE 0.075 0.008 -1.01% -23.44% 23.44% 0.0935Beta QMLE 0.086 0.006 12.71% -5.97% 32.68% 0.0917





Monte Carlo Results

Model Incremental Effect St. Dev. Mean % Bias Lower % Bias Upper % Bias RMSE

DGP 4:True Effect 0.075 0.002Two-stage Approach 0.075 0.003 -0.22% -10.58% 11.14% 0.0966OLS 0.083 0.004 10.32% -2.40% 24.52% 0.0968Beta MLE 0.083 0.005 10.71% -2.67% 25.71% 0.0969Beta QMLE 0.082 0.004 9.20% -3.23% 22.88% 0.0968DGP 5:True Effect 0.062 0.002Two-stage Approach 0.061 0.003 -0.28% -11.20% 11.19% 0.0916OLS 0.072 0.004 16.70% 2.21% 32.65% 0.0920Beta MLE 0.070 0.004 13.03% -1.05% 28.53% 0.0919Beta QMLE 0.070 0.004 13.46% -0.26% 28.56% 0.0919





General Case of Selection on ObservableVariables

-.00

50

.005

.01

.015

Dev

iatio

n fr

om T

rue

Effe

ct

0 10 20 30 40 50

Degree of Selection

-.01

-.00

50

.005

.01

Dev

iatio

n fr

om T

rue

Effe

ct

0 10 20 30 40 50

Degree of Selection

OLS 2SEDGP 1: Homogeneous Effects across Domains

-.00

50

.005

.01

.015

.02

Dev

iatio

n fr

om T

rue

Effe

ct

0 10 20 30 40 50

Degree of Selection

-.01

-.00

50

.005

.01

Dev

iatio

n fr

om T

rue

Effe

ct

0 10 20 30 40 50

Degree of Selection

OLS 2SEDGP 2: Heterogeneous Effects across Domains





With Simulated Data based on the SF-6D

Random Treatment Assignment Selection on Observed VariablesModel Treatment Effect St. Dev. RMSE Treatment Effect St. Dev. RMSE

DGP 1: δ = 1.5 × I6×1True Effect 0.142 0.005 0.142 0.0052SE 0.143 0.006 0.054 0.143 0.007 0.054OLS 0.143 0.007 0.066 0.151 0.010 0.068Beta MLE 0.169 0.012 0.082 0.174 0.021 0.080Beta QMLE 0.143 0.007 0.067 0.146 0.011 0.066DGP 2: δ = 3 × I6×1True Effect 0.264 0.007 0.264 0.0072SE 0.264 0.007 0.046 0.263 0.009 0.046OLS 0.265 0.008 0.077 0.284 0.010 0.091Beta MLE 0.296 0.010 0.075 0.378 0.018 0.067Beta QMLE 0.264 0.008 0.061 0.320 0.013 0.056

DGP 3: δ = [2, 1, 0.5, 2.5, 0, 1]′

True Effect 0.104 0.004 0.104 0.0042SE 0.104 0.005 0.055 0.104 0.007 0.055OLS 0.104 0.006 0.063 0.088 0.009 0.064Beta MLE 0.117 0.012 0.087 0.083 0.023 0.083Beta QMLE 0.104 0.006 0.070 0.079 0.011 0.070





With Simulated Data based on the SF-6D

Random Treatment Assignment Selection on Observed VariablesModel Treatment Effect St. Dev. RMSE Treatment Effect St. Dev. RMSE

DGP 4: interaction terms with δ = 1.5 × I6×1True Effect 0.122 0.006 0.122 0.0062SE 0.122 0.007 0.048 0.122 0.010 0.048OLS 0.122 0.008 0.084 0.137 0.014 0.094Beta MLE 0.133 0.011 0.096 0.234 0.023 0.085Beta QMLE 0.122 0.008 0.074 0.165 0.015 0.073DGP 5: interaction terms with δ = 3 × I6×1True Effect 0.220 0.007 0.220 0.0072SE 0.220 0.007 0.043 0.220 0.010 0.043OLS 0.220 0.008 0.096 0.266 0.014 0.132Beta MLE 0.231 0.011 0.081 0.332 0.022 0.080Beta QMLE 0.220 0.008 0.068 0.272 0.015 0.065

DGP 6: interaction terms with δ = [2, 1, 0.5, 2.5, 0, 1]′

True Effect 0.102 0.005 0.102 0.0052SE 0.102 0.006 0.047 0.102 0.009 0.047OLS 0.102 0.007 0.078 0.098 0.013 0.079Beta MLE 0.114 0.012 0.109 0.210 0.024 0.090Beta QMLE 0.102 0.008 0.081 0.137 0.015 0.081




Data SummaryResults

Data

Data collected prospectively on adult scoliosis patients from over10 participating members of the International Spine Study Group(ISSG).

Variable Mean StandardDeviation

Age 56.76 14.51BMI 26.59 5.84Baseline SF-6D 0.61 0.12Follow-up SF-6D 0.66 0.12

Count Percent

Operative 193 53%Female 309 85%




Data SummaryResults

Summary Statistics

Baseline Follow-upCount Percent Count Percent

Physical Functioning DomainPF=1 0 0% 0 0%PF=2 35 10% 54 15%PF=3 117 32% 121 33%PF=4 96 27% 83 23%PF=5 100 28% 95 26%PF=6 14 4% 9 2%

Role Limitations DomainRL=1 41 11% 53 15%RL=2 115 32% 144 40%RL=3 10 3% 11 3%RL=4 196 54% 154 42%

Social Functioning DomainSF=1 110 30% 156 43%SF=2 72 20% 77 21%SF=3 99 27% 86 24%SF=4 56 15% 30 8%SF=5 25 7% 13 4%




Data SummaryResults

Summary Statistics

Baseline Follow-upCount Percent Count Percent

Pain DomainP=1 5 1% 19 5%P=2 34 9% 47 13%P=3 79 22% 123 34%P=4 85 23% 88 24%P=5 109 30% 66 18%P=6 50 14% 19 5%

Mental Health DomainMH=1 76 21% 130 36%MH=2 127 35% 132 36%MH=3 89 25% 61 17%MH=4 53 15% 32 9%MH=5 17 5% 7 2%

Vitality DomainV=1 13 4% 15 4%V=2 73 20% 123 34%V=3 107 30% 108 30%V=4 94 26% 74 20%V=5 75 21% 42 12%




Data SummaryResults

Average Treatment Effect of Surgery

OLS Beta Beta 2SEMLE QMLE

Outcome: QALY QALY QALY PF RL SF P MH V

Surgery 0.03*** 0.17*** 0.15*** -0.06 -0.06 0.14 0.54*** 0.28** 0.26**(0.01) (0.05) (0.05) (0.12) (0.12) (0.12) (0.12) (0.12) (0.12)

Age 0.00* 0.00 0.00* -0.00 -0.01 0.00 0.01** 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00)

Female -0.02 -0.10 -0.09 -0.12 -0.27 0.07 -0.09 -0.47*** -0.31*(0.01) (0.07) (0.07) (0.16) (0.17) (0.17) (0.16) (0.18) (0.17)

BMI -0.00 -0.00 -0.00 -0.00 -0.02** 0.01 -0.02 -0.00 0.00(0.00) (0.00) (0.00) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)

ATE on QALY 0.033*** 0.038*** 0.032*** 0.029***(0.011) (0.011) (0.011) (0.010)

RMSE 0.098 0.111 0.098 0.097




Intuition

Collapsing multi-dimensional profile into a single summarymeasure introduces floor/ceiling effects and nonlinearities thatare difficult to accommodate in a single equation framework.

With selection into treatment (whether on observables orunobservables), standard methods relying only on QALYsprovide biased estimates of true treatment effect.

An alternative approach is to estimate coefficients based onthe full health profile and then re-interpret effects in theQALY domain based on predicted probabilities in thefirst-stage regressions.




Thank You




Bibliography I

Austin, P.C. 2002. A comparison of methods for analyzing health-related quality-of-life measures. Value in Health,5(4), 329–337.

Austin, P.C., Escobar, M., & Kopec, J.A. 2000. The use of the Tobit model for analyzing measures of healthstatus. Quality of Life Research, 9(8), 901–910.

Basu, A., & Manca, A. 2012. Regression Estimators for Generic Health-Related Quality of Life andQuality-Adjusted Life Years. Medical Decision Making, 32(1), 56–69.

Brazier, J., & Ratcliffe, J. 2007. Measuring and valuing health benefits for economic evaluation. Oxford UniversityPress, USA.

Brazier, J., Roberts, J., & Deverill, M. 2002. The estimation of a preference-based measure of health from theSF-36. Journal of health economics, 21(2), 271–292.

Devlin, N.J., Parkin, D., & Browne, J. 2010. Patient-reported outcome measures in the NHS: new methods foranalysing and reporting EQ-5D data. Health economics, 19(8), 886–905.

Drummond, M.F., Sculpher, M.J., & Torrance, G.W. 2005. Methods for the economic evaluation of health careprogrammes. Oxford University Press, USA.

Gutacker, N., Bojke, C., Daidone, S., Devlin, N., & Street, A. 2012. Analysing Hospital Variation in HealthOutcome at the Level of EQ-5D Dimensions.

Manca, A., Hawkins, N., & Sculpher, M.J. 2005. Estimating mean QALYs in trial-based cost-effectiveness analysis:the importance of controlling for baseline utility. Health economics, 14(5), 487–496.

Mortimer, D., & Segal, L. 2008. Comparing the incomparable? A systematic review of competing techniques forconverting descriptive measures of health status into QALY-weights. Medical decision making, 28(1), 66.

Parkin, D., Rice, N., & Devlin, N. 2010. Statistical analysis of EQ-5D profiles: does the use of value sets biasinference? Medical Decision Making, 30(5), 556–565.

Richardson, G., & Manca, A. 2004. Calculation of quality adjusted life years in the published literature: a review ofmethodology and transparency. Health economics, 13(12), 1203–1210.


robust methods for health-related quality-of-life assessment

Healthcare

assessment of health

healthrelated quality

qalys marginal eects

dimensionsdomains of

common methodology

social functioning sf

mental health mh mh

inconsistent marginal