comparing rtw outcomes between vocational rehabilitation providers after adjusting … · 2010. 5....
TRANSCRIPT
![Page 1: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/1.jpg)
Comparing RTW outcomes between vocational rehabilitation providers
after adjusting for case mixusing statistical models
Jim Gaetjens
![Page 2: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/2.jpg)
ContextWorkCover SA Non-exempt claims expenditure 2008/09• Total = $575m• Income maintenance = $198m• Vocational rehabilitation = $22m• 60% of IM claims have vocational rehab
> % of IM cost much morebased on 2007/08 IM claims vocational rehabilitation to date
![Page 3: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/3.jpg)
Judging provider performance
Who is best at achieving RTW?
• RTW adjusted for case mix= performance + residual bits
• Other information> e.g. file reviews, service costs
![Page 4: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/4.jpg)
Measuring RTW
• income maintenance reduction• except retirement, redemption and death
“Reduction” includes full and partial RTW
![Page 5: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/5.jpg)
Measuring RTW
Injury date Referral date
Baselineperiod
3 months
3 monthoutcome period
6 monthoutcome period
3 months 3 month
Outcomes measured 3, 6, 9 and 12 months after referral date
![Page 6: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/6.jpg)
Measuring RTW
Injury date Referral date
Baselineperiod
3 months
3 monthoutcome period
6 monthoutcome period
3 months 3 month
“Incapacity” = IM paid ÷Full IM entitlement
“RTW” = Baseline incapacity minusOutcome period incapacity
![Page 7: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/7.jpg)
Measuring RTW
Injury date Referral date
Baselineperiod
3 months
3 monthoutcome period
6 monthoutcome period
3 months 3 month
Special treatment of redeemed claims• Outcome IM at pre-redemption level• pre-redemption partial RTW recognised
![Page 8: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/8.jpg)
Measuring RTWMeasures are driven by
1. provider performance> full, partial and sustained RTW
2. claim characteristics
3. other
![Page 9: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/9.jpg)
Adjusting for claim characteristics
2/3 of variation between providersfrom claim characteristics
Measures are driven by
1. provider performance> full, partial and sustained RTW
2. claim characteristics
3. other
![Page 10: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/10.jpg)
Adjusting for claim characteristics
Remove 2. => better indicator of 1.
Measures are driven by
1. provider performance> full, partial and sustained RTW
2. claim characteristics
3. other
![Page 11: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/11.jpg)
Adjusting for claim characteristicsFor each provider ...
• Expected RTW= predicted (or “scheme average”)
given the claim characteristics
• Performance indicator (“CAPO”) = Actual RTW minus Expected RTW
![Page 12: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/12.jpg)
Adjusting for claim characteristics
“CAPO” stands for“Characteristic Adjusted Performance Outcome”
For each provider ...
• Expected RTW= predicted (or “scheme average”)
given the claim characteristics
• Performance indicator (“CAPO”) = Actual RTW minus Expected RTW
![Page 13: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/13.jpg)
Adjusting for claim characteristics
An indicator of relative performanceRelative to the average of other providers
For each provider ...
• Expected RTW= predicted (or “scheme average”)
given the claim characteristics
• Performance indicator (“CAPO”) = Actual RTW minus Expected RTW
![Page 14: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/14.jpg)
Adjusting for claim characteristics
Positive CAPO = better than averageNegative CAPO = worse than average
given the claim characteristics
![Page 15: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/15.jpg)
Adjusting for claim characteristics
Actual RTW = Baseline Incapacity minusActual Outcome Incapacity
Expected RTW = Baseline Incapacity minusExpected Outcome Incapacity
![Page 16: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/16.jpg)
Calculating Expected RTWFor an individual claim ...
Expected RTW = Baseline Incapacity – Expected incapacity
Expected incapacity =
outcome
outcome
![Page 17: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/17.jpg)
Calculating Expected RTW
Expected outcome incapacityconstrained between 0 and 1
For an individual claim ...
Expected RTW = Baseline Incapacity – Expected incapacity
Expected incapacity =
outcome
outcome
![Page 18: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/18.jpg)
Calculating Expected RTW
Provider result is an average over all their claims
For an individual claim ...
Expected RTW = Baseline Incapacity – Expected incapacity
Expected incapacity =
outcome
outcome
![Page 19: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/19.jpg)
Calculating Expected RTW
Claim characteristics must be• recorded on administrative database• measured accurately and consistently• available for all in-scope claims• measurable as at referral date
![Page 20: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/20.jpg)
Calculating Expected RTWVariables analysed -• Baseline incapacity• Worker age• Sex• Occupation• Claim duration• Nature of injury• Body location
• RTW objective:Pre-injury vs Newemployer
• Employer size• Employer industry• Metro vs country• Expenditure by type
![Page 21: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/21.jpg)
Calculating Expected RTW
Criteria to select variables• Statistical significance• Practical significance• Significance judged by partial residual plots• Improved fit judged by partial residual plots
![Page 22: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/22.jpg)
Calculating Expected RTWVariables selected -• Baseline incapacity• Worker age• Claim duration (log transformation)• Selected nature of injury / body location• Income maintenance last 6 months• Medical costs last 6 months• RTW objective: Pre-injury vs New employer
![Page 23: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/23.jpg)
Calculating Expected RTW
Example: 6 month outcome, Pre-injury employer
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Baseline incapacity
Out
com
e in
capa
cityOutcome incapacity - actual
predicted
![Page 24: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/24.jpg)
Calculating Expected RTW
Example: 6 month outcome, Pre-injury employer
0%
20%
40%
60%
80%
100%
15 30 45 60Age at referral date
Out
com
e in
capa
cityOutcome incapacity - actual
predicted
![Page 25: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/25.jpg)
Calculating Expected RTW
Example: 6 month outcome, Pre-injury employer
0%
20%
40%
60%
80%
100%
0 100 200 300 400 500Claim duration at referral date (w eeks)
Out
com
e in
capa
city
Outcome incapacity - actual
predicted
![Page 26: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/26.jpg)
Application
Example: 6 month outcome, Pre-injury employer
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Prov
1
Prov
2
Prov
3Pr
ov4
Prov
5Pr
ov6
Prov
7
Prov
8Pr
ov9
Prov
10Pr
ov11
Prov
12
Prov
13Pr
ov14
Prov
15
Prov
16Pr
ov17
Prov
18Pr
ov19
Prov
20
Prov
21Pr
ov22
Prov
23Pr
ov24
Prov
25
Prov
26
RTW_Actual
RTW_ExpectedProportion of provider varianceremoved by model = 64%
![Page 27: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/27.jpg)
Application
Example: 6 month outcome, Pre-injury employer
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Prov
1
Prov
2
Prov
3Pr
ov4
Prov
5Pr
ov6
Prov
7
Prov
8Pr
ov9
Prov
10Pr
ov11
Prov
12
Prov
13Pr
ov14
Prov
15
Prov
16Pr
ov17
Prov
18Pr
ov19
Prov
20
Prov
21Pr
ov22
Prov
23Pr
ov24
Prov
25
Prov
26
RTW_Actual
RTW_ExpectedProv21 RTW looks fantastic
![Page 28: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/28.jpg)
Application
Example: 6 month outcome, Pre-injury employer
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Prov
1
Prov
2
Prov
3Pr
ov4
Prov
5Pr
ov6
Prov
7
Prov
8Pr
ov9
Prov
10Pr
ov11
Prov
12
Prov
13Pr
ov14
Prov
15
Prov
16Pr
ov17
Prov
18Pr
ov19
Prov
20
Prov
21Pr
ov22
Prov
23Pr
ov24
Prov
25
Prov
26
RTW_Actual
RTW_ExpectedProv21 RTW looks fantasticIn fact, is due to claim characteristics
![Page 29: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/29.jpg)
Application
Example: 6 month outcome, Pre-injury employer
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Prov
1
Prov
2
Prov
3Pr
ov4
Prov
5Pr
ov6
Prov
7
Prov
8Pr
ov9
Prov
10Pr
ov11
Prov
12
Prov
13Pr
ov14
Prov
15
Prov
16Pr
ov17
Prov
18Pr
ov19
Prov
20
Prov
21Pr
ov22
Prov
23Pr
ov24
Prov
25
Prov
26
RTW_Actual
RTW_ExpectedProv19 is the only one withsignificantly (barely) good CAPO
(p-value = 0.078)
![Page 30: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/30.jpg)
Application
Example: 6 month outcome, Pre-injury employer
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Prov
1
Prov
2
Prov
3Pr
ov4
Prov
5Pr
ov6
Prov
7
Prov
8Pr
ov9
Prov
10Pr
ov11
Prov
12
Prov
13Pr
ov14
Prov
15
Prov
16Pr
ov17
Prov
18Pr
ov19
Prov
20
Prov
21Pr
ov22
Prov
23Pr
ov24
Prov
25
Prov
26
RTW_Actual
RTW_ExpectedSeveral significantly poor
![Page 31: Comparing RTW outcomes between vocational rehabilitation providers after adjusting … · 2010. 5. 18. · Jim Gaetjens. Context WorkCover SA Non-exempt claims expenditure 2008/09](https://reader036.vdocuments.us/reader036/viewer/2022071423/611e31553685cf6da9097d6d/html5/thumbnails/31.jpg)
Conclusions
• Objective comparison of providers• Measures full, partial and sustained RTW• Much non-performance variation removed• Must be supplemented by other information• Influences referral patterns• Overall system rewards best performance
and checks poor performance