the reliability and unreliab ility of sub-group analyses
DESCRIPTION
THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES JEFFREY L. PROBSTFIELD, MD, FACP, FACC, FAHA, FESC, FSCT Professor of Medicine (Cardiology) University of Washington Research grants- Abbott, Boehringer Ingelheim, King, Sanofi-Aventis Pharmaceuticals, NHLBI, NCI; - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/1.jpg)
THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES
JEFFREY L. PROBSTFIELD, MD, FACP, FACC, FAHA, FESC, FSCTProfessor of Medicine (Cardiology)
University of Washington
Research grants- Abbott, Boehringer Ingelheim, King, Sanofi-Aventis Pharmaceuticals, NHLBI, NCI;
Consultantship-King Pharmaceuticals; no stocks, options or BOD positions
![Page 2: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/2.jpg)
DILEMMA
“The response of the average patient to therapy is not necessarily the response of
the patient being treated.”
Bernard, 1865
![Page 3: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/3.jpg)
SUBGROUP ANALYSES DRIVEN BY:
“Should all patients be given XYZ before, during, or after ABC or
can/should treatment be limited to a select group?”
![Page 4: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/4.jpg)
SUBGROUPS- PETO
• Only one thing is worse than doing subgroup analyses---believing the results
![Page 5: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/5.jpg)
PETO: HOW TO SPOIL A GOOD TRIAL RESULT
1. Undertake many data-dependent subgroup analyses.
2. Find some subgroups where treatment has no significant effect (or even, perhaps, no apparent effect whatsoever).
3. Publish the findings in such a way that many readers believe them.
![Page 6: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/6.jpg)
CARE RESULTS - NO CHD RISK REDUCTION BELOW LDL-C 125 mg/dL
“…Although our finding cannot be considered definitive and requires confirmation, it suggests that an LDL cholesterol level of 125 mg per deciliter may be an approximate lower boundary for a clinically important influence of the LDL cholesterol level on coronary heart disease…”
NEJM 1996;335:1001-1009
![Page 7: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/7.jpg)
![Page 8: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/8.jpg)
CHOLESTEROL LEVELS AND CHD RISK REDUCTION
Cholesterol and Recurrent Events (CARE)
4159 Participants
Participants
Plasma With Events % Risk
LDL-C Placebo Pravastatin Reduction
137 269 210 23 (8 to 36)
> 137 280 220 24 (10 to 36)
NEJM 1996;335:1001-1009
![Page 9: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/9.jpg)
CHOLESTEROL LEVELS AND CHD RISK REDUCTION
Cholesterol and Recurrent Events (CARE)
4159 Participants
Participants
Plasma with Event % Risk
LDL-C Placebo Pravastatin Reduction
125 93 89 -3 (-38 to 23)
125-150 311 239 26(13 to 38)
> 150-175 145 102 35(17 to 50)
NEJM 1996;335:1001-1009
![Page 10: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/10.jpg)
CARE SUBGROUP ANALYSIS RISK REDUCTION BELOW BASELINE LDL-C
mg/dLConcerns about divisions described
Plasma Participants %CHD Risk
LDL-C N Reduction 95% CI
> x(20%) 850 N/A N/A
> 150mg/dL 953 35 17 to 50
125 - 150 2355 26 13 to 38
< 137.5 2090 23 8 to 36
< 130 1386 15 N/A
< 127 1034 10 N/A
< 125(20%) 851 -3 -38 to 23
![Page 11: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/11.jpg)
HPSLDL-C N RRR CHD
<3 (<116) 6793 33%>3<3.5 5063 25% >3.5 8680 42%
LDL-C N Δ LDL-C E RRR CHD
<100 3421 -35 69 22%100-130 7068 -37 86 28%>130 9927 -39 104 24%
No interaction with Vitamin Cocktail
![Page 12: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/12.jpg)
PROPER SUBGROUP
“A common set
of baseline parameters”
![Page 13: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/13.jpg)
IMPROPER SUBGROUP:
“Characterized by a variable
measured after randomization”
![Page 14: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/14.jpg)
Hypokalemia Associated With Diuretic Use and CV Events in SHEP
Franse, et al. Franse, et al. Hypertension. 2000;35:1025-1030
![Page 15: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/15.jpg)
CHD Event Rate by Year 1 KCHD Event Rate by Year 1 K++ Strata Strata
0.00
0.03
0.06
0.09
0.12
0.15
Cu
mu
lati
ve C
HD
Eve
nt
Rat
e
0 1 2 3 4 5
Years to CHD
Y1 K+ = < 3.5
Y1 K+ = 3.5-5.4
Y1 K+ = > 5.4
HR 95% CIHyper/Normo-K+ 1.28 0.69, 2.40Hypo/Normo-K+ 1.03 0.82, 1.30
![Page 16: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/16.jpg)
EXAMPLES OF IMPROPER SUBGROUPS:
1. Responders vs. nonresponders
2. Adherers vs Non-adherers
![Page 17: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/17.jpg)
FIVE-YEAR MORTALITY ACCORDING TO BASE-LINE CHOLESTEROL AND CHANGE
FROM BASE-LINE, ADJUSTED FOR 40 BASE-LINE CHARACTERISTICS
Treatment Group
Clofibrate Placebo
BaselineCholesterol
MG/DL
CholesterolChange
No. of Pts. % mortality No. of Pts. % mortality
< 250 All men 507 20.0 ± 1.8 1319 19.9 ± 1.1
> 250 All men 490 17.5 ± 1.7 1216 20.6 ± 1.2
All men Fall 680 17.2 ± 1.4 1376 20.7 ± 1.1
All men Rise 317 22.2 ± 2.3 1159 19.7 ± 1.2
< 250 Fall 295 16.0 ± 2.1 614 21.2 ± 1.6
< 250 Rise 212 25.5 ± 3.0 705 18.7 ± 1.5
> 250 Fall 385 18.1 ± 2.0 762 20.2 ± 1.5
> 250 Rise 105 15.5 ± 3.5 454 21.3 ± 1.9
![Page 18: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/18.jpg)
FIVE-YEAR MORTALITY: PATIENTS GIVEN CLOFIBRATE OR PLACEBO, ACCORDING TO CUMULATIVE ADHERENCE TO PROTOCOL
PRESCRIPTION
Treatment Group
Clofibrate Placebo
No. of Pts. % mortality No. of Pts. % mortality
< 80% 357 24.6 ± 2.3(22.5)
882 28.2 ± 1.5(25.8)
> 80% 708 15.0 ± 1.3(15.7)
1813 15.1 ± 0.8(16.4)
Total studygroup
1065 18.2 ± 1.2(18.0)
2695 19.4 ± 0.8(19.5)
![Page 19: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/19.jpg)
STRUCTURED HYPOTHESES
• Carefully state hypothesis
• Allow analyses to capture the effect
![Page 20: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/20.jpg)
INTERACTION (Differential Subgroup Effect)
“A treatment effect that
differs by subgroup.”
![Page 21: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/21.jpg)
QUANTATIVE INTERACTION
Different amount (quantity) of
benefit in various subgroups.
![Page 22: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/22.jpg)
QUALITATIVE INTERACTION
True Benefit in some subgroups
and True Harm in others
(1 of over 700)
![Page 23: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/23.jpg)
QUALITATIVE DIFFERENCES -WHY NOT?
• Extremes excluded
• Lack of replication in other studies
![Page 24: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/24.jpg)
BIASES AND ERRORS IN DETERMINING SUBGROUP
EFFECTS
1. Subgroups lack statistical power
2. Random variation - widely divergentestimates of treatment benefit
3. Statistical multiplicity
4. Post-hoc analyses - extreme results theproduct of random errors
5. Replication - a posteriori vs. a priori
![Page 25: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/25.jpg)
FURBERG AND BYINGTONCIRCULATION 1983;67:I98-I101
• 146 Subgroups in BHAT: Few defined a priori
• Distribution of subgroup results - Gaussian
• Impact of change on data set inversely related to sample size. (Participants or deaths gives similar distribution)
![Page 26: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/26.jpg)
![Page 27: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/27.jpg)
3 CRITERIA FOR CONFIDENCE IN SUBGROUP FINDING
• Dose response relationship
• Independent findings within the study
• Replication by outside trial
![Page 28: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/28.jpg)
EXPECTED EFFECTS OF TRIAL SIZE ON TRIAL RESULTS
Total no. of deaths in trial
(treated+control)
(Approx. no. of patients
randomized if risk 10%
Approx Probability of failing to achieve
1 P<0.01 significance if true risk reduction is
1/4
Comments that might be made of size before trial
begins
0-50 (under 500) Over 0.9 Utterly inadequate
50-150 (1000) 0.7-0.9 Probably inadequate
150-350 (3000) 0.3-0.7 Probably adequate, possibly not
350-650 (6000) 0.1-0.3 Possibly adequate
Over 650 (10,000) Under 0.1 Definitely Adequate
![Page 29: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/29.jpg)
Actual effects of trial size on trial results. Relationship between the total number of deaths in the two treatment groups and the result actually attained, in the 24 trials of a treatment (long-term beta-
blockade) that reduces the odds of death by about 22 + 4%
Total no. of deaths in
trial
(β-bl.+plac.)
(Mean no. of patients
randomized)
Statistical power
P<0.5 against
Non-sigt. against
Non-sigt. favorable
P<0.5 favorable
0-50 (255) Utterly Inadequate
0 5 5 0
50-150 (861) Probably Inadequate
0 1 9 1
150-350 (2925) Possibly adequate,
probably not
0 0 1 2
350-650 (No such β -bl. trials
exist
Probably Adequate
- - - -
Over 650 No such β -bl. trials
exist
Definitely Adequate
- - - -
TOTAL (866) Inadequate separately,
adequate only in aggregate
0 6 15 3
No. of trials resulting in:
![Page 30: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/30.jpg)
SUBGROUP EFFECT
Treatment effect in a specific proper subgroup.
Must be significantly different from
overall effect!!
![Page 31: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/31.jpg)
HYPOTHETICAL SUBGROUP EFFECTS ILLUSTRATING THE “PLAY OF CHANCE” IN A TRIAL THAT SHOWS
CLEAR OVERALL BENEFIT
(%) (%) RiskDecrease
(%)
p Value
Overallresult
240/3,000(8)
300/3,000(10)
20 < 0.01
Subgroup A 80/1,000(8)
100/1,000(10)
20 NS
Subgroup B 70/1,000(7)
110/1,000(11)
36 < 0.001
Subgroup C 90/1,000(9)
90/1,000(9)
0 NS
![Page 32: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/32.jpg)
MULTIPLE COMPARISONSExample:
• 1,000 participants Mortality Rate = 10%
Treat A Treat B Treatments equally effective• 10 subgroups (equal size) randomly formed
Relative Risk Probability
Reduction to: (percent)
.5 99
.33 80
.1 5• Nominal p value - inappropriate
• Conservative approach - p/Sn
• Especially important in trial where main outcome is
not statistically significant
![Page 33: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/33.jpg)
1.25
Subgroup
% pts
Overall
100
Beta blocker (yes)
35
no beta blocker
65
ACEI (yes)
93
no ACEI*
7
1.00.750.50
.87.77 .97
Valsartan better Valsartan worse
Mortality and Morbidity
44.0% , P=.0002
Cohn. N Engl J Med. 2001; *FDA analysis/package insert
RR of death P
no ACEI 41%<0.05*
ACEI + Beta blocker 42% 0.009
Subgroup Results
*n=366
![Page 34: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/34.jpg)
Diabetes No 680/2715 815/2721Yes 470/1088 495/1075
Hyper- No 484/1710 579/1703tension Yes 666/2093 731/2093
ACEIs No 586/2230 688/2244 Yes 564/1573 622/1552
Beta No 611/1701 710/1695blocker Yes 539/2102 600/2101
Spirono- No 880/3160 1041/3167lactone Yes 270/643 269/629
Overall 1150/3803 1310/3796
Test for interaction
P=0.09
P=0.51
P=0.32
P=0.19
P=0.17
candesartan better
Hazard ratio
placebo better
0.6 0.8 1.0 1.2 1.4
Candesartanevent/n
Placeboevent/n
CV Death or Hospitalization for CHF
![Page 35: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/35.jpg)
EXAMPLE OF “SUBGROUPING” IN INTERNATIONAL SOCIETY FOR THE
INVESTIGATION OF STRESS-2: ASTROLOGY AND ASPIRIN
Vascular Mortality at Week 5
Aspirin (%) Placebo (%) OddsDecrease(% ± SD)
Patients bornunder Libraand Gemini
150/1,357(11.1)
147/1,442(10.2)
8% adverse(NS)
Patients bornunder other“birth signs”
654/7,228(9.0)
868/7,157(12.1)
26% ± 5(p<0.0001)
Overall results 804/8,587(9.4)
1,016/8,600(11.8)
23% ± 4(p<0.0001)
![Page 36: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/36.jpg)
ORDERED SUBGROUPS
• Strong biological rationale
• Reflects natural ordering
• Correct for multiplicity
• Only indicate those as significant which have a p value less than p/Sn
![Page 37: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/37.jpg)
Reduction of Stroke According to Sex
N Active Rx
Placebo % Difference
Males 2046 5.5% 8.7% -38%
Females 2690 4.7% 7.3% -31%
Stroke Rates
![Page 38: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/38.jpg)
GISSI-1• Overall result - streptokinase treatment,
20% reduction in total mortality
• Benefit confined to:
– Anterior MI
– Age 65 years
– Treatment 6 hours
• Subsequent trials and pooled results do not confirm
![Page 39: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/39.jpg)
HYPOTHETICAL EXAMPLE OF ORDERED SUBGROUPS: RELATIVE RISK AS A
FUNCTION OF TIME OF THROMBOLYTIC THERAPY
14-d Mortality (%) Hours After Pain Treated Control
Relative Risk
P
≤ 2 10 20 .50 .01
> 2 – 4 11 16 .70 NS
> 4 – 8 17 22 .77 NS
> 8 – 12 20 25 .80 NS
> 12 23 24 .95 NS
Overall 14 21 .67 .001
![Page 40: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/40.jpg)
SUBGROUPS DEFINED A-PRIORI
• Suggestive differential subgroup effect
• State in design of new trial
• Publish (multiplicity, design analysis, plan over-sampling)
• Test within an existing data set
![Page 41: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/41.jpg)
STROKE SUBGROUP HYPOTHESIS
On BP Meds at ICV Off BP Meds at ICV
35% of participants 65% of participants
Net reduction in Net reduction in
stroke rate =10% stroke rate =40%
80% power to detect 30% treatment difference
![Page 42: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/42.jpg)
STROKE EVENTS BY MEDICATION STATUS
GROUP N NFS FS CSNot on MedicationsActive 1584 64 5 67Placebo 1577 88 11 96
6.72= 2א Relative Risk (active/placebo)= 0.69P = .0096 95% CI = 0.51-0.95
On MedicationsActive 781 32 5 36Placebo 794 61 3 63
5.11 = 2א Relative Risk (active/placebo)= 0.57 P = .0237 95% CI = 0.38-0.85
![Page 43: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/43.jpg)
MRFIT RESEARCH GROUP, AM J CARDIOL 1985;55:1-15
ECG ABNORMALITIES AT BASELINE
Present Absent
PersonYrs.
No. ofDeaths
Per 1000Per Yrs.
PersonYrs.
No. ofDeaths
Per 1000Per Yrs.
Not onDiuretic
6074 9 1.48 17,356 35 2.02
On Diuretic 6433 38 5.91 14,399 33 2.29
![Page 44: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/44.jpg)
SUBGROUP HYPOTHESIS
Will the treatment of ISH reduce the frequency of major coronary events more in those free of baseline ECG
abnormalities than in those with such abnormalities?
![Page 45: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/45.jpg)
OTHER COMBINED EVENTS BY TREATMENT GROUP
Event Active Placebo Rel. Risk 95% CI
Nonfatal MI/CHD Death
104 141 0.73 0.57-0.94
Stroke/MI/CHD/Death
199 289 0.67 0.56-0.81
CHD 140 184 0.75 0.60-0.94
CVD 289 414 0.68 0.58-0.79
![Page 46: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/46.jpg)
NONFATAL MI & CHD DEATH BY BASELINE ECG ABNORMALITIES
TreatmentNumber Events Rate per 100 (SE)With baseline ECG abnormalitiesActive 1429 67 6.0 (0.8)Placebo 1426 96 8.0 (0.9)
5.73 = 2א Rel. Risk = 0.69P = 0.02 95% CI = 0.50-0.94
Without baseline ECG abnormalitiesActive 903 35 4.5 (0.8)Placebo 922 43 4.6 (0.7)
0.70 = 2א Rel. Risk = 0.83P = 0.40 95% CI = 0.53-1.29
![Page 47: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/47.jpg)
SUDDEN DEATH BY BASELINE ECG ABNORMALITIES
TreatmentNumber Events Rate per 100 (SE)With baseline ECG abnormalitiesActive 1429 15 1.2 (0.3)Placebo 1426 17 1.3 (0.4)
0.14 = 2א Rel. Risk = 0.88P = 0.71 95% CI = 0.44-1.75
Without baseline ECG abnormalitiesActive 903 8 1.2 (0.4)Placebo 922 5 0.6 (0.3)
0.77 = 2א Rel. Risk = 1.64P = 0.38 95% CI = 0.54-5.01
![Page 48: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/48.jpg)
![Page 49: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/49.jpg)
SUBGROUPS DEFINED A-POSTERIORI
• “Grist” for formulating hypothesis
• Watch for alternative definitions!
• Should be clearly stated and reported as an estimate of effect with appropriate confidence interval
![Page 50: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/50.jpg)
SUBGROUPS AND MONITORING TRIALS
• Use statistically sound monitoring method
• Interference with main trial endpoint - rare
• Formulate hypothesis and test prospectively
• Terminate subgroup
• “Mega trials” - special problems
![Page 51: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/51.jpg)
BUCHWALD AND COLLEAGUES: POSCH, NEJM1990;323;946-955
• Cholesterol lowering by ileal bypass for secondary prevention of CHD
• 838 participants randomized
• Primary outcome-Fatal and non-fatal CHD– 62 vs 49, p=0.164
• Subgroup, EF<50% vs >50%, – Final 24 vs 39, p=0.021– post-hoc analysis 6 vs 17– after observed phenomenon - 18 vs 22
![Page 52: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/52.jpg)
ESSENTIALS IN SUBGROUP REPORTING
• PRESPECIFICATION OF SUBGROUPS • NUMBER OF SUBGROUPS• NUMBER OF SUBGROUP OUTCOMES• STATISTICAL METHODS (INTERACTION)• NUMBER OF SIGNIFICANT SUBGROUPS FOUND• EMPHASIS OF SUBGROUP VS PRIMARY
OUTCOME
![Page 53: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/53.jpg)
Hernandez AV, et al. Am Heart J. 2006;151:257-64.
SUGGESTIONS TO APPROPRIATELY PERFORMAND INTERPRET SUBGROUP ANALYSIS
![Page 54: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/54.jpg)
DICTUM 1
“The treatment effect in all subgroups of patients without obvious contraindications to treatment is likely to be qualitatively (in the same direction) similar.”
![Page 55: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/55.jpg)
DICTUM 2
“The treatment effect in all subgroups of patients without obvious contraindications to treatment is likely to be quantitatively (difference in degree) dissimilar even when effects appear to be identical.”
![Page 56: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/56.jpg)
DICTUM 3
“Estimates of treatment effect within a subgroup chosen for special emphasis are usually ‘biased’ and so the most appropriate estimate in a subgroup is closer to the overall result.”
![Page 57: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/57.jpg)
KEY POINTS IN SUBGROUP ANALYSES & INTERPRETATION I
Design
1. State clearly a few important and plausible subgroup hypotheses in advance. Include the direction of the expected effect.
2. Rank the subgroup hypotheses in order or plausibility.
3. Calculate power to detect key subgroup effects. If it is inadequate, consider building adequate power to detect key subgroup effects.
4. State whether the trial will be continued even after the overall results are convincing but the subgroup effects are not significant. Decide whether the primary method of monitoring will focus on the subgroups or on the overall trial.
5. State primary analytical methods in advance.
![Page 58: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/58.jpg)
KEY POINTS IN SUBGROUP ANALYSES AND INTERPRETATION II
Monitoring a trial
1. Rigorous evidence of benefit or harm in subgroups postulated a priori: consider selective discontinuation of that subgroup.
2. Evidence of benefit or harm in unexpected subgroups: postulate a hypothesis to be tested in the remaining part of the study.
![Page 59: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/59.jpg)
KEY POINTS IN SUBGROUP ANALYSES & INTERPRETATION III
Analyses and interpretation
1. Use statistical methods that capture the framework of the prior hypotheses.
2. Place greater emphasis on the overall results than on what may be apparent within a particular subgroup.
3. Distinguish between prior and data-derived hypotheses. Do not calculate p values for data-derived hypotheses because such p values usually bear little resemblance to what could occurs if the hypothesis were tested independently in another study.
4. Use tests of “interactions” and/or correct for multiplicity of statistical comparisons. (“Nominal” p values are usually misleading.)
5. Interpret the results in the context of similar data from other trials, from the architecture of the entire set of data on all patients, and from principles of biological coherence.
![Page 60: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/60.jpg)
KEY POINTS IN SUBGROUP ANALYSES AND INTERPRETATION IV
Improper subgroups and analyses
1. Avoid analyses of subgroups based on post-randomization response, adherence, etc.
2. Avoid emphasizing nominal p values.
3. Do not emphasize data-derived analyses or analyses based on post-hoc definitions of subgroups.
![Page 61: THE RELIABILITY AND UNRELIAB ILITY OF SUB-GROUP ANALYSES](https://reader035.vdocuments.us/reader035/viewer/2022062409/56814887550346895db599d2/html5/thumbnails/61.jpg)