the treachery of sub-group analyses in cvd clinical trials
DESCRIPTION
The Treachery of Sub-group Analyses in CVD Clinical Trials. JEFFREY L. PROBSTFIELD, MD, FACP, FACC, FAHA, FESC, FSCT DIRECTOR, CLINICAL TRIALS SERVICE UNIT PROFESSOR OF MEDICINE (CARDIOLOGY) UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE ADJUNCT PROFESSOR OF EPIDEMIOLOGY - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/1.jpg)
The Treachery of Sub-group Analyses The Treachery of Sub-group Analyses in CVD Clinical Trialsin CVD Clinical Trials
JEFFREY L. PROBSTFIELD, MD, FACP, FACC, FAHA, FESC, FSCT
DIRECTOR, CLINICAL TRIALS SERVICE UNITPROFESSOR OF MEDICINE (CARDIOLOGY)UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINEADJUNCT PROFESSOR OF EPIDEMIOLOGYUNIVERSITY OF WASHINGTON SCHOOL OF PUBLIC HEALTH AND COMMUNITY MEDICINE
![Page 2: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/2.jpg)
DILEMMA
“The response of the average patient to therapy is not necessarily the response of
the patient being treated.”
Bernard, 1865
![Page 3: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/3.jpg)
SUBGROUP ANALYSES DRIVEN BY:
“Should all patients be given XYZ before, during, or after ABC or
can/should treatment be limited to a select group?”
![Page 4: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/4.jpg)
SUBGROUPS- PETO
• Only one thing is worse than doing subgroup analyses---believing the results
![Page 5: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/5.jpg)
PETO: HOW TO SPOIL A GOOD TRIAL RESULT
1. Undertake many data-dependent subgroup analyses.
2. Find some subgroups where treatment has no significant effect (or even, perhaps, no apparent effect whatsoever).
3. Publish the findings in such a way that many readers believe them.
![Page 6: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/6.jpg)
CARE RESULTS - NO CHD RISK REDUCTION BELOW LDL-C 125 mg/dL
“…Although our finding cannot be considered definitive and requires confirmation, it suggests that an LDL cholesterol level of 125 mg per deciliter may be an approximate lower boundary for a clinically important influence of the LDL cholesterol level on coronary heart disease…”
NEJM 1996;335:1001-1009
![Page 7: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/7.jpg)
![Page 8: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/8.jpg)
CHOLESTEROL LEVELS AND CHD RISK REDUCTION
Cholesterol and Recurrent Events (CARE)
4159 Participants
Participants
Plasma with Event % Risk
LDL-C Placebo Pravastatin Reduction
125 93 89 -3 (-38 to 23)
125-150 311 239 26(13 to 38)
> 150-175 145 102 35(17 to 50)
NEJM 1996;335:1001-1009
![Page 9: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/9.jpg)
CHOLESTEROL LEVELS AND CHD RISK REDUCTION
Cholesterol and Recurrent Events (CARE)
4159 Participants
Participants
Plasma With Events % Risk
LDL-C Placebo Pravastatin Reduction
137 269 210 23 (8 to 36)
> 137 280 220 24 (10 to 36)
NEJM 1996;335:1001-1009
![Page 10: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/10.jpg)
CARE SUBGROUP ANALYSIS RISK REDUCTION BELOW BASELINE LDL-C
mg/dLConcerns about divisions described
Plasma Participants %CHD Risk
LDL-C N Reduction 95% CI
> x(20%) 850 N/A N/A
> 150mg/dL 953 35 17 to 50
125 - 150 2355 26 13 to 38
< 137.5 2090 23 8 to 36
< 130 1386 15 N/A
< 127 1034 10 N/A
< 125(20%) 851 -3 -38 to 23
![Page 11: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/11.jpg)
HPSLDL-C N RRR CHD
<3 (<116) 6793 33%>3<3.5 5063 25% >3.5 8680 42%
LDL-C N Δ LDL-C E RRR CHD
<100 3421 -35 69 22%100-130 7068 -37 86 28%>130 9927 -39 104 24%
No interaction with Vitamin Cocktail
![Page 12: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/12.jpg)
PROPER SUBGROUP
“A common set
of baseline parameters”
![Page 13: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/13.jpg)
IMPROPER SUBGROUP:
“Characterized by a variable
measured after randomization”
![Page 14: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/14.jpg)
Hypokalemia Associated With Diuretic Use and CV Events in SHEP
Franse, et al. Franse, et al. Hypertension. 2000;35:1025-1030
![Page 15: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/15.jpg)
CHD Event Rate by Year 1 KCHD Event Rate by Year 1 K++ Strata Strata
0.00
0.03
0.06
0.09
0.12
0.15
Cu
mu
lati
ve C
HD
Eve
nt
Rat
e
0 1 2 3 4 5
Years to CHD
Y1 K+ = < 3.5
Y1 K+ = 3.5-5.4
Y1 K+ = > 5.4
ALLHAT HR 95% CIHyper/Normo-K+ 1.28 0.69, 2.40Hypo/Normo-K+ 1.03 0.82, 1.30
![Page 16: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/16.jpg)
EXAMPLES OF IMPROPER SUBGROUPS:
1. Responders vs. nonresponders
2. Adherers vs Non-adherers
![Page 17: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/17.jpg)
FIVE-YEAR MORTALITY ACCORDING TO BASE-LINE CHOLESTEROL AND CHANGE
FROM BASE-LINE, ADJUSTED FOR 40 BASE-LINE CHARACTERISTICS
Treatment Group
Clofibrate Placebo
BaselineCholesterol
MG/DL
CholesterolChange
No. of Pts. % mortality No. of Pts. % mortality
< 250 All men 507 20.0 ± 1.8 1319 19.9 ± 1.1
> 250 All men 490 17.5 ± 1.7 1216 20.6 ± 1.2
All men Fall 680 17.2 ± 1.4 1376 20.7 ± 1.1
All men Rise 317 22.2 ± 2.3 1159 19.7 ± 1.2
< 250 Fall 295 16.0 ± 2.1 614 21.2 ± 1.6
< 250 Rise 212 25.5 ± 3.0 705 18.7 ± 1.5
> 250 Fall 385 18.1 ± 2.0 762 20.2 ± 1.5
> 250 Rise 105 15.5 ± 3.5 454 21.3 ± 1.9
![Page 18: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/18.jpg)
FIVE-YEAR MORTALITY: PATIENTS GIVEN CLOFIBRATE OR PLACEBO, ACCORDING TO CUMULATIVE ADHERENCE TO PROTOCOL
PRESCRIPTION
Treatment Group
Clofibrate Placebo
No. of Pts. % mortality No. of Pts. % mortality
< 80% 357 24.6 ± 2.3(22.5)
882 28.2 ± 1.5(25.8)
> 80% 708 15.0 ± 1.3(15.7)
1813 15.1 ± 0.8(16.4)
Total studygroup
1065 18.2 ± 1.2(18.0)
2695 19.4 ± 0.8(19.5)
![Page 19: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/19.jpg)
STRUCTURED HYPOTHESES
• Carefully state hypothesis
• Allow analyses to capture the effect
![Page 20: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/20.jpg)
INTERACTION (Differential Subgroup Effect)
“A treatment effect that
differs by subgroup.”
![Page 21: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/21.jpg)
QUANTATIVE INTERACTION
Different amount (quantity) of
benefit in various subgroups.
![Page 22: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/22.jpg)
QUALITATIVE INTERACTION
True Benefit in some subgroups
and True Harm in others
(1 of over 700)
![Page 23: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/23.jpg)
QUALITATIVE DIFFERENCES -WHY NOT?
• Extremes excluded
• Lack of replication in other studies
![Page 24: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/24.jpg)
BIASES AND ERRORS IN DETERMINING SUBGROUP
EFFECTS
1. Subgroups lack statistical power
2. Random variation - widely divergentestimates of treatment benefit
3. Statistical multiplicity
4. Post-hoc analyses - extreme results theproduct of random errors
5. Replication - a posteriori vs. a priori
![Page 25: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/25.jpg)
FURBERG AND BYINGTONCIRCULATION 1983;67:I98-I101
• 146 Subgroups in BHAT: Only a few defined a priori
• Distribution of subgroup results - Gaussian
• Impact of change on data set inversely related to sample size. (Participants or deaths gives similar distribution)
![Page 26: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/26.jpg)
![Page 27: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/27.jpg)
3 CRITERIA FOR CONFIDENCE IN SUBGROUP FINDING
• Dose response relationship
• Independent findings within the study
• Replication by outside trial
![Page 28: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/28.jpg)
EXPECTED EFFECTS OF TRIAL SIZE ON TRIAL RESULTS
Total no. of deaths in trial
(treated+control)
(Approx. no. of patients
randomized if risk 10%
Approx Probability of failing to achieve
1 P<0.01 significance if true risk reduction is
1/4
Comments that might be made of size before trial
begins
0-50 (under 500) Over 0.9 Utterly inadequate
50-150 (1000) 0.7-0.9 Probably inadequate
150-350 (3000) 0.3-0.7 Probably adequate, possibly not
350-650 (6000) 0.1-0.3 Possibly adequate
Over 650 (10,000) Under 0.1 Definitely Adequate
![Page 29: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/29.jpg)
Actual effects of trial size on trial results. Relationship between the total number of deaths in the two treatment groups and the result actually attained, in the 24 trials of a treatment (long-term beta-
blockade) that reduces the odds of death by about 22 + 4%
Total no. of deaths in
trial
(β-bl.+plac.)
(Mean no. of patients
randomized)
Statistical power
P<0.5 against
Non-sigt. against
Non-sigt. favorable
P<0.5 favorable
0-50 (255) Utterly Inadequate
0 5 5 0
50-150 (861) Probably Inadequate
0 1 9 1
150-350 (2925) Possibly adequate,
probably not
0 0 1 2
350-650 (No such β -bl. trials
exist
Probably Adequate
- - - -
Over 650 No such β -bl. trials
exist
Definitely Adequate
- - - -
TOTAL (866) Inadequate separately,
adequate only in aggregate
0 6 15 3
No. of trials resulting in:
![Page 30: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/30.jpg)
SUBGROUP EFFECT
Treatment effect in a specific proper subgroup.
Must be significantly different from
overall effect!!
![Page 31: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/31.jpg)
HYPOTHETICAL SUBGROUP EFFECTS ILLUSTRATING THE “PLAY OF CHANCE” IN A TRIAL THAT SHOWS
CLEAR OVERALL BENEFIT
(%) (%) RiskDecrease
(%)
p Value
Overallresult
240/3,000(8)
300/3,000(10)
20 < 0.01
Subgroup A 80/1,000(8)
100/1,000(10)
20 NS
Subgroup B 70/1,000(7)
110/1,000(11)
36 < 0.001
Subgroup C 90/1,000(9)
90/1,000(9)
0 NS
![Page 32: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/32.jpg)
MULTIPLE COMPARISONSExample:
• 1,000 participants Mortality Rate = 10%
Treat A Treat B Treatments equally effective• 10 subgroups (equal size) randomly formed
Relative Risk Probability
Reduction to: (percent)
.5 99
.33 80
.1 5• Nominal p value - inappropriate
• Conservative approach - p/Sn
• Especially important in trial where main outcome is
not statistically significant
![Page 33: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/33.jpg)
1.25
Subgroup
% pts
Overall
100
Beta blocker (yes)
35
no beta blocker
65
ACEI (yes)
93
no ACEI*
7
1.00.750.50
.87.77 .97
Valsartan better Valsartan worse
Mortality and Morbidity
44.0% , P=.0002
Cohn. N Engl J Med. 2001; *FDA analysis/package insert
RR of death P
no ACEI 41%<0.05*
ACEI + Beta blocker 42% 0.009
Subgroup Results
*n=366
![Page 34: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/34.jpg)
34
CHARM-AddedPre-specified Subgroups, CV death,
or CHF Hosp.
Beta Yes 223/702 274/711blocker No 260/574 264/561
Recom. Yes 232/643 275/648dose of No 251/633 263/624ACEI
All patients 483/1276 538/1272
Candesartan Placebo
candesartan better
Hazard ratio
placebo better
0.6 0.8 1.0 1.2 1.4
P-value fortreatment interaction
0.14
0.26
![Page 35: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/35.jpg)
35
Diabetes No 680/2715 815/2721Yes 470/1088 495/1075
Hyper- No 484/1710 579/1703tension Yes 666/2093 731/2093
ACEIs No 586/2230 688/2244 Yes 564/1573 622/1552
Beta No 611/1701 710/1695blocker Yes 539/2102 600/2101
Spirono- No 880/3160 1041/3167lactone Yes 270/643 269/629
Overall 1150/3803 1310/3796
Test for interaction
P=0.09
P=0.51
P=0.32
P=0.19
P=0.17
candesartan better
Hazard ratio
placebo better
0.6 0.8 1.0 1.2 1.4
Candesartanevent/n
Placeboevent/n
CV Death or Hospitalization for CHF
![Page 36: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/36.jpg)
EXAMPLE OF “SUBGROUPING” IN INTERNATIONAL SOCIETY FOR THE
INVESTIGATION OF STRESS-2: ASTROLOGY AND ASPIRIN
Vascular Mortality at Week 5
Aspirin (%) Placebo (%) OddsDecrease(% ± SD)
Patients bornunder Libraand Gemini
150/1,357(11.1)
147/1,442(10.2)
8% adverse(NS)
Patients bornunder other“birth signs”
654/7,228(9.0)
868/7,157(12.1)
26% ± 5(p<0.0001)
Overall results 804/8,587(9.4)
1,016/8,600(11.8)
23% ± 4(p<0.0001)
![Page 37: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/37.jpg)
ORDERED SUBGROUPS
• Strong biological rationale
• Reflects natural ordering
• Correct for multiplicity
• Only indicate those as significant which have a p value less than p/Sn
![Page 38: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/38.jpg)
Reduction of Stroke According to Sex
N Active Rx
Placebo % Difference
Males 2046 5.5% 8.7% -38%
Females 2690 4.7% 7.3% -31%
Stroke Rates
![Page 39: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/39.jpg)
GISSI-1• Overall result - streptokinase treatment,
20% reduction in total mortality
• Benefit confined to:
– Anterior MI
– Age 65 years
– Treatment 6 hours
• Subsequent trials and pooled results do not confirm
![Page 40: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/40.jpg)
HYPOTHETICAL EXAMPLE OF ORDERED SUBGROUPS: RELATIVE RISK AS A
FUNCTION OF TIME OF THROMBOLYTIC THERAPY
14-d Mortality (%) Hours After Pain Treated Control
Relative Risk
P
? 2 10 20 .50 .01
> 2 – 4 11 16 .70 NS
> 4 – 8 17 22 .77 NS
> 8 – 12 20 25 .80 NS
> 12 23 24 .95 NS
Overall 14 21 .67 .001
![Page 41: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/41.jpg)
SUBGROUPS DEFINED A-PRIORI
• Suggestive differential subgroup effect
• State in design of new trial
• Publish (multiplicity, design analysis, plan over-sampling)
• Test within an existing data set
![Page 42: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/42.jpg)
STROKE SUBGROUP HYPOTHESIS
On BP Meds at ICV Off BP Meds at ICV
35% of participants 65% of participants
Net reduction in Net reduction in
stroke rate =10% stroke rate =40%
80% power to detect 30% treatment difference
![Page 43: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/43.jpg)
STROKE EVENTS BY MEDICATION STATUS
GROUP N NFS FS CSNot on MedicationsActive 1584 64 5 67Placebo 1577 88 11 96
6.72= 2א Relative Risk (active/placebo)= 0.69P = .0096 95% CI = 0.51-0.95
On MedicationsActive 781 32 5 36Placebo 794 61 3 63
5.11 = 2א Relative Risk (active/placebo)= 0.57 P = .0237 95% CI = 0.38-0.85
![Page 44: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/44.jpg)
MRFIT RESEARCH GROUP, AM J CARDIOL 1985;55:1-15
ECG ABNORMALITIES AT BASELINE
Present Absent
PersonYrs.
No. ofDeaths
Per 1000Per Yrs.
PersonYrs.
No. ofDeaths
Per 1000Per Yrs.
Not onDiuretic
6074 9 1.48 17,356 35 2.02
On Diuretic 6433 38 5.91 14,399 33 2.29
![Page 45: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/45.jpg)
SUBGROUP HYPOTHESIS
Will the treatment of ISH reduce the frequency of major coronary events more in those free of baseline ECG
abnormalities than in those with such abnormalities?
![Page 46: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/46.jpg)
OTHER COMBINED EVENTS BY TREATMENT GROUP
Event Active Placebo Rel. Risk 95% CI
Nonfatal MI/CHD Death
104 141 0.73 0.57-0.94
Stroke/MI/CHD/Death
199 289 0.67 0.56-0.81
CHD 140 184 0.75 0.60-0.94
CVD 289 414 0.68 0.58-0.79
![Page 47: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/47.jpg)
NONFATAL MI & CHD DEATH BY BASELINE ECG ABNORMALITIES
TreatmentNumber Events Rate per 100 (SE)With baseline ECG abnormalitiesActive 1429 67 6.0 (0.8)Placebo 1426 96 8.0 (0.9)
5.73 = 2א Rel. Risk = 0.69P = 0.02 95% CI = 0.50-0.94
Without baseline ECG abnormalitiesActive 903 35 4.5 (0.8)Placebo 922 43 4.6 (0.7)
0.70 = 2א Rel. Risk = 0.83P = 0.40 95% CI = 0.53-1.29
![Page 48: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/48.jpg)
SUDDEN DEATH BY BASELINE ECG ABNORMALITIES
TreatmentNumber Events Rate per 100 (SE)With baseline ECG abnormalitiesActive 1429 15 1.2 (0.3)Placebo 1426 17 1.3 (0.4)
0.14 = 2א Rel. Risk = 0.88P = 0.71 95% CI = 0.44-1.75
Without baseline ECG abnormalitiesActive 903 8 1.2 (0.4)Placebo 922 5 0.6 (0.3)
0.77 = 2א Rel. Risk = 1.64P = 0.38 95% CI = 0.54-5.01
![Page 49: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/49.jpg)
![Page 50: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/50.jpg)
![Page 51: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/51.jpg)
SUBGROUPS DEFINED A-POSTERIORI
• “Grist” for formulating hypothesis
• Watch for alternative definitions!
• Should be clearly stated and reported as an estimate of effect with appropriate confidence interval
![Page 52: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/52.jpg)
SUBGROUPS AND MONITORING TRIALS
• Use statistically sound monitoring method
• Interference with main trial endpoint - rare
• Formulate hypothesis and test prospectively
• Terminate subgroup
• “Mega trials” - special problems
![Page 53: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/53.jpg)
BUCHWALD AND COLLEAGUES: POSCH, NEJM1990;323;946-955
• Cholesterol lowering by ileal bypass for secondary prevention of CHD
• 838 participants randomized
• Primary outcome-Fatal and non-fatal CHD– 62 vs 49, p=0.164
• Subgroup, EF<50% vs >50%, – Final 24 vs 39, p=0.021– post-hoc analysis 6 vs 17– after observed phenomenon - 18 vs 22
![Page 54: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/54.jpg)
BETA CAROTENE THERAPY FOR CHRONIC STABLE ANGINA :
PHYSICIAN’S HEALTH STUDY
• 333 men with chronic stable angina or coronary revascularization
• 50 mg Beta carotene on alternate days
• 44% reduction, major CHD events, p=0.046
• 49% reduction, major CVD events
• Adjusted for age, aspirin assignment and CHD risk factors
![Page 55: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/55.jpg)
SUBGROUP EFFECT FROM DIFFERENT TRIALS
• Meta-analysis
• Clear definition of subgroup
• e.g.– Tamoxifen in women >50 years for breast
cancer is beneficial– Beta blocker- those without ISA being more
effective than those with ISA in the treatment of patients with acute MI- not true in pooled data or prospective trials
![Page 56: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/56.jpg)
BETA BLOCKERS AND NON Q WAVE MI: GHEORGHIADE AND COLLEAGUES, AJC
1990;66:220-222
No benefit of propranolol (BHAT) in 601 participants with non-Q-wave infarction
• Concerns– Overview of 20,000 shows reduction in mortality
-23% (95%CI 0.69-0.85)– Overview, nonfatal MI -26% (95%CI 0.66-0.83)– BHAT, non-fatal MI (NS)
![Page 57: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/57.jpg)
BETA BLOCKERS AND NON Q WAVE MI: GHEORGHIADE AND COLLEAGUES, AJC
1990;66:220-222
• Four potential reasons for spurious results– Statistical power-roughly 5%– Not replicated -Timolol Study– Intervenous trials show benefit. Decrease
mortality similar in those with and without ST elevation at entry.
– Result in subgroups not specifically different from each other. (Non-Q-wave, small subgroup)
![Page 58: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/58.jpg)
MISCLASSIFICATION AND MISSING DATA
• Regression to the mean
• Measurement error and misclassification remain independent-unbiased
• “Sickest patients”-missing data
• Sensitivity analysis vs imputation
• Completion vs randomization cohort
![Page 59: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/59.jpg)
DICTUM 1
“The treatment effect in all subgroups of patients without obvious contraindications to treatment is likely to be qualitatively (in the same direction) similar.”
![Page 60: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/60.jpg)
DICTUM 2
“The treatment effect in all subgroups of patients without obvious contraindications to treatment is likely to be quantitatively (difference in degree) dissimilar even when effects appear to be identical.”
![Page 61: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/61.jpg)
DICTUM 3
“Estimates of treatment effect within a subgroup chosen for special emphasis are usually ‘biased’ and so the most appropriate estimate in a subgroup is closer to the overall result.”
![Page 62: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/62.jpg)
KEY POINTS IN SUBGROUP ANALYSES & INTERPRETATION I
Design
1. State clearly a few important and plausible subgroup hypotheses in advance. Include the direction of the expected effect.
2. Rank the subgroup hypotheses in order or plausibility.
3. Calculate power to detect key subgroup effects. If it is inadequate, consider building adequate power to detect key subgroup effects.
4. State whether the trial will be continued even after the overall results are convincing but the subgroup effects are not significant. Decide whether the primary method of monitoring will focus on the subgroups or on the overall trial.
5. State primary analytical methods in advance.
![Page 63: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/63.jpg)
KEY POINTS IN SUBGROUP ANALYSES AND INTERPRETATION II
Monitoring a trial
1. Rigorous evidence of benefit or harm in subgroups postulated a priori: consider selective discontinuation of that subgroup.
2. Evidence of benefit or harm in unexpected subgroups: postulate a hypothesis to be tested in the remaining part of the study.
![Page 64: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/64.jpg)
KEY POINTS IN SUBGROUP ANALYSES & INTERPRETATION III
Analyses and interpretation
1. Use statistical methods that capture the framework of the prior hypotheses.
2. Place greater emphasis on the overall results than on what may be apparent within a particular subgroup.
3. Distinguish between prior and data-derived hypotheses. Do not calculate p values for data-derived hypotheses because such p values usually bear little resemblance to what could occurs if the hypothesis were tested independently in another study.
4. Use tests of “interactions” and/or correct for multiplicity of statistical comparisons. (“Nominal” p values are usually misleading.)
5. Interpret the results in the context of similar data from other trials, from the architecture of the entire set of data on all patients, and from principles of biological coherence.
![Page 65: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/65.jpg)
KEY POINTS IN SUBGROUP ANALYSES AND INTERPRETATION IV
Improper subgroups and analyses
1. Avoid analyses of subgroups based on post-randomization response, adherence, etc.
2. Avoid emphasizing nominal p values.
3. Do not emphasize data-derived analyses or analyses based on post-hoc definitions of subgroups.
![Page 66: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/66.jpg)
![Page 67: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/67.jpg)
ESSENTIALS IN SUBGROUP REPORTING
• PRESPECIFICATION OF SUBGROUPS • NUMBER OF SUBGROUPS• NUMBER OF SUBGROUP OUTCOMES• STATISTICAL METHODS (INTERACTION)• NUMBER OF SIGNIFICANT SUBGROUPS FOUND• EMPHASIS OF SUBGROUP VS PRIMARY
OUTCOME
![Page 68: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/68.jpg)
INTERACTION VS SEPARATE SUBGROUP TESTS
INTERACTION TEST --DIRECTLY ASSESSES DIFFERENCES IN TREATMENT EFFECTS BETWEEN COMPLEMENTARY SUBGROUPS AND INVOLVES 1 STATISTICAL TEST IRRESPECTIVE OF THE NUMBER OF SUBGROUPS-- APPROPRIATE, BUT UNDERUTILIZED
SEPARATE SUBGROUP METHOD-- INVOLVES MORE THAN 2 TEST-- CONSIDERED INAPPROPRIATE STATISTICALLY
![Page 69: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/69.jpg)
Hernandez AV, et al. Am Heart J. 2006;151:257-64.
SUBGROUP ANALYSES: CHARACTERISTICS OF CVD RCTS
![Page 70: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/70.jpg)
Hernandez AV, et al. Am Heart J. 2006;151:257-64.
DESCRIPTION OF SUBGROUP ANALYSIS REPORTING
![Page 71: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/71.jpg)
Hernandez AV, et al. Am Heart J. 2006;151:257-64.
SUGGESTIONS TO APPROPRIATELY PERFORMAND INTERPRET SUBGROUP ANALYSIS
![Page 72: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/72.jpg)
![Page 73: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/73.jpg)
SHEP: PREVENTION OF CHF
In SHEP-older participants, men, those with higher SBP, or History or ECG evidence of MI at greater risk for CHF
SHEP Intervention Provided
• Overall result: 49% reduction fatal and non-fatal CHF (95% CI 0.37-0.71)
• Those with history or ECG evidence of MI:
81%reduction in CHF (95% CI 0.06- 0.53)
![Page 74: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/74.jpg)
INCIDENCE OF FIRST STROKE BY TYPE AND SUBTYPE, AND BY RANDOMIZED GROUP
Number ofParticipants
Differences BetweenActive Treatment andPlacebo Participants
Active Placebo Risk Ratio 95%Confidence
IntervalTotal SHEP participants 2365 2371
All first strokes 103 159 0.63 0.49 to 0.81
Ischemic strokes 85 132 0.63 0.48 to 0.82
Lacunar 23 43 0.53 0.32 to 0.88
Embolic 9 16 0.56 0.25 to 1.27
Atherosclerotic 13 13 0.99 0.46 to 2.15
Other/unknown 40 60 0.64 0.43 to 0.96
Hemorrhagic strokes 9 19 0.46 0.21 to 1.02
Subarachnoid 1 4 0.25 0.03 to 2.26
Intraparenchymal 8 15 0.52 0.22 to 1.22
Unknown type stroke 9 8 1.05 0.40 to 2.73
![Page 75: The Treachery of Sub-group Analyses in CVD Clinical Trials](https://reader038.vdocuments.us/reader038/viewer/2022110210/56812b1c550346895d8f110a/html5/thumbnails/75.jpg)
DAYS OF REDUCED ACTIVITY BY RAMDOMIZED GROUP FOR STROKE VICTIMS AT THREE TIMES DURING THE TRIAL
(SELF-REPORTED FOR THE TWO WEEKS IMMEDIATELY BEFORE VISIT)
First Post-Stroke Report
First Report >6 Months
Post-Stroke
Last TrialReport
PART A: DAYS OF REDUCED ACTIVITY (including “days in bed”)
61 Stroke victims, active treatment groupPercent with days of reduced activity 16.4% 13.1% 11.5%Mean days of reduced activity 1.11 days 0.79 days 0.46 days
92 Stroke victims, placebo group
Percent with days of reduced activity 22.8% 21.7% 18.5%Mean days of reduced activity 1.89 days 1.93 days 1.89 days
PART B: DAYS IN BED (a subset of days of reduced activity)
61 Stroke victims, active treatment groupPercent with days in bed 9.8% 4.9% 3.1%Mean days in bed 0.59 days 0.30 days 0.25 days
92 Stroke victims, placebo group
Percent with days in bed 8.8% 9.9% 14.1%Mean days in bed 0.90 days 0.83 days 1.18 days