Top Trial Cheats

Dr Rod Stables

The Cardiothoracic Centre

Liverpool UK

No Conflicts of Interest to Declare

Presentation Outline

• Examples without accusation

• Three topic areas

• Establishing the denominators

• Lost or not evaluated at follow-up

• Odds ratio v Risk ratios

• Composite outcome measures

• Mixing safety and efficacy measures

Identify the Denominator: Tracking Trial Subjects

• Need to follow the fate of ALL patients

• From inclusion to outcomes

• Warning signs !

• Results expressed as simple %age

• Smokescreen phrases :

• ‘In patients with outcome data …’

• ‘In evaluable (surviving) patients …’

• ‘When assessed at follow-up …’

Identify the Denominator: Examples

• Rickards and Stables CABRI long term follow-up

• Widely presented (but never published)

• Every result presented as a % value

• Masked incomplete follow-up

• No data on >120 Dutch patients

• More sinister impact - reported outcome measures

Not Assessed for the Outcome Measure ?

• Trial of drug A - exercise capacity in heart failure

• 100 patients randomised 1 : 1 drug v placebo

• Outcome measure

• Mean exercise duration at FUP

• Baseline exercise duration - equal

• Follow up Drug 3.2 mins Placebo 2.4 mins

• Clinical and statistical significance

• Mechanism ?

Not Assessed for the Outcome Measure ?

• Widespread implications

• Incomplete QCA follow up in angiographic trials

• Numerical handling of extreme examples

• QCA - MLD of occluded vessel ? = 0

• QoL - score of dead patient = 0 (?excluded)

• Competing outcomes

• ↑ mortality = ↓ repeat revascularisation

• Event curve analysis - No of patients at risk

SoS Trial: Repeat Revasc Event Curve

BASIL Trial : Amputation - free Survival

Solutions ?

• Clear exposition of patient flows

Trial Profile and Patient Flow Diagram

Solutions ?

• Clear exposition of patient flows

• All event rates expressed: n / d (p%) (CIs)

• Describe status of patients not assessed for

outcome measures

• Express rates with correct denominators eg

• Repeat revasc rate per 100 patient years of Fup

• Full description of complex data handling methods

• QCA QoL etc

Odds and Risk

Odds and Risk

• Throw one of a pair of dice - Aim - To get a 6

• Odds against = 5:1 (Odds)

• Event occurs 1:6 of all throws (Risk)

• Scope for confusion and manipulation of data

• ……. and audience

• Most observers interpret odds ratio as relative risk

Example: Relative Risk in a Classic RCT

• Randomised trial Tx A v Tx B

• Each group comprises 100 patients

• Death rate on Tx A = 50/100 (= 0.5)

• Death rate on Tx B = 25/100 (= 0.25)

• Relative risk = 0.5 / 0.25 = 2

• (p values and confidence intervals)

Odds Ratio Calculation

• Tx A: No dead = 50 (a) No alive = 50 (c)

• Tx B: No dead = 25 (b) No alive = 75 (d)

• Odds ratio = ad/bc = (50 x 75)/(25 x 50)

• = 3

• Sounds more !!

From the Thesis of Dr R H Stables DM (Oxon) …..

Table 5.6 shows the proportion of PTCA patients in each of the two consultant

groupings. In the interventionist group, 15 / 27 (56%) patients were recommended

PTCA compared to 12 / 47 (25.5%) patients in the non-interventionist group. The

odds ratio for these proportions is 3.7 (95% CI, 1.3 – 9.9) : p = 0.01.

Table 5.6 Proportion of PTCA Patients in Each of the two Consultant Groupings

I NI p Value OR (95% CI)

Tx With PTCA 15/27 (56%) 12/47 (25.5%) 0.01 3.7 (1.3 - 9.9)

Relative Risk and Odds Ratio

• If event rate is low OR is approx = RR

• If OR is interpreted as RR it will overstate the risk (benefit)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Odds Ratio

Risk Ratio

Re

por

ted

R

atio

Event Rate %

Some Basic Rules

• Odds ratio correct in 2 x settings

• Backward investigation - Case control

Case Control Studies

• Breast cancer patients - series of 200

• History of OCP = 50 (a) No OCP = 150 (c)

• Odds = a/c = 0.33

• Age sex matched controls - series of 200

• History of OCP = 25 (b) No OCP = 175 (d)

• Odds = b/d = 0.14

• Odds Ratio = (a/c) / (b/d) = a/c x d/b

• Odds ratio = ad/bc = (50 x 175)/(150 x 25) = 2.3

Some Basic Rules

• Odds ratio correct in 2 x settings

• Backward investigation - Case control

• Multiple regression analyses

• Report independent association strength

• factor and outcome

• In all others - RCT and cohort studies

• Risk ratio applicable and best

Mixing Efficacy and Safety Outcomes

Mixing Efficacy and Safety Outcomes

• Developing area - Subject of continued debate

• CVA addition to MACE in revascularisation trials

• PCI trials - MACE outcomes

• ? Add in bleeding complications

• Usually PCI related (puncture ± adjunct drugs)

• Seems reasonable ?

• Value in trials of antithrombotic therapy ?

Mixing Efficacy and Safety Outcomes

• Cloud and confuse :

• Data analysis and reporting

• Procedural development

• BENESTENT 1

• Stent puncture site complications 7% !!

• Unrelated to coronary efficacy of stent

• Reduced to current levels - changes in

• Drugs devices techniques - experience

Mixing Efficacy and Safety Outcomes

• Efficacy and safety issues

• Parallel tracks - causation and solution

• Further complicate analysis and reporting

• Minefield of composite outcomes

• Perhaps best avoided

Questions and Discussion