www.cebm.net

Systematic Reviews and access to all clinical trial data

Dr Carl Heneghan Director CEBM & Clinical Reader

Dept of Primary Health Care, University of Oxford

www.cebm.net

Fallout from a troubled drug Vioxx

2000

Clinical trials show five times the number of heart attacks among patients taking Vioxx……

• Informed FDA in 2001 there was no increase in risk of death while they knew internally it was 3 times that of placebo

Merck, Rofecoxib & Alzheimers

Internal company data April 2001 from 2 Vioxx trials submitted to the FDA in July 2001 used on‐treatment analysis minimized the appearance of any mortality risk

December 2001, FDA raised safety questions & Merck Responded that the rofecoxib placebo findings are

“small numeric differences ... most consistent with chance fluctuations..

Psaty et al. JAMA 2008;299:1813‐7

On treatment analysis

April 2001, Company's internal intention-to-treat analyses of pooled data from these 2 trials identified a significant increase in total mortality

HR = 4.43 (95% CI, 1.26-15.53) - protocol 091 HR = 2.55 (95% CI, 1.17-5.56) - protocol 078

Overall mortality of 34 deaths among 1069 rofecoxib patients and 12 deaths among 1078 placebo patients

HR = 2.99 (95% CI, 1.55-5.77).

Fallout from a troubled drug

FDA approves Vioxx

Vioxx label is revised to include potential heart risks

Vioxx withdrawn from the market

Vioxx jury awards Texas widow $235 million

1999 2002 2004

Stock price $80

www.cebm.net

The 5 steps of EBM

1. Formulate an answerable question2. Track down the best evidence 3. Critically appraise the evidence for validity, clinical

relevance and applicability 4. Individualize, based clinical expertise and patient

concerns5. Evaluate your own performance

Rosiglitazone timelineMay 1999 Approved by the FDA, ADOPT as Phase IV trial

July 2000 Approved by European Commission, RECORD as Phase IVJan 2004 WHO signal document on TZDs and cardiac disease

Apr 2004 Sponsor forms working group on analysis of CV events

Oct 2005 Summary of first meta-analysis (n=37) submitted to FDA

Aug 2006 Final report of 2nd meta-analysis (n=42) submitted to FDA

Oct 2006 European product label incorporates meta-analysis results

Dec 2006 ADOPT trial published

May 2007 Nissen meta-analysis published (OR=1.43)

June 2007 Sponsor publishes interim data from RECORD trial

Nov 2007 FDA adds boxed warning on myocardial ischemia

July 2010 Completed results from RECORD trial published

Sept 2010 EMA suspends sales; FDA announces REMS restrictions

FDA medical review

“Whether RSG favorably affects the natural history of type 2 diabetes is an open question. However, the increase in body weight and undesirable effects on serum lipids is a cause for concern….Heart disease … is a major cause of morbidity and mortality … and it cannot be assumed that RSG will decrease the [CVD] risk…. My concern about deleterious long term effects on the heart should be addressed…. A postmarketing study to address these issues needs to be a condition of approval.”

• Misbin RI. CDER, 21-071 (rosiglitazone), Apr 1999.

Should I give Tamiflu to my children if they have a fever ?

www.cebm.net

UN report How many predicted deaths would Bird Flu cause 1) world-wide2) In the UK

www.cebm.net

2006 Britain halfway through receiving an order of 14.6 million courses of the drug, which the Government hopes will avert some of the 700,000 deaths that might be expected.

13 days Swine Flu Tuberculosis

Death (who.int)

News (news.google.com)

News/Death ratio

13 days Swine Flu Tuberculosis

Death (who.int) 31

News (news.google.com)

News/Death ratio

13 days Swine Flu Tuberculosis

Death (who.int) 31 63 066

News (news.google.com)

News/Death ratio

13 days Swine Flu Tuberculosis

Death (who.int) 31 63 066

News (news.google.com) 254 442

News/Death ratio

13 days Swine Flu Tuberculosis

Death (who.int) 31 63 066

News (news.google.com) 254 442 6 501

News/Death ratio

13 days Swine Flu Tuberculosis

Death (who.int) 31 63 066

News (news.google.com) 254 442 6 501

News/Death ratio 8 176

13 days Swine Flu Tuberculosis

Death (who.int) 31 63 066

News (news.google.com) 254 442 6 501

News/Death ratio 8 176 0.1

Main results and role of chanceWe included four randomised trials for treatment of influenza (two oseltamivir, two zanamivir) involving 1766 children (1243 with confirmed influenza)

and three randomised trials for post-exposure prophylaxis (one oseltamivir, two zanamivir) involving 863 children.

Systematic review of unpublished Clinical Study Reports – the case of neuraminidase inhibitors for influenza

Tom Jefferson

Rokuro Hama

Carl Heneghan

Chris Del Mar

Peter Doshi

Mark Jones

Matthew Thompson

History of the review

1999 – 1st review2003 – update2005 – update2009 – update2011 – update current2013 – forthcoming

Hayashi ‘s criticism• complications; • reliance on Kaiser 2003• no ‘raw’ data; • authors Roche; • some unpublished

www.cebm.net

Kaiser et al. (Archives of Internal Medicine 2003)

www.cebm.net

Cochrane review on Neuraminidase Inhibitors (NIs), 1999, 2006, 2008, 2009.....

Dr Keiji Hayashi We have some questions on the conclusion in your Oseltamivir review especially about the prevention of complication. You described that “Oseltamivir 150 mg daily prevented lower respiratory tract complications (OR 0.32, 95% CI 0.18 to 0.57).” (in abstract). However, we have found that this conclusion is based on the other review (Kaiser2003) and not on your own data analysis. The authors of the review were four employees of F. Hoffman-La Roche Ltd, one paid consultant to F. Hoffman-La Roche Ltd and Kaiser. We cannot find any raw data about this conclusion from your review. Kaiser’s review included 10 RCTs; two RCTs (Nicholson 2000 and Treanor 2003) were published as articles in the peer-reviewed medical journal (JAMA and Lancet), but other 8 RCTs were proceedings of congress (5 RCTs), abstracts of the congress (one RCT) and meeting (one RCT) and data on file, Hoffmann-La Roche, Inc, Nutley, NJ (one RCT). The lower respiratory tract complication rates of these articles were summarized on table: there was nosignificant difference between Oseltamivir and placebo, and their Odds Ratio’s (ORs) were 1.81. But ORs of other 8 RCTs were 4.37. We strongly suppose that the reviewer’s conclusion about the complications was mainly determined by these 8 RCTs, we should appraise the 8 trials rigidly. Without this process it’s difficult to conclude that oseltamivir can prevent lower respiratory tract complications

Comment posted 14 July 2009

cour

tesy

of S

wis

s TV

Professor Laurent KaiserProfessorHayden

“ I have searched but cannot find the original files related to this 2003 publication. Before and again after my 2+ years at WHO in Geneva, I was obliged to move offices at the University several times and downsize.  The files appear to have been discarded.  My co-author Laurent Kaiser, now professor at the University of Geneva, is copied on this reply, as he may have his own sources.  The questions posed by the inquirer are not clear to me, but if original data or unpublished study reports are required, they will likely need to come from Roche, the sponsor of these studies”.

Email 14 August 2009

Professor John Treanor

“…[Treanor] told the BMJ that as far as he could remember, the trial published in JAMA was the only large study of oseltamivir he had ever participated in. ...Channel 4 News put it to Roche that Professor Treanor said that he didn’t actually participate in study M76001 and doesn’t remember presenting it a meeting in 2000. Dr David Reddy, Roche’s Global Pandemic Taskforce leader, said: “It’s not infrequent that you may have somebody who authors but they don’t actually present it at a conference, it depends upon their availability.” (D Cohen, BMJ 2009)

Professor Karl Nicholson

..Nicholson said he did not recall seeing the primary data. He said that the statistical analysis had been conducted by Roche and he analysed the summary data.“While Roche has admitted that “medical writers were used to draft some of the above papers” and Nicholson said that Roche did employ a medical writer to draft the manuscript, they both argued that at the time of submission—before the 2003 Good Publication Practice Guidelines, produced with the help of the drug industry and recently updated —it was standard practice for unnamed medical writers to be used”. (Cohen, BMJ 2009)

“I did not perform an independent analysis of the primary data, which was not required or requested by JAMA at the time of submission, and I do not have access to the primary data, which I also never requested.”

Treanor quoted in Cohen, BMJ 2009

…did not recall seeing the primary data…the statistical analysis had beenconducted by Roche and he analysed the summary data...

Nicholson quoted in Cohen, BMJ 2009

Unanswered questions

• How many trials are there?• Who is responsible for each trial?• Why were large phase III trials (e.g. M76001, NAIA 3002) not

published?• Who is responsible for the decision not to publish studies in

which humans were randomised?• What are the harms and benefits of NIs?• Why are trials published 10 years after completion?• Are the Clinical Study Reports (CSRs) a reliable source of

evidence?• Are we going to get the full CSRs?• How can regulatory approval be based on selected trials instead

of totality of evidence?• Is the body of NI pharmaceutical evidence reliable?• Why do we have divergent indications across regulators (TF)?

www.cebm.net

New methods?

New Methods

Identify all trials (trial programme) Agree

Identify and retrieve all CSRs and regulatory material

Agree

Table of Contents of the Evidence (TOCE) Agree

Weave evidence of trial programmes together Disagree

Assess – if reliable, analyse What does reliable mean?

Complete? Trustworthy?

(Both?)

www.cebm.net

Study IDM Size in pdf pages (Module)

Type of RCT Phase Report dates Trial done

NP15757 225 (M1?) Study of the Pharmacokinetics and Pharmacodynamics of the Neuraminidase Inhibitor Ro 64-0796 (GS4104) in the Prophylaxis of Influenza in Volunteers experimentally Inoculated with the Human Influenza B Virus

II 1999 20/May/1998 – 06/Jul/1998

NV16871 348 (M1? plus list of investigators, consent forms etc)

A double-blind, randomized, stratified, placebo –controlled study of oseltamivir in the treatment of influenza in children with asthma.

IIIb 2003?

WP16263 8545 (M1-4) A randomized, double blind, parallel group, placebo controlled study of the effect of oseltamivir on ECG intervals in healthy subjects

III 9 Jan 2001 Aug-Sept 2000

WV16193 289 (partial M1) A randomized, open-label, parallel group study of oseltamivir used for the management of influenza in households

III 14 Jun 2002 5 Jan 2001 - 14 May 2001

WV15758 513 (M1) A double-blind, randomized, stratified, placebo controlled study of Ro 64-0796 (also known as GS 4104) in the treatment of children with influenza.

III 8 March 2000 Dec 98 – Apr 99

WV15799 (two PDFs)

253 and 278 (M1) A double-blind, randomized, placebo controlled study of Ro 64-0796 (also known as GS4104) used for the prevention of clinical influenza post exposure in families

III 19 Jan 2000 22 December 1998 – 19 March 1999

WV15759 & WV15871 (1 vol)

286 (partial M2) Protocol, amendments, Case reports forms (CRF), list of ethics committees, informed consent, investigators.

WV15759:A double-blind, randomized, stratified, placebo-controlled study of oseltamivir phosphate (Ro 64-0796, also known as GS 4104) in the treatment of influenza in children with chronic asthma*---------------------------------------------WV15871: A double-blind, randomized, stratified, placebo-controlled study of Ro 64-0796 (also known as GS4104) in the treatment of influenza in children with chronic asthma*

III 17 Apr 2000 Unclear.

www.cebm.net

Module II - CSR WV15707

www.cebm.net

www.cebm.net

the two most two cited published trials of oseltamivir either did not mention serious adverse events (Nicholson 2000), or stated that "... there were no drug-relatedserious adverse events" (Treanor 2000).

Finding repeated by bodies such as the UK NHS "No serious adverse events were noted in the major trials and no significant changes were noted in laboratory parameters”

Clinical study reports' Module 1 describe 10 serious adverse events (in nine participants) in the two trials (WV15670; WV15671),

UNEXPECTED FINDINGS

www.cebm.net

www.cebm.net

www.cebm.net

www.cebm.net

www.cebm.net

www.cebm.net

THANK YOU

@cebmblogwww.cebm.net