c o p e washington 9 nov09

Published ResearchFlawed, misleading,

deceitful ?John Hoey

COPE U.S Seminar 2009Washington, DC

johnhoeymd@gmail.comwww.slideshare.com/h

oey

Role’s, Responsibilities of Editors

Research manuscripts

Bailar’s pyramid of faults

Examples

What can an editor do?

Reporting guidelines

Conflict of Interest checklists

Outline

COPECode of Conduct

General duties and responsibilities of Editors

Be responsible for everything published in their journals. • Strive to meet the needs of readers and authors • constantly improve the journal • Ensure the quality of the material they publish • champion freedom of expression • Maintain the integrity of the academic record• Preclude business needs from compromising intellectual standards• always be willing to publish corrections,clarifications, retractions and apologies whenneeded.

Maintaining the Integrity of the Scientific Record.......why?

•Ethics - Nuremburg Trials - Helsinki Declaration.

•Harm to patients and the public

•Physical harm

•Financial harm

Editorial Ethical Responsibilites

• Helsinki Declarations

“Both authors and publishers have ethical obligations.

In publication of the results of research, the investigators are obliged to preserve the accuracy of the results.

Negative as well as positive results should be published or otherwise publicly available.

Sources of funding, institutional affiliations and any possible conflicts of interest should be declared in the publication.

Reports of experimentation not in accordance with the principles laid down in this Declaration should not be accepted for publication.”

$$ Conflict

of Interest

Carelessness

Incompetence

John Bailar’s Pyramid of Manuscript Problems

a few examplesof problems editors face

Kay Dickinson, Reporting and other biases in studies of Neurontin for migraine, psychiatric/bipolar disorders, nociceptive pain, and neuropathic pain.August, 2008

http://dida.library.ucsf.edu/pdf/oxx18r10

Reporting Bias

Some definitions

• Positive= Study shows results favourable to the interests of the author/sponsor. Usually statistically significant.

•Negative = Study shows results unfavourable to the interest of the the author/sponsor - may or may not be statistically significant.

Reporting Biases

•Non-publication of negative or neutral results

•Selective publication of results - outcome bias

• Multiple publication bias

•Language bias - and publishing in the grey literature

•Time lag bias

•Undeclared conflicts of interest

•Ghost writingDickinson

Hypothesis testing or Hypothesis generating?

Hypothesis testing Hypothesis generating

Prior specification before study begins

eg. RCT, observational studies, etc.

Finding an interesting result among man possible results

eg. Survey, cohort study etc..

Selective publication - Outcome biaspublishing the more interesting (usually

positive) result

Was there an hypothesis? A plan for analysis and reporting

of data?

In an RCT, this is the primary outcome

RCT

RRx A

Placebo

Outcome

Study design - RCT

RRx A

Placebo

Outcome

Primary Outcome -Specified in Protocol?or fishing expedition?

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Consort Statement•Poor quality of reporting RCTs.

•Could not be trusted for use in clinical practice

•Could not be combined in systematic reviews

•Identified 22 essential items needed to conduct a RCT.

Ann Intern Med. 2001;134:663-694. www.annals.org David Moher

Selective publication - Outcome bias(publishing the more interesting result)

1402 outcomes31% - 59% incompletely reported(40% not reported at all)

Chan, A.-W. et al. CMAJ 2004;171:735-740

48 RCTs funded by national granting

agency

Selective publication - Outcome bias(publishing the more interesting result)

Primary Outcome

Diabetic control 6 months after the end of intensive multithearpy

Selective publication - Outcome bias(publishing the more interesting result)

Interpretation: Intensive multitherapy for patients with poorly controlled type 2 diabetes is successful in helping patients meet most of the goals set by a national diabetes association.

However, 6 months after intensive therapy stopped and patients returned

to usual care the benefits had vanished,

However, 6 months after intensive therapy stopped and patients returned to usual care the benefits had vanished.

does any of this matter?

images from Wikipedia which also has a nice summary of court proceedings and results

Neurontin(gabapenti

n)

For minor seizuresFDA approved 1994By 2003 one of Pfizer’s best selling drugsOff-label uses account for 90% of sales

Multiple small RCTs Benefit for other disorderse.g. migraine

P Wessely, C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987

Bias Example

Publication Final negative primary results not published, only positive preliminary results

Selective outcome reporting

Outcome reported was not primary or secondary outcome

Selective statistical analyses

2 nonrandomized patients assigned to neurotin were include with those randomized

Spin Emphasis on “positive” outcomes

A seriously flawed RCT - Accepted for publication by someone

16 Citations P Wessely, C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987

Does it matter? General Principles of Migraine Management: The Changing Role of PreventionE Loder, D Biondi - Headache: The Journal of Head and Face Pain, 2005 - Blackwell Synergy

Preventive treatment of migraine - SD Silberstein - Trends in Pharmacological Sciences, 2006 - Elsevier

Migraine preventionDW Dodick, SD Silberstein - British Medical Journal, 2007 - pn.bmj.com

Neuromodulators for Migraine PreventionR Kaniecki - Headache: The Journal of Head and Face Pain, 2008 - Blackwell Synergy

Practice parameter: Evidence-based guidelines for migraine headache (an evidence-based review)

Stephen D. Silberstein, MD, FACP, for the US Headache Consortium

Neurology 2000;55:754-762

a typical editor

and their support staff...

the editor’s desk

BMJ Editorial Staff

Editor in chiefFiona GodleeemailT: + 44(0)20 7383 6102Deputy editorsJane SmithemailT:+44 (0)20 7383 6009Tony DelamotheemailT: +44 (0) 20 7383 6006Trish GrovesemailT:+44 (0)20 7383 6018Magazine editorTrevor Jackson emailT:+44 (0)20 7383 6677bmj.com editorDavid Payne emailT:+44 (0)20 7383 6532Editorials editorGiselle Jones

Research papers editorsKristina Fister email

Trish Groves email

Elizabeth Loder email

Christopher Martyn email

Alison Tonks email

Primary care editorDomhnall MacAuley email

News editorsAnnabel Ferriman emailT: 44 (0)20 7383 6035

Zosia Kmietowicz email

Features editorsDeborah Cohen emailT: +44 (0)20 7383 6183

Rebecca Coombes emailT: +44 (0)20 7383 6243

Clinical reviews editorKirsten Patrick email

Practice editorMabel Chew email

Analysis editorTessa Richards emailT: +44 (0) 20 7383 61

Letters and obituaries editorSharon Davies emailT:+44 (0)20 7383 6716

Observations and reviews editorTrevor JacksonemailT:+44 (0)20 7383 6677

Roger Robinson editorial registrarHelen MacdonaldemailT: + 44 (0)20 7 874 7022

Senior researcherSara Schroter emailT: +44 (0)20 7383 6744Patient editorPeter Lapsley email

Web teamEditor bmj.comDavid PayneemailAssistant editor, bmj.comBirte Twisselmann emailT: +44 (0)20 7383 6720

Print journal teamMagazine editorTrevor Jacksonemail

DesignerJane Walkeremail

Senior art workerAdam di Chiaraemail

Picture editorVanessa Fletcheremail

Deputy managing editorLucy Banham emailT:+44 (0)20 7383 6365

Technical editorsJackie AnnisemailT:+44 (0)20 7383 6658

Maggie Butler emailT:+44 (0)20 7383 6074

Sally Carter emailT:+44 (0)20 7383 6659

Margaret Cooter emailT:+44 (0)20 7383 6657

Greg Cotton emailT:+44 (0)20 7383 6685

Clare Griffith emailT: +44 (0)20 7383 6051

Richard Hurley emailT: + 44 (0)20 7383 6051

Elizabeth Payne emailT: + 44 (0)20 7383 6449

Karl Sharrock emailT:+44 (0)20 7383 6658

Barbara Squire emailT:+44 (0)20 7383 6658

Julia Thompson emailT:+44 (0)20 7383 6691

Douglas Kamerow email

PA to editor in chiefJulia Burrell emailT:+44 (0)20 7383 6102

Departmental administratorChelsey White emailT:+44 (0)20 7383 6109

Benchpress database managerGary BryanemailT:+44 (0)20 7383 6304

Benchpress administratorSue MinnsemailT:+44 (0)20 7383 6309

Chief production editorJohn MayoremailT:+44 (0)20 7383 6355

Assistant production editorEdwyn MayhewemailT:+44 (0)20 7383 6145

Production editorJett AislabieemailT:+44 (0)20 7874 7014

Malcolm Brown (maternity cover)T:+44 (0)20 7874 7014

IllustratorAnthea WilkieemailT:+44 (0)1737 215143

Copyright administratoremail

Career FocusEdward Davies emailT: + 44 (0) 20 7383 6562

studentBMJJessie Colquhounstudenteditor@bmj.comT: +44 (0)20 7874 7016

Visiting editorsJennifer Leaning (USA)Ray Moynihan (USA)Joanne Roberts (USA)Charlie Wilson (USA)

Editorial advisersSteven ReidIan MaconochiePeter LemanNick DunnFrank SullivanPippa OakeshottAziz SheikhLucy ChappellChristopher WhittyScott MurrayJulia Hippisley-CoxJosip CarRobin FoxSue Morgan

Statistical advisersDoug AltmanTim ColeHazel InskipJulie MorrisDeborah AshbyJon Deek

The Lancet Editorial office

“The Apartment” Billy Wilder, Jack Lemnon, 1959

EditorDavid Bevan MBToronto

Photo by Waldo www.flickr.com/photos/waldo4/2178788631/

Manuscripts are messy & confusing

Editorial checklists for authors

RCTs Consort guideline

Checklists for KEY elements of a study that need to be reported in published papers.

Minimal required content

www.equator-network.org

? CONSORT

Enrolment

Allocation

Follow-up

Analysis

Editorial checklists for authors

Guidelines for research reportingConsortPrismaOthers ~ 60 known guidelines

www.equator-network.org

Editorial checklists for authors

www.equator-network.org

Notions Key elements of research that ought to be reported For use by

authors in writing manuscriptsresearchers writing grantsgrant review committeesEDITORS

What can editor’s do to improve quality of what they publish?

• Explicit contract

• Helsinki Declaration

• WAME/ICMJE

• Publisher goals

• Disclosure of contract

• Disclosure of editorial & publisher Conflicts of interest

Publisher/Editor Author/Editor

• Aim for quality

• Publish less

• Author guidelines/instructions

•Use reporting guidelines

•Require authors use them

•Use checklists with submissions

Conflict of Interest -

Conflict of Interest -

Conflict of Interests - Who was responsible for?

•Study

•design

•data collection

•analysis

•write up

•decision to publish

•etc

1109 Canadian clinicaltrials.gov732 investigators11 ideal practices to mitigate fCOI

- control over design- data collection- analysis- interpretation-write up-authorship-decision to publish-etc.

Only 6% of investigators met all ideal practices

Rochon P, et al. - manuscript in review

Thank-you

www.equator-network.org

johnhoeymd@gmail.comwww.slideshare.com/h

oey