medical literature

Medical Literature:

Reading, Interpreting, and Writing

Muhammad Mamdani, PharmD, MA, MPHDirector, Applied Health Research Centre

LKSKI of St. Michael’s HospitalAssociate Professor – University of Toronto

Adjunct Faculty – King Saud UniversityFebruary 2010

Objectives

To better understand the basic purpose of research and the context under which research is conducted

To review basic clinical research study designs

To review two standard study review tools

To review basic principles in writing clinical research manuscripts

Clinical Research: Basic Purpose and Context

The Common Goal of Research

‘The’ GoalAttempt to estimate the ‘truth’

TRUTH

Study 3Study 1 Study 2

Perception: Reality vs Belief

Variability in Perceptions

The Trouble with ResearchExample: H. pylori eradication and non-ulcer dyspepsia

McColl et al (NEJM, 1998)Treatment (n=154): Omeprazole 20 mg bid + amoxicillin 500 mg tid + 400 mg metronidazole tid x 2 weeks

Comparison (n=154): omeprazole 20 mg bid x 2 weeksSymptom resolution at 1 year: Tx = 21% vs. Comparison = 7% (p<0.001)

Blum et al (NEJM, 1998)Treatment (n=164): Omeprazole 20 mg bid + amoxicillin 1000 mg bid + 500 mg clarithromycin bid x 1 week

Comparison (n=164): omeprazole 20 mg bid x 1 weekSymptom resolution at 1 year: Tx = 27% vs. Comparison = 21% (p=0.17)

The Evolving Nature of Research

The Evolving Nature of Research

Analyzed 115 articles published in 1990-2003 in the 3 major general medical journals (NEJM, JAMA, Lancet) and specialty journals that had received over 1000 citations each by August 2004

49 reported evaluations of health care interventions; 45 claimed that the interventions were effective.

By 2004 5/6 non randomised studies and 9/39 randomised trials were already contradicted or found to be exaggerated

Ioannidis JP. JAMA 2005

But How Do I Know About the Quality and Validity

of the Study?

Understanding Clinical Research: What do I Need to Fully Interpret a

Clinical Study?

Understand Clinical Research Methodology: Epidemiology

Analytical ExpertiseQuantitative: BiostatisticsQualitative: Qualitative Analytics

Clinical SenseMedical / Healthcare Training

The Medical Research Spectrum(Sung, JAMA, 2003)

The Medical Research Spectrum(Sung, JAMA, 2003)

Basic Clinical Research Methodology

Comparative Clinical Research: A Simplistic Overview(Melot, CCM, 2009)

Approaches to Clinical Research

Two Major ApproachesExperimental Studies: Randomized Trials

Randomly allocate subjects to different treatment strategies and follow them up to assess outcomes

Operationally complex, analytically simple

Observational Studies Operationally simple, analytically complex

Randomized Clinical Trial: Design(Melot, CCM, 2009)

Validity and Biases in Clinical Trials(Melot, CCM, 2009)

Major Observational Study Types

Cross-sectional Study

Cohort Study

Case-Control Study

Case-Cross-Over Study

TIME

Basic Schematic for Cross-Sectional StudyBasic Schematic for Cross-Sectional Study

Data for analysis

Major Issues:1)Temporal Sequencing2)Selection Bias3)Confounding

TIME

Look-back Window

Index Entry Date

for Exposure

End of Follow-up Date

Observation Window

Basic Schematic for Cohort StudyBasic Schematic for Cohort Study

Major Issues:1)Selection Bias2)Confounding

Basic Schematic for Case-Control StudyBasic Schematic for Case-Control Study

Time Period

Event

No Event

Cases

Controls

Time Period

Major Issues:1)Selection Bias2)Confounding3)Only indirect estimates of time effects

Basic Schematic for Case-Crossover StudyBasic Schematic for Case-Crossover Study

Time Period A Time Period B

Event

Compare exposure in Time Period A vs. Time Period B

only among patients with an event and exposure in either period

Major Issues:1)Largely used for exposures with immediate effects2)Small number3)‘Reverse protopathic’ bias

How Do I Know Which Study Design is Best?

Level of EvidenceLevel of Evidence Study TypeStudy Type

Level 1Level 1 RCTsRCTs

Level 2Level 2 Cohort StudiesCohort Studies

Level 3Level 3 Case-Control Case-Control StudiesStudies

Level 4Level 4 Case SeriesCase Series

Level 5Level 5 Expert OpinionExpert OpinionOxford Centre for Evidence-Based Medicine, 2002

Analytical Aspects

Numeric Literacy(Horton and Switzer, NEJM, 2005)

Sophistication of statistical methods or articles published in the NEJM has been increasing over time

Some statistical tests T-tests Contingency tables Non-parametric tests Epidemiologic statistics Pearson’s correlation Nonparametric correlation (e.g. spearman’s correlation) Simple linear regreasion Analysis of variance Transformations

Only 21% of articles published between January 2004 – June 2005 would be considered accessible by those with basic training in biostatistics

Numerical Methods

Risks: Proportions of Patients Absolute Risk (AR) Number Needed to Treat (NNT)

Ratio of Risks Relative Risk (RR) Hazard Ratio (HR) Odds Ratio (OR)

Measures of Association: An Example

PlaceboPlacebo TreatmentTreatment

MIMI 100100 3030

No MINo MI 1990019900 1997019970

Total n = 40,000

N per study group = 20,000

Absolute Risks

AR (Placebo) = 0.5%

AR (Treatment) = 0.15%

ARR = 0.5% - 0.15% = 0.35%

NNT = 1/0.35% = 296

Relative Risks

RR = 0.15% / 0.5% = 0.30

RRR = 1/RR = 70%

OR (CC: Logistic Regression)

HR (Cohort: Cox PH Models)

Issues with Numerical Approaches

Lacy et al, Am J Cardiol, 2001

Assessed 400 health professionals for their willingness to prescribe a drug based on different measures for reporting the same likelihoods

Literature Evaluation Tools

Standards for Assessing Quality of Research

Clinical Trials CONSORT statement http://www.consort-statement.org/

Observational Research STROBE Statement http://www.strobe-statement.org/

Consort Statement

STROBE

Group Activity: Pitt et al

Group Exercise20 Minutes Group Review

One person from each group to present for up to 2 minutes

Use CONSORT

GroupsGroup 1: Abstract and IntroductionGroup 2: MethodsGroup 3: ResultsGroup 4: Comment – make recommendation

Pitt et al: Study Overview

RALES: Randomized ALdactonE StudyPitt et al, N Engl J Med, 1999

Double-blind randomized controlled trial in patients with congestive heart failure

Primary endpoint: all-cause mortality

Screened (n=?)

Randomize

Spironolactone (n=822)

25 mg po qd

- Increase to 50 mg po qd where tolerated

Placebo (n=841)

Inclusion Exclusion

RALES Criteria

Inclusion / ExclusionNYHA class III or IV at time of enrolmentLVEF < 35% within 6 monthsExclude patients with serum creatinine 2.5 mg/dL or serum potassium > 5 mmol/L

Follow-upLab and clinic follow-up at 4 weeks and 3 and 6 months

Appropriate use of ACE inhibitors and beta-blockers

D/c K-sparing diuretics and K+ supplementsHolding spironolactone for hyperkalemia or creatinine > 4 mg/dL

RALES - Results

OutcomeOutcome Absolute RiskAbsolute Risk Relative RiskRelative Risk

DeathDeath Pl=46% Pl=46% Spir=35%Spir=35%

p<0.001p<0.001

0.70 0.70

(0.60-0.82)(0.60-0.82)

ReadmissioReadmission for HFn for HF

Pl=36% Pl=36% Spir=26%Spir=26%

p<0.001p<0.001

0.650.65

(0.54-0.77)(0.54-0.77)

Serious Serious HyperkalemHyperkalem

iaia

Pl=1.2% Pl=1.2% Spir=1.7%Spir=1.7%

p=0.42p=0.42

NSNS

What Happened in Actual Practice?

Juurlink et al, NEJM, 2004

Spironolactone Prescription Uptake1994-2001

0

20

40

60

80

100

120

140

160

Year

Pre

scri

bin

g r

ate

(p

er

1000 p

ati

en

ts)

RALES

Hospital Admission Associated with K+

0

2

4

6

8

10

12

14

Year

Ad

mis

sio

n r

ate

(per

100

0 p

atie

nts

)

RALES

Admission Associated with K+ Ending in Death

0

0.5

1

1.5

2

2.5

Year

Death

rate

(p

er

1000 p

ati

en

ts)

RALES

What about the expected benefits?

Readmission for Heart Failure

0

50

100

150

200

250

Year

Ad

mis

sio

n r

ate

(p

er

10

00

p

ati

en

ts)

RALES

All-Cause Mortality

0

10

20

30

40

50

60

70

Year

Rat

e o

f dea

th fr

om

an

y ca

use

(per

100

0 p

atie

nts

) RALES

What Happened???

In the ‘Real World’….Different patients may get the drug

> 50% of population that would normally use the drugs in clinical practice often do not meet inclusion / exclusion criteria of RCTs (e.g. Gill et al, CJCP, 2004)

Patients may not be monitored as carefully

Patients may not take their drug as they should

Patients may take interacting drugs that they shouldn’t be on

Patients may not adhere to their drugs optimally

What’s the Problem?

Anton et alJ Clin Pharm Ther 2003; 28: 285-7

Anton et al. J Clin Pharm Ther 2003;28:285-7

Retrospective cohortSingle hospital110 patients Rx spironolactone + ACEI

Mean age 71 years, half with DM

OutcomesCessation of spironolactoneHyperkalemia

Findings

24% developed K+ (> 5.5 mEq/L) at 1 year

Many stopped spironolactone

J Clin Pharm Ther 2003;28:285-7

Bozkurt et alJ Am Coll Cardiol 2003;41:211-4

CriteriaCriteria Bozkurt et Bozkurt et alal

(n=104)(n=104)

RALESRALES

(n=822)(n=822)

NHYA (% patients)NHYA (% patients)

II

IIII

IIIIII

IVIV

UndocumentedUndocumented

4.5%4.5%

4.5%4.5%

15.3%15.3%

10.3%10.3%

65.4%65.4%

0%0%

0.5%0.5%

72%72%

27%27%

0%0%

% Patients with LVEF < % Patients with LVEF < 35%35%

54.8%54.8% 100%100%

% Patients with renal % Patients with renal insufficiency at insufficiency at baselinebaseline

30.7%30.7% ExcludedExcluded

Use of beta-blockersUse of beta-blockers 34.6%34.6% 11%11%

In a highly selected group of patients with severe heart failure who are largely free of other risk factors for hyperkalemia and who can be monitored closely, the addition of spironolactone to standard therapy (as defined in 1994) decreases hospitalization for heart failure and saves lives, with no significant risk of hyperkalemia.

What RALES said…

What we heard…• In patients with heart failure, spironolactone saves lives

Interpreting Evidence: Key Points

Interpretation of evidence depends on:

Availability of evidence

Ability to understand, assimilate, and interpret evidence and its limitations

Putting It All Together

What Drives Interpretation?

Study design / methodologyAppropriate design Inclusiveness / generalizability

Statistical analysis and reporting

Clinical relevance and ‘sense’

Where Does Evidence Fit Into the Decision-Making Process?

BELIEFS

BEHAVIOUR

Evidence Personal Values / Experience

Societal Values ‘Other’ Factors

Clinical Research:Writing a Scientific

Manuscript

What Do Journal Editors Look For?

Characteristic Implications

Clinical Relevance

Research question should be relevant:Ask other clinicians about question’s clinical importance

Originality Research question should ideally be unique:Perform a PubMed search

Clarity Research question should be simpleStudy design should be well explained with ONE primary endpointBefore beginning study – know which 4 or 5 figures and/or tables will be produced; make mock figures/tablesKnow your audience: simple language

Good Science Assemble a team: study lead, methodologist, statistician, clinician, at least one senior researcherMust have good design, sufficient sample size, and analysis plan

Brevity Keep the manuscript short: 2,000 – 3,000 wordsConsider different formats: brief reports, research letters

What will maximize the journal’s IMPACT FACTOR?

Fogg, BJ, et al, What makes Websites Credible? Stanford Persuasive Technology Lab

Author Photo Results

believable

trustworthy

competent

credible

unbiased

expert

COMPOSITE

casual photo

no photo

formal photo

Fogg, BJ, et al, What makes Websites Credible? Stanford Persuasive Technology Lab

The Anatomy of a Clinical Research Question

Is the question simple?The question must be easy to understand and the results must be easy to convey

Start with thinking about the 4 or 5 Tables or Figures that would result from your research

Is the question important?It should be important to others besides the researcher

Dependent on:Funding sourceIntended audience Expected impact

The Anatomy of a Clinical Research Question

Is it the question unique?Must have a solid background / contextExtensive literature search to assess novelty of the research

Is the question specific?ExposureOutcome: reflects only ONE well-defined primary endpoint

Nature of assessment (e.g. relationship vs. difference – need to state intervention)

Research ImpactHigh Impact

Great questions Great questionsGreat findings Poor findings

Low Impact

Poor questions

Great findings

No ImpactNo Impact

Poor questions Poor questions

Poor findingsPoor findings

Other ConsiderationsBefore beginning your study, know which journal(s) you would like to targetCarefully review their format requirements (usually on website)

Review sample studies from that journal

Focus on a good abstractMany journals screen articles for review based on the abstract

Review past article’s abstracts for format and wording style

Make cover letter brief but relevantProvide clinical context and rationale for the studyBriefly summarize study findingsBriefly summarize implications of study findings on clinical practice and/or public health

Try to keep it brief – maximum 1 page

Research and Communications: An Example

BackgroundA particular group of antibiotics called ‘fluoroquinolone’ antibiotics have become the most widely prescribed group of antibiotics About 22 million prescriptions dispensed annually in US for fluoroquinolone

antibiotics 16 individual fluoroquinolone antibiotics have been available on the

market; In 2005, gatifloxacin (Tequin) was the fluoroquinolone antibiotic of choice for US public health system

Some of these drugs have problems: serious adverse events have led to the withdrawal or restriction of several fluoroquinolones in recent years Temafloxacin (blood sugar and kidney problems) Grepafloxacin and sparfloxacin (heart problems) Trovafloxacin (liver problems)

In 2005, a student noticed major changes in blood glucose levels in patients receiving Tequin and approached researchers to investigate this observation

So We Conducted a Study (Park-Wyllie et al, NEJM, 2006)

PopulationExamined records of over 1.4 million elderly residents of Ontario age 66 years and older between 2002-2004

Analysis limited to individuals who were using one of several selected antibiotics

OutcomesHospital admission related to severe changes in blood glucose levels

What Were the Results?

Hospital admissions related to severe drops in blood glucose levelsGatifloxacin associated with 4x the risk of hospital admission compared to other antibiotics

Hospital admissions related to severe increases in blood glucose levelsGatifloxacin associated with almost 17x the risk of hospital admission compared to other antibiotics

The Basic Communications Plan

Raise the profile of the issue

Make the numbers tangible and meaningful

Target the right journal

Identify the appropriate audiences

Making the Numbers Tangible

If we consider the 1.4 million elderly residents of Ontario during the study timeframe:

There were nearly 17,000 courses of gatifloxacin treatment administered

For every 100 courses of gatifloxacin, we may expect 1 hospital admission for dysglycemia

On average, it was estimated that at least 1 elderly person in Ontario was hospitalized every week as a result of dysglycemia that was likely associated with gatifloxacin

ImpactPublished early online (March 1st, 2006) in the New England Journal of Medicine given its clinical relevance

Recognized nationally as a significant research contribution:Canadian Society of Clinical Pharmacology

Best Publication Award for 2006

Gatifloxacin withdrawn from market in May 2006