figure: karolinska institute how to ... - virology...

How to do a systematic review: An introduction

Joseph Tucker, Matthew Law, Nathan Ford

Figure: Karolinska Institute

Overview

1) Introduction & Examples

2) Purpose

3) Steps & Outputs

4) Resources

Figure: Centre for Evidence-Based Medicine, Oxford

Learning Objectives

- To highlight the difference between a systematic review and other types of reviews- To provide examples of the range of research questions to which systematic reviews can be applied- To summarize the key steps to undertaking a systematic review & meta-analysis- To summarize the main graphical elements that can be found in a systematic review- To provide key resources for further reference

Introduction: Key Definitions

Review The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic

Scoping review

A rapid review to identify literature on a topic for a specific purpose, but not rigorous enough to be a systematic review

Systematic review

A review that strives to comprehensively identify and track down all the literature on a given topic; typically follows PRISMA checklist and is registered (may or may not include meta-analysis)

Sackett, D et al. Clinical Epidemiology: A basic science for clinical medicine 2nd ed. Little, Brown & Company, 1991.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Introduction: Why a systematic review?

• Massive amounts of scientific data

• Decision makers only want the key bits

• Efficiency (cheaper than another RCT)

• Generalisability, consistency, and inconsistency

• Power and precision

• Accuracy

Murrow, BMJ, 1994

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Quick Quiz 1

Your mentor just finished a massive RCT and asks you to systematically search the literature on the same topic. You consult a librarian, find some good databases, and double extract the data. You follow tick off the PRISMA checklist boxes, but don’t register it in PROSPERO. You don’t have enough similar outcomes to combine results of several studies into single estimates. You will write up this review and wonder what you should call it:

Quick Quiz 1

a) Opinion

b) Scoping review

c) Systematic review (no meta-analysis)

d) Systematic review with meta-analysis

e) Other

Examples of Systematic Reviews

1. Comparative efficacy and safety

• Drugs

• Public health interventions

2. Prevalence

• Disease

• Exposures

• Outcomes

3. Diagnostic test accuracy

4. Qualitative research

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Comparative safety & efficacy:HIV ART adherence interventions

ConclusionsCounseling, peer support and SMS messaging improve adherence to treatment

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Outcome prevalence: Causes of hospitalization among PLHIV

ConclusionsTB and severe bacterial infections Leading causes of mortality among adults and children worldwide

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Diagnostic test accuracy: Syphilis test sensitivity, specificity

ConclusionRapid syphilis tests are accurate compared to standard syphilis tests.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Qualitative Research: Experiences Using HIV Self-Testing

ConclusionsHIV self-testing can empower users, but may also be associated with pressured testing in some contexts.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Quick Quiz 2

Which of the following is NOT an example of a potential systematic review:1) Comparing HIV outcomes from selected studies2) Exploring themes in published selected syphilis

studies3) Determining the sensitivity of a new test

compared to a gold standard based on selected studies

4) Using mathematical modelling to estimate HIV transmission

Purposes of systematic review

1. To provide a summary of the literature on a specific question using a standardized methodology that can be replicated

2. To assess risk of bias within the existing literature

3. To inform experts and policymakers as they make decisions about guidelines and guidance

4. To outline directions for future research (gaps in the literature)

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Steps in a Systematic Review

1. Develop a protocol (with librarian if serious) and register it on PROSPERO

• Databases to search

• Inclusion and exclusion criteria• Population, Intervention, Comparitor, Outcome

• Risk of bias

• Analytical methods to be used

2. Run searches (in duplicate if serious)

3. Extract data (in duplicate if serious)

4. Analyse data (with statistician if serious)

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Quick Quiz 3

You finish the searching and are now extracting data for the review. Your mentor remembers that you need to register it on PROSPERO. What is the appropriate time to register a review on PROSPERO?1) Immediately, as soon as you have a broad topic2) Once you have a review protocol3) After you finished the data extraction for the

review4) After you finish the review

Outputs of a Systematic Review

1. Flow diagram: search strategy

2. Table: Study characteristics

3. Table: risk of bias assessment

4. Table: summary of outcomes

5. Forest plot: meta-analysis

6. Other

• Funnel plot (publication bias)

• Meta-regression

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Flow diagram

11806 titles screened

33 studies included

Reasons for exclusion

Zhou et al., Interventions to optimize the care continuum for chronic viral hepatitis: A systematic review and meta-analysis. Lancet ID. 2016.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Study Characteristics

Zhou et al., Interventions to optimize the care continuum for chronic viral hepatitis: A systematic review and meta-analysis. Lancet ID. 2016.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Risk of bias

Sinclair et al, Artesunate versus quinine for treating severe malaria (Review). Cochrane systematic reviews 2012.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Summary of outcomes table

Tucker et al. Accelerating worldwide syphilis screening through rapid testing: A systematic review. Lancet ID. 2010.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Forest plot

Zhou et al., Interventions to optimize the care continuum for chronic viral hepatitis: A systematic review and meta-analysis. Lancet ID. 2016.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Meta-regression% risk reduction in major cardiovascular events against the difference inachieved systolic blood pressure between study treatment groups

Ettahad et al, Blood pressure lowering for prevention of cardiovascular disease and death: a systematic review and meta-analysis. Lancet 2015

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

When to do a meta-analysis?

Why do a meta-analysis?• Existing studies gave disparate results• Bigger study population (sample size) can increase

power, generalisability and precision of findings (effect estimate)

• Subgroup analyses may generate new hypotheses

Does it make sense?• Did the individual studies address the same research

question?• Are the studies included in the meta-analysis of

comparable quality?• Are the studies comparable (eg. population,

duration/dosage of treatment)?

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Resources

Cochrane Handbook (how-to manual)http://training.cochrane.org/handbook

Cochrane database of systematic reviews (for examples)http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/

PRISMA STATEMENT (checklist for publicationhttp://prisma-statement.org

PROSPERO (registration of protocols)https://www.crd.york.ac.uk/prospero/

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Resources

Khan et al., Five steps to conducting a systematic review, Journal of the Royal Society of Medicine, 2013, 96:118-121.

Lewin et al., Using Qualitative Evidence in Decision Making for Health and Social Interventions: GRADE-CERQual, PLoSMedicine, 2015

Moher et al., Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement, Systematic Reviews, 2015, 4:1.

Ried, K. Interpreting and understanding meta-analysis graphs, Australian Family Physician, 2006; 35: 635-638.

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Future Directions

• Network meta-analysis

• Individual patient data meta-analysis

• Living systematic review

• Crowdsourcing for systematic reviews

1) Introduction & Examples2) Purpose3) Steps & Outputs4) Resources

Small Group Activity

• Often the most difficult part of a systematic review can be finding a good topic. Divide into four groups (HIV, STD, hepatitis, other) and come up with two PICOs (Population, Intervention, Comparitor, Outcome) for systematic reviews within your field

• Be prepared to report back to the large group in 20 minutes.