Systematic Review Module 2: Systematic Review Module 2: Analytic FrameworksAnalytic Frameworks

Melissa McPheeters, PhD, MPH Melissa McPheeters, PhD, MPH Associate Director for Methods, Associate Director for Methods,

Vanderbilt University EPCVanderbilt University EPCAssistant Professor, Institute for Medicine and Public HealthAssistant Professor, Institute for Medicine and Public Health

Assistant Professor, Obstetrics & GynecologyAssistant Professor, Obstetrics & Gynecology

Learning ObjectivesLearning Objectives

To understand the use of analytic frameworks within the systematic review process

To develop an understanding of principles for developing analytic frameworks

To review examples of formats for analytic frameworks

2

CER Process OverviewCER Process Overview

3

Prepare topic:

· Refine key questions

· Develop analytic frameworks

Search for and select

studies:

· Identify eligibility criteria

· Search for relevant studies

· Select evidence for inclusion

Abstract data:

· Extract evidence from studies

· Construct evidence tables

Analyze and synthesize data:

· Assess quality of studies

· Assess applicability of studies

· Apply qualitative methods

· Apply quantitative methods (meta-analyses)

· Rate the strength of a body of evidence

Present findings

What Is an AnalyticWhat Is an AnalyticFramework?Framework?

A type of evidence model, linking and defining clinical concepts, evidence, populations, and interventions that affect health outcomes (AHRQ Methods Guide; White and Ovid, 2009).

Also known as– Conceptual frameworks

– Influence diagrams

– Theoretical frameworks

– Simulation models

– Causal pathways

4

Why Use Analytic Why Use Analytic Frameworks? Frameworks?

Frames the review in the clinical context Allows the reviewer to easily identify

logical flaws as key questions are being developed

Provides the clinical and biological underpinnings for the mechanism of action of the intervention under review

Clarifies implicit assumptions in the review

5

The Importance of an The Importance of an Analytic FrameworkAnalytic Framework

Helps describe an appropriate set of outcomes

Identifies likely modifiers of effectiveness Can clarify likely links between

intermediate and health outcomes, such as cholesterol and myocardial infarctions (Woolf et al., 2006)

6

When to Use Analytic When to Use Analytic FrameworksFrameworks

Directing literature searches – (Bravata et al., 2005)

Constructing key questions– (Woolf et al.,1996)

Targeting future research– (Bravata et al., 2005)

Demonstrating logic– (Harris et al., 2001)

Making knowledge gaps visible– (Mulrow, Langhorne, and Grimshaw, 1997)

7

Building the Framework Building the Framework

8

Harris et al. (2001)Harris et al. (2001)

We most often focus on the treatment component of the overall analytic framework model

Model CER Framework for Model CER Framework for Behavioral OutcomesBehavioral Outcomes

9

Sample Screening Sample Screening andand Treatment FrameworkTreatment Framework

10

Hartmann et al. (2009)

Population Intervention Outcomes

Capturing the ContextCapturing the Context

11

Analytic frameworks can provide a means of capturing contextual issues important to a review

Defining the FrameworkDefining the FrameworkModelModel

Graphical chain of logicGraphical chain of logic– ArrowsArrows

● LinkagesLinkages

● Preventive service or treatmentPreventive service or treatment

● QuestionsQuestions

– Dotted linesDotted lines● AssociationsAssociations

– RectanglesRectangles● Intermediate outcomesIntermediate outcomes

Rounded cornersRounded corners

● Health statesHealth states Square cornersSquare corners

– Curved arrowsCurved arrows● Lead to ovalsLead to ovals

HarmsHarms

12

Harris et al. (2001)Harris et al. (2001)

ExerciseExercise

13

Overall functioning Therapy Sustained health behavior change

Treatment Intervention Individual diagnosed with condition

Harms Person at risk Long-term outcomes

Patient presenting symptoms Reduced morbidity or mortality Short-term outcomes

Intermediate outcomes Adverse effects Surrogate outcomes

Key Research QuestionsKey Research Questions

Key questions should be numbered on the Key questions should be numbered on the diagram in the corresponding area of the diagram in the corresponding area of the framework (Whitlock et al., 2002).framework (Whitlock et al., 2002).

14

11

22

33 44

Building the FrameworkBuilding the Framework

Analytic frameworks can be developed more easily by adapting them from previously established similar frameworks, which is particularly helpful for young research topics (Bravata et al., 2005).

Using PICOTS, make sure you can easily identify at least the population, intervention(s), outcomes, and timing on the framework; sometimes the comparator and setting are also made clear

Place your key questions on the draft framework if they don’t fit easily; either the questions or framework may be off target

Stand back and consider: the process of developing an analytic framework and associated key questions is iterative

15

Using the FrameworkUsing the Framework

Don’t build it and forget it—use the framework throughout the review process to– Revisit inclusion/exclusion criteria

– Keep a handle on scope

– Reconsider the appropriateness of key questions

Let the framework help in structuring the report and results

16

SummarySummary

Analytic frameworks provide patients, reviewers, decisionmakers, and providers with a model of evidence linking populations, interventions, and outcomes

Frameworks direct literature searches, pinpoint key questions, target future research, demonstrate logic, and build arguments for legitimacy of therapeutic care

Frameworks are created by building a graph of arrows, dotted lines, rectangles, and ovals, and by plugging in corresponding key research questions– These can be easily adapted from previously constructed

frameworks in relevant topic areas

17

References References Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews:

Synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005 142(12 Pt. 2):1056-65.

Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: A review of the process. Am J Prev Med 2001 20(3 Suppl):21-35.

Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of overactive bladder in women. Evidence Report/Technology Assessment, No. 187. Rockville, MD: Agency for Healthcare Research and Quality. 2009 Aug. AHRQ Publication No. 09-E017.

Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997 127(11):989-95.

Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: An evidence-based approach. Am J Prev Med 2002;22(4):267-84.

White CM, Ovid SP. AHRQ methods guide. Rockville, MD: Agency for Healthcare Research and Quality. Search 2009 July 23. Chapter 2, Finding evidence.

Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: Lessons learned by the US Preventive Services Task Force. Annu Rev Public Health 1996 17:511-38.

18