seminaar on meta analysis

META-ANALYSIS

SPEAKER:- PREETI RAI

TEACHER I/C:- DR. B. L VERMA

DATE:- 12/11/2013

DRINKING RED WINE IS BENEFICIAL TO HEALTH-US STUDY

USE OF MOBILE PHONES LEAD TO

NEUROMAS-GERMANY RED WINE IS

HARMFUL-US

MOBILE PHONES CAUSE NO HARM-

EUROPE

SOFT DRINKS CONTAIN HARMFUL

PESTICIDES

SOFT DRINKS ARE SAFE

CHOCOLATES ARE GOOD FOR HEART

CHOCOLATES ARE HARMFUL

WHAT IS THE ANSWER ??

HOW TO TAKE DECISION ??

Meta-analysis A meta-analysis is a quantitative study

wherein a set of statistical procedure is used to summarize and synthesize results of a number of independently conducted research studies. If done well it can be very valuable to a researchers , because it provides an extensive bibliography of existing research on a topic ,while also providing a combined analysis of a number of study.

Meta-analysis is a very time consuming, understanding and usually conducted by a team of researchers.

Thus, Meta Analysis is -

A Quantitative approachFor

systematically combining the results of

previous researches to

arrive at conclusions about

the body of research.

What does it mean?

Quantitative : numbersSystematic : methodicalCombining : putting together

Previous research : already done

Conclusions : new knowledge

Important

1- MA falls under a broader classification of reviews known as Systematic Reviews

2- There are two types of SRs – (a) Qualitative(b) Quantitative (meta-analysis)

Important

Both follow the same rigorous steps, EXCEPT that

a qualitative review does not combine the endpoints for statistical analysis, usually because it’s not appropriate to combine them into any type of common Metric.

The Etymology

"Meta" implies something occurring later, more comprehensive.

Alternative terms are less specific — for example, "overview" is also used for traditional reviews, & "pooling" incorrectly implies that the source data are merged .

Historical notes

Karl Pearson (1904) - Use of formal techniques to combine data from different samples.

Glass (1976) coined the term meta-analysis

Rationale

“by combining the samples of the individual studies,

the overall sample size is increased, thereby improving the statistical power of the analysis

as well as the precision of the estimates

of treatment effects”

Objectives

The benefits or hazards that might not be detected in small studies can be found .

Integrating the findings. Identifying the treatment effect (or

effect size) when it is consistent from one study to the next.

Identifying the reason for the variation when the effect varies from one study to the next .

Importance MA

Too much scientific information. A Well-informed clinical decision is difficult to

reach, time consuming & cost-ineffective. Decisions about the utility of an intervention or

the validity of a hypothesis can’t be based on results of a single study.

Information from many studies with less effort & hassle .

Advantages of MA Saves effort and time

Increases sample size – Gain in statistical power by reducing Random errors.

Enhances reliability (precision) & accuracy (validity).

Explores & Reduces bias.

Resolves controversies &conflicting reports

Identifies crucial areas & questions that have not been adequately addressed with past research.

Generalizes study results.

May explain heterogeneity & its sources between the results of individual studies.

Answers questions about whether an overall study result varies among subgroups—for example, among men and women, older and younger patients, or subjects with different degrees of severity of disease.

Reproducible numerical values – no place of unhelpful descriptors such as "no relation," "some evidence of a trend”, "a weak relation," and "a strong relation“.

Applications OF M A

Clinicians & applied researchers - to determine which interventions work, and which ones work best.

Basic research - to evaluate the evidence in diverse areas.

Planning new studies. Some funding agencies now

require it as part of the grant application to fund new research.

CAUTION

Not used or meaningless, when –

1. Studies are different in terms of their population, intervention or how outcomes were measured.

2. Treatments, evaluated in the individual studies are different.

3. The findings of individual studies differ significantly – because combining widely differing results to produce an average effect would fail to represent the great variation in the outcomes .

Types of MA

1. Literature-based MA (LBMA)- most frequent type of MA- may be misleading as Data extraction &

analysis may be less accurate 2. Individual patient data MA (IPDMA)

- Gold-standard, but has problems- inability of investigators to supply data- the increased costs

Principles of MA

1-The need to consider the totality of evidence.

2-Requirement for Reproducibility Transparent, explicit & systematic approach.

3-Principles of reliable detection of the effects of health care interventions

The Process

The process simply involves:1. Calculation of the treatment effect i.e. OR/RR.2. Calculation of the 95% Confidence interval

around the individual OR/RR.3. Giving a weight to the individual OR/RR (shown

as the size of the box in the forest plot). The weight is calculated as the inverse of the square of the standard error of each OR/RR (1/SE2).

How do we conduct MA

1. Write a protocol – the blueprint2. Locating relevant studies 3. Selecting & appraising studies for inclusion 4. Data extraction from selected studies 5. Statistical methods to combine the effect

measures extracted from primary studies 6. Addressing biases and limitations 7. Results in graphical form – the Forest Plot

Recipe of a good protocol

1. Purpose of meta-analysis.2. Design a research question.3. Search for studies.4. Specify study selection (inclusion & exclusion) &

appraisal criteria.5. Decide data extraction procedures (including statistical

reanalysis).6. Select an analytical strategy (use of models & sensitivity

analysis).7. Anticipate systematic errors (biases)/limitations.8. Present & disseminate results .

Locating relevant studies

Systematic approach.

Primary objective – Strategically locate as much of the completed research on the topic as possible.

Document strategy in sufficient detail to allow others to critique it’s quality.

Usually include e-databases (MEDLINE, CINAHL, Psyclit, Embase, Cochrane Library) and others


1. Write a protocol – the blueprint.2. Locating relevant studies. 3. Selecting & appraising studies for inclusion . 4. Data extraction from selected studies. 5. Statistical methods to combine the effect

measures extracted from primary studies 6. Addressing biases and limitations. 7. Results in graphical form – the Forest Plot.

Selecting & appraising studies for inclusion

Selecting - Judge the relevance of the studies to the review question.

Appraising - Judge numerous features of design & analysis .

Methodical, impartial and reliable strategies are necessary as MA are retrospective exercises & are therefore susceptible to both random & systematic sampling errors .

Rationale - by excluding lesser quality studies the risk of error/bias will be lessened.


1. Write a protocol – the blueprint.2. Locating relevant studies. 3. Selecting & appraising studies for inclusion. 4. Data extraction from selected studies. 5. Statistical methods to combine the effect

measures extracted from primary studies. 6. Addressing biases and limitations. 7. Results in graphical form – the Forest Plot.

Data extraction

Eligibility criteria for inclusion of data .

Data collection in standardized record form.

2 independent observers extract the data, to avoid errors.

Blinding observers to the names of the authors, their institutions, the names of the journals, sources of funding, and acknowledgments leads to more consistent scores.

How to Extract Data

Create a spreadsheet (Excel) For each study, create the following columns:

name of the study name of the author, year published number of participants who received intervention number of participants who were in control arm number who developed outcomes in intervention number who developed outcomes in control arm

We got like 22 studies to do our meta analysis, after all

Statistical Methods

They attempt to answer basic questions: (a) Are results of different studies similar? -Check for heterogeneity.

(b) To what extent that they are similar? -Calculate the amount of heterogeneity.

(c) What is the best overall estimate?

- Combine the effect measures using suitable model & calculate the summary effect size & its CI.

(d) How precise & robust is this estimate? - Do Sensitivity Analysis.

(e) Finally, can dissimilarities be explained?

Heterogeneity if present, should not simply be ignored after a statistical test is applied; rather, it should be scrutinized and explained.

More weight is given to – (a) larger trials(b) Studies with narrow CI

• Assess the heterogeneity of effect size across the studies• Decide the type of model for combining the effect size of all studies. • 2 models to adjust the potential confounding effects of study – (1) Fixed Effect model . (When the combined trials are a homogeneous set) (2) Random Effect model. (When heterogeneity is detected)



measures extracted from primary studies.6. Addressing biases and limitations .7. Results in graphical form – the Forest Plot.

BIASES & LIMITATIONS

Publication bias Selection bias Data extraction bias Database bias Citation Bias Data provision Bias

Publication Bias Synthesis of published data can yield an

exaggerated effect as studies that yield relatively large/beneficial treatment are more likely to publish .

English language bias, citation bias, & multiple

publication bias- In English ,studies are more likely to be cited, and more likely to be published repeatedly.

High likelihood of publishing – - Studies sponsored by government or NGO.- Multi-centric studies.

Many authors may not submit studies with negative findings because they anticipate rejection.

Other Biases

Selection/Inclusion Bias-Manipulation of the inclusion criteria could lead to selective inclusion of studies with positive findings.

Database bias – Selective inclusion of studies from developed countries.

Citation bias – Ease of locating and contacting authors from reference lists.

Data provision Bias – due to willingness of investigators to make their data available.

Testing for bias – Funnel Plot

The presence of bias should be examined in sensitivity analyses and funnel plots.

Funnel plot – It is graphical test for any type of bias that is associated with sample size. Results from small studies will scatter widely at the

bottom of the graph. The spread will narrow as precision increases among larger studies.

In the absence of bias, the plot should thus resemble a symmetrical inverted funnel.

If the plot shows an asymmetrical & skewed shape, bias may be present .

Funnel Plot: what & how to read

To study a funnel plot, look at its LOWER LEFT corner, that’s where negative or null studies are located

If EMPTY, this indicates “PUBLICATION BIAS”

Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias



measures extracted from primary studies. 6. Addressing biases and limitations 7. Results in graphical form – the Forest Plot.

Forest plots

Effective way of presenting results: Studies, effect sizes, confidence intervals Provides an overview of consistency of

effects Summarizes an overall effect (with

confidence interval) Useful visual model of a meta-analysis

Anatomy of a forest plot…

Studies

N of study

Line of no effect

C.I

Study effect size

Pooled effect size

Pooled effect size

Study effect size (with C.I.)

Weighting of study in meta-analysis

When individual studies are inconclusive deficient or its not possible to do Multicentric RCT , Money problem ,Time nahi mila.

56

Weighting studies

More weight to the studies which give us more information More participants More events Lower variance

Weight is closely related to the width of the study confidence interval: wider confidence interval = less weight

3+

2+

1-

4+

5+

EFFECT OF β-BLOCKADE AFTER MI

SG1 = MIX POP

SG2 = MIX POP

SG3 = GERMAN POP

SG4 = MIX POP

SG5 = MIX POP

META ANALYSIS

BENEFICIAL EFFECT OF β-BLOCKADE

13+

Can’t improve the quality or reporting of the original studies.

Limitations arising from mis-applications: - when study diversity is ignored or mishandled , and - when variability of patient populations’, quality of data & potential for underlying biases are not addressed.

Publication bias is a major limitation.

Limitations of MA

Some clinicians consider it as "a tool that has become a weapon” & which represents "the unacceptable face of statisticism" & "should be stifled at birth”

At the other end of the spectrum, the application has been hailed “Newtonian”.

Some reject and see it as "mega-silliness”

MA continues to be controversial technique ?

The mixed reception is not surprising

The pooling of results from a particular set of studies may be inappropriate from a clinical point of view, producing a population " average" effect.

Meta-analyses of the same issue may reach opposite conclusions.

Still, Meta Analyses hold promise….

If original studies of the effects of clot busters after heart attacks had been systematically reviewed, the benefits of therapy would have been apparent as early as the mid-1970s.

Traditional approaches were inadequate in summarizing the current state of knowledge & omitted mention of effective therapies.

Popularity of Meta Analyses

0

500

1000

1500

2000

2500

3000

93-94 94-95 95-96 96-97 97-98 98-99 99-00 2000-1 2001-2 2002-3 2003-4

Nu

mb

er o

f P

ub

licati

on

s

Year of Publications

Refrences Lipsey, M.W., Wilson, D. B. Practical meta-analysis. Thousand Oaks, CA Sage; (Applied Social

Research Methods Series; 49), 2001.

Petitti, D. B. Meta-analysis, decision analysis, and cost-effectiveness analysis: Methods for quantitative synthesis in medicine (2nd ed.). New York Oxford University Press; 2000.

Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods of meta-analysis in medical research. West Sussex, England Wiley, 2000. The Handbook of Research Synthesis-Harris Cooper & L Hedges

Statistical Methods for Meta-Analysis – L Hedges & I OlkinLike Practical Meta-Analysis - Mark Lipsey and David Systematic Reviews in Health Care: Meta-Analysis in context - M Egger, G Davey-Smith, D G

Altman, Foreword by Iain Methods for Meta-Analysis in Medical Research- AJ Sutton, K R Abrams, DR Jones, TA Sheldon, F

SongLike Meta-Analysis in Medicine and Health Policy - DK Stangl, DBerry Publication Bias in Meta-Analysis - H Rothstein, A Sutton, M Borenstein How Science Takes Stock: The Story of Meta-Analysis - Morton Methods of Meta-Analysis: Correcting Error and Bias in Research Findings - John E. Hunter and

Frank L. Schmidt Synthesizing Research - A Guide for Literature Reviews - Harris Meta-Analysis of Controlled Trials - Anne Whitehead

THANK YOU