a random effect model with quality score for meta … · a random effect mode1 with quality score...
TRANSCRIPT
A RANDOM EFFECT MODEL WITH
QUALITY SCORE FOR
META-ANALYSIS
Hongmei Cao
A thesis submit ted in conformity with the requirements
for the degree of Master of Science
Graduate Department of Community Health
Cniversity of Toronto
@Copyright by Hongmei Cao 200 1
National Library I*l of Canada Bibliothèque nationale du Canada
Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 WeUington Street 395. rue Wellington OtfawaON K1AON4 OttawaON K1AON4 Canada Canada
The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant a la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sell reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de
reproduction sur papier ou sur format électronique.
The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substanhd extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.
A Random Effect Mode1 with Quality Score for
Meta-analysis
Hongmei Cao, h1.S~.
Department of Cornrnunity Health
University of Toronto, 200 1
Abstract
!deta-anaiysis is a set of statistical maneuvers to quantitatively sumrnarize rnulti-
ple related studics. There are two major stat ist ical approaches to niet a-analysis. One
is fised effect models. d i i c h assume that al1 studies arc governeci by a cornmon treat-
ment effect. and take the studies be analyzed as the iiniversc of intercst: the other
is randorn effect models. which allow ciifferent treatrnent efI'ects for different studies.
and treat these studies as representing a sample from a larger population of possible
studies. Because the reliability of data used in the meta-analysis is different. in or-
der to get more accurate results. quality scores can be introduced to mode1 the data
reliabilit. In this thesis. 1 study a randorn effect mode1 incorporating quality score
for met a-analysis in comparison s i t h t hree ot her models. 1 apply Iikelihood met hods
and the bootstrap to estimate the study effect. and the bias and standard errors of
the estimators. 1 apply the mode1 to a real data set. and to simulated data sets.
By analyzing systemiitic cornputational results. 1 find that the random effect mode1
with quaiity score can offer better accuracy. precision and coverage of the estimate of
study effect than the other three models.
Acknowledgment s
1 thank my supervisor. Prof. David Tritchler. a h o has offered me valuable guid-
ance and assistance during the course of my MSc. study.
1 thank the niembers of rny departmental oral esamination cornmittee. Prof. David
Tritchler. Prof. Salomon SIinkin. Prof. Paul Corey and Prof. George Tomlinson. for
their valuable comments antl feeclback on the nianuscript of rny thesis.
1 thank rny hiisband. Jiangbin Yang. a h o has acconipanicd me for many days and
nights of hart1 work. nho has understood and appreciated me. nho has sliarcd many
things witli nie. antl who has supported ancl encouraged me.
Contents
Abstract 1
. O O Acknowledgments 111
1 Introduction 1
1.1 Historical Devrloprncnt of Meta-Analysis . . . . . . . . . . . . . . . . 1
1.2 Objective and Steps of 'rieta-.\nalysis . . . . . . . . . . . . . . . . . . :3
1.3 Two Approaches to Statist ical Iieta-Xnalysis . . . . . . . . . . . . . 4
1.4 Quality of Stiiclies in Ileta-Analysis . . . . . . . . . . . . . . . . . . . S
1.5 A Probability Mode1 to be Studied in This Thesis . . . . . . . . . . . 9
1.6 Contents and Organization of the Thesis . . . . . . . . . . . . . . . . 12
2 Methods and Techniques for Analysis 13
'2.1 Derivation of Likelihood Functions . . . . . . . . . . . . . . . . . . . 14
2.2 Monte Carlo Simulation . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Bootstrap
. . . . . . . . . . . . . . . . . 2.3.1 Concept and Idea of Bootstrap
. . . . . . . . . . . . . . . 2.3.2 Bootstrap Procedure in This Thesis
3 Application of the Mode1 to Simulation Data
. . . . . . . . . . . . . . . 3.1 Cornputational Results of Simulation Data
3.2 Analysis of the Computational Results for Cornparison of the SIodels
4 Application of the Mode1 to a Real Data Set
. . . . . . . . . . . . . . . . . . . . . . . 4.1 Background of the Data Set
4.2 Application of the Mode1 to the Data Set . . . . . . . . . . . . . . . .
5 Conchsion
A S-plus hinctions
. . . . . . . . . . . . . . . . . . . . . . 1 Object Function of the Models
. . . . . . . . . . . . .-\ . 1.1 Ranclom Effect !dodel a i t h Quality Score
. . . . . . . . . . . . . .A . 1.2 Fixed Effect SIodel with Quality Score
.A . 3 Random Effect SIodel wit hout Quality Score . . . . . . . . . .
. . . . . . . . . . . . . . . A.2 Information SIatris (Function) for .\ Iodels
. . . . . . . . . . . . 4 . 2 . 1 Random Effect Slodel mith Quality Score
. . . . . . . . . . . . . A.2.2 Fixed Effect .\ Iode1 a i th Quality Score
-4.3.3 Random Effect Mode1 without Quality Score . . . . . . . . . . 70
-1.3 Bootstrap Procedure for Estimation of Biases and Standard Errors . 71
-- A.4 Nain Simulation Fiinction . . . . . . . . . . . . . . . . . . . . . . . . , a
Bibliography 81
List of Tables
1.1 Arrangement of data and table notation for application of Slantel-
Haenszel and Peto methods . . . . . . . . . . . . . . . . . . . . . . . .. *3
3.1 Simulation results for r" O. 1 x e x p ( 4 . 3 ) and 0.5 < Q, c 1 . . . . . 27
3.2 Simulation results for r 2 = 0.25 x e r p ( 4 . 3 ) and 0.5 < Q, < 1 . . . . 28
3.3 Simulation results for r2 = 0.5 x exp( -2 .3 ) and 0.5 < Q, < 1 . . . . 29
3.4 Simulation results for r' = 0.73 x e x p ( 4 . 3 ) and O.: < Q, < 1 . . . . 30
3.5 Simulation results for r' = 1 .O0 x e r p ( - 2 . 3 ) and 0.5 < Q, < 1 . . . . 31
3.6 Simulation results for r' = 2.00 x exp(-'1.3) and 0.5 < Q, < 1 . . . . 32
3.7 Simulation results for r' = 0.10 x erp(- '1.3) and 0.75 < Q, < 1 . . . 33
3.8 Simulation results for r' = 0.25 x e x p ( 4 . 3 ) and 0.7.5 < Q, < 1 . . . 34
3.9 Simulation results for T' = 0.50 x e r p ( 4 . 3 ) and 0.7.3 < Q, < 1 . . . 35
3.10 Simulation results for r2 = 0.75 x e r p ( - 2 . 3 ) and 0.7.5 < Q, < 1 . . . 36
3.11 Simulation results for r2 = 1.00 x exp( -2 .3 ) and 0.75 < Q, < 1 . . . 37
3.12 S i m u i a t i o n r e s u l t s f o r r ~ = 2 x e ~ p ( - 2 . 3 ) a n d 0 . 7 . 5 < Q , < 1 . . . . . 35
vii
3.13 Simulation results for T' = 0.10 x e x p ( 4 . 3 ) and 0.99 < Q, < 1 . . . 39
3.14 Simulation results for r' = 0.25 x e x p ( 4 . 3 ) and 0.99 < &, < 1 . . . 40
3.13 Simulation results for r' = 0.50 x e x p ( 4 . 3 ) and 0.99 < Q, < 1 . . . 41
3.16 Simulation results for r' = 0.75 x e r p ( 4 . 3 ) and 0.99 < Q, < 1 . . . 42
3.17 Simulation results for T' = 1.00 x e x p ( 4 . 3 ) and 0.99 < Q, < 1 . . . 43
3.18 Simulation results for r' = 2.00 x exp(-2.3) and 0.99 < &, < 1 . . . 44
4 Pa r t Io f the rea l c i a t a sc t . . . . . . . . . . . . . . . . . . . . . . . . . 61
-4.2 Part II of the real data set. . . . . . . . . . . . . . . . . . . . . . . . . 62
4.3 Slodcling results of the real data . . . . . . . . . . . . . . . . . . . . 64
. *. V l l l
List of Figures
3.1 Plots of the bias of the estiniateti stucly effect with respect to r' of
the four niodels with and without bootstrap atljiistmcnt for the thrce
ranges of quality scores. . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 Plots of the absoiutc values of bias of the estirnatetl study effitct with
respect to T? of the four moclels a i t li and [rit hoiit bootstrap iicljustrncnt
for the three ranges of qiiality scores. . . . . . . . . . . . . . . . . . 30
3.3 Plots of SISE of the estimated study effect with respect to r- of the four
niodels with and without bootstrap adjustment for the three ranges of
quality scores. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 l
3.4 Plots of the 95% coverage of the estimated stiidy effect tvith respect to
r' of the four rnodels with and withaiit bootstrap adjustment for the
three ranges of quality scores. . . . . . . . . . . . . . . . . . . . . . . 52
3.5 Plots of the 90% coverage of the estimated study effect with respect to
r' of the four models with and without bootstrap adjustment for the
three ranges of quality scores. . . . . . . . . . . . . . . . . . . . . . . 53
3.6 Plots of the b i s of the estimated stridy effect with respect to r2. for
the three ranges of quality scores. of the four niodels a i th and nithout
bootstrap adjustment. . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.7 Plots of the absolute rdlues of bias of the estiniated study effect with
respect to T'. for the three ranges of quality scores. of the four models
- - with and without bootstrnp adjustment. . . . . . . . . . . . . . . . . a;,
3.8 Plots of SISE of the ~s t imatrd stiidy ~ffecit rvit h respert to r'. for the
three ranges of quality scores. of the four niociels witli ancl aitliorit
bootstrap adjustment. . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.9 Plots of the 95% CI coverages of the est imated st iidy effect nit ti respect
to r? for the threc ranges of quality scores. of the four models with
-- and without bootstrap adjustment. . . . . . . . . . . . . . . . . . . . 9 1
3.10 Plots of the 90% CI coverages of the estimated study effect wit h respect
to 7'. for the three ranges of quality scores. of the four models with
andwithout bootstrapadjustrnent. . . . . . . . . . . . . . . . . . . . 523
Chapter 1
Introduction
1.1 Historical Development of Meta-Analysis
\Vhat is niet ;wmalysis? The terni 'm~ta-analysis' was inventecl by GS' . Glass in
1976 (rcfcrriice 10). where 'meta' is a Greek prefis mliich nieans 'transceiiding'. and
.analysis' is the root. A definition of meta-analysis given by Diana B. Petitti is
as foilons: 'Meta-analysis is a quantitative approach for systematically conibining
the results of previous research in order to arrive .et conclusions about the body of
research. S tudies of a topic are first systemat ically ident ified. Cri teria for including
and excluding studies are defined. and data from the eligible st udies are abstracted.
L a t . the data are comhined statistically. yielding a quantitative estimate of the size
of the effect of treatment and a test of homogeneity in the estimate of effect size
(reference 2 1) .' However, before the word .met a-analysis' was coined. the st at ist ical
combination of data from previous studies in the same topic had been widely used.
Because literally hundreds of studies esisted for the same topics in the mid-1970s.
the techniques of combining data from man' studies of the sanie topic have became
more and niore popiilar anci important since late 1970s. During the late 1970s and
the early 1980s. meta-aniilysis and its statistical metliocls were developed in the re-
search work of social scientists. inclucling Rosenthal (1978). Gliiss. '\lcGaw. and Smith
(1% 1). and Heclges ( 1982. 1983). Hunter. Schmidt ancl .Jackson (198'2). Light ( 1983).
and Ligtit and Pillcniitr ( 1984). It ivas important that they espandecl the goal of the
analysis to inclticIr the attcnipt to systematically iti~ntify the stiidirs to br cornhineti.
not just the cornhination of data. It nas eclually important that they consicirr the
estimation of effect size. not jiist statistical significance which had bcen a primary
aim of niet a-arialysis.
The ilse of nieta-analysis in medical research became popiilar right after its p o p
ularization in the social sciences. In the late 1980s. descriptions of the methods of
meta-analysis appeared almost simultaneoiisly in three influential general medical
journals. the New England Journal o/ ibiedicine. Lancet. and the ilnnal.9 O/ Interna1
Mededicine (L'Abbe. Detsky. O'Rourke 1987: Sack et al. 1987: Btilpitt 1988). Due to
the increasing focus on randomized clinical trials. meta-analusis \vas used sidely in
medical research and benefited from the rising level of concern about the interpreta-
t ion of small and individually inconclusive clinical t rails.
1.2 Objective and Steps of Meta-Analysis
The overall objective of meta-analysis is to combine the results of previous studies
to analyze them and get sumrnary conclusions about a topic of research. Especially
when the individual sttidies are small. rneta-analysis is the most iiseful maneuver in
siimmarizing them to reach a valid conclusion.
There are four steps in a meta-analysis:
1. Identify the studies with relevant data in the same topic for the interided meta-
analysis. The development of sys temat ic and esplici t procedures for itlcnt ifying
the stuclies with relevant data in the sanie topic is ver- important for proper a p
plication of meta-analysis rnethods. The systernaticriess ancl esplicitness of the
procedures For st udy iclentification make a clist inct ion beriveen rneta-analysis.
which is quantitative. and qualitative literature revieiv. The systernatic nature
of procedures will reduce bias and help guarantee reproducibility.
2. Define eligibility criteria for the meta-analusis. Sot al1 the studies can or should
be included in the meta-analysis. To ensure reproducibility of the rneta-analysis
and to minimize the bias in choosing studies for the meta-analysis. the eligibility
criteria for inclusion and eschsion of the studies should be defined. after the
studies with relevant information have been identified.
3. Abstract data. There are tmo levels of data abstraction in meta-analysis. First.
the studies that are eligible for the meta-analysis after identification need to be
abstracted from al1 the studies identified. Then. for al1 eligible studies. data
on relevant outcornes of the study and the characteristics of the studies are
abstracted.
4. hnalyze the data statistically. This step is the rnost important part of the
meta-analysis. In this step. statistical analysis is applied to thc cornbined data
in order to arrive at a summary estimate of the effect size. a rneasure of its
variance and a confidence interval, a suninian statistic whicli can be usecl for
tiypot hesis testing. and a test of the hypot hesis t hat the cffects are Iiornogenrous.
1.3 Two Approaches to Statistical Meta- Analysis
There are two major approaches to the statistical analysis of meta-analysis. One is
fixed effect models. nhich assume that al1 stiidies are governed by a common treatment
effect. and take the studies to be analyzed as the iiniverse of interest: the other. is
the random effect modeis. which allow different treat ment effects for different st udies.
and treat t hese studies as representing a sample from a larger population of possible
studies. The Mantel-Haenszel method (!dantel and Haenszel. 1939). the Peto met hods
(Yusuf et al.. 1983). and gneral variance-based rnethods (Wolf. 1986) are basecl on
fixed effect models. The methods described by Dersirnonian and Laird (1986) are
based on random effect models.
- - -
Esposed Sot Exposed Total
Total
Diseascd
Not Diseascd
--
Table 1.1: Arrangement of data and table notation for applicatioii o f SIantel-Haenszel
and Peto methods
1 b 1 a, -+ 6,
ct dl cl + dl
The Mantel-Haenszel method can be used wheri the measrire of cffcct is a ratio
effcct. especially an odds ratio. If the data from a stuciy are arranged as shown in
Table 1.1. the formula to estimate the sunirnary odds ratio using Slantel-Haenszel
methods is
where ORrnh is the Mantel-Haenszel summary odds ratio. OR, is the odds ratio of
study i and
a, x di OR, = -.
h, x ci
IL', is the weight n-hich is equal to 4. and of is the variance of odds ratio in study i
and
The Peto method is a modification of the kmtel-Haenszel method. Like Alantel-
Haenszel mettiod. it has been used frequently in meta-analysis of randoniized trials.
The formula to estinlate a sumrnary measure of effect is
In OR, = mol - Et) L 0:
where OR, is the sumrnary odds ratio using Peto niethod. 0, is the ol~servcd tiiimber
of events. Et is the espected ones in the treatnient group for study i ancl
(a , - bJ x (4 + c,) El =
t
of is the variance of the observed minus espectecl for the stiidy i and
The application of the Slantel-Haenszel method and the Peto met hod reqitire data
from each study to form a 2 x 2 table. If the data from a stiidy is not enougb to
complete a 2 x 2 table. the study must be escluded. Esclusion nieans the potential
to cause bias. This is the limitation of the two rnethods. Howver. the general
variance-based met hods can avoid t his limitation.
Let p, clenote the generic rneasure of the effect of study i. and let u , denote the
reciprocal of its variance. Then. p. the estimator under the assumption of fised effects
by applying the general variance-based met hods is
with the variance of as
The general variance-bascd methods dori't require data froni each stiicly to corn-
plett. a 2 x :! table. -4ctually. the Slantel-Haenszel niethod and the Peto niethod are
special cases of variance bascd methods.
DerSiniotiian and Laird (1986) proposed a statistical mettiod which nas bilsetl on
the random efkct rnodels. The Dersirnonian-Laird siirnrnary cstiniator is
where M.; is ttir DerSirnonian-Liiircl wcighting factor for the itti stiidy. and IL, is the
gerieric nir!asiirc of the effect of study i . 11.;. is calciilatcd as
where 7' denotes the variance. the effect size of the studies. or the measure of ranclorn
effect .
Calculation of 7' is cornplicated. First. we need to calculate Q. n-hich denotes the
statistic for rneasuring study-to-stiidy variation in effect size. and is giwn b -
nhere iLyi and are calculated from the geiieral variance-basecl rnethods. Then. the
nieasure of random effect. T"S
and
., [Q- (n -1 ) j rE w, -- - I - , , )Cr . if Q > TI - 1.
where n is the sanipie s ix .
Becaiisc x, a; < - x, ic, with equality only wtien T' = O. a rantlorn effect rrioclcl
irnplies more uncertairity than a fixed effcct motlcl. and the confidence interval for
the surnmary estimator using random effect rnoclel is widcr ttiari that c*alctilated from
a fised effect model. Generally. a randoni efFect rnorlel is riiorc suitable than a fiscd
effect model.
1.4 Quality of Studies in Meta-Analysis
Often. the design and execution of studies in a meta-analysis differ. This leads to the
the question to the reliability of the data from different studies and the variation in
quaiity of the studies. which can Vary widely. There are several possible approaches
to dealing with this quality problem:
1. Ignore the quality variation.
2. Only include studies meeting a specified standard of quality and ignore the
quality variation wit hin the included st udies.
3. Introduce a quality score to the model in a descriptive way. to see if the summary
st atistic is associated with the quality score.
4. Ipply a quality score to the probability model and adjust the oiitcome of nieta-
analysis to get a niore precise summary statistic ancl makc more accuratc infer-
ences.
In this thesis. I will adopt the foiirth approadi.
1.5 A Probability Mode1 to be Studied in This
Thesis
Tritchler (1999) proposed a fised effect mode1 with quality score. Let p be the pa-
rameter of interest. i.e.. the effect of study. Then his mode1 is
where n is the number of studies. For each study i. x, is the point estimate of IL
and assunieci to be normally distributed mith mean p and variance of. The quality
score Q, for the ith study is definetl to be the probability that the study has not
been influenced by uncontrolled factors. At's are independent of one another and also
independent of p.
The rnodel I will study in this thesis is a random effect model \vit h qualit- score.
where T' denotes the variance of the studies. or the rneasure of random effect, This
rnodel is extended from TritchIer's fiscd effect model. The definitions of p. 1,. O,. A,
and Q, are the same as in fised effect niodel. \lé will assume that a study is irrclcvant
to the parameter of interest if the uncontrolled lactors affect the stiicly in an unknown
fashion.
Q, is the probability that the i th stiidy is relevant (and cinbiased for the param-
eter of interest). and 1 - Q, is the probability that the ith study is i r re l~vmt . An
irreievant study bears no relationship to parameter of interest. Khen the ith stiidy
is relevant (with probability Q,). the rnean of r , is p. If the ith study is irrelc~ant
(ivith probability 1 - Q,). the ith study is assunied to corne from a unique normal
population with mean A,. where A, is independent of IL and al1 other A,. Therefore.
an irrele\.int study does not relate to the pararneter of interest. QI is assunied to
have the following characteristics:
1. The quality score of a study is a function of its design. and the assessnient of
quality is independent of the outcome of the study.
3. Without additional assumptions. a relevant study cannot be distinguished froni
an irrelevant study on the basis of outcorne. That is. ffawed studies do not
arise from a common population. nor need they arise from a population that
appears different from the population of unbiased studies. Al1 Ive specify is that
its clistribution is not tleterrnined by the parameter of intcrest. but hy 0 t h .
uncont rollect factors.
The randoni effect model with quality score will be stiitliecl in cornparison to three
other models. which arc special cases of the random effect model a i t h quality score
( 1.4):
1. Tlie fisccl cffect nioclel with qiiality score. which is Mode1 ( 1.3). or Ilocle1 ( 1.4)
with r2 = 0.
2. A random effect mode1 withoiit quality score. which is !dodrl ( 1.4) with Q, = 1
for i = 1. . .Y to ignore the quality variation. Tha t is:
3. h fised effect aithout quality score. ahich is Slodel ( 1.4) with r2 = O and
Q, = 1 for i = 1. . . to take away hoth the random effect and the differencc in
quality score. That is:
1.6 Contents and Organization of the Thesis
Csing the ~naxirnum likelihood method. 1 will estimate the parameter of interest 11
in the random effect mode1 with qiiality score. Csing the information matris of the
log likeliiiood at @. 1 will conipute the variance of f i . the estirriate of p. Csing large
saniple theory. 1 will conipiite the confidence intemal of 6. 1 will also ilse a bootstrap
to cstirniitc the variance of f i .
in Ctiapter 2 . I s i 1 1 t i r r iv~ the likelihood fiinctions and information matrices of the
nioclels. prrsrrit the simulation and bootstrap procecliircs. In Chapter 3. I will present
t lie coniputat ional resiilts froni t lie siniulation data. analyze t hem. iind compare t hr
p d o r n i a n w of the four moclels. In Chaptrr 4. 1 will apply the niodel to a r d data
set. t r i Chaptcr 3. 1 will priwnt concliiding reniarks.
Chapter 2
Methods and Techniques for
Analysis
Besides maximum likelihood and large sariiplc t heory. tn-O ot her major stat istical
techniques 1 use are Monte Carlo simulation ancl bootstrap. Csing Monte Carlo
simulation. 1 generate data according to the randorn effect mode1 with qiiality score.
Csing the bootstrap procedore. 1 compute variance. bias. coverage. and confidence
intervals of the estimator of the ranclom effect. I use S-Plus. a statistical software
package. to do the simulation and computation. In the rest of this chapter. 1 will
discuss the techniques in detail.
2.1 Derivation of Likelihood Functions
Recall the random effect mode1 with quality score ( 1.4). For each study i. there is
an associatecl quality score Q, that is the probability that the study is relevant. The
point estimate of study effect is r,.
- ( p . + ) if the ith stucly is relevant.
and
, Y ( . O). if the i th study is irrelevant.
~vherc p is the parmieter of interest. r 2 is the ranclom effect of the stiidy. <if is the
variaricc of obsenation. and A, is independent of 11 and other A, 'S.
Cnder the full niotlel ( 1.4). the likelihood function is
\\é get the profile likelihood for p by siibstituting the maximum estimate of A for
fised p. If Q, < 1. the maximum likelihood estiniate of A, esists and equals to r,. So
the profile likelihood is:
The log profile likelihood is:
Then the score functions under the full mode1 ( 1.4) can be obtained as
and ( x , - P F - 0
BlogL(p . r ' ) QI 2 ( T ~ + 2 2 ( & f ) Sr' (p. r 2 ) = 8 - 2 =? 0, +
1= 1 fi
And t hc inforniation mat ris:
- - - asp ( p . ?} a+
Since the other three niodels for coniparison are special cases of the full niodel
( 1 4. t heir likelihooci fiinctions. score functions and information matrices can be
obtainecl by supplying r' = O and/or QI = 1 for i = 1. - - . . .V to ( 1.4).
1. For the fised effect moclel with quality score. supply r2 = 0.
3. For the randorn effect model without qiiality score. supply Q, = 1 for i =
1.---.-Y.
3. For the fixed effect model without quality score. supply r' = O and Q, = I for
i = 1.. . S. Since the MLE computed from ( 1.6).
is the same as p computed from the general rarianccbased methoci disciissed
in 5 1.3. 1 ni11 use p of ( 1.1) and I -a r (p ) of ( 1.2) as f i and I ' u r ( b ) for the fised
effect mode1 nithout qualit? score in my simulation procedure.
2.2 Monte Carlo Simulation
Simulation is to imitate or reproduce certain conditions. The term simulation can be
used to cover a wide variety of activities ranging from the development of mat hemati-
cal relations describing a systcm. to the construction of a physical mode1 or niock-up.
A defiriit ion of siniulat ion vas given by Koskossidis and Brennan( 1984) as follows.
Sinidation is the techriiqiie of constructing and running a mode1 of real
systcni in order to stridy its behavior without disrupting the environment
of the rcal system.
A Alonte Carlo simulation is to generatc pseudo randoni variates according to a
probability nioclcl.
The Moritc Carlo simiilation in this thesis generatcs rancloni variates according to
the random t + k ~ t niodel with quality score. 1 choose parameters for mu sirniilntrd
data the sanie as Tritchl~r (1999). Described below is oiir Nonte Carlo sinidation
procedure.
1. Generate Q,. i = 1. . . . . 24 independently from a Cniform(mir2p. 1) distribution.
where Q, is the probability that the ith study is relevant. 1 -Q, is the probability
that the ith study is irrele~ant. and minp is chosen from 0.5. 0.7.3. 0.99.
2. For each i = 1. - . . .21. generate Ii from a Bernoulli(Q,) distribut ion such t hat :
Pr(I, = 1) = Qi and Pr(1, = O) = 1 - Qi:
3. Generate 0:. i = 1:-2-4 independently such that log(oT) has a Y(-'2.3.0.31)
distribution.
4. Generate study effect p from a Lniform(-2. 9) distribution.
5 . For i = 1.. . .A:
If 1, = 1 (relevant study). theri generate r , from X ( p . of + T-). where
r' is chosen from 0.10 x erp(-2.3). 0.25 x erp(-2.3). 0.50 x erp(-2 .3) .
0.75 x e.rp(4.3) . 1.00 x e s p ( 4 . 3 ) and 2.00 x exp(-2.3).
If 1, = O (irrelevant study). t hen generate A, from a Cniform(-2. 2 ) distri-
butioii and r, from .Y(,\,. of).
That is. tvc tvill have:
I f t e r each sample {(Q,. op. 1,): i = 1.2. . .24} is generated. ive ni11 compute /1.
the LILE of p. according to the random effect mode1 with qualit? score and the three
other models for comparison as well. The asyrnptotic likelihood-based variance ni11 be
used to const ruc t confidence intervals bnsed on the normal distribution assumpt ion.
1s
Comparing jl with the true p that is used to generate the random sample. 1 can
get the hias of b. From each simulation sample. 1 can get
bins, = ,iij - /i,.
90% CI, : Li, k 1.65 x 1-nr(jr,)
for each motlel. where j = 1. =.=*. .\i. .Vs is the total niimber of sarnples generated.
Finally. 1 can get hias. niean squared error (LISE). 95% and 90% coverage For each
moclel from al1 the sirnulat ion sarnples:
2.3 Bootstrap
2.3.1 Concept and Idea of Bootstrap
Bootstrap is a technique for making certain kinds of statistical inferences from a data
sample developed by Efron (1979). The term "bootstrap" derives h m the phrase to
19
pull onesel/ up by one 's bootstrap. Bootstrap is a computer-based method to obtain
statistical inferences and assign rneasures of accuracy to stat istical estimates. which
can solre many statistical questions wit hout formulas. It is a compiiter-intensive
technique.
The basic idea behind the bootstrap is as follows. Suppose that the vector
X = (xl.xz. -. x,} clenotes a set of data points X I . z2. . . . . .L, which are o h
served indepenclently frorn an unknown probability distribution F. ancl ive rvish to
estirnate a parameter of interest 0 = t ( F ) on the hasis of X. We calculate an estimate
é = s(X) from X. To estiinate how accurate the estirnator 6' is. the bootstrap nu
introduced in 1979 as a cornpiitcr-basecl methocl to rstimate the standard rrror o f O .
A bootstrap sarnplc X' = {xi. xi. - . . x*,} is generated from t ht. original data
points r l . x-. - . 1,. by randomly sampling rl times a i t h replacenient. That is. a
bootstrap data set {xi. r:. . . . . n,} consists of rnembers of the original data set
{q. x?. . - . r, }. where sonie members don3 appear. some appear once. somc appear
twice. etc.
A large number of inclependent bootstrap samples X;. X;. . .. Xb with sanlple
size n. are generated. The number of bootstrap saniples. B. typically ranges from
-50 to 200 for standard error estimation. For each bootstrap sample. a bootstrap
replication of s which is denoted by 9'(b) = .s(Xg). the value of the statistic s for
X;. is calculated correspondingly. The bootstrap estimate of standard error of s(X)
is given by:
4 x - 1 where s(.) = ~ f ! ! = , +.
Standard error is iiot the tmly measure of acciiracy Another measure of acciiracy
is bias. which is the difference between the espectütion of an estimator $ and the
quantity 0 being estimated. The bootstrap algorithm c m reach the estiniates of bias
as well as standard error. The bootstrap estimate O/ bias is defined as follows.
The bootstrap espectation E&(X)I can be approsiniated by avcraging the bootstrap
replications e 8 ( b ) = s(Xmb). Therefore. we can get the bootstrap estimate of bias
binss with &(a) substituted forE&(X)]:
biksbo,,t = I? (.) - t ( F ) .
.ifter ne get an estimate 0 and an estimated standard error i e . ne can get the
95% confidence interval by
1 +O. 95 - where t,--, denotes the y - perceritile of the Student's t distribution with n - 1
degrees of Freedom. and n is the sample size of the original data set. Then we can
get the 95% coverage. rvhich is the prohability that the 95% confidence interval can
cover the true parameter value.
hnother bootstrap confidence inten-als is based on percentiles of the bootstrap
distribution of O'. -1 1 - ?CI percentile interval is:
wliere &'(") is the 100 nth percentile of the bootstrap distribution. In the simula-
tion stiidy in Chapter 3. the bootstrap confidence intervals are Studentized. In the
application to a real data set in Chapter -4. both Studentizecl bootstrap confidence
intervals and percentile bootstrap confidence intervals are computed.
\\-hm the original data sets contain strata. where there are n, observations in the
i th stratiim. it will be niore appropriate to generate bootstrap samplcs by stratified
sampling in wliicli ri , observations arc taken with ecpal probability froni the i th
stratiim. Actiially. 1 atlapt stratified sampling to generate bootstrap saniples.
Sow tve can see that the bootstrap is actiially a data-based sirncilation method
for statistical infcrences. which can be used to produce variance. bias. confidence
intervals. The bootstrap estimates of the rneasures of statistical accuracy are non-
parametric. and its computation is feasible no mat ter how mat hernat ically compli-
cated the estirnator = s(X) may be. This is one of the most charrning characteristics
of bootstrap.
2.3.2 Bootstrap Procedure in This Thesis
The bootstrap procedure for estiniatiori of the standard error. bias and confidence
interval of the estimator of the study effect based on each of the four models has been
developed as follows.
1. Generate a bootstrap sarnple of size Zi = 24 by ranclomly sarnpling with replace-
nient from the original set of studies {(x,. O:. Q,): i = 1. - - - . .\-}. and deriote it
{(x;. O:'. Q:): i = 1 .2 . *. - . 'i). Similarly. a large number of indcpendcnt boot-
strap sarnples with size .V are generatecl . and tve denote t hem {(r;l. a:". Q;' ):
i = 1.2,. . . .V}. {(r;2,a'm',Q:2): i = 1.2. . O . . . -v}. . .-., {(r;*.o~*B. Q ; R ) : i =
1. 2. --. *. .Y}.
2. Foreach study i in theset ofsttidies {(x,.a~.Q,):i = l.2.-8-..V}.
i ) Generatc a random variable 1, h m a Bernoulli distribution witti siiccess
probability Q;. that is:
Pr([, = 1) = Q, and Pr([, = O) = 1 - Q,:
ii) Divide the original sample into twa subgroups by 1, . the sribgroup Good is
{(x,. op. Qi): i = 1.. .-. . .\-,} for Ii = 1 and the subgroup Bad is {(r,. a:. Q,): i =
1. *----. .V} for I, = 0. where .V, + -\ib = 3.
iii) Generate a bootstrap sample of size .V, by random sampling LI-ith replace-
ment from the subgroup Good and denote it {(x;. of'. Q;): i = 1. - - . . &}.
23
Generate a bootstrap sample of size -Vb by random sampling with replace-
ment from the subgroup Bad and denote it {(x;. O;*. Q:): i = 1. . - - . &}.
Slerge the two samples together and then 1 can get a new bootstrap sample
of size .V which is denoted { (2: . of*. Q;) : i = 1. - . . . .Y).
iv) Compute by mavimizing the log likelihoocl function of the mode1 I intro-
duced in this paper for the modifieti bootstrap saniple { ( 5 5 , . nmb,. Q; ): 1 =
1.2. -. . . *VI.
3. Compute the bootstrap estiniates of the measures of accuracy - standard error.
biczs. and confidence interval:
and
mhere F m ( . ) = $$.
Bootstrap estimates of the bias and standard errors of the ULE are computed
for each of the four models. Then we can obtain the bootstrap bias adjusted AILE
a . Csing the bootst rap variance. we can const ruct confidence intervals for bboat
based on the t-distribution with S - 1 = '23 degrees of freedoni. Comparing fiaoot
to the triie 11 that is useci to generate the random saniple. I can get the bias of
bsoot. Then. the &S. SISE. 95% and 90% coverages for each mode1 with bootstrap
adjustment can be obtained.
Chapter 3
Application of the Mode1 to
Simulation Data
3.1 Computational Results of Simulation Data
Csing the simulation procedure described in 8 2.2. I generate sers of 2-4 observations
of study effect according to the random effect mode1 ( 1.4) for 6 x 3 combinations
of r2 = tt x e x p ( 4 . 3 ) and ranges of Q,. where tt is 0.1. 0.25. 0.5. 0.75. 1 or 2. and
0.5 < Qt < 1. 0.75 < Q, < 1 or 0.99 < Q, < 1. 1000 simulation data sets are
generated for each combination of tt and range of Q,.
For each simulation data set. tve calculate ji. iiboat. and the bias. mean squared
errors (AISE). 95% and 90% confidence intervals of p and fihot of the four models:
random effect model with quality score.
fked effect mode1 with quality score.
random effect mode1 without quality score.
fised effect model wit hout quality score.
Al1 of the simiilations reported were based on 1000 random data sets. The bootstrap
calculations are based 011 200 bootstrap sarnples. The results are presented in the
folloning tables.
random effect mociel
with quality score
fised effect niodel
with quality score
randorn effect mode1
rvit hout quality score
fised effect model
n i t hout quali ty score
bias LISE 95% coverage 90% coveragc
ilht -0.00249 0.0059 0.936 0.900
Table 3.1: Simulation results for r2 = 0.1 x e r p ( - 2 . 3 ) and 0.5 < Q, < 1.
random effect mode1
n i t h quality score
fixed effect model
with quality score
random effect model
without quality score
fised effect mode1
n i t hout quality score
bias SISE 95% coverage 90% coverage
Table 3.2: Simulation resuits for Ï' = 0.25 x e s p ( - ? . 3 ) and 0.3 < Q, < 1
bias AISE 95% coverage 90% coverage
random effect mode1 1 fihot -0.00Xli 0.0165 O. 922 0.865
wit h quality score / a -0.00089 0.0142 O. 926 O. 892
fixed effect niodel
aitliout quality score 1 ji -0.001 17 O. 1 128 0.69 1 0.553
-0.00389 0.0202 0.920 O. 869
random cffect mode1
with qiiality score , jL -0.00221 0.0154 0.922 0.85 1 -
b o t -0.00137 0.1 127 O. 132 O. 39.5
Table 3.3: Simuiation resiilts for r' = 0.5 x e x p ( - 2 . 3 ) and 0.5 < Q, < 1
fixed effect mode1
without quality score
0.00114 0.1247 0.753 0.6'26
fi 0.00109 0.1'235 0.212 0 . 2 7
bias .\ISE 05% cowrage 90% coverage
randotn effect mode1 / jlb,,,( 0.00286 0.0190 0.916 0.871
ni thout cpality score -0.02200 0.1213 0.212 O. 23-4
Table 3.4: Simulation resiilts for r' = 0.75 x eip(-2.3) and 0.5 < Q, c 1
1 wit h quality score 1 fi 0.00022 0.0199 0.906 0.848
random effect mode1
with qiiaiity score 1 0.00-iX 0.0243 0.871 O. 782
bias .\ISE 9.5% cocerage 90% coverage
f i o o 0.00256 0.0238 0.910 0.8Z9
randorn effeci mode1 /iaooi -0.00778 0.1086 0.135 0.611
l wit hout cluality score 1 jr -0.00679 0.1088 0.70 1 0.354
ai thout quaiity score -0.00893 O. I 198 0.259 0.256
Table 3.5: Simulation restilts for r2 = 1.00 x erp(-2.3) and 0.5 < Q, < 1
random effect mode1
with quality score
fisetl effect mode1
with cluality score
biaç SISE 95% coverage 908 coverage
Cbool -0.004Z9 0.0260 0.923 0.866
random effect mode1
without quality score
fised effect mode1
nit hoiit quality score
Table 3.6: Simulation resiilts For T' = 2.00 x e x p ( 4 . 3 ) ancl 0.5 < Q, < 1
wit h quality score 1 f i 0.00075 O.OOS7 0.956 0.924
random effect niodel
I fixed effect mode1 1 uboot 0.00096 0.0061 O. 930 0.366
bias SISE 95% coverage 90% coverage
fiboal 0.00076 0.0061 O. 935 0.872
with quality score 1 f i 0.00010 0.00158 0.963 0.918
randoni effect rnodel 1 -0.00134 0.0337 0.906 0.804
without qunlity score 1 f i -0.00146 0.0353 0.884 0,792
without quality score 1 @ -0.00235 0.0407 0.499 0.450
with quiility score 1 ji -0.00160 0.0073 0.946 0.904
randorn effect mode1
\vit h qualit? score / -0.001 72 0.0076 0.945 0.898
bias SISE 95% coverage 90% coverage
-0.00198 0.0077 0.935 0.855
ralidorn effect rnodcl 1 /laoot 0.00104 0.0420 0.586 0.169
ivithoiit qiiality score i o.ooo96 0.0-136 0.491 0.424
nithout quality score
. .-
Tihle 3.8: Simulation results for T' = 0.25 x e x ~ ( - 2 . 3 ) and 0.73 < Q, < 1
/l 0.00115 0.0421 0.855 0.734
with quality score / f i -0.00322 0.0105 0.934 0.877
random effect mode1
with quality score / jl -0.00368 0.01 15 0.910 0.826
bias SISE 9-556 coverage 90% coverage
-0.00279 0.01 14 0.929 0.846
without quality score / ji -0.00468 0.0402 0.861 O. Z54
without quality score 1 ji -0.00208 0.0482 0.470 0.396
Table 3.9: Simulation results for T' = 0.X) x e r p ( - 2 . 3 ) and 0.75 < Q, c 1
- -- - --
wirh qiiality score 1 ji 0.00416 0.0112 0.9'23 0.873
random effect mode1
l fised effect inode1 fiboot 0.00477 0.0162 0.927 0.86 1
bias LISE 95% coverage 90% coverage
f i 0.00462 0.0112 0.9'26 0.868
a i t h qiiality score / ,Li 0.00464 0.0135 0.88 1 0.806
withour qiiality score ( i' 0.00184 0.0411 0.563 0.766
Table 3.10: Simulation results for r' = 0.75 x e z p ( 4 . 3 ) and 0.73 < Q, < 1
bias XISE 95% coverage 90% coverage
fixed efkct mode1 1 f i b -0.00702 0.020S 0.9'28 0.876
wit h qualit? score fi -0.00114 0.0127 0.923 0.818
random effect mode1 fibaot -0.00486 0.0390 0.895 0.805
t
wit h quaiity score
- --
fi -0,007'20 0.0159 0.879 0.804
fised effect mode1 1 b o t -0.00137 0.0455 0.905 0.8 18
aithout quality score
witliaui qiiality score 1 -0.00101 0.0447 0 3 10 0.443
-0.00424 0.0388 0.S7.3 0.782
Table 3.11: Siniulation restilts for r' = 1.00 x erp( -2 .3) and 0.75 < Q, < 1
random effect model
with quality score
Lwd effect model
with quality score
randoni effect model
wi t hou t cliiali tu score
fised effect mode1
wit hout quali tu score
bias USE 95% coverage 90% coverage
bbool -0.00439 0.0260 0.923 0.566
Table 3.12: Simulation results for r' = 2 x e x p ( 4 . 3 ) and 0.72 c Q, < 1
hias SISE 9.5% coverage 90% coverage
random effect mode1 1 jib0,, 0.00107 0.00427 0.946 O. 896
1
raridoni effect moclel 1 /iboot 0.00032 0.00478 0.946 0.391 I
witlioiit quality score 1 fi 0.00036 0.00473 0.960 0.908
v i t h o t l i t s r 0.00086 0.00480 O. 9% 0.8'74
Table 3.13: Simulation resiilts for r' = 0.10 x e x p ( - 2 . 3 ) and 0.99 < Q, < 1
with qiiality score 1 ji -0.00157' 0.00504 0.9'23 0.822
random effect mode1
ai thout qual i ty score / i, -0.00232 0.00546 0.946 0.88:3
bias SISE 95% coverage 90% coverage
fibot -0.00184 0.00310 0.94'1 0.898
withoiir qiiiiliry score / fi -0.00210 0.00343 0.593 0.82.5
fisecl effect mode1
Table 3.14: Simiilatiori resiilts for T' = 0.2; x e r p ( - 2 . 3 ) and 0.99 < Q, < 1
/iboot -0.00301 0.00545 0.953 0.902
bias SISE 95% corerage 90% coverage
with quality score 1 f i -0.00073 0.00657 0.924 0.880 4
wittmut qiiiility score 1 fi 0.00129 0.00688 0.9'21 0.877
fixed effect mode1
with quality score
bb,,, -0.00124 0.00749 0.924 O. 860
ji -0.00149 0.001'24 0.859 0.790
Table 3.1.3: Simulation results for r' = 0.50 x exp( -2 .3 ) and 0.99 < Q, < 1
mit hout quality score
--
jl 0.00 163 0.001'2'2 0.84 1 0.737
randorn efFect mode1 1 fiboot -0.00392 0.00ii19 0.938 0.893
with quality score / f i -0.00381 0.00736 0.9'26 0.875
with qualiry score / f i -0.00166 0.00842 0.830 0.111
n-ithoui quality score / f i -0.00297 0.00513 O ,822 0.732
random effect mode1
Table 3.16: Simulation results for r' = 0.75 x e r p ( 4 . 3 ) and 0.99 < Q, < 1
-0.00351 0.00780 0.935 0.894
bias SISE 95% coverage 90% coverage
with quality score -0.00334 0.OiOZ 0.82.5 0.744
random effect mode1
with quality score
randorn rffect mode1 1 /Ibt -0.00269 0.00910 0.945 OS88
f i o t -0.00331 0.00893 0.938 0.886
-0.00332 0.00813 0.9'21 0.575
wirhoiir qiiality score 1 fi -0.00236 11.00900 0.928 0.883
Table 3-17: Siniulation results for Ï' = 1.00 x e r p ( 4 . 3 ) arid 0.90 < Q, < 1
fisccl effect mode1
wit hout qiiality score
bbOot -0.0028'1 0.010'24 0.92:3 0.879
-0.0023'1 0.00995 0.780 0.687
bias SISE 95% coverage 90% coverage
random effect mode1 / bboal -0.00941 0.0121 0.953 0.909
wir h qiiality score 1 -0.00975 0.0 119 0.940 0.890
wir h qualit- score 1 fi 0 . 0 11 19 0.0 176 0.747 0.673
nithout quality score / b -0.00762 0.0136 0.687 0.623
without cluality score
Table 3.15: Simulation results for r' = 2-00 x r x p ( - 2 . 3 ) and 0.99 < Q, < 1
-0.01020 0.01 19 0.947 0.891
3.2 Analysis of the Computational Results for Com-
parison of the Models
Recall that 1 liave generated simulation data sets according to random effect niodels
with quality scores. Csing the model. as well as the three other niodels. with and
aithout bootstrap adjustrnent. 1 have estimated the study effect of the simulateci
random effect mociels a i th quality scores. and computed the bias. SISE. 95% and
90% coveriges of j~ and i lbaol. The computational results have been presented in the
tables of the last section. In this section. 1 compare thesr coniputational results with
respect to r' bctween the random effect models and the fiseci cffect niodels. betiveen
t hc modcls wit h and wit hout qualit? scores. betweeii the moclels iising bootstrap
acijristmmt and not. ancl also among the three ranges of quality scores.
Figures 3.1. 3.2. 3.3. 3.4 and 3 3 present plots of the bias. absolute d u e s
of hias. USE. 95% and 90% coverages of the estimated study effect with respect to
r2 of the four moclels with and without bootstrap adjiistrnent for the three ranges
of quality scores. ahere 'rc1.b' denotes the random efFect mociel witli quality score
and adjusted by bootstrap. 'rq' denotes the random effect moclel with quality score.
*fq.bg denotes the fised effect niodel wit h quality score and adjusted by bootstrap. fq '
denotes the fised effect model ivith quality score. 'rnq.bS denotes the randorn effect
rnodel a i t hout quality score and adjusted by bootstrap. m q ' denotes the random
effect model without quality score. .fnq.b' denotes the fixed effect model without
quality score and adjusted by bootstrap. and 'fnq' denotes the fised effect rnodel
without quality score. Figures 3.6. 3.7. 3.8. 3.9 and 3.10 present plots of the bias.
absolute value of bias. SISE. 95% and 90% coverages of the estirnated study effect
ni th respect to T'. for the three ranges of quality scores (0.5 5 Q, 5 1. 0.75 5 Q, 5 1
and 0.99 5 Q, 5 1). of the four models with and without bootstrap adjtistnient.
From the figures. we can rnake the following observations.
Froni Figures 3.1. 3.2. 3.6. and 3.7. WC c m see that the absolute values of bias
froni the random cffect model with qtiality score arp roiighly the smallest aniong
al1 the rnodels. The random effect rnoclel a i t h quülity score performs the hest
on bias wfien low quality score esists (for 0.5 5 Q, 5 1) . The bias from id1
these rnodels are pretty close when al1 the quality score is rery close tu 1 (for
0.99 5 QI 5 1 ) l l though the absolute values of hias from the randoni effect
model a i t h qtiality score are higher t han t hesc froni t hc rnociels [vit hout cpali ty
score for 0.75 5 Q, 5 1. the M E S from the random effect models are the
smallest comparing with the other models (from Figure 3.3).
2. From Figures 3.3 and 3.5. al1 the >ISE values turn larger and larger almost
linearly as r' goes from 0.1 t e x p ( 4 . 3 ) to 2 + e x p ( 4 . 3 ) escept tliese from
the fised effect models with and without quality score for 0.5 < Qi 5 1. The
SISE values from the randorn effect rnodel with quality score are smaller than
46
t hese from the other models. This indicates t hat the random effect mode1 wit h
quality score produces more precise results than the fised effect models with
quality score.
From Figures 3.4 and 3.9. the 95% coverage from the random effect moclel with
quality score is basically above 0.9 for ail values of r2. the 95% covcrage from
the fised effect model with quality score is good when r' is sniall. but cirops
rapiclly as T' increases. The 95% coverage from the raridom effect mode1 wit hoiit
quality score is good for high qtiality scores. but not for low qiiality scores. The
95% covcragc from the fixccl effect model withoiit quality score rouglily is the
lowest and drops most rapiclly. Therefore. the estimate of stiitly effcct from the
randoni effect model with quality score provides bctter coverage of p tlim the
others when the ranclom efkct is large or when the quality score is low. For
fised effect models wit h and n i t hout quality score. random effect mode1 without
qiiality score. bootstrap acijustment provide improves the coverages.
4. The 1 s t observation is also obvious from Figures 3.3 and 3.10. The fixed effect
models ni th quality score don't work well if the random effect is large. The
random effect model without quality score doesn't work weII when the quality
score is low. The fised effect mode! works the worst under al1 conclitions. The
random effect model n i th quality score appears the most proper one no niatter
how the random effect or the quality score is.
From Figure 3.5. the M E decreases as the quality score becomes higher. This
means that we ail1 get niore precise results when the quality score is high.
From Figure 3.9 and 3.10. the results froiri stiidies with higher quality providc
bet ter coverages in nieta-analysis.
Based on these observations. we can sec that the random effect mode1 with cluality
score ( 1.4) is about the hest among a11 the four niodels.
For 0.50 < Q < 1 For 0.75 < O c 1 For 0.99 c Q =z 1
0 0 0 5 1 0 1 s 2 0
tau
Figure 3.1: Plots of the bias of the estimated study effect with respect to Ï' of the
four models with and ivithout bootstrap adjustment for the tliree ranges of quality
scores.
For 0.50 < Q c 1 For 0.75 c Q c 1 For 0.99 c Q < 1
0 0 0 5 1 0 1 5 2 0
tau
0 0 0 5 1 0 1 5 2 0
tau
0 0 0 5 10 1 5 2 0
tau
Figure 3.2: Plots of the absolute values of bias of the estimated study effect with
respect to r2 of the four models with and without bootstrap adjustment for the three
ranges of qiiality scores.
For 0.50 < Q < 1 For 0.75 c Q < 1 For 0.99 -z Q < 1
- m D rq - - - fq b - - fq -- rnq - - * mq --- fnq D
'nq
0 0 0 5 1 0 1 5 2 0
tau
0 0 0 5 1 0 1 5 2 0
tau
Figure 3.3: Plots of SISE of the estiniated study effect with respect to r' of the four
models with and without bootstrap adjustment for the three ranges of quality scores.
For 0.50 c Q c 1 For 0.75 -Z O c 1 For 0.99 e Q -Z 1
O 0 0 5 1 0 1 5 2 0
tau
0 0 0 5 1 0 1 5 2 0
tau
0 0 o s 1 0 15 2 0
tau
Figure 3.4: Plots of the 95% coverage of the estirnated stiidy effect with respect to
r2 of the four models with and without bootstrap adjustment for the three ranges of
quality scores.
For 0.50 c Q < 1 For 0.75 -z Q < 1 For 0.99 c Q < 1
Figure 3.5: Plots of the 90% coverage of the estimated study effect with respect to
r' of the four models n-ith and aithout bootstrap adjustment for the three ranges of
quality scores.
'fq'
Uu
'rnq. b'
U Y
'rnq'
Figure 3.6: Plots of the bias of the estimated study effect s i t h respect to 7'. for
the three ranges of quality scores. of the four models with and without bootstrap
adjustment.
'mq. b'
F i e 3 : Plots of the absolute values of bias of the estimated study effect with
respect to r2. for the three ranges of quality scores. of the four models n i th and
n i t hout bootstrap adjiist ment.
Uu
'mq .e'
f*"
'rnq' 'fnq'
Figure 3.5: Plots of SISE of the estimated study effect wit h respect to r'. For t lie t hree
ranges of quality scores. of the Four models with and without bootstrap adjustment.
'fq'
Leu
'rnq . b'
Le"
'mq' 'fnq'
Figure 3.9: Plots of the 95% CI coverages of the estimated study effect with respect
to r2. for the three ranges of quality scores. of the four models with and without
bootstrap adjustment .
'fq. b'
'rnq . b'
m.,
'mq' 'fnq'
Figure 3.10: Plots of the 90% CI coverages of the estimated study effect with respect
to r2. for the three ranges of qualit? scores. of the four rnodels n i th and without
bootstrap adjustment .
Chapter 4
Application of the Mode1 to a Real
Data Set
4.1 Background of the Data Set
Breast cancer is one of the most commori cause of death from cancer in worrieri in
most of the Kestern world. Especially. it is the ieading cause of death from ail causes
for the tromen aged less than JO (Boring et al.. 1993). Breast cancer incidence varies
nidely arnong coiintries. The international differences in the frequency of breast
cancer are not only due to genetic differences between populations but also due to
some difference in the environment. Difference in diet could be one of the responsible
environment factors. Cohon and case control studies 11-hich esamined the relationship
between dietary fat and breast cancer risk have given inconsisteut results. To address
this inconsistency meta-analysis is needed by to develop a quantitative sumrnary of
the elcisting literature.
The real data set we use in this paper is h m 24 stiidies of dietary fat and breast
cancer risk. which was assembled by Boyd et al. (1993). A total of 24 cstimates
for total fat intake were obtained from 23 independent studies included in the meta-
analusis. Each study aas assigned a quality score based on some predetermined
met hotlological standards. Table 4.1 and Table 4.2 s h o w the selecteci ciiaractwistics
of the real data set ive lise in meta-analysis iri this paper.
Relative Risk(RR) r , = Iog(RR,) a: Quality Score(Q,)
Table 4.1: Part i of the real data set.
Study ( i ) Relative Risk(RR) xi = log(RRi) o: Quality Score(Qi)
- ..
Table 4.2: Part II of the real data set.
4.2 Application of the Mode1 to the Data Set
Table -4.3 shon-s the results from our meta-analysis. For Boyd's meta-analysis in 1993.
they used a random effect mode1 to calculate a summary relative risk for dietary fat.
and it was 1.12 (93% CI 1.04 - 1.21). Comparing that 114th what 1 get by iising the
random effect model in this thesis. the bboot ( the SILE adjusted by bootstrap) is 1.18
with Studentized 9576 CI (0.98. 1.41) and 90% CI (1.01. 1.37). and percentile 95% CI
(0.96. 1.38) and 90% CI (0.98. 1.33). and the (the AILE of the model) is 1.18 with
93% C I (1.01. 1.38) and 90% CI (1.03. 1.34). 'rly estimates of the sunimary relative
risk for dietary fat are higher than Boyd's. and my estimates of the confidence intervals
are wider than B0yd.s. The wider confidence intervals indicate that the randoni effect
coniputed from my random effect model is p a t e r than Boyci's. since it models mort.
uncertainty and inconsistency. If ive do a hypothesis to test if dietary fat intake
increases breast cancpr ri& with Ho : RR = 1. w~ will rojcct the Ho arvorciing to
the Boyd's results at 0.0.5 levcl. \\é d l reject the Ho at 0.05 level too. according to
the confidence intervals of b. However. ive ivill fail to reject the Ho at 0.05 level and
reject the Ho at 0.1 levcl according to the Studentizecl bootstrap confidence intemals
and n e nill fail to reject the Ho at 0.1 level according to the percentile bootstrap
confidence intervals. If WC just look at the relative risk. ive will concludc that the
dietary fat intake will affect breast cancer risk. which is the same conclusion which
Boyd got.
Relative Risk 95% CI 90% CI --
1.146 Studentized (0.98. 1.41) (1.01. 1.37)
Percentile (0.96. 1.38) (0.98. 1.33)
115th quality score 1.177 (1.01. L.38) (1.03. 1.34)
with quality score 1.088 (0.94. 1.26) (0.96. 1.23)
fised effect mode1 fiboot 1.033 Studentized (0.80. 1.34) (0.84. 1.29)
Percentile (0.95. L.41) (0.96. 1.35)
. - - - - - -
wit hout quslity score 1.169 (1.04. 1.32) (1.06. 1.29)
randoni cffect mode1 bboot 1.17.5 Studentized (1.01. 1.33) (1.04. 1.31)
Percentile (1.02. 1.33) (1.05. 1.30)
aithout quality score / ji 1.151 (1.06. 1.25) (1.08. 1-23)
fised effect rnodel
Table 4.3: Slocle!ing results of the real data
[ L ~ ~ ~ ~ 1.149 Studentized (0.95. 1.33) (1.01. 1.30)
Percentile (1.01. 1.33) (1.03. 1.31)
Chapter 5
Lonclusion
Since niost of the stuclies in meta-analysis are done on diffcrcnt stiidy populations
using clifferrnt niethocls. a rantloni effect modd is appropriate. Due to tlic ïariation
of the met liodological quali ty of the st iidies. cluality score 1 baseci on pr~determincd
methodological standard ni- help to get more accurate results. Theoretically. a
ranclom effect model witli qualit- score should be the niost appropriate approacli to
met a-analysis.
In Chapter '2. 1 have derived the likelihood functions and information matrices
of the rnodels. developed the simulation and bootstrap proceclures for stiiciy. In
Chapter 3 and Chapter 4. I have conipared the random effect model with quality
score n i t h three other related models using simiihtion data sets and a real data set.
From the results and analysis. we can see that Lxed effect models ivith and n-ithout
quality score d o i t nork well when the random effect is large. and the random effect
mode1 without qiiûlity score doesn't work well when i o n quality score exists. However.
the random effect mode1 with cpality score works pretty weli even when the random
effect is large and low quality score exists. The mode1 is pas>. to iniplenient in S-Plus
too. Therefore. the random effect mode1 with quality score is a gootl approach to
met a-analysis.
Sonie possible disaclvantages wheri we apply the randoni cffcfcct nioclel \vit li quality
score to rcal data scts are as follo~vs:
1. It. is more coniplicatetl than the ot hcr thrce moclels nictiorird in this thesis. \\é
have to use stat istical software to est imate the stiicly rffwt.
2 it t a k ~ s 11s some tinw to assign a clunlity score to eacli stiirly iri tripta-iirialysis.
3. I l qiiality scores aren't assignecl prop~rly. ae ni- get inacciiratr resiilts.
Appendix A
S-plus Functions
A.l Object Function of the Models
A.1.1 Random Effect Model with Quality Score
metafrn. obj-function(parms, x, ss, q) <
u-p-s Cl1 tt - p m s C21
.exprl C- x - u
.expr2 <- .exprla2
.expr4 C- ss + tt
. expr7 <- exp ( (( ( - . expr2) / . expr4) / 2 ) )
.expr8 c- q * .expr7
.expr9 <- .expr4-0.5
.exprl4 <- (.expr8/.expr9) + ((1 - q)/(ssa0.5))
A.1.2 Fixed Effect Model with Quality Score
metafq. obj-function(u, x , ss, q) C
A.1.3 Random Effect Model without Quality Score
s u d . value)
A.2 Information Mat rix (Function) for Models
A.2.1 Random Effect Model with Quality Score
metafrn.hes-function(u, tt, meta.x, rneta.~, meta.p) <
A.2.2 Fixed Effect Model with Quality Score
metafq. hes-function(u, x , ss , q)
A.2.3 Random Effect Model without Quality Score
metarnq.hes,function(u, tt, x , ss)
A.3 Bootstrap Procedure for Estimation of Biases and Standard Errors
metafour .fun, function(x, s, p, nb, ttO)
boot-numerichb) boot . r,numeric(nb) boot.f,numeric(nb) boot.nqr-numericbb) boot.nqf-numeric(nb1 tt . boot-numeric (nb) n-length(x) confz95-qt (0.975, n - 2 ) confz90-qt (0.95, n - 2) mu.cf-sum((x * p)/s)/sum(p/s)
max1ik.r-nlminb(c(mu. cf, ttO) , metafrn. obj , scale=2, lower=c (-Inf ,O. 001) , upper=c (Inf , Inf ) , X= X , S S = S, q = p)
mutt.r,maxlik.r$parameters mu. r-mutt . r Cl] tt . r-mutt . r CS1 hessian-metafrn.hes(mu.r, tt-r, x , s, p) mu. var-solve (hessian) senm.r,abs(rnu.var[1,1])'0.5
max1ik.f-nlminb(c(mu.cf), metafq.obj, x = x , ss = s, q = p) mu.f-rnaxlik.f$parameters mu.f-mutt . f [il tt .f,mutt . f [2j
hessian-metafq.hes(mu.f, x , s, p) mu. var,l/abs (hessian) senm.f,mu.var"O.S
# nq is "without quality score"
maxlik.nqr-nlminb(c(mu.cf, ttO), metarnq.obj, scale=2, lower=c(-Inf ,O. 001) , upper=c(Inf , Inf) , x = x , ss = s)
mutt.nqr-maxlik.nqr$parameters mu. nqr-mutt . nqr Cl] tt . nqr-mutt . nqr C21 hessian-metarnq.hes(mu.nqr, tt-nqr, x, s) mu. var-solve (hessian) senm.nqr-abs(mu.var[l, l] )"O -5
Ibnmg5 .nqr- mu .nqr - qnorm(0.975)*senm.nqr ubm95.nqr- mu.nqr + qnom(0.975)*senm.nqr lbnm90. nqr- mu. nqr - qnorm(0.95) *senm. nqr ubnm90.nqr- mu.nqr + qnorm(0.95)*sem.nqr
ib-rbinom(n, 1, p) xgood-x [ib==l] xbad-x [ib==O] sgood-s [ib==l] sbad-s [ib==O] qgood-p [ib==l] qbad-p [ib==O]
igood-sample (1 : length(xgood) , length(xgood) , replace = T) ibad-sample(l:length(xbad), length(xbad), replace = T)
xb-c ( xgood [igood] , xbad [ibad] ) sb-c ( sgood [igood] , sbad [ibad] ) qb-c ( qgood [igood] , qbad [ibad] )
#print(list(x=x,s=s,q=q, xb=xb, sb=sb, qb=qb, igood=igood))
mu. cf -sum(xb*qb/sb) /sum(qb/sb) maxbo0t.r-nlminb(c(mu.cf, ttO), metafrn.obj,
scale=2, lower=c(-Inf,0.001), upper=c(Inf,Inf), x = xb, ss = sb, q = qb)
muboot-r-maxboot.r$parameters boot . r [il ,muboot . r [Il
maxboot.f~nlminb~c(mu.cf), metafq.obj, x = xb, ss = sb, q = qb)
boot . f Ci] -maxboot. f $parameters
iboot-sample(1 :n, n, replace-T) xb-x [iboot] sb-s [iboot]
boot .nqf Cil -sum(xb/sb) /sum(l/sb)
mu. cf -boat . nqf [il
x = xb, ss = sb) muboot.nqr~maxboot.nqr$parameters boot . nqr Ci] -muboot. nqr Ci]
bootbias.nqr-mean(boot.nqr) - mu.nqr bootse .nqr-(var(boot .nqr) )*O .5 bootmu.nqr,mu.nqr - bootbias.nqr lb95.nqr-bootmu.nqr - confz95 * bootse.nqr ub95.nqr,bootmu.nqr + confz95 * bootse.nqr lb90.nqr-bootmu.nqr - confz9O * bootse.nqr ub9O.nqr-bootmu.nqr + confz90 * bootse-nqr
bootbias-nqf-mean(boot.nqf) - mu.nqf bootse . nqf _ (var (boot . nqf) ) -0.5 bootmu.nqf,mu.nqf - bootbiasaqf Ib95.nqf-bootmu-nqf - confz95 * bootseaqf ub95.nqf-bootmu.nqf + confz95 * bootse-nqf lb90.nqf-bootmu.nqf - confz9O * bootse.nqf ub9O.nqf-bootmu.nqf + confz9O * bootse-nqf
Main Simulation Funct ion simufour~function(ns=100,nb=200, tau=0.1, minp=0.5) < #
# Simulation of the random effect mode1 # Hongmei Cao, July 4, 2000 #
bootmu.r-1:ns mu. r-1: ns
bootmu.nqr,l:ns mu. nqr, 1 : ns lb95.nqr-1:ns ub95.nqr-1:ns lbnm95.nqr-l:ns ubm95. nqr- 1 : ns lb90.nqr-i:ns ub90.nqr-1:ns lbnm90.nqr,l:ns ubnm90.nqr-1:ns
tt-tau*exp(-2.3)
#print( l i s t ( ns = n s , nb=nb, tau= tau, minp-ninp))
for (j in 1:ns) print(list(j=j))
meta.out-metafour.fun(x, s , p , nb, tt)
lbm95. f [j] -meta.out$lbnm95. f ubnm95. f [j] -meta. out$ubnm95. f lb9O .f [j] -meta. out81b90. f ub90.f [jl-meta.out$ub90.f lbnm9O. f [j] -meta. out$lbnm90. f ubnm90.f [j]-meta.out$ubnm90.f
bootmu .nqr[j] -meta. out$bootmu .nqr mu.nqr[j]~meta.out$mu.nqr lb95 .nqr [j] _meta.out$lb95 .nqr ub95. nqr [ j] -meta. out$ub95. nqr lbnm95 .nqr [j] -meta. out$lbm95. nqr ubnm95. nqr [ j 1 -meta. out8ubnm95. nqr lb9O .nqr[j] -meta.out$lb90 .nqr ub9O. nqr [j] -meta. out$ub90. nqr lbnm9O .nqr [j] -meta. outFlbnm90 .nqr ubnrn90.nqrCjI-meta.out$ubnm90.nqr
bootmu.nqf[j]~meta.out$bootmu.nqf mu.nqf [jl-meta.out$mu.nqf lb95. nqf [ j] -meta. out0lb95. nqf ub95. nqf [ j] -meta. out$ub95. nqf lbnm95 .nqf [j] -meta. out$lbnm95. nqf ubnm95.nqf[j]-meta.out$ubnm95.nqf lb90 .nqf [j] _meta.out$lb90 .nqf ub9O .nqf [j] -meta.out$ub90 .nqf lbnm9O.nqf[j]-meta.out$lbnm9O.nqf ubnm90. nqf [ j] -meta. out$ubnm90. nqf
c95b.r-sum(ugenc=ub95.r % ugen>=lb95.r)/ns c95m.r-sum(ugen~=ubm95.r & ugen>=lbnn95.r)/ns c95b.nqr-sum(ugen<=ub95.nqr & ugen>=lb95.nqr)/ns c95nm. nqr-sum (ugen<=ubnm95. nqr & ugen>=lbnm95. nqr) /ns c95b.f-sum(ugen<=ub95.f & ugen>=lb95.f)/ns c95m.f-sum(ugen<=ubnm95.f & ugen>=lbnm95.f)/ns c95b.nqf-sum(ugen<=ub95.nqf t ugen>=lb95.nqf)/ns c95m.nqf-sum(ugen<=ubnm95.nqf & ugen>=lbnm95.nqf)/ns
c90b. r-sum (ugenc=ub90. r & ugen>=lbgO. r) /ns c9Onm. r-sum(ugenc=ubnm90. r & ugen>=lbnm90. r) /ns c90b. nqr-sum(ugen<=ub90. nqr & ugen>=lb90. nqr) /ns c9Onm.nqr-sum(ugen<=ubnm90.nqr & ugen>=lbnm90.nqr)/ns c90b.f-sum(ugen<=ub90.f & ugen>=lbgO.f)/ns c9Onm. f -sum(ugen<=ubnm90. f & ugen>=lbnm90. f ) /ns c90b.nqf-swn(ugen<=ub9O.nqf & ugen>=lbgO.nqf)/ns c90nm.nqf-sum(ugen~=ubnm9O.nqf & ugen>=lbnm90.nqf)/ns
b0otbias.r-mean(bootmu.r - ugen) bias.r-rnean(mu.r - ugen) bo0tmse.r-mean( (bo0tmu.ï - ugen)-2) mse .r-mean( (mu. r - ugen) -2 1
bootbias.f~mean(bootmu.f - ugen) bias . f -meadmu. f - ugen) bootmse .f -mean( (bo0tmu.f - ugen) -2) mse . f -mean( (mu. f - ugen) -2
bootbias . nqr-meadbootmu . nqr - ugen) bias .nqr-mean(mu.nqr - ugen) bootmse . nqr-mean( (bootmu. nqr - ugen) -2) mse. nqr-mead (mu.nqr - ugen)-2 )
bootbias.nqf~rnean(bootmu.nqf - ugen) bias . nqf -mean (mu. nqf - ugen) bootmse . nqf -mean( (bootmu. nqf - ugen) ̂2) mse.nqf-mean( (mu-nqf - ugenl-2
effect-c("rq.bM, "rq.nmI1, "fq.b","fq.nm", "nqr.bU, "nqr.nmU, "nqf.b","nqf.nm") bias-c(bootbias.r, bias.r, bootbias.f, bias.f, bootbias.nqr, bias-nqr,
bootbias . nqf , bias . nqf) mse,c(bootmse.r, mse-r, bootmse.f, mse.f, bootmse.nqr, mse.nqr,
bootmse . nqf , mse . nqf ) c95,c(c95b.r, c95nm.r, c95b.f, c95nm.f, c95b.nqr, c95nm.nqr,
c95b.nqfJ c95nm.nqf) c90,c(c90b.r, c90nm.r, c90b.f, ~90nm.f~ c90b.nqr, c90nm.nqr,
c90b. nqf , c9Onm. nqf)
input-data.frame(ns=ns, nb=nb, tau-tau, minp=minp)
result-data.frame(effect=effect, bias=bias, mse=mse, c95=c95, c90=c90)
return(input , result) 3
Bibliography
[1] Agresti. A. (1996). .An Introduction to Categorieal Data .4nalgs2s. Kiley-
Intcrscience Publication. Sen. York.
[2] Boring. C C . Squircs. T.S.. Tong. T. (1993). ..Cancer Statistics". CA: -4 Cancer
Journal for Clinic.ians. 43. 7-26.
[3] Boycl. S.F.. Martin. L.J.. Soffel. S.. Lockwood. C.A.. Tritctiler. D.L. (1993). .A
meta-analysis of studies of dietary fat ancl hreast cancer risk-. Bntish .Journal
of Cancer. 68. 627-636.
[A] Bulpitt. C..J. (1985). *''\leta-analysis". The Lancet. ii. 93-94.
[5] Davison. A.C.. Hinkley. DS'. (19%). Bootstrap Methods and Their ilpplicution.
Combndge University Press. United Iiingdom.
[6] Dersirnonian. R.. Laird. S. (1986). .*Sleta-analysis in Clinical Trials". Controlled
Clinical Trials. 7. 177- 188.
(71 Efron. B. (1979). "Bootstrap Methods: Another Look at the Jacknife". The
.&mals of Statistics. 7. 1-26.
[SI Efron. B.. Tibshirani. R.J. (1993). ,472 Introducta'on to the Bootstrap. Chapman
and Hall. London.
[9] Fleiss. .J.L. ( 1993). +*The S tatistical Basis of Ueta-analysis" . Statisticul Methods
in Medical Research. 2. 1'2 1-145.
[IO] G l a s G. Lw. ( 1976). .*Priniary. Secondary and Meta-analusis of Researcli" . Edu-
cationul Reseurch. 5 , 3-8.
[11] Glass. G. K. IlcGaw. B.. Smith. M. L. (1981). Metu-malys is in Socid Research.
Sage Publications. Beverly Hills. Calif.
[12] Hedges. L. Y. (1982). .*Estimation of Effect Sizc from a Scries of Independent
Esperiments" . Psychological Bulletin. 92. -490-499.
1131 Hedges. L. Y. (1983). fombining Independent Estimators in Research Synthe-
sis". British Journal of Mathematical 8 Statistical Psychology. 36. 123-131.
[l4] Hunter. J . E.. Schmidt. F. L.. .Jackson. B. G. (1982). Meta-analyszs: Cumulating
Research Findings Across Studzes. Sage Publications. Beverly Hills. Calif.
[KI Koskossidis, DA.. Brennan, C. J. (1984). "A Revietv of Simulation Techniques
and 1Iodeling". Proceedings of the 1984 Summer Computer Simulation ConIer-
ence (W.D. CVade.ed., Boston). Vo1.1 . 55-58.
[16] L'Abbe. 1i.A.. Detsky. A. S.. O'Rourke. 11;. (1987). .-Slcta-analusis in Clinical
Research". h n a l s of Internai Medicine. 107. 224-233.
[l T ] Lehman. S . ( 1 9 ) Computer Simulation aiid .Llodeling:.4rr Iritroductiori.
Lawrence Eribaum .\ssociates. Inc.. Publishers. Hillsdale. Sew .Jersey.
[l S] Light . R. .J. ( 1983). Evaluation Studies. Reuie#w .-l nnual. vol 8. Sage Pti hlicat ions.
Beveriy Hills. Calif.
[19] Light. R. .J.. Pillenier. D. B. (1984). strrnrning Up: The S c i e r m O/ Reuiewing
Research. Han-ard Cniversity Press. Cambridge. SIas.
[30] Slantel. S.. Haenszel. W. (1959). 5tatistical Aspects of the .innlysis of Data
from Retrospectiïe Studies of Disease". .Journal 01 iVationaL Cancer h s t i t u t e .
22. 119-748.
[2 11 Petit ti. D. B. ( 9 ) . Meta-analysis. Decision .-lnahjsis.and Cost-effectiveness
Analysis. Oxford University Press. Oxford and Sew York.
[22] Rosenthal. R. (1978). "Combing Results of Independent Studies". Psychological
Bulletin. 85. 18;-193.
p3] Sacks. H.S.. Berrier. .J.. Reitman. D.. Ancona-Berk. \-.A.. Chalmers. T.C. (1987).
"Meta-analysis of Randomized Controlled trials". The i V e . ~ England Journal of
Medicine. 316. 450-435.
[21] Tritchler. D. (1999). 9Iodeling Stucly Quality in Sleta-analusis" . Statistics in
Medicine. 18. 2 135-2 1 6
[E] Wolf. F..1l. (1986). :2.1eta-andy.sis: Quaditatiue Methods for Reseurch S~lrrthesis.
Sage Publications. Sewbury Park. Calif.
[-61 Yusuf. S.. Peto. R.. Lewis. -1 . . Collins. R.. Sleight. P. (1985). **Beta Blockade
cliiririg and after Slocardial Infraction: an Owrview of the Ranclomized Trials".
Proyress in Cardio~uascdar Diseuses. 27. 33.5-371.