nhs r&d hta programmepure-oai.bham.ac.uk/ws/files/17501744/deeks... · random samples of...
Post on 27-Apr-2020
1 Views
Preview:
TRANSCRIPT
Health Technology Assessm
ent 2005;Vol. 9: No. 26
Indirect comparisons of com
peting interventions
Indirect comparisons of competinginterventions
AM Glenny, DG Altman, F Song, C Sakarovitch, JJ Deeks, R D’Amico, M Bradburn and AJ Eastwood
Health Technology Assessment 2005; Vol. 9: No. 26
HTAHealth Technology AssessmentNHS R&D HTA Programme
The National Coordinating Centre for Health Technology Assessment,Mailpoint 728, Boldrewood,University of Southampton,Southampton, SO16 7PX, UK.Fax: +44 (0) 23 8059 5639 Email: hta@soton.ac.ukhttp://www.ncchta.org ISSN 1366-5278
FeedbackThe HTA Programme and the authors would like to know
your views about this report.
The Correspondence Page on the HTA website(http://www.ncchta.org) is a convenient way to publish
your comments. If you prefer, you can send your comments to the address below, telling us whether you would like
us to transfer them to the website.
We look forward to hearing from you.
July 2005
How to obtain copies of this and other HTA Programme reports.An electronic version of this publication, in Adobe Acrobat format, is available for downloading free ofcharge for personal use from the HTA website (http://www.hta.ac.uk). A fully searchable CD-ROM isalso available (see below).
Printed copies of HTA monographs cost £20 each (post and packing free in the UK) to both public andprivate sector purchasers from our Despatch Agents.
Non-UK purchasers will have to pay a small fee for post and packing. For European countries the cost is£2 per monograph and for the rest of the world £3 per monograph.
You can order HTA monographs from our Despatch Agents:
– fax (with credit card or official purchase order) – post (with credit card or official purchase order or cheque)– phone during office hours (credit card only).
Additionally the HTA website allows you either to pay securely by credit card or to print out yourorder and then post or fax it.
Contact details are as follows:HTA Despatch Email: orders@hta.ac.ukc/o Direct Mail Works Ltd Tel: 02392 492 0004 Oakwood Business Centre Fax: 02392 478 555Downley, HAVANT PO9 2NP, UK Fax from outside the UK: +44 2392 478 555
NHS libraries can subscribe free of charge. Public libraries can subscribe at a very reduced cost of £100 for each volume (normally comprising 30–40 titles). The commercial subscription rate is £300 per volume. Please see our website for details. Subscriptions can only be purchased for the current orforthcoming volume.
Payment methods
Paying by chequeIf you pay by cheque, the cheque must be in pounds sterling, made payable to Direct Mail Works Ltdand drawn on a bank with a UK address.
Paying by credit cardThe following cards are accepted by phone, fax, post or via the website ordering pages: Delta, Eurocard,Mastercard, Solo, Switch and Visa. We advise against sending credit card details in a plain email.
Paying by official purchase orderYou can post or fax these, but they must be from public bodies (i.e. NHS or universities) within the UK.We cannot at present accept purchase orders from commercial companies or from outside the UK.
How do I get a copy of HTA on CD?
Please use the form on the HTA website (www.hta.ac.uk/htacd.htm). Or contact Direct Mail Works (seecontact details above) by email, post, fax or phone. HTA on CD is currently free of charge worldwide.
The website also provides information about the HTA Programme and lists the membership of the variouscommittees.
HTA
Indirect comparisons of competinginterventions
AM Glenny,1* DG Altman,2 F Song,3
C Sakarovitch,2 JJ Deeks,2 R D’Amico,2
M Bradburn2 and AJ Eastwood4
In collaboration with the International Stroke Trial Collaborative Group
1 Cochrane Oral Health Group, Dental School, University of Manchester, UK
2 Cancer Research UK, Centre for Statistics in Medicine, Wolfson College Annexe, Oxford, UK
3 School of Medicine Health Policy and Practice, University of East Anglia,Norwich, UK
4 Centre for Reviews and Dissemination, University of York, UK
*Corresponding author
Declared competing interests of authors: none
Published July 2005
This report should be referenced as follows:
Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D’Amico R, et al. Indirectcomparisons of competing interventions. Health Technol Assess 2005;9(26).
Health Technology Assessment is indexed and abstracted in Index Medicus/MEDLINE,Excerpta Medica/EMBASE and Science Citation Index Expanded (SciSearch®) and Current Contents®/Clinical Medicine.
NHS R&D HTA Programme
The research findings from the NHS R&D Health Technology Assessment (HTA) Programme directlyinfluence key decision-making bodies such as the National Institute for Health and Clinical
Excellence (NICE) and the National Screening Committee (NSC) who rely on HTA outputs to help raisestandards of care. HTA findings also help to improve the quality of the service in the NHS indirectly inthat they form a key component of the ‘National Knowledge Service’ that is being developed to improvethe evidence of clinical practice throughout the NHS.
The HTA Programme was set up in 1993. Its role is to ensure that high-quality research information onthe costs, effectiveness and broader impact of health technologies is produced in the most efficient wayfor those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined toinclude all interventions used to promote health, prevent and treat disease, and improve rehabilitationand long-term care, rather than settings of care.
The HTA programme commissions research only on topics where it has identified key gaps in theevidence needed by the NHS. Suggestions for topics are actively sought from people working in theNHS, the public, consumer groups and professional bodies such as Royal Colleges and NHS Trusts.
Research suggestions are carefully considered by panels of independent experts (including consumers)whose advice results in a ranked list of recommended research priorities. The HTA Programme thencommissions the research team best suited to undertake the work, in the manner most appropriate to findthe relevant answers. Some projects may take only months, others need several years to answer theresearch questions adequately. They may involve synthesising existing evidence or designing a trial toproduce new evidence where none currently exists.
Additionally, through its Technology Assessment Report (TAR) call-off contract, the HTA Programme isable to commission bespoke reports, principally for NICE, but also for other policy customers, such as aNational Clinical Director. TARs bring together evidence on key aspects of the use of specifictechnologies and usually have to be completed within a limited time period.
The research reported in this monograph was commissioned by the HTA Programme as project number96/51/99. The contractual start date was in February 1999. The draft report began editorial review in May2002 and was accepted for publication in November 2004. As the funder, by devising a commissioningbrief, the HTA Programme specified the research question and study design. The authors have beenwholly responsible for all data collection, analysis and interpretation, and for writing up their work. TheHTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like tothank the referees for their constructive comments on the draft document. However, they do not acceptliability for damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the HTAProgramme or the Department of Health.
Editor-in-Chief: Professor Tom WalleySeries Editors: Dr Peter Davidson, Professor John Gabbay, Dr Chris Hyde,
Dr Ruairidh Milne, Dr Rob Riemsma and Dr Ken SteinManaging Editors: Sally Bailey and Caroline Ciupek
ISSN 1366-5278
© Queen’s Printer and Controller of HMSO 2005
This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals providedthat suitable acknowledgement is made and the reproduction is not associated with any form of advertising.
Applications for commercial reproduction should be addressed to NCCHTA, Mailpoint 728, Boldrewood, University of Southampton, Southampton, SO16 7PX, UK.
Published by Gray Publishing, Tunbridge Wells, Kent, on behalf of NCCHTA.Printed on acid-free paper in the UK by St Edmundsbury Press Ltd, Bury St Edmunds, Suffolk.
Criteria for inclusion in the HTA monograph seriesReports are published in the HTA monograph series if (1) they have resulted from work commissionedfor the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the refereesand editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search,appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit thereplication of the review by others.
M
Objectives: To survey the frequency of use of indirectcomparisons in systematic reviews and evaluate themethods used in their analysis and interpretation. Alsoto identify alternative statistical approaches for theanalysis of indirect comparisons, to assess theproperties of different statistical methods used forperforming indirect comparisons and to compare directand indirect estimates of the same effects withinreviews.Data sources: Electronic databases.Review methods: The Database of Abstracts ofReviews of Effects (DARE) was searched for systematicreviews involving meta-analysis of randomisedcontrolled trials (RCTs) that reported both direct andindirect comparisons, or indirect comparisons alone. Asystematic review of MEDLINE and other databaseswas carried out to identify published methods foranalysing indirect comparisons. Study designs werecreated using data from the International Stroke Trial.Random samples of patients receiving aspirin, heparinor placebo in 16 centres were used to create meta-analyses, with half of the trials comparing aspirin andplacebo and half heparin and placebo. Methods forindirect comparisons were used to estimate thecontrast between aspirin and heparin. The wholeprocess was repeated 1000 times and the results werecompared with direct comparisons and also theoreticalresults. Further detailed case studies comparing theresults from both direct and indirect comparisons ofthe same effects were undertaken.Results: Of the reviews identified through DARE,31/327 (9.5%) included indirect comparisons. A furtherfive reviews including indirect comparisons were
identified through electronic searching. Few reviewscarried out a formal analysis and some based analysison the naive addition of data from the treatment armsof interest. Few methodological papers were identified.Some valid approaches for aggregate data that could beapplied using standard software were found: theadjusted indirect comparison, meta-regression and, forbinary data only, multiple logistic regression (fixedeffect models only). Simulation studies showed that thenaive method is liable to bias and also produces over-precise answers. Several methods provide correctanswers if strong but unverifiable assumptions arefulfilled. Four times as many similarly sized trials areneeded for the indirect approach to have the samepower as directly randomised comparisons. Detailedcase studies comparing direct and indirect comparisonsof the same effect show considerable statisticaldiscrepancies, but the direction of such discrepancy isunpredictable.Conclusions: Direct evidence from good-quality RCTsshould be used wherever possible. Without thisevidence, it may be necessary to look for indirectcomparisons from RCTs. However, the results may besusceptible to bias. When making indirect comparisonswithin a systematic review, an adjusted indirectcomparison method should ideally be used employingthe random effects model. If both direct and indirectcomparisons are possible within a review, it isrecommended that these be done separately beforeconsidering whether to pool data. There is a need toevaluate methods for the analysis of indirectcomparisons for continuous data and for empiricalresearch into how different methods of indirect
Health Technology Assessment 2005; Vol. 9: No. 26
iii
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Abstract
Indirect comparisons of competing interventions
AM Glenny,1* DG Altman,2 F Song,3 C Sakarovitch,2 JJ Deeks,2 R D’Amico,2
M Bradburn2 and AJ Eastwood4
In collaboration with the International Stroke Trial Collaborative Group
1 Cochrane Oral Health Group, Dental School, University of Manchester, UK
2 Cancer Research UK, Centre for Statistics in Medicine, Wolfson College Annexe, Oxford, UK3 School of Medicine Health Policy and Practice, University of East Anglia, Norwich, UK4 Centre for Reviews and Dissemination, University of York, UK *Corresponding author
comparison perform in cases where there is a large treatment effect. Further study is needed intowhen it is appropriate to look at indirect comparisons and when to combine both direct andindirect comparisons. Research into how evidence from indirect comparisons compares to that from
non-randomised studies may also be warranted.Investigations using individual patient data from a meta-analysis of several RCTs using different protocols and anevaluation of the impact of choosing different binaryeffect measures for the inverse variance method wouldalso be useful.
Abstract
iv
Health Technology Assessment 2005; Vol. 9: No. 26
v
List of abbreviations .................................. vii
Executive summary .................................... ix
1 Background ............................................... 1
2 Research questions .................................... 5
3 Indirect comparisons in published systematic reviews ..................................... 7Methods ...................................................... 7Results ........................................................ 8Summary .................................................... 14
4 Statistical methods for indirect comparisons ............................................... 17Systematic review of the literature ............. 17Statistical methods for indirect comparisons ............................................... 18Comments .................................................. 25
5 An empirical investigation of the properties of different statistical methods used forperforming indirect comparisons .............. 27The International Stroke Trial (IST) ......... 27Adaptations of the trial .............................. 27Analyses ...................................................... 28Some theoretical results ............................. 29Results of empirical studies ........................ 29Summary .................................................... 40
6 Statistical discrepancy between the direct and indirect estimate: empirical evidence from published meta-analyses ................... 43Methods ...................................................... 43Results ........................................................ 44Commentary ............................................... 44Summary .................................................... 49
7 Detailed case studies ................................. 51Five cases with significant discrepancy ....... 51Relative efficacy of antimicrobial prophylaxis in colorectal surgery ............... 55Summary .................................................... 63
8 Discussion ................................................... 65Indirect comparisons based on randomisedtrials: current practice ................................ 65Performance of different methods of making indirect comparisons ..................... 65
Quality of evidence from RCTs (directcomparisons) .............................................. 66What to do when direct evidence is available but insufficient ............................ 67What to do when there is no direct evidence ...................................................... 67Are indirect comparisons of RCTs preferable to direct comparisons from non-randomised trials? .............................. 68Comparing multiple interventionssimultaneously ............................................ 69
9 Conclusions ................................................ 71Implications for practice of systematic reviews ........................................................ 71Implications for clinical research ............... 71Recommendations for methodological research ...................................................... 71
Acknowledgements .................................... 73
References .................................................. 75
Appendix 1 Reviews of effectivenessprescreening form ...................................... 81
Appendix 2 Data extraction form ............. 85
Appendix 3 Table of systematic reviews making indirect comparisons ..................... 87
Appendix 4 List of excluded reviews ........ 109
Appendix 5 Search strategy development ............................................... 111
Appendix 6 Additional electronic searchstrategies ..................................................... 115
Appendix 7 Illustrative analyses ............... 119
Appendix 8 Identified meta-analyses providing sufficient data for both direct and indirect comparisons ........................... 127
Health Technology Assessment reportspublished to date ....................................... 135
Health Technology Assessment Programme ................................................ 145
Contents
ACE angiotensin-converting enzyme
AIC adjusted indirect comparison
AIIA angiotensin II antagonist
ALT alanine transaminase
AP aerolised pentamidine
APSAC anisoylated plasminogenstreptokinase activator complex
ASA aminosalicylate
ATC Antiplatelet Trialists’ Collaboration
CAD coronary artery disease
CDSR Cochrane Database of SystematicReviews
Cefot-M cefotaxime plus metronidazole
Cefur-M cefuroxime plus metronidazole
CI confidence interval
Co-A co-amoxiclav
CRD Centre for Reviews andDissemination
CT chemotherapy
ctrl control
D dapsone
DARE Database of Abstracts of Reviews ofEffects
DC direct comparison
Dip dipyridamole
DP D-penicillamine
DVT deep vein thrombosis
EE ethinyl estradiol
ENRIS Evaluating Non-RandomisedIntervention Studies
ESRD end-stage renal disease
FTT Fibrinolytic Therapy Trialists
5-FU 5-fluorouracil
GI gastrointestinal
GLMM generalised linear mixed model
GORD gastro-oesophageal reflux disease
H2RA H2-receptor antagonist
IBS irritable bowel syndrome
IC indirect comparison
IPD individual patient data
IST International Stroke Trial
IVE important vascular events
LMWH low molecular weight heparin
MAP mean arterial pressure
NA not applicable
NNT number needed to treat
NRT nicotine replacement therapy
ns not significant
NSAID non-steroidal anti-inflammatorydrug
OA osteoarthritis
OGD oesophagogastroduodenoscopy
OR odds ratio
P pyrimethamine
PE pulmonary embolism
PEP polyestradiol phosphate
PPI proton pump inhibitor
RA rheumatoid arthritis
RCT randomised controlled trial
RemRR remedication rate ratio
ResRR response rate ratio
RPA reteplase
RR relative risk
RT radiotherapy
continued
Health Technology Assessment 2005; Vol. 9: No. 26
vii
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
List of abbreviations
List of abbreviations continued
SD standard deviation
SE standard error
SEM standard error of the mean
SK streptokinase
SMX sulfamethoxazole
SPID sum of pain intensity difference
SSRI selective serotonin reuptakeinhibitor
SSZ sulfasalazine
SWI surgical wound infection
TMP trimethoprim
TOTPAR total pain relief
t-PA tissue plasminogen activator
UFH unfractionated heparin
List of abbreviations
viii
All abbreviations that have been used in this report are listed here unless the abbreviation is well known (e.g. NHS), or it has been used only once, or it is a non-standard abbreviation used only in figures/tables/appendices in which case the abbreviation is defined in the figure legend or at the end of the table.
Health Technology Assessment 2005; Vol. 9: No. 26
ix
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
BackgroundThe randomised controlled trial (RCT) is the mostvalid design for evaluating the relative efficacy ofhealthcare technology. However, many competinginterventions have not been directly compared inRCTs and indirect methods have been commonlyused in meta-analyses. Such indirect comparisonsare subject to greater bias (especially selectionbias) than head-to-head randomised comparisons,as the benefit of randomisation does not holdacross trials. Therefore, it is essential to evaluatesuch bias that may lead to inaccuracies in theestimates of treatment effects and result ininappropriate policy decisions.
ObjectivesThe objectives of this study were:
� to survey the frequency of use of indirectcomparisons in systematic reviews and evaluatethe methods used in their analysis andinterpretation
� to identify alternative statistical approaches forthe analysis of indirect comparisons
� to assess the properties of different statisticalmethods used for performing indirectcomparisons
� to carry out empirical work comparing directand indirect estimates of the same effects withinreviews.
MethodsThe Database of Abstracts of Reviews of Effects(DARE) (1994 to March 1999) was searched forsystematic reviews involving meta-analysis of RCTsthat reported both direct and indirectcomparisons, or indirect comparisons alone. Asystematic review of MEDLINE (1966 to February2001) and other databases was carried out toidentify published methods for analysing indirectcomparisons.
Study designs were created using data from theInternational Stroke Trial. Random samples ofpatients receiving aspirin, heparin or placebo in16 centres were used to create meta-analyses, with
half of the trials comparing aspirin and placeboand half heparin and placebo. Methods forindirect comparisons were used to estimate thecontrast between aspirin and heparin. The wholeprocess was repeated 1000 times and the resultswere compared with direct comparisons and alsotheoretical results.
Further detailed case studies comparing the resultsfrom both direct and indirect comparisons of thesame effects were undertaken.
ResultsOf the reviews identified through DARE thatincluded meta-analyses of two or more RCTs,31/327 (9.5%) included indirect comparisons. Afurther five reviews including indirect comparisonswere identified through electronic searching. Fewreviews carried out a formal analysis. Some reviewsbased analysis on the naive addition of data fromthe treatment arms of interest. Interpretation ofindirect comparisons was not always appropriate.
Few methodological papers were identified. Somevalid approaches for aggregate data that could beapplied using standard software were found: theadjusted indirect comparison, meta-regressionand, for binary data only, multiple logisticregression (fixed effect models only).
Simulation studies showed that the naive methodis liable to bias and also produces over-preciseanswers. Several methods provide correct answersif strong but unverifiable assumptions are fulfilled.Four times as many similarly sized trials areneeded for the indirect approach to have the samepower as directly randomised comparisons.
Detailed case studies comparing direct andindirect comparisons of the same effect showconsiderable statistical discrepancies, but thedirection of such discrepancy is unpredictable.
ConclusionsWhen conducting systematic reviews to evaluatethe effectiveness of interventions, direct evidence
Executive summary
x
from good-quality RCTs should be used wherever possible. If little or no such evidenceexists, it may be necessary to look for indirectcomparisons from RCTs. The reviewer needs,however, to be aware that the results may besusceptible to bias.
When making indirect comparisons within asystematic review, an adjusted indirect comparisonmethod should ideally be used using the randomeffects model. If both direct and indirectcomparisons are possible within a review, it isrecommended that these be done separatelybefore considering whether to pool data.
Recommendations for researchThere is a need for evaluation of methods foranalysis of indirect comparisons for continuousdata.
There is a need for empirical research into how different methods of indirect comparison
perform in cases where there is a large treatment effect.
Further research is required to consider how todetermine when it is appropriate to look atindirect comparisons and how to judge when tocombine both direct and indirect comparisons.Research into how evidence from indirectcomparisons compares to that from non-randomised studies may also be warranted.
Empirical investigations were based on one large,multicentre trial with a common protocol acrosseach centre. It would be useful to repeat theinvestigations using individual patient data from ameta-analysis of several RCTs using differentprotocols.
The odds ratio was used as the measure of effectwithin this simulation study. Although logisticregression calls for the effect measure to be theodds ratio, it would be interesting to evaluate theimpact of choosing different binary effectmeasures for the inverse variance method.
Executive summary
Well-designed randomised controlled trials(RCTs) generally provide the most reliable
evidence of effectiveness as observed differencesbetween the trial arms can, in general, beconfidently attributed to differences in thetreatment(s) being evaluated.1,2 However, in manyareas, available trials may not have directlycompared the specific treatments or regimens ofinterest. A common example is where there is aclass of several drugs, each of which has beenstudied in placebo-controlled RCTs, but there areno trials (or very few) in which the drugs havebeen directly compared with each other. Forexample, in a systematic review of antibioticprophylaxis for preventing surgical woundinfections after colorectal surgery, only a limitednumber of antibiotics or combinations ofantibiotics (more than 70 regimens in total) weredirectly compared (head-to-head comparisons) in147 randomised trials.3 With the increasing useand development of meta-analytical techniques,comparisons of arms of different RCTs are beingundertaken. Such indirect comparisons are subjectto greater bias (especially selection bias) thanhead-to-head randomised comparisons, as thebenefit of randomisation does not hold acrosstrials. It is vital, therefore, to evaluate such biasesthat may lead to inaccuracies in the estimates oftreatment effects and result in inappropriatepolicy decisions. By identifying the presence,magnitude and, if possible, methods to overcomesuch bias, the accuracy and interpretation ofestimates of the effectiveness of health
technologies will be enhanced. In addition, suchevaluations may give an insight into theusefulness of indirect comparisons in cases wheredirect comparisons are impossible, for example,when examining the placebo effects ofsubcutaneous and oral medication.4
In a review exploring the types of evidence used tosupport the prescribing of one drug over another,McAlister and colleagues5 suggest a hierarchy ofevidence for grading studies that compare a drugwith another of the same class (Table 1).
As expected, direct comparisons from head-to-head RCTs measuring clinically importantoutcomes (referring to long-term efficacy data) areat the top of the hierarchy of evidence at level 1,followed by head-to-head RCTs using validatedsurrogate outcomes (level 2). Comparisons madeacross placebo-controlled RCTs of different drugsare, however, also classified at level 2, despite theincreased threats to validity due to likelydifferences between trials in terms of end-pointdefinitions, inclusion criteria, patientcharacteristics, setting or baseline risk ofoutcomes. McAlister and colleagues5 acknowledgethat the strength of inference from such indirectcomparisons is limited.
It could be argued that since the randomisationelement of the RCT is not (or not fully) usedduring indirect comparison, such methods mayhave important implications on the use of data
Health Technology Assessment 2005; Vol. 9: No. 26
1
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 1
Background
TABLE 1 Suggested levels of evidence for comparing the efficacy of drugs within the same class
Level Comparison Study patients Outcomes
1 Within a head-to-head RCT Identical (by definition) Clinically important
2 Within a head-to-head RCT Identical (by definition) Validated surrogate
2 Across RCTs of different drugs Similar or different (in disease status Clinically important or vs placebo and risk factor status) validated surrogate
3 Across subgroup analyses from RCTs Similar or different Clinically important or of different drugs vs placebo surrogate
3 Across RCTs of different drugs vs Similar or different Unvalidated surrogateplacebo
4 Between non-randomised studies Similar or different Clinically important
Adapted from McAlister et al.5
from other types of non-randomised(‘observational’) study design.
Indirect comparisons are not only used to makecomparisons between drugs in the same class orduring subgroup analyses when comparing, forexample, different dosages of the same drug.Comparisons between very different interventions,such as pharmacological interventions versussurgery, or between different classes of drugs havealso been made using indirect comparisons.Examples of each type of comparison can be seenin the health research literature.
For example, an indirect comparison of differentclasses of drugs was undertaken in a meta-analysisof second-line drugs used to treat rheumatoidarthritis.6 The drugs compared were antimalarialdrugs, auranofin, injectable gold, methotrexate, D-penicillamine and sulfasalazine. Sixty-six trialsexamining the efficacy of second line drugs wereincluded. For each drug the results were combinedacross treatment groups. Means for each drugtreatment were generated by weighting eachtreatment group by size at the end of the trial. Tocompare drugs, analysis of variance wasundertaken, weighted by treated group size,multiplied by study quality and adjusted forcovariates shown to have significant associationwith each outcome. A fixed effects model wasused. The outcomes of interest were the tenderjoint count, the erythrocyte sedimentation rateand grip strength. For each outcome, resultsshowed that auranofin tended to be weaker thanother second line drugs. No attempt, however, wasmade to provide data from direct comparisons.
An example of indirect comparisons being used tocompare drugs within a specific class can be seenin the systematic review by Garg and Yusuf.7 Theauthors of the review used indirect comparisons toevaluate the effects of angiotensin-convertingenzyme (ACE) inhibitors on mortality andmorbidity in patients with symptomatic congestiveheart failure. Thirty-two placebo-controlled trialsof ACE inhibitors, including 7105 patients, wereused to make adjusted indirect comparisons ofestimates of effect for the different agentsevaluated. The authors state that “Similar benefitswere observed with several different ACEinhibitors, although the data were largely based onenalapril maleate, captopril, ramipril, quinaprilhydrochloride, and lisinopril”.
Indirect comparisons of the effect of differentdoses of a drug can be illustrated with the reviewby the Homocysteine Lowering Trialists’
Collaboration.8 The review included onlyrandomised trials (with an untreated controlgroup) that assessed the effects of different dosesof folic acid, with or without the addition ofvitamin B12 or B6, on blood homocysteineconcentrations. Twelve trials, with data on 1114patients, were included in the review. Trials weregrouped according to the folic acid regimen used(<1, 1–3 or >3 mg daily). The proportionalreductions in blood homocysteine in the treatedgroups compared with the control groups wereevaluated by an analysis of covariance whichestimated the difference in the post-treatment,log-transformed homocysteine values afteradjustment for baseline homocysteine levels. Thismodel was extended to allow the extent of thisadjustment to vary between studies according tofactors such as folic acid dose, additional vitaminB6 or B12, age, gender or duration of treatment.After adjusting for pretreatment bloodconcentrations of homocysteine and folate, therewas no significant difference between daily doses.Such indirect comparison of results from subgroupanalyses is not uncommon in systematic reviewsand meta-analyses. Boersma and colleagues9
evaluated the effect of delayed thrombolytictreatment following acute myocardial infarctionand short-term mortality. They included 22randomised trials of over 50,000 patients. Thetrials were grouped according to time tofibrinolytic therapy (0–1, ≥ 1–2, ≥ 2–3, ≥ 3–6,≥ 6–12, ≥ 12–24 hours) rather than dosage. Theresults of the subgroup analysis were used toconclude that the “beneficial effect of fibrinolytictherapy is substantially higher in patientspresenting within two hours after symptom onsetcompared to those presenting later”.
The above examples illustrate both unadjusted(naive) and adjusted indirect comparisons. In thenaive approach6 data are pooled across treatmentarms, ignoring the fact that the studies are RCTs(and discarding data from some treatmentgroups). Given that comparisons are being madeacross trial arms in the naive indirect comparisons,negating the randomised nature of the trial, theexclusion of non-randomised or observationalstudies can be questioned. Such methods mayhave important implications for the use of datafrom other types of non-randomised(observational) study design.
In the adjusted indirect comparison, thecomparison of the interventions of interest isadjusted by the results of their direct comparisonwith a common control group (e.g. placebo),partially using the strength of the RCT. The
Background
2
methods used to undertake such adjusted indirectcomparisons vary. In the simplest case, one may beinterested in comparing two interventions B andC. Indirect evidence can be obtained from RCTs ofeither B or C versus a common comparator,perhaps placebo (RCT 1 and RCT 2, Figure 1).One could use the results from one or more trialsto estimate the effect of each intervention relativeto control. An indirect measure of the relativeefficacy of B and C could be made subjectively orusing some statistical procedure. This informationcould be combined with data from any direct Bversus C comparison, again either qualitatively orstatistically. In more complex situations, one maybe interested in trying to estimate simultaneouslythe (relative) effectiveness of several treatments(e.g. various �-blockers and other preventivetreatments for patients who have had a myocardialinfarction, or numerous analgesic drugs used in avariety of pain relief settings).
Several ad hoc approaches have been adopted tohandle such situations. For example, in the casewhere there are a large number of placebo-controlled trials of various drugs a commonapproach is to carry out separate meta-analyses ofthe trials of each drug. The estimated effects arethen compared explicitly or implicitly, ignoringthe fact that the studies (or the patients in them)may not be strictly comparable. This procedure islike producing a league table, and results of thistype appear increasingly in Bandolier10 andelsewhere.11 Such comparisons may be of little useif tables are ranked according to number neededto treat (NNT) estimates in which no-treatmentgroups, placebo-controlled trials and head-to-headcomparisons are included.12
Meta-analyses using both randomised and non-randomised studies have also been undertaken.For example, in a review of thromboprophylaxis
after total hip replacement, Murray and colleaguesanalysed treatment arms from both controlled anduncontrolled studies.13 Such indirect comparisonsbetween non-randomised groups are made on theassumption that these groups are similar acrossthe different studies. However, this may not be areasonable assumption, so analyses of this kind areopen to several sources of bias. For example, asthe efficacy of a treatment may vary amongsubpopulations of patients, differences in baselinecharacteristics between groups within differenttrials (variations in case-mix) may lead to biasedestimates of treatment effect. Even when patientcharacteristics are similar, other aspects may varybetween trials, such as ancillary treatment or otheraspects of patient care (the actual treatment mayvary too).
There is a need to investigate the properties ofsuch procedures and to address the ways in whichsuch meta-analyses can best be carried out. Mostobviously, if trials have all used a common controlintervention (maybe placebo) then there is thepotential to use the control groups as astandardising factor. Bucher and colleagues14
presented a model for undertaking indirectcomparisons that preserves the randomisation ofthe originally assigned patient groups. Theirmodel was tested using a meta-analysis of RCTscomparing two experimental regimens against thestandard regimen for the prevention ofPneumocystis carinii pneumonia in patients withHIV infection. Trials providing a direct estimate ofthe relative effectiveness of the two experimentaldrugs were also identified, and an overall estimateof effect of these trials was calculated. Both theindirect and direct comparisons favouredtrimethoprim–sulfamethoxazole overdapsone/pyrimethamine, but the magnitude ofdifference was less in the direct comparison. Theodds ratio (OR) from the indirect comparison was0.37 [95% confidence interval (CI) 0.21 to 0.65]favouring trimethoprim-sulfamethoxazole,compared with 0.64 (95% CI 0.45 to 0.90) for thedirect comparison using the fixed effects model.Using the random effects model the odds ratiofrom the direct comparison was 0.43 (95% CI 0.21to 0.89), which was similar to that from theindirect comparison. The model presented mayprotect against some of the biases that arisethrough the use of indirect comparisons. Theauthors concluded that only where directcomparisons are unavailable should indirectcomparison meta-analysis be carried out. In suchcases the limitations of such procedures should beexamined thoroughly. However, the approachclearly needs further evaluation.
Health Technology Assessment 2005; Vol. 9: No. 26
3
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
RCT 1 RCT 2
Placebo PlaceboB C
Indirectcomparison
FIGURE 1 Indirect comparison
Indirect comparisons are being used in systematicreviews to evaluate the relative effectiveness ofalternative interventions even though such indirectcomparisons may be less accurate than head-to-head randomised comparisons. It is vital,
therefore, to evaluate the properties of differentstatistical approaches to indirect comparisons toensure that inaccuracies in the estimates oftreatment effects do not result in inappropriatepolicy decisions.
Background
4
The aims of the project were:
� to survey the frequency of use of indirectcomparisons in systematic reviews and evaluatethe methods used in their analysis andinterpretation
� to identify alternative statistical approaches for the analysis of indirect comparisons
� to assess the properties of different statisticalmethods used for performing indirectcomparisons
� to carry out empirical work comparing directand indirect estimates of the same effects withinreviews.
To achieve these aims the project involved a reviewof the literature and methodological and empiricalinvestigations.
Chapter 3 presents a survey of published indirectcomparisons. Chapter 4 contains a systematicreview of the literature identifying differentstatistical approaches to the analysis of indirectcomparisons. Chapters 5 and 6 present themethodological and empirical investigations.Detailed case studies are given in Chapter 7. Eachchapter contains a description of the methodsused, the results and a brief overview of thefindings. Chapter 8 gives an overall discussion,drawing on the findings from Chapter 3–7, andthe implications are discussed in Chapter 9.
Health Technology Assessment 2005; Vol. 9: No. 26
5
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 2
Research questions
This chapter summarises the findings of asurvey of frequency of use of indirect
comparisons in published systematic reviews.
MethodsSearch strategyTo survey the frequency of use of indirectcomparisons in published systematic reviews it wasunnecessary to conduct a comprehensive search ofthe research literature. Rather, a careful search ofdatabases of systematic reviews was performed. Inaddition, it was considered impossible to develop asearch strategy to identify relevant publishedreviews.
Two databases were readily available which providea source of meta-analyses: the Database ofAbstracts of Reviews of Effects (DARE) [availablethrough a number of sources, including the Centrefor Reviews and Dissemination (CRD) websitehttp://www.york.ac.uk/inst/crd/darehp.htm and theCochrane Library] and the Cochrane Database ofSystematic Reviews (CDSR) (on the CochraneLibrary, available to all members of the NHSthrough the National Electronic Library forHealth, www.nelh.nhs.uk). It was felt appropriateto focus initially on a search of DARE. DARE is adatabase of quality-assessed systematic reviews.The reviews are identified through regularsearching of a number of electronic databases(including MEDLINE, CINAHL, CurrentContents Clinical Medicine and BIOSIS), byhandsearching key major journals and scanninggrey literature. To be included on DARE, thereviews undergo a rigorous quality assessment andmust meet set criteria with regard to the reviewquestion, the search strategy, validity assessment ofthe primary studies and the presentation andpooling of the primary studies. A hard copy of allreviews published on DARE (1994 to December1998) was obtained, and each review screenedaccording to the inclusion criteria (see below).
Inclusion criteriaAll systematic reviews including at least one meta-analysis were assessed to see whether they used:
1. RCTs2. indirect comparisons3. direct comparisons.
The following types of meta-analysis wererecorded:
� those incorporating elements 1 and 2, with thepossibility of comparing them with other studiesmaking direct comparisons of the sameinterventions. A note was made of whether theauthors interpreted the results as if directcomparisons had been made
� those incorporating all three elements, toexamine the differences in estimates of effectobtained using direct and indirect comparisons
� those incorporating elements 1 and 3, andproviding sufficient data to try to undertake anindirect comparison of specific interventions.
All other systematic reviews were excluded.
Assessment of relevanceAll systematic reviews listed on DARE wereassessed using a piloted prescreening form (seeAppendix 1) designed to allow for identification ofarticles meeting the above inclusion criteria andalso those for the Evaluating Non-RandomisedIntervention Studies (ENRIS) project.15 Tworeviewers independently assessed each article anddisagreements were resolved by discussion. Alldata were managed using Microsoft Excel 97.
Data extractionFor all reviews assessed as relevant, data wereextracted using a predefined data extraction form(Appendix 2). This process was undertakenindependently by two reviewers and discrepancieswere resolved through discussion. The results ofthe data extraction process were tabulated. Themethod used to carry out the indirect comparisonand the appropriateness of the interpretation ofresults were noted. The interpretation was deemedappropriate if the findings of direct comparisonswere given greater weight in the conclusions, orthe potential biases associated with the findings ofthe indirect comparisons were discussed.
Health Technology Assessment 2005; Vol. 9: No. 26
7
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 3
Indirect comparisons in published systematic reviews
ResultsIn total, 734 systematic reviews were available onDARE for screening in March 1999. Of these, 327included a meta-analysis of RCTs. The initialscreening identified ten reviews that were coded asbeing definite examples of indirect comparisonsand 46 reviews that were coded as possiblyincluding an indirect comparison. A further 12potentially relevant reviews were located throughthe electronic searches for the systematic review(Chapter 4).
After detailed assessment, 13 of these 68 reviewswere identified as including both direct andindirect comparisons of competing interventionsand 23 included indirect comparisons only. Thirty-one of the reviews were identified from the searchof DARE and five from the electronic searches.Detailed summaries of the 36 included reviewsappear in Appendix 3. The remainder of thereviews did not include suitable data and so wereexcluded (see Appendix 4).
Reviews including both direct andindirect comparisonsThirteen of the identified systematic reviews usedboth direct and indirect comparisons.16–28 Tenpresented adjusted indirect comparisons, firstestimating the pooled effects of each treatmentarm against placebo or a control group and thencomparing the two estimates. The interpretationof the results was appropriate in each case, withconclusions being based largely on the directcomparisons, or problems associated with indirectcomparisons being discussed.16–20,22,24,26–28 Theother three reviews presented naive, unadjusted,indirect comparisons.21,23,25 The interpretation ofthe results was classed as appropriate in two of thereviews, with emphasis given to results from directcomparisons when available.21,25 There was someuncertainty surrounding the appropriateness ofthe interpretation of the results presented in thethird review23 (Table 2).
Each of these reviews is discussed in the followingsections.
Adjusted indirect comparisons (see Appendix 3,Table 19)Adjusted indirect comparisons were considered tohave been used when estimates of effect werecompared for interventions that had beenevaluated using a common comparator (e.g.placebo). Three of the reviews undertaking anadjusted indirect comparison were conducted bythe Antiplatelet Trialists’ Collaboration andexamined the effects of antiplatelet therapy onvarious outcomes measures (including theprevention of death, myocardial infarction, stroke,venous thrombosis and pulmonary embolism) indifferent categories of patients.16–18 All threereviews were rigorous in their methodology. Thereviews all presented adjusted indirectcomparisons of aspirin plus dipyridamole andaspirin alone, using a common control group. Theresults for the indirect comparisons werepresented separately from those of the directcomparison, and used to enhance the findings ofthe reviews. Conclusions were drawn cautiously ineach review.
A fourth review comparing aspirin plusdipyridamole with aspirin alone was conducted byLowenthal and Buyse, examining the effectivenessof the drugs for the secondary prevention ofcerebrovascular accidents.19 The outcomes ofinterest were total mortality, vascular mortality,total strokes, fatal strokes, and a composite end-point consisting of vascular death or non-fatalstroke or non-fatal myocardial infarction(‘important vascular events’). Double-blind RCTscomparing aspirin with placebo, aspirin plusdipyridamole with placebo, or aspirin plusdipyridamole with aspirin alone were included inthe review. Adjusted indirect comparison wasmade using the placebo groups. Overall oddsratios for aspirin versus placebo were plottedalongside odds ratios for aspirin plus
Indirect comparisons in published systematic reviews
8
TABLE 2 Reviews using both direct and indirect comparisons
No. of Appropriate interpretation of results Agreement between IC and DC studies (estimate of effect in same direction)
Yes No Uncertain Yes No Uncertain
Adjusted IC 10 10 – – 7 2 1Naive IC 3 2 – 1 1 1 1
DC, direct comparison; IC, indirect comparison.
dipyridamole versus placebo for all outcomes. Therisk reduction for each outcome was consistentlybetter for the combination of drugs whencompared with aspirin alone. Chi-squared testswith one degree of freedom were calculated to testwhether the differences observed could beascribed to chance alone, and a statisticallysignificant difference was demonstrated in favourof the combination for three of the five outcomesmeasured [important vascular events (riskreduction 18% versus 40%; �2 = 7.30, p = 0.007),all strokes (risk reduction 17% versus 42%;�2 = 7.15, p = 0.007) and fatal strokes (riskreduction –10% versus 43%; �2 = 4.60, p = 0.03)].No such statistically significant differences werenoted in the trials comparing the two treatmentregimens directly. The authors of the reviewinterpret the results with caution. They state thatthe results “suggest that the combination therapy ofaspirin with dipyridamole may be superior toaspirin alone”. However, they discuss that theresults from the indirect comparisons may reflectdifferences in selection criteria or otherconfounding factors, rather than a truly greatertreatment effect of combination therapy.
Zhang and Li Wan Po conducted a systematicreview to assess the efficacy of paracetamol and itscombination with codeine or caffeine incomparison to paracetamol alone.27 An adjustedindirect comparison was made by estimating thepooled effects of each treatment arm againstplacebo and then by comparing the two estimates.Second, a direct comparison was made betweenparacetamol–dextropropoxyphene andparacetamol using head-to-head trials (ignoringthe placebo group in the three-armed studies).The results were expressed as the difference inpercentage improvement of total pain relief(TOTPAR%) and the sum of pain intensitydifference. The proportions of patients obtainingmoderate to excellent pain relief relative toplacebo and the ratio of patients requiringanalgesic remedication were also estimated. Theresults of the indirect comparison were similar tothose of the head-to-head comparisons anddemonstrated an enhanced analgesic efficacywhen codeine (60 mg) was used in addition toparacetamol (600 mg) (using TOTPAR% as theoutcome measure).
The efficacy of paracetamol and its combinationwith codeine was also assessed by Moore andcolleagues.24 They discussed the results of theindirect comparison (between paracetamol pluscodeine and paracetamol alone, using placebo asthe common control) and direct comparison
separately, making no attempt to combine them.In both cases an increased response rate was notedfor the combination therapy, although the indirectcomparison gave a greater estimate of effect.
Li Wan Po and Zhang conducted a systematicreview of RCTs to evaluate the comparativeefficacy and tolerability ofparacetamol–dextropropoxyphene combinationand paracetamol alone.20 The main outcomemeasures used in this review were the sum ofdifference in pain intensity, the response rate ratioand difference in response rate; and the rate ratioand rate difference of side-effects. Theparacetamol–dextropropoxyphene combinationwas compared with paracetamol alone bothdirectly and indirectly (adjusted indirectcomparison), as in their previous review.27 Theresults of the indirect comparison were used tosupport the findings of the direct comparisons.The mean (95% CI) difference in the sum ofdifference in pain intensity, as illustrated by thedirect comparisons, was 7.3% (–0.2 to 14.9%)(fixed effect model), in favour of the combination.The results of the indirect comparisons wereconsistent with the head-to-head comparisons andthe authors concluded that on the evidence ofboth direct and indirect comparisons “there islittle objective evidence to support prescribing acombination of paracetamol anddextropropoxyphene in preference to paracetamolalone in moderate pain such as that after surgery”.
The benefits of a longer duration interferonregimen (3 MU three times per week for 12months) in comparison to the standard 6-monthregimen for patients with chronic hepatitis C wasevaluated by Poynard and colleagues.26 Theypresented results of an adjusted indirectcomparison, comparing the pooled odds ratios forthe different regimens in comparison to undefinedcontrols. For example, the odds ratio for thesustained alanine transaminase (ALT) versuscontrol at the end of the 12-month regimen was0.35 (95% CI 0.28 to 0.43). This was shown to begreater than for the 6-month regimen, whichproduced an OR of 0.21 (95% CI 0.13 to 0.28),although the difference was not statisticallysignificant (p = 0.06). Direct comparison of a 12-month (or more) regimen and a 6-month regimenshowed a statistically significant duration effect onthe sustained response rate at 3 MU, OR 0.16(95% CI 0.9 to 0.23) (p < 0.01) in favour of the12-month regimen.
Matchar and colleagues did not make anyreference to the use of indirect comparisons within
Health Technology Assessment 2005; Vol. 9: No. 26
9
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
their review of medical treatments for strokeprevention, but the pooled relative risks forwarfarin and aspirin versus a control/placebogroup were compared, as were different doses ofaspirin versus placebo.22 The findings of theindirect comparisons were similar to, and used toreinforce, those of the direct comparison(although only one head-to-head study of warfarinversus aspirin was included).
Piccinelli and co-workers examined the efficacy ofdrug treatment in obsessive–compulsivedisorders.28 The review found considerabledifferences between the results from the indirectcomparisons and direct comparisons. The authorsrecognise that the increase in improvement ratewas greater for clomipramine than for selectiveserotonin reuptake inhibitors (SSRIs) whencompared with placebo, but highlight the fact thatdirect (head-to-head) comparisons showed similartherapeutic efficacy on obsessive–compulsivesymptoms. Possible reasons for discrepancies arecovered in Chapter 6.
Naive indirect comparisons (see Appendix 3,Table 20)Three other reviews included both direct andindirect comparisons, but did not use an adjustedindirect comparison.21,23,25 Marshall and Irvineundertook a systematic review to establish the roleof rectal corticosteroids in the management ofactive distal ulcerative colitis.21 They includedRCTs that assigned patients to two or moretreatment groups, with rectal corticosteroids in atleast one arm. Pooled response rates for each typeof corticosteroid and control therapy werecalculated across all trials. The pooled responserates for the conventional rectal corticosteroids,the topically active corticosteroids,aminosalicylates and placebo were compared.Conventional rectal and topically activecorticosteroids produced similar response rates forsymptoms, endoscopy, histology and remission.The aminosalicylates showed improvements in allresponse rates. The authors of the review sensiblyfocused on the results of direct comparisons intheir discussions, concluding that rectal 5-ASA (anaminosalicylate) is superior to rectalcorticosteroids in the management of distalulcerative colitis. Although the authors did notdraw heavily on the findings from the indirectcomparisons, the results of these are fairlysupportive of the conclusions.
Pope and colleagues also used a naive indirectcomparison technique to investigate thehypertensive effects of non-steroidal anti-
inflammatory drugs (NSAIDs) and ranked them bymagnitude of change in mean arterial pressure(MAP).25 They did this by extracting data fromeach NSAID treatment arm across all trials. Dataon possible confounders (including age, trialquality, dietary salt intake, hypertensive ornormotensive patients) were recorded andadjusted for in the calculation of the averagechange in MAP for each NSAID. The results of theindirect comparison are used greatly within thereview, but are compared with the results of head-to-head comparisons when available. The findingsfrom the direct and indirect comparisons were notalways consistent.
The results from indirect comparisons are notalways used cautiously. The comparativetolerability and rate of withdrawal from clinicaltrials of roxithromycin and erythromicin inpatients with lower respiratory tract infectionswere examined in a systematic review, again usingboth direct and indirect comparisons.23 ThreeRCTs included in the review were head-to-headcomparisons of roxithromycin and erythromicin.All other trials (n=22) compared eitherroxithromycin or erythromicin with anothermacrolide or other agent commonly used as firstline therapy for patients with lower respiratorytract infections. Summary statistics of reportedadverse events and withdrawals based on datafrom arms of all trials (both head-to-head andindirect comparisons) were calculated and used toformulate the review’s conclusions. Although theresults of the three head-to-head trials arepresented, summary statistics for these trials aloneare not calculated. The authors do discuss thepossibility of potential confounding factors, butconclude that there is no significant differencebetween groups in terms of clinical efficacy, age,gender, settings, duration of treatment, indicationsor year of publication.
Reviews using indirect comparisonsaloneOf the 23 reviews presenting results of indirectcomparisons only, 15 used an adjusted method ofcomparison,7–9,11,29–39 and eight performed anaive indirect comparison6,40–46 (Table 3).
Adjusted indirect comparisons (see Appendix 3,Table 21)Of the 15 studies including an adjusted indirectcomparison, only six of these interpreted theresults in an appropriate way.8,29–33 For example,Zalcberg and colleagues undertook a systematicreview of RCTs to determine the effect of 5-fluorouracil (5-Fu) dose in the adjuvant therapy
Indirect comparisons in published systematic reviews
10
of colorectal cancer.29 The trials included in thereview compared a 5-FU-containing regimen witha no-chemotherapy control group. For each study,the observed and expected number of deaths ontreatment was calculated and an estimated oddsratio of mortality obtained for combined studiesusing a fixed effect model. Forest plots werepresented for trials comparing a 5-FU-containingregimen (separated into four categories: ≥ 10 g,between 8 and 10 g, <8 g and oral chemotherapy)with no-treatment controls, and also for trialscomparing 5-FU and levamisole or another 5-FUregimen with no-treatment controls. The authorswere cautious in their interpretation of the resultsstating that, owing to the fact that findings werebased on indirect comparisons, confounding bythe type of patient being studied in each trial is apossibility.
A review by the Homocysteine Lowering Trialists’Collaboration also used a no-treatment controlgroup as the common comparator to enableindirect comparisons to be made for differentdoses of folic acid.8 The aim of the review was todetermine the size of reduction in homocysteineconcentrations produced by dietarysupplementation with folic acid and with vitaminB12 or B6. Twelve trials were included in thereview, with individual data on 1114 patients. Aforest plot was used to illustrate the reductions inblood homocysteine concentrations with varyingdoses of folic acid. The findings suggest that awide range of doses (0.5–5 mg) is similarlyeffective. The authors of the review providedetailed implications for future research in thisarea.
In a comparison of the efficacy of home-administrated low molecular weight heparin(LMWH) in the treatment of deep vein thrombosis(DVT) with that of hospital-administered LMWH,Leizorovicz used an unfractionated heparin (UFH)group as the common comparator.30 Twosubgroups of studies were identified (LMWHadministered at home versus UFH and LMWHadministered at hospital versus UFH) and the
odds ratios for recurrent thromboembolic events,mortality and major haemorrhage were presentedin a forest plot. The authors report similar efficacyresults for the recurrence of thromboembolicevents or death, but acknowledge that thisapproach reduced the statistical power for eachsubgroup.
Pignon and co-workers conducted a meta-analysisusing individual patient data from RCTscomparing chemotherapy alone withchemotherapy combined with thoracicradiotherapy.31 They clearly acknowledge thatindirect comparisons were used to compare trialsof early versus late radiotherapy, and also trialswith or without sequential radiotherapy. Thecomparisons did not reveal any optimal time fortreatment. The authors conclude that in order toidentify the optimal combination of chemotherapyand radiotherapy, further trials, of directcomparisons, are required.
Similarly, a meta-analysis of individual patientdata to evaluate the effect of different cytotoxicchemotherapy regimens on patients with non-small cell lung cancer stated that from the trialsincluded it was not possible to recommend oneparticular regimen over another, and that “Furtherrandomised trials are needed to determine whichregimens are the most effective of the modernchemotherapies studied”.32
Rossouw examined angiographic trials to assess theoverall effects of lipid reduction on angiographicoutcomes and clinical events.33 The trials includedin the review compared various interventions(lifestyle changes, resins, statins, combinations ofdrugs, and surgery) with control groups. Theauthor did not discuss the possibility of biasoccurring due to the use of indirect comparisons.However, the review demonstrated no evidence ofa class effect, with all classes of interventionappearing to have beneficial effects on bothangiographic and cardiovascular outcomes.
A review of antiemetic drugs for the prevention ofvomiting following paediatric strabismus surgerywas conducted by Tramer and colleagues.11 Thereview included 24 RCTs that were placebocontrolled or no-treatment controlled, or includedan unspecified control group. The drugsexamined were droperidol (varying doses),metoclopramide (varying doses), dixyrazine,ondansetron, lignocaine, hyoscine, atropine,lorazepam and propofol. Only three of the drugs(droperidol, metoclopramide and propofol) wereused to draw conclusions, owing to insufficient
Health Technology Assessment 2005; Vol. 9: No. 26
11
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TABLE 3 Reviews using indirect comparisons alone
No. of Appropriate interpretation studies of results
Yes No Unclear
Adjusted IC 15 6 3 6Naive IC 8 – 7 1
data for the remaining drugs. Adjusted indirectcomparisons were made for these three drugs,with odds ratios and 95% confidence intervalsbeing calculated using a fixed effects model, andNNT also calculated. The findings for droperidolsuggested a dose–response relationship(10–75 �g kg–1), with 75 �g kg–1 being the onlydose to have an odds ratio greater than one. Theauthors do interpret the result with caution;however, this caution is mainly due to the smallsample sizes being examined and not the potentialbiases that can occur through indirect comparisons.
The appropriateness of the conclusions is unclearin five other systematic reviews also presentingadjusted indirect comparisons. For example, Kochand colleagues34 conducted a review of H2 blockersand misoprostol. The trials included had to have aplacebo arm to act as the common comparator. Therate difference for each active drug in comparisonto placebo was calculated, as were NNTs. Forestplots were presented. The authors claimed that“gastric ulcer was found to be significantly reducedby misoprostol – both in short-term and long-termNSAID treatment – but not by H2 blockers”. In thediscussion, the authors did not explicitly discussbiases that can occur through indirectcomparisons, but they did highlight the fact thatthere were differences in the characteristics ofpatients studied in the misoprostol and H2-blockertrials (those included in the misoprostol trials wereat higher risk of gastric ulcer).
Holme reviewed the association between totalmortality outcome or coronary artery disease(CAD) incidence and the amount of cholesterolreduction in randomised cholesterol-loweringtrials that were performed before and afterinclusion of statin trials.35 An adjusted indirectcomparison was undertaken comparing diet,statins and hormones. The common comparatorwas fibrates. Multiple regression analysis wasundertaken, adjusted by cholesterol changes andbaseline risk of CAD. The statistical analyses weredone by weighted multiple linear regressionmodels with a fixed effects variance assumption.Diet versus fibrates gave an odds ratio of 0.975 fortotal mortality and 1.07 (p < 0.05) for CADincidence. Odds ratios of 1.088 and 1.185(p < 0.05) were obtained for hormones versusfibrates, and statins versus fibrates showed an oddsratio of 0.833 (p < 0.05) and 0.875 (p < 0.05) fortotal mortality and CAD incidence, respectively.The authors concluded that “Fibrate trials as agroup had the least favourable outcome profilesfor CAD and all cause mortality of all other drugtrials (except hormones)”.
Garg and Yusuf7 conducted a systematic reviewevaluating the effect of ACE inhibitors on mortalityand morbidity in patients with symptomaticcongestive heart failure. Thirty-two placebo-controlled trials of ACE inhibitors, including 7105patients, were used to make adjusted indirectcomparisons of estimates of effect for the differentagents evaluated. Trials making head-to-headcomparisons of different ACE inhibitors wereexcluded from the review unless they also includeda placebo group. The authors state that “Similarbenefits were observed with several different ACEinhibitors, although the data were largely based onenalapril maleate, captopril, ramipril, quinaprilhydrochloride, and lisinopril”.
A systematic review to compare the effectivenessand safety of oral tramadol with standardanalgesics using a meta-analysis of individualpatients’ data included 3453 postoperativepatients.36 Tramadol (50, 75, 100, 150 and 200 mg), codeine (60 mg), aspirin (650 mg) pluscodeine (60 mg) and acetaminophen (650 mg)plus proxyphene (100 mg) were all compared withplacebo, and then adjusted indirect comparisonsmade. Relative risks and NNT were presented with95% confidence intervals, and illustratedgraphically. Again, the authors do not discuss thepotential biases associated with indirectcomparisons. Head-to-head comparisons werepossible in certain cases, although data were notpresented.
The sixth study for which it was unclear whetherthe interpretation of the results was appropriate ornot was conducted by Lefering and Neugebauer.39
They compared studies of low-dose corticosteroidversus control with studies of high-dosecorticosteroid versus control. The outcomeassessed was mortality. The control groups wereunspecified, and there was wide variation in themortality rates among the control groups (7–69%).Pooled rate differences were calculated for the low-dose and high-dose studies, and the resultscompared. Low-dose corticosteroids showed aneffect of –1.9% (95% CI –20.0 to 16.2%), whilehigh-dose corticosteroid trials had a pooled effectof 3.6% (95% CI –2.5 to 9.8%). The findings wereinconclusive and the authors state that “Neitherthe type of steroid used nor the separation intolow-dose or high-dose regimen indicated aremarkable difference between the steroid groupand control group.” No discussion about the roleof indirect comparisons was presented.
Boersma and co-workers examined earlythrombolytic treatment in acute myocardial
Indirect comparisons in published systematic reviews
12
infarction and used indirect comparisons toexamine the relationship between time totreatment, from onset of symptoms, and mortalityup to 35 days.9 Odds ratios were calculated andpresented graphically. Both linear and non-linearregression analyses were undertaken. The authorsconclude that the beneficial effect of fibrinolytictherapy is substantially higher in patientspresenting with 2 hours after symptom onsetcompared with those presenting later. They didnot mention the potential problems of indirectcomparisons. However, a previous meta-analysis ofthe same studies argued that “if patient categoriescan be arranged in some meaningful order then… may be reasonably reliably informative …”.47
The reviews illustrate the usefulness of indirectcomparisons when there is a lack of directcomparison information.
The purpose of the systematic review conductedby Aro37 was to compare the cardiovascular andall-cause mortality of two oestrogen regimens[polyestradiol phosphate (PEP) alone, or incombination with oral ethinyl estradiol (EE)] andorchidectomy with those of the Finnish malepopulation. The review also compared the age-standardised mortality for all three treatmentforms. Only two trials were included in the review,Finnprostate I (PEP plus EE versus orchidectomy)and Finnprostate II (PEP versus orchidectomy).Age-specific person-years at risk were computedfor each treatment group at 5-year intervals. Theauthors concluded that “intramuscular PEPmonotherapy is associated with low cardiovascularmortality and with an all-cause and prostaticcancer mortality equal to orchidectomy”. Theresults and conclusions of the review arequestioned in an article by Ekbom and Taube,48
who highlight the fact that the two RCTs wereconducted in different years and with differentinclusion criteria. Indeed, there were statisticallysignificant differences between patientcharacteristics in the two trials. Ekbom and Taubesuggested that the observed low mortality in PEPalone may be due to “flaws in the methodology”.48
The objective of the meta-analysis carried out byPoynard and colleagues was to comparelansoprazole with raniditine or famotidine in acuteduodenal ulcer, and to compare indirectlylansoprazole with other drugs (omeprazole,nizatidine, cimetidine and sucralfate).38 Raniditineor famotidine was used as a common comparator.Four-week healing rates (OR, 95% CI) werecalculated for each drug in comparison to thecommon comparator, and ranked according toefficacy. The authors mention an RCT directly
comparing lansoprazole versus omeprazole inacute duodenal ulceration; however, this trial isnot included in the meta-analysis.
Naive indirect comparisons (see Appendix 3,Table 22)Eight studies, presenting results of indirectcomparisons alone, used the naive approach.6,40–46
None of the studies presented results from directcomparisons (even if this were possible), andpotential biases associated with indirectcomparisons were rarely discussed.
Coulter and colleagues41 reviewed RCTs of drugsused to treat menorrhagia. A variety of NSAIDs,antifibrinolytics, hormones and intrauterinedevices was examined. The drugs were listedaccording to the percentage reduction inmenstrual blood loss. The review did not presentthe data from the head-to-head comparisons madewithin some of the RCTs. Half of the trialsincluded a placebo group and could have beenused to carry out an adjusted indirect comparison,but the results of the placebo controls were notreported.
The comparative efficacy and toxicity of secondline drugs in rheumatoid arthritis was examinedby Felson and colleagues.6 For each trial, thetreatment arms of interest were identified anddata extracted. To compare the drugs they usedanalysis of variance, weighted by treated groupsize, multiplied by study quality and adjusted forthose covariates that had a significant associationwith each outcome (tender joint count, erythrocytesedimentation rate and grip strength). For eachoutcome auranofin was shown to be weaker thanthe other second line drugs (methotrexate,injectable gold, D-penicillamine, sulfasalazine andantimalarial drugs). It is unclear whether theadjustment for covariates actually reduced orincreased biases. Within-study comparisons were ignored, as were studies comparing drugsdirectly.
The review by Felson and colleagues6 was quotedby Imperiale and Speroff45 in support of usingnaive indirect comparisons of RCT data. Theyundertook a meta-analysis to examine the efficacyof thromboprophylaxis following total hipreplacement. The naive indirect comparison usedin the review was based on the premise that thetreatment groups were clinically homogeneous incomposition. The authors give the impression thattheir conclusions were based on evidence fromRCTs, even though only between-studycomparisons were made.
Health Technology Assessment 2005; Vol. 9: No. 26
13
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Bansal and Beto42 compared the efficacy oftherapeutic agents used in the treatment of lupusnephritis using outcomes of end-stage renaldisease (ESRD) and total mortality, using the samemethod of indirect comparison as Imperiale andSperoff.45 They undertook a simple indirectcomparison of results of different arms across thestudies included in the review. Theappropriateness of pooling the data was examinedby two methods (z-score and heterogeneity test),but the results of the tests were not presented. Thereview included RCTs and quasi-RCTs, but theresults of the studies were used to make between-study comparisons only, therefore losing the powerand rigour of the randomisation process. Again,the authors do not mention the potentialproblems associated with such indirectcomparisons, and no evidence from directcomparisons is provided.
Similar problems occur in a review ofantihypertensive agents to reduce left ventricularhypertrophy.43 The study had strict inclusioncriteria in that only double-blind RCTs wereincluded. However, analysis was undertaken bycombining all treatment arms of the same drugclass and weighting them according to the numberof patients in each individual study, thus breakingthe randomisation procedure. The authors dodiscuss the importance of randomisation and thevalidity of studies, although this is not taken intoaccount in the analysis. Within-study comparisonswere ignored and no attempt was made to carryout an adjusted indirect comparison using acommon placebo group.
Unge and Berstad44 studied anti-Helicobacter pyloriregimens. They included both RCTs andobservational data, and pooled data into groupsaccording to the combination of drugs used,regardless of, for example, dosage or duration.The authors argued that “a formal meta-analysis isof limited value due to the substantial variation oftherapeutic options”. There was no discussion ofpotential biases, and direct comparisons were notincluded in the review.
Chiba and colleagues40 conducted a meta-analysisto evaluate the speed of healing and symptomrelief in grade II–IV gastrooesophageal refluxdisease (GORD). The review included onlyrandomised trials, but did not directly comparedifferent interventions. Data from the includedstudies were grouped by drug class, decided apriori to be placebo, proton pump inhibitors(PPIs) and H2-receptor antagonists (H2RAs). Foreach study arm the overall healing proportion
reported at the final evaluation time-point wasused to calculate the overall healing proportion to12 weeks. Data were pooled within each drug classirrespective of dose, duration of treatment orspecific drug. Groups were then compared usinganalysis of variance (no further details given).Within-study comparisons were not made,although sufficient data to make directcomparisons were presented (see Chapter 6 forfurther details).
A review comparing the antihypertensive efficacyof available drugs in the angiotensin II antagonist(AIIA) class included 43 trials.46 Most of the trialswere placebo controlled, although some werehead-to-head trials. Analysis was based ontreatment arms, regardless of which othertreatments were included in the trials. Theestimate of blood pressure reduction assessedwithin the review is likely to be overestimatedbecause of regression to the mean effect, as theplacebo group (or other comparators) wasignored. The review did not include anyrecognition of the weakness of this approach.
SummaryIndirect comparisons are commonly used forevaluating the relative effectiveness of alternativeinterventions. Approximately 9.5% (31/327) ofmeta-analyses of RCTs identified through DAREincluded some form of indirect comparison. Afurther five reviews including some form ofindirect comparison were identified. The majorityof the reviews included in this chapter werepublished before 1998, owing to the nature of thesearching. An update search was not undertakenas it was felt the sample of reviews was sufficientlylarge. In addition, there was no reason to supposethat the frequency with which indirectcomparisons are used in meta-analyses has altered.
The methods used for the indirect comparisonsincluded the naive indirect comparison (11/36,31%) and the adjusted indirect comparison (25/36,69%). Although the identified meta-analyses oftenincluded only randomised trials, the strength ofthe randomisation procedure is completely brokenwhen making naive or unadjusted indirectcomparisons, thus providing data that are perhapsequivalent or even inferior to those obtainedthrough non-randomised studies. Such indirectcomparisons may be subject to bias (especiallyselection bias) compared with head-to-headrandomised comparisons as the benefit ofrandomisation does not hold across trials. In 50%
Indirect comparisons in published systematic reviews
14
(18/36) of the systematic reviews examined therewas no mention of the potential biases associatedwith the findings of the indirect comparisons.
Results obtained through indirect comparisonwere not always consistent with the findingsobtained by direct comparisons. Thirteen (36%) ofthe identified systematic reviews used both directand indirect comparisons. The results of directand indirect comparisons within these 13 reviewswere different in three meta-analyses and similar(same direction but not necessarily magnitude ofeffect) in eight meta-analyses. Because data fromthe same trials have often been used in both directand indirect comparisons in a review, the
difference between the direct and indirectestimates may have been underestimated in thereviews.
The indirect comparisons were sometimes carriedout implicitly and the results of indirectcomparisons interpreted as if from directcomparisons within randomised trials. Further, the findings of direct comparisons were sometimes ignored, even when data wereavailable. The misuse of indirect methods andinappropriate interpretation of results of indirectcomparison may result in misleading assessmentsof relative efficacy of competing healthcareinterventions.
Health Technology Assessment 2005; Vol. 9: No. 26
15
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Systematic review of the literatureA systematic review of the research literature wasundertaken to identify and evaluate statisticalapproaches to the analysis of indirectcomparisons. The CRD guidelines were used as astarting point for the systematic review protocol.49
However, as the review was a methodologicalreview rather than a review of the effectiveness ofan intervention, it was noted that these guidelineswould not be strictly applicable and would needadapting. In particular, the review was notrestricted to consideration of publicationsidentified by the formal searching.
Search strategyA thorough search was undertaken, including bothcomputerised and manual searching, to identifyrelevant literature. It was recognised that relevantmaterial may also be published in textbooks.Constructing a literature search strategy formethodological papers is problematic owing to thelack of suitable indexing terms in the electronicdatabases. The development of any search strategyis essentially an iterative process whereby theinitial strategy is refined and developed accordingto its recall and precision. The development of thesearch strategy in this instance involved five roundsof searching MEDLINE, reviewing the results andadapting the search strategy. The MEDLINE searchwas run from 1966 to March 1999. It was updatedto include records published by February 2001.Details of this process are given in Appendix 5.
Search strategy 5 (Appendix 5) was used as a basisfrom which to develop strategies to use in otherdatabases. Amendments were made regardingthesaurus terms and subject indexing whereappropriate. In general, the strategies used reliedheavily on the use of free text terms because of alack of adequate MeSH or other thesaurus termsto describe the concepts of research methodology.The strategies for PsycLIT (1887 to February2001), EMBASE (1980 to April 1999), ERIC (1966to February 1999) and MathSCI (1940 toSeptember 1999) are listed in Appendix 6.
The project team considered that it might beuseful to carry out searches of some of the
databases covering the agricultural literature.Potential databases to search were identified asbeing BIOSIS, Agricola, Agris International andCAB Abstracts. Some test search strategies wererun on the BIOSIS database, but the recordsretrieved were studies reporting the results ofsystematic reviews rather than articles discussingmethodological issues. It was decided not topursue this source further.
In addition to the electronic searches, thefollowing key journals were initially handsearched:
� Statistics in Medicine (1984–1999)� Controlled Clinical Trials (1984–1999)� Journal of Clinical Epidemiology (1991–1999)� Psychological Bulletin (1995–1999)� Psychological Methods (1995–1999).
The searches of Statistics in Medicine, ControlledClinical Trials and the Journal of ClinicalEpidemiology were subsequently updated to July2004. The reference lists of all relevant articleswere examined to identify further studies andattempts made to uncover grey and unpublishedliterature. Contact was also made with thoseworking in the field, both nationally andinternationally, including the Cochrane MethodsGroups for Empirical Methodological Studies (nowthe Cochrane Methodology Review Group),Statistics, and Individual Patient Data Meta-analyses. Papers identified ad hoc were alsoincluded.
Finally, given the difficulty in identifying relevantarticles through electronic searching, an updatesearch was conducted in the form of a citationsearch run on all of the relevant papers previouslyidentified (April 2004).
Inclusion criteriaThe searches were used to identify all papers thataddressed the following issues:
� methodology for carrying out indirectcomparisons
� methodology for identifying and assessingbiases that arise from indirect comparisons
� methodology for avoiding (or even adjustingfor) biases arising from indirect comparisons.
Health Technology Assessment 2005; Vol. 9: No. 26
17
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 4
Statistical methods for indirect comparisons
To provide examples for the empirical work (seeChapter 5), any reviews meeting the followingcriteria were also identified:
� systematic reviews reporting both direct andindirect evidence
� examples of systematic reviews where treatmentarms from different RCTs are compared (i.e.randomisation has been broken).
Assessment of relevanceAll titles and abstracts of articles identifiedthrough the searches were screened independentlyby two reviewers. All papers considered to berelevant by at least one reviewer were retrieved forfurther appraisal. Retrieved articles were againassessed independently by two reviewers and anydisagreements taken to a third party.
Results of searchesIn total, 3034 titles and abstracts were identifiedthrough the electronic searches. Followingscreening 29 full text articles were obtained, andfurther screening resulted in the identification ofonly six papers for inclusion.14,50–54 The majorityof papers included in this chapter, including twobook chapters,55,56 were identified on an ad hocbasis.
As noted below, the problem of indirectcomparisons is closely related to other problemsfor which articles were not specifically beingsought: meta-regression, subgroup analysis, andactive–control equivalence trials. Although thesearch certainly did not detect all relevantpublications, it is probable that all of the mainmethods of analysis were covered.
Statistical methods for indirectcomparisonsBackgroundAs noted in Chapter 1, an indirect comparisoninvolves the comparison of the results from sets ofstudies making different treatment comparisons. Itis thus a combination of two (or more) meta-analyses and thus shares all the methodologicaldifficulties associated with the use of meta-analysisto combine the data from several studies. Multiplestudies may vary in numerous ways including, butnot restricted to, the precise interventions beingcompared, characteristics of participants,methodological quality (including aspects oftreatment allocation and blinding), concomitantinterventions, length of follow-up, outcomemeasures and amount of loss to follow-up. In
addition to concerns about the comparability oftrials making the same comparison, thecomparability of different sets of trials must alsobe considered. Two further issues are the choice ofsummary outcome measure57 and theheterogeneity of the results of the trials, issues thatmay be related.
The statistical methods for carrying out anindirect comparison can be derived from methodsfor investigating heterogeneity in a meta-analysis.In meta-analysis it is common to examineseparately subgroups of trials, for example,defined by the clinical or demographiccharacteristics of participants. Such analysis mayserve as a means of exploring and possiblyexplaining statistical heterogeneity of results.58–60
Subgroups may also be defined by characteristicsof one of the treatments. For example, trials mayhave used two or more different doses of an activetreatment, or members of a class of drugs, or mayhave compared a single treatment against differenttypes of standard or inert treatment. Although it iscommon to perform separate meta-analyses foreach subgroup, a formal approach requires acomparison of the treatment effects in each subsetof trials, which assesses the treatment by subgroupinteraction. When subgroups define differenttreatments the analysis is exactly the same as anindirect comparison. Comparison of twoindependent estimates is a standard statisticalanalysis, yielding an estimate of the comparativeeffect in the subgroups, with a confidence interval,and a p-value. The calculation is slightly morecomplex when the analyses have been of relativemeasures (odds ratio, relative risk) and thus on thelog scale.61 Alternatively, a regression model canbe used to examine whether heterogeneity isexplained by one or more study characteristics,known as meta-regression.60,62 The adjustedindirect comparison can thus be seen to be thesimplest form of meta-regression, with a singlebinary trial factor.
Indirect comparisons are inherent in the use ofactive–control equivalence drug trials, in which theaim is to demonstrate that a new active drug isequivalent to (i.e. not very different in efficacyfrom) an already available active drug, which itselfhas been shown to be superior to placebo.63,64
Although indirect comparisons can arise in thesedifferent contexts, the statistical analysis optionsare the same whichever scenario applies. Theinitial focus is on the simplest, and common, casein which results are available from one or moreRCTs comparing A versus B (AvB) and one or
Statistical methods for indirect comparisons
18
more RCTs comparing AvC, and the aim is toestimate the difference in effect of BvC. Thefollowing five sections consider different statisticalapproaches that have been suggested for indirectcomparisons. First, statistical methods usingaggregate data are discussed for each study,mainly simple two-step methods. Second,modelling approaches based on individual patientdata (IPD) are considered. Although full IPD arerarely available, in the case of binary outcomes thefrequencies in a 2 × 2 table are effectivelyindividual observations and thus such studies areamenable to a wide range of possible analysismethods. In these first two sections the emphasisis on classical frequentist methods that are in wideuse and do not require specialist software. Third,Bayesian and likelihood-based methods areconsidered. Fourth, the assumptions that underlieall indirect comparisons are explored. Finally,various extensions of the approach, including thecombination of direct and indirect evidence in asingle analysis, are discussed.
The deeply flawed ‘naive method’, in which therandomisation is broken (described in Chapter 1)is not considered in this chapter, although itsperformance is evaluated in Appendix 7 and inChapter 5.
Classical methods using aggregate dataMeta-analysis using summarised (or aggregate)data extracted from published studies is a two-stage process involving the extraction orcalculation of an appropriate summary statistic foreach of a set of studies, followed by the weightedcombination of these statistics to provide anoverall estimate of effect (i.e. the contrast betweentwo treatments). Many familiar methods of meta-analysis are of this type, includingMantel–Haenszel methods for binary data and thegeneric inverse variance method, used for anytype of data.60
For an indirect comparison these ideas areeffectively extended to a three-step approach, inwhich the third step is to combine the results oftwo separate meta-analyses into an overallcomparison. A standard statistical result is that thevariance of the difference between twoindependent estimates is the sum of the twovariances (the variance is the square of thestandard error).65,66 This relation underlies thetwo-sample t-test, for example. It also applies toan indirect comparison, as the two sets of data arefrom different studies. Thus, given two estimatedeffects �AB and �AC for comparisons of AvB andAvC, respectively, the effect for the comparison
BvC is estimated as �BC = �AB – �AC, and var(�BC)= var(�AB) + var(�AC). A 95% confidence intervalfor �BC is obtained as �BC ± 1.96√[var(�BC)]. Theestimates of effect, denoted �, relate to the scaleon which the data would be analysed; examplesare risk difference, log risk ratio and log oddsratio for binary data, means for continuous data,and log hazard ratio for time-to-event data. Themethod in which two separate meta-analyses arecombined is referred to as an adjusted indirectcomparison. The adjustment here is for a commoncomparison group. The possibility of furtheradjustment for covariates is discussed later.
Because the basic method relies on a standardstatistical result, it is not possible to say who firstapplied it in the context of indirect comparison.The earliest methodological discussion found hereis by Eddy and colleagues in 1992,67 but earlierapplied examples are likely to exist, especially inthe framework of comparing subgroups of trials ina meta-analysis.
The first methodological article explicitlydiscussing adjusted indirect comparisons seems tobe that of Bucher and co-workers.14 For trials witha binary outcome they suggested combining oddsratios from separate meta-analyses, ORAB andORAC, so that logORBC is estimated as logORAB –logORAC, and its variance as var(log ORBC) =var(logORAB) + var(logORAC). From thesecalculations it is simple to obtain a confidenceinterval for logORBC and hence, by transformation,an estimate of ORBC with a confidence interval.The adjusted indirect comparison method is quitegeneral, and this formulation is clearly a specificexample of the general method described above.The approach has been used to combine trialsusing other effect measures, such as risk ratios,68
risk differences,68 hazard ratios or means.69
Fisher and colleagues discussed the use of adjustedindirect comparisons to address the specificquestion of estimating the superiority of a drug toplacebo when no placebo-controlled trials havebeen done.70 This situation arises when one drughas been shown to be effective and it becomesunethical to conduct further placebo-controlledtrials of new agents. Baker and Kramer71 presentthe same idea from a conceptual perspective. Allof their examples are hypothetical and they do notdiscuss the specifics of analysis.
In a meta-regression analysis, the estimatedtreatment effect is modelled as a function of oneor more study characteristics as predictorvariables.59,62 Least squares regression or a
Health Technology Assessment 2005; Vol. 9: No. 26
19
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
maximum likelihood method is used and eacheffect estimate is weighted by the reciprocal of itsvariance. For a single binary study factor, thisanalysis is exactly the same as an adjusted indirectcomparison. The method can be used to compareresults from trials comparing AvB and AvC wherethe study characteristic is the choice ofcomparator. The estimated effect comparing BvCis the coefficient for the indicator variabledenoting which comparison was made.
All of the authors cited used a fixed effect meta-analysis to combine the results of trials of the samecomparison, but the same method can be applied tocombine estimates from two random effects meta-analyses. Meta-regression can also be performedallowing estimation of a random effect to describeresidual heterogeneity. Only Fisher and co-workersdiscussed this possibility, and they gave variousreasons for preferring a fixed effect meta-analysis.70
In all meta-analyses a choice has to be madebetween a fixed effect or random effects analysis; itis not specific to indirect comparisons or meta-regression. However, in this more complex situationthere is more than one way to implement a randomeffects analysis. In a random effects analysis thedifferent trials are assumed to be estimatingdifferent, but related quantities that are distributedaround some typical value with a variance that isestimated from the data (�2). In an indirectcomparison, for some models �2 is set the same foreach component comparison of two treatments,while for other models it is estimated separately.
Methods using generalised linear(mixed) modelsThe previous section described the analysis of anadjusted indirect comparison in terms of weightedcombinations of meta-analyses of sets of trialsmaking different treatment comparisons. Anindirect comparison can also be analysed in aregression framework using generalised linearmodels, but this approach requires the availabilityof IPD.
For a binary outcome, the 2 × 2 frequency tablefor a trial effectively reflects IPD and so such datacan be analysed using logistic regression, asdiscussed by Hasselblad.52
The same principles apply to meta-analyses ofstudies with continuous or time-to-event outcomes,but IPD will need to be obtained from the authors,which is generally a major undertaking.72
Statistical methods for these outcomes have beenconsidered by Higgins and colleagues73 and TudurSmith and colleagues.74
For binary and other outcomes, random effectsanalysis requires fitting generalised linear mixedmodels (GLMMs),75 which can be done forexample in Stata (Release 8.0; Stata Corporation,Texas, USA, 2003) using the program gllamm, orin SAS (version 9.1) using PROC MIXED (for acontinuous outcome) or PROC NLMIXED (for abinary outcome).
These general methods can be used to handleboth simple indirect comparisons and morecomplex networks of treatment comparisons, asdiscussed below (section ‘Extensions to morecomplex situation’, p. 22).
Bayesian and likelihood-based methodsSome authors have presented flexible Bayesianmethods that can be used to analyse indirectcomparisons as well as more complex datastructures. Higgins and Whitehead53 used aBayesian approach to investigate the relativeeffectiveness of �-blockers and sclerotherapy toprevent bleeding in patients with cirrhosis.Twenty-four trials had compared one of the activetreatments against control, and two trials hadthree arms.
The confidence profile method67 is a very generalmethod for combining almost any evidencerelating to a particular question. As well asincorporating studies making different treatmentcomparisons it can encompass different designs,outcomes and measures of effect, and allowsexplicit modelling of biases. Although oftenconsidered as a fully Bayesian model, it can alsobe formulated without prior distributions andfitted using maximum likelihood.
Ades76 recently proposed a Bayesian Markov chainMonte Carlo method to handle the general case.His method of multiparameter evidence synthesisis also very general in allowing other types ofevidence to be included in the model. Earlier,Dominici and colleagues51 presented ahierarchical Bayes grouped random effects model.They used Markov chain Monte Carlo simulationto apply this approach to a highly complex set of46 trials evaluating treatments for migraine fromthree classes: �-blockers, calcium channel blockersand biofeedback therapy.
Sutton and colleagues77 give an overview of theadvantages and disadvantages of Bayesianmethods in meta-analysis. One advantage of fullyBayesian methods is that they place a distributionon the heterogeneity �2 and do not assume thatthe estimate of �2 is without error.78 This issue is
Statistical methods for indirect comparisons
20
especially relevant when there are rather few trialsmaking a particular comparison.
Fully Bayesian models offer the greatest flexibility,particularly in modelling random effects. Theyrequire specialist software and a deep statisticalunderstanding, taking them beyond the scope ofmany research groups. Attention in this study isfocused on simpler approaches, which can be usedfor many of the more common scenarios, but thereviewers indicate later where complex methodsare needed.
Assumptions Many study and patient factors (case-mix)influence the outcome of patients in a specifictreatment group in a particular trial. The essenceof the indirect comparison is that the effect for thecomparison BvC is estimated using a commoncomparator A by contrasting the estimated effectsfor comparisons of AvB and AvC. Implicit here isthe notion that the variation in the observedresults for patients treated with the commoncomparator, treatment A, will (to some extent)account for differences between studies inmethodology, case-mix, and so on. The validity ofthe indirect comparison will thus depend on theextent to which this assumption is reasonable.Adjustment for a common comparator (A) will beexpected to reduce, if not remove bias in thecomparison of B and C.
The underlying assumption of a fixed effect meta-analysis is that the various studies are allestimating the same effect, such as the effect of Arelative to that of B. The same considerationsapply to trials comparing AvC. An indirectcomparison shares with other observationalepidemiological investigations the risk of bias byconfounding.62 The key additional assumption ofan indirect comparison using the results of trialsof AvB and AvC is that there should be noimportant differences between the two sets of trialswith respect to aspects that could influence (bias)the estimated treatment effect of BvC, that is,there is no confounding of the comparison bysome trial characteristic. As an example, Lim andcolleagues79 reported a comparison of low- andmedium-dose aspirin therapy after coronarysurgery using an indirect comparison of placebo-controlled trials, with outcome evaluated byangiography. There were three randomised trialsof low-dose aspirin, with patients receivingangiography at an average of 10, 130 and 180days, and two trials of medium-dose aspirin, withangiography at an average of 363 and 367 days.Subsequent correspondence80 considered the
possible relation between the time to angiographyand the observed effect of aspirin.
Another formulation of this argument is that thetwo sets of trials should be exchangeable, in thesense that there is no reason to suppose that theresults as a whole would be different had thevarious trialists kept the same protocol andpatients, but chosen to study a different treatmentcomparison.81 Baker and Kramer71 showgraphically that the validity of the indirectcomparison depends on the consistency of thetreatment effect over settings with different eventrates (different case-mix). Phillips82 consideredsome ways in which this assumption might fail.Clearly, one important way in whichexchangeability could fail is when the treatmenteffect is influenced by some factor that itself variesacross the different treatment comparisons, suchas the clinical setting or length of follow-up.Adjustment for study covariates can help to reducesuch an effect and make exchangeability morelikely. The analysis is only possible when the factorvaries within each set of trials.
Similar issues have been discussed in the contextof non-inferiority trials, in which a new treatmentis compared with a standard treatment with theaim of showing that the new treatment is no lesseffective than the standard treatment, within someprespecified margin (these trials are also calledactive–control equivalence trials).64,83,84 Often thestandard treatment will previously have beenshown to be better than placebo. In essence, thereis an implicit indirect comparison when inferringfrom a trial that shows non-inferiority that the newtreatment is better than placebo. It is recognisedthat the validity of such an inference relies onaspects of the non-inferiority trial being closelysimilar to the prior placebo-controlled trials of thestandard treatment. Particular examples includepatient characteristics, treatment dose, outcomemeasures and length of follow-up.85
In addition to concerns about the averagetreatment effect, the issue of heterogeneity has tobe considered. Hirotsu and Yamada54 presented amethod that is equivalent to adjusted indirectcomparison using inverse variance meta-analysis.They suggested testing for homogeneity of trialresults both within treatment comparison andbetween direct and indirect estimates beforepooling.54 However, an indirect comparison doesnot require homogeneity, and modellingapproaches exist that include random effects termsthat estimate the degree of heterogeneity in thecomparisons. The model of Higgins and
Health Technology Assessment 2005; Vol. 9: No. 26
21
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Whitehead assumes that the degree of heterogeneityis similar in all comparisons,53 whereas othermethods allow separate estimates of heterogeneityfor each comparison. Further, in most situationsthere are too few trials for each paired comparisonto allow reliable assessment of whether there isexcess heterogeneity, or estimation of separaterandom effects for each component comparison.
As with any meta-analysis, there is the possibilityof fitting random effects models to take account ofbetween-trial heterogeneity. Under a randomeffects model the treatment effect is allowed tovary among trials making the same treatmentcomparison (usually assuming that the distributionof effects is normal). Models that apply therandom effect to each study arm rather than eachcomparison between arms are not sensible. Inmany situations it may be reasonable to assumethat the variability of the treatment effects is thesame for all paired treatment comparisons.53
Several of the papers describing the analysis ofindirect comparisons have given rather littleconsideration to the underlying assumptions, andthose statements that have been made are oftenquestionable. For example, the only assumptionmentioned by Hasselblad and Kong68 is that thepopulations of the individual studies contain somesubpopulation in common. While this is sensible,and echoes concerns about equivalence trials, it isby no means a sufficient requirement. Bucher andcolleagues14 noted that “The only requirement isthat the magnitude of the treatment effect isconstant across differences in the populations’baseline characteristics”. In their appendix theymentioned the additional assumption of notreatment by study interaction in each set of trials,but as noted above, the requirement is rather oneof equal heterogeneity.
The following section considers various morecomplex situations involving multiple comparisonsof multiple treatments. It is clear that theassumptions become more numerous and tenuousin relation to increasing complexity of the datastructure. Most obviously, the notion ofexchangeability may need to extend across manysets of trials (which may have been carried outacross a long period).
Lastly, in a meta-analysis of trials with a binaryoutcome, it is well known that the choice of effectmeasure may have a considerable impact on theanalysis, and also on the degree of observedheterogeneity. Empirical studies show that forbinary outcomes measures of relative effect (odds
ratios and relative risk) are more likely to beconsistent across trials than measures of absoluteeffect (risk difference).65,86 This issue is of majorrelevance to indirect comparisons of two or moresets of trials. None of the papers cited aboveconsidered the impact of the choice of effectmeasure in making indirect comparisons, with theodds ratio used in almost all cases. Anotherassumption of an indirect comparison meta-analysis, therefore, is that the effect measure isappropriate.
Extensions to more complex situationsThis section considers six ways in which theavailable data may be more complex than thesimple case outlined above. First, three furthersituations are considered where all trials still havetwo treatment arms, and then two cases where atleast one trial has more than two arms. Lastly, thecase where study or patient characteristics aretaken into account is considered.
More than one common comparison treatment(e.g. AvB, AvC, DvB and DvC)All methods for adjusted indirect comparisonsextend simply to the case where there are two ormore sets of indirect evidence for the comparisonof interest, such as comparing B and C using datafrom trials comparing AvB and AvC and also trialscomparing DvB and DvC. Separate estimates ofBvC from adjusted indirect comparisons using twodifferent sets of two-arm trials can be combinedusing inverse variance weighting. The analysis ofsuch a set of trials is illustrated in Chapter 8.87
Lumley88 suggests comparing the results derivedfrom the different comparators, and suggests aparameter to measure the ‘incoherence’ of thesystem, which considers the consistency of a specificestimated contrast between two treatments withthe rest of the system (here the estimate from theother route). As with other tests in this context,there may be too few trials of each treatmentcomparison for this test to have much power.
Combining direct and indirect evidence (e.g. AvB,AvC and BvC)Often there is both direct and indirect evidence.Traditionally, meta-analysts focus only on directevidence and do not seek indirect evidence, orthey disregard it. Indirect comparisons havemainly been used when there is no directevidence, or perhaps very little. Thus, there arefew examples of indirect and direct evidence beingcombined. Such a combined analysis is consideredhere; whether such an analysis is sensible isconsidered in Chapter 8.
Statistical methods for indirect comparisons
22
The adjusted indirect comparison method doesnot explicitly generalise to allow inclusion of directevidence (RCTs of BvC). It is simple, however, tocombine the estimate from the adjusted indirectcomparison with data from one or more directcomparisons using inverse variance weighting.The separate meta-analysis estimates need notalso be obtained from inverse variance meta-analyses.
Hasselblad showed how logistic regression couldbe used to analyse a mixture of direct and indirectcomparisons.52 The approach works well for fixedeffect models. However, because of the way themodel is set up, the random effects model hedescribed allows the outcome per treatment armto vary randomly across trials rather than thetreatment effect itself, so that this approach is notrecommended. A proper random effects combinedanalysis requires the use of mixed models, asdiscussed above.75 This issue is also considered inAppendix 7.
Hirotsu and Yamada presented a fixed effectmethod for estimating odds ratios from a mixtureof direct and indirect comparisons, noting that theproblem can be seen as an example of anincomplete block design.54 Although presentedrather differently, their method is equivalent toadjusted indirect comparison (possibly combinedwith direct comparisons) using inverse variancemeta-analysis.
A sequence of comparisons (e.g. AvB, BvC, CvDand DvE)The case of a sequence, or chain, of pairedcomparisons may arise in the drug developmentprocess, particularly where the emphasis is onestablishing equivalence of new drugs rather thansuperiority. If A and B are found to bebioequivalent (often defined as an effect ratio inthe range 0.8–1.25), and from separate studies soare B and C and C and D, what can be said aboutthe equivalence of A and D?63
Such questions may arise primarily in smallbioequivalence studies looking at uptake of drugsvia measures such as total and peak exposure,63
but in principle can also arise in studies of clinicalequivalence, and may arise in superiority trialstoo. An example of such a chain is shown inTable 18 in Chapter 8.
A simple chain such as AvB, BvC, CvD and DvEcan be handled by most of the methods of analysisdescribed. For this example, the contrast AvE canbe evaluated via a combination of multiple
applications of the simple adjusted indirectcomparison.68
While such an analysis is clearly easy to carry out,it extends the assumption of exchangeabilityacross three or more sets of trials in the chain,which many may consider to be rather tenuous.
One or more multiarm trials (e.g. AvB, AvC andAvBvC)An extension of the previous case is when thereare multiarm trials comparing all three of thetreatments of interest. Although the standardapproaches to meta-analysis relate to two-armtrials, trials with three or more treatment arms aresurprisingly common, making up about a quarterof parallel group trials in a MEDLINErepresentative sample.89 Multiarm trials also causedifficulties in conventional meta-analysis.
The adjusted indirect comparison method cannottake account of trials with three or more armswithout either splitting or discarding groups.Multiarm trials can be handled using logisticregression, as this method treats the treatmentarm as the unit of data,52 and by more complexmethods.
Gleser and Olkin considered the case where thereis a set of RCTs in each of which one or more ofseveral active treatments has been evaluatedagainst a common control.55 They proposedregression models for obtaining estimates of eitherthe risk difference or (log) odds ratio betweeneach treatment and control, and also for contrastsbetween the active treatments. Their method isconceptually similar to the adjusted indirectcomparison.
When a single multiarm trial is used to estimatetwo or more paired comparisons those estimateswill be correlated when they contain a commonarm (e.g. AvB and BvC). This correlation wasincluded in the models presented by Gleser andOlkin55 and Higgins and Whitehead;53 it is nottaken into consideration in logistic regression.
An arbitrary set of all combinations The most general case allows for trials comparingvarious sets of two, three or more of manytreatments. (The further possibility of theinclusion of other trial designs, such as cluster orcross-over trials, is not considered here, althoughin principle these could also be added.) Anexample is used in Appendix 7 as the basis forillustration of some of the methods of analysis.This scenario goes beyond the main focus of this
Health Technology Assessment 2005; Vol. 9: No. 26
23
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
report, but such data sets can be analysed usingseveral very general methods, such as that of Eddyand colleagues.67
Lumley88 mentioned the possibility of includingmultiarm trials, but did not show how his networkmeta-analysis method extends to that case. Theassumptions of a single analysis of a complex setof trials are considerable, and the inner features ofthe data will be obscured. If such an approach isused it seems desirable also to examine simpleranalyses of subsets of treatments and to exploreconsistency of results.
Adjustment for study and/or patientcharacteristics Concerns about the validity of the assumptionsmay lead to the wish to incorporate study-levelcovariates in the analysis. These may relate toaspects of the study, such as length of follow-up orwhether or not double blind, or study-levelsummaries of patient characteristics such as meanage. Such adjustment may be felt to remove orreduce meta-confounding and make the studiesmore likely to conform to the assumption ofexchangeability. Adjustment will be more usefulfor study characteristics as there is very poorpower to detect the effect of patient characteristicsusing study-level summaries.90,91
Regression methods, including logistic regression,can be used to adjust for study characteristics. Inthe rare case where full individual patient data areavailable, then the whole analysis could beperformed using individual patient characteristics,for example using multilevel (hierarchical)modelling.92
Summary of available analysesIt has been noted that some complex statisticalmethods are available to analyse any set of trials.Such methods are inaccessible to manyresearchers, so it is useful to summarise the typesof data that can be analysed using simpler, widelyavailable methods.
The method of adjusted indirect comparison canbe extended to any case where all of the trials havejust two treatment arms, where necessary by usinginverse variance weighted combination of separateindirect estimates. For example, separate indirectand direct estimates of a common treatment effectevidence can be combined. Likewise, indirect anddirect estimates can also be combined using theresults from separate meta-regression analyses.Adjusted indirect comparison and meta-regressioncan be used to combine fixed or random effects
estimates from subsets of trials, but they cannothandle trials with more than two treatment arms(multiarm trials). However, an advantage of bothapproaches is that they can be used for any type ofoutcome measure, including odds ratios, relativerisk and risk difference for binary data as well asmeans or hazard ratios.
For a binary outcome, logistic regression can beused to perform a fixed effect analysis for all ofthe cases discussed in the previous section,including multiarm trials. Here the odds ratio isthe only available effect measure.
When there is at least one multiarm trial, randomeffects analysis requires hierarchical (mixed)models, and these can also be used for continuousoutcomes. Such models appropriately consider therandom effects to apply to the treatment effect, incontrast to some of the models discussed above.Their use ideally requires expert statisticalassistance.
As noted earlier, several more flexible but complexapproaches can be used to handle any mixture ofdata structures, although not all can deal with anytype of outcome measure.
Related literature The focus of interest here was the combination ofresults from RCTs, and the authors would notadvocate including data from uncontrolled studies.However, methods are available to combine datafrom RCTs with results of uncontrolled studies.Begg and Pilote93 proposed a random effectsmodel for combining controlled and uncontrolledstudies in the context of incorporating historicalcontrols into a meta-analysis of comparativestudies. Their method assumes a distribution forthe control group event rate (rather than for thetreatment effect, as in conventional random effectsmeta-analysis) and a fixed treatment effect. Therelative weight given to the controlled studiesdepends on the observed degree of heterogeneity.They illustrated their method using fourcontrolled trials (274 patients) and 12uncontrolled studies (1708 patients) comparingbone marrow transplantation and chemotherapyfor acute non-lymphocytic leukaemia. The methodwas extended by Li and Begg.94 Raghunathan95
presented a conceptually similar approach forcombining the results of case–control studies withdata from controls in previous studies.
Berkey and colleagues50 presented a method forcombining multiple outcomes in a meta-analysis,to allow a composite analysis when not all
Statistical methods for indirect comparisons
24
outcomes were assessed (reported) in all trials.Their generalised least squares regression modelalso extended to analysis of single outcomes fromtrials comparing multiple treatments, not all ofwhich were studied in each trial. They presentedan analysis comparing gold and auranofin forreducing tender joints in patients with rheumatoidarthritis. Of the 44 randomised trials, only ninecompared directly the two treatments of interest.Their method does not preserve the link betweentreatment arms in a given trial and thus cannot berecommended.
Lastly, Büchner and colleagues96,97 described theprospective design of several trials with a commonstandard arm, to allow indirect comparison via thecommon treatment. However, they suggest that ineach trial only a small proportion of patients arerandomised to the common arm and all thepatients across the various trials are used as thecommon comparator. This method mixesrandomised and non-randomised comparisonsand runs a serious risk of bias.98
CommentsAlthough indirect comparisons are common,surprisingly few methodological papers haveproposed or discussed methods for handling suchdata. Searching for such papers was hampered bythe lack of recognised terminology for indirectcomparisons; other terms used in the papersidentified include cross-trial (or cross-study)comparison,82,99 connected comparativeexperiment,54 network meta-analysis,88 mixedcomparison,76 and virtual comparison.100 Hardlyany of the papers identified cited any of theothers. It is thus quite possible that othermethodological papers were missed, although it isunlikely that there are important omissionsregarding methods of analysis.
Some valid approaches were identified foraggregate data that could be applied to simpleproblems using standard software: the adjustedindirect comparison, meta-regression and, forbinary data only, multiple logistic regression (butonly fixed effect models).
A particular advantage of a regression modellingapproach is the possibility to adjust for othervariables available for each study. The hope wouldbe that such adjustment may help to explain someof the heterogeneity within and between groups oftrials making the same comparisons. Adjustmentfor aggregated patient-level variables (such asaverage age) would be expected not to have mucheffect in comparison with adjustment with thesame variables at the individual level.90,91 All suchanalyses are open to all of the problems inherentin meta-regression analysis.62,91 The regressionapproach may be seen as a simple version of someof the more complex approaches that can be used.
The use of simple adjusted indirect comparisonsto combine two-arm trials is being used moreoften. Some of these methods can be extended tomore complex situations. At the extreme, someadvanced methods exist that can handle generalproblems of networks of comparisons, andsometimes also combine different outcomes, studydesigns and perhaps external information.Whether it is sensible to include such variedinformation in a single analysis is open toquestion. It would seem desirable that, if used,such complex analyses are supplemented bysimpler analyses.
Chapter 5 reports on empirical investigations intothe performance of several methods of makingindirect comparisons, using both fixed andrandom effects, with and without adjustment forstudy covariates.
Health Technology Assessment 2005; Vol. 9: No. 26
25
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Comparisons of indirect and directcomparisons, while interesting, cannot
provide reliable information about the propertiesof the indirect approaches. Apart from theproblem of small numbers of trials, the maindifficulty is the fact that the difference inestimated treatment effect between the two sets oftrials may be confounded with differencesbetween the studies.
To compare direct and indirect estimates withoutsuch confounding an extensive empiricalinvestigation was carried out using data from alarge multicentre randomised controlled trial, theInternational Stroke Trial (IST).101 The centres inthe study were grouped by region to representseparate trials and results from these ‘trials’ usedto generate both direct and indirect estimates ofthe same treatment comparison.
As all of the centres used exactly the sameprotocol, including eligibility criteria, the results ofthe studies can be said to represent a somewhatoptimistic case as, in general, further variabilitywill be introduced by variation in protocols acrossstudies.
The International Stroke Trial(IST)The aim of the trial was to assess the separate andcombined effect of aspirin (300 mg daily) and ofsubcutaneous heparin [5000 IU twice daily (lowdose) and 12,500 IU twice daily (medium dose)]administered for 14 days. Between January 1991and May 1996 19,435 patients with suspectedacute ischaemic stroke entering 467 hospitals in 36countries were assigned centrally to a treatmentusing minimisation within 48 hours of symptomsonset.
Details of the study methods and interventionshave been described elsewhere.101 In brief, using afactorial design, half of all patients were randomly
allocated to receive aspirin and half to ‘avoidaspirin’. For each of these two groups, half of thepatients were allocated heparin (low or mediumdose) and half to ‘avoid heparin’. Placebos werenot used.
The primary outcomes were death within 14 daysand death or dependency at 6 months. At6 months fewer patients in the aspirin group weredead or dependent [62.2% versus 63.5%, p = 0.07;a difference of 13 (SD 7) per 1000]. Afteradjustment for baseline stroke severity, the benefitfrom aspirin was statistically significant [14 eventsprevented per 1000 patients (SD 6), p = 0.03].Heparin was not found to have any effect (eventrate 62.9% in groups who did or did not receiveheparin). There was no detectable interactionbetween aspirin and heparin in the mainoutcomes.
Adaptations of the trialOutcome and treatmentDeath and dependence at 6 months wasconsidered as the main outcome. Aspirin versusheparin (either dose) was taken as the comparisonof interest. The four treatment groups werelabelled as follows:
� no treatment: A � aspirin: B� heparin: C� aspirin and heparin: D.
CountriesData from the different countries in the trial wereused to represent multiple studies in a meta-analysis. Only countries with more than 100patients for each arm were considered. As Italyand the UK enrolled a large number of patientsboth were split into three new ‘countries’ byregion. Thus, data from 16 countries wereanalysed. As a consequence of omitting somesmaller centres, the overall comparison between
Health Technology Assessment 2005; Vol. 9: No. 26
27
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 5
An empirical investigation of the properties of different statistical methods used for performing
indirect comparisons
aspirin and heparin is slightly different from thatobtained in the actual trial.
Sampling schemeFor each comparison, a sample of 100 patientswas drawn at random with replacement from each relevant treatment arm in each country.Thus, the samples drawn were all of the same sizeregardless of the number of patients actually inthe trial in that country. This eliminatedcomplications arising from variation in samplesize across trials.
For indirect comparisons, random samples ofpatients receiving aspirin (B), heparin (C) orneither (A) were used to create meta-analyses withk ‘trials’ comparing AvB and k comparing AvC,where k = 8, 4, 2 or 1. Various methods forindirect comparisons were used to estimate thecontrast between aspirin and heparin. Randomsamples comparing BvC were also taken from thesame 2k countries. The whole process wasrepeated 1000 times.
All analyses were done in Stata 6.0 (StataCorporation, Texas, USA, 2000). Meta-analyseswere undertaken using the metan command.102,103
Random effects meta-regression was fitted usingthe metareg command.102,104
AnalysesThe researchers were interested in thecomparison of aspirin and heparin (BvC).Situations were examined where the comparisonbetween aspirin and heparin could be estimatedusing indirect comparisons. The first situation(case 1) is examined in most detail as it is themost likely to occur. However, it was recognisedthat indirect comparisons can occur in manydifferent ways. Three possibilities are examined incases 2–4 using the same basic approach as incase 1. Each of these cases also used data frompatients in the fourth arm of the IST trial whohad received both aspirin and heparin (denoted D).
Case 1: Indirect comparisons by oneroute (trials of AvB and AvC)Estimates based on indirect comparison (using ktrials of AvB and k trials of AvC) were comparedwith estimates from direct comparisons from thesame 2k countries.
The specific methods used to estimate the BvCtreatment effect were as follows.
� Indirect comparison of BvC using each of thefollowing methods
Method Short name used in tables
Fixed effect adjusted indirect Adjusted indirect (fixed effect)comparison14 (ratio of odds ratios for AvB and AvC)
Random effects adjusted Adjusted indirect (random indirect comparison effects)(DerSimonian and Laird)
Logistic regression Logistic regression(fixed effect)
Random effects meta- Meta-regression (random regression effects)
Naive method (adding Naivenumerators and denominators for treatment arms)
� Direct comparison of BvC in the same 2kcountries used for indirect comparison
Method Short name used in tables
Inverse variance meta-analysis Meta-analysis (fixed effect)
Logistic regression Logistic regression
Random effects meta-analysis Meta-analysis (random (DerSimonian and Laird) effects)
In addition, logistic regression analyses wereperformed for both direct and indirectcomparisons adjusting for the three covariatesgender, age and risk score, both at the individuallevel and using study-level summaries.
Analyses were performed for k = 8, 4, 2 or 1, but some methods could not be used reliably forsmall k.
The scenario just described is asymmetricregarding the three treatments, focusing on BvCas the comparison of interest. As all of the datawere available, it was also possible to set up thesame analyses with, in turn, AvB or AvC as thecomparison of interest. The extent to which thesethree analyses would indicate the sameperformance of different methods of analysisdepends on the similarity of the amount ofheterogeneity between studies for the differentcomparisons.
Additional analyses were performed to examinethe effect of adjusting for patient covariates.Logistic regression was used with adjustment forpatients’ gender, age and symptom score. Analyseswere done using either individual patient data
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
28
(k = 8, 4, 2 or 1) or trial-level summary statistics(k = 8 or 4).
Case 2: Indirect comparison of BvC bytwo routes (trials of AvB, AvC and DvB,DvC)The same basic approach to sampling was used,with k trials of each of the four comparisons.Indirect comparison of BvC was made usinglogistic regression. Analyses were performed fork = 4, 2 or 1.
Case 3: Indirect comparisons of BvC bytwo steps (trials of BvD, DvA and AvC)Here, k trials of each of three comparisons weregenerated, and indirect comparisons madecombining two adjusted indirect comparisons.Thus, trials of BvD and DvA were combined toobtain an indirect estimate of the comparisonBvA, and then this estimate was combined withthe estimate for AvC to give an estimate of thecomparison BvC. Analyses were performed fork = 4, 2 or 1.
Case 4: Indirect comparisons of BvCfrom multiarm trials (trials of AvBvDand AvCvD)Here, k trials of each of two three-armcomparisons were generated, and indirectcomparisons made using logistic regression.Analyses were performed for k = 8, 4, 2 or 1.
Some theoretical resultsPrecision of indirect estimatesSuppose that a set of trials has been performed onsets of patients randomly sampled from the samepopulation. The symbol � was used to indicate theestimated log odds ratio and a subscript in squarebrackets to indicate the number of trials beingcombined to obtain this estimate. For one trial,suppose that the estimated treatment effect � hasvariance �2 (= [SE(�)]2). Then for a meta-analysisof 2k trials, all of the same size and assuming acommon true treatment effect, an inverse variancemeta-analysis would provide an estimated varianceof the treatment effect of [SE(�[2k])]
2 = �2/2k. Theexpected variance from an indirect comparisonbased on k trials for each comparison is
�2 �2 2�2
[SE(�AB[k] – �AC[k])]2 = — + — = ——.
k k k
Thus, one directly randomised trial is as precise asan indirect comparison based on four randomisedtrials of the same size. Put differently, four times
as many similarly sized trials are needed for theindirect approach to have the same power asdirectly randomised comparisons. This relationwill be approximately true when � is estimatedfrom k trials of varying sizes.
The 4:1 ratio depends on certain assumptions,including equal variances for comparisons AvB,AvC and BvC. Using �2 to denote the between-trialvariance, one requires
�2AB + �2
AB = �2AC + �2
AC = �2BC + �2
BC
This assumption cannot be tested unless one hasdata from all three two-way comparisons, and eventhen there are unlikely to be enough trials to allowreliable estimation of all of these variances.
Fixed effect methodsAs with standard meta-analysis, fixed effectsmethods for indirect comparisons (the method ofBucher and colleagues14 and logistic regression)will underestimate SE2(�) if there is excessheterogeneity. For the indirect comparisonbetween B and C,
2�2
[SE(�AB[k] – �AC[k])]2 = —– + �2
AB + �2AC
k
rather than 2�2/k given earlier. Heterogeneity in atleast one of the sets of trials will lead to theestimated SE2(�) being too small. Random effectsanalyses would therefore seem to be a safer optionfor indirect comparisons.
The naive methodThe naive method treats the data as if they camefrom a single trial and completely ignores between-trial variance. To take the simplest case with noexcess heterogeneity, SE2(�) for a single trial of sizen is �2, as before. A naive comparison between karms of treatment A and k arms of treatment B isequivalent to a single trial of size kn, so that SE2(�)would be estimated as �2/k, which is half of thevariance from the adjusted indirect comparison bythe method of Bucher and colleagues.14 Thus, inthe case with no heterogeneity the naive methodwill give standard errors that are too small by afactor of 1/√2; that is, about 30% too small. Whenthere is between-trial heterogeneity, theunderestimation will be even greater.
Results of empirical studiesTables 4–10 summarise results of analyses of 1000simulated data sets, constructed as described inthe previous section.
Health Technology Assessment 2005; Vol. 9: No. 26
29
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
The content of the columns in Tables 4–6 isdescribed in the box below. Tables 7–10 contain thesame information apart from the estimated �2.Tables 8–10 also do not show estimated coverage.In the tables the notation k + k is used to denoteindirect comparisons based on two sets of k trials,and similarly for three or four sets of trials inTables 8 and 9.
Case 1: Indirect comparisons by oneroute (trials of AvB and AvC)Table 4 summarises direct and indirectcomparisons of aspirin and heparin (BvC) fordirect comparisons based on 16, 8, 4 and 2 trials
and indirect comparisons based on 8 + 8, 4 + 4,2+2 and 1 + 1 trials. Thus, for example, thedirect comparison involving 16 trials of BvC isbased on exactly the same amount of data as theindirect comparison derived from eight trials ofAvB and 8 of AvC. For the direct comparisonsresults are shown for an inverse variance meta-analysis, logistic regression and random effectsmeta-analysis (DerSimonian and Laird). For theindirect comparisons analyses were made using the same methods (the inverse variancemeta-analysis approach is that suggested byBucher and colleagues14), and also by the naivemethod. Random effects analyses with fewer thanfour trials per treatment comparison were notperformed.
Several features of the results can be noted:
� The estimated odds ratio is effectively the samewhichever method was used. The samplingprocedures used should lead to this result.
� As predicted, the results from indirectcomparisons were less precise than those fromdirect comparisons. As noted above, a fixedeffect indirect comparison of k+k trials wouldbe expected to give estimates with twice thevariance as a direct comparison based on 2ktrials (all of the same size). Thus, the standarderror of the indirect estimate is expected to beabout √2 = 1.41 larger than that of the directestimate.
� Similarly, the standard error for the indirectcomparison of k+k trials would be expected tohave the same precision as the direct estimatefrom k/2 trials. Table 4 shows that the standarderrors of the fixed effect indirect estimates from8+8 trials are indeed very similar to those from4 direct trials, and likewise for indirect 4+4 anddirect 2.
� The standard error obtained from the fixedeffect analysis will be too small if there isbetween-trial heterogeneity (beyond randomvariation). Column 6 shows that there was suchheterogeneity in all cases except for k<4. Forboth direct and indirect fixed effect estimates(first two rows of each section) the empiricalstandard deviations of the 1000 estimated logodds ratios (column 5) exceed the averagestandard errors (column 4). An exception hereis the direct comparisons using all 16 trials, forwhich there is no obvious explanation.
� Equivalent random effects methods seem toover-correct for heterogeneity, giving standarderrors that are larger than the empiricalstandard deviations of estimated log oddsratios.105
Column
2 Estimated odds ratio (obtained as exponential ofthe value in column 3)
3 Estimated log odds ratio (mean of 1000 estimatedvalues), denoted logOR
----------
4 Standard error of the estimated log odds ratio (mean of 1000 estimated values), denoted
SE––
(logOR) –––––––
5 Standard deviation of 1000 estimates log oddsratios (provides a non-parametric estimate ofuncertainty of the estimated log odds ratio),denoted SD(logOR
----------)
Comparison with the standard error (previouscolumn) indicates the extent to which the methodof analysis (e.g. logistic regression) underestimatesthe imprecision of the estimated treatment effect
6 Median value of �2 from 1000 analyses (forrandom effects models only). For indirect values, this is the median of the average of thetwo values of �2 for the two componentcomparisons Positive values indicate heterogeneity abovechance variation between studies. Differencesbetween values for different comparisons (e.g. comparing Tables 4 and 5) indicate non-constant heterogeneity for differentcomparisons
7,8 Coverage of the 95% confidence intervals:percentages of 1000 confidence intervals for thelog odds ratio that were wholly below or whollyabove the value in column 2 For a good method there should be about 2.5%of confidence intervals that lie wholly below orwholly above the true value
9,10 Percentages of 1000 trials for which thecomparison of two treatments was statisticallysignificant (p < 0.05) in favour of each treatmentThe percentages for indirect comparisons can be compared with the direct comparisons toshow overoptimistic or conservative methods
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
30
� The results for the adjusted indirect inversevariance meta-analysis are almost identical tothose using logistic regression. Likewise, theresults from the random effects methods(DerSimonian and Laird meta-analysis andrandom effects meta-regression) agree closelyfor these data.
� The naive method gives standard errors that aremuch smaller than those from all of the validmethods. Further, the empirical standarddeviations using the naive method were verymuch larger than those using properapproaches. As a consequence, the apparentstandard error of the estimates from the naive method is only about 40% of the correct size.
� Fixed effect direct and indirect methods do notgive the right coverage: the proportion ofoccasions when 95% confidence intervals do not include the correct value (i.e. the directestimate based on all the trials) exceeds 5% as a consequence of the excess heterogeneity. By contrast, for random effects models thecoverage is about 5%. Again, only the direct analyses based on 16 trials do not follow this pattern. The coverage of the naive method is awful, with over 40% ofconfidence intervals not including the correctvalue.
� The probability of obtaining a significant result (power) in favour of aspirin was similar for indirect and direct comparisons of equivalent strength (e.g. direct 4 and indirect 8+8). Random effects analyses had a rather lower power than fixed effectmethods, but also a reduced risk of a significant result in favour of heparin. As the number of studies reduces, as expected, the power falls and the probability of asignificant result in favour of heparin increases.
� The naive method not only gave a muchinflated probability of a significant result, butalso gave a high risk, 12–20%, of a significantresult in favour of heparin. About half of theanalyses using the naive method werestatistically significant in one direction or the other, for all numbers of trialsconsidered.
Tables 5 and 6 show the same information as inTable 4, but for analyses of no treatment versus heparin and no treatment versus aspirin,respectively. Although the broad picture is the same as in Table 4, some differences arisefrom variation in the degree of heterogeneity of the different comparisons. In particular,
for the direct comparison between no treatmentand heparin (Table 5) the between-trialheterogeneity was much less than for the othertwo comparisons. In fact, �2 for this comparisonwas about one-quarter of �2 for the other twocomparisons, indicating that the spread (standard deviation) of the distribution ofestimates across trials was twice as great for aspirin versus heparin and no treatment versusaspirin as for no treatment versus heparin. This disparity leads to the underestimation ofstandard errors of estimates being greatest for no treatment versus heparin. In most realapplications one would not be able to estimate �2 for all three two-way comparisons, so any such variability would be hidden.
The effect of adjustment for patient characteristicswas also explored. Table 7 shows for the aspirinversus heparin comparison the results ofadditional logistic regression analyses in whichadjustment was made for three patient variables:age, gender and prognostic score. The scoreindicates the number of symptoms (deficits) thatpatients presented at randomisation. Onlysymptoms that increased the risk at the 5% level ofstatistical significance in univariate analysis wereconsidered. The score was thus based on thefollowing deficits: face, arm/hand, leg/foot,dysphasia, hemianopia and visuospatial. Separate adjusted analyses were made usingindividual data and also using trial-levelsummaries (only for analyses of at least 8 trials).For comparison purposes, the results ofunadjusted logistic regression analyses arerepeated from Table 4. Only fixed effects analyseswere used.
Adjustment for individual patient data had a smalleffect on the estimated log odds ratio in bothdirect and indirect analyses. There was also a small inflation of the standard error of the logodds ratio. Adjustment for study-level summarydata had a rather greater effect on the estimatedlog odds ratio in both direct and indirect analyses, giving results nearer to no effect (OR=1)than the analyses based on individual data. There was also a further inflation of the standarderror of the log odds ratio in comparison to the analyses using individual data. The standarderrors for analyses with adjustment for study-level summaries were about 50% larger than the standard errors of unadjusted estimates.In all cases the standard errors were smaller than the empirical standard deviations ofestimates.
Health Technology Assessment 2005; Vol. 9: No. 26
31
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
32 TA
BLE
4Re
sults
of d
irect
and
indi
rect
com
paris
ons
of a
spiri
n ve
rsus
hep
arin
usin
g da
ta fr
om d
iffer
ent
num
bers
of c
ount
ries
as s
epar
ate
tria
ls (r
esul
ts b
ased
on
1000
resa
mpl
ings
bas
ed o
n th
e IS
Tst
udy;
pat
ient
out
com
e w
as d
eath
or d
epen
denc
e)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s Fa
vour
sva
lue
(%)
valu
e (%
)as
piri
nhe
pari
n
Dir
ect
16 t
rial
sM
eta-
anal
ysis
(fixe
d ef
fect
)0.
88–0
.131
0.07
50.
070
2.0
1.9
41.7
0Lo
gist
ic r
egre
ssio
n 0.
88–0
.132
0.07
50.
071
2.0
2.4
42.5
0M
eta-
anal
ysis
(ran
dom
effe
cts)
0.88
–0.1
300.
097
0.07
20.
058
0.6
0.6
21.7
0
Indi
rect
(8)
+ (
8)a
tria
lsA
djus
ted
indi
rect
(fix
ed e
ffect
)0.
88–0
.131
0.14
90.
171
4.8
4.5
17.2
0.4
Logi
stic
reg
ress
ion
0.87
–0.1
340.
148
0.17
14.
24.
317
.30.
8M
eta-
regr
essio
n (r
ando
m e
ffect
s)0.
87–0
.138
0.18
10.
171
0.03
71.
03.
512
.10.
1A
djus
ted
indi
rect
(ran
dom
effe
cts)
0.88
–0.1
290.
189
0.17
10.
050
1.2
2.2
9.1
0.2
Nai
ve
0.89
–0.1
150.
102
0.25
522
.320
.138
.112
.1
Dir
ect
8 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.88
–0.1
330.
106
0.12
33.
43.
927
.70.
7Lo
gist
ic r
egre
ssio
n0.
88–0
.128
0.10
50.
123
5.3
3.8
26.9
0.5
Met
a-an
alys
is (r
ando
m e
ffect
s)0.
88–0
.131
0.13
70.
125
0.04
82.
32.
716
.20.
2
Indi
rect
(4)
+ (
4) t
rial
sA
djus
ted
indi
rect
(fix
ed e
ffect
)0.
88–0
.131
0.21
10.
248
5.6
4.1
12.4
1.7
Logi
stic
reg
ress
ion
0.88
–0.1
240.
210
0.25
45.
14.
014
.22.
1M
eta-
regr
essio
n (r
ando
m e
ffect
s)0.
88–0
.125
0.25
90.
245
0.02
72.
52.
57.
50.
9A
djus
ted
indi
rect
(ran
dom
effe
cts)
0.88
–0.1
310.
273
0.25
80.
048
2.5
2.8
7.2
0.6
Nai
ve
0.88
–0.1
310.
145
0.38
023
.323
.235
.213
.9
Dir
ect
4 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.88
–0.1
260.
149
0.18
46.
14.
918
.51.
0Lo
gist
ic r
egre
ssio
n0.
87–0
.134
0.14
90.
189
6.3
6.2
21.3
1.1
Met
a-an
alys
is (r
ando
m e
ffect
s)0.
88–0
.130
0.19
40.
187
0.02
83.
23.
512
.20.
8
Indi
rect
(2)
+ (
2) t
rial
sA
djus
ted
indi
rect
(fix
ed e
ffect
)0.
88–0
.123
0.29
90.
356
5.6
5.2
10.4
1.8
Logi
stic
reg
ress
ion
0.88
–0.1
310.
298
0.35
54.
84.
79.
42.
9M
eta-
regr
essio
n (r
ando
m e
ffect
s)0.
88–0
.128
0.37
90.
357
0.00
13.
13.
76.
61.
7N
aive
0.
89–0
.116
0.20
80.
545
21.5
22.0
28.6
17.3
cont
inue
d
Health Technology Assessment 2005; Vol. 9: No. 26
33
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
4Re
sults
of d
irect
and
indi
rect
com
paris
ons
of a
spiri
n ve
rsus
hep
arin
usin
g da
ta fr
om d
iffer
ent
num
bers
of c
ount
ries
as s
epar
ate
tria
ls (r
esul
ts b
ased
on
1000
resa
mpl
ings
bas
ed o
n th
e IS
Tst
udy;
pat
ient
out
com
e w
as d
eath
or d
epen
denc
e) (
cont
’d)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s Fa
vour
sva
lue
(%)
valu
e (%
)as
piri
nhe
pari
n
Dir
ect
2 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.88
–0.1
300.
212
0.27
06.
46.
414
.52.
6Lo
gist
ic r
egre
ssio
n0.
88–0
.132
0.21
10.
273
5.4
6.5
16.2
2.1
Indi
rect
(1)
+(1
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
0.87
–0.1
390.
426
0.52
96.
75.
29.
94.
2Lo
gist
ic r
egre
ssio
n 0.
87–0
.135
0.42
50.
533
5.6
5.8
8.9
3.5
Naï
ve
0.89
–0.1
160.
302
0.82
424
.121
.031
.119
.8
a(N
umbe
r of
stu
dies
invo
lved
in t
he c
ompa
rison
asp
irin
vers
us n
o tr
eatm
ent)
+ (N
umbe
r of
stu
dies
invo
lved
in h
epar
in v
ersu
s no
tre
atm
ent)
.
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
34 TA
BLE
5Re
sults
of d
irect
and
indi
rect
com
paris
ons
of n
o tr
eatm
ent
vers
us h
epar
in u
sing
data
from
diff
eren
t nu
mbe
rs o
f cou
ntrie
s as
sep
arat
e tr
ials
(res
ults
bas
ed o
n 10
00 re
sam
plin
gs b
ased
on
the
IST
stud
y; p
atie
nt o
utco
me
was
dea
th o
r dep
ende
nce)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s no
Favo
urs
valu
e (%
)va
lue
(%)
trea
tmen
the
pari
n
Dir
ect
16 t
rial
sM
eta-
anal
ysis
(fixe
d ef
fect
)0.
95–0
.049
0.07
50.
074
1.8
2.3
9.4
0.7
Logi
stic
reg
ress
ion
0.96
–0.0
450.
074
0.07
32.
42.
09.
20.
3M
eta-
anal
ysis
(ran
dom
effe
cts)
0.96
–0.0
460.
085
0.07
30.
017
1.3
1.3
6.7
0.2
Indi
rect
(8)
+ (
8)a
tria
lsA
djus
ted
indi
rect
(fix
ed e
ffect
)0.
96–0
.040
0.15
00.
190
5.2
6.7
8.8
5.9
Logi
stic
reg
ress
ion
0.96
–0.0
400.
148
0.19
48.
05.
710
.44.
7M
eta-
regr
essio
n (r
ando
m e
ffect
s)0.
95–0
.055
0.19
40.
196
0.05
83.
63.
24.
52.
1A
djus
ted
indi
rect
(ran
dom
effe
cts)
0.95
–0.0
520.
200
0.18
70.
066
2.6
2.1
4.5
0.9
Nai
ve
0.96
–0.0
360.
102
0.24
323
.718
.725
.416
.9
Dir
ect
8 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.95
–0.0
530.
106
0.11
13.
03.
38.
51.
3Lo
gist
ic r
egre
ssio
n 0.
96–0
.046
0.10
50.
111
3.7
3.3
6.9
1.4
Met
a-an
alys
is (r
ando
m e
ffect
s)0.
96–0
.046
0.12
20.
115
0.01
12.
72.
05.
40.
9
Indi
rect
(4)
+(4
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
0.94
–0.0
560.
212
0.27
65.
96.
358.
74.
2Lo
gist
ic r
egre
ssio
n 0.
95–0
.049
0.21
00.
275
6.1
7.3
9.9
4.4
Met
a-re
gres
sion
(ran
dom
effe
cts)
0.94
–0.0
580.
277
0.26
70.
045
3.6
3.4
5.0
1.9
Adj
uste
d in
dire
ct (r
ando
m e
ffect
s)0.
96–0
.044
0.28
80.
280
0.06
43.
12.
94.
62.
0N
aive
0.
95–0
.051
0.14
50.
363
21.8
2327
.318
.1
Dir
ect
4 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.95
–0.0
530.
150
0.16
22.
73.
17.
30.
9Lo
gist
ic r
egre
ssio
n 0.
96–0
.037
0.14
90.
168
5.1
3.0
5.6
2.8
Met
a-an
alys
is (r
ando
m e
ffect
s)0.
96–0
.037
0.17
80.
162
0.00
12.
51.
54.
51.
1
Indi
rect
(2)
+ (
2) t
rial
sA
djus
ted
indi
rect
(fix
ed e
ffect
)0.
94–0
.056
0.30
10.
412
7.2
6.5
10.3
4.6
Logi
stic
reg
ress
ion
0.96
–0.0
410.
298
0.40
28.
56.
88.
16.
6M
eta-
regr
essio
n (r
ando
m e
ffect
s)0.
94–0
.067
0.39
90.
396
0.02
14.
24.
86.
13.
3N
aive
0.
96–0
.039
0.20
80.
509
21.9
19.2
22.5
18.4
cont
inue
d
Health Technology Assessment 2005; Vol. 9: No. 26
35
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
5Re
sults
of d
irect
and
indi
rect
com
paris
ons
of n
o tr
eatm
ent
vers
us h
epar
in u
sing
data
from
diff
eren
t nu
mbe
rs o
f cou
ntrie
s as
sep
arat
e tr
ials
(res
ults
bas
ed o
n 10
00 re
sam
plin
gs b
ased
on
the
IST
stud
y; p
atie
nt o
utco
me
was
dea
th o
r dep
ende
nce)
(co
nt’d
)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
--SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s no
Favo
urs
valu
e (%
)va
lue
(%)
trea
tmen
the
pari
n
Dir
ect
2 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
0.95
–0.0
510.
212
0.24
02.
94.
15.
72.
5Lo
gist
ic r
egre
ssio
n0.
96–0
.046
0.21
00.
236
3.9
3.9
5.8
2.0
Indi
rect
(1)
+(1
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
0.94
–0.0
660.
426
0.57
26.
56.
98.
85.
8Lo
gist
ic r
egre
ssio
n 0.
95–0
.055
0.42
50.
577
6.0
8.1
8.9
5.2
Nai
ve0.
96–0
.038
0.30
20.
784
22.1
20.9
23.5
20.2
a(N
umbe
r of
stu
dies
invo
lved
in t
he c
ompa
rison
asp
irin
vers
us n
o tr
eatm
ent)
+ (N
umbe
r of
stu
dies
invo
lved
in h
epar
in v
ersu
s as
pirin
).
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
36 TA
BLE
6Re
sults
of d
irect
and
indi
rect
com
paris
ons
of n
o tr
eatm
ent
vers
us a
spiri
n us
ing
data
from
diff
eren
t nu
mbe
rs o
f cou
ntrie
s as
sep
arat
e tr
ials
(res
ults
bas
ed o
n 10
00 re
sam
plin
gs b
ased
on
the
IST
stud
y; p
atie
nt o
utco
me
was
dea
th o
r dep
ende
nce)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s no
Favo
urs
valu
e (%
)va
lue
(%)
trea
tmen
tas
piri
n
Dir
ect
16 t
rial
sM
eta-
anal
ysis
(fixe
d ef
fect
)1.
080.
079
0.07
50.
072
2.4
2.2
0.1
16.5
Logi
stic
reg
ress
ion
1.09
0.08
60.
074
0.07
52.
91.
80.
121
.9M
eta-
anal
ysis
(ran
dom
effe
cts)
1.09
0.08
40.
099
0.07
50.
065
0.3
0.8
08.
4
Indi
rect
(8)
+(8
)atr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
1.08
0.07
40.
150
0.16
35.
03.
41.
39.
1Lo
gist
ic r
egre
ssio
n 1.
090.
090
0.14
90.
170
5.4
3.4
1.3
11.4
Met
a-re
gres
sion
(ran
dom
effe
cts)
1.09
0.08
50.
181
0.16
50.
037
2.5
1.5
0.4
6.2
Adj
uste
d in
dire
ct (r
ando
m e
ffect
s)1.
090.
089
0.18
60.
167
0.04
42.
41.
70.
35.
8N
aive
1.
090.
093
0.10
20.
246
24.3
19.4
11.8
34.5
Dir
ect
8 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
1.08
0.07
80.
106
0.12
45.
24.
31.
014
.3Lo
gist
ic r
egre
ssio
n 1.
080.
080
0.10
50.
129
6.1
5.8
1.6
15.0
Met
a-an
alys
is (r
ando
m e
ffect
s)1.
090.
085
0.14
00.
129
0.05
92.
34.
10.
29.
5
Indi
rect
(4)
+(4
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
1.08
0.07
40.
212
0.23
65.
34.
11.
87.
8Lo
gist
ic r
egre
ssio
n 1.
080.
076
0.21
00.
243
3.9
4.6
1.9
6.9
Met
a-re
gres
sion
(ran
dom
effe
cts)
1.07
0.07
20.
262
0.24
80.
027
3.2
2.8
1.4
6.0
Adj
uste
d in
dire
ct (r
ando
m e
ffect
s)1.
090.
083
0.26
90.
239
0.04
52.
01.
70.
94.
5N
aive
1.
080.
077
0.14
50.
375
23.7
22.9
16.6
31.0
Dir
ect
4 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
1.08
0.07
70.
149
0.18
86.
17.
42.
513
.5Lo
gist
ic r
egre
ssio
n 1.
090.
088
0.14
90.
189
7.0
4.6
2.5
14.9
Met
a-an
alys
is (r
ando
m e
ffect
s)1.
080.
077
0.19
80.
192
0.03
92.
84.
21.
39.
8
Indi
rect
(2)
+(2
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
1.10
0.09
40.
301
0.35
35.
55.
42.
88.
4Lo
gist
ic r
egre
ssio
n 1.
090.
088
0.29
90.
355
5.8
3.7
1.8
8.4
Met
a-re
gres
sion
(ran
dom
effe
cts)
1.09
0.08
80.
377
0.36
10
4.4
2.8
1.9
6.3
Nai
ve
1.08
0.08
10.
207
0.54
222
.222
.018
.027
.2
cont
inue
d
Health Technology Assessment 2005; Vol. 9: No. 26
37
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
6Re
sults
of d
irect
and
indi
rect
com
paris
ons
of n
o tr
eatm
ent
vers
us a
spiri
n us
ing
data
from
diff
eren
t nu
mbe
rs o
f cou
ntrie
s as
sep
arat
e tr
ials
(res
ults
bas
ed o
n 10
00 re
sam
plin
gs b
ased
on
the
IST
stud
y; p
atie
nt o
utco
me
was
dea
th o
r dep
ende
nce)
(co
nt’d
)
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)M
edia
n �
2B
elow
tru
e A
bove
tru
e Fa
vour
s no
Favo
urs
valu
e (%
)va
lue
(%)
trea
tmen
tas
piri
n
Dir
ect
2 tr
ials
Met
a-an
alys
is (fi
xed
effe
ct)
1.08
0.08
20.
212
0.28
14.
77.
93.
811
.6Lo
gist
ic r
egre
ssio
n1.
090.
085
0.21
00.
277
7.4
6.1
4.0
12.2
Indi
rect
(1)
+(1
) tr
ials
Adj
uste
d in
dire
ct (f
ixed
effe
ct)
1.10
0.09
20.
428
0.50
54.
25.
43.
77.
3Lo
gist
ic r
egre
ssio
n 1.
080.
080
0.42
70.
499
5.1
4.5
3.1
6.2
Nai
ve1.
100.
094
0.30
00.
821
23.5
22.1
20.7
27.5
a(N
umbe
r of
stu
dies
invo
lved
in t
he c
ompa
rison
asp
irin
vers
us h
epar
in) +
(Num
ber
of s
tudi
es in
volv
ed in
hep
arin
ver
sus
no t
reat
men
t).
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
38 TA
BLE
7Re
sults
of d
irect
and
indi
rect
com
paris
ons
of a
spiri
n ve
rsus
hep
arin
usin
g lo
gist
ic re
gres
sion
with
and
with
out
adju
stm
ent
for p
atie
nt fa
ctor
s at
stu
dy a
nd in
divid
ual l
evel
Cov
erag
e of
95%
CI
% R
esul
ts w
ith
p<
0.05
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)B
elow
tru
e A
bove
tru
e Fa
vour
sFa
vour
sva
lue
(%)
valu
e (%
)as
piri
nhe
pari
n
Dir
ect
16 t
rial
sLo
gist
ic r
egre
ssio
n 0.
88–0
.132
0.07
50.
071
2.0
2.4
42.5
0A
djus
ted
for
gend
er, a
ge, s
core
(stu
dy le
vel)
0.92
–0.0
790.
089
0.08
94.
01.
114
.10.
1A
djus
ted
for
gend
er, a
ge, s
core
(ind
ivid
ual l
evel
)0.
89–0
.113
0.08
20.
080
1.7
2.4
27.6
0
Indi
rect
(8)
+ (
8)a
tria
lsLo
gist
ic r
egre
ssio
n 0.
87–0
.134
0.14
80.
171
4.2
4.3
17.3
0.8
Adj
uste
d fo
r ge
nder
, age
, sco
re (s
tudy
leve
l)0.
92–0
.087
0.17
10.
183
3.8
2.8
9.0
1.0
Adj
uste
d fo
r ge
nder
, age
, sco
re (i
ndiv
idua
l lev
el)
0.90
–0.1
080.
164
0.18
74.
95.
612
.51.
1
Dir
ect
8 tr
ials
Logi
stic
reg
ress
ion
0.88
–0.1
280.
105
0.12
35.
33.
826
.90.
5A
djus
ted
for
gend
er, a
ge, s
core
(stu
dy le
vel)
0.92
–0.0
780.
154
0.18
13.
92.
711
.91.
0A
djus
ted
for
gend
er, a
ge, s
core
(ind
ivid
ual l
evel
)0.
89–0
.113
0.11
60.
132
3.5
5.1
20.0
0.2
Indi
rect
(4)
+ (
4) t
rial
sLo
gist
ic r
egre
ssio
n 0.
88–0
.124
0.21
00.
254
5.1
4.0
14.2
2.1
Adj
uste
d fo
r ge
nder
, age
, sco
re (s
tudy
leve
l)0.
93–0
.071
0.31
80.
376
4.9
2.6
5.3
2.0
Adj
uste
d fo
r ge
nder
, age
, sco
re (i
ndiv
idua
l lev
el)
0.89
–0.1
150.
232
0.26
74.
34.
88.
81.
9
Dir
ect
4 tr
ials
Logi
stic
reg
ress
ion
0.87
–0.1
340.
149
0.18
96.
36.
221
.31.
1A
djus
ted
for
gend
er, a
ge, s
core
(ind
ivid
ual l
evel
)0.
89–0
.116
0.16
50.
206
5.2
6.1
16.3
1.4
Indi
rect
(2)
+ (
2) t
rial
sLo
gist
ic r
egre
ssio
n 0.
88–0
.131
0.29
80.
355
4.8
4.7
9.4
2.9
Adj
uste
d fo
r ge
nder
, age
, sco
re (i
ndiv
idua
l lev
el)
0.89
–0.1
130.
331
0.39
75.
45.
68.
72.
8
Dir
ect
2 tr
ials
Logi
stic
reg
ress
ion
0.88
–0.1
320.
211
0.27
35.
46.
516
.22.
1A
djus
ted
for
gend
er, a
ge, s
core
(ind
ivid
ual l
evel
)0.
89–0
.120
0.23
60.
298
4.7
7.3
13.8
2.1
Indi
rect
(1)
+(1
) tr
ials
Logi
stic
reg
ress
ion
0.87
–0.1
350.
425
0.53
35.
65.
88.
93.
5A
djus
ted
for
gend
er, a
ge, s
core
(ind
ivid
ual l
evel
)0.
89–0
.116
0.47
70.
556
4.1
4.7
6.4
2.8
a(N
umbe
r of
stu
dies
invo
lved
in t
he c
ompa
rison
asp
irin
vers
us n
o tr
eatm
ent)
+ (N
umbe
r of
stu
dies
invo
lved
in h
epar
in v
ersu
s no
tre
atm
ent)
.
Health Technology Assessment 2005; Vol. 9: No. 26
39
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
8Re
sults
of i
ndire
ct c
ompa
rison
s of
asp
irin
vers
us h
epar
in (
BvC)
by
two
rout
es (
tria
ls of
AvB
, AvC
, DvB
and
DvC
) (c
ase
2) (
In p
aren
thes
es a
re s
how
n re
sults
from
Tab
le 4
for t
he s
ame
num
bers
of p
atie
nts
in s
impl
e in
dire
ct c
ompa
rison
)
% R
esul
ts w
ith
p<
0.0
5
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)Fa
vour
s as
piri
nFa
vour
s he
pari
n
Logi
stic
reg
ress
ion
Indi
rect
4 +
4 +
4 +
4 (i
ndire
ct 8
+ 8
)0.
88–0
.131
(–0.
134)
0.14
8 (0
.148
)0.
176
(0.1
71)
16.6
(17.
3)0.
5 (0
.8)
Indi
rect
2 +
2 +
2 +
2 (i
ndire
ct 4
+ 4
)0.
88–0
.127
(–0.
124)
0.21
1 (0
.210
)0.
259
(0.2
54)
14.6
(14.
2)1.
5 (2
.1)
Indi
rect
1 +
1 +
1 +
1 (i
ndire
ct 2
+ 2
)0.
88–0
.129
(–0.
131)
0.30
0 (0
.298
)0.
375
(0.3
55)
11.3
(9.4
)2.
8 (2
.9)
Adj
uste
d in
dire
ct (
rand
om e
ffect
s)In
dire
ct 4
+ 4
+ 4
+ 4
0.87
–0.1
340.
188
0.18
511
.30.
4
TA
BLE
9Re
sults
of i
ndire
ct c
ompa
rison
s of
asp
irin
vers
us h
epar
in (
BvC)
by
two
step
s (t
rials
of t
rials
of B
vD, D
vA a
nd A
vC)
(cas
e 3)
% R
esul
ts w
ith
p<
0.0
5
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)Fa
vour
s as
piri
nFa
vour
s he
pari
n
Adj
uste
d in
dire
ct (
fixed
effe
ct)
Indi
rect
4 +
4 +
40.
89–0
.118
0.25
80.
304
11.4
1.8
Indi
rect
2 +
2 +
20.
86–0
.156
0.36
60.
447
10.4
2.6
Indi
rect
1 +
1 +
10.
89–0
.114
0.52
20.
637
8.0
3.3
Adj
uste
d in
dire
ct (
rand
om e
ffect
s)In
dire
ct 4
+ 4
+ 4
0.88
–0.1
230.
335
0.31
75.
70.
5
TA
BLE
10
Resu
lts o
f ind
irect
com
paris
ons
of a
spiri
n ve
rsus
hep
arin
(Bv
C) fr
om m
ultia
rm t
rials
(tria
ls of
AvB
vD a
nd A
vCvD
) by
logi
stic
regr
essio
n (c
ase
4) (
in p
aren
thes
es a
re s
how
n re
sults
from
Tabl
e 4
for t
he s
impl
e in
dire
ct c
ompa
rison
bas
ed o
n Av
B an
d Av
C)
% R
esul
ts w
ith
p<
0.0
5
OR
logO
R---
------
---SE
(log
OR
)SD
(log
OR
------
------
)Fa
vour
s as
piri
nFa
vour
s he
pari
n
Logi
stic
reg
ress
ion
Indi
rect
8+
80.
88–0
.131
(–0.
134)
0.12
8 (0
.148
)0.
152
(0.1
71)
21.4
(17.
3)0.
6 (0
.8)
Indi
rect
4+
40.
88–0
.131
(–0.
124)
0.18
2 (0
.210
)0.
225
(0.2
54)
15.4
(14.
2)1.
5 (2
.1)
Indi
rect
2+
20.
88–0
.124
(–0.
131)
0.25
8 (0
.298
)0.
334
(0.3
55)
12.8
(9.4
)3.
0 (2
.9)
Indi
rect
1+
10.
89–0
.120
(–0.
135)
0.36
9 (0
.425
)0.
468
(0.5
33)
9.7
(8.9
)2.
8 (3
.5)
Case 2: Indirect comparison of BvC bytwo routes (trials of AvB, AvC and DvB,DvC)Tables 4–7 consider the simplest indirectcomparison, in which the contrast BvC isestimated using results from trials of AvB and AvC.Table 8 shows results from analyses that included asecond pathway for the indirect comparison ofBvC, namely trials of DvB and DvC. Results forlogistic regressions are shown for 4, 2 and 1 trialper comparison. For comparison the results fromTable 4 are repeated here relating to a singlepathway of twice as many trials: each of the firstfour rows of the table thus compares analyses ofthe same amount of data. The two sets of resultsare very similar, suggesting that the twoapproaches are equivalent. In theory they are, butin practice additional heterogeneity variances areplaying a part and in general one might expectthat the inclusion of additional links wouldincrease the risk of obtaining an erroneous result.
Table 8 also shows the results of a random effectsanalysis for the case of 4 + 4 + 4 + 4 trials. Aswas seen in Table 4, the standard error of theestimated log odds ratio is larger for the randomeffects analysis than for the fixed effect analysis(first row).
Case 3: Indirect comparisons of BvC bytwo steps (trials of BvD, DvA and AvC)Table 9 shows results from three sets of trialsinvolving four treatments. In effect, this methodinvolves two applications of the adjusted indirectapproach in sequence. These results are broadlycompatible with those in the previous tables.Again, a random effects analysis is shown for thelargest number of trials (4+4+4).
Case 4: Indirect comparisons of BvCfrom multiarm trials (trials of AvBvDand AvCvD)Lastly, Table 10 shows results from indirectcomparisons of two sets of three-arm trials.Although these analyses also make use oftreatment D, in this set-up the comparison isinternal (and randomised), unlike in case 2.Although there is the same number of trials thenumber of patients is 50% greater than in Table 8. As a result, there is a 25% reduction invariance of the estimated log odds ratio compared with the results in Table 8. Similarly,there is a reduction in variance compared with thesimple indirect comparison based on AvB andAvC, which for comparison is repeated here fromTable 4.
SummarySeveral methods have been proposed to estimatethe different effectiveness of treatments notcompared directly in randomised trials. Thisinvestigation focused on variations of widelyavailable methods: the adjusted indirectcomparison (fixed or random effects), meta-regression and logistic regression, and alsoevaluated the inappropriate ‘naive’ method.
The results of the resampling studies show that allof the appropriate methods are unbiased and willgive the right answer on average across many suchapplications. However, the correctness of theestimated standard error will depend on strongbut unverifiable assumptions. Little difference wasfound in the performance of inverse variancemethods (adjusted indirect comparison) andlogistic regression. But, for this data set, theresults support the use of random effects versionsof either of these approaches. Such a finding isnot surprising given that there are (at least) threeheterogeneity variances that could have an impacton the results of even the simplest indirectcomparison.
With binary outcomes, logistic regression mayseem preferable to the adjusted indirectcomparison. The results of the two methods areextremely close when both can be applied. Anadvantage of the adjusted indirect comparison forbinary data is that the meta-analysis is notrestricted to the use of the odds ratio, although itmay be possible to use other types of regressionmodel. The main advantages of logistic regressionare that it can be extended to more complexsituations and it can be used to adjust forindividual- or study-level covariates. In practice,however, IPD are unlikely to be available and theuse of study-level patient covariates (such as meanage) is in general too blunt a method to beinformative. Certainly, in this investigationadjustment for study-level summaries of patientcharacteristics led to biased estimates of treatmenteffect with inflated standard errors. By contrast,the ability to include study-level covariates thatrelate to the study itself can be valuable, as in thestudy by Packer and colleagues.69
This resampling study used the odds ratio as ameasure of effect. While the odds ratio hasdesirable statistical properties, it may not alwaysbe the preferred effect measure for a meta-analysis.57 Adjusted indirect comparisons based oncontrasting separate meta-analyses, such as theapproach of Bucher and colleagues,14 can be
An empirical investigation of the properties of different statistical methods used for performing indirect comparisons
40
extended very simply to risk ratio or riskdifference, but logistic regression and some morecomplex methods do require the effect measure tobe the odds ratio. For continuous data theadjusted indirect comparison approach can beused for either actual or standardised meandifferences, as in the study by Packer andcolleagues.69 The choice here is perhaps simpler,with a strong preference for using the actual meandifference as long as the outcome being measuredis the same in the different studies.
These analyses were all based on the data from asingle large trial, with a unified protocol andhence consistent inclusion criteria. In general,more variation would be expected between
independent component studies contributing toan indirect comparison. In particular, there maybe differences between the participants in thetrials (and aspects of the trials themselves)between the two sets of trials being combined.
In addition, the present analyses compared atreatment with a small benefit (aspirin) with onewith no apparent benefit (heparin). The extent towhich our findings would apply in the morecommon case where indirect comparisons aremade when each of two active drugs is better thanplacebo (or no treatment) is unknown. Also,differences between direct and indirect estimatesmay be less important where the effect of interestwas large.
Health Technology Assessment 2005; Vol. 9: No. 26
41
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
The objective of this chapter is to summariseempirical evidence about the validity of
indirect comparison by measuring discrepancybetween the direct and the indirect estimates in asample of published meta-analyses. It builds onthe findings of the survey of the use of indirectcomparison for evaluating relative efficacy ofcompeting interventions presented in Chapter 3.(Note: data in this chapter have previously beenpublished.106)
MethodsIdentification of relevant reviewsDetailed strategies for identifying relevant meta-analyses have been described in Chapter 3. Inbrief, the hard copies of reviews on DARE (1994 toDecember 1998) were available and read. Inaddition, the reviews on The Cochrane Library(Issue 3, 2000) were screened. Included meta-analyses had to meet two criteria: competinginterventions could be compared both directly andindirectly, and the same primary studies were notused in both the direct and indirect comparison.
Comparison methodsThe relative efficacy of competing interventionswas estimated using three comparative methods:the direct (head-to-head) comparison, the adjustedindirect comparison and the naive indirectcomparison. For the direct comparisons, therelative efficacy (TAB) of intervention A versus Bwas estimated by comparing the result of group Aand the result of group B within a randomisedtrial. When there were two or more similar trialsthat compared the same interventions, results ofindividual trials were weighted by the inverse ofcorresponding variances and then quantitativelycombined. Whenever possible, the random effectsmodel was used for the quantitative pooling.
The naive indirect estimate of relative efficacy wasobtained simply by comparing the result oftreatment A in trial 1 with the result oftreatment B in trial 2. That is, in a naive indirect
comparison, results of individual arms betweendifferent trials are compared as if they are from asingle trial. The variance in the naive indirectcomparison will be similar to the variance in thedirect comparison with the same sample size (seeChapter 5). When more than one study wasavailable for a treatment, the results of individualstudies were weighted by the number ofparticipants in the corresponding arms and thenquantitatively combined (thus, between-trialvariability was not considered).
Adjusted indirect comparisons were conductedusing the method suggested by Bucher andcolleagues.14 In brief, the indirect comparison ofinterventions A and B was adjusted by the resultsof their direct comparisons with a commonintervention C. Suppose TAC is the estimate ofdirect comparison of intervention A versus C, andTBC is the direct comparison of intervention Bversus C. Then the estimate of the adjustedindirect comparison of intervention A versus B(T�AB, e.g. log relative risk, mean difference) isestimated by
T�AB = TAC – TBC
and its variance is
Var(T�AB) = Var(TAC) + Var(TBC)
This adjusted method aims to overcome thepotential problem of different prognostic factorsbetween study participants in different trials, andit is valid if the relative efficacy of interventions isconsistent in patients across different trials. Itshould be noted that the variance in the adjustedindirect comparison using four trials is equivalentto the variance in the direct comparison withinone trial of the same size (see Chapter 5). Multiplestudies used in the adjusted indirect comparisonare combined by a random effects modelwhenever possible.
Measures of discrepancyThe relative efficacy was measured using meandifference for continuous data and log relative risk
Health Technology Assessment 2005; Vol. 9: No. 26
43
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 6
Statistical discrepancy between the direct and indirect estimate: empirical evidence from published
meta-analyses
for binary data. The discrepancy between thedirect estimate (TBC) and the adjusted indirectestimate (T�BC) was measured by the difference (X)between the two estimates:
X = TBC – T�BC
and its standard error is
SE(X) = √—————————–— SE(TBC)2 + SE (T�BC)2
where SE(TBC) and SE(T�BC) are the estimatedstandard errors for the direct estimate and theadjusted indirect estimate, respectively. The 95%confidence interval for the estimated discrepancywas calculated by X ± 1.96 * SE(X). The estimateddiscrepancy can also be standardised by itsstandard error to obtain a value of z = X/SE(X).
In addition, the results of meta-analyses werecategorised as statistically non-significant(p > 0.05) or statistically significant (p 0.05).The statistically significant effect can be furtherseparated according to whether intervention B wasless or more effective than intervention C. Thedegree of agreement in statistical conclusionsbetween the direct and indirect method wasassessed by a weighted kappa.
ResultsThe searches identified 28 systematic reviews inwhich both the direct and indirect comparison ofcompeting interventions could be conducted,although indirect comparison was not explicitlyused in many of these meta-analyses (seeAppendix 8). Some systematic reviews assessedmore than two active interventions, and a total of44 comparisons could be made using data fromthe 28 systematic reviews.
Figure 2 summarises the statistical discrepancies(z statistics) between the direct and both naive andadjusted indirect estimates for 43 meta-analyses(note that one meta-analysis in which the naiveindirect comparison was not available was notincluded in Figure 2). There are several significantdiscrepancies between the direct and indirectestimates, but the direction of the difference isinconsistent. The relative efficacy of an interventionmay be overestimated or underestimated by theindirect comparison, as compared with the resultsof the direct comparison.
Significant discrepancy (|z| ≥ 1.96) was observedin three of the 44 comparisons between the direct
and adjusted indirect estimate (7%). Between thedirect and the naive indirect estimate, 11 of the 43comparisons showed statistically significantdiscrepancy (26%). The statistical discrepanciesbetween the direct and the naive indirect estimateare generally greater than those between the directand the adjusted indirect estimate (Figure 2).
Figure 3 shows the relation between the statisticaldiscrepancy (z) and the number of trials used inindirect comparisons. Visually, statisticaldiscrepancies tended to be smaller when thenumber of trials was large (>60) than when thenumber of trials was small (<40). However, suchtendency was not consistent, as the discrepancybetween the direct and indirect estimate may besignificant even when more than 40 trials havebeen used for the indirect comparison (e.g. meta-analysis 44).
Figure 4 summarises the differences (and their 95%confidence intervals) between the direct and theadjusted indirect estimates. Significant discrepancy(p < 0.05) was observed in three of the 44comparisons (i.e. the 95% confidence interval didnot include zero). In four other meta-analyses, thediscrepancy between the direct and the adjustedindirect estimate was borderline significant (i.e.p < 0.1). The relative efficacy of an interventionwas equally likely to be overestimated orunderestimated by the indirect comparison, ascompared with the results of the direct comparison.
There was a moderate agreement in statisticalconclusions between the direct and the adjustedindirect method (weighted kappa = 0.53) (Table 11).In terms of statistical conclusions, 32 of the 44indirect estimates fell within the same categories ofthe direct estimates. According to the directcomparison, 19 of the 44 comparisons suggested astatistically significant difference (p < 0.05) betweencompeting interventions. Compared with directestimates, the adjusted indirect estimates were lesslikely to be statistically significant. Ten of the 19significant direct estimates became statistically non-significant in the adjusted indirect comparison,whereas only two of the 25 non-significant directestimates were significant in the adjusted indirectcomparison. The agreement between the direct andthe naive indirect comparison is much poorer(weighted kappa = 0.28).
CommentaryA wide range of medical topics has been coveredby the 44 meta-analyses used in this study
Statistical discrepancy between the direct and indirect estimate: empirical evidence from published meta-analyses
44
Health Technology Assessment 2005; Vol. 9: No. 26
45
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
6420–2–4–6
z value
FIGURE 2 Statistical discrepancy between the direct and the indirect estimate: empirical evidence from published meta-analyses.Solid bars, discrepancy between the direct and adjusted indirect estimate; blank bars, discrepancy between the direct and naive indirectestimate.
(Appendix 8). The patient categories includethose with an increased risk of vascular occlusion,HIV-infected patients, those with GORD,postoperative pain or dyspepsia, and cigarettesmokers. These meta-analyses were used to obtaindata for examining discrepancies between thedifferent comparative methods. The study did notcritically appraise the methodological quality ofthese meta-analyses and of primary trials in thesemeta-analyses.
The findings presented in this paper suggest thatthe direction of discrepancy between the directand the indirect estimate is unpredictable, but thediscrepancy in the adjusted indirect comparison isgenerally less than that in the naive indirectcomparison. The observed significantdiscrepancies between the direct and the indirectestimates may be explained by many possiblefactors, such as random errors, baseline prognosticcharacteristics, interventions other than those
Statistical discrepancy between the direct and indirect estimate: empirical evidence from published meta-analyses
46
10
8
6
4
2
0
–2
–4
–6
–8
–10
0 15 30 45 60 75 90
No. of trials for indirect comparison
Adjusted
Naivez
valu
e
FIGURE 3 Statistical discrepancy (z value) and the number of trials used in indirect comparison
TABLE 11 Methods of comparison and number of significant findings in 44 meta-analyses of competing interventions
Adjusted indirect estimate Naive indirect estimate
Direct estimate Significant Non- Significant Significant Non- Significant effect (–) significant effect (+) effect (–) significant effect (+)(n = 6) effect (n = 33) (n = 5) (n = 9) effect (n = 19) (n = 15)
Significant effect (–) 5 3 0 3 5 0(n = 8)
Non-significant effect 1 23 1 5 11 8(n = 25 or 24)
Significant effect (+) 0 7 4 1 3 7(n = 11)
Non-significant effect: difference between intervention groups is statistically non-significant (p > 0.05); significant effect(p 0.05) is separated according to whether the intervention A is less (–) or more effective (+) than intervention B. Agreement between the direct and the adjusted indirect estimate: weighted kappa value 0.53; agreement between thedirect and the naive indirect estimate: weighted kappa value 0.28.
Health Technology Assessment 2005; Vol. 9: No. 26
47
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
–2–10 1 2
Met
a-an
alys
es
Discrepancy(95% CI)
123456789
1011121314151617181920212223242526272829303132333435363738394041424344
FIG
UR
E 4
Disc
repa
ncy
betw
een
the
dire
ct a
nd t
he a
djus
ted
indi
rect
com
paris
on: e
mpi
rical
evid
ence
from
44
publ
ished
met
a-an
alys
es. N
umbe
ring
of m
eta-
anal
yses
(1–
44)
follo
ws
the
orde
r as
inAp
pend
ix 8
. The
disc
repa
ncy
betw
een
the
dire
ct a
nd t
he in
dire
ct e
stim
ate
was
def
ined
as
diffe
renc
e in
est
imat
ed lo
g re
lativ
e ris
k (m
eta-
anal
yses
1–3
9), d
iffer
ence
in e
stim
ated
sta
ndar
dise
d m
ean
diffe
renc
e (m
eta-
anal
ysis
40)
or d
iffer
ence
in e
stim
ated
mea
n di
ffere
nce
(met
a-an
alys
es 4
1–44
).
compared, follow-up period and methods foroutcome measurement.
Direct versus naive indirect estimateEvidence from the naive indirect comparisons canbe considered to be equivalent to results fromobservational or non-randomised controlledresearch. To explain the discrepancies between thedirect and the naive indirect estimates, the mostimportant factor that should be considered is lackof comparability between patients in differentstudies. The observed difference betweentreatment groups in the naive indirect comparisonmay be due not to different interventions ofinterest, but to different prognostic characteristicsamong patients in the different studies(confounding).
Random error may be a cause of discrepancybetween the direct and the naive indirect estimate.When |z| = 1.96 is used as the cut-point forstatistical significance, the chance of a type I erroris 5%, that is, observing significant discrepancyeven if the null hypothesis is true. Because thenaive indirect estimates tend to be overprecise (seeChapter 5), the statistical discrepancies betweenthe direct and the naive indirect method will begreat, compared with those between the direct andthe adjusted indirect method.
Eleven of the 43 comparisons show statisticallysignificant discrepancy between the direct and thenaive indirect estimate. In seven of these 11examples with significant discrepancy, the pointestimate by the naive indirect comparison is in theopposite direction to that by the directcomparison. Because of the high frequency ofstatistically significant discrepancy andunpredictable direction of such discrepancy, thenaive indirect method should be avoided in theanalysis of data from randomised trials.
Direct versus adjusted indirect estimateDiscrepancies between the direct and the adjustedindirect estimate may also be due to random errors.However, because of the wider confidence intervalprovided by the adjusted indirect comparison, thediscrepancies between the direct and the adjustedindirect estimate are less likely to be statisticallysignificant than those between the direct and thenaive indirect estimate. Indeed, the statisticallysignificant results by using the direct comparisonoften become statistically non-significant in theadjusted indirect comparison (Table 11).
More importantly, perhaps, prognosticcharacteristics of participants in different studies
have been taken into account partially by theadjusted indirect method. The adjusted indirectmethod has partially preserved the rigour ofrandomisation by considering direct comparisonsof interventions of interest with the same controlintervention. An underlying assumption in thisadjusted method is that the relative efficacy of anintervention is consistent in participants indifferent studies. It is important to examine thegeneralisability of results of trials in the adjustedindirect comparison.
Of the 44 comparisons, three showed significantdiscrepancy (p < 0.05) between the direct andadjusted indirect estimates.27,40,107 These threecases will be further discussed in Chapter 7.
Combination of the direct and theadjusted indirect estimatesIt is often the case that direct evidence is availablebut not sufficient. In such cases, the adjustedindirect comparison may provide supplementaryinformation in evaluating relative efficacy ofcompeting interventions.53 Sixteen of the 44 directcomparisons included are based on onerandomised trial, while the adjusted indirectcomparisons were based on a median of 19 trials(range 2–86). Such a large amount of dataavailable for adjusted indirect comparisons couldbe used to strengthen conclusions based on thedirect comparisons, especially when there areconcerns about the methodological quality of thesingle direct comparison trial.
Results of the direct and the adjusted indirectcomparison could be quantitatively combined toincrease statistical power or precision, when thereis no important discrepancy between the twoestimates. The statistically non-significant relativeeffect estimated by the direct comparison maybecome statistically significant by combining thedirect and the adjusted indirect estimate, forexample, in two of the 44 meta-analyses (Figure 5).
It is also possible that the significant relative effectestimated by the direct comparison would becomenon-significant after being combined with theadjusted indirect estimate. H2RA versus sucralfatefor non-ulcer dyspepsia from a Cochranesystematic review provides an example.108 In thisexample, the direct comparison based on onerandomised trial found that H2RA was lesseffective than sucralfate [relative risk (RR) 2.74,95% CI 1.25 to 6.02], while the adjusted indirectcomparison indicates that H2RA was as effective assucralfate (RR 0.99, 95% CI 0.47 to 2.08). Thediscrepancy between the direct and the adjusted
Statistical discrepancy between the direct and indirect estimate: empirical evidence from published meta-analyses
48
indirect estimate is statistically significant(z = 1.84). The combination of the direct andadjusted indirect estimate provided a statisticallynon-significant relative risk of 1.56 (95% CI 0.93to 2.75). In cases like this, one may questionwhether it is appropriate to combine the twoestimates. If it is inappropriate, then one needs toinvestigate sources of significant discrepancy anddetermine which estimate is more believable.
SummaryForty-four analyses from 28 published meta-analyses were available to compare competinginterventions both directly and indirectly. Therewere considerable statistical discrepancies between the direct and the indirect estimate, but the direction of such discrepancy wasunpredictable. The relative efficacy may be
overestimated or underestimated by the indirectcomparison compared with results of the directcomparison.
The naive indirect comparison should be avoidedowing to its unpredictable nature and very highfrequency of statistically significant discrepanciesbetween the direct and the naive indirect estimate(11/43). In contrast, the adjusted indirect methodhas two advantages: partially taking intoconsideration patient prognostic characteristicsand wider confidence intervals. Empiricalevidence presented in this chapter has confirmedthese theoretical advantages. There is nostatistically significant discrepancy between thedirect and the adjusted indirect estimate in mostcases (41/44). When direct evidence is available butnot sufficient, the direct and the adjusted indirectestimate could be combined to obtain a moreprecise estimate.
Health Technology Assessment 2005; Vol. 9: No. 26
49
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
–2
–1
0
1
2T
reat
men
t effe
ct(9
5% c
onfid
ence
inte
rval
)
Direct estimateAdjusted indirectCombined estimate
Soo et al. Trindade et al.
FIGURE 5 Combining the direct and the adjusted indirect estimates in two meta-analyses
This chapter first examines five comparisonsfrom the sample of meta-analyses included in
Chapter 6.27,40,107–109 The reason for selecting thefive cases is that they showed a statisticallysignificant discrepancy (z > 1.64) between thedirect and the adjusted indirect estimate. It maybe interesting to note that four of these fiveexamples were about the treatment with H2RA orPPI.40,107–109
Following this, further empirical evidence isprovided about indirect comparisons, using asystematic review of antimicrobial prophylaxis incolorectal surgery.3
Five cases with significantdiscrepancyCase 1: Chiba and colleagues (1997)40
This meta-analysis evaluated speed of healing andsymptom relief in grade II–IV GORD. The reviewincluded only randomised trials but did notdirectly compare different interventions, althoughit has presented sufficient data for both direct andindirect comparison of H2RA and PPI.
By pooling data from 13 trials that comparedH2RA and PPI directly, the healing proportion inpatients receiving H2RA was lower than that inpatients receiving PPI (RR 0.56, 95% CI 0.48 to0.66). The adjusted indirect comparison betweenH2RA and PPI was carried out using 11 trials thatcompared H2RA versus placebo and two trials thatcompared PPI versus placebo (Figure 6). Theadjusted indirect estimate (RR 0.26, 95% CI 0.14to 0.48) was statistically significantly different fromthe direct estimate (z = 2.35). However, there maybe no important clinical implication associatedwith this statistically significant discrepancy. Thediscrepancy may be quantitative rather thanqualitative since both the direct and adjustedindirect estimates are in the same direction. In thisexample, the naive indirect estimate is alsostatistically significantly different from the directestimate (z = –2.95) (Figure 6a).
Figure 6(b) presents results of each interventiongroup from trials used in the direct and indirectcomparisons of H2RA and PPI. Patients receiving
placebo had worse outcome in two PPI trials(12.0%) than those receiving placebo in 11 H2RAtrials (35.3%). One explanation for thisobservation may be that the patients in the twoPPI trials were more severe than those in theH2RA trials. However, further investigation isimpossible because detailed data on patientcharacteristics were not available.
Case 2: Rostom and colleagues (2000)107
This systematic review assessed the effectiveness ofinterventions for the prevention of NSAID-induced upper gastrointestinal toxicity. Theinterventions of interest include misoprostol,H2RA and PPI. Although the authors of thisreview did not make any indirect comparisons, thedata from the individual studies are sufficient tocompare PPI and H2RA both directly andindirectly. In a trial that directly compared PPIand H2RA in reducing total endoscopic ulcers,significant difference in favour of PPI wasobserved (RR 0.28, 95% CI 0.15 to 0.51). Theadjusted indirect comparison was conducted usingthree trials that compared PPI versus placebo andsix trials that compared H2RA versus placebo.Significant discrepancy was observed between thedirect estimate and the adjusted indirect estimate,and between the direct and the naive indirectestimate (Figure 7).
In this review the authors concluded that low-doseH2RA was less effective than high-dose H2RA.Because low dose H2RA was used in the trial ofdirect comparison, the indirect comparisons werealso conducted after separating H2RA trials intosubsets of high dose and low dose (according toauthors’ original definition). Results of thesensitivity analysis indicate that the discrepancybetween the direct and the adjusted indirectestimate was reduced but still statisticallysignificant when low-dose H2RA was comparedwith PPI. The naive indirect estimate in thisexample is also significantly different from thedirect estimate. According to the naive indirectestimate, there is no significant difference in totalendoscopic ulcers between the PPI and H2RAtreatment.
Figure 7(b) presents the results of each interventiongroup from the trials included. It can be seen that
Health Technology Assessment 2005; Vol. 9: No. 26
51
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 7
Detailed case studies
patients receiving PPI had better outcome whilepatients receiving H2RA had worse outcome in thetrial that directly compared PPI and H2RA,compared with patients receiving the sameintervention in placebo-controlled trials.Table 12 presents some characteristics of studiesinvolved in the direct and indirect comparisons. Itseems that patients in PPI trials and in H2RA trialswere similar. The results of individual trials andmeta-analyses are shown in Figure 8. There was nosignificant heterogeneity across trials for any set oftrials. Thus, the causes of the statisticallysignificant discrepancy were unknown.
The observed statistical discrepancy between thedirect and adjusted indirect estimate may not beclinically important in this case. The results of the
direct and adjusted indirect method bothsuggested that PPI was superior to H2RA. Sincethe direct estimate was based on only one trial thatwas not double blinded, the relative efficacy of PPIversus H2RA in preventing NSAID-inducedendoscopic ulcers may not be as great as had beensuggested by the direct estimate.
Case 3: Soo and colleagues (2000)108
This Cochrane systematic review evaluatedpharmacological interventions for non-ulcerdyspepsia. The authors attempted indirectcomparisons of different drugs and used multipleregression in these (adjusted) indirectcomparisons. Only one comparison had sufficientdata for both a direct and an adjusted indirectestimate, and this is discussed below.
Detailed case studies
52
100
80
60
40
20
0
Hea
ling
prop
ortio
n (%
)
49.6
84.0
56.9
35.3
78.1
12.0
H2RA vs PPI H2RA vs placebo PPI vs placebo
0.1 1
RR (95% CI)
10
13 (731/884)
11 (1817/959) vs 2 (334/75)
11 (1817/) vs 2 (334)(a)
(b)
FIGURE 6 Chiba et al.:40 H2RA versus PPI for GORD (healing proportion). (a) Solid circle, direct estimate; square, adjusted indirect;cross, naive indirect. The numbers shown in the figure are the number of trials (patients). (b) Results of each intervention group fromtrials used in the direct and indirect comparison of H2RA and PPI.
One trial compared H2RA and sucralfate directlyand observed a significant difference in globalsymptom assessment between the twointerventions (RR 2.74, 95% CI 1.25 to 6.02). Theadjusted indirect comparison involved eightplacebo-controlled H2RA trials and two placebo-controlled sucralfate trials (Figure 9). The adjustedindirect estimate indicates no difference betweenH2RA and sucralfate (RR 0.99, 95% CI 0.47 to2.08). The discrepancy between the direct estimateand the adjusted indirect estimate is statisticallysignificant (z = 1.85) (Figure 10).
Figure 10(b) indicates that the result of sucralfatearm in the direct trial is extremely good comparedwith trials used in the adjusted indirectcomparison. This cannot be explained easily byfactors such as different underlying risk because
the result of H2RA arm in the direct trial wassimilar to that in the trials involved in the adjustedindirect comparison. The scrutiny ofcharacteristics of studies in Table 13 provided noobvious explanation for the observed discrepancybetween the direct and indirect estimates andheterogeneity among trials. In this case, thediscrepancy between the direct and adjustedindirect comparison was clinically important. Theresult of the direct comparison by the small andopen trial may not be more believable than theresult of the adjusted indirect comparison.
Case 4: van Pinxteren and colleagues(2000)109
This Cochrane systematic review evaluated short-term treatment for GORD-like symptoms. For thecomparison of empirical treatment of heartburn
Health Technology Assessment 2005; Vol. 9: No. 26
53
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
FIGURE 7 Rostom et al.:107 PPI versus H2RA for preventing chronic NSAID-induced upper gastrointestinal toxicity (endoscopic ulcers).(a) Solid circle, direct estimate; square, adjusted indirect; cross, naive indirect. The numbers shown in the figure are the number oftrials (patients). (b) Results of each intervention group from trials used in the direct and indirect comparison of H2RA and PPI.
RR (95% CI)
0.1 1 10
1 (210/215)
3 (443/331) vs 6 (645/634)
3 (443) vs 6 (645)
Indirect: adequate H2RA
Indirect: low H2RA
(a)
(b)
60
50
40
30
20
10
0
5.7
20.5
11.1
29.6
10.9
20.2
PPI vs H2RA PPI vs placebo H2RA vs placebo
Tot
al e
ndos
copi
c ul
cers
(%)
remission with PPI or H2RA, the direct estimate(RR 0.67, 95% CI 0.57 to 0.80) was statisticallysignificantly different from the adjusted indirectestimate (RR 0.45, 95% CI 0.31 to 0.66). Thisstatistical discrepancy may have no actual clinicalimportance in this review, since both estimatesindicated a significant benefit of PPI versus H2RAin the empirical treatment for heartburn remission(Figure 11).
Case 5: Zhang and Li-Wan-Po (1996)27
This meta-analysis evaluated the analgesic efficacyof paracetamol plus codeine in surgical pain. (Theauthor, WY Zhang, supplied data for individualstudies that were not available in the publishedarticle.) The direct comparison indicated thatparacetamol plus codeine was more efficacious
that paracetamol alone (mean difference in thesum of pain intensity difference 6.97, 95% CI 3.56to 10.37). The adjusted indirect comparison didnot show significant difference betweenparacetamol plus codeine and paracetamol alone(mean difference –1.16, 95% CI –6.95 to 4.64)(Figure 12).
Results of each intervention group from trialsinvolved in the direct and the indirectcomparisons are presented in Figure 12(b). Theimprovement in pain intensity was much less inpatients receiving placebo in placebo-controlledtrials of paracetamol plus codeine than in placebo-controlled trials of paracetamol alone. It suggeststhat patients in the placebo-controlled trials ofparacetamol plus codeine may be different to
Detailed case studies
54
Review: The validity of indirect comparisons for estimating the relative efficacy of competing interventionsComparison: 01 PPI vs H2RA for preventing NSAID-induced upper gastrointestinal ulcers (Rostom et al., 2000)Outcome: 01 Total endoscopic ulcers (3–12 months)
Studyor subcategory
Arm 1n/N
Arm 2n/N
RR (random)95% CI
RR (random)95% CI
01 PPI vs H2RA YeomansSubtotal (95% CI)Total events: 12 (Arm 1), 44 (Arm 2)Test for heterogeneity: not applicableTest for overall effect: Z = 4.10 (p < 0.0001)
02 PPI vs placebo Cullen Ekstrom HawkeySubtotal (95% CI)Total events: 49 (Arm 1), 98 (Arm 2)Test for heterogeneity: χ2 = 0.61, df = 2 (p = 0.74), I2 = 0%Test for overall effect: Z = 7.16 (p < 0.00001)
03 H2RA (high dose) vs placebo Hudson Taha Ten WoldeSubtotal (95% CI)Total events: 22 (Arm 1), 53 (Arm 2)Test for heterogeneity: χ2 = 0.39, df = 2 (p = 0.82), I2 = 0%Test for overall effect: Z = 4.03 (p < 0.0001)
04 H2RA (low dose) vs placebo Ehsanullah Levine TahaSubtotal (95% CI)Total events: 48 (Arm 1), 75 (Arm 2)Test for heterogeneity: χ2 = 0.38, df = 2 (p = 0.83), I2 = 0%Test for overall effect: Z = 2.67 (p = 0.007)
12/210 44/215 210 215
3/83 14/85 4/86 15/9142/274 69/155 443 331
10/39 21/39 9/97 24/93 3/15 8/15 151 147
10/151 17/14624/248 34/24814/95 24/93 494 487
0.28 (0.15 to 0.51)0.28 (0.15 to 0.51)
0.22 (0.07 to 0.74)0.28 (0.10 to 0.82)0.34 (0.25 to 0.48)0.33 (0.24 to 0.45)
0.48 (0.26 to 0.87)0.36 (0.18 to 0.73)0.38 (0.12 to 1.15)0.42 (0.27 to 0.64)
0.57 (0.27 to 1.20)0.71 (0.43 to 1.15)0.57 (0.32 to 1.03)0.63 (0.45 to 0.88)
0.1 0.2 0.5 1 2 5 10 Favours arm 1 Favours arm 2
FIGURE 8 Results of individual trials involved in the direct and indirect comparisons in the meta-analysis by Rostom et al.107
those in the placebo-controlled trials ofparacetamol alone. The possible difference inbaseline risk is taken into consideration in theadjusted indirect comparison, which indicates thatthere is no difference between paracetamol pluscodeine and paracetamol alone. When thedifferent baseline risk has not been considered,the naïve indirect comparison yields a result that isopposite to the direct estimate (Figure 12a).
In this case, the significant discrepancy may bedue to different types of surgical pain and/ordifferent doses of paracetamol/codeine used indifferent trials. When the analysis was restricted totrials of dental surgery, the discrepancy betweenthe direct and the adjusted indirect estimate wasstill significant (Table 14). Further scrutiny of theincluded trials found that the majority of the trials(n = 10) in the direct comparison used a dose of600–650 mg for paracetamol and 60 mg forcodeine, whereas many placebo-controlled trials
(n = 29) used a dose of 300 mg for paracetamoland 30 mg for codeine. When the analysisincluded only trials that used paracetamol600–650 mg and codeine 60 mg, the adjustedindirect estimate (5.72, 95% CI –5.37 to 16.81)was no longer significantly different from thedirect estimate (7.28, 95% CI 3.69 to 10.87). Thus,the significant discrepancy between the direct andthe indirect estimate based on all trials could beexplained by the fact that many placebo-controlledtrials used low doses of paracetamol (300 mg) andcodeine (30 mg). This example suggests that thesimilarity of trials involved in the adjusted indirectcomparison should be carefully assessed.
Relative efficacy of antimicrobialprophylaxis in colorectal surgery(Note: this section is based on an article publishedin Controlled Clinical Trials.87)
Health Technology Assessment 2005; Vol. 9: No. 26
55
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Review: The validity of indirect comparisons for estimating the relative efficacy of competing interventionsComparison: 02 H2RA vs sucralfate for non-ulcer dyspepsia (Soo et al., 2000)Outcome: 01 Global symptom assessment at the end of treatment
01 H2RA vs sucralfate MisraSubtotal (95% CI)Total events: 17 (Arm 1), 7 (Arm 2)Test for heterogeneity: not applicableTest for overall effect: Z = 2.51 (p = 0.01)
02 H2RA vs placebo Delattre Gotthard Hadi Hansen Kelbaek Nesland Saunders SingalSubtotal (95% CI)Total events: 199 (Arm 1), 304 (Arm 2)Test for heterogeneity: χ2 = 27.85, df = 7 (p = 0.0002), I2 = 74.9%Test for overall effect: Z = 2.25 (p = 0.02)
03 Sucralfate vs placebo Gudjonsson KairaluomaSubtotal (95% CI)Total events: 34 (Arm 1), 46 (Arm 2)Test for heterogeneity: χ2 = 3.19, df = 1 (p = 0.07), I2 = 68.7%Test for overall effect: Z = 0.99 (p = 0.32)
2.74 (1.25 to 6.02)2.74 (1.25 to 6.02)
0.53 (0.40 to 0.69)0.74 (0.53 to 1.04)0.03 (0.00 to 0.43)1.20 (0.88 to 1.64)1.19 (0.62 to 2.29)0.75 (0.53 to 1.06)0.50 (0.32 to 0.78)0.57 (0.32 to 0.99)0.70 (0.52 to 0.96)
1.03 (0.57 to 1.86)0.51 (0.32 to 0.83)0.71 (0.36 to 1.40)
0.1 0.2 0.5 1 2 5 10 Favours arm 1 Favours arm 2
Studyor subcategory
Arm 1n/N
Arm 2n/N
RR (random)95% CI
RR (random)95% CI
17/47 7/53 47 53
54/209 102/20929/63 34/55 0/26 17/2551/111 42/11011/24 10/2623/44 32/4621/103 48/11810/271 9/29 607 618
16/50 14/4518/79 32/72 129 117
FIGURE 9 Results of individual trials involved in the direct and indirect comparisons in the meta-analysis by Soo et al.108
In a systematic review of antibiotic prophylaxis forpreventing surgical wound infection aftercolorectal surgery,3 147 randomised trials wereidentified in which more than 70 differentantibiotics or combinations of antibiotics wereassessed. Only a limited number of antibiotics wasdirectly compared within the trials. If all 70options had been directly compared, over 2400trials would have been required, withoutconsidering different dosages, routes and timingof administration of the same drug.
Using this systematic review as an example, thissection aims to explore the potential usefulnessand limitations of indirect comparison inevaluating the relative efficacy of competinginterventions.
MethodFrom the systematic review of antimicrobialprophylaxis in colorectal surgery 11 sets of trials
were identified in which different antibiotics couldbe compared both directly and indirectly.3 Eachset of trials contains at least three trials that testedthree different antibiotics (or combinations ofantibiotics). For example, suppose a set of trialsincludes a trial that compared antibiotic A with B,a trial that compared A with C, and a trial thatcompared B with C. Then the trials in this setcould be used for three different comparisons: A versus B, A versus C and B versus C.
For each comparison the relative efficacy ofantimicrobial prophylaxis (i.e. odds ratio forsurgical wound infection) was estimated usingthree different methods: direct comparison,adjusted indirect comparison and naive indirectcomparison. The method suggested by Bucherand colleagues was used to carry out the adjustedindirect comparison.14 Results from more than onetrial were weighted by the inverse ofcorresponding variances and then quantitatively
Detailed case studies
56
RR (95% CI)
0.1 1 10
1 (47/53)
8 (607/618) vs 2 (129/117)
8 (607) vs 2 (129)
(a)
(b)
H2RA vs Sucr H2RA vs placebo Sucr vs placebo
60
50
40
30
20
10
0
Patie
nts
with
mod
erat
e or
seve
re d
yspe
psia
(%)
36.2
13.2
32.8
49.2
26.4
39.3
FIGURE 10 Soo et al.:108 H2RA versus sucralfate (Sucr) for non-ulcer dyspepsia (global symptom assessment). (a) Solid circle, directestimate; square, adjusted indirect; cross, naive indirect. The numbers shown in the figure are the number of trials (patients). (b) Results of each intervention group from trials used in the direct and indirect comparison of H2RA and sucralfate.
pooled to obtain an overall estimate. The naiveindirect method compared the results of singlearms included in the trials that had been used inthe adjusted indirect comparison. In the naiveindirect comparison, when relevant the results ofmore than one trial were pooled by adding up thenumber of surgical wound infections and thenumber of patients.
As an example, Table 15 presents data from a set ofthree trials that could be used to conduct threedifferent comparisons: cefuroxime plusmetronidazole (Cefur-M) versus co-amoxiclav (Co-A),110 cefuroxime plus metronidazole (Cefur-M)versus cefotaxime plus metronidazole (Cefot-M)111
and co-amoxiclav versus cefotaxime plusmetronidazole.112 Table 16 shows the results ofdifferent analyses for each of the three
comparisons. For instance, Cefur-M and Co-Awere directly compared in the trial conducted byPalmer and colleagues.110 The indirectcomparison of Cefur-M and Co-A was based onthe other two trials that included the commonintervention Cefot-M.111,112 Likewise, the directcomparison of Cefur-M and Cefot-M was based onthe results obtained by Rowe-Jones andcolleagues,111 and the indirect comparison wasbased on the two trials with a commonintervention, Co-A.110,112
The results of the two indirect methods werecompared with the results from the gold standardof direct comparison. In this example, thediscrepancy was defined as the absolute value ofdifference in log odds ratio between the directmethod and the indirect method. To take into
Health Technology Assessment 2005; Vol. 9: No. 26
57
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
0
10
20
30
40
50
60
70
80
Hea
rtbu
rn r
emiss
ion
(%)
PPI vs H2RA H2RA vs placeboPPI vs placebo
46.7
66.1
24.2
69.8
45.4
59.4
(b)
(a)
0.1 1 10
3 (1228/664)
1 (161/159) vs 2 (511/502)
1 (161) vs 2 (511)
RR (95% CI)
FIGURE 11 van Pinxteren et al.:109 PPI versus H2RA for reflux disease-like symptoms (heartburn remission). (a) Solid circle, directestimate; square, adjusted indirect; cross, naive indirect. The numbers shown in the figure are the number of trials (patients). (b) Results of each intervention group from trials used in the direct and indirect comparison of H2RA and PPI.
account the precision of estimated discrepancy, the corresponding z statistic was calculated bydividing the difference in log odds ratio by thesquare root of the sum of the variances. The cut-off point for statistical significance wasarbitrarily considered to be z = 1.64 (or two-sidedp-value = 0.10).
The difference in log odds ratio can be convertedinto the ratio of odds ratios (i.e. the antilogarithmof difference in log odds ratio). For example, whenthe difference in log odds ratio is 1.0, thecorresponding ratio of odds ratios is 2.72. It canbe mathematically shown that the absolute valueof discrepancy between the adjusted indirectmethod and the direct method will be the samefor three different comparisons using the same setof trials. For example, it was 1.636 for all threecomparisons in Table 16 (this example correspondsto trial set 5 in Figure 13).
ResultsThe results presented in Figure 13 indicate thatconsiderable discrepancies exist between the directand indirect comparisons. In each set of trials, thediscrepancies between the direct and the adjustedindirect method were the same in three differentcomparisons. In contrast, the discrepanciesbetween the direct and the naive indirectcomparisons were unpredictable and variedgreatly across the different comparisons.
The discrepancies between the direct and thenaive indirect comparisons may be either smalleror greater than those between the direct and theadjusted indirect comparisons (Figure 13),depending on which interventions have beencompared using a given set of trials. In nine of the11 sets of trials, there is a naive indirectcomparison with a greater discrepancy than theadjusted indirect comparison. The significant
Detailed case studies
58
22.63
9.6712.74
4.12
17.01
24.00
Para + Code vs Para
Para + Code vs Placebo
Para vs Placebo
40
35
30
25
20
25
21
5
0
Mea
n di
ffere
nce
(sum
of d
iffer
ence
in p
ain
inte
nsity
)
Mean difference (95% CI)
–15 –10 –5 0 5 10
13
12 vs 31
12 vs 31(a)
(b)
FIGURE 12 Zhang and Li Wan Po:27 paracetamol (Para) plus codeine (Code) versus paracetamol alone in surgical pain (analgesicefficacy). (a) Solid circle, direct estimate; square, adjusted indirect; cross, naïve indirect. The numbers shown in the figure are thenumber of trials (patients). (b) Results of each intervention group from used in the direct and indirect comparison of paracetamol pluscodeine and paracetamol alone.
Health Technology Assessment 2005; Vol. 9: No. 26
59
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
12
Char
acte
ristic
s of
stu
dies
invo
lved
in t
he d
irect
and
the
indi
rect
com
paris
ons
of P
PI a
nd H
2RA
for p
reve
ntin
g N
SAID
-indu
ced
uppe
r gas
troi
ntes
tinal
tox
icity
(to
tal e
ndos
copi
c ul
cers
)107
Stud
yIn
terv
enti
on (
dura
tion
)Pa
rtic
ipan
tsQ
ualit
ya
Yeom
ans,
199
8O
mep
razo
le 2
0 m
g pe
r da
yPa
tient
s w
ith R
A a
nd O
A w
ho n
eede
d m
aint
enan
ce t
reat
men
t af
ter
succ
essf
ul
Jada
d sc
ale:
3Ra
nitid
ine
2×
150
mg
trea
tmen
t of
ulc
ers
or e
rosio
ns in
eith
er t
he s
tom
ach
or d
uode
num
Allo
catio
n (6
mon
ths)
Mea
n ag
e 56
yea
rsco
ncea
lmen
t: D
Hel
icon
acte
r pyl
ori:
posit
ive:
50%
Cul
len,
199
8O
mep
razo
le 2
0 m
g pe
r da
yPa
tient
s ta
king
NSA
IDs
regu
larly
, chr
onic
ally
and
abo
ve d
efin
ed m
inim
um d
oses
Jada
d sc
ale:
2Pl
aceb
oA
lloca
tion
(6 m
onth
s)co
ncea
lmen
t: D
Ek
stro
m, 1
996
Om
epra
zole
20
mg
per
day
Patie
nts
with
dys
peps
ia o
r hi
stor
y of
pep
tic u
lcer
dise
ase
Jada
d sc
ale:
3Pl
aceb
oM
ean
age
58 y
ears
Allo
catio
n (3
mon
ths)
Prev
ious
pep
tic u
lcer
: 27%
conc
ealm
ent:
D
Hel
icob
acte
r pyl
orip
ositi
ve: 5
3%H
awke
y, 1
998
Om
epra
zole
20
mg
Patie
nt w
ith R
A a
nd O
A. M
aint
enan
ce t
hera
py fo
r pa
tient
s in
who
m t
reat
men
t w
as
Jada
d sc
ale:
3M
isopr
osto
l 400
�g
succ
essf
ulA
lloca
tion
Plac
ebo
Out
patie
nts
conc
ealm
ent:
D(6
mon
ths)
Mea
n ag
e 58
yea
rsLe
ngth
of N
SAID
s: >
6 m
onth
sPr
evio
us p
eptic
ulc
ers:
29%
Hel
icob
acte
r pyl
orip
ositi
ve: 4
2%H
udso
n, 1
997
Fam
otid
ine
40 m
g b.
d. (h
igh
dose
)Pa
tient
s w
ith R
A o
r O
A. N
SAID
use
rs w
ith h
eale
d ul
cers
Jada
d sc
ale:
3Pl
aceb
oH
elic
obac
ter p
ylor
ipos
itive
: 18/
39A
lloca
tion
(6 m
onth
s)co
ncea
lmen
t: D
Ta
ha, 1
996
Fam
otid
ine
20 m
g b.
d. (l
ow d
ose)
Patie
nts
with
out
pept
ic u
lcer
s w
ho w
ere
rece
ivin
g lo
ng-t
erm
NSA
ID t
hera
py fo
r Ja
dad
scal
e: 4
Fam
otid
ine
40 m
g b.
d. (h
igh
dose
)RA
(82%
) or
OA
Allo
catio
n Pl
aceb
oco
ncea
lmen
t: D
(6
mon
ths)
Ten
Wol
de, 1
996
Rani
tidin
e 30
0 b.
d. (h
igh
dose
)Pa
tient
s w
ith R
A a
nd a
hist
ory
of p
eptic
ulc
er d
iseas
eJa
dad
scal
e: 3
Plac
ebo
Allo
catio
n (T
he s
tudy
was
sto
pped
afte
r a
conc
ealm
ent:
D
blin
ded
inte
rim a
naly
sis)
Ehsa
nulla
h, 1
988
Rani
tidin
e 15
0 m
g b.
d. (l
ow d
ose)
Patie
nts
with
RA
or
OA
age
d >
18 y
ears
with
out
lesio
ns in
the
sto
mac
h an
d Ja
dad
scal
e: 5
Plac
ebo
duod
enum
at
base
line
endo
scop
y (a
fter
1 w
eek
with
out
taki
ng N
SAID
s).
Allo
catio
n (2
mon
ths)
Tho
se t
akin
g ot
her
antir
heum
atic
age
nts,
con
com
itant
ulc
erog
enic
dru
gs o
r tr
eatm
ent
conc
ealm
ent:
D
for
pept
ic u
lcer
s w
ithin
the
pre
viou
s 30
day
s w
ere
excl
uded
Mea
n ag
e 57
yea
rsLe
vine
, 199
3N
izat
idin
e 15
0 m
g b.
d.Pa
tient
s w
ith O
A w
ho w
ere
taki
ng N
SAID
s. E
ndos
copy
to
rule
out
the
pre
senc
e Ja
dad
scal
e: 3
Plac
ebo
of a
n ac
ute
ulce
r A
lloca
tion
(3 m
onth
s)co
ncea
lmen
t: D
aA
s co
ded
in t
he o
rigin
al r
evie
w.10
7
OA
, ost
eoar
thrit
is; R
A, r
heum
atoi
d ar
thrit
is.
Detailed case studies
60 TA
BLE
13
Char
acte
ristic
s of
stu
dies
invo
lved
in t
he d
irect
and
the
indi
rect
com
paris
ons
of H
2RA
and
sucr
alfa
te fo
r non
-ulc
er d
yspe
psia
108
Stud
yIn
terv
enti
ons
Part
icip
ants
Qua
litya
Misr
a, 1
992
Rani
tidin
e 15
0 m
gPa
tient
s w
ith 1
mon
th o
f abd
omin
al s
ympt
oms
refe
rabl
e to
the
upp
er G
I tra
ct.
Rand
omise
d, o
pen,
(In
dia)
Sucr
alfa
te 1
g q
.d.
Patie
nts
with
GO
RD o
r IB
S w
ere
excl
uded
. 87%
com
plet
ed t
rial
cont
rolle
d cl
inic
al t
rial
(4 w
eeks
) A
lloca
tion
conc
ealm
ent:
BD
elat
tre,
198
5C
imet
idin
e 20
0 m
g q.
d.Pa
tient
s w
ith n
on-u
lcer
dys
peps
ia
RCT,
dou
ble-
blin
d,
(USA
)Pl
aceb
o pl
aceb
o-co
ntro
lled,
(4
wee
ks)
mul
ticen
tre
tria
l A
lloca
tion
conc
ealm
ent:
B G
otth
ard,
198
8C
imet
idin
e 40
0 m
g b.
d.3/
12 o
f dys
peps
ia o
f unk
now
n or
igin
. Aci
d ou
tput
stu
dies
per
form
ed.
RCT,
dou
ble-
blin
d,
(Sw
eden
)A
ntac
id 1
0 m
l q.d
. 16
% d
uode
nitis
pl
aceb
o-co
ntro
lled
tria
l Pl
aceb
oA
lloca
tion
conc
ealm
ent:
B (6
wee
ks)
Had
i, 19
89Ra
nitid
ine
300
mg
daily
Dur
atio
n of
dys
peps
ia u
ncle
ar. G
astr
itis
on a
ll O
GD
. Dro
pout
rat
e fo
r pl
aceb
o RC
T, d
oubl
e-bl
ind,
pla
cebo
-(In
done
sia)
Plac
ebo
was
23%
and
for
rani
tidin
e w
as 4
%
cont
rolle
d tr
ial
(4 w
eeks
)A
lloca
tion
conc
ealm
ent:
B H
anse
n, 1
998
Cisa
prid
e 10
mg
t.d.
Prim
ary
care
rec
ruitm
ent.
Mea
n du
ratio
n of
dys
pept
ic s
ympt
om w
as 8
8 m
onth
s.
RCT,
dou
ble-
blin
d, p
lace
bo-
(Den
mar
k)N
izat
idin
e 30
0 m
g no
cte
Four
sub
grou
ps: u
lcer
-like
(13%
), re
flux-
like
(23%
), dy
smot
ility
-like
(46%
) and
co
ntro
lled
tria
l Pl
aceb
o un
clas
sifie
d (1
8%).
Incl
uded
sup
erfic
ial e
rosio
ns o
n O
GD
. 85%
com
plet
ed t
rial
Allo
catio
n co
ncea
lmen
t: A
(2
wee
ks)
Kelb
aek,
198
5C
imet
idin
e 20
0 m
g t.d
. and
Pr
imar
y ca
re r
ecru
itmen
t. Pa
tient
s w
ith 1
mon
th o
f epi
gast
ric p
ain.
Aci
d ou
tput
RC
T, d
oubl
e-bl
ind,
pla
cebo
-(D
enm
ark)
400
mg
noct
e st
udie
s pe
rfor
med
. Had
OG
D. 1
4 pa
tient
s w
ho w
ere
sym
ptom
free
at
end
of
cont
rolle
d tr
ial
Plac
ebo
trea
tmen
t ha
d 3
mon
ths
of fo
llow
-up.
96%
com
plet
ed t
rial
Allo
catio
n co
ncea
lmen
t: B
(3 w
eeks
)N
esla
nd, 1
985
Cim
etid
ine
400
mg
b.d.
Patie
nts
with
6 m
onth
s of
pre
dom
inan
tly u
lcer
-like
pai
n w
ith e
rosiv
e pr
epyl
oric
RC
T, d
oubl
e-bl
ind,
pla
cebo
-(N
orw
ay)
Plac
ebo
chan
ges.
90%
com
plet
ed t
rial
cont
rolle
d tr
ial
(4 w
eeks
)A
lloca
tion
conc
ealm
ent:
BSa
unde
rs, 1
986
Rani
tidin
e 15
0 m
g b.
d.
Prim
ary
care
rec
ruitm
ent.
88%
com
plet
ed t
rial.
One
-yea
r fo
llow
-up,
but
the
RC
T, d
oubl
e-bl
ind,
pla
cebo
-(U
K)
Plac
ebo
resu
lts in
clud
ed o
ther
pep
tic d
iseas
e co
ntro
lled
mul
ticen
tre
tria
ls (6
wee
ks)
Allo
catio
n co
ncea
lmen
t: A
Si
ngal
, 198
9C
imet
idin
e 40
0 m
g b.
d.Pa
tient
s w
ith 1
mon
th o
f prim
ary
sym
ptom
of u
pper
abd
omin
al d
iscom
fort
. RC
T, d
oubl
e-bl
ind,
pla
cebo
-(In
dia)
Plac
ebo
IBS
excl
uded
co
ntro
lled
tria
l (4
wee
ks)
Allo
catio
n co
ncea
lmen
t: B
Gud
jons
son,
199
3 Su
cral
fate
1 g
q.d
.Pr
ivat
e pr
actic
e re
crui
tmen
t. Pa
tient
s w
ith 1
mon
th o
f dys
peps
ia. H
ad O
GD
. RC
T, d
oubl
e-bl
ind,
pla
cebo
(Ic
elan
d)
Plac
ebo
91%
com
plet
ed t
rial
cont
rolle
d tr
ial
(3 w
eeks
)A
lloca
tion
conc
ealm
ent:
BKa
iralu
oma,
198
7Su
cral
fate
1 g
t.d
. Pa
tient
s w
ith 3
mon
ths
of d
yspe
psia
. Had
OG
D. 6
% h
ad d
uode
nitis
. 86%
RC
T, d
oubl
e-bl
ind,
pla
cebo
-(F
inla
nd)
Plac
ebo
com
plet
ed t
rial
cont
rolle
d tr
ial
(4 w
eeks
)A
lloca
tion
conc
ealm
ent:
D
aA
s co
ded
in t
he o
rigin
al r
evie
w10
8
GI,
gast
roin
test
inal
; IBS
, irr
itabl
e bo
wel
syn
drom
e; O
GD
, oes
opha
goga
stro
duod
enos
copy
.
Health Technology Assessment 2005; Vol. 9: No. 26
61
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TABLE 14 Direct and adjusted indirect estimates of efficacy of paracetamol plus codeine versus paracetamol alone for surgical pain27
Trials Mean difference in standardised score for the sum of pain Discrepancy intensity difference (95% CI) between the
direct and adjusted Direct comparison Adjusted indirect comparison indirect estimates
All trials 6.97 (3.56 to 10.37) –1.16 (–6.95 to 4.64) 8.13 (p = 0.018)n = 13 n1 = 12, n2 = 31
Dental surgery trials only 7.07 (3.37 to 10.78) –1.40 (–8.27 to 5.46) 8.47 (p = 0.033)n = 11 n1 = 7, n2 = 15
Trials that used paracetamol 7.28 (3.69 to 10.87) 5.72 (–5.37 to 16.81) 1.56 (p = 0.793)600–650 mg and codeine n = 10 n1 = 2, n2 = 1260 mg
n, n1 and n2 are the number of trials used in the direct comparison and indirect comparison, respectively. Trials included dental surgery, episiotomy, postpartum uterine cramp, orthopaedic and other surgery. Doses of paracetamolranged from 300 to 1000 mg and the dose of codeine was 30 or 60 mg.
TABLE 15 Example: a set of trials that could be used to compare different antibiotic regimens both directly and indirectly forpreventing surgical wound infection in colorectal surgery
Trial Cefur-M Co-A Cefot-M SWI/n SWI/n SWI/n
Palmer et al., 1994110 2/79 8/69 –
Rowe-Jones et al., 1990111 33/454 – 32/453
Kwok et al., 1993112 – 7/76 8/88
n, number of patients; SWI, surgical wound infection.
TABLE 16 Example: results of different methods for each comparison between antibiotics, using the trials presented in Table 15
Comparison Direct method Naive indirect method Adjusted indirect methodlnOR (95% CI) lnOR (95% CI) lnOR (95% CI)
Cefur-M vs Co-A –1.619 (–3.205 to –0.034) –0.258 (–1.112 to 0.596) 0.016 (–1.162 to 1.194)∆ = 1.361 ∆ = 1.636z = 1.481 z = 1.623
Cefur-M vs Cefot-M 0.031 (–0.474 to 0.535) –1.348 (–2.929 to 0.233) –1.605 (–3.514 to 0.305)∆ = –1.379 ∆ = –1.636z = –1.629 z = –1.623
Co-A vs Cefot-M 0.0144 (–1.050 to 1.079) 0.545 (–0.275 to 1.365) 1.650 (–0.014 to 3.314)∆ = 0.531 ∆ = 1.636z = 0.775 z = 1.623
lnOR, log odds ratio; ∆, difference in log odds ratio between the indirect method and the direct method.The z statistic was calculated by dividing the difference in log odds ratio by the square root of the sum of the variances.
discrepancies (|z| ≥ 1.64) occurred in five of the11 sets for the naive indirect comparison (45%)and two of the 11 for the adjusted indirectcomparison (18%).
The observed discrepancy between the direct andthe indirect comparison may be explained by
many possible factors, including chance error,study quality and other factors associated with theinternal and external validity of studies. Becauseof the relatively small sample size (number of trialsand patients) in many sets of trials, thediscrepancies between the direct and the indirectcomparisons may have been exacerbated in this
Detailed case studies
62
0.0(1.0)
1.0(2.7)
2.0(7.4)
3.0(20.1)
4.0(54.6)
5.0(148.4)
1 3 510
2 7 959
3 3 310
4 4 1173
5 3 1219
6 3 297
7 4 857
8 3 460
9 3 557
10 3 364
11 4 396
Tria
l set
No.
of
tria
ls
No.
of
Patie
nts
Absolute difference in log odds ratio(Ratio of odds ratios)
FIGURE 13 Absolute difference in log odds ratio between the direct and the adjusted indirect comparisons, and between the directand the naive indirect comparisons. Solid bars, difference between the direct and the adjusted indirect comparison; open bars,difference between the direct and the naive indirect comparison; dotted lines, standard error of difference in log odds ratio. Results arefrom 11 sets of trials of antimicrobial prophylaxis in colorectal surgery. For each set of trials, the absolute discrepancies by the adjustedindirect comparisons are the same and therefore only one solid bar is presented. Conversely, for each set of trials, absolutediscrepancies by three naive indirect comparisons often vary greatly.
example. In general, the discrepancy between thedirect and the adjusted indirect comparisonshould be investigated by examining the externaland internal validity of the trials involved.
SummaryFive examples from Chapter 6Although statistically significant discrepancy hasbeen observed between the direct and the adjustedindirect estimate, these examples indicate thatsuch discrepancy may not be clinically important.The direct and the adjusted indirect estimates maybe in the same direction and the difference isquantitative rather than qualitative inthree40,107,109 of the five examples. The remainingtwo examples suggest that the discrepancybetween the direct and the adjusted indirectestimate may be clinically important,27,108 in whichstatistically significant relative efficacy observed bythe direct comparison was not confirmed by theadjusted indirect comparison.
Study-level characteristics reported in thepublished meta-analyses are not sufficient fordetailed investigation of sources of significant
discrepancy between the direct and the indirectestimate of relative efficacy of competinginterventions.
Example of antimicrobial prophylaxis incolorectal surgeryFrom a systematic review of antimicrobialprophylaxis in colorectal surgery, 11 sets ofrandomised trials were identified that could beused to compare antibiotics both directly andindirectly. The discrepancy between the direct andthe indirect comparison is defined as the absolutevalue of difference in log odds ratio.
Considerable discrepancies exist between thedirect and the adjusted indirect comparisons. Theadjusted indirect comparison has the advantagesthat the prognostic factors of participants indifferent trials can be partially taken into accountand more uncertainty be incorporated into itsresult by providing a wider confidence interval.The findings of this section indicate that thediscrepancy between the direct and the naiveindirect estimate is more unpredictable and morelikely to be statistically significant than thatbetween the direct and the adjusted indirectestimate.
Health Technology Assessment 2005; Vol. 9: No. 26
63
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Indirect comparisons based onrandomised trials: currentpracticeIndirect comparisons are commonly used forevaluating the relative effectiveness of alternativeinterventions, with approximately 9.5% of meta-analyses of RCTs identified through DAREincluding some form of indirect comparison(Chapter 3). Even when only randomised trials areincluded, the strength of the randomisationprocedure is weakened when making indirectcomparisons because of the need to combineinformation across separate sets of trials.
Indirect comparisons are sometimes carried outimplicitly and the results interpreted as if fromdirect comparisons within randomised trials.Indeed, the findings of direct comparisons aresometimes ignored, even when data are available.The use of inferior indirect methods and theinappropriate interpretation of results of indirectcomparison may result in misleading estimates ofrelative efficacy of competing healthcareinterventions. Poor methods of analysis could evenyield results that are inferior to those obtainedthrough non-randomised studies (as discussedbelow).
The survey of published meta-analyses indicatesthat results obtained through indirect comparisonsare not always consistent with the findingsobtained by direct comparisons. Data available in28 identified meta-analyses allowed 44 analyses tobe undertaken comparing competinginterventions both directly and indirectly (Chapter6). Indirect comparisons can be broadly classifiedas either naive or adjusted. The naive approachrefers to the comparison of results of single armsbetween different trials. In the adjusted indirectcomparison, the comparison of the interventionsof interest is adjusted by the results of their directcomparison with a common control group (e.g.placebo). Empirical evidence from the 40 analysesundertaken in Chapter 6 shows the naiveapproach to be a highly unpredictable method formaking indirect comparisons, with a very highfrequency of statistically significant discrepanciesfrom the direct estimate. By contrast, for adjustedindirect comparisons there was no clear evidence
of significant differences from the direct estimatesbeyond expectation by chance. However, theamount of direct randomised evidence was ratherlimited in many of the systematic reviews in thesample.
Performance of different methodsof making indirect comparisonsMany systematic reviews were found to includeindirect comparisons, yet few reviewers performformal analyses. Perhaps this lack reflects the factthat there are few publications discussing theanalysis of such data (Chapter 4). While the validmethods make assumptions that cannot easily betested, it may be better to use these explicit formalapproaches than for reviewers (and readers) tomake such comparisons informally.
The recommended methods can all be describedas adjusted indirect comparisons, in which twotreatments are compared via their relative effectversus a common comparator. The main types ofanalysis are the simple weighted combination ofseparate estimates (e.g. as suggested by Bucherand colleagues14), meta-regression and generalisedlinear models (e.g. logistic regression). Althoughseveral examples were found of the use of a naivemethod in published reviews, no methodologicalpaper with a recommendation to use this strategywas found.
The simulation studies (Chapter 5) showed that allof these methods are unbiased and will give theright answer on average across many suchapplications. However, the correctness of thestandard error of the estimated treatment effectwill depend on strong but unverifiableassumptions. Little difference was found in theperformance of fixed effect methods for adjustedindirect comparisons, but they tended to giveconfidence intervals that were too narrow. Partlyfor this reason, and also in view of additionalstudy heterogeneity compared with the usual caseof direct evidence only, a random effects methodwill usually be appropriate if formal meta-analysisis applied (with the usual provisos regarding theapplication of such models, such as having enoughtrials).
Health Technology Assessment 2005; Vol. 9: No. 26
65
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 8
Discussion
The analyses described in Chapter 5 were allbased on the data from a single, large trial, with aunified protocol and hence consistent inclusioncriteria. Thus, the usual sources of heterogeneitywere not present, except for variation in location(country) and some associated case-mix variation.In general, one would expect more variationbetween independent component studiescontributing to an indirect comparison as a resultof variation in aspects of the study design. Inparticular, there may be differences between thetwo sets of trials being combined with respect tothe participants and perhaps also regarding theactual treatments given.
The naive approach, in which the numbers areadded as if there was just one trial making eachcomparison, is problematic even when consideringdirectly randomised trials, and may severelymislead owing to ‘Simpson’s paradox’.113 Forindirect comparisons, such inappropriatecombination is compounded by the discarding ofwithin-trial comparison groups, such that thewhole point of randomisation is lost. The resultsof such analysis are completely untrustworthy, andnaive comparisons should never be made.
Quality of evidence from RCTs(direct comparisons)Evidence from RCTs is, in general, considered tobe the best. However, such evidence may beimperfect for several reasons. First, the problem ofpatient comparability exists in RCTs.114 Thebaseline comparability of patients between groupsmay become problematic owing to a lack ofallocation concealment in randomised trials.115 Itis also possible that, purely by chance, the patientsrandomly allocated to different groups havedifferent prognostic characteristics, particularlywhen the sample size is small.
Second, after participants have been enrolledthere is considerable possibility that biases (such asperformance bias, attrition bias and detection bias)may be introduced into trials. For example, patientsmay drop out after randomisation for variousreasons or may be excluded from the analysis.Such withdrawal or exclusion may not be randomor balanced across groups. Therefore, even thoughthe patients in different treatment groups arecomparable at the time of randomisation, thatcomparability may disappear owing to non-random exclusion or withdrawal. In addition, lackof blinding in assessment of the outcome may leadto overestimation of treatment effects.90,115
Third, empirical evidence has confirmed thatpublished randomised trials may be a biasedsample of all trials that have been conducted,owing to publication and related biases.116 Trialswith non-significant or negative results are lesslikely to be published than trials with significant orpositive results. Recently, additional evidence hasdemonstrated the added problem of selectivereporting of outcomes.117 Therefore, evidence fromrandomised trials still needs to be interpreted withcaution and the possibility of publication biasshould be investigated and excluded if possible.
Finally, there may be considerable difference inresults among randomised trials, especially smalltrials. Empirical evidence indicates thatsubsequent large trials may sometimes overturnconclusions based on small published trials.118–120
It has been argued that “randomisation is notsufficient for comparability”.119,121 Here,comparability refers not only to baseline patientcharacteristics, but also to other studycharacteristics. It should not be assumed that theadjective ‘randomised’ is a guarantee of highquality. As a consequence, it is recommended thatmethodological quality is assessed in a systematicreview, although there is no consensus on how todo this or how to make use of the information.122
Nonetheless, the problems just described applyalso to non-RCTs, so while the methodology ofdirectly randomised trials may not always be ideal,such trials do offer the most reliable evidence oftreatment efficacy (especially if studies ofunacceptably poor quality are discounted).
Thus, when there is a substantial or modestamount of direct evidence the customary practiceof ignoring non-randomised studies and indirectrandomised evidence will generally be correct.Inevitably, however, in some circumstances theremay be few or no trials that have compareddirectly treatments of particular interest. Thus, thepossibility of using indirect evidence becomesimportant. This report has examined methods forperforming indirect comparisons, focusing onindirect evidence from RCTs. The next sectionsconsider issues relating to the use of indirectevidence when direct randomised evidence iseither insufficient or non-existent.
Similar problems may apply when performingsubgroup analyses within a meta-analysis of trialsall making the same direct comparison. If thesubgrouping factor relates to the intervention,such as the dose of active treatment, or choice ofcomparator treatment, then comparison of
Discussion
66
subgroups is an indirect comparison of exactly thesame type as discussed.
What to do when direct evidenceis available but insufficientEven when some direct randomised evidence isavailable it is often insufficient. In such cases theadjusted indirect comparison may providesupplementary information in evaluating relativeefficacy of competing interventions.53 Fourteen ofthe 40 direct comparisons in Chapter 3 are basedon one randomised trial, and a median of 18 trials(range 2–96) is incorporated in the correspondingadjusted indirect comparison (Appendix 3). Suchlarge amounts of data available for adjustedindirect comparisons could be used to attempt tostrengthen conclusions based on the directcomparisons. Results of the direct and theadjusted indirect comparison can be quantitativelycombined to increase statistical power or precision.The statistically non-significant relative effectestimated by the direct comparison may becomestatistically significant by combining the direct andthe adjusted indirect estimate, as happened inthree of the 40 comparisons (Table 11).
It is also possible that the significant relative effectestimated by the direct comparison becomes non-significant after it has been combined with theadjusted indirect estimate. H2RA versus sucralfatefor non-ulcer dyspepsia from a Cochranesystematic review is an example.108 Here, thedirect comparison based on one randomised trialfound that H2RA was less efficacious thansucralfate (RR 2.74, 95% CI 1.25 to 6.02), whilethe adjusted indirect comparison indicates thatH2RA was as effective as sucralfate (RR 0.99, 95%CI 0.47 to 2.08). The discrepancy between thedirect and the adjusted indirect estimates ismarginally statistically significant (z = 1.84). Thecombination of the direct and adjusted indirectestimate provided a statistically non-significantrelative risk of 1.56 (95% CI 0.93 to 2.75).
A general question raised by cases like this is:‘when is it appropriate to combine direct andindirect estimates?’ Bucher and colleagues14
concluded that only where direct comparisons areunavailable should indirect comparison meta-analysis be carried out. This is certainly thestandard approach, but it is not clear that this willalways be the best advice. Even when there isdirect randomised evidence, there may be farmore information available from indirectcomparisons. For example, Song and colleagues87
examined a case in which one head-to-headrandomised trial and six trials contributed to anindirect comparison (via two differentcomparators). In addition, the use of indirect datamay be particularly indicated when it is felt thatthe methodological quality of the trials makingdirect comparisons is low.
It will be a matter of judgement whether and howto take into account the indirect evidence.Although a combination of direct and indirectcomparisons may appear to strengthenconclusions (by increasing the quantity of data),the increase in precision must be balanced againsta loss of confidence in the certainty with whichbias is avoided. Few would argue that direct andindirect estimates should always be combined.Rather, many would feel that while presentingseparate estimates is necessary, combination willonly sometimes be suitable. Some criteria areneeded on which to base such a judgement. A verysimilar problem arises in other contexts withinsystematic reviews; for example, reviewers ponderwhether it is reasonable to combine parallel andcross-over trials, or in epidemiologicalinvestigations whether to combine case–controland cohort studies. It is not desirable to base suchdecisions on whether or not the differencebetween the two estimates is statisticallysignificant, although this is the easiest approach. Amore constructive approach would be to base thedecision on the similarity of the participants in thedifferent trials and the comparability of theinterventions. Such judgement applies to thedirectly randomised trials and each subset of trialscontributing to the indirect comparison.
What to do when there is nodirect evidenceIt is not unusual to find that different treatmentoptions have not been directly compared withinrandomised trials, and conclusions on relativeefficacy often end up based entirely on indirectevidence. The adjusted indirect method may beespecially useful to obtain some indirect evidenceabout the relative efficacy of competinginterventions. The validity of the indirect estimatewas discussed in the previous section. Althoughthe absence of any direct evidence avoids thequestion of whether to combine, it means that allof the available evidence is indirect. The reliabilityof that evidence is then of particular concern.
Evidence from an adjusted indirect comparisonshould be interpreted with caution. The internal
Health Technology Assessment 2005; Vol. 9: No. 26
67
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
validity of the trials involved in the adjustedindirect comparison should be examined becausebias in trials will inevitably infect the results of theadjusted indirect comparison. In addition, for theadjusted indirect estimate to be valid, the keyassumption here is that the relative efficacy of anintervention is consistent in patients included indifferent trials. That is, the estimated relativeefficacy should be generalisable. However,generalisability (external validity) of trial results isoften questionable because of restricted inclusioncriteria, exclusion of patients and the higher levelof clinical settings where trials were carried out.123
Such assessments are more complex still whencomparing sets of trials evaluating differenttreatment comparisons. As in many othersituations, interpretation is a matter of judgementand there are no rules applicable across allcircumstances.
Are indirect comparisons of RCTspreferable to direct comparisonsfrom non-randomised trials?As noted above, in the absence of directrandomised evidence one could seek eitherindirect randomised evidence or non-randomisedstudies that examine directly the comparison ofinterest. The authors have not found or generatedany empirical evidence to investigate this issue, sothis discussion relies on knowledge of themechanisms that would render both types ofcomparison biased, and the comparativelikelihood that such mechanisms exist.
Non-randomised evaluations of healthcareinterventions involve making comparisonsbetween two groups of participants who receivedifferent interventions. If there are any otherdifferences between the groups that are themselveslinked to outcome, the comparison will beconfounded and potentially biased. For example,the case-mix in the participant groups could differin terms of age, gender, disease severity or co-morbidities, there could be other differences intreatments between the groups, or the way inwhich outcomes are defined and assessed couldvary. Judgement of the validity of the comparisondepends on the degree to which one is assuredthat ‘like is being compared with like’ such thatone knows that there are no differences betweenthe groups in all factors other than theintervention received. Although many devices areused in non-randomised studies to reduce thepotential for such confounding (such asmeasurement of change scores, matching,
stratification and statistical risk adjustment ofresults), the degree to which these are successful inany individual study is largely inestimable.Empirical studies have found, however, thatadjustment may fail to remove the bulk ofselection bias.15 In addition, as there is always alikelihood of prognostic factors that are unknownor unmeasurable, adjustment methods cannotcope with all eventualities. Indirect comparisonsalso feature comparisons between non-randomisedgroups; however, in this instance the groups arenot cohorts receiving one or other treatment, butrandomised trials within which differentcomparisons are made. Now the same differencesin case-mix, concomitant therapy and follow-upcould exist between the trials in an indirectcomparison in the same way as they do betweenthe groups within a non-randomised study.However, even if the magnitudes of thesedifferences are similar (and there are reasons toargue that they are most likely to be less inindirect comparisons), there are reasons to believethat the bias they introduce would be less than in anon-randomised study.
For binary data the mechanism by which thisprediction is reached is as follows. Variations inpotentially confounding factors will change theaverage event rate observed in the trial as they doin a non-randomised study, as they relate to thefrequency of outcome. However, it is likely thattheir impact on outcome would affect both groupsin each trial in a proportionate manner. If thishappens, and if the treatment effect is calculatedusing an appropriate metric (probably a risk ratioor an odds ratio), very little variation in thetreatment effect will be observed between trials ofthe same comparison among participants withvarying baseline risks. When this is the case theindirect comparison will be unbiased. Thisobservation is fundamental to the practice ofmeta-analysis.57
The case of continuous outcomes seems not tohave been considered in the same light. Althoughthe broad concerns are the same, the impact ofeffect modifiers, including case-mix variation, isunpredictable; it may work in additive ormultiplicative manners, affecting either or both ofthe mean and standard deviation.
An indirect comparison will, however, be biased ifthe differences in baseline risk between trials arelinked to differences in the observed treatmenteffect. This would occur if any of the factorsvarying between the trials are known effectmodifiers (or, in meta-analysis terminology, sources
Discussion
68
of heterogeneity). Thus, it would be wrong to claim,based on the argument of constancy of treatmenteffects, that indirect comparisons are alwaysunbiased. There are examples where such effectmodifiers exist such that indirect comparisons willbe biased. Therefore, it will always be desirable toshow similar distributions of confounding factors inthe trials included in an indirect comparison,rather than rely on the assumption of constancy ofeffect across varying baseline risk.
The resampled results (Chapter 5), despitedemonstrating heterogeneity between trials,probably underestimate the potential for such bias,as all the trials in the reconstructed analysis usedexactly the same protocol. Such uniformity ofprotocol is unlikely in reality.
In conclusion, bias is less likely in indirectcomparisons than within a non-randomised study,as an indirect comparison between treatmenteffects estimated in trials with different baselinerisks is not necessarily biased, whereas the samedifference in baseline risks between groups in anon-randomised comparison is sufficient to renderit biased. The present authors thus agree with therelative placing of these levels of evidence byMcAlister and colleagues,5 as discussed inChapter 1.
Comparing multiple interventionssimultaneouslyThe focus of this review and empirical researchhas been the use of indirect comparisons tocompare two prespecified healthcareinterventions. As noted in Chapter 1, reviewerssometimes consider simultaneously severalinterventions with the natural desire to saysomething about their relative efficacy. The resultsof such a review may be presented in the form of aleague table of efficacy.
This situation was not studied in the presentempirical work, although it did consider indirectcomparison of two specific treatments from trialsinvolving four different treatments (Chapter 5).Here, some different situations within this broadframework are briefly summarised, and anindication is given of how analysis might proceed.Distinction is made between two rather differentcontexts in which multiple treatments areconsidered simultaneously.
First, there may be several competinginterventions, each of which has been compared
against the same comparator in one or moreRCTs. The most obvious example is where each ofseveral drugs has been evaluated in placebo-controlled trials. There are several such examplesin pain research.124,125 The common comparatormakes the comparison of the various drugs seemrelatively simple. In the particular case of trials ofpain relief there is also much greater consistencyof trial methodology (notably standardisedoutcome measure) than is seen in most medicalareas. However, the quoted results are often simplyseparate meta-analyses of each active drug versusplacebo, ranked by treatment effect (perhapsNNT); this display does not allow an easyassessment of the difference between any twoparticular treatments, and so may give a falseimpression of their relative merits. Also, when datasets are presented in this way, there may be asuspicion that any direct comparisons have beenomitted to simplify the presentation.
Sometimes researchers making such comparisonsuse the naive approach by summarising treatmentarms across trials,6 or ignore the control groupsaltogether.46 The strong warnings given againstthe naive approach apply even more strongly inthe multiple treatment case.
This data structure is similar to that where thereare several trials of the same intervention versuscontrol, but where that intervention was deliveredin different ways in different trials; for example,dose, regimen or route may vary across trials.However, the aim of meta-analysis in such cases isusually to evaluate the treatment against thecomparator rather than to study explicitly theimportance of the mode of delivery.
The second case is where there are multiplestudies that have each compared two or more of aset of treatments, but without a commoncomparator. This more general situation isexemplified by the trial of antithrombotic therapyconsidered in Chapter 4 and also a set of trials ofthrombolytic therapy discussed by Hasselblad andKong,68 which is shown in Table 17.
The aim here may be to estimate the effect of aparticular drug versus placebo when no such trialhas been done, but such data may also be the basisof an attempt to rank the treatments. Data such asthese can arise when new treatments areintroduced over a period of several years, an effectseen more clearly in the subset of trials shown inTable 18. This subset perhaps makes it clearer thatestimation is based on a chain of inference. Forexample, adjusted indirect comparisons or logistic
Health Technology Assessment 2005; Vol. 9: No. 26
69
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
regression could be used to estimate the effect ofaccelerated t-PA versus placebo. Such an analysiswould link the results of five trials, three of whichdid not investigate either of the treatments ofinterest. Like all chains, its strength depends onthe weakest link. Here TEAM-3 was a very smalltrial of 325 patients, with an imprecise estimate oftreatment effect: it would therefore beunsatisfactory to rely on this chain of inference.ISIS-3 enrolled over 17,000 patients to make thesame comparison, so there would be no need torely on TEAM-3. The amount of information ineach component of such an inferential chainwould certainly be a concern, even though the
uncertainty of the final estimate would takesample size into account. Other concerns wouldinclude the methodological quality and the degreeof comparability of the treatments, participantsand protocols of the component trials. Hasselbladand Kong rather underplay the assumptionsbehind such an analysis. They suggest that themethod is acceptable as long as there is a commonsubpopulation of participants in the differenttrials.68 In fact, the more links there are thestronger will be the assumptions of adjustedincorrect comparisons. Recently, other authorshave proposed new models for analysing suchnetworks of comparisons.76,88
Discussion
70
TABLE 17 Treatments studied in 17 randomised trials reporting 30-day mortality examining thrombolytic therapy for patients withacute coronary syndrome (treatment within 6 hours of onset) (see Hasselblad and Kong68 for further details)
Trial Placebo t-PA APSAC SK RPA Accelerated t-PA
ASSENT � �ECGS � �AIMS � �Bassand-1 � �GISSI � �ISIS-2 � �ISAM � �TEAM-3 � �Bassand-2 � �GISSI-2 � �ISG � �ISIS-3 � � �TIMI-4 � �TAPS � �INJECT � �GUSTO I � �GUSTO III � �
APSAC, anisoylated plasminogen streptokinase activator complex; RPA, reteplase; SK, streptokinase; tPA, tissue plasminogenactivator.
TABLE 18 Subset of trials from Table 17, showing sequential evaluation of new treatments
Trial Placebo t-PA APSAC SK RPA Accelerated t-PA
ASSENT � �TEAM-3 � �ISIS-3 � �INJECT � �GUSTO III � �
APSAC, anisoylated plasminogen streptokinase activator complex; RPA, reteplase; SK, streptokinase; tPA, tissue plasminogenactivator.
Implications for practice ofsystematic reviewsWhen conducting systematic reviews to evaluatethe effectiveness of interventions, direct evidencefrom good-quality RCTs should be used whereverpossible. If no such evidence exists a call forfurther trials may be necessary (if they are deemedethical). If further research is not feasible, it maybe necessary to look for direct comparisons innon-randomised studies and/or indirectcomparisons from RCTs, which would requireadditional searches of the literature. The reviewerneeds, however, to be aware that both aresusceptible to bias, although bias is less likely inindirect comparisons. The use of non-randomisedstudies within a systematic review is, in addition,perhaps more problematic in terms of identifyingthe studies. The development of better indexingand sensitive search strategies is required to aidthe identification of study designs other thanRCTs. It should also be noted, however, that thedevelopment of search strategies for theidentification of RCTs may actually exclude datathat could be used when making indirectcomparisons, particularly if a search is drug ortreatment specific. Ideally, indirect comparisonsshould be prespecified in a review’s protocol andthe search strategy developed so as to include allrelevant drug/treatment comparisons.
When making indirect comparisons in a systematicreview, the reviewer needs to be clear that that iswhat they are doing, and interpret the resultsappropriately. The naive approach, comparing theresults of single arms between trials, should beavoided as empirical evidence shows it to have ahigh frequency of statistically significantdiscrepancies from the direct estimate. Ideally, anadjusted indirect comparison method should beused, using the random effects model (given theincrease in study heterogeneity). The main typesof adjusted indirect comparison, showing littledifference in performance, are the adjustedindirect comparison, meta-regression and logisticregression.
If both direct and indirect comparisons arepossible within a review, these should be doneseparately before considering whether to pool
data. Whether or not the two sets of informationare combined, or when only indirect comparisonsare available, interpretations should be made evenmore cautiously than in a standard meta-analysisof just head-to-head randomised trials, in view ofthe partially observational nature of thecomparisons.
Implications for clinical researchIndirect comparisons can be used to overcomecertain problems that may arise in the design andconduct of an RCT. An example can be seen in aproposed European multicentre trial of surgicaltechniques for repairing cleft lip and palate.126
Each centre approached used a different surgicaltechnique and was only willing to participate in thetrial if their ‘preferred’ surgical technique was oneof the treatment arms to which patients would berandomised. To overcome this, a common protocolwas devised that allowed each centre to randomiseto one of two surgical techniques: their usual,preferred technique, and an alternative techniquethat was common to all participating centres. Thealternative technique was planned to be used as thecommon comparator, allowing adjusted indirectcomparisons to be made between the centres’preferred surgical techniques. A head-to-headcomparison of the different surgical techniquesmay have been preferable, but the indirectcomparison was able to provide an acceptabledesign for the surgeons, who otherwise may nothave been willing to participate in the trial.
When considering the use of an indirectcomparison when designing such a multicentreRCT, consideration should be given to devising aunified protocol with consistent inclusion criteriaacross all centres.
Recommendations formethodological researchThe majority of the indirect comparisonsidentified were made within meta-analyses ofbinary data. There is a need for evaluation ofalternative methods for analysis of indirectcomparisons for continuous data.
Health Technology Assessment 2005; Vol. 9: No. 26
71
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Chapter 9
Conclusions
In addition, there is a need for empirical researchinto how different methods of indirect comparisonperform in cases where there is a large treatmenteffect.
Further research is required to consider how todetermine when it is appropriate to look at indirectcomparisons and how to judge when to combineboth direct and indirect comparisons. Researchinto how evidence from indirect comparisonscompares to evidence from non-randomisedstudies may also be warranted. (This question hasnot been considered in the current project.)
The empirical investigations (Chapter 5) werebased on one large, multicentre trial101 with acommon protocol across each centre. It would beuseful to repeat the investigations using individualpatient data from a meta-analysis of several RCTsusing different protocols.
The odds ratio was used as the measure of effectwithin the simulation study. Although logisticregression calls for the effect measure to be theodds ratio, it would be interesting to evaluate theimpact of choosing different binary effectmeasures for the inverse variance method.
Conclusions
72
The authors would like to thank Kath Wright(CRD, York) for her assistance with the
development of the electronic search strategies,and members of the International Stroke TrialCollaborative Group for allowing data from theirmulticentre RCT to be used for the empiricalinvestigations within this report. We would alsolike to thank the referees who provided valuablecomments on the draft report.
Contribution of authorsDG Altman (Professor of Statistics in Medicine), JJ Deeks (Senior Medical Statistician), AJ Eastwood (Senior Research Fellow), AM Glenny(Lecturer in Evidence Based Oral Health Care)and F Song (Reader in Research Synthesis)developed the structure of the report.
AJE and AMG coordinated the project.
In developing the search strategy, the initialsearches were developed and undertaken by
Kath Wright (Centre for Reviews and Dissemination,University of York) and the update searches wereundertaken by AMG.
Handsearching was carried by DGA (Chapter 4),AJE (Chapter 3), AMG (Chapter 3), and FS(Chapters 3 and 6).
DGA, JJD, AJE, AMG and FS carried out thescreening of search results and retrieved papersagainst inclusion criteria.
The analysis of data was carried out by DGA(Chapters 4 and 5), M Bradburn (Statistician)(Chapter 5), R D’Amico (Research Fellow)(Chapter 5), JJD (Chapters 4 and 5), AJE (Chapter 3), AMG (Chapter 3), C Sakarovitch(Biostatistician) (Chapter 5) and FS (Chapters 3, 6 and 7).
Production of the full report was carried out byDGA, JJD, AJE, AMG and FS.
Health Technology Assessment 2005; Vol. 9: No. 26
73
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Acknowledgements
1. Pocock SJ. Clinical trials: a practical approach. NewYork: John Wiley; 1996.
2. Moher D, Schulz KF, Altman D. CONSORT Group(Consolidated Standards of Reporting Trials). TheCONSORT statement: revised recommendationsfor improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987–91.
3. Song F, Glenny AM. Antimicrobial prophylaxis incolorectal surgery: a systematic review ofrandomised controlled trials. Health Technol Assess1998;2(7).
4. de Craen AJM, Tijssen JGP, de Gans J, Kleijnen J.Placebo effect in the acute treatment of migraine:subcutaneous placebos are better than oralplacebos. J Neurol 2000;247:183–8.
5. McAlister FA, Laupacis A, Wells GA, Sackett DL.Users’ Guides to the Medical Literature: XIX.Applying clinical trial results B. Guidelines fordetermining whether a drug is exerting (morethan) a class effect. JAMA 1999;282:1371–7.
6. Felson DT, Anderson JJ, Meenan RF. Thecomparative efficacy and toxicity of second-linedrugs in rheumatoid arthritis. Results of twometaanalyses. Arthritis Rheum 1990;33:1449–61.
7. Garg R, Yusuf S, for the Collaborative Group onACE Inhibitor Trials. Overview of randomizedtrials of angiotensin-converting enzyme inhibitorson mortality and morbidity in patients with heartfailure. JAMA 1995;273:1450–6.
8. Homocysteine Lowering Trialists’ Collaboration.Lowering blood homocysteine with folic acid basedsupplements: meta-analysis of randomised trials.BMJ 1998;316:894–8.
9. Boersma E, Maas AC, Deckers JW, Simoons ML.Early thrombolytic treatment in acute myocardialinfarction: reappraisal of the golden hour. Lancet1996;348:771–5.
10. Moore A, McQuay H, Gray M, editors. Bandolier.Oxford: Pain Relief Unit. URL:http://www.jr2.ox.ac.uk/bandolier/
11. Tramer MR, Moore RA, McQuay HJ. Preventionof vomiting after paediatric strabismus surgery: asystematic review using the numbers-needed-to-treat method. Br J Anaesth 1995;75:556–61.
12. de Craen AJM, Vickers AJ, Tijssen JGP, Kleijnen J.Number-needed-to treat and placebo-controlledtrials. Lancet 1998;351:310.
13. Murray DW, Britton A, Bulstrode CJ.Thromboprophylaxis and death after total hip
replacement. J Bone Joint Surg Am 1996;78-B:863–70.
14. Bucher HC, Guyatt GH, Griffith LE, Walter SD.The results of direct and indirect treatmentcomparisons in meta-analysis of randomizedcontrolled trials. J Clin Epidemiol 1997;50:683–91.
15. Deeks JJ, Dinnes J, D’Amico R, Sowden AJ,Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health TechnolAssess 2003;7(27).
16. Antiplatelet Trialists’ Collaboration. Collaborativeoverview of randomised trials of antiplatelettherapy. II. Maintenance of vascular graft orarterial patency by antiplatelet therapy. BMJ 1994;308:159–68.
17. Antiplatelet Trialists’ Collaboration. Collaborativeoverview of randomised trials of antiplatelettherapy. I. Prevention of death, myocardialinfarction, and stroke by prolonged antiplatelettherapy in various categories of patients. BMJ1994;308:81–106.
18. Antiplatelet Trialists’ Collaboration. Collaborativeoverview of randomised trials of antiplatelettherapy. III. Reduction in venous thrombosis andpulmonary embolism by antiplatelet prophylaxisamong surgical and medical patients. BMJ 1994;308:235–46.
19. Lowenthal A, Buyse M. Secondary prevention ofstroke: does dipyridamole add to aspirin? ActaNeurol Belg 1994;94:24–34.
20. Li Wan Po A, Zhang WY. Systematic overview of co-proxamol to assess analgesic effects of additionof dextropropoxyphene to paracetamol. BMJ1997;315:1565–71.
21. Marshall JK, Irvine EJ. Rectal corticosteroidsversus alternative treatments in ulcerative colitis: a meta-analysis. Gut 1997;40:775–81.
22. Matchar DB, McCrory DC, Barnett HJ, Feussner MD. Medical treatment for strokeprevention. Ann Intern Med 1994;121:41–53.
23. Milne RJ, Olney RW, Gamble GD, Turnidge J.Tolerability of roxithromycin vs erythromycin incomparative clinical trials in patients with lowerrespiratory tract infections. Clinical DrugInvestigation 1997;14:405–17.
24. Moore A, Collins S, Carroll D, McQuay H.Paracetamol with and without codeine in acutepain: a quantitative systematic review. Pain1997;70:193–201.
Health Technology Assessment 2005; Vol. 9: No. 26
75
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
References
25. Pope JE, Anderson JJ, Felson DT. A meta-analysisof the effects of nonsteroidal anti-inflammatorydrugs on blood pressure. Arch Intern Med 1993;153:477–84.
26. Poynard T, Leroy V, Cohard M, Thevenot T,Mathurin P, Opolon P, et al. Meta-analysis ofinterferon randomized trials in the treatment ofviral hepatitis C: effects of dose and duration.Hepatology 1996;24:778–89.
27. Zhang WY, Li Wan Po A. Analgesic efficacy ofparacetamol and its combination with codeine andcaffeine in surgical pain – a meta-analysis. J ClinPharm Ther 1996;21:261–82.
28. Piccinelli M, Pini S, Bellantuono C, Wilkinson G.Efficacy of drug treatment in obsessive–compulsivedisorder: a meta-analytic review. Br J Psychiatry1995;166:424–43.
29. Zalcberg JR, Siderov J, Simes J. The role of 5-fluorouracil dose in the adjuvant therapy ofcolorectal cancer. Ann Oncol 1996;7:42–6.
30. Leizorovicz A. Comparison of the efficacy andsafety of low molecular weight heparins andunfractionated heparin in the initial treatment ofdeep venous thrombosis. An updated meta-analysis. Drugs 1996;7:30–7.
31. Pignon JP, Arriagada R, Ihde DC, Johnson DH,Perry MC, Souhami RL, et al. A meta-analysis ofthoracic radiotherapy for small-cell lung cancer. N Engl J Med 1992;327:1618–24.
32. Non-small Cell Lung Cancer Collaborative Group.Chemotherapy in non-small cell lung cancer; ameta-analysis using updated data on individualpatients from 52 randomised clinical trials. BMJ1995;311:899–909.
33. Rossouw JE. Lipid-lowering interventions inangiographic trials. Am J Cardiol 1995;76:86–92C.
34. Koch M, Dezi A, Ferrario F, Capurso L. Preventionof nonsteroidal anti-inflammatory drug-inducedgastrointestinal mucosal injury. Arch Intern Med1996;156:2321–32.
35. Holme I. Cholesterol reduction and its impact oncoronary artery disease and total mortality. Am JCardiol 1995;76:10–17C.
36. Moore RA, McQuay HJ. Single-patient data meta-analysis of 3453 postoperative patients: oraltramadol versus placebo, codeine and combinationanalgesics. Pain 1997;69:287–94.
37. Aro J. Cardiovascular and all-cause mortality inprostatic cancer patients treated with estrogens ororchiectomy as compared to the standardpopulation. Prostate 1991;18:131–7.
38. Poynard T, Lemaire M, Agostini H. Meta-analysisof randomized clinical trials comparinglansoprazole with ranitidine or famotidine in thetreatment of acute duodenal ulcer. Eur JGastroenterol Hepatol 1995;7:661–5.
39. Lefering R, Neugebauer EAM. Steroid controversyin sepsis and septic shock: a meta-analysis. CritCare Med 1995;23:1294–303.
40. Chiba N, de-Gara CJ, Wilkinson FM, Hunt RH.Speed of healing and symptom relief in grade IIto IV gastroesophageal reflux disease: a meta-analysis. Gastroenterology 1997;112:1798–810.
41. Coulter A, Kelland J, Peto V, Rees MC. Treatingmenorrhagia in primary care: an overview of drugtrials and a survey of prescribing practice. Int JTechnol Assess Health Care 1995;11:456–71.
42. Bansal VK, Beto JA. Treatment of lupus nephritis:a meta-analysis of clinical trials. Am J Kidney Dis1997;29:193–9.
43. Schmieder RE, Martus P, Klingbeil A. Reversal ofleft ventricular hypertrophy in essentialhypertension. A meta-analysis of randomizeddouble-blind studies. JAMA 1996;275:1507–13.
44. Unge P, Berstad A. Pooled analysis of anti-Helicobacter pylori treatment regimens. Scand JGastroenterol Suppl 1996;220:27–40.
45. Imperiale TF, Speroff TS. A meta-analysis ofmethods to prevent venous thromboembolismfollowing total hip replacement. JAMA 1994;271:1780–5.
46. Conlin PR, Spence JD, Williams B, Ribeiro AB,Saito I, Benedict C, et al. Angiotensin IIantagonists for hypertension: are there differencesin efficacy? Am J Hypertens 2000;13:418–26.
47. Fibrinolytic Therapy Trialists’ (FTT) CollaborativeGroup. Indications for fibrinolytic therapy insuspected acute myocardial infarction:collaborative overview of early mortality and majormorbidity results from all randomised trials ofmore than 1000 patients. Lancet 1994;343:311–22.
48. Ekbom A, Taube A. Methodological flaws in studyon prostatic cancer. Prostate 1994;25:53–4.
49. NHS Centre for Reviews and Dissemination.Undertaking systematic reviews of research oneffectiveness. CRD guidelines for those carrying out orcommissioning reviews. Report No. 4. York: NHSCentre for Reviews and Dissemination; 1996.
50. Berkey CS, Anderson JJ, Hoaglin DC. Multiple-outcome meta-analysis of clinical trials. Stat Med1996;15:537–57.
51. Dominici F, Parmigiani G, Wolpert RL, Hasselblad V. Meta-analysis of migraine headachetreatments: combining information fromheterogeneous designs. Journal of the AmericanStatistical Association 1999;94:16–28.
52. Hasselblad V. Meta-analysis of multitreatmentstudies. Med Decis Making 1998;18:37–43.
53. Higgins JP, Whitehead A. Borrowing strength fromexternal trials in a meta-analysis. Stat Med1996;15:2733–49.
References
76
54. Hirotsu C, Yamada Y. Estimating odds ratiosthrough the connected comparative experiments.Communications in Statistics – Theory and Methods1999;28:905–29.
55. Gleser LJ, Olkin I. Meta-analysis for 2 × 2 tableswith multiple treatment groups. In Stangl DK,Berry DA, editors. Meta-analysis in medicine andhealth policy. New York: Marcel Dekker; 2001. pp. 179–89.
56. Gleser LJ, Olkin I. Stochastically dependent effect sizes. In Cooper H, Hedges LV, editors. The handbook of research synthesis. New York: RussellSage; 1994. pp. 340–55.
57. Deeks JJ. Issues in the selection of a summarystatistic for meta-analysis of clinical trials withbinary outcomes. Stat Med 2002;21:1575–600.
58. Davey Smith G, Egger M. Going beyond the grandmean: subgroup analysis in meta-analysis ofrandomised trials. In Egger M, Davey Smith G,Altman DG, editors. Systematic reviews in healthcare.Meta-analysis in context. 2nd ed. London: BMJBooks; 2001. pp. 143–56.
59. Thompson SG. Why and how sources ofheterogeneity should be investigated. In Egger M,Davey Smith G, Altman DG, editors. Systematicreviews in healthcare. Meta-analysis in context. 2nd ed.London: BMJ Books; 2001. pp. 157–75.
60. Deeks JJ, Altman DG, Bradburn MJ. Statisticalmethods for examining heterogeneity andcombining results from several studies in meta-analysis. In Egger M, Davey Smith G, Altman DG,editors. Systematic reviews in health care: Meta-analysisin context. London: BMJ Books; 2001. pp. 285–312.
61. Altman DG, Bland JM. Interaction revisited: thedifference between two estimates. BMJ 2003;326:219.
62. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken andinterpreted? Stat Med 2002;21:1559–73.
63. Anderson S, Hauck WW. The transitivity ofbioequivalence testing: potential for drift. Int JClin Pharmacol Ther 1996;34:369–74.
64. Hauck WW, Anderson S. Some issues in the designand analysis of equivalence trials. Drug InformationJournal 1999;33:109–18.
65. Mainland D. The treatment of clinical and laboratorydata. Edinburgh: Oliver and Boyd; 1938. pp. 151–3.
66. Armitage P, Berry G. The sampling error of adifference. In Statistical methods in medical research.2nd ed. Oxford: Blackwell; 1987. pp. 88–90.
67. Eddy DM, Hasselblad V, Schachter R. Meta-analysisby the confidence profile method. Boston, MA:Academic Press; 1992.
68. Hasselblad V, Kong DF. Statistical methods forcomparison to placebo in active–control trials.Drug Information Journal 2001;35:435–49.
69. Packer M, Antonopoulos GV, Berlin JA, Chittams J,Konstam MA, Udelson JE. Comparative effects ofcarvedilol and metoprolol on left ventricularejection fraction in heart failure: results of a meta-analysis. Am Heart J 2001;141:899–907.
70. Fisher LD, Gent M, Butler HR. Active–controltrials: how would a new agent compare withplacebo? A method illustrated with clopidogrel,aspirin, and placebo. Am Heart J 2001;141:26–32.
71. Baker SG, Kramer BS. The transitive fallacy forrandomized trials: if A bests B and B bests C inseparate trials, is A better than C? BMC Med ResMethodol 2002;2:13.
72. Clarke MJ, Stewart LA. Obtaining data fromrandomised controlled trials: how much do weneed for reliable and informative meta-analysis?BMJ 1994;309:1007–10.
73. Higgins JP, Whitehead A, Turner RM, Omar RZ,Thompson SG. Meta-analysis of continuousoutcome data from individual patients. Stat Med2001;20:2219–41.
74. Smith CT, Williamson PR, Marson AG.Investigating heterogeneity in an individualpatient data meta-analysis of time to eventoutcomes Stat Med 2005;24:1307–19.
75. Brown H, Prescott R. Applied mixed models inmedicine. Chichester: John Wiley; 1999.
76. Ades AE. A chain of evidence with mixedcomparisons: models for multi-parametersynthesis and consistency of evidence. Stat Med2003;22:2995–3016.
77. Sutton AJ, Abrams KR, Jones DR, Sheldon TA,Song F. Methods for meta-analysis in medical research.Chichester: John Wiley; 2000.
78. Thompson SG, Sharp SJ. Explainingheterogeneity in meta-analysis: a comparison ofmethods. Stat Med 1999;18:2693–708.
79. Lim E, Ali A, Routledge T, Edmonds L, AltmanDG, Large S. Indirect comparison meta-analysis ofaspirin therapy after coronary surgery. BMJ 2003;327:1309–11.
80. Lim E. First principles or evidence based critique?URL: http://bmj.bmjjournals.com/cgi/eletters/327/7427/1309#46233 (accessed September 2004).
81. Spiegelhalter DJ, Abrams KR, Myles JP. Bayesianapproaches to clinical trials and health-care evaluation.Chichester: John Wiley; 2004.
82. Phillips A. Trial and error: cross-trial comparisonsof antiretroviral regimens. AIDS 2003;17:619–23.
83. Senn S. Active control equivalence trials. Statisticalissues in drug development. Chichester: John Wiley;2002. pp. 207–17.
Health Technology Assessment 2005; Vol. 9: No. 26
77
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
84. Siegel JP. Equivalence and non-inferiority trials.Am Heart J 2000;139:S166–70.
85. ICH Harmonised Tripartite Guideline. Choice ofcontrol group and related issues in clinical trials.URL: http://www.nihs.go.jp/dig/ich/efficacy/e10/e10step4.pdf (accessed March 2005).
86. Engels EA, Schmid CH, Terrin N, Olkin I, Lau J.Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses.Stat Med 2000;19:1707–28.
87. Song F, Glenny AM, Altman DG. Indirectcomparison in evaluating relative efficacy –illustrated by antimicrobial prophylaxis in colorectalsurgery. Control Clin Trials 2000;21:488–97.
88. Lumley T. Network meta-analysis for indirecttreatment comparisons. Stat Med 2002;21:2313–24.
89. Chan A-W, Altman DG. Epidemiology andreporting of randomised trials published inPubMed journals. Lancet 2005;365:1159–62.
90. Lambert PC, Sutton AJ, Jones DR. A comparisonof summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol 2002;55:86–94.
91. Berlin JA, Santanna J, Schmid CH, Szczech LA,Feldman HI. Individual patient- versus group-leveldata meta-regression for the investigation oftreatment effect modifiers: ecological bias rears itsugly head. Stat Med 2002;21:371–87.
92. Turner RM, Omar RZ, Yang M, Goldstein H,Thompson SG. A multilevel model framework formeta-analysis of clinical trials with binaryoutcomes. Stat Med 2000;19:3417–32.
93. Begg CB, Pilote L. A model for incorporatinghistorical controls into a meta-analysis. Biometrics1991;47:899–906.
94. Li Z, Begg CB. Random effects models forcombining results from controlled anduncontrolled studies in a meta-analysis. Journal ofthe American Statistical Association 1994;89:1523–7.
95. Raghunathan TE. Pooling controls from differentstudies. Stat Med 1991;10:1417–26.
96. Büchner T, Dohner H, Ehninger G, Ganser A,Hasford J. Up-front randomization and commonstandard arm: a proposal for comparing AMLtreatment strategies between different studies.Leuk Res 2002;26:1073–5.
97. Büchner T, Dohner H, Ehninger G, Ganser A,Niederwieser D, Hasford J. Cross-trial networkingin AML: a step forward rather than corner cutting.Leuk Res 2004;28:649–50.
98. Hills RK, Richards SM, Wheatley K. Cornercutting compromises clinical trials: the inherentproblems with up-front randomisation and acommon standard arm. Leuk Res 2003;27:1071–3.
99. Tsong Y, Wang SJ, Hung HM, Cui L. Statisticalissues on objective, design, and analysis ofnoninferiority active-controlled clinical trial. J Biopharm Stat 2003;13:29–41.
100. Wang SJ, Hung HM, Tsong Y. Utility and pitfallsof some statistical methods in active controlledclinical trials. Control Clin Trials 2002;23:15–28.
101. Sandercock PAG, Collins R, Counsell C, Farrell B,Peto R, Slattery J, et al. The International StrokeTrial (IST): a randomised trial of aspirin,subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke. Lancet1997;349:1569–81.
102. Sterne JAC, Bradburn MJ, Egger M. Meta-analysisin StataTM. In Egger M, Davey Smith G, AltmanDG, editors. Systematic reviews in health care. Meta-analysis in context. London: BMJ Books; 2001. pp. 347–85.
103. Bradburn MJ, Deeks JJ, Altman DG. Metan-analternative meta-analysis command (sbe24)[correction appears in Stata Technical Bulletin, No. 45, 21 (sbe24.1)]. Stata Technical Bulletin 1998;44:4–15.
104. Sharp S. sbe23: Meta-analysis regression. StataTechnical Bulletin 1998;42:16–24.
105. Robinson LD, Jewell NP. Some surprising resultsabout covariate adjustment in logistic-regressionmodels. International Statistical Review 1991;59:227–40.
106. Song F, Altman DG, Glenny AM, Deeks JJ. Validityof indirect comparison for estimating efficacy ofcompeting interventions: empirical evidence frompublished meta-analyses. BMJ 2003;326:472–5.
107. Rostom A, Wells G, Tugwell P, Welch V, Dube C,McGowan J. Prevention of chronic NSAID inducedupper gastrointestinal toxicity. In The CochraneLibrary (Issue 3). Oxford: Update Software; 2000.
108. Soo S, Moayyedi P, Deeks J, Delaney B, Innes M,Forman D. Pharmacological interventions for non-ulcer dyspepsia. In The Cochrane Library (Issue 3).Oxford: Update Software; 2000.
109. van Pinxteren B, Numans ME, Bonis PA, Lau J.Short-term treatment with proton pumpinhibitors, H2-receptor antagonists and prokineticsfor gastro-oesophageal reflux disease-likesymptoms and endoscopy negative reflux disease.In The Cochrane Library (Issue 3). Oxford: UpdateSoftware; 2000.
110. Palmer BV, Mannur KR, Ross WB. An observerblind trial of co-amoxiclav versus cefuroxime plusmetronidazole in the prevention of postoperativewound infection after general surgery. J Hosp Infect1994;26:287–92.
111. Rowe-Jones DC, Peel ALG, Kingston RD, Shaw JFL,Teasdale C, Cole DS. Single dose cefotaxime plusmetronidazole versus three dose cefuroxime plus
References
78
metronidazole as prophylaxis against woundinfection in colorectal surgery: multicentreprospective randomised study. BMJ 1990;300:18–22.
112. Kwok SPY, Lau WY, Leung KL, Ku KW, Ho WS, Li AKC. Amoxycillin and clavulanic acid versuscefotaxime and metronidazole as antibioticprophylaxis in elective colorectal resectionalsurgery. Chemotherapy 1993;39:135–9.
113. Altman DG, Deeks JJ. Meta-analysis, Simpson’sparadox, and the number needed to treat. BMCMed Res Methodol 2002;2:3.
114. Altman DG, Dore CJ. Randomisation and baselinecomparisons in clinical trials. Lancet 1990;335:149–53.
115. Schulz KF, Chalmers I, Hayes RJ, Altman DG.Empirical evidence of bias: dimensions ofmethodological quality associated with estimates oftreatment effects in controlled trials. JAMA 1995;273:408–12.
116. Song F, Eastwood A, Gilbody S, Duley L, Sutton A.Publication and related biases. Health Technol Assess2000;4(10).
117. Chan A-W, Hròbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence forselective reporting of outcomes in randomizedtrials: comparison of protocols to publishedarticles. JAMA 2004;291:2457–65.
118. Villar J, Carroli G, Belizan JM. Predictive ability ofmeta-analyses of randomised controlled. Lancet1995;345:772–6.
119. Cappelleri JC, Ioannidis JPA, Schmid CH, Ferranti SD de, Aubert M, Chalmers TC, et al.Large trials vs meta-analysis of smaller trials. Howdo their results compare? JAMA 1996;276:1332–8.
120. LeLorier J, Gregoire G, Benhaddad A, Lapierre J,Derderian F. Discrepancies between meta-analysesand subsequent large randomized, controlledtrials. N Engl J Med 1997;337:536–42.
121. Abel U, Koch A. The role of randomization inclinical studies: myths and beliefs. J Clin Epidemiol1999;52:487–97.
122. Jüni P, Altman DG, Egger M. Assessing the qualityof randomised controlled trials. In Egger M,Davey Smith G, Altman DG, editors. Systematicreviews in health care. Meta-analysis in context.London: BMJ Books; 2001. pp. 87–108.
123. Black N. Why we need observational studies toevaluate the effectiveness of health care. BMJ1996;312:1215–18.
124. McQuay HJ, Moore RA. An evidence-based resource forpain relief. Oxford: Oxford University Press; 1998.
125. Bandolier. The Oxford league table of analgesicefficacy. URL: http:// www.jr2.ox.ac.uk/bandolier/booth/painpag/acutrev/analgesics/lftab.html(accessed September 2004).
126. European Collaboration on CraniofacialAnomalies (Eurocran). URL:http://www.eurocran.org/content.asp?contentID=1345(accessed September 2004).
127. Sackett DL. Rules of evidence and clinicalrecommendations on the use of asymptomaticatherosclerosis. Stroke 1989;20:844–9.
128. Poynard T. Evaluation de la qualite methodolgiquedes essais therapeutiques randomises. Presse Med1988;17:315–88.
129 Marshall JK, Irvine EJ. Rectal aminosalicylatetherapy for distal ulcerative colitis: a meta-analysis.Aliment Pharmacol Ther 1995;9:293–300.
130. Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D et al. A method for assessing the quality of a randomizedcontrol trial. Control Clin Trials 1981;2:31–49.
131. Liberati A, Himel HN, Chalmers TC. A qualityassessment of randomized control trials of primarytreatment of breast cancer. J Clin Oncol 1986;4:942–51.
132. Evans M, Pollock AV. A score system for evaluatingrandom control clinical trials of prophylaxis ofabdominal surgical wound infection. Br J Surg1985;72:265–70.
133. Jadad AR, Moore RA, Carrol D, Jenkinson C,Reynolds DJM, Gavaghan DJ, et al. Assessing thequality of reports of randomised clinical trials: isblinding necessary? Control Clin Trials 1996;17:1–12.
134. Poynard T, Pignon JP. Duodenal ulcer. Analysis of 293randomized clinical trials. London: Montrouge, JohnLibbey Eurotext; 1989.
135. DerSimonian R, Charette LJ, McPeek B, Mosteller F. Reporting on methods in clinicaltrials. N Engl J Med 1982;306:1332–7.
136. Schmieder RE, Schlaich MP, Klingbeil AU, Martus P. Update on reversal of left ventricularhypertrophy in essential hypertension (a meta-analysis of all randomized double-blind studiesuntil December 1996). Nephrol Dial Transplant1998;13:564–9.
137. Cheng L, Gulmezoglu AM, Ezcurra E, van-Look PFA. Interventions for emergencycontraception (Cochrane Review). In The CochraneLibrary (Issue 3). Oxford: Update Software; 2000.
138. Collins SL, Moore RA, McQuay HJ, Wiffen PJ,Edwards JE. Single dose oral ibuprofen anddiclofenac for postoperative pain (CochraneReview). In The Cochrane Library (Issue 3). Oxford:Update Software; 2000.
139. Delaney BC, Innes MA, Deeks JJ, Wilson S, OakesR, Moayyedi P, et al. Initial management strategiesfor dyspepsia (Cochrane Review). In The CochraneLibrary (Issue 3). Oxford: Update Software; 2001
Health Technology Assessment 2005; Vol. 9: No. 26
79
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
140. Di Mario F, Battaglia G, Leandro G, Grasso G,Vianello F, Vigneri S. Short-term treatment ofgastric ulcer. A meta-analytical evaluation of blindtrials. Dig Dis Sci 1996;41:1108–31.
141. Handoll HHG, Farrar MJ, McBirnie J,Tytherleigh-Strong G, Awal KA, Milne AA, et al.Heparin, low molecular weight heparin andphysical methods for preventing deep veinthrombosis and pulmonary embolism followingsurgery for hip fractures (Cochrane Review). InThe Cochrane Library (Issue 4). Oxford: UpdateSoftware; 2002.
142. Horn J, Limburg M. Calcium antagonists for acuteischemic stroke (Cochrane Review). In TheCochrane Library (Issue 3). Oxford: UpdateSoftware; 2001.
143. McIntosh HM, Olliaro P. Artemisinin derivativesfor treating uncomplicated malaria (CochraneReview). In The Cochrane Library (Issue 3). Oxford:Update Software; 2001.
144. Silagy C. Physician advice for smoking cessation(Cochrane Review). In The Cochrane Library (Issue3). Oxford: Update Software; 2001.
145. Silagy C, Mant D, Fowler G, Lancaster T. Nicotinereplacement therapy for smoking cessation(Cochrane Review). In The Cochrane Library (Issue3). Oxford: Update Software; 2001.
146. Trindade E, Menon D. Selective serotoninreuptake inhibitors (SSRIs) for major depression.
Part I. Evaluation of the clinical literature. ReportNo. 3E. Ottawa, Ontario: Canadian CoordinatingOffice for Health Technology Assessment; 1997August 1997.
147. Zhang WY, Li Wan Po A. Efficacy of minoranalgesics in primary dysmenorrhoea: a systematicreview. Br J Obstet Gynaecol 1998;105:780–9.
148. Ausejo M, Saenz A, Pham B, Kellner JD, Johnson DW, Moher D, et al. Glucocorticoids forcroup (Cochrane Review). In The Cochrane Library(Issue 3). Oxford: Update Software; 2001.
149. Sauriol L, Laporta M, Edwardes MD, Deslandes M,Ricard N, Suissa S. Meta-analysis comparing newerantipsychotic drugs for the treatment ofschizophrenia: evaluating the indirect approach.Clin Ther 2001;23:942–56.
150. Hart RG, Benavente O, McBride R, Pearce LA.Antithrombotic therapy to prevent stroke inpatients with atrial fibrillation: a meta-analysis.Ann Intern Med 1999;131:492–501.
151. Singer DE. Antithrombotic therapy to preventstroke in patients with atrial fibrillation. Ann InternMed 2000;132:841–2.
152. EAFT (European Atrial Fibrillation Trial) StudyGroup. Secondary prevention in non-rheumaticatrial fibrillation after transient ischaemic attack orminor stroke. Lancet 1993;342:1255–62.
References
80
Health Technology Assessment 2005; Vol. 9: No. 26
81
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 1
Reviews of effectiveness prescreening form
Appendix 1
82 Aut
hor/
year
R
elev
ant
Use
ful
Type
of s
tudy
des
ign
M-A
ENR
ISIC
Com
men
tsM
etho
dolo
gy
DA
RE
Acc
essi
on
proj
ect
(�/?
/✕)
cita
tion
snu
mbe
r(E
/I/N
)
EI
BR
CTs
Com
pN
on-
VA
CF
ICD
C/I
CSu
ff.co
mp
done
done
data
Health Technology Assessment 2005; Vol. 9: No. 26
83
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Key
:1
.A
uth
or/
year
an
d D
AR
E a
cces
sio
n n
um
ber
if
rele
van
t.2
.R
elev
ant
pro
ject
E
: E
NR
IS;
I: i
nd
irec
t co
mp
aris
on
N:
nei
ther
No
te:
can
be
bo
th E
an
d I
.3
.U
sefu
l�
: if
ad
equ
ate
dat
a ar
e p
rese
nte
d f
or
use
in
eit
her
rev
iew
?:
if
un
sure
or
insu
ffic
ien
t d
ata
are
pre
sen
ted
, bu
t th
ere
is a
po
ssib
ilit
y o
f o
bta
inin
g f
urt
her
dat
a, o
r if
met
a-an
alys
is o
fn
on
-RC
Ts
bu
t n
o v
alid
ity
asse
ssm
ent
or
con
tro
l fo
r co
nfo
un
din
g v
aria
ble
s h
as b
een
un
der
take
n (
spec
ific
ally
fo
rE
NR
IS)
X:
if t
her
e ar
e in
suff
icie
nt
dat
a fo
r u
se i
n e
ith
er r
evie
w (
mar
k r
elev
ant
colu
mn
)B
: m
ark i
f th
e p
aper
is
use
ful
bu
t o
nly
fo
r th
e bac
kg
rou
nd
/dis
cuss
ion
.4
. In
clu
ded
stu
dy
des
ign
En
ter
nu
mber
of
RC
Ts,
co
mp
arat
ive
and
no
n-c
om
par
ativ
e st
ud
ies.
5.
M-A
�
: if
met
a-an
alys
is h
as b
een
un
der
take
nX
: if
no
met
a-an
alys
is h
as b
een
un
der
take
n (
nar
rati
ve r
evie
w).
6.
EN
RIS
VA
: n
eed
to
kn
ow
wh
eth
er s
om
e fo
rm o
f va
lid
ity
asse
ssm
ent
has
bee
n u
nd
erta
ken
CF:
nee
d t
o k
no
w w
het
her
th
ey h
ave
con
tro
lled
fo
r co
nfo
un
din
g v
aria
ble
s7
.IC
IC d
on
e: h
as a
n i
nd
irec
t co
mp
aris
on
bee
n c
on
du
cted
in
th
e re
view
?IC
/DC
do
ne:
do
es t
he
revi
ew p
rese
nt
bo
th d
irec
t an
d i
nd
irec
t co
mp
aris
on
s o
f d
ata?
Su
ff.
dat
a: a
re t
her
e su
ffic
ien
t p
rim
ary
dat
a (i
.e.
nu
mber
of
even
ts a
nd
sam
ple
siz
e fo
r ea
ch a
rm o
f th
e in
clu
ded
tria
ls)
or
sum
mar
y st
atis
tics
an
d S
E/C
Is,
in o
rder
to
fo
rce
an i
nd
irec
t co
mp
aris
on
, w
her
e a
dir
ect
com
par
iso
n h
as b
een
do
ne?
If
so,
stat
e co
mp
aris
on
s fo
r w
hic
h a
n I
C i
s p
oss
ible
in
th
e co
mm
ents
fie
ld.
8.
Co
mm
ents
9.
Met
ho
do
log
y ci
tati
on
sL
ist
cita
tio
ns
of
po
ten
tial
ly r
elev
ant
met
ho
do
log
y p
aper
s ci
ted
in
th
e re
view
(re
fere
nce
lis
ts o
f al
l re
view
s sh
ou
ld b
eex
amin
ed).
NB
. A
ll c
olu
mn
s sh
ou
ld b
e co
mp
lete
d.
Author (year)
Title
Journal
Objectives
COMPARISONS BEING MADE FOR RCTs
INDIRECT Done Not Done*�
Intervention A / ctrl Intervention B / ctrl No. of trialsA / B
DIRECT (only those that are relevant to ICs) Done Not Done�
Intervention A Intervention B No. of trials
METHODS USED FOR CONDUCTING IC
Health Technology Assessment 2005; Vol. 9: No. 26
85
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 2
Data extraction form
RESULTS OF IC
Intervention A / ctrl Intervention B /ctrl Summary statistic
RESULTS OF DC
Intervention A Intervention B Summary statistic
INTERPRETATION OF RESULTSIC interpreted appropriately? Yes � No � Unsure �How were ICs interpreted? Details:
Other COMMENTS
Appendix 2
86 *But sufficient data presented in paper to force IC.
Health Technology Assessment 2005; Vol. 9: No. 26
87
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 3
Table of systematic reviews making indirect comparisons
Appendix 3
88 TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
ATC
17
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Onl
y un
conf
ound
ed t
rials
dem
onst
ratin
g co
ncea
led
trea
tmen
tal
loca
tion
wer
e in
clud
ed
Ass
essm
ent
of h
eter
ogen
eity
:A
naly
ses
wer
e pe
rfor
med
sep
arat
ely
acco
rdin
g to
pat
ient
gro
ups
and
outc
ome
mea
sure
s. A
naly
ses
wer
eal
so p
erfo
rmed
sep
arat
ely
acco
rdin
gto
age
, gen
der,
dias
tolic
blo
odpr
essu
re a
nd d
iabe
tes
Met
hod
used
:A
djus
ted
ICC
omm
on c
ontr
ol.
Resu
lts p
rese
nted
ina
grap
h
Com
pari
sons
mad
e:a.
Asp
irin+
Dip
/ctr
l (n
= 3
4)b.
Asp
irin/
ctrl
(n=
46)
c.H
igh-
dose
asp
irin/
ctrl
(n=
30)
d.M
ediu
m-d
ose
aspi
rin/c
trl
(n=
19)
Res
ults
:%
odd
s re
duct
ion
(SD
)a.
28%
(5)
b.25
% (2
) c.
21%
(4)
d. 2
8% (3
) (1
60–3
25 m
g pe
r da
y)26
% (1
1) (<
160
mg
per
day)
Com
pari
sons
mad
e:e.
Asp
irin+
Dip
/asp
irin
(n=
14)
f.H
igh-
dose
aspi
rin/m
ediu
m-d
ose
aspi
rin (n
=3)
Res
ults
:%
odd
s re
duct
ion
(SD
)e.
–1%
(9)
f.5%
(11)
Indi
rect
(adj
uste
d) e
vide
nce
used
to
enha
nce
dire
ct e
vide
nce
“Suc
h in
dire
ct c
ompa
rison
s ne
ed t
obe
inte
rpre
ted
mor
e ca
utio
usly,
for
alth
ough
man
y of
the
bia
ses
inhe
rent
in n
on-r
ando
m m
etho
ds (s
uch
asth
ose
invo
lvin
g hi
stor
ic c
ontr
ols)
are
avoi
ded,
som
e po
tent
ial f
or b
ias
rem
ains
”
No
stat
istic
ally
sig
nific
ant
diffe
renc
e in
the
resu
lts b
etw
een
the
indi
rect
and
dire
ct c
ompa
rison
s w
as n
oted
App
ropr
iate
inte
rpre
tati
on?
Yes
Add
ition
alco
mpa
rison
sw
ere
mad
e.T
hose
with
grea
test
sam
ple
size
pres
ente
dhe
re
ATC
16
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Onl
y un
conf
ound
ed t
rials
dem
onst
ratin
g co
ncea
led
trea
tmen
tal
loca
tion
wer
e in
clud
ed
Ass
essm
ent
of h
eter
ogen
eity
:Se
para
te a
naly
ses
wer
e un
dert
aken
acco
rdin
g to
pat
ient
gro
ups
and
whe
ther
the
ant
ipla
tele
t ag
ent
was
star
ted
befo
re, w
ithin
24
hour
s, o
r>
24 h
ours
afte
r th
e pr
oced
ure
Met
hod
used
:A
djus
ted
ICC
omm
on c
ontr
ol,
grap
hic
plot
Com
pari
sons
mad
e:a.
Asp
irin+
Dip
/ctr
l (n
=20
)b.
Asp
irin/
ctrl
(n=
13)
Res
ults
:%
odd
s re
duct
ion
(SD
)a.
40%
(6)
b.48
% (7
)
Com
pari
sons
mad
e:c.
Asp
irin+
Dip
/asp
irin
(n=
9)R
esul
ts:
% o
dds
redu
ctio
n (S
D)
c.–1
% (1
1)
Indi
rect
(adj
uste
d) e
vide
nce
used
to
enha
nce
dire
ct e
vide
nce
“Thi
s ov
ervi
ew p
rovi
des
som
e di
rect
and
indi
rect
ran
dom
ised
com
paris
ons
of t
he e
ffect
s of
diff
eren
t dr
ugre
gim
ens
on t
he p
reve
ntio
n of
occl
usio
n bu
t fin
ds n
o ev
iden
ce o
f any
diffe
renc
es in
effi
cacy
. The
num
bers
of
patie
nts
stud
ied
and
the
num
bers
of
even
ts t
hat
occu
rred
wer
e, h
owev
er,
not
larg
e en
ough
to
excl
ude
som
esm
all b
ut r
eal d
iffer
ence
s in
effi
cacy
betw
een
diffe
rent
dru
g re
gim
ens”
App
ropr
iate
inte
rpre
tati
on?
Yes
Add
ition
alco
mpa
rison
sw
ere
mad
e.T
hose
with
grea
test
sam
ple
size
pres
ente
dhe
re
Health Technology Assessment 2005; Vol. 9: No. 26
89
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
ATC
18
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Onl
y un
conf
ound
ed t
rials
dem
onst
ratin
g co
ncea
led
trea
tmen
tal
loca
tion
wer
e in
clud
ed
Ass
essm
ent
of h
eter
ogen
eity
:D
ata
from
pat
ient
s un
derg
oing
gene
ral,
trau
mat
ic o
rtho
paed
ic a
ndel
ectiv
e or
thop
aedi
c su
rger
y, a
ndhi
gh-r
isk m
edic
al p
atie
nts
wer
ean
alys
ed s
epar
atel
y. S
epar
ate
anal
yses
wer
e al
so u
nder
take
n fo
r tr
ials
inw
hich
pat
ient
s di
d or
did
not
rec
eive
hepa
rin
Met
hod
used
:A
djus
ted
IC u
sing
ano
-tre
atm
ent
com
paris
on g
roup
.Ill
ustr
ated
with
plo
t
Com
pari
sons
mad
e:O
n D
VT:
a.A
spiri
n+D
ip/c
trl (
n =
18)
b.A
spiri
n/ct
rl (n
= 1
6)
On
PE:
c.A
spiri
n+D
ip/c
trl (
n =
18)
d.A
spiri
n/ct
rl (n
= 2
1)
Res
ults
:O
dds
redu
ctio
na.
56%
b.
23%
c.43
%d.
67%
Com
pari
sons
mad
e:O
n D
VT:
e.A
spiri
n+D
ip/a
spiri
n(n
= 9
)
On
PE:
f.A
spiri
n+D
ip/a
spiri
n(n
= 1
1)
Res
ults
:O
dds
redu
ctio
ne.
52%
The
disc
ussio
n of
asp
irin
plus
Dip
vers
us a
spiri
n al
one
was
form
ed o
nev
iden
ce fr
om d
irect
com
paris
ons.
The
con
clus
ions
wer
e ca
utio
us
Resu
lts fr
om t
he IC
sup
port
tho
sefr
om d
irect
com
paris
ons
App
ropr
iate
inte
rpre
tati
on?
Yes
Low
enth
al a
nd B
uyse
19
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
of h
eter
ogen
eity
:�
2te
st fo
r he
tero
gene
ity w
asca
lcul
ated
for
each
end
-poi
nt a
nd fo
rea
ch g
roup
of t
rials
(asp
irin
vspl
aceb
o, a
spiri
n pl
us d
ipyr
imad
ole
vspl
aceb
o). N
o sig
nific
ant
hete
roge
neity
show
n
Met
hod
used
:A
djus
ted
IC, u
sing
plac
ebo
asco
mpa
rato
r.D
ispla
yed
in g
raph
Com
pari
sons
mad
e:a.
Asp
irin+
Dip
/pla
cebo
(n=
2)b.
Asp
irin/
plac
ebo
(n=
9)
Res
ults
:Ri
sk r
educ
tion
(SD
)A
ll de
aths
: a.
30%
(11)
b.10
% (8
)(n
s)Va
scul
ar d
eath
s:a.
24%
(13)
b.–4
% (1
0)(n
s)A
ll st
roke
sa.
42%
(9)
b.17
% (7
)(p
=0.
007)
Fata
l str
okes
:a.
43%
(18)
b.–1
0% (2
1)(p
= 0
.03)
IVE:
a.40
% (8
)b.
18%
(6)
(p=
0.00
7)
Com
pari
sons
mad
e:c.
Asp
irin+
Dip
/asp
irin
(n=
2)
Res
ults
:Ri
sk r
educ
tion
(95%
CI)
All
deat
hs: –
19%
(–77
to 2
0%)
Vasc
ular
dea
ths:
–11
%(–
78 t
o 31
%)
All
stro
kes:
13%
(–22
to
38%
)Fa
tal s
trok
es: 2
% (–
129
to 8
%)
IVE:
4%
(–28
to
28%
)
Indi
rect
(adj
uste
d) e
vide
nce
used
to
enha
nce
dire
ct e
vide
nce.
“It
mus
t be
str
esse
d th
at t
hese
res
ults
are
base
d on
an
indi
rect
com
paris
onbe
twee
n tw
o gr
oups
of t
rials,
and
may
the
refo
re r
efle
ct d
iffer
ence
s in
sele
ctio
n cr
iteria
or
othe
rco
nfou
ndin
g fa
ctor
s ra
ther
tha
n a
trul
y gr
eate
r tr
eatm
ent
effe
ct o
fco
mbi
natio
n th
erap
y”
“The
se r
esul
ts s
ugge
stth
at t
heco
mbi
natio
n th
erap
y of
asp
irin
with
dipy
ridam
ole
may
be
supe
rior
toas
pirin
alo
ne”
App
ropr
iate
inte
rpre
tati
on?
Yes
Resu
ltssu
bseq
uent
lyco
nfirm
ed b
yla
rger
RC
T
Appendix 3
90 TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Li W
an P
o an
d Z
hang
20
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
of h
eter
ogen
eity
:St
atist
ical
het
erog
enei
ty a
cros
sin
divi
dual
stu
dies
tes
ted
usin
g th
e Q
stat
istic
. If t
rials
wer
e st
atist
ical
lyhe
tero
gene
ous
a ra
ndom
effe
cts
mod
el w
as u
sed
Met
hod
used
:A
djus
ted
IC u
sing
plac
ebo
asco
mpa
rato
r
Com
pari
sons
mad
e:a.
Para
ceta
mol
+de
xtro
poro
pxyp
hene
/pl
aceb
o (n
=9)
b.Pa
race
tam
ol/p
lace
bo(n
=17
)
Res
ults
:M
ean
diffe
renc
e in
per
cent
age
sum
of d
iffer
ence
s in
pai
nin
tens
ity (9
5% C
I)
Fixe
d ef
fect
: a.
12.7
% (9
.2 t
o 16
.2%
)b.
9.4%
(6.9
to
11.9
%)
Rand
om e
ffect
: a.
13.5
% (8
.8 t
o 18
.3%
)b.
9.4%
(6.6
to
12.2
%)
Com
pari
sons
mad
e:c.
Para
ceta
mol
/pla
cebo
(n=
6, o
nly
thre
eus
ed in
met
a-an
alys
is)
Res
ults
:M
ean
diffe
renc
e in
perc
enta
ge s
um o
fdi
ffere
nces
in p
ain
inte
nsity
(95%
CI)
c.Fi
xed
effe
ct: 7
.3%
(–0.
2 to
14.
9%)
Rand
om e
ffect
s:7.
4% (–
0.4
to15
.1%
)
Indi
rect
(adj
uste
d) e
vide
nce
used
to
enha
nce
dire
ct e
vide
nce
Aut
hors
con
clud
ed t
hat
“On
the
basis
of d
ata
on a
nalg
esic
effi
cacy
and
acu
tesa
fety
in b
oth
head
to
head
and
indi
rect
com
paris
ons,
the
re is
litt
leob
ject
ive
evid
ence
to
supp
ort
pres
crib
ing
a co
mbi
natio
n of
para
ceta
mol
and
dext
ropr
opox
yphe
ne in
pre
fere
nce
topa
race
tam
ol a
lone
in m
oder
ate
pain
such
as
that
afte
r su
rger
y”
App
ropr
iate
inte
rpre
tati
on?
Yes
Oth
er o
utco
me
mea
sure
s al
sopr
esen
ted
in t
here
view
Mat
char
et
al.22
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Onl
y RC
Ts in
clud
ed a
ccor
ding
to
prev
ious
ly p
ublis
hed
crite
ria12
7
Ass
essm
ent
of h
eter
ogen
eity
:N
o fo
rmal
tes
t of
het
erog
enei
tyre
port
ed, a
lthou
gh it
is s
tate
d th
at a
rand
om e
ffect
s m
etho
d w
as u
sed
asap
prop
riate
Met
hod
used
:A
djus
ted
IC u
sing
plac
ebo/
ctrl
grou
p
Com
pari
sons
mad
e:N
on-v
alvu
lar
atria
l fib
rilla
tion;
a.W
arfa
rin/c
trl (
n =
5)
b.A
spiri
n/pl
aceb
o (n
= 2
)
Res
ults
:RR
for
stro
ke (9
5% C
I) a.
0.33
(0.2
2 to
0.5
0)b.
0.67
(0.4
5 to
0.9
9)
Com
pari
sons
mad
e:N
on-v
alvu
lar
atria
lfib
rilla
tion;
c.W
arfa
rin/a
spiri
n(n
=1)
Res
ults
:RR
for
stro
ke (9
5% C
I) c.
0.34
(0.1
1 to
0.8
7)
The
res
ults
of t
he IC
wer
e us
ed t
oen
hanc
e th
e fin
ding
s of
the
dire
ctco
mpa
rison
App
ropr
iate
inte
rpre
tati
on?
Yes
Det
ails
on o
ther
com
paris
ons
and
outc
ome
mea
sure
s ar
epr
esen
ted
in t
here
view
Health Technology Assessment 2005; Vol. 9: No. 26
91
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Moo
re e
t al
.24
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
with
blin
ded
desig
n in
clud
ed
Ass
essm
ent
of h
eter
ogen
eity
:Te
sts
used
not
rep
orte
d. A
ll bu
t tw
oco
mbi
natio
ns o
f tria
ls w
ere
hom
ogen
eous
Met
hod
used
:A
djus
ted
IC u
sing
plac
ebo
com
para
tor
Com
pari
sons
mad
e:a.
Para
ceta
mol
+co
dein
e/pl
aceb
o (n
=18
)b.
Para
ceta
mol
/pla
cebo
(n=
13)
Res
ults
:>
50%
max
TO
TPA
R. R
iskra
tio (9
5% C
I)a.
2.6
(2.1
to
3.2)
b.1.
7 (1
.3 t
o 2.
2)
Sum
mar
y st
atist
ic (R
R): 1
.53
Com
pari
sons
mad
e:c.
Para
ceta
mol
+co
dein
e/ p
arac
etam
ol(n
= 1
1)
Res
ults
:>
50%
max
TO
TPA
R.Ri
sk r
atio
(95%
CI)
c.1.
19 (0
.98
to 1
.44)
Resu
lts o
f adj
uste
d IC
and
dire
ctco
mpa
rison
are
disc
usse
d se
para
tely,
with
no
atte
mpt
to
com
bine
the
resu
lts
The
IC g
ave
a gr
eate
r es
timat
e th
anth
e di
rect
com
paris
on in
ter
ms
ofef
ficac
y of
par
acet
amol
plu
s co
dein
eco
mpa
red
with
par
acet
amol
alo
ne
App
ropr
iate
inte
rpre
tati
on?
Yes
Non
-sig
nific
ant
resu
lts o
f dire
ctco
mpa
rison
cou
ldbe
com
e sig
nific
ant
if th
e re
sults
of
the
adju
sted
ICar
e in
corp
orat
ed
Picc
inel
li et
al.28
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
doub
le-b
lind
RCTs
of
4 w
eeks
’ dur
atio
n in
clud
ed
Ass
essm
ent
of h
eter
ogen
eity
:�
2te
st o
f het
erog
enei
ty w
asun
dert
aken
. Fix
ed e
ffect
mod
el u
sed
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Clo
mip
ram
ine/
plac
ebo
(n=
9)SS
RI/p
lace
bo (n
= 8
)
Res
ults
:O
bses
sive/
com
pulsi
vesy
mpt
oms
(effe
ct s
ize
=g)
(95%
CI)
a.g
= 1
.31
(1.1
5 to
1.4
7)b.
g=
0.4
7 (0
.33
to 0
.61)
Com
pari
sons
mad
e:c.
Clo
mip
ram
ine/
SSRI
s(n
=3)
Res
ults
:c.
–0.0
4 (–
0.43
to
0.35
)
The
aut
hors
con
clud
ed t
hat
“alth
ough
the
incr
ease
in im
prov
emen
t ra
teov
er p
lace
bo w
as g
reat
er fo
rcl
omip
ram
ine
than
for
SSRI
s, d
irect
com
paris
on b
etw
een
thes
e dr
ugs
show
ed t
hat
they
had
sim
ilar
ther
apeu
tic e
ffica
cy o
nob
sess
ive–
com
pulsi
ve s
ympt
oms”
App
ropr
iate
inte
rpre
tati
on?
Yes
Con
sider
able
diffe
renc
eob
serv
ed in
indi
rect
com
paris
on b
utno
t in
dire
ctco
mpa
rison
Appendix 3
92 TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Poyn
ard
et a
l.26
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Use
of p
revi
ously
val
idat
edqu
estio
nnai
re12
8
Ass
essm
ent
of h
eter
ogen
eity
:�
2te
st o
f het
erog
enei
ty w
asun
dert
aken
. Bot
h fix
ed a
nd r
ando
mef
fect
mod
els
wer
e us
ed
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Com
plet
e A
LTa.
3 M
U 1
2 m
onth
s/ct
rl(n
=7)
b.3
MU
6 m
onth
s/ct
rl(n
=7)
Sust
aine
d A
LTc.
3 M
U 1
2 m
onth
s/ct
rl(n
=5)
d.3
MU
6 m
onth
s/ct
rl(n
=6)
Res
ults
:Re
spon
se r
ate
(95%
CI)
a.48
%b.
45%
(35
to 5
5%)
Diff
eren
ce in
res
pons
e ra
te:
3% c.35
% (2
8 to
43%
)d.
21%
(13
to 2
8%)
Diff
eren
ce in
res
pons
e ra
te:
14%
Com
pari
sons
mad
e:C
ompl
ete
ALT
e.3
MU
12
mon
ths/
3MU
6m
onth
s (n
=4)
Sust
aine
d A
LTf.
3 M
U 1
2 m
onth
s/3
MU
6 m
onth
s(n
=4)
Res
ults
:e.
11%
f.16
% (9
to
23%
)
IC u
sed
to e
nhan
ce t
he r
esul
ts o
f the
dire
ct c
ompa
rison
s
App
ropr
iate
inte
rpre
tati
on?
Yes
Dos
e–ef
fect
dat
aav
aila
ble
in p
aper
Health Technology Assessment 2005; Vol. 9: No. 26
93
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
19
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
ATC
, Ant
ipla
tele
t Tr
ialis
ts’ C
olla
bora
tion;
ctr
l, co
ntro
l; D
ip, d
ipyr
idam
ole;
DVT
, dee
p ve
in t
hrom
bosis
; IVE
, im
port
ant
vasc
ular
eve
nts;
ns,
not
sig
nific
ant;
PE, p
ulm
onar
y em
bolis
m;
Rem
RR, r
emed
icat
ion
rate
rat
io; R
esRR
, res
pons
e ra
te r
atio
; SPI
D, s
um o
f pai
n in
tens
ity d
iffer
ence
.
Zha
ng a
nd L
i Wan
Po27
Val
idit
y as
sess
men
t un
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
doub
le-b
lind
RCTs
incl
uded
Ass
essm
ent
of h
eter
ogen
eity
:�
2te
st o
f het
erog
enei
ty w
asun
dert
aken
. A r
ando
m e
ffect
s m
odel
was
use
d w
hen
hete
roge
neity
was
pres
ent
Met
hod
used
:A
djus
ted
IC u
sing
plac
ebo
com
para
tor
Com
pari
sons
mad
e:a.
Para
ceta
mol
+co
dein
e/pl
aceb
o (n
=37
)b.
Para
ceta
mol
/pla
cebo
(n=
13)
c.Pa
race
tam
ol+
caffe
ine/
plac
ebo
(n=
228)
d.Pa
race
tam
ol/p
lace
bo(n
=10
)
Res
ults
:D
iffer
ence
in T
OT
PAR%
a.d 2
= 2
3.18
(SE
2.67
)b.
d 1=
15.
06 (S
E 0.
90)
d 2–
d 1=
8.1
2 (S
E 2.
82)
c.d 2
= 1
7.36
(SE
1.89
)d.
d 1=
13.
91 (S
E 1.
52)
d 2–
d 1=
3.4
5 (S
E 2.
43)
Com
pari
sons
mad
e:e.
Para
ceta
mol
+co
dein
e/ p
arac
etam
ol(n
=13
)f.
Para
ceta
mol
+ca
ffein
e/ p
arac
etam
ol(n
=10
)
Res
ults
:D
iffer
ence
in T
OT
PAR%
e.d
= 7
.39
(SE
2.43
)f.
d=
3.9
7 (S
E 1.
73)
IC u
sed
to s
uppo
rt r
esul
ts fr
om d
irect
com
paris
ons
“The
ana
lges
ic e
ffica
cy o
f par
acet
amol
600
mg
was
enh
ance
d w
ith t
head
ditio
n of
cod
eine
60
mg
(usin
gT
OT
PAR%
as
outc
ome)
in b
oth
indi
rect
and
hea
d-to
-hea
dco
mpa
rison
s”
App
ropr
iate
inte
rpre
tati
on?
Yes
Resu
lts fo
rSP
ID%
, Res
RRan
d Re
mRR
also
avai
labl
e in
the
revi
ew.
The
obs
erve
ddi
ffere
nce
inT
OT
PAR
was
not
conf
irmed
usin
gth
e m
ore
clin
ical
lym
eani
ngfu
lRe
mRR
Appendix 3
94 TA
BLE
20
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Mar
shal
l and
Irvi
ne21
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
30-p
oint
sco
ring
syst
emus
ed.12
9Va
lidity
sco
re fo
r ea
chin
clud
ed t
rial p
rese
nted
. RC
Tson
ly in
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
Hom
ogen
eity
with
in g
roup
s of
tria
ls co
nfirm
ed u
sing
Bres
low
–Day
tes
t
Met
hod
used
:N
aive
pre
sent
atio
n of
pool
ed r
espo
nse
rate
sac
ross
all
tria
ls fo
r ea
chtr
eatm
ent
Com
pari
sons
mad
e:Re
ctal
cor
ticos
tero
ids/
plac
ebo
or o
ther
tre
atm
ent
(n=
16)
5-A
SA/o
ther
tre
atm
ent
(n=
9)
Res
ults
:Po
oled
impr
ovem
ent
rate
s by
sym
ptom
atic
, end
osco
pic
and
hist
olog
ical
crit
eria
:Re
ctal
cor
ticos
tero
ids
77%
,66
% a
nd 5
2%, r
espe
ctiv
ely;
5-A
SA 8
2%, 7
3% a
nd 6
6%,
resp
ectiv
ely
Com
pari
sons
mad
e:Re
ctal
cor
ticos
tero
ids/
5-A
SA (n
=7)
Res
ults
:Po
oled
OR
(95%
CI)
Sym
ptom
atic
impr
ovem
ent:
1.36
(0.8
8to
2.0
9)
Endo
scop
icim
prov
emen
t: 1.
06 (0
.61
to 1
.85)
Hist
olog
ical
impr
ovem
ent:
2.27
(1.2
2 to
4.2
7)
Emph
asis
give
n to
res
ults
from
dire
ct c
ompa
rison
App
ropr
iate
inte
rpre
tati
on?
Yes
Miln
e et
al.23
a
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. C
ompa
rativ
ecl
inic
al s
tudi
es in
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
�2
test
of h
eter
ogen
eity
was
unde
rtak
en. N
o sig
nific
ant
hete
roge
neity
was
sho
wn
Met
hod
used
:N
aive
ICSu
mm
ary
stat
istic
of
repo
rted
adv
erse
eve
nts
and
with
draw
als
base
d on
data
from
arm
s of
all
tria
ls
Com
pari
sons
mad
e:a.
Roxi
thro
myc
in/o
ther
mac
rolid
e or
age
ntco
mm
only
use
d as
firs
t lin
eth
erap
y (n
=13
)b.
Eryt
hrom
ycin
/oth
erm
acro
lide
or a
gent
com
mon
ly u
sed
as fi
rst
line
ther
apy
(n=
15)
Res
ults
:A
dver
se e
vent
s, r
ate
% (9
5%C
I):a.
10%
(8 t
o 12
%)
b.24
.8%
(22
to 2
7%)
Diff
eren
ce (S
E) 1
4% (1
.5%
)
a.2.
0% (1
to
3%)
b.7.
1% (6
to
9%)
Diff
eren
ce (S
E) 5
% (0
.045
%)
Com
pari
sons
mad
e:c.
Ro
xith
rom
ycin
/er
ythr
omyc
in (n
=3)
Res
ults
:N
o su
mm
ary
stat
istic
for
DC
alo
ne p
rese
nted
.A
dver
se e
vent
s:Ro
xith
rom
ycin
12.
8%(1
0.5
to 1
7.5%
)Er
ythr
omyc
in 2
7.15
% (8
to 5
1.3%
)
With
draw
als:
Ro
xith
rom
ycin
: 1.8
%Er
ythr
omyc
in: 2
.65%
Aut
hors
disc
usse
d th
epo
ssib
ility
of s
yste
mat
ic b
ias
due
to d
issim
ilar
patie
ntgr
oups
, but
con
clud
ed t
hat
this
is un
likel
y in
thi
s ca
se
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Pote
ntia
l con
foun
ding
fact
ors
disc
usse
d an
dco
nclu
ded
no s
igni
fican
tdi
ffere
nce
betw
een
grou
ps in
ter
ms
ofcl
inic
al e
ffica
cy, a
ge,
gend
er, s
ettin
gs,
dura
tion
of t
reat
men
t,in
dica
tions
or
year
of
publ
icat
ion
Health Technology Assessment 2005; Vol. 9: No. 26
95
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
20
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
and
dire
ct c
ompa
rison
s (c
ont’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
Pope
et
al.25
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Stud
ies
wer
e as
sess
ed fo
rbl
indi
ng, r
ando
misa
tion
and
cont
rol g
roup
. Ind
ivid
ual
qual
ity a
sses
smen
t fo
r ea
chst
udy
not
repo
rted
Ass
essm
ent
ofhe
tero
gene
ity:
Het
erog
enei
ty w
as e
xam
ined
and
a ra
ndom
effe
cts
mod
elus
ed
Met
hod
used
:N
aive
IC. A
djus
ted
for
diet
ary
salt
inta
ke
Com
pari
sons
mad
e:a.
Nap
roxe
n (n
=4)
b.In
dom
etha
cin
(n=
57)
c.Pi
roxi
cam
(n=
4)d.
Sulin
dac
(n=
23)
e.A
spiri
n (n
=4)
f.Ib
upro
fen
(n=
6)g.
Plac
ebo
(n=
10)
(n =
num
ber
of t
reat
men
tgr
oups
for
hype
rten
sive
patie
nts)
Res
ults
:M
ean
MA
P ±
SEM
(adj
uste
dfo
r sa
lt in
take
):b.
3.59
±1.
12
d.–0
.16
±1.
45M
ean
diffe
renc
e in
MA
P 3.
75 ±
1.83
Mea
n M
AP
±SE
M (a
djus
ted
for
salt
inta
ke):
b.3.
59 ±
1.12
g.
–2.5
9 ±
1.78
Mea
n di
ffere
nce
in M
AP
6.18
±2.
10
Com
pari
sons
mad
e:h.
Indo
met
haci
n/su
linda
c(n
=16
)i.
Indo
met
haci
n/pl
aceb
o(n
=11
)
Res
ults
:M
ean±
SEM
diff
eren
ce in
MA
P:a.
4.1
5±1.
00b.
3.9
3±1.
42
The
res
ults
of t
he n
aive
ICw
ere
com
pare
d w
ith t
hose
of
dire
ct c
ompa
rison
s w
hen
avai
labl
e
App
ropr
iate
inte
rpre
tati
on?
Yes
Resu
lts fo
r ot
her
NSA
IDs
are
avai
labl
e in
the
revi
ew
aN
ot id
entif
ied
thro
ugh
DA
RE.
Appendix 3
96 TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Aro
37a
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
No
form
al t
est
ofhe
tero
gene
ity r
epor
ted
Met
hod
used
:A
djus
ted
ICA
ge-s
tand
ardi
sed
Com
pari
sons
mad
e:a.
PEP+
EE/o
rchi
dect
omy
(n=
1)b.
PEP/
orch
idec
tom
y (n
=1)
Res
ults
:A
ge-s
tand
ardi
sed
deat
h ra
tera
tio (9
5% C
I)A
ll-ca
use
mor
talit
y:a.
2.31
(1.9
2 to
2.7
9)b.
1.50
(1.0
6 to
2.1
1)
CH
D d
eath
s:a.
1.51
(1.1
0 to
2.0
8)b.
0.17
(0.0
5 to
0.5
7)
Com
pari
sons
mad
e:N
ot d
one
No
tria
l inc
lude
d in
the
revi
ew d
irect
lyco
mpa
red
PEP
+EE
with
PEP
alo
ne
Res
ults
:N
ot a
pplic
able
Aut
hors
con
clud
ed t
hat
“int
ram
uscu
lar
PEP
mon
othe
rapy
is a
ssoc
iate
dw
ith lo
w c
ardi
ovas
cula
rm
orta
lity
and
with
an
all-c
ause
and
pros
tatic
can
cer
mor
talit
yeq
ual t
o or
chid
ecto
my”
App
ropr
iate
inte
rpre
tati
on?
No
A s
ubse
quen
tpu
blic
atio
n48hi
ghlig
hts
the
maj
or d
iffer
ence
s in
incl
usio
n cr
iteria
betw
een
the
two
stud
ies
incl
uded
in t
here
view
. The
obs
erve
dlo
w m
orta
lity
in P
EPal
one
may
be
due
tofla
ws
in t
hem
etho
dolo
gy
Boer
sma
et a
l.9
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
udin
g
100
patie
nts
wer
ein
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
Het
erog
enei
ty w
as e
xam
ined
usin
g th
e Br
eslo
w–D
ay t
est.
Sens
itivi
ty a
naly
sis w
asun
dert
aken
whe
nhe
tero
gene
ity e
xist
ed
Met
hod
used
:A
djus
ted
IC, p
rese
nted
info
rest
plo
tLi
near
and
non
-line
arre
gres
sion
Com
pari
sons
mad
e:T
ime
to fi
brin
olyt
ic t
hera
py(h
ours
:)a.
0–1/
ctrl
b.≥
1–2/
ctrl
c.≥
2–3/
ctrl
d.≥
3–6/
ctrl
e.≥
6–12
/ctr
lf.
≥12
–24/
ctrl
Tota
l n =
22
Res
ults
:Su
bgro
up r
esul
ts. A
bsol
ute
redu
ctio
n (S
D) i
n m
orta
lity
per
1000
a.
65 (1
4)b.
37 (9
)c.
26 (6
)d.
29 (5
)e.
18 (6
)f.
9 (7
)
Not
don
eT
he a
utho
rs c
oncl
ude
that
“The
ben
efic
ial e
ffect
of
fibrin
olyt
ic t
hera
py is
subs
tant
ially
hig
her
in p
atie
nts
pres
entin
g w
ithin
2 h
afte
rsy
mpt
om o
nset
com
pare
d to
thos
e pr
esen
ting
late
r”
The
aut
hors
do
not
men
tion
the
pote
ntia
l pro
blem
sas
soci
ated
with
ICs.
How
ever
,in
an
earli
er m
eta-
anal
ysis
(upo
n w
hich
thi
s re
view
isba
sed)
it is
arg
ued
that
“If
patie
nt c
ateg
orie
s ca
n be
arra
nged
in s
ome
mea
ning
ful
orde
r th
en …
may
be
reas
onab
ly r
elia
bly
info
rmat
ive
…”
App
ropr
iate
inte
rpre
tati
on?
No
Alte
rnat
ive
anal
ysis
ofpr
evio
us m
eta-
anal
ysis47
Health Technology Assessment 2005; Vol. 9: No. 26
97
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Gar
g an
d Yu
suf7
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
�2
test
for
hete
roge
neity
was
unde
rtak
en. N
o sig
nific
ant
hete
roge
neity
sho
wn
for
tota
lm
orta
lity
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Diff
eren
t A
CE
inhi
bito
rs:
a.Be
naze
pril
hydr
ochl
orid
e/ct
rl (n
=2)
b.C
apto
pril/
ctrl
(n=
6)c.
Cila
zapr
il/ct
rl (n
=1)
d.En
alap
ril m
alea
te/c
trl
(n=
7)e.
Lisin
opril
/ctr
l (n
=4)
f.Pe
rindo
pril/
ctrl
(n=
1)g.
Qui
napr
ilhy
droc
hlor
ide/
ctrl
(n=
5)h.
Ram
ipril
/ctr
l (n
=6)
Diff
eren
t du
ratio
n of
follo
w-
up:
�90
day
s/ct
rl (n
=32
)>
90 d
ays/
ctrl
(n=
12)
Res
ults
:To
tal m
orta
lity
(OR)
Diff
eren
t A
CE
inhi
bito
rs:
a.0.
36b.
0.79
c.0.
12d.
0.78
e.0.
62f.
0.14
g.0.
79h.
0.67
Dur
atio
n of
follo
w-u
p:�
90 d
ays/
ctrl
0.56
(0.4
4, 0
.70)
> 9
0 da
ys/c
trl 0
.87
(0.7
5,1.
01)
Not
don
e: n
o tr
ials
mak
ing
dire
ctco
mpa
rison
bet
wee
ndi
ffere
nt A
CE
inhi
bito
rsre
port
ed in
the
rev
iew
Aut
hors
con
clud
ed t
hat
“sim
ilar
bene
fits
wer
eob
serv
ed w
ith s
ever
al d
iffer
ent
AC
E in
hibi
tors
, alth
ough
dat
aw
ere
larg
ely
base
d on
ena
lapr
ilm
alea
te, c
apto
pril,
ram
ipril
,qu
inap
ril h
ydro
chlo
ride
and
lisin
opril
”
“The
gre
ates
t ef
fect
was
see
ndu
ring
the
first
3 m
onth
s, b
utad
ditio
nal b
enef
it w
asob
serv
ed d
urin
g fu
rthe
rtr
eatm
ent”
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
It is
poss
ible
to
mak
ew
ithin
-tria
l com
paris
onfo
r du
ratio
n of
follo
w-
up a
nd e
ffect
, alth
ough
the
conc
lusio
n is
unlik
ely
to b
e di
ffere
nt
Appendix 3
98 TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Hol
me35
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
with
6-m
onth
follo
w-u
pin
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
Met
a-re
gres
sion
unde
rtak
en
Met
hod
used
:M
eta-
regr
essio
n (m
ultip
le)
adju
sted
by
chol
este
rol
chan
ges
and
base
line
risk
ofC
AD
Com
pari
sons
mad
e:D
iet/
plac
ebo
(n=
12)
Hor
mon
es/p
lace
bo (n
=3)
Fibr
ates
/pla
cebo
(n=
7)St
atin
s/pl
aceb
o (n
=3)
Oth
er (o
ther
drug
s/su
rger
y)/p
lace
bo (n
=6)
Res
ults
:To
tal m
orta
lity:
Die
t/fib
rate
s O
R 0.
975
(p>
0.05
), z
=0.
84
Stat
ins/
fibra
tes
OR
0.83
3(p
<0.
05),
z=
3.37
Not
don
e: n
o tr
ials
mak
ing
dire
ctco
mpa
rison
bet
wee
ndi
ffere
nt in
terv
entio
nsre
port
ed in
the
rev
iew
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Hom
ocys
tein
e Lo
wer
ing
Tria
lists
’ Col
labo
ratio
n8
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
Het
erog
enei
ty a
cros
s st
udie
sas
sess
ed b
y m
ultiv
aria
tere
gres
sion
anal
ysis
Met
hod
used
:A
djus
ted
IC u
sing
a no
-tr
eatm
ent
cont
rol a
sco
mm
on c
ompa
rato
r.Ill
ustr
ated
in g
raph
Com
pari
sons
mad
e:D
iffer
ent
folic
aci
d re
gim
en:
a.1
mg/
ctrl
(n=
5)b.
1–3
mg/
ctrl
(n=
5)
c.>
3 m
g/ct
rl (n
=4)
Res
ults
:%
red
uctio
n in
blo
odho
moc
yste
ine
(95%
CI):
a.26
% (2
35 t
o 29
%)
b.25
% (2
0 to
29%
)c.
255
(21
to 2
8%)
Not
don
eA
utho
rs s
ugge
st a
wid
e ra
nge
of d
oses
(0.5
–5 m
g) t
o be
simila
rly e
ffect
ive
App
ropr
iate
inte
rpre
tati
on?
Yes
Health Technology Assessment 2005; Vol. 9: No. 26
99
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Koch
et
al.34
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Valid
ity a
sses
sed
usin
gpr
evio
usly
pub
lishe
dcr
iteria
.130,
131
Med
ian
qual
itysc
ore
for
all t
rials
pres
ente
d.O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
A L
’Abb
e pl
ot a
nd t
he Q
stat
istic
wer
e us
ed t
o ex
amin
ehe
tero
gene
ity
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
H2
bloc
kers
/pla
cebo
(nin
etr
ials)
Miso
pros
tol/p
lace
bo (t
entr
ials)
Res
ults
:Ra
te d
iffer
ence
(95%
CI)
Shor
t-te
rm r
isk fo
r ga
stric
ulce
r:
H2
bloc
kers
/pla
cebo
: –0.
9%(–
4.0
to 2
.2%
)M
isopr
osto
l/pla
cebo
: –13
.3%
(–25
.7 t
o –0
.9%
)
Long
-ter
m r
isk fo
r ga
stric
ulce
r:H
2bl
ocke
rs/p
lace
bo: –
0.3%
(–2.
9 to
2.2
%)
Miso
pros
tol/p
lace
bo: –
8.4%
(–17
.7 t
o –1
.0%
)
Not
don
e: n
o tr
ials
mak
ing
dire
ctco
mpa
rison
bet
wee
n H
2bl
ocke
rs a
nd m
isopr
otol
repo
rted
in t
he r
evie
w
Aut
hor’
s cl
aim
tha
t “g
astr
icul
cer
was
foun
d to
be
signi
fican
tly r
educ
ed b
ym
isopr
osto
l, bo
th in
sho
rt-
term
and
long
-ter
m N
SAID
trea
tmen
t, bu
t no
t by
H2
bloc
kers
”
“We
foun
d di
scre
panc
ies
betw
een
the
resu
lts o
f H2
bloc
ker
and
miso
pros
tol t
rials.
Stud
ies
on m
isopr
osto
lad
mitt
ed s
ubje
cts
with
ahi
gher
risk
for
the
deve
lopm
ent
of g
astr
icda
mag
e”
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Resu
lts a
lso p
rese
nted
for
gast
ric le
sions
,du
oden
al u
lcer
s an
ddu
oden
al le
sions
Lefe
ring
and
Neu
geba
uer39
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
A p
revi
ously
pub
lishe
d qu
ality
chec
klist
was
use
d.13
2A
qua
lity
scor
e w
as p
rese
nted
for
each
tria
l. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
A t
est
of h
eter
ogen
eity
was
cond
ucte
d ac
cord
ing
toC
ochr
an a
nd a
ran
dom
effe
cts
mod
el u
sed
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
a.Lo
w-d
ose
cort
icos
tero
id/c
trl (
n=
5)b.
Hig
h-do
seco
rtic
oste
roid
/ctr
l (n
=5)
Res
ults
:M
orta
lity
rate
% (9
5% C
I)a.
–1.9
% (–
20.0
to
16.2
%)
b.3.
6% (2
.5 t
o 9.
8%)
Not
don
e. N
o he
ad-t
o-he
ad t
rials
of h
igh
vs lo
wdo
se a
vaila
ble
“Nei
ther
the
typ
e of
ste
roid
used
nor
the
sep
arat
ion
into
low
-dos
e or
hig
h-do
sere
gim
en in
dica
ted
are
mar
kabl
e di
ffere
nce
betw
een
the
ster
oid
grou
p an
dco
ntro
l gro
up”
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Appendix 3
100 TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Leiz
orov
icz30
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
No
signi
fican
t he
tero
gene
ityw
as o
bser
ved
Met
hod
used
:N
o fo
rmal
met
hod
used
.C
ompa
rison
of O
R,pr
esen
ted
on fo
rest
plo
t
Com
pari
sons
mad
e:a.
LMW
H h
ospi
tal/U
FHho
spita
l (n
=18
)b.
LMW
H h
ome/
UFH
hos
pita
l(n
=2)
Res
ults
:Re
curr
ent
thro
mbo
embo
licev
ents
(OR)
:a.
0.76
b.0.
79
Mor
talit
y:a.
0.66
b.0.
75
Not
don
e. T
rials
ongo
ing
“Alth
ough
thi
s ap
proa
chre
duce
d th
e st
atist
ical
pow
erfo
r ea
ch s
ubgr
oup,
effi
cacy
resu
lts w
ere
simila
r fo
rre
curr
ence
of t
hrom
boem
bolic
even
ts o
r de
ath”
App
ropr
iate
inte
rpre
tati
on?
Yes
IC a
lso m
ade
for
rout
eof
adm
inist
ratio
n
Moo
re a
nd M
cQua
y36
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Valid
ity a
sses
sed
usin
g th
eJa
dad
scor
e133
All
tria
ls sc
ored
max
imum
5 p
oint
s
Ass
essm
ent
ofhe
tero
gene
ity:
No
form
al e
xam
inat
ion
ofhe
tero
gene
ity p
rese
nted
Met
hod
used
:A
djus
ted
IC, u
sing
plac
ebo
as c
ompa
rato
r. T
he s
ame
plac
ebo
grou
p m
ay h
ave
been
use
d to
com
pare
with
diffe
rent
act
ive
drug
s w
ithin
tria
ls
Com
pari
sons
mad
e:a.
Cod
eine
(60
mg)
/pla
cebo
b.Tr
amad
ol (5
0 m
g)/p
lace
boc.
Tram
adol
(75
mg)
/pla
cebo
d.Tr
amad
ol (1
00 m
g)/p
lace
boe.
Tram
adol
(150
mg)
/pla
cebo
f.A
ceta
min
ophe
n (6
50 m
g)pl
us p
ropo
xyph
ene
(100
mg)
/pla
cebo
g.A
spiri
n (6
50 m
g) p
lus
code
ine
(60
mg)
/pla
cebo
Res
ults
:RR
(95%
CI)
for
patie
nts
achi
evin
g
50%
of %
max
-T
OT
PAR
Den
tal p
ain:
c.1.
3 (0
.8 t
o 2.
1)d.
2.9
(1.6
to
5.2)
e.2.
7 (1
.1 t
o 6.
5)f.
3.8
(2.4
to
5.8)
g.4.
8 (2
.1 t
o 11
.1)
h.4.
0 (1
.7 t
o 9.
4)i.
3.8
(2.2
to
6.8)
Not
don
e, a
lthou
ghpo
ssib
le fr
om d
ata
pres
ente
d
Inte
rpre
ted
as if
dire
ctco
mpa
rison
s w
ere
mad
e
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Dat
a on
pos
tsur
gica
lpa
in a
lso p
rese
nted
inpa
per
Health Technology Assessment 2005; Vol. 9: No. 26
101
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Non
-sm
all C
ell L
ong
Can
cer
Col
labo
rativ
e G
roup
32
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
�2
test
for
hete
roge
neity
was
unde
rtak
en fo
r bo
th b
etw
een
and
with
in c
hem
othe
rapy
cate
gorie
s. W
hen
gros
she
tero
gene
ity w
as d
etec
ted,
the
ratio
nale
for
com
bini
ngda
ta w
as q
uest
ione
d an
dso
urce
s of
het
erog
enei
tyex
amin
ed, r
athe
r th
an u
sing
ara
ndom
effe
cts
mod
el
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
a.Su
rger
y pl
us lo
ng-t
erm
alky
latin
g ag
ent/
surg
ery
alon
eb.
Surg
ery
plus
cisp
latin
/sur
gery
alo
nec.
Radi
cal r
adio
ther
apy
(RT
)pl
us a
lkyl
atin
gag
ents
/rad
ical
RT
alo
ned.
Radi
cal R
T p
lus
cisp
latin
/rad
ical
RT
alo
ne
Res
ults
:O
–E d
eath
s (v
aria
nce)
a.55
.53
(394
.74)
b.–2
1.58
(151
.83)
c.–2
.83
(140
.23)
d.–5
7.08
(411
.18)
Not
don
e: n
o tr
ials
mak
ing
dire
ctco
mpa
rison
bet
wee
nch
emot
hera
py d
rugs
repo
rted
in t
he r
evie
w
“Fur
ther
ran
dom
ised
tria
ls ar
ene
eded
to
dete
rmin
e w
hich
regi
men
s ar
e th
e m
ost
effe
ctiv
e of
the
mod
ern
chem
othe
rapi
es s
tudi
ed”
App
ropr
iate
inte
rpre
tati
on?
Yes
Oth
er c
ompa
rison
spr
esen
ted
in a
rtic
le.
Onl
y m
ain
resu
ltsex
trac
ted
Pign
on e
t al
.31
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
Test
s of
het
erog
enei
ty w
ere
perf
orm
ed; h
owev
er, t
heau
thor
s st
ate
that
“su
bsta
ntia
lhe
tero
gene
ity d
oes
not
inva
lidat
e th
e re
sults
of a
met
a-an
alys
is”
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
a.Ea
rly R
T/c
trl (
n=
7)b.
Late
RT
/ctr
l (n
=6)
c.W
ith s
eque
ntia
l RT
/ctr
l(n
=8)
d.W
ithou
t se
quen
tial R
T/c
trl
(n=
5)
Res
ults
:O
R (9
5% C
I)a.
0.88
(0.7
8 to
0.9
8)b.
0.81
(0.6
9 to
0.9
4)c.
0.86
(0.7
5 to
1.0
0)d.
0.85
(0.7
5 to
0.9
6)
Not
don
e. N
o tr
ials
dire
ctly
com
parin
gea
rly/la
te, w
ith/w
ithou
tse
quen
tial R
T a
vaila
ble
“Ind
irect
com
paris
on o
f ear
lyw
ith la
te r
adio
ther
apy
and
ofse
quen
tial w
ith n
on-s
eque
ntia
lra
diot
hera
py d
id n
ot r
evea
l any
optim
al ti
me
for
trea
tmen
t”
“The
sel
ectio
n of
an
optim
alsc
hedu
le o
f CT
com
bine
d w
ithRT
tha
t w
ould
lead
to
a m
ajor
incr
ease
in s
urvi
val w
ithm
inim
al t
oxic
ity is
the
prin
cipa
lch
alle
nge
raise
d by
our
stud
y….W
e ho
pe t
hat
the
resu
lts o
f fut
ure
tria
ls w
illse
ttle
thi
s qu
estio
n”
App
ropr
iate
inte
rpre
tati
on?
Yes
Appendix 3
102 TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Poyn
ard
et a
l.38
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
The
met
hodo
logi
cal q
ualit
y of
each
tria
l was
ass
esse
d us
ing
a14
-item
que
stio
nnai
re a
ndsc
ored
bet
wee
n –2
and
26.
Abr
eakd
own
of t
he s
corin
g fo
rea
ch t
rial i
s pr
esen
ted
Ass
essm
ent
ofhe
tero
gene
ity:
Sens
itivi
ty a
naly
sis w
aspe
rfor
med
by
stra
tific
atio
nac
cord
ing
to H
2bl
ocke
r us
ing
both
ran
dom
and
fixe
d ef
fect
s
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Lans
opra
zole
/ran
itidi
ne o
rfa
mot
idin
e (n
=5)
Oth
er d
rugs
/ran
iditi
ne o
rfa
mot
idin
e (n
=?)
Res
ults
:O
R (9
5% C
I) fo
r 4-
wee
khe
alin
g ra
te in
com
paris
onw
ith r
aniti
dine
or
fam
otid
ine
Lans
opra
zole
: 2.5
(1.7
to
3.7)
Om
epra
zole
: 2.9
(1.5
to
5.7)
Rani
tidin
e or
fam
otid
ine:
0.9
(0.7
to
1.2)
Niz
atid
ine:
1.0
(0.8
to
1.4)
Cim
etid
ine:
1.8
Sucr
alfa
te: 1
.0 (0
.7 t
o 1.
4)
Lans
opra
zole
dire
ctly
com
pare
d w
ithra
nitid
ine
or fa
mot
idin
e.A
utho
rs m
entio
n an
RCT
dire
ctly
com
parin
gla
nsop
razo
le v
ersu
som
epra
zole
in a
cute
duod
enal
ulc
erat
ion;
how
ever
, thi
s tr
ial i
s no
tin
clud
ed in
the
met
a-an
alys
is
The
indi
rect
com
paris
on w
asus
ed t
o ra
nk e
ffica
cy. I
t w
asco
nclu
ded
that
“th
ere
was
asig
nific
ant
diffe
renc
e be
twee
nth
e ef
ficac
y of
om
epra
zole
and
lans
opra
zole
on
the
one
hand
and
all t
he o
ther
gro
ups
on t
heot
her”
App
ropr
iate
inte
rpre
tati
on?
No
A p
revi
ous
met
a-an
alys
is us
ed t
he s
ame
indi
rect
met
hod.
134
Ross
ouw
33
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
The
aut
hors
ack
now
ledg
e th
atcl
inic
al h
eter
ogen
eity
exi
sts
betw
een
the
incl
uded
stu
dies
.C
orre
latio
n an
alys
es w
ere
perf
orm
ed t
o as
sess
whe
ther
the
ORs
for
dise
ase
chan
gew
ere
rela
ted
to b
asel
ine
or in
-tr
ial l
ow-d
ensit
y lip
opro
tein
leve
ls or
to
rela
tive
orab
solu
te d
iffer
ence
s be
twee
ntr
eatm
ent
and
cont
rol g
roup
sdu
ring
the
tria
l
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Vario
us in
terv
entio
ns/c
trl:
a.Li
fest
yle/
ctrl
(n=
3)b.
Resin
s/ct
rl (n
=2)
c.St
atin
s/ct
rl (n
=4)
d.C
ombi
natio
n/ct
rl (n
=5)
e.Su
rger
y/ct
rl (n
=1)
Res
ults
:O
R (9
5% C
I) fo
rca
rdio
vasc
ular
eve
nts
pres
ente
d in
fore
st p
lot.
Onl
y th
e de
tails
of t
he o
vera
llO
R fo
r al
l int
erve
ntio
nspr
esen
ted
in t
he t
ext
0.53
(0.4
5 to
0.6
3)
Not
don
eA
utho
rs c
oncl
uded
tha
t “t
here
is no
con
clus
ive
evid
ence
of a
clas
s ef
fect
”
App
ropr
iate
inte
rpre
tati
on?
Yes
Health Technology Assessment 2005; Vol. 9: No. 26
103
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
21
Syst
emat
ic re
view
s re
port
ing
adju
sted
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
aN
ot id
entif
ied
thro
ugh
DA
RE.
CH
D, c
oron
ary
hear
t di
seas
e; C
T, c
hem
othe
rapy
; RT,
rad
ioth
erap
y.
Tram
er e
t al
.11
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
The
Jada
d sc
ore
was
use
d133
Resu
lts n
ot p
rese
nted
Ass
essm
ent
ofhe
tero
gene
ity:
No
form
al e
xam
inat
ion
ofhe
tero
gene
ity p
rese
nted
Met
hod
used
:A
djus
ted
ICC
ompa
riso
ns m
ade:
Diff
eren
t do
ses
of d
rope
ridol
: a.
10 �
g kg
–1(n
= 1
)b.
20 �
g kg
–1(n
= 1
)c.
50 �
g kg
–1(n
= 2
)d.
75 �
g kg
–1(n
= 1
0)
Res
ults
:A
bsen
ce o
f ear
ly v
omiti
ng:
OR
(95%
CI)
a.1.
6 (0
.4 t
o 7.
2)b.
1.9
(0.7
to
5)c.
1.5
(0.7
to
3.2)
d.3.
3 (2
.4 t
o 4.
7)
Not
don
eT
he a
utho
rs’ c
autio
n ab
out
the
adju
sted
IC w
as m
ainl
y du
e to
smal
l sam
ple
size.
Pot
entia
lbi
as w
as n
ot m
entio
ned
“The
num
ber
of c
hild
ren
stud
ied
at t
he lo
wes
t do
ses
was
sm
all a
nd t
he c
onfid
ence
inte
rval
s w
ide;
nev
erth
eles
sth
ere
wou
ld s
eem
to
besu
ffici
ent
info
rmat
ion
tosu
gges
t th
at t
he u
se o
fsu
bmax
imal
dos
es is
not
wor
thw
hile
”
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
Det
ails
of o
ther
com
paris
ons
and
outc
ome
mea
sure
spr
esen
ted
in t
he r
evie
w
Zal
cber
g et
al.29
a
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
No
form
al e
xam
inat
ion
ofhe
tero
gene
ity p
rese
nted
Met
hod
used
:A
djus
ted
ICRe
gres
sion
anal
ysis
was
also
used
Com
pari
sons
mad
e:a.
≥10
g 5
-FU
/ctr
l (n
=3)
b.8–
9 g
5-FU
/ctr
l (n
=7)
c.<
8 g
5-FU
/ctr
l (n
=5)
d.O
ral C
T/c
trl (
n=
2)e.
5-FU
+le
vam
isole
/ctr
l(n
=2)
f.5-
FU/c
trl (
n=
15)
Res
ults
:O
R (9
5% C
I)a.
0.71
b.0.
79c.
0.93
d.1.
04e.
0.64
(0.4
9 to
0.8
5)f.
0.86
Not
don
e. N
o he
ad-t
o-he
ad t
rials
pres
ente
d in
the
revi
ew
“It
shou
ld b
e po
inte
d ou
t th
atth
e re
lativ
e ef
fect
s of
5-F
Udo
se a
nd o
f lev
amiso
le a
reba
sed
on in
dire
ct, n
on-
rand
omise
d co
mpa
rison
s in
this
anal
ysis,
so
that
conf
ound
ing
by t
he t
ype
ofpa
tient
s be
ing
stud
ied
in e
ach
tria
l is
a po
ssib
ility
”
The
aut
hors
also
con
clud
e th
atan
RC
T is
req
uire
d to
com
pare
5-F
U+
leva
miso
lew
ith 5
-FU
alo
ne
App
ropr
iate
inte
rpre
tati
on?
Yes
The
dos
e–re
spon
sere
latio
nshi
p is
quite
conv
inci
ng. H
owev
er,
the
bene
fit o
f add
ition
alle
vam
isole
to
5-FU
isun
clea
r as
the
tria
ls th
atin
clud
ed a
dditi
onal
leva
miso
le a
lso u
sed
high
er d
oses
of 5
-FU
Appendix 3
104 TA
BLE
22
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
only
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Bans
al a
nd B
eto42
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. P
rosp
ectiv
eco
ntro
lled
tria
ls w
ithtr
eatm
ent
allo
catio
n by
rand
om a
ssig
nmen
t or
cons
ecut
ive
enro
lmen
t
Ass
essm
ent
ofhe
tero
gene
ity:
Het
erog
enei
ty w
as a
sses
sed
and
a ra
ndom
effe
cts
mod
elus
ed. T
he a
ppro
pria
tene
ss o
fpo
olin
g w
as a
lso a
sses
sed
byco
mpa
ring
the
resu
lts b
etw
een
thos
e tr
ials
with
mat
ched
cont
rol a
nd a
ll tr
ials
Met
hod
used
:N
aive
IC o
f res
ults
of
diffe
rent
arm
s ac
ross
stud
ies
The
app
ropr
iate
ness
of
pool
ing
was
exa
min
ed b
ytw
o m
etho
ds (Z
-sco
re a
ndhe
tero
gene
ity t
est)
, but
the
resu
lts o
f the
tes
ts w
ere
not
pres
ente
d
Com
pari
sons
mad
e:Tr
eatm
ent
arm
s ac
ross
stu
dies
wer
e po
oled
to
com
pare
vario
us im
mun
osup
pres
sive
agen
ts p
lus
oral
pre
dniso
new
ith p
redn
isone
alo
ne (n
=?)
Res
ults
:A
bsol
ute
risk
diffe
renc
e (9
5%C
I) fo
r al
l im
mun
osup
pres
sive
agen
ts w
ithpr
edni
sone
/pre
dniso
ne a
lone
To
tal m
orta
lity:
13.
2% (2
.5 t
o23
.9%
)ES
RD: 1
2.9%
(2.2
to
23.6
%)
Not
don
eT
he c
oncl
usio
n w
as b
ased
on
ana
ive
IC w
ithou
t an
y ef
fort
to
adju
st fo
r po
tent
ial b
ias
and
conf
ound
ing.
Res
ults
wer
ein
terp
rete
d as
tho
ugh
DC
unde
rtak
en w
ith n
o di
scus
sion
of p
oten
tial b
iase
s
App
ropr
iate
inte
rpre
tati
on?
No
The
rev
iew
incl
uded
RCTs
or
quas
i-RC
Ts,
but
the
resu
lts o
f the
stud
ies
wer
e us
ed t
om
ake
betw
een-
stud
yco
mpa
rison
s, lo
sing
the
pow
er/r
igou
r of
rand
omisa
tion
Chi
ba e
t al
.40
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Blin
ding
and
met
hod
ofra
ndom
isatio
n w
ere
the
mai
nqu
ality
item
s as
sess
ed. O
nly
singl
e- o
r do
uble
-blin
d RC
Tsw
ere
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
No
exam
inat
ion
ofhe
tero
gene
ity r
epor
ted
Met
hod
used
:N
aive
IC
For
each
dru
g cl
ass
linea
rre
gres
sion
anal
ysis
estim
ated
the
ave
rage
perc
enta
ge o
f pat
ient
s w
how
ere
heal
ed a
nd h
eart
burn
free
per
wee
k
Com
pari
sons
mad
e:43
stu
dies
use
d to
com
pare
PPIs
(om
epra
zole
,la
nsop
razo
le a
ndpa
ntop
razo
le),
H2R
As
(cim
etid
ine,
niz
atid
ine,
rant
idin
e an
d fa
mot
idin
e),
sucr
alfa
te, p
roki
netic
s, p
lace
boan
d ot
her
Res
ults
:M
ean
over
all h
ealin
gpr
opor
tion:
PPIs
: 83.
6 ±
11.4
%
H2R
A: 5
1 ±
17.1
%Su
cral
fate
: 39.
2 ±
22.4
%Pl
aceb
o: 2
8.2
±15
.6%
Not
don
e bu
t po
ssib
leT
he a
utho
rs c
oncl
ude
that
“mor
e co
mpl
ete
esop
hagi
tishe
alin
g an
d he
artb
urn
relie
f is
obse
rved
with
PPI
s vs
H2-
RAs”
App
ropr
iate
inte
rpre
tati
on?
No
The
rev
iew
incl
udes
only
sin
gle-
or
doub
le-
blin
d RC
Ts, b
ut n
ow
ithin
-stu
dyco
mpa
rison
was
mad
e
It is
not
clea
r w
heth
er it
is a
stan
dard
err
or o
rst
anda
rd d
evia
tion
follo
win
g th
e es
timat
edov
eral
l pro
port
ion
Health Technology Assessment 2005; Vol. 9: No. 26
105
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
22
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Con
lin e
t al
.46
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
doub
le-
blin
d RC
Ts w
ere
incl
uded
Ass
essm
ent
ofhe
tero
gene
ity:
No
exam
inat
ion
ofhe
tero
gene
ity r
epor
ted
Met
hod
used
:N
aive
IC
Ana
lysis
bas
ed o
n tr
eatm
ent
arm
s. T
he a
bsol
ute
(non
-pl
aceb
o co
rrec
ted)
wei
ghte
d av
erag
e bl
ood
pres
sure
red
uctio
n w
asca
lcul
ated
for
each
AIIA
Com
pari
sons
mad
e:43
tria
ls w
ere
used
to
com
pare
the
effi
cacy
of
losa
rtan
, val
sart
an, i
rbes
arta
nan
d ca
ndes
arta
n
Res
ults
:T
he a
bsol
ute
wei
ghte
dav
erag
e re
duct
ions
in d
iast
olic
and
syst
olic
blo
od p
ress
ure
wer
e co
mpa
rabl
e fo
r al
l AIIA
s
Not
don
e, a
lthou
ghpo
ssib
le t
o do
so
Aut
hors
con
clud
e th
at “
this
anal
ysis
sugg
ests
tha
t A
IIAlo
wer
blo
od p
ress
ure
with
simila
r ef
ficac
y w
hen
adm
inist
ered
at
thei
r us
ual
dose
s fo
r th
e tr
eatm
ent
ofhy
pert
ensio
n”
App
ropr
iate
inte
rpre
tati
on?
No
The
rev
iew
incl
udes
only
RC
Ts, b
ut n
ow
ithin
-stu
dyco
mpa
rison
was
mad
e.O
vere
stim
atio
n of
bloo
d pr
essu
rere
duct
ion
beca
use
ofre
gres
sion
to m
ean
effe
ct, o
win
g to
the
igno
ring
of t
he p
lace
bogr
oups
Cou
lter
et a
l.41
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
RCTs
wer
e in
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
Clin
ical
het
erog
enei
tybe
twee
n tr
ials
was
not
ed. N
ote
sts
of h
eter
ogen
eity
wer
ere
port
ed
Met
hod
used
:N
aive
ICC
ompa
riso
ns m
ade:
31 s
tudi
es u
sed
to c
ompa
reth
e ef
ficac
y of
a v
arie
ty o
fdr
ugs
used
to
trea
tm
enor
rhag
ia
Res
ults
:D
rugs
list
ed a
ccor
ding
to
perc
enta
ge r
educ
tion
inm
enst
rual
blo
od lo
ss
Not
don
e, a
lthou
ghpo
ssib
le t
o do
so
Aut
hors
men
tion
limita
tions
such
as
lack
of p
lace
boco
ntro
ls, b
ut c
laim
the
rev
iew
of R
CTs
is s
atisf
acto
ry fo
rco
mpa
ring
rela
tive
effic
acy
ofdr
ugs
App
ropr
iate
inte
rpre
tati
on?
Unc
lear
The
rev
iew
incl
udes
only
RC
Ts, b
ut n
ow
ithin
-stu
dyco
mpa
rison
was
mad
e.H
alf o
f the
tria
lsin
clud
ed a
pla
cebo
grou
p, b
ut r
esul
ts o
fth
e pl
aceb
o co
ntro
lsw
ere
not
repo
rted
Appendix 3
106 TA
BLE
22
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Felso
n et
al.6
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Mod
ifica
tion
of a
pre
viou
slypu
blish
ed c
heck
list.13
5O
nly
RCTs
wer
e in
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
�2
test
of h
eter
ogen
eity
was
unde
rtak
en a
nd a
ran
dom
effe
cts
mod
el u
sed
whe
nap
prop
riate
Met
hod
used
:N
aive
ICA
djus
tmen
t w
as m
ade
for
som
e co
varia
tes,
but
not
clea
r ho
w a
djus
tmen
t w
asm
ade
Com
pari
sons
mad
e:Tr
ials
of s
econ
d lin
e dr
ugs
totr
eat
rheu
mat
oid
arth
ritis
wer
e in
clud
ed:
a.Pl
aceb
o (n
=22
)b.
Ant
imal
aria
l dru
gs (n
=11
)c.
Aur
anof
in (n
=23
)d.
Inje
ctab
le g
old
(n=
29)
e.M
etho
trex
ate
(n=
7)f.
DP
(n=
19)
g.SS
Z (n
=6)
Res
ults
:Ef
ficac
y:A
uran
ofin
was
foun
d to
be
signi
fican
tly w
eake
r th
anm
etho
trex
ate,
inje
ctab
le g
old,
DP
and
SSZ
, and
slig
htly,
but
not
signi
fican
tly w
eake
r th
anan
timal
aria
l age
nts
Toxi
city
:In
ject
able
gol
d ha
d hi
gher
toxi
city
rat
es a
nd h
ighe
r to
tal
drop
out
than
any
oth
er d
rug
Com
pari
sons
mad
e:N
ot d
one,
alth
ough
poss
ible
to
do s
o
With
in-s
tudy
com
paris
ons
igno
red
Info
rmat
ion
from
all
outc
ome
mea
sure
s an
d fr
om e
ach
tria
lco
mbi
ned
into
a c
ompo
site
mea
sure
of t
reat
men
t ef
fect
App
ropr
iate
inte
rpre
tati
on?
No
Thi
s re
view
was
bee
nqu
oted
by
Impe
riale
and
Sper
off a
s pr
oof o
fus
ing
naiv
e IC
s of
RC
Tda
ta
Dire
ct c
ompa
rison
stud
ies
avai
labl
e
Health Technology Assessment 2005; Vol. 9: No. 26
107
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
22
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
cont
inue
d
Met
hod
used
:N
aive
IC b
ased
on
prem
iseth
at t
he t
reat
men
t gr
oups
wer
e cl
inic
ally
hom
ogen
eous
inco
mpo
sitio
n
Com
pari
sons
mad
e:a.
LMW
H/c
trl (
n=
20)
b.W
arfa
rin/c
trl (
n=
10)
c.C
ompr
essio
n st
ocki
ngs/
ctrl
(n=
6)
Res
ults
:N
NT
(95%
CI)
for
all D
VTa.
3.2
(2.9
to
3.7)
b.4.
3 (3
.0 t
o 8.
0)c.
3.9
(3.1
to
5.1)
Not
don
eA
ppro
pria
te in
terp
reta
tion
?N
oA
utho
rs g
ive
the
impr
essio
n th
at t
heir
conc
lusio
ns w
ere
base
don
evi
denc
e fr
om R
CTs
,ev
en t
houg
h on
lybe
twee
n-st
udy
com
paris
ons
wer
em
ade
Quo
tes
revi
ew b
yFe
lson
et a
l.6to
sup
port
use
of IC
met
hod
Mor
e ou
tcom
es a
ndco
mpa
rison
s av
aila
ble
inth
e re
view
Schm
iede
r et
al.43
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. O
nly
doub
le-
blin
d RC
Ts in
clud
ed
Ass
essm
ent
ofhe
tero
gene
ity:
Hom
ogen
eity
of d
ata
coul
dno
t be
con
firm
ed fo
r al
ltr
eatm
ent
arm
s, s
o se
nsiti
vity
anal
ysis
was
und
erta
ken
Met
hod
used
:N
aive
IC
All
trea
tmen
t ar
ms
of t
hesa
me
drug
wer
e co
mbi
ned
Com
pari
sons
mad
e:a.
Diu
retic
s (n
= 1
3)b.
�-B
lock
ers
(n =
21)
c.C
alci
um-c
hann
el b
lock
ers
(n=
19)
d.A
CE
inhi
bito
rs (n
=18
)e.
Plac
ebo
(n=
13)
Res
ults
:M
ean
(SD
) % d
ecre
ase
insy
stol
ic/d
iast
olic
blo
odpr
essu
rea.
10.7
(1.8
)/13.
1 (3
.5)
b.12
.8 (3
.8)/
15.4
(2.8
)c.
10.3
(3.6
)/13.
4 (2
.3)
d.11
.9 (4
.6)/
13.2
(5.4
)
Not
don
e, a
lthou
ghpo
ssib
le t
o do
so
The
aut
hors
disc
uss
the
impo
rtan
ce o
f ran
dom
isatio
nan
d sc
ient
ific
qual
ity o
f stu
dies
.H
owev
er, t
hey
igno
re w
ithin
-st
udy
com
paris
ons
and
mak
eno
att
empt
to
adju
st t
he IC
acco
rdin
g to
a c
omm
onpl
aceb
o gr
oup
App
ropr
iate
inte
rpre
tati
on?
No
See
upda
ted
revi
ew b
yth
e sa
me
auth
ors13
6
Impe
riale
and
Spe
roff45
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Onl
y RC
Ts in
clud
ed. T
rials
asse
ssed
on
incl
usio
n/ex
clus
ion
crite
ria; b
asel
ine
simila
rity;
blin
d ad
min
istra
tion
ofin
terv
entio
n; d
escr
iptio
n of
co-in
terv
entio
ns; d
rop-
outs
and
with
draw
als.
Sum
mar
ysc
ore
(max
. 10)
ass
igne
d to
each
tria
l
Ass
essm
ent
ofhe
tero
gene
ity:
Ana
lysis
bas
ed o
n pr
emise
tha
ttr
eatm
ent
grou
ps w
ere
clin
ical
ly h
omog
enou
s.Ra
ndom
effe
cts
mod
el u
sed
Appendix 3
108 TA
BLE
22
Syst
emat
ic re
view
s re
port
ing
naive
indi
rect
com
paris
ons
only
(co
nt’d
)
Stud
yM
etho
d of
IC
Indi
rect
com
pari
son
Dir
ect
com
pari
son
Des
crip
tion
of i
nter
pret
atio
nC
omm
ents
DP,
D-p
enic
illam
ine;
SSZ
, sul
fasa
lazi
ne.
Ung
e an
d Be
rsta
d44
Val
idit
y as
sess
men
tun
dert
aken
wit
hin
the
revi
ew:
Non
e re
port
ed. S
tudy
des
igns
uncl
ear
Ass
essm
ent
ofhe
tero
gene
ity:
Sens
itivi
ty a
naly
sis o
n m
ajor
diffe
renc
es in
dos
e, d
osag
ean
d du
ratio
n w
as p
erfo
rmed
Met
hod
used
:N
aive
ICC
ompa
riso
ns m
ade:
17 d
iffer
ent
trea
tmen
t gr
oups
For
exam
ple:
a.
Om
epra
zole
+ a
mox
ycill
in+
cla
rithr
omyc
in (n
=59
)b.
Bism
uth
+ n
itroi
mid
azol
e+
tet
racy
clin
e (n
=87
)
Res
ults
:Er
adic
atio
n ra
te:
a.87
% (r
ange
72–
100%
b.82
% (r
ange
43–
100%
)
Not
don
e, a
lthou
ghpo
ssib
le t
o do
so
The
aut
hors
con
clud
e th
at“o
mep
razo
le/c
larit
hrom
ycin
base
d tr
iple
reg
imen
s ar
e th
em
ost
effe
ctiv
e an
ti-H
. pyl
ori
ther
apeu
tic s
trat
egy,
slig
htly
supe
rior
to b
ismut
h tr
iple
regi
men
s”
App
ropr
iate
inte
rpre
tati
on?
No
Obs
erva
tiona
l stu
dies
and
RCTs
are
incl
uded
in t
he r
evie
w. D
etai
ls of
othe
r co
mpa
rison
s ar
epr
esen
ted
in t
he r
evie
w
The following reviews were identified from thesearches as potentially including indirect
comparisons or both direct and indirectcomparison of competing interventions. However,after assessment they did not include suitable dataand so were excluded.
Arriagada R, Pignon JP, Ihde DC, Johnson DH,Perry MC, Souhami RL, et al. Effect of thoracicradiotherapy on mortality in limited small celllung cancer. A meta-analysis of 13 randomizedtrials among 2,140 patients. Anticancer Res 1994;14(1B):333–5.
van Balkom AJ, Nauta MC, Bakker A. Meta-analysis on the treatment of panic disorder withagoraphobia; review and re-examination. ClinPsychol Psychother 1995;2:1–14.
Bhansali M, Vaidya J, Bhatt R, Patil P, Badwe R,Desai P. Chemotherapy for carcinoma of theesophagus: a comparison of evidence from meta-analyses of randomized trials and of historicalcontrol studies. Ann Oncol 1996;7:355–9.
Boyer W. Serotonin uptake inhibitors are superiorto imipramine and alprazolam in alleviating panicattacks: a meta-analysis. Int Clin Psychopharmacol1995;10:45–9.
Chang D, Wilson S. Meta-analysis of the clinicaloutcome of carbapenem monotherapy in theadjunctive treatment of intra-abdominalinfections. Am J Surg 1997;174:284–90.
Childhood ALL Collaborative Group. Durationand intensity of maintenance chemotherapy inacute lymphoblastic leukaemia: overview of 42trials involving 12 000 randomised children.Lancet 1996;347:1783–8.
de Craen A, Di Giulio G, Lampe-SchoenmaechersA, Kessels A, Kleijnen J. Analgesic efficacy andsafety of paracetamol-codeine combinations versusparacetamol alone: a systematic review. BMJ1996;313:321–5.
Droitcour J, Silberman G, Chelimsky E. A newform of meta-analysis for combining results fromrandomized clinical trials and medical-practice
databases. Int J Technol Assess Health Care 1993;9:440–9.
Eriksson S, Langstrom G, Rikner L, Carlsson R,Naesdal J. Omeprazole and H2 receptorantagonists in the acute treatment of duodenalulcer, gastric ulcer and reflux oesophagitis: ameta-analysis. Eur J Gastroenterol Hepatol 1995;7:467–75.
Golzari H, Cebul R, Bahler R. Atrial fibrillation:restoration and maintenance of sinus rhythm andindications for anticoagulation therapy. Ann InternMed 1996;125:311–23.
Halliday H. Overview of clinical trials comparingnatural and synthetic surfactants. Biol Neonate1995;67(Suppl):32–47.
Held P, Yusuf S. Calcium anatagonists in thetreatment of ischemic heart disease: myocardialinfarction. Coron Artery Dis 1994;5:21–6.
Hoes A, Grobbee D, Lubsen J. Does drugtreatment improve survival? Reconciling the trialsin mild-to-moderate hypertension. J Hypertens1995;13:805–11.
Hoes A, Grobbee D, Peet T, Lubsen J. Do non-potassium-sparing diuretics increase the risk ofsudden cardiac death in hypertensive patients?Drugs 1994;47:711–33.
Hooks M. Tacrolimus, a new immunosuppressant– a review of the literature. Ann Pharmacother1994;28:501–11.
Koes B, Assendelft W, van der Heijden G, Bouter L. Spinal manipulation for low back pain.An updated systematic review of randomizedclinical trials. Spine 1996;21:2860–71.
Linde K, Ramirez G, Mulrow C, Pauls A,Weidenhammer W, Melchart D. St John's wort fordepression – an overview and meta-analysis ofrandomised clinical trials. BMJ 1996;313:253–8.
MacRae H, McLeod R. Comparison ofhemorroidal treatment modalities: a meta-analysis.Dis Colon Rectum 1995;38:687–94.
Health Technology Assessment 2005; Vol. 9: No. 26
109
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 4
List of excluded reviews
McQuay H, Carroll D, Jadad AR, Wiffen P, Moore A. Anticonvulsant drugs for managementof pain: a systematic review. BMJ 1995;311:1047–52.
McQuay H, Tramer M, Nye B, Carroll D, Wiffen P,Moore R. A systematic review of antidepressants inneuropathic pain. Pain 1996;68:217–27.
Meunier F, Paesmans M, Autier P. Value ofantifungal prophylaxis with antifungal drugsagainst oropharyngeal candidiasis in cancerpatients. Eur J Cancer 1994;30B:196–9.
Ofman J, Koretz R. Clinical economics review:nutritional support. Aliment Pharmacol Ther 1997;11:453–71.
Patrono C, Roth GJ. Aspirin in ischemiccerebrovascular disease. How strong is the case fora different dosing regimen? Stroke 1996;27:756–60.
Piccinelli M, Pini S, Bellantuono C, Wilkinson G.Efficacy of drug treatment in obsessive–compulsivedisorder: a meta-analytic review. Br J Psychiatry1995;166:424–43.
Rahlfs V, Macciocchi A, Monti T. Brodimoprim inupper respiratory tract infections. Two meta-analyses of randomised, controlled clinical trials inacute sinusitis and otitis media. Clinical DrugInvestigation 1996;11:65–76.
Rains C, Noble S, Faulds D. Sulfasalazine. A reviewof its pharmacological properties and therapeutic
efficacy in the treatment of rheumatoid arthritis.Drugs 1995;50:137–56.
Riedemann P, Bersinic S, Cuddy L, Torrance G,Tugwell P. A study to determine the efficacy andsafety of tenoxicam versus piroxicam, diclofenacand indomethacin in patients with osteoarthritis: ameta-analysis. J Rheumatol 1993;20:2095–103.
Tramonte S, Brand M, Mulrow C, Amato M,O'Keefe M, Ramirez G. The treatment of chronicconstipation in adults. A systematic review. J GenIntern Med 1997;12:15–24.
Vaitkus PT, Berlin JA, Schwartz JS, Barnathan ES.Stroke complicating acute myocardial infarction. A meta-analysis of risk modification byanticoagulation and thrombolytic therapy. ArchIntern Med 1992;152:2020–4.
Voogel A, van der Meulen J, van Montfrans G.Effects of antihypertensive drugs on the circadianblood pressure profile. J Cardiovasc Pharmacol1996;28:463–9.
Wade C, Kramer G, Grady J, Fabian T, Younes R.Efficacy of hypertonic 7.5% saline and 6%dextran-70 in treating trauma: a meta-analysis ofcontrolled clinical studies. Surgery 1997;122:609–16.
Wood M. The comparative efficacy and safety ofteicoplanin and vancomycin. J Antimicrob Chemother1996;37:209–22.
Appendix 4
110
Search strategy 1The process involved an initial ‘One Search’ onDialog to gauge the amount of literature onindirect comparisons and to identify suitabledatabases for future searching (search strategy 1)
Search strategy 1:
S1 randomized controlled trials/ab,ti,deS2 trial?/ab,ti,deS3 s1 or s2S4 (indirect(2w)comparison?)/ab,ti,deS5 (direct(2w)comparison?)/ab,ti,deS6 (indirect(2w)evaluat?)/ab,ti,deS7 (direct(2w)evaluat?)/ab,ti,deS8 (treatment(2w)arm?)/ab,ti,deS9 (compet?(2w)technolog?)/ab,ti,de
S10 (compet?(2w)intervention?)/ab,ti,deS11 s4:s10S12 s3 and s11
Search strategy 1 was run across all the databaseslisted below up to October 1997 and retrievedrecords as follows:
Database Records retrievedMEDLINE 723ERIC 1PsycINFO 29EMBASE 901Dissertation Abstracts 33MathSci 8
Search strategy 2The next step was to run a strategy on MEDLINEvia Ovid (Search strategy 2) to retrieve the full textof each record to help in the process of identifyingsuitable terms for inclusion in the final searchstrategy. This strategy identified 759 records onMEDLINE when run for the period 1966 toOctober 1997.
Search strategy 2:
1 exp RANDOMIZED CONTROLLED TRIALS/2 trial$.tw.3 1 or 2
4 (indirect adj2 comparison$).tw.5 (direct adj2 comparison$).tw.6 (indirect adj2 evaluat$).tw.7 (direct adj2 evaluat$).tw.8 (treatment adj2 arm$).tw.9 (compet$ adj2 technolog$).tw.
10 (compet$ adj2 intervention$).tw.11 4 or 5 or 6 or 7 or 8 or 9 or 1012 3 and 11
Search strategy 3After further consideration of how ‘competinginterventions’ or indirect comparisons weredescribed in individual studies and which MeSHheadings had been used to index records, thestrategy was then further developed (searchstrategy 3).
Search strategy 3 was run for the year 1997 andretrieved 1690 records. When these records wereexamined by the reviewers it was found that manyof them were reports of single RCTs, rather thandiscussion of the methodology of RCTs.
Search strategy 3:
1 RANDOMIZED CONTROLLED TRIALS/2 controlled clinical trials.sh.3 CLINICAL TRIALS/4 clinical trials.tw.5 trial$.tw.6 meta-analysis.sh.7 meta-analysis.tw.8 metaanalys$.tw.9 (meta adj analys$).tw.
10 RESEARCH DESIGN/11 data interpretation,statistical.sh.12 models,statistical.sh.13 (indirect adj2 comparison$).tw.14 (direct adj2 comparison$).tw.15 (indirect adj2 evaluat$).tw.16 (direct adj2 evaluat$).tw.17 (compet$ adj2 technolog$).tw.18 (compet$ adj2 intervention$).tw.19 (treatment adj2 arm$).tw.20 (treatment adj2 group$).tw.21 (randomi$ adj2 group$).tw.22 (randomi$ adj2 comparison$).tw.
Health Technology Assessment 2005; Vol. 9: No. 26
111
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 5
Search strategy development
23 (therapeutic adj2 arm).tw.24 (therapeutic adj2 arms).tw.25 (study adj2 arm).tw.26 (study adj2 arms).tw.27 4 limb study.tw.28 four limb study.tw.29 (trial adj2 arm).tw.30 (trial adj2 design).tw.31 (placebo adj2 arm).tw.32 (preventive adj2 arm).tw.33 (preventative adj2 arm).tw.34 or/1-935 or/10-3336 34 and 35
Search strategy 4In an attempt to remove records describing singlestudies from the search results the strategy wasamended. The desired outcome was to retrieverecords in which there was reference to trials andmeta-analysis and one of the possible free textterms used for describing competinginterventions/indirect comparisons (searchstrategy 4).
Search strategy 4:
1 RANDOMIZED CONTROLLED TRIALS/2 controlled clinical trials.sh.3 CLINICAL TRIALS/4 clinical trials.tw.5 trial$.tw.6 meta-analysis.sh.7 meta-analysis.tw.8 metaanalys$.tw.9 (meta adj analys$).tw.
10 RESEARCH DESIGN/11 data interpretation,statistical.sh.12 models,statistical.sh.13 (indirect adj2 comparison$).tw.14 (direct adj2 comparison$).tw.15 (indirect adj2 evaluat$).tw.16 (direct adj2 evaluat$).tw.17 (compet$ adj2 technolog$).tw.18 (compet$ adj2 intervention$).tw.19 (treatment adj2 arm$).tw.20 (treatment adj2 group$).tw.21 (randomi$ adj2 group$).tw.22 (randomi$ adj2 comparison$).tw.23 (therapeutic adj2 arm).tw.24 (therapeutic adj2 arms).tw.25 (study adj2 arm).tw.26 (study adj2 arms).tw.27 4 limb study.tw.28 four limb study.tw.
29 (trial adj2 arm).tw.30 (trial adj2 design).tw.31 (placebo adj2 arm).tw.32 (preventive adj2 arm).tw.33 (preventative adj2 arm).tw.34 or/1-535 or/6-936 or/10-3337 34 and 35 and 36
Search strategy 5Search strategy 4 was run on MEDLINE for theperiod 1995 to December 1998 and retrieved 238records. There was still some uncertainty, however,about the recall of the search strategy, and in anattempt to improve this additional free text termswere included in the third facet of the strategy(search strategy 5). The inclusion of the additionalterms resulted in the retrieval of some additional131 records.
Search strategy 5:
1 RANDOMIZED CONTROLLED TRIALS/2 controlled clinical trials3 CLINICAL TRIALS/4 clinical trials.tw.5 trial$.tw.6 1 or 2 or 3 or 4 or 57 meta-analysis.sh.8 meta-analysis.tw.9 metaanalys$.tw.
10 (meta adj analys$).tw.11 7 or 8 or 9 or 1012 RESEARCH DESIGN/13 data interpretation, statistical.sh.14 models, statistical.sh.15 (indirect adj2 comparison$).tw.16 (direct adj2 comparison$).tw.17 (indirect adj2 evaluat$).tw.18 (direct adj2 evaluat$).tw.19 (compet$ adj2 technolog$).tw.20 (compet$ adj2intervention$).tw.21 (treatment adj2 arm$).tw.22 (treatment adj2group$).tw.23 (randomi$ adj2group$).tw.24 (randomi$ adj2comparison$).tw.25 (therapeutic adj2 arm).tw.26 (therapeutic adj2 arms).tw.27 (study adj2 arm).tw.28 (study adj2 arms).tw.29 4 limb study.tw.30 four limb study31 (trial adj2 arm).tw.32 (trial adj2 design).tw.
Appendix 5
112
33 (placebo adj2 arm).tw.34 (preventive adj2arm).tw.35 (preventative adj2 arm).tw.36 multiple arm study.tw.37 multiple arms study.tw.38 multiple arm studies.tw.39 multiple arms studies.tw.40 multiple arm.tw.41 multiple arms.tw.42 multi arm.tw.43 multi arms.tw.44 (multi adj2 arm).tw.45 (multi adj2 arms).tw.46 (multiple adj2 arm).tw.47 (multiple adj2 arms).tw.48 ((three arm or three arms or 3 arm or 3 arms
or three limb or three limbs or 3 limb or3limbs) adj5 (trial$ or stud$ or random$))
49 ((four arm or four arms or 4 arm or 4 arms orfour limb or four limbs or 4 limb or 4 limbs)adj5 (trial$ or stud$ or random$))
50 (competing adj2 therap$).tw.51 (multi$ adj2 (study or studies)).tw.52 or/12-5153 6 and 11 and 52
Before 1989, records in MEDLINE referring tometa-analysis were indexed using the terms:Outcome and Process Assessment (1977–1979),Follow-Up Studies (1977–1979), Research(1980–1982), Research Design (1980–1988) andStatistics (1980–1988). It was not clear what effectthis would have on running the existing strategyon these earlier years, so various trial strategieswere undertaken. These included (1) omitting themeta-analysis facet from the strategy entirely; (2) replacing the meta-analysis terms with the termsresearch, research design, follow-up studies,statistics, outcome and process assessment using theexplosion facility; and (3) replacing the meta-analysis terms with the terms research, researchdesign, follow-up studies, statistics, outcome andprocess assessment not using the explosion facility.None of these search strategies resulted in anyadditional studies being identified. Strategy 5 wastherefore used as a basis from which to developstrategies to use in other databases withamendments as appropriate regarding thesaurusterms and subject indexing. The MEDLINE searchwas run from 1966 to March 1999. It was updatedto include records published by February 2001.
Health Technology Assessment 2005; Vol. 9: No. 26
113
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
PsycLITThe PsycLIT database was searched viaSilverPlatter (1887 to March 1999) and 287records were retrieved. The search was updated toinclude records published up to February 2001,identifying a further 40 records. The searchstrategy used was:
1 randomized controlled trials2 randomi* control* trial*3 control* clinical trial*4 clinical trial*5 trial*6 exact{EMPIRICAL-STUDY} in PT7 1 or 2 or 3 or 4 or 5 or 68 multiple arm study9 multiple arms study
10 multiple arms studies11 multiple arms studies12 multiple arm13 multiple arms14 multi arm15 multi arms16 multi near2 arm17 multi near2 arms18 multiple near2 arm19 multiple near2 arms20 three arm or three arms or 3 arm or 3 arms or
three limb or three limbs or 3 limb or 3 limbs21 four arm or four arms or 4 arm or 4 arms or
four limb or four limbs or 4 limb or 4 limbs22 multi* near3 (study or studies)23 8 or 9 or 10 or 11or 12 or 13 or 14 or 15 or
16 or 17 or 18 or 19 or 20 or 21 or 2224 7 or 2325 explode META-ANALYSIS26 meta-analy*27 metaanaly*28 meta near analy*29 25 or 26 or 27 or 2830 24 and 2931 explode EXPERIMENTAL-DESIGN32 explode STATISTICAL-ANALYSIS33 indirect near2 comparison*34 direct near2 comparison*35 indirect near2 evaluat*36 direct near2 evaluat*37 compet* near2 technolog*38 compet* near2 intervention*
39 treatment near2 arm*40 treatment near2 group*41 randomi* near2 group*42 randomi* near2 comparison*43 therapeutic near2 arm*44 study near2 arm*45 trial near2 arm*46 trial near2 design*47 placebo near2 arm*48 preventive near2 arm*49 preventative near2 arm*50 competing near2 therap*51 indirect near evaluat*52 31 or 32 or 33 or 34 or 35 or 36 or 37 or 38
or 39 or 40 or 41 or 42 or 43 or 44 or 45 or46 or 47 or 48 or 49 or 50 or 51
53 30 and 52
ERICThe ERIC database was searched using the Ovidinterface via BIDS (1966 to February 1999) and 72records were retrieved. The search strategy usedwas:
1 randomized controlled trial.tw.2 randomi#ed control? trial?.tw.3 control? clinical trial?.tw.4 clinical trial?.tw.5 trial?.tw.6 exp MATCHED GROUPS/7 exp EXPERIMENTAL GROUPS/8 1 or 2 or 4 or 5 or 6 or 79 multiple arm study.tw.
10 multiple arms study.tw.11 multiple arm studies.tw.12 multiple arms studies.tw.13 multiple arm.tw.14 multiple arms.tw.15 multi arm.tw.16 multi arms.tw.17 (multi adj2 arm).tw.18 (multi adj2 arms).tw.19 (multiple adj2 arm).tw.20 (multiple adj2 arms).tw.21 (three arm or three rms or 3 arm or 3 arms or
three limb or three limbs or 3 limb).mp. or 3 limbs.tw. [mp=abstract, title, heading word,identifiers]
Health Technology Assessment 2005; Vol. 9: No. 26
115
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 6
Additional electronic search strategies
22 (three arm or three arms or 3 arm or 3 armsor three limb or three limbs or 3 limb).mp. or3 limbs.tw. [mp=abstract, title, heading word,identifiers]
23 (four arm or four arms or 4 arm or 4 arms orfour limb or four limbs or 4 limb).mp. or 4limbs.tw. [mp=abstract, title, heading word,identifiers]
24 (multi? adj3 (study or studies)).tw.25 8 or 23 or 2426 meta-analysis.tw.27 (meta adj analysis).tw.28 metaanalysis.tw.29 meta-analytic.tw. 30 (meta adj analytic).tw.31 metaanalytic.tw.32 exp META ANALYSIS/33 exp LITERATURE REVIEWS 34 26 or 27 or 28 or 29 or 30 or 31 or 32 or 3335 25 and 34
EMBASEThe EMBASE database was searched using theSilverplatter interface (1980 to April 1999) and282 records were retrieved. The search strategyused was:
1 explode RANDOMIZED-CONTROLLED-TRIAL/ all subheadings
2 explode CONTROLLED-STUDY/ allsubheadings
3 explode CLINICAL-TRIAL/ all subheadings 4 trial*5 clinical trials 6 1 or 2 or 3 or 4 or 57 multiple arm study 8 multiple arms study 9 multiple arm studies
10 multiple arms studies 11 multiple arm 12 multiple arms 13 multi arm 14 multi arms 15 multi near2 arm 16 multi near2 arms 17 multiple near2 arm 18 multiple near2 arms 19 three arm or three arms or 3 arm or 3 arms or
three limb or three limbs or 3 limb or 3 limbs 20 #19 near5 (trial* or stud* or random*) 21 four arm or four arms or 4 arm or 4 arms or
four limb or four limbs or 4 limb or 4 limbs 22 #21 near5 (trial* or stud* or random*) 23 multi* near3 (study or studies) 24 #7 or #8 or #9 or #10 or #11 or #12 or #13
or #14 or #15 or #16 or #17 or #18 or #20or #22 or #23
25 #6 or #24 26 explode META-ANALYSIS/ all subheadings 27 meta-analysis 28 metaanalys* 29 meta near2 analys* 30 #26 or #27 or #28 or #29 31 #25 and #30 32 indirect near2 comparison* 33 direct near2 comparison* 34 indirect near2 evaluat* 35 direct near2 evaluat* 36 compet* near2 technolog* 37 compet* near2 intervention* 38 treatment near2 arm* 39 treatment near2 group* 40 randomi* near2 group* 41 randomi* near2 comparison* 42 therapeutic near2 arm 43 therapeutic near2 arms 44 study near2 arm 45 study near2 arms 46 trial near2 arm 47 trial near2 design 48 placebo near2 arm 49 preventive near2 arm 50 preventative near2 arm 51 competing near2 therap* 52 #32 or #33 or #34 or #35 or #36 or #37 or
#38 or #39 or #40 or #41 or #42 or #43 or#44 or #45 or #46 or #47 or #48 or #49 or#50 or #51
53 #31 and #52
MathSci The MathSci database was searched using Dialog(up to September 1999) and 1817 records were retrieved. The search strategyused was:
1 s research(W)design2 s statist?(W)models3 s indirect(2W)comparison?4 s direct(2W)comparison?5 s indirect(2W)evaluat?6 s direct(2W)evaluat?7 s compet?(2W)technolog?8 s compet?(2W)intervention?9 s treatment(2W)arm?
10 s treatment(2W)group?11 s randomi?(2W)group?12 s randomi?(2W)comparison?13 s therapeutic(2W)arm14 s therapeutic(2W)arms
Appendix 6
116
15 s study(2W)arm16 s study(2W)arms17 s 4(W)limb(W)study18 s four(W)limb(W)study19 s trial(2W)arm20 s trial(2W)design21 s placebo(2W)arm22 s preventive(2W)arm23 s preventative(2W)arm24 s multiple(W)arm(W)study25 s multiple(W)arms(W)study26 s multiple(W)arm(W)studies27 s multiple(W)arms(W)studies28 s multiple(W)arm29 s multiple(W)arms30 s multi(W)arm
31 s multi(W)arms32 s multi(2W)arm33 s multi(2W)arms34 s multiple(2W)arm35 s multiple(2W)arms36 s ((three(W)arm) or (three(W)arms) or
(3(W)arm) or (3(W)arms) or (three(W)limb) or(three(W)limbs) or (3(W) limb) or (3(W)limbs))(5W) (trial? or stud? or random?)
37 s ((four(W)arm) or (four(W)arms) or (4(W)arm)or (4(W)arms) or (four(W)limb) or(four(W)limbs) or (4(W)limb) or (4(W)limbs))(5W) (trial? or stud? or random?)
38 s competing(2W)therap?39 s multitreatment(2W)(study or studies)40 s s1:s39
Health Technology Assessment 2005; Vol. 9: No. 26
117
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Data setSeveral of the possible analyses are illustratedusing a set of trials of antithrombotic therapy toprevent strokes in patients with atrial fibrillation150
(Table 23). The emphasis is on methods that canbe applied using widely available software. Asthese trials are used for illustration only theimpact of certain criticisms of the systematicreview has been disregarded.151
From this set of data subsets of trials are taken toillustrate:
� an indirect comparison of BvC using A as acommon control (studies 3, 4 and 5 are two-armed comparisons of AvB, studies; 7, 8, 9 and10 are two-armed comparisons of AvC)
� a combination of the above indirect comparisonwith direct two-armed trials to estimate BvC(trial 11 is a two-armed trial of BvC, which iscombined with the indirect comparison usingthe seven trials listed above)
� a combination of the above indirect and directcomparisons with multiarmed trials of A, B andC (trials 1, 2 and 6 combined with the eighttrials listed above)
� a combination of all 15 trials and all treatments.
These subsets of trials are created for the purposeof illustrating sequentially more complexalternative models, and the results should not beregarded as definitive analyses of the above dataset.
AnalysesAnalyses were undertaken using Stata version 8 foradjusted indirect comparisons and logisticregression, and using the PROC NLMIXEDprocedure in SAS version 9.1 for the mixed models.The SAS procedure is chosen for presentation inpreference over the Stata gllamm command(which can also be used for fitting mixed models),for reasons of computational speed.
Health Technology Assessment 2005; Vol. 9: No. 26
119
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 7
Illustrative analyses
TABLE 23 Event rates for stroke in trials of antithrombotic therapy for patients with atrial fibrillation (from Hart et al.150)
Trial Placebo Adjusted- Aspirin Low- or fixed- Low- or fixed-dose warfarin dose warfarin dose warfarin
+ aspirinA B C D E
1 AFASAK 19/336 9/335 16/3362 SPAF 19/211 8/210 25/5523 BAATAF 13/208 3/2124 CAFA 9/191 6/1875 SPINAF 23/290 7/2816 EAFT ia 50/214 20/225 49/230a
7 EAFT iia 40/164 39/174a
8 ESPS II 23/107 17/1049 LASAF 6/182 5/194b
10 UK-TIA 8/30 8/34b
11 SPAF II 19/358 21/35712 AFASAK II 11/170 9/169 14/167 11/17113 PATAF 3/131 4/141 4/12214 SPAF III 14/523 48/52115 MWNAF 1/153 5/150
a EAFT152 included two clinical subgroups randomised to different treatment options according to eligibility foranticoagulation. The results for aspirin were combined in the publication (88/404); here, the data have been split bymaking the event rates equal.
b Combination of two groups randomised to different doses of aspirin.
Format of data setsFour different formats are required for alternativeanalyses:
(a) One row per trial (see Table 24)Each row states for treatment and control groupsthe numbers of events (event_t, event_c) and thenumber of participants (sample_t, sample_c).
This data structure is only suitable for two-armedtrials. For adjusted indirect comparisons (AICs)meta-analyses are undertaken on different subsetsof the data for each component meta-analysisindicated by an additional variable [here a variablecompb is used, indicating whether A wascompared with B (value 1) or C (value 0)]. Here,the ‘A’ arm of the trials acts as the commoncomparator and is called ‘control’.
(b) Two (or more) rows per trial, oneper trial arm (see Table 25)
Each row states the number of events (event) andparticipants (sample) and study arm (arm). Thisdata structure is the standard format for fittinglogistic and mixed models.
In Stata a data set of format (a) can be changedinto format (b) using the following commands:
rename event_c event0
rename event_t event1
rename sample_c sample0
rename sample_t sample1
reshape long event sample, i(trial)
j(treat)
generate arm=”C”
replace arm=”A” if treat==1 & compb==0
replace arm=”B” if treat==1 & compb==1
(c) Four (or more) rows per trial, tworows per trial arm (see Table 26)
Each row states the number in each outcomecategory (n) and indicator variables for whether
they suffered a stroke or not (stroke) and studyarm (arm). This data structure is only required forthe models fitted in the Stata mixed modelscommand gllamm.
In Stata a data set of format (b) can be changedinto format (c) using the following commands:
rename event n1
gene n0=sample-n1
reshape long event sample, i(trial arm)
j(stroke)
Appendix 7
120
TABLE 24 Example data set for indirect comparisons analysis
ID Trial event_c sample_c event_t sample_t compb
1 BAATAF 3 212 13 208 12 CAFA 6 187 9 191 13 SPINAF 7 281 23 290 14 EAFT ii 39 174 40 164 05 ESPS II 17 104 23 107 06 LASAF 5 194 6 182 07 UK-TIA 8 34 8 30 0
TABLE 25 Example data set for full analysis
ID Trial Event Sample Arm
1 AFASAK 19 336 A2 AFASAK 9 355 B3 AFASAK 16 336 C4 SPAK 19 211 A5 SPAK 8 210 B
… … … … …35 MWNAF 1 153 A36 MWNAF 5 150 D
TABLE 26 Example data set for full analysis in gllamm
ID Trial n Stroke Arm
1 AFASAK 19 1 A2 AFASAK 317 0 A3 AFASAK 9 1 B4 AFASAK 346 0 B5 AFASAK 16 1 C6 AFASAK 320 0 C7 SPAK 19 1 A8 SPAK 192 0 A9 SPAK 8 1 B
10 SPAK 202 0 B… … … …69 MWNAF 1 1 A70 MWNAF 152 0 A71 MWNAF 5 1 D72 MWNAF 145 0 D
(d) One row per participant, n rows pertrial (see Table 27)
Each row states whether the participant suffered astroke or not (stroke) and their study arm (arm).This data structure is only required for the modelsfitted in the Stata xtlogit.
In Stata a data set of format (c) can be changedinto format (d) using the following commands:
expand n
drop n
Indirect comparisonsNaive methodThe naive analysis is based on summing the eventsand participants in the B and C arms of the sevencomponent trials, and computing an odds ratioand confidence interval as if the data had arisen ina single study.
Across the three B trial arms 16 out of 680participants experienced a stroke. Across the fourC trial arms 69 out of 506 participantsexperienced a stroke. This comparison yields anodds ratio of 0.15 (95% CI 0.09 to 0.27), and ishighly statistically significant (z = –6.61,p < 0.00001).
Adjusted indirect comparisons andmeta-regressionAdjusted indirect comparisons are undertaken byperforming separate meta-analyses on the datasets of trials of AvB and AvC. One approach is touse the Stata meta-analysis command metan103 foreach meta-analysis (lines 5 and 9), and store thevalues of the log odds ratio (lines 6 and 10) andstandard error (lines 7 and 11) from each separatemeta-analysis. The odds ratio for indirectcomparison is estimated as the exponential of the
difference in log odds ratios from the two meta-analyses (lines 13 and 14). The standard error ofthe indirect comparison is estimated from thesquare root of the sum of the squared standarderrors (line 15).
The code below assumes that the data are informat (a):
* STATA CODE for indirect comparison by
adjusted indirect comparison
1 * compute values of the four cells for
each trial
2 gene noevent_t=sample_t-event_t
3 gene noevent_c=sample_c-event_c
4 * meta-analysis of trials of BvA (combp
equal to 1)
5 metan event_t noevent_t event_c
noevent_c if compb==1, or fixedi
nograph
6 local logor1=log($S_1)
7 local se1=$S_2
8 * meta-analysis of trials of CvA (combp
equal to 0)
9 metan event_t noevent_t event_c
noevent_c if compb==0, or fixedi
nograph
10 local logor2=log($S_1)
11 local se2=$S_2
12 * computation of log OR, OR and se for
indirect comparison
13 local logor_aic=`logor1'-`logor2'
14 local or_aic=exp(`logor_aic')
15 local se_aic=sqrt(`se1'^2+`se2'^2)
16 * computation of confidence intervals,
z-value and P-value
17 local ll_aic=exp(`logor_aic'-
(1.96*`se_aic'))
18 local
ul_aic=exp(`logor_aic'+(1.96*`se_aic')
)
19 local z_aic=`logor_aic'/`se_aic'
20 if `z_aic'>0 local p_aic=2*(1-
norm(`z_aic'))
21 if `z_aic'<=0 local
p_aic=2*norm(`z_aic')
The above analysis produces inverse varianceestimates. Alternative meta-analysis models areobtained by changing the inverse variance fixedeffect option (fixedi) on the metan command lines(lines 5 and 9) to indicate Mantel–Haenszel fixedeffects (fixed) or DerSimonian and Laird random
Health Technology Assessment 2005; Vol. 9: No. 26
121
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TABLE 27 Example data set for full analysis in xtlogit
ID Trial Stroke Arm
1 AFASAK 1 A2 AFASAK 1 A3 AFASAK 1 A
… … …19 AFASAK 1 A20 AFASAK 0 A21 AFASAK 0 A
… … …8139 MWNAF 0 D8140 MWNAF 0 D
effects (randomi) models. Risk ratio models can beobtained by changing the summary statistic fromodds ratio (or) to the risk ratio (rr). Riskdifference estimates are produced using the riskdifference (rd) option. If risk differences arepooled, use of a logarithmic scale is not required,requiring alterations to lines 6, 10, 14, 17 and 18of the above code. Continuous outcomes can alsobe pooled using the metan command.
Meta-regression can also be used to estimate thedifference between the groups. To undertakemeta-regression, estimates of the log odds ratioand standard error are computed for each study.Using the values stored by metan provides a short-cut for doing this (lines 5–7). A weightedregression model is then fitted with the log oddsratio as the outcome and the comparison (compb)as the predictor. The model weights each studyaccording to the inverse variance (line 8). A simpleapproach to fit the model involves weighted linearregression (line 10). A more appropriate method isto use random effects meta-regression (line 12)that allows for unexplained variability betweengroups, as provided by the metareg command.78
* STATA CODE for indirect comparison
by meta-regression
1 * compute values of the four cells for
each trial
2 gene noevent_t=sample_t-event_t
3 gene noevent_c=sample_c-event_c
4 * meta-analysis of all trials to
compute logOR and se
5 metan event_t noevent_t event_c
noevent_c, or fixedi nograph notable
6 local logor=log($S_1)
7 local se=$S_2
8 local wt=1/`se’^2
9 * weighted linear regression
10 regress logor compb [aw=wt]
11 * random effects meta-regression
12 metareg logor compb, wsse(se)
The three AIC models (inverse variance,Mantel–Haeszel, and DerSimonian and Laird) andtwo regression models (fixed and random) givesimilar estimates of the indirect comparison(Table 28), but all are very different from theflawed naive method.
The AIC DerSimonian and Laird and randomeffects meta-regression use different approaches to estimate the unexplained between studyvariation, �2. The DerSimonian and Laird method estimates separate �2 values forcomparisons of AvC (�2 = 0) and AvB (�2 = 0.023).The meta-regression model estimates a single �2 todescribe the residual heterogeneity among alltrials having accounted for the difference inestimates between trials comparing AvB and trialscomparing AvC (�2 = 0). The meta-regressionapproach is likely to be more efficient and preciseas it involves one fewer parameter and estimates �2
using data from all trials. The alternativeDerSimonian and Laird approach may be moreappropriate when there are reasons to assumedifferent �2 values in the two component analyses.
Generalised linear modelsA logistic regression model can be fitted to data informat (b). The model includes two zero–oneindicator variables, armA and armB, to indicatestudy arm (lines 1–5). Arm C is designated thebaseline category with which comparisons aremade. An estimate of the indirect comparison isobtained from the parameter estimate for armB(comparison of B with baseline C). To obtain ananalysis that is stratified by trial, indicatorvariables for trial are included to allow each trialto have a different control group risk (lines 7–8).In Stata the command blogit is used to fit alogistic regression model indicating the number ofoutcomes (event), the number of participants(people) and the dependent variables (armC,armA and trial):
* STATA CODE for indirect comparison by
logistic regression
Appendix 7
122
TABLE 28
Approach Method OR 95% CI z/t-Value p-Value
AIC Inverse variance 0.43 (0.22 to 0.87) z = –2.36 0.0184AIC Mantel–Haenszel 0.42 (0.21 to 0.83) z = –2.48 0.0131AIC DerSimonian and Laird 0.43 (0.21 to 0.89) z = –2.28 0.0225Meta-regression Weighted linear regression 0.43 (0.23 to 0.82) t = –3.37 0.0198Meta-regression Random effects meta-regression 0.43 (0.22 to 0.87) z = –2.36 0.0184
1 * generate indicator variables for armA
and armB
2 gene armA=0
3 gene armB =0
4 replace armA=1 if arm=="A"
5 replace armB=1 if arm=="B"
6 * fit a logistic regression model
7 * xi: automatically adds indicator
variables for trial
8 xi:blogit event people armC armA
i.trial, or
Alternatively, the Stata command xtlogit can beused for data in format (d). Again indicatorvariables are generated for study arms, which takevalues of 0 or –1 (lines 1–5). The i(trial) optioncombined with the fixed effect option fe (line 7)performs an analysis stratified by trial thatproduces the same results as the above logisticregression model.
The xtlogit command includes an option toperform random effects analysis, using the reoption (line 9). This analysis replaces the trialindicator variables with a distribution of effects(making an assumption that the logit controlgroup risks are a random sample from a normaldistribution). It does not, as is usually desired in ameta-analytical random effects analysis, place therandom effect on the treatment contrasts (seeChapter 4, section ‘Classical methods usingaggregate data’, p. 19).
* STATA CODE for indirect comparisonusing xtlogit1 * generate indicator variables forarmA and armB
2 gene armA=03 gene armB =04 replace armA=-1 if arm=="A"5 replace armB=-1 if arm=="B"
6 * fit a fixed effect logisticregression model
7 xtlogit stroke armC armA, i(trial)fe or
8 * fit a random effects logisticregression model
9 xtlogit stroke armC armA, i(trial)re or
Two standard software packages can be used forestimating a random effect for the treatmentcontrast for binomial data: SAS PROC NLMIXEDand the gllamm command in Stata. The use of the
SAS option is demonstrated here; this usesadaptive quadrature to identify maximumlikelihood solutions, a more advanced and fastermethod than used by gllamm. WinBugs software,81
using a fully Bayesian model specification, providesthe greatest flexibility for fitting these models.
Two alternative models can be fitted using data informat (b) with indicator variables armA and armBindicating the treatment. In the first, trial effectsare fitted as random (using the term trialr in lines4 and 7) and a random effect for treatmentcontrasts is included assumed constant acrosstreatment comparisons (using the term het in lines4 and 7). The procedure requires appropriatestarting values to be stated (line 3), which may beobtained from a fixed effect analysis.
* SAS code for indirect comparison using
PROC NLMIXED
1 * random effects for trial and
treatment contrasts
2 proc nlmixed data=indirect;
3 parms base=-2.3 a=0.2 b=-1
s2trialr=0.7 s2het=0.1;
4 logitp= (base+trialr)+ armA*(a+het) +
armB*(b+het);
5 p = exp(logitp)/(1+exp(logitp));
6 model stroke ~ binomial(n,p);
7 random trialr het ~
normal([0,0],[s2trialr,0,s2het])
subject=trial;
8 run;
This analysis estimates the trial effect variance tobe 0.68, and the random effect for treatmentcontrasts to be 0.09. If desired, separate randomeffects can be estimated for the AvC and BvCtreatment contrasts by specifying two separateheterogeneity parameters, although there arelikely to be estimation problems unless there aremany trials. This model requires the assumptionthat the logit baseline event rates are randomlysampled from a normal distribution. Thatassumption can be avoided by estimating a fixedeffect for each trial while including a randomeffect for treatment contrast. This model requiresestimation of many more parameters (additionalparameters t3–t10 are estimated for each trial inlines 7 and 8) and is less likely to produce a stablesolution. Again separate heterogeneity parameterscould be included for each treatment contrast.
* SAS code for indirect comparison
using PROC NLMIXED
1 * fixed effect for trial, random
effects for treatment contrasts
Health Technology Assessment 2005; Vol. 9: No. 26
123
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
2 proc nlmixed data=indirect;
3 parms a=0.2 b=-0.9
4 t3=-3 t4=-3 t5=-2.7 t7=-1.3
t8=-1.6 t9=-3.6 t10=-1.2
5 s2het=1;
6 logitp=a*(armA+het) + b*(armB+het)
+
7 t3*trial3 + t4*trial4 +
t5*trial5 +
8 t7*trial7 + t8*trial8 + t9*trial9
+ t10*trial10;
9 p = exp(logitp)/(1+exp(logitp));
10 model stroke ~ binomial(n,p);
11 random het ~ normal([0],[s2het])
subject=trial;
12 run;
In the application to the example data set, theestimate of among-study heterogeneity is close tozero, but the estimates of the treatment contrastsare sensitive to the choice of starting value for theheterogeneity statistic (Table 29).
Combining indirect and directcomparisonsTrial 11 (SPAF II) is a two-armed trial of BvC. Theestimated treatment effect in this trial is OR=0.90(95% CI 0.45 to 1.79) (p=0.74). This section looksat analyses that combine this trial with the sevenindirect trials.
Combining with adjusted indirectcomparisonsA weighted combination of the results from theAIC and the direct comparison is computed as aninverse variance weighted average.
For the direct trial:
lnOR = ln [(19/339) / (21/336)] = ln(0.90) = –0.1090Var(lnOR) = 1/19 + 1/339 + 1/21 + 1/336 =0.1062
For the indirect comparison (from the inversevariance solution)
lnOR = –0.8358
Var(lnOR) = 0.1257
Inverse variance weights for the indirect and directcomparisons are 7.96 and 9.42, respectively. Theweighted average and variance of the combinationare thus:
lnOR = [(9.42 � –0.1090) + (7.96 � –0.8358)] / (7.96
+ 9.42) = –0.4456
Var(lnOR) = 1 / (7.96 + 9.42) = 0.0576
giving an overall estimate of OR = 0.64 (95% CI0.40 to 1.02) (p = 0.06).
It is also possible to test whether there is asignificant difference in the finding of the AIC andthe direct comparison, by dividing the difference inthe logOR (–0.1090 – –0.8358 = 0.7268) by thestandard error of the difference [√(0.1062+0.1257)= 0.4816], and comparing the resulting number(0.7268/0.4816 = 1.51) with a standard normaldistribution (z) to obtain a p-value (p = 0.13).
It is possible to undertake the analyses combiningany of the estimates using AICs or meta-regressionmodels with the results of the direct trial. If therewere more than one direct two-armed trial, resultsof a meta-analysis of the direct trials could becombined with the AIC.
Appendix 7
124
TABLE 29
Method Random effects OR 95% CI z/t-Value p-Value
Logistic regression None 0.42 (0.21 to 0.83) z = –2.49 0.0129(blogit, xtlogit fe)
Logistic regression Trials 0.41 (0.22 to 0.78) z = –2.71 0.007(xtlogit re)
Mixed model Trials 0.38 (0.16 to 0.94)a t = –2.76 0.03(NLMIXED)
Mixed model Trials and treatments 0.38 (0.16 to 0.94)a t = –2.76 0.04(NLMIXED)
Mixed model Treatments Unstable estimates(NLMIXED) (variance estimate is very small)
a Treatment contrast calculated using t statistic.
Generalised linear modelsThe models described above can all incorporatedata from the additional direct trial withoutchanging the code other than inclusion of anadditional indicator variable for the extra trial.The results for these models are given in Table 30.
Including three-armed AvBvC trialsTrials with more than two arms cannot be used inAICs without discarding some treatment arms.However, they can be included naturally in basicgeneralised linear models, again withoutadjustment to the code beyond addition ofindicator variables for the additional trials. There are challenges in appropriately modellingrandom effects for multiarmed trials, as outlinedby Higgins and Whitehead.53 Appropriatemodelling of random effects involves realising a different random effect for each comparison.
SAS PROC NLMIXED assumes that allcomparisons from the same trial involve the samerealisation of the treatment comparison randomeffect, which is not ideal. Correct modelling canbe undertaken using a fully Bayesian model inWinBUGS.81
Analysis of all trials and alltreatmentsAICs can be combined to include comparisonswith alternative control groups only if all trials aretwo-armed trials. Here this is not the case, andincorporation of data from trials 12–15 can onlybe done using generalised linear models. In thesesituations additional indicator variables need to beadded to the model for treatments D and E, aswell as for the additional trials. The same issuesarise in the correct modelling of random effects asdiscussed in the previous section.
Health Technology Assessment 2005; Vol. 9: No. 26
125
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TABLE 30
Approach Method OR 95% CI z/t-Value p-Value
AIC Inverse variance 0.64 (0.40 to 1.02) z = –1.86 0.0632AIC Mantel–Haenszel 0.63 (0.40 to 1.01) z = –1.94 0.0530AIC DerSimonian and Laird 0.65 (0.40 to 1.04) z = –1.78 0.0747Meta-regression Weighted linear regression 0.56 (0.38 to 0.83) t = –2.91 0.0036Meta-regression Random effects meta-regression 0.64 (0.40 to 1.02) z = –1.86 0.0632Logistic regression Fixed effect 0.62 (0.39 to 0.99) z = –2.00 0.0460Logistic regression Random effect for trials 0.59 (0.37 to 0.93) z = –2.28 0.0230Mixed model Random effect for trials 0.59 (0.34 to 1.03) t = –2.24 0.0604Mixed model Random effect for trials and treatment 0.59 (0.33 to 1.05) t = –2.24 0.0667Mixed model Random effect for treatment Unstable estimates
(variance estimate is very small)
TABLE 31
Approach Method OR 95% CI z/t-Value p-Value
Logistic regression Random effect for trials 0.57 (0.43 to 0.75) z = –3.99 0.0001Mixed model Random effect for trials 0.53 (0.38 to 0.75) t = –4.09 0.0022Mixed model Random effect for trials and treatment 0.53 (0.37 to 0.75) t = –4.09 0.0027Mixed model Random effect for treatment Unstable estimates
(variance estimate is very small)
TABLE 32
Approach Method OR 95% CI z/t-Value p-Value
Logistic regression Random effect for trials 0.56 (0.43 to 0.73) z = –4.37 <0.0001Mixed model Random effect for trials 0.56 (0.41 to 0.75) t = –4.15 0.001Mixed model Random effect for trials and treatment 0.55 (0.41 to 0.75) t = –4.15 0.001Mixed model Random effect for treatment Unstable estimates
(variance estimate is very small)
Health Technology Assessment 2005; Vol. 9: No. 26
127
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Appendix 8
Identified meta-analyses providing sufficient data for both direct and indirect comparisons
Appendix 8
128 TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
1 ATC
-I17
Patie
nts
with
an
incr
ease
d ris
k of
occ
lusiv
eva
scul
ar d
iseas
e (e
.g. p
rior
or a
cute
CH
D,
stro
ke, p
erip
hera
l vas
cula
r di
seas
e)
(Vas
cula
r ev
ents
)
Hig
h-do
se a
spiri
n vs
med
ium
-dos
e as
pirin
:3
(121
2/12
13)
Hig
h-do
se a
spiri
n vs
con
trol
: 30
(13,
667/
11,5
65)
Med
ium
-dos
e as
pirin
vs
cont
rol:
19 (2
5,37
6/25
,417
)
RR Dire
ct e
stim
ate:
0.
96 (0
.81
to 1
.15)
Adj
uste
d in
dire
ct:
1.08
(0.9
4 to
1.2
4)N
aive
indi
rect
: 1.
64 (1
.54
to 1
.74)
2 ATC
-I17
Sam
e as
abo
veA
spiri
n+di
pyrid
amol
e vs
asp
irin:
16
(282
9/28
40)
Asp
irin+
dipy
ridam
ole
vs c
ontr
ol:
23 (4
757/
4694
)A
spiri
n vs
con
trol
: 42
(40,
013/
37,5
38)
RR Dire
ct e
stim
ate:
1.
01 (0
.87
to 1
.16)
Adj
uste
d in
dire
ct:
0.91
(0.8
0 to
1.0
5)N
aive
indi
rect
: 1.
12 (1
.03
to 1
.22)
3 ATC
-I17
Sam
e as
abo
veSu
lfinp
yraz
one
vs a
spiri
n:4
(507
/656
)Su
lfinp
yraz
one
vs c
ontr
ol:
17 (2
108/
2135
) A
spiri
n vs
con
trol
:49
(41,
656/
38,7
99)
RR Dire
ct e
stim
ate:
1.
17 (0
.88
to 1
.54)
Adj
uste
d in
dire
ct:
1.02
(0.8
7 to
1.2
0)N
aive
indi
rect
: 1.
32 (1
.17
to 1
.47)
4 ATC
-I17
Sam
e as
abo
veT
iclo
pidi
ne v
s as
pirin
: 3
(173
0/17
41)
Tic
lopi
dine
vs
cont
rol:
27 (2
936/
2955
) A
spiri
n vs
con
trol
: 52
(42,
248/
39,2
30)
RR Dire
ct e
stim
ate:
0.
71 (0
.38
to 1
.34)
Adj
uste
d in
dire
ct:
0.90
(0.7
8 to
1.0
4)N
aive
indi
rect
: 0.
91 (0
.81
to 1
.03)
5 ATC
-II16
Patie
nts
with
an
incr
ease
d ris
k of
vas
cula
roc
clus
ion
(e.g
. cor
onar
y or
leg
arte
ryby
pass
gra
fting
or
angi
opla
sty)
(Vas
cula
r oc
clus
ion)
Hig
h-do
se v
s m
ediu
m-d
ose
aspi
rin:
1 (1
55/1
54)
Hig
h-do
se a
spiri
n vs
con
trol
: 8
(744
/753
)M
ediu
m-d
ose
aspi
rin v
s co
ntro
l: 4
(489
/496
)
RR Dire
ct e
stim
ate:
1.
15 (0
.76
to 1
.74)
Adj
uste
d in
dire
ct:
0.93
(0.5
8 to
1.4
8)N
aive
indi
rect
: 0.
59 (0
.46
to 0
.77)
6 ATC
-II16
Sam
e as
abo
veA
spiri
n+di
pyrid
amol
e vs
asp
irin:
10 (1
264/
1267
)A
spiri
n+di
pyrid
amol
e vs
con
trol
:14
(137
1/13
03)
Asp
irin
vs c
ontr
ol:
7 (5
97/6
10)
RR Dire
ct e
stim
ate:
1.
03 (0
.84
to 1
.27)
Adj
uste
d in
dire
ct:
1.26
(0.8
5 to
1.8
6)N
aive
indi
rect
: 1.
53 (1
.21
to 1
.92)
7 ATC
-II16
Sam
e as
abo
veSu
lfinp
yraz
one
vs a
spiri
n:
2 (1
67/3
26)
Sulfi
npyr
azon
e vs
con
trol
: 5
(276
/285
) A
spiri
n vs
con
trol
: 12
(123
3/12
49)
RR
Dire
ct e
stim
ate:
1.
01 (0
.71
to 1
.45)
Adj
uste
d in
dire
ct:
1.15
(0.7
3 to
1.8
0)N
aive
indi
rect
: 0.
81 (0
.58
to 1
.13)
Health Technology Assessment 2005; Vol. 9: No. 26
129
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
8 ATC
-II16
Sam
e as
abo
veT
iclo
pidi
ne v
s as
pirin
: 2
(41/
41)
Tic
lopi
dine
vs
cont
rol:
12 (5
46/5
42)
Asp
irin
vs c
ontr
ol:
13 (1
542/
1402
)
RR Dire
ct e
stim
ate:
1.
16 (0
.43
to 3
.16)
Adj
uste
d in
dire
ct:
1.12
(0.8
0 to
1.5
6)N
aive
Indi
rect
: 0.
75 (0
.59
to 0
.96)
9 ATC
-II16
Sam
e as
abo
veA
spiri
n+di
pyrid
amol
e vs
sul
finpy
razo
ne:
1(16
2/14
8)A
spiri
n+di
pyrid
amol
e vs
con
trol
: 19
(200
4/19
42)
Sulfi
npyr
azon
e vs
con
trol
:5
(276
/285
)
RR
Dire
ct e
stim
ate:
0.
94 (0
.62
to 1
.43)
Adj
uste
d in
dire
ct:
1.06
(0.6
9 to
1.6
5)N
aive
indi
rect
: 1.
54 (1
.13
to 2
.12)
10 ATC
-III18
Surg
ical
and
hig
h-ris
k m
edic
al p
atie
nts
(DVT
)
Asp
irin+
dipy
ridam
ole
vs a
spiri
n:
9 (2
63/2
18)
Asp
irin+
dipy
ridam
ole
vs c
ontr
ol:
11 (3
94/4
22)
Asp
irin
vs c
ontr
ol:
9 (6
49/5
97)
RR Dire
ct e
stim
ate:
0.
67 (0
.51
to 0
.89)
Adj
uste
d in
dire
ct:
0.77
(0.4
4 to
1.3
3)N
aive
indi
rect
: 0.
94 (0
.73
to 1
.20)
11 Buch
er e
t al
.14
HIV
-infe
cted
pat
ient
s
(Pne
umoc
ystis
car
inii
pneu
mon
ia)
TM
P+SM
X v
s D
+P
or D
: 8
(803
/815
)T
MP+
SMX
vs
AP:
9
(681
/613
)D
+P
or D
vs
AP:
5
(732
/718
)
RR Dire
ct e
stim
ate:
0.
45 (0
.22
to 0
.91)
Adj
uste
d in
dire
ct:
0.43
(0.2
5 to
0.7
5)N
aive
indi
rect
: 0.
55 (0
.35
to 0
.87)
12 Che
ng e
t al
.137
Wom
en a
tten
ding
ser
vice
s fo
r em
erge
ncy
cont
race
ptio
n
(No.
of p
regn
anci
es)
Levo
norg
estr
el v
s m
ifepr
iston
e:
1 (6
43/6
33)
Levo
norg
estr
el v
s yu
zpe:
2
(138
6/14
21)
Mife
prist
one
vs y
uzpe
: 2
(597
/589
)
RR Dire
ct e
stim
ate:
2.
19 (1
.00
to 4
.77)
Adj
uste
d in
dire
ct:
5.40
(0.5
3 to
54.
82)
Nai
ve in
dire
ct:
19.8
3 (1
.21
to 3
26.0
3)
13 Chi
ba e
t al
.40
Patie
nts
with
GO
RD
(Hea
ling
rate
)
H2R
A v
s PP
I: 13
(731
/884
)H
2RA
vs
plac
ebo:
11
(181
7/95
9)PP
I vs
plac
ebo:
2
(334
/75)
RR Dire
ct e
stim
ate:
0.
56 (0
.48
to 0
.66)
Adj
uste
d in
dire
ct:
0.26
(0.1
4 to
0.4
8)N
aive
indi
rect
: 0.
73 (0
.68
to 0
.78)
14 Col
lins
et a
l.138
Patie
nts
with
pos
tope
rativ
e pa
in
(No.
of p
atie
nts
with
>50
% p
ain
relie
f)
Ibup
rofe
n 40
0 m
g vs
200
mg:
5 (1
99/2
02)
Ibup
rofe
n 40
0 m
g vs
con
trol
: 26
(140
7/10
43)
Ibup
rofe
n 20
0 m
g vs
con
trol
: 3
(204
/133
)
RR Dire
ct e
stim
ate:
1.
39 (1
.08
to 1
.79)
Adj
uste
d in
dire
ct:
0.74
(0.2
7 to
2.0
2)N
aive
indi
rect
: 1.
16 (1
.00
to 1
.34)
Appendix 8
130 TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
15 Del
aney
et
al.13
9
Patie
nts
with
dys
peps
ia
(Glo
bal a
sses
smen
t)
PPI v
s H
2RA
:3
(633
/634
)PP
I vs
algi
nate
/ant
acid
:2
(595
/591
)H
2RA
vs
algi
nate
/ant
acid
:1
(119
/136
)
RR Dire
ct e
stim
ate:
0.
64 (0
.49
to 0
.82)
Adj
uste
d in
dire
ct:
0.73
(0.5
6 to
0.9
6)N
aive
indi
rect
: 0.
86 (0
.71
to 1
.05)
16 Di M
ario
et
al.14
0
Patie
nts
with
pre
viou
sly u
ntre
ated
gas
tric
ulce
r
(End
osco
pic
heal
ing)
Cim
etid
ine
vs r
aniti
dine
: 5
(tot
al 6
36)
Cim
editi
ne v
s pl
aceb
o:13
(tot
al 8
52)
Rani
tidin
e vs
pla
cebo
:8
(tot
al 7
56)
RR Dire
ct e
stim
ate:
1.21
(0.8
8 to
1.6
7)A
djus
ted
indi
rect
:0.
65 (0
.35
to 1
.20)
N
aive
indi
rect
: N
ot a
vaila
ble
17 Han
doll
et a
l.141
Patie
nts
unde
rgoi
ng s
urge
ry fo
r hi
pfr
actu
res
(DVT
)
LMW
H v
s U
FH:
3 (1
36/1
11)
LMW
H v
s pl
aceb
o:
2 (1
04/1
10)
UFH
vs
plac
ebo:
10
(407
/409
)
RR Dire
ct e
stim
ate:
0.
91 (0
.36
to 2
.31)
Adj
uste
d in
dire
ct:
1.05
(0.5
2 to
2.1
3)N
aive
indi
rect
: 0.
68 (0
.44
to 1
.07)
18 Hor
n an
dLi
mbu
rg14
2
Patie
nts
with
acu
te is
chem
ic s
trok
e
(Poo
r ou
tcom
e: d
eath
or
depe
nden
cy in
activ
ities
of d
aily
livi
ng)
Mim
odip
ine
240
mg
vs 1
20 m
g:2
(340
/341
)M
imod
ipin
e 24
0 m
g vs
con
trol
: 1
(73/
69)
Mim
odip
ine
120
mg
vs c
ontr
ol:
13 (2
081/
2106
)
RR Dire
ct e
stim
ate:
1.
07 (0
.94
to 1
.22)
Adj
uste
d in
dire
ct:
0.97
(0.6
3 to
1.4
8)N
aive
indi
rect
: 1.
06 (0
.79
to 1
.42)
19 Mar
shal
l and
Irvi
ne21
Patie
nts
with
ulc
erat
ive
colit
is
(End
osco
pic
rem
issio
n)
5-A
SA v
s re
ctal
cor
ticos
tero
ids:
7 (t
otal
682
)5-
ASA
vs
R. b
udes
onid
e:
2 (t
otal
154
)Re
ctal
cor
ticos
tero
ids
vs r
ecta
l bud
eson
ide:
5 (t
otal
463
)
OR
Dire
ct e
stim
ate:
0.
53 (0
.36
to 0
.78)
Adj
uste
d in
dire
ct:
0.92
(0.3
6 to
2.3
6)
Nai
ve in
dire
ct:
1.13
(0.8
6 to
1.4
9)
20 McI
ntos
h an
dO
lliar
o143
Patie
nts
with
unc
ompl
icat
ed m
alar
ia
(Par
asite
cle
aran
ce a
t da
y 28
)
Art
emisi
nin
vs a
rtes
unat
e:1
(20/
19)
Art
emisi
nin
vs q
uini
ne:
1 (2
7/22
)A
rtes
unat
e vs
qui
nine
: 1
(47/
39)
RR Dire
ct e
stim
ate:
0.
82 (0
.55
to 1
.22)
Adj
uste
d in
dire
ct:
0.70
(0.3
8 to
1.2
8)
Nai
ve in
dire
ct:
0.42
(0.2
6 to
0.6
6)
21 Moo
re e
t al
.24
Patie
nts
with
sev
ere
post
oper
ativ
e pa
in
(>50
% p
ain
relie
f)
Para
ceta
mol
+co
dein
e vs
par
acet
amol
: 10
(309
/313
)Pa
race
tam
ol+
code
ine
vs p
lace
bo:
5 (9
8/11
0)Pa
race
tam
ol v
s pl
aceb
o:
7 (2
81/2
54)
RR Dire
ct e
stim
ate:
1.
24 (1
.01
to 1
.54)
Adj
uste
d in
dire
ct:
1.74
(0.5
9 to
5.1
8)
Nai
ve in
dire
ct:
1.03
(0.7
8 to
1.3
5)
Health Technology Assessment 2005; Vol. 9: No. 26
131
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
22 Pagl
iaro
(use
d by
Hig
gins
and
Whi
tehe
ad53
)
Patie
nts
with
cirr
hosis
and
oeso
phag
ogas
tric
var
ices
(Rat
e of
firs
t bl
eedi
ng)
�-B
lock
ers
vs s
cler
othe
rapy
:2
(111
/115
)�
-Blo
cker
s vs
con
trol
:7
(378
/394
)Sc
lero
ther
apy
vs c
ontr
ol:
17 (7
23/7
36)
RR Dire
ct e
stim
ate:
0.
53 (0
.12
to 2
.36)
Adj
uste
d in
dire
ct:
1.00
(0.5
3 to
1.8
9)N
aive
indi
rect
: 0.
69 (0
.53
to 0
.91)
23 Poyn
ard
et a
l.26
Patie
nts
with
vira
l hep
atiti
s C
(Sus
tain
ed A
LT r
espo
nse
rate
)
Inte
rfer
on (3
MU
) 12
mon
ths
vs 6
mon
ths:
4 (2
56/2
49)
Inte
rfer
on (3
MU
) 12
mon
ths
vs c
ontr
ol:
5 (1
61/1
57)
Inte
rfer
on (3
MU
) 6 m
onth
s vs
con
trol
:6
(132
/131
)
RR Dire
ct e
stim
ate:
2.
20 (1
.52
to 3
.17)
Adj
uste
d in
dire
ct:
1.49
(0.3
5 to
6.3
1)
Nai
ve in
dire
ct:
1.67
(1.1
5 to
2.4
2)
24 Rost
om e
t al
.107
Patie
nts
who
had
tak
en N
SAID
s fo
r >
3w
eeks
(Tot
al e
ndos
copi
c ul
cers
)
PPI v
s H
2RA
: 1
(210
/215
)PP
I vs
plac
ebo:
3
(443
/331
)H
2RA
vs
plac
ebo:
6
(645
/541
)
RR Dire
ct e
stim
ate:
0.
28 (0
.15
to 0
.51)
Adj
uste
d in
dire
ct:
0.61
(0.4
0 to
0.9
3)
Nai
ve in
dire
ct:
1.02
(0.7
2 to
1.4
4)
25 Sila
gy14
4
Smok
ers
(Sm
okin
g ce
ssat
ion
rate
)
>1
visit
vs
1 vi
sit:
5 (7
33/5
21)
>1
visit
vs
cont
rol:
4 (1
931/
1529
)1
visit
vs
cont
rol:
15 (7
551/
5826
)
RR Dire
ct e
stim
ate:
1.
51 (1
.08
to 2
.12)
Adj
uste
d in
dire
ct:
1.51
(0.9
0 to
2.5
6)
Nai
ve in
dire
ct:
2.34
(2.0
2 to
2.7
1)
26 Sila
gy e
t al
.145
Patie
nts
unde
rgoi
ng N
RT
(Sm
okin
g ce
ssat
ion
rate
)
Nic
otin
e pa
tch
24 h
ours
vs
16 h
ours
1 (5
1/55
)N
icot
ine
patc
h 25
hou
rs v
s co
ntro
l:24
(541
5/43
04)
Nic
otin
e pa
tch
16 h
ours
vs
cont
rol:
8 (4
368/
1737
)
RR Dire
ct e
stim
ate:
0.
70 (0
.36
to 1
.35)
Adj
uste
d in
dire
ct:
0.82
(0.5
6 to
1.2
0)
Nai
ve in
dire
ct:
1.07
(0.9
7 to
1.1
8)
27 Sila
gy e
t al
.145
Sam
e as
abo
veN
RT w
eani
ng v
s no
wea
ning
:1
(68/
56)
NRT
wea
ning
vs
cont
rol:
24 (7
571/
4598
)N
RT n
o w
eani
ng v
s co
ntro
l:5
(701
/648
)
RR Dire
ct e
stim
ate:
0.
97 (0
.68
to 1
.38)
Adj
uste
d in
dire
ct:
0.75
(0.5
3 to
1.0
6)N
aive
indi
rect
: 0.
88 (0
.74
to 1
.05)
28 Soo
et a
l.108
Patie
nts
with
non
-ulc
er d
yspe
psia
(Glo
bal s
ympt
om a
sses
smen
t)
H2R
A v
s su
cral
fate
: 1
(47/
53)
H2R
A v
s pl
aceb
o:
8 (6
07/6
18)
Sucr
alfa
te v
s pl
aceb
o:
2 (1
29/1
17)
RR Dire
ct e
stim
ate:
2.
74 (1
.25
to 6
.02)
Adj
uste
d in
dire
ct:
0.99
(0.4
7 to
2.0
8)N
aive
indi
rect
: 1.
24 (0
.91
to 1
.70)
Appendix 8
132 TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
29 Soo
et a
l.108
Sam
e as
abo
vePr
okin
etic
s vs
H2R
A:
2 (2
08/2
15)
Prok
inet
ics
vs p
lace
bo:
10 (3
26/2
64)
Sucr
alfa
te v
s pl
aceb
o:
7 (4
96/5
08)
RR Dire
ct e
stim
ate:
0.
54 (0
.22
to 1
.33)
Adj
uste
d in
dire
ct:
0.66
(0.4
1 to
1.0
5)
Nai
ve in
dire
ct:
1.04
(0.8
4 to
1.2
8)
30 Trin
adad
e an
dM
enon
146
Patie
nts
with
maj
or d
epre
ssio
n
(No.
of d
ropo
uts)
Fluo
xetin
e vs
par
oxet
ine:
5
(327
/328
)Fl
uoxe
tine
vs c
ontr
ols:
40
(250
0/22
34)
Paro
xetin
e vs
con
trol
s:33
(147
1/14
23)
RR Dire
ct e
stim
ate:
1.
00 (0
.73
to 1
.37)
Adj
uste
d in
dire
ct:
1.00
(0.8
6 to
1.1
7)N
aive
indi
rect
: 1.
24 (1
.13
to 1
.35)
31 Trin
dade
and
Men
on14
6
Sam
e as
abo
veFl
uoxe
tine
vs fl
uvox
amin
e:
1 (4
9/51
)Fl
uoxe
tine
vs c
ontr
ols:
45
(290
9/23
78)
Fluv
oxam
ine
vs c
ontr
ols:
41
(116
8/11
66)
RR Dire
ct e
stim
ate:
0.
78 (0
.18
to 3
.31)
Adj
uste
d in
dire
ct:
0.89
(0.7
7 to
1.0
2)N
aive
indi
rect
: 1.
14 (1
.03
to 1
.26)
32 Trin
dade
and
Men
on14
6
Sam
e as
abo
vePa
roxe
tine
vs fl
uvox
amin
e:
1 (5
6/64
)Pa
roxe
tine
vs c
ontr
ols:
34
(133
5/12
88)
Fluv
oxam
ine
vs c
ontr
ols:
35
(102
5/10
19)
RR Dire
ct e
stim
ate:
0.
80 (0
.47
to 1
.35)
Adj
uste
d in
dire
ct:
0.77
(0.6
3 to
0.9
3)N
aive
indi
rect
: 0.
92 (0
.81
to 1
.03)
33 Trin
dade
and
Men
on14
6
Sam
e as
abo
veSe
rtra
line
vs fl
uvox
amin
e:
1 (4
8/49
)Se
rtra
line
vs c
ontr
ols:
9
(816
/759
)Fl
uvox
amin
e vs
con
trol
s:
23 (6
85/6
51)
RR
Dire
ct e
stim
ate:
0.
40 (0
.18
to 0
.86)
Adj
uste
d in
dire
ct:
0.81
(0.6
0 to
1.1
0)N
aive
indi
rect
: 0.
94 (0
.81
to 1
.09)
34 Trin
dede
and
Men
on14
6
Sam
e as
abo
veFl
uoxe
tine
vs s
ertr
alin
e:
3 (2
66/2
73)
Fluo
xetin
e vs
con
trol
s:35
(250
2/19
67)
Sert
ralin
e vs
con
trol
s:
9 (8
16/7
59)
RR Dire
ct e
stim
ate:
0.
90 (0
.38
to 2
.14)
Adj
uste
d in
dire
ct:
0.88
(0.7
0 to
1.1
1)N
aive
indi
rect
:1.
10 (0
.98
to 1
.23)
35 van
Pinx
tere
net
al.10
9
Patie
nts
with
GO
RD-li
ke s
ympt
oms
(Hea
rtbu
rn r
emiss
ion)
PPI v
s H
2RA
: 3
(122
8/66
4)PP
I vs
plac
ebo:
1
(161
/159
)H
2RA
vs
plac
ebo:
2
(511
/502
)
RR Dire
ct e
stim
ate:
0.
67 (0
.57
to 0
.80)
Adj
uste
d in
dire
ct:
0.45
(0.3
1 to
0.6
6)
Nai
ve in
dire
ct:
0.53
(0.4
0 to
0.7
1)
Health Technology Assessment 2005; Vol. 9: No. 26
133
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
cont
inue
d
36 Zha
ng a
nd
Li W
an P
o147
Patie
nts
with
dys
men
orrh
oea
(No.
of p
atie
nts
with
at
leas
t m
oder
ate
pain
relie
f)
Nap
roxe
n vs
ibup
rofe
n:
3 (1
22/1
13)
Nap
roxe
n vs
pla
cebo
: 17
(904
/877
)Ib
upro
fen
vs p
lace
bo:
10 (3
45/3
46)
RR Dire
ct e
stim
ate:
1.
08 (0
.79
to 1
.48)
Adj
uste
d in
dire
ct:
1.40
(0.9
4 to
2.0
9)N
aive
indi
rect
: 0.
82 (0
.75
to 0
.89)
37 Zha
ng a
nd
Li W
an P
o147
Sam
e as
abo
veN
apro
xen
vs m
efen
amic
aci
d:
1 (2
4/20
)N
apro
xen
vs p
lace
bo:
18 (9
42/9
16)
Mef
enam
ic v
s pl
aceb
o:
4 (3
07/3
07)
RR Dire
ct e
stim
ate:
2.
40 (1
.39
to 4
.13)
Adj
uste
d in
dire
ct:
1.53
(1.1
1 to
2.1
2)N
aive
indi
rect
: 0.
88 (0
.80
to 0
.97)
38 Zha
ng a
nd
Li W
an P
o147
Sam
e as
abo
veN
apro
xen
vs a
spiri
n:
1 (3
2/32
)N
apro
xen
vs p
lace
bo:
17 (8
90/8
72)
Asp
irin
vs p
lace
bo:
5 (2
20/2
23)
RR Dire
ct e
stim
ate:
2.
29 (1
.16
to 4
.52)
Adj
uste
d in
dire
ct:
2.45
(1.6
5 to
3.6
4)N
aive
indi
rect
: 1.
83 (1
.49
to 2
.23)
39 Zha
ng a
nd
Li W
an P
o147
Sam
e as
abo
veIb
upro
fen
vs a
spiri
n:1
(43/
43)
Ibup
rofe
n vs
pla
cebo
: 10
(322
/327
)A
spiri
n vs
pla
cebo
: 5
(187
/184
)
RR Dire
ct e
stim
ate:
1.
90 (1
.30
to 2
.77)
Adj
uste
d in
dire
ct:
1.80
(1.1
2 to
2.8
9)N
aive
indi
rect
: 2.
32 (1
.84
to 2
.93)
40 Aus
ejo14
8
Chi
ldre
n w
ith c
roup
(Impr
ovem
ent
in c
roup
sev
erity
sco
re)
Bude
soni
de v
s de
xam
etha
sone
:1
(65/
69)
Bude
soni
de v
s pl
aceb
o:
5 (1
66/1
61)
Dex
amet
haso
ne v
s pl
aceb
o:
8 (3
65/3
74)
Stan
dard
ised
mea
n di
ffere
nce
Dire
ct e
stim
ate:
0.
09 (–
0.25
to
0.43
)A
djus
ted
indi
rect
: 0.
32 (–
0.52
to
1.16
)N
aive
indi
rect
: 0.
20 (0
.02
to 0
.38)
41 Li W
an P
o an
dZ
hang
20
Patie
nts
with
pos
tsur
gica
l pai
n
(Sum
of d
iffer
ence
in p
ain
inte
nsity
)
Para
ceta
mol
+de
xam
etha
sone
vs
para
ceta
mol
:3
(103
/99)
Para
ceta
mol
+de
xam
etha
sone
vs
plac
ebo:
5
(181
/178
)Pa
race
tam
ol v
s pl
aceb
o:
14 (5
58/5
34)
Mea
n di
ffere
nce
Dire
ct e
stim
ate:
1.
22 (0
.00
to 2
.45)
Adj
uste
d in
dire
ct:
0.51
(–0.
43 t
o 1.
45)
Nai
ve in
dire
ct:
0.13
(–0.
61 t
o 0.
88)
42 Pack
er e
t al
.69
Patie
nts
with
hea
rt fa
ilure
(Cha
nges
in le
ft ve
ntric
ular
eje
ctio
nfr
actio
n)
Car
vedi
lol v
s m
etop
rolo
l: 4
(123
/125
)C
arve
dilo
l vs
plac
ebo:
9
(534
/668
)M
etop
rolo
l vs
plac
ebo:
6
(376
/408
)
Mea
n di
ffere
nce
Dire
ct e
stim
ate:
0.
029
(0.0
07 t
o 0.
051)
Adj
uste
d in
dire
ct:
0.02
7 (0
.013
to
0.04
1)
Nai
ve in
dire
ct:
0.01
6 (0
.012
to
0.02
04)
Appendix 8
134 TA
BLE
33
Iden
tifie
d m
eta-
anal
yses
with
suf
ficie
nt d
ata
for b
oth
dire
ct a
nd in
dire
ct c
ompa
rison
s (c
ont’d
)
Met
a-an
alys
isPa
tien
ts (
outc
ome)
Inte
rven
tion
s co
mpa
red:
no.
of t
rial
s (p
atie
nts)
Rel
ativ
e ef
ficac
y (9
5% C
I)
AP,
aero
lised
pen
tam
idin
e; D
, dap
sone
; NA
, not
ava
ilabl
e; N
RT, n
icot
ine
repl
acem
ent
ther
apy;
P, p
yrim
etha
min
e; S
MX
, sul
fam
etho
xazo
le; T
MP,
trim
etho
prim
.
43 Saur
iol e
t al
.149
Patie
nts
with
sch
izop
hren
ia
(Cha
nges
in b
rief p
sych
iatr
ic r
atin
g sc
ale)
Ola
nzap
ine
vs r
isper
idon
e:
1 (1
72/1
67)
Ola
nzap
ine
vs p
lace
bo:
3 (1
620/
786)
Risp
erid
one
vs p
lace
bo:
8 (1
044/
416)
Mea
n di
ffere
nce
Dire
ct e
stim
ate:
1.
80 (–
1.43
to
5.03
)A
djus
ted
indi
rect
: 1.
33 (–
0.63
to
3.29
) N
aive
indi
rect
: 1.
07 (–
0.28
to
2.42
)
44 Zha
ng a
nd
Li W
an P
o147
Patie
nts
with
sur
gica
l pai
n
(Sum
of d
iffer
ence
in p
ain
inte
nsity
)
Para
ceta
mol
+co
dein
e vs
par
acet
amol
: 13
(449
/448
)Pa
race
tam
ol+
code
ine
vs p
lace
bo:
12 (N
A)
Para
ceta
mol
vs
plac
ebo:
31
(NA
)
Mea
n di
ffere
nce
Dire
ct e
stim
ate:
6.
97 (3
.56
to 1
0.37
)A
djus
ted
indi
rect
: –1
.16
(–6.
95 t
o 4.
64)
Nai
ve in
dire
ct:
–9.8
9 (–
11.6
5 to
–8.
13)
Health Technology Assessment 2005; Vol. 9: No. 26
145
Health Technology AssessmentProgramme
Prioritisation Strategy GroupMembers
Chair,Professor Tom Walley, Director, NHS HTA Programme,Department of Pharmacology &Therapeutics,University of Liverpool
Professor Bruce Campbell,Consultant Vascular & GeneralSurgeon, Royal Devon & ExeterHospital
Dr Edmund Jessop, MedicalAdvisor, National Specialist,Commissioning Advisory Group(NSCAG), Department ofHealth, London
Professor Jon Nicholl, Director,Medical Care Research Unit,University of Sheffield, Schoolof Health and Related Research
Dr John Reynolds, ClinicalDirector, Acute GeneralMedicine SDU, RadcliffeHospital, Oxford
Dr Ron Zimmern, Director,Public Health Genetics Unit,Strangeways ResearchLaboratories, Cambridge
HTA Commissioning BoardMembers
Programme Director, Professor Tom Walley, Director, NHS HTA Programme,Department of Pharmacology &Therapeutics,University of Liverpool
Chair,Professor Jon Nicholl,Director, Medical Care ResearchUnit, University of Sheffield,School of Health and RelatedResearch
Deputy Chair, Professor Jenny Hewison,Professor of Health CarePsychology, Academic Unit ofPsychiatry and BehaviouralSciences, University of LeedsSchool of Medicine
Dr Jeffrey AronsonReader in ClinicalPharmacology, Department ofClinical Pharmacology,Radcliffe Infirmary, Oxford
Professor Deborah Ashby,Professor of Medical Statistics,Department of Environmentaland Preventative Medicine,Queen Mary University ofLondon
Professor Ann Bowling,Professor of Health ServicesResearch, Primary Care andPopulation Studies,University College London
Dr Andrew Briggs, PublicHealth Career Scientist, HealthEconomics Research Centre,University of Oxford
Professor John Cairns, Professorof Health Economics, PublicHealth Policy, London School ofHygiene and Tropical Medicine,London
Professor Nicky Cullum,Director of Centre for EvidenceBased Nursing, Department ofHealth Sciences, University ofYork
Mr Jonathan Deeks, Senior Medical Statistician,Centre for Statistics inMedicine, University of Oxford
Dr Andrew Farmer, SeniorLecturer in General Practice,Department of Primary Health Care, University of Oxford
Professor Fiona J Gilbert,Professor of Radiology,Department of Radiology,University of Aberdeen
Professor Adrian Grant,Director, Health ServicesResearch Unit, University ofAberdeen
Professor F D Richard Hobbs,Professor of Primary Care &General Practice, Department ofPrimary Care & GeneralPractice, University ofBirmingham
Professor Peter Jones, Head ofDepartment, UniversityDepartment of Psychiatry,University of Cambridge
Professor Sallie Lamb, Professor of Rehabilitation,Centre for Primary Health Care, University of Warwick
Professor Stuart Logan,Director of Health & SocialCare Research, The Peninsula Medical School, Universities of Exeter &Plymouth
Dr Linda Patterson, Consultant Physician,Department of Medicine,Burnley General Hospital
Professor Ian Roberts, Professorof Epidemiology & PublicHealth, Intervention ResearchUnit, London School ofHygiene and Tropical Medicine
Professor Mark Sculpher,Professor of Health Economics,Centre for Health Economics,Institute for Research in theSocial Services, University of York
Dr Jonathan Shapiro, SeniorFellow, Health ServicesManagement Centre,Birmingham
Ms Kate Thomas,Deputy Director,Medical Care Research Unit,University of Sheffield
Ms Sue Ziebland,Research Director, DIPEx,Department of Primary HealthCare, University of Oxford,Institute of Health Sciences
Current and past membership details of all HTA ‘committees’ are available from the HTA website (www.ncchta.org)
© Queen’s Printer and Controller of HMSO 2005. All rights reserved.
Health Technology Assessment Programme
146
Diagnostic Technologies & Screening PanelMembers
Chair,Dr Ron Zimmern, Director ofthe Public Health Genetics Unit,Strangeways ResearchLaboratories, Cambridge
Ms Norma Armston,Lay Member, Bolton
Professor Max BachmannProfessor of Health Care Interfaces, Department of Health Policy and Practice,University of East Anglia
Professor Rudy BilousProfessor of Clinical Medicine &Consultant Physician,The Academic Centre,South Tees Hospitals NHS Trust
Dr Paul Cockcroft, Consultant MedicalMicrobiologist and ClinicalDirector of Pathology,Department of ClinicalMicrobiology, St Mary'sHospital, Portsmouth
Professor Adrian K Dixon,Professor of Radiology,University Department ofRadiology, University ofCambridge Clinical School
Dr David Elliman, Consultant Paediatrician/Hon. Senior Lecturer,Population Health Unit, Great Ormond St. Hospital,London
Professor Glyn Elwyn,Primary Medical Care Research Group,Swansea Clinical School,University of Wales Swansea
Mr Tam Fry, HonoraryChairman, Child GrowthFoundation, London
Dr Jennifer J Kurinczuk,Consultant ClinicalEpidemiologist,National PerinatalEpidemiology Unit, Oxford
Dr Susanne M Ludgate, MedicalDirector, Medicines &Healthcare Products RegulatoryAgency, London
Professor William Rosenberg,Professor of Hepatology, LiverResearch Group, University ofSouthampton
Dr Susan Schonfield, Consultantin Public Health, SpecialisedServices Commissioning NorthWest London, HillingdonPrimary Care Trust
Dr Phil Shackley, SeniorLecturer in Health Economics,School of Population andHealth Sciences, University ofNewcastle upon Tyne
Dr Margaret Somerville, PMSPublic Health Lead, PeninsulaMedical School, University ofPlymouth
Dr Graham Taylor, ScientificDirector & Senior Lecturer,Regional DNA Laboratory, TheLeeds Teaching Hospitals
Professor Lindsay WilsonTurnbull, Scientific Director,Centre for MR Investigations &YCR Professor of Radiology,University of Hull
Professor Martin J Whittle,Associate Dean for Education,Head of Department ofObstetrics and Gynaecology,University of Birmingham
Dr Dennis Wright, Consultant Biochemist &Clinical Director, Pathology & The KennedyGalton Centre, Northwick Park & St Mark’sHospitals, Harrow
Pharmaceuticals PanelMembers
Chair,Dr John Reynolds, ChairDivision A, The John RadcliffeHospital, Oxford RadcliffeHospitals NHS Trust
Professor Tony Avery, Head of Division of PrimaryCare, School of CommunityHealth Services, Division ofGeneral Practice, University ofNottingham
Ms Anne Baileff, ConsultantNurse in First Contact Care,Southampton City Primary CareTrust, University ofSouthampton
Professor Stirling Bryan,Professor of Health Economics,Health Services Management Centre,University of Birmingham
Mr Peter Cardy, ChiefExecutive, Macmillan CancerRelief, London
Professor Imti Choonara,Professor in Child Health,Academic Division of ChildHealth, University ofNottingham
Dr Robin Ferner, ConsultantPhysician and Director, WestMidlands Centre for AdverseDrug Reactions, City HospitalNHS Trust, Birmingham
Dr Karen A Fitzgerald,Consultant in PharmaceuticalPublic Health, National PublicHealth Service for Wales,Cardiff
Mrs Sharon Hart, Head of DTB Publications, Drug &Therapeutics Bulletin, London
Dr Christine Hine, Consultant inPublic Health Medicine, SouthGloucestershire Primary CareTrust
Professor Stan Kaye,Cancer Research UK Professor of Medical Oncology,Section of Medicine, The Royal Marsden Hospital,Sutton
Ms Barbara Meredith,Lay Member, Epsom
Dr Andrew Prentice, SeniorLecturer and ConsultantObstetrician & Gynaecologist,Department of Obstetrics &Gynaecology, University ofCambridge
Dr Frances Rotblat, CPMPDelegate, Medicines &Healthcare Products RegulatoryAgency, London
Professor Jan Scott, Professor of Psychological Treatments,Institute of Psychiatry,University of London
Mrs Katrina Simister, AssistantDirector New Medicines,National Prescribing Centre,Liverpool
Dr Richard Tiner, MedicalDirector, Medical Department,Association of the BritishPharmaceutical Industry,London
Dr Helen Williams,Consultant Microbiologist,Norfolk & Norwich UniversityHospital NHS Trust
Current and past membership details of all HTA ‘committees’ are available from the HTA website (www.ncchta.org)
Health Technology Assessment 2005; Vol. 9: No. 26
147
Therapeutic Procedures PanelMembers
Chair, Professor Bruce Campbell,Consultant Vascular andGeneral Surgeon, Departmentof Surgery, Royal Devon &Exeter Hospital
Dr Aileen Clarke,Reader in Health ServicesResearch, Public Health &Policy Research Unit, Barts &the London School of Medicine& Dentistry, London
Dr Matthew Cooke, Reader inA&E/Department of HealthAdvisor in A&E, WarwickEmergency Care andRehabilitation, University ofWarwick
Dr Carl E Counsell, ClinicalSenior Lecturer in Neurology,Department of Medicine andTherapeutics, University ofAberdeen
Ms Amelia Curwen, ExecutiveDirector of Policy, Services andResearch, Asthma UK, London
Professor Gene Feder, Professorof Primary Care R&D,Department of General Practiceand Primary Care, Barts & theLondon, Queen Mary’s Schoolof Medicine and Dentistry,London
Professor Paul Gregg,Professor of OrthopaedicSurgical Science, Department ofGeneral Practice and PrimaryCare, South Tees Hospital NHSTrust, Middlesbrough
Ms Bec Hanley, Co-Director,TwoCan Associates,Hurstpierpoint
Ms Maryann L Hardy, Lecturer, Division ofRadiography, University ofBradford
Professor Alan Horwich,Director of Clinical R&D,Academic Department ofRadiology, The Institute ofCancer Research, London
Dr Simon de Lusignan,Senior Lecturer, Primary Care Informatics,Department of CommunityHealth Sciences,St George’s Hospital MedicalSchool, London
Professor Neil McIntosh,Edward Clark Professor of Child Life & Health,Department of Child Life &Health, University of Edinburgh
Professor James Neilson,Professor of Obstetrics andGynaecology, Department ofObstetrics and Gynaecology,University of Liverpool
Dr John C Pounsford,Consultant Physician,Directorate of Medical Services,North Bristol NHS Trust
Karen Roberts, NurseConsultant, Queen ElizabethHospital, Gateshead
Dr Vimal Sharma, ConsultantPsychiatrist/Hon. Senior Lecturer,Mental Health Resource Centre,Cheshire and Wirral PartnershipNHS Trust, Wallasey
Dr L David Smith, ConsultantCardiologist, Royal Devon &Exeter Hospital
Professor Norman Waugh,Professor of Public Health,Department of Public Health,University of Aberdeen
Current and past membership details of all HTA ‘committees’ are available from the HTA website (www.ncchta.org)
Health Technology Assessment Programme
148Current and past membership details of all HTA ‘committees’ are available from the HTA website (www.ncchta.org)
Expert Advisory NetworkMembers
Professor Douglas Altman,Director of CSM & CancerResearch UK Med Stat Gp,Centre for Statistics inMedicine, University of Oxford,Institute of Health Sciences,Headington, Oxford
Professor John Bond,Director, Centre for HealthServices Research, University ofNewcastle upon Tyne, School ofPopulation & Health Sciences,Newcastle upon Tyne
Mr Shaun Brogan, Chief Executive, RidgewayPrimary Care Group, Aylesbury
Mrs Stella Burnside OBE,Chief Executive, Office of theChief Executive. TrustHeadquarters, AltnagelvinHospitals Health & SocialServices Trust, Altnagelvin AreaHospital, Londonderry
Ms Tracy Bury, Project Manager, WorldConfederation for PhysicalTherapy, London
Professor Iain T Cameron,Professor of Obstetrics andGynaecology and Head of theSchool of Medicine,University of Southampton
Dr Christine Clark,Medical Writer & ConsultantPharmacist, Rossendale
Professor Collette Clifford,Professor of Nursing & Head ofResearch, School of HealthSciences, University ofBirmingham, Edgbaston,Birmingham
Professor Barry Cookson,Director, Laboratory ofHealthcare Associated Infection,Health Protection Agency,London
Professor Howard Cuckle,Professor of ReproductiveEpidemiology, Department ofPaediatrics, Obstetrics &Gynaecology, University ofLeeds
Dr Katherine Darton, Information Unit, MIND – The Mental Health Charity,London
Professor Carol Dezateux, Professor of PaediatricEpidemiology, London
Mr John Dunning,Consultant CardiothoracicSurgeon, CardiothoracicSurgical Unit, PapworthHospital NHS Trust, Cambridge
Mr Jonothan Earnshaw,Consultant Vascular Surgeon,Gloucestershire Royal Hospital,Gloucester
Professor Martin Eccles, Professor of ClinicalEffectiveness, Centre for HealthServices Research, University ofNewcastle upon Tyne
Professor Pam Enderby,Professor of CommunityRehabilitation, Institute ofGeneral Practice and PrimaryCare, University of Sheffield
Mr Leonard R Fenwick, Chief Executive, Newcastleupon Tyne Hospitals NHS Trust
Professor David Field, Professor of Neonatal Medicine,Child Health, The LeicesterRoyal Infirmary NHS Trust
Mrs Gillian Fletcher, Antenatal Teacher & Tutor andPresident, National ChildbirthTrust, Henfield
Professor Jayne Franklyn,Professor of Medicine,Department of Medicine,University of Birmingham,Queen Elizabeth Hospital,Edgbaston, Birmingham
Ms Grace Gibbs, Deputy Chief Executive,Director for Nursing, Midwifery& Clinical Support Services, West Middlesex UniversityHospital, Isleworth
Dr Neville Goodman, Consultant Anaesthetist,Southmead Hospital, Bristol
Professor Alastair Gray,Professor of Health Economics,Department of Public Health,University of Oxford
Professor Robert E Hawkins, CRC Professor and Director ofMedical Oncology, Christie CRCResearch Centre, ChristieHospital NHS Trust, Manchester
Professor Allen Hutchinson, Director of Public Health &Deputy Dean of ScHARR,Department of Public Health,University of Sheffield
Dr Duncan Keeley,General Practitioner (Dr Burch& Ptnrs), The Health Centre,Thame
Dr Donna Lamping,Research Degrees ProgrammeDirector & Reader in Psychology,Health Services Research Unit,London School of Hygiene andTropical Medicine, London
Mr George Levvy,Chief Executive, MotorNeurone Disease Association,Northampton
Professor James Lindesay,Professor of Psychiatry for theElderly, University of Leicester,Leicester General Hospital
Professor Julian Little,Professor of Human GenomeEpidemiology, Department ofEpidemiology & CommunityMedicine, University of Ottawa
Professor Rajan Madhok, Medical Director & Director ofPublic Health, Directorate ofClinical Strategy & PublicHealth, North & East Yorkshire& Northern Lincolnshire HealthAuthority, York
Professor David Mant, Professor of General Practice,Department of Primary Care,University of Oxford
Professor Alexander Markham, Director, Molecular MedicineUnit, St James’s UniversityHospital, Leeds
Dr Chris McCall, General Practitioner, TheHadleigh Practice, Castle Mullen
Professor Alistair McGuire,Professor of Health Economics,London School of Economics
Dr Peter Moore, Freelance Science Writer, Ashtead
Dr Sue Moss, Associate Director,Cancer Screening EvaluationUnit, Institute of CancerResearch, Sutton
Mrs Julietta Patnick, Director, NHS Cancer ScreeningProgrammes, Sheffield
Professor Tim Peters,Professor of Primary CareHealth Services Research,Academic Unit of PrimaryHealth Care, University ofBristol
Professor Chris Price, Visiting Chair – Oxford, ClinicalResearch, Bayer DiagnosticsEurope, Cirencester
Professor Peter Sandercock,Professor of Medical Neurology,Department of ClinicalNeurosciences, University ofEdinburgh
Dr Eamonn Sheridan,Consultant in Clinical Genetics,Genetics Department,St James’s University Hospital,Leeds
Dr Ken Stein,Senior Clinical Lecturer inPublic Health, Director,Peninsula TechnologyAssessment Group, University of Exeter
Professor Sarah Stewart-Brown, Professor of Public Health,University of Warwick, Division of Health in theCommunity Warwick MedicalSchool, LWMS, Coventry
Professor Ala Szczepura, Professor of Health ServiceResearch, Centre for HealthServices Studies, University ofWarwick
Dr Ross Taylor, Senior Lecturer, Department ofGeneral Practice and PrimaryCare, University of Aberdeen
Mrs Joan Webster, Consumer member, HTA –Expert Advisory Network
How to obtain copies of this and other HTA Programme reports.An electronic version of this publication, in Adobe Acrobat format, is available for downloading free ofcharge for personal use from the HTA website (http://www.ncchta.org). A fully searchable CD-ROM isalso available (see below).
Printed copies of HTA monographs cost £20 each (post and packing free in the UK) to both public andprivate sector purchasers from our Despatch Agents, York Publishing Services.
Non-UK purchasers will have to pay a small fee for post and packing. For European countries the cost is£2 per monograph and for the rest of the world £3 per monograph.
You can order HTA monographs from our Despatch Agents, York Publishing Services by:
– fax (with credit card or official purchase order) – post (with credit card or official purchase order or cheque)– phone during office hours (credit card only).
Additionally the HTA website allows you either to pay securely by credit card or to print out yourorder and then post or fax it.
Contact details are as follows:York Publishing Services Email: ncchta@yps-publishing.co.ukPO Box 642 Tel: 0870 1616662YORK YO31 7WX Fax: 0870 1616663UK Fax from outside the UK: +44 1904 430868
NHS libraries can subscribe free of charge. Public libraries can subscribe at a very reduced cost of £100 for each volume (normally comprising 30–40 titles). The commercial subscription rate is £300 per volume. Please contact York Publishing Services at the address above. Subscriptions can only bepurchased for the current or forthcoming volume.
Payment methods
Paying by chequeIf you pay by cheque, the cheque must be in pounds sterling, made payable to York PublishingDistribution and drawn on a bank with a UK address.
Paying by credit cardThe following cards are accepted by phone, fax, post or via the website ordering pages: Delta, Eurocard,Mastercard, Solo, Switch and Visa. We advise against sending credit card details in a plain email.
Paying by official purchase orderYou can post or fax these, but they must be from public bodies (i.e. NHS or universities) within the UK.We cannot at present accept purchase orders from commercial companies or from outside the UK.
How do I get a copy of HTA on CD?
Please use the form on the HTA website (www.ncchta.org/htacd.htm). Or contact York PublishingServices (see contact details above) by email, post, fax or phone. HTA on CD is currently free of chargeworldwide.
The website also provides information about the HTA Programme and lists the membership of the variouscommittees.
HTA
Health Technology Assessm
ent 2005;Vol. 9: No. 26
Indirect comparisons of com
peting interventions
Indirect comparisons of competinginterventions
AM Glenny, DG Altman, F Song, C Sakarovitch, JJ Deeks, R D’Amico, M Bradburn and AJ Eastwood
Health Technology Assessment 2005; Vol. 9: No. 26
HTAHealth Technology AssessmentNHS R&D HTA Programme
The National Coordinating Centre for Health Technology Assessment,Mailpoint 728, Boldrewood,University of Southampton,Southampton, SO16 7PX, UK.Fax: +44 (0) 23 8059 5639 Email: hta@soton.ac.ukhttp://www.ncchta.org ISSN 1366-5278
FeedbackThe HTA Programme and the authors would like to know
your views about this report.
The Correspondence Page on the HTA website(http://www.ncchta.org) is a convenient way to publish
your comments. If you prefer, you can send your comments to the address below, telling us whether you would like
us to transfer them to the website.
We look forward to hearing from you.
July 2005
top related