n engl j med 367;14 nejm.org october 4, 2012 1353

T h e n e w e ngl a nd j o u r na l o f m e dic i n e

s t a t i s t i c s i n m e d i c i n e

Missing DataJames H. Ware, Ph.D., David Harrington, Ph.D., David J. Hunter, M.B., B.S., M.P.H., Sc.D.,

and Ralph B. D’Agostino, Sr., Ph.D.

Missing data threaten the validity of many clini-cal trials. In this issue of the Journal, the mem-bers of an expert panel convened by the National Research Council (NRC) provide recommenda-tions regarding the design, conduct, and analysis of studies to minimize that threat.1 The authors define missing data as “values that are not avail-able and that would be meaningful for analysis if they were observed.” They find that there is no analytic approach that can assuredly produce un-biased estimates of treatment effects when rele-vant data are missing and therefore recommend that investigators place increased emphasis on strategies for designing and conducting studies to minimize missing data.

The members of the panel also find that when some data are missing, methods of analysis that are model-based or that use appropriate weight-ing are usually superior to complete-case analy-sis or single-imputation methods such as the last observation carried forward. When the degree of missingness in the data raises questions about the validity of preferred methods such as model-based multiple imputation, they recommend that sensitivity analyses be performed to determine whether the conclusions are sensitive to assump-tions about the missing-data mechanism.

Though the panel was convened in response to a request from the Food and Drug Adminis-tration to provide advice about missing data in clinical trials, the issues raised by the panel apply with equal force to many observational studies. We consider it important, however, that the panel’s recommendations not be interpreted as creating a barrier to the conduct and dissemina-tion of the best possible research in populations with disorders that make follow-up especially difficult — for example, substance dependence or some mental illnesses. Moreover, the recom-mendation of the panel to “Target a population that is not adequately served by current treat-

ments and hence has an incentive to remain in the study” could be pragmatically desirable but ethically challenging in some settings. For exam-ple, an institutional review board might rule that potential study participants cannot give true in-formed consent if the only way they can access care is to participate in a placebo-controlled, randomized trial.

In response to the NRC report, we have re-viewed Journal policies and practices with regard to the review of clinical trials and observational studies and have reached the following conclu-sions. First, the statistical consultants to the Journal will pay greater attention to the features of the design and conduct of trials and observa-tional studies that reduce the extent and poten-tial impact of missing data. Second, when authors use complete-case analysis or single-imputation methods to analyze incomplete data, we will re-quire justification that the assumptions required for the validity of those methods are reasonable. We support the recommendation of the expert panel that weighting or model-based methods are usually preferred because they require less restrictive assumptions about the missing-data mechanism.

Third, when the missing data are sufficiently extensive to raise questions about the robust-ness of the results to unverifiable assumptions, we may require authors to conduct sensitivity analyses. We recognize, however, that this is a relatively undeveloped area of statistical analysis and that no clear guidelines are currently avail-able for defining the appropriate sensitivity analysis.

Fourth, the preferred methods of analysis (e.g., weighted estimating equations and multi-ple imputation) often involve complex modeling. It will be important for Journal authors who use these methods to explain clearly both the ration-ale for the choice of a model and the details of

The New England Journal of Medicine Downloaded from nejm.org on February 10, 2014. For personal use only. No other uses without permission.

Copyright © 2012 Massachusetts Medical Society. All rights reserved.

n engl j med 367;14 nejm.org october 4, 20121354

statistics in medicine

variables used in the model. Clearly written sup-plemental appendixes will be helpful, perhaps even essential, for outside reviewers, statistical consultants, and our readers.

Fifth, the report urges consideration of the effects of missing data and analytic strategies at the design stage. As statistical reviewers, we ex-pect that these elements of the design will be described in the registration filed with one of the World Health Organization–compliant clinical trials databases (International Clinical Trials Registry Platform [ICTRP]), or with a database that is an authorized contributor to the ICTRP, and in the formal statistical analysis plan (SAP). If it becomes necessary to account for missing data in ways that were not evident at the design stage, modifications to the analysis plan should be posted as a protocol or SAP amendment.

Missing data are common in clinical research and can complicate interpretation or even invali-date an otherwise important study. We welcome the report from the NRC panel and its thought-ful recommendations regarding strategies for strengthening future studies.

Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.

Dr. D’Agostino served on the NRC expert panel.

From the Departments of Biostatistics ( J.H.W.) and Epidemiol-ogy (D.J.H.), Harvard School of Public Health, the Department of Biostatistics and Computational Biology, Dana–Farber Can-cer Institute (D.H.), and the Department of Mathematics and Statistics, Boston University (R.B.D.) — all in Boston.

1. Little RJ, D’Agostino R, Cohen ML, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med 2012; 367:1355-60.

DOI: 10.1056/NEJMsm1210043Copyright © 2012 Massachusetts Medical Society.

an nejm app for iphone

The NEJM Image Challenge app brings a popular online feature to the smartphone. Optimized for viewing on the iPhone and iPod Touch, the Image Challenge app lets

you test your diagnostic skills anytime, anywhere. The Image Challenge app randomly selects from 300 challenging clinical photos published in NEJM, with a new image added each week. View an image, choose your answer,

get immediate feedback, and see how others answered. The Image Challenge app is available at the iTunes App Store.

The New England Journal of Medicine Downloaded from nejm.org on February 10, 2014. For personal use only. No other uses without permission.

Copyright © 2012 Massachusetts Medical Society. All rights reserved.