research techniques made simple articles 49–60 (2016–2017) … · 2018. 4. 6. · research...

Research Techniques Made SimpleArticles 49–60 (2016–2017)

www.jidonline.org

JID_v2_i1_COVER.indd 1JID_v2_i1_COVER.indd 1 08-11-2017 19:01:4108-11-2017 19:01:41

TH

E JO

UR

NA

L OF IN

VE

STIGA

TIVE

DE

RM

ATO

LOG

Y R

esearch Techniques M

ade Sim

ple

Articles 49–60 (2016–2017)

ELSE

VIE

R

JOURNAL OF INVESTIGATIVE DERMATOLOGYThe official journal of The Society for Investigative Dermatology and European Society for Dermatological Research

Volume 137 RTMS 49e60 December 2017

EditorMark C. Udey, St. Louis, MO

Principal Deputy EditorThomas Krieg, Cologne, Germany

Deputy EditorsLeena Bruckner-Tuderman, Freiburg, GermanyKilian Eyerich, Munich, GermanyDavid E. Fisher, Boston, MAJoel M. Gelfand, Philadelphia, PAValerie Horsley, New Haven, CTSarah E. Millar, Philadelphia, PATony Oro, Stanford, CAVincent Piguet, Toronto, Canada

Section EditorsMartine Bagot, Paris, FranceIsaac Brownell, Bethesda, MDKeith A. Choate, New Haven, CTMichael Detmar, Zurich, SwitzerlandJames T. Elder, Ann Arbor, MIJohn E. Harris, Worcester, MADaniel H. Kaplan, Pittsburgh, PAEthan A. Lerner, Boston, MACarien M. Niessen, Cologne, GermanyAimee S. Payne, Philadelphia, PAMartin Roecken, Tubingen, GermanyThomas Schwarz, Kiel, GermanyPhyllis I. Spuls, Amsterdam, The NetherlandsMarjana Tomic-Canic, Miami, FLXiao-Jing Wang, Denver, CO

Statistical EditorChao Xing, Dallas, TX

Managing EditorElizabeth Nelson Blalock, Chapel Hill, NC

Editorial Process ManagerSarah Forgeng, Chapel Hill, NC

Medical WriterHeather Yarnall Schultz, Huntington, WV

JID Connector EditorLynn A. Cornelius, St. Louis, MO

Cells to Surgery Quiz EditorKeyvan Nouri, Miami, FL

Meet the Investigator EditorAyman Grada, Boston, MA

Meeting Reports Section EditorJouni Uitto, Philadelphia, PA

Podcast EditorsAbigail Baird Waldman, Chicago, ILRobert Dellavalle, Denver, COOlivier Gaide, Lausanne, Switzerland

Research Techniques Made SimpleJodi Lynn Johnson, Chicago, IL, Coordinating EditorBrian Kim, St. Louis, MO, Contributing Editor

SnapshotDx Quiz EditorMariya Miteva, Miami, FL

Editors EmeritiMarion B. Sulzberger, 1938-1949Naomi M. Kanof, 1949-1967Richard B. Stoughton, 1967-1972Irwin M. Freedberg, 1972-1977Ruth K. Freinkel, 1977-1982Howard P. Baden, 1982-1987David A. Norris, 1987-1992Edward J. O’Keefe, 1992-1997Conrad Hauser, 1997-2002Lowell A. Goldsmith, 2002-2007Paul R. Bergstresser, 2007-2012Barbara A. Gilchrest, 2012-2017

Editorial ConsultantsMasayuki Amagai, Tokyo, JapanMaryam Asgari, Boston, MAJurgen Becker, Graz, AustriaMark Berneburg, Tubingen, GermanyTilo Biedermann, Munich, GermanyWendy B. Bollag, Augusta, GAVladimir Botchkarev, Bradford, UKJoke Bouwstra, Leiden, The NetherlandsPaul E. Bowden, Cardiff, UKJulide Celebi, New York, NYAngela M. Christiano, New York, NYCheng-Ming Chuong, Los Angeles, CAThomas N. Darling, Bethesda, MDJeffrey M. Davidson, Nashville, TNRobert Dellavalle, Denver, COMitchell F. Denning, Chicago, ILRichard L. Eckert, Baltimore, MDTatiana Efimova, Washington, DCAlexander H. Enk, Heidelberg, GermanyGary J. Fisher, Ann Arbor, MIMayumi Fujita, Aurora, CORichard Gallo, San Diego, CASpiro Getsios, Collegeville, PAMichel F. Gilliet, Lausanne, SwitzerlandMatthias Goebeler, Wurzburg, Germany

Kathleen J. Green, Chicago, ILMichael Hertl, Marburg, GermanyAlain Hovnanian, Paris, FranceSam Hwang, Sacramento, CARivkah Isseroff, Davis, CAAndrew Johnston, Ann Arbor, MIKenji Kabashima, Kyoto, JapanVeli-Matti Kahari, Turku, FinlandTatsuyoshi Kawamura, Yamanashi, JapanReinhard Kirnbauer, Vienna, AustriaHeidi H. Kong, Bethesda, MDJo Lambert, Ghent, BelgiumAlexander G. Marneros, Boston, MACaterina Missero, Napoli, ItalyMaria Morasso, Bethesda, MDAkimichi Morita, Nagoya, JapanKeisuke Nagao, Bethesda, MDPaul Nghiem, Seattle, WATamar Nijsten, Rotterdam, The NetherlandsManabu Ohyama, Tokyo, JapanAmy S. Paller, Chicago, ILAndrey A. Panteleyev, Moscow, RussiaCarlo Pincelli, Modena, ItalyGraca Raposo, Paris, FranceDennis Roop, Denver, COSarbjit S. Saini, Baltimore, MD

Fernanda Sakamoto, Boston, MAHelmut Schaider, Brisbane, AustraliaChristoph Schlapbach, Berne, SwitzerlandMartin Schmelz, Berne, SwitzerlandVijayasaradhi Setaluri, Madison, WIJohn Seykora, Philadelphia, PAJan C. Simon, Leipzig, GermanyEli Sprecher, Tel Aviv, IsraelRobert S. Stern, Boston, MAGeorg Stingl, Vienna, AustriaMakoto Sugaya, Tokyo, JapanRobert Swerlick, Atlanta, GASergey M. Troyanovsky, Chicago, ILHensin Tsao, Boston, MAErwin Tschachler, Vienna, AustriaJouni Uitto, Philadelphia, PAMaurice van Steensel, Dundee, UKBaoxi Wang, Beijing, ChinaNicole L. Ward, Cleveland, OHWendy Weinberg, Bethesda, MDThomas Werfel, Hannover, GermanyTraci Wilgus, Columbus, OHGiovanna Zambruno, Rome, ItalyXuejun Zhang, Heifei, ChinaBin Zheng, Charlestown, MADetlef Zillikens, Lubeck, Germany

JOURNAL OF INVESTIGATIVE DERMATOLOGYwww.jidonline.org

Copyright ª 2017 Society for Investigative Dermatology, Inc.ISSN 0022-202X

SCOPEThe Journal of Investigative Dermatology is published monthly in printand online. The journal provides an international forum for the pub-lication of high-quality, original articles. JID features information on allaspects of cutaneous biology and skin disease.

This journal is covered by Adonis, BIOSIS, CAB Abstracts, ChemicalAbstracts Databases, CurrentContents/ClinicalMedicine, CurrentCon-tents/Life Sciences, Derwent Journals Abstracted, EBSCO, Embase/Excerpta Medica, Global Health, Index Medicus/MEDLINE, InternationalPharmaceutical Abstracts, PASCAL, Reference Update, Science CitationIndex, SciSearch/SCI Expanded, Sociedad Iberoamericana de Informa-cion Cientifica (SIIC) Database.

EDITORIALAll correspondence should be addressed to: Elizabeth Blalock, ManagingEditor for The Journal of Investigative Dermatology, P.O. Box 429,Chapel Hill, NC 27514. Tel: +1 919 932 0140. Fax: +1 216 619 9980.E-mail: [email protected]. All manuscripts should be submitted onlineat: http://jid.manuscriptcentral.com.

SOCIETYFor information, contact the Society for Investigative Dermatology [email protected] or the European Society for Dermatological Research [email protected]. Detailed instructions to authors are available at thejournal website, www.jidonline.org.

CUSTOMER SERVICEAddress orders, claims, change of address to: Elsevier Health SciencesDivision, Subscription Customer Service, 3251 Riverport Lane, MarylandHeights, MO 63043. Tel: +1 800 654 2452 (toll free US and Canada);+1 314 447 8871 (outside US and Canada). Fax: +1 314 447 8029.E-mail: [email protected] (for print support);[email protected] (for online support). Addresschanges must be submitted four weeks in advance.

SUBSCRIPTIONSInstitutional print & electronic subscriptions: $1,396 US, $1,836 Rest ofWorld.Personal print & electronic subscriptions: $899 US, $1,199 Rest ofWorld.

Further information on this journal is available from the Publisher orfrom this journal’s Web site, www.jidonline.org. Information on otherElsevier products is available through Elsevier’s Web site, www.elsevier.com.Contact information: Tel: +1 800 654 2452 (toll free US & Canada), +1314 447 8871 (Rest of World). E-mail: [email protected].

Prices include postage and are subject to change without notice. Singleissues of The Journal of Investigative Dermatology are available.

INFORMATION FOR ADVERTISERSAdvertising orders and inquiries can be sent to: US, Canada, and SouthAmerica, Roxana Muniz, Elsevier, 230 Park Avenue, Suite 800, New York,NY 10169; Tel: +1 347 702 0380; Fax: +1 212 633 3820; E-mail: [email protected]. Classified advertising orders and inquiries can be sent toAdamMoorad, 230 Park Avenue, Suite 800, New York, NY 10169; Tel: +1212 633 3122; Fax: +1 212 633 3820; E-mail: [email protected]& the rest of theworld, Carol Clark, E-mail: [email protected].

SUPPLEMENTSInquiries concerning supplements should be addressed to: Craig Smith,Tel: +1 212 462 1933; E-mail: [email protected].

INFORMATION FOR AUTHORSFor inquiries relating to the submission of articles, please visit http://authors.elsevier.com. To submit a manuscript to The Journal of Inves-tigative Dermatology, please visit https://mc.manuscriptcentral.com/jid.

This site also provides detailed Information for Authors. Contact detailsfor questions arising after acceptance of an article, especially thoserelating to proofs, are provided after registration of an article forpublication.

REPRINTSFor queries about author offprints, e-mail [email protected] order 100 or more reprints for educational, commercial, or promo-tional use, contact Derrick Imasa at Elsevier Inc, 230 Park Avenue, Suite800, New York, NY 10169; Tel: +1 215 633 3874; Fax: +1 212 462 1935;E-mail: [email protected]. Reprints of single articles available onlinemay be obtained by purchasing Pay-Per-View access for $31.50 perarticle on the journal Web site, www.jidonline.org.

PERMISSIONSThis journal and the individual contributions contained in it are protectedunder copyright, and the following terms and conditions apply to theiruse in addition to the terms of any Creative Commons or other userlicense that has been applied by the publisher to an individual article:

Photocopying: Single photocopies of single articles may be made forpersonal use as allowed by national copyright laws. Permission is notrequired for photocopying of articles published under the CC BY licensenor for photocopying for non-commercial purposes in accordance withany other user license applied by the publisher. Permission of the pub-lisher and payment of a fee is required for all other photocopying,including multiple or systematic copying, copying for advertising orpromotional purposes, resale, and all forms of document delivery. Spe-cial rates are available for educational institutions that wish to makephotocopies for non-profit educational classroom use.

Derivative Works: Users may reproduce tables of contents or pre-pare lists of articles including abstracts for internal circulation withintheir institutions or companies. Other than for articles published underthe CC BY license, permission of the publisher is required for resale ordistribution outside the subscribing institution or company. For anysubscribed articles or articles published under a CC BY-NC-ND license,permission of the publisher is required for all other derivative works,including compilations and translations.

Storage or Usage: Except as outlined above or as set out in therelevant user license, no part of this publication may be reproduced,stored in a retrieval system or transmitted in any form or by any means,electronic, mechanical, photocopying, recording or otherwise, withoutprior written permission of the publisher.

Address permissions requests to: Elsevier Rights Department, at the faxand e-mail addresses noted below. For information on how to seek per-mission visit www.elsevier.com/permissions or call: +44 1865 843830(UK) / +1 215 239 3804 (US). In the US, users may clear permissions andmake payments through the Copyright Clearance Center, Inc, 222Rosewood Drive, Danvers, MA 01923, USA; Tel: +1 978 750 8400. Inthe UK, users may clear permissions through the Copyright LicensingAgency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road,London W1P 0LP, UK; Tel: +44 20 7631 5555. Other countries may havea local reprographic rights agency for payments.

Printed on acid-free paper, effective with Volume 126, Issue 1, 2006

NOTICENo responsibility is assumed by the Society for Investigative Dermatol-ogy, the editors, the publisher, or their respective employees, officers, oragents for any injury and/or damage to persons or property as a matter ofproduct liability, negligence or otherwise, or from any use or operation ofany methods, products, instructions or ideas contained in the materialherein. These are the responsibility of the contributor. Because of rapidadvances in the medical sciences, in particular, independent verificationof diagnoses and drug dosages should be made.

Although all advertising material is expected to conform to ethical(medical) standards, inclusion in this publication does not constitute aguarantee or endorsement of the quality or value of such product or ofthe claims made of it by its manufacturer.

Journal of Investigative Dermatology (2017), Volume 137 ª 2017 Society for Investigative Dermatology

JOURNAL OF INVESTIGATIVE DERMATOLOGY

Research Techniques Made Simple Articles 49e60 (2016e2017)

EDITORIALSJID Connector: Your Link to Content and ColleaguesLA Cornelius

The Challenge of Effective Communication among ScientistsTR Matos

RTMS ARTICLESArticle 49 Research Techniques Made Simple: Laser Capture Microdissection in Cutaneous

ResearchE Chen Gonzalez and JS McGee

Article 50 Research Techniques Made Simple: Assessing Risk of Bias in Systematic ReviewsAM Drucker, P Fleming and A-W Chan

Article 51 Research Techniques Made Simple: Workflow for Searching Databases to ReduceEvidence Selection Bias in Systematic ReviewsL Le Cleach, E Doney, KA Katz, HC Williams and L Trinquart

Article 52 Research Techniques Made Simple: Mouse Models of Autoimmune Blistering DiseasesR Pollmann and R Eming

Article 53 Research Techniques Made Simple: Analysis of Collective Cell Migration Using theWound Healing AssayA Grada, M Otero-Vinas, F Prieto-Castrillo, Z Obagi and V Falanga

Article 54 Research Techniques Made Simple: Identification and Characterization of LongNoncoding RNA in Dermatological ResearchD Antonini, MR Mollo and C Missero

Article 55 Research Techniques Made Simple: Experimental Methodology for Single-CellMass CytometryTR Matos, H Liu and J Ritz

Article 56 Research Techniques Made Simple: Mass Cytometry Analysis Tools for Decryptingthe Complexity of Biological SystemsTR Matos, H Liu and J Ritz

Article 57 Research Techniques Made Simple: High-Throughput Sequencing of the T-CellReceptorTR Matos, MA de Rie and MBM Teunissen

Article 58 Research Techniques Made Simple: Cost-Effectiveness AnalysisCR Shi and VE Nambudiri

Article 59 Research Techniques Made Simple: An Introduction to Use and Analysis of Big Datain DermatologyMR Wehner, KA Levandoski, M Kulldorff and MM Asgari

Article 60 Research Techniques Made Simple: Bioinformatics for Genome-Scale BiologyAC Foulkes, DS Watson, CEM Griffiths, RB Warren, W Huber and MR Barnes

i Answers

xiii Supplementary Information

JID Connector: Your Link to Contentand Colleagues

A ccording to Malcolm Gladwell (2000),author of The Tipping Point, “Connec-tors [are] people with a special gift for

bringing the world together..” We all knowthese people, and they just seem to initiatecontacts among individuals or groups that buildnew, and oftentimes lasting social and profes-sional networks. It is therefore fitting that one ofthe quintessential “Connectors” in investigativedermatology, Barbara Gilchrest, created the JIDConnector at the beginning of her tenure asJID Editor on the 75th anniversary of the JID in2012. In this effort, Barbara led a challenge toall of us in dermatology to utilize the JID as aconduit for connectivity among cutaneousbiology investigators, academicians, clinicians,and trainees. In the years that followed, the Edi-tors of the JID Connector have focused oncreating content for clinicians and trainees thatmakes basic science more accessible and rele-vant. We would like to thank Lilly for their initial,and continued support, of these efforts.

It is a privilege to serve as the Editor of the JIDConnector, following in the footsteps of KavithaReddy, MD, of Boston University. The Connectorwill continue to highlight the highly popular andeducational Research Techniques Made Simpleseries. This section will remain under the direc-tion of our Coordinating Editor, Jodi Johnson(Northwestern), who is now joined in this role byContributing Editor Brian Kim (Washington Uni-versity). Kudos to all who contribute to thesepublications, and please know that you arecontinuing in the spirit of the Connector in a veryspecial way—bringing your expertise in investi-gative techniques to other investigators, clini-cians, and trainees.

We will also continue the “Meet the Investi-gator” feature. Ayman Grada (Boston University)has done a terrific job featuring our juniorinvestigators in the field and bringing theseamazing individuals to life. Ayman, pleaseaccept our thanks in championing these seg-ments and we look forward to “meeting” manymore rising stars. And, a special thanks to BobDellavalle (University of Colorado Denver), whohas graciously agreed to continue to serve as ourPodcast Editor.

The SnapShot Dx and the Cells to SurgeryQuizzes “connect” clinical dermatology to sci-entific discovery. We are truly fortunate to haveKeyvan Nouri and Mariya Miteva, faculty fromthe University of Miami, Department of Derma-tology, continue in their roles as ContributingEditors. These quizzes are frequently accessedand appeal to trainees and experienced clini-cians alike. In the spirit of expanding theConnector network, we have extended an invi-tation to faculty from several dermatology de-partments to serve as Contributing Editors. Ournew Contributing Editors include Eva Hurst andMilan Anadkat (Washington University); EmilyChu and Jeremy Etzkorn (Unviersity of Pennsyl-vania); and Ben Chong and Rajiv Nijhawan(University of Texas Southwestern). Going for-ward, our quizzes will be based on JID publi-cations from the previous 6 months, allowingmore time for quiz preparation, review, andrevision. To that point, it is impressive thatKeyvan and Mariya have turned these aroundeach month with such quality and consistency.We sincerely appreciate everyone’s willingnessto serve—it is a great opportunity for faculty andtrainees from various institutions to deliver clin-ical “pearls” that have been extracted from thework of the “family” of investigators who publishin our journal.

We have also put much thought into how to beeven more purposeful in incorporating new waysto connect and share information, whileproviding further service to our (expanding) JIDcommunity. Some of our proposals are listedbelow:

1. With the help of our publishers at Elsevier, theConnector is considering the development ofa new venture called “Research Methods,”where frequently cited protocols could besubmitted, published, and archived inconjunction with the Research TechniquesMade Simple articles under the broad desig-nation of “Research Techniques.” These“Method” publications would follow specificguidelines that have been similarly adoptedby other scientific journals.

2. We are also encouraging all JID authors toinclude their Twitter handles with the contactinformation submitted with their manu-scripts—it will provide an easy way for othersto give these individuals a quick “shout out,”

Journal of Investigative Dermatology (2017) 137, 2243e2244.doi:10.1016/j.jid.2017.09.012

ª 2017 The Author. Published by Elsevier, Inc. on behalf of the Society for Investigative Dermatology. www.jidonline.org 2243

EDITORIAL

initiate a conversation surrounding their work, and, whoknows, possibly spur new collaborations. Along theselines, we encourage our readership and trainees to con-nect to the JID via social media either through Twitter byfollowing @JIDJournal or through Facebook at https://www.facebook.com/JournalOfInvestigativeDermatology/.An easy way to get started is to publish a tweet encour-aging others in the JID community to get involved insharing engaging JID content with each other, by using thislink https://goo.gl/FkMN3P to help craft your first Tweet!The JID is an international journal and our contributorsand readership span the globe. By connecting throughsocial media you will increase your connectivity to col-leagues and trainees both nationally and internationally.Let the JID serve as your “Connector” to these interestingand engaged individuals.

3. We are also exploring the use of Elsevier’s “CollectionsTool” that allows collections of articles to be devel-oped thematically—possibilities include specificdisease-focused articles that shaped the field, recentpapers of special interest, and Editor’s Picks from theJID archives.

I write this on 21 August 2017—a day when scientists andnon-scientists alike are inspired by an astronomical spectaclethat reminds us all of how interconnected we are with eachother and the cosmos. The path of totality crosses acrossMissouri, touching the edges of St. Louis as it passes by. I amsimilarly inspired, with you, to work toward new and invig-orated connectivity across a growing JID community!

CONFLICT OF INTERESTThe author states no conflict of interest.

Lynn A. Cornelius1

JID Connector Editor1Department of Dermatology, Washington University St. Louis,St. Louis, Missouri, USA

Correspondence: Lynn A. Cornelius, Division of Dermatology, WashingtonUniversity School of Medicine, 660 South Euclid Ave, St. Louis, Missouri63130. E-mail: [email protected]

REFERENCE

Gladwell M. The tipping point: How little things can make a big difference.Boston, MA: Little, Brown and Company; 2000.

This is a reprint of an article that originally appeared in the November 2017 issue of the Journal of Investigative Dermatology. It retains its original paginationhere. For citation purposes, please use these original publication details: Cornelius LA. JID Connector: Your Link to Content and Colleagues. J Invest Dermatol2017;137(11):2243e2244. doi:10.1016/j.jid.2017.09.012

EDITORIAL

Journal of Investigative Dermatology (2017), Volume 1372244

The Challenge of EffectiveCommunication among Scientists

A t a time when scientific knowledge israpidly growing within a multitude ofsubspecialties, it is difficult to remain

familiar with the language, techniques, and rele-vance of all available research approaches. Still, acommon language is essential for successfultranslational and team research. Having recentlymoved to Amsterdam (The Netherlands), I amacutely aware that communicating effectively isnot like riding a bicycle. That is, communicatingsuccessfully once does not guarantee mastery forall time. Communication is a highly complex,dynamic, and evolving skill.

Ever since starting my medical degree inLisbon (Portugal), I consistently sought to gainexperience in a wide range of basic and trans-lational research areas. I would spend holidaysand weekends working in a lab, driven by theidea of possibly generating results that couldcontribute in any small way to understanding ofhuman biology. After completing my medicaldegree, my passion for research led me to moveto Boston (USA) to focus exclusively on trans-lational research. The experiences of working indifferent countries and research areas as well aspresenting and discussing diverse clinical cases,new research hypotheses, methodologies, re-sults, and even teaching younger students mademe believe that I had more or less mastered theart of communication.

However, on recently moving to Amsterdam topursue a residency program in dermatology, Ihave been confronted anew with the importanceand difficulty of communicating effectively. Iexperienced the obvious language barrier, stilllearning Dutch, but also struggled to grab theinterest of my colleagues when discussing theimpact of my studies and results. My researchseemed too abstract and complex because thetechniques that I used were barely known withinmy new institution.

The Research Techniques Made Simple(RTMS) manuscripts were precisely the tool I wasmissing to bridge the gap between the work ofthe lab-based researchers and clinicians. Theopportunity to co-author three manuscripts forthe RTMS series proved to be a much moreenriching experience than I could have ever

anticipated. In a two-part series, we reviewed theexperimental methodology for single-cell masscytometry (CyTOF) (Matos et al., 2017a) andCyTOF’s analysis tools for decrypting thecomplexity of biological systems (Matos et al.,2017b). CyTOF enables the detection andquantification of more than 40 markers at asingle-cell resolution. The 135 available detec-tion channels allow a simultaneous study ofadditional characteristics within complex bio-logical systems across millions of cells. A thirdmanuscript reviewed the technique of high-throughput deep sequencing of the T-cell re-ceptor (Matos et al., 2017c), which allowssensitive and accurate identification and quanti-fication of every distinct T-cell clone presentwithin any biological sample.

Writing these RTMS manuscripts forced me tobe creative in the way I translated these complextechniques into a language understandable bythe diverse readership of the Journal of Investi-gative Dermatology. The RTMS format includesnot only the manuscript but also bullet pointshighlighting the pros and cons, examples of howto use the technique for diverse research ques-tions, a quiz to test mastery of the informationprovided, and a PowerPoint presentation allow-ing readers to promptly share and teach thetechniques. Hence, a single manuscript chal-lenges authors to communicate through differentformats, not only by writing but also by designingcoherent explanatory figures and creating thefoundation of an oral presentation. Workingthrough authoring these different componentsmade me further realize the potential, sensitivity,and applicability of those techniques—knowl-edge that greatly benefited my forthcomingresearch. The manuscripts’ writing process wasvery gratifying, with continuous help from theEditor, Jodi L. Johnson, PhD, and ManagingEditor, Elizabeth Blalock, and the chance todevelop appealing figures with the Journal ofInvestigative Dermatology illustrators. The crit-ical comments provided by both the reviewersand editors really made me think deeply abouthow to properly write review manuscripts in aconcise, engaging, and comprehensible way,helping to shape our manuscripts to be their best.

After the manuscripts’ publication and pre-senting the corresponding PowerPoints at journalclub sessions, I noticed an increasing attendance

Journal of Investigative Dermatology (2017) 137, e183ee184.doi:10.1016/j.jid.2017.09.018

ª 2017 The Author. Published by Elsevier, Inc. on behalf of the Society for Investigative Dermatology. www.jidonline.org e183

� EDITORIAL

to my research presentations by both lab colleagues andclinicians. The audience became interactive with numerousquestions and observations. It was obvious that what wasonce a breakdown in communication had turned around, andfluency in the scientific language had been restored. Under-standing applications of the research methodology made thestudies appear more interesting and accessible, therebystrengthening the collaborative research within our depart-ment. In addition, these manuscripts led to prolific collabo-rations between distinct academic institutions and industry,and an invitation to give lectures to Masters’ students at auniversity. I also feel that these manuscripts strengthened asubsequent successful grant submission by attesting to myunderstanding of the techniques proposed in the application.It is with great delight that I keep reading the RTMS articleswritten by so many excellent author teams, learning andrevisiting various research techniques. I have tested myself onmany of those techniques and included them in my researchprojects. I hope that fellow researchers are now aware of thetechniques we covered and that these become more widelyapplied within investigative dermatology.

The challenge of effective communication persists amongscientists, in part because of uneven knowledge of funda-mental basic science concepts and laboratory techniques.The unparalleled rapid development of science has resultedin innovative treatments and cutting-edge technologies being

used in health care today. Thus, a basic common scienceliteracy is essential to help each individual successfullycommunicate his or her ideas and to foster state-of-the-artdermatologic care. The RTMS manuscripts not onlyprovided me with new professional opportunities, but alsobecame a language tool. These manuscripts simultaneouslyshare fundamental research knowledge and the vocabularynecessary to create effective communication betweenclinicians and researchers.

CONFLICT OF INTERESTThe author states no conflict of interest.

Tiago R. Matos11Department of Dermatology, Academic Medical Center, Universityof Amsterdam, Amsterdam, The Netherland

Correspondence: Tiago R. Matos, Department of Dermatology, AcademicMedical Center, University of Amsterdam, Meibergdreef 9, 1105 AZ,Amsterdam, The Netherlands. E-mail: [email protected]

REFERENCES

Matos TR, de Rie MA, Teunissen MB. RTMS: high-throughput sequencing ofthe T-cell receptor. J Invest Dermatol 2017c;137:e131e8.

Matos TR, Hongye L, Ritz J. RTMS: mass cytometry analysis tools fordecrypting the complexity of biological systems. J Invest Dermatol2017a;137:e43e51.

Matos TR, Hongye L, Ritz J. RTMS: experimental methodology for single-cellmass cytometry. J Invest Dermatol 2017b;137:e31e8.

This is a reprint of an article that originally appeared in the November 2017 issue of the Journal of Investigative Dermatology. It retains its original paginationhere. For citation purposes, please use these original publication details: Matos TR. The Challenge of Effective Communication among Scientists. J InvestDermatol 2017;137(11):e183ee184. doi:10.1016/j.jid.2017.09.018

EDITORIAL �

Journal of Investigative Dermatology (2017), Volume 137e184

Research Techniques Made Simple: LaserCapture Microdissection in Cutaneous ResearchEstela Chen Gonzalez1 and Jean Suh McGee1

In cutaneous research, we aim to study the molecular signature of a diseased tissue. However, such a study ismet with obstacles due to the inherent heterogeneous nature of tissues because multiple cell types residewithin a tissue. Furthermore, there is cellular communication between the tissue and the neighboring extra-cellular matrix. Laser capture microdissection is a powerful technique that allows researchers to isolate cells ofinterest from any tissue using a laser source under microscopic visualization, thereby circumventing the issueof tissue heterogeneity. Target cells from fixed preparations can be extracted and examined without disturbingthe tissue structure. In live cultures, a subpopulation of cells can be extracted in real time with minimaldisturbance of cellular communication and molecular signatures. Here we describe the basic principles of thetechnique, the different types of laser capture microdissection, and the subsequent downstream analyses. Thisarticle will also discuss how the technique has been employed in cutaneous research, as well as futuredirections.

Journal of Investigative Dermatology (2016) 136, e99ee103; doi:10.1016/j.jid.2016.08.005

CME Activity Dates: September 21, 2016Expiration Date: September 20, 2017Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: All speakers,planning committee members, CME committee members andstaff involved with this activity as content validation reviewershave no financial relationship(s) with commercial interests todisclose relative to the content of this CME activity.

Commercial Support Acknowledgment: This CME activity issupported by an educational grant from Lilly USA, LLC.

Description: This article, designed for dermatologists, resi-dents, fellows, and related healthcare providers, seeks toreduce the growing divide between dermatology clinicalpractice and the basic science/current research methodologieson which many diagnostic and therapeutic advances are built.

Objectives: At the conclusion of this activity, learners shouldbe better able to:� Recognize the newest techniques in biomedical research.� Describe how these techniques can be utilized and theirlimitations.

� Describe the potential impact of these techniques.

CME Accreditation and Credit Designation: This activity hasbeen planned and implemented in accordance with theaccreditation requirements and policies of the AccreditationCouncil for Continuing Medical Education through the jointprovidership of William Beaumont Hospital and the Societyfor Investigative Dermatology. William Beaumont Hospital isaccredited by the ACCME to provide continuing medicaleducation for physicians.William Beaumont Hospital designates this enduring materialfor a maximum of 1.0 AMA PRA Category 1 Credit(s)�.Physicians should claim only the credit commensurate withthe extent of their participation in the activity.

Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Oct16. Fax or other copies will not beaccepted. To receive credits, learners must review the CMEaccreditation information; view the entire article, complete thepost-test with a minimum performance level of 60%; andcomplete the online evaluation form in order to claimCME credit. For questions about CME credit email [email protected].

PRINCIPLES OF LCMEmmert-Buck et al. developed laser capture microdissection(LCM) in 1996 at the National Institutes of Health to supportthe Cancer Genome Anatomy Project (Emmert-Buck et al.,1996). The goal of the Cancer Genome Anatomy Project

was to develop a high caliber expression library of humancancers and precancerous lesions. Such an undertakingcalled for an isolation of specific tumor cells from solidtumors without disturbing the integrity of biomolecules (DNA,RNA, protein) within the collected cells. To accomplish thistask, the team developed a microscope-based microdissec-tion platform, now known as LCM.

LCM is a technology used to isolate a single cell ora specific cell population from a heterogeneous tissuesection, cytological preparation, or live cell culture bydirect visualization of the cells (Emmert-Buck et al., 1996).There are two main classes of laser capture microdissectionsystems: infrared LCM (IR-LCM) and ultraviolet LCM (UV-LCM). IR-LCM instruments are available as manual or

1Department of Dermatology, Boston University School of Medicine, Boston,Massachusetts, USA

Correspondence: Jean Suh McGee, Department of Dermatology, BostonUniversity School of Medicine, Boston Medical Center, Boston,Massachusetts, 02118, USA. E-mail: [email protected]

Abbreviations: EVA, ethylene-vinyl acetate; FFPE, formalin-fixed paraffin-embedded; IHC, immunohistochemical; IR-LCM, infrared LCM; LCM, lasercapture microdissection; UV-LCM, ultraviolet LCM

ª 2016 The Authors. Published by Elsevier, Inc. on behalf of the Society for Investigative Dermatology. www.jidonline.org e99

� RESEARCH TECHNIQUES MADE SIMPLE

automated systems. Other available platforms include an IR/UV combined system. Regardless of the platform used, theprincipal steps of LCM are the visualization of cells bymicroscopy, the transfer of laser energy to isolated cells ofinterest, and the collection of cells of interest from the tissuesection (Espina et al., 2006).

LCM can be applied to a variety of preparations includinghistological specimens (formalin-fixed paraffin-embedded[FFPE] or fresh-frozen sections) and cytology preparations(direct smears, touch preps, or cell block). Samples can bestained, unstained, or tagged by immunohistochemistry.Frozen tissue effectively preserves RNA, DNA, and proteins,but may distort histologic differentiation. The standardmethod for preservation of tissue morphology is FFPE. How-ever, it causes unwanted crosslinking between proteins andnucleic acids and proteins are not extractable from FFPEsamples (Liu, 2010).

INFRARED LCMThis technique uses a lower energy laser in the IR spectrum of810 hm to activate a 100-mm, transparent, and thermosensi-tive film containing ethylene-vinyl acetate (EVA) saturatedwith a dye that absorbs IR laser energy. The thermosensitivefilm is positioned over a stained frozen or FFPE tissue section,which can be visualized with an inverted microscope. Themicroscope is connected to a computer for laser control andimage archiving. A laser beam is directed at the cells of in-terest, but only the thermosensitive film absorbs the energy ofthe laser. Consequently, there is no damage to the underlyingcells or biomolecules within the cells. The focused pulse fromthe IR laser produces a conformational change in the EVA

polymer, which becomes fixed to the cells of interest under-neath. The adhesive force of cells to the film exceeds theadhesive force to the slide, enabling selective removal of cells(Figure 1). Once removed, the cells are transferred to amicrocentrifuge tube containing DNA, RNA, or enzymebuffer where the cellular material detaches from the film(Emmert-Buck et al., 1996).

ULTRAVIOLET LCMThe LCM technique using an ultraviolet cutting laser is alsoknown as laser microbeam microdissection. Laser microbeammicrodissection uses a high-energy UV laser (355 hm)capable of cutting tissues. The laser is used to cut around thecells of interest, in contrast to IR capture that focuses the laseron the cells. In UV-LCM, the surrounding unwanted tissue isphotoablated whereas the desired cells remain intact (Schutzeand Lahr, 1998). Target cells are retrieved through a variety ofmethods depending on the instrument. The cells can becollected by photonic pressure from a second laser shot thatcatapults them into a collection cap (PALM/Zeiss system,Oberkochen, Germany), by gravity that deposits them into acollection cap (Leica Microsystems, Wetzlar, Germany), or bya sticky cap to which they are glued after LCM (MMI In-struments, Eching, Germany) (Espina et al., 2006; Liu et al.,2014).

IMMUNO-LCMImmuno-LCM uses immunohistochemical (IHC) staining toidentify and isolate a specific cell population that is chal-lenging to discern visually. For instance, cells that aremorphologically similar but immunologically distinct such asB and T lymphocytes can be distinguished using IHC stainingfor a type-specific antigen before LCM. The common IHCreagents do not adversely affect downstream analysis usingassays such as PCR (Fend et al., 1999). RNA degradation fromIHC staining can be prevented by prelabeling cells, forexample, by injecting animals with a fluorogold label beforeharvesting the tissue (Yao et al., 2005). However, immuno-LCM is not optimal for studying protein expression, as theprotein of interest is bound by antibodies (both primary andsecondary) during IHC staining. These bound antibodies caninterfere with downstream methods such as polyacylamidegel electrophoresis, western blotting, and mass spectrometry.

DOWNSTREAM ANALYSISOnce collected, DNA can be subjected to sequencing, DNAmethylation assays, and loss of heterozygosity studies. RNAcan be used for sequencing and constructing a cDNA library,as well as in gene expression arrays, real-time RT-PCR, andquantitative PCR. Protein can be studied with westernblotting, 2D gel electrophoresis, mass spectrometry, andreverse-phase protein microarray. It is important to note thatproteomic tests require more material than DNA and RNAanalyses (Espina et al., 2006).

ALTERNATIVE METHODSAn alternative approach to isolate and concentrate cells of interest isby cell sorting techniques such as FACS and magnetic-activated cellsorting. These methods require that cells be processed in fluidsuspensions, which are suitable for the analysis of hematopoieticand circulating cells but not ideal in the analysis of solid tissue.

WHAT LCM DOES� LCM is a technique that isolates cells of interestor even a single cell from a heterogeneous tissuespecimen using a laser source under microscopicvisualization.

� Cells isolated by LCM contain intact DNA, RNA,and proteins for downstream molecular analysis.

� LCM is capable of isolating diseased cells fromthe primary lesion without altering theirmolecular signatures.

� LCM can be applied to a wide variety of tissueand cellular preparations.

LIMITATIONS� In the absence of a cover slip, the opticalresolution of complex tissues may be limited.

� The reliance on visual identification of targetcells creates room for human error.

� Unlike IR-LCM, UV-LCM is limited by thepotential to induce UV damage in the circum-ferential cells, which may be subsequentlycollected for analysis.

RESEARCH TECHNIQUES MADE SIMPLE �


Intercellular adhesions and the extracellular matrix prevent thedisaggregation of cells. Disruption of intact tissue alters gene andprotein expression and renders the subsequent interpretation ofmolecular studies difficult (Holle et al., 2016). In this regard, LCM isadvantageous because it is capable of isolating diseased cells fromthe primary lesion without altering molecular signatures.

ADVANTAGES AND LIMITATIONSAdvantages of LCM include speed, precision, ease of use, andversatility. With the IR-LCM, because laser pulses are deliv-ered through an optical cap, the pulses can be repeatedacross the cap surface to collect thousands of cells per cap.IR-LCM can also be used to collect cells sequentially from thesame tissue section, as the directed pulse does not alteradjacent cells. In contrast, the high-energy UV laser is usefulfor microdissection of thick specimens (up to 200 mm thick-ness). The UV-LCM laser has a finer beam diameter (0.5 mm)compared with that of the IR-LCM laser (7.5 mm), makingUV-LCM more precise in microdissection of a single cell.However, UV-LCM is more time consuming. Additionally,UV-LCM can induce UV damage in neighboring cells, whichmay limit analysis of sequentially collected cells (Espina et al.,2006).

There are a few limitations that apply to both IR-LCM andUV-LCM. Both require the use of noncoverslip slides to allow

physical access to the tissue surface for microdissection. Inthe absence of a cover slip, the tissue section has a limitedoptical resolution, which can make precise dissection ofcomplex tissues very difficult. Staining the cell population tobe isolated or avoided is a common way to address this issue.Another major limitation of LCM is the reliance on visualdiscrimination of the target cells. For large-scale molecularprofiling projects that involve lesions or cells that are difficultto discern, consultation with a trained pathologist may berequired. Further limitations or errors may result from theperishability of tissue specimens, as well as tissue stainingprotocols and fixation techniques that are not compatiblewith the downstream analysis. For instance, unwantedcrosslinking of nucleic acids and proteins in FFPE tissuesections limits downstream analysis of proteins and RNA(Espina et al., 2006, Fend and Raffeld, 2000).

LCM IN CUTANEOUS RESEARCHLCM was employed by Masterson et al. (2014) to identifyprognostic biomarkers of the rare and poorly understoodcutaneous malignancy, Merkel cell carcinoma. IR-LCM andUV-LCM were used to isolate tumor cells for subsequent RNAexpression analysis using Affymetrix GeneChip arrays. Inaddition to the 191 genes demonstrating differential expres-sion, A2 group X, kinesin family member 3A, tumor protein

Figure 1. Schematic representation ofinfrared laser capture microdissection(IR-LCM). (a) The IR-LCM setupincludes an inverted microscope,an infrared laser, a cap withthermosensitive film on the bottomsurface, and a tissue section ona slide without a cover slip. (b)Thermosensitive ethylene-vinylacetate (EVA) film under the cap.An IR laser melts the EVA film. Cellsof interest are captured by polymer-cell adhesion. Reprinted from Espinaet al. (2006) with permission fromMacmillan Publishers Ltd, and fromFend and Raffeld (2000) withpermission from BMJ PublishingGroup Ltd.

Figure 2. Quantitation of mRNA levelby real-time PCR after laser capturemicrodissection. In both young andaged skin, the epidermis and thedermis were separated and collectedby laser capture microdissection.The levels of mRNA for both PTGES1(a) and COX2 (b) are increased in thedermis of the aged skin compared withthat of the young skin, whereas therewas no difference between the youngand old skin in expression of the twogenes in the epidermis. *P < 0.05.Reprinted from Li et al. (2015) withpermission from Elsevier.

www.jidonline.org e101


D52, mucin 1, and KIT were identified as novel genes upre-gulated in the tumor cells of the patients with poor prognoses.New clinical prognostic markers and therapeutic modalitiesmay be discovered with continued investigation of thesepromising targets (Masterson et al., 2014).

Wouters et al. (2014) used LCM technology, cDNA libraryconstruction, and qRT-PCR to examine the molecular

phenotype of melanoma cells undergoing metastatic trans-formation. Fibronectin 1 is an epithelial-to-mesenchymaltransition marker of melanoma. Fibronectin 1 high mela-noma cells were found to reside in hypoxic environmentssuch as melanoma lesions with ischemic necrosis. Thisassociation suggests that the hypoxic tumor microenviron-ment may induce melanoma cells to become migratory andmore invasive (Wouters et al., 2014).

Li et al. (2015) used LCM and subsequent qPCR to studychanges in aging skin. Simply comparing the tissue samplesbetween young and old skin did not initially yield any sig-nificant differences. However, using LCM to separate thedermis from the epidermis, followed by qPCR to assess geneexpression, Li et al. showed that dermal expression of PTGES1and COX2 genes was significantly higher in aged skin(Figure 2). PTGES1 and COX2 contribute to aging skin byincreasing levels of PGE2, which inhibits collagen productionleading to thinning of the skin. Notably, the therapeuticinhibition of PGE2 may help combat age-associated collagendecrease in human skin (Li et al., 2015).

Goldstein et al. (2015) used LCM, RNA amplification, andqRT-PCR to better understand the effectiveness of narrowband UVB in the treatment of vitiligo. Narrow band UVBtreatment was correlated with an increase in gene transcrip-tion and subsequent protein expression of certain markers ofmelanocyte differentiation in treated skin. The examination ofmolecular changes in activated and mobilized melanocytes isessential for understanding the mechanism of this autoim-mune condition as well as for the development of moreevidence-based therapies (Goldstein et al., 2015).

SUMMARY AND FUTURE DIRECTIONSLCM is used in cutaneous research to study molecular profilesof a specific cell or population within heterogeneous tissue.Future directions for improvement of the technique includeautomation to increase efficiency and ease of use, integrationof cell recognition software to reduce human errors, andoptimization of protocols for sample preparation to aidmicrodissection itself and to preserve the integrity of thebiomolecules for subsequent studies. In the near future, wecan anticipate the use of LCM in clinical and research settingswith a wide range of applications.

CONFLICT OF INTERESTThe authors state no conflict of interest.

SUPPLEMENTARY MATERIALSupplementary material is linked to this paper. Teaching slides are availableas supplementary material.

REFERENCESEmmert-Buck MR, Bonner RF, Smith PD, Chuaqui RF, Zhuang Z,

Goldstein SR, et al. Laser capture microdissection. Science 1996;274:998e1001.

Espina V, Wulfkuhle JD, Calvert VS, VanMeter A, Zhou W, Coukos G, et al.Laser-capture microdissection. Nat Protoc 2006;1:586e603.

Fend F, Emmert-Buck MR, Chuaqui R, Cole K, Lee J, Liotta LA, et al. Immuno-LCM: laser capture microdissection of immunostained frozen sections formRNA analysis. Am J Pathol 1999;154:61e6.

Fend F, Raffeld M. Laser capture microdissection in pathology. J Clin Pathol2000;53:666e72.

Goldstein NB, Koster MI, Hoaglin LG, Spoelstra NS, Kechris KJ, Robinson SE,et al. Narrow band ultraviolet B treatment for human vitiligo is associated

MULTIPLE CHOICE QUESTIONS1. What does LCM do?

A. Sorts cells based on morphology (size,granularity, density)

B. Sorts cells with either IR or UV lasertechnology

C. Collects cells of interest with lasertechnology

D. Photoablates cells of interest with lasertechnology

2. What is the thinnest diameter of the UV-LCMlaser beam?

A. 0.5 mm

B. 5.0 mm

C. 7.5 mm

D. 30 mm

3. Which technique uses a thermosensitive EVAfilm to sequester cells of interest?

A. IR-LCM

B. UV-LCM

C. Laser microbeam microdissection

D. FACS

4. In the absence of a cover slip, the opticalresolution of complex tissues may be limited.This issue can be addressed by:

A. Using a temporary cover slip

B. Increasing the thickness of the tissue section

C. Decreasing the thickness of the tissuesection

D. Staining with immunohistochemistry

5. Which of the following statements regarding theLCM technique is NOT true?

A. UV-LCM is better suited for single cellmicrodissection

B. IR-LCM is more time consuming thanUV-LCM

C. The EVAmembrane undergoes conformationalchange when exposed to IR laser energy

D. The downstream analysis of proteins islimited in FFPE tissue sections due to unde-sirable protein and nucleic acid crosslinking



with proliferation, migration, and differentiation of melanocyte pre-cursors. J Invest Dermatol 2015;135:2068e76.

Holle AW, Young JL, Spatz JP. In vitro cancer cell-ECM interactions informin vivo cancer treatment. Adv Drug Deliv Rev 2016;97:270e9.

Li Y, Lei D, Swindell WR, Xia W,Weng S, Fu J, et al. Age-associated increase inskin fibroblast-derived prostaglandin E2 contributes to reduced collagenlevels in elderly human skin. J Invest Dermatol 2015;135:2181e8.

Liu A. Laser capture microdissection in the tissue biorepository. J Biomol Tech2010;21:120e5.

Liu H, McDowell TL, Hanson NE, Tang X, Fujimoto J, Rodriguez-Canales J.Laser capture microdissection for the investigative pathologist. Vet Pathol2014;51:257e69.

Masterson L, Thibodeau BJ, Fortier LE, Geddes TJ, Pruetz BL, Malhotra R, et al.Gene expression differences predict treatment outcome of Merkel cellcarcinoma patients. J Skin Cancer 2014;2014:596459.

Schutze K, Lahr G. Identification of expressed genes by laser-mediatedmanipulation of single cells. Nat Biotechnol 1998;16:737e42.

Wouters J, Stas M, Govaere O, Barrette K, Dudek A, Vankelecom H, et al.A novel hypoxia-associated subset of FN1 high MITF low melanomacells: identification, characterization, and prognostic value. Mod Pathol2014;27:1088e100.

Yao F, Yu F, Gong L, Taube D, Rao DD, MacKenzie RG. Microarray analysisof fluoro-gold labeled rat dopamine neurons harvested by laser capturemicrodissection. J Neurosci Methods 2005;143:95e106.

This is a reprint of an article that originally appeared in the October 2016 issue of the Journal of Investigative Dermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Gonzalez EC,McGee JS. Research TechniquesMade Simple: Laser CaptureMicrodissectionin Cutaneous Research. J Invest Dermatol 2016;136(10):e99ee103. doi:10.1016/j.jid.2016.08.005



Research Techniques Made Simple: AssessingRisk of Bias in Systematic ReviewsAaron M. Drucker1, Patrick Fleming2 and An-Wen Chan3

Systematic reviews are increasingly utilized in the medical literature to summarize available evidence on aresearch question. Like other studies, systematic reviews are at risk for bias from a number of sources. Asystematic review should be based on a formal protocol developed and made publicly available before theconduct of the review; deviations from a protocol with selective presentation of data can result in reportingbias. Evidence selection bias occurs when a systematic review does not identify all available data on a topic. Thiscan arise from publication bias, where data from statistically significant studies are more likely to be publishedthan those that are not statistically significant. Systematic reviews are also susceptible to bias that arises in anyof the included primary studies, each of which needs to be critically appraised. Finally, competing interests canlead to bias in favor of a particular intervention. Awareness of these sources of bias is important for authors andconsumers of the scientific literature as they conduct and read systematic reviews and incorporate theirfindings into clinical practice and policy making.


CME Activity Dates: October 22, 2016Expiration Date: October 21, 2017Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: All authors, plan-ning committee members, CME committee members and staffinvolved with this activity as content validation reviewershave no financial relationship(s) with commercial interests todisclose relative to the content of this CME activity.


Description: This article, designed for dermatologists, resi-dents, fellows, and related healthcare providers, seeks toreduce the growing divide between dermatology clinicalpractice and the basic science/current research methodolo-gies on which many diagnostic and therapeutic advances arebuilt.




Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Nov16 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCMEcredit. The CME credit code for this activity is: 21310. Forquestions about CME credit email [email protected].

INTRODUCTIONSystematic reviews are comprehensive overviews of theexisting evidence on a specific research question. If appro-priate, they can include a pooled statistical summary of

available data called a meta-analysis. Systematic reviews andmeta-analyses are becoming increasingly prevalent in medi-cal journals; a PubMed search using “systematic reviews” as apublication type filter in the Journal of Investigative Derma-tology, Journal of the American Academy of Dermatology,JAMA Dermatology, and British Journal of Dermatologyreturned 7 results published in 2010 compared with 27 in2015, although these figures may capture some narrative re-views as well. The results of systematic reviews can helpguide clinicians, patients, and policy makers by providingmore precise and comprehensive information than individualstudies alone. They can also be used to identify gaps inknowledge and suggest areas for future research. A previouspaper in the Research Techniques Made Simple series

1Department of Dermatology, Warren Alpert Medical School, BrownUniversity, Providence, Rhode Island, USA; 2Division of Dermatology,University of Toronto, Toronto, Ontario, Canada; and 3Division ofDermatology, University of Toronto and Women’s College Research Institute,Toronto, Ontario, Canada

Correspondence: Aaron M. Drucker, Department of Dermatology, BrownUniversity, Box G-D, Providence, Rhode Island 02903, USA.E-mail:[email protected]

Abbreviation: PRISMA-P, Preferred Reporting Items for Systematic Reviewsand Meta-Analyses Protocol

Journal of Investigative Dermatology (2016), Volume 136 ª 2016 The Authors. Published by Elsevier, Inc. on behalf of the Society for Investigative Dermatology.e109


discussed the methodology and utility of systematic reviewsand meta-analyses in dermatology (Abuabara et al., 2012). Inthis article, we discuss the various types of bias that can occurin systematic reviews so that they can be avoided oracknowledged by review authors, and critically assessed byusers of the dermatology literature.

REPORTING BIAS AND THE IMPORTANCE OF PROTOCOLSReporting bias refers to the selective dissemination of researchfindings based on the nature of the results (Kirkham et al.,2010). For example, the choice of review outcomes orincluded studies might be changed to highlight significantfindings. The selective inclusion of outcomes or studies withmore significant results after exploring the data will bias theresults of the review toward positive findings.

To help identify and deter reporting bias, it is critical forsystematic reviews to be conducted in accordance with aprotocol written before beginning the review. As with othertypes of research, the protocol defines the researchquestion—including the population, intervention or expo-sure, and outcomes of interest—and describes the method-ology in sufficient detail to allow replication by others. Toavoid a data-driven hypothesis, the research question shouldbe formulated in advance based primarily on clinical rele-vance rather than knowledge of available evidence. ThePreferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA-P) statement is a valuableevidence-based resource that defines the key content of areview protocol, including a description of search strategiesand data sources, eligibility criteria, method of studyscreening and selection, primary and secondary outcomes,data extraction, and any planned analyses (Shamseer et al.,2015). Similarly, it is recommended that authors adhere tothe comprehensive PRISMA guidance when actually prepar-ing reports of systematic reviews (Moher et al., 2009). ThePRISMA statement contains an evidence-based checklist ofitems to address in the manuscript itself and has beenendorsed by many medical journals.

Public availability of the review protocol facilitates criticalappraisal of the methods and identification of protocol

deviations and selective reporting of results. It is importantthat protocols be prospectively registered online atPROSPERO—an online database of systematic reviews (http://www.crd.york.ac.uk/prospero/). Alternatively, protocols maybe published in their entirety (as with Cochrane reviews).Subsequent publications of systematic reviews should statewhere the protocol was registered and where a copy of theprotocol can be found. Protocol deviations do not necessarilylead to bias but must be explained in the Methods section ofthe systematic review report. For example, the search strategymight be modified if the results obtained from the originalsearch were too broad or narrow. A recently published meta-analysis by Atzmony et al. (2015) concerning adjuvant

SUMMARY POINTS� It is important for authors of systematic reviews to:

B Register a protocol before conducting thereview and explain any deviations from it

B Utilize the PRISMA-P and PRISMA guidanceB Search comprehensively beyond thepublished literature

B Assess risk of bias in included primary studiesB Disclose competing interests.

� It is important for consumers of systematicreviews to be aware of those same issues whenreading review reports and when interpreting theimplications of their findings on clinical practiceand policy.

A

B

Figure 1. Representative examples of funnel plots. Funnel plots are scatterplots representing effect estimates on the x-axis compared with study precision(often the standard error of effect estimates) on the y-axis. (a) A symmetricalfunnel plot adapted from a meta-analysis on the use of sirolimus in renaltransplant recipients (Knoll et al., 2014). In this plot, the x-axis (log hazardratio [HR]) is a proxy for effect estimates and the y-axis (standard error) isinversely related to the study sample size. The data points (red circles) eachrefer to a specific study. In a symmetrical funnel plot, the data points should bescattered symmetrically within the funnel (blue lines), suggesting a low risk ofpublication bias. (b) In this fictional plot (modified from Knoll et al., 2014),there is clear asymmetry within the funnel, with missing data points fromunpublished trials in the lower-left portion of the funnel, suggesting a high riskof publication bias. Reproduced from Knoll et al., 2014 with permission fromBMJ Publishing Group Ltd.



therapy for pemphigus was reported based on PRISMA andwas registered on PROSPERO (http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID¼CRD42014014160).

EVIDENCE SELECTION BIASA key goal of a systematic review is to identify all relevantdata to answer the research question. Data sources caninclude journal databases (e.g., PubMed), trial registries (e.g.,clinicaltrials.gov), and direct communication with authors(Chan, 2012). Although electronic databases such as PubMedand EMBASE have made it much easier to identify publishedarticles, systematic reviews are still prone to evidence selec-tion bias from missed studies.

One source of this selection bias is publication bias. Sub-stantial research has shown that only half of studies con-ducted are ever published. Statistically significant findings aremore likely to be published and are published a year earlier,on average, than studies with nonsignificant findings (Dwanet al., 2013; Hopewell et al., 2007). Excluding unpublished,statistically nonsignificant data will bias a systematic reviewtoward positive findings.

Although evidence selection bias due to nonpublicationcan be difficult to identify, there are a number of strategiesthat systematic reviewers can employ to search for all existingdata (Chan, 2012; Liberati et al., 2009). Clinical trial regis-tries, regulatory agency websites, and conference abstractscan be searched to identify unpublished studies or any out-comes that may have been selectively omitted from a studypublication. The World Health Organization’s clinical trialssearch portal (http://apps.who.int/trialsearch) is a useful toolto search across multiple registries. In a systematic review onthe epidemiology of angiolymphoid hyperplasia, Adler et al.(2016) completed a comprehensive search that includedtraditional databases (PubMed and EMBASE) in addition toconference abstracts, Google Scholar, and manual searchesof reference lists. Their search strategy was published as anappendix to the main report.

There are graphical and statistical methods that can be usedto assess publication bias, though these have limitations. Theyrely on the assessment of the relationship between effect es-timates (the magnitide of the exposure’s effect on theoutcome, such as the relative risk of infection between twointerventions) and some measure of sample size for studiesincluded in the systematic review (Higgins and Green, 2011).As a general rule, the precision of an effect estimate increases(and its confidence interval decreases) as the sample sizeincreases. As a result, increased between-study variability ineffect estimates is expected among smaller studies. Graphi-cally, this can be represented by a funnel plot. Figure 1ashows a funnel plot modified from a meta-analysis on the useof sirolimus in renal transplant recipients (Knoll et al., 2014).In this plot, data points (representing individual studies) tendto scatter more horizontally in a symmetric funnel shape asthe inverse of standard error (which is related to sample size)on the y-axis increases. This visual symmetry and funnelshape suggest a low risk of publication bias. This is in contrastto a fictional asymmetrical funnel plot shown in figure 1b,which suggests a higher risk of publication bias. Statisticaltests of funnel plot asymmetry, such as Egger’s test for

Figure 2. Tabular representation of risk of bias in individual studies. Theauthors of this systematic review on the efficacy of systemic treatments forpsoriasis used the Cochrane risk of bias tool to assess potential sources of biasin included clinical trials, rating each as low (�), high (þ), or unclear (?) risk ofbias. Reprinted with permission from Elsevier from Nast et al. (2015).



continuous outcomes, can help assess whether the associa-tion between effect estimates and standard error is statisticallysignificant (Higgins and Green, 2011; Sedgwick and Marston,2015). Although funnel plots and statistical tests are usefultools, they have limitations. Statistical tests have low power todetect asymmetry if there are less than 10 studies. Further,there are other causes of asymmetric funnel plots aside frompublication bias, such as the inclusion of studies with het-erogeneous patient populations, different study designs, orpoor methodological quality (Sedgwick and Marston, 2015).

RISK OF BIAS IN PRIMARY STUDIESGiven that systematic reviews rely on data from other studies,the evidence in a systematic review is only as good as, or asfree from bias as, the primary data sources. As such, eachindividual study included in a systematic review should beassessed for key sources of bias. Selection bias refers to theexistence of systematic differences in baseline characteristicsbetween the groups compared in a study. In randomized tri-als, selection bias can arise from inadequate generation of arandom allocation sequence or inadequate concealment ofallocations before group assignment. Detection bias arisesfrom differences in outcome assessment due to knowledge oftreatment allocation by unblinded outcome assessors. Per-formance bias refers to a systematic difference betweengroups in terms of how they are treated, or differences in thebehavior of participants due to knowledge of the allocatedinterventions. Attrition bias refers to systematic differences indropouts between groups. Finally, outcome reporting biasoccurs when published trials selectively report only a subsetof measured outcomes (Chan et al., 2014). A more detaileddiscussion of bias in primary studies can be found in theCochrane Handbook for Systematic Reviews of Interventions(Higgins and Green, 2011).

It is important that review authors report the methods used toassess the risk of bias in individual studies, aswell as thefindingsof the assessment. Figure 2 presents an assessment using theCochrane Collaboration’s tool for assessing risk of bias in asystematic review of systemic treatments for psoriasis (Higginset al., 2011; Nast et al., 2015). In this example, each study isgraded as low (�), high (þ), or unclear (?) risk of bias acrossdifferent types of bias. The Cochrane risk of bias tool provides adomain-based qualitative description of critical areas of po-tential bias in clinical trials (Higgins et al., 2011). For meta-analyses, authors can conduct sensitivity analyses thatexclude trials at high risk of bias to determine the effect on theresults. Use of the Cochrane risk of bias tool is strongly recom-mended over the use of quality scales because the latter do notprovide reliable measures of bias, and the summary scores theyproduce are difficult to interpret due to uncertainty over howeach scale item should beweighted (Higgins andGreen, 2011).

COMPETING INTERESTSSystematic reviews conducted with ties to industry, particu-larly those funded by an industry sponsor, have the potentialfor bias in favor of the sponsor’s product. Using Oxman andGuyatt’s quality index for bias, Jørgensen et al. (2006) foundthat industry-sponsored reviews were generally of poorerquality than those conducted independently or sponsored bynot-for-profit organizations. In addition, despite similar effect

estimates, none of the industry sponsored reviews versus all ofthe non-industry-sponsored reviews expressed reservations inrecommending use of the studied interventions (Jørgensenet al., 2006). To acknowledge the potential for bias relatedto industry involvement, all conflicts of interest must be dis-closed by the authors of a systematic review, including whosponsored the study and what role the sponsor had in itsdesign, conduct, and reporting.

Clinicians, patients, and policy makers sometimes rely onabstracts or summaries rather than the full systematic reviewreport. However, misleading conclusions or “spin” often ap-pears in summaries and abstracts of industry-sponsored sys-tematic reviews (Yavchitz et al., 2016). Spin can also be usedby academic authors who may overstate conclusions to in-crease the likelihood of their study being accepted for pub-lication. Although spin may at times be obvious, it is notalways readily apparent.

Spin in systematic reviews has been classified into threemajor categories:

1. Misleading reporting (e.g., not fully reporting the methodsused to collect data)

2. Misleading interpretation (e.g., discussing nonsignificantresults as if they were significant)

3. Inappropriate extrapolation (e.g., application of the studyresults to a patient population not actually studied in thesystematic review) (Yavchitz et al., 2016).

It is important for authors to avoid spin when writing reportsof systematic reviews and for readers to recognize spin byconsidering the full reports of systematic reviews in contextbefore interpreting the conclusions.

OTHER QUALITY INDICATORSApart from sources of bias, there are a number of other quality-related issues to consider.When interpreting systematic reviewfindings, authors should discuss the precision or uncertainty ofa meta-analysis in terms of the 95% confidence interval of thesummary effect estimate (Guyatt et al. 2011). It is also importantfor authors to evaluate the degree to which the results of pri-mary studies included in the systematic review are consistentwith each other. Heterogeneity refers to differences in resultsbetween primary studies that are greater than expected bychance alone (Higgins and Green, 2011). It arises from differ-ences in various aspects of study design and conduct, such asthe patient populations, interventions, outcome measurementmethods, and quality. Heterogeneity can be quantified usingthe I2 statistic (Higgins et al., 2003). A substantial degree ofheterogeneity can be explored and addressed in a variety ofways. For example, the a priori selection of either a fixed- orrandom-effects model is an important consideration whenconducting ameta-analysiswith the potential for heterogeneity(Higgins and Green, 2011; Riley et al., 2011). A full discussionon the methodology and utility of meta-analyses in derma-tology is available in a priorResearch TechniquesMade Simplearticle (Abuabara et al., 2012).

SUMMARYAlthough systematic reviews andmeta-analyses are invaluablefor synthesizing available evidence, they are susceptible to



multiple forms of bias. Authors of systematic reviews canminimize the risk of bias and promote transparency by regis-tering and publishing the protocol before starting the reviewand by adhering to the PRISMA-P and PRISMA statements. It isimportant to explain protocol deviations and to searchcomprehensively for published and unpublished studies.Further, assessing the risk of bias in the included primarystudies provides an indication of the quality of available evi-dence. Finally, review authors ought to acknowledge othersources of bias, including industry sponsorship. It is importantfor consumers of systematic reviews, including clinicians, pa-tients, and policy makers, to be aware of these potential biaseswhen reading systematic reviews and assessing the evidencethey provide to address a clinical or policy question.

CONFLICT OF INTERESTAMD is an investigator for Regeneron and Sanofi (no compensation received)and has received honoraria (speaker) from Astellas Canada.


REFERENCESAbuabara K, Freeman EE, Dellavalle R. The role of systematic reviews and

meta-analysis in dermatology. J Invest Dermatol 2012;132:e2.

Adler BL, Krausz AE, Minuti A, Silverberg JI, Lev-Tov H. Epidemiology andtreatment of angiolymphoid hyperplasia with eosinophilia (ALHE): asystematic review. J Am Acad Dermatol 2016;74:506e1.

Atzmony L, Hodak E, Leshem YA, Rosenbaum O, Gdalevich M, Anhalt GJ,et al. The role of adjuvant therapy in pemphigus: a systematic review andmeta-analysis. J Am Acad Dermatol 2015;73:264e71.

Chan AW. Out of sight but not out of mind: how to search for unpublishedclinical trial evidence. BMJ 2012;344:d8013.

Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al.Increasing value and reducing waste: addressing inaccessible research.Lancet 2014;383:257e66.

Dwan K, Gamble C, Williamson PR, Kirkham JJ; Reporting Bias Group. Sys-tematic review of the empirical evidence of study publication bias andoutcome reporting bias—an updated review. PloS One 2013;8:e66844.

Guyatt G, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al.GRADE guidelines 6. Rating the quality of evidence-imprecision. J ClinEpidemiol 2011;64:1283e93.

Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. TheCochrane Collaboration’s tool for assessing risk of bias in randomisedtrials. BMJ 2011;343:d5928.

Higgins J, Green S. Cochrane handbook for systematic reviews ofinterventions, version 5.1.0. The Cochrane Collaboration; 2011.

Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency inmeta-analyses. BMJ 2003;327:557e60.

MULTIPLE CHOICE QUESTIONS1. The protocol for a systematic review should.

A. Be written and made publicly availablebefore conducting the review

B. Contain information on the sources of datathat will be used

C. Contain information on the outcomes thatwill be assessed

D. Contain information on the criteria used toinclude and exclude studies

E. All of the above

2. Publication bias occurs because.

A. The peer review process takes too long

B. Studies with statistically nonsignificantfindings are less likely to be published

C. Journals prefer to publish studies withnonsignificant findings rather than thosewith statistically significant findings

D. Systematic reviewers change their outcomeof interest after designing their protocol

3. Because they involve searches of the existingliterature and pooling of multiple primarystudies, systematic reviews.

A. Are not prone to bias because they have alarger sample size than primary studies

B. Are always able to find all available data ona topic

C. Have their own sources of bias in addition tobiases that exist in any primary studies

D. Are easy to conduct and can beaccomplished without significant effort

4. Which of the following is not a type of “spin”?

A. Discussing limitations (e.g., explainingpotential sources of missing data)

B. Misleading reporting (e.g., not fully reportingthe methods used to collect data)

C. Misleading interpretation (e.g., discussingnonsignificant results as if they weresignificant)

D. Inappropriate extrapolation (e.g., applicationof the study results to a patient populationnot actually studied in the systematic review)

5. Which of the following is true regardingcompeting interests?

A. Authors of systematic reviews shoulddisclose all potential competing interests

B. If a nonindustry organization funded theconduct of a systematic review, but did notdesign, conduct, or write the review, then thefunding does not need to be declared

C. Although declaring competing interests isstill important, funding from industry hasbeen shown to have no effect on theresults of systematic reviews

D. Systematic reviews without industry fundingnever use “spin” when writing systematicreview reports



Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. TheCochrane Collaboration’s tool for assessing risk of bias in randomisedtrials. BMJ 2011;343:d5928.

Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for results ofclinical trials. Cochrane Database Syst Rev 2007;(2):MR000011.

Jørgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared withindustry supported meta-analyses and other meta-analyses of the samedrugs: systematic review. BMJ 2006;333:782.

Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specifiedoutcomes during the systematic review process. PloS One 2010;5:e9810.

Knoll GA, Kokolo MB, Mallick R, Beck A, Buenaventura CD, Ducharme R,et al. Effect of sirolimus on malignancy and survival after kidney trans-plantation: systematic review and meta-analysis of individual patientdata. BMJ 2014;349:g6679.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al.The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanationand elaboration. PLoS Med 2009;6:e1000100.

Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group. PreferredReporting Items for Systematic Reviews and Meta-Analyses: the PRISMAstatement. BMJ 2009;339:b2535.

Nast A, Jacobs A, Rosumeck S, Werner RN. Efficacy and safety of systemiclong-term treatments for moderate-to-severe psoriasis: a systematic re-view and meta-analysis. J Invest Dermatol 2015;135:2641e8.

Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses.BMJ 2011;342:d549.

Sedgwick P, Marston L. How to read a funnel plot in a meta-analysis. BMJ2015;351:h4718.

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al.Preferred reporting items for systematic review and meta-analysis pro-tocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;349:g7647.

Yavchitz A, Ravaud P, Altman DG, Moher D, Hrobjartsson A, Lasserson T,et al. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J ClinEpidemiol 2016;75:56e65.

This is a reprint of an article that originally appeared in theNovember 2016 issue of the Journal of InvestigativeDermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Drucker AM, Fleming P, Chan A-W. Research Techniques Made Simple: Assessing Risk ofBias in Systematic Reviews. J Invest Dermatol 2016;136(11):e109ee114. doi:10.1016/j.jid.2016.08.021



Research Techniques Made Simple: Workflowfor Searching Databases to Reduce EvidenceSelection Bias in Systematic ReviewsLaurence Le Cleach1, Elizabeth Doney2, Kenneth A. Katz3, Hywel C. Williams2 and Ludovic Trinquart4

Clinical trials and basic science studies without statistically significant results are less likely to be publishedthan studies with statistically significant results. Systematic reviews and meta-analyses that omit unpublisheddata are at high risk of distorted conclusions. Here, we describe methods to search beyond bibliographicaldatabases to reduce evidence selection bias in systematic reviews. Unpublished studies may be identified bysearching conference proceedings. Moreover, clinical trial registries—databases of planned and ongoingtrials—and regulatory agency websites such as the European Medicine Agency (EMA) and the United StatesFood and Drug Administration (FDA) may provide summaries of efficacy and safety data. Primary andsecondary outcomes are prespecified in trial registries, thus allowing the assessment of outcome reportingbias by comparison with the trial report. The sources of trial data and documents are still evolving, withongoing initiatives promoting broader access to clinical study reports and individual patient data. There iscurrently no established methodology to ensure that the multiple sources of information are incorporated.Nonetheless, systematic reviews must adapt to these improvements and cover the new sources in theirsearch strategies.


CME Activity Dates: November 21, 2016Expiration Date: November 21, 2017Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: Kenneth A. Katz isfounder of Prevention Health Labs, Inc. and also has stockownership in Synta Pharmaceuticals, Madrigal Pharmaceuti-cals, and Arrowhead Pharmaceuticals. All other authors,planning committee members, CME committee members andstaff involved with this activity as content validation reviewershave no financial relationships with commercial interests todisclose relative to the content of this CME activity.






Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of InvestigativeDermatology website: http://www.jidonline.org/current. Testsfor CME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Dec16 e click ‘CME onDemand’ and locate the article to complete the test. Fax orother copies will not be accepted. To receive credits, learnersmust review the CME accreditation information; view theentire article, complete the post-test with a minimum perfor-mance level of 60%; and complete the online evaluation formin order to claim CME credit. The CME credit code for thisactivity is: 21310. For questions about CME credit [email protected].

1Department of Dermatology, AP-HP, Hôpitaux Universitaires Henri Mondor, Université Paris-Est, EA EpiDermE, INSERM, Créteil, France; 2Cochrane SkinGroup, The University of Nottingham, Centre of Evidence-Based Dermatology, School of Medicine, University of Nottingham, Nottingham, UK; 3Department ofDermatology, Kaiser Permanente, San Francisco, California, USA; and 4Cochrane France, INSERM U1153 METHODS team, Paris, France

Correspondence: Laurence Le Cleach, Service de Dermatologie, Hôpital Henri Mondor, 51 Avenue du Maréchal de Lattre de Tassigny, 94010 Créteil, France.E-mail: [email protected]

Abbreviations: EMA, European Medicine Agency; FDA, United States Food and Drug Administration



INTRODUCTIONAs highlighted in related Research Techniques Made Simplearticles, reporting bias remains one of the greatest threats to thevalidity of systematic reviews (Abuabara et al., 2012; Druckeret al., 2016). To obtain a fair assessment of the effects of anintervention, systematic reviews of interventions for skindiseases should use stringent efforts to include all relevantevidence. An exhaustive search of trials is the most importantstep in systematic review methodology to reduce evidenceselection bias. However, many published articles labeledas “systematic reviews” search only a fraction of the evidenceby limiting the search to one or two convenient databases.

In this article, we describe a workflow for searching sourcesbeyond bibliographical databases (Figure 1). These tech-niques will be useful for systematic reviewers for planning anoptimal search strategy and for readers of systematic reviewsto judge whether suboptimal methods of identifying trials mayhave introduced bias.

FIRST, FIND THE PUBLISHED TRIALSFor a systematic review of dermatological interventions, theleast one can do is to make every effort to identify all pub-lished randomized trials. Searching the Cochrane CentralRegister of Controlled Trials, MEDLINE, and EMBASE willlikely allow the researcher to find most published trials. TheCochrane Central Register of Controlled Trials is particularlyimportant to search because it offers a concentrated source ofreports of randomized trials. Other specialized bibliograph-ical databases may be relevant to specific topics (seeSupplementary Appendix S1 online). Searching bibliograph-ical databases should follow the methodological principlesfor information retrieval (Lefebvre et al., 2011). In particular,search equations should seek increased sensitivity and use adhoc filters to identify randomized trials (such as the CochraneHighly Sensitive Search Strategies or filters listed at https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home).Such a search should be complemented by screening thereference lists of all selected trials and by searching for pre-vious systematic reviews on the same topic and screening thelists of selected trials.

NEXT, FIND THE UNPUBLISHED TRIALSAbout 50% of clinical trial results that are presentedat meetings and congresses remain unpublished (Schereret al., 2007). As a consequence, conference abstractsshould be searched to identify trials with unpublished results.Data reported in conference abstracts may be not be as reli-able as full publications, because abstracts may contain pre-liminary results and may not contain sufficient information toassess methodological quality. However, abstracts allowdocumenting the existence of unpublished trials (more spe-cifically, their number and sample size) and unpublishedoutcomes. It allows statistical analysis to gauge the sensitivityof the systematic review conclusions to the nondisseminationof these trials.

Some databases index conference proceedings. However,there is currently no centralized registry of abstracts from allconferences. Systematic reviewers most frequently handsearch or electronically search abstracts made available bythe corresponding societies (e.g., American Academy of

Dermatology, European Society for Dermatological Research,Society for Investigative Dermatology, Japanese Society forInvestigative Dermatology) through journal supplements oron their websites. The Cochrane Skin Group has handsearched and added to its Specialized Register 42 journalsand 28 conference proceedings (see SupplementaryAppendix S1).

THE VALUE OF TRIAL REGISTRIES FOR IDENTIFYINGMISSING OUTCOME DATAClinical trial registries—databases of planned and ongoingtrialsehave become essential sources for identifying unpub-lished trials. In 2005, the International Committee of MedicalJournal Editors stated that to be considered for publication,trials need to have been registered in a public, InternationalCommittee of Medical Journal Editors-approved registrybefore the beginning of enrollment. Systematic review authorscan search the World Health Organization InternationalClinical Trials Registry Platform Search Portal, which gathersrecords of trials registered on 16 data providers, includingclinicaltrials.gov and the European Union Clinical TrialsRegister. Besides institutional registries, pharmaceuticalcompanies have also developed clinical trial registries. Whena relevant completed trial is identified but no publishedarticle can be matched, the systematic review authors cancontact the trialists or sponsors to inquire about the trial statusand ask for results. Some researchers have even suggestedthat only prospectively registered trials should be included inmeta-analyses because the risk of bias with any other form oftrial is too great (Roberts et al., 2015).

SUMMARY POINTS� Trials without statistically significant results areless likely to be published than trials that showapparent differences (publication bias). More-over, trial outcomes that do not support the useof the new treatment are less likely to bepublished than those that do support itsuse (outcome reporting bias).

� Systematic reviews and meta-analyses that omitunpublished data are at high risk of biased con-clusions. To increase their validity, systematicreviews should rely on a thorough search forpublished and unpublished trials.

� The Cochrane Central Register of ControlledTrials, MEDLINE, and EMBASE should besearched for published trials.

� Sources for finding unpublished trials haveexpanded recently. Conference proceedings,clinical trial registries, regulatory agency reviews,and health technology assessment reportsshould be searched for unpublished trials.

� A limitation is that there is no standard method-ology yet to decide which sources of unpub-lished trials to search and how to search them.



Trial registries also allow identification of unreported out-comes, because the primary and secondary outcomes aredocumented in each trial record. In cases of publication, onecan compare the reported outcomes with the registered out-comes and assess selective outcome reporting bias, that is,when negative outcomes remain unreported (Nankervis et al.,2012). An example of outcome reporting bias is the Multi-center Selective Lymphadenectomy Trial (Williams, 2015),one study that sought to determine whether wide excisionfollowed by sentinel node biopsy and immediate lymphade-nectomy for nodal metastases is better than wide excisionfollowed by nodal observation for melanoma. The trial pro-duced much valuable data, yet the primary outcome ofoverall survival, which was identified in the original trialregistration, was never published in the final report. Deriva-tion of overall survival data from the study report suggestedno overall survival increase for sentinel biopsy plus selectivelymphadenectomy (Williams, 2015).

Clinical trial registries may also contain summary trial data.At clinicaltrials.gov, the results of applicable clinical trials, asdefined by section 801 of the FDA Amendments Act, arerequired to be posted, and the results of many other trials are

also posted voluntarily. For systematic reviewers, it is there-fore crucial to use clinicaltrials.gov to find trial results, inparticular safety information. The EMA has also enacted aproactive publication of summary results through the Euro-pean Union Clinical Trials Register. Some pharmaceuticalcompanies have also developed their own clinical trial resultdatabases.

THE UNTAPPED DATA BURIED IN REGULATORY AGENCYWEBSITESRegulatory agencies, such as the FDA and the EMA, also offeraccess to additional data through the pharmaceutical com-panies’ approval applications (see Supplementary AppendixS2 online). The FDA provides a searchable catalog ofapproved drug products. These unpublished trial data aredirectly usable for systematic reviews, and their inclusion canresult in modification of the conclusions. In a re-analysis of 41meta-analyses based on published data only, the addition ofunpublished FDA trial data changed the outcome to a lowertreatment effect in 46.3% of meta-analyses, did not changethe estimate in 7.4%, and changed the outcome to a largertreatment effect in 46.3% (Hart et al., 2012). The EMA

Figure 1. Summary workflow for searching databases in a systematic review. A Cochrane systematic review about oral antiviral therapy for prevention of genitalherpes outbreaks in immunocompetent and nonpregnant patients is used as an illustration of this workflow (Le Cleach et al., 2014). Details can be found inSupplementary Appendix S1 online, and a tutorial to search FDA drug approval packages and EMA public assessment reports can be found in SupplementaryAppendix S2 online.



publishes European Public Assessment Reports for everymedicine application, whether it has been granted or refuseda marketing authorization. A comparison of FDA and EMAdata for 27 drugs has shown that detailed data on efficacy andharms were available; the information was easier to find onthe EMA than on the FDA website, but more data on harmswere available on the latter (Schroll et al., 2015).

The benefit of searching regulatory agency websites isexemplified in studies on use of imiquimod cream for mol-luscum contagiosum. In a Cochrane review published in2009 (van der Wouden et al., 2009), the one published trialcomparing imiquimod with placebo in 23 patients showed arelative risk of 3.67 (95% confidence interval ¼ 0.48e28.0)for complete clearance of lesions. However, three industry-sponsored unpublished trials were included in a FDA’spublicly available review (Papadopoulos, 2007). These threetrials randomized a total of 827 patients. When added to thepublished trial, the pooled relative risk was 0.93 (95% con-fidence interval ¼ 0.73e1.19), suggesting that imiquimod isineffective for that indication.

Finally, health technology assessment agencies, throughtheir requests to industry, may have access to unpublisheddata and make them publicly available by publishing benefitassessment dossiers online (see Supplementary Appendix S1).

LIMITATIONS OF STATISTICAL DIAGNOSIS ORCORRECTION FOR BIASA comprehensive search is even more important whenconsidering that no statistical method allows complete doc-umenting or excluding of reporting bias in a systematic reviewwith certainty. Asymmetry of the funnel plot may indicate thatsmaller trials give different findings than larger trials, but funnelplot asymmetry has several possible causes, and its presenceor absence cannot be equated with the presence or absence ofreporting bias. Moreover, many statistical methods havebeen introduced to detect or adjust for reporting bias, but theiruse is inappropriate in most meta-analyses because of too fewtrials or excessive heterogeneity (Ioannidis, 2008).

POTENTIAL CHALLENGES TO HANDLE WITH MULTIPLESOURCES OF DATAComprehensive searching adds to the resources needed tocomplete the systematic review, but searching some sourcesmay not always yield additional evidence. Among 114 sys-tematic reviews that searched FDA documents, unpublished

MULTIPLE CHOICE QUESTIONS1. Which of the following would result in publica-

tion bias?

A. Trials with negative results were notpublished and could not be selected inthe systematic review.

B. Trials with statistically significant results werecited more often by subsequent articles,increasing the likelihood of being selectedin the systematic review.

C. Trials were published in languages other thanEnglish and could not be selected in thesystematic review.

D. Trials were published more than once,increasing the likelihood of the trial beingselected in the systematic review.

E. All of the above

2. Searching beyond bibliographical databases fora systematic review potentially reduces whichof the following?

A. Publication bias

B. Validity of the systematic review

C. Outcome reporting bias

D. Labor intensity of the search

E. A and C

3. The sources to search for published trialsinclude which of the following?

A. MEDLINE only

B. The Cochrane Central Register of ControlledTrials

C. The Cochrane Database of SystematicReviews

D. EMBASE

E. B, C, and D

4. The sources to search for unpublished trialsinclude which of the following?

A. clinicaltrials.gov

B. alltrials.net

C. Drugs@FDA

D. Proceedings to the American Academy ofDermatology Annual Meeting

E. A, C, and D

5. Which of the following are some limitations ofsources of unpublished trials?

A. Clinical trial registries include ongoing andcompleted trials and potentially postedtrial results.

B. Reviews obtained from regulatory agenciestypically lack sufficient detail to assess therisk of bias for a trial.

C. Conference abstracts are not restricted bytreatment type (pharmacological andnonpharmacological).

D. Searching conference abstracts, clinical trialregistries, and regulatory and healthtechnology assessment agency websitesis burdensome.

E. B and D

Continued



data were available from the FDA for 17% (McDonagh et al.,2013). The extent and depth of the search strategy might beadapted according to the review question and context. Forexample, in a systematic review of a drug for an unapprovedindication, searching the FDA documents is unlikely toprovide unpublished evidence. Specific indications or guide-lines for reviews that will most likely benefit from searchingadditional sources such as the FDA do not yet exist.

Another challenge is that multiple reports for the same trialmay be identified, and discrepancies for results can existbetween different sources (Hartung et al., 2014). Systematicreview authors then have to link all reports of the same trialtogether and decide and describe clearly which report is to bechosen as the primary source of information. Although thereis no established consensus, an order of priority may beprespecified. For instance, FDA-prepared documents may beconsidered as more reliable than journal articles. In fact, FDAstatistical reviewers reanalyze raw data, whereas journalarticles may be affected by selective reporting of a subsetof statistical analyses based on the results.

THE WAY FORWARDTrial registration is now a legal requirement in the UnitedStates, European Union, and many countries, but complianceis far from perfect. Enhanced transparency is encouraged bythe alltrials.net campaign, an initiative of several organiza-tions such as Cochrane, The BMJ, and the Centre forEvidence-Based Medicine, calling for registration andreporting of results of all clinical trials. Another project,OpenTrials.net, will aggregate information from a wide vari-ety of existing sources to provide a comprehensive picture ofall the data and documents available for all trials. One keysource of trial data is clinical study reports, which are pre-pared by trial sponsors and transmitted to regulators. Thesedocuments are still infrequent but are becoming increasinglypublicly available through requests to the EMA and FDA.Moreover, the goal to obtain reporting transparency will bereached as prominent journals continue to establish clearrequirements for making trial data available (Taichman et al.,2016). The clinicalstudydatarequest.com and the Yale Uni-versity Open Data Access (http://yoda.yale.edu/, accessed 5October 2016) websites allow researchers to request accessto individual patient data and supporting documents fromindustry-sponsored clinical trials. Moreover, the EuropeanMedicines Agency policy has released guidance on thepublication of clinical data for medicinal products. This pol-icy has entered into force in 2015 for the publication ofclinical reports, but in a later stage it will also concern thepublication of individual patient data.

Systematic reviews must adapt to these improvements andcover the multiple new information sources in their searchstrategies. Conference proceedings, clinical trial registries,

regulatory agency reviews, and health technology assess-ment reports contain unpublished evidence that can beessential in resolving publication bias and selective outcomereporting.



REFERENCESAbuabara K, Freeman EE, Dellavalle R. The role of systematic reviews and

meta-analysis in dermatology. J Invest Dermatol 2012;132:e2.

Drucker AM, Fleming P, Chan A-W. Research techniques made simple:assessing risk of bias in systematic reviews. J Invest Dermatol 2016;136:e109e14.

Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drugtrials: reanalysis of meta-analyses. BMJ 2012;344:d7202.

Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M.Reporting discrepancies between the ClinicalTrials.gov results databaseand peer-reviewed publications. Ann Intern Med 2014;160:477e83.

Ioannidis JP. Interpretation of tests of heterogeneity and bias in meta-analysis.J Eval Clin Pract 2008;14:951e7.

Le Cleach L, Trinquart L, Do G, Maruani A, Lebrun-Vignes B, Ravaud P,Chosidow O. Oral antiviral therapy for prevention of genital herpesoutbreaks in immunocompetent and nonpregnant patients. CochraneDatabase Syst Rev 2014;8:CD009036.

Lefebvre C, Manheimer E, Glanvill J. Searching for studies. In: Higgins J,Green S, editors. Cochrane handbook for systematic reviews ofinterventions: version 5.1.0. London: The Cochrane Collaboration; 2011.

McDonagh MS, Peterson K, Balshem H, Helfand M. US Food and DrugAdministration documents can provide unpublished evidence relevant tosystematic reviews. J Clin Epidemiol 2013;66:1071e81.

Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective regis-tration and outcome-reporting bias in randomized controlled trials ofeczema treatments: a systematic review. J Invest Dermatol 2012;132:2727e34.

Papadopoulos E. Clinical review NDA 20723 Aldara, imiquimod 5% cream,http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM428714.pdf; 2007 (accessed 16May 2016).

Roberts I, Ker K, Edwards P, Beecher D, Manno D, Sydenham E. Theknowledge system underpinning healthcare is not fit for purpose andmust change. BMJ 2015;350:h2463.

Scherer RW, Langenberg P, von Elm E. Full publication of results initiallypresented in abstracts. Cochrane Database Syst Rev 2007;MR000005.

Schroll JB, Abdel-Sattar M, Bero L. The Food and Drug Administration reportsprovided more data but were more difficult to use than the EuropeanMedicines Agency reports. J Clin Epidemiol 2015;68:102e7.

Taichman DB, Backus J, Baethge C, Bauchner H, de Leeuw PW, Drazen JM,et al. Sharing clinical trial data: a proposal from the International Com-mittee of Medical Journal Editors. Ann Intern Med 2016;164:505e6.

van der Wouden JC, van der Sande R, van Suijlekom-Smit LWA, Berger M,Butler CC, Koning S. Interventions for cutaneous molluscum con-tagiosum. Cochrane Database Syst Rev 2009;CD004767.

Williams HC. Place your bet and show us your hand. Br J Dermatol 2015;173:1104e5.

This is a reprint of an article that originally appeared in the December 2016 issue of the Journal of Investigative Dermatology. It retains its original paginationhere. For citation purposes, please use these original publication details: Le Cleach L, Doney E, Katz KA, Williams HC, Trinquart L. Research TechniquesMade Simple: Workflow for Searching Databases to Reduce Evidence Selection Bias in Systematic Reviews. J Invest Dermatol 2016;136(12):e125ee129.doi:10.1016/j.jid.2016.09.019



Research Techniques Made Simple: MouseModels of Autoimmune Blistering DiseasesRobert Pollmann1 and Rüdiger Eming1

Autoimmune blistering diseases are examples of autoantibody-mediated, organ-specific autoimmune disor-ders. Based on a genetic susceptibility, such as a strong HLA-class II association, as yet unknown triggeringfactors induce the formation of circulating and tissue-bound autoantibodies that are mainly directed againstadhesion structures of the skin and mucous membranes. Compared with other autoimmune diseases, espe-cially systemic disorders, the pathogenicity of autoimmune blistering diseases is relatively well described.Several animal models of autoimmune blistering diseases have been established that helped to uncover theimmunological and molecular mechanisms underlying the blistering phenotypes. Each in vivo model focuseson specific aspects of the autoimmune cascade, from loss of immunological tolerance on the level of T and Bcells to the pathogenic effects of autoantibodies upon binding to their target autoantigen. We discuss currentmouse models of autoimmune blistering diseases, including models of pemphigus vulgaris, bullous pemphi-goid, epidermolysis bullosa acquisita, and dermatitis herpetiformis.


CME Activity Dates: December 20, 2016Expiration Date: December 20, 2017Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: All authors,planning committee members, CME committee membersand staff involved with this activity as content validationreviewers have no financial relationship(s) with commercialinterests to disclose relative to the content of this CMEactivity.






Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of InvestigativeDermatology website: http://www.jidonline.org/current. Testsfor CME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Jan17 e click ‘CME onDemand’ and locate the article to complete the test. Fax orother copies will not be accepted. To receive credits, learnersmust review the CME accreditation information; view theentire article, complete the post-test with a minimum perfor-mance level of 60%; and complete the online evaluation formin order to claim CME credit. The CME credit code for thisactivity is: 21310. For questions about CME credit [email protected].

INTRODUCTIONAutoimmune blistering diseases (AIBDs) are a group of rareacquired blistering skin diseases that are divided into fourmajor groups based on clinical appearance and pathology:pemphigus diseases, including the most common clinical

subtypes pemphigus vulgaris (PV) and pemphigus foliaceus,the pemphigoid diseases like bullous pemphigoid (BP), epi-dermolysis bullosa acquisita (EBA), and dermatitis herpeti-formis (DH). These diseases share the common featureof being caused by circulating autoantibodies targeting

1Department of Dermatology and Allergology, Philipps University Marburg, Marburg, Germany

Correspondence: Rüdiger Eming, Department of Dermatology and Allergology, Philipps University Marburg, Baldingerstrasse, D-35043 Marburg, Germany.E-mail: [email protected]

Abbreviations: AIDB, autoimmune blistering disease; BP, bullous pemphigoid; COL-17, type XVII collagen; COL-7, type VII collagen; DH, dermatitisherpetiformis; EBA, epidermolysis bullosa acquisita; MHC, major histocompatibility complex; PV, pemphigus vulgaris



disease-specific autoantigens in the human skin, resulting inpainful blisters of the skin and/or mucous membranes. Severalmouse models of AIBD have been generated, allowing re-searchers to investigate key pathophysiological mechanisms.

These models are either passive, based on the transfer ofpreviously generated autoantibodies into mice to generate ablistering phenotype in vivo, or active, based on immuniza-tion of wild-type or genetically modified mice with the

Figure 1. Mouse models ofautoimmune blistering diseases.Autoantigens and mouse strains used forimmunization are shown for eachrespective autoimmune blisteringdisease. (a)Current animalmodels for PV,BP, EBA, and DH use wild-type orhumanized HLA-transgenic mice. Somemodels are limited because of weakhomology between human and mouseproteins and established self-tolerance toautoantigens in mice. (b) To avoid thedifficulty of self-tolerance, preventing themouse immune system from reactingdestructively against autoantigens of theskin, enhancedmodels for PV andBPuseimmunization of mice lacking theautoantigen. After subsequent adoptivetransfer of splenocytes (containingautoreactive T and B cells) in Rag2-knockout recipient animals expressingthe autoantigen, an autoimmuneresponse is initiated that resemblescertain aspects of the human disease. BP,bullous pemphigoid; COL7, type VIIcollagen; COL17, type XVII collagen;DH, dermatitis herpetiformis; Dsg3,desmoglein 3; EBA, epidermolysisbullosa acquisita; PV, pemphigus vulgaris.

Table 1. Models of AIBD using active immunization with the respective antigensBlistering Disease Purpose of Model Methods Used in the Model References

PV Generation of mouse Dsg3-specific Tand B cells and characterization ofproduced autoantibodies; inductionof a clinical phenotype in mouse

Immunization of Dsg3 knockout micewith mouse Dsg3 and subsequenttransfer of splenocytes into Dsg3-competent immunodeficient Rag2-

knockout recipients

Amagai et al., 2000; Tsunoda et al., 2003;Takahashi et al., 2008

Transfer of Dsg3 knockoutsplenocytes into immunodeficientRag2-knockout, Dsg3-competent

recipients

Aoki-Ota et al., 2004; Kawasaki et al., 2006

Study the role of HLA moleculesin loss of self-tolerance against

human Dsg3

Immunization of humanized HLA-transgenic, MHC class II knock-outDBA/1J mice with human Dsg3

Eming et al., 2014; Schmidt et al., 2016

BP Characterize human COL17-specificT and B cells in initiation and effector

phases of disease

Human COL17 immunization ofwild-type mice by skin grafting fromhumanized COL17-transgenic mice

Olasz et al., 2007

Human COL17 immunization ofwild-type mice by skin grafting fromhumanized COL17-transgenic mice

and subsequent transfer ofsplenocytes in COL17-humanized

Rag2-knockout mice

Ujiie et al., 2010

EBA Characterize loss of self-toleranceagainst COL7 and the mechanisms ofautoantibody-induced tissue damage

Immunization of SJL/J mice withmouse GST-tagged COL7C

Ludwig et al., 2011; Sitaru et al., 2006

DH Study the role of HLA moleculesin disease induction after

gluten-sensitization

Gluten-sensitization of HLA-DQ8etransgenic, MHC class II knockout

NOD mice

Marietta et al., 2004

Abbreviations: BP, bullous pemphigoid; COL, collagen; DH, dermatitis herpetiformis; Dsg, desmoglein; EBA, epidermolysis bullosa acquisita; GST,glutathione S-transferase; MHC, major histocompatibility complex; NOD, nonobese diabetic; PV, pemphigus vulgaris.



autoantigen to induce an autoimmune response (see Iwataet al., 2015 for a comprehensive review).

In this article we will describe current active mouse modelsfor AIBD that use immunization of wild-type or geneticallymodified mice (Figure 1 and Table 1). The models help toshow certain key elements of disease pathogenesis as follows:(a) the loss of tolerance to self-antigens leading to the gen-eration of autoreactive immune cells, (b) the T- and B-celledependent production of autoantibodies, and (c)autoantibody-dependent tissue damage. Genetic modifica-tion of mice can be defined as (i) the introduction of exoge-nous genes, like human autoantigens, into the genome ofmice (gain of function); (ii) the knockout of endogenous genesin mice (loss of function), and (iii) the knockin of modifiedendogenous genes (change of function). These techniqueshave been described comprehensively in previous ResearchTechniques Made Simple articles (Griffin et al., 2015;Günschmann et al., 2014; Scharfenberger et al., 2014;Tellkamp et al., 2014).

MOUSE MODELS FOR PVIn PV, autoantibodies directed against desmogleins (Dsg3 andDsg1) cause loss of keratinocyte adhesion, resulting in blistersand erosions of the skin and mucous membranes. In most PV

patients, autoantibody titers correlate with the clinical activity,indicating a critical role of autoantibodies in disease patho-genesis. Moreover, several in vitro and in vivo studies haveclearly shown the pathogenic relevance of Dsg3-reactive IgGautoantibody (Amagai and Stanley, 2012). To study themechanisms leading to generation of pathogenic autoanti-bodies in PV, Amagai et al. (2000) developed an active diseasemodel using Dsg3e/e mice (Koch et al., 1997) that lack anestablished self-tolerance against Dsg3. Isolated splenocytesfromDsg3-immunized or naïve Dsg3e/emice were transferredinto Rag2-knockout, but Dsg3-competent, recipients to inducea Dsg3-specific autoimmune response in vivo (Amagai et al.,2000; Aoki-Ota et al., 2004). This model allowed the stableproduction of a panel ofDsg3-specific autoantibodies that bindto Dsg3 in vivo. Some of these autoantibodies were also able toinduce a PV-like phenotype in wild-type mice.

Using this active mouse model, the same group demon-strated that single Dsg3-specific CD4þ T-cell clones are ableto induce a clinical phenotype in recipient mice by activatingDsg3-reactive B cells (Takahashi et al., 2008). Further workwith monoclonal autoantibodies isolated from B-cell hybrid-omas that were generated in this model showed that eachDsg3-specific autoantibody has distinct pathogenic potencies(Tsunoda et al., 2003). Among the different isolated

Figure 2. Dsg3-specificautoantibodies show a synergisticeffect when used in combination. Inan active disease model of PVestablished by Amagai et al. (2000),Dsg3e/e mice were immunized withmouse Dsg3, and splenocytes weresubsequently transferred into Rag2immunodeficient knockout mice togenerate B cells producing a panel ofpolyclonal Dsg3-specific antibodies.Mice inoculated with hybridoma cellsproducing a weakly pathogenic Dsg3-specific antibody (NAK1) did notdevelop an apparent PV phenotype. Incontrast, when mice were inoculatedwith a combination of severalhybridoma cells (NAK1 þ NAK2 þNAK7 þ NAK11) they developed aphenotype similar to PV includingpatchy hair loss, IgG deposition inthe oral mucosa, and suprabasilaracantholysis in the skin. Scale bar ¼50 mm. Adapted from Kawasaki et al.(2006) with permission from Elsevier.Dsg3, desmoglein 3; PV, pemphigusvulgaris.



monoclonal autoantibodies, AK23 could induce blister for-mation after passive transfer of AK23 into neonatal wild-typemouse or by inoculation of AK23-producing hybridomacells into Rag-2e/e recipient mice. AK23 recognizes acalcium-dependent conformational epitope located at theadhesive interface in the N-terminal domain of Dsg3(Tsunoda et al., 2003), indicating that one highly potentpathogenic autoantibody alone can induce a blisteringphenotype in PV. However, in another study by Kawasaki

et al. a combination of several weakly pathogenic autoanti-bodies generated in the same model was shown to alsoinduce a PV phenotype in mice, pointing to potential syner-gistic effects of polyclonal Dsg3-specific autoantibodies indisease induction (Figure 2) (Kawasaki et al., 2006).

Because PV patients show a high prevalence of distinctHLA-DRB1 alleles such as HLA-DRB1*04:02, DRB1*14:04,and DQB1*05:03, Eming et al. (2014) generated a humanizedHLA-class IIetransgenic mouse in which antigen presentation

Figure 3. Genotypying of HLAtransgenes in a humanized PV mousemodel. Eming et al. (2014) establishedan HLA-transgenic mouse model byintroducing HLA-DRB1, HLA-DQB1,and human CD4 co-receptor into micelacking endogenous expression ofmouse major histocompatibilitycomplex class II (I-Ab). Transgeneexpression on the immune cell surfacewas validated by flow cytometricanalysis. Transgenic mice (solid line)express human CD4, HLA-DR, andHLA-DQ but do not express I-Abcompared with wild-type mice (dottedline), which showed no transgeneexpression. Adapted with permissionfrom Eming et al. (2014), Copyright2014. The American Association ofImmunologists, Inc. FITC, fluoresceinisothiocyanate; PE, phycoerythrin; PV,pemphigus vulgaris.

Figure 4. Disease severity is dependent on MHC haplotype in an immunization-induced mouse model of EBA. Different mouse strains were immunized with animmunogenic peptide from the murine collagen type VII (COL7C) to induce experimental EBA with erosions, crusts, and alopecia mainly localized at the ears,snout, and around the eyes. Representative phenotypes from three different strains are shown. After COL7C immunization of SJL/J mice carrying the MHChaplotype H2s a severe disease phenotype was induced, whereas C57Bl/6 mice carrying a different MHC haplotype showed no clinical signs of EBA. FemaleMRL/MpJ mice carrying the H2k haplotype were also susceptible to EBA induction. Adapted from Ludwig et al. (2011), with permission from Elsevier. EBA,epidermolysis bullosa aquisita; MHC, major histocompatibility complex.



to CD4þ T cells is restricted to human HLA alleles, which arehighly prevalent in PV, thus allowing characterization of theloss of self-tolerance against human Dsg3 by CD4þ T cells inan HLA-restricted in vivo model system (Eming et al., 2014).Mice were generated by introducing transgenicconstructs containing HLA-DRA1*01:01, -DRB1*04:02, andHLA-DQA1*03:01, -DQB1*03:02 (DQ8) into mice lackingfunctional endogenous murine major histocompatibilitycomplex (MHC) class II (I-Aße/e) (Figure 3). After immuni-zation of HLA-DRB1*04:02-transgenic mice with immuno-dominant Dsg3 peptides, a CD4þ T cell-dependent immuneresponse against human Dsg3 with the production of patho-genic Dsg3 reactive IgG antibodies was observed. However,

immunization of mice transgenic for a PV-unrelated HLA-molecule (HLA-DRB1*04:01) did not induce a Dsg3-specificantibody response, indicating that recognition of distinctDsg3 peptides by CD4þ T cells is highly specific for certainHLA molecules that are highly prevalent in PV (Eming et al.,2014). With the same model it was recently shown thatCD4þCD25þFoxP3þ T regulatory cells exert a down-regulatory effect on the humoral Dsg3-specific immuneresponse, which supports the hypothesis that the Dsg3-specific CD4þ T-celledependent immune pathogenesis ofPV is modulated by T regulatory cells (Schmidt et al., 2016).

ANIMAL MODELS FOR BPBP is a subepidermal blistering disease characterized by au-toantibodies against antigens in the epidermal basementmembrane, mainly type XVII collagen (COL-17)/BP180 (BPantigen II of 180 kDa) and the intracellular plakin BP230 (BPantigen I of 230 kDa). Autoantibodies from BP patients fail torecognize mouse COL-17 in passive transfer models due to ofdifferences in the amino acid sequences between humansand mice. Therefore, humanized mouse models for BP havebeen used to study disease mechanisms in vivo (Nishie et al.,2007). Olasz et al. (2007) established an active BP modelusing transgenic mice expressing human COL-17 in themurine basement membrane. Transgenic mice were gener-ated by crossing COL-17 knockout mice with animalsexpressing human COL-17 under the control of the humankeratin-14 promotor, allowing tissue-specific expression ofhuman COL-17 only in the basement membrane of transgenicmice (Olasz et al., 2007). Skin grafts from COL-17-transgenicmice were then transplanted onto syngeneic wild-typerecipients to induce a strong COL-17especific IgG responsewith autoantibodies able to induce subepidermal blistering inthe skin graft.

The model was further developed by Ujiie et al. (2010), whotransferred splenocytes from human COL-17-immunizedwild-type mice into immunodeficient Rag-2e/e/COL17ehumanized recipients. In this model, the depletion of CD4þ Tcells from the COL-17eimmunized mice suppressed theproduction of COL-17especific IgG antibodies, whereas thedepletion of CD8þ T cells showed no effects, indicatingthat CD4þ T cells, but not CD8þ T cells, are essential forthe production of antibodies against human COL-17 in thehumanized BP model.

ANIMAL MODELS FOR EBAIn EBA, autoantibodies bind to type VII collagen (COL-7), ananchoring fibril protein of the dermal-epidermal junction,leading to skin fragility and blistering of the skin and mucousmembranes. Most EBA sera recognizes the noncollagenous-1domain of COL-7. Animal models for EBA are mainly basedeither on the passive transfer of COL-7especific antibodies(derived from human or other species like rabbits) or on thedirect immunization of mice with the autoantigen (seeKasperkiewicz et al. [2016] for review). In the activeimmunization-induced EBA mouse model, wild-type mice areimmunized with an immunogenic peptide of the COL-7epitope from the murine noncollagenous-1 domain. After4e8weeksmice start showing a phenotype similar to EBA,withsubepidermal blister formation mainly located at the ears,

MULTIPLE CHOICE QUESTIONS1. Which of the following is not characteristic of all

autoimmune blistering diseases?

A. Blisters on the skin and/or mucousmembranes

B. IgG autoantibodies

C. Autoantibodies against autoantigens in theskin

D. Loss of self-tolerance

2. Which knockout mice are immunized withautoantigen in the active disease model ofpemphigus vulgaris?

A. Dsg1e/e mice

B. COL7e/e mice

C. Dsg3e/e mice

D. Dsg2e/e mice

3. Which domain of type VII collagen is used forthe immunization-induced model forepidermolysis bullosa acquisita?

A. NC1

B. NC2

C. NC3

D. NC4

4. What is/are the main autoantigen(s) in bullouspemphigoid?

A. COL17/BP180

B. BP230

C. COL17/BP180 and BP230

D. COL17/BP180 and BP250

5. Which Dsg3-specific antibody can induce apemphigus vulgaris-resembling phenotype inwild-type mice?

A. AK7

B. AK47

C. AK3

D. AK23



snout, and around the eyes (Ludwig, 2012; Sitaru et al., 2006).This model is used to study both the initial autoimmune eventsin loss of self-tolerance leading to development of autoreactiveCOL-7especific T and B cells and to study mechanisms ofautoantibody-induced tissue damage and inflammation. Forinstance, Ludwig et al. (2011) showed that the induction andthe severity of the EBA-like phenotype strongly depends onthe mouse’s MHC haplotype, because mouse strains carryingthe H2s haplotype are more prone to develop experimentalEBA after COL-7 immunization and show the highest diseaseseverity compared with other inbred mouse strains (Ludwiget al., 2011) (Figure 4).

MOUSE MODELS FOR DHDH is a blistering skin disease strongly associated with glutenintolerance that is clinically characterized by an intensivelypruritic papulovesicular rash on the skin. Immunofluores-cence shows IgA deposition at the tips of the papillary dermis,and gluten-induced IgA autoantibodies are directed againsttransglutaminase-2 and -3 (Kárpáti, 2012). On the basis of thestrong genetic association of DH with HLA-DQ2 and HLA-DQ8, Marietta et al. (2004) used autoimmune-prone non-obese diabetic mice lacking the endogenous murine MHCclass II (I-Aße/e) and introduced the human HLA-DQ8transgene to establish a transgenic model in which antigenpresentation to CD4þ T cells is restricted to HLA-DQ8. Blisterformation, neutrophil infiltration in the dermis, and depositionof IgA antibodies at the dermal-epidermal junction wereobserved in 16% of HLA-DQ8etransgenic nonobese diabeticmice that were sensitized to gluten, whereas no blistering orIgA deposition was observed in noneHLA-DQ8etransgenicmice, indicating that HLA-DQ8 is required for blister forma-tion (Marietta et al., 2004).

CONCLUSIONThe current understanding of the pathophysiology of AIBDshas been greatly increased by studies that have been per-formed in mouse models for these disorders. The currentmodels summarized in this article are based on the activeimmunization with recombinant autoantigens. The mousemodels focus on various aspects of the autoimmune cascadefinally resulting in the production of antibodies directedagainst the antigen of interest. In most cases, the mice developa clinical phenotype resembling certain aspects of the humandisease. However, so far there is no spontaneous mousemodelfor AIBD, which limits the significance of the establishedin vivo systems with regard to modeling the human situation.



REFERENCESAmagai M, Stanley JR. Desmoglein as a target in skin disease and beyond. J Invest

Dermatol 2012;132:776e84.

Amagai M, Tsunoda K, Suzuki H, Nishifuji K, Koyasu S, Nishikawa T. Use ofautoantigen-knockout mice in developing an active autoimmune diseasemodel for pemphigus. J Clin Invest 2000;105:625e31.

Aoki-OtaM, Tsunoda K, Ota T, Iwasaki T, Koyasu S, Amagai M, et al. Amouse modelof pemphigus vulgaris by adoptive transfer of naive splenocytes from desmo-glein 3 knockout mice. Br J Dermatol 2004;151:346e54.

Eming R, Hennerici T, Bäcklund J, Feliciani C, Visconti KC, Willenborg S, et al.Pathogenic IgG antibodies against desmoglein 3 in pemphigus vulgaris areregulated by HLA-DRB1* 04: 02erestricted T cells. J Immunol 2014;193:4391e9.

Griffin RL, Kupper TS, Divito SJ. Humanized mice in dermatology research. J InvestDermatol 2015;135:e39e43.

Günschmann C, Chiticariu E, Garg B, Hiz MM, Mostmans Y, Wehner M, et al.Transgenic mouse technology in skin biology: inducible gene knockout inmice. J Invest Dermatol 2014;134:1e4.

Iwata H, Bieber K, Hirose M, Ludwig RJ. Animal models to investigate patho-mechanisms and evaluate novel treatments for autoimmune bullous derma-toses. Curr Pharm Des 2015;21:2422e39.

Kárpáti S. Dermatitis herpetiformis. Clin Dermatol 2012;30:56e9.

Kasperkiewicz M, Sadik CD, Bieber K, Ibrahim SM, Manz RA, Schmidt E, et al.Epidermolysis bullosa acquisita: from pathophysiology to novel therapeuticoptions. J Invest Dermatol 2016;136:24e33.

Kawasaki H, Tsunoda K, Hata T, Ishii K, Yamada T, Amagai M. Synergistic path-ogenic effects of combined mouse monoclonal anti-desmoglein 3 IgG anti-bodies on pemphigus vulgaris blister formation. J Investigative Dermatol2006;126:2621e30.

Koch PJ, Mahoney MG, Ishikawa H, Pulkkinen L, Uitto J, Shultz L, et al. Targeteddisruption of the pemphigus vulgaris antigen (desmoglein 3) gene in micecauses loss of keratinocyte cell adhesion with a phenotype similar topemphigus vulgaris. J Cell Biol 1997;137:1091e102.

Ludwig RJ. Model systems duplicating epidermolysis bullosa acquisita: a method-ological review. Autoimmunity 2012;45:102e10.

Ludwig RJ, Recke A, Bieber K, Müller S, de Castro Marques A, Banczyk D, et al.Generation of antibodies of distinct subclasses and specificity is linked to H2sin an active mouse model of epidermolysis bullosa acquisita. J Invest Dermatol2011;131:167e76.

Marietta E, Black K, Camilleri M, Krause P, Rogers RS, David C, et al. A new modelfor dermatitis herpetiformis that uses HLA-DQ8 transgenic NOD mice. J ClinInvest 2004;114:1090e7.

Nishie W, Sawamura D, Goto M, Ito K, Shibaki A, McMillan JR, et al. Humanizationof autoantigen. Nat Med 2007;13:378e83.

Olasz EB, Roh J, Yee CL, Arita K, Akiyama M, Shimizu H, et al. Human bullouspemphigoid antigen 2 transgenic skin elicits specific IgG in wild-type mice.J Invest Dermatol 2007;127:2807e17.

Scharfenberger L, Hennerici T, Király G, Kitzmüller S, Vernooij M, Zielinski JG.Transgenic mouse technology in skin biology: generation of complete ortissue-specific knockout mice. J Invest Dermatol 2014;134:1e5.

Schmidt T, Willenborg S, Hünig T, Deeg CA, Sonderstrup G, Hertl M, et al.Induction of T regulatory cells by the superagonistic anti-CD28 antibody D665leads to decreased pathogenic IgG autoantibodies against desmoglein 3 in aHLA-transgenic mouse model of pemphigus vulgaris. Exp Dermatol 2016;25:293e8.

Sitaru C, Chiriac MT, Mihai S, Büning J, Gebert A, Ishiko A, et al. Induction ofcomplement-fixing autoantibodies against type VII collagen results in sub-epidermal blistering in mice. J Immunol 2006;177:3461e8.

Takahashi H, Amagai M, Nishikawa T, Fujii Y, Kawakami Y, Kuwana M. Novelsystem evaluating in vivo pathogenicity of desmoglein 3-reactive T cell clonesusing murine pemphigus vulgaris. J Immunol 2008;181:1526e35.

Tellkamp F, Benhadou F, Bremer J, Gnarra M, Knüver J, Schaffenrath S, et al.Transgenic mouse technology in skin biology: generation of knockin mice.J Invest Dermatol 2014;134:1e3.

Tsunoda K, Ota T, Aoki M, Yamada T, Nagai T, Nakagawa T, et al. Induction ofpemphigus phenotype by a mouse monoclonal antibody against the amino-terminal adhesive interface of desmoglein 3. J Immuol 2003;170:2170e8.

Ujiie H, Shibaki A, Nishie W, Sawamura D, Wang G, Tateishi Y, et al. A novelactive mouse model for bullous pemphigoid targeting humanized pathogenicantigen. J Immunol 2010;184:2166e74.

This is a reprint of an article that originally appeared in the January 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Pollmann R, Eming R. Research Techniques Made Simple: Mouse Models of AutoimmuneBlistering Diseases. J Invest Dermatol 2017;137(1):e1ee6. doi:10.1016/j.jid.2016.11.003



Research Techniques Made Simple: Analysisof Collective Cell Migration Using the WoundHealing AssayAyman Grada1, Marta Otero-Vinas1,2, Francisco Prieto-Castrillo3, Zaidal Obagi4 and Vincent Falanga1,5

Collective cell migration is a hallmark of wound repair, cancer invasion and metastasis, immune responses,angiogenesis, and embryonic morphogenesis. Wound healing is a complex cellular and biochemical processnecessary to restore structurally damaged tissue. It involves dynamic interactions and crosstalk betweenvarious cell types, interaction with extracellular matrix molecules, and regulated production of soluble medi-ators and cytokines. In cutaneous wound healing, skin cells migrate from the wound edges into the wound torestore skin integrity. Analysis of cell migration in vitro is a useful assay to quantify alterations in cell migratorycapacity in response to experimental manipulations. Although several methods exist to study cell migration(such as Boyden chamber assay, barrier assays, and microfluidics-based assays), in this short report we willexplain the wound healing assay, also known as the “in vitro scratch assay” as a simple, versatile, and cost-effective method to study collective cell migration and wound healing.


CME Activity Dates: January 19, 2017Expiration Date: January 19, 2018Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: All authors, plan-ning committee members, CME committee members andstaff involved with this activity as content validation re-viewers have no financial relationship(s) with commercialinterests to disclose relative to the content of this CMEactivity.






Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of InvestigativeDermatology website: http://www.jidonline.org/current. Testsfor CME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Feb17 e click ‘CME onDemand’ and locate the article to complete the test. Fax orother copies will not be accepted. To receive credits, learnersmust review the CME accreditation information; view theentire article, complete the post-test with a minimum perfor-mance level of 60%; and complete the online evaluation formin order to claim CME credit. The CME credit code for thisactivity is: 21310. For questions about CME credit [email protected].

COLLECTIVE CELL MIGRATIONCell migration is defined as the actual movement of individualcells, cell sheets, and clusters from one location to another. Theterm “cell motility” is often used interchangeably, but maytechnically imply a less coordinated and purposeful movementof cells. Two principal types of cell migration have been iden-tified: single cell migration and collective cell migration.Depending on the cell type, cytoskeletal structure, and thecontext in which it is migrating, the cell can migrate in differentmorphological variants such as mesenchymal, amoeboidmotility modes (Friedl and Wolf, 2003). Collective migration is

1Department of Dermatology, Boston University School of Medicine, Boston,Massachusetts, USA; 2Department of Systems Biology, The Tissue Repair andRegeneration Laboratory, Universitat de Vic—Universitat Central deCatalunya, Vic, Spain; 3MIT Media Lab, Massachusetts Institute ofTechnology, Cambridge, Massachusetts, USA; 4College of Medicine and LifeSciences, University of Toledo, Toledo, Ohio, USA; and 5Department ofBiochemistry, Boston University School of Medicine, Boston, Massachusetts,USA

Correspondence: Ayman Grada, Department of Dermatology, BostonUniversity School of Medicine, 609 Albany Street, A600, Boston,Massachusetts 02118, USA. E-mail: [email protected]

Abbreviation: RWD, relative wound density



the coordinatedmovement of a group of cells thatmaintain theirintercellular connections and collective polarity. Depending onthe anatomical and physiological context, collective migrationcan manifest as (i) two-dimensional locomotion across a tissuesurface (also known as sheet migration) where cells migrate asflat monolayer sheets, such as epidermal keratinocytes duringwound healing, or (ii) three-dimensional locomotion across atissue scaffold where cells are organized as a network ofmulticellular strands (Friedl and Gilmour, 2009).

WOUND HEALINGThere are four main phases in wound healing: coagulation,inflammation, migration-proliferation (including matrix depo-sition), and remodeling (Falanga, 2005). These phases do notrepresent distinct events, but rather overlap and are contin-uous. After tissue injury and under the influence of variousgrowth factors and cytokines, keratinocytes at the rear of thewoundmargins may display a high proliferative activity. Thesecells thenmigrate forward onto thewound bed andhelp restorethe epidermal barrier structure and function. This overall pro-cess involves cell migration, proliferation, and differentiation.In smaller wounds, the critical event is keratinocyte migrationrather than proliferation (Falanga, 2005). Cell migration beginsseveral hours after injury. Epidermal cells adjacent to thewound margin become polarized (driven by the actin

cytoskeleton) and develop pseudopodium-like projectionspreferentially oriented outward, into the free space, and within24 hours, the cells detach from the basal lamina and are readyfor migration. Lamellipodial crawling refers to the pattern ofmotion that epidermal cells exhibit during migration (Ridleyet al., 2003). Although we have predominantly referred tokeratinocytes, onemust recognize that studies of cell migrationin skin processes and disease also involve other resident skincells, including fibroblasts, microvascular endothelial cells,and melanocytes, among others.

IN VITRO WOUND HEALING ASSAYStudying the collective migration of cells in a two-dimensionalconfluent monolayer in highly controlled in vitro conditionsallows investigators to simulate and explore critical mecha-nisms of action involved in the process. A variation of thismethod that tracks the migration of individual cells has beendescribed in the literature (Rodriguez et al., 2005). There isargument about whether the assay can be equated to an actualwound, which is obviously more complex, but the assay doesallow modeling and testing of cell movement under well-defined conditions. This assay is suitable for cell types suchas keratinocytes and skin fibroblasts that exhibit collectivemigration, also known as “sheet migration” (Bindschadler andMcGrath, 2007). The technique involves making a linear thinscratch “wound” (creating a gap) in a confluent cell monolayer(Figure 1) and subsequently capturing at regular time intervalsimages of the cells filling the gap (Cory, 2011). One can thenanalyze the images to quantify migration. Live cell imagingusing time-lapse microscopy allows recording of spatial andtemporal information and allows for investigation of dynamicprocesses in living cells (Supplementary Movie S1 online). Themeasurements are generally taken for 24 hours in an attempt tolimit the study to migration and minimize the contribution ofcell proliferation to gap filling. However, the time frame shouldbe adjusted according to the particular cell type to be studied.To further reduce the risk of cell proliferation confounding thestudy of migration, a low dose of the proliferation inhibitormitomycin C can be used. Mitomycin C is an antitumor anti-biotic that inhibits DNA synthesis. The dose needs to becarefully optimized to avoid toxic effects that may affect cellmigration. Using low serum concentrations in cell medium(serum starvation) is the most common method to suppress cellproliferation in wound healing assays. However, the durationof serum starvation and the required serum concentrationsneed to be rigorously determined for each studied cell line.Serum starvation can elicit complex, unpredictable time-dependent, and cell-type-dependent effects (Pirkmajer andChibalin, 2011).

APPLICATIONS OF THE WOUND HEALING ASSAY

� Quantitative and qualitative analysis of collective cellmigration under different experimental conditions.

� Studying the effects of cell-matrix and cell-cell interactionson cell migration.

� High-throughput screening for genes involved in cancercell migration (Simpson et al., 2008), small moleculescreening (Yarrow et al., 2005), and drug discovery(Hulkower and Herber, 2011).

SUMMARYAdvantages:� Relatively inexpensive and easy to perform.

� Allows observation of cell movement andmorphology throughout the experiment.

� Testing conditions can be easily adjusted fordifferent purposes.

� Creates a strong directional migratory response.

� Ability to coat assay surface with an appropriateextracellular matrix.

� Amenable to high throughput screeningplatforms.

Limitations:� May not be suitable for studying specializedprimary cells because a relatively large numberof cells are required for the assay.

� Not suitable for chemotaxis studies or fornonadherent cells.

� Lack of standardization in its application makes itdifficult to reproduce experiments.

� Scratching introduces mechanical injury to thecells, leading to release of cellular contents intothe surroundings and potentially influencing themigration process.

� Cell proliferation may interfere with themeasurement of cell migration. Therefore,suppression of proliferation is a recommendedintervention.



METRICS TO QUANTIFY CELL MIGRATIONThe rate of cell migration can be quantfied using a singlemetric or a combination of metrics. The following are themost commonly used metrics:

1. Wound width can be calculated as the average distancebetween the edges of the scratch. Manual quantificationcan be time consuming. The wound width shoulddecrease as cell migration progresses over time. Migrationrate can be quantified by dividing the change in woundwidth by the time spent in migration:

RM ¼ Wi �Wf

tRM ¼ Rate of cell migration ðnm=hÞWi ¼ initial wound width ðnmÞWf ¼ final wound width ðnmÞt ¼ duration of migration ðhourÞ

2. Wound area can be calculated by manually tracing thecell-free area in captured images using the ImageJ publicdomain software (NIH, Bethesda, MD). Under normalconditions, the wound area will decrease over time. Themigration rate can be expressed as the change in thewound area over time. Rotzer et al. (2016) showed howthe scratch wound assay can be used to assess themigration capacity of keratinocytes under different exper-imental conditions (Figure 2). Alternatively, the migrationrate can be expressed as the percentage of area reductionor wound closure. The closure percentage will increase ascells migrate over time:

Wound Closure % ¼�At¼0h � At¼Dh

At¼0h

��100%

At¼0h is the area of the wound measured immediatelyafter scratching ðt ¼ 0hÞ

At¼Dh is the area of the wound measured h hours afterthe scratch is performed

3. Relative wound density (RWD) can also be used to mea-sure cell migration. This metric is employed in live cellimaging platforms such as the IncuCyte system (EssenBioScience, Ann Arbor, MI), and is the percentage ofspatial cell density in the wound area relative to the spatialcell density outside of the wound area at each time point(Johnston et al., 2015). Under normal conditions, RWDwill increase as cells migrate over time. This metric has theadvantage of allowing normalization for changes in celldensity caused by proliferation and pharmacologicaleffects.

% RWD ðtÞ ¼ wt �w0

ct �w0� 100

wt ¼ Density of the wound area at time t

ct ¼ Density of the cell area at time t

MATERIALS AND EQUIPMENT

� Standard cell culture protocols are available for most cell lines andcan be obtained from the online literature.

� Plastic-bottomed multiwell tissue culture plates, 10 ml pipette tips,razor or extra fine permanent marker, cell culture incubator, CO2

supply, and, in the case of testing hypoxic conditions, a nitrogensupply.

� Imaging equipment: bright-field microscope connected to a digitalcamera. Recently, an automated live cell content imaging platformhas become available that allows time-lapse microscopy(including of green fluorescent protein [GFPþ] cells) and quanti-tative image analysis. In this system, the entire unit is placed on atray in the tissue culture incubator connected to an externaldedicated computer.

PROCEDUREProtocols will of course vary according to the cell type beingstudied. However, there are some basic fundamental steps(Figure 3) that are applicable for almost all cell types: (i) cellculture preparation, (ii) scratch-making, (iii) data acquisition,and (iv) data analysis.

Figure 1. The in vitro wound healing assay. (a) Human skin fibroblasts forming a confluent monolayer. (b) In vitro “wound” was created by a straight line scratchacross the fibroblast monolayer. Black arrows are pointed toward wound edges.



1. Cell culture preparation:a. The required number of cells to form a confluent

monolayer need to be determined according to theparticular cell type and the size of wells.

b. Place the culture dishes inside the incubator until aconfluent monolayer is formed (Figure 1).

c. To inhibit cell proliferation, add an optimized (nontoxic)dose of mitomycin C for a few hours after cells reachconfluence and then removing mitomycin-C by washingbefore making the scratch (Chen et al., 2013).

2. Scratch-making:a. A sterile plastic micropipette tip or razor blade can be

used to simulate an in vivo wound by creating astraight-edged, cell-free zone across the cell monolayerin each well. A gap width of 0.5 mm allows observation

at �4 or �10 magnification. It is important to angle thepipette correctly and apply consistent pressure to createa consistent gap width. The gap should have relativelysmooth edges and little cellular debris. A fine perma-nent marker tip can be used to draw several referencepoints close to the scratch. Manual scratching mayreduce reproducibility due to well-to-well variation ingap width. Commercially available “wound maker”allows for uniform scratch-making (SupplementaryTeaching Slides online).

b. After creating the scratch, the monolayer is washed withbasal medium to remove cell debris, and complete me-dium is added. Most experiments are performed with cellsin a tissue culture incubator set at 37 �C,5%CO2, and95%air. These conditions can be altered, for example, to study

Figure 2. Migration is increased in desmoglein 3 (Dsg3)-depleted keratinocytes in a p38MAPK-dependent manner. (a) Representative bright-field images showthat silencing of Dsg3 resulted in significantly increased migration speed compared with nontarget siRNA controls. This acceleration of gap closure was alsoprevented by p38MAPK inhibition using SB20. (b) Wound closure expressed as the remaining area uncovered by the cells. The scratch area at time point 0 hourswas set to 1 (n ¼ 4e6; *P < 0.05, #P < 0.05 vs. respective DMSO condition). (c, d) Scratch-wound closure monitored over time in cells isolated from Dsg3þ/þ

and Dsg3�/� mice. Bar is 150 mm (n ¼ 10e15 from four independent isolation procedures; *P < 0.05 vs. Dsg3þ/þ DMSO, #P < 0.05 vs. respective DMSOcondition). The black bar at the right lower corner is 150 mm. MAPK, mitogen-activated protein kinase; siRNA, small interfering RNA. Reprinted from Rotzeret al. (2016), with permission from Elsevier.



the effects of hypoxia on migration, whereby nitrogen isinfused to decrease the air/oxygen concentration. The timeneeded for incubation should be determined empiricallyfor the particular cell type to be investigated. Each exper-imental condition is typically evaluated in a triplicate.

3. Data acquisition:a. Snapshot method: Migration progress can be docu-

mented by taking sequential digital photographs of thegap using bright-field microscopy. A reasonableapproach is to captures three images per well per timepoint. Wound area can be calculated using the ImageJpublic domain software.

b. Live cell imaging: Automated live-cell imaging plat-forms are well suited for long-term monitoring of cellbehavior because the microscope can be placed insidethe incubator itself. Cells are therefore imaged underoptimal physiological conditions for the duration of theexperiment. The system allows the end-point readout ofcellular events and exploration of kinetic, functional,and quantitative measurement of living cells’ behavior.

4. Data analysis:Summary statistics can be calculated, and a line chart can beused to plot the mean migration rate versus time. To test ahypothesis or compare migration rates of different cell pop-ulations, Student’s t-test is used to compare two samples andanalysis of variance with multiple testing corrections shouldbeperformed for comparing threeormoregroups of data.AP-value < 0.05 is used to define statistical significance. If thedata satisfy tests for normality and equal variance, then a t-test

should be performed to compare the mean of two groups; atwo-tailed unpaired t-test (if the samples are independent), ora two-tailed paired t-test (if the samples are related) can beperformed. If the data fail the normality test, then nonpara-metric tests such as theWilcoxon-Mann-Whitney test may beused or Kruskal-Wallis nonparametricwhen comparingmorethan two groups. One drawback of nonparametric tests is thattheyhave less power thanconventional tests suchasStudent’st-test and analysis of variance. Tomitigate this issue, a smallerP-value can be used to define significance (P< 0.01). To helpusers perform statistical analysis, we provided instructionsand R code in the Supplementary Materials (online). Recentadvances in network science, machine learning, andcomputational modeling can be utilized to model and simu-late collective cell migration and “learn” patterns of complexcellular behavior on a large scale, and perhaps predict re-sponses to perturbations. Such in silico models can comple-ment traditional experimental research and help in narrowingresearch questions (Masuzzo et al., 2016).

CONCLUSIONSThe in vitro wound healing assay is a convenient andeconomical method to assess and quantify collective cellmigration under different experimental conditions. Collectivecell migration is a hallmark of many physiological andpathological processes pertaining to skin such as woundrepair and cancer metastasis. One can enhance accuracy andreproducibility of the assay by creating cell monolayers withthe same degree of confluence and making uniform in vitro“wounds” in terms of size and geometry.

Figure 3. Graphical abstract summarizing the workflow of the in vitro wound healing assay. The technique involves basic steps applicable to almost all celltypes: 1) cell seeding and preparation; 2) making a linear thin scratch “wound” (creating a gap) in a confluent cell monolayer; 3) data acquisition throughmicroscopic image capturing and gap measurement at each time point; and 4) data analysis.




SUPPLEMENTARY MATERIALSupplementary material is linked to this paper. Teaching slides and R Codefiles are available as supplementary material.

REFERENCESBindschadler M, McGrath JL. Sheet migration by wounded monolayers as an

emergentpropertyof single-cell dynamics. JCell Sci 2007;120(Pt5):876e84.

Chen L, Guo S, Ranzer MJ, DiPietro LA. Toll-like receptor 4 has an essentialrole in early skin wound healing. J Invest Dermatol 2013;133:258e67.

Cory G. Scratch-wound assay. Methods Mol Biol 2011;769:25e30.

Falanga V. Wound healing and its impairment in the diabetic foot. Lancet2005;366:1736e43.

Friedl P, Gilmour D. Collective cell migration in morphogenesis, regenerationand cancer. Nat Rev Mol Cell Biol 2009;10:445e57.

Friedl P, Wolf K. Tumour-cell invasion and migration: diversity and escapemechanisms. Nat Rev Cancer 2003;3:362e74.

Hulkower KI, Herber RL. Cell migration and invasion assays as tools for drugdiscovery. Pharmaceutics 2011;3:107e24.

Johnston ST, Shah ET, Chopin LK, McElwain DS, Simpson MJ. Estimating celldiffusivity and cell proliferation rate by interpreting IncuCyte ZOOM�assay data using the Fisher-Kolmogorov model. BMC Syst Biol 2015;9:1.

Masuzzo P, Van Troys M, Ampe C, Martens L. Taking aim at moving targets incomputational cell migration. Trends Cell Biol 2016;26:88e110.

Pirkmajer S, Chibalin AV. Serum starvation: caveat emptor. Am J Physiol CellPhysiol 2011;301:C272e9.

Ridley AJ, Schwartz MA, Burridge K, Firtel RA, Ginsberg MH, Borisy G, et al.Cell migration: integrating signals from front to back. Science 2003;302:1704e9.

Rodriguez LG, Wu X, Guan J-L. Wound-healing assay. Methods Mol Biol2005;294:23e9.

Rotzer V, Hartlieb E, Winkler J, Walter E, Schlipp A, Sardy M, et al.Desmoglein 3-dependent signaling regulates keratinocyte migration andwound healing. J Invest Dermatol 2016;136:301e10.

Simpson KJ, Selfors LM, Bui J, Reynolds A, Leake D, Khvorova A, et al.Identification of genes that regulate epithelial cell migration using ansiRNA screening approach. Nat Cell Biol 2008;10:1027e38.

Yarrow JC, Totsukawa G, Charras GT, Mitchison TJ. Screening for cellmigration inhibitors via automated microscopy reveals a Rho-kinaseinhibitor. Chem Biol 2005;12:385e95.

MULTIPLE CHOICE QUESTIONS1. Which of the following treatments can be used

to suppress cell proliferation so that it does notinterfere with in vitro measurement of cellmigration?

A. Mitomycin C

B. Paclitaxel

C. Serum starvation

D. Vinblastine

E. A and C

2. The wound healing assay is performed in thefollowing sequence:

A. Cell culture, image collection, scratch-making,sequencing

B. Cell culture, scratch-making, data acquisition,data analysis

C. Scratch-making, cell culture, freezing, imagecollection

D. Cell coating, DNA sequencing, alignment toa reference genome

E. Data acquisition, culture preparation,scratch-making, data analysis

3. Advantages of the wound healing assay includeall of the following except:

A. Affordable and easy to set up

B. High reproducibility

C. It does not require the use of specificchemoattractants or gradient chambers

D. Suitable for chemotaxis studies

E. B and D

4. Applications of the wound healing assay mayinclude:

A. Helping to identify therapies to promote cellmigration in wound healing

B. Evaluation of the effects of inhibitors/enhancers on the migratory capacity of aparticular cell population

C. Investigating the mechanisms regulatingcancer cell migration and evaluating theefficacy of potential therapeutic drugs

D. Studying regulation of actin cytoskeletalstructures and cell polarity

E. All of the above

5. Which of the following measures can enhancereproducibility of results when performing thein vitro wound healing assay?

A. Always seed cells at the same density andstart the assay at the same degree ofconfluence.

B. If a manual scratch must be made, useconsistent pressure and pipette tip angleto create uniform scratch sizes and shapes.

C. Incubate cells for no more than 24 hours.

D. Increase the sample number.

E. A and B

This is a reprint of an article that originally appeared in the February 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Grada A, Otero-Vinas M, Prieto-Castrillo F, Obagi Z, Falanga V. Research Techniques MadeSimple: Analysis of Collective Cell Migration Using the Wound Healing Assay. J Invest Dermatol 2017;137(2):e11ee16. doi:10.1016/j.jid.2016.11.020



Research Techniques Made Simple: Identificationand Characterization of Long Noncoding RNA inDermatological ResearchDario Antonini1, Maria Rosaria Mollo2 and Caterina Missero2,3

Long noncoding RNAs (lncRNAs) are a functionally heterogeneous and abundant class of RNAs acting in allcellular compartments that can form complexes with DNA, RNA, and proteins. Recent advances inhigh-throughput sequencing and techniques leading to the identification of DNA-RNA, RNA-RNA, and RNA-protein complexes have allowed the functional characterization of a small set of lncRNAs. However, charac-terization of the full repertoire of lncRNAs playing essential roles in a number of normal and dysfunctionalcellular processes remains an important goal for future studies. Here we describe the most commonly usedtechniques to identify lncRNAs, and to characterize their biological functions. In addition, we provide examplesof these techniques applied to cutaneous research in healthy skin, that is, epidermal differentiation, and indiseases such as cutaneous squamous cell carcinomas and psoriasis. As with protein-coding RNA transcripts,lncRNAs are differentially regulated in disease, and can serve as novel biomarkers for the diagnosis andprognosis of skin diseases.


CME Activity Dates: February 22, 2017Expiration Date: February 22, 2018Estimated Time to Complete: 1 hour







Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Mar17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

INTRODUCTIONThe sequencing and functional analysis of the humangenome has demonstrated that while well-characterizedprotein-coding genes account for only 2% of the genome,approximately 80% of the genome is transcribed from one orboth strands, resulting in a large number of RNAs with little orno protein coding potential. Long noncoding RNAs (lncRNAs)are defined as RNA transcripts equal to or longer than 200nucleotides that do not encode for proteins (Geisler andColler, 2013). Regulation of their expression occurs by

1IRCSS SDN, Napoli, Italy; 2CEINGE Biotecnologie Avanzate, Center forGenetic Engineering, Napoli, Italy; and 3Department of Biology, University ofNaples Federico II, Napoli, Italy

Correspondence: Caterina Missero, CEINGE Biotecnologie Avanzate, via G.Salvatore 486, 80145 Napoli, Italy. E-mail: [email protected]

Abbreviations: cSCC, cutaneous squamous cell carcinoma; FISH,fluorescence in situ hybridization; lncRNA, long noncoding RNA; PICSAR,p38 inhibited cutaneous squamous cell carcinoma associated lincRNA;RNA-seq, RNA sequencing; TINCR, tissue differentiation-inducing non-proteincoding RNA



mechanisms similar to those observed for coding genes.Similar to coding genes, lncRNA genes are often spliced,although with fewer exons, capped at their 50 end, and pol-yadenylated at their 30 end, although some lncRNAs do notcontain a poly(A) tail.

LncRNAs have versatile functions due to their ability to pairwith other nucleic acids and to form secondary and tertiarystructures that can serve as scaffolds for multiple proteincomplexes. Several studies have revealed the functionalrelevance of lncRNAs in normal physiology, and their clinicalimplication in a number of malignancies as well as in otherpathologies (Leucci et al., 2016; Schmitt and Chang, 2016;Yan et al., 2015).

IDENTIFICATION AND VALIDATION OF lncRNAsTo generate a comprehensive and global expression profile oflncRNAs in a given cell type or tissue, the most accurate andhigh-resolution method is RNA sequencing (RNA-seq) thattakes advantage of high-throughput next-generationsequencing technologies (Figure 1a). The experimental designis crucial to obtain quantitative data and the appropriatecomparison of different samples, especially when considering

healthy and patient tissues. The first step is to create a cDNAlibrary from RNA samples by depleting the highly abundantribosomal RNA (for details see Whitley et al., 2016). Aftersequencing, basic bioinformatics analysis, and data extrac-tion, the sequenced reads can be mapped to the corre-sponding genome or transcriptome reference, using severaldatabanks including reference sequence (RefSeq; https://www.ncbi.nlm.nih.gov/refseq/), GENCODE (https://www.gencodegenes.org), lncrbadb (http://lncrnadb.com), andNONCODEv4 (http://noncode.org). An advantage of RNA-seq as opposed to microarrays is the potential of discov-ering novel transcribed regions and alternatively splicedforms of known genes by transcriptome reconstruction.

Using RNA-seq to compare undifferentiated and differenti-ated human keratinocytes, Kretz et al. (2013) identified the firstlncRNA (terminal differentiation-induced ncRNA [TINCR])that controls human epidermal differentiation by a post-transcriptional mechanism. PolyA-selected RNA was usedto generate cDNA, and high-throughput transcriptomesequencing was undertaken using the Illumina HiSeq platform.Differential expression analysis was performed using humanRefSeq transcripts as a reference transcriptome.

The largest expression dataset of lncRNAs in skin generatedto date is from polyAþ RNA-derived cDNA from 216 samplesof lesional, nonlesional psoriatic skin, and normal skin (Tsoiet al., 2015). More recently, Gupta et al. (2016) performedRNA-seq to compare the expression of lncRNAs in normalskin from healthy individuals and in lesional skin from pa-tients with psoriasis before and after treatment with adali-mumab, a humanized monoclonal antibody against tumornecrosis factor-alpha. cDNA was generated from ribosomal-depleted RNA from more than 15 individuals for eachgroup, and sequences were obtained using the IlluminaHiSeq. Interestingly, in this case, lncRNAs were mapped us-ing a combined dataset derived from RefSeq, GENCODE, anda previously generated lncRNA catalog (Hangauer et al.,2013). Data were validated by reanalyzing previously pub-lished RNA-seq data obtained from an independent set ofpsoriatic skin (Li et al., 2014) with the combined database.This approach validated the top-scoring lncRNA identified byGupta et al., underlying the crucial importance of publiclyavailable RNA-seq raw data that can be reanalyzed in sub-sequent experiments by other groups.

Once specific lncRNAs of interest have been identified, avalidation step is required (Figure 1b). Quantitative reversetranscription polymerase chain reaction approaches areuseful to confirm lncRNA expression levels under differentconditions. Although less quantitative and sensitive, Northernblot can provide reliable visual evidence for the abundanceand length of the transcripts (Kretz et al., 2013). A usefultechnique to determine the subcellular localization oflncRNAs is single-molecule RNA fluorescence in situhybridization (RNA FISH) analysis (Kretz et al., 2013;Piipponen et al., 2016) (Figure 2c). Visualization of the sub-cellular localization of a given lncRNA by FISH technologycan shed light on its putative functions. This fluorescentmethod led to the demonstration that the lncRNA TINCR, acrucial regulator of keratinocyte differentiation, is present atlow levels both in the nucleus and in the cytoplasm of un-differentiated human keratinocytes, whereas its expression

SUMMARYAdvantages:� RNA-seq coupled with advanced bioinformaticstools allow detection of even low abundanceof known or previously unidentified lncRNAs,determining their primary structure and expres-sion pattern.

� Well-established techniques such as quantitativereverse transcription polymerase chain reaction,Northern blots, and RNA FISH can be applied tovalidate expression, length, and to identify thelocalization of lncRNAs.

� Recent advances in genomics and proteomicscan be applied to the study of lncRNAs byidentifying in a high-throughput fashionlncRNA-associating DNA, RNA, and proteins.

� Because lncRNAs can serve as crucial componentsof large complexes, their functional characteriza-tion can be instrumental for therapeuticintervention.

Limitations:� lncRNAs have highly heterogeneous functionsrequiring ad hoc studies for each single lncRNA.

� lncRNAs form complicated secondary andtertiary structures that can depend on theirinteracting molecules; therefore the function isnot easily predictable by their primary structure.

� lncRNAs may be part of different complexes;therefore identification of protein, RNA, andDNA partners may not be entirely predictiveof their functions.



becomes abundant in the cytoplasm of differentiated humankeratinocytes, indicating that it plays a cytoplasmic functionas confirmed by its ability to bind to and stabilize differenti-ation mRNAs (Kretz et al., 2013).

FUNCTIONAL STUDIES OF lncRNAsLncRNAs have been shown to participate in many biologicalprocesses including the control of gene transcription, DNAreplication, RNA splicing and stability, protein synthesis, and

Figure 1. Schematic experimentalprocedure for the (a) identification,(b) validation, and (c) functionalcharacterization of lncRNAs. RNAinteractome analysis with high-throughput sequencing (RIA-Seq);protein microarray analysis (PMA);RNA binding protein (RBP) pull-down.FISH, fluorescence in situhybridization; lncRNA, longnoncoding RNA; qRT-PCR,quantitative reverse transcriptionpolymerase chain reaction.



protein modification. LncRNAs’ function is not easily pre-dictable by their primary structure, given their ability to foldinto complicated secondary and tertiary structures that candepend on their interacting molecules, which may be part ofdifferent complexes. In the nucleus, lncRNAs can recruitproteins to chromatin sites through RNA-DNA base pairing,can function as scaffolds to create discrete protein complexes,or act as decoys to remove proteins from target DNA. Thesemultifaceted functions are due to the biochemical versatilityof RNA, which can directly pair by the base-base interactionwith other nucleic acids, and fold into three-dimensionalstructures providing complex and dynamic recognition sur-faces. Although so far molecular and functional studies havebeen performed only for a limited number of lncRNAs,several approaches can be undertaken, including isolation ofother nucleic acids or proteins with which the lncRNA in-teracts (Figure 1c).

If the lncRNA is chromatin bound and interacts withgenomic DNA, RNA-DNA FISH can be used to simulta-neously reveal the localization of a specific genomic regionwith the RNA of interest. In addition, various genomicmethods are being developed to map the functional associ-ation of lncRNAs to distinct regions of the genome. Chro-matin isolation by RNA purification followed by deepsequencing is based on pull-down assays, a method forselectively isolating macromolecules based on affinity

purification. In this specific case, biotinylated oligonucleo-tides complementary to the lncRNA of interest are used as a“handle” to bring down associated chromatin to catalog thebinding sites of novel RNA molecules in a genome (Chu et al.,2015).

To discover RNA interacting with the lncRNA of interest,RNA interactome analysis followed by deep sequencing isalso based on a pull-down assay. RNA interactome analysisfollowed by deep sequencing has been used to identify RNAbinding to the lncRNA TINCR (Kretz et al., 2013), revealingthat TINCR interacts with and stabilizes a number ofdifferentiation-associated mRNAs through a 25-nucleotide“TINCR box” motif.

Another important strategy that can shed light on thebiological function of lncRNAs is the analysis of interactingprotein partners. LncRNA pull-down assays allow purifica-tion of RNA-protein complexes after which proteins can bedetected by mass spectrometry (Feng and Zhang, 2016). Asan alternative approach, Kretz et al. (2013) have usedTINCR RNA-labeled probes to hybridize commerciallyavailable human protein microarrays containing 9,400spotted proteins. Data obtained from protein microarrayanalysis revealed that Staufen1, a RNA-binding protein,displayed the strongest TINCR RNA binding signal (Kretzet al., 2013). Impaired differentiation of epidermal tissuewas observed in the absence of both TINCR and STAU1,

Figure 2. Expression of the lncRNA PICSAR is specifically upregulated in cSCC cells and its depletion suppresses growth of cSCC xenografts. (a) The heatmapof whole transcriptome analysis showing significantly (P < 0.05) regulated lncRNAs in primary (Prim; n ¼ 5) and metastatic (Met; n ¼ 3) cSCC cell lines and inNHEKs (n ¼ 4). (b) Expression of PICSAR in cSCC (n ¼ 6) and in normal skin (n ¼ 7) was determined by quantitative reverse transcription polymerase chainreaction (qRT-PCR). (c) Expression of PICSAR in cSCC and NHEK was determined by RNA FISH (in red). Scale bar ¼ 10 mm. (d) PICSAR siRNA or negative controltransfected cSCC cells were injected subcutaneously into the back of immunodeficient mice. Xenografts were harvested 18 days after inoculation and weighed.Mean � SEM is shown; *P < 0.05. (e) Histology of the tumors was analyzed by hematoxylin and eosin (H&E) staining. The proliferation marker Ki-67 wasdetected in xenografts by immunohistochemistry. Hematoxylin was used as a counterstain. Scale bar ¼ 100 mm. cSCC, cutaneous squamous cell carcinoma;FISH, fluorescence in situ hybridization; lncRNA, long noncoding RNA; NHEK, normal human keratinocyte; PICSAR, p38 inhibited cutaneous squamous cellcarcinoma associated lincRNA; SEM, standard error of the mean; siRNA, small interfering RNA. Figure reproduced with permission from Piipponen et al. (2016).



suggesting that the TINCR-STAU1 complex mediates stabi-lization of differentiation mRNAs and is required forepidermal differentiation.

In another recent study, Piipponen et al. (2016) identifiedPICSAR (p38 inhibited cutaneous squamous cell carcinomaassociated lincRNA, or LINC00162) as the highest expressedlncRNA in cutaneous squamous cell carcinoma (cSCC) cellscompared with normal human keratinocytes (Figure 2a).Increased expression levels of PICSAR were also demon-strated in tissue derived from cSCCs, compared with normalskin (Figure 2b).

To assess the biological function of aberrant expressionof PICSAR, Piipponen et al. designed siRNA specifically toknock down PICSAR in cSCC cells. They determined thatPICSAR knockdown caused a significant inhibition in tumorgrowth associated with reduced cell proliferation in xeno-graft models (Figure 2d and e) (Piipponen et al., 2016).Accordingly, PICSAR-depleted cSCC cells exhibitedimpaired proliferation and migration, and decreasedextracellular signal-regulated kinase 1/2 activity. To deter-mine the molecular effects of PICSAR on cSCC cells,Piipponen et al. performed RNA-seq analysis in PICSAR-depleted cSCC cells compared with controls. Among themost upregulated genes after PICSAR depletion were thedual-specificity phosphatases DUSP1 and DUSP6 whoseproduct dephosphorylate and inactivate mitogen-activatedprotein kinases (Piipponen et al., 2016). Interestingly, inthe presence of a DUSP6 inhibitor, PICSAR knockdown hadno effect on extracellular signal-regulated kinase 1/2 acti-vation, indicating that DUSP6 may be the link betweenPICSAR and the extracellular signal-regulated kinase 1/2signaling pathway (Piipponen et al., 2016). The exactmolecular mechanism by which PICSAR regulates mRNAexpression of several genes remains to be explored, andfurther studies will benefit from the identification ofPICSAR-interacting molecules.

In conclusion, in this overview, we discuss how to deter-mine the relevance of lncRNAs in normal physiology anddiseases of the skin, using multidisciplinary approachesincluding RNA-seq, RNA FISH, and RNA and protein inter-actome analysis. Future studies aimed at determininglncRNAs altered in skin diseases will help identifying novelpotential biomarkers of these diseases as well as targets oftherapeutic treatments.


ACKNOWLEDGMENTSThis work was supported by grants from the Italian Association for CancerResearch (AIRC IG2015-17079) and from Fondazione Telethon (GEP15096).


REFERENCESChu C, Spitale RC, Chang HY. Technologies to probe functions and mecha-

nisms of long noncoding RNAs. Nat Struct Mol Biol 2015;22:29e35.

Feng Y, Zhang L. Long non-coding RNAs: methods and protocols, vol. 1402.New York: Humana Press, Springer; 2016. p. xii, 298pp.

Geisler S, Coller J. RNA in unexpected places: long non-coding RNA functionsin diverse cellular contexts. Nat Rev Mol Cell Biol 2013;14:699e712.

MULTIPLE CHOICE QUESTIONS1. Identification of a novel lncRNA may be

performed by:

A. mQuantitative reverse transcriptionpolymerase chain reaction (qRT-PCR)

B. RNA fluorescence in situ hybridization(RNA FISH)

C. Chromatin isolation by RNA purificationfollowed by deep sequencing (ChIRP-seq)

D. RNA sequencing (RNA-seq)

2. Validation of novel identified lncRNAs may bedetermined by:

A. Quantitative reverse transcriptionpolymerase chain reaction (qRT-PCR)

B. RNA interactome analysis followed by deepsequencing (RIA-seq)

C. Chromatin isolation by RNA purificationfollowed by deep sequencing (ChIRP-seq)

D. lncRNA interacting protein analysis

3. RNA fluorescence in situ hybridization (RNAFISH) is capable of:

A. Identifying RNA-interacting proteins

B. Identifying the RNA interactome

C. Determining the subcellular localizationof RNA

D. Identifying the functional role of RNA

4. Identification of lncRNA-binding proteins can beachieved by:

A. RNA interactome analysis followed by deepsequencing (RIA-seq)

B. Chromatin isolation by RNA purificationfollowed by deep sequencing (ChIRP-seq)

C. RNA pulldown followed by MassSpectrometry

D. Specific knockdown of lncRNA

5. RNA interactome analysis followed by deepsequencing (RIA-seq) is capable of:

A. Localizing the cellular compartment in whichthe lncRNA is expressed

B. Identifying the RNAs that interact withlncRNA

C. Identifying novel transcribed regions andalternative spliced forms of annotated genes

D. Identifying the RNA interacting proteins



Gupta R, Ahn R, Lai K, Mullins E, Debbaneh M, Dimon M, et al. Landscape oflong noncoding RNAs in psoriatic and healthy skin. J Invest Dermatol2016;136:603e9.

Hangauer MJ, Vaughn IW, McManus MT. Pervasive transcription of the hu-man genome produces thousands of previously unidentified long inter-genic noncoding RNAs. PLoS Genet 2013;9:e1003569.

Kretz M, Siprashvili Z, Chu C, Webster DE, Zehnder A, Qu K, et al. Control ofsomatic tissue differentiation by the long non-coding RNA TINCR. Na-ture 2013;493:231e5.

Leucci E, Vendramin R, Spinazzi M, Laurette P, Fiers M, Wouters J, et al.Melanoma addiction to the long non-coding RNA SAMMSON. Nature2016;531:518e22.

Li B, Tsoi LC, Swindell WR, Gudjonsson JE, Tejasvi T, Johnston A, et al. Tran-scriptome analysis of psoriasis in a large case-control sample: RNA-seqprovides insights into disease mechanisms. J Invest Dermatol 2014;134:1828e38.

Piipponen M, Nissinen L, Farshchian M, Riihilä P, Kivisaari A, Kallajoki M,et al. Long noncoding RNA PICSAR promotes growth of cutaneoussquamous cell carcinoma by regulating ERK1/2 activity. J Invest Der-matol 2016;136:1701e10.

Schmitt AM, Chang HY. Long noncoding RNAs in cancer pathways. CancerCell 2016;29:452e63.

Tsoi LC, Iyer MK, Stuart PE, Swindell WR, Gudjonsson JE, Tejasvi T, et al.Analysis of long non-coding RNAs highlights tissue-specific expressionpatterns and epigenetic profiles in normal and psoriatic skin. Genome Biol2015;16:24.

Whitley SK, Horne WT, Kolls JK. Research techniques made simple: meth-odology and clinical applications of RNA sequencing. J Invest Dermatol2016;136:e77e82.

Yan X, Hu Z, Feng Y, Hu X, Yuan J, Zhao SD, et al. Comprehensive genomiccharacterization of long non-coding RNAs across human cancers. Can-cer Cell 2015;28:529e40.

This is a reprint of an article that originally appeared in theMarch 2017 issue of the Journal of InvestigativeDermatology. It retains its original pagination here. Forcitation purposes, please use these original publication details: Antonini D, Mollo MR, Missero C. Research Techniques Made Simple: Identification andCharacterization of Long Noncoding RNA in Dermatological Research. J Invest Dermatol 2017;137(3):e21ee26. doi:10.1016/j.jid.2017.01.006



Research Techniques Made Simple: ExperimentalMethodology for Single-Cell Mass CytometryTiago R. Matos1,2,3, Hongye Liu1,2 and Jerome Ritz1,2

Growing recognition of the complexity of interactions within cellular systems has fueled the developmentof mass cytometry. The precision of time-of-flight mass spectrometry combined with the labeling ofspecific ligands with mass tags enables detection and quantification of more than 40 markers at a single-cellresolution. The 135 available detection channels allow simultaneous study of additional characteristics ofcomplex biological systems across millions of cells. Cutting-edge mass cytometry by time-of-flight (CyTOF)can profoundly affect our knowledge of cell population heterogeneity and hierarchy, cellular state, multi-plexed signaling pathways, proteolysis products, and mRNA transcripts. Although CyTOF is currently scarcelyused within the field of investigative dermatology, we aim to highlight CyTOF’s utility and demystify thetechnique. CyTOF may, for example, uncover the immunological heterogeneity and differentiation ofLangerhans cells, delineate the signaling pathways responsible for each phase of the hair cycle, or elucidatewhich proteolysis products from keratinocytes promote skin inflammation. However, the success of masscytometry experiments depends on fully understanding the methods and how to control for variations whenmaking comparisons between samples. Here, we review key experimental methods for CyTOF that enableaccurate data acquisition by optimizing signal detection and minimizing background noise and sample-to-sample variation.


CME Activity Dates: 21 March 2017Expiration Date: 20 March 2018Estimated Time to Complete: 1 hour







Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Apr17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

1Division of Hematologic Malignancies, Dana-Farber Cancer Institute, Boston, Massachusetts, USA; 2Harvard Medical School, Boston, Massachusetts, USA; and3Academic Medical Center, Department of Dermatology, University of Amsterdam, Amsterdam, The Netherlands

a The abbreviation “CyTOF”, in addition to being the name of this technique, is also the name of a commercial product that enables researchers to use themethod. The authors are in no way endorsing any specific commercial products

Correspondence: Tiago R. Matos, Department of Dermatology, Room L3-119, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZAmsterdam, The Netherlands. E-mail: [email protected]

Abbreviations: CyTOF, mass cytometry by time-of-flight; MCA, metal-conjugated antibody; MCB, mass-tag cellular barcoding



INTRODUCTIONGrowing recognition of the complexity of interactions withincellular systems has fueled development of new technologiescapable of a broad, holistic scope of analysis. In masscytometry by time-of-flight (CyTOF)a, cells are probed withmetal-conjugated antibodies (MCAs). Tagged cell suspen-sions are then passed through a droplet nebulizer to enter intoargon plasma, where individual cells are atomized andionized, and abundant common ions are removed. Then,time-of-flight mass spectrometry detects the ionized metaltags through 135 detection channels and can measure over45 parameters in each cell (Bandura et al., 2009) (Table 1).Mass cytometry has been described in detail in previous re-views (Bendall et al., 2012; Doan et al., 2015). Despite theinnovative applications of this technique, it is currentlyscarcely used within the field of investigative dermatology.CyTOF technology may lead to a greater comprehension ofcutaneous cellular phenotype heterogeneity, development,hierarchy, and relationship to other tissues. CyTOF may allowsimultaneous study of cell state (such as proliferation, hyp-oxia, or enzymatic activity) and deeper understanding ofexpression of mRNA transcripts, cytokines, growth factors, ortranscription factors within cell subsets. Examples of ques-tions that may be addressed using this technique are abun-dant. Which signaling pathways of innate lymphoid cellseffectively regulate immune homeostasis or contribute toautoimmunity? Which proteolysis products of keratinocytespromote skin inflammation? Which cancer cells have pre-dictive value for early diagnosis, prognosis, development ofdrug resistance, or relapse?

This review aims to highlight CyTOF’s novelty and utility,demystify the technique, and provide guidance on design ofthe multistep experimental methodology that requiresdetailed understanding and planning to ensure accurate andconsistent results. We will focus on specific considerationsneeded when designing a panel of desired markers,

optimizing the staining protocol, performing metal conjuga-tion of antibodies, and barcoding multiple samples.

EXPERIMENTAL DESIGNMass cytometry experiments include precise and lengthymultistep protocols that generate immense amounts of data atsingle-cell resolution (Figure 1). It is therefore imperative toestablish a meticulous experimental strategy and to have clearobjectives at the onset of each experiment. After definingspecific experimental aims, it is important to define the typesof cells that will be studied, experimental conditions,comparative groups, and controls. In some cases, FACS ormagnetic-activated cell sorting is needed to enrich for raresubsets of cells to avoid long CyTOF acquisition times (sam-ple throughput: flow cytometry ¼ 25,000 vs. CyTOF ¼500e2,000 cells/second). At least 300 events of the rarestpopulation should be acquired for analysis.

Watanabe et al. (2015) recently used mass cytometry tocompare the relative functional capacities (including TNF-a,IL-2, IL-4, IL-13, IFN-g, IL-17, IL-22, and IL-10) of skin-tropic(CLAþ) central memory, migratory memory, and effectormemory T cells from human blood. T cells were isolated fromperipheral blood of healthy individuals and stimulated withphorbol 12-myristate 13-acetate and ionomycin. The oppor-tunity to simultaneously study those many markers from asingle sample enabled the authors to conclude that effector

SUMMARY POINTS� The success of mass cytometry experiments isdependent on wellethought-out goals, a detailedexperimental design, and practice with CyTOFtechnology and protocols.

� When designing custom metal-conjugatedantibody (MCA) panels, account for targetantigen abundance and signal crosstalk.

� Custom MCAs need to be thoroughly validatedand titrated.

� Staining protocols may need to be tested tooptimize marker signal detection. Metal nucleicacid intercalators should be included in theexperiment for accurate single-cell identificationand live:dead discrimination.

� To minimize sample-to-sample variation, it isimportant to normalize samples based on beadstandards and/or a sample barcoding strategy.

Table 1. Summary description of mass cytometrytechnology, advantages and limitationsAdvantages Limitations

� Possible to simultaneouslyanalyze over 45 parameters(e.g., 40 antibody-taggedmarkers, cell viability, andDNA content)

� Possible to study cell death,cytokine production, andcell signaling simultaneously

� Minimal background noise fromsignal overlap or endogenouscellular components

� Cost per probe pertest z $1.50e$3.001

� Cost per analyzedcell z 0.005 cents2

� A single dataset can be analyzedsimultaneously by variousanalysis methods to testmultiparameter hypotheses

� Cells are destroyed throughthe CyTOF process; thus, it isnot feasible to further cultureor analyze cells after dataacquisitionSlow sample throughput(maximum of 2,000events/second), whereasflow cytometry can operate25e50 times faster

� Some cellular propertiescannot be measured(e.g., pH or ion concentration)

� Low efficiency (only 30e60% ofcells of a sample are measured)

� CyTOF’s multiplexed,high-dimension data requiresnew analysis tools

Abbreviation: CyTOF, mass cytometry by time-of-flight.CyTOF allows the characterization and quantification of over 40 markerssimultaneously on millions of individual cells. Metal-tagged antibodies areused to label multiple internal and external cellular markers of interest,which can be quantified by time-of-flight mass spectrometry at a single-cell resolution. It can lead to unprecedented breakthroughs ofunderstanding the complex differentiation process and interactionbetween cell subpopulations, new cell types, functional profiles, andbiomarkers.1Estimated based on the price of commercially conjugated reagents orunconjugated antibodies and commercial conjugation kits, in contrast to$2.00e$8.00 for fluorescent flow cytometry (Bendall et al., 2012).2The cost of reagents, disposables, and data acquisition, in contrast tow$22 to measure cells by single-cell RNA sequencing using the FluidigmC1 system (Fluidigm, San Francisco, CA) and molecular identifiers (Spitzeret al., 2016).



memory T cells secrete more T helper type 1, T helper type17, and T helper type 22 proinflammatory effector cytokines,whereas central memory T cells have higher T helper type 2cytokine production.

It is important to start an experimental design by listing themarkers necessary to define the cell populations that will bestudied. Further markers can then be added to the panel forspecific experimental measurements. Complementary stainingpanels can even be combined to study more than 40 markersfrom the same sample. For example, Bendall et al. (2011)aimed to develop an immune system map of healthy humanbone marrow cells, allowing comparisons with bone marrowcells after drug treatment or in cases of disease. This couldallow mechanistic studies and pharmacologic intervention.The authors wanted to simultaneously analyze 52 differentsingle-cell parameters, which are more markers than eitherCyTOF (w40) or flow cytometry (w15e20) can detect at asingle time. Two complementary staining panels were used for

the same samples. (Each panel had the same 16 core markersand an additional 18 panel-specificmarkers). Merging the dataallowed analysis of the biochemical intracellular signaling inrare and diverse subsets of human bone marrow cells.

A recent study showed that each CyTOF instrument has itsown signal sensitivity profile for metal isotopes, suggesting thatit may be difficult to accurately compare data acquired bydifferent instruments. This should be taken into account inexperimental design, especially for multicenter studies usingdifferent CyTOF instruments (Tricot et al., 2015). Even though itis currently possible to normalize variation over time relative toa single CyTOF instrument by including calibration beads witheach run, it is not yet possible to normalize data acquired bydifferent instruments. An alternative is to include a standard/control sample (having several vials of the same sample frozen)together with each run of the samples of interest. Each sample’sdata can later be normalized according to the control samplethat was analyzed within that same run. It has also been

Figure 1. Schematic representation of mass cytometry experiment design. See text for details.



suggested that beads should be embedded with elementsrepresenting the complete detectable mass range to allow analgorithm to be developed to normalize output for eachchannel and instrument (Tricot et al., 2015).

METAL-CONJUGATED ANTIBODY PANEL DESIGNDesign of mass cytometry panels requires the assignment of adistinct metal isotope to each of the antibodies from the list ofdesired markers. There are several commercially available,ready-to-use panels that have been designed to study distinctcell types and functional systems. However, when customizingpanels by adding new markers or designing a new panel,careful planning and awareness of multiple factors that caninfluence signal acquisition are needed to optimize signaldetection and minimize background noise (Table 2). The na-ture of the current CyTOF mass window results in an optimaldetection of metals in the range of 153e176 Da. Therefore,low-abundance targets should be tagged with metals withinthis mass range (Figure 2). In CyTOF, the main sources ofbackground noise come from environmental contaminationwith untagged metals or the measurement of the isotope signalin undesired channels, referred to as crosstalk. Environmentalcontamination of samples can drastically compromise dataacquisition. This is reduced by the use of water purified byreverse osmosis (e.g., Milli Q water; Merck Millipore, Darm-stadt, Germany) and certifiedmetal-free buffers and containers.Sources of crosstalk include oxidation, isotopic purity, andsignal abundance sensitivity. Several ions oxidize at low fre-quencies (�0.1e3.0%), resulting in a mass gain that is thenread by the corresponding channel. Oxidation can be mini-mized by optimizing the plasma temperature, which is part ofthe daily CyTOF tuning procedure. Isotopic purity is generallyhigh (95e99%), but nevertheless, the signal from residualmetal isotopes can bleed and be read by correspondingchannels. Online tools are available to help researchers mini-mize crosstalk and optimize study-specific MCAs.

METAL-ANTIBODY CONJUGATIONOver 500 metal conjugated antibodies are commerciallyavailable to study human and mouse cells. It is also possible toconjugate in-house additional IgG antibodies to one of theexisting 36 metal isotopes or request a custom conjugationservice, available through commercial vendors or institutionalcore facilities (Lou et al., 2007). Each new customMCA shouldbe validated with relevant positive and negative control celllines with known expression of the marker being tested. It isalso often necessary to compare the results acquired by CyTOFwith conventional flow cytometry, using a fluorochrome-tagged antibody identical to the purified clone used for themetal-antibody conjugation (Figure 3). To avoid ion detectorsaturation, new MCA and commercial ready-to-use MCAconcentrationsmust be titrated. To facilitate these comparisonsit is useful to store several vials of the same target cell popu-lation that can be thawed and used as a control, because thenthe expected expression of a specific marker would already beknown from previous tests. When purchasing custom MCAs itis important to know what validations and titrations wereconducted, to determine whether these are relevant to thespecific research project at hand.

CELL-STAINING PROTOCOLThe cell-staining protocol can also dependon the experimentalgoals; for example, consideration should be given to whethercells need to be stimulated or whether the study involves cellssurface markers, intercellular markers (cytokines and chemo-kines), intranuclear markers (transcription factors), or tetrame-ters (T-cell specificity). Online resources, for example at http://web.stanford.edu/group/nolan, offer CyTOF-validated pro-tocols for cell staining. Protocols must occasionally be slightlyadjusted to optimize signal sensitivity. For instance, a possibleadjustment may be due to the down-regulation of some surfacemarkers (e.g., chemokine receptors) when cells are stimulated.Wong et al. (2015) noticed a 10% difference in frequency ofCXCR5 expression on tonsillar and peripheral CD4þ T cellsupon phorbol 12-myristate 13-acetate/ionomycin stimulation,whichwas prevented by includingmetal-tag antibodies againstthe trafficking receptors (i.e., CXCR5), together with the stim-ulation medium. This is likely because metal-tag antibodieshave a higher antibody conjugate stability compared withfluorophore-tag antibodies during cell incubation (Wong et al.,2015).

If necessary to achieve a better separation between cellsubsets expressing or not expressing a marker, a secondaryantibody can be used (e.g., antieKi67-PE followed by antiePE-metal) to amplify the signal. Some antigens can be taggedusing either surface or intracellular staining protocols (e.g.,CTLA-4), resulting in different signal intensities. In our expe-rience, protocol adjustments are based on studying the samemarkers by traditional flow cytometry or by testing distinctstaining protocols side by side. Because protocol optimizationmay take several rounds of trial and error, we use controlsamples (e.g., cell lines or healthy control samples) beforeanalyzing precious experimental samples.

For discrimination between live and dead cells, rhodium103 nucleic acid intercalators or cisplatin (platinum-195,-194, or -198) should be included in the experimental pro-tocol. Rhodium and cisplatin label only dead cells, because

Figure 2. Metal-conjugated antibody panel design. Example of mass responsecurve with optimal detection of metals in the range of 153e176 Da (pink bar).

Table 2. Factors that can influence signal detectionFactors to account for signal detection with minimum background

Antigen abundanceSystem sensitivity to metal isotopeBackground signal (environment)Crosstalk (abundance sensitivity, metal isotope purity, metal ion oxidation)



they have compromised cell membranes. Resulting data canthen be gated on rhodium-negative or cisplatin-negative cellsto ensure analysis of live cells only.

For accurate single-cell identification, cells can be stainedwith two distinct iridium isotopes (iridium-191 and iridium-193) after fixation and permeabilization. Iridium intercalatesDNA with high affinity, allowing detection of all DNA-containing cells and aiding the identification of single-cellevents (excludes doublets).

MASS-TAG CELLULAR BARCODINGUsing a newly developed mass-tag cellular barcoding (MCB)strategy, it is possible to individually barcode up to 20

samples that can then be processed together in a single tube.After data acquisition, software deconvolutes samples basedon the barcode present in each cell, allowing the analysis andcomparison of each original sample separately. It guaranteesconsistent quality of signal detection and better efficiency ofdata acquisition by reducing the time needed to wash theinstrument between samples. When the MCB is done beforecell staining, it also reduces antibody consumption andstaining protocol workflow time and adds consistencythroughout sample processing and staining that is crucial toproviding accurate comparisons among samples.

Each sample can be barcoded with a unique combinationof two or three of six available palladium isotopes (Figure 4).

Figure 3. Example of validation of custom metal antibody conjugation (ICOS-154Sm). (a) An initial validation of custom metal-conjugated antibodies can becarried out by assessing the expression of the marker in previously known populations (e.g., by searching for information published in the literature; Ito et al.,2008). (b) Results from identical cell samples analyzed by CyTOF and flow cytometry can be compared for a limited number of markers. Here, a fluorochrome-tagged antibody identical to the purified clone used for the metal-antibody conjugation was used for FACS. Alternatively, positive and negative control cell linesshould be used. DNA2 stands for iridium-193. CyTOF, mass cytometry by time-of-flight; SSC, side scatter.



Alternatively, osmium and ruthenium tetroxides (OsO4 andRuO4), which bind covalently with fatty acids in cellularmembranes and aromatic amino acids in proteins, can beused independently or combined with palladium isotopes forMCB of cells (Catena et al., 2016). When studying peripheralblood mononuclear cells, it is also possible to use six distinctanti-CD45 antibodies conjugated with different metal iso-topes (Lai et al., 2015).

Some drawbacks of MCB are that cell yields can bereduced by up to 50% after barcoding. There are severalfactors that can influence cell loss, such as low initial cellnumber, cell type, container type, procedures for liquidtransfer, and mixing. One MCB protocol put forth by Zunderet al. (2015) includes an early sample pooling step,which creates a single high-abundance sample rather thanmultiple low-abundance samples, improving the cell yield.

Additionally, MCB requires fixation and permeabilization ofcells that, when done before antibody staining, may affect theability to detect some epitopes. Epitopes that are sensitive tothe paraformaldehyde fixation step should be stained withantibodies before barcoding. Alternatively, it is necessary toidentify a paraformaldehyde-compatible epitope for the samemarker of interest (Behbehani et al., 2014; Zunder et al.,2015) (Table 3).

CALIBRATION BEADSTo minimize the variance in mass cytometry instrument per-formance over time that may produce sample-to-samplesignal variation, calibration beads should be added to thesample immediately before the sample is injected into theinstrument for CyTOF acquisition. After acquisition, datanormalization can be carried out using an algorithm to

Figure 4. Single-cell barcode anddeconvolution. (a) Each sample isbarcoded with a unique combinationof three of six available palladium (Pd)isotopes. (b) Five events from a 6-choose-3 mass-tag cellular barcoding(MCB)-multiplexed fluorescent cellstandard (FCS) file are shown in single-cell format displayed on a verticaldashed line. Events 1, 2, and 3correspond to barcode single cells asindicated by the isotope symbols,Event 4 is a barcode doublet, andEvent 5 is classified as debris. The redline segments indicate barcodeseparation, assuming the 6-choose-3scheme, which is always set as thedistance between the third and fourthhighest barcode intensities. Withoutthis assumption, the last two eventswould have larger barcode separationsbut would still be discarded becausetheir barcodes would not match anyin the 20-sample scheme. (c) Examplesof a barcode singlet (three positivebarcode channels) and a barcodedoublet (more than three positivebarcode channels) as seen in the time-of-flight spectra used to visualize cellswhile acquiring data at the instrument.ms, mass spectra. Adapted withpermission from Macmillan PublishersLtd: Zunder et al., 2015.

Table 3. Advantages and limitations of mass-tag barcodingAdvantages Limitations

� Reduces technical variability (cell-to-antibody ratio-dependent effects onstaining, pipetting errors, sample acquisition, etc.)

� Reduces antibody consumption, staining protocol workflow time,and data acquisition time

� Allows staining of valuable samples with low cell numbers bycombining numerous samples together

� Excludes debris and cross-sample doublets (e.g., cell eventswith more or less than three palladium isotopes)

� Barcoding reagents can be costly� Low cell yield (w50%)� Barcoding method can affect quality of antibody stainingfor some epitopes



analyze the calibration bead signal variation that wascaptured simultaneously with the cell samples, enablingcorrection of both short- and long-term signal fluctuations(Finck et al., 2013). This normalization algorithm is currentlypart of the general CyTOF software package.

CONCLUSIONThe success of mass cytometry-based experiments is depen-dent on wellethought-out goals, detailed experimentaldesign, and knowledge of potential technical pitfalls andlimitations. A methodical approach is essential to control forexperimental noise when conducting precise comparisonsbetween samples. This approach facilitates the ability toharness the full potential of mass cytometry to characterizecomplex biological systems at single-cell resolution. CyTOFmay lead to key discoveries in investigative dermatology,including identification of signaling phenotypes with predic-tive value for early diagnosis, prognosis, or relapse, and athorough characterization of intratumor heterogeneity anddisease-resistant cell populations that may ultimately unveilnovel therapeutic approaches.


ACKNOWLEDGMENTSWe would like to thank Jodi L. Johnson for helpful comments, critical readingof the manuscript, and editorial assistance. Funded by the Harvard MedicalSchoolePortugal Program in Translational Research HMSP-ICT/0001/201.

CME ACCREDITATIONThis activity has been planned and implemented in accordance with theaccreditation requirements and policies of the Accreditation Council forContinuing Medical Education through the joint providership of WilliamBeaumont Hospital and the Society for Investigative Dermatology. WilliamBeaumont Hospital is accredited by the ACCME to provide continuing med-ical education for physicians. William Beaumont Hospital designates thisenduring material for a maximum of 1.0 AMA PRA Category 1 Credit(s)�.Physicians should claim only the credit commensurate with the extent of theirparticipation in the activity.

To participate in the CE activity, follow the link provided.https://beaumont.cloud-cme.com/RTMS-Apr17


REFERENCESBandura DR, Baranov VI, Ornatsky OI, Antonov A, Kinach R, Lou X, et al.

Mass cytometry: technique for real time single cell multitarget immu-noassay based on inductively coupled plasma time-of-flight mass spec-trometry. Anal Chem 2009;81:6813e22.

Behbehani GK, Thom C, Zunder ER, Finck R, Gaudilliere B, Fragiadakis GK,et al. Transient partial permeabilization with saponin enables cellularbarcoding prior to surface marker staining. Cytometry A 2014;85:1011e9.

Bendall SC, Nolan GP, Roederer M, Chattopadhyay PK. A deep profiler’sguide to cytometry. Trends Immunol 2012;33:323e32.

Bendall SC, Simonds EF, Qiu P, Amir ED, Krutzik PO, Finck R, et al. Singlecell mass cytometry of differential immune and drug responses acrossthe human hematopoietic continuum. Science 2011;332(6030):687e96.

Catena R, Özcan A, Zivanovic N, Bodenmiller B. Enhanced multiplexingin mass cytometry using osmium and ruthenium tetroxide species.Cytometry 2016;89:491e7.

Doan H, Chinn GM, Jahan-Tigh RR. Flow cytometry II: mass and imagingcytometry. J Invest Dermatol 2015;135:e36.

Finck R, Simonds EF, Jager A, Krishnaswamy S, Sachs K, Fantl W, et al.Normalization of mass cytometry data with bead standards. Cytometry A2013;83:483e94.

MULTIPLE CHOICE QUESTIONS1. Which of the following are two main consider-

ations when designing a CyTOF panel?

A. Antigen abundance and crosstalk

B. Use only ready-to-use panel kits andcommercially available MCAs

C. Low number of available probes and samplecell type

D. Vast variability of signal detection acrosschannels and low isotopic purity

2. What are the principal sources of signalbackground in CyTOF?

A. Concentration of metal-conjugatedantibodies and reagents during staining

B. Environmental contamination and crosstalk

C. Plasma temperature and instrument state ofmaintenance

D. Endogenous cell signal and spectral overlap

3. Which cell-staining protocol should be used forCyTOF experiments?

A. Standard CyTOF protocol

B. Same protocol as for flow cytometry

C. CyTOF staining protocol tested for specificexperiment panel

D. Any CyTOF-validated protocol

4. What can be done to ensure an accurate analysisof live single cells?

A. Use only fresh and filtered cell samples.

B. Use nucleic acid intercalators.

C. Acquire data on the same day as performingthe staining protocol.

D. Use calibration beads.

5. Which CyTOF-specific strategies should be usedto control for sample-to-sample variation?

A. Use a commercial, ready-to-use panel kit andhigh-quality reagents.

B. Incorporate nucleic acid intercalators anduse the same cell concentration in allsamples.

C. Make sure to use appropriate statistical testsand the same analysis software for all samples.

D. Normalize samples based on bead standardsand barcode samples.



Ito T, Hanabuchi S, Wang YH, Park WR, Arima K, Bover L, et al. Two func-tional subsets of FOXP3þ regulatory T cells in human thymus andperiphery. Immunity 2008;28:870e80.

Lai L, Ong R, Li J, Albani S. A CD45-based barcoding approach to multiplexmass-cytometry (CyTOF). Cytometry A 2015;87:369.

Lou X, Zhang G, Herrera I, Kinach R, Ornatsky O, Baranov V, et al. Polymer-based elemental tags for sensitive bioassays. Angew Chem Int Ed2007;46:6111e4.

Spitzer MH, Nolan GP. Mass cytometry: single cells, many features. Cell2016;165:780e91.

Tricot S, Meyrand M, Sammicheli C, Elhmouzi-Younes J, Corneau A,Berholet S, et al. Evaluating the efficiency of isotope transmissionfor improved panel design and a comparison of the detection

sensitivities of mass cytometer instruments. Cytometry A 2015;87:357e68.

Watanabe R, Gehad A, Yang C, Scott LL, Teague JE, Schlapbach C, et al.Human skin is protected by four functionally and phenotypically discretepopulations of resident and recirculating memory T cells. Sci Transl Med2015;7:279ra39.

WongMT, Chen J, Narayanan S, LinW,Anicete R, KiaangHT, et al. Mapping thediversity of follicular helper T cells in human blood and tonsils using highdimensional mass cytometry analysis. Cell Reports 2015;11:1822.

Zunder ER, Finck R, Behbehani GK, Amir el-AD, Krishnaswamy S,Gonzalez VD, et al. Palladium-based mass tag cell barcoding with adoublet-filtering scheme and single-cell deconvolution algorithm. NatProtocols 2015;10:316e33.

This is a reprint of an article that originally appeared in the April 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Matos TR, Liu H, Ritz J. Research Techniques Made Simple: Experimental Methodologyfor Single-Cell Mass Cytometry. J Invest Dermatol 2017;137(4):e31ee38. doi:10.1016/j.jid.2017.02.006



Research Techniques Made Simple: MassCytometry Analysis Tools for Decryptingthe Complexity of Biological SystemsTiago R. Matos1,2,3, Hongye Liu1,2 and Jerome Ritz1,2

Mass cytometry by time-of-flight experiments allow analysis of over 40 functional and phenotypic cellularmarkers simultaneously at the single-cell level. The data dimensionality escalation accentuates limitations,inherent to manual analysis, as being subjective, labor-intensive, slow, and often incapable of showing thedetailed features of each unique cell within populations. The subsequent challenge of examining, visualizing,and presenting mass cytometry data has motivated continuous development of dimensionality reductionmethods. As a result, an increasing recognition of the inherent diversity and complexity of cellular networks isemerging, with the discovery of unexpected cell subpopulations, hierarchies, and developmental pathways,such as those existing within the immune system. Here, we briefly review some frequently used and accessiblemass cytometry data analysis tools, including principal component analysis (PCA); spanning-tree progressionanalysis of density-normalized events (SPADE); t-distributed stochastic neighbor embedding (t-SNE)ebasedvisualization (viSNE); automatic classification of cellular expression by nonlinear stochastic embedding(ACCENSE); and cluster identification, characterization, and regression (CITRUS). Mass cytometry, usedtogether with these innovative analytic tools, has the power to lead to key discoveries in investigativedermatology, including but not limited to identifying signaling phenotypes with predictive value for earlydiagnosis, prognosis, or relapse and a thorough characterization of intratumor heterogeneity and disease-resistant cell populations, that may ultimately unveil novel therapeutic approaches.


CME Activity Dates: 12 April 2017Expiration Date: 11 April 2018Estimated Time to Complete: 1 hour







Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-May17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

1Division of Hematologic Malignancies, Dana-Farber Cancer Institute, Boston, Massachusetts, USA; 2Harvard Medical School, Boston, Massachusetts, USA; and3Academic Medical Center, Department of Dermatology, University of Amsterdam, Amsterdam, The Netherlands

Correspondence: Tiago R. Matos, Department of Dermatology, Room L3-119, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZAmsterdam, The Netherlands. E-mail: [email protected]

Abbreviations: ACCENSE, automatic classification of cellular expression by nonlinear stochastic embedding; CITRUS, cluster identification, characterization, andregression; CyTOF, mass cytometry by time-of-flight mass spectrometry; FCS, Flow Cytometry Standard; PCA, principal component analysis; SPADE, spanning-tree progression analysis of density-normalized events; t-SNE, t-distributed stochastic neighbor embedding; viSNE, t-distributed stochastic neighborembeddingebased visualization



INTRODUCTIONNew methods are being developed to examine, visualize, andpresent the multidimensional complexity of cellular functionand identity and the role of individual cells within biologicalsystems. Mass cytometry by time-of-flight mass spectrometry(CyTOF)1 currently has the capacity to allow investigation of40 or more distinct parameters at the single-cell level(Figure 1). Although the technique has not yet been widelyadopted within the field of investigative dermatology, it haspotential to, for example, allow identification of cell signalsfor early diagnosis in cutaneous T-cell lymphoma, allow earlydetection or predict relapse in psoriasis and atopic dermatitis,and allow thorough characterization of drug-resistant cellpopulations in skin cancer, eventually unveiling new thera-pies. The large amount of data generated and potential of thetechnique to delineate rare cell subsets has driven the need todevelop dimensionality reduction methods and analysisalgorithms to best analyze and represent mass cytometry data.A significant limitation of traditional data clustering methodsthrough biaxial plots and histograms, such as has been usedto represent traditional flow cytometry data, is that pre-existing knowledge of the defining markers of each popula-tion is required. This limits the ability of researchers todiscover unexpected cellular subsets and does not allow ex-amination of system-level phenotypic diversity. Furthermore,

manual analysis of individual markers and combinations ofmarkers is a subjective, slow, and labor-intensive process,which results in a significant scalability restriction and canintroduce several inherent biases. Although CyTOF technol-ogy and experimental methodology have been described indetail in previous reviews (Doan et al., 2015; Matos et al.,2017), comprehensive understanding is also required withrespect to the tools available for analysis of high-dimensionaldatasets to make meaningful use of the results. In this shortreview, we focus on some of the most commonly used andaccessible novel CyTOF data analysis tools, including prin-cipal component analysis (PCA), spanning-tree progressionanalysis of density-normalized events (SPADE), t-distributedstochastic neighbor embedding (t-SNE)ebased visualization(viSNE), automatic classification of cellular expression bynonlinear stochastic embedding (ACCENSE), and clusteridentification, characterization, and regression (CITRUS).

DIMENSION REDUCTION AND VISUALIZATIONALGORITHMSPCAPCA is a well-established and widely used tool for visual-izing multidimensional data that was adopted to analyzelarge mass cytometry datasets (Bendall et al., 2011;Jackson, 1991; Newell et al., 2012). PCA identifies thoseparameters among a certain dataset that present the mostvariance by generating linear combinations from a large listof parameters into new compound variables (principalcomponents). As a result, the quotient of the relative vari-ation of each principle component over the total variancegives an idea of the effectiveness of each component inseparating out data points. In addition, PCA results inmodels that can be used to project new data points inlinear time. For example, it allows graphical visualizationof the expression intensity of several functional markers (y-axis) throughout the cell differentiation process (x-axis)(Figure 2). PCA also allows visualization of the data inthree-dimensional space, often prominently displaying thefirst three data components of maximal variance. However,this feature can also be a limitation, because it may masknoteworthy biological differences that are more subtlevariances in the data. Another constraint is the inherentassumption that the given data are parametric. PCA alsorepresents the data through linear projections, which maynot be representative of the inherent structure of the orig-inal data. To overcome this constraint, nonlinear methodssuch as t-SNE (described in following sections) weredeveloped for high-dimensional data analysis. Newell atal. (2012) used PCA to represent simultaneously 25markers from a single cell sample, hence quantifying theexpression of functional markers among several CD8þ T-cell subsets. This representation method displayed agreater phenotypic and functional complexity amongCD8þ T cells than previously appreciated (Figure 2). Theholistic study of many functional and phenotypic markersand their expression levels through several differentiationsubsets would not be possible by conventional manualanalysis. This study also observed that subsets that developin response to different viruses have distinct combinatorialpatterns of cytokine expression, showing the remarkable

SUMMARY POINTS� New methods are being continuously developedto analyze and best represent multidimensional,complex CyTOF data.

� Principal component analysis (PCA) provides avisualization of the data in three-dimensionalspace and identifies the parameters with themost variance among the dataset.

� Spanning-tree progression analysis of density-normalized events (SPADE) clusters cells into aminimum-spanning hierarchical tree for two-dimensional visualization.

� In t-distributed stochastic neighbor embedding(t-SNE)ebased visualization (viSNE) and auto-matic classification of cellular expression bynonlinear stochastic embedding (ACCENSE),each single cell data point has a unique locationin a two-dimensional representation, reflectingthe cells’ immunophenotypic similarity or differ-ences in high-dimensional space.

� Cluster identification, characterization, andregression (CITRUS) identifies cellular featuresthat correlate to an experimental endpoint ofinterest.

1 The abbreviation “CyTOF”, in addition to being the name of this technique, is alsothe name of a commercial product that enables researchers to use the method. Theauthors are in no way endorsing any specific commercial products.



Figure 1. Mass cytometry experiment workflow. The experimental procedure can be separated into multiple steps, including sample preparation, cell staining,cell barcoding, acquisition of CyTOF data, data processing (randomization, normalization, and cell de-barcoding), and high-dimensional data analysis. Briefly,



flexibility of CD8þ T cells in responding to pathogens. Asimilar approach could be used to study the diversity andcomplexity of skin-specific T cells or innate lymphoid cells,complementing the recent in situ topographic character-ization of innate lymphoid cells in human skin (Brüggenet al., 2016).

SPADEIn contrast to PCA, which draws out the underlying vari-ance within a dataset, the goal of clustering algorithms is tovisualize common patterns within datasets. In the contextof immune-phenotyping, automatic clustering algorithmsaim to define the most prevalent cell populations by clus-tering cells based on markers expression similarity (Levineet al., 2015).

SPADE was the first algorithm specifically developed formass cytometry data analysis that includes both clusteringand dimension reduction. In SPADE, cells are clustered into ahierarchical tree shape for two-dimensional visualization(Bendall et al., 2011; Qiu et al., 2011). Each cluster of aSPADE tree is graphically represented by a circular node inwhich the node size symbolizes the frequency of data pointswithin that population (number of hits), and the node colorshows the signal intensity of a selected marker (intensity ofhit). Each node is connected to related nodes by a minimum-spanning tree algorithm. Minimum-spanning tree is a classiccomputer algorithm that searches for a tree-like graph througha set of spatial nodes by using the least possible number ofconnections. Such a tree might mimic a relational map ofimmune cell types in their cell development. This method

cells are isolated from blood or solid tissue samples. After the optional enrichment step, the cell suspension is stained with cisplatin or rhodium to distinguishdead cells. Principally, cells were probed with surface, intracellular, or intranuclear markers after tetramer staining (if applicable). Cells can be further barcodedby mass-tag cell barcoding (after cell fixation) or CD45-Pd cell barcoding (before cell fixation) systems. Fixed cells are stained with iridium or rhodium DNA-interchelator and resuspended in deionized water for subsequent acquisition on CyTOF. Collected data are converted into an FCS file, and metal signals arenormalized. De-barcoded samples are loaded onto a bioinformative platform (usually in a MATLAB or R environment or online server such as Cytobank.org) ofchoice for high-dimensional cytometric analysis. Adapted from Cheng and Newell, 2016, with permission from Elsevier. CyTOF, time-of-flight mass cytometry;FCS, Flow Cytometry Standard.

=

Figure 2. 3D-principal componentanalysis (PCA) elucidates thephenotypic and functional complexityof CD8D T-cell memorydifferentiation. In this example, PCAwas used to represent simultaneously25 markers from a single cell sample,hence quantifying the expression offunctional markers among severalmemory CD8þ T-cell subsets. (a) Thecytometry dataset is plotted on the firstthree principal component axes andshown from three differentperspectives (rotated around the PC2-axis). Cells are gated according to theirsurface memory markers, naive(green), central memory (Tcm, yellow),effector memory (Tem, blue), andshort-lived effector (Tslec, red),which show main phenotypic clusters.(b and c) To analyze only memorycells, cells were gated to excludethe naive compartment (cells withlow value for PC1). The averageexpressions for each (b) phenotypicand (c) functional parameter werenormalized and plotted as a functionof normalized PC2 values. In this way,the phenotypic progression of CD8þ

memory T cells are represented by thex-axis (0 ¼ early differentiated Tcm,1 ¼ mature Tslec), and the y-axisrepresents the average expression ofeach marker. The functionalprogression of these numerousmarkers during CD8þ T-cell memorydifferentiation would not be possibleby conventional manual gating.Reprinted from Newell et al., 2012,with permission from Elsevier.



allows comparison between expression of markers amongclusters and across distinct samples in a dimension-reduce,big-picture view of diverse cell populations (Figure 3). Alimitation of SPADE is that it cannot reproduce the samerepresentation of results when the same dataset is analyzedmore than once, because the algorithm involves several sto-chastic steps. The rigid connections established within thedata representation structure (graph) may mislead the posi-tioning of some cell nodes, possibly obscuring the underlyingbiology. Lee et al. (2015) recently used CyTOF and SPADE todefine in detail the phenotype and functional characteristicsof distinct subsets of nasal dendritic cells in mice. SPADEclearly portrayed a map of the vast heterogeneity of nasaldendritic cells and identified new subsets that were notperceptible by manual gating on the basis of canonicalmarker expression. Moreover, it enabled simultaneous com-parison of all subsets before and after stimulating factor (FMS)-related tyrosine 3 kinase ligand treatment, showing whichsubsets became functionally active and/or expanded innumber of cells.

viSNE and ACCENSEt-SNE allows visualization of high-dimensional datathrough a nonlinear reduction where each data point isgiven a location in a two- or three-dimensional map (Van

der Maaten et al., 2008). Newer t-SNEebased strategieshave been developed to visualize complex mass cytometrydata: viSNE and ACCENSE. In contrast to PCA, thesealgorithms effectively capture nonlinear relationshipsamong the data, and unlike SPADE, they do not clustercells into exclusive nodes. Each single-cell data point has aunique location in a two-dimensional representation,similar to a biaxial scatterplot, reflecting their proximity inhigh-dimensional space. viSNE is a graphical user-interfaced tool based on t-SNE, whereas ACCENSE hasextended t-SNE with clustering of the resulting two-dimensional scatter data into density-based clusters.Thus, close proximity between any two cells is based ontheir immunophenotypic similarity, predefined by inputmarkers. For example, to generate a map of the variousmemory T-cell subpopulation markers such as CD31,CD45RA, CD45RO, CCR7, or CD62-L can be selected. Thealgorithm then clusters cells according to their similaritywithin these markers.

A color scale can then be used to visualize each marker’srelative expression in the population. However, this type ofrepresentation may obscure subtle density differences amongcell populations. Another drawback of these algorithms is theinability to currently analyze large numbers of differentsamples simultaneously. For example, the current version of

Figure 3. Flowchart of a SPADE treeconstruction and result visualization.This example illustrates how a SPADEtree is generated from raw cytometrydata and how it displays the datasetresults. (a) The cytometry datasetanalysis by two parameters detects onerare population and three abundantpopulations. (b) The original data issubjected to density-density down-sampling. (c) Agglomerative clusteringresults of the down-sampled cells. (d)Minimum spanning-tree algorithmconnects the cell clusters. (e) ColoredSPADE trees. Nodes are colored by themedian intensities of markers, allowingvisualization of themarkers’ expressionacross the numerous heterogeneouscell populations. The final SPADE treerepresentation enables determinationof how many different subsets arepresent within a dataset, the relativepopulation density (size), theexpression of various markers (color)within each subset, and the relationshipamong subsets (links). Reprinted bypermission from Macmillan PublishersLtd: Nature Biotechnology (Qiu et al.,2011), copyright 2011. SPADE,spanning-tree progression analysis ofdensity-normalized events.



the commercial software Cytobank (Cytobank, Inc., MountainView, CA) can analyze up to 2 million cells.

Healthy and cancerous bone marrow samples wererecently studied using viSNE. Healthy samples weregraphically represented and contrasted with canceroussamples that exhibited an abnormal map. Marker expres-sion patterns were analyzed from diagnosis to relapse,allowing identification of disease-specific subsets (Amiret al., 2013) (Figure 4). ACCENSE helped further charac-terize CD8þ T-cell subpopulations and identify new sub-sets that were not noticeable on biaxial plots (Shekharet al., 2014) (Figure 5).

CLUSTERING AND CORRELATION WITH CLINICALOUTCOMECITRUSCITRUS aims to not only showcase biological mechanismsand hierarchies and define population subsets, similar topreviously described algorithms, but also to identifycellular features that correlate to an experimental endpointof interest. Bruggner et al. (2014) showed that CITRUSaccurately identified subsets of cells associated with AIDS-free survival in HIV-infected patients. CITRUS clusters cellsbased on similarity of cell markers. Such methods candelineate clusters of cells with specific behavioral

Figure 4. viSNE creates a map of the immune system. (a) In this example, viSNE projects a one-dimensional curve embedded in three dimensions (left) to twodimensions (right). The color gradient shows that nearby points in three dimensions remain close in two dimensions. (b) Application of viSNE to a healthy humanbone marrow sample. viSNE automatically separates cells based on their subtype. Each point in the viSNE map represents an individual cell, and its colorrepresents its immune subtype based on independent manual gating. The axes are in arbitrary units. (c) Biaxial plots representing the same data shown in b. Selectsubpopulations are shown with canonical markers where the square color matches the subtype in b. The actual gating used is more complex and uses a series ofbiaxial plots for each population. Note, unlike b, these plots do not separate between all subtypes in a single viewpoint. (d) The same viSNE map represented in b,but this time each cell is colored based on CD11b expression. Gated cells are all CD33 high and show a CD11b (maturity) gradient. Many of these cells were notclassified as monocytes by manual gating (grey cells b). Reprinted by permission from Macmillan Publishers Ltd: Nature Biotechnology (Amir et al., 2013),copyright 2013. viSNE, t-distributed stochastic neighbor embeddingebased visualization.



characteristics such as those that have activated specificsignaling pathways, which can potentially be associatedwith clinical outcomes such as time to recovery or overallsurvival after a drug treatment or surgery. By specifying theoutcome of interest for each sample from the vast cytom-etry data uploaded, regularized statistic algorithms canrecognize cells exhibiting behavior predictive of outcome.The phenotype and behaviors of each cluster are delin-eated through distinct data representations including con-ventional biaxial plots that can result in a predictive modelfor the analyses or validation of future samples. To runcorrelation analysis with enough statistical power, CITRUSrequires input from more than eight samples for eachgroup. Gaudillière et al. (2014) applied this method tomonitor patients undergoing hip replacement surgery tocharacterize the phenotypical and functional immune re-sponses predictive of recovery from surgical trauma(Figure 6). Because the patients’ clinical histories of post-surgical recovery were known, CITRUS identified intra-cellular signaling markers that strongly correlated withrecovery from fatigue, functional hip impairment, and painafter surgery. Likewise, CITRUS could be used to analyzeimmune cell populations and molecular markers from pa-tients with inflammatory or autoimmune diseases beforeand after receiving biologic treatments. It could showwhich cell subset or markers are predictive of therapeuticoutcome, allowing early adjustments to treatment pro-tocols in patients predicted to be nonresponders, reducingadverse effects and costs.

AVAILABILITY OF SOFTWAREAfter mass cytometry data acquisition, each respectivesample must be deconvoluted if samples were initially

barcoded and normalized based on calibration beads.CyTOF data can be exported in the form of Flow CytometryStandard (FCS) files, which can be analyzed by standardflow cytometry software such as FCS Express (De NovoSoftware, Los Angeles, CA) and FlowJo (FlowJo, LLC,Ashland, OR), or by using cloud-based analysis tools, suchas Cytobank. Cytobank allows transfer and storage ofmultiple CyTOF FCS files; attachment of related dataincluding protocols, presentations, annotations, and im-ages; and sharing of data and analysis with chosen col-laborators. Additional analytical tools may be needed,including but not limited to dose response, heat maps,SPADE, viSNE, CITRUS, and dot and histogram overlays.viSNE can be publicly licensed to academic users as aMATLAB-based tool from the Dana Pe’er Lab of Compu-tational Systems of Biology webpage (http://www.c2b2.columbia.edu/danapeerlab/html/software.html). The NolanLaboratory also publically offers some of these algorithms,such as SPADE and CITRUS (https://github.com/nolanlab).ACCENSE is freely offered at http://www.cellaccense.com,and a PCA algorithm is included in the open-sourced Rbasic package (The R Foundation, Vienna, Austria).These analysis tools may provide different types of bio-logical information about the same dataset, making itacceptable to concurrently use distinct tools in a singleexperiment (Table 1). Nevertheless, the accessibility toCyTOF analysis tools continues to improve, with numerousgroups developing similar and new analysis methods forCyTOF data.

CONCLUSIONThe continuous development and enhancement of analysistools for mass cytometry expands our ability to study

Figure 5. ACCENSE quantifies distinct subsets of CD8D T cells. In this example, ACCENSE was able to stratify cells into phenotypic subsets by the localprobability density of cells (ACCENSE map location of hits) based on the 35 markers studied. It identified a novel subset with a unique multivariate phenotype thatis not distinguishable on a biaxial plot of markers. (a) Illustration of a sample mass cytometry dataset. Rows correspond to different cells, and columns correspondto the different markers. Entries correspond to transformed values of mass-charge ratios that indicate expression levels of each marker. (b) Biaxial plots exemplifythe manual gating approach to identify cell subsets. (c) The two-dimensional t-SNE map of CD8þ T cells, where each point represents a cell (n ¼ 18,304) derivedby down-sampling the original dataset. (d) A composite map depicting the local probability density of cells as embedded in panel c, computed using a kernel-based transform. Local maxima in this two-dimensional density map represent centers of phenotypic subpopulations and were identified using a standard peak-detection algorithm. viSNE also generates a similar two-dimensional t-SNE map; however, each subpopulation has to be identified and demarcated manually,whereas ACCENSE automatically defines subsets based on local cell density in the map. Reprinted from Shekhar et al., 2014, in accordance with permissionrights to reprint material published in PNAS. ACCENSE, automatic classification of cellular expression by nonlinear stochastic embedding. t-SNE, t-distributedstochastic neighbor embedding; viSNE, t-distributed stochastic neighbor embeddingebased visualization.



complex and heterogeneous biological systems at the levelof individual cells. This enables our understanding of theprogression and development of healthy and pathologiccells, such as in psoriasis, atopic dermatitis, and vitiligo.For example, why do some lesions reoccur only in thesame anatomical site? Why are some areas of the body

more commonly affected? What distinguishes pathologicalcells in a stable versus a progressive disease? Disease-specific cell subsets can be identified, characterized,monitored during treatment, and perhaps screened forearly biomarkers predictive of relapse risk. Differences maybe revealed among cells responsible for the clinical het-erogeneity of cutaneous T-cell lymphoma, ultimatelyunveiling disease-specific biomarkers and personalizednovel therapeutic approaches. Innovative therapies can bestudied to specifically target malignant cell populationsresistant to conventional treatments, such as in melanoma.Ultimately, the goal of this article is to demystify thedeveloping tools for mass cytometry and its data analysis sothese technologies can be adopted and the results under-stood to address these and other important research ques-tions in the coming years.


Table 1. Analysis algorithms for mass cytometry dataAlgorithmName

Type ofInformation Advantages Limitations Reference

PCA Parameterswith mostvariancewithindataset

Visualizationin 3D space

� May miss sub-tle varianceswithin data

� Assumes thatdata areparametric

� Data represen-tation isrestricted tolinearprojections

Jackson,1991;Newellet al.,2012

SPADE Cellpopulationhierarchies

� Delineates thepresence ofrare cell types

� Can compareclusters andexpressionmarkersamong cellsubsets andacrosssamples

� Lacksreproducibility

� Rigid structureconnectivity

Qiu et al.,2011

t-SNEebasedviSNE andACCENSE

Cell subsetheterogeneity

Single-cellrepresentation(withoutclustering)

Unfeasible toanalyze vastnumbers of cells1

Amir et al.,2013;Shekharet al.,2014

CITRUS Allowscorrelationbetweensample andclinicaloutcome

Correlation toexperimentalendpoint ofinterest

More than eightsamples pergroup arerequired

Bruggneret al.,2014

Abbreviations: 3D, three-dimensional; ACCENSE, automatic classificationof cellular expression by nonlinear stochastic embedding; CITRUS, clusteridentification, characterization, and regression; PCA, principal componentanalysis; SPADE, spanning-tree progression analysis of density-normalizedevents; t-SNE, t-distributed stochastic neighbor embedding; viSNE,t-distributed stochastic neighbor embeddingebased visualization.1The cloud-based software Cytobank viSNE can currently handle up to2 million cells.

Figure 6. Overview of CITRUS. CITRUS enables correlation of the multitudeof cellular parameters studied with clinical outcomes information. Cells from(a) all samples are combined (b) and clustered using hierarchical clustering. (c)Descriptive features of identified cell subsets are calculated on a per-samplebasis and (d) used in conjunction with additional experimental metadata totrain (e) a regularized regression model predictive of the experimentalendpoint. (f) Predictive subset features are plotted as a function ofexperimental endpoint, (g) along with scatter or density plots of thecorresponding informative subset. In this example, the abundance of cells insubset A was found to differ between healthy and diseased samples (f; subsetA abundance in healthy patients and diseased patients). Scatter plots showthat cells in subset A have high expression of marker 1 and low expression ofmarker 2 relative to all other cells (shown in gray). In this study, CITRUSidentified T-cell subsets whose abundance is predictive of AIDS-free survivalrisk in patients with HIV. Reprinted from Bruggner et al., 2014, in accordancewith permission rights to reprint material published in PNAS.



ACKNOLWEDGMENTSWe would like to thank Dr. Jodi L. Johnson for helpful comments, criticalreading of the manuscript, and editorial assistance.


REFERENCESAmir E-AD, Davis KL, Tadmor MD, Simonds EF, Levine JH, Bendall SC, et al.

viSNE enables visualization of high dimensional single-cell data andreveals phenotypic heterogeneity of leukemia. Nat Biotechnol 2013;31:545e52.

Bendall SC, Simonds EF, Qiu P, Amir ED, Krutzik PO, Finck R, et al.Single cell mass cytometry of differential immune and drug responsesacross the human hematopoietic continuum. Science 2011;332(6030):687e96.

Brüggen MC, Bauer WM, Reininger B, Clim E, Captarencu C, Steiner GE, et al.In situ mapping of innate lymphoid cells in human skin: evidence forremarkable differences between normal and inflamed skin. J InvestDermatol 2016;136:2396e405.

Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automatedidentification of stratifying signatures in cellular subpopulations. ProcNatl Acad Sci USA 2014;111:E2770e7.

Cheng Y, Newell EW. Deep profiling human T cell heterogeneity by masscytometry. Adv Immunol 2016;131:101e34.

Doan H, Chinn GM, Jahan-Tigh RR. Flow cytometry II: mass and imagingcytometry. J Invest Dermatol 2015;135:e36.

Gaudillière B, Fragiadakis GK, Bruggner RV, Nicolau M, Finck R, Tingle M,et al. Clinical recovery from surgery correlates with single-cell immunesignatures. Sci Transl Med 2014;6:255ra131.

Jackson JE. PCA with more than two variables. In: Jackson JE, editor. User’sguide to principal component analysis. New York: John Wiley and Sons;1991. p. 26e62.

LeeH, RuaneD, Law K,Ho Y, Garg A, Rahman A, et al. Phenotype and functionof nasal dendritic cells. Mucosal Immunol 2015;8:1083e98.

Levine JH, Simonds EF, Bendall SC, Davis KL, Amir el-AD, Tadmor MD, et al.Data-driven phenotypic dissection of AML reveals progenitor-like cellsthat correlate with prognosis. Cell 2015;162:184e97.

Matos TR, Liu H, Ritz J. Experimental methodology for single-cell masscytometry. J Invest Dermatol 2017;137:e31e8.

Newell EW, Sigal N, Bendall SC, Nolan GP, Davis MM. Cytometry by time-of-flight shows combinatorial cytokine expression and virus-specific cellniches within a continuum of CD8þ T cell phenotypes. Immunity2012;36:142e52.

Qiu P, Simonds EF, Bendall SC, Gibbs KD Jr, Bruggner RV, Linderman MD,et al. Extracting a cellular hierarchy from high-dimensional cytometrydata with SPADE. Nat Biotechnol 2011;29:886e91.

Shekhar K, Brodin P, Davis MM, Chakraborty AK. Automatic classification ofcellular expression by nonlinear stochastic embedding (ACCENSE). ProcNatl Acad Sci USA 2014;111:202e7.

Van der Maaten L, Hinton J. Visualizing data using t-SNE. J Mach Learn Res2008;9(85):2579e605.

MULTIPLE CHOICE QUESTIONS1. What is the best CyTOF data analysis tool?

A. Same methods as for flow cytometry

B. The analysis method depends on specificexperimental goals.

C. Manual clustering methods through biaxial plots

D. Comparisons of marker expression usinghistograms

2. Identify one advantage of principal componentanalysis (PCA).

A. Displays data in two-dimensionalrepresentation

B. Results are represented through linearprojections

C. Identifies parameters with the most variance

D. Capable of analyzing only a few parameters

3. Which of the following is a limitation ofspanning-tree progression analysis ofdensity-normalized events (SPADE)?

A. Incapable of reproducing the samerepresentation of results when analyzed morethan once

B. Represents cell subset hierarchies

C. Assumes that data is parametric

D. Does not allow comparing markerexpression among subsets and samples

4. Select one advantage of t-distributed stochasticneighbor embedding (t-SNE)ebased visualization(viSNE) and automatic classification of cellularexpression by nonlinear stochastic embedding(ACCENSE).

A. It captures only nonlinear relationshipsamong the dataset.

B. It clusters cells into exclusive nodes.

C. It allows representation of single cellswithout clustering

D. It allows the researcher to identify clusterscapable of predicting the sample’s outcome.

5. What is the novel application of analyzing datausing cluster identification, characterization, andregression (CITRUS)?

A. It is capable of displaying data inthree-dimensional representations.

B. It identifies rare cell subsets with the highestexpression of studied markers.

C. It allows the researcher to define cellpopulation hierarchies.

D. It identifies cellular features that correlate toan experimental endpoint of interest.

This is a reprint of an article that originally appeared in the May 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Matos TR, Liu H, Ritz J. Research Techniques Made Simple: Mass Cytometry Analysis Toolsfor Decrypting the Complexity of Biological Systems. J Invest Dermatol 2017;137(5):e43ee51. doi:10.1016/j.jid.2017.03.002



Research Techniques Made Simple:High-Throughput Sequencing of the T-Cell ReceptorTiago R. Matos1, Menno A. de Rie1 and Marcel B.M. Teunissen1

High-throughput sequencing (HTS) of the T-cell receptor (TCR) is a rapidly advancing technique that allowssensitive and accurate identification and quantification of every distinct T-cell clone present within any bio-logical sample. The relative frequency of each individual clone within the full T-cell repertoire can also bestudied. HTS is essential to expand our knowledge on the diversity of the TCR repertoire in homeostasis orunder pathologic conditions, as well as to understand the kinetics of antigen-specific T-cell responses that leadto protective immunity (i.e., vaccination) or immune-related disorders (i.e., autoimmunity and cancer). HTS canbe tailored for personalized medicine, having the potential to monitor individual responses to therapeuticinterventions and show prognostic and diagnostic biomarkers. In this article, we briefly review the method-ology, advances, and limitations of HTS of the TCR and describe emerging applications of this technique in thefield of investigative dermatology. We highlight studying the pathogenesis of T cells in allergic dermatitis andthe application of HTS of the TCR in diagnosing, detecting recurrence early, and monitoring responses totherapy in cutaneous T-cell lymphoma.


CME Activity Dates: 19 May 2017Expiration Date: 18 May 2018Estimated Time to Complete: 1 hour







Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-June17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

INTRODUCTIONT lymphocytes form an essential component of the adaptiveimmune system. Each individual T cell expresses a unique T-cell receptor (TCR) that specifically recognizes a single

antigenic determinant. Taking the whole T-cell populationtogether, the adaptive immune system has a spectacularlylarge and highly diverse TCR repertoire at its disposal,capable of identifying unlimited numbers of antigens, hence

1Academic Medical Center, Department of Dermatology, University of Amsterdam, Amsterdam, The Netherlands

Correspondence: Tiago R. Matos, Department of Dermatology, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, TheNetherlands. E-mail: [email protected]

Abbreviations: CDR3, complementarity determining region 3; CTCL, cutaneous T-cell lymphoma; HTS, high-throughput sequencing; LAM-PCR, linearamplification-mediated PCR; nrLAM-PCR, nonrestrictive linear amplification-mediated PCR; TCR, T-cell receptor; TRM, resident memory T cells; V(D)J, variable,diversity, joining



providing protection against constant diverse pathogenicthreats. TCRs are heterodimer molecules consisting of either acombination of a and b chains, the most common TCR, or acombination of g and d chains (Figure 1). The superb speci-ficity and diversity occurs during lymphocyte development byrandomized combinations of DNA from distinct variable,diversity, joining (V(D)J) gene segments and by deletion and/or insertion of nucleotides at the junctions of these segments,

which in particular takes place in the hypervariablecomplementarity determining region 3 (CDR3). This hyper-mutation process of TCR genes can lead to over 1018 differentab TCRs, making it highly improbable that two TCRs with anidentical nucleotide CDR3 sequence will be generated(Murphy, 2011). Consequently, the TCR nucleotide sequenceof each T cell is akin to a barcode that enables recognitionand tracking of each specific T-cell clone (Figure 1).

OVERVIEW OF METHODOLOGYA recently developed methodology, known as immunose-quencing, combines bias-controlled multiplexed PCR withhigh-throughput sequencing (HTS) of the CDR3 of the TCR.Subsequent innovative bioinformatic analysis enables thesimultaneous identification, quantification, and tracking ofeach individual lymphocyte and of the entire repertoirewithin any sample of interest (Figure 2). To describe HTS ofthe TCR briefly, DNA or mRNA is extracted from the bio-logic sample of interest (e.g., blood or skin), and the CDR3region is amplified from the isolated DNA or cDNA (syn-thetized by reverse transcription of mRNA). Then, in asecond PCR step, bias-controlled V and J gene primers areused to further amplify the rearranged V(D)J segments,which finally are subjected to HTS. Clustering algorithmsare subsequently used to correct the raw data forsequencing errors (Figure 3). Lastly, the unique CDR3segments and the V(D)J genes within each rearrangementare identified and quantified (Figure 2), based on previ-ously described sequences, which can be accessed withinthe IMGT data bank (http://www.imgt.org), offering acommon standard for nomenclature, numbering, andannotation (Lefranc et al., 2009; Robins et al., 2009). Theseresults allow identification of somatic allelic mutations,study of the lymphoid clonality and diversity in healthy andmalignant tissues, and tracking of each clone over time(e.g., as a biomarker during disease progression or therapy)among different anatomical sites or within definedpopulations.

VARIANTS OF PCR AND OTHER TECHNIQUES TO STUDYTHE TCRMultiple variants of PCR can be applied to analyze theCDR3, and for a better understanding of the literature, somerelevant methods will be briefly explained. Most PCR pro-tocols require an initial treatment with restriction enzymesto reduce the DNA length or to enable linking of definedoligonucleotides and use double-strand DNA synthesisduring amplification, known as linear amplification-mediated PCR (LAM-PCR). As an alternative approach,nonrestrictive LAM-PCR (nrLAM-PCR) avoids using restric-tion enzymes while the single DNA strands are amplifieddirectly by competitive exponential PCR (Gabriel, 2009).Because the time span of the linear DNA synthesis step islimited, the resultant nrLAM-PCR products display substan-tial variability in length, because size is no longer deter-mined by restriction enzymes. As a consequence, analysisby gel electrophoresis is not possible (PCR products appearas a smear instead of a sharp band), but nrLAM-PCR allowsunbiased HTS by restriction site motifs. The overall sensi-tivity and efficiency of nrLAM-PCR is superior compared

SUMMARY POINTSWhat HTS of the TCR DoesHTS of the TCR accurately identifies and quantifies eachand every T cell present in a certain biological sample,allowing study of each unique clone and the full T-cellrepertoire, including over time and among differenttissues and/or individuals.

Advantages of HTS of TCR� High clone detection sensitivity. Approximately100-fold greater than other current technologies(e.g., flow cytometry).

� Extremely accurate, with lower rate of falsenegatives and false positives than analysis ofthe TCR by PCR.

� Capable of successfully studying any type oftissue and small samples, including standard-sizepunch biopsy or shave biopsy samples, becauseit is a cDNA- or DNA-based technology, and theamplification step enables the detection ofscarce cells.

� Technique does not require radioactive agents asin the previously commonly used Southernblotting.

� Can be used for a wide range of applications,from expanding our knowledge of the adaptiveimmune system to monitoring responses totherapeutic interventions and studying newdiagnostic and prognostic biomarkers.

Limitations of HTS of TCR� Technology’s sensitivity is limited only by theamount of DNA probed. For example, if the DNAof a million cells is analyzed, then the clonedetection sensitivity is about 1:1,000,000.

� Variations in tissue processing can lead to DNAdegradation. Section thickness and sample cellsize may affect accurate amplification and rep-resentation of all gene segments. A certain genemay also be lost as a cell undergoes malignanttransformation.

� Pairing the awith b or g with d chains of a specificTCR is generally possible only when analyzingsingle cells.

� It is generally still not possible to match thestudied TCR sequences to their specific epitope.



with other LAM-PCR techniques if the amount of input DNAis not a strict limitation (Gabriel, 2009).

Rapid amplification of cDNA ends, also known as one-sided PCR or anchored PCR, starts with the generation ofthe first strand of cDNA with a specific primer targeting aknown sequence within the mRNA transcript (e.g., C-generegion of the TCR), and the other terminus is covalently linked

to an “anchor” oligonucleotide sequence. Then, double-strand cDNA synthesis by PCR is performed by usinga primer that specifically binds to the anchor and to a secondprimer of interest.

Multiplex PCR uses multiple primer sets within a singlePCR mixture, resulting in multiple PCR products (amplicons)of varying sizes. For example, multiple primers covering all

Figure 2. High-throughput TCRsequencing. (a) The biological sampleof interest is collected. (b) DNA isextracted or cDNA is synthetized. (c)Bias-controlled multiplexed PCRamplifies and sequences the CDR3from the DNA or cDNA. Then, bias-controlled V and J gene primers areused to amplify the rearranged V(D)Jsegments. (d) Bioinformatics can thenbe used to identify, quantify, and trackeach individual lymphocyte and theentire repertoire within any sample ofinterest. It is possible to identify andquantify the unique CDR3 segmentsand the V(D)J genes within eachrearrangement, based on previouslydescribed sequences that can beaccessed within data banks. CDR3,complementarity determiningregion 3; TCR, T-cell receptor; V(D)J,variable, diversity, joining.

T cell

Variable region

Constant region

T cell receptor (TCR)

or

α β γ δ

CV D JGenes

mRNA

Germline genome

Rearranged DNA of mature cell line

Transcrip on

Transla on

Single pep de chain

a b

Figure 1. The T-cell receptor (TCR) can function as a unique identifying bar code of T cells. (a) In the germline genome, the multiple gene segments of the TCRhave not yet been rearranged. Specificity and diversity occurs during lymphocyte development by combining the distinct variable (V), diversity (D), and joining (J)gene segments and by deletion and/or insertion of nucleotides at the junctions of those segments (C, Constant gene segments). This randomized process makes ithighly improbable that two T-cell receptors with the same complimentary determining region 3 (CDR3) nucleotide sequence will be generated. Thus, the uniqueTCR nucleotide sequence of each T cell is akin to a bar code that enables recognition and tracking of each specific T-cell clone. The unique mRNAs aretranslated into the peptide chains of the TCR. (b) TCR proteins heterodimerize to form either a combination of a and b chains or g and d chains.



Figure 3. High-throughput TCR CDR3 sequencing captures entire T-cell diversity. (a) Comparison of standard TCRb spectratype data and calculated TCRbCDR3 length distributions for sequences using representative TCR Vb gene segments. CDR3 length is plotted along the x-axis, and the number of unique CDR3sequences with that length or the relative intensity of the corresponding peak in the spectratype is plotted along the y-axis. The length of the differently coloredsegments within each bar of the histograms indicates the fraction of unique CDR3 sequences that was observed 1e5 times (black), 6e10 times (blue), 11 e100times (green), or more than 100 times (red). (b) A representative spectratype of TCRb CDR3 cells that use the Vb10 gene segment. The CDR3 sequences were



known V genes can be used when studying the lengths andsequences of the CDR3 of the TCR repertoire. The design ofthe primers for multiplex PCR is of utmost importance: theprimer length (18e22 bases) and the annealing and meltingtemperature must be optimized to act accurately within asingle reaction mixture. In addition, binding of primers toeach other (primer dimers) must be excluded, and a linearamplification protocol must be applied to prevent bias be-tween the primers.

The TCR repertoire can also be analyzed by other tech-niques besides PCR, including flow cytometry or masscytometry (Table 1). However, comprehensive analysis of theTCR repertoire by cytometry is restricted because of thelimited availability of anti-TCR antibodies.

TERMS TO DESCRIBE TCR REPERTOIRE DIVERSITYThe lexicon of terms applied to describe the diversity of T-cellclones in biological samples includes metric terms such asclonality, richness, and entropy.

Clonality is a metric of relative abundance that allowsevaluation of clonal expansion based on the probability offinding the same sequence in two different replicates. Ahigher clonality score reflects greater clonal expansion andcan be attributed to expansion within the memory compart-ment, an uneven homeostatic proliferation of the naive T-cellrepertoire, or may be indicative of a T-cell tumor. Forexample, cutaneous T-cell lymphoma (CTCL) samplescommonly have a high clonality value, because there is asingle pathogenic T-cell clone largely predominating in skinlesions (Chitgopeker et al., 2014).

Richnessmeasures how many distinct T-cell clones (uniqueTCRs) are present in a sample. The diversity richness of therepertoire is strongly linked to a healthy immune system,ensuring continuous surveillance and response to unlimitedforeign antigens, controlling for acute and chronic infections,and preventing mutant cells from unrestricted proliferation.

Entropy provides a theoretic measurement of the proba-bility of a certain clone being present within the total T-cellrepertoire, combining information on abundance and rich-ness. Entropy is the degree of uncertainty associated withidentifying clones in the total repertoire based on the totalnumber of clones, their identity, and their relativeabundances.

APPLICATIONS OF HIGH-THROUGHPUT TCR SEQUENCINGIN DERMATOLOGYHigh-throughput TCR sequencing is increasingly becomingan important tool in the field of investigative dermatology,including monitoring the immune response to infectiousdiseases or vaccines, studying pathogenesis of T-celleassociated diseases, facilitating diagnosis and earlyrecurrence detection, and assessing T-cell responses duringtherapies.

Monitoring the immune response to infectious diseases orvaccinesThe exposure to pathogens or vaccines induces a specificadaptive immune reaction, which can be monitored by HTS.The study of the TCRs of all present T cells allows analyzingthe efficacy of inducing persistent T-cell memory by trackingunique sequences that were initially present in the naive

sorted by CDR3 length into a frequency histogram, and the sequences within each length were then color-coded on the basis of their Jb use. The inset representsCDR3 sequences having a length of 39 nucleotides (nt), as well as the number of times that each of these sequences was observed in the data. The origin of thenucleotides in each sequence is color-coded as follows: Vb gene segment, red; template-independent N nucleotide, black; Db gene segment, blue; Jb genesegment, green. (c) Shown are the results of TCRg HTS of lesional skin from a patient with stage III cutaneous T-cell lymphoma. The two rearranged TCRg allelesequences of the malignant clone are indicated by asterisks. A healthy diverse population of benign infiltrating T cells was present. Parts a and b were adaptedfrom Robins et al. (2009), with permission from the American Society of Hematology. Part c was adapted from Kirsch et al. (2015), with permission from TheAmerican Association for the Advancement of Science. CDR3, complementarity determining region 3; D, diversity; HTS, high-throughput sequencing; J, joining;TCR, T-cell receptor; V, variable.

=

Table 1. Comparison of approaches for analysis of the TCRType of Information Flow Cytometry Mass Cytometry HTS of the TCR

Protein/mRNA/DNA level Protein Protein mRNA/DNATissue sample preparation Obligatory to prepare single-cell

suspension from tissueObligatory to prepare single-cell

suspension from tissueDirect extraction of mRNA/DNA

from tissueCellular throughput(approximate maximum)

25,000 events/second 2,000 events/second Not applicable

Combined analysis of othermarkers

Up to �20 Up to �40 Not possible1

Dependent on availability ofspecific antibodies

Yes Yes No

Antigen specificity possible Up to �30 (combinationof tetramers)

Up to �100 (combinationof tetramers)

No

TCR repertoire information Limited2 Limited2 CompleteFurther informationabout TCR sequence repertoire

Cells can be sorted andthen subjected to HTS

No, cells are destroyedthrough the mass cytometry process

Can identify and quantify the V(D)Jgenes and the nucleotide and amino

acid sequences

Abbreviations: HTS, high-throughput sequencing; TCR, T-cell receptor; V(D)J, variable, diversity, joining.1Possible only when starting with sorted single cells or cloned cells.2Availability of antibodies to TCR repertoire is incomplete.



T-cell population but appear in the T-cell population with amemory phenotype after an immune response. Moreover, itallows tracking the memory T cells over time, unveilingwhether the vaccine or infection provides long-lasting pro-tection. This cumulative knowledge may allow researchers toidentify T-cell clones that exclusively react to an antigen ofinterest, which might eventually be used as a biomarker ofexposure or for early detection and prevention of diseasespread.

Gaide et al. (2015) used HTS of the TCR to clarify theclonal origin of both central memory T cells in lymphnodes and resident memory T (TRM) cells in peripheraltissues after skin vaccination. They noted that after skinimmunization, a mutual native T-cell precursor gives rise

to both antigen-reactive skin TRM cells and lymph nodecentral memory T cells with overlapping TCR repertoires,hence creating memory T cells with different effectorproperties but with identical antigen specificity in distincttissue compartments. In addition, TRM cells could rapidlygenerate a contact hypersensitivity response, whereascentral memory T cells were responsible for a moderateand delayed response. These data indicate that TRM cellsmediate allergic contact dermatitis, which explains thesite-specificity, recurrence, and refractory characteristics ofthis disease. Similar approaches may help in the study ofhuman diseases for which the clinical features resembleTRM-mediated diseases, including psoriasis, vitiligo, andfixed drug eruption.

Figure 4. High-throughput TCRb CDR3 region sequencing identifies expanded T-cell clones and discriminates CTCL from benign inflammatory skin disorders.(a) Clonality of lesional skin T cells increased with advanced stage of CTCL. (b, c) TCR sequencing identified expanded populations of clonal malignant T cells inCTCL skin lesions. (b) The V versus J gene usages of T cells from a lesional skin sample are shown. The green peak includes the clonal malignant T-cellpopulation. (c) The individual T-cell clone sequence is shown with detailed information on the CDR3 amino acid sequence and V and J gene usage. The ninemost frequent TCR sequences of benign infiltrating T cells are also shown. In this patient, the malignant T-cell clone made up 10.3% of the total T-cell populationin lesional skin. (d, e) The most frequent T-cell clone expressed as the fraction of total nucleated cells successfully discriminates CTCL from benign inflammatoryskin diseases. The most frequent TCR sequence expressed as a fraction of total nucleated cells is shown for (d) individual samples and (e) aggregate data. Thisanalysis allowed discrimination of CTCL from benign inflammatory skin diseases and healthy skin. Reprinted from Kirsch et al. (2015), with permission from TheAmerican Association for the Advancement of Science. ACD, allergic contact dermatitis; CDR3, complementarity determining region 3; CTCL, cutaneous T-celllymphoma; ED, eczematous dermatitis; J, joining; MF, mycosis fungoides; Nml, normal; TCR, T-cell receptor; V, variable.



Studying pathogenesis of T-cell diseasesBesides elucidating the pathogenesis of allergic contactdermatitis, HTS also allowed demonstration that allopurinol-induced severe cutaneous adverse reactions are driven byclonotype-specific T cells in a dose-dependent response tooxypurinol (Chung et al., 2015). Studies of the TCR g gene byHTS showed that CTCL is caused by mature memory T cellsafter undergoing normal thymic maturation and not bylymphoid progenitor cells or immature T cells (Kirsch et al.,2015). By means of a modified HTS technique, Ruggieroet al. (2015) assessed the TCR a- and b-chain diversity inSézary syndrome, and correlations between the restriction ofthe repertoire and clinical severity of CTCL skin involvementwere found.

Facilitating diagnosis and early recurrence detectionHigh-throughput TCR sequencing has been shown to facili-tate the diagnosis of CTCL. TCRg PCR is currently the com-mon diagnostic method of CTCL; however, it detects only themalignant T-cell clone in a subgroup of patients. Kirsch et al.(2015) showed that HTS of the TCR b and g alleles is moresensitive and specific than TCRg PCR in detecting the path-ogenic expanded T-cell clone. HTS can function as an earlydefinitive diagnostic tool for CTCL, and importantly, distin-guish early stages of CTCL from benign inflammatory skindiseases (Figure 4).

Assessing immune response to therapyIn a clinical trial testing topical resiquimod gel (an agonist fortoll-like receptors 7 and 8) as a treatment for CTCL, it wasnoted that HTS of the TCR was more specific in assessingmalignant T-cell clone clearance than clinical score evalua-tion (Rook et al., 2015). HTS showed that the malignant T-cellclone was reduced in 90% of the treated patients and thatmalignant T-cell eradication was correlated with the recruit-ment and expansion of new benign T cells.

SUMMARY AND FUTURE DIRECTIONSHTS of the TCR is a highly sensitive and precise techniquethat allows quantification of the relative frequency of eachclone within the full T-cell repertoire. It enables concomitantstudy of each unique T cell and all clonal populations overtime and comparison of different biologic tissues of the sameindividual or among different individuals who may be eitherhealthy or suffering from a certain disease.

HTS technology is based on the specific detection of theCDR3 region on just one of the two chains of the TCR het-erodimer. An important next step forward would be estab-lishment of technology that is able to simultaneously defineboth chains of the TCR. Currently, the most common way todefine both peptide chains of the same TCR is throughsingle-cell analysis. However, a recently developed methodcalled pairSEQ seems to be able to accurately pair TCRa andTCRb sequences out of hundreds of thousands of lympho-cytes without the need for single-cell technologies (Howieet al., 2015). Once established, this development maysignificantly contribute to another highly anticipated goal:being able to link TCR sequences to their specific targetantigenic peptides. These new methods will augment TCRdatasets with complementary information on the targetepitope recognized by each TCR and link these TCR

sequences to lymphocytic phenotypic markers. In addition,these new methods build on the current applications of HTSas a monitoring tool for lymphoid malignancies and he-matopoietic transplants and potentiate the identification anddevelopment of lymphocytes specific for tumor antigens orself-antigens for anticancer or autoimmune therapeutics,respectively.

HTS is meaningfully expanding our understanding of thecomplex and exquisite role of T cells in the immune systemand may lead us in groundbreaking personalized medicine

MULTIPLE CHOICE QUESTIONS1. What does HTS of the TCR identify?

A. Identifies the DNA sequence of the entireTCR

B. Identifies the specific epitope of each TCR

C. Identifies the variable and constant region ofeach TCR

D. Identifies and quantifies each and everyT cell present in a sample

2. All of the following are advantages of HTS of theTCR, except the following:

A. Uses highly accurate radioactive agents

B. High clone detection sensitivity

C. Can be applied to any biologic tissue andsmall samples

D. Low rate of false positives and false negatives

3. All of the following steps are part of the HTS ofthe TCR methodology, except the following:

A. Bias-controlled multiplexed PCR

B. Western blot

C. Advanced bioinformatics

D. HTS of the CDR3 of the TCR

4. What does the clonality score measure?

A. The probability of a clone being presentwithin the total cell repertoire

B. How much a sample is dominated by clonalexpansion

C. How many distinct clones are present in asample

D. How many T cells belong to each clonalpopulation

5. HTS can be used to study the following:

A. The immune response to infectious diseasesor vaccine

B. Pathogenesis of T-celleassociated diseases

C. Diagnosis biomarkers and/or immuneresponse to therapies.

D. All of the above



by accurately monitoring and improving therapeutic in-terventions and possibly uncovering diagnostic and prog-nostic biomarkers.


ACKNOWLEDGMENTSWe would like to thank Dr. Jodi L. Johnson for helpful comments, criticalreading of the manuscript, and editorial assistance, and Dr. Rachael A. Clarkfor the valuable scientific advice and guidance. This work was supported byFondation René Touraine.


REFERENCESChitgopeker P, Sahni D. T-cell receptor gene rearrangement detection in

suspected cases of cutaneous T-cell lymphoma. J Invest Dermatol2014;134:e19.

Chung WH, Pan RY, Chu MT, Chin SW, Huang YL, Wang WC, et al. Oxy-purinol-specific T cells possess preferential TCR clonotypes and expressgranulysin in allopurinol-induced severe cutaneous adverse reactions.J Invest Dermatol 2015;135:2237e48.

Gabriel R, Eckenberg R, Paruzynski A, Bartholomae CC, Nowrouzi A,Arens A, et al. Comprehensive genomic access to vector integration inclinical gene therapy. Nat Med 2009;15:1431e6.

Gaide O, Emerson RO, Jiang X, Gulati N, Nizza S, Desmarais C, et al.Common clonal origin of central and resident memory T cells followingskin immunization. Nat Med 2015;21:647e53.

Howie B, Sherwood AM, Berkebile AD, Berka J, Emerson RO,Williamson DW, et al. High-throughput pairing of T cell receptor a and bsequences. Sci Transl Med 2015;7:301ra131.

Kirsch IR, Watanabe R, O’Malley JT, Williamson DW, Scott LL, Elco CP, et al.TCR sequencing facilitates diagnosis and identifies mature T cells as thecell of origin in CTCL. Sci Transl Med 2015;7:308ra158.

Lefranc MP, Giudicelli V, Ginestoux C, Jabado-Michaloud J, Folch G,Bellahcene F, et al. IMGT, the international ImMunoGeneTics informa-tion system. Nucleic Acid Res 2009;37:D1006e12.

Murphy K. Janeway’s immunobiology, 8th ed. New York: Garland: Science;2011.

Robins HS, Campregher PV, Srivastava SK, Wacher A, Turtle CJ, Kahsai O,et al. Comprehensive assessment of T-cell receptor beta-chain diversityin alphabeta T cells. Blood 2009;114:4099e107.

Rook AH, Gelfand JC,WysockaM, Troxel AB, Benoit B, Surber C, et al. Topicalresiquimod can induce disease regression and enhance T-cell effectorfunctions in cutaneous T-cell lymphoma. Blood 2015;126:1452e61.

Ruggiero E, Nicolay JP, Fronza R, Arens A, Paruzynski A, Nowrouzi A, et al.High-resolution analysis of the human T-cell receptor repertoire. NatCommun 2015;6:8081.

This is a reprint of an article that originally appeared in the June 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here. Forcitation purposes, please use these original publication details: Matos TR, de Rie MA, Teunissen MBM. Research Techniques Made Simple: High-ThroughputSequencing of the T-Cell Receptor. J Invest Dermatol 2017;137(6):e131ee138. doi:10.1016/j.jid.2017.04.001



Research Techniques Made Simple:Cost-Effectiveness AnalysisConnie R. Shi1,2 and Vinod E. Nambudiri2

Cost-effectiveness analysis (CEA) is a research method used to determine the clinical benefit-to-cost ratio of agiven intervention. CEA offers a standardized means of comparing cost-effectiveness among interventions.Changes in quality-adjusted life-years, disability-adjusted life-years, or survival and mortality are some of thecommon clinical benefit measures incorporated into CEA. Because accounting for all associated costs andbenefits of an intervention is complex and potentially introduces uncertainty into the analysis, sensitivity an-alyses can be performed to test the analytic model under varying conditions. CEA informs the identification ofhigh-value clinical practices and can be used to evaluate preventative, diagnostic, and therapeutic interventionsin dermatology.


CME Activity Dates: 21 June 2017Expiration Date: 20 June 2018Estimated Time to Complete: 1 hour







Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-July17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

WHAT IS COST-EFFECTIVENESS ANALYSIS?Cost-effectiveness analysis (CEA) is a research method thatcharacterizes the costs of interventions relative to theamount of benefit that they yield. CEA provides a stan-dardized means of comparing interventions to identifythose that provide maximal clinical effect per incrementalunit of cost. CEA can be applied to preventive, diagnostic,and therapeutic interventions. Outcomes captured by such

analyses can include mortality benefit, symptom reduction,or improved quality of life after a treatment or procedure.CEA is one type of economic analysis used in health ser-vices research; other related but separate concepts areoutlined in Table 1.

Given the interest in delivering high-value care across allclinical specialties including dermatology, research identi-fying clinical practices that deliver a high level of

1Harvard Medical School, Boston, Massachusetts, USA; and 2Department of Dermatology, Brigham and Women’s Hospital, Harvard Medical School, Boston,Massachusetts, USA

Correspondence: Vinod E. Nambudiri, 221 Longwood Avenue, Boston, Massachusetts, 02115 USA E-mail: [email protected]

Abbreviations: CEA, cost-effectiveness analysis; DALY, disability-adjusted life-year; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year



effectiveness at a relatively lower cost can be valuable inguiding policy on allocation of health care resources. It is thusincreasingly relevant for dermatologists to understand CEAand demonstrate cost-effectiveness in current practice.

METHODS IN COST-EFFECTIVENESS ANALYSISCore elements of cost-effectiveness analysis include identi-fying clinical interventions, accounting for all associated costs,

and defining outcome measures for analysis. The Panel onCost-Effectiveness in Health and Medicine provides recom-mendations on variables that should be included in cost andoutcome definitions used in CEA (Sanders et al., 2016). Costcalculations should include not only the price of administeringan intervention but also costs associated with facility and staffresources, intervention adverse effects, and indirect costs ofpatient suffering and lost productivity, among others.

Table 2 outlines various outcome measures used in CEA.The most commonly used outcome measures are thedisability-adjusted life-year (DALY) and quality-adjustedlife-year (QALY). For both DALYs and QALYs, a value of1 is assigned to a single year lived with perfect health. Todetermine the DALYs associated with a condition, adisability weight is assigned based on the level of impair-ment caused by the condition, with larger disabilityweights correlated with greater impairments to health(Jamison et al., 2006). The disability weight is then sub-tracted from 1 to determine the DALY. QALYs are calcu-lated in a similar fashion but incorporate quality of lifechanges into the measurement. Standardized quality of lifesurveys such as the EuroQol five dimensions questionnaire(EQ-5D) are commonly used to derive QALY values (Prietoand Sacristán, 2003). DALY and QALY determinations areinformed by standardized disease severity, symptom, andquality of life measurements and in many cases are pref-erable markers of health outcomes over simply countinglife-years prolonged (Jamison et al., 2006). Dermatology-related instruments that have been developed and vali-dated include the Dermatology Life Quality Index (DLQI),Children’s Dermatology Life Quality Index, Psoriasis Areaand Severity Index (PASI), SCOring Atopic Dermatitis(SCORAD), and Functional Assessment of Cancer TherapyMelanoma (FACT-M), among others.

Decision analysis models are used to analyze largevolumes of patient outcomes in CEA. A decision tree,which allows visualization of the different clinical path-ways being compared and their possible outcomes, is anexample of a simple decision analysis model (Figure 1).Probabilities of an intervention’s success or failure areestimated from existing literature on efficacy, and the de-cision tree allows for the incorporation of outcomes suchas cost and QALYs. However, the decision tree is less welladapted to handling recurrent conditions and longer-termoutcomes. The Markov model, an iterative model that ac-commodates transitions among various disease states,can be better suited for representing conditions thatrecur, evolve, and progress over time (Sonnenberg andBeck, 1993).

Table 1. Comparison of concepts in health economicsanalysisConcept Definition

Cost analysis Calculation of the costs associated with anintervention

Cost-benefit analysis Characterization of the cost of an interventionrelative to the monetary benefit of its outcome

Cost-effectivenessanalysis

Characterization of the cost of an interventionrelative to the clinical benefits of the outcome,

measured in nonmonetary valuesComparativeeffectiveness research1

A field of research that aims to discriminateamong clinical interventions according to theirclinical effectiveness, cost effectiveness, and

appropriateness1Nambudiri and Qureshi, 2013.

Table 2. Definitions of selected outcome measures used in CEA1

Outcome measure Definition

Mortality (deaths averted) The number of deaths prevented by an interventionLife-years gained/lost The remaining life expectancy at the time of an averted death, weighted in favor of younger persons.Disability-adjusted life-years gained/lost A unit of the amount of health lost because of a condition, taking into account the burden of morbidity

associated with the conditionQuality-adjusted life-years gained/lost A unit of the years of life saved and adjusted for health-related quality of life with that condition1Adapted from Jamison et al., 2006.

SUMMARY POINTSWhat is cost-effectiveness analysis?Cost-effectiveness analysis (CEA) is a research methodused to determine the clinical benefit-to-cost ratio of in-terventions. CEA offers a standardized means ofcomparing cost effectiveness among interventions.

Changes in quality-adjusted life-years (QALYs),disability-adjusted life-years (DALYs), or survival andmortality are some of the common outcome measures ofclinical benefit incorporated into CEA.

CEA can be used to evaluate screening, preventative,diagnostic, and therapeutic interventions.

Limitations of cost-effectiveness analysisAccounting for all associated costs and benefits of anintervention is complex, potentially introducing uncer-tainty into the analysis. Sensitivity analyses can be per-formed to test the analytic model under varyingconditions.

There remains no universally accepted standard forcost-effectiveness thresholds.



There is inherent uncertainty in estimations of the inputsand outputs in CEA. Sensitivity analyses are incorporated intoCEA to test decision analysis models under varying condi-tions. A sensitivity analysis might include testing outcomesafter changing the estimate of a treatment cost or under as-sumptions of lesser treatment efficacy than reported in theliterature.

The calculated incremental cost-effectiveness ratio (ICER)allows for comparison of cost-effectiveness between in-terventions. The basic formula for the ICER of an interventionX relative to a comparison intervention Y (e.g., the currentstandard of care, the control treatment, or no intervention ifthere is no current available therapy) is as follows:

ICER

¼ ðcost of intervention X � cost of comparison intervention YÞðeffect of intervention X � effect of comparison intervention YÞ

ICER is commonly reported in units of $/DALY or $/QALY.Historically, a commonly used threshold below which anintervention is considered cost effective has been $50,000/QALY, but recommended cost-effectiveness thresholds varywidely. For instance, the National Institute for Health andCare Excellence in the United Kingdom uses a thresholdrange of £20,000e£30,000/QALY (approximately$25,351e$38,026/QALY) (McCabe et al., 2008), whereas theWorld Health Organization recommends thresholds of lessthan three times a nation’s per capita gross domestic product/QALY (Marseille et al., 2015). The threshold for cost effec-tiveness remains an area of active discussion in health eco-nomics research (Neumann et al., 2014).

CEA IN DERMATOLOGY: PREVENTIONA large-scale retrospective CEA published by Gordon et al. inthe Journal of Investigative Dermatology in 2009 investigatedthe cost effectiveness of routine sunscreen use for skin cancerprevention versus usual practice (i.e., discretionary sunscreen

use) in Australia. Program participants were supplied withsunscreen to use over the course of the study.

The net societal costs per person—which included totalcosts incurred by patients (i.e., time to attend provider visits),providers (i.e., provider salaries, facility costs, sunscreen costs),and the Australian government, the predominant payer in theAustralian health care system—over the 5-year period were$405 and $275 for the daily sunscreen treatment interventionand usual practice groups, respectively. However, the authorsnoted that the intervention represented a net savings for theAustralian government because of decreased need for treat-ment of squamous cell carcinoma, basal cell carcinoma, andactinic keratosis in the intervention group, yielding a totalsavings of $88,203 over the 5-year period. At an investment ofjust $0.74 per person per year, the authors concluded that theintervention was favorably cost effective for Australiacompared with other public health prevention interventions.

Variables examined in the sensitivity analyses includedmisdiagnosis costs, costs associated with sunscreen purchase,costs associated with medical visits (which showed widevariation), and proportion of actinic keratoses treated, forwhich there is no single standardized clinical pathway(Table 3). The intervention consistently offered net cost sav-ings to the government across the sensitivity analyses; how-ever, variation in medical costs and in the proportion ofactinic keratoses treated had significant effects on estimatedcost-effectiveness ratios (Table 3), illustrating how calculatedcost-effectiveness ratios can change by varying assumptionsand inputs in these analyses. Although the generalizability ofthe study to other countries remains to be seen, this articleprovides an example of a CEA with direct impact on derma-tology and skin cancer prevention.

In another recent CEA related to dermatology, a studypublished in JAMA Pediatrics examined the cost effectivenessof six different daily total-body moisturizer treatments used asa means of prevention for the first 6 months of life amongnewborns at high risk for developing atopic dermatitis(Xu et al., 2016). Unlike the aforementioned skin cancer

Figure 1. Representative decisiontree of management pathways forverruca vulgaris. The figure illustratesan example of a clinical decision treefor the management of a commondermatologic condition using selectstrategies currently used in clinicalpractice. It does not exhaustively showall therapeutic options.



study, which used observed clinical outcomes from a ran-domized controlled trial as the effect measure, this CEAadapted cost and effect findings from a previous report thatshowed the relative risk reduction of atopic dermatitis aftermoisturizer treatment among newborns (Simpson et al.,2014). Sensitivity analyses examined treatment effectsranging from 28%e90% relative risk reductions. The costeffectiveness of moisturizer treatments ranged from $353/QALY to $8,386/QALY for the least and most expensivemoisturizers included in the study, both well belowcommonly accepted thresholds for cost effectiveness in theUnited States.

The authors noted that moisturizers, despite evidence ofclinical cost- effectiveness, are not included in insurancecoverage and thus are frequently out-of-pocket expenses thatpose economic burdens for patients with atopic dermatitis.The two preventive studies cited here highlight cost-effectiveinterventions for which intervention costs are most oftenborne by patients, despite the benefits and cost savingsaccruing to government and insurance payers. This discor-dance represents a challenging dilemma in cost-effectivenessresearch but potentially also an opportunity for advocacy inexpanding payment coverage for these beneficial, evidence-based, cost-effective interventions in disease prevention.

CEA IN DERMATOLOGY: THERAPEUTICSGiven the range of treatment options that exist for dermato-logic diseases and concerns of escalating costs associatedwith novel therapies, CEA is increasingly conducted to eval-uate emerging treatments. For example, in an analysis oftrametinib plus dabrafenib in the treatment of BRAF V600-

mutated melanoma in Switzerland, the authors found thatbased on current pricing, trametinib plus dabrafenib had anICER of 385,603 Swiss francs/QALY (approximately$379,624/QALY), making it a less cost-effective treatmentthan vemurafenib monotherapy, despite potentially beingmore clinically effective (Matter-Walstra et al., 2015). It is

MULTIPLE CHOICE QUESTIONS1. This unit of cost effectiveness is defined as

ðcost of intervention X � cost of comparison intervention YÞðeffect of intervention X � effect of comparison intervention YÞ

A. Cost-benefit ratio

B. Quality-adjusted life-year

C. Incremental cost-effectiveness ratio

D. Average cost effectiveness

2. What type of analysis is performed to simulatereal-world uncertainty in the parameters of thecost-effectiveness analysis and test assumptionsunder varying conditions?

A. Cost-benefit analysis

B. Sensitivity analysis

C. Comparative effectiveness analysis

D. Chi-square analysis

3. This type of model can be useful in cost-effectiveness analysis for simulating the com-plex course of chronic disease or conditions inwhich there is transition back and forth betweendisease states.

A. Decision tree analysis

B. Logistic regression model

C. Cox proportional hazards model

D. Markov model

4. Which of the following is a commonly usedthreshold for valuing a single quality-adjustedlife-year (QALY) in cost-effectiveness analysis?

A. $500/QALY

B. $5,000/QALY

C. $50,000/QALY

D. $500,000/QALY

5. Which of the following costs should be factoredin as part of a cost-effectiveness analysis for anewly developed pharmaceutical treatment?

A. Retail price of the drug

B. Physician time necessary to administer thedrug to patients

C. Patient time out of work to recover fromadverse effects of the drug

D. All of the above

Table 3. Sensitivity analyses of the incremental cost-effectiveness ratio of population-wide routinesunscreen use under varying conditions

Incremental cost-effectiveness ratio (mean

per person)

Base analysis 3.72Costs involved in skin cancermisdiagnoses (positive predictivevalue)

0.60 3.62

0.801 3.79Medical costs ($) Low 3.81

High2 0.33Time to visit a GP ($) Low 3.79

High 3.70Time to apply sunscreen ($) Low 3.77

High 3.66Sunscreen purchases ($) Low 3.52

High 3.93Out-of-pockets for GP visit ($) Low 3.77

High 3.70Proportion of AKs treated 0% 6.31

25% 5.02100% 1.15

Abbreviations: AKs, actinic keratosis; GP, general practitioner.1Includes the scenario of higher accuracy and less resource use in expe-rienced or specialist doctors.2Includes the scenario of higher costs of skin cancers treated in hospitals.From Gordon et al., 2009, reprinted with permission.



important to appreciate, however, that because treatmentprices change with time, so too may cost-effectiveness cal-culations. Thus, updating historical analyses to reflect currentcosts may prove informative.

CEA has also been used to identify cost-effective in-terventions in current practice. In a CEA of topical regimensfor mild to moderate localized psoriasis, a Markov model wasconstructed to simulate clinical pathways for psoriasismanagement (Sawyer et al., 2013). Use of the iterativeMarkov model in this study illustrates its value in analysesof chronic and frequently relapsing diseases such aspsoriasis. The authors concluded that the most cost-effectivefirst-line therapy for psoriasis of the body was twice-dailypotent corticosteroids, with an ICER of £20,000/QALY(approximately $25,351/QALY), whereas very potent corti-costeroids were the most cost-effective treatment for scalppsoriasis.

As these examples show, CEA identifies interventionswithin dermatology that confer benefits to patients, providecost savings for health systems, and inform policy decisions.CEA of both preventive and therapeutic measures brings intogreater focus the relative advantages and disadvantages ofimplementing various interventions from both clinical andcost perspectives.

FUTURE DIRECTIONS IN CEA AND DERMATOLOGYDecreases in melanoma mortality after a population-basedmelanoma screening program in Schleswig-Holstein, Ger-many (Breitbart et al., 2012) have led to renewed interest inevaluating whether similar comprehensive screening pro-grams could be implemented cost-effectively in other coun-tries. CEA is increasingly included in clinical trials to enabledemonstration of favorable cost-effectiveness profiles inaddition to evidence of therapeutic value and has beenapplied to the analysis of care models themselves, such asinvestigations of the cost-effectiveness of teledermatologyprograms. Clinicians and investigators alike should befamiliar with CEA methodology and the many ways in whichCEA can be used in the evaluation of preventive, diagnostic,and therapeutic measures for skin diseases. UnderstandingCEA research techniques can aid physicians and researchersas they design clinical trials, engage in policy-related advo-cacy, and make clinical decisions affecting the care of in-dividuals with dermatologic diseases.


AUTHOR CONTRIBUTIONSCS is a medical student/trainee and VEN is faculty. Both authors wereresponsible for manuscript conception, design, drafting, and critically revisingcontent. Both authors gave final approval of the version to be published andagree to be accountable for all aspects of the work.


REFERENCESBreitbart EW, Waldmann A, Nolte S, Capellaro M, Greinert R, Volkmer B,

et al. Systematic skin cancer screening in Northern Germany. J Am AcadDermatol 2012;66:201e11.

Gordon LG, Scuffham PA, van der Pols JC,McBride P,Williams GM,Green AC.Regular sunscreen use is a cost-effective approach to skin cancerprevention in subtropical settings. J Invest Dermatol 2009;129:2766e71.

Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, et al.Cost-effectiveness analysis, https://www.ncbi.nlm.nih.gov/books/NBK10253/; 2006 (accessed 2016 December 8).

Marseille E, Larson B, Kazi DS, Kahn JG, Rosen S. Thresholds for the cost-effectiveness of interventions: alternative approaches. Bull World HealthOrgan 2015;93:118e24.

Matter-Walstra K, Braun R, Kolb C, Ademi Z, Dummer R, Pestalozzi BC, et al.A cost-effectiveness analysis of trametinib plus dabrafenib as first-linetherapy for metastatic BRAF V600-mutated melanoma in the Swisssetting. Br J Dermatol 2015;173:1462e70.

McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold: whatit is and what that means. Pharmacoeconomics 2008;26:733e44.

Nambudiri VE, Qureshi A. Comparative effectiveness research. J Invest Der-matol 2013;133:1e4.

Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness—Thecurious resilience of the $50,000-per-QALY threshold. N Engl J Med2014;371:796e7.

Prieto L, Sacristán JA. Problems and solutions in calculating quality-adjustedlife years (QALYs). Health Qual Life Outcomes 2003;1:80.

Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al.Recommendations for conduct, methodological practices, and reportingof cost-effectiveness analyses: second panel on cost-effectiveness inhealth and medicine. JAMA 2016;316:1093e103.

Sawyer L, Samarasekera EJ, Wonderling D, Smith CH. Topical therapies forthe treatment of localized plaque psoriasis in primary care: a cost-effectiveness analysis. Br J Dermatol 2013;168:1095e105.

Simpson EL, Chalmers JR, Hanifin JM, Thomas KS, Cork MJ, McLean WHI,et al. Emollient enhancement of the skin barrier from birth offerseffective atopic dermatitis prevention. J Allergy Clin Immunol2014;134:818e23.

Sonnenberg FA, Beck JR. Markov models in medical decision making: apractical guide. Med Decis Making 1993;13:322e38.

Xu S, Immaneni S, HazenGB, Silverberg JI, Paller AS, Lio PA. Cost-effectivenessof Prophylactic Moisturization for Atopic Dermatitis. JAMA Pediatr2016;171:e163909.

This is a reprint of an article that originally appeared in the July 2017 issue of the Journal of Investigative Dermatology. It retains its original pagination here. Forcitation purposes, please use these original publication details: Shi CR, Nambudiri VE. Research Techniques Made Simple: Cost-Effectiveness Analysis. J InvestDermatol 2017;137(7):e143ee147. doi:10.1016/j.jid.2017.03.004



Research Techniques Made Simple:An Introduction to Use and Analysis ofBig Data in DermatologyMackenzie R. Wehner1, Katherine A. Levandoski2, Martin Kulldorff3 and Maryam M. Asgari2

Big data is a term used for any collection of datasets whose size and complexity exceeds the capabilities oftraditional data processing applications. Big data repositories, including those for molecular, clinical, andepidemiology data, offer unprecedented research opportunities to help guide scientific advancement. Ad-vantages of big data can include ease and low cost of collection, ability to approach prospectively and retro-spectively, utility for hypothesis generation in addition to hypothesis testing, and the promise of precisionmedicine. Limitations include cost and difficulty of storing and processing data; need for advanced techniquesfor formatting and analysis; and concerns about accuracy, reliability, and security. We discuss sources of bigdata and tools for its analysis to help inform the treatment and management of dermatologic diseases.


CME Activity Dates: 20 July 2017Expiration Date: 19 July 2018Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: Maryam Asgarireceived research grant support from Pfizer, Inc and ValeantPharmaceuticals. All other authors, planning committeemembers, CME committee members and staff involved withthis activity as content validation reviewers have no financialrelationships with commercial interests to disclose relative tothe content of this CME activity.






Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-August17 e click ‘CME on Demand’and locate the article to complete the test. Fax or other copieswill not be accepted. To receive credits, learners must reviewthe CME accreditation information; view the entire article,complete the post-test with a minimum performance level of60%; andcomplete theonline evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

WHAT ARE BIG DATA?Big data are commonly defined as data so large or complexthat traditional data processing and analytic approachesare inadequate. The 3 Vs that characterize big data arevolume (amount of data), velocity (speed at which data aregenerated and processed), and variety (types of data)

(Laney, 2001), all of which have been growing rapidly(Figure 1). Although there is no predefined threshold forvolume, in general, anything 1 petabyte (1015 bytes, or theapproximate size of 1 million human genomes) or greater isconsidered big data (Figure 2). The ability to monitor, re-cord, and store information from large populations from

1Department of Dermatology, University of Pennsylvania, Philadelphia, Pennsylvania, USA; 2Department of Dermatology, Massachusetts General Hospital,Harvard Medical School, Boston, Massachusetts, USA; and 3Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brighamand Women’s Hospital and Harvard Medical School, Boston, Massachusetts, USA

Correspondence: Maryam M. Asgari, Department of Dermatology, Massachusetts General Hospital, 50 Staniford Street, Suite 230A, Boston, Massachusetts02114, USA. E-mail: [email protected]

ª 2017 The Authors. Published by Elsevier, Inc. on behalf of the Society for Investigative Dermatology. This is an open accessarticle under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). www.jidonline.org e153


sources including electronic medical records, insuranceclaims, surveys, disease registries, biospecimens, apps andsocial media, the internet, and personal monitoring deviceshas shepherded the era of big data into use in health care.The volume of health care data in the United States in 2017is rapidly approaching zettabyte levels (iHT2, 2013). Thiswealth of structured and unstructured data has the poten-tial to substantially affect health care delivery throughimproved risk assessment, surveillance, diagnosis, andtreatment methods.

WHAT ARE SOME BIG DATA SOURCES IN HEALTH CARE?There are many big data sources in health care. OptumLabs(https://www.optumlabs.com), an open collaborative researchcenter, provides de-identified clinical data from electronichealth records and claims data for over 100 millioninsured members (Borah, 2016). Sentinel (https://www.sentinelinitiative.org), a US Food and Drug Administration

initiative, uses data from electronic health records, insuranceclaims, and registries to monitor postmarketing, real-worldsafety of medicines. Sentinel data were used to estimate thevalidity of International Classification of DiseaseseNinthRevision codes (Centers for Disease Control, 1998) forascertaining Stevens-Johnson syndrome and toxic epidermalnecrolysis in 12 collaborating research units, covering almost60 million people (Davis et al., 2015). UK Biobank and KaiserPermanente Biobank are examples of medical data and tissuesamples collected for research purposes. UK Biobank (www.ukbiobank.ac.uk) is a cohort of 500,000 participants in theUK who have provided baseline information and blood,urine, and saliva samples and who are being followed pro-spectively through their regular care. The Kaiser PermanenteResearch Biobank (https://www.dor.kaiser.org/external/DORExternal/rpgeh) is composed of 220,000 health planmembers who have contributed genetic and electronic healthrecord data. This was recently used in a large genome-wideassociation study of cutaneous squamous cell carcinoma,which identified 10 single-nucleotide polymorphisms asso-ciated with cutaneous squamous cell carcinoma at genome-wide significance and provided new insights into the ge-netics of heritable cutaneous squamous cell carcinoma risks(Asgari et al., 2016). For genomic data, such as those found inbiobanks, the National Center for Biotechnology Informationhas developed the Gene Expression Omnibus (https://www.ncbi.nlm.nih.gov/geo), which acts as a public archive andrepository of microarray, next-generation sequencing, andhigh-throughput functional genomic data. Geographic infor-mation systems, such as the National Cancer InstituteGeographic Information Systems and Science for CancerControl (https://gis.cancer.gov), capture geographic data thatallow for mapping of disease trends. Solar UV radiation dataare available through this system, and the association be-tween cutaneous melanoma incidence rates and county-levelUV exposure has been examined (Richards et al., 2011).

Figure 1. The 3 Vs of big data. The 3Vs of big data are volume (amount ofdata), velocity (speed at which data isgenerated), and variety (number oftypes of data), all of which have beengrowing rapidly. After “The 3Vs ThatDefine Big Data,” Diya Soubra,Data Science Central, http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data. GPS,global positioning system.

SUMMARY POINTS� Big data describes any collection of datasetswhose size and complexity exceeds the capabil-ities of traditional data processing applications.

� Big data has the potential to help inform thetreatment and management of dermatologicdiseases through improved risk assessment, sur-veillance, diagnosis, and treatment methods.

� While big data presents spectacular researchopportunities, there are important limitations toconsider, including storage costs, processingchallenges, and concerns about accuracy, reli-ably, and security.



Computer-based geographic information systems, web-basedgeospatial technologies such as global positioning systems insmartphones, and geospatial modeling can be used to followdisease trends and to examine mobility and social networksand their impact on disease (Birch, 2016; Ray et al., 2016).

To enhance the utility of biomedical big data from thesediverse sources, the National Institutes of Health establishedBig Data to Knowledge (https://datascience.nih.gov/bd2k). Itaims to make digital data “findable, accessible, interoperable,and reusable (FAIR),” with the following specific goals: (i) toimprove the ability to find and use big data, (ii) to developanalysis tools for big data, (iii) to increase training in datascience, and (iv) to establish centers of excellence in datascience (Margolis et al., 2014). Big Data to Knowledge hasfunding opportunities in many areas, including curating,coordinating, and organizing big data, developing big dataeducational curricula, and improving big data standards(https://www.nlm.nih.gov/ep/BD2KGrants.html).

HOW DO ANALYTIC TECHNIQUES FOR BIG DATA DIFFERFROM THOSE FOR TRADITIONAL DATA?Although big data can be used for traditional hypothesistesting and can be especially valuable for research on rarediseases or exposures, big data analyses are often hypothesisgenerating. Rather than test a hypothesis, they can provideevidence for new hypotheses that can later be tested with

traditional techniques. Big data analyses often center onidentifying patterns. Unlike traditional predictive modelingbased on a small number of covariates, big data predictivemodeling often involves variables that are not preselected.Thus, compared with traditional data analysis, big dataanalysis has the potential to be more exploratory. Given themultiplicity inherent in the many potential patterns evaluated,such big data analyses benefit from special statistical methodsthat account for this multiple testing using P-value adjust-ments or false discovery rates.

ANALYTIC TECHNIQUES FOR BIG DATAThere are many computational and statistical methods used toanalyze big data. Data mining is a process through whichdata are analyzed from different perspectives to identify un-suspected patterns. Using insurance claims, data mining withTreeScan software was used to explore unsuspected adversereactions associated with antifungal drug exposure (Kulldorffet al., 2013). TreeScan is free data mining software avail-able for download online (TreeScan, Boston, MA; https://www.treescan.org). Cluster analysis focuses on groupingsimilar patients or observations by demographics, medicalhistory, genetics, or geography. For example, the spatialscan statistic was used to detect geographic clusters of basalcell carcinomas in a Northern California population withthe goal of targeting screening and prevention efforts

Figure 2. Logarithmic scale depicting volume of big data. The relative scale of different datasets is depicted. There is no predefined threshold for volume thatdefines big data, but in general, anything one petabyte or greater is considered big data.



(Ray et al., 2016). Another example is cluster analysis ofdifferent quality-of-life scoring systems in psoriasis patients,which showed lack of correlation of disease severity withpsychological distress instruments (Sampogna et al., 2004).

Machine learning allows algorithms to learn from a trainingdataset to make predictive models without specifying themodel in advance. Machine learning is currently beingexplored to track pigmented lesions over time and identifylesions at higher risk for malignancy (Li et al., 2016). Machinelearning was recently used to develop a diagnosis algorithmfor skin cancer based on clinical images (Esteva et al., 2017).The algorithm, which uses only pixels and disease labels asinputs, matches the performance of dermatologists in identi-fying cancerous and noncancerous lesions (Esteva et al.,2017). Deployable on mobile devices, machine learningalgorithms that train computers to make reliable diagnosesdirectly from clinical images hold the potential to make asignificant clinical impact by extending the reach of derma-tologists beyond the clinic (Esteva et al., 2017). Decision treelearning is a type of machine learning in which the inde-pendent variables are used to create a hierarchical treestructure with leaves and branches, which can predict anoutcome (see Figure 3 for example). There are two main typesof decision tree analyses: classification tree analysis, wherethe predicted outcome is dichotomous such as for melanomamortality, and regression tree analysis, where the predictedoutcome is a continuous variable such as age at melanomadiagnosis. Both classification and regression tree analyseswere used to identify histological features of melanomaassociated with CDKN2A germline mutations (Sargen et al.,2015). Bayesian networks are another type of machinelearning that use probabilistic graphs to explore relationshipsbetween, for example, symptoms and disease, to be used inclinical decision making or diagnosis. Cognitive computing isa type of machine learning that tries to mimic the functioningof the human brain. Natural language processing algorithmsallow computers to extract useful information from text, suchas electronic health records, well enough to yield meaningfuldata. Such algorithms can identify mentions of a risk factor or

of an outcome disease in clinic notes, recognizing that thesame exposure or diagnosis can be expressed in manydifferent ways and with potential misspellings and dis-tinguishing a positive diagnosis from a rule-out diagnosis.Natural language processing has been used in dermatologyresearch to find nonmelanoma skin cancer diagnoses inelectronic pathology reports (Eide et al., 2012).

ANALYTIC PLATFORMS FOR BIG DATAThere are two approaches to analytic platforms for big data: (i)a divide-and-conquer approach (distributed data) and (ii) acentralized approach using a platform that provides bothdatabase storage and analytics in a centralized fashion, suchas SAP HANA (SAP, Walldorf, Germany; http://www.sap.com/product/technology-platform/hana.html). SAP HANA isa computing platform that offers tools for storing, managing,and analyzing big data. When big data are in differentphysical locations, distributed data analysis can be used withsome of the analysis conducted locally on the complete datawhile the final analysis occurs centrally using summary datafrom each site. The advantage of distributed data for medicalinformation is that data remain at local sites, minimizingstorage costs and maximizing data integrity and patientprivacy.

SUMMARY AND FUTURE DIRECTIONS IN DERMATOLOGYThe term big data is more than just very large data or a largenumber of data sources but encompasses a new approach tocomplex data. It offers a new, hypothesis-generatingframework to conduct research and requires novel analysismethods. It has significant advantages but also has limita-tions (Table 1), and traditional data analytics are still

Figure 3. Decision-tree learning to predict melanoma mortality(hypothetical). Hypothetical example illustrating the utility of decision-treelearning for melanoma mortality prediction showing “leaves” (independentvariables) such as tumor thickness, ulceration, and tumor location, andprobability of survival (outcome).

Table 1. Advantages and limitations of big dataAdvantages Limitations

� Large sample size� Data can be inexpensive to collectand acquire: in many cases the datahave already been collected throughroutine clinical care (electronic healthrecords) or through the participantsthemselves (internet searches orpersonal monitoring devices)

� Both retrospective and prospectiveapproaches are often available

� Multiple data points from differentsources can be combined, leveragingthe advantages of different collectionsources or smaller datasets

� Storage: datasets canrequire considerableresources to store

� Formatting and datacleaning: advancedcomputer science can berequired before the datais analyzable

� Quality control: can bedifficult and often has tobe done through smallrepresentative samples

� Security and privacyconcerns: often morecomplex than fortraditional datasets

� Accuracy and consistencyof methods: manyapproaches are relatively new and imperfect,although these maycontinue to improveover time



crucially important. In dermatology, big data can be used toimprove risk prediction models, support targeted screeningfor high-risk individuals (e.g., targeted skin cancerscreening), optimize management of a variety of skin dis-eases, and offer clinical decision support (e.g., assistance indeciding whether to biopsy a pigmented lesion). We canfurther investigate the genetics of skin disease (e.g., genome-

wide association studies) (Asgari et al., 2016; Frelinger,2015) and examine distinct disease phenotypes within het-erogeneous diseases that could benefit from tailored thera-pies (e.g., in psoriasis or eczema). Big data may be anexcellent way to perform surveillance and evaluate safety ofmedications and devices, especially for rarer outcomes. Bigdata in dermatology present spectacular opportunities,allowing researchers to maximize the potential of existingdata sources and opening up new, efficient, and powerfulmethods for future research.

CONFLICT OF INTERESTMA has received research funding to her institution from Pfizer, Inc. andValeant Pharmaceuticals, but these associations have not influenced our workon this article. The authors have no other potential conflicts of interest todisclose.

ACKNOWLEDGMENTSThis research was supported by National Institutes of Health grantsR01CA166672 (MA) and K24AR069760 (MA). We would like to acknowl-edge Susan Gruber for her assistance with reviewing the content of thismanuscript.


REFERENCESAsgari MM, Wang W, Ioannidis NM, Itnyre J, Hoffmann T, Jorgenson E, et al.

Identification of susceptibility loci for cutaneous squamous cell carci-noma. J Invest Dermatol 2016;136:930e7.

Birch P. Powering geospatial analysis: public geo datasets now on GoogleCloud, https://cloudplatform.googleblog.com/2016/10/powering-geospatial-analysis-public-geo-datasets-now-on-Google-Cloud.html; 2016 (accessed6 January 2017).

Borah BJ. Optum Labs overview, https://http://www.allianceforclinicaltrialsinoncology.org/main/cmsfile?cmsPath¼/Public/Annual Meeting/files/Prevention-Optum Labs Overview.pdf; 2016 (accessed 14 December2016).

Centers for Disease Control. International Classification of DiseaseseNinthRevision. ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/ICD-9/ucod.txt. Published April 9, 1998. Accessed 22 June 2017.

Davis RL, Gallagher MA, Asgari MM, Eide MJ, Margolis DJ, Macy E, et al.Identification of Stevens-Johnson syndrome and toxic epidermal necrol-ysis in electronic health record databases. Pharmacoepidemiol Drug Safe2015;24:684e92.

Eide MJ, Tuthill JM, Krajenta RJ, Jacobsen GR, Levine M, Johnson CC. Vali-dation of claims data algorithms to identify nonmelanoma skin cancer.J Invest Dermatol 2012;132:2005e9.

Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatol-ogist-level classification of skin cancer with deep neural networks.Nature 2017;542:115e8.

Frelinger JA. Big data, big opportunities, and big challenges. J Investig Der-matol Symp Proc 2015;17:33e5.

iHT2. Transforming health care through big data, http://c4fd63cb482ce6861463-bc6183f1c18e748a49b87a25911a0555.r93.cf2.rackcdn.com/iHT2_BigData_2013.pdf; 2013 (accessed 14 December 2016).

Kulldorff M, Dashevsky I, Avery TR, Chan AK, Davis RL, Graham D, et al.Drug safety data mining with a tree-based scan statistic. Pharmacoepi-demiol Drug Saf 2013;22:517e23.

Laney D. 3D data management: controlling data volume, variety and velocity.Application Delivery Strategies 2001;6 Feb:949.

Li Y, Esteva A, Kuprel B, Novoa R, Ko J, Thrun S. Skin cancer detection andtracking using data synthesis and deep learning. arXiv 2016:161201074.

Margolis R, Derr L, Dunn M, Huerta M, Larkin J, Sheehan J, et al. The NationalInstitutes of Health’s Big Data to Knowledge (BD2K) initiative: capitalizingon biomedical big data. J Am Med Inform Assoc 2014;21:957e8.

MULTIPLE CHOICE QUESTIONS1. What are the 3 Vs that characterize big data?

a. Value, viability, and variety

b. Volume, velocity, and viability

c. Volume, velocity, and variety

d. Volume, value, and variety

2. What distinguishes big data analyses fromtraditional data analyses?

a. They can be used to both test and generatehypotheses.

b. Variables are often not preselected forprediction modeling.

c. They often center around identifying andevaluating patterns.

d. All of the above

3. What analytic technique focuses on groupingsimilar patients by characteristics such asdemographics, genetics, or geography and canbe used to inform geographically targetedscreening and prevention efforts?

a. Cluster analysis

b. Decision-tree learning

c. Bayesian networks

d. Cognitive computing

4. Which of the following is NOT a limitation of bigdata?

a. Storage may require considerable resources.

b. Formatting and analysis may requireadvanced computer science.

c. Big data can be used only for retrospectiveanalyses.

d. Big data have more complex security andinformation privacy concerns than traditionaldatasets.

5. Which of the following is NOT a potentialapplication of big data?

a. Improve risk prediction for very rare diseases

b. Identify distinct disease phenotypes inheterogeneous diseases that may meritdifferent therapies

c. Identify causal associations

d. Perform drug and medical device surveillance



Ray GT, Kulldorff M, Asgari MM. Geographic clusters of basal cell carcinomain a northern California health plan population. JAMA Dermatol2016;152:1218e24.

Richards TB, Johnson CJ, Tatalovich Z, Cockburn M, Eide MJ, Henry KA, et al.Association between cutaneous melanoma incidence rates among whiteUS residents and county-level estimates of solar ultraviolet exposure.J Am Acad Dermatol 2011;65:S50e7.

Sampogna F, Sera F, Abeni D. IDI Multipurpose Psoriasis Research on VitalExperiences (IMPROVE) Investigators. Measures of clinical severity,quality of life, and psychological distress in patients with psoriasis: acluster analysis. J Invest Dermatol 2004;122:602e7.

Sargen MR, Kanetsky PA, Newton-Bishop J, Hayward NK, Mann GJ,Gruis NA, et al. Histologic features of melanoma associated withCDKN2A genotype. J Am Acad Dermatol 2015;72:496e507.e7.

This is a reprint of an article that originally appeared in theAugust 2017 issue of the Journal of InvestigativeDermatology. It retains its original pagination here. Forcitation purposes, please use these original publication details: Wehner MR, Levandoski KA, Kulldorff M, Asgari MM. Research Techniques Made Simple: AnIntroduction to Use and Analysis of Big Data in Dermatology. J Invest Dermatol 2017;137(8):e153ee158. doi:10.1016/j.jid.2017.04.019

This work is licensed under a Creative CommonsAttribution-NonCommercial-NoDerivatives 4.0

International License. To view a copy of this license, visithttp://creativecommons.org/licenses/by-nc-nd/4.0/



Research Techniques Made Simple:Bioinformatics for Genome-Scale BiologyAmy C. Foulkes1, David S. Watson2, Christopher E.M. Griffiths1, Richard B. Warren1, Wolfgang Huber3

and Michael R. Barnes2

High-throughput biology presents unique opportunities and challenges for dermatological research. Drawingon a small handful of exemplary studies, we review some of the major lessons of these new technologies. Wecaution against several common errors and introduce helpful statistical concepts that may be unfamiliar toresearchers without experience in bioinformatics. We recommend specific software tools that can aid der-matologists at varying levels of computational literacy, including platforms with command line and graphicaluser interfaces. The future of dermatology lies in integrative research, in which clinicians, laboratory scientists,and data analysts come together to plan, execute, and publish their work in open forums that promote criticaldiscussion and reproducibility. In this article, we offer guidelines that we hope will steer researchers towardbest practices for this new and dynamic era of data intensive dermatology.


CME Activity Dates: 21 August 2017Expiration Date: 20 August 2018Estimated Time to Complete: 1 hour

Planning Committee/Speaker Disclosure: Amy Foulkes is aconsultant/advisor for AbbVie, Almirral, Eli Lilly, Leo Pharma,Novartis, Pfizer, Janssen and UCB. Christopher Griffiths is onthe speakers’ bureau and is a consultant/advisor for AbbVie,GSK, Janssen, Pfizer, Lilly, Novartis, Celgene, Leo Pharma,UCB, Sun Pharmaceuticals, and Almirral; in addition, Dr.Griffiths receives research grant support from AbbVie, GSK,Janssen, Pfizer, Lilly, Novartis, Sandoz, Celgene, and LeoPharma. All other authors, planning committee members,CME committee members and staff involved with this activityas content validation reviewers have no financial relation-ships with commercial interests to disclose relative to thecontent of this CME activity.



Objectives: At the conclusion of this activity, learners shouldbe better able to:� Recognize the newest techniques in biomedical research.

� Describe how these techniques can be utilized and theirlimitations.



Method of Physician Participation in Learning Process: Thecontent can be read from the Journal of Investigative Derma-tology website: http://www.jidonline.org/current. Tests forCME credits may only be submitted online at https://beaumont.cloud-cme.com/RTMS-Sept17 e click ‘CME on Demand’ andlocate the article to complete the test. Fax or other copies willnot be accepted. To receive credits, learners must review theCME accreditation information; view the entire article, com-plete the post-test with a minimum performance level of 60%;and complete the online evaluation form in order to claimCME credit. The CME credit code for this activity is: 21310.For questions about CME credit email [email protected].

INTRODUCTIONModern dermatology has been revolutionized by themany so-called ‘omic’ profiling platforms enabled byhigh-throughput sequencing (HTS, also referred to as next-

generation sequencing). Plunging data generation costshave enabled dermatology researchers to generate genomescale data relating to genome sequence variation (Scott et al.,2013), epigenomes (Zhou et al., 2016), and transcriptomes

1The Dermatology Centre, Salford Royal NHS Foundation Trust, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK;2William Harvey Research Institute, Centre for Translational Bioinformatics, Barts and The London School of Medicine and Dentistry, Charterhouse Square,London, UK; and 3European Molecular Biology Laboratory, Heidelberg, Germany

Correspondence: A.C. Foulkes, NIHR Academic Clinical Lecturer in Dermatology, The Dermatology Centre, Salford Royal NHS Foundation Trust, The Universityof Manchester, Manchester Academic Health Science Centre, M6 8HD. E-mail: [email protected]

Abbreviations: HTS, high-throughput sequencing; RNA-seq, RNA sequencing



(Li et al., 2014; Swindell et al., 2016), and these de-velopments have increased the dermatology-relevant dataopenly available in repositories (Table 1).

Bioinformatics refers to the tools used to collect, classify,and analyze such datasets, collectively enabling the field ofcomputational biology. Bioinformatics techniques have beendeveloped to make sense of the output of omic platforms,including HTS, microarrays, liquid chromatography-massspectrometry, and others (Kimball et al., 2012).

Physicians are key instigators of research data collectionrequiring computational biology. Structured and validatedanalysis pipelines for most omic data have been implementedfor researchers at various levels of complexity. Software hasbeen designed for all ranges of computational ability, fromsimple “point and click” graphic user interfaces to highlycustomizable command line interfaces, with the latterapproach offering superior flexibility and analyticalcomplexity. Although programming may seem like a dauntingchallenge for those without backgrounds in math, computerscience, or statistics, with practice, computational methodsfor exploratory and inferential analytics can become afamiliar part of the research toolkit. Of course, there is nosubstitute for expertise, and we advise all research teamsworking with omic data to consult a bioinformatician earlyand often. Here we highlight several points of special rele-vance to the dermatologist and dermatology researcher,based on the first-hand experience of a junior clinician.

CONSIDERATIONS BEFORE DATA COLLECTIONExperimental DesignResearchers in dermatology use a wide variety of HTS tech-niques, many of which have been discussed previously in theResearch Techniques Made Simple series. These includetranscriptome analysis with RNA sequencing (RNA-seq)(Antonini et al., 2017; Whitley et al., 2016), immunose-quencing (Matos et al., 2017), genome-wide epigenetics(Capell and Berger, 2013), proteomics, metabolomics, meta-genomics, and assessment of the microbiome (Jo et al., 2016).Additionally, the Molecular Revolution in Cutaneous Biologyseries provided an overview of HTS techniques (Anbunathanand Bowcock, 2017; Botchkareva, 2017; Johnston et al.,2017; Kong and Segre, 2017; Sarig et al., 2017), as didGrada and Weinbrecht (2013) in an earlier Research Tech-niques Made Simple publication. However, researchers oftendo not reach out to data analysts until a study is practicallycomplete. At that point, they may look for a mathematicallyinclined colleague to fill in the blanks of a statistical modeland provide a friendly P-value suitable for publication. Thisorder of events is all wrong. As Ronald Fisher famously put itback in 1938, “To consult the statistician after an experimentis finished is often merely to ask him to conduct a post-mortem examination. He can perhaps say what the experi-ment died of” (Fisher, 1938).

The data analysis strategy, including the choice of statisticalapproaches, should be integral to planning any researchstudy. Hypothesis testing, regression, and other statisticalmethods rely on rigorous collection and quality of the data,and any lapses here usually cannot be fixed retrospectively.How many samples are required to adequately power yourexperiment? If samples cannot be processed all at once, doesit matter how they are grouped into separate batches? If thedata do not corroborate your hypothesis, can a modifiedresearch question generate interesting results? Failure toconsider these questions before data collection may doom astudy before it even begins. Statistical expertise is required toanswer these questions, which is why we urge researchersto team up with a data analyst who can help guide themthrough these tricky issues. This will typically either be astatistician, with a background in math and statistics, or a

Table 1. High-throughput sequencing repositoriesRepository Website Curator

EuropeEuropean NucleotideArchive (ENA)

http://www.ebi.ac.uk/ena

European BioinformaticsInstitute

ArrayExpress http://www.ebi.ac.uk/arrayexpress


European Genome-phenome Archive (EGA)

https://www.ebi.ac.uk/ega/home


United StatesdbGAP https://www.ncbi.

nlm.nih.gov/gapThe National Center for

Biotechnology InformationGene ExpressionOmnibus (GEO)

https://www.ncbi.nlm.nih.gov/geo

The National Center forBiotechnology Information

Short Read Archive (SRA) https://www.ncbi.nlm.nih.gov/sra

The National Center forBiotechnology Information

ADVANTAGES� Bioinformatics methods allow efficient andpowerful analysis of multi-omic data in a way thatcould not be achieved using simpler methods.

� Bioinformatics software are customizable to allranges of computational ability; however, someinformatics tasks are difficult and requireexperience.

� Involving bioinformatician colleagues fromproject conception should improve projectdesign, maximizing the opportunity to detectrelevant association.

� Sharing data, metadata, and code, andpropagating the culture of bioinformaticians,will fuel best practices in dermatology research,promoting open research and reproducibility.

LIMITATIONS� Some statistical analysis methods require anunderstanding of underlying assumptions—erroneous assumptions can lead to false results.

� The use of some analytical pipelines requiresaccess to high-performance computing facilities:this may be achieved by access to omic corefacilities that provide researchers withcompressed datasets that are amenable tocomputer-based analysis.



bioinformatician, more likely with a background in computerscience and machine learning. Although there is considerableoverlap in their respective areas of expertise, statisticians andbioinformaticians may offer differing (and sometimes com-plementary) perspectives on a given biological question.

One of the most fundamental tools in statistical analysis ishypothesis testing. The principles of hypothesis testing areillustrated in Table 2, which highlights the work of Li et al.(2014) as an exemplar study in the field (see SupplementarySlides online). In this exploratory study, RNA-seq was usedto evaluate the transcriptomes of lesional psoriatic and

normal skin (from a large cohort of 174 individuals). A subsetof these samples has been studied previously using micro-arrays, allowing for comparison of the methodologies; RNA-seq identified many more differentially expressed transcriptsenriched in immune system processes.

Detailed discussion of requirements for testing a hypothesiswill facilitate better downstream clinical data collection, ul-timately maximizing the opportunity to detect a clinicallyrelevant association. Several key themes tend to dominateexperimental design considerations, including selection ofappropriate numbers of biological replicates (Schurch et al.,2016), minimization of batch effects (Leek et al., 2010), andappropriate correction for multiple testing (Allison et al.,2006). For a general overview of issues related to HTS studydesign, we recommend other excellent reviews (Allison et al.,2006; Conesa et al., 2016).

The steps outlined in Table 1 apply to most forms of omicdata. Methods for computing test statistics vary depending onthe data and underlying statistical assumptions. Commondata types and test statistics used in dermatological researchare discussed elsewhere (Silverberg, 2015).

Batch EffectsOften a study’s sample size exceeds the maximum number ofsamples that can be simultaneously processed by the availableequipment. In such cases, it is common to process the samplesin multiple batches. This inevitably introduces batch effects, inwhich technical artifacts become significant, perhaps evendominant drivers of variation in a dataset. There are severalmethods for batch adjustment (Oytam et al., 2016).

Each method has its merits, but none can overcomepoor study design. If a batch is confounded with a clinicalcovariate—say, all disease samples were processed in Batch A,and all healthy samples were processed in Batch B—then thereis no way to disentangle the technical from the biologicalvariation. Ideally, each batch would represent a microcosm of

Table 2. Principles of hypothesis testing from Li et al.(2014)Step in Hypothesis Testing Example

Ask a clinically relevant, testablequestion

Is there a significant differencebetween this set of genes expressedin subjects with psoriasis versus

those without?Choose an experimental designand statistical framework

Gene expression is modeled as alinear function of disease condition

Set up a null hypothesis, that is,a testable claim that becomes thetarget of statistical analysis

There is no significant differencebetween the average expression ofgene g in subjects with and without

psoriasisFix a rejection region, that is, thedegree of evidence against thenull hypothesis at which it maybe rejected

Genes whose t-statistics correspondto false discovery rates � 5% aredeclared differentially expressed

Conduct the experiment: collectdata, compute the test statistics

Expression levels for each gene giare regressed onto one or severalclinical predictors, generating a

vector of t-statisticsReport results: all and only thosegenes that fall within the rejectionregion are declared differentiallyexpressed

A number of genes weresignificantly differentially expressed

in plaques of psoriasis whencompared with control samples

Figure 1. Common methodology forprocessing of short reads.



the experiment itself, with proportionate numbers of samplesfrom all relevant groups. Although this cannot always be donein practice, the closer researchers come to attaining this goal,the more accurate their results will be.

CONSIDERATIONS AFTER DATA COLLECTIONSoftware and Workflows for Omic AnalysisAs a rule of thumb, processing of raw HTS data, includinggenome alignment and assembly, is likely to require access toone or several devoted computers that can execute jobs inparallel. However, once the initial data processing is complete,in most cases the biological downstream analysis can be per-formed using a laptop. The analysis of omic data, includingHTS, is supported by a range of widely used software packagesthat can be arranged into analysis workflows. Many packageshave been made freely available by their authors with an opensource license, and in this field there is very little correlationbetween the price of software and its usefulness. A workflow isa software pipeline that takes raw data as input, transforms andsummarizes the data, conducts exploratory and/or inferentialanalytics, and exports results ready for biological interpreta-tion. Command line genomic analysis tools can be scaled touse available computing resources and are highly custom-izable to meet the requirements of an experiment. Many stan-dard analysis tools can also be accessed remotely using theGalaxy workflow environment (https://usegalaxy.org). Galaxyoffers users a simple but highly customizable graphic userinterface environment to perform many bioinformatics tasks.Galaxy is also well documented and serves as an excellentintroduction to HTS analysis pipelines.

Processing HTS DataThe short read is the common currency of HTSmethods, but theway the read is processed is highly dependent on the analysisobjective (Figure 1). In most cases processing commences withalignment to a reference genome using a tool, such as Burrows-Wheeler Aligner or bowtie2, producing binary alignment mapfiles. The alignment files can serve as the input to many otherprocesses; in genetics they are used for variant calling, in epi-genomics for peak calling, and in transcriptomics to estimatetranscript abundance. A recent revolution in transcriptomics isalignment-free mapping methods, such as Kallisto (Bray et al.,2016) and Salmon (Patro et al., 2017). These tools circumventthe cumbersome alignment step and directly estimate transcriptabundance; they are several orders of magnitude faster than

alignment-based methods and so computationally efficient thatthey can be run on a laptop computer. The workflow used by Liet al. (2014) is illustrated in Figure 2.

Programming EnvironmentsAlthough many programming environments are used in bio-informatics, the most popular choices tend to be R (R CoreTeam, 2014) and Python (Python Software Foundation,2013). Software packages for these languages are oftenreleased under open source licenses, which means the toolsare free to use and the code is publicly accessible. Large usercommunities have developed around these languages, and Rin particular has become a lingua franca for bio-informaticians. This has been aided in no small part by theBioconductor project (Huber et al., 2015), a major repositoryfor biostatistical software based primarily on R. The site alsohosts discussion forums, encouraging active user engagementand collaborative learning.

Several programming environments are widely used inbioinformatics, including R, Matlab (Mathworks, 2012), andJava (see Table 3). These are open source and freely available,enabling statistical and graphical data manipulation withinlarge, active user communities.

Hypothesis Testing in the Age of Big DataHypothesis tests and P-values are the workhorses of medicalresearch, but some additional complexities enter the scenewhen we do not perform only one or a few tests, but thousandsor millions. Interpreting P-values is quite different in omiccontexts than in more traditional low-throughput research. Sayyou test 10,000 genes in search of biomarkers to distinguishbetween case and control samples. You find 500 with P-valuesbelow 0.05, not to mention 10 with p-values below 0.001.Not bad, right? Wrong! Because P-values are uniformlydistributed under the null hypothesis, we should expect 5% ofall tests to reach the nominal significance level of 0.05 bychance alone. That’s a manageable problem when testing oneor two hypotheses, but in omic experiments we typically testsomething on the order of thousands to millions of hypotheses.

Some early articles attempted to mitigate the issue bycontrolling the family-wise error rate, defined as the proba-bility of finding at least one false positive in a series of hy-pothesis tests. For example, the Bonferroni correction used by

Figure 2. Example bioinformatic pipeline used by Li et al. (2014).

Table 3. Open source programming languages andresources for bioinformatics analysis of omic dataOpen Source Resource URL

Analysis code repositoriesBioconductor bioconductor.orgCRAN www.cran.orgBioperl bioperl.orgBiopython biopython.github.ioGitHub github.comBioJulia github.com/BioJulia

Workflow toolsGalaxy usegalaxy.org

VisualizationShinyR Shiny.rstudio.comPlotly plot.ly



Li et al. (2014) strongly controls the family-wise error rate bysetting the significance threshold as the quotient of the type Ierror a and the total number of hypothesis tests m, so that alland only tests with P � a/m are declared significant. Althoughthe Bonferroni correction is guaranteed to control the family-wise error rate, it is an overly conservative method that islikely to lead to many false negatives as m grows.

Current practice is to control not the false positive rate (i.e.,the proportion of truly null features that are nominally sig-nificant) but the false discovery rate (i.e., the proportion ofnominally significant features that are truly null). This lattervalue is typically estimated using the Benjamini-Hochbergalgorithm (Hochberg and Benjamini, 1990) or some variantthereof. This method takes a list of P-values as input andreturns a matched list of adjusted P-values, also known as Q-values. Applying a 5% false discovery rate threshold meansthat 1 in 20 genes in the hit list will be a false positive. Given10,000 uniformly distributed P-values, as hypothesizedearlier, minimum Q-values are typically greater than 0.5.

VISUALIZATIONThe communication of results is key for data exploration,summarization, and ultimately publication. Readers can morereadily absorb a well-made graphic than any table ofnumbers. Visualizing HTS results can be challenging becauseof the data’s high dimensionality, but projection techniqueslike principal component analysis (Pearson, 1901), multi-dimensional scaling (Torgerson, 1952), and t-distributed sto-chastic neighbor embedding (van der Maaten et al., 2008)can render large matrices as easily digestible two-dimensionalor three-dimensional scatterplots. Matos et al. (2017) showhow these methods can give powerful insights for dermato-logical research. More recent interactive tools such as plotly(https://plot.ly/), shiny (https://shiny.rstudio.com/), and ggvis(http://ggvis.rstudio.com/) can also aid in data exploration oreven create widgets for HTML publication.

CODE SHARING AND REPRODUCIBILITYA number of studies have found an alarming lack of repro-ducibility in modern omic and clinical research (OpenScience Collaboration, 2015). Many factors contribute tothis problem, including the widespread failure to publishanalysis code (Baker, 2016). Although some inroads havebeen made toward establishing best practices in molecularbiology (Brazma et al., 2001), script sharing remains rareoverall. Results may vary greatly depending on subtle, un-stated analytic choices that are invisible without access toboth raw data and the complete analysis script. Code sharing

is a critical ingredient for open science; this will be apparentto researchers who have tried to reuse data in repositories,where code is absent and subject data are often incomplete,making reproduction challenging at best. Excellent platformsexist for publishing code. Taking advantage of sites likeGitHub (https://github.com) can assist during peer review,enabling precise debate on the merits of particular methods.Set-up can be technically challenging, but user-friendlyguides exist (http://happygitwithr.com). Researchers shouldensure that they or their bioinformatician colleagues docu-ment and archive code, analogous to the use of a laboratorybook as a record of research. This will ensure that bio-informatician turnover will not prevent ongoing analysis,because code will be clear, maintained, and transferable.

Figure 3. Reproducibility: Creating a virtuous circle.

MULTIPLE CHOICE QUESTIONS1. Which is an accurate description of batch effect?

A. Technical source of variation added tosamples during handling

B. An uncommon problem in HTS experiments

C. Where proportionate samples areanalyzed in each experiment

D. A problem that is not possible to adjustfor using bioinformatic techniques

2. The relevant significance measure inomic data is

A. the P-value.

B. the false discovery rate.

C. the false positive rate.

D. the family-wise error rate.

3. Which of the following is an analysiscode repository?

A. GEO

B. R

C. Galaxy

D. GitHub

4. Which of the following statements istrue regarding sharing of analysis code?

A. This allows reproducibility of an analysis.

B. Sharing of analysis code istechnically challenging.

C. Analysis code is required alongsidesubmission of data and metadata forsubmission of original articles tomajor journals.

D. There is no code sharing repository.

5. Which of the following is a major repositoryfor biostatistical software?

A. ShinyR

B. Plotly

C. Ggvis

D. Bioconductor



SUMMARY AND FUTURE DIRECTIONSEmbedding biostatisticians and computational biologistswithin clinical and academic research teams, as well aspromoting better data and code sharing practices, will allowdermatologists to better document and communicate theirresearch. The days of assembly line research—in which cli-nicians recruit patients, laboratory scientists process samples,and analysts crunch numbers—are coming to an end. The ageof big data demands a rigorous, integrated approach.Appropriate statistical design and analysis methods should bediscussed and decided on up front to meet most researchobjectives. By incorporating good experimental design andanalytical work practice early, research quality and repro-ducibility will improve, and peer review by journals and grantawarding bodies is likely to be more favorable (Figure 3).Patients will be the ultimate beneficiaries of dermatology’sdrive to the forefront of life science research.

ORCIDAmy C Foulkes: http://orcid.org/0000-0003-2680-750X

CONFLICT OF INTERESTACF has received educational support to attend conferences from or acted as aconsultant or speaker for Abbvie, Almirall, Eli Lilly, Leo Pharma, Novartis, Pfizer,Janssen, and UCB. CEMG has acted as a consultant and/or speaker for Abbvie,Janssen,Novartis, Sandoz, RockCreek Pharma, Pfizer, Eli Lilly,UCB, LeoPharma,Galderma,andCelgene.RBWhasactedasaconsultantand/or speaker forAbbvie,Amgen, Almirall, Boehringer, Medac, Eli Lilly, Janssen, Leo Pharma, Pfizer,Novartis, Sun Pharma, Valeant, Schering-Plough (nowMSD), and Xenoport.

ACKNOWLEDGMENTSThis forms part of the research themes contributing to the translational researchportfolio of Barts and the London Cardiovascular Biomedical Research Centre,which is supported and funded by the National Institute of Health Research.


REFERENCESAllison DB, Cui X, Page GP, Sabripour M. Microarray data analysis: from disarray to

consolidation and consensus. Nat Rev Genet 2006;7:55e65.

Anbunathan H, Bowcock AM. The molecular revolution in cutaneous biology: theera of genome-wide association studies and statistical, big data, and compu-tational topics. J Invest Dermatol 2017;137:e113e8.

Antonini D, Mollo MR, Missero C. Research techniques made simple: identificationand characterization of long noncoding RNA in dermatological research.J Invest Dermatol 2017;137(3):e21e6.

Baker M. 1,500 scientists lift the lid on reproducibility. Nature 2016;533(7604):452e4.

Botchkareva NV. The molecular revolution in cutaneous biology: noncoding RNAs:new molecular players in dermatology and cutaneous biology. J Invest Der-matol 2017;137:e105e11.

Bray NL, Pimentel H, Melsted P, Pachter L. Near-optimal probabilistic RNA-seqquantification. Nat Biotechnol 2016;34:525e7.

Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, et al.Minimum information about a microarray experiment (MIAME)-toward stan-dards for microarray data. Nat Genet 2001;29:365e71.

Capell BC, Berger SL. Genome-wide epigenetics. J Invest Dermatol 2013;133:e9.

Conesa A, Madrigal P, Tarazona S, Gomez-Cabrero D, Cervera A, McPherson A, et al.A survey of best practices for RNA-seq data analysis. Genome Biol 2016;17:13.

Fisher RA. Presidential address to the first Indian Statistical Congress. Sankhya1938;4:14e7.

Grada A, Weinbrecht K. Next-generation sequencing: methodology and applica-tion. J Invest Dermatol 2013;133:e11.

Hochberg Y, Benjamini Y. More powerful procedures for multiple significancetesting. Stat Med 1990;9:811e8.

Huber W, Carey VJ, Gentleman R, Anders S, Carlson M, Carvalho BS, et al.Orchestrating high-throughput genomic analysis with Bioconductor. NatureMethods 2015;12:115e21.

Jo JH, Kennedy EA, Kong HH. Research techniques made simple: bacterial 16Sribosomal RNA gene sequencing in cutaneous research. J Invest Dermatol2016;136:e23e7.

Johnston A, Sarkar MK, Vrana A, Tsoi LC, Gudjonsson JE. The molecular revolutionin cutaneous biology: the era of global transcriptional analysis. J Invest Der-matol 2017;137:e87e91.

Kimball AB, Grant RA, Wang F, Osborne R, Tiesman JP. Beyond the blot: cuttingedge tools for genomics, proteomics and metabolomics analyses and previoussuccesses. Br J Dermatol 2012;166(Suppl. 2):1e8.

Kong HH, Segre JA. The molecular revolution in cutaneous biology: investigatingthe skin microbiome. J Invest Dermatol 2017;137(5):e119e22.

Leek JT, Scharpf RB, Bravo HC, Simcha D, Langmead B, Johnson WE, et al. Tacklingthe widespread and critical impact of batch effects in high-throughput data.Nat Rev Genet 2010;11:733e9.

Li B, Tsoi LC, Swindell WR, Gudjonsson JE, Tejasvi T, Johnston A, et al. Tran-scriptome analysis of psoriasis in a large case-control sample: RNA-seq pro-vides insights into disease mechanisms. J Invest Dermatol 2014;134:1828e38.

MathWorks. MATLAB and Statistics Toolbox Release. Boston, MA; 2012.

Matos TR, de Rie MA, Teunissen MBM. Research techniques made simple: high-throughput sequencing of the T-cell receptor. J Invest Dermatol 2017;137:e131e8.

Open Science Collaboration. PSYCHOLOGY. Estimating the reproducibility ofpsychological science. Science 2015;349(6251):aac4716.

Oytam Y, Sobhanmanesh F, Duesing K, Bowden JC, Osmond-McLeod M, Ross J.Risk-conscious correction of batch effects: maximising information extractionfrom high-throughput genomic datasets. BMC Bioinformatics 2016;17:332.

Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C. Salmon provides fast and bias-aware quantification of transcript expression. Nat Methods 2017;14:417e9.

Pearson K. On lines and planes of closest fit to systems of points in space. Philo-sophical Magazine 1901;2:559e72.

Python Software Foundation. Python Language Reference. Wilmington, DE; 2013.

R Core Team. R: A language and environment for statistical computing. Vienna,Austria: R Foundation for Statistical Computing; 2014. http://www.R-project.org/. Accessed 2 August 2017.

Sarig O, Sprecher E. The molecular revolution in cutaneous biology: era of next-generation sequencing. J Invest Dermatol 2017;137:e79e82.

Schurch NJ, Schofield P, Gierlinski M, Cole C, Sherstnev A, Singh V, et al. How manybiological replicates are needed in an RNA-seq experiment and which differ-ential expression tool should you use? RNA 2016;22:839e51.

Scott CA, Plagnol V, Nitoiu D, Bland PJ, Blaydon DC, Chronnell CM, et al. Targetedsequence capture and high-throughput sequencing in the molecular diagnosisof ichthyosis and other skin diseases. J Invest Dermatol 2013;133:573e6.

Silverberg JI. Study designs in dermatology: practical applications of study designs andtheir statistics in dermatology. J Am Acad Dermatol 2015;73:733e40.

Swindell WR, Sarkar MK, Liang Y, Xing X, Gudjonsson JE. Cross-disease tran-scriptomics: unique IL-17A signaling in psoriasis lesions and an autoimmunePBMC signature. J Invest Dermatol 2016;136:1820e30.

Torgerson WS. Multidimensional scaling: I. Theory and method. Psychometrika1952;17:401e19.

Whitley SK, Horne WT, Kolls JK. Research techniques made simple: methodologyand clinical applications of RNA sequencing. J Invest Dermatol 2016;136:e77e82.

van der Maaten L, Hinton G, van der Maaten GH. Visualizing Data using t-SNE.Journal of Machine Learning Research 2008;9:2579e605.

Zhou F, Wang W, Shen C, Li H, Zuo X, Zheng X, et al. Epigenome-wide associationanalysis identified nine skin DNA methylation loci for psoriasis. J InvestDermatol 2016;136:779e87.

This is a reprint of an article that originally appeared in the September 2017 issue of the Journal of InvestigativeDermatology. It retains its original pagination here.For citation purposes, please use these original publication details: Foulkes AC, Watson DS, Griffiths CEM, Warren RB, Huber W, Barnes MR. Research Tech-niques Made Simple: Bioinformatics for Genome-Scale Biology. J Invest Dermatol 2017;137(9):e163ee168. doi:10.1016/j.jid.2017.07.095



RTMS Article 49, October 2016 Research TechniquesMade Simple: Laser Capture Microdissection inCutaneous Research

QUESTIONS

1. What does LCM do?A. Sorts cells based on morphology (size, granularity,

density)B. Sorts cells with either IR or UV laser technologyC. Collects cells of interest with laser technologyD. Photoablates cells of interest with laser technology

2. What is the thinnest diameter of the UV-LCM laser beam?A. 0.5 mmB. 5.0 mmC. 7.5 mmD. 30 mm

3. Which technique uses a thermosensitive EVA film tosequester cells of interest?A. IR-LCMB. UV-LCMC. Laser microbeam microdissectionD. FACS

4. In the absence of a cover slip, the optical resolution ofcomplex tissues may be limited. This issue can beaddressed by:A. Using a temporary cover slipB. Increasing the thickness of the tissue sectionC. Decreasing the thickness of the tissue sectionD. Staining with immunohistochemistry

5. Which of the following statements regarding the LCMtechnique is NOT true?A. UV-LCM is better suited for single cell microdissectionB. IR-LCM is more time consuming than UV-LCMC. The EVA membrane undergoes conformational change

when exposed to IR laser energyD. The downstream analysis of proteins is limited in FFPE

tissue sections due to undesirable protein and nucleicacid crosslinking

ANSWERS

1. C.

2. A.

3. A.

4. D.

5. B.

ª 2016 Society for Investigative Dermatology www.jidonline.org i

� RTMS ANSWERS

RTMS Article 50, November 2016 Research TechniquesMade Simple: Assessing Risk of Bias in Systematic Reviews

QUESTIONS

1. The protocol for a systematic review should.A. Be written and made publicly available before

conducting the reviewB. Contain information on the sources of data that will be

usedC. Contain information on the outcomes that will be

assessedD. Contain information on the criteria used to include and

exclude studiesE. All of the above

2. Publication bias occurs because.A. The peer review process takes too longB. Studies with statistically nonsignificant findings are less

likely to be publishedC. Journals prefer to publish studies with nonsignificant

findings rather than those with statistically significantfindings

D. Systematic reviewers change their outcome of interestafter designing their protocol

3. Because they involve searches of the existing literatureand pooling of multiple primary studies, systematicreviews.A. Are not prone to bias because they have a larger

sample size than primary studiesB. Are always able to find all available data on a topicC. Have their own sources of bias in addition to biases

that exist in any primary studiesD. Are easy to conduct and can be accomplished without

significant effort

4. Which of the following is not a type of “spin”?A. Discussing limitations (e.g., explaining potential

sources of missing data)B. Misleading reporting (e.g., not fully reporting the

methods used to collect data)C. Misleading interpretation (e.g., discussing

nonsignificant results as if they were significant)D. Inappropriate extrapolation (e.g., application of the

study results to a patient population not actuallystudied in the systematic review)

5. Which of the following is true regarding competinginterests?A. Authors of systematic reviews should disclose all

potential competing interestsB. If a nonindustry organization funded the conduct of

a systematic review, but did not design, conduct, orwrite the review, then the funding does not need to bedeclared

C. Although declaring competing interests is stillimportant, funding from industry has been shown tohave no effect on the results of systematic reviews

D. Systematic reviews without industry funding never use“spin” when writing systematic review reports

ANSWERS

1. E.

2. B.

3. C.

4. A.

5. A.

Journal of Investigative Dermatology (2016), Volume 136 ª 2016 Society for Investigative Dermatologyii

RTMS ANSWERS �

RTMS Article 51, December 2016 Research TechniquesMade Simple: Workflow for Searching Databases toReduce Evidence Selection Bias in Systematic Reviews

QUESTIONS

1. Which of the following would result in publication bias?A. Trials with negative results were not published and

could not be selected in the systematic review.B. Trials with statistically significant results were cited

more often by subsequent articles, increasing thelikelihood of being selected in the systematic review.

C. Trials were published in languages other than Englishand could not be selected in the systematic review.

D. Trials were published more than once, increasing thelikelihood of the trial being selected in the systematicreview.

E. All of the above

2. Searching beyond bibliographical databases for asystematic review potentially reduces which of thefollowing?A. Publication biasB. Validity of the systematic reviewC. Outcome reporting biasD. Labor intensity of the searchE. A and C

3. The sources to search for published trials include whichof the following?A. MEDLINE onlyB. The Cochrane Central Register of Controlled TrialsC. The Cochrane Database of Systematic ReviewsD. EMBASEE. B, C, and D

4. The sources to search for unpublished trials includewhich of the following?A. clinicaltrials.govB. alltrials.netC. Drugs@FDAD. Proceedings to the American Academy of Dermatology

Annual MeetingE. A, C, and D

5. Which of the following are some limitations of sources ofunpublished trials?A. Clinical trial registries include ongoing and completed

trials and potentially posted trial results.B. Reviews obtained from regulatory agencies typically

lack sufficient detail to assess the risk of bias for a trial.C. Conference abstracts are not restricted by treatment

type (pharmacological and nonpharmacological).D. Searching conference abstracts, clinical trial registries,

and regulatory and health technology assessmentagency websites is burdensome.

E. B and D

ANSWERS

1. A.

2. E.

3. E.

4. E.

5. E.

ª 2016 Society for Investigative Dermatology www.jidonline.org iii

� RTMS ANSWERS

RTMS Article 52, January 2017 Research Techniques MadeSimple: Mouse Models of Autoimmune Blistering Diseases

QUESTIONS

1. Which of the following is not characteristic of allautoimmune blistering diseases?A. Blisters on the skin and/or mucous membranesB. IgG autoantibodiesC. Autoantibodies against autoantigens in the skinD. Loss of self-tolerance

2. Which knockout mice are immunized with autoantigen inthe active disease model of pemphigus vulgaris?A. Dsg1e/e miceB. COL7e/e miceC. Dsg3e/e miceD. Dsg2e/e mice

3. Which domain of type VII collagen is used for theimmunization-induced model for epidermolysisbullosa acquisita?A. NC1B. NC2C. NC3D. NC4

4. What is/are the main autoantigen(s) in bullouspemphigoid?A. COL17/BP180B. BP230C. COL17/BP180 and BP230D. COL17/BP180 and BP250

5. Which Dsg3-specific antibody can induce a pemphigusvulgaris-resembling phenotype in wild-type mice?A. AK7B. AK47C. AK3D. AK23

ANSWERS

1. B.

2. C.

3. A.

4. C.

5. D.

Journal of Investigative Dermatology (2017), Volume 137 ª 2016 Society for Investigative Dermatologyiv

RTMS ANSWERS �

RTMS Article 53, February 2017 Research TechniquesMade Simple: Analysis of Collective Cell Migration Usingthe Wound Healing Assay

QUESTIONS

1. Which of the following treatments can be used tosuppress cell proliferation so that it does not interferewith in vitro measurement of cell migration?A. Mitomycin CB. PaclitaxelC. Serum starvationD. VinblastineE. A and C

2. The wound healing assay is performed in the followingsequence:A. Cell culture, image collection, scratch-making,

sequencingB. Cell culture, scratch-making, data acquisition, data

analysisC. Scratch-making, cell culture, freezing, image collectionD. Cell coating, DNA sequencing, alignment to a

reference genomeE. Data acquisition, culture preparation, scratch-making,

data analysis

3. Advantages of the wound healing assay include all of thefollowing except:A. Affordable and easy to set upB. High reproducibilityC. It does not require the use of specific chemoattractants

or gradient chambersD. Suitable for chemotaxis studiesE. B and D

4. Applications of the wound healing assay may include:A. Helping to identify therapies to promote cell migration

in wound healingB. Evaluation of the effects of inhibitors/enhancers on the

migratory capacity of a particular cell populationC. Investigating the mechanisms regulating cancer cell

migration and evaluating the efficacy of potentialtherapeutic drugs

D. Studying regulation of actin cytoskeletal structures andcell polarity

E. All of the above

5. Which of the following measures can enhancereproducibility of results when performing thein vitro wound healing assay?A. Always seed cells at the same density and start the

assay at the same degree of confluence.B. If a manual scratch must be made, use consistent

pressure and pipette tip angle to create uniformscratch sizes and shapes.

C. Incubate cells for no more than 24 hours.D. Increase the sample number.E. A and B

ANSWERS

1. E.

2. B.

3. E.

4. E.

5. E.

ª 2016 Society for Investigative Dermatology www.jidonline.org v

� RTMS ANSWERS

RTMS Article 54, March 2017 Research Techniques MadeSimple: Identification and Characterization of LongNoncoding RNA in Dermatological Research

QUESTIONS

1. Identification of a novel lncRNA may be performed by:A. mQuantitative reverse transcription polymerase chain

reaction (qRT-PCR)B. RNA fluorescence in situ hybridization (RNA FISH)C. Chromatin isolation by RNA purification followed by

deep sequencing (ChIRP-seq)D. RNA sequencing (RNA-seq)

2. Validation of novel identified lncRNAs may bedetermined by:A. Quantitative reverse transcription polymerase chain

reaction (qRT-PCR)B. RNA interactome analysis followed by deep

sequencing (RIA-seq)C. Chromatin isolation by RNA purification followed by

deep sequencing (ChIRP-seq)D. lncRNA interacting protein analysis

3. RNA fluorescence in situ hybridization (RNA FISH) iscapable of:A. Identifying RNA-interacting proteinsB. Identifying the RNA interactomeC. Determining the subcellular localization of RNAD. Identifying the functional role of RNA

4. Identification of lncRNA-binding proteins can beachieved by:A. RNA interactome analysis followed by deep

sequencing (RIA-seq)B. Chromatin isolation by RNA purification followed by

deep sequencing (ChIRP-seq)C. RNA pulldown followed by Mass SpectrometryD. Specific knockdown of lncRNA

5. RNA interactome analysis followed by deep sequencing(RIA-seq) is capable of:A. Localizing the cellular compartment in which the

lncRNA is expressedB. Identifying the RNAs that interact with lncRNAC. Identifying novel transcribed regions and alternative

spliced forms of annotated genesD. Identifying the RNA interacting proteins

ANSWERS

1. D.

2. A.

3. C.

4. C.

5. B.

Journal of Investigative Dermatology (2017), Volume 137 ª 2017 Society for Investigative Dermatologyvi

RTMS ANSWERS �

RTMS Article 55, April 2017 Research Techniques MadeSimple: Experimental Methodology for Single-Cell MassCytometry

QUESTIONS

1. Which of the following are two main considerations whendesigning a CyTOF panel?A. Antigen abundance and crosstalkB. Use only ready-to-use panel kits and commercially

available MCAsC. Low number of available probes and sample cell typeD. Vast variability of signal detection across channels and

low isotopic purity

2. What are the principal sources of signal background inCyTOF?A. Concentration of metal-conjugated antibodies and

reagents during stainingB. Environmental contamination and crosstalkC. Plasma temperature and instrument state of

maintenanceD. Endogenous cell signal and spectral overlap

3. Which cell-staining protocol should be used for CyTOFexperiments?A. Standard CyTOF protocolB. Same protocol as for flow cytometryC. CyTOF staining protocol tested for specific experiment

panelD. Any CyTOF-validated protocol

4. What can be done to ensure an accurate analysis of livesingle cells?A. Use only fresh and filtered cell samples.B. Use nucleic acid intercalators.C. Acquire data on the same day as performing the

staining protocol.D. Use calibration beads.

5. Which CyTOF-specific strategies should be used tocontrol for sample-to-sample variation?A. Use a commercial, ready-to-use panel kit and

high-quality reagents.B. Incorporate nucleic acid intercalators and use the same

cell concentration in all samples.C. Make sure to use appropriate statistical tests and the

same analysis software for all samples.D. Normalize samples based on bead standards and

barcode samples.

ANSWERS

1. A.

2. B.

3. C.

4. B.

5. D.

ª 2017 Society for Investigative Dermatology www.jidonline.org vii

� RTMS ANSWERS

RTMS Article 56, May 2017 Research Techniques MadeSimple: Mass Cytometry Analysis Tools for Decryptingthe Complexity of Biological Systems

QUESTIONS

1. What is the best CyTOF data analysis tool?A. Same methods as for flow cytometryB. The analysis method depends on specific experimental

goals.C. Manual clustering methods through biaxial plotsD. Comparisons of marker expression using histograms

2. Identify one advantage of principal component analysis(PCA).A. Displays data in two-dimensional representationB. Results are represented through linear projectionsC. Identifies parameters with the most varianceD. Capable of analyzing only a few parameters

3. Which of the following is a limitation of spanning-treeprogression analysis of density-normalized events(SPADE)?A. Incapable of reproducing the same representation of

results when analyzed more than onceB. Represents cell subset hierarchiesC. Assumes that data is parametricD. Does not allow comparing marker expression among

subsets and samples

4. Select one advantage of t-distributed stochastic neighborembedding (t-SNE)ebased visualization (viSNE) andautomatic classification of cellular expression bynonlinear stochastic embedding (ACCENSE).A. It captures only nonlinear relationships among the

dataset.B. It clusters cells into exclusive nodes.C. It allows representation of single cells without

clusteringD. It allows the researcher to identify clusters capable of

predicting the sample’s outcome.

5. What is the novel application of analyzing data usingcluster identification, characterization, and regression(CITRUS)?A. It is capable of displaying data in three-dimensional

representations.B. It identifies rare cell subsets with the highest expression

of studied markers.C. It allows the researcher to define cell population

hierarchies.D. It identifies cellular features that correlate to an

experimental endpoint of interest.

ANSWERS

1. B.

2. C.

3. A.

4. C.

5. D.

Journal of Investigative Dermatology (2017), Volume 137 ª 2017 Society for Investigative Dermatologyviii

RTMS ANSWERS �

RTMS Article 57, June 2017 Research Techniques MadeSimple: High-Throughput Sequencing of the T-CellReceptor

QUESTIONS

1. What does HTS of the TCR identify?A. Identifies the DNA sequence of the entire TCRB. Identifies the specific epitope of each TCRC. Identifies the variable and constant region of each TCRD. Identifies and quantifies each and every T cell present

in a sample

2. All of the following are advantages of HTS of the TCR,except the following:A. Uses highly accurate radioactive agentsB. High clone detection sensitivityC. Can be applied to any biologic tissue and small

samplesD. Low rate of false positives and false negatives

3. All of the following steps are part of the HTS of the TCRmethodology, except the following:A. Bias-controlled multiplexed PCRB. Western blotC. Advanced bioinformaticsD. HTS of the CDR3 of the TCR

4. What does the clonality score measure?A. The probability of a clone being present within the total

cell repertoireB. How much a sample is dominated by clonal expansionC. How many distinct clones are present in a sampleD. How many T cells belong to each clonal population

5. HTS can be used to study the following:A. The immune response to infectious diseases or vaccineB. Pathogenesis of T-celleassociated diseasesC. Diagnosis biomarkers and/or immune response to

therapies.D. All of the above

ANSWERS

1. D.

2. A.

3. B.

4. B.

5. D.

ª 2017 Society for Investigative Dermatology www.jidonline.org ix

� RTMS ANSWERS

RTMS Article 58, July 2017 Research Techniques MadeSimple: Cost-Effectiveness Analysis

QUESTIONS

1. This unit of cost effectiveness is defined as

ðcost of intervention X � cost of comparison intervention YÞðeffect of intervention X � effect of comparison intervention YÞ

A. Cost-benefit ratioB. Quality-adjusted life-yearC. Incremental cost-effectiveness ratioD. Average cost effectiveness

2. What type of analysis is performed to simulate real-worlduncertainty in the parameters of the cost-effectivenessanalysis and test assumptions under varying conditions?A. Cost-benefit analysisB. Sensitivity analysisC. Comparative effectiveness analysisD. Chi-square analysis

3. This type of model can be useful in cost-effectivenessanalysis for simulating the complex course of chronicdisease or conditions in which there is transition backand forth between disease states.A. Decision tree analysisB. Logistic regression modelC. Cox proportional hazards modelD. Markov model

4. Which of the following is a commonly used thresholdfor valuing a single quality-adjusted life-year (QALY)in cost-effectiveness analysis?A. $500/QALYB. $5,000/QALYC. $50,000/QALYD. $500,000/QALY

5. Which of the following costs should be factored in as partof a cost-effectiveness analysis for a newly developedpharmaceutical treatment?A. Retail price of the drugB. Physician time necessary to administer the drug to

patientsC. Patient time out of work to recover from adverse effects

of the drugD. All of the above

ANSWERS

1. C.

2. B.

3. D.

4. C.

5. D.

Journal of Investigative Dermatology (2017), Volume 137 ª 2017 Society for Investigative Dermatologyx

RTMS ANSWERS �

RTMS Article 59, August 2017 Research Techniques MadeSimple: An Introduction to Use and Analysis of Big Data inDermatology

QUESTIONS

1. What are the 3 Vs that characterize big data?A. Value, viability, and varietyB. Volume, velocity, and viabilityC. Volume, velocity, and varietyD. Volume, value, and variety

2. What distinguishes big data analyses from traditional dataanalyses?A. They can be used to both test and generate hypotheses.B. Variables are often not preselected for prediction

modeling.C. They often center around identifying and evaluating

patterns.D. All of the above

3. What analytic technique focuses on grouping similarpatients by characteristics such as demographics,genetics, or geography and can be used to informgeographically targeted screening and prevention efforts?A. Cluster analysisB. Decision-tree learningC. Bayesian networksD. Cognitive computing

4. Which of the following is NOT a limitation of big data?A. Storage may require considerable resources.B. Formatting and analysis may require advanced

computer science.C. Big data can be used only for retrospective analyses.D. Big data have more complex security and information

privacy concerns than traditional datasets.

5. Which of the following is NOT a potential application ofbig data?A. Improve risk prediction for very rare diseasesB. Identify distinct disease phenotypes in heterogeneous

diseases that may merit different therapiesC. Identify causal associationsD. Perform drug and medical device surveillance

ANSWERS

1. C.

2. D.

3. A.

4. C.

5. C.

ª 2017 Society for Investigative Dermatology www.jidonline.org xi

� RTMS ANSWERS

RTMS Article 60, September 2017 Research TechniquesMade Simple: Bioinformatics for Genome-Scale Biology

QUESTIONS

1. Which is an accurate description of batch effect?A. Technical source of variation added to samples during

handlingB. An uncommon problem in HTS experimentsC. Where proportionate samples are analyzed in each

experimentD. A problem that is not possible to adjust for using

bioinformatic techniques

2. The relevant significance measure in omic data isA. the P-value.B. the false discovery rate.C. the false positive rate.D. the family-wise error rate.

3. Which of the following is an analysis code repository?A. GEOB. RC. GalaxyD. GitHub

4. Which of the following statements is true regardingsharing of analysis code?A. This allows reproducibility of an analysis.B. Sharing of analysis code is technically challenging.C. Analysis code is required alongside submission of data

and metadata for submission of original articles tomajor journals.

D. There is no code sharing repository.

5. Which of the following is a major repository forbiostatistical software?A. ShinyRB. PlotlyC. GgvisD. Bioconductor

ANSWERS

1. A.

2. B.

3. D.

4. A.

5. D.

Journal of Investigative Dermatology (2017), Volume 137 ª 2017 Society for Investigative Dermatologyxii

RTMS ANSWERS �

RTMS Articles 49e60 (2016e2017) Teaching Slides

Teaching slides are available for every RTMS article. Please visit the URLs below:

Article 49, October 2016 http://dx.doi.org/10.1016/j.jid.2016.08.005

Article 50, November 2016 http://dx.doi.org/10.1016/j.jid.2016.08.021

Article 51, December 2016 http://dx.doi.org/10.1016/j.jid.2016.09.019

Article 52, January 2017 http://dx.doi.org/10.1016/j.jid.2016.11.003

Article 53, February 2017 http://dx.doi.org/10.1016/j.jid.2016.11.020

Article 54, March 2017 http://dx.doi.org/10.1016/j.jid.2017.01.006

Article 55, April 2017 http://dx.doi.org/10.1016/j.jid.2017.02.006

Article 56, May 2017 http://dx.doi.org/10.1016/j.jid.2017.03.002

Article 57, June 2017 http://dx.doi.org/10.1016/j.jid.2017.04.001

Article 58, July 2017 http://dx.doi.org/10.1016/j.jid.2017.03.004

Article 59, August 2017 http://dx.doi.org/10.1016/j.jid.2017.04.019

Article 60, September 2017 http://dx.doi.org/10.1016/j.jid.2017.07.095

ª 2017 Society for Investigative Dermatology www.jidonline.org xiii

RTMS SUPPLEMENTARY INFORMATION

research techniques made simple articles 49–60 (2016–2017) … · 2018. 4. 6. · research...

Documents