REVIEW

Deep Learning and Neurology: A Systematic Review

Aly Al-Amyn Valliani . Daniel Ranti . Eric Karl Oermann

Received: June 27, 2019 / Published online: August 21, 2019� The Author(s) 2019

ABSTRACT

Deciphering the massive volume of complexelectronic data that has been compiled by hos-pital systems over the past decades has thepotential to revolutionize modern medicine, aswell as present significant challenges. Deeplearning is uniquely suited to address thesechallenges, and recent advances in techniquesand hardware have poised the field of medicalmachine learning for transformational growth.The clinical neurosciences are particularly wellpositioned to benefit from these advances giventhe subtle presentation of symptoms typical ofneurologic disease. Here we review the variousdomains in which deep learning algorithmshave already provided impetus for change—ar-eas such as medical image analysis for theimproved diagnosis of Alzheimer’s disease andthe early detection of acute neurologic events;medical image segmentation for quantitativeevaluation of neuroanatomy and vasculature;

connectome mapping for the diagnosis of Alz-heimer’s, autism spectrum disorder, and atten-tion deficit hyperactivity disorder; and miningof microscopic electroencephalogram signalsand granular genetic signatures. We addition-ally note important challenges in the integra-tion of deep learning tools in the clinical settingand discuss the barriers to tackling the chal-lenges that currently exist.

Keywords: Artificial intelligence; Biomedicalinformatics; Computer vision; Connectomemapping; Deep learning; Genomics; Machinelearning; Neurology; Neuroscience

INTRODUCTION

Twenty-first century healthcare is marked by anabundance of biomedical data and the devel-opment of high-performance computing toolscapable of analyzing these data. The availabilityof data and increased speed and power ofcomputer systems together present bothopportunities and challenges to researchers andhealthcare professionals. Most significantly,they provide the potential to discover new dis-ease correlates and translate these insights intonew data-driven medical tools that can improvethe quality and delivery of care. However, suchadvancements require the navigation of high-dimensional, unstructured, sparse, and often

Enhanced digital features To view enhanced digitalfeatures for this article go to https://doi.org/10.6084/m9.figshare.9272951.

Aly Al-Amyn Valliani and Daniel Ranti contributedequally to this article.

A. A.-A. Valliani � D. Ranti � E. K. Oermann (&)Department of Neurological Surgery, Mount SinaiHealth System, 1 Gustave Levy Pl, New York, NY10029, USAe-mail: eric.oermann@mountsinai.org

Neurol Ther (2019) 8:351–365

https://doi.org/10.1007/s40120-019-00153-8

incomplete data sources, with the outcomesbeing cumbersome to track. Identifying novelclinical patterns amidst this complexity is defi-nitely not a trivial task [1–3].

Modern representation learning methodsenable the automatic discovery of representa-tions needed to generate insights from raw data[4]. Deep learning algorithms are an example ofsuch representation learning approaches thathierarchically compose nonlinear functions totransform raw input data into more sophisti-cated features that enable the identification ofnovel patterns [5]. Such approaches haveproved to be essential in modern engineeringbreakthroughs—from face recognition and self-driving cars to chat-bots and language transla-tion [6–12]. In medicine, the successful appli-cation of deep learning algorithms to routinetasks has enabled a flood of academic andcommercial research, with publications on var-ious applications growing from 125 publishedpapers identified as machine learning publica-tions in arXiv, the electronic scientific andengineering paper archive, in 2000, to morethan 3600 by November of 2018 (see Fig. 1).

The multidiscipline of clinical neuroscienceshas similarly experienced the beginnings of animpact from deep learning, with movement

towards the development of novel diagnosticand prognostic tools. Deep learning techniquesare particularly promising in the neuroscienceswhere clinical diagnoses often rely on subtlesymptoms and complicated neuroimagingmodalities with granular and high-dimensionalsignals. In this article, we discuss the applicationsof deep learning in neurology and the ongoingchallenges, with an emphasis on aspects relevantto the diagnosis of common neurologic disor-ders. However, our aim is not to provide com-prehensive technical details of deep learning orits broader applications. We begin with a briefoverview of deep learning techniques followedby a review of applications in the clinical neuro-science field. We conclude the review with ashort discussion on existing challenges and alook to the future. This article is based on previ-ously conducted studies and does not containany studies with human participants or animalsperformed by any of the authors.

FUNDAMENTALS OF DEEPLEARNING

Machine learning is a subset of artificial intelli-gence that learns complex relationships among

Fig. 1 Machine learning publications in PubMed by year through 2018 showing the exponential growth of interest in thefield, as reported by the US National Library of Medicine of the National Institutes of Health [13]

352 Neurol Ther (2019) 8:351–365

variables in data [14]. The power of machinelearning comes from its ability to derive pre-dictive models from large amounts of data withminimal or, in some cases, entirely without theneed for prior knowledge of the data or anyassumptions about the data. One of the mostwidely discussed modern machine learningalgorithms, the artificial neural network (ANN),draws inspiration from biological neural net-works that constitute mammalian brains. Thefunctional unit of the ANN is the perceptron,which partitions input data into separable cat-egories or classes [15]. When hierarchicallycomposed into a network, the perceptronbecomes an essential building block for moderndeep neural networks (DNNs), such as multi-layer perceptron classifiers. Similar examples ofcommonly used traditional machine learningalgorithms include linear regression (LR), logis-tic regression, support vector machines (SVMs),and the Naıve Bayes classifier (Fig. 2).

These traditional machine learning methodshave been important in furthering advance-ments in medicine and genomics. As an exam-ple, LR has proven useful in the search forcomplex, multigene signatures that can beindicative of disease onset and prognosis, taskswhich are otherwise too intricate and cumber-some even for researchers with professionaltraining [16]. Although such tools have beenvery effective in parsing massive datasets andidentifying relationships between variables ofinterest, traditional machine learning

techniques often require manual feature engi-neering and suffer from overhead that limitstheir utility in scenarios that require near real-time decision-making.

Deep learning differs from traditionalmachine learning in how representations areautomatically discovered from raw data. Incontrast to ANNs, which are shallow featurelearning techniques, deep learning algorithmsemploy multiple, deep layers of perceptronsthat capture both low- and high-level repre-sentations of data, enabling them to learn richerabstractions of inputs [5]. This obviates the needfor manual engineering of features and allowsdeep learning models to naturally uncover pre-viously unknown patterns and generalize betterto novel data. Variants of these algorithms havebeen employed across numerous domains inengineering and medicine.

Convolutional neural networks (CNNs) havegarnered particular attention within computervision and imaging-based medical research[17, 18]. CNNs gather representations acrossmultiple layers, each of which learns specificfeatures of the image, much like the humanvisual cortex is arranged into hierarchical layers,including the primary visual cortex (edgedetection), secondary visual cortex (shapedetection), and so forth [19]. CNNs consist ofconvolutional layers in which data features arelearned: pooling layers, which reduce thenumber of features, and therefore computa-tional demand, by aggregating similar or

Fig. 2 Breakdown of algorithm types in the machine learning family that are commonly used in medical subdomainresearch and analyses

Neurol Ther (2019) 8:351–365 353

redundant features; dropout layers, whichselectively turn off perceptrons to avoid over-reliance on a single component of the network;and a final output layer, which collates thelearned features into a score or class decision,i.e., whether or not a given radiograph showssigns of ischemia. These algorithms haveachieved rapid profound success in image clas-sification tasks and, in some cases, have mat-ched board-certified human performance[20–24].

Recurrent neural networks and variants, suchas long short-term memory (LSTM) and gatedrecurrent units, have revolutionized the analy-sis of time-series data that can be found invideos, speech, and texts [25]. These algorithmssequentially analyze each element of input dataand employ a gating mechanism to determinewhether to maintain or discard informationfrom prior elements when generating outputs.In this manner, they efficiently capture long-term dependencies and have revolutionizedmachine translation, speech processing, andtext analysis.

Autoencoders (AEs) are a class of unsuper-vised learning algorithms that discover mean-ingful representations of data by learning alower-dimensional mapping from inputs tooutputs [26, 27]. They are composed of anencoder, which learns a latent representation ofthe input, and a decoder, which reconstructsthe input from the latent representation. Byconstraining the latent representation to alower dimensionality than the input, AEs areable to learn a compressed representation ofdata that contains only the features necessary toreconstruct the input. Such algorithms are oftenemployed to learn features that can be subse-quently utilized in conjunction with the deeplearning techniques previously discussed.

Generative adversarial networks are a newerclass of algorithms aimed at generating noveldata that statistically mimic input data byapproximating a latent distribution for the data[28]. Such algorithms are composed of twocompeting (‘‘adversarial’’) networks: a genera-tor, which produces synthetic data from noiseby sampling from an approximated distribu-tion, and a discriminator, which aims to dif-ferentiate between real and synthetic instances

of data. As the two networks engage in thisadversarial process, the fidelity of the generateddata gradually improves. In some contexts, theresulting data have been utilized to augmentexisting datasets [29].

These strides in deep learning are largely dueto breakthroughs in computing capabilities andthe open-source nature of research in the field.The application of graphics processing units todeep learning research has dramatically accel-erated the size and complexity of algorithmarchitectures and simultaneously reduced thetime to train such algorithms from months tothe order of days. The consequence has beenhigh-throughput research characterized byrapid experimentation, ultimately enablingmore efficacious algorithms. In addition, therise of open-source deep learning frameworks,such as TensorFlow, Keras, PyTorch, Caffe, andothers, has increased accessibility to technicaladvances and facilitated the sharing of ideasand their rapid application across variousdomains [30, 31]. The truly collaborative natureof deep learning research has led to surprisinginnovations and changed the landscape ofmedical research and care.

LITERATURE REVIEW

In this article, we review and summarize pub-lished literature on the application of deeplearning to the clinical neurosciences. We usedsearch engines and repositories such as GoogleScholar, PubMed, ScienceDirect, and arXiv toidentify and review existing literature and per-formed keyword searches of these databasesusing the following terms: ‘‘deep learning,’’‘‘machine learning,’’ ‘‘neurology,’’ ‘‘brain,’’ and‘‘MRI.’’ Following a comprehensive review ofthe literature initially retrieved, we identified312 articles as containing one or more keywordsassociated with our queries. Of these articles,134 were subsequently identified as being rele-vant to the subject of this review. Followingcollation of the relevant articles, we groupedarticles first into broad modalities, namelyimage classification, image segmentation,functional connectivity and classification ofbrain disorders, and risk prognostication.

354 Neurol Ther (2019) 8:351–365

Within these areas, we then grouped publica-tions into disease applications. We focused ourdiscussion on the clinical implications of thedevelopments in the field.

DEEP LEARNING IN NEUROLOGY

The deep learning techniques described aboveare playing an increasingly crucial role in neu-rological research, tackling problems withinseveral subdomains. First, radiological imageclassification and segmentation has been a tra-ditional locus of deep learning developmentefforts. Image classification and segmentationtasks are uniquely suited to deep learning due tothe high-dimensional nature of neuroimagingdata which is unfavorable to manual analysis,combined with the naturally digital nature ofmost modern imaging. Secondly, deep learninghas been applied to functional brain mappingand correlational studies using functionalmagnetic resonance imaging (fMRI) data fortasks such as prediction of postoperative sei-zure. Lastly, diagnostic prognostication withdeep learning using multiple data types,including lab values, images, notes, amongothers, has been used to assign disease risk. Inthe following sections, we discuss the successesand challenges inherent in the deep learningapproaches adopted towards these tasks, as wellas the limitations and difficulties that suchmethods face within the field of neurology andwithin medicine as a whole.

Medical Image Classification

The first application of deep learning in medi-cine involved the analysis of imaging modali-ties, especially those for the detection ofAlzheimer’s disease (AD) and other cognitiveimpairments. A variety of publicly availabledatabases, such as the Alzheimer’s DiseaseNeuroimaging Initiative (ADNI) and BrainTumor Segmentation Benchmark (BraTS), havebecome available to spur advancements inneuroimaging analysis [32, 33].

Early approaches used AEs in conjunctionwith a classifier to distinguish AD, mild cogni-tive impairments (MCI) and healthy controls.

Among the first such applications, Suk and Shenutilized a stacked AE to learn multimodal brainrepresentations from structural MRI and posi-tron emission tomography (PET), and incorpo-rated those features with cerebrospinal fluidbiomarker data and clinical scores from theMini-Mental State Examination (MMSE) andAlzheimer’s Disease Assessment Scale-Cognitivesubscale (ADAS-Cog) to train an SVM classifierthat improved diagnostic accuracy [34]. Otherapproaches pre-trained a stacked AE using nat-ural images (everyday images) prior to trainingon brain MR images in order to learn morehigh-fidelity anatomical features, such as graymatter and structural deformities, for incorpo-ration into a CNN [35]. Variations on theseapproaches have been used to incrementallyimprove diagnostic performance [36–42].

Whereas older approaches were limited totwo-dimensional (2D) slices of medical imagesdue to computational constraints, newer appli-cations have been able to incorporate the full3D volume of an imaging modality for ADdetection. Among the first such examples waswork by Payan and Montana in which theytrained a sparse AE on 3D patches of MRI scansto learn a volumetric brain representation thatwas used to pre-train a 3D CNN for AD diagnosis[43]. More recently, Hosseini-Asl et al. used anadaptable training regime with a 3D CNN pre-trained by a convolutional AE to learn general-izable AD biomarkers [44, 45]. This approachwas notable because it allowed the transfer oflearned features from the source CADDementiadataset to the target ADNI dataset, resulting instate-of-the-art AD diagnosis accuracy on anexternal dataset. Analogous work with volu-metric data has been conducted in the com-puted tomography (CT) domain to differentiateAD from brain lesions and the processes ofnormal aging [46].

The most recent work has built on existingwork in AD diagnosis and focused on predictingthe onset of AD in at-risk patients in order tostem progression of the disease. Ding et al. usedfluorine-18-fluorodeoxyglucose PET scans of thebrain derived from the ADNI database to train aCNN to diagnose AD [47]. Unlike many inves-tigators before them, however, the authorsevaluated the efficacy of their algorithm against

Neurol Ther (2019) 8:351–365 355

data from the long-term follow-up of patientswho did not have AD at the time. Interestingly,they found that the algorithm predicted onsetof AD on average 75.8 months prior to the finaldiagnosis on an independent dataset, whichsurpassed the diagnostic performance of threeexpert radiologists.

Deep learning-based image classification hasalso been applied in the diagnosis of acuteneurologic events, such as intracranial hemor-rhage (ICH) and cranial fractures, with the aimof reducing time to diagnosis by optimizingneuroradiology workflows. Titano et al. traineda 3D CNN in a weakly supervised manner on37,236 CT scans to identify ICH for the pur-poses of triaging patient cases [48]. They lever-aged a natural language processing algorithmtrained on 96,303 radiology reports to generatesilver-standard labels for each imaging studyand validated the efficacy of their CNN on asubset of studies with gold standard labels gen-erated by manual chart review [49]. The inves-tigators conducted a double-blind randomizedcontrol trial to compare whether the algorithmor expert radiologists could more effectivelytriage studies in a simulated clinical environ-ment and found that the CNN was 150-foldfaster in evaluating a study and significantlyoutperformed humans in prioritizing the mosturgent cases. Subsequent studies have similarlydemonstrated the potential for deep learning tooptimize radiology workflows in the diagnosisof ICH and detect as many as nine criticalfindings on head CT scans with sensitivitycomparable to that of expert radiologists[50–52].

Medical Image Segmentation

Segmentation of radiological brain images iscritical for the measurement of brain regions,including shape, thickness, and volume, thatare important for the quantification of struc-tural changes within the brain that occur eithernaturally or due to various disease processes[53]. Accurate structural classification is partic-ularly important in patients with gliomas, themost common brain tumor type, with less thana 2-year survival time [54, 55]. Manual

segmentations by expert raters show consider-able variation in images obscured by field arti-facts or where intensity gradients are minimal,and rudimentary algorithms struggle to achieveconsistency in an anatomy that can vary con-siderably from patient to patient [33]. In light ofthese factors, deep learning segmentation ofneuroanatomy has become a prime target forefforts in deep learning research.

Measurement of the performance of neu-roanatomical segmentation algorithms hasbeen standardized by the BraTS, which wasestablished at the 2012 and 2013 Medical ImageComputing and Computer Assisted Interven-tions (MICCAI) conference [33]. Prior to theestablishment of this challenge, segmentationalgorithms were often evaluated on privateimaging collections only, with variations in theimaging modalities incorporated and the met-rics used to evaluate effectiveness. The estab-lishment of BraTS has been critical instandardizing the evaluation of various modelsfor the determination of which to pursue inclinical practice. At the time of BraTS estab-lishment, the models being evaluated were lar-gely simple machine learning models, includingfour random forest-based segmentation models[33]. Since then, there has been considerableadvancement in performance, largely based onthe adoption of CNNs for anatomicalsegmentation.

The traditional computational approach tosegmentation is to employ an atlas-based seg-mentation, namely the FreeSurfer software,which assigns one of 37 labels to each voxel in a3D MRI scan based on probabilistic estimates[56]. In a recent comparative study, Wachingeret al. designed and applied a deep CNN, calledDeepNAT, for the purposes of segmenting neu-roanatomy visualized in T1-weighted MRI scansinto 25 different brain regions. The authorsused the MICCAI Multi-Atlas Labeling chal-lenge, consisting of 30 T1-weighted images, inaddition to manually labeled segmentations[53, 57]. When the authors compared the cur-rent clinical standard, FreeSurfer, which uses itsown anatomical atlas to assign anatomic labels,to DeepNAT, they found that DeepNAT showedstatistically significant performance improve-ments. Performance in segmentation was

356 Neurol Ther (2019) 8:351–365

measured using a Dice volume overlap score,with DeepNAT achieving a Dice score of 0.906,in comparison to FreeSurfer’s 0.817 [53].

In addition to tissue-based segmentationefforts, vascular segmentation has been an areaof deep learning research aimed at quantifyingbrain vessel status. Traditional vessel segmen-tation relies on either manual identification orrule-based algorithms since there is no equiva-lent atlas-based method for brain vessels asthere is for neuroanatomy. In their recent studyon blood vessel segmentation, Livne et al.applied a U-net model to labeled data from 66patients with cerebrovascular disease and thencompared the method to the traditional vascu-lar segmentation method of graph-cuts. TheU-net model outperformed graph-cuts, achiev-ing a Dice score of 0.891 compared to 0.760 forgraph-cuts [58]. Of note, the model, which wastrained on 3T MRI time-of-flight images, failedto generalize well to 7T images [58].

Quantification of changes in white matter asbiomarkers for disease processes has been athird area of deep learning segmentation effortsin neurology. Perivascular spaces (PVSs) aresmall spaces surrounding blood vessels that canbe caused by the stress-induced breakdown ofthe blood–brain barrier by various inflamma-tory processes [59, 60]. While PVSs have beenimplicated in a wide range of disease processes,the quantification of these spaces is difficult dueto their tubular and low-contrast appearanceeven on those clinical MRI machines with thehighest-approved resolution [61]. In one 2018study, Lian et al. used a deep CNN to evaluatePVSs in 20 patients scanned on a 7T MRImachine, comparing these to gold-standardmanual labels. Their deep CNN outperformedunsupervised algorithmic methods, such as aFrangi filter, as well as a U-net deep learningmodel, achieving a positive predictive value(PPV) of 0.83 ± 0.05, compared to a PPV of0.62 ± 0.08 for the Frangi filter and 0.70 ± 0.10for the U-net.

U-net models have also been leveraged inquantifying white matter hyperintensities asbiomarkers for age-related neurologic disorders[62]. White matter changes have been shown tobe involved in various forms of corticaldementia, such as AD, and manifest themselves

as high-intensity regions in T2-fluid-attenuatedinversion recovery (FLAIR) MRI scans [63]. Inaddition to quantifying PVSs, U-nets have beenused in segmentation efforts to identify regionsof abnormally intense white matter signals. In2019, Jeong et al. proposed a sailiency U-net, aU-net combined with simple regional maps,with the aim to lower the computationaldemand of the architecture while maintainingperformance in order to identify areas of signalintensity in T2-FLAIR MRI scans of patients withAD [62, 64]. Their model achieved a Dice coef-ficient score of 0.544 and a sensitivity of 0.459,indicating the utility of such a model to aug-ment clinical image analysis [62]. The effortsdescribed above in neuroanatomical segmenta-tion and anomaly detection highlight the ver-satility of deep learning in analyzing aninherently complex organ system.

Functional Connectivityand Classification of Brain Disorders

Research in diagnostic support using multiplemodalities has been a key area of focus in deeplearning research, particularly in disease spacessuch as AD, autism spectrum disorder (ASD),and attention deficit hyperactivity disorder(ADHD). For all of these diseases, the onset canbe insidious, and diagnosis is reliant on non-specific symptoms, such as distractibility andhyperactivity in the case of ADHD, whichresults in poor sensitivity and specificity forclinical diagnostic testing; in fact, the sensitiv-ity of the American Psychiatric Association’sDiagnostic and Statistical Manual testing forADHD is between 70 and 90% [65]. Further-more, delays in diagnosis inevitably delaytreatment, resulting in the treatment being lesseffective or entirely ineffective [65]. Using fMRIand connectome mapping alongside clinicaland demographic data points, multidisciplinaryteams have sought to improve upon the accu-racy of currently utilized neurological tests.

Within the realm of AD and disordersimplicated in MCIs, deep learning has beenincreasingly adopted as a method to analyzeneural connectivity information. Althoughmuch of the work in connectome mapping has

Neurol Ther (2019) 8:351–365 357

relied on less complex classifiers, recent publi-cations have explored the benefits of deeplearning [66, 67]. When applied to fMRI data,deep learning has several advantages over sim-pler SVMs and Lasso models, and exhibits anexponential gain in accuracy over simplermodels with increasing volumes of training data[5, 68]. Meszlenyi et al. utilized a variant of aconvolutional neural network called a connec-tome convolutional neural network (CCNN) toclassify MCI in a relatively small dataset offunctional connectivity data from 49 patients[67]. Although accuracies were comparablebetween the deep learning and less complexclassifiers (53.4% accuracy for the CCNN com-pared to 54.1% for the SVM), the authors pos-tulate that the accuracy benefits of the CCNNarchitecture are well suited to fMRI tasks asdataset sizes expand [67].

Deep learning classifiers have been appliednumerous times toward the accurate diagnosisof ASD using fMRI data. In one study publishedin 2015, Iidaka et al. selected 312 patients withASD and 328 control patients from the AutismBrain Imaging Data Exchange (ABIDE), togetherwith 90 regions of interest, and used a proba-bilistic neural network to classify individualswith ASD. Their method achieved a classifica-tion accuracy of 90% [69]. Additionally, Chenet al. published a classifier based on a con-structed functional network and additional datafrom the ABIDE dataset in a clustering analysisaimed at grouping discriminative features andfound that many discriminative features clus-tered into the Slow-4 band [70].

In the realm of ADHD, several efforts havebeen made to use publicly available imagingdata and deep learning algorithms for diagnosis.In a study published in 2014, Kuang et al.attempted to classify ADHD using a deep beliefnetwork, comprised of stacked Boltzmann’smachines trained on the public ADHD-200dataset [71]. Using time-series fMRI data, thedeep belief network achieved an accuracy of35.1%. While each of the above classifiers haveachieved results that are either on-par or lessaccurate than clinical diagnoses using fMRIdata, methods are expected to improve dra-matically as the quantity of labeled data con-tinues to grow [71].

Risk Prognostication

In addition to widespread research on deeplearning applications for image classificationand segmentation, researchers have applieddeep learning to a variety of other neurology-specific and general medicine data for the pur-poses of risk prognostication. These efforts havebeen applied to electroencephalogram (EEG)signals and genetic biomarkers in the hope ofpredicting clinically meaningful events. Neu-rologists frequently rely on EEG data for themanagement and diagnosis of neurologicaldysfunction, in particular epilepsy and epilepticevents. Several studies using deep learningmethods have investigated its utility whenapplied to preictal scalp EEGs as a predictivetool for seizures [72–74]. The most successful ofthese efforts included a LSTM network, which isparticularly useful for interpreting time-seriesdata, allowing a model to allocate importanceto previously seen data in a sequence wheninterpreting a given datapoint. These algo-rithms are uniquely suited to large sequences ofdata and have proved their efficacy in predict-ing epileptic events [73].

In their 2018 study, Tsiouris et al. used a two-layer LSTM-based algorithm to predict epilepticseizures using the publicly available CHB-MITscalp EEG database. While previous efforts hadbeen made using CNNs and scalp EEGs to pre-dict epileptic events, the novel use of an LSTMset a new state-of-the-art over traditionalmachine learning algorithms and other deeplearning algorithms. Following feature extrac-tion, the LSTM was provided several meaningfulfeatures, including statistical moments, zerocrossings, Wavelet Transform coefficients,power spectral density, cross-correlation, andgraph theory, to use in the prediction of sei-zures. Notably, the authors compared the pre-dictive ability of the raw EEG data to theextracted features and determined that featureextraction improved model performance [73].This model configuration achieved a minimumof 99.28% sensitivity and 99.28% specificityacross the 15-, 30-, 60-, and 120-min preictalperiods, as well as a maximum false positive rateof 0.11/h. Similar experiments on the CHB-MITscalp EEG database using CNNs, as opposed to

358 Neurol Ther (2019) 8:351–365

LSTMs, achieved worse results, namely poorersensitivity and a higher hourly rate of falsepositives [75, 76].

Genetic data has been another importantarea of research and development for precisionmedicine. Predictive tasks in large-scale geno-mic profiles face high-dimensional datasets thatare often pared down by experts who hand-se-lect a small number of features for trainingpredictive models [77]. In ASD, deep learninghas played a particularly important role indetermining the impact of de-novo mutations,including copy number variants and pointmutations, on ASD severity [78]. Using a deepCNN, Zhou et al. modeled the biochemicalimpact of observed point mutations in 1790whole-genome sequenced families with ASD, onboth the RNA and DNA levels [78]. Thisapproach revealed that both transcriptional andpost-transcriptional mechanisms play a majorrole in ASD, suggesting biological convergenceof genetic dysregulation in ASD.

Genomic data, either alone or in conjunc-tion with neuroimaging and histopathology,has provided cancer researchers a wealth of dataon which to perform cancer-related predictivetasks [77, 79, 80]. Deep learning offers severaladvantages when working simultaneously withmultiple data modalities, removing subjectiveinterpretations of histological images, accu-rately predicting time-to-event outcomes, andeven surpassing gold standard clinical para-digms for glioma patient survival [80]. Usinghigh-powered histological slices and geneticdata, namely IDH mutation status and 1p/19qcodeletion, on 769 patients from The CancerGenome Atlas (TCGA), Mobadersaney et al.used a survival CNN (SCNN) to predict time-to-event outcomes. The histological and geneticmodel performed on par with manual histologicgrading or molecular subtyping [80]. In a sec-ond paper by this group, SCNNs were shown tooutperform other machine learning algorithms,including random forest, in classification tasksusing genetic data from multiple tumor types,including kidney, breast, and pan-glioma can-cers [77]. The ability of deep learning algo-rithms to reduce subjectivity in histologicgrading and disentangle complex relationshipsbetween noisy EEG or genetic data, has the

potential to improve current standards for pre-dicting clinical events.

CHALLENGES

Despite the profound biomedical advances dueto deep learning algorithms, there remain sig-nificant challenges that must be addressedbefore such applications gain widespread use.We discuss some of the most critical hurdles inthe following sections.

Data Volume

Deep neural networks are computationallyintensive, multilayered algorithms with param-eters on the order of millions. Convergence ofsuch algorithms requires data commensuratewith the number of parameters. Although thereare no strict rules governing the amount of datarequired to optimally train DNNs, empiricalstudies suggest that tenfold more training datarelative to the number of parameters is requiredto produce an effective model. It is no surprisethen that domains, such as computer vision andnatural language processing, have seen the mostrapid progress due to deep learning given thewide availability of images, videos, and free-form text on the Internet.

Biomedical data on the other hand is mostlydecentralized—stored locally within hospitalsystems—and subject to privacy constraints thatmake such data less readily accessible forresearch. Furthermore, given the complexity ofpatient presentations and disease processes,reliable ground truth labels for biomedicalapplications are extremely expensive to obtain,often requiring the efforts of multiple highlyspecialized domain experts. This paucity oflabeled data remains an important bottleneck inthe development of deep learning applicationsin medicine.

Data Quality

Healthcare data are fundamentally ill-suited fordeep learning applications. Electronic medicalrecords are highly heterogeneous, being

Neurol Ther (2019) 8:351–365 359

composed of clinical notes, a miscellany ofvarious codes, and other patient details thatmay often be missing or incomplete. Clinicalnotes consist of nuanced language and acro-nyms that often vary by specialty and containredundant information that provides an inac-curate temporal representation of disease onsetor progression. Diagnosis codes suffer from asimilar fate as they track billing for insurancepurposes instead of health outcomes. Thisinherent complexity makes it impossible fordeep learning algorithms to parse signal fromnoise.

Generalizability

Although existing deep learning applicationshave garnered success in silico, their widespreadadoption in real-world clinical settings remainslimited due to concerns over their efficacyacross clinical contexts. Much of the concern isbased on the tendency of deep learning algo-rithms to overfit to the statistical characteristicsof the training data, rendering them hyper-specialized for a hospital or certain patientdemographic and less effective on the popula-tion at-large [81, 82]. The siloed existence ofhealthcare data in hospitals and the hetero-geneity of data across healthcare systems makethe task of developing generalizable modelseven more difficult. And even when multi-in-stitutional data are acquired, the data are oftenretrospective in nature, which prevents practi-cal assessment of algorithm performance.

Interpretability

The power of deep learning algorithms to mapcomplex, nonlinear functions can render themdifficult to interpret. This becomes an impor-tant consideration in healthcare applicationswhere the ability to identify drivers of outcomesbecomes just as important as the ability toaccurately predict the outcome itself. In theclinical setting, where clinical decision supportsystems are designed to augment the decision-making capacity of healthcare professionals,interpretability is critical to convince healthcareprofessionals to rely on the recommendations

made by algorithms and enable their wide-spread adoption. As such, major efforts withinthe deep learning community to tackle prob-lems of interpretability and explainability havethe potential to be particularly beneficial forfacilitating the use of deep learning methods inhealthcare.

Legal

Medical malpractice rules govern standards ofclinical practice in order to ensure the appro-priate care of patients. However, to date, nostandards have been established to assign cul-pability in contexts where algorithms providepoor predictions or substandard treatment rec-ommendations. The establishment of such reg-ulations is a necessary prerequisite for thewidespread adoption of deep learning algo-rithms in clinical contexts.

Ethical

Incidental introduction of bias must be care-fully evaluated in the application of deeplearning in medicine. As discussed previously,deep learning algorithms are uniquely adept atfitting to the characteristics of the data onwhich they are trained. Such algorithms havethe capability to perpetuate inequities againstpopulations underrepresented in medicine and,by extension, in the very healthcare data usedto train the algorithms. Furthermore, recentresearch evaluating algorithmic bias in a com-mercial healthcare algorithm provides a cau-tionary tale on the importance of criticallyevaluating the very outcomes algorithms aretrained to predict [83].

CONCLUSION

Deep learning has the potential to fundamen-tally alter the practice of medicine. The clinicalneurosciences in particular are uniquely situ-ated to benefit given the subtle presentation ofsymptoms typical of neurologic disease. Here,we reviewed the various domains in which deeplearning algorithms have already provided

360 Neurol Ther (2019) 8:351–365

impetus for change—areas such as medicalimage analysis for improved diagnosis of ADand the early detection of acute neurologicevents; medical image segmentation for quan-titative evaluation of neuroanatomy and vas-culature; connectome mapping for thediagnosis of AD, ASD, and ADHD; and miningof microscopic EEG signals and granular geneticsignatures. Amidst these advances, however,important challenges remain a barrier to inte-gration of deep learning tools in the clinicalsetting. While technical challenges surroundingthe generalizability and interpretability ofmodels are active areas of research and progress,more difficult challenges surrounding data pri-vacy, accessibility, and ownership will necessi-tate conversations in the healthcareenvironment and society in general to arrive atsolutions that benefit all relevant stakeholders.The challenge of data quality, in particular, mayprove to be a uniquely suitable target foraddressing using deep learning techniques thathave already demonstrated efficacy in imageanalysis and natural language processing.Overcoming these hurdles will require theefforts of interdisciplinary teams of physicians,computer scientists, engineers, legal experts,and ethicists working in concert. It is only inthis manner that we will truly realize thepotential of deep learning in medicine to aug-ment the capability of physicians and enhancethe delivery of care to patients.

ACKNOWLEDGEMENTS

Funding. No funding or sponsorship wasreceived for this study or publication of thisarticle.

Authorship. All named authors meet theInternational Committee of Medical JournalEditors (ICMJE) criteria for authorship for thisarticle, take responsibility for the integrity ofthe work as a whole, and have given theirapproval for this version to be published.

Disclosures. Aly Al-Amyn Valliani, DanielRanti and Eric Karl Oermann have nothing todisclose.

Compliance with Ethics Guidelines. Thisarticle is based on previously conducted studiesand does not contain any studies with humanparticipants or animals performed by any of theauthors.

Data availability. Data sharing is notapplicable to this article as no datasets weregenerated or analyzed during the current study.

Open Access. This article is distributedunder the terms of the Creative CommonsAttribution-NonCommercial 4.0 InternationalLicense (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommer-cial use, distribution, and reproduction in anymedium, provided you give appropriate creditto the original author(s) and the source, providea link to the Creative Commons license, andindicate if changes were made.

REFERENCES

1. Jensen PB, Jensen LJ, Brunak S. Mining electronichealth records: towards better research applicationsand clinical care. Nat RevGenet. 2012;13(6):395–405.

2. Luo J, Wu M, Gopukumar D, Zhao Y. Big dataapplication in biomedical research and health care:a literature review. Biomed Inform Insights.2016;19(8):1–10.

3. Kohli MD, Summers RM, Geis JR. Medical imagedata and datasets in the era of machine learning-whitepaper from the 2016 C-MIMI meeting datasetsession. J Digit Imaging. 2017;30(4):392–9.

4. Bengio Y, Courville A, Vincent P. Representationlearning: a review and new perspectives. IEEE TransPattern Anal Mach Intell. 2013;35(8):1798–828.

5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nat-ure. 2015;521(7553):436–44.

6. Li H, Lin Z, Shen X, Brandt J, Hua G. A convolu-tional neural network cascade for face detection. In:Proceedings of IEEE conference on computer visionand pattern recognition. Boston, MA. 2015.pp. 5325–34.

Neurol Ther (2019) 8:351–365 361

7. Gilani SZ, Mian A. Learning from millions of 3Dscans for large-scale 3D face recognition. 2017.http://arxiv.org/abs/1711.05942.

8. Ramanishka V, Chen Y-T, Misu T, Saenko K. Towarddriving scene understanding: a dataset for learningdriver behavior and causal reasoning. In: Proceed-ings of IEEE conference on computer vision andpattern recognition. Salt Lake City, UT. 2018.pp. 7699–707.

9. Maqueda AI, Loquercio A, Gallego G, Garcia N,Scaramuzza D. Event-based vision meets deeplearning on steering prediction for self-driving cars.2018. http://arxiv.org/abs/1804.01310.

10. Mazare P-E, Humeau S, Raison M, Bordes A. Train-ing millions of personalized dialogue agents. 2018.http://arxiv.org/abs/1809.01984.

11. Zhang S, Dinan E, Urbanek J, Szlam A, Kiela D,Weston J. Personalizing dialogue agents: I have adog, do you have pets too? 2018. http://arxiv.org/abs/1801.07243.

12. Wu Y, Schuster M, Chen Z, et al. Google’s neuralmachine translation system: bridging the gapbetween human and machine translation. 2016.http://arxiv.org/abs/1609.08144.

13. US National Library of Medicine National Institutesof Health. PubMed. 2019. https://www.ncbi.nlm.nih.gov/pubmed/?term=Machine?Learning.

14. Mitchell TM. The discipline of machine learning,vol. 9. Pittsburgh: School of Computer Science,Carnegie Mellon University; 2006.

15. Rosenblatt F. The perceptron: A probabilistic modelfor information storage and organization in thebrain. Psychol Rev. 1958;65:386–408. http://dx.doi.org/10.1037/h0042519.

16. Ogutu JO, Schulz-Streeck T, Piepho H-P. Genomicselection using regularized linear regression models:ridge regression, lasso, elastic net and their exten-sions. BMC Proc. 2012;6[Suppl 2]:S10.

17. Krizhevsky A, Sutskever I, Hinton GE. ImageNetclassification with deep convolutional neural net-works. In: Pereira F, Burges CJC, Bottou L, Wein-berger KQ, editors. Advances in neural informationprocessing systems, vol. 25. New York: CurranAssociates, Inc.; 2012; 1097–105.

18. Szegedy C, Liu W, Jia Y, et al. Going deeper withconvolutions. 2014.http://arxiv.org/abs/1409.4842.

19. Saba L, Biswas M, Kuppili V, et al. The present andfuture of deep learning in radiology. Eur J Radiol.2019;114:14–24.

20. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D,Narayanaswamy A, et al. Development and valida-tion of a deep learning algorithm for detection ofdiabetic retinopathy in retinal fundus photographs.JAMA. 2016;316(22):2402–10.

21. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neuralnetworks. Nature. 2017;542(7639):115–8.

22. Haenssle HA, Fink C, Schneiderbauer R, et al. Managainst machine: diagnostic performance of a deeplearning convolutional neural network for dermo-scopic melanoma recognition in comparison to 58dermatologists. Ann Oncol. 2018;29(8):1836–42.

23. De Fauw J, Ledsam JR, Romera-Paredes B, et al.Clinically applicable deep learning for diagnosisand referral in retinal disease. Nat Med.2018;24(9):1342–50.

24. Poplin R, Varadarajan AV, Blumer K, et al. Predic-tion of cardiovascular risk factors from retinal fun-dus photographs via deep learning. Nat BiomedEng. 2018;2(3):158–64.

25. Lipton ZC, Berkowitz J, Elkan C. A critical review ofrecurrent neural networks for sequence learning.2015. http://arxiv.org/abs/1506.00019.

26. Rumelhart DE, McClelland JL. Learning internalrepresentations by error propagation. In: Paralleldistributed processing: explorations in themicrostructure of cognition: foundations. Wacht-endonk: MITP Verlags-GmbH & Co. KG; 1987.pp. 318–62.

27. Hinton GE, Salakhutdinov RR. Reducing thedimensionality of data with neural networks. Sci-ence. 2006;313(5786):504–7.

28. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al.Generative adversarial networks. 2014. http://arxiv.org/abs/1406.2661.

29. Shin H-C, Tenenholtz NA, Rogers JK, et al. Medicalimage synthesis for data augmentation andanonymization using generative adversarial net-works. In:Proc Third International Workshop,SASHIMI 2018, held in conjunction with MICCAI2018, Granada, Spain, September 16, 2018. In:Gooya A, Goksel O, Oguz I, Burgos N, editors.Simulation and synthesis in medical imaging.Cham: Springer International Publishing;2018:1–11.

30. Shi S, Wang Q, Xu P, Chu X. Benchmarking state-of-the-art deep learning software tools. 2016.http://arxiv.org/abs/1608.07249.

31. Liu J, Dutta J, Li N, Kurup U, Shah M. Usabilitystudy of distributed deep learning frameworks for

362 Neurol Ther (2019) 8:351–365

convolutional neural networks. 2018. https://www.kdd.org/kdd2018/files/deep-learning-day/DLDay18_paper_29.pdf.

32. Petersen RC, Aisen PS, Beckett LA, et al. Alzheimer’sDisease Neuroimaging Initiative (ADNI): clinicalcharacterization. Neurology. 2010;74(3):201–9.

33. Menze BH, Jakab A, Bauer S, et al. The MultimodalBrain Tumor Image Segmentation Benchmark(BRATS). IEEE Trans Med Imaging.2015;34(10):1993–2024.

34. Suk H-I, Shen D. Deep learning-based feature rep-resentation for AD/MCI classification. Med ImageComput Comput Assist Interv. 2013;16(Pt2):583–90.

35. Gupta A, Ayhan M, Maida A. Natural image bases torepresent neuroimaging data. In: Proceedings of30th international conference on machine learn-ing. vol. 28. Atlanta, GA. 2013. pp. 987–94.

36. Li F, Tran L, Thung K-H, Ji S, Shen D, Li J. Robustdeep learning for improved classification of AD/MCI Patients. Machine learning in medical imag-ing. New York: Springer International Publishing;2014:240–7.

37. Liu S, Liu S, Cai W, Pujol S, Kikinis R, Feng D. Earlydiagnosis of Alzheimer’s disease with deep learning.In: 2014 IEEE 11th international symposium onbiomedical imaging (ISBI). Beijing, China. 2014.pp. 1015–8. http://ieeexplore.ieee.org.

38. Liu S, Liu S, Cai W, et al. Multimodal neuroimagingfeature learning for multiclass diagnosis of Alzhei-mer’s disease. IEEE Trans Biomed Eng.2015;62(4):1132–40.

39. Suk H-I, Lee S-W, Shen D. Alzheimer’s diseaseneuroimaging initiative. Latent feature representa-tion with stacked auto-encoder for AD/MCI diag-nosis. Brain Struct Funct. 2015;220(2):841–59.

40. Sarraf S, Tofighi G. Classification of Alzheimer’sdisease using fMRI data and deep learning convo-lutional neural networks. 2016. http://arxiv.org/abs/1603.08631.

41. Suk H-I, Lee S-W, Shen D. Alzheimer’s diseaseneuroimaging initiative. Deep sparse multi-tasklearning for feature selection in Alzheimer’s diseasediagnosis. Brain Struct Funct. 2016;221(5):2569–87.

42. Valliani A, Soni A. Deep residual nets for improvedAlzheimer’s diagnosis. In: BCB. Boston, MA. 2017.p. 615.

43. Payan A, Montana G. Predicting Alzheimer’s dis-ease: a neuroimaging study with 3D convolutional

neural networks. 2015. http://arxiv.org/abs/1502.02506.

44. Hosseini-Asl E, Gimel’farb G, El-Baz A. Alzheimer’sdisease diagnostics by a deeply supervised adapt-able 3D convolutional network. 2016. http://arxiv.org/abs/1607.00556.

45. Hosseini-Asl E, Ghazal M, Mahmoud A, et al. Alz-heimer’s disease diagnostics by a 3D deeply super-vised adaptable convolutional network. FrontBiosci. 2018;1(23):584–96.

46. Gao XW, Hui R. A deep learning based approach toclassification of CT brain images. In: 2016 SAIcomputing conference (SAI). London, UK. 2016.pp. 28–31. http://ieeexplore.ieee.org.

47. Ding Y, Sohn JH, Kawczynski MG, et al. A deeplearning model to predict a diagnosis of Alzheimerdisease by using 18F-FDG PET of the brain. Radiol-ogy. 2019;290(2):456–64.

48. Titano JJ, Badgeley M, Schefflein J, et al. Automateddeep-neural-network surveillance of cranial imagesfor acute neurologic events. Nat Med.2018;24(9):1337–41.

49. Zech J, Pain M, Titano J, et al. Natural language-based machine learning models for the annotationof clinical radiology reports. Radiology.2018;30:171093.

50. Arbabshirani MR, Fornwalt BK, MongelluzzoGJ,et al. Advanced machine learning in action:identification of intracranial hemorrhage on com-puted tomography scans of the head with clinicalworkflow integration. NPJ Digit Med. 2018;1(1):9.

51. Chilamkurthy S, Ghosh R, Tanamala S, et al. Deeplearning algorithms for detection of critical findingsin head CT scans: a retrospective study. Lancet.2018;392(10162):2388–96.

52. Lee H, Yune S, Mansouri M, et al. An explainabledeep-learning algorithm for the detection of acuteintracranial haemorrhage from small datasets. NatBiomed Eng. 2018;5:6. https://doi.org/10.1038/s41551-018-0324-9.

53. Wachinger C, Reuter M, Klein T. DeepNAT: deepconvolutional neural network for segmenting neu-roanatomy. Neuroimage. 2018;15(170):434–45.

54. Ohgaki H, Kleihues P. Population-based studies onincidence, survival rates, and genetic alterations inastrocytic and oligodendroglial gliomas. J Neu-ropathol Exp Neurol. 2005;64(6):479–89.

55. Holland EC. Progenitor cells and glioma formation.Curr Opin Neurol. 2001;14(6):683–8.

Neurol Ther (2019) 8:351–365 363

56. Fischl B, Salat DH, Busa E, Albert M, Dieterich M,Haselgrove C, et al. Whole brain segmentation:automated labeling of neuroanatomical structuresin the human brain. Neuron. 2002;33(3):341–55.

57. Landman B, Warfield S. MICCAI 2012 workshop onmulti-atlas labeling. In: Medical image computingand computer assisted intervention conference.Nice, France. October 1–5, 2012.

58. Livne M, Rieger J, Aydin OU, et al. A U-Net deeplearning framework for high performance vesselsegmentation in patients with cerebrovascular dis-ease. Front Neurosci. 2019;28(13):97.

59. Loftis JM, Huckans M, Morasco BJ. Neuroimmunemechanisms of cytokine-induced depression: cur-rent theories and novel treatment strategies. Neu-robiol Dis. 2010;37(3):519–33.

60. Menard C, Pfau ML, Hodes GE, et al. Social stressinduces neurovascular pathology promotingdepression. Nat Neurosci. 2017;20(12):1752–60.

61. Lian C, Zhang J, Liu M, et al. Multi-channel multi-scale fully convolutional network for 3D perivas-cular spaces segmentation in 7T MR images. MedImage Anal. 2018;46:106–17.

62. Jeong Y, Rachmadi MF, Valdes-Hernandez MDC,Komura T. Dilated saliency U-Net for white matterhyperintensities segmentation using irregularityage map. Front Aging Neurosci. 2019;27(11):150.

63. Gootjes L, Teipel SJ, Zebuhr Y, et al. Regional dis-tribution of white matter hyperintensities in vas-cular dementia, Alzheimer’s disease and healthyaging. Dement Geriatr Cogn Disord.2004;18(2):180–8.

64. Karargyros A, Syeda-Mahmood T. Saliency U-Net: Aregional saliency map-driven hybrid deep learningnetwork for anomaly segmentation. In: Medicalimaging 2018: computer-aided diagnosis. Interna-tional Society for Optics and Photonics. Houston,TX. 2018. 105751T.

65. Kuang D, He L. Classification on ADHD with deeplearning. In: 2014 international conference oncloud computing and big data. Wuhan, China.2014. pp. 27–32. http://ieeexplore.ieee.org.

66. Suk H-I, Wee C-Y, Lee S-W, Shen D. State-spacemodel with deep learning for functional dynamicsestimation in resting-state fMRI. Neuroimage.2016;1(129):292–307.

67. Meszlenyi RJ, Buza K, Vidnyanszky Z. Resting statefMRI functional connectivity-based classificationusing a convolutional neural network architecture.Front Neuroinform. 2017;17(11):61.

68. Montufar GF, Pascanu R, Cho K, Bengio Y. On thenumber of linear regions of deep neural networks.In: Ghahramani Z, Welling M, Cortes C, LawrenceND, Weinberger KQ, editors. Advances in neuralinformation processing systems, vol. 27. Red Hook:Curran Associates, Inc.; 2014:2924–32.

69. Iidaka T. Resting state functional magnetic reso-nance imaging and neural network classified autismand control. Cortex. 2015;63:55–67.

70. Chen H, Duan X, Liu F, et al. Multivariate classifi-cation of autism spectrum disorder using fre-quency-specific resting-state functionalconnectivity—a multi-center study. Prog Neu-ropsychopharmacol Biol Psychiatry.2016;4(64):1–9.

71. Kuang D, Guo X, An X, Zhao Y, He L. Discrimina-tion of ADHD based on fMRI data with deep beliefnetwork. Intelligent computing in bioinformatics.New York: Springer International Publishing;2014:225–32.

72. Tjepkema-Cloostermans MC, de Carvalho RCV, vanPutten MJAM. Deep learning for detection of focalepileptiform discharges from scalp EEG recordings.Clin Neurophysiol. 2018;129(10):2191–6.

73. TsiourisJL, Pezoulas VC, Zervakis M, Konitsiotis S,Koutsouris DD, Fotiadis DI. A long short-termmemory deep learning network for the predictionof epileptic seizures using EEG signals. Comput BiolMed. 2018;1(99):24–37.

74. Acharya UR, Oh SL, Hagiwara Y, Tan JH, Adeli H.Deep convolutional neural network for the auto-mated detection and diagnosis of seizure using EEGsignals. Comput Biol Med. 2018;1(100):270–8.

75. Truong ND, Nguyen AD, Kuhlmann L, Bonyadi MR,Yang J, Kavehei O. A generalised seizure predictionwith convolutional neural networks for intracranialand scalp electroencephalogram data analysis.2017. http://arxiv.org/abs/1707.01976.

76. Khan H, Marcuse L, Fields M, Swann K, Yener B.Focal onset seizure prediction using convolutionalnetworks. IEEE Trans Biomed Eng.2018;65(9):2109–18.

77. Yousefi S, Amrollahi F, Amgad M, et al. Predictingclinical outcomes from large scale cancer genomicprofiles with deep survival models. Sci Rep.2017;7(1):11707.

78. Zhou J, Park CY, Theesfeld CL, et al. Whole-genomedeep-learning analysis identifies contribution ofnoncoding mutations to autism risk. Nat Genet.2019;51(6):973–80.

364 Neurol Ther (2019) 8:351–365

79. Buda M, Saha A, Mazurowski MA. Association ofgenomic subtypes of lower-grade gliomas withshape features automatically extracted by a deeplearning algorithm. Comput Biol Med.2019;109:218–25.

80. Mobadersany P, Yousefi S, Amgad M, et al. Pre-dicting cancer outcomes from histology and geno-mics using convolutional networks. Proc Natl AcadSci USA. 2018;115(13):E2970–9.

81. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ,Oermann EK. Variable generalization performanceof a deep learning model to detect pneumonia in

chest radiographs: a cross-sectional study. PLoSMed. 2018;15(11):e1002683.

82. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ,Oermann EK. Confounding variables can degradegeneralization performance of radiological deeplearning models. 2018. http://arxiv.org/abs/1807.00431.

83. Obermeyer Z, Mullainathan S. Dissecting racial biasin an algorithm that guides health decisions for 70million people. In: Proceedings of conference onfairness, accountability, and transparency. NewYork: ACM; 2019. p. 89.

Neurol Ther (2019) 8:351–365 365