computational medicine, present and the future: obstetrics

Expert Reviews ajog.org

Computational medicine, present and the future:obstetrics and gynecology perspective
Radek Bukowski, MD, PhD; Karl Schulz, PhD; Kelly Gaither, PhD; Keri K. Stephens, PhD; Dave Semeraro, PhD;Justin Drake, PhD; Gordon Smith, MD, PhD; Craig Cordola, FACHE; Thaleia Zariphopoulou, PhD;Thomas J. R. Hughes, PhD; Christopher Zarins, MD; Dimitri Kusnezov, PhD; Rep. Donna Howard, MA;Tinsley Oden, PhD
From the Department of Women’s Health, Dell Medical School, The University of Texas at Austin,Austin, TX (Drs Bukowski, Schulz, Gaither, Semeraro, and Drake); Oden Institute for ComputationalEngineering and Sciences, Austin, TX (Drs Schulz, Hughes, and Oden); Texas Advanced ComputingCenter, Austin, TX (Drs Gaither, Semeraro, and Drake); Moody College of Communication, TheUniversity of Texas at Austin, Austin, TX (Dr Stephens); Department of Obstetrics & Gynaecology,University of Cambridge, Cambridge, United Kingdom (Dr Smith); Ascension, St. Louis, MO (MrCordola); Department of Mathematics and IROM, McCombs School of Business, The University ofTexas at Austin, Austin, TX (Dr Zariphopoulou); HeartFlow, Inc, RedwoodCity, CA (Dr Zarins); DeputyUnder Secretary Artificial Intelligence & Technology Office, US Department of Energy, Washington,DC (Dr Kusnezov); and Texas House of Representatives, Representative for the 48th District, Austin,TX (Ms Howard).

Received April 21, 2020; revised Aug. 5, 2020; accepted Aug. 20, 2020.

The authors report no conflict of interest.

Corresponding author: Radek Bukowski MD, PhD. [email protected]

0002-9378/$36.00 � ª 2020 Elsevier Inc. All rights reserved. � https://doi.org/10.1016/j.ajog.2020.08.057

Medicine is, in its essence, decision making under uncertainty; the decisions aremade abouttests to be performed and treatments to be administered. Traditionally, the uncertainty indecision making was handled using expertise collected by individual providers and, morerecently, systematic appraisal of research in the form of evidence-based medicine. Thetraditional approach has been used successfully in medicine for a very long time. However, ithas substantial limitations because of the complexity of the system of the human body andhealthcare. The complex systems are a network of highly coupled components intenselyinteracting with each other. These interactions give those systems redundancy and thusrobustness to failure and, at the same time, equifinality, that is, many different causativepathways leading to the same outcome. The equifinality of the complex systems of thehuman body and healthcare system demand the individualization of medical care, medicine,and medical decision making. Computational models excel in modeling complex systemsand, consequently, enabling individualization of medical decision making and medicine.Computational models are theory- or knowledge-based models, data-driven models, ormodels that combine both approaches. Data are essential, although to a different degree, forcomputational models to successfully represent complex systems. The individualizeddecision making, made possible by the computational modeling of complex systems,has the potential to revolutionize the entire spectrum of medicine from individualpatient care to policymaking. This approach allows applying tests and treatments toindividuals who receive a net benefit from them, for whom benefits outweigh the risk,rather than treating all individuals in a population because, on average, the populationbenefits. Thus, the computational modelingeenabled individualization of medicaldecision making has the potential to both improve health outcomes and decrease thecosts of healthcare.

Key words: computation, data, data-driven models, machine learning, modeling,physics-based models, theory-based models, uncertainty

16 American Journal of Obstetrics & Gynecology JANUARY 2021

Downloaded for Anonymous User (n/a) at University of Texas at Austin School of Nursing from ClinicalFor personal use only. No other uses without permission. Copyright ©2021. Elsevier In

Introduction“Medicine is a science of uncertainty andan art of probability.”1 This is how SirWilliam Osler described medicine over acentury ago, a statement that resonateswith us as strongly today. The consider-able and ever-present multifaceted un-certainty, in comparison with otherdisciplines, is what makes the practice ofmedicine so challenging. This is mainlybecause in medicine we face a very com-plex system and we lack fundamentalnatural laws describing its function toinform our models. The uncertainty andcomplexity are especially pronounced inthe practice of obstetrics, where 2 ormoreyoung and usually otherwise healthy pa-tients with long life expectancies and withfrequently conflicting health interests arebeing cared for and where the stakes andexpectations are very high.2,3 Thus,medicine is, in its essence, decisionmaking under uncertainty, decisionsabout the tests to be performed andtreatments to be administered. The ines-capable and pervasive uncertainty is theconsequence of 2 issues facing everymedical decision, although to a differentdegree: the incomplete information tomake the decision and the element ofchance, randomness, or luck. As aconsequence, there are neither perfecttests nor treatments. Although not intu-itive, the false-positive rate of a test—theprobability that the positive test is false—varies widely among patients dependingon the patient’s prior (before the test)probability of having a disease. One of themost accurate tests available is the humanimmunodeficiency virus (HIV) antibodytest. The false-positive rate of theHIV testvaries from 3.2% in populations with ahigh risk for HIV to 99.5% in low-riskpopulations.4 Thus, if a patient is at lowrisk for HIV, a positive HIV test is falselypositive in 99.5% of cases. However,

Key.com by Elsevier on March 01, 2021.c. All rights reserved.

http://crossmark.crossref.org/dialog/?doi=10.1016/j.ajog.2020.08.057&domain=pdf

mailto:[email protected]

https://doi.org/10.1016/j.ajog.2020.08.057

http://www.AJOG.org

ajog.org Expert Reviews

when the risk level or prior probability ofthe disease is not known before testing,the uncertainty of testing vastly affects thedecisionmaking for an individual patient.Similarly, among the top 10 grossingmedications in the US, only 1 in 4 to 1 in25 patients receiving them benefit fromthe treatment.5 Thus, some patientsbenefit but many do not, and it is un-certain who will respond and who willnot. Therefore, in choosing a test ortreatment, uncertainty plays a consider-able role and substantially affects the de-cision making.

Patients, physicians, and policymakershave been shown to have difficulty withinterpretation of the meaning ofnumbers, especially probabilities. Thisdifficulty is common and has seriousconsequences for healthcare.6 A study of160 obstetricians and gynecologists hasshown that 80% of them incorrectlyinterpreted the risk of breast cancerassociated with a positive mammogram.Most of them grossly overestimated theprobability of cancer, overestimating therisk by almost an order of magnitude,80% to 90% instead of 10% in the givenclinical scenario.7 US obstetricians andgynecologists also had difficulty ininterpretation uncertainty and proba-bilities related to ovarian cancerscreening.8 The difficulty in interpretinguncertainty and the related probabilitiesby obstetricians, gynecologists, andother specialty physicians and patientshas been shown regarding ovarian can-cer screening and other areas.8,9

Because of the uncertainty, the bestdecision is not the one which results in agood outcome, but one which carefullyconsiders all potential future outcomes,their probabilities, and consequences, aswell as relates to the decision maker’sindividual preferences.10 Two approachesemerge to handle uncertainty in medi-cine. The traditional approach has beenused in medicine from its origins. Yet,recent developments of computationalmethods and computer capabilities haveplaced us at a point in time when theseadvances can revolutionize medical deci-sion making and medicine in general.

The traditional approach to handlinguncertainty has been successfully used inmedicine for a very long time. For

Downloaded for Anonymous User (n/a) at UFor personal use only. No

centuries, it was based on expertise,experience collected in the process ofpracticing medicine. Expertise is un-doubtedly very effective and savescountless lives. It is especially useful,however, when used with the awarenessof its limitations. These limitations ofexpertise are due to the heuristics we usein intuitive decision making. They werefirst described by Tversky and Kahne-man11 over 40 years ago, and the latterreceived for it the Nobel Memorial Prizein Economic Sciences. Heuristics areused by every human being and probablyconstitute an evolutionary advantage.They allow quick decision making andare usually very effective. However, theymay also lead us to make predictablemistakes, resulting in a bias in our de-cision making.One of these biases is our tendency to

overestimate the probability of events wecan more easily retrieve from our mem-ory, events that are common in ourexperience, and events that are emotion-ally charged. That is why we all have abiased tendency to overestimate theprobability of good outcomes, especiallyin obstetrics, or specific rare adverseoutcomes that have happened to one ofour patients.2,3 The use of expertise in thehandling of uncertainty in medical prac-tice is also hindered by the limitations ofindividual experience. An obstetrician inthe US delivers, on average, about 140patients a year.12 This means, on average,an obstetrician will experience 1 cerebralpalsy associated with intrapartum hyp-oxia in 174 years,13,14 1 permanentbrachial plexus palsy in 40 years,15 1 ce-rebral palsy because of uterine rupture in694 years,16,17 and 1 maternal death in 38years.18,19 Thus, it is very difficult tocollect sufficient experience and use iteffectively in estimating uncertainty inmedical decision making. The limitedpossible individual experiences, togetherwith cognitive biases, limit the effective-ness of expertise in handling uncertaintyin decision making. These limitations ofexpertise are well illustrated by wordsspoken in an area outside of medicine butvery well suited to it, by Captain EdwardSmith in 1907, “I never saw a wreck andnever have been wrecked, nor was I everin any predicament that threatened to

JANUARY 2021 A

niversity of Texas at Austin School of Nursing from Clinical other uses without permission. Copyright ©2021. Elsevier In

end in disaster.”20 Five years later, onApril14, 1912, he was the captain of the RoyalMail Ship Titanic.

Mainly in response to the limitationsof expertise, as well as the quality and useof evidence, evidence-based medicinewas introduced in the 1990s. It wasdeveloped with the goal to guide clinicalpractice by using the results of high-quality evidence, especially in random-ized controlled trials (RCTs). Thisapproach was expected to minimize theuncertainty in medical decision makingand consequently improve outcomesthrough consolidation of a high qualityof evidence in systematic reviews andclinical guidelines. Much progress inmedicine has been achieved in the last 30years thanks to the evidence-basedmedicine paradigm.21 However, its lim-itations and misapplications have alsobecome evident.22 Over 2 decades later, areview of the state of evidence-basedmedicine shows that many evidence-based guidelines are not based onRCTs, perhaps as few as 11% in someareas of medicine.23 A review ofapproximately 3000 treatments classified50% of them as having insufficientsupporting evidence. Among the otherhalf, judged to be evidence-based, 24%of treatments were considered likely tobe beneficial, 7% required tradeoffs be-tween benefits and harms, 5% were un-likely to be beneficial, 3% were likely tobe ineffective or harmful, and only 11%were clearly beneficial.23

Perhaps even more importantly, whenthe clinical guidelines make recommen-dations based on RCTs, the recommen-dations frequently differ between theguidelines even when the same RCTsare cited as evidence for the differentrecommendations.24,25 This is a conse-quence of the focus of the guidelines’appraisal tools on the internal validity ofthe referenced RCTs. These appraisaltools focus on the methodology andquality of reporting of the cited studiesrather than on their external validity,generalizability, and clinical relevanceand applicability besides their internalvalidity.26 Therefore, the RCTs may notbe applicable to the populations, in-terventions, and outcomes specified inthe recommendations made in the

merican Journal of Obstetrics & Gynecology 17Key.com by Elsevier on March 01, 2021.c. All rights reserved.

http://www.AJOG.org


guidelines. Analysis of national clinicalguidelines issued by professional orga-nizations from the US, Canada, andEurope, showed that, of the 338 treat-ment recommendations made in the 9guidelines, a third had not been based onevidence from RCTs, considered to bethe highest quality level of evidence.Another third of recommendations didcite RCTs in support but were found toprovide evidence of low quality. The lowquality of the evidence in those recom-mendations was due to lack of applica-bility of the RCTs to the populationtargeted by the recommendations orbecause the cited trials reported surro-gate outcomes rather than the outcomesaddressed by the recommendations.26

RCTs are traditionally considered tobe the strongest form of evidence forclinical decision making. However, 20%of all published medical research wasshown to have methodological flaws,with RCTs having as many limitations asother studies.27 Some estimates of thenonreproducibility of RCTs are evenhigher. Furthermore, even the highestquality medical evidence is itself uncer-tain. An analysis of 49 studies, each citedover 1000 times and published in theleading journals, showed that a third oftheir findings could not be reproducedby subsequent studies of similar or largersize or the effect sizes of the subsequentstudies were substantially smaller.28

RCTs are widely considered the goldstandard of evidence, and their integra-tion, the metaanalyses, are thought bymany to be the platinum standard ofevidence-based medicine.29 They areconsidered to be the highest level of ev-idence and thus the best way to handleuncertainty in medicine. However,despite many tremendous contributionsmade to the practice of medicine, RCTsand their metaanalyses have a largenumber of potentially serious limita-tions, making them less than optimal tobe the sole source of evidence in man-aging uncertainty in medicine. A. L.Cochrane, the pioneer of the use of RCTsin medicine, warned in 1971, “Betweenmeasurements based on RCTs andbenefit . . . in the community there is agulf which has been much under-estimated.”30

18 American Journal of Obstetrics & Gynecology


Today it is evident that treatments usedacross the medical specialties are consis-tently less effective in clinical practicethan they are reported in RCTs.30e36 Thelow effectiveness of medications in clin-ical practice appears to result from poorexternal validity or generalizability ofclinical trials. Research into the internalvalidity of RCTs dwarfs the evaluation oftheir generalizability and their use inclinical practice. The RCTs’ appraisaltools evaluating their performance, andthus the quality of their evidence, usuallydo not consider generalizability of RCTs,the quality critical to managing uncer-tainty in clinical practice. In RCTs, only asubset of a population in which anintervention or medication is appliedbenefits from it.34,37,38 Frequently, thebenefiting subset is a minority of patientsreceiving a medication.34 A still smallersubset of patients may receive a netbenefit from medication, the benefitminus harm related to medication.39,40

The low generalizability or external val-idity of clinical trials and, consequently,low effectiveness of medications in clin-ical practice is due in large part to theheterogeneity of treatment effect (HTE),which has been observed across thespectrum of interventions and domainsof medicine.30,36,41,42 HTE has 4 sources:heterogeneity of baseline disease risk;heterogeneity of treatment effect; het-erogeneity of treatment-related harm;and heterogeneity of competing risks,risks related to conditions other than theone studied.33,34,36,43,44

The limited generalizability of RCTs isalso because of a very narrow selection ofstudy populations in RCTs and theextensive inclusion and exclusioncriteria applied.45

Thus, RCTs and the guidelines cannotalways be assumed to provide high-quality evidence for the recommenda-tions they make. RCTs and their meta-analyses answer the question “does itwork?” rather than the question morecritical to medical decision making, “inwhom might it work?” Evidence-basedmedicine in general and RCTs specif-ically are substantially limited inhandling uncertainty, which is critical inmedical decision making and the prac-tice of medicine.

JANUARY 2021


Complex systemsThe traditional approaches to handlinguncertainty, based on expertise orevidence-based, are shown to have severelimitations. One could argue that theselimitations underlie a large part of thehealthcare crisis by enabling abuses ofthe fee-for-service system and defensivemedicine. The difficulty in managing theuncertainty inherent in medicine comesfrom the traditional understanding ofthe human body and healthcare as linearcause and effect systems. However, thehuman body and healthcare are complexsystems, systems composed of a vastnumber of relatively simple compo-nents, which intensely interact with eachother, and lead to the emergence ofunique system behaviors. The complexsystems are networks of tightly coupledcomponents (eg, genes, proteins, cells)interacting in a nonlinear manner.46e49

The unique behaviors of the systememerge from the interactions of thesimple components. The number ofpossible interactions grows exponen-tially with the number of elements in thesystem. To put this in a proper perspec-tive, a moderate number of 25 elements,for example, risk factors and protectivecharacteristics, could have up to 225, or33,554,432, interactions among them. Ifone considers that the human body ismade of an order of 1013 cells, and eachcell contains 42�106 protein molecules,the number of potential interactions isgenuinely staggering. Interactions wherealmost everything affects everything elseand does so in a nonlinear manner resultin redundancies that make the complexsystems robust to failures, such as diseasein an individual patient or inability toprovide healthcare to patients in need ina healthcare system.50 However, theseredundancies also result inmultiple waysthat failure can occur rather than a singleroot cause leading to failure. Because ofthat structure, complex systems havemany different causative pathways andthus many different ways an outcomecan occur, the phenomenon known asequifinality.51 Equifinality demandsindividualization of care, the individu-alization of decision making and theindividualization of testing and treat-ments (Figure 1). For example, in a


http://www.AJOG.org

FIGURE 1Representation of the complex system, such as the human body orhealthcare system

The complex system is made of a vast number of components highly interacting with each other.

Complex systems have, because of their structure, many different ways or causative pathways an

outcome can occur, the phenomenon known as equifinality. Equifinality demands individualization in

approach to the complex systems, such as individualization of medical decision making.

Bukowski. Computational Medicine. Am J Obstet Gynecol 2021.


predictive model using a combination of13 continuous and categorical pre-dictors, 80% of the 2M pregnant womenhave a unique combination of thosepredictors, risk factors and protectivecharacteristics.

Therefore, any contemplated preven-tive or therapeutic intervention, test, orhealth policy considered must be basedon a balance between individual associ-ated risks and benefits and an individualdecision maker’s risk preference. Thehuman mind is ill-equipped to under-stand andmanage the level of complexityneeded for optimal medical decisionmaking. Despite that, these complexdecisions are made daily by a broadspectrum of decision makers rangingfrom an individual patient to a healthpolicymaker.

Computational models are uniquelysuited to handle the intensity of in-teractions and the uncertainty andcomplexity associated with them. Theyexcel in modeling complex systems. Thewordmodel in this context is a simplifiedapproximate representation of the com-plex system, which allows analysis of thecomplex system behavior. The compu-tational part is the mathematical andquantitative representation of the com-plex system, which prevents ambiguityof the model specification and repre-sentation in a form that is executableusing a computer. Computationalmodeling of complex systems can beachieved by using data-driven and the-ory- or knowledge-driven models. Invery general terms, the data-drivenmodels require a limited amount ofknowledge about the system but a largeamount of data. The theory- orknowledge-driven models require alimited amount of data but substantialknowledge of the system. The compu-tational models predict the behavior ofthe system (eg, patients) in the future,and this ability is expressed as externalvalidation of the model. The computa-tional model, which predicts well, alsoprovides a good understanding of thecomplex system it represents. Predictionis an essential component of medicaldecisionmaking. Because of the inherentuncertainty in medical decision makingand complex systems in general, the


computational models are stochastic.That means that the models limit thenumber of possible outcomes by elimi-nating some possibilities and giving ahigher probability to other outcomesthat are among a pool of potential out-comes known before modeling and onesthat become a possibility with themodel.Familiar examples of computationalmodels are weather forecasts or hurri-cane predictions.The computational models excel in

modeling complex systems and at solv-ing these kinds of complex problems.They are ideally suited to address thecritical and complex elements of indi-vidualized medical decision making.Computational methods are unrivaledand can play a central role in individualpredictions and weighing individual,frequently conflicting, risks and benefitswith their preferences. Computationalapproaches can account for these typesof complexities, aiding optimal medicaldecision making for an individual pa-tient and simulating a suite of decisionsfor policymakers.Computational medicine can thus be

defined as the discipline that usesadvanced mathematical approaches to

JANUARY 2021 A


model complex systems along a spec-trum from the human body to thehealthcare system. To accurately repre-sent these complex systems, the modelsneed to capture the individuality ofhealth and disease for accurate decisionmaking at all levels. Ranging from thepatient to the policy, these require sub-stantial computational capabilities tomake accurate decisions. The modelscan be theory- or knowledge-driven,data-driven, or a combination of these.

To define the future direction ofcomputational medicine, the Computa-tional Health Conference, held in Austin,Texas in 2018, brought together keystakeholders in healthcare and experts incomputation and health from academia,government, industry, philanthropy, andcommunities with the goal of identifyingfuture directions and opportunities forcomputational health and medicine(Figure 2). In addition to establishing aconsensus that advances in computationare creating a new paradigm for medi-cine and healthcare, a primary outcomeof the conference was a recommendationto create a multidisciplinary center orthink tank to research and developcomputational solutions to problems of


http://www.AJOG.org

FIGURE 2The Computational Health Conference, 2018

The conference held in Austin, TX in October 2018 brought together key stakeholders in healthcare and

experts in computation and health from academia, government, industry, philanthropy, and communities

with the goal of identifying future directions and opportunities for computational health and medicine.



interest for healthcare stakeholders fromacademia, government, industry, phi-lanthropy, and communities. Such acollaborative partnership would mostefficiently use computational methods toexploit data at a deeper level and assurethe use of computational solutions for arange of the stakeholders’ interests, fromclinical practice to policymaking.

Computational models allow us toexplore a very large number of predictorsand many more interactions amongthem, inherent to every complex system,to extract a deeper level of informationcontained in the data. The deeper, morerefined insights consequently allow bet-ter modeling of complex systems bycapturing their individual facets andthus allowing individualization of med-icine through individualized decisionmaking.

Data are essential, although to adifferent degree, for computationalmodels to successfully model complexsystems. However, having more data onan individual level, higher granularitydata, does not necessarily equate to moreinformation. Indeed, there is no perfect



data source, neither for traditional norfor computational solutions. Each datasource in medicine has its own advan-tages and limitations. However, compu-tational methods seem to extract moreinformation from the same data thantraditional solutions.

DataTraditional sources of dataUsing information to make and informclinical decisions is innate within medi-cine. Teaching and personal experiencelead to decision making that can beinterpreted in light of statistical theory.Bayesian statistical methods estimateposterior risk by combining prior riskwith disease associations, which increaseor decrease the odds of disease. Clini-cians do this every day without formal-izing or possibly even thinking about theanalysis. The experience of sudden onsetdyspnea is a possible presenting symp-tom of pulmonary embolism. The pro-portional increase in risk (the positivelikelihood ratio) will be greater still if it isassociated with pleuritic chest pain. Thelikelihood of the diagnosis of pulmonary

JANUARY 2021


embolism will also depend on the priorrisk. It will be higher in a pregnantwomanwith a recent history of long haultravel, whereas it will be lower in a febrilebut previously healthy child.

Computational methods allow data tobe explored at a deeper level by exploringa very large number of predictors andmany more interactions among them.Hence, complex combinations of posi-tive and negative factors can be com-bined into a single estimate of risk in away that an individual clinicianwould beunable to achieve. Moreover, whereas aclinician may be biased by individualexperience of a relatively small numberof cases, computational methods attachweights to risk factors that are based onthe observed data and can use far morecases than any clinician could ever see.However, there is no perfect data source,either for analytical or for numeric so-lutions. Each data source inmedicine hasits own advantages and limitations.

Real-world dataEvery data source can be criticized, andthere is no such thing as a perfect datasource; they are all imperfect, but theyare imperfect in inherently differentways.

Administrative databases. The strength ofadministrative databases is that they tendto have large numbers of subjects andthey are available for analysis, that is, it isusually straightforward to obtainpermission for research and the datausually come in a format that is easy toaccommodate. This means that rareoutcomes can be studied and that thecosts of the research are often modestwhen compared with alternative ap-proaches. There are, however, multipledrawbacks of using administrative data-sets. The data are generally of rather lowgranularity. Hence, outcomes such aspreeclampsia (PE) will simply be recor-ded in a binary fashion with little or noinformation of the different phenotypes.Key exposure data might also be absent.Consequently, associations may beobserved through residual confoundingby rather obvious and well-recognizedconfounders. The lack of granularityalso affects covariates. For example, a


http://www.AJOG.org


womanmay be documented as a smoker.However, it might only be recorded at 1stage in pregnancy, the number or typeof tobacco might not be documentedand there may be no information onwhether she subsequently quit smokingduring the pregnancy. The other majorissue with administrative data sources isthe quality. Taking the previous example,not only is the diagnosis of PE binary, inmany cases the definition of PE may becompletely incorrect, being documentedas absent in a woman with severe diseaseand documented as present in a healthywoman. There could also be conflictinginformation contained in the differentparts of patient records.

The strengths of administrative datamean that it is very likely that they willcontinue to be used for the purposes ofresearch. The appropriate response toweaknesses is their recognition by thosegenerating and using research. Theresearcher may use multiple sensitivityanalyses to determine whether their as-sociations are likely to be true. Forexample, where there is an analysis of anassociation with PE, they might performexploratory analyses to determinewhether the pattern is consistent with areal association. Was the diagnosis of PEassociated with known risk factors, suchas nulliparity and obesity? When theoutcome was confined to women withother features consistent with the dis-ease, such as preterm birth and fetalgrowth restriction, were the key obser-vations still present?

Electronic medical records. The futureprospects for researchers planning acareer in the analysis of routinelycollected data look quite bright, and thisis due to the increasing use of the elec-tronic medical record (EMR). Adminis-trative databases, as describedpreviously, have existed since the secondhalf of the 20th century and would haveinvolved records staff entering data to adedicated database from a patient’s pa-per case record. However, with thedevelopment of the EMR, in some hos-pitals every piece of information that isheld on a patient may be kept in an EMR.This means that the information avail-able is much greater in scope and much


more detailed. Moreover, the data arebeing entered by clinicians who have amuch more highly developed clinicalknowledge than records staff. However,research analysis of the data is a sec-ondary purpose of the EMR. As theprimary purpose is to facilitate the de-livery of and billing for care, the data areentered prioritizing this end. Moreover,although the clinical staff will havegreater knowledge, data entry is a meansto an end and not their primary purpose.Hence, when compared with recordsstaff, they may apply definitions lessconsistently and incompletely. A greatstrength of analysis of the EMR is theaccess to observed numeric data. Thisincludes vital signs (eg, temperature,blood pressure, and respiratory rate) andthe result of lab tests, (eg, biochemistry,hematology, and microbiology). Hence,if studying PE, a researcher might hesi-tate to accept the presence or absence ofthe diagnosis being documented in theEMR as defining the presence or absenceof the condition. However, consideringthe American College of Obstetriciansand Gynecologists Guidelines52 fordefinition, every single blood pressuremeasurement recorded during a staycould be analyzed to define hyperten-sion; every point of care and laboratoryanalysis of urine could be used to defineproteinuria; and every laboratory testperformed during admission (such ascreatinine, platelet count, or alaninetransaminase) could be used to definethe renal, hematological, or hepatic fea-tures, respectively, of severe disease.The practical utility of EMR data also

depends on other issues. Whereasadministrative databases are typicallyavailable in a simple spreadsheet formatof columns and rows and the very largedatabases may have a more complexstructure, EMR data will reflect thecomplexity of the clinical environment.Some members of a cohort of pregnantpatients may have 10 measurements ofblood pressure, others may have 200.The researcher needs to considerwhether they wish for a raw data dumpor for the information to be pre-processed. Whatever the case, additionalresources will be required to format thedataset before analysis when compared

JANUARY 2021 A


with an administrative dataset. Statisticalpower is likely to be better than in manyresearch studies, as the entire populationattending a hospital will, over a shortperiod of time, generate the sorts ofnumbers of patients that would beextremely expensive to study in a pro-spective fashion. However, outcomessuch as perinatal or maternal death maystill be too infrequent for the analysis of asingle institution to be informative.Perhaps one of the most promising av-enues is the analysis of a common EMRused by a network of hospitals, perhapscovered by the same provider, or wheredifferent providers have used the sameEMR. For example, a national RCT inthe United Kingdom is going to collectoutcome data from a widely usedneonatal intensive care EMR to ascertainoutcomes.53

Research studies. Study design is keywhen assessing the evidence around agiven belief in medicine. This has led tothe widely adopted use of levels of evi-dence (Figure 3). However, the prioriti-zation of study design above many otherimportant issues is open to criticism.54

For example, for some outcomes, themain challenge might be to perform astudy that has a large sample size, as theoutcome in question is rare, which istrue for the most important outcomes,such as mortalities or severe morbidities.RCTs are biased to report negative resultsfor rare outcomes, as conducting a trialthat is powered is prohibitively expen-sive. Similar arguments apply for remoteoutcomes, because of the expense oflong-term follow-up. Hence, the pyra-mid of evidence could be an unhelpfulmetaphor, as it oversimplifies a complexquestion.

Observational studies. There are multipleobservational study designs, and adescription of these is outside the scopeof this review. The key issue aroundobservational designs is that the expo-sure of interest is observed but notdetermined by the investigator. This is incontrast to the RCT (described here-after), where the exposure is appliedexperimentally. The major weakness ofobservational study designs is that


http://www.AJOG.org

FIGURE 3Schematic representation of the levels of evidence

RCT, randomized controlled trial; SR, systematic review.



associations between the given exposureand the outcome are not causal but arerelated to their mutual dependence on athird characteristic, a confounder. Con-trolling for confounding can beattempted statistically, for example, us-ing multivariable statistical models.However, the concern is that an associ-ation that is observed after statisticaladjustment for the measured potentialconfounders could be explained by anunmeasured confounder. For thisreason, the associations described inobservational studies are interpretedcautiously.

The reality is, however, that manymajor questions can only be assessedusing observational studies. Sometimes,this is because women will not agree tobe randomized to a trial. Virtually theentirety of the evidence around vaginalbirth after cesarean (VBAC) is based onobservational data. A group whoattempted a large-scale RCTof VBAC vsplanned repeat cesarean delivery in 14Australian maternity hospitals onlymanaged to recruit 22 women out of acohort of 2345 (ie, <1% of thoserecruited).55 Other questions will onlyever be addressable using observationalstudies because an adequately poweredtrial is prohibitively expensive. For



example, there is strong evidence fromobservational studies that there is anincreased risk of perinatal death of thesecond twin at term but not preterm.56

Observational studies also indicate areduced risk of perinatal death of twinswith planned cesarean at term.56 How-ever, a multicenter, international RCTwas only able to study about 1500 termtwin births,57 less than a quarter of therequired sample size to study the effect ofplanned cesarean on the risk of perinataldeath.58 A recent reanalysis of the TwinBirth Study has shown that planned ce-sarean delivery reduced the risk of acomposite adverse outcome, includingdeath, when delivery occurred at termbut not preterm (as predicted by obser-vational studies).59

Randomized controlled trials. There is nodoubt that an adequately powered RCT,conducted in a methodologicallyrigorous fashion (eg, prospective regis-tration, prespecified primary outcome,predefined analysis plan, overseen byindependent steering and data moni-toring committees) provides the stron-gest evidence in relation to the effects ofinterventions in medicine. Furthermore,metaanalysis of multiple such trials,conducted in different settings, will

JANUARY 2021


generate more precise estimates of effectsize and assess whether the interventionworks consistently in different settings.However, a minority of clinical decisionsin obstetrics and gynecology are madeusing such an evidence base.

An increasingly recognized concern isthat we may end up making clinical de-cisions based on metaanalysis of RCTsbut the decisions made may be flawedbecause of issues with the evidence base,despite being composed of RCTs. Asrecently stated by the Editor of TheLancet, “But what if the astonishing en-ergy, commitment, and productivity ofthe systematic review community arepoisoning rather than nourishing med-ical practice?”60 The issue beingaddressed was concerns about the qual-ity and accuracy of many of the smalltrials included in systematic reviews.However, even well-conducted trialshave the potential to mislead. Thewomen recruited to a trial might not berepresentative of the general population;this undermines the external validity ofthe conclusions. RCTs are difficult andexpensive; hence, the number of womenrecruited may have been limited by cost,meaning that only relatively commonoutcomes could be studied. Therefore, atreatment might be recommended ornot based on its effect on mild adverseoutcomes that are common when it hasthe opposite effect on severe adverseoutcomes that are rare. Cost may alsolimit the duration of follow-up. Hence,an intervention might be recommendedon the basis of a short-term benefit butin the absence of evidence about its long-term effect.

Further issues relate to type 1 andtype 2 statistical error. Whereas bestpractice recognizes that reliable con-clusions from RCTs can only be drawnfrom the prespecified primary outcome,p-hacking still occurs. Multiple hy-pothesis tests make it more likely that anull hypothesis is incorrectly rejected(type 1 error).61 Conversely, type 2 er-ror is the major concern in relation tostatistical power. If the sample size is toosmall, a study will be biased to produce anegative result. In an ideal world, cli-nicians would look at the 95% confi-dence interval of the effect and


http://www.AJOG.org

FIGURE 4Ownership of cellphones and smartphones

Reproduced, with permission, from Pew Research Center.64



understand that no safe conclusion canbe drawn. However, with an over-emphasis on P-values, the statisticallyuninformed may equate absence of ev-idence with evidence of absence andutter evidence-based medicine’s 6 mostdangerous words62: “there is no evi-dence to suggest that...”

In summary, clinical decisions involvedrawing conclusions from a large bodyof evidence. For most clinical decisions,the task of relating the research evidenceto making a given decision is highlycomplex. Analysis of routinely collecteddata, whether administrative or EMR,has the advantage that the entire popu-lation can be studied but has the draw-back that the approach is much morecomplicated, and the output can bedifficult to assess. RCTs are attractive bytheir relative simplicity and their exper-imental design. However, consideringcomplex questions about recommend-ing a given approach in a simplisticmanner, for example, study design is theonly major concern, has the capacity tolead to widespread harm.

Novel sources of dataDigital phenotyping.Digital phenotypingis an all-encompassing term for the trailof digital data that people leave behind intheir daily lives, interactions with theinternet, social media, and technology.These data are largely untapped and havesubstantial potential for use in thehealthcare industry. People generate anenormous amount of digital data eachday, and this moment-by-momentcomputation of an individual’s pheno-type measured in situ from ubiquitouspersonal devices has enormous potentialto revolutionize the way we understandand make sense of health and health-related conditions.63

A total of 81% of Americans now ownsmartphones, and this rate has dramat-ically increased over time (Figure 4)64.The ubiquity of smartphones makesthem ideal to collect detailed patterns oftrue behavior from individuals in anobjective manner. In addition, passivemethods for collecting data—thosemechanisms that automatically collect,with the individual’s consent, butwithout the need for explicit input—


provide a continuousmeans of collectingdata in the background and allow for amore fine-grained collection of behav-ioral, health-related, environmental, andlifestyle data.65 These data, in combina-tion with traditional clinical data, pro-vide a powerful tool to understand anddevelop a digital phenotype of bothhealth and disease at the individual levelbut also across a given population(Figure 5). With rapid advances in thebroad availability of machine learningand the application of numeric simula-tions, digital phenotyping holdstremendous promise to provide aplethora of insight not previouslypossible.Novel approaches such as digital

phenotyping allow wider applications ofN-of-1 trials or studies and are moreeffectively performed using computa-tional methods. By connecting aggregatemultilevel and multiscale clinical,biomedical, personal, social, contextual,environmental, and organizational dataand using it to individualize decisionmaking in health and medicine, wewould get not only a more informed

JANUARY 2021 A


picture of an individual’s health, but alsoa greater promise for transforming andoptimizing healthcare for populations aswell.

Wearables and internet-connected sensors.Current and emerging wearable andinternet-connected devices and sensorsare poised to address many deficienciesof traditional data sources by enablingthe longitudinal collection of contextu-ally rich, multifaceted, and individual-level data at unprecedented scales. Theuse of wearables or smart devices (eg,activity trackers, smartwatches, gluc-ometers) for health and wellness israpidly proliferating among consumers,from an estimated 9% in 2014 to 33% in2018.66 They also continue to be incor-porated into clinical practice as a meansof remote patient monitoring (eg,chronic disease management, acutehealth event detection) and population-based health research.4 A detailed re-view of these devices is beyond the scopeof this article, and we refer the reader toDunn et al67 for a thorough discussion ofhealth-related applications. The diverse


http://www.AJOG.org

FIGURE 5Digital phenotype data streamed in real time from a patient’s cell phone to the supercomputer



data generated by sensors embedded inwearables or smart devices can reporton, for example, an individual’s mobilityand physical behaviors using acceler-ometers, gyroscopes, and global posi-tioning system; physiological parameterslike heart rate, temperature, and oxygensaturation; biologic analyte concentra-tions like blood glucose measured fromcontinuous, internet-connected glucosemonitors; and real-time air pollutionexposure via personal air quality moni-tors.68 Furthermore, an individual’s so-cial behaviors and mental health may beassessed, for example, through theanalysis of phone call and text messagelogs (ie, frequency, duration, incomingand outgoing)69 or through psycholin-guistic analyses of user-generated socialmedia content.70 These technologieshave been used in a variety of healthdomains including weight management,metabolic and cardiovascular disease,maternal and neonatal care, sleep qualityand assessment, and neurology, amongothers.67,71

Although it is likely that the nextdecade will see networks of wearabledevices being routinely used in medical



practice, there are a number of chal-lenges to be addressed. These include,but are not limited to, the developmentof a robust cyber-infrastructure capableof the real-time fusion and analysis ofdisparate data streams and flexibleenough to accommodate the rapid paceof consumer technology development;development of analytical methods andalgorithms to extract or discover signa-tures (ie, digital biomarkers) or leadingindicators of health or disease statusfrom real-world, messy data; executionof large RCTs to assess efficacy of usingthese novel data sources in treatments orinterventions; and interoperability withexisting EMRs and medical decisionmaking workflows.Contrary to popular belief, the data,

or big data itself, are not the focus ofcomputational medicine. Its focus is themodeling of complex systems such as thehuman body or the healthcare system.Data in this context are an essentialprerequisite to building a model, but ashighlighted in the section on the theory-based model, a large amount of data andespecially big data is not a necessarycondition for those models.72

JANUARY 2021


Currently, data gathering is primarilydriven by convenience and is empiricalin nature. Such an approach mustnecessarily lead tomany areas where dataare either not collected or not usable formodel development. It has been pro-posed that the models, especially thecomputational models, should definewhat data are needed and direct thedesign of experiments allowing bettermodel development. The area wheresuch an approach would be especiallyuseful is genomics.72,73

ModelsHowever, evenwhen the data contain thenecessary information, it is by itselfinsufficient to individualize decisionmaking andmedicine in general. Indeed,an accuratemodel of the complex systemhas to be developed, a model that cap-tures the equifinality, the multiple caus-ative pathways in the system leading tothe same outcome, requiring individu-alization of medicine. The capture ofequifinality and enabling of individuali-zation of medicine can be achieved bydata being used by computationalmodels, theory- or knowledge-based


http://www.AJOG.org


models, data-driven models, or modelsthat combine both approaches. Asmentioned earlier and using broadstrokes, the data-drivenmodels require alimited amount of knowledge about thesystem but a large amount of data. Thetheory- or knowledge-based modelsrequire a limited amount of data butsubstantial knowledge of the system.

Theory- or knowledge-based models.What is a model? A mouse model forlaboratory experiments that try tomimic human physiology, a toy airplanemodel, a model community? No, thenotion of a model addressed here is amathematical construct that representsan abstraction of physical phenomenadescribed by a scientific theory—amathematical characterization of a setof inductive hypotheses, often based onobservation and moving toward gener-alized conclusions, put forth to explainevents that occur in the physical uni-verse. Such scientific models thus formthe essence of the second pillar of sci-ence, alongside empirical observations,as a fundamental source of knowledge.To qualify as a scientific theory or as ameaningful model based on theory, thetheory must be falsifiable, according tophilosopher Karl Popper—that is, itmust be capable of being contradictedand abandoned if predictions contraryto theoretical predictions areobserved.74 Today, we deal withcomputational models to make pre-dictions of the behavior of complexsystems; they are corruptions (dis-cretizations) of mathematical modelsconstructed so as to be implemented ondigital computers. The selection of amodel to describe a class of physicalrealities is the most important anddifficult component of predictive sci-ence. Models involve parameters thatare usually unknown and can berandom variables; the calibration ofmodel parameters, the adjustment ofthe model parameters for the modelto better reflect the modeled complexsystem, requires the acquisition ofdata that can be noisy and expensive toaccess, and the discretization of themodel to produce a viable computa-tional model introduces additional


uncertainties. The quantification ofthese uncertainties is essential for reli-able model predictions.Science and scientific prediction

without models are meaningless. It mustbe emphasized that the development ofpredictive computational models iscritical to the advancement of science; itis a fundamental challenge that must bemet in all areas of science, medicine, andtechnology.The construction of predictive

computational models in medical sci-ence and practice could lead to one of themost important developments in humanhistory. For example, the development ofpredictive computational models ofcancer75 and of the effects of variouscancer therapies for specific individuals,where patient-specific observationaldata are used to calibrate and validatemodels, would make possible break-through effective and noninvasive treat-ments of the disease, revolutionizemedicine worldwide, and forever enrichand expand the scope of medical science.

Data-driven models.Data-driven modelsare those for which predictions are per-formed by examining relationships be-tween a number of available statevariables (predictors) related to aparticular quantity (outcome) of inter-est. Unlike theory- or knowledge-basedmodels, explicit knowledge of the un-derlying physical, biologic, and psycho-logical mechanisms impacting theoutcome is not a prerequisite. Instead,the goal is to develop predictive modelsthat infer relationships directly fromavailable data. Fundamental to thedevelopment of these models is theavailability of sufficient data containinginput features (predictors, confounders,and mediators) that have observed in-fluence (either directly or indirectly) onthe outcome of interest. Historically,data-driven models used in healthcarehave been fairly simple via the deploy-ment of regression models leveraging arelatively small number of input pa-rameters. These regression models typi-cally have very modest computationalrequirements. More recently, moreadvanced machine-learning models arebeing exploited in a variety of healthcare

JANUARY 2021 A


areas. Relevant examples of modelingalgorithms in this space include featuredetection and pattern matching (usefulin imaging and radiology), clustering(the detection of like groups and struc-tures), and classification (formulatingpredictions into a predefined set ofrelevant outcome classes). Clustering isan example of unsupervised learningwhere labeled data (data with definedoutcomes) are not required as an input,whereas classification is an example ofsupervised learning that requires a set ofdata with known outcomes that serves asthe basis for training the classificationalgorithm. Performance and range ofapplicability of these machine-learningmodels often improves with a largeramount of data, and consequently, thesetypes of data-driven models can lead tomore demanding computational re-quirements requiring parallel processingand high-speed input/output sub-systems during their formulation,particularly for model training in su-pervised learning approaches. Althoughvery promising in a variety of industries,there are challenges in the US healthcareindustry related to development andadoption of machine-learning models.Examples of these challenges includedata inconsistencies and limited wide-spread availability of high-quality datafor model development because of theproprietary nature of electronic healthrecord systems, lack of understandingand exposure to more advanced dataanalytics within the medical community,and difficulty in disseminating machine-learning results by healthcare pro-fessionals that can be perceived as black-box models.

Combined theory- and knowledge-basedand data-driven models. The pro-ponents of the 2 types of computationalmodels, theory- or knowledge-basedmodels and data-driven models, oftenargue about their relative merits, butexperience indicates both are veryimportant in computational medicine.An example is illustrative.

HeartFlow, Inc is a pioneeringcomputational medicine companyproviding individualized, noninvasivediagnosis of coronary artery disease to


http://www.AJOG.org


determine whether or not coronary ar-tery narrowings are obstructing bloodflow and would benefit from coronaryartery stenting or coronary bypass sur-gery. The primary source of patient datais a coronary computed topography(CT) scan, which is uploaded by way of asecure web-based interface. The firstaspect of modeling is construction of anindividual quantitative 3-dimensionalanatomic model of the aortic root andcoronary arteries. This step, performedusing data-driven modeling, specificallydeep learning, is referred to as “seg-mentation” and is particularly chal-lenging. Construction of a high-accuracygeometric model of the coronary vesselson a patient-by-patient basis, even withthe assistance of computer algorithms,may take an inordinate amount of timeand is prone to error. However, becauseof the compilation of an enormous datatrove of coronary trees of over 50,000patients, deep learning algorithms couldbe trained to develop the data-drivenmodels to more quickly and accuratelysegment coronary arteries of new patientdata. Data-driven models, developedusing machine learning, for example,deep learning, are very effective for vi-sual tasks, and constructing geometryfrom a CTscan is in essence a visual task.

Once the coronary arteries aresegmented, an analysis of blood flow isperformed through theory- orknowledge-based modeling. The Navier-Stokes equations of fluid dynamics the-ory are employed because they arecapable of accurately representing bloodflow phenomena in the coronary ar-teries. Additional data are not needed toestablish this fact. The Navier-Stokesequations are based on the foundationof Newtonian mechanics and have beencorroborated through untold numbersof physical experiments over hundredsof years. Very accurate, efficient, androbust computer algorithms areemployed for their solution. The result isa precise prediction of flow velocity andpressure in the coronary arteries. Clini-cians are particularly interested in frac-tional flow reserve, which is the pressuredrop across obstructed regions in thecoronaries caused by disease underconditions of maximal coronary blood



flow (hyperemia). This predicts whetheror not coronary revascularization isneeded and is the gold standard forguiding treatment. During cardiac cath-eterization, hyperemia is induced byadministering a drug (adenosine).HeartFlow analysis simulates this he-modynamic condition computationally,thus making it possible to determinenoninvasively which patients should betreated medically and who should besent for coronary angiography andpossible revascularization. The compu-tational model can also be manipulatedto simulate stent implantation withprediction of the potential outcome ofstenting before the procedure is per-formed. The benefits of HeartFlow’ssynthesis of theory- or knowledge-basedand data-driven modeling have beendemonstrated by the Platform Trials.76

Among patients with planned invasivecoronary angiography (ICA), 73%showed no considerable blockage orobstruction, and in 61% of patients, theuse of computational modeling resultedin the cancellation of a planned ICA.After 1 year, none of the 117 patientswho had ICA cancelled had suffered anadverse clinical event. There are over 1million ICAs in the US each year, over 2million in Japan, and a similar numberin Europe. The cost of an ICA in the USis over $12,000; about two-thirds seemto be unnecessary and can be eliminated,representing potential savings in billionsof dollars. In summary, there are well-established and highly reliable theory-or knowledge- based models that can beeffectively used in computational medi-cine, but where these models do notexist, we can utilize powerful data-drivenmodeling approaches.Computational models, as any other

models in medicine, are developed to beclinically useful to make optimal med-ical decisions and thus improve pa-tients’ health. To improve decisionmaking, the models need to enable bothindividual decision making and deter-mination of individual net benefit forcontemplated tests or treatments.77 Toaid in clinical decision making andespecially in individualized decisionmaking, the models need to have highdiscriminatory power and be well

JANUARY 2021


calibrated.78 Discrimination is theability of the model to predict a higherprobability of the event in patients whowill ultimately experience the event oroutcome than in patients who will nothave it.79 If a model always predicts ahigher probability of the event in pa-tients who have it than in patients whodo not experience it, then the discrim-ination of this model is perfect, and itsmeasures, area under the receiveroperating characteristic curve and c-statistics, are equal to 1.0. However, therelationship between true positives andfalse positives for different cutoff pointsof the model predictions those statisticsdescribe has to be balanced. Thus,choosing a cutoff that maximizes truepositives leads to an increase in falsepositives and vice versa.79 Calibration,on the other hand, is the ability of themodel to predict probabilities of theevent (the absolute risk) that are inagreement with observed frequencies ofthe event.79 In other words, if a well-calibrated model predicts risk of theoutcome of 60% in 10 women, 6 of the10 or 60% of them will experience thepredicted event.

Good discrimination is an essentialfirst step, without which the model isgenerally not useful for decision mak-ing. However, good discrimination isinsufficient for effective decision mak-ing. The measures of discriminationassume an equal value of sensitivity andspecificity, of false negatives and falsepositives. In real life, however, theconsequences of false negatives (eg,missed diagnosis) are more severe thanconsequences of false positives (eg,unnecessary tests).77 As a consequence,a model with good discrimination canstill have an unacceptable rate of falsenegatives (missed diagnoses).77 Apoorly calibrated model can lead to asituation where an individual with ahigh risk of disease has assigned a lowprobability of it occurring and thusmisses the opportunity of effectivepreventive intervention.77 Althoughboth the discrimination and calibrationare critical for the clinical usefulness ofthe model for medical decision making,they are rarely both provided in theliterature.79 A systematic review in


http://www.AJOG.org


cardiovascular medicine showed thatdiscrimination was reported in 63%and calibration in 36% of the studies.80

Good discrimination and good cali-bration of the computational model orany other model is necessary for themodel’s clinical usefulness for individ-ual patient and individualization ofmedical decision making.78,81 The rarerthe event the model predicts, the moredifficult it is to achieve good calibration.This is because the model is much morelikely to be right by predicting theoverwhelmingly more commonnonevent, and thus the prediction ofhigher risk of the event is under-estimated. However, the outcomes ofhighest interest are the rare, most severemorbidities and mortalities rather thanmore common proxy outcomes. Finally,because decision making is about theprediction of future outcomes in in-dividuals and populations, the model’sdiscrimination, calibration, and use-fulness in decision making can only beaccurately evaluated in the process ofexternal validation.82 That is in thepopulation of individuals whose datawere not used in model developmentand who are from a population thatwould be a potential target populationfor the clinical use of the model.Therefore, external validation should bethe final adjudicator of the model’sperformance and clinical usefulness fordecision making.

Clinical practice can only be informedby very good models that represent thecomplex system of the human body inhealth and disease accurately. The dataare critically important but are, concep-tually, insufficient to the development ofsuch models. Building meaningfulmodels requires a much broaderanalytical and quantitative medicalexpertise, much beyond empirical in-puts. Novel sources of data, the signals ofwearables, digital phenotype dataincluding environmental or social mediadata, etc., carry a promise of improvingthe accuracy of the models and, as aconsequence, individualization ofhealthcare that utilizes those models.However, at this point, it is not clearwhich sources of data and to what extentwill inform the development of the


models in general and computationalmodels specifically. In short, more datadoes not mean more information and,thus, better healthcare.Computational models offer an op-

portunity to integrate the differentsources of multimodal data into action-able information, which will inform theclinical practice. Computational modelswere shown to provide real insights, forexample, into breast cancer therapy.83

However, although promising, the clin-ical usefulness of those models has notyet been demonstrated, including thoseproposed in obstetrics andgynecology.77,81,84,85

Medical Decision MakingAs we have argued in the beginning, thehuman body and the healthcare systemare complex systems. Those systems aremade of a very large number ofintensely interacting elements forminghighly coupled networks with a vastnumber of feedback loops and re-dundancies. This structure makescomplex systems robust to failure butalso, because of equifinality, difficult topredict their behavior. Because of thispervasive and inescapable uncertainty,medicine is, in its essence, decisionmaking under uncertainty. The de-cisions about tests to be performed andtreatments to be administered range theentire spectrum of healthcare from de-cisions regarding individual patients topolicymaking. Traditionally, decisionsare being made based on experience andtraditional evidence from researchstudies, both limited by a number offactors including limited possible per-sonal experience of a single physicianand average effects reported by theclinical research studies, respectively. Incomplex systems, characterized bymany causative pathways, basing thedecision on an average effect in a pop-ulation of patients is limiting the opti-mality of the decision. Becausemedications are only effective in arelatively small fraction of patientsreceiving them5 and tests’ predictiveperformance depend strongly on thepatients’ characteristics defining theirpretest risk of the disease tested for,4

decision making has to be

JANUARY 2021 A


individualized to be optimal. Specif-ically, it has to be optimal under thecondition of uncertainty, when the de-cision is being made prospectively andnot after the outcome is alreadyknown,10 and for an individual or apopulation of individuals and not for apopulation average. However, to indi-vidualize medical decision making re-quires accounting for very largenumbers of constellations of risk factorsand protective characteristics. Thenumber of those unique combinationsgrows exponentially. For example, just 5characteristics with 3 categories eachwould result in 243 unique combina-tions of those characteristics potentiallyaffecting the individual outcome(Figure 6).

Traditional methods of analysis arelimited in handling this type of complexdata. However, computational modelsexcel in modeling the behavior ofcomplex systems, enabling individuali-zation of decision making. The indi-vidualized medical decision making isdependent on the individual predictionof outcomes, individual weighing ofprobabilities of outcomes, individual(patient to policymaker) preferences,and individual risk communication(Figure 7).

Individualized prediction of outcomes“Prediction is an essential feature ofnonarbitrary decision-making”51 andindividual prediction is an essentialfeature of individual decision making,critical in medicine. Although the per-fect prediction of the behavior of com-plex systems may never be achievable,computational models can eliminate,from a pool of potential alternatives,ones that are inconsistent with the datafor an individual and assign individualprobabilities to the remaining alterna-tives.51 This probabilistic approach isvery effective in real-world individual-ized decisionmaking, especially whenweaccept that deterministic solutions (0%or 100% probabilities) are not possiblein predicting the behavior of a complexsystem such as the human body.86

Perfectly accurate tests or perfectlyeffective interventions are not and arenot likely to ever be available.


http://www.AJOG.org

FIGURE 6The role of computation in medicine

The human body and the healthcare system are complex systems, networks of highly coupled

components intensely interacting with each other. These interactions give those systems redundancy

and thus robustness to failure and, at the same time, equifinality, many different causative pathways

leading to the same outcome. The equifinality demands individualization of medical care, which is

urgently needed. Computational models excel in accounting for a very large number of interactions

and modeling complex systems, and therefore they enable the individualization of medicine. They

have the potential to enable individualization of medical decision making and, consequently, better

health outcomes and lower costs.



Decision theory: weighing individualprobabilities of outcomes andindividual (from the patient’s to thepolicymaker’s) preferencesBuilding models of individual optimaldecision making under uncertainty inhealth and medical care is of tantamountimportance. The modeling cornerstone,besides the individual outcome predic-tion, is to describe the preferences towarduncertainty, capturing individuals’ atti-tudes toward risk; loss; incomplete in-formation; and other ingredients that



affect personal choices, both statically anddynamically.86 Building risk-preferencemodels is a challenging task as similarnotions in economics (expected utilitytheory, behavioral finance, bounded ra-tionality, rational inattention, etc.), albeitfoundational, cannot be directly appliedbecause of how detrimental certain risksmight be for the health and wellbeing ofthe patient. Indeed, traditional riskcriteria, based on averaging andsmoothing formulations, are not verysuitable to model loss aversion, fear,

JANUARY 2021


prudence, impatience, and other(frequently acute and also path-dependent) sentiments arising in thecourse of a medical treatment. Further-more, traditional criteria are typically 1-dimensional and, thus, cannot capturethe multiattribute risks a patient faces.

An additional challenge is how tosolve the associated stochastic optimi-zation problems. Indeed, such complexrisk preferences criteria often give rise totime-inconsistency, a well-documentedphenomenon even for financial and in-surance risks. These problems are veryhard to solve because all classicaloptimization approaches fail and newtechniques, both analytical and compu-tational, need to be developed. In addi-tion, these problems need to be analyzedin real time, for model decay always oc-curs, and adaptive optimization criterianeed to be incorporated to captureincoming information.

Overall, individualized medical deci-sion making under uncertainty in dy-namic settings is a wide-open area with aplethora of new research directions.Although, as mentioned previously,modeling patients’ risk preferences willborrow considerably from fundamentalnotions in financial economics, buildingboth sophisticated dynamic models andsolving the associated stochastic controlproblems present many challenges but, atthe same, a very fertile ground for bothcross- and interdisciplinary collabora-tions. Furthermore, there is a pressingneed to support such developments as, atpresent, there is a rather sizable discrep-ancy between the sophistication that ex-ists in medical science models and thesimplicity, if not absence, of evaluationcriteria from the patients’ point of view.

The literature and evidence in the areaof decision theory in medicine is verylimited, and new computational- andcomputer scienceebased approachespromise to offer progress in this area.

Individualized risk communicationHowever, individual prediction andweighing of individual probabilities ofoutcomes and individual preferences areinsufficient for individual decisionmaking in medicine. Those individualprobabilities and preferences have to be


http://www.AJOG.org

FIGURE 7Individualized decision making in medicine

Individualized decision making in medicine can be thought of as composed of 3 consecutive parts:

prediction of the individual probability of outcome; weighing of those individual probabilities of

outcomes and individual patient preferences for outcomes; and communication of the risk, the

probability of the outcome, and the burden of this outcome.



appraised and communicated efficientlyto make effective decisions. The domi-nant way risk communication has beenviewed is as an individual-level percep-tion. Uncertainty is presumed to driverisk, and its perception exists when anindividual perceives information to beunavailable, inaccessible, or inconsis-tent.87 Most theories of risk communi-cation conceptualize risk as anindividualized perception (eg, PlannedRisk Information Seeking Model,88 The-ory of Planned Behavior,89 Theory ofMotivated Information Management,90

Extended Parallel Processing Model,91

Health Information AcquisitionModel,92 Risk Information SeekingModel,93 and Comprehensive Model ofInformation Seeking94). In addition tothe view that risk communication is anindividual-level construct, it is importantto realize that perceptions of risk areinfluenced by others or socially con-structed.95,96 Past research has acknowl-edged this in several communicationmodels (eg, subjective norms that influ-ence individual behavior), but these per-ceptions are still measured on anindividual level. A more recent argumentis for health-related risk studies to thinkof risk communication as the “exchangeof information among individuals,groups, and institutions related to theassessment, characterization, and man-agement of risk.”97 This is an approachthat resonates with healthcare providersbecause they are naturally part of the riskcommunication process. However, theimportant others are expanding, and thisis especially relevant as mobile and socialmedia become a part of the fabric of oursociety. Specifically, individuals can maketheir risks visible to others by posting onsocial media or sending a photo or textto a trusted friend or family member, andothers can directly respond, thus influ-encing how individuals internalize andpotentially act regarding their own risk.Thus, risk decision making is a combi-nation of individualized perceptions andthe influences of others around them.

Medical decision making relies on theunderstanding of potential outcomesand their probabilities. A rational deci-sion would theoretically try to maximizethe probability of a positive outcome


while minimizing the risk, the proba-bility of an adverse outcome. Of course,the reality is more complex. There aremany factors that impact medical deci-sion making that may cause a patient tochoose a riskier procedure to achieve adesired outcome. The preferences,values, and biases of the individual pa-tient have a large influence on theperception of risk, the value of potentialoutcomes, and ultimately the individualdecision making. For example, a patientmay choose to deliver the baby at homerather than in the hospital even thoughshe knows that the risks for home de-livery are higher.98 Here it is the personalpreference and the value placed on homedelivery that cause the patient to placeless importance on the associated risk.Knowing these individual biases can helpthe clinician formulate an effectivecommunication strategy. For instance,knowing that the patient prefers homedelivery, it would be prudent to ensurethat the patient truly understands theassociated risks of home delivery,particularly if she has other risk factors.Computer game technology has beenused in the past to educate patients aboutthe risks and possible outcomes ofvarious screenings and procedures. In thefuture, immersive game technology willbe used to automatically determine how apatient places value on certain outcomesand learn what bias the patient hasregarding associated risks. The use ofvirtual environments will allow the pa-tient to be less inhibited about their re-sponses to questions asked throughinteractive scenarios. The game itself will

JANUARY 2021 A


use these responses to adapt the presen-tation of information regarding the risksand outcomes of the procedure in ques-tion. Ultimately, the decision lies with thepatient, but with improved communica-tion provided by computational solu-tions, the clinician and patient will be ableto formulate a plan that meets the biasand value structure of the individual pa-tient and minimizes the risks involved.

Individualized decision making, madepossible by the computational modelingof complex systems, has the potential torevolutionize the entire spectrum ofmedicine from individual patient care tohealthcare policymaking. In patient care,computational models can enable indi-vidual decision making based on the pa-tient’s individual net benefit ofcontemplated tests and interventions. Inmaking decisions on a strategic level of ahospital, a system of hospitals, or on thelevel of government policy, the compu-tational models enable individualizeddecision making for a population. Thoseindividualized decisions on the popula-tion level are based on the benefits ofindividuals comprising the populationrather than based on the average benefitof the population. This approach allowsapplying tests and treatments exclusivelyto individuals who receive net benefit, inwhom benefits outweigh the risk, ratherthan to all individuals in the populationregardless if they do or do not receive thenet benefit. Computational modelsenable individual medical decision mak-ing that can transform medicine and thehealthcare system, both in urgent need ofa substantial disruptive transformation.


http://www.AJOG.org


Future of Computational Medicine:Healthcare Stakeholders’PerspectivesThe substantial potential benefits ofcomputational medicine can only berealized if they solve problems impor-tant to healthcare stakeholders. Thiswill ensure that the computationalmedicine solutions will be implementedand thus could transform healthcare.The potential to transform healthcareexists at all levels of the system, from thehospital and hospital system levels togovernment policy. This is becausehealthcare is a complex system, whichcan be thought of as a highly couplednetwork system in which constituentsaffect each other.47,49

AcademiaFrom an academic standpoint, it is anextremely exciting time to be investing inresearch and educational programs incomputational medicine. The conflu-ence of computing power, scalable al-gorithms, availability of patient data, andincreased understanding of physicalmechanisms positions us for a revolu-tion in the way medical advances areachieved and in the way medicine isdelivered. The methodological founda-tions for concepts such as a digital twinare being laid in fields as diverse asaviation and medicine, built on syner-gistic combinations of predictivephysics-based models and data andproviding an enabling technology forachieving asset and patient-specific data-driven decisions. The research andeducational programs that will enablethis revolution must be highly interdis-ciplinary in nature. The fields of artificialintelligence and data science will play abig role, particularly with machine-learning methods, but we cannot losesight of the critical importance ofphysics-based and mechanism-basedmodeling. Educational programs incomputational medicine must blendthese perspectives, training students atthe interfaces of mathematical modeling,computing, data science, and medicine.Partnerships have never been moreimportant, including the sharing of dataand digital infrastructure, both partner-ships among units within the university



system and partnerships across univer-sities, government, and healthcareproviders.

Hospital systemIn healthcare, we are awash in data andinformation. However, it is often indisparate systems, discrete data, andlargely does not tell the entire story of thepatient. With computer technology andthe introduction of the EMR, we havelost sight of the entirety of patient care,invested more in small, discrete data el-ements to enter into the system, often tonever be harvested. Over the decades wehave stored more patient-specific datathan ever, but at what cost? Very few, ifany, health systems maximize the use ofdata to truly improve quality, patientsafety, or health outcomes or to ulti-mately improve the health of the com-munities we serve. We have the ability tomake profound changes in how care isdelivered. Given the current state ofhealthcare, the variability in care deliv-ered, and the continued rise in overallcost of care, we need a dramatically newapproach, and the data we have beenfeverishly storing may hold the answer tothe changes we need.The discrete data inputs, when pro-

vided to computational models, maylead to information that results in betterdecision making, more decisive and co-ordinated care, and, ultimately, betteroutcomes for patients and communities.With computational models availableand more robust computing capacity,care delivery can be more predictive,efficient, and effective in the diagnosisand care of an individual patient, whichimproves outcomes for all parties,payers, providers, and patients. As healthsystems look ahead toward affordablecare and value-based payment systems,effective and efficient care of patients istantamount to health and longevity ofthe organization and the right thing todo for the patient. However, that alone isnot enough. To create change and toultimately impact patients, these ad-vances must make it to the bedside andtruly impact the care of the patient attheir most vulnerable time and in themost expensive setting in healthcare.Once these advances cross the chasm and

JANUARY 2021


change the way care is delivered inside,the hospital (in a sustainable and lastingway for the patients, families, and pro-viders) then it will be transformative andwill truly change the hospital systemsand healthcare for the better.

State governmentResponsible stewardship of taxpayerdollars is arguably the paramount dutyof lawmakers throughout the USwho aretasked with using those dollars tomaintain government operations whilealso ensuring that appropriations aremade to achieve expectations of tax-payers. Achieving this balance is nosimple matter. Texas in particular facesadditional difficulties because of certainlimitations, including a constitutionalbalanced budget requirement99 and theadoption of a biennial budget that ne-cessitates substantial estimations.100 Toaid with these constraints, the TexasLegislature often involves subject matterexperts to educate lawmakers on how tobest achieve desired policy outcomeswith limited resources. However, thistype of short-term partnership—where,for example, experts are only called on totestify for a single committee hearing—has the potential to fall short as facts anddata are either forgotten in the deluge oflegislative issues or overlooked for po-litical expediency. Avariety of factors cancause this, but notably, the lack of astrong and well-established partnershipbetween government and experts is a keycontributor.

It is empirically proven in Texas thatlong-term partnerships, particularlywith industry and academia, are botheffective and fruitful. The Cancer Pre-vention and Research Institute of Texas(CPRIT), which was established by theLegislature and approved by Texas votersin 2007 to aid in cancer research andimplement the Texas Cancer Plan, is aprime example. Since its inception,CPRIT reports recruiting 192 cancer re-searchers and labs, producing a NobelPrize recipient, and awarding 1452grants totaling $2.4 billion.101 It has alsoprovided the Legislature with evidence-based prevention interventions and ser-vices with 66 active projects that, com-bined, impact every single one of the


http://www.AJOG.org

FIGURE 8The Texas Advanced Computing Center at The University of Texas atAustin

The University is home to Frontera, the fastest supercomputer at any university and the fifth most

powerful system in the world. This is the type of computational infrastructure that will allow

breakthroughs in computational medicine.



state’s 254 counties.102 This partnershiphas produced both desired health out-comes and considerable cost savings forthe Legislature and Texans, which likelycontributed to the Legislature’s over-whelming support, and eventual voterapproval, for Proposition 6 in 2019 toincrease CPRIT’s ability to award fund-ing from a total of $3 billion to $6billion.103 In addition, Texas is nowseeking to replicate its success withCPRIT in the mental health arena withthe recent passage of Senate Bill 11 by the86th Legislature in 2019 to create theTexas Child Mental Health CareConsortium.104

The benefits of a strong partnershipare clear: improved research capabil-ities, healthier outcomes, and cost-efficient investments. As such, a part-nership on computational medicinebetween the Texas government andother healthcare stakeholders hasvirtually limitless possibilities. Everyarea of medicine can benefit from amore individualized approach, andestablishing the infrastructure forvarious computational models pavesthe path to achieving desired healthoutcomes, whether that outcome islowering maternal mortality rates orsimply increasing wellness visits toemphasize preventive care. Further, acomputational medicine partnershipcould assist the Legislature with itsongoing efforts to contain healthcarecosts in Texas, particularly in Medicaid.The most recent iteration of thisinitiative is delineated in the 86th Leg-islature’s House Bill 1, the state’s budgetfor fiscal years 2020 and 2021, and re-quires the Health and Human ServicesCommission (HHSC) realize at least$350 million in savings.105 Currently,HHSC is limited to achieving theseservices by addressing systemic fraud,waste, and abuse as well as maximizingthe use of federal Medicaid dollars.With computational medicine, the op-tions expand as, for instance, expensivetreatments are avoided with individu-alized decision making and preventivecare, medications best for a patient areprescribed, and unnecessary proceduresare cut back with individual best prac-tices. Simply put, having the ability to


take a pertinent health issue and pro-duce cost-efficient solutions that willensure a healthier population withstriking accuracy would make the Leg-islature’s job elementary, to the benefitof lawmakers and, more importantly,Texans.

Federal governmentFederal agencies can serve as enablers oreven doers for issues with societal im-plications that are beyond visible marketforces. There are elements of computa-tional medicine that may require suchactions from the federal government.The Human Genome Project, whichevolved into a multiagency, interna-tional, and public-private partnership, isan example of a large-scale effort thatwas driven by a federal recognition thattechnological convergence of advance-ments in robotics, image processing,database restructuring, computing, la-sers, and so forth, all outside of medi-cine, could be foundationallytransformative to genomics. The scale ofdata that became to be generated fromsequencing has been pivotal to drivingdata-centered analysis into the fabric ofmedicine. Learning from rich data setsremains a challenge, as the questionsbeing asked are more complex and thegrowth of available data continues tostress the leading edge technologies,including artificial intelligence (AI)systems.

JANUARY 2021 A


The promise of computational med-icine resides in the complex technicallandscape that spans more traditionalsupercomputing, through data sciences,cognitive computing, and AI. The ad-vancements in computational medicinewill be furthered when we recognizewhich efforts are hindered by viewingthe computational demands as a posthoc add-on. Our notable successes inthe application of computational sci-ence to decision making in highconsequence situations have, at theircore, teams of specialists from theoutset that codevelop everything fromthe technologies to tools and share inthe responsibilities of the outcomes. It isquite likely that AI-based analog willneed to develop in the same way. Manytough problems remain in predictionfrom models or data or both, problemsthat do not have visible economicdrivers behind them. It is in this spacethat federal agencies can play a role infilling gaps that can help in makingprogress in the areas we have discussed.It is likely that at the federal level, thatprogress here will require 2 or morefederal agencies participating. There aremany means to organize such efforts,and finding the suitable champions is akey ingredient.

As enablers, the federal agencies canalso help develop the technical base fromacademic programs to support thefuture workforce and key areas such as


http://www.AJOG.org


uncertainty quantification for AI, datatrust and integrity, and decision support.The US has a unique ability today toshape this future. Defining the rightpartnerships and working together cantransform health research and healthcarefor us all (Figure 8). -

REFERENCES

1. Osler W, Bean RB, Bean WB. Sir WilliamOsler aphorisms: from his bedside teachingsand writings. New York, NY: Henry Schuman,Inc; 1950.2. Cheng YW, Snowden JM, Handler SJ,Tager IB, Hubbard AE, Caughey AB. Litigation inobstetrics: does defensive medicine contributeto increases in cesarean delivery? J Matern FetalNeonatal Med 2014;27:1668–75.3. Wolf JH. Risk and reputation: obstetricians,cesareans, and consent. J Hist Med Allied Sci2018;73:7–28.4. Liu P, Shi Z, Wang C, et al. The false-positiveand false-negative predictive value of HIV anti-body test in the Chinese population. J MedScreen 2008;15:72–5.5. Schork NJ. Personalized medicine: timefor one-person trials. Nature 2015;520:609–11.6. Gigerenzer G, Gaissmaier W, Kurz-Milcke E,Schwartz LM, Woloshin S. Helping doctors andpatients make sense of health statistics. PsycholSci Public Interest 2007;8:53–96.7. Wegwarth O, Gigerenzer G. The barrier toinformed choice in cancer screening: statisticalilliteracy in physicians and patients. Recent Re-sults Cancer Res 2018;210:207–21.8. Wegwarth O, Gigerenzer G. US gynecolo-gists’ estimates and beliefs regarding ovariancancer screening’s effectiveness 5 years afterrelease of the PLCO evidence. Sci Rep 2018;8:17181.9. Gigerenzer G, Edwards A. Simple tools forunderstanding risks: from innumeracy to insight.BMJ 2003;327:741–4.10. Fischhoff B. Hindsight not equal to fore-sight: the effect of outcome knowledge onjudgment under uncertainty. 1975. Qual SafHealth Care 2003;12:304–11;discussion311e2.11. Tversky A, Kahneman D. Judgment underuncertainty: heuristics and biases. Science1974;185:1124–31.12. Carpentieri AM, Lumalcuri JJ, Shaw J,Joseph GF Jr. Overview of the 2015 AmericanCongress of Obstetricians and Gynecologists’survey on professional liability. 2015. Availableat: https://protectpatientsnow.org/wp-content/uploads/2016/02/ACOG2015PLSurveyNationalSummary11315.pdf. Accessed September 11,2020.13. Nelson KB, Dambrosia JM, Ting TY,Grether JK. Uncertain value of electronic fetalmonitoring in predicting cerebral palsy. N Engl JMed 1996;334:613–8.



14. Turner JM, Mitchell MD, Kumar SS. Thephysiology of intrapartum fetal compromiseat term. Am J Obstet Gynecol 2020;222:17–26.15. Executive summary: neonatal brachialplexus palsy. Report of the American College ofObstetricians and Gynecologists’ Task Force onNeonatal Brachial Plexus Palsy. Obstet Gynecol2014;123:902–4.16. Gibbins KJ, Weber T, Holmgren CM,Porter TF, VarnerMW,Manuck TA.Maternal andfetal morbidity associated with uterine rupture ofthe unscarred uterus. Am J Obstet Gynecol2015;213:382.e1–6.17. Al-Zirqi I, Daltveit AK, Vangen S. Infantoutcome after complete uterine rupture. Am JObstet Gynecol 2018;219:109.e1–8.18. Hoyert DL, Miniño AM. Maternal mortality inthe United States: changes in coding publica-tion, and data release, 2018. Natl Vital Stat Rep2020;69:1–18.19. Kramer MR, Strahan AE, Preslar J, et al.Changing the conversation: applying a healthequity framework to maternal mortality reviews.Am J Obstet Gynecol 2019;221:609.e1–9.20. Captain Edward Smith. Available at: https://www.quotes.net/quote/35858. AccessedOctober 21, 2020.21. Djulbegovic B, Guyatt GH. Progress inevidence-based medicine: a quarter century on.Lancet 2017;390:415–23.22. Greenhalgh T, Howick J, Maskrey N. Evi-dence Based Medicine Renaissance Group.Evidence basedmedicine: amovement in crisis?BMJ 2014;348:g3725.23. EvidenceBC.Clinical Evidence. Howmuchdowe know?, vol. 2018;2013. Available at: https://www.ncims.com/wp-content/uploads/2016/01/HWClinical-Evidence.pdf.24. McAlister FA, Campbell NR, Zarnke K,Levine M, Graham ID. The management of hy-pertension in Canada: a review of currentguidelines, their shortcomings and implicationsfor the future. CMAJ 2001;164:517–22.25. McMurray J, Swedberg K. Treatment ofchronic heart failure: a comparison betweenthe major guidelines. Eur Heart J 2006;27:1773–7.26. McAlister FA, van Diepen S, Padwal RS,Johnson JA, Majumdar SR. How evidence-based are the recommendations inevidence-based guidelines? PLoS Med2007;4:e250.27. Steen RG, Dager SR. Evaluating the evi-dence for evidence-based medicine: are ran-domized clinical trials less flawed than otherforms of peer-reviewed medical research?FASEB J 2013;27:3430–6.28. Ioannidis JP. Contradicted and initiallystronger effects in highly cited clinical research.JAMA 2005;294:218–28.29. Stegenga J. Is meta-analysis the platinumstandard of evidence? Stud Hist Philos BiolBiomed Sci 2011;42:497–507.30. Rothwell PM. External validity of randomisedcontrolled trials: “to whom do the results of thistrial apply?”. Lancet 2005;365:82–93.

JANUARY 2021


31. Goldstein DB, Need AC, Singh R,Sisodiya SM. Potential genetic causes of het-erogeneity of treatment effects. Am J Med2007;120(Suppl1):S21–5.32. Greenfield S, Kravitz R, Duan N,Kaplan SH. Heterogeneity of treatment ef-fects: implications for guidelines, payment,and quality assessment. Am J Med2007;120(Suppl 1):S3–9.33. Kent DM, Hayward RA. Limitations ofapplying summary results of clinical trials to in-dividual patients: the need for risk stratification.JAMA 2007;298:1209–12.34. Kent DM, Rothwell PM, Ioannidis JP,Altman DG, Hayward RA. Assessing andreporting heterogeneity in treatment effects inclinical trials: a proposal. Trials 2010;11:85.35. Rothwell PM. Can overall results of clinicaltrials be applied to all patients? Lancet1995;345:1616–9.36. Rothwell PM, Mehta Z, Howard SC,Gutnikov SA, Warlow CP. Treating individuals 3:from subgroups to individuals: general principlesand the example of carotid endarterectomy.Lancet 2005;365:256–65.37. Glasziou PP, Irwig LM. An evidence basedapproach to individualising treatment. BMJ1995;311:1356–9.38. Ioannidis JP, Lau J. The impact of high-riskpatients on the results of clinical trials. J ClinEpidemiol 1997;50:1089–98.39. Hayward RA, Kent DM, Vijan S, Hofer TP.Reporting clinical trial results to inform providers,payers, and consumers. Health Aff (Millwood)2005;24:1571–81.40. Kent DM, Ruthazer R, Selker HP. Are somepatients likely to benefit from recombinanttissue-type plasminogen activator for acuteischemic stroke even beyond 3 hours fromsymptom onset? Stroke 2003;34:464–7.41. Rothwell PM. Treating individuals 2. Sub-group analysis in randomised controlled trials:importance, indications, and interpretation.Lancet 2005;365:176–86.42. Smeeth L, Haines A, Ebrahim S. Numbersneeded to treat derived from meta-analyses–sometimes informative, usually misleading.BMJ 1999;318:1548–51.43. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment ef-fects, and the trouble with averages. Milbank Q2004;82:661–87.44. Kent DM, Alsheikh-Ali A, Hayward RA.Competing risk and heterogeneity of treatmenteffect in clinical trials. Trials 2008;9:30.45. Kennedy-Martin T, Curtis S, Faries D,Robinson S, Johnston J. A literature reviewon the representativeness of randomizedcontrolled trial samples and implications forthe external validity of trial results. Trials2015;16:495.46. Jayasinghe S. Complexity science toconceptualize health and disease: is it relevant toclinical medicine? Mayo Clin Proc 2012;87:314–9.47. Lipsitz LA. Understanding health care as acomplex system: the foundation for


http://refhub.elsevier.com/S0002-9378(20)30885-1/sref1











































https://protectpatientsnow.org/wp-content/uploads/2016/02/ACOG2015PLSurveyNationalSummary11315.pdf
































https://www.quotes.net/quote/35858

https://www.quotes.net/quote/35858








https://www.ncims.com/wp-content/uploads/2016/01/HWClinical-Evidence.pdf


































































































http://www.AJOG.org


unintended consequences. JAMA 2012;308:243–4.48. Ma’ayan A. Complex systems biology. J RSoc Interface 2017;14:20170391.49. Plsek PE, Greenhalgh T. Complexity sci-ence: the challenge of complexity in health care.BMJ 2001;323:625–8.50. Cook RI. How complex systems fail. 2000.Available at: https://how.complexsystems.fail/.Accessed September 11, 2020.51. Symons J, Boschetti F. How computationalmodels predict the behavior of complex sys-tems. Found Sci 2013;18:809–21.52. American College of Obstetricians and Gy-necologists, Task Force on Hypertension inPregnancy. Hypertension in pregnancy. Reportof the American College of Obstetricians andGynecologists’ Task Force on Hypertension inPregnancy. Obstet Gynecol 2013;122:1122–31.53. Gale C, Modi N, Jawad S, et al. TheWHEATpilot trial-WithHolding enteral feeds Aroundpacked red cell Transfusion to prevent necrot-ising enterocolitis in preterm neonates: a multi-centre, electronic patient record (EPR),randomised controlled point-of-care pilot trial.BMJ Open 2019;9:e033543.54. Smith GC, Rowitch D, Mol BW. The role ofprenatal steroids at 34-36 weeks of gestation.Arch Dis Child Fetal Neonatal Ed 2017;102:F284–5.55. Crowther CA, Dodd JM, Hiller JE,Haslam RR, Robinson JS; Birth After CaesareanStudy Group. Planned vaginal birth or electiverepeat caesarean: patient preference restrictedcohort with nested randomised trial. PLoS Med2012;9:e1001192.56. Smith GC, Fleming KM,White IR. Birth orderof twins and risk of perinatal death related todelivery in England, Northern Ireland, andWales,1994-2003: retrospective cohort study. BMJ2007;334:576.57. Barrett JF, Hannah ME, Hutton EK, et al.A randomized trial of planned cesarean orvaginal delivery for twin pregnancy. N Engl JMed2013;369:1295–305.58. Smith GC, Pell JP, Dobbie R. Birth order,gestational age, and risk of delivery relatedperinatal death in twins: retrospective cohortstudy. BMJ 2002;325:1004.59. Zafarmand MH, Goossens SMTA, Tajik P,et al. Planned cesarean or planned vaginal de-livery for twins: a secondary analysis of a ran-domized controlled trial. Ultrasound ObstetGynecol 2019 [Epub ahead of print].60. Horton R. Offline: the gravy train of sys-tematic reviews. Lancet 2019;394:1790.61. Prior M, Hibberd R, Asemota N,Thornton JG. Inadvertent P-hacking among tri-als and systematic reviews of the effect of pro-gestogens in pregnancy? A systematic reviewand meta-analysis. BJOG 2017;124:1008–15.62. Braithwaite RS. A piece of my mind. EBM’ssix dangerous words. JAMA 2013;310:2149–50.


63. Torous J, Kiang MV, Lorme J, Onnela JP.New tools for new research in psychiatry: ascalable and customizable platform to empowerdata driven smartphone research. JMIR MentHealth 2016;3:e16.64. Mobile fact sheet. Pew Research Center.2019. Available at: https://www.pewresearch.org/internet/fact-sheet/mobile/. AccessedSeptember 11, 2020.65. Skinner AL, Attwood AS, Baddeley R,Evans-Reeves K, Bauld L, Munafò MR. Digitalphenotyping and the development and deliveryof health guidelines and behaviour change in-terventions. Addiction 2017;112:1281–5.66. Accenture Consulting. 2018 Consumersurvey on digital health. 2018. Available at: https://www.accenture.com/t20180306t103559z__w__/us-en/_acnmedia/pdf-71/accenture-health-2018-consumer-survey-digital-health.pdf. AccessedSeptember 11, 2020.67. Dunn J, Runge R, SnyderM.Wearables andthe medical revolution. Per Med 2018;15:429–48.68. Larkin A, Hystad P. Towards personal ex-posures: how technology is changing air pollu-tion and health research. Curr Environ HealthRep 2017;4:463–71.69. Reinertsen E, Clifford GD. A review ofphysiological and behavioral monitoring withdigital sensors for neuropsychiatric illnesses.Physiol Meas 2018;39:05TR01.70. De Choudhury M, Counts S, Horvitz E.Predicting postpartum changes in emotion andbehavior via social media. Proceedings of theSIGCHI Conference on Human Factors inComputing Systems. Paris, France: Associa-tion for Computing Machinery; 2013. p.3267–76.71. Vegesna A, Tran M, Angelaccio M,Arcona S. Remote patient monitoring vianon-invasive digital technologies: a sys-tematic review. Telemed J E Health2017;23:3–17.72. Coveney PV, Dougherty ER, Highfield RR.Big data need big theory too. Philos Trans AMath Phys Eng Sci 2016;374:20160153.73. Iyengar R, Altman RB, Troyanskya O,FitzGerald GA. MEDICINE. Personalization inpractice. Science 2015;350:282–3.74. Popper K. The logic of scientific discovery.New York, NY: Routledge; 2002.75. Yankeelov TE, AtuegwuN, Hormuth D, et al.Clinically relevant modeling of tumor growth andtreatment response. Sci Transl Med 2013;5:187ps9.76. Douglas PS, Pontone G, Hlatky MA, et al.Clinical outcomes of fractional flow reserve bycomputed tomographic angiography-guideddiagnostic strategies vs. usual care in patientswith suspected coronary artery disease: theprospective longitudinal trial of FFR(CT):outcome and resource impacts study. Eur HeartJ 2015;36:3359–67.77. Holmberg L, Vickers A. Evaluation of pre-diction models for decision-making: beyond

JANUARY 2021 A


calibration and discrimination. PLoS Med2013;10:e1001491.78. Olchanski N, Cohen JT, Neumann PJ,Wong JB, Kent DM. Understanding the value ofindividualized information: the impact of poorcalibration or discrimination in outcome predic-tion models. Med Decis Making 2017;37:790–801.79. Alba AC, Agoritsas T, Walsh M, et al.Discrimination and calibration of clinical predic-tion models: users’ guides to the medical litera-ture. JAMA 2017;318:1377–84.80. Wessler BS, Lai Yh L, Kramer W, et al.Clinical prediction models for cardiovasculardisease: Tufts Predictive Analytics andComparative Effectiveness clinical predictionmodel database. Circ Cardiovasc Qual Out-comes 2015;8:368–75.81. Escobar GJ, Gupta NR, Walsh EM,Soltesz L, Terry SM, Kipnis P. Automated earlydetection of obstetric complications: theoreticand methodologic considerations. Am J ObstetGynecol 2019;220:297–307.82. Riley RD, Ensor J, Snell KI, et al. Externalvalidation of clinical prediction models using bigdatasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ2016;353:i3140.83. McKenna MT, Weis JA, Brock A,Quaranta V, Yankeelov TE. Precision medicinewith imprecise therapy: computational modelingfor chemotherapy in breast cancer. Transl Oncol2018;11:732–42.84. Steyerberg EW, Vergouwe Y. Towardsbetter clinical prediction models: seven steps fordevelopment and an ABCD for validation. EurHeart J 2014;35:1925–31.85. Venkatesh KK, Strauss RA, Grotegut CA,et al. Machine learning and statistical models topredict postpartum hemorrhage. Obstet Gyne-col 2020;135:935–44.86. Sommers BD, Zeckhauser R. Probabili-ties and preferences: what economics canteach doctors and patients making difficulttreatment decisions. Urol Oncol 2008;26:669–73.87. Brashers DE. Communication and uncer-tainty management. J Commun 2001;51:477–97.88. Kahlor L. PRISM: a planned risk informationseeking model. Health Commun 2010;25:345–56.89. Ajzen I. The theory of planned behaviour:reactions and reflections. Psychol Health2011;26:1113–27.90. Afifi WA, Weiner JL. Toward a theory ofmotivated information management. CommunTheor 2004;14:167–90.91. Witte K. Putting the fear back into fear ap-peals: the extended parallel process model.Commun Monogr 1992;59:329–49.92. Freimuth VS, Stein JA, Kean TJ. Searchingfor Health Information: the Cancer InformationService model. Philadelphia: University ofPennsylvania Press; 1989.









https://how.complexsystems.fail/





























































https://www.pewresearch.org/internet/fact-sheet/mobile/

https://www.pewresearch.org/internet/fact-sheet/mobile/






https://www.accenture.com/t20180306t103559z__w__/us-en/_acnmedia/pdf-71/accenture-health-2018-consumer-survey-digital-health.pdf


















































































































http://www.AJOG.org


93. GriffinRJ,DunwoodyS,NeuwirthK.Proposedmodel of the relationship of risk information seekingand processing to the development of preventivebehaviors. Environ Res 1999;80:S230–45.94. Johnson JD. Cancer-related informationseeking. Creskill, NJ: Hampton Press; 1997.95. Douglas M, Wildavsky AB. Risk and culture:an essay on the selection of technical and envi-ronmental dangers. Berkeley: University of Cali-fornia Press; 1983.96. Stephens KK, Robertson BW, Murthy D.Throw me a lifeline: articulating mobile socialnetwork dispersion and the social constructionof risk in rescue communication. Mob MediaCommun 2020;8:149–69.97. McComas KA. Defining moments in riskcommunication research: 1996e2005. J HealthCommun 2006;11:75–91.



98. Committee on Obstetric Practice. Commit-tee Opinion no. 697: planned home birth. ObstetGynecol 2017;129:e117–22.99. Texas constitution: Article III, Sec 49a.Justia. Available at: https://law.justia.com/constitution/texas/sections/cn000300-49-g00.html. Accessed September 11, 2020.100. The Texas Constitution. Article III, Sec 5.Available at: https://statutes.capitol.texas.gov/Docs/CN/pdf/CN.3.pdf. Accessed September11, 2020.101. State of Texas. Cancer Prevention &Research Institute of Texas, 2019. Available at:https://www.cprit.state.tx.us. AccessedSeptember 11, 2020.102. Cancer Prevention & Research Insti-tute of Texas. Populations Served by CPRITPrevention Projects, 2019. Available at:

JANUARY 2021


https://www.cprit.state.tx.us/media/1998/current_programs_may_2019.pdf. AccessedSeptember 11, 2020.103. State of Texas. Proposition 6: IncreasingCPRIT’s bond authority from$3 billion to $6 billion,2019. Available at: https://hro.house.texas.gov/pdf/focus/amend86.pdf. Accessed September11, 2020.104. State of Texas. Senate Bill 11: Texas ChildMental Health Care Consortium, 2019. Availableat: https://capitol.texas.gov/tlodocs/86R/billtext/pdf/SB00011F.pdf#navpanes=0,%20pages%2042-50. Accessed September 11, 2020.105. State of Texas. General appropriations actfor the 2020-21 biennium, 2019:174. Avail-able at: http://www.lbb.state.tx.us/Documents/GAA/General_Appropriations_Act_2020_2021.pdf. Accessed September 11, 2020.























https://law.justia.com/constitution/texas/sections/cn000300-49-g00.html



https://statutes.capitol.texas.gov/Docs/CN/pdf/CN.3.pdf

https://statutes.capitol.texas.gov/Docs/CN/pdf/CN.3.pdf

https://www.cprit.state.tx.us

https://www.cprit.state.tx.us/media/1998/current_programs_may_2019.pdf

https://www.cprit.state.tx.us/media/1998/current_programs_may_2019.pdf

https://hro.house.texas.gov/pdf/focus/amend86.pdf

https://hro.house.texas.gov/pdf/focus/amend86.pdf

https://capitol.texas.gov/tlodocs/86R/billtext/pdf/SB00011F.pdf#navpanes=0,%20pages%2042-50



http://www.lbb.state.tx.us/Documents/GAA/General_Appropriations_Act_2020_2021.pdf



http://www.AJOG.org